Every once in a while I get clients who ask me why NServiceBus doesn’t support content-based routing. My answer sometimes surprises them, “because it is a dangerous pattern that should usually be avoided”.

Content-Based Routing and ESBs

Since content-based routing often appears on feature lists for various ESBs, many people consider them to be a necessary part of systems built on SOA principles. The pattern also appears in the book Enterprise Integration Patterns, which apparently is also a convincing reason to use it, even though the book specifically states:

“When implementing a Content-Based Router, special caution should be taken to make the routing function easy to maintain as the router can become a point of frequent maintenance.”

That’s right – maintenance nightmare.

For Example

Let’s take a real-world example – a trading system where users can put orders for stock and for forex (money from other countries).

We’d start by creating a message that can be sent by a client:

public class PlaceOrder : IMessage
{
    public OrderTypeEnum OrderType { get; set; }
    public string Code { get; set; }
    public double Amount { get; set; }
}

public enum OrderTypeEnum
{
    Stock,
    Forex
}

Clients would send messages by using code like the following:

Bus.Send<PlaceOrder>(m => {
    m.OrderType = OrderTypeEnum.Stock;
    m.Code = "MSFT";
    m.Amount = 300;
    m.SetHeader("AccountId", myAccountId);
});

The header is the way that clients identify themselves in the system.

Let’s say that the logic for handling the different types of orders is different, but also that we’d like the logic to be deployed to different endpoints. One reason we might want to do this is so that we can independently scale each of these endpoints. This is where it would appear we’d need some content-based routing – having some code that looks at the OrderType property and decides where to route based on its value.

Before we get into solutions, let’s make this more involved.

Not only do we want to route based on OrderType, but we want to use the account ID in the header to check in our database (or via a web service) if the account belongs to one of our VIP customers, and if so, it should be given higher priority.

Content-Based Routing and Brokers

We can see that if we were to go with a content-based routing solution, this would drive our architecture to a hub-and-spoke model where the hub becomes quite large and complex, as well as likely becoming a bottleneck in terms of performance.

This logically centralized place through which all communication flows defines the Broker architectural style – not that of a Bus. In the Bus architectural style, there is no logical (or physical) hub. You can think of it as a kind of peer-to-peer setup. Just like you wouldn’t want ethernet getting involved in applicative routing decisions, neither should your bus get involved.

Business-Topology Mapping Solutions

Instead, when following the Bus architectural style – we look at mapping stable business characteristics to bus-level topology. At one level, that would mean defining two different message types for our different types of trades:

public abstract class PlaceOrder
{
    public string Code { get; set; }
    public double Amount { get; set; }
}

public class PlaceStockOrder : PlaceOrder { }

public class PlaceForexOrder : PlaceOrder { }

Once we have two different message types, then we can configure the client to have those statically sent to different endpoints like this:

<UnicastBusConfig>
  <MessageEndpointMappings>
    <add Messages="Messages.PlaceStockOrder, Messages" Endpoint="Stock" />
    <add Messages="Messages.PlaceForexOrder, Messages" Endpoint="Forex" />
  </MessageEndpointMappings>
</UnicastBusConfig>

And when it comes to handling our VIP customers, the recommendation would be to have those customers be served by a different set of web servers – we wouldn’t want a sudden flux in regular customers stealing all the HTTP connections from our VIP customers. Then we’d statically configure our VIP front-end to talk to our VIP back-end like this:

<UnicastBusConfig>
  <MessageEndpointMappings>
    <add Messages="Messages.PlaceStockOrder, Messages" Endpoint="VIP_Stock" />
    <add Messages="Messages.PlaceForexOrder, Messages" Endpoint="VIP_Forex" />
  </MessageEndpointMappings>
</UnicastBusConfig>

And the logic to identify VIP customers would be done in the login screen which, from there, would direct the user to the appropriate web server farm.

Static vs. Dynamic

It may appear that the above statically configured solution is less flexible then the afore-mentioned hub-and-spoke content-based routing solution. And that’s probably correct. But the question is, are the business requirements (better called “objectives” in this case) likely to change? If we make the technological solution 10x more flexible than the business needs, but at the cost of maintainability (read time to market), we probably haven’t used the right tool for the job.

Many times we can find stable business objectives and align the topology of our solution with them. Not all that is dynamic and flexible is necessarily better than that which is static.

I’d go so far to say that if a solution makes heavy use of content-based routing, it is likely a more fragile solution as implementations of stable business objectives and volatile requirements are mixed up together.

In Closing

NServiceBus intentionally does not support content-based routing so as not to make it easy for developers to make architectural blunders that could require full-system rewrites a couple of years down the road.

If you want to learn more about these kinds of architectural principles, I suggest coming to my course. I’m afraid that New York City is already sold out, but I’ll be coming back to the US again around October. Stockholm, London, Sydney, and Oslo are now all open for registration.

Hope to see you there.

One of the things I cover early on in my course is the problem with traditional layered architecture driving people to create a business logic layer made up of a bunch of inter-related entities. I see this happening a lot, even though nowadays people are calling that bunch of inter-related entities a “domain model”.

Let me just say this upfront – most inter-related entity models are NOT a domain model.
Here’s why: most transactions don’t respect entity boundaries.

That being said, you don’t always need a domain model.
The domain model pattern’s context is “if you have complicated and everchanging business rules” – right there on page 119 of Patterns of Enterprise Application Architecture.

Persisting the customer’s first name, last name, and middle initial – and later reading and showing that data does not sound either complicated or that it is really going to change that much.

Then there are things like credit limits, that may be on the customer entity as well. It is likely that there are business requirements that expect that value to be consistent with the total value of unpaid orders – data that comes from other entities.

The problem that is created is one of throughput.

Since databases lock an entire row/entity at a time, if one transaction is changing the customer’s first name, the database would block another transaction that tried to change the same customer’s credit limit.

The bigger your entities, the more transactions will likely need to operate on them in parallel, the slower your system will get as the number of transactions increases. This feeds back in on itself as often those blocked transactions will have operated already on some other entity, leaving those locked for longer periods of times, blocking even more transactions.

And the absurd thing is that the business never demanded that the customer’s first name be consistent with the credit limit.

What if we didn’t have a single Customer entity?

What if we had one that contained first name, last name, middle initial and another that contained things like credit limit, status, and risk rating. These entities would be correlated by the same ID, but could be stored in separate tables in the database. That would do away with much of the cascading locking effects drastically improving our throughput as load increases.

And you know what? That division would still respect the 3rd normal form.

Which of these entities do you think would be classified by the business under the “complicated and everchanging rules” category?

And for those entities that are just about data persistence – do you think it’s justified to use 3 tiers? Do we really need a view model which we transform to data transfer objects which we transform to domain objects which we transform to relational tables and then all the way back? Wouldn’t some simpler 2-tier programming suffice – dare I say datasets? Ruby on Rails?

Are we ready to leave behind the assumption that all elements of a given layer must be built the same way?

One of the questions that came up from my NServiceBus – .NET Service Bus Smackdown post was about the Polymorphic Message Dispatch and Polymorphic Message Routing features. People wanted to know what those are, why they’re important, and if other technologies (specifically WCF and BizTalk) support them.

Messaging Basics

First of all, when building a system using messaging, you don’t have methods that are invoked on some remote object (a.k.a “service”) to which you pass parameters. Instead, you use some generic piece of infrastructure (in the world of Java, this is most commonly a Message Broker) to send a message where a message can be thought of as a serializable class. Here’s an example of a message:

public class UserCreated : IMessage
{
    public Guid UserId { get; set; }
    public string Name { get; set; }
}

This message would be published using NServiceBus like this:

bus.Publish<UserCreated>( m =>
{
    m.UserId = Guid.NewGuid();
    m.Name = "John Smith";
});

This can be contrasted with RPC models like WCF where you need to define a “service” that has methods on it, where those methods accepts parameters. Sometimes developers try to make this more generic by having a single “service” with one method on it named something like “Process” where the parameter it accepts is of the type “object”, or they introduce generics like this: Process<T>(T message);

This is where Polymorphic Message Dispatch comes in:

Polymorphic Message Dispatch

While you can pull of the WCF generics thing, one thing that is more difficult (without writing your own dispatch model) is to have a pipeline of classes which can be invoked based on their relationship to the type passed in. Using NServiceBus, both of the following message handlers will be invoked when UserCreated arrives:

public class Persistence : IHandleMessages<UserCreated>
{
    public void Handle(UserCreated message) { }
}

public class Audit : IHandleMessages<IMessage>
{
    public void Handle(IMessage message) { }
}

Now some might say that WCF, BizTalk, and the .NET Service Bus allow you to do auditing in their own internal pipeline, and that’s true. The place where this becomes more powerful is when you need to build V2 of your system, and the publisher now publishes a slightly different event – that a user was created as a part of a campaign, requiring the subscriber to register statistics about the campaign. Of course, this event also means that a user was created. Here’s how you’d do that with NServiceBus:

public class UserCreatedFromCampaign : UserCreated
{
    public Guid CampaignId { get; set; }
}

//publisher code
bus.Publish<UserCreatedFromCampaign>( m =>
{
    m.UserId = Guid.NewGuid();
    m.Name = "John Smith";
    m.CampaignId = theCampaignId;
}

//subscriber code
public class Statistics : IHandleMessages<UserCreatedFromCampaign>
{
    public void Handle(UserCreatedFromCampaign message) { }
}

The important part is what you don’t see – since UserCreatedFromCampaign inherits from UserCrated, the Persistence handler we had from V1 will also be invoked, and so will the Audit handler of course. You don’t have to make your new code call the old code like you would in a method based dispatch model. This makes sure that the coupling in your service layer code remains constant over time as you grow the functionality of your system.

This was one of the main benefits mentioned by Rackspace in their use of NServiceBus (here):

“The main benefit NServiceBus has brought us so far is developer scalability due to lower coupling and higher consistency in our code.”

But, when looking at the above scenario, we can obviously expect that all sorts of things can happen in relation to campaigns – it is a separate concern, and thus should be handled by a separate subscriber. And this bring us to…

Polymorphic Message Routing

The challenge that we have here is that we no longer have a hierarchy where something clearly belongs on top of something else. We have users created and activities happening related to campaigns – that may happen in any combination. By having separate subscribers, we could then introduce new handlers/subscribers to our environment without touching or taking down any of the other subscribers. Here’s what the subscribers would look like:

public class Persistence : IHandleMessages<UserCreated>
{
    public void Handle(UserCreated message) { }
}

public class Statistics : IHandleMessages<CampaignActivityOccurred>
{
    public void Handle(CampaignActivityOccurred message) { }
}

But if each of the above messages were a class, how could we define a message which inherited from both?

Before answering that, we need to understand why the publisher wouldn’t just publish both of the above messages. You see, the publisher can’t make any assumptions about its subscribers – it could be that one of them has logic that correlates across both of these messages that could end up counting the occurrence as happening twice rather than once, possibly charging the account associated with the campaign twice. Publishing two messages results in two transactions when there really should have been one.

So, here’s how to define messages so that we can have multiple inheritance:

public interface UserCreated : IMessage
{
    Guid UserId { get; set; }
    string Name { get; set; }
}

public interface CampaignActivityOccurred : IMessage
{
    Guid CampaignId { get; set; }
    Guid ActivityId { get; set; }
}

public interface UserCreatedFromCampaign
                 : UserCreated,
                   CampaignActivityOccurred
{
}

And when the publisher publishes UserCreatedFromCampaign, the event would be routed to both the UserCreated subscriber and the CampaignActivityOccurred subscriber. The power of this approach is felt as we handle new requirements around purchases made related to a campaign. Now we can have another event which inherits from CampaignActivityOccurred and not have to worry since the existing subscriber will be routed those messages automatically.

Since WCF doesn’t have publish/subscribe capabilities, we might as well move along.

Not to throw a burning match on an ocean of oil, but REST doesn’t really support this either.

Not Content-Based Routing

This may sound like the content-based router pattern from EIP (CBR), but it’s not. The important difference is that there isn’t some part of the routing that depends on the structure of the messages. The major drawback of CBR is that it creates a central place in your system that needs to be changed any time *syntactic* changes happen to message structure *in addition to* to changes in the subscribers.

Now, this is where the BizTalk guys would say that “that’s why we can do message transformations”, and then the subscribers wouldn’t need to be changed. However, can we really know when getting a requirement that the change is syntactic and not semantic? I mean, it’s quite common that changes to message structures happen together with changes to processing logic.

You may be beginning to get the feeling that more and more logic is being sucked out of the subscribers into some monolithic black hole that is likely going to be unmaintainable and quite slow.

This is one of the main differences between using a bus and a broker – a bus supports the correct distribution of logic keeping the system loosely coupled; brokers are useful integration engines when you absolutely can’t change the applications being integrated. Enterprise Application Integration (EAI) brokers don’t usually make good Enterprise Service Bus (ESB) technology.

In Closing

NServiceBus has all sorts of features you didn’t know you needed until you saw what life could be like when you had them. Most of these features don’t have snazzy drag-and-drop demos that make people ooh-and-aah and TechEds and PDCs, but they’re really necessary to avoid finding yourself in yet another big-ball-of-mud code base telling your manager/customer (again) that it would be faster to rebuild the system from scratch than to implement that new requirement in the old one.

Take NServiceBus for a spin and see for yourself.

One of the better known analysts in the enterprise software area, JP Morgenthal, wrote this post about the relationship between SOA, BPM, and EA. In it he defines SOA as follows:

“SOA is a practice that focuses on modeling the entities, and relationships between entities, that comprise the business as a set of services. This can be done on a small or large scale. Typically, the relationships in this model represent consumer/provider relationships.”

I have some serious concerns about the ramifications of this definition/description.

First of all, when reading “entities”, many people will interpret that to mean the entities found in Entity Relationship Diagrams [ERD] or in Object Oriented Analysis & Design [OOAD]. In both, these entities are identified as the “nouns” of the domain. Examples of these ERD/OOAD-type entities include things like Customer, Order, and Product.

These are almost always the wrong place to start for identifying services in SOA.

Second, on the consumer/provider relationship: on the one had, this fits very well with how web services can consume (or call) other web services. However, the downsides of using web services as services in SOA is becoming well enough known that even in the same post we see this warning:

“Web Services is not SOA, it is merely a standardized approach to accessing functionality on remote systems.”

But the question remains, if a producer/consumer relationship is OK for SOA-type services, why doesn’t that hold for web services? And the answer is… it depends on the type of producer/consumer relationship. The typical relationship is one of synchronous calls from consumer to producer, this is not OK for SOA-type services either.

You see, this synchronous producer/consumer implies a model where services are not able to fulfill their objectives without calling other services. In order for us to achieve the IT/Business alignment promised by SOA, we need services which are autonomous, ie. able to fulfill their objectives without that kind of external help.

Instead, we need to look for a more loosely coupled producer/consumer relationship – like publish/subscribe, where the producer emits events, and the consumer subscribes and handles those events. The reason that this kind of relationship doesn’t hurt autonomy is that it disconnects services on the dimension of time. In order for a service to be able to make a decision autonomously without synchronously calling any other service, using only information provided by events it received in the past, it must be strongly aligned with the business.

Most projects which bandy about the SOA acronym aren’t actually made up of services – they’re made up of XML over HTTP functions calling other XML over HTTP functions, eventually calling XML over HTTP databases. You can layer as much XML and HTTP as you want on top of it, but at the end of the day, most projects are just functions calling functions calling databases – in other words, procedural programming in the large, and no amount of SOAP will wash away the stink.

Here’s a different definition of services for SOA that may communicate a bit better what it’s all about:

A service is the technical authority for a specific business capability.
Any piece of data or rule must be owned by only one service.

What this means is that even when services are publishing and subscribing to each other’s events, we always know what the authoritative source of truth is for every piece of data and rule.

Also, when looking at services from the lense of business capabilities, what we see is that many user interfaces present information belonging to different capabilities – a product’s price alongside whether or not it’s in stock. In order for us to comply with the above definition of services, this leads us to an understanding that such user interfaces are actually a mashup – with each service having the fragment of the UI dealing with its particular data.

Ultimately, process boundaries like web apps, back-end, batch-processing are very poor indicators of service boundaries. We’d expect to see multiple business capabilities manifested in each of those processes.

I know that this may be more confusing than the traditional web services approach but, to paraphrase Donald Rumsfeld, it is better to know that you don’t know, than to not know that you don’t know

One architectural misunderstanding I see repeatedly in my work with clients is in the relationship between logical and physical architecture. The most common building-block of these misunderstandings is the web service (or it’s “upgraded” .net counterpart – the WCF service).

Don’t get me wrong, sometimes there is a place for a web service, just not everywhere.

So, what’s the problem?

Well, when developers and architects use web services as the building blocks of their designs, they are creating the same architecture for both the logical and physical elements of their system. Back in 1995, Philippe Kruchten documented his 4 + 1 Architectural View Model in which he outlined 4 + 1 different views that should be used to describe an architecture.

Even though since 1995 the number and types of recommended views of software architecture has evolved (with things like the Zachman Framework for enterprise architecture numbering some 30 views), there is broad agreement that (at the very least) the logical and physical artifacts should likely be designed differently.

Just because two distinct logical components have been identified in the architecture, that doesn’t necessarily mean they should be hosted separately (for example by making each one a web/wcf service). In fact, there are significant disadvantages to doing so (as described in the Fallacies of Distributed Computing).

In some cases, this mistake is exacerbated by a mistaking these components with SOA-type services, resulting in an attempt by developers to have each component have its own contract, which can then be independently versioned. This often results in the need for transformation between the structure of these so-called contracts, but not within the components themselves (oh-no, they’re “autonomous”), but rather in between them using some kind of “ESB” technology.

This architectural style is known as the Broker, Hub and Spoke, Mediator, and most importantly – not SOA. If you find a technology that fits this style perfectly (like BizTalk), that technology is not a Bus, not a Service Bus, and definitely not an Enterprise Service Bus.

One of the problems of this approach is that when any “service” contract changes, you have to change all the transformations in your broker that involve it. Unfortunately, most brokers have no unit-testing facility so it’s very much trial and error, and error, and error. The matter is even more serious since most brokers don’t enable you to have your transformations or orchestrations in source control, so you can’t diff to see what changed from the previous version.

It’s really amazing how much pain can be traced back to that one original misunderstanding.

Not in the business world anyway.

The problem is that, as software developers, we’re all too quick to accept them at face value. We don’t question the requirements – in all fairness, it was never our job to do so. We were the ones that implemented them, preferably quickly.

For example

Let’s say we get the requirement the following requirements:

1. If the order was already shipped, don’t let the user cancel the order.
2. If the order was already cancelled, don’t let the user ship the order.

The race condition here is when we have two users who are looking at the same order, which is neither cancelled nor shipped yet, and each submits a command – one to ship the order, the other to cancel it.

In these cases, the code is simple – just an if statement before performing the relevant command.

So what’s the problem

A microsecond difference in timing shouldn’t make a difference to core business behaviors. Which means that we’ve actually got here is a bug in the requirements. Users are actually dictating solutions here rather than requirements.

Let’s ask our stakeholders, “why shouldn’t we let users cancel a shipped order? I mean, the users don’t want the products.”

And the stakeholders would respond with something like, “well, we don’t want to refund the user’s money then. Or, at least, not all their money. Well, maybe if they return the products in their original packaging, *then* we could give a full refund.”

And as we drilled deeper, “when do refunds need to be given? Right away, in the same transaction?”

The stakeholders would explain, “no, refunds don’t need to be given right away.”

It turns out we were missing the concept of a refund, as well as assuming that all things needed to be processed and enforced immediately. Once we dug into the requirements, we found that there is actually plenty of time to allow both transactions to go through. We just need to add some checks during shipping’s long-running process to see if the order was cancelled, and then to cut the process short.

So is everything a long-running process then?

That’s actually a fair question – long-running processes are a lot more common than at first appears.

What we’re seeing is that cancellation is now a command that has no reason to fail – just like CQRS tells us. When this command is performed, it publishes the OrderCancelled event, which the billing service subscribes to.

Billing then starts a long-running process (a saga, in NServiceBus lingo), also listening to events from the shipping process, ultimately making a decision when a refund should be given, and for how much.

Deeper business analysis

As we discuss matters more with our business stakeholders, we hear that most orders are actually cancelled within an hour of being submitted. It is quite rare for orders to be cancelled days later.

In which case, we could look at modeling the acceptance of an order as a long-running process itself.

When a user places an order, we don’t immediately publish an event indicating the acceptance of an order, instead a saga is kicked off – which opens up a timeout for an hour later. If a cancellation command arrives during that period of time, the user gets a full refund (seeing as we didn’t charge anything since billing didn’t get the accepted event to begin with), and the saga just shuts itself down. If the timeout occurs an hour later, and the saga didn’t get a cancel command, then the order is actually accepted and the event is published.

Yes, sagas are everywhere, once you learn to see with business eyes, and no race conditions are left.

In closing

Any time you see requirements that indicate a race condition, dig deeper.

What you’re likely to find are some additional business concepts as well as the introduction of time and the creation of long-running business processes. The implementation at that point will pivot from being trivial if-statements to being richer sagas.

Keep an eye out.

This post will be less of a big-concept type posts I usually do, and more of a tip for people building and maintaining infrastructure and frameworks either open-source or internally for their companies. I’m going to illustrate this with NServiceBus as it is a large enough code base to have significant complexity and open so that you can go and take a look yourself. Trying to include some example in here would be just too small to be useful or for the point to come across.

Some background

As a cohesive framework, NServiceBus makes it quite easy for developers to pick and choose which settings they want turned on and off. Being built as a loosely-coupled set of components that don’t know about each other has always kept the internal complexity low. But as the NServiceBus API has been evolving over the years, and the functionality offered has increased, some interesting challenges have popped up as the codebase has been refactored.

The challenge

The UnicastBus class has grown too large and it’s time to refactor something out. Coincidentally, users have been asking for a better “header” story for messages – the ability to specify static headers that will be appended to all messages being sent (useful for things like security tokens), as well as per message headers. So, we want to refactor all the header management out to its own component independent of the UnicastBus class.

So, here’s the issue. So far, users have specified “.UnicastBus()” as a part of the fluent code-configuration, and shouldn’t have to change that – they shouldn’t need to know that header management is now a separate component. But then how can the new component bootstrap itself into the startup, such that it gets all the dependency injection facilities of the rest of the framework? Remember that the component doesn’t know which container technology is being used (since the user can swap it out) or when the container has been set.

The solution

The only part of the framework that knows about when all DI configuration is set is the configuration component, thus it will have to be the one that invokes the new component (without knowing about it). Introduce an interface (say INeedInitialization) and scan all the types loaded looking for classes which implement that type, register them into the container, and invoke them. Have the new component implement that interface, and in its initialization have it hook into the events and/or pipelines of other parts of the system.

Other uses

One historically problematic area in NServiceBus has been people forgetting to call “.LoadMessageHandlers()”. This can now be wired in automatically by a class in the UnicastBus component via the same mechanism.

A new feature coming in the next version is the “data bus”, a component which will allow sending large quantities of data through the bus without going through the messaging pipelines. This will help people get around the 4MB limit of MSMQ and, even more importantly, the much smaller 8KB limit of Azure. We will be able to introduce the functionality transparently with the same mechanism.

As an extension point, developers can now enrich the NServiceBus framework with their own capabilities and make those available via the contrib project to the community at large. This is better than the IWantToRunAtStartup interface that was only available for those using the generic host (which excluded web apps) and gives a consistent extensibility story for all uses.

Summary

Extensibility has always been a challenge when writing object-oriented code and dependency injection techniques have helped, but sometimes you need a bit more to take things to the next level while maintaining a backwards-compatible API.

Like I said, not a ground-shaking topic but something quite necessary in creating loosely-coupled frameworks and applications. Once you know it’s there, it isn’t really a big deal. If you didn’t know to do it, you may have been contorting your codebase in all kinds of ways to try to achieve similar things.

If you want to take a look at the code, you can find the SVN repository here: https://nservicebus.svn.sourceforge.net/svnroot/nservicebus/trunk/

OK – this is the last one, I promise. Well, for now, anyway.

Earlier this month at TechEd North America I gave a fairly new presentation that was only delivered once before (at the Connected Systems User Group in London) and I’m happy to say is now online for your viewing pleasure.

High Availability – A Contrarian View

Comments? Thoughts? Let me know.

In my work with clients the topic of how to handle the movement of software from one environment to another inevitably comes up. Sometimes this is in the context of NServiceBus but the problem is more generic. The faster that an organization is able to get software out the door, the more agile they can be.

Unfortunately, there is one tiny little mistake that I see almost everywhere that gets in the way, and that’s going to be the topic of this post.

The Problem

Let’s say you have a standard web app environment – some web servers, application servers, and a database server. Your web servers need to send messages to the application servers. So far, so good.

In your test environment, you have an application server called AS_01_Test, and your web servers are configured to send it messages. However, in your staging environment the application server fulfilling that same role is called AS_01_Stage. This creates a configuration problem – you need to change the config of your web servers as you move the web app from Test to Staging.

I’ve seen companies doing all sorts of creative things to get around this problem – some of them involve putting all configuration settings in a database so that they can be centrally managed and visualized. I’d like to suggest an alternative approach.

What if…

What if server names were the same across all environments?

Well, you wouldn’t need to change configuration as you moved the system between environments. That’s a good thing.

But how can that be? Wouldn’t there be a conflict if there were two machines with the same name?

The answer is that there wouldn’t be a conflict if the machines were on different networks. Not all machines have to be on the same network. We can set up as many networks / virtual networks as we like. And it is clear that we don’t need machines in one environment / network to talk to machines in another environment. I mean, under no circumstances would we want web servers in our test environment to talk to application servers in the production environment.

These separate networks provide much needed isolation, beyond solving the server naming problem.

In closing

It’s really a tiny thing when you think about – multiple networks. But that’s exactly why software developers overlook it so often – because it’s not a “software solution” to the configuration problem we perceive as a “software problem”.

I wrote about related multi-environment configuration issues in this earlier post: Convention over Configuration – The Next Generation

I’m happy to say that this functionality is now in NServiceBus called “profiles” and you can read more about how they work here.

How are you handling the flow of moving software through to production? Leave your comments below.

With the growing interest in Command/Query Responsibility Segregation (CQRS), more people are starting to ask questions about how to apply it to their applications. CQRS is actually in danger of reaching “best practice” status at which point in time people will apply it indiscriminately with truly terrible results.

One of the things that I’ve been trying to do with my presentations around the world on CQRS was to explain the why behind it, just as much as the what. The problem with the format of these presentations is that they’re designed to communicate a fairly closed message: here’s the problem, here’s how that problem manifests itself, here’s a solution.

In this post, I’m going to try to go deeper.

The hitchhiker’s guide to the galaxy

In this most excellent book, one of the things that struck me was the theme that made it’s way through the whole book – starting with the answer to life, the universe, and everything: 42. By the time you get to the end of the book, you find out that the real question to life, the universe, and everything is “what do you get when you multiply 6 by 9″. And that’s how the book leaves it.

To us engineers, we can’t just accept the fact that the book would say that 6*9 = 42 when we know it’s 54. After bashing our heads on the rigid rules of math, we realize that not all math problems are necessarily in base 10, and that if we switch to base 13, the number 42 is 4*13 + 2 = 54. So, the book was right – but that’s not the point.

What’s the point?

The hitchhiker’s guide is an example of a teaching technique which presents an apparent paradox, leaving the student to dig up unspoken and unthought assumptions in order to resolve it. Key to this technique are rigid rules which do not allow any compromise or shortcuts on the student’s part.

The purpose of this technique is not for the student to learn the answer, but to gain deeper understanding, which in turn changes the way they go about thinking about problems in the future.

So, when given the problem 4*5, we do not just immediately answer 20, instead we clarify in which numeric base the question is being phrased, and only then go to solve the problem. In base 13, the answer would be 17. In hex, the answer would be 14.

The externally visible change is that we know which questions to ask in order to arrive at the right answer – not that we know the answer ahead of time.

Making an “ass” out of “u” and “me”

Let’s start at the end – one of the unspoken assumptions that has been causing problems:

All businesses can be treated the same from the perspective of software.

In our previous example, we assumed that all math problems use base 10. It turns out that different bases are useful for different domains (like base 2 for computers). We can say similar things about degrees and radians in geometry. The more we look at the real world, the more we see this repeating itself. There’s no reason that software should be any different.

Base 10 is not a ubiquitous best practice. We shouldn’t be surprised that there really aren’t best practices for software either.

Here’s another problematic assumption:

“The business” can (and do) tell us what they need in a way we can understand.

So many software fads have been built on the quicksand of this assumption. OOAD – on verbs and nouns. 4GL and other visual tools that “the business” will use directly. SOA – on IT business alignment. I expect we haven’t seen the end of this.

Some of you may be wondering why this is false, others are sagely nodding their heads in agreement.

The myth of “the business”

Unless you have a single user, who is also the CEO paying for the development, there is no “the”. It’s an amalgam of people with different backgrounds, skills, and goals – there is no homogeneity. Even if no software was involved, many business organizations are dysfunctional with conflicting goals, policies, and politics.

To some extent, we technical people have hidden ourselves away in IT to avoid the scary world of business whose rules we don’t understand. With the rise in importance of information to the world, we’ve been pulled back – being forced to talk to people, and not just computers. Luckily, we’ve been able to create a buffer to insulate ourselves – we’ve taken the less successful technical people from our heard and nominated them “business analysts”. No, not all companies do it this way, but we do need to take a minute to reflect on how information flows between the business Mars into and out of the IT Venus.

On human communication

Even if we made this insulation layer more permeable, allowing and encouraging more technical people and business people to cross its boundary, we still need to deal with the problem of two humans communicating with each other. There are enough books that have been written on this topic, so I won’t go into that beyond recommending (strongly) to technical people to read (some of) them.

Rather, I’d like to focus on the environment in which these discussions take place. IT has been around long enough, and users have used computers long enough, that a certain amount of tainting has taken place. If the world was a trial, the evidence would have been thrown out as untrustworthy.

When users tell you what they want, they’re usually framing that with respect to the current system that they’re using. “Like the old system – but faster, and with better search, and more information on that screen, and…”

At this point, business analysts write down and formalize these “requirements” into some IT-sanctioned structure (use cases, user stories, whatever), at which point developers are told to build it. Users only know what they didn’t want when developers deliver exactly what was asked.

How can that be?

These are not the “requirements” you are looking for

Users ultimately dictate solutions to us, as a delta from the previous set of solutions we’ve delivered them. That’s just human psychology – writer’s block when looking at a blank page, as compared to the ease with which we provide “constructive criticism” on somebody else’s work.

We need to get the real requirements. We need to probe beyond the veneer:

• Why do you need this additional screen?
• What real-world trigger will cause you to open it?
• Is there more than one trigger?
• How are they different?
• etc, etc, etc…

This is real work – different work than programming. It requires different skills. And that’s not even getting into the political navigation between competing organizational forces.

But let’s say that you don’t have (enough) people with these skills in your organization. What then?

Enter CQRS

CQRS gives us a set of questions to ask, and some rigid rules that our answers must conform to. If our answers don’t fit, we need to go back to the drawing board and move things around and/or go back to “the business” and seek deeper understanding there.

For each screen/task/piece of data:

Will multiple users be collaborating on data related to this task?
Look at every shred of raw data, not just at the entity level.
Are there business consistency requirements around groups of raw data?

If “the business” answers no – ask them if they see that answer changing, and if so, in what time frame, and why. What changing conditions in the business environment would cause that to change – what other parts of the system would need to be re-examined under those conditions.

After understanding all that and you find a true single-user-only-thing, then you can use standard “CRUD” techniques and technologies. There are no inherent time-propagation problems in a single-user environment – so eventual consistency is beyond pointless, it actually makes matters worse.

On the other hand, if the business-data-space is collaborative, the inherent time-propagation of information between actors means they will be making decisions on data that isn’t up-to-the-millisecond-accurate anyway. This is physics, gravity – you can’t fight it (and win).

The rule for collaboration

Actors must be able submit one-way commands that will fail only under exceptional business circumstances.

The challenge we have is how to achieve the real business objectives uncovered in our previous “requirements excavation” activities and follow this rule at the same time. This will likely involve a different user-system interaction than those implemented in the past. UI design is part of the solution domain – it shouldn’t be dictated by the business (otherwise it’s like someone asking you to run a marathon, but also dictating how you do so, like by tying your shoelaces together).

Many of the technical patterns I described in my previous blog post describe the tools involved. BTW, hackers can be considered “exceptional actors” – the business actually wants their commands to fail.

In Summary

The hard and fast rule of CQRS about one-way commands is relevant for collaborative domains only. This domain has inherent eventual consistency – in the real world. Taking that and baking it into our solution domain is how we align with the business.

The process we go through, until ultimately arriving at one-way-almost-always-successful-commands is business analysis. Rejecting pre-formulated solutions, truly understanding the business drivers, and then representing those as directly as possible in our solution domain – that’s our job.

After doing this enough times and/or in more than one business domain, we may gain the insight that there is no cookie-cutter, one-size-fits-all, best-practice solution architecture for everything. Each problem domain is distinct and different – and we need to understand the details, because they should shape the resulting software structure.

The next time the business tell us to implement 42, we’ll use CQRS along with other questioning techniques until we can get “6 x 9″ out of them, learning from the exercise what are the significant and stable parts of the business – ultimately helping us to “build the right system, and to build the system right”.

Don’t Panic