Download as PDF – this is quite a long post.

Why CQRS

Before describing the details of CQRS we need to understand the two main driving forces behind it: collaboration and staleness.

Collaboration refers to circumstances under which multiple actors will be using/modifying the same set of data – whether or not the intention of the actors is actually to collaborate with each other. There are often rules which indicate which user can perform which kind of modification and modifications that may have been acceptable in one case may not be acceptable in others. We’ll give some examples shortly. Actors can be human like normal users, or automated like software.

Staleness refers to the fact that in a collaborative environment, once data has been shown to a user, that same data may have been changed by another actor – it is stale. Almost any system which makes use of a cache is serving stale data – often for performance reasons. What this means is that we cannot entirely trust our users decisions, as they could have been made based on out-of-date information.

Standard layered architectures don’t explicitly deal with either of these issues. While putting everything in the same database may be one step in the direction of handling collaboration, staleness is usually exacerbated in those architectures by the use of caches as a performance-improving afterthought.

A picture for reference

I’ve given some talks about CQRS using this diagram to explain it:

The boxes named AC are Autonomous Components. We’ll describe what makes them autonomous when discussing commands. But before we go into the complicated parts, let’s start with queries:

Queries

If the data we’re going to be showing users is stale anyway, is it really necessary to go to the master database and get it from there? Why transform those 3rd normal form structures to domain objects if we just want data – not any rule-preserving behaviors? Why transform those domain objects to DTOs to transfer them across a wire, and who said that wire has to be exactly there? Why transform those DTOs to view model objects?

In short, it looks like we’re doing a heck of a lot of unnecessary work based on the assumption that reusing code that has already been written will be easier than just solving the problem at hand. Let’s try a different approach:

How about we create an additional data store whose data can be a bit out of sync with the master database – I mean, the data we’re showing the user is stale anyway, so why not reflect in the data store itself. We’ll come up with an approach later to keep this data store more or less in sync.

Now, what would be the correct structure for this data store? How about just like the view model? One table for each view. Then our client could simply SELECT * FROM MyViewTable (or possibly pass in an ID in a where clause), and bind the result to the screen. That would be just as simple as can be. You could wrap that up with a thin facade if you feel the need, or with stored procedures, or using AutoMapper which can simply map from a data reader to your view model class. The thing is that the view model structures are already wire-friendly, so you don’t need to transform them to anything else.

You could even consider taking that data store and putting it in your web tier. It’s just as secure as an in-memory cache in your web tier. Give your web servers SELECT only permissions on those tables and you should be fine.

Query Data Storage

While you can use a regular database as your query data store it isn’t the only option. Consider that the query schema is in essence identical to your view model. You don’t have any relationships between your various view model classes, so you shouldn’t need any relationships between the tables in the query data store.

So do you actually need a relational database?

The answer is no, but for all practical purposes and due to organizational inertia, it is probably your best choice (for now).

Scaling Queries

Since your queries are now being performed off of a separate data store than your master database, and there is no assumption that the data that’s being served is 100% up to date, you can easily add more instances of these stores without worrying that they don’t contain the exact same data. The same mechanism that updates one instance can be used for many instances, as we’ll see later.

This gives you cheap horizontal scaling for your queries. Also, since your not doing nearly as much transformation, the latency per query goes down as well. Simple code is fast code.

Data modifications

Since our users are making decisions based on stale data, we need to be more discerning about which things we let through. Here’s a scenario explaining why:

Let’s say we have a customer service representative who is one the phone with a customer. This user is looking at the customer’s details on the screen and wants to make them a ‘preferred’ customer, as well as modifying their address, changing their title from Ms to Mrs, changing their last name, and indicating that they’re now married. What the user doesn’t know is that after opening the screen, an event arrived from the billing department indicating that this same customer doesn’t pay their bills – they’re delinquent. At this point, our user submits their changes.

Should we accept their changes?

Well, we should accept some of them, but not the change to ‘preferred’, since the customer is delinquent. But writing those kinds of checks is a pain – we need to do a diff on the data, infer what the changes mean, which ones are related to each other (name change, title change) and which are separate, identify which data to check against – not just compared to the data the user retrieved, but compared to the current state in the database, and then reject or accept.

Unfortunately for our users, we tend to reject the whole thing if any part of it is off. At that point, our users have to refresh their screen to get the up-to-date data, and retype in all the previous changes, hoping that this time we won’t yell at them because of an optimistic concurrency conflict.

As we get larger entities with more fields on them, we also get more actors working with those same entities, and the higher the likelihood that something will touch some attribute of them at any given time, increasing the number of concurrency conflicts.

If only there was some way for our users to provide us with the right level of granularity and intent when modifying data. That’s what commands are all about.

Commands

A core element of CQRS is rethinking the design of the user interface to enable us to capture our users’ intent such that making a customer preferred is a different unit of work for the user than indicating that the customer has moved or that they’ve gotten married. Using an Excel-like UI for data changes doesn’t capture intent, as we saw above.

We could even consider allowing our users to submit a new command even before they’ve received confirmation on the previous one. We could have a little widget on the side showing the user their pending commands, checking them off asynchronously as we receive confirmation from the server, or marking them with an X if they fail. The user could then double-click that failed task to find information about what happened.

Note that the client sends commands to the server – it doesn’t publish them. Publishing is reserved for events which state a fact – that something has happened, and that the publisher has no concern about what receivers of that event do with it.

Commands and Validation

In thinking through what could make a command fail, one topic that comes up is validation. Validation is different from business rules in that it states a context-independent fact about a command. Either a command is valid, or it isn’t. Business rules on the other hand are context dependent.

In the example we saw before, the data our customer service rep submitted was valid, it was only due to the billing event arriving earlier which required the command to be rejected. Had that billing event not arrived, the data would have been accepted.

Even though a command may be valid, there still may be reasons to reject it.

As such, validation can be performed on the client, checking that all fields required for that command are there, number and date ranges are OK, that kind of thing. The server would still validate all commands that arrive, not trusting clients to do the validation.

Rethinking UIs and commands in light of validation

The client can make of the query data store when validating commands. For example, before submitting a command that the customer has moved, we can check that the street name exists in the query data store.

At that point, we may rethink the UI and have an auto-completing text box for the street name, thus ensuring that the street name we’ll pass in the command will be valid. But why not take things a step further? Why not pass in the street ID instead of its name? Have the command represent the street not as a string, but as an ID (int, guid, whatever).

On the server side, the only reason that such a command would fail would be due to concurrency – that someone had deleted that street and that that hadn’t been reflected in the query store yet; a fairly exceptional set of circumstances.

Reasons valid commands fail and what to do about it

So we’ve got a well-behaved client that is sending valid commands, yet the server still decides to reject them. Often the circumstances for the rejection are related to other actors changing state relevant to the processing of that command.

In the CRM example above, it is only because the billing event arrived first. But “first” could be a millisecond before our command. What if our user pressed the button a millisecond earlier? Should that actually change the business outcome? Shouldn’t we expect our system to behave the same when observed from the outside?

So, if the billing event arrived second, shouldn’t that revert preferred customers to regular ones? Not only that, but shouldn’t the customer be notified of this, like by sending them an email? In which case, why not have this be the behavior for the case where the billing event arrives first? And if we’ve already got a notification model set up, do we really need to return an error to the customer service rep? I mean, it’s not like they can do anything about it other than notifying the customer.

So, if we’re not returning errors to the client (who is already sending us valid commands), maybe all we need to do on the client when sending a command is to tell the user “thank you, you will receive confirmation via email shortly”. We don’t even need the UI widget showing pending commands.

Commands and Autonomy

What we see is that in this model, commands don’t need to be processed immediately – they can be queued. How fast they get processed is a question of Service-Level Agreement (SLA) and not architecturally significant. This is one of the things that makes that node that processes commands autonomous from a runtime perspective – we don’t require an always-on connection to the client.

Also, we shouldn’t need to access the query store to process commands – any state that is needed should be managed by the autonomous component – that’s part of the meaning of autonomy.

Another part is the issue of failed message processing due to the database being down or hitting a deadlock. There is no reason that such errors should be returned to the client – we can just rollback and try again. When an administrator brings the database back up, all the message waiting in the queue will then be processed successfully and our users receive confirmation.

The system as a whole is quite a bit more robust to any error conditions.

Also, since we don’t have queries going through this database any more, the database itself is able to keep more rows/pages in memory which serve commands, improving performance. When both commands and queries were being served off of the same tables, the database server was always juggling rows between the two.

Autonomous Components

While in the picture above we see all commands going to the same AC, we could logically have each command processed by a different AC, each with it’s own queue. That would give us visibility into which queue was the longest, letting us see very easily which part of the system was the bottleneck. While this is interesting for developers, it is critical for system administrators.

Since commands wait in queues, we can now add more processing nodes behind those queues (using the distributor with NServiceBus) so that we’re only scaling the part of the system that’s slow. No need to waste servers on any other requests.

Service Layers

Our command processing objects in the various autonomous components actually make up our service layer. The reason you don’t see this layer explicitly represented in CQRS is that it isn’t really there, at least not as an identifiable logical collection of related objects – here’s why:

In the layered architecture (AKA 3-Tier) approach, there is no statement about dependencies between objects within a layer, or rather it is implied to be allowed. However, when taking a command-oriented view on the service layer, what we see are objects handling different types of commands. Each command is independent of the other, so why should we allow the objects which handle them to depend on each other?

Dependencies are things which should be avoided, unless there is good reason for them.

Keeping the command handling objects independent of each other will allow us to more easily version our system, one command at a time, not needing even to bring down the entire system, given that the new version is backwards compatible with the previous one.

Therefore, keep each command handler in its own VS project, or possibly even in its own solution, thus guiding developers away from introducing dependencies in the name of reuse (it’s a fallacy). If you do decide as a deployment concern, that you want to put them all in the same process feeding off of the same queue, you can ILMerge those assemblies and host them together, but understand that you will be undoing much of the benefits of your autonomous components.

Whither the domain model?

Although in the diagram above you can see the domain model beside the command-processing autonomous components, it’s actually an implementation detail. There is nothing that states that all commands must be processed by the same domain model. Arguably, you could have some commands be processed by transaction script, others using table module (AKA active record), as well as those using the domain model. Event-sourcing is another possible implementation.

Another thing to understand about the domain model is that it now isn’t used to serve queries. So the question is, why do you need to have so many relationships between entities in your domain model?

(You may want to take a second to let that sink in.)

Do we really need a collection of orders on the customer entity? In what command would we need to navigate that collection? In fact, what kind of command would need any one-to-many relationship? And if that’s the case for one-to-many, many-to-many would definitely be out as well. I mean, most commands only contain one or two IDs in them anyway.

Any aggregate operations that may have been calculated by looping over child entities could be pre-calculated and stored as properties on the parent entity. Following this process across all the entities in our domain would result in isolated entities needing nothing more than a couple of properties for the IDs of their related entities – “children” holding the parent ID, like in databases.

In this form, commands could be entirely processed by a single entity – viola, an aggregate root that is a consistency boundary.

Persistence for command processing

Given that the database used for command processing is not used for querying, and that most (if not all) commands contain the IDs of the rows they’re going to affect, do we really need to have a column for every single domain object property? What if we just serialized the domain entity and put it into a single column, and had another column containing the ID? This sounds quite similar to key-value storage that is available in the various cloud providers. In which case, would you really need an object-relational mapper to persist to this kind of storage?

You could also pull out an additional property per piece of data where you’d want the “database” to enforce uniqueness.

I’m not suggesting that you do this in all cases – rather just trying to get you to rethink some basic assumptions.

Let me reiterate

How you process the commands is an implementation detail of CQRS.

Keeping the query store in sync

After the command-processing autonomous component has decided to accept a command, modifying its persistent store as needed, it publishes an event notifying the world about it. This event often is the “past tense” of the command submitted:

MakeCustomerPerferredCommand -> CustomerHasBeenMadePerferredEvent

The publishing of the event is done transactionally together with the processing of the command and the changes to its database. That way, any kind of failure on commit will result in the event not being sent. This is something that should be handled by default by your message bus, and if you’re using MSMQ as your underlying transport, requires the use of transactional queues.

The autonomous component which processes those events and updates the query data store is fairly simple, translating from the event structure to the persistent view model structure. I suggest having an event handler per view model class (AKA per table).

Here’s the picture of all the pieces again:

Bounded Contexts

While CQRS touches on many pieces of software architecture, it is still not at the top of the food chain. CQRS if used is employed within a bounded context (DDD) or a business component (SOA) – a cohesive piece of the problem domain. The events published by one BC are subscribed to by other BCs, each updating their query and command data stores as needed.

UI’s from the CQRS found in each BC can be “mashed up” in a single application, providing users a single composite view on all parts of the problem domain. Composite UI frameworks are very useful for these cases.

Summary

CQRS is about coming up with an appropriate architecture for multi-user collaborative applications. It explicitly takes into account factors like data staleness and volatility and exploits those characteristics for creating simpler and more scalable constructs.

One cannot truly enjoy the benefits of CQRS without considering the user-interface, making it capture user intent explicitly. When taking into account client-side validation, command structures may be somewhat adjusted. Thinking through the order in which commands and events are processed can lead to notification patterns which make returning errors unnecessary.

While the result of applying CQRS to a given project is a more maintainable and performant code base, this simplicity and scalability require understanding the detailed business requirements and are not the result of any technical “best practice”. If anything, we can see a plethora of approaches to apparently similar problems being used together – data readers and domain models, one-way messaging and synchronous calls.

Although this blog post is over 3000 words (a record for this blog), I know that it doesn’t go into enough depth on the topic (it takes about 3 days out of the 5 of my Advanced Distributed Systems Design course to cover everything in enough depth). Still, I hope it has given you the understanding of why CQRS is the way it is and possibly opened your eyes to other ways of looking at the design of distributed systems.

Questions and comments are most welcome.

There’s this belief that if we just reused more code, everything would be better.

Some even go so far as saying that the whole point of object-orientation was reuse – it wasn’t, encapsulation was the big thing. After that component-orientation was the thing that was supposed to make reuse happen. Apparently that didn’t pan out so well either because here we are now pinning our reuseful hopes on service-orientation.

Entire books of patterns have been written on how to achieve reuse with the orientation of the day.
Services have been classified every which way in trying to achieve this, from entity services and activity services, through process services and orchestration services. Composing services has been touted as the key to reusing, and creating reusable services.

I might as well let you in on the dirty-little secret:

Reuse is a fallacy

Before running too far ahead, let’s go back to what the actual goal of reuse was: getting done faster.

That’s it.

It’s a fine goal to have.

And here’s how reuse fits in to the picture:

If we were to write all the code of a system, we’d write a certain amount of code.
If we could reuse some code from somewhere else that was written before, we could write less code.
The more code we can reuse, the less code we write.
The less code we write, the sooner we’ll be done!

However, the above logical progression is based on another couple of fallacies:

Fallacy: All code takes the same amount of time to write

Fallacy: Writing code is the primary activity in getting a system done

Anyone who’s actually written some code that’s gone into production knows this.

There’s the time it takes us to understand what the system should do.
Multiply that by the time it takes the users to understand what the system should do
Then there’s the integrating that code with all the other code, databases, configuration, web services, etc.
Debugging. Deploying. Debugging. Rebugging. Meetings. Etc.

Writing code is actually the least of our worries.
We actually spend less time writing code than…

Rebugging code

Also known as bug regressions.

This is where we fix one piece of code, and in the process break another piece of code.
It’s not like we do it on purpose. It’s all those dependencies between the various bits of code.
The more dependencies there are, the more likely something’s gonna break.
Especially when we have all sorts of hidden dependencies,
like when other code uses stuff we put in the database without asking us what it means,
or, heaven forbid, changing it without telling us.

These debugging/rebugging cycles can make stabilizing a system take a long time.

So, how does reuse help/hinder with that?

Here’s how:

Dependencies multiply by reuse

It’s to be expected. If you wrote the code all in one place, there are no dependencies. By reusing code, you’ve created a dependency. The more you reuse, the more dependencies you have. The more dependencies, the more rebugging.

Of course, we need to keep in mind the difference between…

Reuse & Use

Your code uses the runtime API (JDK, .NET BCL, etc).
Likewise other frameworks like (N)Hibernate, Spring, WCF, etc.

Reuse happens when you extend and override existing behaviors within other code.
This is most often done by inheritance in OO languages.

Interestingly enough, by the above generally accepted definition, most web services “reuse” is actually really use.

Let’s take a look at the characteristics of the code we’re using and reusing to see where we get the greatest value:

The value of (re)use

If we were to (re)use a piece of code in only one part of our system, it would be safe to say that we would get less value than if we could (re)use it in more places. For example, we could say that for many web applications, the web framework we use provides more value than a given encryption algorithm that we may use in only a few places.

So, what characterizes the code we use in many places?

Well, it’s very generic.

Actually, the more generic a piece of code, the less likely it is that we’ll be changing something in it when fixing a bug in the system.

That’s important.

However, when looking at the kind of code we reuse, and the reasons around it, we tend to see very non-generic code – something that deals with the domain-specific behaviors of the system. Thus, the likelihood of a bug fix needing to touch that code is higher than in the generic/use-not-reuse case, often much higher.

How it all fits together

Goal: Getting done faster
Via: Spending less time debugging/rebugging/stabilizing
Via: Having less dependencies reasonably requiring a bug fix to touch the dependent side
Via: Not reusing non-generic code

This doesn’t mean you shouldn’t use generic code / frameworks where applicable – absolutely, you should.
Just watch the number of kind of dependencies you introduce.

Back to services

So, if we follow the above advice with services, we wouldn’t want domain specific services reusing each other.
If we could get away with it, we probably wouldn’t even want them using each other either.

As use and reuse go down, we can see that service autonomy goes up. And vice-versa.
Luckily, we have service interaction mechanisms from Event-Driven Architecture that enable use without breaking autonomy.
Autonomy is actually very similar to the principle of encapsulation that drove object-orientation in the first place.
Interesting, isn’t it?

In summary

We all want to get done faster.

Way back when, someone told us reuse was the way to do that.

They were wrong.

Reuse may make sense in the most tightly coupled pieces of code you have, but not very much anywhere else.

When designing services in your SOA, stay away from reuse, and minimize use (with EDA patterns).

The next time someone pulls the “reuse excuse”, you’ll be ready.

Further Reading

• Additional logic required for service autonomy
• Self-contained events & SOA
• Autonomous Services and Enterprise Entity Aggregation [MS Architecture Journal]
• Does SOA mean the end of OO? [Podcast]

The common e-commerce example

We accept orders, bill the customer, and then ship them the product.

Fairly straight-forward.

Since each part of that process can be quite complex, let’s have each step be handled by a service:

Sales, Billing, and Shipping. Each of these services will publish an event when it’s done its part. Sales will publish OrderAccepted containing all the order information – order Id, customer Id, products, quantities, etc. Billing will publish CustomerBilledForOrder containing the customer Id, order Id, etc. And Shipping will publish OrderShippedToCustomer with its data.

So far, so good. EDA and SOA seem to be providing us some value.

Where’s the saga?

Well, let’s consider the behavior of the Shipping service. It shouldn’t ship the order to the customer until it has received the CustomerBilledForOrder event as well as the OrderAccepted event. In other words, Shipping needs to hold on to the state that came in the first event until the second event comes in. And this is exactly what sagas are for.

Let’s take a look at the saga code that implements this. In order to simplify the sample a bit, I’ll be omitting the product quantities.

   1:      public class ShippingSaga : Saga<ShippingSagaData>,

   2:          ISagaStartedBy<OrderAccepted>,

   3:          ISagaStartedBy<CustomerBilledForOrder>

   4:      {

   5:          public void Handle(OrderAccepted message)

   6:          {

   7:              this.Data.ProductIdsInOrder = message.ProductIdsInOrder;

   8:          }

   9:   

  10:          public void Handle(CustomerBilledForOrder message)

  11:          {

  12:               this.Bus.Send<ShipOrderToCustomer>(

  13:                  (m =>

  14:                  {

  15:                      m.CustomerId = message.CustomerId;

  16:                      m.OrderId = message.OrderId;

  17:                      m.ProductIdsInOrder = this.Data.ProductIdsInOrder;

  18:                  }

  19:                  ));

  20:   

  21:              this.MarkAsComplete();

  22:          }

  23:   

  24:          public override void Timeout(object state)

  25:          {

  26:              

  27:          }

  28:      }

First of all, this looks fairly simple and straightforward, which is good.

It’s also wrong, which is not so good.

One problem we have here is that events may arrive out of order – first CustomerBilledForOrder, and only then OrderAccepted. What would happen in the above saga in that case? Well, we wouldn’t end up shipping the products to the customer, and customers tend not to like that (for some reason).

There’s also another problem here. See if you can spot it as I go through the explanation of ISagaStartedBy<T>.

Saga start up and correlation

The “ISagaStartedBy<T>” that is implemented for both messages indicates to the infrastructure (NServiceBus) that when a message of that type arrives, if an existing saga instance cannot be found, that a new instance should be started up. Makes sense, doesn’t it? For a given order, when the OrderAccepted event arrives first, Shipping doesn’t currently have any sagas handling it, so it starts up a new one. After that, when the CustomerBilledForOrder event arrives for that same order, the event should be handled by the saga instance that handled the first event – not by a new one.

I’ll repeat the important part: “the event should be handled by the saga instance that handled the first event”.

Since the only information we stored in the saga was the list of products, how would we be able to look up that saga instance when the next event came in containing an order Id, but no saga Id?

OK, so we need to store the order Id from the first event so that when the second event comes along we’ll be able to find the saga based on that order Id. Not too complicated, but something to keep in mind.

Let’s look at the updated code:

   1:      public class ShippingSaga : Saga<ShippingSagaData>,

   2:          ISagaStartedBy<OrderAccepted>,

   3:          ISagaStartedBy<CustomerBilledForOrder>

   4:      {

   5:          public void Handle(CustomerBilledForOrder message)

   6:          {

   7:              this.Data.CustomerHasBeenBilled = true;

   8:   

   9:              this.Data.CustomerId = message.CustomerId;

  10:              this.Data.OrderId = message.OrderId;

  11:   

  12:              this.CompleteIfPossible();

  13:          }

  14:   

  15:          public void Handle(OrderAccepted message)

  16:          {

  17:              this.Data.ProductIdsInOrder = message.ProductIdsInOrder;

  18:   

  19:              this.Data.CustomerId = message.CustomerId;

  20:              this.Data.OrderId = message.OrderId;

  21:   

  22:              this.CompleteIfPossible();

  23:          }

  24:   

  25:          private void CompleteIfPossible()

  26:          {

  27:              if (this.Data.ProductIdsInOrder != null && this.Data.CustomerHasBeenBilled)

  28:              {

  29:                  this.Bus.Send<ShipOrderToCustomer>(

  30:                     (m =>

  31:                     {

  32:                         m.CustomerId = this.Data.CustomerId;

  33:                         m.OrderId = this.Data.OrderId;

  34:                         m.ProductIdsInOrder = this.Data.ProductIdsInOrder;

  35:                     }

  36:                     ));

  37:                  this.MarkAsComplete();

  38:              }

  39:          }

  40:      }

And that brings us to…

Saga persistence

We already saw why Shipping needs to be able to look up its internal sagas using data from the events, but what that means is that simple blob-type persistence of those sagas is out. NServiceBus comes with an NHibernate-based saga persister for exactly this reason, though any persistence mechanism which allows you to query on something other than saga Id would work just as well.

Let’s take a quick look at the saga data that we’ll be storing and see how simple it is:

   1:      public class ShippingSagaData : ISagaEntity

   2:      {

   3:          public virtual Guid Id { get; set; }

   4:          public virtual string Originator { get; set; }

   5:          public virtual Guid OrderId { get; set; }

   6:          public virtual Guid CustomerId { get; set; }

   7:          public virtual List<Guid> ProductIdsInOrder { get; set; }

   8:          public virtual bool CustomerHasBeenBilled { get; set; }

   9:      }

You might have noticed the “Originator” property in there and wondered what it is for. First of all, the ISagaEntity interface requires the two properties Id and Originator. Originator is used to store the return address of the message that started the saga. Id is for what you think it’s for. In this saga, we don’t need to send any messages back to whoever started the saga, but in many others we do. In those cases, we’ll often be handling a message from some other endpoint when we want to possibly report some status back to the client that started the process. By storing that client’s address the first time, we can then “ReplyToOriginator” at any point in the process.

The manufacturing sample that comes with NServiceBus shows how this works.

Saga Lookup

Earlier, we saw the need to search for sagas based on order Id. The way to hook into the infrastructure and perform these lookups is by implementing “IFindSagas<T>.Using<M>” where T is the type of the saga data and M is the type of message. In our example, doing this using NHibernate would look like this:

   1:      public class ShippingSagaFinder : 

   2:          IFindSagas<ShippingSagaData>.Using<OrderAccepted>,

   3:          IFindSagas<ShippingSagaData>.Using<CustomerBilledForOrder>

   4:      {

   5:          public ShippingSagaData FindBy(CustomerBilledForOrder message)

   6:          {

   7:              return FindBy(message.OrderId)

   8:          }

   9:   

  10:          public ShippingSagaData FindBy(OrderAccepted message)

  11:          {

  12:              return FindBy(message.OrderId)

  13:          }

  14:   

  15:          private ShippingSagaData FindBy(Guid orderId)

  16:          {

  17:              return sessionFactory.GetCurrentSession().CreateCriteria(typeof(ShippingSagaData))

  18:                  .Add(Expression.Eq("OrderId", orderId))

  19:                  .UniqueResult<ShippingSagaData>();

  20:          }

  21:   

  22:          private ISessionFactory sessionFactory;

  23:   

  24:          public virtual ISessionFactory SessionFactory

  25:          {

  26:              get { return sessionFactory; }

  27:              set { sessionFactory = value; }

  28:          }

  29:      }

For a performance boost, we’d probably index our saga data by order Id.

On concurrency

Another important note is that for this saga, if both messages were handled in parallel on different machines, the saga could get stuck. The persistence mechanism here needs to prevent this. When using NHibernate over a database with the appropriate isolation level (Repeatable Read – the default in NServiceBus), this “just works”. If/When implementing your own saga persistence mechanism, it is important to understand the kind of concurrency your business logic can live with.

Take a look at Ayende’s example for mobile phone billing to get a feeling for what that’s like.

Summary

In almost any event-driven architecture, you’ll have services correlating multiple events in order to make decisions. The saga pattern is a great fit there, and not at all difficult to implement. You do need to take into account that events may arrive out of order and implement the saga logic accordingly, but it’s really not that big a deal. Do take the time to think through what data will need to be stored in order for the saga to be fault-tolerant, as well as a persistence mechanism that will allow you to look up that data based on event data.

If you feel like giving this approach a try, but don’t have an environment handy for this, download NServiceBus and take a look at the samples. It’s really quick and easy to get set up.

I’ve been to too many clients where I’ve been brought in to help them with their problems around service versioning when the solution I propose is simply to have version N+1 of the system be backwards-compatible with version N. If two adjacent versions of a given system aren’t compatible with each other, it is practically impossible to solve versioning issues.

Here’s what happens when versions aren’t compatible:

Admins stop the system from accepting any new requests, and wait until all current requests are done processing. They take a backup/snapshot of all relevant parts of the system (like data in the DB). Then, bring down the system – all of it. Install the new version on all machines. Bring everything back up. Let the users back in.

If, heaven-forbid, problems were uncovered with the new version (since some problems only appear in production), the admins have to roll back to the previous version – once again bringing everything down.

This scenario is fairly catastrophic for any company that requires not-even high availability, but pretty continuous availability – like public facing web apps.

If adjacent versions were compatible with each other, we could upgrade the system piece-meal – machine by machine, where both the old and new versions will be running side by side, communicating with each other. While the system’s performance may be sub-optimal, it will continue to be available throughout upgrades as well as downgrades.

This isn’t trivial to do.

It impacts how you decide what is (and more importantly, what isn’t) nullable.

It may force you to spread certain changes to features across more versions (aka releases).

As such, you can expect this to affect how you do release and feature planning.

However, if you do not take these factors into account, it’s almost a certainty that your versioning problems will persist and no technology (new or old) will be able to solve them.

Coming next… Units of versioning – inside and outside a service.

Also, “passing reference data in a message makes the message-consuming systems dependent on the knowledge and availability of actual persistent data that is stored “somewhere”. This data must separately be accessed for the sake of understanding the event that is represented by the message.”

The discussion of self-contained events can be compared to integration databases vs application databases.

Centralized Integration – Pros & Cons

If everything in a system can access a central datastore, it is enough for one party to publish an event containing only the ID of an entity that that party previously entered/updated. Upon receiving that event, a subscriber would go to the central datastore and get the fields its interested in for that ID. The advantage of this approach is that the minimal amount of data necessary crosses the network, as subscribers only retrieve the fields that interest them. Martin Fowler describes the disadvantages as:

“An integration database needs a schema that takes all its client applications into account. The resulting schema is either more general, more complex or both. The database usually is controlled by a separate group to the applications and database changes are more complex because they have to be negotiated between the database group and the various applications.”

This is far from being aligned with the principle of autonomy so important to SOA. In that respect, the architectural principle of self-contained messages points us away from those problems and towards more autonomous services.

However, once we have these autonomous business services in place, we may find that we don’t need 100% fully self-contained messages anymore.

A Real-World Example

Let’s say we have 3 business services, Sales, Fulfillment, and Billing.

Sales publishes an OrderAccepted event when it accepts an order. That event contains all the order information.

Both Fulfillment and Billing are subscribed to this event, and thus receive it.

Fulfillment does not ship products to the customer until the customer has been billed, so it just stores the order information internally, and is done.

Billing starts the process of billing the customer for their order, possibly joining several orders into a single bill. After completing this process, it publishes a CustomerBilled event containing all billing information, as well as the IDs of the orders in that bill. It does not put all the order information in that event, as it is not the authoritative owner of that data.

When Fulfillment receives the CustomerBilled event, it uses the IDs of the orders contained in the event to find the order information it previously stored internally. It does not need to call the Sales service for this information or contact some central Master Data Management system. It uses the data it has, and goes about fulfilling the orders and shipping the products to the customer, finally publishing its own OrderShipped event.

Notice, as well, that in the original OrderAccepted event there were the IDs of products the customer ordered. These product IDs originated from another service, Merchandising, responsible for the product catalog. The same thing can be said for the customer ID originating from another service – Customer Care.

The Issue of Time

One could argue that since subscribers use previously cached data when processing new events, that data might not be up to date. Also, we may have race conditions between our services. In the above example, if Billing was extremely fast and more highly available than Fulfillment. Billing could have received the OrderAccepted event, processed it, and published the CustomerBilled event before Fulfillment had received the OrderAccepted event. In short, the CustomerBilled and OrderAccepted messages could be out of order in Fulfillment’s queue.

What would Fulfillment do when trying to process the CustomerBilled message when it doesn’t have the order information?

Well, it knows that the world is parallel and non-sequential, so it does NOT return/log an error, but rather puts that message in the back of the queue to be processed again later (or maybe in some other temporary holding area). This enables the OrderAccepted message to be processed before the CustomerBilled message is retried. When the retry occurs, well, everything’s OK – it’s worked itself out over time.

In the case where we retry again and again and things don’t work themselves out (maybe the OrderAccepted event was lost), we move that message off to a different queue for something else to resolve the conflict (maybe a person, maybe software). If/when the conflict is resolved (got the Sales system / messaging system to replay the OrderAccepted event), the conflict resolver returns the CustomerBilled message to the queue, and now everything works just fine.

As all of this is occurring, the only thing that’s visible to external parties is that it happens to be taking longer than usual for the OrderShipped event to be published. In other words, time is the only difference.

Summary

The problem of non-self-contained events is mitigated first and foremost by business services in SOA, and the apparent issue of time-synchronization by business logic inside these services.

Don’t be afraid to put IDs in your messages and events.

Do be afraid of using those IDs to access datastores shared by multiple “services”.

Using IDs to correlated current events to data from previous events is not only OK, it’s to be expected.

The architectural principle of fully self-contained messages steers us away from the problems of Integration Databases and towards Application Databases, autonomous services, and a better SOA implementation. From there, following the principle of autonomy from a business perspective, will lead us to services not publishing data in their messages that is owned by other services, taking us the next step of our journey to SOA.

Related Content

[Podcast] Message Ordering – Is it cost effective?

Don’t EDA between existing systems

[Podcast] Handling dependencies between subscribers in SOA

Durable Messaging

In order to prevent a message from being lost when it is sent from a publisher to a subscriber, the message is written to disk on the publisher side, and then forwarded to the subscriber, where it is also written to disk. This store-and-forward mechanism enables our systems to gracefully recover from either side being temporarily unavailable.

In my MSDN article on this topic, I outlined some problems with this approach. These problems are exacerbated for publishers. Imagine a publisher with 40 subscribers, publishing 10 messages a second, each containing 1MB of XML. If 10 of the subscribers are unavailable, that’s 100MB of data being written to the publisher’s disk every second, 6GB every minute. That’s liable to bring down a publisher before an administrator brews a cup of coffee.

Publishers have no choice but to throw away messages after a certain period of time.

Publisher Contracts

The whole issue of contracts and schema is considered one of the better understand parts of SOA. Unfortunately, the operational aspects of service contracts is hardly ever taken into account.

On top of the schema of the messages a service publishers, additional information is needed in the contract:

1. How big will this message be?
2. How often will it be published?
3. How long will this message be stored if a subscriber is unavailable?

This first two pieces of information are important for subscribers to do load and capacity planning. The last one is the most important as it dictates the required availability and fault-tolerance characteristic of subscribers.

For Example

In the canonical retail scenario, when our sales service accepts an order, it publishes an order accepted event. Other services subscribed to this event include shipping, billing, and business intelligence.

While shipping and billing are highly available and able to keep up with the rate at which orders are accepted, the business intelligence service is not. BI has two main parts to it – a nightly batch that does the number crunching, and a UI for reporting off of the results of that number crunching. Some even do the reporting in a semi-offline fashion, emailing reports back to the user when they’re ready.

Furthermore, nobody’s going to invest in servers for making BI highly available.

And wasn’t the whole point of this publish/subscribe messaging to keep our services autonomous? That not all services have to have the same level uptime?

Houston, do we have a problem.?

Data Freshness

There is a glimmer of light in all this doom and gloom.

Not all services have the same data freshness requirements.

The business intelligence service above doesn’t need to know about orders the second they’re accepted. A daily roll-up would be fine, and an hourly roll-up bring us that much closer to “real time business intelligence”.

So, while BI is ready to accept the sales message schema, it would like a slightly different contract around it – less messages per unit of time, more data in each message.

From the operational perspective of the sales service, it would be cost effective to have less “online” subscribers. It could even take things a few steps further. Instead of using the regular messaging backbone for transmitting these hourly messages, it could use FTP. The data could even be zipped to take up even less space. Since the total data size is less than the corresponding online stream, is stored on cheaper, large storage, and the number of subscribers for this zipped, hourly update is fairly small, these messages can be kept around far longer.

If you’ve heard about consumer-driven contracts, this is it.

Note that we’re still talking about the same logical message schema.

Summary

It’s not that lost notifications aren’t a problem.

It’s that they feed the design process in such a way that the resulting service ecosystem is set up in such a way that notifications won’t get lost. I know that that sounds kind of recursive, but that’s how it works. Either subscribers take care of their SLA allowing them to process the online stream of events, or they should subscribe to a different pipe (which will have different SLA requirements, but maybe they can deal with those).

It make sense to have multiple pipes for the same logical schema.

It’s practically a necessity to make pub/sub a feasible solution.

Related Content

MSDN article on messaging and lost messages

Durable messaging dilemmas

Additional logic required for service autonomy

More in depth example on events and pub/sub between services

Consumer-Driven Contracts

Dennis asked this:

There are different design choices in a SOA, even when you already have identified the services. I have a simple example that I would like to share:

Imagine a order-to-cash process. One part of that process is to register an order. Suppose we have two services, Order Service and Inventory Service. The task is to register the order and make a corresponding reservation of the stock level. I would be pleased to have the groups view on the following 3 design options (A, B, C):

A.
1. The “process/application” sends a message (sync or async) to “registerOrder” on the Order Service.
2. The “process/application” sends another message (sync or async) to “reserveStock” on the the Inventory Service.

B.
1. The “process/application” sends a message (sync or async) to “registerOrder” on the Order Service.
2. The Order Service sends a message (sync or async) to “reserveStock” on the the Inventory Service.

C.
1. The “process/application” sends a message (sync or async) to “registerOrder” on the Order Service.
2. The Order Service publishes an “orderReceived” event.
3. The Inventory Service subscribes to the “orderReceived” event .

On the whole “already identified the services” thing – naming a service doesn’t mean much. It’s all about allocating responsibility, and until that’s been done, those “services” don’t give us very much information.

Business Services

If we were to view this example in light of business services, and look at the business events that make up this process, maybe we’d get a different perspective.

Three business services: Sales, Inventory, and Shipping.

In Sales, many applications and people may operate, including the person and the application he used to submit the order. When the order is submitted and goes through all the internal validation stuff, Sales raises an OrderTentativelyAccepted event.

Inventory and Orders

Inventory, which is subscribed to this event, checks if it has everything in stock for the order. For every item in the order on stock, it allocates that stock to the order and publishes the InventoryAllocatedToOrder event for it. For items/quantities not in stock, it starts a long running process which watches for inventory changes.

When an InventoryChanged event occurs, it matches that against orders requiring allocation – if it finds one that requires stock, based on some logic to choose which order gets precedence, it publishes the InventoryAllocatedToOrder event.

Sales, which is subscribed to the InventoryAllocatedToOrder event, upon receiving all events pertaining to the order tentatively accepted, will publish an OrderAccepted event.

Orders and Shipping

When Inventory receives the OrderAccepted event, it generates the pick list to bring all the stock from the warehouses to the loading docks, finally publishing the PickListGenerated event containing target docks.

When Shipping receives the PickListGenerated event, it starts the yard management necessary to bring the needed kinds of trucks to the docks.

What else is possible

I could go on, talking about things like the maximum amount of time stock of various kinds can wait to be loaded on trucks, subscribing to earlier events to employ all kinds of optimization and prediction algorithms, having a Customer Care service notifying the customer about what’s going on with their order (probably different for different kinds of customers and preferred communication definitions). Obviously, we’d need a Billing service to handle the various kinds of billing procedures, whether or not the customer has credit, pays upon delivery, etc.

It turns out that many business domains map very well to this join of SOA and EDA.

What an ESB is for

When we have these kinds of business services primarily publishing events and subscribing to those of other services, you don’t need much else from your “enterprise service bus”. All sorts of transformation, routing, and orchestration capabilities don’t come into play at all.

In all truthfullness, those bits of functionality are really just a historical artifact of their broker heritage.

Don’t get me wrong, sometimes a broker is a nice thing to have – behind a service boundary in order to perform some complex integration between existing legacy applications.

Just keep that stuff in its place – not between services.

Complex Event Processing

We can look at how Sales transitions an order from being tentatively accepted to being accepted as requiring event correlation around InventoryAllocatedToOrder events. This isn’t exactly “complex” in its own right. If there were some kind of CEP engine that did this for us out of the box, it might be a possible technology choice for implementing this logic within our service.

As we add more concerns, like time, we may find new ways to make use of this engine. For instance, if the time to provide the order to the customer is approaching, we may choose to split the order into two – accepting one for which we have all the stock allocated, and leaving the second as tentatively accepted.

Summary

While it is difficult to move forward on service responsibility without discussing the events it raises and those it subscribe to, the whole issue of CEP can be postponed for a while.

Although there aren’t many who would say that EDA is necessary for driving down coupling in SOA, or that SOA won’t likely provide much value without EDA, or that SOA is necessary for providing the right boundaries for EDA, it’s been my experience that that is exactly the case.

CEP, while being a challenging engineering field, and managing the technical risks around it necessary for a project to succeed in some circumstances, and really shines when used under the SOA/EDA umbrella, it should not be taken by itself and used at the topmost architectural levels.

Related Content

SOA and Enterprise Processes

How client interaction fits with SOA

Time and SOA

Durable Messaging for Fault-Tolerant Services

And if you’re wondering about how to handle all that complexity inside services (different kinds of billing, periodic tests for electronics inventory, etc), you might like listening to this podcast about business components.

Watch that first step

Although sometimes described as the first step of an organization moving to SOA, web-service-izing everything results in synchronous, blocking, request/response interaction between services. The problem being that if one service were to become unavailable, all consumers of that service would not be able to perform any work. With the deep service “call stacks” this architectural style condones, the availability and performance of the entire organization will be dictated by the weakest link.

So, while I’d agree that many organizations do need to take this step, I’d caution against going into production at this step.

Pub/Sub Considered Helpful

When services interact with each other using publish/subscribe semantics we don’t have that technical problem of blocking. Subscribers cache the data published to them (either in memory or durably depending on their fault-tolerance requirements) thus enabling them to function and process requests even if the publisher is unavailable.

Consider the following scenario:

Let’s say we have an e-commerce site, a part of our Sales service responsible for selling products. Another service, let’s call it merchandising, is responsible for the catalog of products, and how much each product costs. Sales is subscribed to price update events published by Merchandising and saves (caches) those prices in its own database. When a customer orders some products on the site, Sales does not need to call Merchandising to get the price of the product and just uses the previously saved (cached) price. Thus, even if Merchandising is unavailable, Sales is able to accept orders. This is a big win as our merchandising application is not nearly as robust as our sales systems.

Yet, there are scenarios where data freshness requirements prevent this.

Too Much of a Good Thing?

Technically, the above story is accurate. There is nothing technically preventing Sales from accepting orders. Yet consider a scenario where Merchandising is down or unavailable for an extended period of time. While this may not be entirely likely for two servers in the same data center, consider physical kiosks which customers can use to buy products. Those kiosks may not receive updates for days. Should they accept orders?

That’s really a question to the business. If pricing data is stale for a time period greater than X, do not sell that item. The value of X may even be different for different kinds of products. Keep in mind that this issue only arose since we architected our services to be fully autonomous. In a synchronous systems architecture, this issue would not come up. As such, it is our responsibility as architects to go digging for these requirements as well as explaining to the business what the tradeoffs are.

In order to have more up to date data, we need to invest in more available hardware, networks, and infrastructure. This needs to be balanced against the predicted increase in revenue that more up to date (read higher) prices would give us.

You Can Get What You Pay For

Beyond the additional cost of writing that additional logic, and the perceived increased complexity, another difference to note between this architectural style and the synchronous/traditional one is that it puts control of spending back in the hands of business.

In a synchronous architecture, in order to achieve required performance and availability, all systems need to be performant requiring across the board investments in servers, networks, and storage. Without investing everywhere, the weakest link is liable to undo all other investments. In other words, your developers have made your investment choices for you. Scary, isn’t it.

A more prudent investment strategy would prefer spending on services that give the biggest bang for the buck, better known as return on investment. A pub/sub based architecture allows investing in data-freshness where it makes the most sense. For example, in sales of high profit products to strategic customers rather than inventory management of raw materials for products slated to be decommissioned.

That sounds a lot like IT-Business Alignment.

Maybe there’s something to this SOA thing after all…

Read more about:

7 Questions for Service Selection

7 Questions around data freshness 

Event-Driven Architecture and Legacy Applications

Autonomous Services and Enterprise Entity Aggregation

Or listen to a podcast describing Business Components, the connection of pub/sub and SOA.

“We have three high level business services: Engagement, Management, [and] Production. [...] they represent different operational ambitions. Engagement is all about quantity and then filtering. Management is about the quality and Production is about realising the benefits.”

Services are not about “are you being served?”

They’re not about re-use, and barely about use. Events are what it’s all about.

Each service has its own responsibility and does what it needs to do, business-wise, to achieve its goals. Whether it’s about increasing the number of leads, ensuring high-profile clients get good service, or maximizing equipment utilization, services take responsibility.

I know I harp on this a lot.

It’s because it’s that important.

In regular client-server development, the server is responsible for providing the client with all CRUD (create, read, update, and delete) capabilities. However, when users look at data they do not often require it to be up to date to the second (given that they often look at the same screen for several seconds to minutes at a time). As such, retrieving data from the same table as that being used for highly consistent transaction processing creates contention resulting in poor performance for all CRUD actions under higher load.

A Scalable Solution

One of the common answers to this question is for the server/service to publish a message when data changes (say, as the result of processing a message) and for clients to subscribe to these messages. When such a notification arrives at a client, the client would cache the data it needs. Then, when the user wants to see a grid of data, that data is already on the client. Of course, this solution doesn’t work so well for older client machines (like some point of service devices) or if there are millions of rows of data.

The thing is that this solution is one implementation of a more general pattern – command query separation (CQS).

Command Query Separation

Wikipedia describes CQS as a pattern where "… every method should either be a command that performs an action, or a query that returns data to the caller, but not both. More formally, methods should return a value only if they are referentially transparent and hence possess no side effects."

Martin Fowler is less strict about the use of CQS allowing for exceptions: "Popping a stack is a good example of a modifier that modifies state. Meyer correctly says that you can avoid having this method, but it is a useful idiom. So I prefer to follow this principle when I can, but I’m prepared to break it to get my pop."

So, how does separating commands from queries and SOA help at all in getting data to and from a UI? The answer is based on Pat Helland’s thinking as described in his article Data on the Inside vs. Data on the Outside.

Services Cross Boxes

The biggest lie around SOA is that services run.

Let that sink in a second.

Sure services have runnable components, but that’s not why they’re important.

I’ll skip the books of background and cut to the chase:

Services communicate with each other using publish/subscribe and one-way messaging. Services have components inside them. Inside a service, these components can communicate with each using synchronous RPC, or any other mechanism. Also, these components can reside on different machines.

This is broader than just scaling out a service. There can be service components running on the client as well as the server.

SOA & CQS

Combining these two concepts together, here’s what comes out:

In this solution there are two services that span both client and server – one in charge of commands (create, update, delete), the other in charge of queries (read). These services communicate only via messages – one cannot access the database of the other.

The command service publishes messages about changes to data, to which the query service subscribes. When the query service receives such notifications, it saves the data in its own data store which may well have a different schema (optimized for queries like a star schema).

The client component which is in charge of showing grids of data to the user behaves the same as it would in a regular layered/tiered architecture, using synchronous blocking request/response to get its data – SOA doesn’t change that.

Composite Applications

Although the client side components of both the command and query services are hosted in the same process, they are very much independent of each other. That being said, from an interoperability perspective (the one that most people attribute to SOA), all of the client-side components will likely be developed using the same technology – although there are already ways to host Java code in .NET and vice-versa.

Of course, once we talk about web UI’s things are a bit different – but still similar. While web-server-side there may be a level of independence, for browser side inter-component communications we’re still likely to target javascript. There, I’ve managed to say something technical supporting mashups and SOA without lying through my teeth.

On the Microsoft side with the recent release of the Composite Application Guidance & Library (pronounced "prism") I hope that more of these principles will be reaching the "smart client". The command pattern is especially critical in maintaining the separation while enabling communication to still occur so I’m glad that, as one of the Prism advisors, I was able to simplify that part (Glenn still has nightmares about that rooftop conversation).

Publish / Subscribe

In the "scalable solution" section up top I mentioned how publish/subscribe to the smart client is really just one implementation of CQS and SOA. So, how different is it really?

Well, there will probably be a different technology mapping. Instead of a star-schema OLAP product, we might simply store the published data in memory on the client. That is, if you designed your components to be technology agnostic.

In terms of the use of nServiceBus, the same component is going to be subscribing to the same type of message – all that’s different is that now every client will be having data pushed to them rather than this occurring server-side only.

You could have the same code deployed differently in the same system – stronger clients subscribing themselves, weaker ones using a remote server. Web servers would probably be considered stronger clients. This kind of flexible deployment has proven to be extremely valuable for my larger clients. The added benefit of enabling users to work (view data) even while offline (somewhere there’s no WIFI) is just icing on the cake.

A Word of Warning

Once the client starts receiving notifications, and handling those on a background thread (as it should) the code becomes susceptible to deadlocks and data races. Juval does a good job of outlining some of those with respect to the use of WCF. Prism doesn’t provide any assurances in this area either.

Summary

NServiceBus is not designed to be used for any and all types of communication in a given architecture. In the examples above, nServiceBus handles the publish/subscribe but leaves the synchronous RPC to existing solutions like WCF. Not only that, but synchronous RPC does have its place in architecture, just not across service boundaries. In all cases, data is served to users from a store different from that which transaction processing logic uses.

Command Query Separation is not only a good idea at the method/class level but has advantages at the SOA/System level as well – yet another good idea from 20 years ago that services build upon. Making use of CQS requires understanding your data and its uses – SOA builds on that by looking into data volatility and the freshness business requirements around it.

Finally, designing the components of your services in such a way that their dependency on technology is limited buys a lot of flexibility in terms of deployment and, consequently, significant performance and scalability gains.

Simple, it is. Easy, it is not.

This innocuous question comes up a lot. Usually I get this question after a short problem domain description. One of these came up on the nServiceBus discussion groups. Ayende took it and ran with it turning it into a nice blog post, An exercise in designing SOA systems. I’ve been meaning to write something myself. Bill put up a response already in his Service Granularity Example. So, I’m late to the party, again, but here we go.

It’s almost impossible to know, right away, which services are appropriate.

So, I’m going to focus more on the process of getting there, rather than describing the solution itself.

The domain deals with a placement agency placing physicians in positions at hospitals.

1. So, what does it actually do?

In Ayende’s post, he describes several services, but I’d rather look at them as use cases: registering an open position, registering a candidate, verifying their credentials, etc. It’s worth going through this requirements process. It doesn’t necessarily translate immediately to services, but there’s value in it.

2. What does it do it to?

We should also be looking at the data model, an entity relationship diagram (ERD) , where we see that we may have placed a certain physician at a number of positions. It’s also important for us to know about under which circumstances a physician finished their employment at a previous position before, say, trying to place them at a position in the same hospital or chain of hospitals. Don’t go thinking that this what the database schema will look like, it’s all about understanding connections between various bits of data.

3. When does that happen?

The next step is to map the uses cases above to the entities in the ERD, which entity is used in which use case. It’s also important to differentiate between entities (or even more importantly, specific fields of entities) that are used in a read-only fashion within a given use case. For instance, when registering a new position, we’ll want to check that against other open positions in the same hospital so we don’t end up registering the same position twice. Also, we might want to suggest verified physicians whose credentials match the position’s requirements. Data we wouldn’t be interested in might be which other physicians we placed at that hospital.

4. What just happened?

Another valuable perspective on the problem domain is the business process view – what are the interesting business events in the system and how they unfold over time. For instance, physician registered, position opened, physician’s credentials verified, and physician placed in position (or position filled by physician) are events that describe a different business perspective than use cases.

5. How do I decide?

Once we know what events there are, we can start looking at what kind of decisions we might want to make when those events occur and what data we’d need to make those decisions. These decisions may be as simple as updating a database or sending an email to a user. They also may include more advanced logic like when the profitability of an agreement with a specific hospital chain changes, prefer placing physicians in positions in that chain over others.

6. How do I deal with all this information?

After we have all of this information, we can start looking for cohesive bunching across all of these axes using these rules:

• Data that is modified by a use case gets published as an event.
• Data that is required by a use case for read-only purposes, arrives as the result of subscribing to some event.

Look for rules that differentiate behaviour based on the properties of data. Look for a correlation to some business concept. For instance, physicians probably won’t be changing their specialization, and open positions often deal with a certain specialization. Therefore, specific data instances tied to two different specializations can be said to be loosely coupled.

7. Which property slices across the domain?

Even though the ERD may not have made it clear, and the use cases didn’t show any particular break-down, nor did the events call out this point, the key to finding the way a business domain decomposes into services lies in decoupling specific data instances.

Actually, at this point we can clump autonomous components (mere technical bits) that handle a single message, into more granular business components.

If you think about it, it makes a lot of sense. The kind of credential checking you’d do for physicians specializing in brain surgery would likely be different than for general practitioners. The kind of information you’d store would, therefore, also be different.

But, which services do I need?

Quite frankly, I don’t have enough information to know.

But if we had continued this conversation, going through issues like transactional consistency, availability requirements, and other non-functional issues we could have  gotten there.

If there’s one thing that I hope you got out of this, it’s that the questions are what’s important. The iterative process of looking at the problem domain from various perspectives, incorporating the new-found knowledge, and asking more questions is what leads us to a solution. But we don’t stop there. We keep looking for characteristics which split services apart into business components, and for consistency requirements that brings autonomous components together into services.

It’s not easy, but by focusing on these simple questions, you can get to a coherent service oriented architecture.

Jim brought the concept of cohesion to the regular SOA discussions around loose coupling in his post Anemic Service Model, which I think, all in all, is a very good idea.

Naïve Service Composition

Jim first calls out a common anti-pattern that seems to have become quite rampant – I’d call it naïve service composition if only the things being composed could even be called services. And I think the tone being set is correct – a service needs to meet a stronger set of criteria than just being able to be composed. Multiple services sharing the same logical data store (in that the same actual rows/data elements are managed by multiple services) probably means there’s an encapsulation problem here. I agree with Jim sentiment here:

“On the one hand we’re inclined, and indeed encouraged by the SOA brigade, to think of this architecture as a good fit for purpose because it is very loosely coupled. Since every component or service is decoupled from every other component or service it should be possible to arrange and re-arrange them in a Lego-style in a myriad of useful ways. Building out “business services” from some more fundamental set of services is how the books tell us to do it. In fact we could even do that quite easily with point-and-[click] BPM tools, ruling out such overheads as developers and change management along the way. Right?”

MVC? There are, like, 6 of them!

However, I disagree with some of the conclusions that Jim draws from that point. Jim states “build your services to implement business processes”, and that services are “just an instance of MVC”. I’m going to leave alone the MVC statement since there are like 6 documented kinds of MVC not including the Front Controller stuff that the web guys are now calling MVC. I’m going to focus on the business process advice. JJ also doesn’t seem to agree with this advice. As Savas has already taken issue with the tone of JJ’s response, I’ll keep my focus on the content.

Visual Cobol

First of all, in my previous conversations with Jim he had already denounced the procedural nature of composing higher-level business processes out of smaller services which implement small bits of common activities. Visual Cobol was how he described it. In JJ’s follow-up post, he called out the necessary aspect of autonomy that jives with Jim’s cohesion principle.

I’m a bit concerned about the way JJ tends to version what SOA means over time. It might make it impossible to have intelligent design discussions without tagging each sentence with “as SOA meant in 2006″. I acknowledge that the accepted meaning of SOA by various vendors has changed over the years. However, I’ve found that meanings rooted in decades of computer science tend to last and provide value that outlasts much of the industry-buzzword-bingo (SOA 2.0 anyone?).

Cohesion, Business Domains, and Business Processes

My view of the original cohesion principles Steve discusses in his 2005 article Old Measures for New Services takes a business spin to Functional Cohesion:

A service should be responsible for one business domain.

If we jump off from this point, we’ll see that certain business processes which occur entirely in one business domain are fully encapsulated, whereas those macro-processes which cross many domains (like Order to Cash) cross multiple services – they do not become a service since that would break the “one business domain” rule. Given that services are loosely coupled, avoiding temporal coupling leads to services raising events. Thus, macro-processes are really just a series of events of various services where each service does its own internal business processes.

Enterprise Processes >> Business Processes

I think that maybe some of the difficulty in discussing concrete SOA guidance has to do with granularity. I’ve started calling those macro-processes something different from business processes, and that may just bring me full circle to Jim’s guidance.

An Enterprise Process is any process which involves multiple business domains.

Under that definition, a service may be responsible for multiple business processes in the same business domain. But still, one business process is usually not a service by itself.

Business Components & Autonomous Components to the Rescue

Finally, by introducing the additional levels of decomposition of business components and autonomous components I’ve found that we can focus the discourse on one concern at a time. My presentation on the topic can be found here. The 30 second pitch is this:

Business domains are inherently partitionable – data and rules. A business component represents one partition. An example of this is the domain of Sales being partitioned by strategic and non-strategic customers. Although the data structure might be similar or the same, the actual rows/data elements are not shared. Rules around discounts are different.

Within a business component, different activities should not interfere with each other. An autonomous component represents one activity. In our example, reporting on orders from strategic customers should not interfere with accepting their orders. As such, those activities should have different messages coming in on different endpoints. Each endpoint could have different characteristics, like durability. Losing a request for a report when a server restarts isn’t a big deal, however not a good idea for orders.

For more information you could check out these episodes from my podcast:

Business and Autonomous Components in SOA

Using Autonomous Components for SLAs in SOA

Questions and comments are always welcome.

Typically, in such an environment [SOA], services tend to be parts of multiple systems. For example, consider a Products service! It might start its life as part of a sales system. Later it might be involved in a purchasing system, a product development system, a marketing system, a warehousing system, and perhaps in several other systems too. This process may take years, and it really never ends. The service is the same, but its responsibilities and its external exposure are increased with each system it’s enrolled in.

One of the core tenets of SOA that all vendors and analysts agree upon is that there should be loose coupling between services. If you were to design such a product service, it’s clear that changing part of its interface could break almost every system in the enterprise. That doesn’t sound like loose coupling to me.

If there’s one place that is the source of loose coupling – it’s the business. Warehousing is viewed by the business as being fairly independent of Marketing. While Sales might make use of data created in Product Development, business wouldn’t want any problems in IT related to Product Development to inhibit Sales ability to accept orders. That is another kind of loose coupling – the ability of one service to make use of “not-accurate-up-to-the-millisecond” data created by another service. That’s known as loose “temporal” coupling, as in loose coupling in the dimension of time.

Loosely-Coupled Services

So, in the example described we’d see the following services:

• Sales
• Purchasing
• Product Development
• Marketing
• Warehousing / Inventory

Product data would flow between the services but each would have a very different internal view of it.

• Product Development would be more interested in managing the schedule and risk around a product’s development.
• Marketing would probably be more focused on its relation to competing products and pricing.
• Purchasing would be maintaining data as to which suppliers are being used to supply raw materials for the production of the product.
• Sales would be looking at actually accepting orders and giving discounts.
• Warehousing would be focused on the storage and transportation aspects of the product.

As you can see, there is very little overlap in the data between these services even on something similar like product data. The logic of each service around the management of its data would be even more different. This leads to services with a high level of autonomy.

There Be Dragons…

Without starting at this business-level loose coupling, I doubt that any technical effort will succeed. That is to say every time I’ve seen this style implemented it has failed, but that’s no proof. Conversely, every time that we did start our SOA efforts by identifying the clear business fracture lines, we were able to maintain loose coupling all the way down. That is not to say that it always will succeed, but the logic is sound.

I suppose that the difference between my view on SOA and Sundblad’s stems from the fact that they put systems at a higher level of abstraction than services, and I put services on top. Regardless, I do agree with their views about architecture and engineering and consider them quite valuable.

Here’s the core list from Joe Armstrong’s book, Programming Erlang:

“In Erlang:

• Creating and destroying processes is very fast.
• Sending messages between processes is very fast.
• Processess behave the same way on all operating systems.
• We can have very large numbers of processes.
• Processes share no memory and are completely independent.
• The only way for processes to interact is through message passing.”

In nServiceBus, we don’t create or destroy processes – that’s a Windows issue. Instead, we just do messaging with endpoints. If there’s a process behind that endpoint, and it responds, then other interesting things can occur.

In the continued list:

• Message passing is asynchronous.
• Processes can monitor each other.
• It is possible to selectively receive messages.
• Remote processes appear largely the same as local processes.

All of this is part of the design philosophy of nServiceBus. While I have yet to see a carrier-grade implementation of nServiceBus, we are enjoying very impressive system-wide uptimes in production. Oh, and the programming model is still plain-old .NET, so you don’t have to learn any new languages or environments (even though I think that you might learn something – I know I did).

Not so.

The other day I sat down to implement and test a non-trivial business process, and the testing was far from easy. Now as developers go, I’m not great, or an expert on unit testing or TDD, but I’m above average. It should not have been this hard. And I tried doing it with Rhino.Mocks, TypeMock, and finally Moq. It seemed like I was in a no-mans-land, between trying to do state-based testing, and setting expectations on the messages being sent (as well as correct values in those messages), nothing flowed.

Until I finally stopped trying to figure out how to test, and focused on what needed to be tested. I mean, it’s not like I was trying to build a generic mocking framework like Daniel.

Here’s an example business process, or actually, part of one, and then we’ll see how that can be tested. By the way, there will be a post coming soon which describes how we go about analysing a system, coming up with these message types, and how these sagas come into being, so stay tuned. Either that, or just come to my tutorial at QCon.

On with the process:

1. When we receive a CreateOrderMessage, whose “Completed” flag is true, we’ll send 2 AuthorizationRequestMessages to internal systems (for managers to authorize the order), one OrderStatusUpdatedMessage to the caller with a status “Received”, and a TimeoutMessage to the TimeoutManager requesting to be notified – so that the process doesn’t get stuck if one or both messages don’t get a response.

2. When we receive the first AuthorizationResponseMessage, we notify the initiator of the Order by sending them a OrderStatusUpdatedMessage with a status “Authorized1”.

3. When we get “timed out” from the TimeoutManager, we check if at least one AuthorizationResponseMessage has arrived, and if so, publish an OrderAcceptedMessage, and notify the initator (again via the OrderStatusUpdatedMessage) this time with a status of “Accepted”.

And here’s the test:

    public class OrderSagaTests
    {
        private OrderSaga orderSaga = null;
        private string timeoutAddress;
        private Saga Saga;     

        [SetUp]
        public void Setup()
        {
            timeoutAddress = "timeout";
            Saga = Saga.Test(out orderSaga, timeoutAddress);
        }     

        [Test]
        public void OrderProcessingShouldCompleteAfterOneAuthorizationAndOneTimeout()
        {
            Guid externalOrderId = Guid.NewGuid();
            Guid customerId = Guid.NewGuid();
            string clientAddress = "client";     

            CreateOrderMessage createOrderMsg = new CreateOrderMessage();
            createOrderMsg.OrderId = externalOrderId;
            createOrderMsg.CustomerId = customerId;
            createOrderMsg.Products = new List<Guid>(new Guid[] { Guid.NewGuid() });
            createOrderMsg.Amounts = new List<float>(new float[] { 10.0F });
            createOrderMsg.Completed = true;     

            TimeoutMessage timeoutMessage = null;     

            Saga.WhenReceivesMessageFrom(clientAddress)
                .ExpectSend<AuthorizeOrderRequestMessage>(
                    delegate(AuthorizeOrderRequestMessage m)
                    {
                        return m.SagaId == orderSaga.Id;
                    })
                .ExpectSend<AuthorizeOrderRequestMessage>(
                    delegate(AuthorizeOrderRequestMessage m)
                    {
                        return m.SagaId == orderSaga.Id;
                    })
                .ExpectSendToDestination<OrderStatusUpdatedMessage>(
                    delegate(string destination, OrderStatusUpdatedMessage m)
                    {
                        return m.OrderId == externalOrderId && destination == clientAddress;
                    })
                .ExpectSendToDestination<TimeoutMessage>(
                    delegate(string destination, TimeoutMessage m)
                    {
                        timeoutMessage = m;
                        return m.SagaId == orderSaga.Id && destination == timeoutAddress;
                    })
                .When(delegate { orderSaga.Handle(createOrderMsg); });     

            Assert.IsFalse(orderSaga.Completed);     

            AuthorizeOrderResponseMessage response = new AuthorizeOrderResponseMessage();
            response.ManagerId = Guid.NewGuid();
            response.Authorized = true;
            response.SagaId = orderSaga.Id;     

            Saga.ExpectSendToDestination<OrderStatusUpdatedMessage>(
                    delegate(string destination, OrderStatusUpdatedMessage m)
                    {
                        return (destination == clientAddress &&
                                m.OrderId == externalOrderId &&
                                m.Status == OrderStatus.Authorized1);
                    })
                .When(delegate { orderSaga.Handle(response); });     

            Assert.IsFalse(orderSaga.Completed);     

            Saga.ExpectSendToDestination<OrderStatusUpdatedMessage>(
                    delegate(string destination, OrderStatusUpdatedMessage m)
                    {
                        return (destination == clientAddress &&
                                m.OrderId == externalOrderId &&
                                m.Status == OrderStatus.Accepted);
                    })
                .ExpectPublish<OrderAcceptedMessage>(
                    delegate(OrderAcceptedMessage m)
                    {
                        return (m.CustomerId == customerId);
                    })
                .When(delegate { orderSaga.Timeout(timeoutMessage.State); });     

            Assert.IsTrue(orderSaga.Completed);
        }
    }

You might notice that this style is a bit similar to the fluent testing found in Rhino Mocks. That’s not coincidence. It actually makes use of Rhino Mocks internally. The thing that I discovered was that in order to test these sagas, you don’t need to actually see a mocking framework. All you should have to do is express how messages get sent, and under what criteria those messages are valid.

If you’re wondering what the OrderSaga looks like, you can find the code right here. It’s not a complete business process implementation, but its enough to understand how one would look like:

using System;
using System.Collections.Generic;
using ExternalOrderMessages;
using NServiceBus.Saga;
using NServiceBus;
using InternalOrderMessages;     

namespace ProcessingLogic
{
    [Serializable]
    public class OrderSaga : ISaga<CreateOrderMessage>,
        ISaga<AuthorizeOrderResponseMessage>,
        ISaga<CancelOrderMessage>
    {
        #region config info     

        [NonSerialized]
        private IBus bus;
        public IBus Bus
        {
            set { this.bus = value; }
        }     

        [NonSerialized]
        private Reminder reminder;
        public Reminder Reminder
        {
            set { this.reminder = value; }
        }     

        #endregion     

        private Guid id;
        private bool completed;
        public string clientAddress;
        public Guid externalOrderId;
        public int numberOfPendingAuthorizations = 2;
        public List<CreateOrderMessage> orderItems = new List<CreateOrderMessage>();     

        public void Handle(CreateOrderMessage message)
        {
            this.clientAddress = this.bus.SourceOfMessageBeingHandled;
            this.externalOrderId = message.OrderId;     

            this.orderItems.Add(message);     

            if (message.Completed)
            {
                for (int i = 0; i < this.numberOfPendingAuthorizations; i++)
                {
                    AuthorizeOrderRequestMessage req = new AuthorizeOrderRequestMessage();
                    req.SagaId = this.id;
                    req.OrderData = orderItems;     

                    this.bus.Send(req);
                }
            }     

            this.SendUpdate(OrderStatus.Recieved);     

            this.reminder.ExpireIn(message.ProvideBy - DateTime.Now, this, null);
        }     

        public void Timeout(object state)
        {
            if (this.numberOfPendingAuthorizations <= 1)
                this.Complete();
        }     

        public Guid Id
        {
            get { return id; }
            set { id = value; }
        }     

        public bool Completed
        {
            get { return completed; }
        }     

        public void Handle(AuthorizeOrderResponseMessage message)
        {
            if (message.Authorized)
            {
                this.numberOfPendingAuthorizations--;     

                if (this.numberOfPendingAuthorizations == 1)
                    this.SendUpdate(OrderStatus.Authorized1);
                else
                {
                    this.SendUpdate(OrderStatus.Authorized2);
                    this.Complete();
                }
            }
            else
            {
                this.SendUpdate(OrderStatus.Rejected);
                this.Complete();
            }
        }     

        public void Handle(CancelOrderMessage message)
        {     

        }     

        private void SendUpdate(OrderStatus status)
        {
            OrderStatusUpdatedMessage update = new OrderStatusUpdatedMessage();
            update.OrderId = this.externalOrderId;
            update.Status = status;     

            this.bus.Send(this.clientAddress, update);
        }     

        private void Complete()
        {
            this.completed = true;     

            this.SendUpdate(OrderStatus.Accepted);     

            OrderAcceptedMessage accepted = new OrderAcceptedMessage();
            accepted.Products = new List<Guid>(this.orderItems.Count);
            accepted.Amounts = new List<float>(this.orderItems.Count);     

            this.orderItems.ForEach(delegate(CreateOrderMessage m)
                                        {
                                            accepted.Products.AddRange(m.Products);
                                            accepted.Amounts.AddRange(m.Amounts);
                                            accepted.CustomerId = m.CustomerId;
                                        });     

            this.bus.Publish(accepted);
        }
    }
}

All this code is online in the subversion repository under /Samples/Saga.

Questions, comments, and general thoughts are always appreciated.

Our long time listener Bill asks:

Hi Udi,

I have a question around processing of messages in proper order. When leveraging multiple threads to process messages in a message queue, it is possible for the second message in the queue to get processed before the first – especially if the first message is considerably larger than the second. I have taken a lot of care to make sure that messages are sent in the correct order, only to find that the receiving system can process them out of order anyway.

Consider a Policy Created notification, which must come before a Policy Approved notification. If both messages are sitting in the queue when the receiving service starts up, the approval message can be processed before the creation message. How can I make sure that message ordering is respected by the receiving system? I am using WCF/MSMQ as the underlying transport by the way. The only way I have found so far is to limit the receiving service to a single thread, which is by no means desirable.

Best Regards,

Bill

Download

Download via the Dr. Dobb’s site

Or download directly here.

Additional References

• Blog post: In-Order Messaging a Myth?
• Blog post: Handling Messages out of Order

Want more?

Check out the “Ask Udi” archives.

Got a question?

Send Udi your question to answer on the show.

The Setup

Just so that the example is in itself secure, we’ll assume that the password is one-way hashed before being stored. Also, given a reasonable network infrastructure our web servers will be isolated in the DMZ and will have to access some application server which, in turn, will communicate with the DB. There’s also a good chance for something like round-robin load-balancing between web servers, especially for things like user login.

Before diving into the meat of it, I wanted to preface with a few words. One of the commonalities I’ve found when people dismiss asynchrony is that they don’t consider a real deployment environment, or scaling up a solution to multiple servers, farms, or datacenters.

The Synchronous Solution

In the synchronous solution, each one of our web servers will be contacting the app server for each user login request. In other words, the load on the app server and, consequently, on the database server will be proportional to the number of logins. One property of this load is its data locality, or rather, the lack of it. Given that user U logged in, the DB won’t necessarily gain any performance benefits by loading all username/password data into memory for the same page as user U. Another property is that this data is very non-volatile – it doesn’t change that often.

I won’t go to far into the synchronous solution since its been analysed numerous times before. The bottom line is that the database is the bottleneck. You could use sharding solutions. Many of the large sites have numerous read-only databases for this kind of data, with one master for updates – replicating out to the read-only replicas. That’s great if you’re using a nice cheap database like mySql (of LAMP), not so nice if you’re running Oracle or MS Sql Server.

Regardless of what you’re doing in your data tier, you’re there. Wouldn’t it be nice to close the loop in the web servers? Even if you are using Apache, that’s going to be less iron, electricity, and cooling all around. That’s what the asynchronous solution is all about – capitalizing on the low cost of memory to save on other things.

The Asynchronous Solution

In the asynchronous solution, we cache username/hashed-password pairs in memory on our web servers, and authenticate against that. Let’s analyse how much memory that takes:

Usernames are usually 12 characters or less, but let’s take an average of 32 to be sure. Using Unicode we get to 64 bytes for the username. Hashed passwords can run between 256 and 512 bits depending on the algorithm, divide by 8 and you have 64 bytes. That’s about 128 bytes altogether. So we can safely cache 8 million of these with 1GB of memory per web server. If you’ve got a million users, first of all, good for you Second, that’s just 128 MB of memory – relatively nothing even for a cheap 2GB web server.

Also, consider the fact that when registering a new user we can check if such a username is already taken at the web server level. That doesn’t mean it won’t be checked again in the DB to account for concurrency issues, but that the load on the DB is further reduced. Other things to notice include no read-only replicas and no replication. Simple. Our web servers are the "replicas".

The Authentication Service

What makes it all work is the "Authentication Service" on the app server. This was always there in the synchronous solution. It is what used to field all the login requests from the web servers, and, of course, allowed them to register new users and all the regular stuff. The difference is that now it publishes a message when a new user is registered (or rather, is validated – all a part of the internal long-running workflow). It also allows subscribers to receive the list of all username/hashed-password pairs. It’s also quite likely that it would keep the same data in memory too.

The same message can be used to publish both single updates, and returning the full list when using NServiceBus. Let’s define the message:

And the message that the web server sends when it wants the full list:

And the code that the web server runs on startup looks like this (assuming constructor injection):

And the code that runs in the Authentication Service when the GetAllUsernamesMessage is received:

And the class on the web server that handles a UsernameInUseMessage when it arrives:

When the app server sends the full list, multiple objects of the type UsernameInUseMessage are sent in one physical message to that web server. However, the bus object that runs on the web server dispatches each of these logical messages one at a time to the message handler above.

So, when it comes time to actually authenticate a user, this the web page (or controller, if you’re doing MVC) would call:

When registering a new user, the web server would of course first check its cache, and then send a RegisterUserMessage that contained the username and the hashed password.

When the RegisterUserMessage arrives at the app server, a new long-running workflow is kicked off to handle the process:

That UsernameInUseMessage would eventually arrive at all the web servers subscribed.

Performance/Security Trade-Offs

When looking deeper into this workflow we realize that it could be implemented as two separate message handlers, and have the email address take the place of the workflow Id. The problem with this alternate, better performing solution has to do with security. By removing the dependence on the workflow Id, we’ve in essence stated that we’re willing to receive a UserValidatedMessage without having previously received the RegisterUserMessage.

Since the processing of the UserValidatedMessage is relatively expensive – writing to the DB and publishing messages to all web servers, a malicious user could perform a denial of service (DOS) attack without that many messages, thus flying under the radar of many detection systems. Spoofing a guid that would result in a valid workflow instance is much more difficult. Also, since workflow instances would probably be stored in some in-memory, replicated data grid the relative cost of a lookup would be quite small – small enough to avoid a DOS until a detection system picked it up.

Improved Bandwidth & Latency

The bottom line is that you’re getting much more out of your web tier this way, rather than hammering your data tier and having to scale it out much sooner. Also, notice that there is much less network traffic this way. Not such a big deal for usernames and passwords, but other scenarios built in the same way may need more data. Of course, the time it takes us to log a user in is much shorter as well since we don’t have to cross back and forth from the web server (in the DMZ) to the app server, to the db server.

The important thing to remember in this solution is doing pub/sub. NServiceBus merely provides a simple API for designing the system around pub/sub. And publishing is where you get the serious scalability. As you get more users, you’ll obviously need to get more web servers. The thing is that you probably won’t need more database servers just to handle logins. In this case, you also get lower latency per request since all work needed to be done can be done locally on the server that received the request.

ETags make it even better

For the more advanced crowd, I’ll wrap it up with the ETags. Since web servers do go down, and the cache will be cleared, what we can do is to write that cache to disk (probably in a background thread), and "tag" it with something that the server gave us along with the last UsernameInUseMessage we received. That way, when the web server comes back up, it can send that ETag along with its GetAllUsernamesMessage so that the app server will only send the changes that occurred since. This drives down network usage even more at the insignificant cost of some disk space on the web servers.

And in closing…

Even if you don’t have anything more than a single physical server today, and it acts as your web server and database server, this solution won’t slow things down. If anything, it’ll speed it up. Regardless, you’re much better prepared to scale out than before – no need to rip and replace your entire architecture just as you get 8 million Facebook users banging down your front door.

So, go check out NServiceBus and get the most out of your iron.

Since the service needs to perform all request processing in near-real-time, it caches all data from the DB in memory (yes, that’s a lot of memory). Since the service needs to handle multiple requests concurrently, we’re using multiple threads (so far, so good). The problem is that we don’t want the service to handle messages received until it’s finished caching everything. Also, we don’t want that check to show up in every message handler (important when you have lots of message types).

This is actually quite easy to do with NServiceBus. Here’s how:

Have a thread-safe class, let’s call it Loader, for the API to the caching. Something along the lines of:

If (!Loader.HasCachedEverything)
  Loader.CacheEverything();

Obviously, the Loader will have internal logic for checking if it has already started loading things from the DB, so that it won’t do the same thing twice.

OK, now on to the interesting stuff.

We’d like to have the above code run no matter which kind of message we’ve received, so we just write a “generic” message handler – which handles “IMessage” like so:

When the message handler calls “HandleCurrentMessageLater”, the bus puts the current message in the back of the queue. If you’ve configured a transactional transport, this will be safe even in the case of a server restart.

Also, notice the “CanContinue = false”. This tells the bus that the message should not be passed on to any other message handlers, even if there are those that are configured to handle it.

We’ll also package this class up by itself, keeping it separate from the core logic of the service – making it easier to version these cross cutting concerns and the service logic. Let’s put it in “CrossCuttingConcerns.dll”

The final thing needed in order to achieve the behavior described above is to configure this message handler to run before any other handler. This is done in the config file of the process, under the “bus” object, in the “MessageHandlerAssemblies” property like so:

        <property name="MessageHandlerAssemblies">
          <list>
            <value>CrossCuttingConcerns</value>
            <value>ServiceLogic</value>
          </list>
        </property>

This is similar to the way HttpHandlers are (were?) configured in IIS – the order of the handlers defines the order in which the bus dispatches messages to them.

And that’s it.

We’re done.

If you have any questions you’d like to ask about NServiceBus, please feel free to send them my way: Questions@NServiceBus.com.

And just in closing I’d like to say that I don’t necessarily think you should be creating stateful services, but that there’s a time and place for everything.

Our long-time listener Bill asks the following:

Hi Udi,

Thank you for your podcast clarifying the concept of autonomous components. It certainly cleared up a lot in terms of how you define an autonomous component and helped position the concept better in my mind.

I am considering building our Policy Administration service with 3 autonomous components as you previously suggested – one for each insurance product family. As you previously mentioned, these autonomous components could either share a database instance or have separate database instances, even though they may share schema (being a part of the same service).

What concerns me however is that there is some data that must be shared between these autonomous components. A perfect example is Region. If a new Region is defined somewhere in the enterprise, a RegionCreatedNotification is published onto the bus. If I have 3 autonomous components sharing the same database, is it best I just choose one to be the subscriber for this event? Or should I create a new separate queue to receive these notifications and put a message handler for it in a separate MessageHandlers assembly? Or should I subscribe all 3 autonomous components and have them each check if the Region has already been created in the database before inserting a new one?

Option (1) seems presumptuous because it assumes all 3 autonomous components are sharing the same database. Option (2) seems a bit strange because it is like creating a 4th autonomous component only to receive the RegionCreatedNotification messages. And Option (3) seems wasteful since there is redundant code and the system is doing extra unnecessary lookups. At the moment it looks like Option (2) is my best bet, but I’m very interested to get your take on it.

Another question is whether you would say that autonomous components should be hosted in different processes. Since I don’t need to run these autonomous components on separate servers at this stage, it would seem wasteful to create a service host for each component. I could always create separate service hosts at a later time if I needed to host one or more autonomous components on other servers.

And the final question I have is in terms of solution structure. I was contemplating the following structure:

ProductFamily1
———————-
Company.PolicyAdministration.ProductFamily1.Domain
Company.PolicyAdministration.ProductFamily1.MessageHandlers
Company.PolicyAdministration.ProductFamily1.Messages
Company.PolicyAdministration.ProductFamily1.Persistence
Company.PolicyAdministration.ProductFamily1.Persistence.Implementation
Company.PolicyAdministration.ProductFamily1.Etc…

ProductFamily2
———————-
Company.PolicyAdministration.ProductFamily2.Domain
Company.PolicyAdministration.ProductFamily2.MessageHandlers
Company.PolicyAdministration.ProductFamily2.Messages
Company.PolicyAdministration.ProductFamily2.Persistence
Company.PolicyAdministration.ProductFamily2.Persistence.Implementation
Company.PolicyAdministration.ProductFamily2.Etc…

ProductFamily3
———————-
Company.PolicyAdministration.ProductFamily3.Domain
Company.PolicyAdministration.ProductFamily3.MessageHandlers
Company.PolicyAdministration.ProductFamily3.Messages
Company.PolicyAdministration.ProductFamily3.Persistence
Company.PolicyAdministration.ProductFamily3.Persistence.Implementation
Company.PolicyAdministration.ProductFamily3.Etc…

Common
———————-
Company.PolicyAdministration.Common.Domain
Company.PolicyAdministration.Common.Persistence
Company.PolicyAdministration.Common.Messages
Company.PolicyAdministration.Common.ServiceHost
Company.PolicyAdministration.Common.Etc…

Do you have any thoughts on the above structure?

Once again thank you very much for your extremely valuable advice!

Best Regards,
Bill

Download via the Dr. Dobbs’ site.

Or download directly here.

Additional References:

• Ask Udi Podcast #017 – Using Autonomous Components for SLAs in SOA
• Ask Udi Podcast #020 – How to structure .NET solutions and components

Want More?

Check out the “Ask Udi” archives.

Got a question?

Send Udi your question and have him answer it on the show: podcast@UdiDahan.com.

It seems to me that nowadays we work more and more with data streams and not data sets. On a transaction-to-transaction basis, I think it’s an uncommon application that uses more data than can fit into several gigabytes of RAM (obvious exception: multimedia data).

While data stream processing is the heartbeat of many verticals, I’m seeing another trend there as well – the use of historical data as a part of that data stream processing. Some people have begun calling this Complex Event-Stream Processing (CEP), and the analysts are already beginning to eat it up. Regardless, the problem is that it is difficult to hold all historical data in memory so that when events arrive we can process them quickly.

So, my bottom line is that we’re being hit on multiple fronts – both the rate at which we need to process events and the amount of data required to process each event. Multiple cores help a bit, but probably not enough to discount scaling up to more machines. All this at the end of the day points out that we should not treat multiple cores any differently than multiple machines.

So, we either need languages to handle this (Erlang for one) or possibly frameworks (NServiceBus is my contribution). All I know is that Layered (Tiered) Architectures won’t cut it.

Let’s say I have a system to handle a call center for financial services or telco. When a customer calls on the phone and asks to be enrolled in “Heavily Advertised Program ABC,” there may need to be three or four systems that interact to make that real.

…

Harry asks me to consider using a ‘logical name’ of the receiver. The sender contacts a logical end point, the addressing infrastructure turns that into a physical end point, and we still have decoupling.

Honestly, I like it but I think it is insufficient. What if we need to contact 20 downstream systems in a complex workflow, but I don’t want a single “orchestration coordinator” to be a bottleneck (or single point of failure). I don’t want to hand the orchestration off from my app to a central orchestration hub.

Let me propose a different approach.

When we use SOA/EDA (the same thing as far as I’m concerned), the top-level building block used is the Service. A service may make use of a number of existing systems to perform its work. The business-level events that we publish (and subscribe to) are done by the service, not the existing systems.

If there’s any orchestration/workflow that needs to be done as a result of a service receiving an event, it is done entirely internal to that service. Inter-service orchestrations don’t really exist, as in there is no orchestration coordinator that is not in a service. And the orchestration coordinators within a service don’t touch other services’ back-end systems – if anything, they publish other business level events.

Be aware: when just starting out on an SOA, you’ll find that multiple Services make use of the same backend systems. This may be necessary, but not a desirable state to stay in for too long since it embodies the most insidious and invisible kind of inter-service coupling there is.

I want to go back to Nick’s original question:

So what if no one picks the message up? Is that an error?

The answer is mu.

If a service publishes a business-event (message) and no other services currently care, that’s fine. It’s not an error. Actually, you’d probably have some kind of infrastructure “queue” where messages that haven’t been received more than X time get sent to, so that the event isn’t “lost”. On the other hand, within a service – if an existing system sends out a message that needs to arrive at another system, and that message doesn’t arrive or isn’t picked up “in time”, that is an error.

This is one of the advantages SOA brings to the table in terms of EDA (again, the same as far as I’m concerned). You get simple messaging semantics between services, while within the “sphere of control” of a service you need, and more importantly can do more complex messaging and orchestration.

Bottom line: you need higher abstractions than your existing systems to employ EDA effectively.

You might also want to check out my podcast on this topic: SOA, ESB, and Events.

ESB’s are a lot about messaging and therefore a better name might be “Enterprise Data Bus”. It’s the asynchronous messaging that needs such an infrastructure with persistency and mediation facilities. All the WS-* standards are about messaging as well, leveraging the message itself to tell the infrastructure how it has to be handled.

I think WS-* will make it possible to have the ESB evolve from a vendor-product to a concept implemented in the operating systems an network devices that understand WS-*. Then you can leave the prefix “Enterprise” and we will be ready for an univeral asynchronous data bus over the Internet (or any other network you like). This will help breaking the current “services centric” idea of SOA into a “messages centric” perspective.

What are your thoughts?

Download via the Dr. Dobb’s site

Or download directly here.

Additional References:

• Podcast on if you need an ESB for SOA
• Podcast on Enterprise Service Discovery
• Blog post on ESBs as a global dataspace
• Blog post about what the brave new world of WS-* brings us

Want more? Check out the “Ask Udi” archives.

Got a question? Have Udi answer it on the podcast – podcast@UdiDahan.com

And the question that this answers is:

Hi Udi,

We’ve been having some discussions about how to implement our latest project using SOA and this is what we came up with:

Every activity is a method, which is broken into a class ( Think separation of concerns ), so we get high reusability.
On top of those classes we have decided to put a WCF layer, so you can expose the method as a web method. If an activity needs data, then it will access a entity service via WCF. To make the activities useful for the business we put them in a class which we have called processes ( a process layer). A process contains one or more activities and is able to call other processes and activities aren’t allowed to call processes or other activities. On top of each process we have decided to put a WCF so the UI can access them.

So it’s pretty close to what you wrote about in the Microsoft Architecture Journal except that we don’t have direct call to the entity services, we wrap it up in an activity before the call, which is wrapped in a WCF-host. Much like the definitions in Ontology and Taxonomy of Services in a Service-Oriented Architecture

I would love to hear your comments and thoughts about this architecture.

With thanks, Ingo

Download via the Dr. Dobbs’ site.

Or download directly here.

Additional References:

• Ontology and Taxonomy of Services in a Service-Oriented Architecture
• Udi’s Architecture Journal article on Autonomous Services
  
• Podcast on how to structure .NET solutions and components
• Blog post covering a discussion on Entity Services

Want More?

Check out the “Ask Udi” archives.

Got a question?

Send Udi your question and have him answer it on the show: podcast@UdiDahan.com.

Q: Could you tell you your thoughts or preference for a distributed or centralized ESB?

DON: there is no such thing as a centralized ESB.

This is the problem with a lot of the products that call themselves ESBs. They are centralized brokers which may be clustered for availability. But they are in no way an implementation of the Bus Architectural Pattern. Please check this before cutting a check to your vendor.

Also, understand that if you do security related things in your ESB, possibly as a part of your routing rules, that if the security infrastructure is centralized that means your ESB is too. Even if it really was distributed to begin with.

Buyer beware.

Download via the Dr. Dobbs’ site

Or download directly here.

And here’s the original question:

Hi Udi,

I thought I’d ask a question about an implementation that seems to be bouncing off my head. We have been thinking about services that can, at the business owners discretion based on SLA expectations, be synchronous, async but within a “human” timeframe ie, the gui could still wait and just fail gracefully if the response took too long, or async where the client must check back later.

Client abc is using application xyz and sends a message via the bus to do something. Logically, client abc is waiting for a response even though application xyz sent it asynchronously. Client abc may wait with the gui in front of them, with a progress bar running or a little ajaxy polling widget. They may wait for an email or an message on their web site for long running operations. How would you arrange that the response to this particular message would get routed to only client abc? If the answer is you shouldn’t think that way, then what would a subscriber to that message type, as shown in your example, do with the client abc’s response? Imagine a password reset example where the result is a temporary password but the service consults or updates multiple systems as a result of the request.

You also use a shared message type library. As we have been studying WCF, we have focused mostly on the features that would help us allow service consumers and suppliers move independently. We have been looking at message versioning and loose coupling techniques. We have settled on the idea that a shared message library would make our message too brittle, once again forcing each consumer to sync up with each service update. What is your take on that?

Philip

Additional Resources

• Joel Spolsky’s posts about how they do asynchronous callbacks over the Web
• Microsoft Biztalk Internet Service Bus for asynchronous callbacks over the Web
• Podcast on loose coupling