I think I’ve just about drove everybody crazy now with my apparent zigzagging on CQRS.

Some people heard about CQRS first from one of my presentations and got all excited about it. Then I did some blogging which further drove people to CQRS (as did Greg Young and some others). As CQRS was just about to hit its stride with the Early Adopters, I started pushing a more balanced view – CQRS not as an answer, but as one of many questions. More recently I’ve pushed more strongly back against CQRS saying that it should be used rarely.

So what’s the missing piece?

If you’re in the Domain-Driven Design camp (as many doing CQRS are), then it’s Bounded Contexts.

If you’re in the Event-Driven SOA camp (a much smaller camp to be sure), then it’s Services.

The problem is the naming, because the DDD guys have their kinds of services which do not fit the definition for Service of the Event-Driven SOA approach.

Let me propose the term Autonomous Business Component for the purposes of this blog post to describe that thing which is both a DDD Bounded Context (have the shared BC part of the acronym) and an SOA Autonomous Services. Resulting in the nice short form: ABC (and everyone knows you need to have a good acronym if you want something to catch on).

What does this have to do with CQRS?

Nothing just yet. Well, at least, nothing directly to do with CQRS.

Although some proponents of CQRS have stated that it can and should be used as the top-most architectural pattern, both myself and Greg Young (arguably the first two to talk about it and the two who ultimately collaborated on naming it – and now Google knows we didn’t means “cars”) always recommended it as a pattern to be used one level down.

Although Greg and I have had many long discussions on the topic and do agree very much about what the overall structure should look like, I’ll try to avoid putting words in his mouth from this point on.

Before talking more about ABCs, let’s discuss the principle upon which they rest: The Single Responsibility Principle (SRP).

What does SRP have to with CQRS?

Many developers are familiar with SRP and have seen good results from using it. What we’re going to do is take this principle to the next level.

In Object Orientation (OO), data is encapsulated in an object. A good object does not expose its data to other objects to do with as they wish. Rather, it exposes methods that other objects can invoke, and those methods operate on the internal data.

SRP would guide us to not have the same data exist in two objects. For example, if we saw the customer’s first name as an internal data member of two objects, we’d be right to question that kind of duplication and move to refactor it away. However, when we see two systems doing the exact same thing – somehow that gets excused.

“Of course we need to be able to see the customer’s first name in the front-end website as well as in the back-end fulfillment system. How could we NOT have the customer’s first name in both those code-bases?”

And there’s the catch.

Who said that a system should be a single code-base?

But what about integration?

Although many times we do need to integrate existing systems together, sometimes we have the ability to change those systems. More importantly, when going to create a new solution, we can avoid getting ourselves into the problems that integration tries to solve.

Integrating with a system that cannot be changed can be done also by composing multiple ABCs, but that’s a topic for another post.

It is better to think of integration as a necessary evil – kind of like regular expressions and multi-threading; things to be avoided unless absolutely necessary.

“If you have a problem that you decide to use a regular expression to solve, you now have 2 problems.” Or so the saying goes. With multi-threading, you have a non-deterministic number of problems to solve.

If you thought you had duplicate responsibilities with 2 systems operating on the same data, how will introducing a 3rd code base (also known as “integration”) help? Remember that Single Responsibility Principle – our goal is to get it down to one.

OK, so how do ABCs do that?

In order for us to get back into alignment with SRP, that would require us to have responsibility for a single piece of data exist in one code base. Note that SRP makes no statements about how many physical places a given code base can be deployed to. Nor does it state that only a single technology can be in play – code that emits HTML can be packaged at design time together with rich-client code in the same solution.

If an ABC is responsible for a piece of data, it is responsible for it everywhere, and forever. No other ABC should see that data. That data should not travel between ABCs via remote procedure call (RPC) or via publish/subscribe. It is the ultimate level of encapsulation – SRP applied at the highest level of granularity.

This results in systems which are the result of deploying the components of multiple ABCs to the same physical place. The ABC which owns the customer name would have the necessary web code to render it in the e-commerce front-end and in the shipping back-end for printing on labels. This would mean that practically every screen in any UI is a composite of widgets owned by their respective ABCs.

This is ultimately what keeps the complexity of each ABC’s code base to a minimum.

But why not just use CQRS as the top-level pattern? ABCs are weird.

Imagine trying to create a single denormalized view model for the entire Amazon.com product page – product name, price, inventory, editorial review, customer comments, other products that customers viewed, other products that customers bought, etc.

Pretty complex, right?

How much duplication would you have for the page shown after you add an item to a cart? Once again, you need to show other products that customers bought, their names, images, prices, and inventory.

And then on the home page – items you might be interested in, names, images, prices.

And that’s only in the front-end system.

It’s not just the duplication, but how complex the code is for each one.

Instead of the duplication that top-level CQRS would bring you, consider an ABC responsible for products names and images that has just about the same view model composed on each of the above screens. The same with another ABC responsible for price.

You may be thinking that this would result in more queries to get the data to show on a page, and you’d be right. But it isn’t necessarily a classical N+1 Select problem, as the queries are bounded to the number of ABCs. Secondly, consider the ability to have well-tuned caching at the granularity of an ABC – something that would be much more difficult when dealing with everything as a single monolithic view model. In short, not only will it not be a performance problem, often it will actually improve performance.

OK – that explains “everywhere”, what about “forever”?

Forever is where things get interesting – or more accurately, when they get interesting.

Let’s talk about things like invoices.

One of the requirements in this area is that immutability. If the customer’s name was Jane Smith when they made their purchase, it doesn’t matter that they’ve since changed their name to Jane Jones, the invoice should still show Jane Smith.

Often developers push these types of requirements on the data warehouse guys – that’s where history gets handled. The only thing is that if your ABC owns the customer’s name, then no other code base can deal with it. If it’s your data, you have to handle all historical representations of it.

On the one hand, this would seem to kill the data warehouse. On the other hand, it means that the principles of data warehouses are now core to every code-base.

This means you don’t ever delete data (see my previous blog post on the subject), and you definitely don’t overwrite it with an update – even if you think you’re in a simple CRUD domain. The only case where you can get away with traditional CRUD is if we’re talking about private data – data that is only ever acted on by a single actor.

This sounds like the collaboration you talk about with CQRS

It’s similar in principle but different in practice.

In a collaborative domain, an inherent property of the domain is that multiple actors operate in parallel on the same set of data. A reservation system for concerts would be a good example of a collaborative domain – everyone wants the “good seats” (although it might be better call that competitive rather than collaborative, it is effectively the same principle).

A customer’s name would not fall under that category. It isn’t an inherent property of the domain for multiple actors to operate on that data. While there can be multiple readers, one can easily enforce a single writer without any adverse effects. Doing that with a reservation system would cause the online system to behave as if users were lining up in front of a box office – not a desirable outcome.

Private data would be something like a user’s shopping cart. Until they make a purchase, that data doesn’t need to be visible anywhere. Here you could theoretically do simple CRUD – that is, until the business realizes that there’s extremely valuable information to be extracted from the historical record of things people do with their carts.

I think you’re ready to make your point, so just make it already

OK – so we now realize that Update and Delete don’t exist in their traditional form. Delete is really just a kind of update, and update is effectively an “upsert” – a combination of update and insert to retain history. This can be done by having ValidFrom and ValidTo columns for our data.

In which case, Create is really just a special case of Upsert, which looks like this:

UPDATE Something SET ValidTo = NOW() WHERE Id=@Id AND ValidTo = NULL; INSERT INTO Something SET { regular values }, Id=@Id, ValidTo = NULL;

And then we’d have 2 forms of Read – reading the current state (ValidTo = NULL), and reading history (ValidFrom <= Instant AND (ValidTo >= Instant OR ValidTo = NULL))

Here we don’t need fancy N-Tier architectures, data transfer objects, service layers, or domain models. A simple 2-Tier approach could probably suffice. We don’t need a task-based UI, events, denormalized view models, or any of that CQRS stuff. This was at the crux of my previous anti-CQRS post.

The only thing is that this is exactly CQRS.

Say what?

Have we not effectively separated the responsibility of commands/upserts and queries/reads?

As Greg Young has said before, “the creation of 2 objects where there previously was one”.

Effectively 2 paths through our ABC.

CQRS.

Let me give you a second to gather your thoughts.

*

You see, CQRS is an approach, a mind-set – not a cookie cutter solution. Frameworks that guide you to applying CQRS exactly the same way everywhere are taking you in the wrong direction. The fact is that you couldn’t possibly know what your Aggregate Roots were before you figured out how to break your system down into ABCs. Attempting to create commands and events for everything will make you overcomplicate your solution.

So the built-in history of this model is event-sourcing?

Well, it’s not event-sourcing in the sense that we don’t necessarily have events. It achieves many of the benefits of event-sourcing by giving us the full history of what happened.

On the whole issue of replaying events to fix bugs – that’s a bit problematic, logically, unless we have a closed system. A closed system is one that doesn’t interact with anything else – no other systems, no users, nothing. As such, closed systems aren’t that common.

In an open system, one with users, let’s say there was a bug. This bug could have caused the wrong data to be written and/or shown to users. As such, users could have submitted subsequent commands based on that erroneous data that they would not have submitted otherwise. There’s no way for us to know.

The problem with replaying events when we fix the bug is that we’re in essence rewriting history – making it as if the user didn’t see the wrong data. The only problem is that we can’t know which events not to replay – we can’t automatically come up with the right events that should have come afterwards. We could try to sit together with our users and have them try to revise history manually, but our organization often isn’t in a bubble. Our users interacted with customers and suppliers. It isn’t feasible to try to undo the real-world impacts of this situation.

Why didn’t you just tell us this from the very beginning?

I did, you just weren’t listening.

You wanted a cookie cutter, and until you tried CQRS out as cookie cutter (and saw it create a bunch of complexity) you wouldn’t listen to anything else.

As developers, we’re trained to solve problems – the faster the better. Unfortunately, this causes us to be blind to things that don’t immediately present themselves as solutions.

When applying CQRS with ABCs, the solutions you end up with are very simple, but the process of getting there is quite hard and takes practice. Finding the boundaries of ABCs such that data isn’t duplicated between them and that data doesn’t travel between them either via RPC or publish/subscribe – it may feel impossible the first several times you try. Keep at it – it is almost always possible.

We haven’t touched on the whole saga/aggregate-root thing yet, but that isn’t as important until you can successfully apply the principles described here.

Also, this post has already gotten long enough, so it looks like now would be a good time to stop.

Until next time…

The problem

Let’s start with an image to describe the scenario:

The main issue we have here is that the values transaction 2 gets for A and B are those from T0 – before either transaction 1 or 3 completed. The reason this is an issue is that these old values (usually together with some message data) are used to calculate what the new state of C should be.

Traditional optimistic concurrency techniques won’t detect any problem if we don’t touch A or B in transaction 2.

In short, systems today are causing inconsistency.

Some solutions

1. Don’t have transactions which operate on multiple entities (which probably isn’t possible for some of your most important business logic).

2. Turn on multi-version concurrency control – this is called snapshot isolation in MS Sql Server.

Yes, you need to turn it on. It’s off by default.

The good news is that this will stop the writing of inconsistent data to your database.
The bad news is that it will probably cause your system many more exceptions when going to persist.

For those of you who are using transaction messaging with automatic retrying, this will end up as “just” a performance problem (unless you follow the recommendations below). For those of you who are using regular web/wcf services (over tcp/http), you’re “cross cutting” exception management will likely end up discarding all the data submitted in those requests (but since that’s what you’re doing when you run into deadlocks this shouldn’t be news to you).

The solution to the performance issues

Eventual consistency.

Funny isn’t it – all those people who were afraid of eventual consistency got inconsistency instead.

Also, it’s not enough to just have eventual consistency (like between the command and query sides of CQRS). You need to drastically decrease the size of your entities. And the best way of doing that is to partition those entities across multiple business services (also known in DDD lingo as Bounded Contexts) each with its own database.

This is yet another reason why I say that CQRS shouldn’t be the top level architectural breakdown. Very useful within a given business service, yes – though sometimes as small as just some sagas.

Next steps

It may seem unusual that the title of this post implies that SOA is the solution, yet the content clearly states that traditional HTTP-based web services are a problem. Even REST wouldn’t change matters as it doesn’t influence how transactions are managed against a database.

The SOA solution I’m talking about here is the one I’ve spent the last several years blogging about. It’s a different style of SOA which has services stretch up to contain parts of the UI as well as down to contain parts of the database, resulting in a composite UI and multiple databases. This is a drastically different approach than much of the literature on the topic – especially Thomas Erl’s books.

Unfortunately there isn’t a book out there with all of this in it (that I’ve found), and I’m afraid that with my schedule (and family) writing a book is pretty much out of the question. Let’s face it – I’m barely finding time to blog.

The one thing I’m trying to do more of is provide training on these topics. I’ve just finished a course in London, doing another this week in Aarhus Denmark, and another next month in San Francisco (which is now sold out). The next openings this year will be in Stockholm, London; Sydney Australia and Austin Texas will be coming in January of next year. I’ll be coming over to the US more next year so if you missed San Francisco, keep an eye out.

I wish there was more I could do, but I’m only one guy.

Hmm, maybe it’s time to change that.

The question

The main touted benefit of these workflow-centric architectures is that we don’t have to change the code of the system in order to change its behavior resulting in ultimate flexibility!

Some of you may have already gone down this path and are shaking your heads remembering how your particular road to hell was paved with the exact same good intentions.

Let me explain why these things tend to go horribly wrong.

What’s behind the curtain

It starts with the very nature of workflow – a flow chart, is procedural in nature. First do this, then that, if this, then that, etc. As we’ve experienced first hand in our industry, procedural programming is fine for smaller problems but isn’t powerful enough to handle larger problems. That’s why we’ve come up with object-oriented programming.

I have yet to see an object-oriented workflow drag-and-drop engine. Yes, it works great for simple demo-ware apps. But if you try to through your most complex and volatile business logic at it, it will become a big tangled ball of spaghetti – just like if you were using text rather than pictures to code it.

And that’s one of the fundamental fallacies about these tools – you are still writing code. The fact that it doesn’t look like the rest of your code doesn’t change that fact. Changing the definition of your workflow in the tool IS changing your code.

On productivity

Sometimes people mention how much more productive it would be to use these tools than to write the code “by hand”. Occasionally I hear about an attempt to have “the business” use these tools to change the workflows themselves – without the involvement of developers (”imagine how much faster we could go without those pesky developers!”).

For those of us who have experienced this first-hand, we know that’s all wrong.

If “the business” is changing the workflows without developer involvement, invariably something breaks, and then they don’t know what to do. They haven’t been trained to think the way that developers have – they don’t really know how to debug. So the developers are brought back in anyway and from that point on, the business is once again giving requirements and the devs are the one implementing it.

Now when it comes to developer productivity, I can tell you that the keyboard is at least 10x more productive than the mouse. I can bang out an if statement in code much faster than draggy-dropping a diamond on the canvas, and two other activities for each side of the clause.

On maintainability

Sometimes the visualization of the workflow is presented as being much more maintainable than “regular code”.

When these workflows get to be to big/nested/reused, it ends up looking like the wiring diagram of an Intel chip (or worse). Check out the following diagram taken from the DailyWTF on a customer friendly system:

The bigger these get, the less maintainable they are.

Now, some would push back on this saying that a method with 10,000 lines of code in it may be just as bad, if not worse. The thing is that these workflow tools guide developers down a path where it is very likely to end up with big, monolithic, procedural, nested code. When working in real code, we know we need to take responsibility for the cleanliness of our code using object-orientation, patterns, etc and refactoring things when they get too messy.

Here is where I’d bring up the SOA/pub-sub approach as an alternative – there is no longer this idea of a centralized anything. You have small pieces of code, each encapsulating a single business responsibility, working in concert with each other – reacting to each others events.

Productivity take 2: testing and version control

If you’re going to take your most complex and volatile business logic and put it into these workflow tools, have you thought about how your going to test it? How do you know that it works correctly? It tends to be VERY difficult to unit-test these kinds of workflows.

When a developer is implementing a change request, how do they know what other workflows might have been broken? Do they have to manually go through each and every scenario in the system to find out? How’s that for productivity?

Assuming something did break and the developer wants to see a diff – what’s different in the new workflow from the old one, what would that look like? When working with a team, the ability to diff and merge code is at the base of the overall team productivity.

What would happen to your team if you couldn’t diff or merge code anymore?
In this day and age, it should be considered irresponsible to develop without these version control basics.

In closing

There are some cases where these tools might make sense, but those tend to be much more rare than you’d expect (and there are usually better alternatives anyway). Regardless, the architectural analysis should start without the assumption of centralized workflow, database, or centralized anything for that matter.

If someone tries to push one of these tools/architectures on you, don’t walk away – run!

I can only speak for myself on this one, but I guess it’s that there’s just so many times you can repeat yourself.

So, why this post?

Well, Richard was able to dig up an old (2004) presentation I gave about SOA in which I said:

“Services run in a separate process from their clients
A boundary must be crossed to get from the client to the service – network, security, …”

And 7 years later I can say, hand on heart, I was wrong.

Luckily, I’ve spent much of those past 7 years trying to correct that recommendation. One blog post in which I tried to do that (in mid-2007) was On Intermediation and SOA in which I described the relationship between systems (i.e process boundaries) and services:

“all of these “systems” might just end up within the same service, or having parts of them being used by multiple services

There can also be multiple services (or, more accurately, parts of multiple services) deployed together in the same system/process.

And this is nothing new – in the 4+1 Architectural View Model by Philippe Kruchten (1995) we can see very clearly the differentiation between the Logical View (our services) and the Physical View (a.k.a the Deployment View).

These views are orthogonal to each other – multiple elements from one view can map to a single element in another view (and vice versa).

This, if anything, makes it that much harder to identify service boundaries – if they have nothing to do with the existing applications and systems, then what are they? In my blog post on The Known Unknowns of SOA I point to the fact that Business Capabilities are much more appropriate constructs than, say, web services which (as it says in the referenced post) “[are] merely a standardized approach to accessing functionality on remote systems”.

As I bring this post to a close, I’m feeling more comfortable rehashing material I’ve published before:

Logical and Physical Architecture

and the rest of the SOA category on my blog here.

Happy boundary hunting.

Please accept my apologies for my part in the overly-complex software being created because of it.

I’ve tried to do what I could to provide a balanced view on the topic with posts like Clarified CQRS and Race Conditions Don’t Exist.

It looks like that wasn’t enough, so I’ll go right out and say it:

Most people using CQRS (and Event Sourcing too) shouldn’t have done so.

Should we really go back to N-Tier?

When not using CQRS (which is the majority of the time), you don’t need N-Tier either.

You see, if you’re not in a collaborative domain then you don’t have multiple writers to the same logical set of data as an inherent property of your domain. As such, having a single database where all data lives isn’t really necessary.

Data is inherently partitioned by who owns it.

Let’s take the online shopping cart as an example. There aren’t any use cases where users operate on each others’ carts – ergo, not collaborative, therefore not a good candidate for CQRS. Same goes for user profiles, and tons of other cases.

So why is it that we need a separate tier to run our business logic?

Originally, the application server tier was introduced for improved scalability, but specifically around managing the connection pool to the database. Increasing numbers of clients (when each had its own user/account for connecting to the database) caused problems. Luckily, most web applications side-step this problem – that is, until someone got the idea that the web server was only supposed to run the UI layer, and the Business Logic layer would be on a separate application server tier.

Rubbish – see Fowler’s First Law of Distribution: Don’t.

Keep it all on one tier. Same goes for smart clients.
No, Silverlight, you don’t count – architecturally speaking, you’re a glorified browser.

But what about scalability?

In a non-collaborative domain, where you can horizontally add more database servers to support more users/requests/data at the same time you’re adding web servers – there is no real scalability problem (caveat, until you’re Amazon/Google/Facebook scale).

Database servers can be cheap – if using MySQL/SQL Express/others.

But what about the built-in event-log CQRS/ES gives us?

Architectural gold-plating / stealing from the business.

Who put you in a position to decide that development time and resources should be diverted from short-term business-value-adding features to support a non-functional requirement that the business didn’t ask for?

If you sat down with them, explaining the long-term value of having an archive of all actions in the system, and they said OK, build this into the system from the beginning, that would be fine. Most people who ask me about CQRS and/or Event Sourcing skip this step.

Finally, you can usually implement this specific requirement with some simple interception and logging. Don’t over-engineer the solution. If using messaging, you can get this by turning on journaling, or if you want to centralize this archive, NServiceBus can forward all messages to a specific queue.

Don’t forget that this storage has a cost – including administration. Nothing is free.

What about the “proof of correctness” in Event Sourcing

I’ve heard statements made that when you use the events that flowed into/through your system AS your system’s data, rather than transforming those events to some other schema (relational or otherwise) and storing the result – you can prove that your system behaves correctly.

Let me put it this way:

No programming technique used by humans will prevent those same humans from creating bugs.
No testing technique used by humans will prevent those same humans from not catching those bugs.
* Automated tests – see programming technique.

While having a full archive of all events can allow us to roll the system back to some state, fix a bug, and roll forwards, that assumes that we’re in a closed system. We have users which are outside the system. If a user made a decision based on data influenced by the bug, there’s no automated way for us to know that, or correct for it as we roll forwards.

In short, we’re interested in the business’ behavior – as composed of user and system behavior. No proof can exist.

Umm, so where should we use it

If you’ve uncovered a scenario where you’re wondering “first-one-wins, or last-one-wins”, that’s often a good candidate for a place where CQRS could make sense. Then re-read my Race Conditions Don’t Exist post.

Also, CQRS should not be your top-level architectural pattern – that would be SOA.
CQRS, if used at all, would be used inside a service boundary only.

Given that SOA guides us away from having a given 3rd normal form entity exist in any one service, it is unlikely that the building blocks of your CQRS design will be those kinds of entities. Most 3rd normal form one-to-many and many-to-many relationships simply do not exist when doing SOA and CQRS properly.

Therefore, I’m sorry to say that most sample application you’ll see online that show CQRS are architecturally wrong. I’d also be extremely wary of frameworks that guide you towards an entity-style aggregate root CQRS model.

In Summary

So, when should you avoid CQRS?

The answer is most of the time.

Here’s the strongest indication I can give you to know that you’re doing CQRS correctly: Your aggregate roots are sagas.

And the biggest caveat – the above are generalizations, and can’t necessarily be true for every specific scenario. If you’re Greg Young, then you probably can (and will) decide on your own on these matters. For everybody else, please take these warnings to heart. There have been far too many clients that have come to me all mixed up with their use CQRS in areas where it wasn’t warranted.

If you want to know everything you need to know to apply CQRS appropriately, please come to my course – there is so much unlearning to do first that just can’t happen via a series of blog posts.

The ugly truth

You don’t need an ESB for integration.

There. I’ve said it.

Most of the ESB products on the market, in focusing on integration, are addressing the wrong problem.

What integration is about

There are three primary components to integration – data format translation, protocol bridging, and logic. Let’s take the Agile “simplest solution that could possibly work” motto and apply it to these elements:

1. Data format translation
   
   • Use something like MapForce (from Altova, the guys behind XMLSpy).
   • At under $1200 for a dev license and handling mapping to/from XML, EDI, flat files, and relational databases, you can host the resulting mapping in any endpoint, as XSLT or even Java/C# – no need to have “the bus” do this for you.
2. Protocol bridging
   
   • Use something like /n software
   • At about $1500 for a dev license, you get solutions for FTP, HTTP, SMTP, POP, IMAP, LDAP, DNS, RSS, SMS, Jabber, SOAP, WebDav, etc and, again – you can host the DLLs in any endpoint you want. Don’t need expensive buses for this
3. Logic
   
   This is all you – no technology can do this for you. If anything, the most important thing is to get all the ugly mapping and protocol bridging stuff out of the way and let you focus on your logic. This is Single Responsibility Principle (un)common sense.

Digging deeper into the logic

The interesting thing about most of the protocol stuff mentioned above is that they’re inherently unreliable and also their performance at runtime is unknowable due to the total load on the target server.

We wouldn’t want any of the above situations to cause our integration to “get stuck”. As such, it is best we think of our integration logic as a long-running process that manages other endpoints which do the actual protocol bridging and data transformations.

This is one of the areas where NServiceBus, with its sagas can actually help a lot. Just like the other pieces of integration mentioned above, these sagas can run on any endpoint.

You could alternatively look at other technologies like BizTalk and Business Process Execution Language (BPEL) engines, though many of those are designed to be physically centralized (just like many of the ESBs out there).

By the way, if you do want to use NServiceBus together with BizTalk, Michael Stephenson has published some great white papers on getting the two to work together. His latest is about integrating NServiceBus into BizTalk’s RFID processes – check it out.

Putting it all together

When we take all of these pieces and look at them in a cohesive architecture, here’s what it can look like:

As you can see, integration is physically distributed across multiple endpoints.

Not only that, but the integration logic is kept separate from the protocol bridging and data transformation, enabling independent versioning of each. Just as important, it makes it much easier to unit test that the integration logic is correct as we don’t have to simulate the target technologies.

Costs

As you can see, you can get integration capabilities just as powerful as if you went with something like BizTalk, but without creating a single point of failure in your architecture. In terms of costs, it’s also quite a bit cheaper.

For a high availability BizTalk deployment, you’ll be paying over $40,000 per CPU for the Enterprise Edition, not including the extra $7000 per CPU for SQL Server Standard Edition (clustered). For a clustered 4-CPU mid-size deployment, you’d be in the area of $200,000.

For the distributed integration solution above, you’d be paying around $2700 in dev licenses (for /n software and Altova MapForce), and $500 per core for NServiceBus Standard Edition. The reason there’s per-core licensing for NServiceBus is that in a virtualized environment, you’ll be provisioning virtual cores to your virtual machines. No reason to pay for a quad-core CPU when all you’re using is a single core. You can also use any number of cores running NServiceBus Express Edition at no cost, so a mid-size deployment with say 8 cores running Standard Edition, and another 24 cores running Express Edition would cost (with the volume discount) an additional $3800 – a total cost of $6,500.

BizTalk: $200,000       NServiceBus: $6,500

‘Nuff said.

In closing

We’ve been bitten by centralized architectures before.

By having our integration distributed, we can version, upgrade, and scale each piece independently.

You’ve seen how we can use simple, lightweight, and inexpensive technologies to create distributed integration solutions just as powerful and robust as the centralized ESB vendor-offerings out there, but at a tiny fraction of the cost.

Next time the topic of integration is brought up, you’ll know not to be suckered in by re-branded EAI brokers.

Learn more about NServiceBus.

Brokers

Message brokers, more broadly known and used on the Java platform, don’t come with this constraint. For example, when using ActiveMQ, you can have any number of endpoints come to the broker and publish a message under a given topic.

So where’s the problem?

It’s all about accountability.

Let’s say you’ve subscribed to a given topic, and have received two events – one telling you that the price of bananas next week will be $1/kg and another telling you that it’ll be $2/kg.

Which one is right?

Especially given that those events may have been published by any other endpoint via the broker.

Is it first one wins? Last one wins? How about first one sent vs. first one received? Ditto for last. As a subscriber, can you really be held accountable for having the logic to choose the right one? Shouldn’t this responsibility have fallen to the publishing side?

This is one of the big drawbacks of the broker, hub and spoke architecture. No responsibility. No single source of truth – unless everybody’s going to some central database, in which case – what’s the point of all this messaging anyway?

Buses

The Bus Architectural Style is all about accountability. If you are going to publish an event, you are accountable for the correctness of the data in that event – there is no central database that a subscriber can go to “just in case”. And the only way that you can be held accountable, is if you have full responsibility – ergo, you’re the only one who can publish that type of event.

If you say bananas are going to cost $1/kg next week, that’s that. Subscribers will not hear from anybody else on that topic.

Now, this is not to say that you can’t have more than one physical publishing endpoint.

You see, buses differentiate between the logical and the physical. Brokers tend to assume that the physical hub-and-spoke topology is also the logical.

In a bus, while there can only be one logical endpoint publishing a given type of event, that endpoint can be physically scaled out across multiple machines. It is the responsibility of the bus to provide infrastructure facilities to allow for that to happen in such a way that to subscribers, it still appears as if there is really only one publishing endpoint.

The same is true about the subscriber – one logical subscribing endpoint may be scaled out across multiple machines.

Product Mix-ups

Unfortunately, there are many broker-style technologies out there that are being marketed under the banner of the Enterprise Service Bus. While some products have the ability to be deployed in both a centralized and distributed fashion (sometimes called “federated” or “embedded” mode), many do not enforce the “single publishing endpoint per event-type” rule.

Without this constraint, it is just too easy to make mistakes.

NServiceBus

By enforcing this constraint, we see the same kind of question appear on the discussion group time and time again:

“I have an Audit event that I’d like all of my machines to publish, and have one machine subscribe to them all, but NServiceBus won’t let me. How do I make NServiceBus support this scenario?”

And the answer is the same every time:

“You should have all the machines Send the Audit message (configured to go to the single machine handling that message), and not Publish. It is not an event until its been handled by the endpoint responsible for it.”

The semantics of the message matter a lot.

When looking at Service-Oriented Architecture, these messages are the contract and, as any lawyer will tell you, contracts need to be explicit and the intentions really need to be spelled out – otherwise the contract is practically worthless.

In closing

Friction is sometimes a good thing – it prevents us from making mistakes. It keeps cars on the road. And because that’s not enough friction, we introduce curbs as well.

If you’re looking for a service bus technology for your next project, check that it’ll give you the friction that you need to keep everybody safe. Really check what it is that the vendors are offering you – more often than not, it’s some ESB lipstick on a broker pig.

To learn more about how NServiceBus supports this kind of publish/subscribe, click here.

Content-Based Routing and ESBs

Since content-based routing often appears on feature lists for various ESBs, many people consider them to be a necessary part of systems built on SOA principles. The pattern also appears in the book Enterprise Integration Patterns, which apparently is also a convincing reason to use it, even though the book specifically states:

“When implementing a Content-Based Router, special caution should be taken to make the routing function easy to maintain as the router can become a point of frequent maintenance.”

That’s right – maintenance nightmare.

For Example

Let’s take a real-world example – a trading system where users can put orders for stock and for forex (money from other countries).

We’d start by creating a message that can be sent by a client:

public class PlaceOrder : IMessage
{
    public OrderTypeEnum OrderType { get; set; }
    public string Code { get; set; }
    public double Amount { get; set; }
}

public enum OrderTypeEnum
{
    Stock,
    Forex
}

Clients would send messages by using code like the following:

Bus.Send<PlaceOrder>(m => {
    m.OrderType = OrderTypeEnum.Stock;
    m.Code = "MSFT";
    m.Amount = 300;
    m.SetHeader("AccountId", myAccountId);
});

The header is the way that clients identify themselves in the system.

Let’s say that the logic for handling the different types of orders is different, but also that we’d like the logic to be deployed to different endpoints. One reason we might want to do this is so that we can independently scale each of these endpoints. This is where it would appear we’d need some content-based routing – having some code that looks at the OrderType property and decides where to route based on its value.

Before we get into solutions, let’s make this more involved.

Not only do we want to route based on OrderType, but we want to use the account ID in the header to check in our database (or via a web service) if the account belongs to one of our VIP customers, and if so, it should be given higher priority.

Content-Based Routing and Brokers

We can see that if we were to go with a content-based routing solution, this would drive our architecture to a hub-and-spoke model where the hub becomes quite large and complex, as well as likely becoming a bottleneck in terms of performance.

This logically centralized place through which all communication flows defines the Broker architectural style – not that of a Bus. In the Bus architectural style, there is no logical (or physical) hub. You can think of it as a kind of peer-to-peer setup. Just like you wouldn’t want ethernet getting involved in applicative routing decisions, neither should your bus get involved.

Business-Topology Mapping Solutions

Instead, when following the Bus architectural style – we look at mapping stable business characteristics to bus-level topology. At one level, that would mean defining two different message types for our different types of trades:

public abstract class PlaceOrder
{
    public string Code { get; set; }
    public double Amount { get; set; }
}

public class PlaceStockOrder : PlaceOrder { }

public class PlaceForexOrder : PlaceOrder { }

Once we have two different message types, then we can configure the client to have those statically sent to different endpoints like this:

<UnicastBusConfig>
  <MessageEndpointMappings>
    <add Messages="Messages.PlaceStockOrder, Messages" Endpoint="Stock" />
    <add Messages="Messages.PlaceForexOrder, Messages" Endpoint="Forex" />
  </MessageEndpointMappings>
</UnicastBusConfig>

And when it comes to handling our VIP customers, the recommendation would be to have those customers be served by a different set of web servers – we wouldn’t want a sudden flux in regular customers stealing all the HTTP connections from our VIP customers. Then we’d statically configure our VIP front-end to talk to our VIP back-end like this:

<UnicastBusConfig>
  <MessageEndpointMappings>
    <add Messages="Messages.PlaceStockOrder, Messages" Endpoint="VIP_Stock" />
    <add Messages="Messages.PlaceForexOrder, Messages" Endpoint="VIP_Forex" />
  </MessageEndpointMappings>
</UnicastBusConfig>

And the logic to identify VIP customers would be done in the login screen which, from there, would direct the user to the appropriate web server farm.

Static vs. Dynamic

It may appear that the above statically configured solution is less flexible then the afore-mentioned hub-and-spoke content-based routing solution. And that’s probably correct. But the question is, are the business requirements (better called “objectives” in this case) likely to change? If we make the technological solution 10x more flexible than the business needs, but at the cost of maintainability (read time to market), we probably haven’t used the right tool for the job.

Many times we can find stable business objectives and align the topology of our solution with them. Not all that is dynamic and flexible is necessarily better than that which is static.

I’d go so far to say that if a solution makes heavy use of content-based routing, it is likely a more fragile solution as implementations of stable business objectives and volatile requirements are mixed up together.

In Closing

NServiceBus intentionally does not support content-based routing so as not to make it easy for developers to make architectural blunders that could require full-system rewrites a couple of years down the road.

If you want to learn more about these kinds of architectural principles, I suggest coming to my course. I’m afraid that New York City is already sold out, but I’ll be coming back to the US again around October. Stockholm, London, Sydney, and Oslo are now all open for registration.

Hope to see you there.

“SOA is a practice that focuses on modeling the entities, and relationships between entities, that comprise the business as a set of services. This can be done on a small or large scale. Typically, the relationships in this model represent consumer/provider relationships.”

I have some serious concerns about the ramifications of this definition/description.

First of all, when reading “entities”, many people will interpret that to mean the entities found in Entity Relationship Diagrams [ERD] or in Object Oriented Analysis & Design [OOAD]. In both, these entities are identified as the “nouns” of the domain. Examples of these ERD/OOAD-type entities include things like Customer, Order, and Product.

These are almost always the wrong place to start for identifying services in SOA.

Second, on the consumer/provider relationship: on the one had, this fits very well with how web services can consume (or call) other web services. However, the downsides of using web services as services in SOA is becoming well enough known that even in the same post we see this warning:

“Web Services is not SOA, it is merely a standardized approach to accessing functionality on remote systems.”

But the question remains, if a producer/consumer relationship is OK for SOA-type services, why doesn’t that hold for web services? And the answer is… it depends on the type of producer/consumer relationship. The typical relationship is one of synchronous calls from consumer to producer, this is not OK for SOA-type services either.

You see, this synchronous producer/consumer implies a model where services are not able to fulfill their objectives without calling other services. In order for us to achieve the IT/Business alignment promised by SOA, we need services which are autonomous, ie. able to fulfill their objectives without that kind of external help.

Instead, we need to look for a more loosely coupled producer/consumer relationship – like publish/subscribe, where the producer emits events, and the consumer subscribes and handles those events. The reason that this kind of relationship doesn’t hurt autonomy is that it disconnects services on the dimension of time. In order for a service to be able to make a decision autonomously without synchronously calling any other service, using only information provided by events it received in the past, it must be strongly aligned with the business.

Most projects which bandy about the SOA acronym aren’t actually made up of services – they’re made up of XML over HTTP functions calling other XML over HTTP functions, eventually calling XML over HTTP databases. You can layer as much XML and HTTP as you want on top of it, but at the end of the day, most projects are just functions calling functions calling databases – in other words, procedural programming in the large, and no amount of SOAP will wash away the stink.

Here’s a different definition of services for SOA that may communicate a bit better what it’s all about:

A service is the technical authority for a specific business capability.
Any piece of data or rule must be owned by only one service.

What this means is that even when services are publishing and subscribing to each other’s events, we always know what the authoritative source of truth is for every piece of data and rule.

Also, when looking at services from the lense of business capabilities, what we see is that many user interfaces present information belonging to different capabilities – a product’s price alongside whether or not it’s in stock. In order for us to comply with the above definition of services, this leads us to an understanding that such user interfaces are actually a mashup – with each service having the fragment of the UI dealing with its particular data.

Ultimately, process boundaries like web apps, back-end, batch-processing are very poor indicators of service boundaries. We’d expect to see multiple business capabilities manifested in each of those processes.

I know that this may be more confusing than the traditional web services approach but, to paraphrase Donald Rumsfeld, it is better to know that you don’t know, than to not know that you don’t know

Don’t get me wrong, sometimes there is a place for a web service, just not everywhere.

So, what’s the problem?

Well, when developers and architects use web services as the building blocks of their designs, they are creating the same architecture for both the logical and physical elements of their system. Back in 1995, Philippe Kruchten documented his 4 + 1 Architectural View Model in which he outlined 4 + 1 different views that should be used to describe an architecture.

Even though since 1995 the number and types of recommended views of software architecture has evolved (with things like the Zachman Framework for enterprise architecture numbering some 30 views), there is broad agreement that (at the very least) the logical and physical artifacts should likely be designed differently.

Just because two distinct logical components have been identified in the architecture, that doesn’t necessarily mean they should be hosted separately (for example by making each one a web/wcf service). In fact, there are significant disadvantages to doing so (as described in the Fallacies of Distributed Computing).

In some cases, this mistake is exacerbated by a mistaking these components with SOA-type services, resulting in an attempt by developers to have each component have its own contract, which can then be independently versioned. This often results in the need for transformation between the structure of these so-called contracts, but not within the components themselves (oh-no, they’re “autonomous”), but rather in between them using some kind of “ESB” technology.

This architectural style is known as the Broker, Hub and Spoke, Mediator, and most importantly – not SOA. If you find a technology that fits this style perfectly (like BizTalk), that technology is not a Bus, not a Service Bus, and definitely not an Enterprise Service Bus.

One of the problems of this approach is that when any “service” contract changes, you have to change all the transformations in your broker that involve it. Unfortunately, most brokers have no unit-testing facility so it’s very much trial and error, and error, and error. The matter is even more serious since most brokers don’t enable you to have your transformations or orchestrations in source control, so you can’t diff to see what changed from the previous version.

It’s really amazing how much pain can be traced back to that one original misunderstanding.

Currently, the video is being graciously hosted by Jon Torresdal on his blog, so let’s hope that the bandwidth holds up.

Get it here.

My latest article has been published in issue 21 of the Microsoft Architecture Journal:

EDA: SOA Through The Looking Glass

Please leave questions and comments here.

Take a listen.

The common e-commerce example

We accept orders, bill the customer, and then ship them the product.

Fairly straight-forward.

Since each part of that process can be quite complex, let’s have each step be handled by a service:

Sales, Billing, and Shipping. Each of these services will publish an event when it’s done its part. Sales will publish OrderAccepted containing all the order information – order Id, customer Id, products, quantities, etc. Billing will publish CustomerBilledForOrder containing the customer Id, order Id, etc. And Shipping will publish OrderShippedToCustomer with its data.

So far, so good. EDA and SOA seem to be providing us some value.

Where’s the saga?

Well, let’s consider the behavior of the Shipping service. It shouldn’t ship the order to the customer until it has received the CustomerBilledForOrder event as well as the OrderAccepted event. In other words, Shipping needs to hold on to the state that came in the first event until the second event comes in. And this is exactly what sagas are for.

Let’s take a look at the saga code that implements this. In order to simplify the sample a bit, I’ll be omitting the product quantities.

   1:      public class ShippingSaga : Saga<ShippingSagaData>,

   2:          ISagaStartedBy<OrderAccepted>,

   3:          ISagaStartedBy<CustomerBilledForOrder>

   4:      {

   5:          public void Handle(OrderAccepted message)

   6:          {

   7:              this.Data.ProductIdsInOrder = message.ProductIdsInOrder;

   8:          }

   9:   

  10:          public void Handle(CustomerBilledForOrder message)

  11:          {

  12:               this.Bus.Send<ShipOrderToCustomer>(

  13:                  (m =>

  14:                  {

  15:                      m.CustomerId = message.CustomerId;

  16:                      m.OrderId = message.OrderId;

  17:                      m.ProductIdsInOrder = this.Data.ProductIdsInOrder;

  18:                  }

  19:                  ));

  20:   

  21:              this.MarkAsComplete();

  22:          }

  23:   

  24:          public override void Timeout(object state)

  25:          {

  26:              

  27:          }

  28:      }

First of all, this looks fairly simple and straightforward, which is good.

It’s also wrong, which is not so good.

One problem we have here is that events may arrive out of order – first CustomerBilledForOrder, and only then OrderAccepted. What would happen in the above saga in that case? Well, we wouldn’t end up shipping the products to the customer, and customers tend not to like that (for some reason).

There’s also another problem here. See if you can spot it as I go through the explanation of ISagaStartedBy<T>.

Saga start up and correlation

The “ISagaStartedBy<T>” that is implemented for both messages indicates to the infrastructure (NServiceBus) that when a message of that type arrives, if an existing saga instance cannot be found, that a new instance should be started up. Makes sense, doesn’t it? For a given order, when the OrderAccepted event arrives first, Shipping doesn’t currently have any sagas handling it, so it starts up a new one. After that, when the CustomerBilledForOrder event arrives for that same order, the event should be handled by the saga instance that handled the first event – not by a new one.

I’ll repeat the important part: “the event should be handled by the saga instance that handled the first event”.

Since the only information we stored in the saga was the list of products, how would we be able to look up that saga instance when the next event came in containing an order Id, but no saga Id?

OK, so we need to store the order Id from the first event so that when the second event comes along we’ll be able to find the saga based on that order Id. Not too complicated, but something to keep in mind.

Let’s look at the updated code:

   1:      public class ShippingSaga : Saga<ShippingSagaData>,

   2:          ISagaStartedBy<OrderAccepted>,

   3:          ISagaStartedBy<CustomerBilledForOrder>

   4:      {

   5:          public void Handle(CustomerBilledForOrder message)

   6:          {

   7:              this.Data.CustomerHasBeenBilled = true;

   8:   

   9:              this.Data.CustomerId = message.CustomerId;

  10:              this.Data.OrderId = message.OrderId;

  11:   

  12:              this.CompleteIfPossible();

  13:          }

  14:   

  15:          public void Handle(OrderAccepted message)

  16:          {

  17:              this.Data.ProductIdsInOrder = message.ProductIdsInOrder;

  18:   

  19:              this.Data.CustomerId = message.CustomerId;

  20:              this.Data.OrderId = message.OrderId;

  21:   

  22:              this.CompleteIfPossible();

  23:          }

  24:   

  25:          private void CompleteIfPossible()

  26:          {

  27:              if (this.Data.ProductIdsInOrder != null && this.Data.CustomerHasBeenBilled)

  28:              {

  29:                  this.Bus.Send<ShipOrderToCustomer>(

  30:                     (m =>

  31:                     {

  32:                         m.CustomerId = this.Data.CustomerId;

  33:                         m.OrderId = this.Data.OrderId;

  34:                         m.ProductIdsInOrder = this.Data.ProductIdsInOrder;

  35:                     }

  36:                     ));

  37:                  this.MarkAsComplete();

  38:              }

  39:          }

  40:      }

And that brings us to…

Saga persistence

We already saw why Shipping needs to be able to look up its internal sagas using data from the events, but what that means is that simple blob-type persistence of those sagas is out. NServiceBus comes with an NHibernate-based saga persister for exactly this reason, though any persistence mechanism which allows you to query on something other than saga Id would work just as well.

Let’s take a quick look at the saga data that we’ll be storing and see how simple it is:

   1:      public class ShippingSagaData : ISagaEntity

   2:      {

   3:          public virtual Guid Id { get; set; }

   4:          public virtual string Originator { get; set; }

   5:          public virtual Guid OrderId { get; set; }

   6:          public virtual Guid CustomerId { get; set; }

   7:          public virtual List<Guid> ProductIdsInOrder { get; set; }

   8:          public virtual bool CustomerHasBeenBilled { get; set; }

   9:      }

You might have noticed the “Originator” property in there and wondered what it is for. First of all, the ISagaEntity interface requires the two properties Id and Originator. Originator is used to store the return address of the message that started the saga. Id is for what you think it’s for. In this saga, we don’t need to send any messages back to whoever started the saga, but in many others we do. In those cases, we’ll often be handling a message from some other endpoint when we want to possibly report some status back to the client that started the process. By storing that client’s address the first time, we can then “ReplyToOriginator” at any point in the process.

The manufacturing sample that comes with NServiceBus shows how this works.

Saga Lookup

Earlier, we saw the need to search for sagas based on order Id. The way to hook into the infrastructure and perform these lookups is by implementing “IFindSagas<T>.Using<M>” where T is the type of the saga data and M is the type of message. In our example, doing this using NHibernate would look like this:

   1:      public class ShippingSagaFinder : 

   2:          IFindSagas<ShippingSagaData>.Using<OrderAccepted>,

   3:          IFindSagas<ShippingSagaData>.Using<CustomerBilledForOrder>

   4:      {

   5:          public ShippingSagaData FindBy(CustomerBilledForOrder message)

   6:          {

   7:              return FindBy(message.OrderId)

   8:          }

   9:   

  10:          public ShippingSagaData FindBy(OrderAccepted message)

  11:          {

  12:              return FindBy(message.OrderId)

  13:          }

  14:   

  15:          private ShippingSagaData FindBy(Guid orderId)

  16:          {

  17:              return sessionFactory.GetCurrentSession().CreateCriteria(typeof(ShippingSagaData))

  18:                  .Add(Expression.Eq("OrderId", orderId))

  19:                  .UniqueResult<ShippingSagaData>();

  20:          }

  21:   

  22:          private ISessionFactory sessionFactory;

  23:   

  24:          public virtual ISessionFactory SessionFactory

  25:          {

  26:              get { return sessionFactory; }

  27:              set { sessionFactory = value; }

  28:          }

  29:      }

For a performance boost, we’d probably index our saga data by order Id.

On concurrency

Another important note is that for this saga, if both messages were handled in parallel on different machines, the saga could get stuck. The persistence mechanism here needs to prevent this. When using NHibernate over a database with the appropriate isolation level (Repeatable Read – the default in NServiceBus), this “just works”. If/When implementing your own saga persistence mechanism, it is important to understand the kind of concurrency your business logic can live with.

Take a look at Ayende’s example for mobile phone billing to get a feeling for what that’s like.

Summary

In almost any event-driven architecture, you’ll have services correlating multiple events in order to make decisions. The saga pattern is a great fit there, and not at all difficult to implement. You do need to take into account that events may arrive out of order and implement the saga logic accordingly, but it’s really not that big a deal. Do take the time to think through what data will need to be stored in order for the saga to be fault-tolerant, as well as a persistence mechanism that will allow you to look up that data based on event data.

If you feel like giving this approach a try, but don’t have an environment handy for this, download NServiceBus and take a look at the samples. It’s really quick and easy to get set up.

I’ve been to too many clients where I’ve been brought in to help them with their problems around service versioning when the solution I propose is simply to have version N+1 of the system be backwards-compatible with version N. If two adjacent versions of a given system aren’t compatible with each other, it is practically impossible to solve versioning issues.

Here’s what happens when versions aren’t compatible:

Admins stop the system from accepting any new requests, and wait until all current requests are done processing. They take a backup/snapshot of all relevant parts of the system (like data in the DB). Then, bring down the system – all of it. Install the new version on all machines. Bring everything back up. Let the users back in.

If, heaven-forbid, problems were uncovered with the new version (since some problems only appear in production), the admins have to roll back to the previous version – once again bringing everything down.

This scenario is fairly catastrophic for any company that requires not-even high availability, but pretty continuous availability – like public facing web apps.

If adjacent versions were compatible with each other, we could upgrade the system piece-meal – machine by machine, where both the old and new versions will be running side by side, communicating with each other. While the system’s performance may be sub-optimal, it will continue to be available throughout upgrades as well as downgrades.

This isn’t trivial to do.

It impacts how you decide what is (and more importantly, what isn’t) nullable.

It may force you to spread certain changes to features across more versions (aka releases).

As such, you can expect this to affect how you do release and feature planning.

However, if you do not take these factors into account, it’s almost a certainty that your versioning problems will persist and no technology (new or old) will be able to solve them.

Coming next… Units of versioning – inside and outside a service.

Greg Young asserts:

“I believe that this shows there to be a rather negligible cost associated with the use of such a model. There is however a small cost, this cost however I believe only exists when one looks at the system in isolation.”

Ayende adds his perspective:

“The cost of messaging, and a very real one, comes when you need to understand the system. In a system where message exchange is the form of communication, it can be significantly harder to understand what is going on.”

Of course, both these intelligent fellows are right. The reason for the apparent disparity in viewpoints has to do with which part of the following graph you look at. Ayende zooms in on the left side:

As systems get larger, though, the only way to understand them is by working at higher levels of abstraction. That’s where messaging really shines, as the incremental complexity remains the same by maintaining the same modularity as before:

In Ayende’s post, he follows the design I described a while back on using messaging for user management and login for a high-scale web scenario. In his comments, he agrees with the above stating:

“I certainly think that a similar solution using RPC would be much more complex and likely more brittle.”

I feel quite conservative in saying the most enterprise solutions fall on the right side of the intersection in the graph.

That being said, don’t underestimate the learning curve developers go through with messaging. While the mechanics are similar, the mindset is very different. Think about it like this:

You’ve driven a car for years in the US. It’s practically second nature. Then you fly to the UK, rent a car, and all of a sudden, your brain is in meltdown. (or vice versa for those going from the UK to the US)

Summary

If you are going down the messaging route, please be aware that there are shades of gray there as well. You don’t have to implement your user management and login the way I outlined in my post if you don’t require such high levels of scalability, but even lower levels of scalability can benefit from messaging.

Just as there isn’t a single correct design for non-messaging solutions, the same is true for those using messaging. Finding the right balance is tricky, and critical.

When the code is simple in every part of the system, and the asynchronous interactions are what provide for the necessary complexity the problem domain requires, that’s when you know you’ve got it just right.

The question of how web-based (or 3rd party) consumers can work with pub/sub based services comes up a lot.

Many developers are used to implementing web services exposing methods on them like GetAllCustomers.

When moving to pub/sub and other more loosely coupled messaging patterns, developers look to implement the same pattern, opting for something like duplex GetCustomersRequest and GetCustomersResponse. The reasoning is simple and straightforward – it is difficult to push data over the web to consumers.

However, there are still ways to disconnect the preparation of the data from its usage thus gaining many of the advantages of pub/sub.

By employing REST principles and modelling our customer list as an explicit resource, web-based consumers would simply perform regular HTTP GET operations on the URI to get the list of customers.

The resource itself could be a simple XML file – it wouldn’t need to be dynamic at all.

You can get all the scalability benefits of pub/sub for web based consumers. All you need is a bit of REST

Also, “passing reference data in a message makes the message-consuming systems dependent on the knowledge and availability of actual persistent data that is stored “somewhere”. This data must separately be accessed for the sake of understanding the event that is represented by the message.”

The discussion of self-contained events can be compared to integration databases vs application databases.

Centralized Integration – Pros & Cons

If everything in a system can access a central datastore, it is enough for one party to publish an event containing only the ID of an entity that that party previously entered/updated. Upon receiving that event, a subscriber would go to the central datastore and get the fields its interested in for that ID. The advantage of this approach is that the minimal amount of data necessary crosses the network, as subscribers only retrieve the fields that interest them. Martin Fowler describes the disadvantages as:

“An integration database needs a schema that takes all its client applications into account. The resulting schema is either more general, more complex or both. The database usually is controlled by a separate group to the applications and database changes are more complex because they have to be negotiated between the database group and the various applications.”

This is far from being aligned with the principle of autonomy so important to SOA. In that respect, the architectural principle of self-contained messages points us away from those problems and towards more autonomous services.

However, once we have these autonomous business services in place, we may find that we don’t need 100% fully self-contained messages anymore.

A Real-World Example

Let’s say we have 3 business services, Sales, Fulfillment, and Billing.

Sales publishes an OrderAccepted event when it accepts an order. That event contains all the order information.

Both Fulfillment and Billing are subscribed to this event, and thus receive it.

Fulfillment does not ship products to the customer until the customer has been billed, so it just stores the order information internally, and is done.

Billing starts the process of billing the customer for their order, possibly joining several orders into a single bill. After completing this process, it publishes a CustomerBilled event containing all billing information, as well as the IDs of the orders in that bill. It does not put all the order information in that event, as it is not the authoritative owner of that data.

When Fulfillment receives the CustomerBilled event, it uses the IDs of the orders contained in the event to find the order information it previously stored internally. It does not need to call the Sales service for this information or contact some central Master Data Management system. It uses the data it has, and goes about fulfilling the orders and shipping the products to the customer, finally publishing its own OrderShipped event.

Notice, as well, that in the original OrderAccepted event there were the IDs of products the customer ordered. These product IDs originated from another service, Merchandising, responsible for the product catalog. The same thing can be said for the customer ID originating from another service – Customer Care.

The Issue of Time

One could argue that since subscribers use previously cached data when processing new events, that data might not be up to date. Also, we may have race conditions between our services. In the above example, if Billing was extremely fast and more highly available than Fulfillment. Billing could have received the OrderAccepted event, processed it, and published the CustomerBilled event before Fulfillment had received the OrderAccepted event. In short, the CustomerBilled and OrderAccepted messages could be out of order in Fulfillment’s queue.

What would Fulfillment do when trying to process the CustomerBilled message when it doesn’t have the order information?

Well, it knows that the world is parallel and non-sequential, so it does NOT return/log an error, but rather puts that message in the back of the queue to be processed again later (or maybe in some other temporary holding area). This enables the OrderAccepted message to be processed before the CustomerBilled message is retried. When the retry occurs, well, everything’s OK – it’s worked itself out over time.

In the case where we retry again and again and things don’t work themselves out (maybe the OrderAccepted event was lost), we move that message off to a different queue for something else to resolve the conflict (maybe a person, maybe software). If/when the conflict is resolved (got the Sales system / messaging system to replay the OrderAccepted event), the conflict resolver returns the CustomerBilled message to the queue, and now everything works just fine.

As all of this is occurring, the only thing that’s visible to external parties is that it happens to be taking longer than usual for the OrderShipped event to be published. In other words, time is the only difference.

Summary

The problem of non-self-contained events is mitigated first and foremost by business services in SOA, and the apparent issue of time-synchronization by business logic inside these services.

Don’t be afraid to put IDs in your messages and events.

Do be afraid of using those IDs to access datastores shared by multiple “services”.

Using IDs to correlated current events to data from previous events is not only OK, it’s to be expected.

The architectural principle of fully self-contained messages steers us away from the problems of Integration Databases and towards Application Databases, autonomous services, and a better SOA implementation. From there, following the principle of autonomy from a business perspective, will lead us to services not publishing data in their messages that is owned by other services, taking us the next step of our journey to SOA.

Related Content

[Podcast] Message Ordering – Is it cost effective?

Don’t EDA between existing systems

[Podcast] Handling dependencies between subscribers in SOA

Durable Messaging

In order to prevent a message from being lost when it is sent from a publisher to a subscriber, the message is written to disk on the publisher side, and then forwarded to the subscriber, where it is also written to disk. This store-and-forward mechanism enables our systems to gracefully recover from either side being temporarily unavailable.

In my MSDN article on this topic, I outlined some problems with this approach. These problems are exacerbated for publishers. Imagine a publisher with 40 subscribers, publishing 10 messages a second, each containing 1MB of XML. If 10 of the subscribers are unavailable, that’s 100MB of data being written to the publisher’s disk every second, 6GB every minute. That’s liable to bring down a publisher before an administrator brews a cup of coffee.

Publishers have no choice but to throw away messages after a certain period of time.

Publisher Contracts

The whole issue of contracts and schema is considered one of the better understand parts of SOA. Unfortunately, the operational aspects of service contracts is hardly ever taken into account.

On top of the schema of the messages a service publishers, additional information is needed in the contract:

1. How big will this message be?
2. How often will it be published?
3. How long will this message be stored if a subscriber is unavailable?

This first two pieces of information are important for subscribers to do load and capacity planning. The last one is the most important as it dictates the required availability and fault-tolerance characteristic of subscribers.

For Example

In the canonical retail scenario, when our sales service accepts an order, it publishes an order accepted event. Other services subscribed to this event include shipping, billing, and business intelligence.

While shipping and billing are highly available and able to keep up with the rate at which orders are accepted, the business intelligence service is not. BI has two main parts to it – a nightly batch that does the number crunching, and a UI for reporting off of the results of that number crunching. Some even do the reporting in a semi-offline fashion, emailing reports back to the user when they’re ready.

Furthermore, nobody’s going to invest in servers for making BI highly available.

And wasn’t the whole point of this publish/subscribe messaging to keep our services autonomous? That not all services have to have the same level uptime?

Houston, do we have a problem.?

Data Freshness

There is a glimmer of light in all this doom and gloom.

Not all services have the same data freshness requirements.

The business intelligence service above doesn’t need to know about orders the second they’re accepted. A daily roll-up would be fine, and an hourly roll-up bring us that much closer to “real time business intelligence”.

So, while BI is ready to accept the sales message schema, it would like a slightly different contract around it – less messages per unit of time, more data in each message.

From the operational perspective of the sales service, it would be cost effective to have less “online” subscribers. It could even take things a few steps further. Instead of using the regular messaging backbone for transmitting these hourly messages, it could use FTP. The data could even be zipped to take up even less space. Since the total data size is less than the corresponding online stream, is stored on cheaper, large storage, and the number of subscribers for this zipped, hourly update is fairly small, these messages can be kept around far longer.

If you’ve heard about consumer-driven contracts, this is it.

Note that we’re still talking about the same logical message schema.

Summary

It’s not that lost notifications aren’t a problem.

It’s that they feed the design process in such a way that the resulting service ecosystem is set up in such a way that notifications won’t get lost. I know that that sounds kind of recursive, but that’s how it works. Either subscribers take care of their SLA allowing them to process the online stream of events, or they should subscribe to a different pipe (which will have different SLA requirements, but maybe they can deal with those).

It make sense to have multiple pipes for the same logical schema.

It’s practically a necessity to make pub/sub a feasible solution.

Related Content

MSDN article on messaging and lost messages

Durable messaging dilemmas

Additional logic required for service autonomy

More in depth example on events and pub/sub between services

Consumer-Driven Contracts

Dennis asked this:

There are different design choices in a SOA, even when you already have identified the services. I have a simple example that I would like to share:

Imagine a order-to-cash process. One part of that process is to register an order. Suppose we have two services, Order Service and Inventory Service. The task is to register the order and make a corresponding reservation of the stock level. I would be pleased to have the groups view on the following 3 design options (A, B, C):

A.
1. The “process/application” sends a message (sync or async) to “registerOrder” on the Order Service.
2. The “process/application” sends another message (sync or async) to “reserveStock” on the the Inventory Service.

B.
1. The “process/application” sends a message (sync or async) to “registerOrder” on the Order Service.
2. The Order Service sends a message (sync or async) to “reserveStock” on the the Inventory Service.

C.
1. The “process/application” sends a message (sync or async) to “registerOrder” on the Order Service.
2. The Order Service publishes an “orderReceived” event.
3. The Inventory Service subscribes to the “orderReceived” event .

On the whole “already identified the services” thing – naming a service doesn’t mean much. It’s all about allocating responsibility, and until that’s been done, those “services” don’t give us very much information.

Business Services

If we were to view this example in light of business services, and look at the business events that make up this process, maybe we’d get a different perspective.

Three business services: Sales, Inventory, and Shipping.

In Sales, many applications and people may operate, including the person and the application he used to submit the order. When the order is submitted and goes through all the internal validation stuff, Sales raises an OrderTentativelyAccepted event.

Inventory and Orders

Inventory, which is subscribed to this event, checks if it has everything in stock for the order. For every item in the order on stock, it allocates that stock to the order and publishes the InventoryAllocatedToOrder event for it. For items/quantities not in stock, it starts a long running process which watches for inventory changes.

When an InventoryChanged event occurs, it matches that against orders requiring allocation – if it finds one that requires stock, based on some logic to choose which order gets precedence, it publishes the InventoryAllocatedToOrder event.

Sales, which is subscribed to the InventoryAllocatedToOrder event, upon receiving all events pertaining to the order tentatively accepted, will publish an OrderAccepted event.

Orders and Shipping

When Inventory receives the OrderAccepted event, it generates the pick list to bring all the stock from the warehouses to the loading docks, finally publishing the PickListGenerated event containing target docks.

When Shipping receives the PickListGenerated event, it starts the yard management necessary to bring the needed kinds of trucks to the docks.

What else is possible

I could go on, talking about things like the maximum amount of time stock of various kinds can wait to be loaded on trucks, subscribing to earlier events to employ all kinds of optimization and prediction algorithms, having a Customer Care service notifying the customer about what’s going on with their order (probably different for different kinds of customers and preferred communication definitions). Obviously, we’d need a Billing service to handle the various kinds of billing procedures, whether or not the customer has credit, pays upon delivery, etc.

It turns out that many business domains map very well to this join of SOA and EDA.

What an ESB is for

When we have these kinds of business services primarily publishing events and subscribing to those of other services, you don’t need much else from your “enterprise service bus”. All sorts of transformation, routing, and orchestration capabilities don’t come into play at all.

In all truthfullness, those bits of functionality are really just a historical artifact of their broker heritage.

Don’t get me wrong, sometimes a broker is a nice thing to have – behind a service boundary in order to perform some complex integration between existing legacy applications.

Just keep that stuff in its place – not between services.

Complex Event Processing

We can look at how Sales transitions an order from being tentatively accepted to being accepted as requiring event correlation around InventoryAllocatedToOrder events. This isn’t exactly “complex” in its own right. If there were some kind of CEP engine that did this for us out of the box, it might be a possible technology choice for implementing this logic within our service.

As we add more concerns, like time, we may find new ways to make use of this engine. For instance, if the time to provide the order to the customer is approaching, we may choose to split the order into two – accepting one for which we have all the stock allocated, and leaving the second as tentatively accepted.

Summary

While it is difficult to move forward on service responsibility without discussing the events it raises and those it subscribe to, the whole issue of CEP can be postponed for a while.

Although there aren’t many who would say that EDA is necessary for driving down coupling in SOA, or that SOA won’t likely provide much value without EDA, or that SOA is necessary for providing the right boundaries for EDA, it’s been my experience that that is exactly the case.

CEP, while being a challenging engineering field, and managing the technical risks around it necessary for a project to succeed in some circumstances, and really shines when used under the SOA/EDA umbrella, it should not be taken by itself and used at the topmost architectural levels.

Related Content

SOA and Enterprise Processes

How client interaction fits with SOA

Time and SOA

Durable Messaging for Fault-Tolerant Services

And if you’re wondering about how to handle all that complexity inside services (different kinds of billing, periodic tests for electronics inventory, etc), you might like listening to this podcast about business components.

Watch that first step

Although sometimes described as the first step of an organization moving to SOA, web-service-izing everything results in synchronous, blocking, request/response interaction between services. The problem being that if one service were to become unavailable, all consumers of that service would not be able to perform any work. With the deep service “call stacks” this architectural style condones, the availability and performance of the entire organization will be dictated by the weakest link.

So, while I’d agree that many organizations do need to take this step, I’d caution against going into production at this step.

Pub/Sub Considered Helpful

When services interact with each other using publish/subscribe semantics we don’t have that technical problem of blocking. Subscribers cache the data published to them (either in memory or durably depending on their fault-tolerance requirements) thus enabling them to function and process requests even if the publisher is unavailable.

Consider the following scenario:

Let’s say we have an e-commerce site, a part of our Sales service responsible for selling products. Another service, let’s call it merchandising, is responsible for the catalog of products, and how much each product costs. Sales is subscribed to price update events published by Merchandising and saves (caches) those prices in its own database. When a customer orders some products on the site, Sales does not need to call Merchandising to get the price of the product and just uses the previously saved (cached) price. Thus, even if Merchandising is unavailable, Sales is able to accept orders. This is a big win as our merchandising application is not nearly as robust as our sales systems.

Yet, there are scenarios where data freshness requirements prevent this.

Too Much of a Good Thing?

Technically, the above story is accurate. There is nothing technically preventing Sales from accepting orders. Yet consider a scenario where Merchandising is down or unavailable for an extended period of time. While this may not be entirely likely for two servers in the same data center, consider physical kiosks which customers can use to buy products. Those kiosks may not receive updates for days. Should they accept orders?

That’s really a question to the business. If pricing data is stale for a time period greater than X, do not sell that item. The value of X may even be different for different kinds of products. Keep in mind that this issue only arose since we architected our services to be fully autonomous. In a synchronous systems architecture, this issue would not come up. As such, it is our responsibility as architects to go digging for these requirements as well as explaining to the business what the tradeoffs are.

In order to have more up to date data, we need to invest in more available hardware, networks, and infrastructure. This needs to be balanced against the predicted increase in revenue that more up to date (read higher) prices would give us.

You Can Get What You Pay For

Beyond the additional cost of writing that additional logic, and the perceived increased complexity, another difference to note between this architectural style and the synchronous/traditional one is that it puts control of spending back in the hands of business.

In a synchronous architecture, in order to achieve required performance and availability, all systems need to be performant requiring across the board investments in servers, networks, and storage. Without investing everywhere, the weakest link is liable to undo all other investments. In other words, your developers have made your investment choices for you. Scary, isn’t it.

A more prudent investment strategy would prefer spending on services that give the biggest bang for the buck, better known as return on investment. A pub/sub based architecture allows investing in data-freshness where it makes the most sense. For example, in sales of high profit products to strategic customers rather than inventory management of raw materials for products slated to be decommissioned.

That sounds a lot like IT-Business Alignment.

Maybe there’s something to this SOA thing after all…

Read more about:

7 Questions for Service Selection

7 Questions around data freshness 

Event-Driven Architecture and Legacy Applications

Autonomous Services and Enterprise Entity Aggregation

Or listen to a podcast describing Business Components, the connection of pub/sub and SOA.

“We have three high level business services: Engagement, Management, [and] Production. [...] they represent different operational ambitions. Engagement is all about quantity and then filtering. Management is about the quality and Production is about realising the benefits.”

Services are not about “are you being served?”

They’re not about re-use, and barely about use. Events are what it’s all about.

Each service has its own responsibility and does what it needs to do, business-wise, to achieve its goals. Whether it’s about increasing the number of leads, ensuring high-profile clients get good service, or maximizing equipment utilization, services take responsibility.

I know I harp on this a lot.

It’s because it’s that important.

In regular client-server development, the server is responsible for providing the client with all CRUD (create, read, update, and delete) capabilities. However, when users look at data they do not often require it to be up to date to the second (given that they often look at the same screen for several seconds to minutes at a time). As such, retrieving data from the same table as that being used for highly consistent transaction processing creates contention resulting in poor performance for all CRUD actions under higher load.

A Scalable Solution

One of the common answers to this question is for the server/service to publish a message when data changes (say, as the result of processing a message) and for clients to subscribe to these messages. When such a notification arrives at a client, the client would cache the data it needs. Then, when the user wants to see a grid of data, that data is already on the client. Of course, this solution doesn’t work so well for older client machines (like some point of service devices) or if there are millions of rows of data.

The thing is that this solution is one implementation of a more general pattern – command query separation (CQS).

Command Query Separation

Wikipedia describes CQS as a pattern where "… every method should either be a command that performs an action, or a query that returns data to the caller, but not both. More formally, methods should return a value only if they are referentially transparent and hence possess no side effects."

Martin Fowler is less strict about the use of CQS allowing for exceptions: "Popping a stack is a good example of a modifier that modifies state. Meyer correctly says that you can avoid having this method, but it is a useful idiom. So I prefer to follow this principle when I can, but I’m prepared to break it to get my pop."

So, how does separating commands from queries and SOA help at all in getting data to and from a UI? The answer is based on Pat Helland’s thinking as described in his article Data on the Inside vs. Data on the Outside.

Services Cross Boxes

The biggest lie around SOA is that services run.

Let that sink in a second.

Sure services have runnable components, but that’s not why they’re important.

I’ll skip the books of background and cut to the chase:

Services communicate with each other using publish/subscribe and one-way messaging. Services have components inside them. Inside a service, these components can communicate with each using synchronous RPC, or any other mechanism. Also, these components can reside on different machines.

This is broader than just scaling out a service. There can be service components running on the client as well as the server.

SOA & CQS

Combining these two concepts together, here’s what comes out:

In this solution there are two services that span both client and server – one in charge of commands (create, update, delete), the other in charge of queries (read). These services communicate only via messages – one cannot access the database of the other.

The command service publishes messages about changes to data, to which the query service subscribes. When the query service receives such notifications, it saves the data in its own data store which may well have a different schema (optimized for queries like a star schema).

The client component which is in charge of showing grids of data to the user behaves the same as it would in a regular layered/tiered architecture, using synchronous blocking request/response to get its data – SOA doesn’t change that.

Composite Applications

Although the client side components of both the command and query services are hosted in the same process, they are very much independent of each other. That being said, from an interoperability perspective (the one that most people attribute to SOA), all of the client-side components will likely be developed using the same technology – although there are already ways to host Java code in .NET and vice-versa.

Of course, once we talk about web UI’s things are a bit different – but still similar. While web-server-side there may be a level of independence, for browser side inter-component communications we’re still likely to target javascript. There, I’ve managed to say something technical supporting mashups and SOA without lying through my teeth.

On the Microsoft side with the recent release of the Composite Application Guidance & Library (pronounced "prism") I hope that more of these principles will be reaching the "smart client". The command pattern is especially critical in maintaining the separation while enabling communication to still occur so I’m glad that, as one of the Prism advisors, I was able to simplify that part (Glenn still has nightmares about that rooftop conversation).

Publish / Subscribe

In the "scalable solution" section up top I mentioned how publish/subscribe to the smart client is really just one implementation of CQS and SOA. So, how different is it really?

Well, there will probably be a different technology mapping. Instead of a star-schema OLAP product, we might simply store the published data in memory on the client. That is, if you designed your components to be technology agnostic.

In terms of the use of nServiceBus, the same component is going to be subscribing to the same type of message – all that’s different is that now every client will be having data pushed to them rather than this occurring server-side only.

You could have the same code deployed differently in the same system – stronger clients subscribing themselves, weaker ones using a remote server. Web servers would probably be considered stronger clients. This kind of flexible deployment has proven to be extremely valuable for my larger clients. The added benefit of enabling users to work (view data) even while offline (somewhere there’s no WIFI) is just icing on the cake.

A Word of Warning

Once the client starts receiving notifications, and handling those on a background thread (as it should) the code becomes susceptible to deadlocks and data races. Juval does a good job of outlining some of those with respect to the use of WCF. Prism doesn’t provide any assurances in this area either.

Summary

NServiceBus is not designed to be used for any and all types of communication in a given architecture. In the examples above, nServiceBus handles the publish/subscribe but leaves the synchronous RPC to existing solutions like WCF. Not only that, but synchronous RPC does have its place in architecture, just not across service boundaries. In all cases, data is served to users from a store different from that which transaction processing logic uses.

Command Query Separation is not only a good idea at the method/class level but has advantages at the SOA/System level as well – yet another good idea from 20 years ago that services build upon. Making use of CQS requires understanding your data and its uses – SOA builds on that by looking into data volatility and the freshness business requirements around it.

Finally, designing the components of your services in such a way that their dependency on technology is limited buys a lot of flexibility in terms of deployment and, consequently, significant performance and scalability gains.

Simple, it is. Easy, it is not.

This innocuous question comes up a lot. Usually I get this question after a short problem domain description. One of these came up on the nServiceBus discussion groups. Ayende took it and ran with it turning it into a nice blog post, An exercise in designing SOA systems. I’ve been meaning to write something myself. Bill put up a response already in his Service Granularity Example. So, I’m late to the party, again, but here we go.

It’s almost impossible to know, right away, which services are appropriate.

So, I’m going to focus more on the process of getting there, rather than describing the solution itself.

The domain deals with a placement agency placing physicians in positions at hospitals.

1. So, what does it actually do?

In Ayende’s post, he describes several services, but I’d rather look at them as use cases: registering an open position, registering a candidate, verifying their credentials, etc. It’s worth going through this requirements process. It doesn’t necessarily translate immediately to services, but there’s value in it.

2. What does it do it to?

We should also be looking at the data model, an entity relationship diagram (ERD) , where we see that we may have placed a certain physician at a number of positions. It’s also important for us to know about under which circumstances a physician finished their employment at a previous position before, say, trying to place them at a position in the same hospital or chain of hospitals. Don’t go thinking that this what the database schema will look like, it’s all about understanding connections between various bits of data.

3. When does that happen?

The next step is to map the uses cases above to the entities in the ERD, which entity is used in which use case. It’s also important to differentiate between entities (or even more importantly, specific fields of entities) that are used in a read-only fashion within a given use case. For instance, when registering a new position, we’ll want to check that against other open positions in the same hospital so we don’t end up registering the same position twice. Also, we might want to suggest verified physicians whose credentials match the position’s requirements. Data we wouldn’t be interested in might be which other physicians we placed at that hospital.

4. What just happened?

Another valuable perspective on the problem domain is the business process view – what are the interesting business events in the system and how they unfold over time. For instance, physician registered, position opened, physician’s credentials verified, and physician placed in position (or position filled by physician) are events that describe a different business perspective than use cases.

5. How do I decide?

Once we know what events there are, we can start looking at what kind of decisions we might want to make when those events occur and what data we’d need to make those decisions. These decisions may be as simple as updating a database or sending an email to a user. They also may include more advanced logic like when the profitability of an agreement with a specific hospital chain changes, prefer placing physicians in positions in that chain over others.

6. How do I deal with all this information?

After we have all of this information, we can start looking for cohesive bunching across all of these axes using these rules:

• Data that is modified by a use case gets published as an event.
• Data that is required by a use case for read-only purposes, arrives as the result of subscribing to some event.

Look for rules that differentiate behaviour based on the properties of data. Look for a correlation to some business concept. For instance, physicians probably won’t be changing their specialization, and open positions often deal with a certain specialization. Therefore, specific data instances tied to two different specializations can be said to be loosely coupled.

7. Which property slices across the domain?

Even though the ERD may not have made it clear, and the use cases didn’t show any particular break-down, nor did the events call out this point, the key to finding the way a business domain decomposes into services lies in decoupling specific data instances.

Actually, at this point we can clump autonomous components (mere technical bits) that handle a single message, into more granular business components.

If you think about it, it makes a lot of sense. The kind of credential checking you’d do for physicians specializing in brain surgery would likely be different than for general practitioners. The kind of information you’d store would, therefore, also be different.

But, which services do I need?

Quite frankly, I don’t have enough information to know.

But if we had continued this conversation, going through issues like transactional consistency, availability requirements, and other non-functional issues we could have  gotten there.

If there’s one thing that I hope you got out of this, it’s that the questions are what’s important. The iterative process of looking at the problem domain from various perspectives, incorporating the new-found knowledge, and asking more questions is what leads us to a solution. But we don’t stop there. We keep looking for characteristics which split services apart into business components, and for consistency requirements that brings autonomous components together into services.

It’s not easy, but by focusing on these simple questions, you can get to a coherent service oriented architecture.

What’s on first?

Many times I see SOA projects integrate existing/legacy systems focusing only on getting those systems to talk to the ESB (bits flowing) using the right structures (canonical schema, oy). However, little attention is often given to where that integration code runs – in other words, which endpoint does the rest of the system talk to? Who’s in charge of the integration?

The answer is usually muddled – sometimes its the ESB itself (serving more as an EAI broker at that point), sometimes its some DLL that the calling service uses, but I VERY rarely hear anything about the actual process that’s hosting that code, the endpoint itself, or anything that will help us deal with Service-Level Agreements.

No. What’s on second.

If you can’t (or don’t want to) change the legacy application at all, I suggest setting up a new endpoint and an additional process which listens on that endpoint. That process is in charge of communicating with the legacy application and translating whatever is going on to the messaging semantics of the SOA environment. Not everything may be publish/subscribe – other systems may send command messages to the endpoint, resulting in API calls on the legacy application.

One of the things that the process can/should do, is to subscribe to events/messages from other services and feed the relevant information to the legacy application. At times this will be done on an as-needed basis from the legacy application’s perspective – it will call some API/web service that will need to communicate with the afore-mentioned process, and the process will return the data needed.

How is playing a different game

From the perspective of all the other services, the legacy application might as well not even be there – they communicate via the regular messaging semantics with everything.

What is important to understand is that developing that kind of process is not a trivial undertaking. In DDD terms, it can be called an Anti-Corruption Layer, as it prevents the legacy from influencing the structure of any other service. This procedure is one of the ways one can go about slowly getting data and functionality off of mainframes and into more versionable and change-friendly (and cheaper) environments.

I don’t give a darn!

Oh, that’s our esb.