So it’s been about 6 months since my last NServiceBus post and since then about 1000 new people have subscribed to this blog so they might not know anything about it. For a bit of history, see the post (from almost exactly a year ago) describing the 1.9 release of NServiceBus here.

What’s New

The quickly approaching next release of NServiceBus will be version 2.0 and is a big step from 1.9. After 2 betas and 2 release candidates, this version has had a longer stabilization period than any of the versions so far (1.4-1.9). Many of my clients are already using it in production and are very pleased with it. I’ve heard similar reports from others in the community (now with over 500 members in the discussion group). There have been almost 10,000 downloads since the version 1.9 release and in every country I visit I meet people using NServiceBus in new and interesting applications.

With my appearance on Hanselminutes, many in the mainstream .NET industry have started taking a look at NServiceBus. That, and the fact that Microsoft’s Oslo technology has now taken a very data-driven turn (rather than its original service-oriented direction).

Interestingly enough, I’ve been hearing more and more reports about people using NServiceBus as a developer-friendly API on top of other technologies. This includes BizTalk and even Neuron. I never thought that people would take the pluggability of NServiceBus that far.

So, what is NServiceBus?

Well, it’s a service bus, y’know, like an ESB – just an open-source one.

All kidding aside, in a nutshell, it gives you an easy way to integrate transactional messaging into your applications.

One of the reasons why you might want to do that is so that you don’t lose messages containing valuable data when IIS recycles your AppDomain, every 15-20 minutes (as I wrote about in this MSDN magazine article).

There are many other nice things in there, like the ability to unit test your service layers and long-running processes but you can read more about that here…

Documentation

One of the biggest differences to NServiceBus in this release is documentation.

A lot of work has gone into the NServiceBus.com site to help developers hit the ground running with NServiceBus, including the more advanced aspects of transparent scale-out with the distributor and multi-site communications.

There is still work to be done in this area but feedback so far has been extremely positive (except for some grumblings from certain old-timers saying that if they could figure it out by themselves, well, you know the rest).

In Closing

If you’re building a distributed enterprise .NET system, take 5 minutes, download it, and see transactional publish/subscribe messaging working on your machine without any big heavy-weight middleware.

www.NServiceBus.com

Yesterday me and Scott virtually sat down to have a chat about NServiceBus and service buses in general. While we didn’t get in to many of the more advanced parts, you may find it an interesting introduction to the topic as well as saving yourself the costly mistake of implementing a broker instead of a bus (yes – they’re actually two different things).

Take a listen.

It’s hard to believe that this continues to pop up even as WCF is reaching its fourth version (emphasis mine):

“A common complaint is that the first call on a client object takes some disproportionately large amount of time, usually ten seconds or more, while successive calls are instantaneous. There are many reasons why this might happen so there’s no generic resolution for this problem.” — Nicholas Allen

The thing is that there IS a generic solution to this problem.

It’s queued messaging.

The only thing is that you have to give up talking to your services as if they were regular objects – calling methods on them and expecting a response. In other words, designing a distributed systems isn’t like designing a regular OO system just with some WCF sprinkled on top.

Even when trying to do fire and forget messaging on top of WCF (void method calls with the OneWay attribute), the underlying channel can still block your thread, as Nick mentioned.

A queue isn’t an implementation detail.
It’s the primary architectural abstraction of a distributed system.

I’ve gotten this question several times already but now companies are beginning to look for performance comparisons in making decisions around the use of nServiceBus. It’s often compared to straight WCF, BizTalk, and now Neuron ESB. In Sam’s recent post he posts to a case study of Neuron doing 28 million messages an hour. That’s far more than I’ve ever heard quoted for BizTalk.

Disclaimer

Before giving some numbers, please keep in mind that high performance of system infrastructure does not necessarily by itself mean that the system above it is running that fast. For instance, you may have server heartbeats running really quickly but the time it takes to save a purchase order borders on a minute. So, please, take all benchmarks with a grain of salt, or two, or a whole shaker-full.

While I’m not at liberty to say on which specific domain/company these numbers were measured, I can say that we had the full gamut of “stateless services”, statefull services (sagas), number crunching, large data sets, many users, complex visualization, etc. Also, this wasn’t the largest installation of nServiceBus that I’m aware of, but its the one I have the most specific numbers for.

Setup

OK, so using the default nServiceBus distribution using MSMQ, on servers where the queue files themselves were on separate SCSI RAID disks, we were pumping around 1000 durable, transactionally processed messages per second, per server. That means that similar to the Neuron case, no messages would be lost in the case of a single fault per server per window (time to replace a failed disk set at 3 hours from failure, through detection, to replacement per site – but that’s more an operational staffing concern, not the technology itself).

So, that’s 3.6 million messages per hour per server, at full load. We had a total of 98 servers doing these kinds of processing, not including web servers, databases, etc. Keep in mind that web servers would be communicating with other servers using nServiceBus, but that would maybe be an unfair comparison to the Neuron numbers.

Server Breakdown

Anyway, the 48 number crunching servers (blade centers) we had were at full load, so we were pumping more than 170 million messages there. Keep in mind that those servers had a really fast backbone so weren’t held up by IO. Your environment may be different.

Another 30 (regular pizza boxes) were doing our sagas. Saga state was stored in a distributed in-memory “cache”, so once again IO wasn’t an issue for processing those messages. We were at about 70% utilization there, coming to just over 100 million messages an hour.

The last 20 were clustered boxes (fairly expensive) that handled the various nServiceBus distributor and timeout manager processes were at full load since they handled control messages for all the servers as well as dynamically routing the load. However, on those boxes we used much higher performance disks for the messages, since they had to feed everything else, capable of doing, on average, around 5000 messages a second. That adds up to 360 million messages an hour.

Unnecessary Durability

Later, we moved a bunch of messages that didn’t need all that durability and transactionality off the disks, pushing the total throughput over 1 billion messages an hour. That was about 100 million per hour durable, 900 million per hour non-durable. You can guess that we were left with plenty of IO to spare at that point while we weren’t yet pushing the limit of our memory.

One thing that’s important to understand is the size of the messages that didn’t require durability was less than 1MB, with most weighing in under 10KB. Also, since most of those messages were published, less state management was required around them, enabling us to further improve performance.

Summary

NServiceBus didn’t give us all that by itself. It was the result of skilled architects, developers, and operations staff working together for many iterations, deploying, monitoring, re-designing, etc. You need to understand your technology, your hardware, and your specific performance, availability, and fault-tolerance requirements if you want to get anywhere.

There’s no magic.

I didn’t see the number or kinds of servers involved in the Neuron case study so this wasn’t ever really a comparison. Nor or we talking about the same system here.

So, please, don’t base your decisions on arbitrary numbers. Spend some time setting up a scaled down version of your target architecture with all the relevant technologies and measure. Be aware that you want high performance end to end, not just of the messaging part. At times, it makes sense to actively throw away messages (of the non-durable, published kind) to help a server come online faster especially after a restart.

Thus ends the tale of another “benchmark”.

Following my last post (From CRUD to Domain-Driven Fluency) a bunch of questions have started popping up. One that I received via email from a client up in Ireland particularly caught my eye, so here it is:

Hi Udi, I think  I see the point about the domain-driven approach but I’m wondering what my messages will look like. If it’s this:

IAppointment InsertInterview(Guid recruiterId, Guid applicantId, Guid appointmentId); OR

IRecuiter UpdateRecuiter(IRecuiter recruiter); (passing in an operation flag attached to the IRecuiter object) OR

IRecuiter UpdateRecuiter(IRecuiter recruiter); (setting a state flag on the relevant object and have the business object check the flag and behave according the state change)

Hope I’m not way off

Sean

Well, Sean, first of all – messages don’t look like functions. They’re a lot more like structures – data transfer objects. In this case, you’d probably be looking at a ScheduleInterviewMessage that had the relevant fields. It would look something like this:

   1:  using System;

   2:  using NServiceBus;

   3:  using System.Xml.Serialization;

   4:   

   5:  namespace Messages

   6:  {

   7:      [Serializable]

   8:      [Recoverable]

   9:      [TimeToBeReceived("0:01:00.000")]

  10:      public class ScheduleInterviewMessage : IMessage

  11:      {

  12:          public Guid InterviewerId;

  13:          public Guid CandidateId;

  14:          public DateTime RequestedTime;

  15:   

  16:          [XmlAnyElement]

  17:          public object extra;

  18:      }

  19:  }

Before we go on, I want to explain what we see. The “recoverable” attribute is the way we indicate to the infrastructure that these messages should not be lost in case a server fails, there are network problems, etc. In essence, it does durable, store-and-forward messaging. This will create an environment in which, in the case of network problems, these messages will be written to disk. That’s a good thing, since once connectivity comes back or the server boots back up, the messages will still be around and can be sent.

Now these messages are fairly small, so even at a relatively high load, we shouldn’t be chewing through too much of our expensive, small, high performance local disks. However, if these messages were bigger, we may fill up our disks before connectivity comes back, and we all know what happens to Windows boxes when there’s no room on the file system left:

In order to prevent our system from Denial-of-Servicing itself we need to make those messages clean themselves up. That’s what the “TimeToBeReceived” attribute is for. The amount of time that if a message had not yet been received by the other side that it will be deleted. This could be that the message even made it to the other machine, but the process handling those messages was down. You wouldn’t want to be filling the other side’s disk either causing them to crash, would you? This protects both parties.

The way to figure out how long to set is by looking at the smallest amount of durable storage you have available at your nodes, divide that by the size of the average message, and then again by the rate you need to process messages – and leave yourself at least 100% spare.

In other words, to build a robust system you not only will need to deal with lost messages, but you will be actively throwing messages away.

Finally, that last “XmlAnyElement” attribute is there for versioning. As we version our system and schema, we’ll be adding fields to the message. However, an old client may be talking to a new server, or vice versa. Since we wouldn’t want data to get lost just because of serialization. In a future post, I’ll show how to set up a message handler pipeline exactly for these issues.

Now that we’ve covered all the intricacies around messaging, we can see how the code that handles that above message looks:

   1:  using System;

   2:  using Messages;

   3:  using NServiceBus;

   4:  using NHibernate;

   5:   

   6:  namespace Server

   7:  {

   8:      public class ScheduleInterviewMessageHandler :

   9:                   BaseMessageHandler<ScheduleInterviewMessage>

  10:      {

  11:          public override void Handle(ScheduleInterviewMessage message)

  12:          {

  13:              using (ISession session = SessionFactory.OpenSession())

  14:              using (ITransaction tx = session.BeginTransaction())

  15:              {

  16:                  ICandidateInterviewer interviewer = session.Get<ICandidateInterviewer>(

  17:                          message.InterviewerId);

  18:                  ICandidate candidate = session.Get<ICandidate>(

  19:                          message.CandidateId);

  20:   

  21:                  interviewer.ScheduleInterviewWith(candidate)

  22:                          .At(message.RequestedTime);

  23:   

  24:                  tx.Commit();

  25:              }

  26:   

  27:              // publish new appointment data

  28:          }

  29:      }

  30:  }

If you’ve read this far and have more questions, please feel free to send them my way. If you’re at a more time-critical part of your project and need an answer quickly, we can set up a skype call. This has been working quite well for many of my overseas clients (shout out to the guys in Ireland and Florida).

Until next time

An interesting development.

A while ago, me and Ron Jacobs (virtually) got together and did a couple “rapid responses” to questions on the MSDN architecture forums, and I just noticed that they’re online. The really great thing is that there are transcripts! For your convenience, I’ve included them here.

By the way, if you’re looking for more Q&A style info, check out the Ask Udi podcast. If you have a pressing question and need a shorter turn around time than the month or so it usually takes me for the podcast, send me an email to OnlineConsultation@UdiDahan.com.

Number 1

Ron: Hey, welcome back to ARCast Rapid Response. This is your host Ron Jacobs and today I’m looking at the MSDN architecture forum where I see this message from “theking2.” Yeah? OK, so “king,” he says, he’s building a distributed architecture that has a number of external systems. These external systems interface through a telnet connection and so they accept commands and return results as ACKS or NACKS.

Typically these systems have limited resources for the number of simultaneous sessions you can open, so, five to fifty depending on the system. What he did to get around this, was, he created some Enterprise Services objects and some pooled objects that set up these connections and then he has some Web services. The Web services are going to receive an incoming message. They’re going to call these pooled COM+ objects and they’re going to make the telnet calls to the external systems. Sounds interesting.

He says, after a year of production it has become apparent that some of the external systems are not performing very well. He says the bulk of the requests, but not all, to the external systems can be done asynchronously. So, he’s opting for a message queue-based solution using pseudosynchronous calls whenever a direct response is needed.

So, the question is, at what layer would message queuing make most sense?

So, should the clients, this Web service that receives the message — should it do a queue? Put a message in the queue and then the COM+ objects would pop off or they have some central Web services that would pop it. So, the central Web services or these Enterprise Service objects? Or maybe just a communication at the top of the telnet. He says this is the first time when he’s using message queuing.

On the line with me I have Udi Dahan, the Software Simplist from Israel.

Udi, this is a very interesting application and my first gut reaction is, does it really matter where you put the queuing?

Udi Dahan: Well, actually I took a look at it as well and I’d have to say that it does because the problem that he’s trying to solve isn’t that clear. We know that there is some sort of performance problem but we’re not quite sure where it is. We know that there are long and varying latencies in the responses but we’re not really quite sure why.

While we know that their external system is a bit slow but our choice of where to put the queue will probably have an impact, obviously on the development model of the clients and the Web services as well as how those external systems would work. So, I’d have to say that choosing the correct place to put the queue is important.

Ron: Well, let me interject something here because what you said just made me think. Now, if the problem is that these external systems are slow and limited number of connections, the first question we ought to ask is, does queuing help this situation at all?

Udi: Well, that’s probably a good first step. I mean every single time someone comes with a solution and then says, “OK, what’s the problem,” it’s always a good thing to check that solution first.

It looks like the problem that he has here has to deal with or the reason that he wants to use a queue is to do some kind of load leveling. He’s getting too many requests or at too high a rate from his clients and external Web services and external web applications more than his back end systems can use. So, using a queue as a load leveling mechanism is definitely the right way to go. So, from that perspective I think that putting a queue somewhere in there is a good idea.

Ron: OK. So then if you put a queue, it seems to me that it’s not going to make that much difference which layer you put the queue, would it?

Udi: Well, it might for the main reason that you really have to look at where his bottleneck is and that’s his back end systems. The bottleneck also has to do with the number of connections that can be opened and the number of sessions that can be opened. The place that I’d be looking at doing that is probably between those pooled COM+ objects and his central Web services for the main reason that that really gives a nice encapsulation in terms of the Web services towards both his organization’s internal services if they are other Web services, web applications or clients and everybody else that’s going on out there while keeping that abstraction out of the way.

So, the choice of using pooled COM objects is one of the ways he does the load leveling now. One of the problems he has is that it doesn’t seem to be doing that much for him because the switches and knobs that are available in COM+ in order to do that load leveling aren’t that great. What I’d be looking at in his situation is to put a queue in there but on the back side of that queue, not talking directly to the external system but doing something with WCF.

WCF has an incredible amount of switches and knobs in order to do the load throttling and the number of threads that are open. He could also do that on a large number of URIs in order to sort of split up the load from that perspective allowing him to cache results quite a bit better. So, that’s where I’d be looking at too. Just throw away those COM+ objects, put WCF in there, use the MSMQ binding and start configuring things from there.

Ron: There’s a lot of stuff in the message, but I think his core concern is performance. He mentions pseudosynchronous calls. I think by that he means, a message comes in to the web service, he’s going to drop something on the queue and then hold that message response until he gets a response back from a queue. So, it’s sort of synchronous but sort of not synchronous. So, in effect he’s kind of waiting on a queue instead of waiting on the pooled object to make this outbound telnet call.

I could agree if you said, “Well, look, our big problem is that we keep getting time outs because when we go to get a COM+ object from the pool, COM+ waits for a while and then it says, “Hey, there’s no object available’ and it returns an error,” then the queue is definitely going to help that problem. But in terms of the sheer through put or performance of the system, this is not going to help at all. It’s going to still be the same performance.

Now if you said, “Oh look, we can do some of this work kind of at a later point in time, ” well queuing doesn’t allow you to time shift the work. Right? So, if you said, “Look we can rethink this solution.” So you get a message in, we stuff something in a queue that we’ll deal with later, and then very quickly return a response like some kind of a number like, hey “your transaction number blah, blah, blah, will be processed later, it’s queued for processing, ” whatever.

I mean that introduces a lot of complexity in the system but it clearly would provide better response at the Web service layer. What do you think?

Udi: Well I think that at the most basic level, his throughput is dictated by his back end systems. From what he seems to be describing, every single request that is going through there, has to hit that back end system. If he has a limited number of back end systems that are supporting a limited number of connections, that’s going to limit his throughput no matter what technology he puts in front of that. So that’s at the core level. You just can’t get away from that.

The one thing that I would agree with you in your description there is the choice of using those COM+ objects. I mean COM+ was a great technology when it came out. The problem occurs, of course, when we start getting into larger and larger delays around the response time and we start getting all sorts of time out exceptions and things like that. So in that respect, I definitely say you know, take a step back from there.

But in terms of everything that he has around there, the queue isn’t going to make the back end system run any faster. What it will do is definitely complicate his system because he’s taking something that used to be synchronous and making it asynchronous. Writing Web services in order to handle that, I mean just adding a bunch of threads in order to listen to queues is not going to make things any simpler.

However, what it might do is to improve the resource usage of those Web services, OK? So instead of having those Web services have a bunch of threads open, waiting for the response coming back from those COM+ pooled objects, those threads could be relinquished and really just be triggered back up when a response comes back from the queue.

So I don’t see an improvement in the kind of solution that MSMQ or queuing would put in there in terms of the latency — how long it would take for a response to get back. However, I do see an improvement in terms of the resource usage of all the other players in the system.

Ron: I would agree with that. I would just say though that if you make the Web server that is hosting these Web services more resource efficient, maybe all you’re going to do is enable it to get more requests in queue the more quickly. Ultimately, this solution I think is going to solve a lot of problems related to time outs and server busy errors and that sort of thing, thread contentions, but not likely to increase overall performance.

But I definitely agree though. I would move this solution forward to WCF. I used to be on the COM+ team. COM+ was rolled into WCF so that it would have similar capabilities for pooling, instancing behavior, transactional support, those sorts of things. I would definitely move that forward into WCF.

OK! So great answer, Udi. Thank you so much for being on this ARCast Rapid Response.

Number 2

Ron: Hey this is Ron Jacobs back with another ARCast.TV Rapid Response. Today I’m joined by Udi Dahan, the Software Simplist from Israel.

Udi, I’m looking at the MSDN Architecture Forum and here’s a question from “blast.” Blast says he’s looking for where to put business rules. He’s developing a WinForm application. He uses data sets as the data layer, he says. He’s thinking about business rules and where to put them.

He says obviously, the more organized and centralized business rules are, the better. He’s tempted to put the business rules in the UI layer especially with the type data set. It makes a lot of sense there but not all rules belong on the client. He says some rules belong on the server, perhaps in a trigger.

So he’s asking where do you put your rules? How do you think about this problem, Udi?

Udi: Well, it looks like what he’s doing here is developing a two-tier client that is using WinForms and using datasets and speaking directly to the database. That in essence is part of his problem in that in terms of performance, he’d like to run more rules in the UI layer so that the user won’t be sending garbage to the database.

He also understands that because he’s building a multi-user system, there is a limited capability, in terms of concurrency, of actually having all the rules run correctly in order to make sure that everything is correct. So, his choice of an architecture, working two-tier is the main problem of why he has to fragment his business rules.

If he were to move towards a three-tier solution, that is put an application server between his smart client and the database, it would be a lot easier to put those business rules there. Now, once the business rules are out of the database, because again, we don’t have to deal with the concurrency issues once we have an application server and we’re using transactions there and we don’t have any disconnected problems, then what we can do is use those same DLLs, that same CLR code that runs the business rules, and deploy it client-side and use it there.

So, in terms of deployment, what we’d have is we’d have the same rules, both running client-side and server-side, whereas from a development perspective, we’d have them organized and centralized. That’s the way that I’d go about it.

Ron: Yeah, you know, I think conceptually I agree with you that a multi-tier solution would be a very good idea here. What I would probably think about conceptually, is breaking down rules into things that really ought to happen on the client-side. In particular, rules related to validation of data, so you know that you’ve got good and complete data before you ship it off to the server-side. Oftentimes you have to do that anyway because you have a button that shouldn’t be enabled until the data is valid, or something like that.

Udi: Absolutely.

Ron: Of course, we all know that if you have middle-tier web services, you must do validation both on the client-side and the server-side, because you must ensure that the valid data is received on the web server. So I agree with you that creating an assembly that you deploy on both sides is a good idea.

I would just expand on what you said a little bit and think about maybe on the server-side using a workflow foundation and business rules and workflow as a way to handle a lot of the heavier lifting, server-side validations and business rules that might require maybe sifting through more data or whatever kind of things, but server-side business rules that are more oriented towards business logic, and even if you have very, very data-intensive roles, then maybe some of those might even happen in the database. Don’t you agree?

Udi: Oh, absolutely. Absolutely. That’s something that I think often gets swept under the rug too much. Things like unique constraints and things like that are kinds of business rules. They protect the referential integrity and if we look at the alternatives, sometimes getting 10 million rows out of the database, in order to do some sort of unique email validation upon them, that’s just going to kill your performance.

There are certain things that it just makes sense to do them in the database, it’s just the best way to do it. The hard part, from a development perspective, is maintaining the coherence of your business rules. When you say, “OK, I want a single perspective, what are all the rules in my system?”

Even though we might try to keep it all CLR based, some of the things like unique constraints, like referential integrity, will be in the database. So, what I sometimes suggest to do is to have a separate solution, in terms of your development team, where you put all your business rules.

This includes both the SQL statements for defining your unique constraints and your referential integrity. Also put in that validation logic, your workflow that you’re going to be running server-side. If it’s AJAX controls and regular expressions that you’re going to be doing client-side in order to validate that data, absolutely make sure you have, from a development perspective, one place where you can go where you can see everything, because if you don’t do that [inaudible] can be running, and when things stop working, you won’t know how to debug it.

Ron: All right. Well, excellent answer. Udi, thank you so much.

I’ve been sitting on this post for a while, waiting, before outlining all the kinds of problems durable messaging doesn’t solve, I wanted to have a solution handy. Harry Pierson begins to outline the goodness that durable messaging brings to SOA, and in a later post on idempotence describes in general terms how it ties back into durable messaging and transaction – in essence describing a saga. Let’s do this in story form.

Since you’re concerned that maybe your shipping company’s servers may be down for some kind of planned (or unplanned) maintenance just as you’re trying to fulfill orders, you use a durable messaging solution there. What happens is that messages get written to disk on your end, and later the messaging tries to transfer the messages until it succeeds. So what’s wrong with that?

Well, let’s say that the shipping company’s servers went up in smoke (true story – broken down air conditioners + poor ventilation, you get the picture). Those servers aren’t going to be coming back online any second now. So, you have all these order messages buffering on your disk. Taking into account all the data, meta-data, XML, SOAP, encryption and everything, we may get up to 1MB per message.

And now’s holiday season and your company’s selling hand over fist, hundreds of orders per second from all over the world. So that means we’re eating up 100MB of disk per second, that’s 6GB a minute, and in under an hour of our shipping company’s servers going down – so do ours.

Durable messaging – yay? We don’t want to lose those orders, right? In short, durable messaging is an important part of the solution, but it’s not the whole solution.

[Continued next time...]

If you’re impatient and just want the solution, yes, nServiceBus give you all the tools you need.

In this podcast we’ll be discussing the issues around multi-threaded processing of messages by a service, specifically that the processing of message received second may be finished before that of the first. This scenario tends to rear its ugly head at higher levels of load and is critical for correctness in high-scalability environments.

Our long time listener Bill asks:

Hi Udi,

I have a question around processing of messages in proper order. When leveraging multiple threads to process messages in a message queue, it is possible for the second message in the queue to get processed before the first – especially if the first message is considerably larger than the second. I have taken a lot of care to make sure that messages are sent in the correct order, only to find that the receiving system can process them out of order anyway.

Consider a Policy Created notification, which must come before a Policy Approved notification. If both messages are sitting in the queue when the receiving service starts up, the approval message can be processed before the creation message. How can I make sure that message ordering is respected by the receiving system? I am using WCF/MSMQ as the underlying transport by the way. The only way I have found so far is to limit the receiving service to a single thread, which is by no means desirable.

Best Regards,

Bill

Download

Download via the Dr. Dobb’s site

Or download directly here.

Additional References

• Blog post: In-Order Messaging a Myth?
• Blog post: Handling Messages out of Order

Want more?

Check out the “Ask Udi” archives.

Got a question?

Send Udi your question to answer on the show.

“Is it still valid to assume it is more expensive to design a scalable system?”

In Gavin Terrill’s news post on InfoQ, Big Architecture Up Front – A Case of Premature Scalaculation? he covers one of the questions so many of my clients deal with:

“How hard will it be to make it scale later?”

This is a valid question, especially for companies/products just beginning their lifecycle. When the product/web site isn’t bringing in any revenue yet, how much money should we spend on getting it ready for that future success?

The answer to that question lies in treating capacity and scalability differently (source).

What I mean by that is designing for scalability, yet separating out all technological aspects of the scaling from the core solution. That way, you can start with simple, low capacity technologies that won’t be too expensive. As you grow, upgrade that infrastructure and plug it in to your solution. Arnon’s recent post on Tier Splitting touches on a project we worked on together where we designed it in a way that we could scale down to a single process on a single machine and scale out to a server farm, all without changing the core system.

Let me take the design of nServiceBus as an example:

One primary property of scalable systems is the explicit treatment of all IO/communication. This can be seen in the one-way messaging exposed by the Bus object. There is no immediately evident way to do synchronous RPC-style request/response. This design decision is taken up front. However, the way that messages are passed around is abstracted behind an ITransport interface. You can deploy the first version of your system on MSMQ, and as load increases, switch to a more performant solution like RV or MQ, just by changing configuration. WCF does this kind of abstraction as well.

Another important element of the scalability of a system is how workflow instances are persisted. This behaviour is also abstracted behind an interface – IWorkflowPersister. Start out persisting workflows to a database. As you grow, swap that out of a replicated in-memory cache. In any case, the interaction between workflow and messaging at the logical level is set up front. All the pieces of the design are there. Up front. Helping you design your core application in a way that won’t limit your scalability in the future.

This is plain Separation-of-Concerns; code that works with your specific ESB kept out of your business logic.

Design first, scale with technology later.

[Full disclosure: I'm an editor for InfoQ]