It seems to me that nowadays we work more and more with data streams and not data sets. On a transaction-to-transaction basis, I think it’s an uncommon application that uses more data than can fit into several gigabytes of RAM (obvious exception: multimedia data).

While data stream processing is the heartbeat of many verticals, I’m seeing another trend there as well – the use of historical data as a part of that data stream processing. Some people have begun calling this Complex Event-Stream Processing (CEP), and the analysts are already beginning to eat it up. Regardless, the problem is that it is difficult to hold all historical data in memory so that when events arrive we can process them quickly.

So, my bottom line is that we’re being hit on multiple fronts – both the rate at which we need to process events and the amount of data required to process each event. Multiple cores help a bit, but probably not enough to discount scaling up to more machines. All this at the end of the day points out that we should not treat multiple cores any differently than multiple machines.

So, we either need languages to handle this (Erlang for one) or possibly frameworks (NServiceBus is my contribution). All I know is that Layered (Tiered) Architectures won’t cut it.

Before going too far ahead, you might want to take a look at my previous post “Space-Based Architectural Thinking, or listen to my podcast Space-Based Architecture for the Web. There’s also a 30 minute webcast online describing SBA more fully here. I’m also going to try to stay away from things concerning Jini this time after already discussing the connection between Jini and SOA, and the tradeoffs between two general approaches: Tasks and Spaces vs Message and Handlers.

OK, so the issue of state-management is a big one. Everybody wants to work stateless, because it scales. The only problem is that the business processes that we are automating are long running, meaning that there are external systems or people involved. This makes these processes inherently stateful. So, we need a way to scale statefully – SBA gives us that. For some background on the “Shared Nothing Architecture”, I suggest reading this post on inter-process SOA and this one as well.

Availability also has to be handled, not only in terms of having enough servers online to handle the required load but in having all the data required to process each request be accessible. This has often been handled by the database using ACID transactions – durability being that which solved availability issues, but also hurting latency the most. The problem with saving the state of our long-running business processes/workflows in the database is the load and the responsiveness requirements. In many verticals – telcos, financial, and defense to name a few, we need millisecond level latency on each stage of the workflow. This is what leads SBA to the in-memory, replicated data grid.

Note that SBA only intends to take these workflows out of the database, and not anything else – especially not Master Data. The lifetime of these workflows is incredibly short compared to that of master data like customers and products. It will have much different backup strategies as well. In terms of load, these workflows will be heavy on reads and writes together in the same transactions, but quite low in terms of just reads. If we have workflows that perform work in parallel, we easily end up with concurrency requirements that make DBAs cringe under the barrage of short transactions.

If you’re worried that Workflow Foundation (WF) won’t scale because of the above, you needn’t be. You can (more or less easily) replace the persistence mechanism of WF with your own, saving your workflow instances to an in-memory replicated data grid.

By enabling the objects in the grid to call back into logic on your servers, you have, in essence, done messaging and more. The added benefit that SBA receives from this is a unification of technology between caching and messaging. This translates directly to savings when it comes time to cluster each of those technology’s environments.

Finally, if we can find an attribute in the incoming stream of messages that creates a nice even distribution, we can then partition our load between our servers by that key. This will work up to the point where the load per key increases beyond a single server’s capacity, and then we have to look at re-partitioning, a non-trivial problem. However, if we put objects in our grid that represent the master data, and tie them to our workflow instances with both of those tied to the key of our load, a smart infrastructure can make sure all that data is already resident on the server that is handling that piece of the load. That decreases latency even more since we no longer have to pay network roundtrips to collect all the data needed before we can process it. That’s a substantial advantage for the above verticals.

But all of this has nothing to do with SOA.

Sure, it’ll change how we implement our Services internally, but it has no impact on their interfaces or the top-level service decomposition. In the Java community, the word “service” is often used to describe the logic of a system. Great significance is placed on keeping these “services” simple, as in Plain-Old Java Objects. The fact of the matter is that the logic of the system should be simple and independent of other concerns like data access and communcations (a la Web Services), but that does not make it a service, not in the SOA sense.

For more information on what Services in SOA are like, check out this podcast on Business and Autonomous Components in SOA. Actually, SBA will probably have the biggest impact on the way autonomous components will handle service-level agreements.

So, it appears that even with SOA, SBA has its place. The former dealing with business level agility, the latter dealing with all the technical aspects of supporting that agility. If you’re tasked with the designing the architecture of a scalable, available, high-throughput, low-latency distributed system, I’d strongly advise you to look at SBA – the technical value is overwhelming. Even if you don’t utilize all elements of SBA and choose the Master Worker Pattern instead of load partitioning, you’ll find the technologies supporting SBA to be quite flexible in that respect.

Will Space-Based Architectures be a part of your future? I don’t know for sure, but they’re a most welcome part of my present.

The ability to introduce a new task at runtime without restarting any servers, and have new clients be able to send those tasks, and existing servers perform them. I never did that before.

This is very important when you’ve already got a system running and you want to expand on it. I think that this covers at least half of all the software work being done on the planet.

The interesting thing is that this ability comes in two parts – one design, the other technology.

In terms of design, in order for existing servers to be able to do the work of the new task without any kind of restart, we need the code for performing the work of the task to reside in the task itself, possibly in some kind of “execute” method. The server would simply take a task out of the “queue” of pending tasks and call that execute method.

Now, those of us trying to do this with Microsoft technology know that if the assembly containing the code for that task was not available on that machine, this wouldn’t work. Technologically speaking, we’d receive some kind of deserialization exception that resulted from a TypeNotFoundException. In other words, in order to support the new task, we’d need to “install” all of its participating assemblies on all our servers.

For those of us who know and use Jini with Java, we get this behavior automatically – the bytecode of the task is downloaded automatically. This is an advantage in terms of operations, but I’m not an operations guy so I can’t say how huge this really is.

The interesting thing for me in this design is that it’s somewhat different from the messaging paradigm I’ve been so successful with. Here we don’t use messages as simple Data Transfer Objects. Tasks contain both data and behavior. This behavior used to belong to message handlers. I like having separate message handlers, it enables me to create very flexible pipelines.

Here’s how I’d trade it off. When using Jini, the tasks style gives me zero footprint deployment. When using .NET, if I’m already installing things on my existing servers, I can deploy new messages and handlers just as well as the task assemblies. The problem is that I’d need some way for the server to pick up on the new handlers – a background thread scanning the deployment directory, or the FileSystemWatcher (given the fact that it sometimes misses things).

Well, it looks like both design styles are feasible on both platforms. The ability to have code download automatically on the Java platform is a plus that is most felt when using tasks.

Bottom line so far, there’s a lot to learn from JavaSpaces, and even if you don’t use Java, Jini, or Space technologies (like this), the design patterns employed there are extremely valuable.

A “JavaSpace” is really just a space, which is, at the end of the day, a queryable distributed in-memory hashtable. Something many of us are already doing for caching. The reason you shouldn’t be doing this yourself is simple. While keeping a single hashtable in memory on a single computer and synchronizing it against changes to your database is simple, doing that in a highly available manner across multiple servers is not. Vendors providing solutions in this space include:

• Gigaspaces
• IBM ObjectGrid
• Alachisoft’s NCache
• GemStone
• Tangosol’s Coherence

But there are others as well. Bottom line: don’t develop one of your own. Do a proof of concept with your short list of vendors and go from there.

The article sums it up nicely like this:

Although JavaSpaces servers are not trivial to set up, they are much easier than any other type of grid computing server. Furthermore, the simplicity of the interface makes the learning curve easier. The greatest advantage of the JavaSpaces approach is the ease with which additional workers can be added to the grid.

It should be clear from the example that there is a lot of extra communication traffic in a JavaSpaces solution so the only reason to use JavaSpaces or any other form of grid computing in support of a Web service is a requirement for computing power or special resources that are not feasible to supply on the server directly.

I have this to add to it. Whereas most traditional systems keep the idea of message-based communication and data caching separate, spaces allow you to kill two birds with one stone. Even if you don’t go the whole Space-Based Architecture route, you’ll find that spaces will fit nicely in your distributed architecture toolkit – I know I did.

Udi,

In a web scenario, how do you correlate multiple events in a SBA that have been created out of a single request? Also, how would you ensure that the response to the user is synchronous and meaningful?

Of course we may use a correlation ID across the events. In this case will the request thread wait until all events for the request are processed?

Get it via the Dr. Dobb’s site here.

Or download directly here.

Additional References

• So, how many machines/CPUs do we need?
• Space-Based Architectural Thinking

Want more? Go to the “Ask Udi” archives.