One question that I get asked about quite a bit with relation to messaging is about search. Isn’t search inherently request/response? Doesn’t it have to return immediately? Wouldn’t messaging in this case hurt our performance?

While I tend to put search in the query camp in the when keeping the responsibility of commands and queries separate, and often recommend that those queries be done without messaging, there are certain types of search where messaging does make sense.

In this post, I’ll describe certain properties of the problem domain that make messaging a good candidate for a solution.

Searching is besides the point – Finding is what it’s all about

Remember that search is only a means to an end in the eyes of the user – they want to find something. One of the difficulties we users have is expressing what we want to find in ways that machines can understand.

In thinking about how we build systems to interact with users, we need to take this fuzziness into account. The more data that we have, the less homogeneous it is, the harder this problem becomes.

When talking about speed, while users are sensitive to the technical interactivity, the thing that matters most is the total time it takes for them to find what they want. If the result of each search screen pops up in 100ms, but the user hasn’t found what they’re looking for after clicking through 20 screens, the search function is ultimately broken.

Notice that the finding process isn’t perceived as “immediate” in the eyes of the user – the evaluation they do in their heads of the search results is as much a part of finding as the search itself.

Also, if the user needs to refine their search terms in order to find what they want, we’re now talking about a multi-request/multi-response process. There is nothing in the problem domain which indicates that finding is inherently request/response.

Relationships in the data

When bringing back data as the result of a search, what we’re saying is that there is a property which is the same across the result elements. But there may be more than one such property. For example, if we search for “blue” on Google Images, we get back pictures of the sky, birds, flowers, and more. Obvious so far – but let’s exploit the obvious a bit.

When the user sees that too many irrelevant results come back, they’ll want to refine their search. One way they can do that is to perform a new search and put in a more specific search phrase – like “blue sky”. Another way is for them to indicate this is by selecting an image and saying “not like this” or “more of these”. Then we can use the additional properties we know about those images to further refine the result group – either adding more images of one kind, or removing images of another.

Here’s something else that’s obvious:

Users often click or change their search before the entire result screen is shown.

It’s beginning to sound like users are already interacting with search in an asynchronous manner. What if we actually designed a system that played to that kind of interaction model?

Data-space partitioning

Once we accept the fact that the user is willing to have more results appear in increments, we can talk about having multiple servers processing the search in parallel. For large data spaces, it is unlikely for us to be able to store all the required meta data for search on one server anyway.

All we really need is a way to index these independent result-sets so that the user can access them. This can be done simply by allocating a GUID/UUID for the search request and storing the result-sets along with that ID.

Browser interaction

When the browser calls a server with the search request the first time, that server allocates an ID to that request, returns a URL containing that ID to the browser, and publishes an event containing the search term and the ID. Each of our processing nodes is subscribed to that event, performs the search on its part of the data-space, and writes its results (likely to a distributed cache) along with that ID.

The browser polls the above URL, which queries the cache (give me everything with this ID), and the browser sees which resources have been added since the last time it polled, and shows them to the user.

If the user clicks “more of these”, that initiates a new search request to the server, which follows the same pattern as before, just that the system is able to pull more relevant information. When implementing “not like this”, this performs a similar search but, instead of adding to the list of items shown, we’re removing items from the list shown based on the response from the server.

In this kind of user-system interaction model, having the user page through the result set doesn’t make very much sense as we’re not capturing the intent of the user, which is “you’re not showing me what I want”. By making it easy for the user to fine tune the result set, we get them closer to finding what they want. By performing work in parallel in a non-blocking manner on smaller sets of data, we greatly decrease the “time to first byte” as well as the time when the user can refine their search.

But Google doesn’t work like that

I know that this isn’t like the search UI we’ve all grown used to.

But then again, the search that you’re providing your users is more specific – not just pages on the web. If you’re a retailer allowing your users to search for a gift, this kind of “more like this, less like that” model is how users would interact with a real sales-person when shopping in a store. Why not model your system after the ways that people behave in the real world?

In closing

If we were to try to make use of messaging underneath “classical” search interaction models, it probably wouldn’t have been the greatest fit. If all we’re doing at a logical level is blocking RPC, then messaging would probably make the system slower. The real power that you get from messaging is being able to technically do things in parallel – that’s how it makes things faster. If you can find ways to see that parallelism in your problem domain, not only will messaging make sense technically – it will really be the only way to build that kind of system.

Learning how to disconnect from seeing the world through the RPC-tinted glasses of our technical past takes time. Focusing on the problem domain, seeing it from the user’s perspective without any technical constraints – that’s the key to finding elegant solutions. More often than not, you’ll see that the real world is non-blocking and parallel, and then you’ll be able to make the best use of messaging and other related patterns.

What are your thought? Post a comment and let me know.

I just finished listening to the Microsoft presentation on how they use the Concurrency & Coordination Runtime (CCR) in MySpace (the stated largest web site running .NET).

Some interesting numbers were stated in the talk.

• Tens of thousands to hundreds of thousands of requests per second
• Over 3 thousand web servers
• Over a thousand mid-tier servers

No wonder most big web sites don’t run .NET. The Windows licenses would put them out of business.

Well, that is if you follow those same architectural practices.

I’ve written in the past of alternative architectural approaches that can scale to those levels at easily an order of magnitude less hardware (I think it’s closer to two OOMs) – here’s one of them on the topic of weather:

Building Super-Scalable Web Systems with REST.

By the way, the client quoted in that post is now well above 60 million users with only small incremental increases in hardware. Oh, and their running everything on Windows and .NET. The question is not “can it scale”, but rather “how much will it cost to scale”.

Architecture pays itself back faster than ever in the Web 2.0 world.

My latest article has been published in issue 21 of the Microsoft Architecture Journal:

EDA: SOA Through The Looking Glass

Please leave questions and comments here.

So, I’ve gotten back from a most enjoyable couple of days in Sweden where I gave two half-day tutorials, the first being the SOA and UI composition talk I gave at the European Virtual ALT.NET meeting (which you can find online here) and the other on DDD in enterprise apps (the first time I’ve done this talk).

I’ve gotten some questions about my DDD presentation there based on Aaron Jensen’s pictures:

Yes – I talk with my hands. All the time.

That slide is quite an important one – I talked about it for at least 2 hours.

Here it is again, this time in full:

You may notice that the nice clean layered abstraction that the industry has gotten so comfortable with doesn’t quite sit right when looking at it from this perspective. The reason for that is that this perspective takes into account physical distribution while layers don’t.

I’ll have some more posts on this topic as well as giving a session in TechEd Europe this November.

Oh – and please do feel free to already send your questions in.

After reading Ayende’s post advocating against “soft deletes” I felt that I should add a bit more to the topic as there were some important business semantics missing. As developers discuss the pertinence of using an IsDeleted column in the database to mark deletion, and the way this relates to reporting and auditing concerns is weighed, the core domain concepts rarely get a mention. Let’s first understand the business scenarios we’re modeling, the why behind them, before delving into the how of implementation.

The real world doesn’t cascade

Let’s say our marketing department decides to delete an item from the catalog. Should all previous orders containing that item just disappear? And cascading farther, should all invoices for those orders be deleted as well? Going on, would we have to redo the company’s profit and loss statements?

Heaven forbid.

So, is Ayende wrong? Do we really need soft deletes after all?

On the one hand, we don’t want to leave our database in an inconsistent state with invoices pointing to non-existent orders, but on the other hand, our users did ask us to delete an entity.

Or did they?

When all you have is a hammer…

We’ve been exposing users to entity-based interfaces with “create, read, update, delete” semantics in them for so long that they have started presenting us requirements using that same language, even though it’s an extremely poor fit.

Instead of accepting “delete” as a normal user action, let’s go into why users “delete” stuff, and what they actually intend to do.

The guys in marketing can’t actually make all physical instances of a product disappear – nor would they want to. In talking with these users, we might discover that their intent is quite different:

“What I mean by ‘delete’ is that the product should be discontinued. We don’t want to sell this line of product anymore. We want to get rid of the inventory we have, but not order any more from our supplier. The product shouldn’t appear any more when customers do a product search or category listing, but the guys in the warehouse will still need to manage these items in the interim. It’s much shorter to just say ‘delete’ though.”

There seem to be quite a few interesting business rules and processes there, but nothing that looks like it could be solved by a single database column.

Model the task, not the data

Looking back at the story our friend from marketing told us, his intent is to discontinue the product – not to delete it in any technical sense of the word. As such, we probably should provide a more explicit representation of this task in the user interface than just selecting a row in some grid and clicking the ‘delete’ button (and “Are you sure?” isn’t it).

As we broaden our perspective to more parts of the system, we see this same pattern repeating:

Orders aren’t deleted – they’re cancelled. There may also be fees incurred if the order is canceled too late.

Employees aren’t deleted – they’re fired (or possibly retired). A compensation package often needs to be handled.

Jobs aren’t deleted – they’re filled (or their requisition is revoked).

In all cases, the thing we should focus on is the task the user wishes to perform, rather than on the technical action to be performed on one entity or another. In almost all cases, more than one entity needs to be considered.

Statuses

In all the examples above, what we see is a replacement of the technical action ‘delete’ with a relevant business action. At the entity level, instead of having a (hidden) technical WasDeleted status, we see an explicit business status that users need to be aware of.

The manager of the warehouse needs to know that a product is discontinued so that they don’t order any more stock from the supplier. In today’s world of retail with Vendor Managed Inventory, this often happens together with a modification to an agreement with the vendor, or possibly a cancellation of that agreement.

This isn’t just a case of transactional or reporting boundaries – users in different contexts need to see different things at different times as the status changes to reflect the entity’s place in the business lifecycle. Customers shouldn’t see discontinued products at all. Warehouse workers should, that is, until the corresponding Stock Keeping Unit (SKU) has been revoked (another status) after we’ve sold all the inventory we wanted (and maybe returned the rest back to the supplier).

Rules and Validation

When looking at the world through over-simplified-delete-glasses, we may consider the logic dictating when we can delete to be quite simple: do some role-based-security checks, check that the entity exists, delete. Piece of cake.

The real world is a bigger, more complicated cake.

Let’s consider deleting an order, or rather, canceling it. On top of the regular security checks, we’ve got some rules to consider:

If the order has already been delivered, check if the customer isn’t happy with what they got, and go about returning the order.

If the order contained products “made to order”, charge the customer for a portion (or all) of the order (based on other rules).

And more…

Deciding what the next status should be may very well depend on the current business status of the entity. Deciding if that change of state is allowed is context and time specific – at one point in time the task may have been allowed, but later not. The logic here is not necessarily entirely related to the entity being “deleted” – there may be other entities which need to be checked, and whose status may also need to be changed as well.

Summary

I know that some of you are thinking, “my system isn’t that complex – we can just delete and be done with it”.

My question to you would be, have you asked your users why they’re deleting things? Have you asked them about additional statuses and rules dictating how entities move as groups between them? You don’t want the success of your project to be undermined by that kind of unfounded assumption, do you?

The reason we’re given budgets to build business applications is because of the richness in business rules and statuses that ultimately provide value to users and a competitive advantage to the business. If that value wasn’t there, wouldn’t we be serving our users better by just giving them Microsoft Access?

In closing, given that you’re not giving your users MS Access, don’t think about deleting entities. Look for the reason why. Understand the different statuses that entities move between. Ask which users need to care about which status. I know it doesn’t show up as nicely on your resume as “3 years WXF”, but “saved the company $4 million in wasted inventory” does speak volumes.

One last sentence: Don’t delete. Just don’t.

Yesterday me and Scott virtually sat down to have a chat about NServiceBus and service buses in general. While we didn’t get in to many of the more advanced parts, you may find it an interesting introduction to the topic as well as saving yourself the costly mistake of implementing a broker instead of a bus (yes – they’re actually two different things).

Take a listen.

Convention over configuration describes a style of development made popular by Ruby on Rails which has gained a great deal of traction in the .net ecosystem. After using frameworks designed in this way, I can say that the popularity is justified – it is much more pleasurable developing this way.

The thing is, when looking at this in light of the full software development lifecycle, there are signs that the waters run deeper than we might have originally thought.

Let’s take things one step at a time though…

What is it?

Wikipedia tells us:

“Convention over Configuration (aka Coding by convention) is a software design paradigm which seeks to decrease the number of decisions that developers need to make, gaining simplicity, but not necessarily losing flexibility. The phrase essentially means a developer only needs to specify unconventional aspects of the application.”

What this means is that frameworks built in this way have default implementations that can be swapped out if needed. So far so good.

For example…

In NServiceBus, there is an abstraction for how subscription data is stored and multiple implementations – one in-memory, another using a durable MSMQ queue, and a third which uses a database. The convention for that part of the system is that the MSMQ implementation will be used, unless something else is specified.

Developers wishing to specify a different implementation can specify the desired implementation in the container – either one that comes out of the box, or their own implementation of ISubscriptionStorage.

Things get more interesting when we consider the full lifecycle.

Lifecycle effects

When developers are in the early phases of writing a new service, they want to focus primarily on what the service does – its logic. They don’t want to muck around with MSMQ queues for storing subscriptions and would much rather use the in-memory storage.

As the service takes shape and the developers want to run the full service on their machine, possibly testing basic fault-tolerance behaviors – kill one service, see that the others get a timeout, bring the service back up, wanting it to maintain all the previous subscriptions.

Moving on from there, our developers want to take the same system they just tested on their machine and move it into a staging environment. There, they don’t want to use the MSMQ implementation for subscription storage, but rather the database implementation – as will be used in the production environment.

While it may not sound like a big deal – changing the code which specifies which implementation to use when moving from one environment to another, consider that on top of just subscription storage, there is logging (output to console, file, db?), saga persistence (in-memory, file-based DB, relational DB), and more.

It’s actually quite likely that something will get missed as we move the system between environments. Can there be a better way?

What if…

What if there was some way for the developer to express their intent to the system, and the system could change its conventions, without the developer having to change any code or configuration files?

You might compare this (in concept) to debug builds and release builds. Same code, same config, but the runtime behaves different between the two.

As I mulled over how we could capture that intent without any code or config changes, the solution that I kept coming to seemed too trivial at first, so I dismissed it. Yet, it was the simplest one that would work for console and WinForms applications, as well as windows services – command line arguments. The only thing is that I don’t think those are available for web applications.

But since we’re still in “what if” land, and I’m more thinking out loud here than providing workable solutions for tomorrow morning, let’s “what if” command line arguments worked for web apps too.

Command-Line Intent

Going back to our original scenario, when developers are working on the logic of the service, they run it using the generic NServiceBus host process, passing it the command line parameter /lite (or whatever). The host then automatically configures all the in-memory implementations.

As the system progresses, when the developer wants to run everything on their machine, they run the processes with /integration. The host then configures the appropriate implementations (MSMQ for subscription storage, SQLite for saga persistence, etc.

When the developers want to run the system in production, they could specify /production (or maybe that could be the default?), and the database backed implementations would be configured.

Imagine…

Imagine being able to move that fluidly from one environment to another. Not needing to pore over configuration files or startup script code which configures a zillion implementation details. Not needing to worry that as you moved the system to staging something would break.

Imagine short, frictionless iterations even for large scale systems.

Imagine – lifecycle-aware frameworks making all this imagination a reality.

In Closing

We’re not there yet – but we’re not that far either. The generic host we’re providing with NServiceBus 2.0 is now being extended to support exactly these scenarios.

It’s my hope that as more of us think about this challenge, we’ll come up with better solutions and more intelligent frameworks. Just as convention came to our rescue before, breaking us out of the pain of endless XML configuration, I hope this new family of lifecycle-aware frameworks will make the friction of moving a system through dev, test, staging, and production a thing of the past.

A worthy problem for us all to solve, don’t you think?

Any ideas on how to make it a reality?
Send them in – leave a comment below.

I’ve been getting more questions from readers as to when I’ll be giving a public course in their area, so I thought I’d put up a quick little post on the topic.

Advanced Distributed Systems Design with SOA & DDD

• 
  Sep 7th, London UK
  
• 
  Sep 21, Brussels Belgium
  
• 
  Nov 30, Austin Texas – USA

Loosely-Coupled Messaging with NServiceBus

• 
  Sep 16th, London UK
  

***

There’s also work afoot to bring the SOA course to Australia in October – if you’re interested, let the organizers know here.

Those interested in a private course for their company can be contact me here.

And a quick, final note in closing – those of you interested in NServiceBus should now be able to find more information about it as well as how it relates to current and future Microsoft technologies on the improved NServiceBus.com site.

My article on “employing the domain model pattern” has been published in the August edition of MSDN Magazine.

Here’s a short excerpt:

“In this article, we’ll go through the reasons to (and not to) employ the domain model pattern, the benefits it brings, as well as provide some practical tips on keeping the overall solution as simple as possible.”

Continue reading…

My previous post on Domain Events left some questions about how aggregate roots should be created unanswered. It would actually be more accurate to say how aggregate roots should *not* be created. It turns out that this is one of the less intuitive parts of domain-driven design and has been the source of many arguments on the matter. Let’s start with the wrong way:

   1:  using (ISession s = sf.OpenSession())

   2:  using (ITransaction tx = s.BeginTransaction())

   3:  {

   4:      Customer c = new Customer();

   5:      c.Name = "udi dahan";

   6:   

   7:      s.Save(c);

   8:      tx.Commit();

   9:  }

I understand that the code above is representative of how much code is written when using an object-relational mapper. Many would consider this code to follow DDD principles – that Customer is an aggregate root. Unfortunately – that is not the case. The code above is missing the real aggregate root.

There’s also the inevitable question of validation – if the customer object isn’t willing to accept a name with a space in it, should we throw an exception? That would prevent an invalid entity from being saved, which is good. On the other hand, exceptions should be reserved for truly exceptional occurrences. But if we don’t use exceptions, using Domain Events instead, how do we prevent the invalid entity from being saved?

All of these issues are handled auto-magically once we have a true aggregate root.

Always Get An Entity

Let’s start with the technical guidance – always get an entity. At least one. Also, don’t add any objects to the session or unit of work explicitly – rather, have some other already persistent domain entity create the new entity and add it to a collection property.

Looking at the code above, we see that we’re not following the technical guidance.

But the question is, which entity could we possibly get from the database in this case? All we’re doing is adding a customer.

And that’s exactly where the technical guidance leads us to the business analysis that was missing in this scenario…

Business Analysis

Customers don’t just appear out of thin air.

Blindingly obvious – isn’t it.

So why would we technically model our system as if they did? My guess is that we never really thought about it – it wasn’t our job. So here’s the breaking news – if we want to successfully apply DDD we do need to think about it, it is our job.

Going back to the critical business question:

Where do customers come from?

In the real world, they stroll into the store. In our overused e-commerce example, they navigate to our website. New customers that haven’t used our site before don’t have any cookies or anything we can identify them with. They navigate around, browsing, maybe buying something in the end, maybe not.

Yet, the browsing process is interesting in its own right:

• Which products did they look at?
• Did they use the search feature?
• How long did they spend on each page?
• Did they scroll down to see the reviews?

If and when they do finally buy something, all that history is important and we’d like to maintain a connection to it.

Actually, even before they buy something, what they put in their cart is the interesting piece. The transition from cart to checkout is another interesting piece. Do they actually complete the checkout process, or do they abandon it midway through?

Add to that when we ask/force them to create a user/login in our system.

Are they actually a customer if they haven’t bought anything?

We’re beginning to get an inkling that almost every activity that results in the creation of an entity or storing of additional information can be traced to a transition from a previous business state.

In any transition, the previous state is the aggregate root.

In the beginning…

Let’s start at the very beginning then – someone came to our site. Either they navigated here from some other web page, they clicked on an email link someone sent them, or they typed in our URL. This can be designed as follows:

   1:  using (ISession s = sf.OpenSession())

   2:  using (ITransaction tx = s.BeginTransaction())

   3:  {

   4:     var referrer = s.Get<Referrer>(msg.URL);

   5:     referrer.BroughtVisitorWithIp(msg.IpAddress);

   6:   

   7:     tx.Commit();

   8:  }

   9:   

And our referrer code could look something like this:

   1:  public void BroughtVisitorWithIp(string ipAddress)

   2:  {

   3:     var visitor = new Visitor(ipAddress);

   4:     this.NewVisitors.Add(visitor);

   5:  }

   6:   

This follows the technical guidance we saw at the beginning.

It also allows us to track which referrer is bringing us which visitors, through tracking those visitors as they become shoppers (by putting stuff in their cart), finally seeing which become customers.

We can solve the situation of not having a referrer by implementing the null object pattern which is well supported by all the standard object-relational mappers these days.

How it works internally

When we call a method on a persistent entity retrieved by the object-relational mapper, and the entity modifies its state like when it adds a new entity to one of its collection properties, when the transaction commits, here’s what happens:

The mapper sees that the persistent entity is dirty, specifically, that its collection property was modified, and notices that there is an object in there that isn’t persistent. At that point, the mapper knows to persist the new entity without us ever having to explicitly tell it to do so. This is sometimes known as “persistence by reachability”.

Where validation happens

Let’s consider the relatively trivial rule that says that a user name can’t contain a space.

Also, keep in mind that a registered user is the result of a transition from a visitor.

Here’s *one* way of doing that:

   1:  public class Visitor

   2:  {

   3:     public void Register(string username, string password)

   4:     {

   5:        if (username.Contains(" "))

   6:        {

   7:           DomainEvents.Raise<UsernameCantContainSpace>();

   8:           return;

   9:        }

  10:   

  11:        var user = new User(username, password);

  12:        this.RegisteredUser = u;

  13:     }

  14:  }

  15:   

This actually isn’t representative of most of the rules that will be found in the domain model, but it illustrates a way of preventing an entity from being created without our service layer needing to know anything. All the service layer does is get the visitor object and call the Register method.

Validation of string lengths, data ranges, etc is not domain logic and is best handled elsewhere (and a topic for a different post). The same goes for uniqueness.

Summary

The most important thing to keep in mind is that if your service layer is newing up some entity and saving it – that entity isn’t an aggregate root *in that use case*. As we saw above, in the original creation of the Visitor entity by the Referrer, the visitor class wasn’t the aggregate root. Yet, in the user registration use case, the Visitor entity was the aggregate root.

Aggregate roots aren’t a structural property of the domain model.

And in any case, don’t go saving entities in your service layer – let the domain model manage its own state. The domain model doesn’t need any references to repositories, services, units of work, or anything else to manage its state.

If you do all this, you’ll also be able to harness the technique of fetching strategies to get the best performance out of your domain model by representing your use cases as interfaces on the domain model like IRegisterUsers (implemented by Visitor) and IBringVisitors (implemented by Referrer).

And spending some time on business analysis doesn’t hurt either – unless customers really do fall out of the sky in your world