“So, which services do I need?”

This innocuous question comes up a lot. Usually I get this question after a short problem domain description. One of these came up on the nServiceBus discussion groups. Ayende took it and ran with it turning it into a nice blog post, An exercise in designing SOA systems. I’ve been meaning to write something myself. Bill put up a response already in his Service Granularity Example. So, I’m late to the party, again, but here we go.

It’s almost impossible to know, right away, which services are appropriate.

So, I’m going to focus more on the process of getting there, rather than describing the solution itself.

The domain deals with a placement agency placing physicians in positions at hospitals.

1. So, what does it actually do?

In Ayende’s post, he describes several services, but I’d rather look at them as use cases: registering an open position, registering a candidate, verifying their credentials, etc. It’s worth going through this requirements process. It doesn’t necessarily translate immediately to services, but there’s value in it.

2. What does it do it to?

We should also be looking at the data model, an entity relationship diagram (ERD) , where we see that we may have placed a certain physician at a number of positions. It’s also important for us to know about under which circumstances a physician finished their employment at a previous position before, say, trying to place them at a position in the same hospital or chain of hospitals. Don’t go thinking that this what the database schema will look like, it’s all about understanding connections between various bits of data.

3. When does that happen?

The next step is to map the uses cases above to the entities in the ERD, which entity is used in which use case. It’s also important to differentiate between entities (or even more importantly, specific fields of entities) that are used in a read-only fashion within a given use case. For instance, when registering a new position, we’ll want to check that against other open positions in the same hospital so we don’t end up registering the same position twice. Also, we might want to suggest verified physicians whose credentials match the position’s requirements. Data we wouldn’t be interested in might be which other physicians we placed at that hospital.

4. What just happened?

Another valuable perspective on the problem domain is the business process view – what are the interesting business events in the system and how they unfold over time. For instance, physician registered, position opened, physician’s credentials verified, and physician placed in position (or position filled by physician) are events that describe a different business perspective than use cases.

5. How do I decide?

Once we know what events there are, we can start looking at what kind of decisions we might want to make when those events occur and what data we’d need to make those decisions. These decisions may be as simple as updating a database or sending an email to a user. They also may include more advanced logic like when the profitability of an agreement with a specific hospital chain changes, prefer placing physicians in positions in that chain over others.

6. How do I deal with all this information?

After we have all of this information, we can start looking for cohesive bunching across all of these axes using these rules:

• Data that is modified by a use case gets published as an event.
• Data that is required by a use case for read-only purposes, arrives as the result of subscribing to some event.

Look for rules that differentiate behaviour based on the properties of data. Look for a correlation to some business concept. For instance, physicians probably won’t be changing their specialization, and open positions often deal with a certain specialization. Therefore, specific data instances tied to two different specializations can be said to be loosely coupled.

7. Which property slices across the domain?

Even though the ERD may not have made it clear, and the use cases didn’t show any particular break-down, nor did the events call out this point, the key to finding the way a business domain decomposes into services lies in decoupling specific data instances.

Actually, at this point we can clump autonomous components (mere technical bits) that handle a single message, into more granular business components.

If you think about it, it makes a lot of sense. The kind of credential checking you’d do for physicians specializing in brain surgery would likely be different than for general practitioners. The kind of information you’d store would, therefore, also be different.

But, which services do I need?

Quite frankly, I don’t have enough information to know.

But if we had continued this conversation, going through issues like transactional consistency, availability requirements, and other non-functional issues we could have  gotten there.

If there’s one thing that I hope you got out of this, it’s that the questions are what’s important. The iterative process of looking at the problem domain from various perspectives, incorporating the new-found knowledge, and asking more questions is what leads us to a solution. But we don’t stop there. We keep looking for characteristics which split services apart into business components, and for consistency requirements that brings autonomous components together into services.

It’s not easy, but by focusing on these simple questions, you can get to a coherent service oriented architecture.

Update: The new and improved solution is now available: Domain Events, Take 2.

Most people getting started with DDD and the Domain Model pattern get stuck on this. For a while I tried answering this on the discussion groups, but here we have a nice example that I can point to next time.

The underlying problem I’ve noticed over the past few years is that developers are still thinking in terms of querying when they need more data. When moving to the Domain Model pattern, you have to “simply” represent the domain concepts in code – in other words, see things you aren’t used to seeing. I’ll highlight that part in the question below so that you can see where I’m going to go with this in my answer:

I have an instance where I believe I need access to a service or repository from my entity to evaluate a business rule but I’m using NHibernate for persistence so I don’t have a real good way to inject services into my entity. Can I get some viewpoints on just passing the services to my entity vs. using a facade?

Let me explain my problem to provide more context to the problem.

The core domain revolves around renting video games. I am working on a new feature to allow customers to trade in old video games. Customers can trade in multiple games at a time so we have a TradeInCart entity that works similar to most shopping carts that everybody is familiar with. However there are several rules that limit the items that can be placed into the TradeInCart. The core rules are:

1. Only 3 games of the same title can be added to the cart.
2. The total number of items in the cart cannot exceed 10.
3. No games can be added to the cart that the customer had previously reported lost with regards to their rental membership.
    a. If an attempt is made to add a previously reported lost game, then we need to log a BadQueueStatusAddAttempt to the persistence store.

So the first 2 rules are easily handled internally by the cart through an Add operation. Sample cart interface is below.

   1:  class TradeInCart{

   2:      Account Account{get;}

   3:      LineItem Add(Game game);

   4:      ValidationResult CanAdd(Game game);

   5:      IList<LineItems> LineItems{get;}

   6:  }

However the #3 rule is much more complicated and can’t be handled internally by the cart, so I have to depend on external services. Splitting up the validation logic for a cart add operation doesn’t seem very appealing to me at all. So I have the option of passing in a repository to get the previously reported lost games and a service to log bad attempts. This makes my cart interface ugly real quick.

   1:  class TradeInCart{

   2:      Account Account{get;}

   3:      LineItem Add(

   4:          Game game, 

   5:          IRepository<QueueHistory> repository, 

   6:          LoggingService service);

   7:   

   8:      ValidationResult CanAdd(

   9:          Game game, 

  10:          IRepository<QueueHistory> repository, 

  11:          LoggingService service);

  12:   

  13:      IList<LineItems> LineItems{get;}

  14:  }

The alternative option is to have a TradeInCartFacade that handles the validations and adding the items to the cart. The façade can have the repository and services injected though DI which is nice, but the big negative is that the cart ends up totally anemic.

Any thought on this would be greatly appreciated.

Thanks,
Jesse

As I highlighted above, the thing that will help you with your business rules is to introduce the Customer object (that you probably already have) with the property GamesReportedLost (an IList<Game>). Your TradeInCart would have a reference to the Customer object and could then check the rule in the Add method.

Before I go into the code, it looks like your Account object might be used the same way, but your description of the domain doesn’t mention accounts, so I’m going to assume that that’s unrelated for now:

   1:  public class Customer{

   2:   

   3:      /* other properties and methods */

   4:   

   5:      private IList<Game> gamesReportedLost;

   6:      public virtual IList<Game> GamesReportedLost 

   7:      { 

   8:          get

   9:          {

  10:              return gamesReportedLost;

  11:          }

  12:          set

  13:          {

  14:              gamesReportedLost = value;

  15:          }

  16:      }

  17:  }

Keep in mind that the GamesReportedLost is a persistent property of Customer. Every time a customer reports a game lost, this list needs to be kept up to date. Here’s the TradeInCart now:

   1:  public class TradeInCart

   2:  {

   3:      /* other properties and methods */

   4:   

   5:      private Customer customer;

   6:      public virtual Customer Customer

   7:      { 

   8:          get { return customer; }

   9:          set { customer = value; }

  10:      }

  11:   

  12:      private IList<LineItem> lineItems;

  13:      public virtual IList<LineItem> LineItems

  14:      {

  15:          get { return lineItems; }

  16:          set { lineItems = value; }

  17:      }

  18:   

  19:      public void Add(Game game)

  20:      {

  21:          if (lineItems.Count >= CONSTANTS.MaxItemsPerCart)

  22:          {

  23:              FailureEvents.RaiseCartIsFullEvent();

  24:              return;

  25:          }

  26:   

  27:          if (NumberOfGameAlreadyInCart(game) >=

  28:              CONSTANTS.MaxNumberOfSameGamePerCart)

  29:          {

  30:              FailureEvents

  31:                .RaiseMaxNumberOfSameGamePerCartReachedEvent();

  32:              return;

  33:          }

  34:   

  35:          if (customer.GamesReportedLost.Contains(game))

  36:              FailureEvents.RaiseGameReportedLostEvent();

  37:          else

  38:              this.lineItems.Add(new LineItem(game));

  39:      }

  40:   

  41:      private int NumberOfGameAlreadyInCart(Game game)

  42:      {

  43:          int result = 0;

  44:   

  45:          foreach(LineItem li in this.lineItems)

  46:              if (li.Game == game)

  47:                  result++;

  48:   

  49:          return result;

  50:      }

  51:  }

  52:   

  53:  public static class FailureEvents

  54:  {

  55:      public static event EventHandler GameReportedLost;

  56:      public static void RaiseGameReportedLostEvent()

  57:      {

  58:           if (GameReportedLost != null)

  59:               GameReportedLost(null, null);

  60:      }

  61:   

  62:      public static event EventHandler CartIsFull;

  63:      public static void RaiseCartIsFullEvent()

  64:      {

  65:           if (CartIsFull != null)

  66:               CartIsFull(null, null);

  67:      }

  68:   

  69:      public static event EventHandler MaxNumberOfSameGamePerCartReached;

  70:      public static void RaiseMaxNumberOfSameGamePerCartReachedEvent()

  71:      {

  72:           if (MaxNumberOfSameGamePerCartReached != null)

  73:               MaxNumberOfSameGamePerCartReached(null, null);

  74:      }

  75:  }

Your service layer class that calls the Add method of TradeInCart would first subscribe to the relevant events in FailureEvents. If one of those events is raised, it would do the necessary logging, external system calls, etc.

As you can see, the API of TradeInCart doesn’t need to make use of any external repositories, nor do you need to inject any other external dependencies in.

One thing I didn’t do in the above code to keep it “short” is to define the relevant custom EventArgs for bubbling up the information as to which game was reported lost or already have 3 of those in the cart. That is something that definitely should be done so that the service layer can pass this information back to the client.

Here’s a look at Service Layer code:

   1:  public class AddGameToCartMessageHandler :

   2:      BaseMessageHandler<AddGameToCartMessage>

   3:  {

   4:      public override void Handle(AddGameToCartMessage m)

   5:      {

   6:          using (ISession session = SessionFactory.OpenSession())

   7:          using (ITransaction tx = session.BeginTransaction())

   8:          {

   9:              TradeInCart cart = session.Get<TradeInCart>(m.CartId);

  10:              Game g = session.Get<Game>(m.GameId);

  11:   

  12:              Domain.FailureEvents.GameReportedLost +=

  13:                gameReportedLost;

  14:              Domain.FailureEvents.CartIsFull +=

  15:                cartIsFull;

  16:              Domain.FailureEvents.MaxNumberOfSameGamePerCartReached +=

  17:                maxNumberOfSameGamePerCartReached;

  18:   

  19:              cart.Add(g);

  20:   

  21:              Domain.FailureEvents.GameReportedLost -=

  22:                gameReportedLost;

  23:              Domain.FailureEvents.CartIsFull -=

  24:                cartIsFull;

  25:              Domain.FailureEvents.MaxNumberOfSameGamePerCartReached -=

  26:                maxNumberOfSameGamePerCartReached;

  27:   

  28:              tx.Commit();

  29:          }

  30:      }

  31:   

  32:      private EventHandler gameReportedLost = delegate { 

  33:            Bus.Return((int)ErrorCodes.GameReportedLost);

  34:          };

  35:   

  36:      private EventHandler cartIsFull = delegate { 

  37:            Bus.Return((int)ErrorCodes.CartIsFull);

  38:          };

  39:   

  40:      private EventHandler maxNumberOfSameGamePerCartReached = delegate { 

  41:            Bus.Return((int)ErrorCodes.MaxNumberOfSameGamePerCartReached);

  42:          };

  43:      }

  44:  }

It’s important to remember to clean up your event subscriptions so that your Service Layer objects get garbage collected. This is one of the primary causes of memory leaks when using static events in your Domain Model. I’m hoping to find ways to use lambdas to decrease this repetitive coding pattern. You might be thinking to yourself that non-static events on your Domain Model objects would be easier, since those objects would get collected, freeing up the service layer objects for collection as well. There’s just on small problem:

The problem is that if an event is raised by a child (or grandchild object), the service layer object may not even know that that grandchild was involved and, as such, would not have subscribed to that event. The only way the service layer could work was by knowing how the Domain Model worked internally – in essence, breaking encapsulation.

If you’re thinking that using exceptions would be better, you’d be right in thinking that that won’t break encapsulation, and that you wouldn’t need all that subscribe/unsubscribe code in the service layer. The only problem is that the Domain Model needs to know that the service layer had a default catch clause so that it wouldn’t blow up. Otherwise, the service layer (or WCF, or nServiceBus) may end up flagging that message as a poison message (Read more about poison messages). You’d also have to be extremely careful about in which environments you used your Domain Model – in other words, your reuse is shot.

Conclusion

I never said it would be easy

However, the solution is simple (not complex). The same patterns occur over and over. The design is consistent. By focusing on the dependencies we now have a domain model that is reusable across many environments (server, client, sql clr, silverlight). The domain model is also testable without resorting to any fancy mock objects.

One closing comment – while I do my best to write code that is consistent with production quality environments, this code is more about demonstrating design principles. As such, I focus more on the self-documenting aspects of the code and have elided many production concerns.

Do you have a better solution?

Something that I haven’t considered?

Do me a favour – leave me a comment. Tell me what you think.

Sagas have always been designed with unit testing in mind. By keeping them disconnected from any communications or persistence technology, it was my belief that it should be fairly easy to use mock objects to test them. I’ve heard back from projects using nServiceBus this way that they were pleased with their ability to test them, and thought all was well.

Not so.

The other day I sat down to implement and test a non-trivial business process, and the testing was far from easy. Now as developers go, I’m not great, or an expert on unit testing or TDD, but I’m above average. It should not have been this hard. And I tried doing it with Rhino.Mocks, TypeMock, and finally Moq. It seemed like I was in a no-mans-land, between trying to do state-based testing, and setting expectations on the messages being sent (as well as correct values in those messages), nothing flowed.

Until I finally stopped trying to figure out how to test, and focused on what needed to be tested. I mean, it’s not like I was trying to build a generic mocking framework like Daniel.

Here’s an example business process, or actually, part of one, and then we’ll see how that can be tested. By the way, there will be a post coming soon which describes how we go about analysing a system, coming up with these message types, and how these sagas come into being, so stay tuned. Either that, or just come to my tutorial at QCon.

On with the process:

1. When we receive a CreateOrderMessage, whose “Completed” flag is true, we’ll send 2 AuthorizationRequestMessages to internal systems (for managers to authorize the order), one OrderStatusUpdatedMessage to the caller with a status “Received”, and a TimeoutMessage to the TimeoutManager requesting to be notified – so that the process doesn’t get stuck if one or both messages don’t get a response.

2. When we receive the first AuthorizationResponseMessage, we notify the initiator of the Order by sending them a OrderStatusUpdatedMessage with a status “Authorized1”.

3. When we get “timed out” from the TimeoutManager, we check if at least one AuthorizationResponseMessage has arrived, and if so, publish an OrderAcceptedMessage, and notify the initator (again via the OrderStatusUpdatedMessage) this time with a status of “Accepted”.

And here’s the test:

    public class OrderSagaTests
    {
        private OrderSaga orderSaga = null;
        private string timeoutAddress;
        private Saga Saga;     

        [SetUp]
        public void Setup()
        {
            timeoutAddress = "timeout";
            Saga = Saga.Test(out orderSaga, timeoutAddress);
        }     

        [Test]
        public void OrderProcessingShouldCompleteAfterOneAuthorizationAndOneTimeout()
        {
            Guid externalOrderId = Guid.NewGuid();
            Guid customerId = Guid.NewGuid();
            string clientAddress = "client";     

            CreateOrderMessage createOrderMsg = new CreateOrderMessage();
            createOrderMsg.OrderId = externalOrderId;
            createOrderMsg.CustomerId = customerId;
            createOrderMsg.Products = new List<Guid>(new Guid[] { Guid.NewGuid() });
            createOrderMsg.Amounts = new List<float>(new float[] { 10.0F });
            createOrderMsg.Completed = true;     

            TimeoutMessage timeoutMessage = null;     

            Saga.WhenReceivesMessageFrom(clientAddress)
                .ExpectSend<AuthorizeOrderRequestMessage>(
                    delegate(AuthorizeOrderRequestMessage m)
                    {
                        return m.SagaId == orderSaga.Id;
                    })
                .ExpectSend<AuthorizeOrderRequestMessage>(
                    delegate(AuthorizeOrderRequestMessage m)
                    {
                        return m.SagaId == orderSaga.Id;
                    })
                .ExpectSendToDestination<OrderStatusUpdatedMessage>(
                    delegate(string destination, OrderStatusUpdatedMessage m)
                    {
                        return m.OrderId == externalOrderId && destination == clientAddress;
                    })
                .ExpectSendToDestination<TimeoutMessage>(
                    delegate(string destination, TimeoutMessage m)
                    {
                        timeoutMessage = m;
                        return m.SagaId == orderSaga.Id && destination == timeoutAddress;
                    })
                .When(delegate { orderSaga.Handle(createOrderMsg); });     

            Assert.IsFalse(orderSaga.Completed);     

            AuthorizeOrderResponseMessage response = new AuthorizeOrderResponseMessage();
            response.ManagerId = Guid.NewGuid();
            response.Authorized = true;
            response.SagaId = orderSaga.Id;     

            Saga.ExpectSendToDestination<OrderStatusUpdatedMessage>(
                    delegate(string destination, OrderStatusUpdatedMessage m)
                    {
                        return (destination == clientAddress &&
                                m.OrderId == externalOrderId &&
                                m.Status == OrderStatus.Authorized1);
                    })
                .When(delegate { orderSaga.Handle(response); });     

            Assert.IsFalse(orderSaga.Completed);     

            Saga.ExpectSendToDestination<OrderStatusUpdatedMessage>(
                    delegate(string destination, OrderStatusUpdatedMessage m)
                    {
                        return (destination == clientAddress &&
                                m.OrderId == externalOrderId &&
                                m.Status == OrderStatus.Accepted);
                    })
                .ExpectPublish<OrderAcceptedMessage>(
                    delegate(OrderAcceptedMessage m)
                    {
                        return (m.CustomerId == customerId);
                    })
                .When(delegate { orderSaga.Timeout(timeoutMessage.State); });     

            Assert.IsTrue(orderSaga.Completed);
        }
    }

You might notice that this style is a bit similar to the fluent testing found in Rhino Mocks. That’s not coincidence. It actually makes use of Rhino Mocks internally. The thing that I discovered was that in order to test these sagas, you don’t need to actually see a mocking framework. All you should have to do is express how messages get sent, and under what criteria those messages are valid.

If you’re wondering what the OrderSaga looks like, you can find the code right here. It’s not a complete business process implementation, but its enough to understand how one would look like:

using System;
using System.Collections.Generic;
using ExternalOrderMessages;
using NServiceBus.Saga;
using NServiceBus;
using InternalOrderMessages;     

namespace ProcessingLogic
{
    [Serializable]
    public class OrderSaga : ISaga<CreateOrderMessage>,
        ISaga<AuthorizeOrderResponseMessage>,
        ISaga<CancelOrderMessage>
    {
        #region config info     

        [NonSerialized]
        private IBus bus;
        public IBus Bus
        {
            set { this.bus = value; }
        }     

        [NonSerialized]
        private Reminder reminder;
        public Reminder Reminder
        {
            set { this.reminder = value; }
        }     

        #endregion     

        private Guid id;
        private bool completed;
        public string clientAddress;
        public Guid externalOrderId;
        public int numberOfPendingAuthorizations = 2;
        public List<CreateOrderMessage> orderItems = new List<CreateOrderMessage>();     

        public void Handle(CreateOrderMessage message)
        {
            this.clientAddress = this.bus.SourceOfMessageBeingHandled;
            this.externalOrderId = message.OrderId;     

            this.orderItems.Add(message);     

            if (message.Completed)
            {
                for (int i = 0; i < this.numberOfPendingAuthorizations; i++)
                {
                    AuthorizeOrderRequestMessage req = new AuthorizeOrderRequestMessage();
                    req.SagaId = this.id;
                    req.OrderData = orderItems;     

                    this.bus.Send(req);
                }
            }     

            this.SendUpdate(OrderStatus.Recieved);     

            this.reminder.ExpireIn(message.ProvideBy - DateTime.Now, this, null);
        }     

        public void Timeout(object state)
        {
            if (this.numberOfPendingAuthorizations <= 1)
                this.Complete();
        }     

        public Guid Id
        {
            get { return id; }
            set { id = value; }
        }     

        public bool Completed
        {
            get { return completed; }
        }     

        public void Handle(AuthorizeOrderResponseMessage message)
        {
            if (message.Authorized)
            {
                this.numberOfPendingAuthorizations--;     

                if (this.numberOfPendingAuthorizations == 1)
                    this.SendUpdate(OrderStatus.Authorized1);
                else
                {
                    this.SendUpdate(OrderStatus.Authorized2);
                    this.Complete();
                }
            }
            else
            {
                this.SendUpdate(OrderStatus.Rejected);
                this.Complete();
            }
        }     

        public void Handle(CancelOrderMessage message)
        {     

        }     

        private void SendUpdate(OrderStatus status)
        {
            OrderStatusUpdatedMessage update = new OrderStatusUpdatedMessage();
            update.OrderId = this.externalOrderId;
            update.Status = status;     

            this.bus.Send(this.clientAddress, update);
        }     

        private void Complete()
        {
            this.completed = true;     

            this.SendUpdate(OrderStatus.Accepted);     

            OrderAcceptedMessage accepted = new OrderAcceptedMessage();
            accepted.Products = new List<Guid>(this.orderItems.Count);
            accepted.Amounts = new List<float>(this.orderItems.Count);     

            this.orderItems.ForEach(delegate(CreateOrderMessage m)
                                        {
                                            accepted.Products.AddRange(m.Products);
                                            accepted.Amounts.AddRange(m.Amounts);
                                            accepted.CustomerId = m.CustomerId;
                                        });     

            this.bus.Publish(accepted);
        }
    }
}

All this code is online in the subversion repository under /Samples/Saga.

Questions, comments, and general thoughts are always appreciated.

Some Technical Difficulties

Ayende and I had an email conversation that started with me asking what would happen if I added an Order to a Customer’s “Orders” collection, when that collection was lazy loaded. My question was whether the addition of an element would result in NHibernate hitting the database to fill that collection. His answer was a simple “yes”. In the case where a customer can have many (millions) of Orders, that’s just not a feasible solution. The technical solution was simple – just define the Orders collection on the Customer as “inverse=true”, and then to save a new Order, just write:

Although it works, it’s not “DDD compliant”

In Ayende’s post Architecting for Performance he quoted a part of our email conversation. The conclusion I reached was that in order to design performant domain models, you need to know the kinds of data volumes you’re dealing with. It affects both internals and the API of the model – when can you assume cascade, and when not. It’s important to make these kinds of things explicit in the Domain Model’s API.

How do you make “transparent persistence” explicit?

The problem occurs around “transparent persistence”. If we were to assume that the Customer object added the Order object to its Orders collection, then we wouldn’t have to explicitly save orders it creates, so we would write service layer code like this:

On the other hand, if we designed our Domain Model around the million orders constraint, we would need to explicitly save the order, so we would write service layer code like this:

But the question remains, how do we communicate these guidelines to service layer developers from the Domain Model? There are a number of ways, but it’s important to decide on one and use it consistently. Performance and correctness require it.

Solution 1: Explicitness via Return Type

The first way is a little subtle, but you can do it with the return type of the “CreateOrder” method call. In the case where the Domain Model wishes to communicate that it handles transparent persistence by itself, have the method return “void”. Where the Domain Model wishes to communicate that it will not handle transparent persistence, have the method return the Order object created.

Another way to communicate the fact that an Order has been created that needs to be saved is with events. There are two sub-ways to do so:

Solution 2: Explicitness via Events on Domain Objects

The first is to just define the event on the customer object and have the service layer subscribe to it. It’s pretty clear that when the service layer receives a “OrderCreatedThatRequiresSaving” event, it should save the order passed in the event arguments.

The second realizes that the call to the customer object may come from some other domain object and that the service layer doesn’t necessarily know what can happen as the result of calling some method on the aggregate root. The change of state as the result of that method call may permeate the entire object graph. If each object in the graph raises its own events, its calling object will have to propagate that event to its parent – resulting in defining the same events in multiple places, and each object being aware of all things possible with its great-grandchild objects. That is clearly bad.

What [ThreadStatic] is for

So, the solution is to use thread-static events.

[Sidebar] Thread-static events are just static events defined on a static class, where each event has the ThreadStaticAttribute applied to it. This attribute is important for server-side scenarios where multiple threads will be running through the Domain Model at the same time. The easiest thread-safe way to use static data is to apply the ThreadStaticAttribute.

Solution 3: Explicitness via Static Events

Each object raises the appropriate static event according to its logic. In our example, Customer would call:

And the service layer would write:

The advantage of this solution is that it requires minimal knowledge of the Domain Model for the Service Layer to correctly work with it. It also communicates that anything that doesn’t raise an event will be persisted transparently behind the appropriate root object.

Statics and Testability

I know that many of you are wondering if I am really advocating the use of statics. The problem with most static classes is that they hurt testability because they are difficult to mock out. Often statics are used as Facades to hide some technological implementation detail. In this case, the static class is an inherent part of the Domain Model and does not serve as a Facade for anything.

When it comes to testing the Domain Model, we don’t have to mock anything out since the Domain Model is independent of all other concerns. This leaves us with unit testing at the single Domain Class level, which is pretty useless unless we’re TDD-ing the design of the Domain Model, in which case we’ll still be fiddling around with a bunch of classes at a time. Domain Models are best tested using State-Based Testing; get the objects into a given state, call a method on one of them, assert the resulting state. The static events don’t impede that kind of testing at all.

What if we used Injection instead of Statics?

Also, you’ll find that each Service Layer class will need to subscribe to all the Domain Model’s events, something that is easily handled by a base class. I will state that I have tried doing this without a static class, and injecting that singleton object into the Service Layer classes, and in that setter having them subscribe to its events. This was also pulled into a base class. The main difference was that the Dependency Injection solution required injecting that object into Domain Objects as well. Personally, I’m against injection for domain objects. So all in all, the static solution comes with less overhead than that based on injection.

Summary

In summary, beyond the “technical basics” of being aware of your data volumes and designing your Domain Model to handle each use case performantly, I’ve found these techniques useful for designing its API as well as communicating my intent around persistence transparency. So give it a try. I’d be grateful to hear your thoughts on the matter as well as what else you’ve found that works.

Related posts:

• Fetching Strategy Design – showing how to separate the concern of eager loading from both your Domain Model and your Service Layer.
• Better Domain-Driven Design Implementation – showing the basics of how valuable interfaces between your Domain Model and the Service Layer can be.
• Lazy Loading, and how messaging fixes everything again – describing the advantage of O/R mapping your message classes as well.
• Query Objects vs Methods on a Repository – discussing the scalability (in terms of number of developers and queries) of Query Objects.

After seeing Mark’s post on Reasons for Isolation describing the ways Layered Architectures break down, and the ways making it more testable can change it, I’ve got to wonder – is Layering just too simplistic to actually work?

Just the other day I was doing a design review for a fairly simple Smart Client whose design was layered. In order to stay away from interfaces that accepted dozens of ints, strings, and dates, they wanted to have each layer talk to the other using “entities”. So where are these entities defined – oh, in a “vertical layer” that all the horizontal layers talk to.

OK, so we’ve taken the simplistic one-dimensional layered architecture and added a dimension. What now?

Well, it seems that having the business logic and the entities in separate layers goes against one of the most basic Object Oriented principles – encapsulation. So, let’s put the entities back in the Business Logic Layer. But then how will the Data Access Layer accept those objects as parameters?

So, that is solved by keeping Entity Interfaces in the “vertical” shared “layer”, and having the entities in the business logic layer implement those interfaces. That way, the data access layer can still accept parameters corresponding to those interfaces:

void InsertCustomer(Shared.Entities.ICustomer customer);

So far so good. Now, we want more testable UI layer code – so we use Model-View-Controller (MVC) – of whichever flavor suits your fancy. I’d say that Supervising Controller is a must. You could also add another presenter for more complex screens as in Passive View, but I’d be less strict on that. So, in which layer do these Controllers/Presenters sit? And is the Business Logic Layer the Model? Or is the Model just part of it?

Well, our Supervising Controllers are those who decide what action to do and when, where to get the data from, etc. That sounds like business logic to me. So let’s put them in the BLL. Presenters for the Passive View are much more UI centered, so let’s put them in the Presentation Layer. But we don’t want them tied to the implementation of the view, so we’ll put them in a separate package, and have them depend only on the view’s interface. So we’ll put the view interfaces in a package separate from the view implementation as well.

If it wasn’t clear up to this point, all the questions raised in this post are architectural in nature – as in they have a substantial impact on the structure and flow of the system, and will definitely have a profound effect on its maintainability. In other words, if you think that Layer Diagram covers your design – you’re probably deluding yourself. Personally, I think that’s why many developers consider architects to be “out of touch with the real world”.

When you have a design that answers these, and other architectural concerns, you’ll find that layering is of little importance. The specific constraints on each package are what counts. The fact that the Presentation Layer can talk to the Business Logic Layer doesn’t mean that the classes in your Views Implementation Package can. A large part of an architects work is to specify these constraints, and communicate them to the team. Tools like FxCop may help in terms of enforcing these constraints, but I believe that getting the team to actually “buy-in” is more effective.

Single-dimensional layered architectures don’t work. They violate Einstein’s maxim:

Make everything as simple as possible, but not simpler.

Layering – “simpler” to the point of simplistic.

First I find out that NHibernate does support “Persistence by Reachability”, even though the docs say it doesn’t. Next, Ayende makes it support multiple queries in a single DB roundtrip, something I’ve been asking all the other O/R mappers out there to do. To top it off, he’s got his sights set on solving the issues I raised in my talk on Complex Business Logic with DDD and O/R Mapping at DevTeach. That’s right, he’s going to give me my decorators and state machines.

I love you, Oren.

I know that the ADO.NET Entity Framework guys are open to this as well, but I’m pretty sure that the “Entity Model” thinking will hold them back. You just can’t divorce data and behavior – not when employing state machines or decorators.

I’m sold.

When implementing a domain model, often object-relational mapping technologies are used. Like many tools, with their use comes the danger of abuse – abuse to the point of invalidating the benefits of the pattern itself.

From some pointers about how to use (and not to use) these tools, see why object-relational mapping sucks.

Martin Fowler’s has this to say about the Domain Model Pattern:

At its worst business logic can be very complex. Rules and logic describe many different cases and slants of behavior, and it’s this complexity that objects were designed to work with. A Domain Model creates a web of interconnected objects, where each object represents some meaningful individual, whether as large as a corporation or as small as a single line on an order form.

In short, using Object-Oriented techniques to handle the complexity.

A more comprehensive discussion about what happens when it is not used can be found under the Anemic Domain Model Anti-Pattern:

The basic symptom of an Anemic Domain Model is that at first blush it looks like the real thing. There are objects, many named after the nouns in the domain space, and these objects are connected with the rich relationships and structure that true domain models have. The catch comes when you look at the behavior, and you realize that there is very little behavior on these objects. Indeed often these models come with design rules that say that you are not to put any domain logic in the the domain objects. Instead there are a set of service objects which capture all the domain logic. These services live on top of the domain model and use the domain model for data.

The fundamental horror of this anti-pattern is that it’s so contrary to the basic idea of object-oriented design; which is to combine data and process together. The anemic domain model is really just a procedural style design…

In terms of Domain-Driven Design, this pattern is also known as Domain Layer.

Domain Models do a lot for encapsulating Business Rules, thus making them amenable to automated testing. This hinges on keeping the Domain Model independent of things related to Data Access.

It is therefore almost required to use some kind of Object/Relational Mapping tool to make it possible to persist objects belonging to the domain model to databases and other kinds of storage.

The use of DataSets in .NET is often a sign of the Anemic Domain Model Anti-Pattern.

One thing to keep in mind when working on a domain model is that you probably won’t get it “right” the first time, and will have re-work the division of responsibility a couple of times. Techniques like Test-Driven Development help out immensely for that.

So, last night I was at the MVP dinner in TechEd and everything was nice. We had a nice meal, conversation was nice, weather was… nice. And then the volume started to rise, slowly at first, so as you don’t quite notice it. After a bit, you kind of stop talking and look around. And then I hear it…

<WWF announcer voice>
Are you ready… to RUMBLE !?!?
</WWF announcer voice>

It was Datasets vs. O/R mapping, a slight twist on the infamous datasets vs. custom objects debate, all over again. They pulled me in, kicking and screaming, I swear, I really do. The lines were drawn, maintainability, performance, all the things that architects like to philosophize about in terms of other people’s work.

Anyway, I won’t give you the play-by-play ‘cause we were there almost all night. I’ll just cut to the chase.

First things first – any comparison of solutions without the context of a problem leads nowhere, fast, and stays there. So the first question I asked (when I got the chance to speak) was “are we talking about querying/reporting here?” and the answer was something like “well, yeah, but a lot of other things too”. So my suggestion was that we discuss the solutions in terms of two contexts – querying/reporting and OLTP.

What I mean by OLTP is the data-updating kind of work that you do on certain items. Examples of this include “insert order”, “change customer address”, and “discount product”. Querying/reporting doesn’t change data, and often involves dealing with large sets of data pulled from different kinds of entities (in ERD terms).

Luckily, my suggestion to deal with them separately was accepted. Secondly, I proposed that an object model (specifically implementing the Domain Model pattern) designed for OLTP would perform poorly when used for querying/reporting – simply because it wasn’t designed for it. The structure of a domain model is such that it makes it possible to define / implement business rules in one place. That’s possible, not easy.

Well, the dataset people weren’t going to just hand me the OLTP side of the equation without a fight, so they mentioned how easy it was to just “AcceptChanges”, and that my way was much more complex. My rebuttal came in the form of a question (are you seeing a pattern here?): Do you just swallow DbConcurrencyExceptions are do you throw all the user’s changes away when it happens? I didn’t quite make out the answer since there was a lot of mumbling going on, but I’m pretty sure they had one. I mean, you can’t develop multi-user systems using datasets without running into this situation.

The example that clinched OLTP was this. Two users perform a change to the same entity at the same time – one updates the customer’s marital status, the other changes their address. At the business level, there is no concurrency problem here. Both changes should go through. When using datasets, and those changes are bundled up with a bunch of other changes, and the whole snapshot is sent together from each user, you get a DbConcurrencyException. Like I said, I’m sure there’s a solution to it, I just haven’t heard it yet.

Now, here’s where things get interesting. I didn’t say that using a domain model automatically solves this problem. Rather, I described how each client could send a specific message, one a ChangeMaritalStatusMessage, the other a ChangeAddressMessage, to the server – in essence, giving the server the context in which each bit of data is relevant. The server could just open a transaction, get the customer object based on its Id, call a method on the customer (ChangeMaritalStatus or ChangeAddress), and commit the transaction. If two of these messages got to the server at the same time, the transactions would just cause them to be performed serially, and both transactions would succeed. The important part here is not losing the context of the changes.

When we talked about querying/reporting, things seemed quite a bit clearer. Datasets, or rather datatables seemed like a fine solution – most 3rd party controls support them out of the box. One guy mentioned that datasets performed poorly for large sets of data and that by designing custom entities for the result set, he could improve performance and memory utilization by, like, 70%. To tell you the truth, I think that if you need the performance, do it, if not, just use datasets. There isn’t much of an issue of correctness.

Just as an ending comment, in response to something someone said about scalability, I asked if they were reporting against the live OLTP data. The response was “yes”. Well, there’s a database scalability problem if I ever saw one. OLTP works most correctly when employing transactions that have an isolation level of serializable. The problem with them is that they lock up the whole table, or get blocked when a table scan is going on. Querying often results in a table scan. You can see the problem. Anyway, a common solution to this problem is to just reduce the isolation level, a quick fix that improves performance almost immediately. You take one hit in that your reports may be showing incorrect data, especially if they do aggregate type work. You might take another hit if your OLTP transactions need to do aggregate type work themselves. That second hit is pretty much unacceptable. A different solution is to accept the fact that the heaviest querying can usually show data that isn’t up to date up to the second.

In such a solution, you would have another database for reporting. It wouldn’t be just a replica of the OLTP database, but rather a lot more denormalized – which is a really not nice way of saying designed for reporting. You could then move the data from your OLTP database to the reporting database in some way (more to come on this topic) and you increase the scalability of your database. Just to define that a bit better – your OLTP database will be able to handle more transactions per unit of time, and reports will run faster, meaning that you will both improve their latency and the number of queries that can be handled per unit of time.

Anyway, I was pretty tired after all that, but if I had to sum it up I’d say something like this: before debating solutions, define the problem, you get a lot more insight into the solutions and you get it faster. That’s just win-win all around.

Domain Driven Design (DDD), alongside its growing popularity, is experiencing some growing pains. The Domain Model pattern, documented in the Patterns of Enterprise Application Architecture book, is at the heart of DDD yet the division of responsibility between it and other DDD patterns like Service Layer isn’t quite clear. To make matters worse, the value of the Domain Model pattern relative to simpler code-generation techniques remains vague. The one thing that has reached a wide consensus is that it requires a higher level of skill to employ these techniques than continue using the widespread procedural programming practices. There is one overwhelming reason to do it anyway, though, and that is that it is the cheapest way to get a system up and running right. The reasoning behind this has to do with business rules.

An exceedingly large number of systems are being built and modified today in order to support business rules. Beyond just computerizing the management of data in the enterprise, today’s systems are required to computerize the business processes that use that data, and these processes are built upon business rules.

A rule is composed of two main elements, a clause and an action. The clause defines under what circumstances the action is to be activated. An example of a rule employed in the business environment might be “if the customer is a preferred customer, then give them a 5% discount on all orders”. The rich behavior of an enterprise is governed by these interacting rules. Consider the result of adding another business rule like “if the customer has ordered $100,000 or more in the last year, then they are to be considered a preferred customer”. If a customer is not preferred, but then sends in a new order that puts them over the limit, how is the system to behave? Conversely, what if a preferred customer cancels an order bringing them under the limit?

I’ve seen too many projects that have been tasked with implementing these kinds of behaviors yet were unable to get the system running right. Customers that should have gotten discounts didn’t in some cases, and those that should not have enjoyed a discount did at times. After much time was spent trying to track down what part of the code was wrong, changing some code, testing, over and over again, an executive decision was made to put the system into production as is. The harm to the business was deemed cheaper in this case then not putting the system in production. Is it any wonder that business is skeptical of IT’s ability to handle the agile enterprise of the future where, not only will the business rules be more complex, they will be changing all the time.

The Domain Model pattern encapsulates these business rules in such a way that they will be run even when not directly invoked. This is especially critical when one rule triggers another. Intelligent use of OO principles when designing the domain can help you altogether avoid the jump in complexity found in Business Rules Engines.

Finally, we need to understand that supporting techniques like Object/Relational (O/R) mapping are but a means, and not the end. The discussions around DDD often get mired down in the relative costs of O/R mapping and procedural code generation. Persistence is a solved problem, a technical problem that has no meaning to business. Is it not time to raise the discussion to the level of business? If the only problem you are trying to solve is the manipulation of data in a database, then don’t bother with DDD and its descendents. It won’t make your life any easier. If, on the other hand, business has gotten sick of IT deciding for them how to run their business; if you are the one tasked with building the right system, you just won’t be able to do it unless you build the system right – DDD won’t be a bother, but a necessity.

After reading this article you just might be thinking that business rules management systems (BRMS’s) are the key to what SOA was supposed to bring (now that we’re in the trough of disillusionment) – that is, if you haven’t tried working with these beasts before.

There are several now-well-known fallacies around BRMS’s. One is that business analysts write the business rules, in plain English, and these rules do not require a programmer to enter/code them into the system; the BRMS handles all that.

“Costly, time-consuming transformation from business terms to programmer requirements and subsequent implementation can be reduced or eliminated. This in turn empowers business analysts to create and maintain the business logic directly, allowing them to make changes with little or no IT intervention.”

But, the author forgets to mention who performs these trivial activities:

“After testing and validation the business logic can be deployed to a business rules server.”

This brings me to the single most important thing anyone should know about the business rules approach. Employing a rules engine opens up the possibility that the system will behave non-deterministically. Consider that only 3 non-linear equations are needed to create chaotic behavior, and try to imagine what will happen in a system with hundreds and thousands of business rules that affect each other.

Yes, business rules often end up changing data which, in turn, causes other business rules to be activated. No, it is not humanly possible to look at such a set of rules and predict what will happen when a certain event occurs. You can fire such an event in a test environment and see what will happen. When the results aren’t what you wanted (that would be all the time), you can use the BRMS’s to show you which rules were run as a result of that event. Let’s see a business analyst debug that. “Plain English” my butt.

Let’s not go too much further on the BRMS on its own, but rather view it in the light of service-oriented architectures.

In the article, an approach is shown that is supposed to bring you the best of both SOA and the Business Rules Philosophy. I’m afraid that my experience has been otherwise. The author states:

“The business rules server may be implemented as a Web Service that is accessible to many SOA enabled applications across the organization. By supporting shared business logic within the SOA architecture that can be addressed by many disparate applications, organizations can reduce redundancy, speed implementation, lessen inconsistency and improve the overall efficiency of both the applications and the business processes they serve.”

This desire to concentrate all business logic into a single BRMS embodies the 11th fallacy of enterprise computing, as put forth by James Gosling “Business logic can and should be centralized”. Not to mention that this goes against the grain of SOA where each service is autonomous and is entirely responsible for all of its data and behavior. There’s another reason why the business rules community is pro centralization – the BRMS’s are so damned expensive.

Anyway, I don’t want to be all doom and gloom today so I’ll end on a positive note. While good OO practices and the use of the Domain Model Pattern will take you VERY far in making business rules explicit, and, therefore, their maintenance and evolution much less costly, sometimes you really neeed a BRMS. Just so you know, the investment you made in the domain model is not written off when you “upsize” to business rules – the rules themselves are written using the concepts exposed in the domain. As for the service, well, its message handlers keep on working the same as they always did; parsing messages and making calls on the domain model. The business rules will just respond to the events raised by the domain; message handlers don’t need to call into the BRMS.

Oh, and the next time somebody tells you how business analysts will be doing the work instead of programmers, ask them if they’re willing to take the debugging too