I hear this too often: “X sounds like a great pattern, but it’s overkill for small applications”. Many patterns have been subjected to this including (but not limited to): SOA, DDD, CQRS, ORM, etc. Often the statement is made by a person without experience in the given pattern (though possibly experienced in other patterns). Let’s take a look at the second part – the “small application”, and ask:

What makes an app small?

Or inversely, what makes an app warrant the “enterprise” moniker?

If there’s one thing that the history of our industry has shown repeatedly, it’s that developers aren’t particularly accurate with their estimates. Like, orders-of-magnitude inaccurate. Knowing this, it’s surprising that the “small app” argument seems to win so many arguments. The same goes for justifications in the form of “we’ve got to have an X, this is a BIG project”.

So, what makes an app small?

Is it a small number of lines of code? Well, what if those lines of code are keeping planes in the air?

Is it a small number of developers? Same as above. Actually, history has shown that some of the most valuable bits of code written were done by small numbers of developers.

Is it that it will only be installed on a single machine?

Is it…

What could it be?

The real issue

The small app argument is a diversionary tactic.

Loosely translated, it means “I’m comfortable where I am and I don’t want to change”.

Moving on…

The real story of size

Once we actually look at the specific context of an app, we tend to see that someone cares a great deal about it, enough to finance its custom development – rather than buying an off-the-shelf alternative. The expected lifetime of business use is easily 3-5 years, if not 7-10, during which many enhancements will likely be requested. Thus, some non-functional properties of the code matter – at the very least maintainability.

In which case, if the given pattern or approach does significantly improve the desired non-functional properties of the app, it only makes sense to use it.

There is one class of software that might possibly be treated as “small” – the one-off script that’s written to automate some IT task. And even then, so many of these scripts end up living longer than the apps themselves that they should be engineered at the same level of quality.

In closing

Don’t counter a “small app” argument with psychology.
It will only make matters worse.

Instead, rephrase the issue around the lifetime of business use.

I’ve found that there are precious few cases where the harsh light of reality doesn’t help the appropriate decisions be made. If indeed this is a small-lifetime-app, just drag-and-drop until you’re done. Otherwise, the time it takes to understand and evaluate the applicability of the given patterns will definitely pay itself back many times over the life of the app.

And managers, keep your ears open for it. The technical risks behind that statement are icebergs waiting to sink your project.

* with thanks to Mike Nichols for pushing my buttons.

After reading Ayende’s post advocating against “soft deletes” I felt that I should add a bit more to the topic as there were some important business semantics missing. As developers discuss the pertinence of using an IsDeleted column in the database to mark deletion, and the way this relates to reporting and auditing concerns is weighed, the core domain concepts rarely get a mention. Let’s first understand the business scenarios we’re modeling, the why behind them, before delving into the how of implementation.

The real world doesn’t cascade

Let’s say our marketing department decides to delete an item from the catalog. Should all previous orders containing that item just disappear? And cascading farther, should all invoices for those orders be deleted as well? Going on, would we have to redo the company’s profit and loss statements?

Heaven forbid.

So, is Ayende wrong? Do we really need soft deletes after all?

On the one hand, we don’t want to leave our database in an inconsistent state with invoices pointing to non-existent orders, but on the other hand, our users did ask us to delete an entity.

Or did they?

When all you have is a hammer…

We’ve been exposing users to entity-based interfaces with “create, read, update, delete” semantics in them for so long that they have started presenting us requirements using that same language, even though it’s an extremely poor fit.

Instead of accepting “delete” as a normal user action, let’s go into why users “delete” stuff, and what they actually intend to do.

The guys in marketing can’t actually make all physical instances of a product disappear – nor would they want to. In talking with these users, we might discover that their intent is quite different:

“What I mean by ‘delete’ is that the product should be discontinued. We don’t want to sell this line of product anymore. We want to get rid of the inventory we have, but not order any more from our supplier. The product shouldn’t appear any more when customers do a product search or category listing, but the guys in the warehouse will still need to manage these items in the interim. It’s much shorter to just say ‘delete’ though.”

There seem to be quite a few interesting business rules and processes there, but nothing that looks like it could be solved by a single database column.

Model the task, not the data

Looking back at the story our friend from marketing told us, his intent is to discontinue the product – not to delete it in any technical sense of the word. As such, we probably should provide a more explicit representation of this task in the user interface than just selecting a row in some grid and clicking the ‘delete’ button (and “Are you sure?” isn’t it).

As we broaden our perspective to more parts of the system, we see this same pattern repeating:

Orders aren’t deleted – they’re cancelled. There may also be fees incurred if the order is canceled too late.

Employees aren’t deleted – they’re fired (or possibly retired). A compensation package often needs to be handled.

Jobs aren’t deleted – they’re filled (or their requisition is revoked).

In all cases, the thing we should focus on is the task the user wishes to perform, rather than on the technical action to be performed on one entity or another. In almost all cases, more than one entity needs to be considered.

Statuses

In all the examples above, what we see is a replacement of the technical action ‘delete’ with a relevant business action. At the entity level, instead of having a (hidden) technical WasDeleted status, we see an explicit business status that users need to be aware of.

The manager of the warehouse needs to know that a product is discontinued so that they don’t order any more stock from the supplier. In today’s world of retail with Vendor Managed Inventory, this often happens together with a modification to an agreement with the vendor, or possibly a cancellation of that agreement.

This isn’t just a case of transactional or reporting boundaries – users in different contexts need to see different things at different times as the status changes to reflect the entity’s place in the business lifecycle. Customers shouldn’t see discontinued products at all. Warehouse workers should, that is, until the corresponding Stock Keeping Unit (SKU) has been revoked (another status) after we’ve sold all the inventory we wanted (and maybe returned the rest back to the supplier).

Rules and Validation

When looking at the world through over-simplified-delete-glasses, we may consider the logic dictating when we can delete to be quite simple: do some role-based-security checks, check that the entity exists, delete. Piece of cake.

The real world is a bigger, more complicated cake.

Let’s consider deleting an order, or rather, canceling it. On top of the regular security checks, we’ve got some rules to consider:

If the order has already been delivered, check if the customer isn’t happy with what they got, and go about returning the order.

If the order contained products “made to order”, charge the customer for a portion (or all) of the order (based on other rules).

And more…

Deciding what the next status should be may very well depend on the current business status of the entity. Deciding if that change of state is allowed is context and time specific – at one point in time the task may have been allowed, but later not. The logic here is not necessarily entirely related to the entity being “deleted” – there may be other entities which need to be checked, and whose status may also need to be changed as well.

Summary

I know that some of you are thinking, “my system isn’t that complex – we can just delete and be done with it”.

My question to you would be, have you asked your users why they’re deleting things? Have you asked them about additional statuses and rules dictating how entities move as groups between them? You don’t want the success of your project to be undermined by that kind of unfounded assumption, do you?

The reason we’re given budgets to build business applications is because of the richness in business rules and statuses that ultimately provide value to users and a competitive advantage to the business. If that value wasn’t there, wouldn’t we be serving our users better by just giving them Microsoft Access?

In closing, given that you’re not giving your users MS Access, don’t think about deleting entities. Look for the reason why. Understand the different statuses that entities move between. Ask which users need to care about which status. I know it doesn’t show up as nicely on your resume as “3 years WXF”, but “saved the company $4 million in wasted inventory” does speak volumes.

One last sentence: Don’t delete. Just don’t.

Convention over configuration describes a style of development made popular by Ruby on Rails which has gained a great deal of traction in the .net ecosystem. After using frameworks designed in this way, I can say that the popularity is justified – it is much more pleasurable developing this way.

The thing is, when looking at this in light of the full software development lifecycle, there are signs that the waters run deeper than we might have originally thought.

Let’s take things one step at a time though…

What is it?

Wikipedia tells us:

“Convention over Configuration (aka Coding by convention) is a software design paradigm which seeks to decrease the number of decisions that developers need to make, gaining simplicity, but not necessarily losing flexibility. The phrase essentially means a developer only needs to specify unconventional aspects of the application.”

What this means is that frameworks built in this way have default implementations that can be swapped out if needed. So far so good.

For example…

In NServiceBus, there is an abstraction for how subscription data is stored and multiple implementations – one in-memory, another using a durable MSMQ queue, and a third which uses a database. The convention for that part of the system is that the MSMQ implementation will be used, unless something else is specified.

Developers wishing to specify a different implementation can specify the desired implementation in the container – either one that comes out of the box, or their own implementation of ISubscriptionStorage.

Things get more interesting when we consider the full lifecycle.

Lifecycle effects

When developers are in the early phases of writing a new service, they want to focus primarily on what the service does – its logic. They don’t want to muck around with MSMQ queues for storing subscriptions and would much rather use the in-memory storage.

As the service takes shape and the developers want to run the full service on their machine, possibly testing basic fault-tolerance behaviors – kill one service, see that the others get a timeout, bring the service back up, wanting it to maintain all the previous subscriptions.

Moving on from there, our developers want to take the same system they just tested on their machine and move it into a staging environment. There, they don’t want to use the MSMQ implementation for subscription storage, but rather the database implementation – as will be used in the production environment.

While it may not sound like a big deal – changing the code which specifies which implementation to use when moving from one environment to another, consider that on top of just subscription storage, there is logging (output to console, file, db?), saga persistence (in-memory, file-based DB, relational DB), and more.

It’s actually quite likely that something will get missed as we move the system between environments. Can there be a better way?

What if…

What if there was some way for the developer to express their intent to the system, and the system could change its conventions, without the developer having to change any code or configuration files?

You might compare this (in concept) to debug builds and release builds. Same code, same config, but the runtime behaves different between the two.

As I mulled over how we could capture that intent without any code or config changes, the solution that I kept coming to seemed too trivial at first, so I dismissed it. Yet, it was the simplest one that would work for console and WinForms applications, as well as windows services – command line arguments. The only thing is that I don’t think those are available for web applications.

But since we’re still in “what if” land, and I’m more thinking out loud here than providing workable solutions for tomorrow morning, let’s “what if” command line arguments worked for web apps too.

Command-Line Intent

Going back to our original scenario, when developers are working on the logic of the service, they run it using the generic NServiceBus host process, passing it the command line parameter /lite (or whatever). The host then automatically configures all the in-memory implementations.

As the system progresses, when the developer wants to run everything on their machine, they run the processes with /integration. The host then configures the appropriate implementations (MSMQ for subscription storage, SQLite for saga persistence, etc.

When the developers want to run the system in production, they could specify /production (or maybe that could be the default?), and the database backed implementations would be configured.

Imagine…

Imagine being able to move that fluidly from one environment to another. Not needing to pore over configuration files or startup script code which configures a zillion implementation details. Not needing to worry that as you moved the system to staging something would break.

Imagine short, frictionless iterations even for large scale systems.

Imagine – lifecycle-aware frameworks making all this imagination a reality.

In Closing

We’re not there yet – but we’re not that far either. The generic host we’re providing with NServiceBus 2.0 is now being extended to support exactly these scenarios.

It’s my hope that as more of us think about this challenge, we’ll come up with better solutions and more intelligent frameworks. Just as convention came to our rescue before, breaking us out of the pain of endless XML configuration, I hope this new family of lifecycle-aware frameworks will make the friction of moving a system through dev, test, staging, and production a thing of the past.

A worthy problem for us all to solve, don’t you think?

Any ideas on how to make it a reality?
Send them in – leave a comment below.

I’ve been hearing from people that have had a great deal of success using the Domain Event pattern and the infrastructure I previously provided for it in Domain Events – Take 2. I’m happy to say that I’ve got an improvement that I think you’ll like. The main change is that now we’ll be taking an approach that is reminiscent to how events are published in NServiceBus.

Background

Before diving right into the code, I wanted to take a minute to recall how we got here.

It started by looking for how to create fully encapsulated domain models.

The main assertion being that you do *not* need to inject anything into your domain entities.

Not services. Not repositories. Nothing.

Just pure domain model goodness.

Make Roles Explicit

I’m going to take the advice I so often give. A domain event is a role, and thus should be represented explicitly:

   1:  public interface IDomainEvent {}

If this reminds you of the IMessage marker interface in nServiceBus, you’re beginning to see where this is going…

How to define domain events

A domain event is just a simple POCO that represents an interesting occurence in the domain. For example:

   1:  public class CustomerBecamePreferred : IDomainEvent 

   2:  {

   3:      public Customer Customer { get; set; }

   4:  }

For those of you concerned about the number of events you may have, and therefore are thinking about bunching up these events by namespaces or things like that, slow down. The number of domain events and their cohesion is directly related to that of the domain model.

If you feel the need to split your domain events up, there’s a good chance that you should be looking at splitting your domain model too. This is the bottom-up way of identifying bounded contexts.

How to raise domain events

In your domain entities, when a significant state change happens you’ll want to raise your domain events like this:

   1:  public class Customer

   2:  {

   3:      public void DoSomething()

   4:      {

   5:          DomainEvents.Raise(new CustomerBecamePreferred() { Customer = this });

   6:      }

   7:  }

We’ll look at the DomainEvents class in just a second, but I’m guessing that some of you are wondering “how did that entity get a reference to that?” The answer is that DomainEvents is a static class. “OMG, static?! But doesn’t that hurt testability?!” No, it doesn’t. Here, look:

Unit testing with domain events

One of the things we’d like to check when unit testing our domain entities is that the appropriate events are raised along with the corresponding state changes. Here’s an example:

   1:  public void DoSomethingShouldMakeCustomerPreferred()

   2:  {

   3:      var c = new Customer();

   4:      Customer preferred = null;

   5:   

   6:      DomainEvents.Register<CustomerBecamePreferred>(

   7:          p => preferred = p.Customer

   8:              );

   9:   

  10:      c.DoSomething();

  11:      Assert(preferred == c && c.IsPreferred);

  12:  }

As you can see, the static DomainEvents class is used in unit tests as well. Also notice that you don’t need to mock anything – pure testable bliss.

Who handles domain events

First of all, consider that when some service layer object calls the DoSomething method of the Customer class, it doesn’t necessarily know which, if any, domain events will be raised. All it wants to do is its regular schtick:

   1:  public void Handle(DoSomethingMessage msg)

   2:  {

   3:      using (ISession session = SessionFactory.OpenSession())

   4:      using (ITransaction tx = session.BeginTransaction())

   5:      {

   6:          var c = session.Get<Customer>(msg.CustomerId);

   7:          c.DoSomething();

   8:   

   9:          tx.Commit();

  10:      }

  11:  }

The above code complies with the Single Responsibility Principle, so the business requirement which states that when a customer becomes preferred, they should be sent an email belongs somewhere else.

Notice that the key word in the requirement – “when”.

Any time you see that word in relation to your domain, consider modeling it as a domain event.

So, here’s the handling code:

   1:  public class CustomerBecamePreferredHandler : Handles<CustomerBecamePreferred>

   2:  { 

   3:     public void Handle(CustomerBecamePreferred args)

   4:     {

   5:        // send email to args.Customer

   6:     }

   7:  } 

This code will run no matter which service layer object we came in through.

Here’s the interface it implements:

   1:  public interface Handles<T> where T : IDomainEvent

   2:  {

   3:      void Handle(T args); 

   4:  } 

Fairly simple.

Please be aware that the above code will be run on the same thread within the same transaction as the regular domain work so you should avoid performing any blocking activities, like using SMTP or web services. Instead, prefer using one-way messaging to communicate to something else which does those blocking activities.

Also, you can have multiple classes handling the same domain event. If you need to send email *and* call the CRM system *and* do something else, etc, you don’t need to change any code – just write a new handler. This keeps your system quite a bit more stable than if you had to mess with the original handler or, heaven forbid, service layer code.

Where domain event handlers go

These handler classes do not belong in the domain model.

Nor do they belong in the service layer.

Well, that’s not entirely accurate – you see, there’s no *the* service layer. There is the part that accepts messages from clients and calls methods on the domain model. And there is another, independent part that handles events from the domain. Both of these will probably make use of a message bus, but that implementation detail shouldn’t deter you from keeping each in their own package.

The infrastructure

I know you’ve been patient, reading through all my architectural blah-blah, so here it is:

   1:  public static class DomainEvents

   2:  { 

   3:      [ThreadStatic] //so that each thread has its own callbacks

   4:      private static List<Delegate> actions;

   5:   

   6:      public static IContainer Container { get; set; } //as before

   7:   

   8:      //Registers a callback for the given domain event

   9:      public static void Register<T>(Action<T> callback) where T : IDomainEvent

  10:      {

  11:         if (actions == null)

  12:            actions = new List<Delegate>();

  13:   

  14:         actions.Add(callback);

  15:     }

  16:   

  17:     //Clears callbacks passed to Register on the current thread

  18:     public static void ClearCallbacks ()

  19:     {

  20:         actions = null;

  21:     }

  22:   

  23:     //Raises the given domain event

  24:     public static void Raise<T>(T args) where T : IDomainEvent

  25:     {

  26:        if (Container != null)

  27:           foreach(var handler in Container.ResolveAll<Handles<T>>())

  28:              handler.Handle(args);

  29:   

  30:        if (actions != null)

  31:            foreach (var action in actions)

  32:                if (action is Action<T>)

  33:                    ((Action<T>)action)(args);

  34:     }

  35:  } 

Notice that while this class *can* use a container, the container isn’t needed for unit tests which use the Register method.

When used server side, please make sure that you add a call to ClearCallbacks in your infrastructure’s end of message processing section. In nServiceBus this is done with a message module like the one below:

   1:  public class DomainEventsCleaner : IMessageModule

   2:  { 

   3:      public void HandleBeginMessage() { }

   4:   

   5:      public void HandleEndMessage()

   6:      {

   7:          DomainEvents.ClearCallbacks();

   8:      }

   9:  }

The main reason for this cleanup is that someone just might want to use the Register API in their original service layer code rather than writing a separate domain event handler.

Summary

Like all good things in life, 3rd time’s the charm.

It took a couple of iterations, and the API did change quite a bit, but the overarching theme has remained the same – keep the domain model focused on domain concerns. While some might say that there’s only a slight technical difference between calling a service (IEmailService) and using an event to dispatch it elsewhere, I beg to differ.

These domain events are a part of the ubiquitous language and should be represented explicitly.

CustomerBecamePreferred is nothing at all like IEmailService.

In working with your domain experts or just going through a requirements document, pay less attention to the nouns and verbs that Object-Oriented Analysis & Design call attention to, and keep an eye out for the word “when”. It’s a critically important word that enables us to model important occurrences and state changes.

What do you think? Are you already using this approach? Have you already tried it and found it broken in some way? Do you have any suggestions on how to improve it?

Let me know – leave a comment below.

Every once in a while this topic pops up, and since the nServiceBus code base doesn’t follow the apparently accepted practice, and I do get asked about it, here goes.

First of all, the conventional wisdom:

“If you don’t choose assembly to represent component, the natural artifact candidate is then namespace.”

There’s only one minor assumption here that deserves being dragged out into the light.

While Visual Studio creates an assembly from every project by default, you can take those assemblies and merge them together into a single assembly using this nice little utility from Microsoft. It is likely that each project would have its own namespace too, so we should still be aligned with the conventional wisdom.

In other words, we could choose a Visual Studio project to represent a logical component and still be in the same camp as Jeremy:

“I’m very firmly in the camp that says you should only split assemblies by deployment targets”

What everyone agrees about seems to be that coupling hurts, and should be managed.

Where does coupling come from? Well, from references between two pieces of code. If we were to represent our logical components as Visual Studio projects, we could easily see those references without the help of any 3rd party tools. The compiler would even yell at us if we were to (accidentally) create an evil circular reference.

While some might complain about the long compile time when we have many projects in a single solution, good componentization often doesn’t require us to put all projects in a single solution. In fact, each component could theoretically have its own solution – since it’s reasonable to assume we’d really only be working on one component at a time. In which case, compile time per developer task would be a non-issue.

Going through the whole code base is usually only needed when doing a full-system debug when trying to track down a problem. This wouldn’t need to be done against a solution with all projects. We’d do this using PDBs of the merged projects (as that’s what actually got delivered, and where the bug was found). After spelunking through those PDBs, we’d eventually find the problematic component (or 2, or 3, or …), and open up developer tasks for each component.

Regardless of if we’re putting out a patch for an existing customer or rolling these changes into a release with other tasks, all the logical components would be built into a physical system (merged as necessary) and the system would be put through QA.

In short, it looks like just a bit of unconventional wisdom gets us a nice balance.

“We need to rewrite the system.”

Thus begins the story of yet another developer trying to convince their manager to adopt test-driven development (or any other methodology or technology). There’s a good chance this developer’s been reading all sorts of stuff on blogs (like those linked here) that have convinced him that salvation lies that way.

Don’t get me wrong.

There’s a good chance the developer’s right.

It’s just that that’s besides the point.

Developers and Managers

There’s a difference between how developers view a practice and how a manager (defined for the purposes of this post as someone in charge of delivering something) view that same practice. From a developer perspective, Ian’s point about unit testing is spot on:

“The problem is that the most important step is not doing it right, but doing it at all.”

Yet, as Ian himself points out in the title, this is a learning issue. If you want to learn to swim, there’s no replacement for jumping in the pool.

The manager’s perspective is a bit different.

Yes, we want our developers to improve their skill set. Yes, we understand that unit testing will ultimately improve quality. Yes, we know developers need to practice these skills as a part of their job. But, and it’s a big ole’ but, when it comes time to sink or swim, and we’ve got a deadline, those desires need to be balanced with delivering. Accounts of unit testing adoption efforts resulting in more (test) code to support with little apparent improvement in quality in the short term, well, they scare us. Arnon’s post gives more links supporting that feeling.

What’s a Unit Test anyway?

Is it any class that happens to have a TestFixture attribute on it?

If we are to “decouple” unit testing from good design, as Roy has described, that’s a not-improbable outcome. If the design of the system is such as there aren’t any real “units”, what exactly are we testing? Regardless of static or dynamic typing, replaceability of code, and other technological things, does the fact that all TestMethods in that TestFixture complete successfully mean anything? In other words, what did the test test?

It is clear that these tests cost something.

It’s more code to write. It’s more code to maintain.

The question is, what value are we getting from these “unit tests that any developer without design skills can write”?

The manager in me doesn’t like this return on investment.

By the way, TDD is as much the evolution of unit testing as the screw driver is the evolution of the hammer. But that’ll have to wait for a different post.

What’s Design Got To Do With It?

If you’re looking for the technical ability to write a test fixture and replace calls to other classes, then design has nothing to it.

If tests are to be valuable – design has everything to do with it.

The difficulty our developer is having unit-testing the system is a symptom of design problems. There’s a good chance that’s why he suggested a rewrite.

By the way, please do a search & replace in your vocabulary on the word “rewrite” with the word “redesign”. The code’s syntax isn’t the problem – it’s not the “m_”, camel case, or anything like that. It’s not that if the code was rewritten under the same design that all problems will go away.

Redesign, or do nothing.

The community’s been discussing the issues of coupling, interfaces, mocking, and tools at length in the context of testability. I won’t reiterate the debate here but I’ll tell you this:

If logic is duplicated, if the code is tightly coupled, if there is no separation of concerns, the unit tests will be useless – even if they “test” the class in isolation.

Cut the coverage crap

Metrics lie.

The fact that there’s a bunch of other code which calls 100% of the system’s code and doesn’t contain false assertions doesn’t mean that the code is high quality or doesn’t contain bugs.

In a well designed system, most “logic” will be contained in two “layers” – the controllers and the domain model. These classes should be independent of any and all technological concerns. You can and should get high unit test coverage on classes in these layers. Shoot for 100%, it’s worth it.

Testing domain models is all about asserting state. While using setters to get the domain objects into a necessary initial state is OK, setters should not be used beyond that. Testing controllers is primarily about interactions – mocks will probably be needed for views and service agents. Commands do not need to be mocked out.

Most other layers have some dependence on technology that makes unit tests relatively less valuable. Coverage on these layers is most meaningless. My guidance is to take the effort that would have been spent on unit testing these other layers and invest it all in automated integration tests. You’re likely to get a much higher return on investment. Much higher.

Much.

Everybody’s Right

Developers aren’t just born knowing good design, testing, or anything else. Universities, colleges, and most vendors do little to change that state of affairs. Books help, a bit, but when learning to swim, you’ve got to get your feet wet, and on the job training is, by and large, all there is. As such, lowering the barrier to entry is important.

Keeping in mind the Dreyfus model of knowledge acquisition, it’s not about “dumbing down” software development, it’s about bringing novices up to speed:

“In the beginning [novices] learn to recognize objective facts and features, relevant to the skill. Characteristic of relevant elements are that they can be recognized context-free, i.e. without reference to the overall situation. The novice acquire basic rules to follow, acting upon those facts and features. The rules are also context-free, i.e. no notice is taken to the surroundings. On account of this the novice feels very little responsibility for the result.” (emphasis mine)

Managers are ultimately responsible for the result.

Managers shouldn’t necessarily sacrifice their projects on this altar of learning. Organizations need to find ways for developers to safely practice these techniques as a part of developing their “human resources”. First of all, this needs to be communicated to everyone – that the organization understands the importance of these techniques, the desires of developers to adopt them, and the projects that need to be delivered.

Some projects may be allocated additional non-functional requirements: the software will be developed test-first, there will be at least 80% unit test coverage, etc. It can make sense to have developers spend some time on these projects after finishing one more delivery focused project and before going onto another one. As more developers become proficient with unit testing and design, the delivery focused projects can start to benefit from these skills.

It’s a gradual process.

The Important Bit

No matter how you go about unit testing, do periodic test reviews.

Just like code reviews.

That’s it.

Related Posts

Business Process Verification

Self documenting and Test-Driven Alien Artifacts

SOA Testing

Update: The next post in this series is now online here.

My previous post on how to create fully encapsulated domain models introduced the concept of events as a core pattern of communication from the domain back to the service layer. In that post, I put up enough code to get the idea across but didn’t address issues like memory leaks and multi-threading. This post will show the solution to those two critical points.

I’ve snipped out one of the events in the previous example for brevity.

Previous API

The previous API looked like this:

   1:  public static class DomainEvents

   2:  {

   3:       public static event EventHandler GameReportedLost;

   4:       public static void RaiseGameReportedLostEvent()

   5:       {

   6:             if (GameReportedLost != null)

   7:                 GameReportedLost(null, null);

   8:       }

   9:   

  10:       public static event EventHandler CartIsFull;

  11:       public static void RaiseCartIsFull()

  12:       {

  13:             if (CartIsFull != null)

  14:                 CartIsFull(null, null);

  15:       }

  16:  }

One thing that we want to keep in the solution is that all the code to define events, their names, and the parameters they bring will be in one place – in this case, the DomainEvents class. One thing that we’d like to fix is the amount of code needed to define an event.

Previous Service Layer

Here’s what our previous service layer code looked like:

   1:  public class AddGameToCartMessageHandler :

   2:      BaseMessageHandler<AddGameToCartMessage>

   3:  {

   4:      public override void Handle(AddGameToCartMessage m)

   5:      {

   6:          using (ISession session = SessionFactory.OpenSession())

   7:          using (ITransaction tx = session.BeginTransaction())

   8:          {

   9:              ICart cart = session.Get<ICart>(m.CartId);

  10:              IGame g = session.Get<IGame>(m.GameId);

  11:   

  12:              Domain.DomainEvents.GameReportedLost +=

  13:                gameReportedLost;

  14:              Domain.DomainEvents.CartIsFull +=

  15:                cartIsFull;

  16:   

  17:              cart.Add(g);

  18:   

  19:              Domain.DomainEvents.GameReportedLost -=

  20:                gameReportedLost;

  21:              Domain.DomainEvents.CartIsFull -=

  22:                cartIsFull;

  23:   

  24:              tx.Commit();

  25:          }

  26:      }

  27:   

  28:      private EventHandler gameReportedLost = delegate { 

  29:            Bus.Return((int)ErrorCodes.GameReportedLost);

  30:          };

  31:   

  32:      private EventHandler cartIsFull = delegate { 

  33:            Bus.Return((int)ErrorCodes.CartIsFull);

  34:          };

  35:      }

  36:  }

Another thing that should be improved is the amount of code needed in the service layer.

Raising an event, though, should still be fairly simple – one line of code similar to DomainEvents.RaiseGameReportedLost().

New API

Here’s what the new API looks like:

   1:  public static class DomainEvents

   2:  {

   3:       public static readonly DomainEvent<IGame> GameReportedLost = 

   4:                                            new DomainEvent<IGame>;

   5:   

   6:       public static readonly DomainEvent<ICart> CartIsFull=

   7:                                            new DomainEvent<ICart>;

   8:  }

It looks like we’ve managed to bring down the complexity of defining an event.

Raising an event is slightly different, but still only one line of code (”this” refers to the Cart class that is calling this API): DomainEvents.CartIsFull.Raise(this);

New Service Layer

The advantage of having a disposable domain event allows us to use the “using” construct for cleanup.

   1:  public class AddGameToCartMessageHandler :

   2:      BaseMessageHandler<AddGameToCartMessage>

   3:  {

   4:      public override void Handle(AddGameToCartMessage m)

   5:      {

   6:          using (ISession session = SessionFactory.OpenSession())

   7:          using (ITransaction tx = session.BeginTransaction())

   8:          using (DomainEvents.GameReportedLost.Register(gameReportedLost))

   9:          using (DomainEvents.CartIsFull.Register(cartIsFull))

  10:          {

  11:              ICart cart = session.Get<ICart>(m.CartId);

  12:              IGame g = session.Get<IGame>(m.GameId);

  13:   

  14:              cart.Add(g);

  15:   

  16:              tx.Commit();

  17:          }

  18:      }

  19:   

  20:      private Action<IGame> gameReportedLost = delegate { 

  21:            Bus.Return((int)ErrorCodes.GameReportedLost);

  22:          };

  23:   

  24:      private Action<ICart> cartIsFull = delegate { 

  25:            Bus.Return((int)ErrorCodes.CartIsFull);

  26:          };

  27:      }

  28:  }

I also want to mention that you don’t necessarily have to have the same service layer object handle these events as that which calls the domain objects. In other words, we can have singleton objects handling these events for things like sending emails, notifying external systems, and auditing.

The Infrastructure

The infrastructure that makes all this possible (in a thread-safe way) is quite simple and made up of two parts, the DomainEvent that we saw being used above, and the DomainEventRegistrationRemover which handles the disposing:

   1:  using System;

   2:  using System.Collections.Generic;

   3:   

   4:  namespace DomainEventInfrastructure

   5:  {

   6:      public class DomainEvent<E> 

   7:      {

   8:          [ThreadStatic] 

   9:          private static List<Action<E>> _actions; 

  10:   

  11:          protected List<Action<E>> actions 

  12:          {

  13:              get { 

  14:                  if (_actions == null) 

  15:                      _actions = new List<Action<E>>(); 

  16:   

  17:                  return _actions; 

  18:              }

  19:          }

  20:   

  21:          public IDisposable Register(Action<E> callback) 

  22:          {

  23:              actions.Add(callback);

  24:              return new DomainEventRegistrationRemover(delegate

  25:                  {

  26:                      actions.Remove(callback);

  27:                  }

  28:              ); 

  29:          }

  30:   

  31:          public void Raise(E args) 

  32:          {

  33:              foreach (Action<E> action in actions) 

  34:                  action.Invoke(args);

  35:          }

  36:      }

  37:  }

  38:   

Note that the invocation list of the domain event is thread static, meaning that each thread gets its own copy – even though they’re all working with the same instance of the domain event.

Here’s the DomainEventRegistrationRemover – even simpler:

   1:  using System;

   2:   

   3:  namespace DomainEventInfrastructure

   4:  {

   5:      public class DomainEventRegistrationRemover : IDisposable 

   6:      {

   7:          private readonly Action CallOnDispose;

   8:   

   9:          public DomainEventRegistrationRemover(Action ToCall) 

  10:          {

  11:              this.CallOnDispose = ToCall; 

  12:          }

  13:   

  14:          public void Dispose() 

  15:          {

  16:              this.CallOnDispose.DynamicInvoke();

  17:          }

  18:      }

  19:  }

For your convenience, I’ve made these available for download here.

I also want to add that if you haven’t looked at the comments on the original post – there’s some really good stuff there (36 comments so far). Take a look.

Don’t.

Not in applicative code anyway.

This follows up on Ayende’s post about the AOP way.

Now, I have nothing against AOP but some developers are leery of it.

In broader terms, all logging goes in framework-level code. For smart clients, one really good place to put logging is in your Command infrastructure – every time a command is invoked, log it and the args. For data access, well, any decent O/R Mapper has a lot of logging already, just use it. For communication, ditto. Funny that just last week this was one of the major bits of feedback I gave in a code review.

The Important Part

Logging is useful for developers to find out why a system isn’t working correctly.

It is terrible for knowing that a system isn’t working correctly.

If you’re entire exception management strategy is “write it to the log”, how will an admin know that something’s wrong? Did you remember to configure your logging library that errors (and maybe warnings too) should be pushed out to a monitoring system? Do you have a monitoring system?

And if the admins don’t know anything’s wrong, they won’t know they need to increase the fidelity of the logs, will they? Are you planning on providing training for your admins telling them this (and all the other things they need to know)? Or maybe this will all be set up as an automatic script?

An Agile Digression

I hope all that’s on your agile (”we can ship at the end of every 2 week iteration”) product backlog (pardon my cynicism). I hope it’s at least something that you’re looking at per release and feeding the relevant features into your iterations. Yes, there’s project work to do (writing training manuals) that isn’t “development” that needs to be handled; if you don’t timebox it into the same iterations, it won’t get done.

Now, back to you’re regularly schedule logging…

Things Logging Doesn’t Address

Logging is a mildly useless tool for pinpointing where in the system the source of a problem is.

“I know the entity isn’t in the database. I can see that. I want to know why it isn’t there.”

Sure, if you had every SQL statement logged you could figure these sorts of things out out. Of course, performance-wise, you wouldn’t put the system into production like that. In which case, the delete statement wouldn’t have been logged leaving you with precious little information to solve the root cause.

Also consider that the more logging you do, the more crap you’ll have to sift through to find the proverbial needle. Developers often don’t think twice about increasing the amount of crap logs they generate…

The Real Problem

The real problem is that developers think too much about logging and not at all nearly enough about designing the system in ways that it’ll be easy possible to answer questions like those above without having to know exactly how the system is built. One of the reasons that developers should care about this is that it’ll decrease the number of times they need to get up at 3:00 am to answer those questions.

A Path to the Solution

Now, if you had some kind of business activity monitoring (BAM) capability in your system, an admin could do a simple search/query [WHEN entity DELETED] and find out answers to the questions above, find out the time that the relevant activities occurred, figure out what the problem is on their own, and maybe even fix it – especially if it has to do with some esoteric configuration variable.Regardless of whether you buy a BAM tool or roll what you need yourself, you need to understand what about the system needs to be monitored. That’s a very different thought-process to go through than “should we log this? Yeah, sure, why not.”

It’s called “Design for Operations”.

Take a holistic perspective on exception management, logging, monitoring, etc. Think about questions like those above and then analyse your use of the relevant tools in that context. Think about all the different kinds of users of the information that’s going to be generated and how quickly their going to need to act on that information. Admins in the data-center in the middle of a crisis are going to have different needs than developers analysing logs on their machine. Think about:

• How will the administrator know that a server has been configured properly?
• If the system is feeling slow, how can the administrator know which server/process is to blame?
• So that maybe they can scale out that part of the system.

In Closing

It’s a mindset.

It takes time to make the shift.

It takes more time to bring the development process to this kind of maturity (god, I hate that word).

Writing exceptions to the log is not a strategy.

At the very best, its a tactic.

What’s your strategy?

One of the things I haven’t like about using IoC containers, AKA dependency injection frameworks, was the string-based configuration model they exposed. In order to set these values, developers had 2 options: either use XML config (usually without the benefit of intellisense or refactoring support), or use code (still quoting property names – again, no intellisense or refactoring support).

In short, there seemed to be a hole in the development model.

Here’s an example from how nServiceBus used to do this:

builder.ConfigureComponent(typeof(HttpTransport), ComponentCallModelEnum.Singleton)
  .ConfigureProperty(”DefaultNumberOfWorkerThreads”, 10)
  .ConfigureProperty(”DefaultNumberOfSenderThreads”, 10);

The problem was that if a developer got the case of the property wrong, misspelled it in some way, or somebody later refactored/renamed that property, the system would break. It would also be very difficult to figure out why.

Then, a couple of weeks ago, it dawned on me.

This was the same problem we used to have with testing using mock objects – before we had today’s more advanced frameworks. So, the solution must be to use the same techniques. The container should give the developer an object that looks just like their class, but that would intercept all calls. Then, that interceptor could turn those into the config calls shown above. Here’s what the new config model looks like: 

HttpTransport transport = builder.ConfigureComponent<HttpTransport>
                                                   (ComponentCallModelEnum.Singleton);

transport.DefaultNumberOfSenderThreads = 10;
transport.DefaultNumberOfWorkerThreads = 10; 

Granted, you’re not going to have tons of code like this. However, for all those parameters which are factory-configured and that customers/integrators shouldn’t tinker with, it makes a difference. The biggest difference is during that time of development where you’ve gotten into preliminary integration tests but the systems components are still being “polished”.

Aside: On the current project that has adopted this model, we’ve probably saved (conservatively) about 3 months of effort with this tiny (?) thing, and this isn’t a huge project. If that’s more than you would’ve thought, well, I was surprised myself. First, understand that in the old config model, everything still compiles and unit tests pass, even though its broken.

Just consider what happens in the lab when this occurs. You have N testers that can’t test the new version, waiting. You have the person who installed the version, trying to figure out what’s wrong. They then call in one of the developers where most of the new development occurred since the previous version. They fiddle around with it, looking at exception traces and whatnot. In the best case, we’re talking about 2 hours from noticing its broken until a new version comes out fixed. Multiply that by N+3 people. Then multiply by the number of versions you do integration tests on in the lab.

Caveat: In the current version, properties must be virtual in order for this to work.

For those of you who want just this feature without nServiceBus, I’ve put up all the binaries here. For the source, you’ll need to go to here.

Let me know what you think – especially if you can take the implementation to the point where it won’t need virtual properties to work

Prism, AKA Composite Application Guidance + Composite Application Library, is rolling towards a release. I’ve been talking with Glenn Block quite a bit about Prism, and am even on the advisory board (what were they thinking?).

One of the topics not covered by Prism is occasional connectivity, and I would like to say a word or two about that. First of all, if you’re building a standalone client (one that doesn’t communicate with anything), then there’s a good chance that Prism isn’t for you, although you could be composing other standalone client modules. So, if your client isn’t communicating with anything, well, then this post probably won’t interest you that much. Let’s start with…

Physics

Networks fail. Period.

This means that your client machine will not always be connected to other servers.

Also, servers fail – critical Windows patches and just regular power outages.

Ergo, your “smart” client will be occasionally connected, whether you planned for it or not.

And please don’t take this post as a “dumping on Prism” post – it isn’t intended that way. Rather, it is about how you should think about designing modules in Prism, and why.

Modules and Connectivity

Consider the case where we have two modules being composed in a single client. Each module communicates with a different server. Let’s call these modules Ma and Mb, and the servers Sa and Sb respectively. Now, let’s discuss what occurs given that the modules weren’t designed with occasional connectivity in mind.

User clicks something in Mb which requires communication.

Mb tries to call Sb, say, over HTTP, using a regular web service invocation.

The calling thread, in this case, the one used for user interaction, is blocked waiting for a response from Sb.

Sometime in this call, Sb fails, connectivity goes down, whatever.

30 seconds after the call, the HTTP connection times out.

If something important were happening in Ma at the same time, the user couldn’t even see it, let alone do anything about it since the user interaction thread is stuck. This is a serious concern for the financial services domain, but in many others as well.

You mean there’s more?

I can go on, but I think that that’s enough to paint the picture that if you are building a smart client, there are a lot more things to think about than just learning Prism. That’s my main concern after witnessing what happened around the CAB. Given the learning curve around these frameworks many developers don’t seek to deepen their understanding beyond just becoming proficient with them. This isn’t just centered on the developers, evangelists in Microsoft tend to paint the picture this way:

Once you understand X (CAB, Prism, BizTalk, whatever), all your problems are solved.

That’s not to say there aren’t good things in those technologies, but that’s just it, they’re just tools. Silver hammers and “laser” guided saws do not a master carpenter make. There’s actually a pretty good chance the regular guy will saw their arm off.

Help

I do hope more “instruction manuals” will be coming out of Microsoft on these topics. That’s not to say there aren’t any. Specifically on the topic of occasional connectivity, there is Chapter 4 of the Smart Client Architecture & Design Guide. Unfortunately, it doesn’t say anything about how that connects with the MVC/MVP being used client side (the bits affected by Prism). Chapter 6 of the same guide deals with the client-side threading, but doesn’t address issues like:

• Which model object instance are views bound to.
• Do other threads have access to that object at the same time.
• Which controller/presenter is responsible for giving that object to the view.
• Do they need to clone it.
• How deep should the clone be.
• How do various controllers/presenters (which may be showing the same object in different views at the same time) communicate changes to their various independent clones.

I haven’t yet documented all the patterns that answer these questions, but until I do (or Microsoft does), let me offer these few resources which I’ve put out over the years:

• Occasionally Connected Systems Architecture: The Client
• Occasionally Connected Systems Architecture: Concurrency
• Threading issues when smart clients reconnect
• [Podcast] Occasionally Connected Smart Clients and ADO.NET Sync Services

There’s also some more links under the Smart Client link of my “First time here?” page.

Also, please join me in asking Microsoft for an update to these guides – comments below or your own blog posts would be great.