Some background

As a cohesive framework, NServiceBus makes it quite easy for developers to pick and choose which settings they want turned on and off. Being built as a loosely-coupled set of components that don’t know about each other has always kept the internal complexity low. But as the NServiceBus API has been evolving over the years, and the functionality offered has increased, some interesting challenges have popped up as the codebase has been refactored.

The challenge

The UnicastBus class has grown too large and it’s time to refactor something out. Coincidentally, users have been asking for a better “header” story for messages – the ability to specify static headers that will be appended to all messages being sent (useful for things like security tokens), as well as per message headers. So, we want to refactor all the header management out to its own component independent of the UnicastBus class.

So, here’s the issue. So far, users have specified “.UnicastBus()” as a part of the fluent code-configuration, and shouldn’t have to change that – they shouldn’t need to know that header management is now a separate component. But then how can the new component bootstrap itself into the startup, such that it gets all the dependency injection facilities of the rest of the framework? Remember that the component doesn’t know which container technology is being used (since the user can swap it out) or when the container has been set.

The solution

The only part of the framework that knows about when all DI configuration is set is the configuration component, thus it will have to be the one that invokes the new component (without knowing about it). Introduce an interface (say INeedInitialization) and scan all the types loaded looking for classes which implement that type, register them into the container, and invoke them. Have the new component implement that interface, and in its initialization have it hook into the events and/or pipelines of other parts of the system.

Other uses

One historically problematic area in NServiceBus has been people forgetting to call “.LoadMessageHandlers()”. This can now be wired in automatically by a class in the UnicastBus component via the same mechanism.

A new feature coming in the next version is the “data bus”, a component which will allow sending large quantities of data through the bus without going through the messaging pipelines. This will help people get around the 4MB limit of MSMQ and, even more importantly, the much smaller 8KB limit of Azure. We will be able to introduce the functionality transparently with the same mechanism.

As an extension point, developers can now enrich the NServiceBus framework with their own capabilities and make those available via the contrib project to the community at large. This is better than the IWantToRunAtStartup interface that was only available for those using the generic host (which excluded web apps) and gives a consistent extensibility story for all uses.

Summary

Extensibility has always been a challenge when writing object-oriented code and dependency injection techniques have helped, but sometimes you need a bit more to take things to the next level while maintaining a backwards-compatible API.

Like I said, not a ground-shaking topic but something quite necessary in creating loosely-coupled frameworks and applications. Once you know it’s there, it isn’t really a big deal. If you didn’t know to do it, you may have been contorting your codebase in all kinds of ways to try to achieve similar things.

If you want to take a look at the code, you can find the SVN repository here: https://nservicebus.svn.sourceforge.net/svnroot/nservicebus/trunk/

Unfortunately, there is one tiny little mistake that I see almost everywhere that gets in the way, and that’s going to be the topic of this post.

The Problem

Let’s say you have a standard web app environment – some web servers, application servers, and a database server. Your web servers need to send messages to the application servers. So far, so good.

In your test environment, you have an application server called AS_01_Test, and your web servers are configured to send it messages. However, in your staging environment the application server fulfilling that same role is called AS_01_Stage. This creates a configuration problem – you need to change the config of your web servers as you move the web app from Test to Staging.

I’ve seen companies doing all sorts of creative things to get around this problem – some of them involve putting all configuration settings in a database so that they can be centrally managed and visualized. I’d like to suggest an alternative approach.

What if…

What if server names were the same across all environments?

Well, you wouldn’t need to change configuration as you moved the system between environments. That’s a good thing.

But how can that be? Wouldn’t there be a conflict if there were two machines with the same name?

The answer is that there wouldn’t be a conflict if the machines were on different networks. Not all machines have to be on the same network. We can set up as many networks / virtual networks as we like. And it is clear that we don’t need machines in one environment / network to talk to machines in another environment. I mean, under no circumstances would we want web servers in our test environment to talk to application servers in the production environment.

These separate networks provide much needed isolation, beyond solving the server naming problem.

In closing

It’s really a tiny thing when you think about – multiple networks. But that’s exactly why software developers overlook it so often – because it’s not a “software solution” to the configuration problem we perceive as a “software problem”.

I wrote about related multi-environment configuration issues in this earlier post: Convention over Configuration – The Next Generation

I’m happy to say that this functionality is now in NServiceBus called “profiles” and you can read more about how they work here.

How are you handling the flow of moving software through to production? Leave your comments below.

One of the things that I’ve been trying to do with my presentations around the world on CQRS was to explain the why behind it, just as much as the what. The problem with the format of these presentations is that they’re designed to communicate a fairly closed message: here’s the problem, here’s how that problem manifests itself, here’s a solution.

In this post, I’m going to try to go deeper.

The hitchhiker’s guide to the galaxy

In this most excellent book, one of the things that struck me was the theme that made it’s way through the whole book – starting with the answer to life, the universe, and everything: 42. By the time you get to the end of the book, you find out that the real question to life, the universe, and everything is “what do you get when you multiply 6 by 9″. And that’s how the book leaves it.

To us engineers, we can’t just accept the fact that the book would say that 6*9 = 42 when we know it’s 54. After bashing our heads on the rigid rules of math, we realize that not all math problems are necessarily in base 10, and that if we switch to base 13, the number 42 is 4*13 + 2 = 54. So, the book was right – but that’s not the point.

What’s the point?

The hitchhiker’s guide is an example of a teaching technique which presents an apparent paradox, leaving the student to dig up unspoken and unthought assumptions in order to resolve it. Key to this technique are rigid rules which do not allow any compromise or shortcuts on the student’s part.

The purpose of this technique is not for the student to learn the answer, but to gain deeper understanding, which in turn changes the way they go about thinking about problems in the future.

So, when given the problem 4*5, we do not just immediately answer 20, instead we clarify in which numeric base the question is being phrased, and only then go to solve the problem. In base 13, the answer would be 17. In hex, the answer would be 14.

The externally visible change is that we know which questions to ask in order to arrive at the right answer – not that we know the answer ahead of time.

Making an “ass” out of “u” and “me”

Let’s start at the end – one of the unspoken assumptions that has been causing problems:

All businesses can be treated the same from the perspective of software.

In our previous example, we assumed that all math problems use base 10. It turns out that different bases are useful for different domains (like base 2 for computers). We can say similar things about degrees and radians in geometry. The more we look at the real world, the more we see this repeating itself. There’s no reason that software should be any different.

Base 10 is not a ubiquitous best practice. We shouldn’t be surprised that there really aren’t best practices for software either.

Here’s another problematic assumption:

“The business” can (and do) tell us what they need in a way we can understand.

So many software fads have been built on the quicksand of this assumption. OOAD – on verbs and nouns. 4GL and other visual tools that “the business” will use directly. SOA – on IT business alignment. I expect we haven’t seen the end of this.

Some of you may be wondering why this is false, others are sagely nodding their heads in agreement.

The myth of “the business”

Unless you have a single user, who is also the CEO paying for the development, there is no “the”. It’s an amalgam of people with different backgrounds, skills, and goals – there is no homogeneity. Even if no software was involved, many business organizations are dysfunctional with conflicting goals, policies, and politics.

To some extent, we technical people have hidden ourselves away in IT to avoid the scary world of business whose rules we don’t understand. With the rise in importance of information to the world, we’ve been pulled back – being forced to talk to people, and not just computers. Luckily, we’ve been able to create a buffer to insulate ourselves – we’ve taken the less successful technical people from our heard and nominated them “business analysts”. No, not all companies do it this way, but we do need to take a minute to reflect on how information flows between the business Mars into and out of the IT Venus.

On human communication

Even if we made this insulation layer more permeable, allowing and encouraging more technical people and business people to cross its boundary, we still need to deal with the problem of two humans communicating with each other. There are enough books that have been written on this topic, so I won’t go into that beyond recommending (strongly) to technical people to read (some of) them.

Rather, I’d like to focus on the environment in which these discussions take place. IT has been around long enough, and users have used computers long enough, that a certain amount of tainting has taken place. If the world was a trial, the evidence would have been thrown out as untrustworthy.

When users tell you what they want, they’re usually framing that with respect to the current system that they’re using. “Like the old system – but faster, and with better search, and more information on that screen, and…”

At this point, business analysts write down and formalize these “requirements” into some IT-sanctioned structure (use cases, user stories, whatever), at which point developers are told to build it. Users only know what they didn’t want when developers deliver exactly what was asked.

How can that be?

These are not the “requirements” you are looking for

Users ultimately dictate solutions to us, as a delta from the previous set of solutions we’ve delivered them. That’s just human psychology – writer’s block when looking at a blank page, as compared to the ease with which we provide “constructive criticism” on somebody else’s work.

We need to get the real requirements. We need to probe beyond the veneer:

• Why do you need this additional screen?
• What real-world trigger will cause you to open it?
• Is there more than one trigger?
• How are they different?
• etc, etc, etc…

This is real work – different work than programming. It requires different skills. And that’s not even getting into the political navigation between competing organizational forces.

But let’s say that you don’t have (enough) people with these skills in your organization. What then?

Enter CQRS

CQRS gives us a set of questions to ask, and some rigid rules that our answers must conform to. If our answers don’t fit, we need to go back to the drawing board and move things around and/or go back to “the business” and seek deeper understanding there.

For each screen/task/piece of data:

Will multiple users be collaborating on data related to this task?
Look at every shred of raw data, not just at the entity level.
Are there business consistency requirements around groups of raw data?

If “the business” answers no – ask them if they see that answer changing, and if so, in what time frame, and why. What changing conditions in the business environment would cause that to change – what other parts of the system would need to be re-examined under those conditions.

After understanding all that and you find a true single-user-only-thing, then you can use standard “CRUD” techniques and technologies. There are no inherent time-propagation problems in a single-user environment – so eventual consistency is beyond pointless, it actually makes matters worse.

On the other hand, if the business-data-space is collaborative, the inherent time-propagation of information between actors means they will be making decisions on data that isn’t up-to-the-millisecond-accurate anyway. This is physics, gravity – you can’t fight it (and win).

The rule for collaboration

Actors must be able submit one-way commands that will fail only under exceptional business circumstances.

The challenge we have is how to achieve the real business objectives uncovered in our previous “requirements excavation” activities and follow this rule at the same time. This will likely involve a different user-system interaction than those implemented in the past. UI design is part of the solution domain – it shouldn’t be dictated by the business (otherwise it’s like someone asking you to run a marathon, but also dictating how you do so, like by tying your shoelaces together).

Many of the technical patterns I described in my previous blog post describe the tools involved. BTW, hackers can be considered “exceptional actors” – the business actually wants their commands to fail.

In Summary

The hard and fast rule of CQRS about one-way commands is relevant for collaborative domains only. This domain has inherent eventual consistency – in the real world. Taking that and baking it into our solution domain is how we align with the business.

The process we go through, until ultimately arriving at one-way-almost-always-successful-commands is business analysis. Rejecting pre-formulated solutions, truly understanding the business drivers, and then representing those as directly as possible in our solution domain – that’s our job.

After doing this enough times and/or in more than one business domain, we may gain the insight that there is no cookie-cutter, one-size-fits-all, best-practice solution architecture for everything. Each problem domain is distinct and different – and we need to understand the details, because they should shape the resulting software structure.

The next time the business tell us to implement 42, we’ll use CQRS along with other questioning techniques until we can get “6 x 9″ out of them, learning from the exercise what are the significant and stable parts of the business – ultimately helping us to “build the right system, and to build the system right”.

Don’t Panic

Java takes the enterprise

When Java originally came out, it was an interesting language that you could use to write applets – code that would run the same everywhere, in the browser, on the desktop, etc. SUN was the keeper of Java. And then came this concept of a container – the thing that would run your Java code, which then grew to handle things like transactions, and became the Enterprise Java Bean – EJB, and that came out of IBM, with SUN adopting it later.

The adoption of Java at that point was important enough that the specs were opened up, and many EJB technologies blossomed. With backing from big companies already inside the enterprise, the only possible fight came from COBOL around Y2K, but that was a dying gasp. Microsoft wasn’t in the game as Windows NT wasn’t competition for UNIX or mainframes.

Multi-vendor as a way of life

With multiple big and medium-sized vendors offering similar, competing, and complementary technologies, all tied together by the promise of backwards compatibility in Java and the specs demanding interoperability, customers could safely go for best-of-breed solutions. This forced technological decision makers to truly evaluate the offerings on merits, not just lineage.

Many attribute the rise of OSS in Java to the fact that the existing containers were so heavyweight. I believe that was a secondary effect. As a result of the fact that the industry had embraced and internalized the values of thinking and choosing for itself, it was willing to look at alternatives with much humbler lineage, ultimately using them on their merits. It was the culture.

The Microsoft ecosystem

This culture was practically nonexistent on the Microsoft side of the border. As the only vendor, Microsoft was put on a pedestal – it was the best, period. The industry hungrily looked to Seattle not only for technology, but also for guidance and leadership. If a developer could get a job at Microsoft, they were “hot stuff”, the best of the best. This isn’t a bad thing – it was just a thing.

This enabled technological decision makers on the Microsoft side to have much shorter thought and decision processes than their counterparts on the Java side.

All of these things got baked into the culture.

About Microsoft

Like all that were ever on a pedestal, the fall was a matter of time. Expectations being that high, it was inevitable. You can’t make all people happy all the time, and the conditions in the industry were changing, and the company had to change to remain competitive.

Let me say this clearly: Microsoft was not at fault.

Sure, it’s easy to say in retrospect they should have communicated more clearly about this, or built that technology differently. If you haven’t yet worked in a big company, you may not know this, but big companies aren’t just bigger small companies. It’s a hodge-podge of competing agenda, initiatives, politics, people, and power. There’s a saying that things only get done despite the organization’s best efforts.

For a company Microsoft’s size, what they manage to get done is incredible.

On acquisitions and OSS

Microsoft has come under fire over the years for offering their own implementations of open source technologies- as if the vendors on the Java side didn’t do this. The Java world was ultra competitive, the big vendors would eat promising upstarts in order to win back lost contracts to key customers. This made the technological decision makers broaden their thought processes to include risk management as a part of managing their technological portfolio. To a large extent, this actually justifies the existence of a C-level role related to technology – the CIO.

Chief Officers of Information and Technology

I found it interesting to see the difference in age, experience, background, and thought processes between people holding the CIO title at organizations that were Microsoft-centric and those with a more heterogeneous technology investment. This was likely influenced to a large extent by the history of technological evolution, age and size of organizations with the resulting culture and hiring practices, among other things. This pattern continued with the CTO as well.

Obviously one wouldn’t expect the same thought processes in the CTO of a 20 person IT shop and the CTO of Ford Motor Company (for example). They shouldn’t be the same.

It appeared that as Microsoft became more focused on innovation they started listening more to the technology leaders of smaller companies, not a bad thing by itself. Choosing A means not choosing B, and in order to stay competitive, a choice must be made.

Fast forward

I think that what happened was necessary, and will be good for the industry and Microsoft. Technological decision making in companies that were traditionally Microsoft-centric has evolved. This has clarified Microsoft’s role as a platform vendor who can be trusted, and whose tools can be used or not used as the situation dictates, with comparable commercial and OSS tooling evaluated on the same criteria.

Just as IBM reinvented itself and now occupies a sustainable role in a combined commercial and OSS ecosystem of platforms, tools, and services, Microsoft appears to have made several big strides partnering with the community in much more productive ways, yet with more strides to be made as well.

A challenge to OSS on the Microsoft platform

In this new and more mature environment, OSS can’t remain the same either. Some code a developer whipped up in their free time and put in an online accessible repository with a decent license just won’t cut it any more.

In my previous post I called out the Linq2Sql support story – the same goes for OSS. Active development is required, and so is support, and so is documentation. The commitment needs to be much more serious.

Also, until usage reaches some critical mass, it is unlikely that a single developer or even a small group of committers will be able to do it without the help of the community. Really the only alternative is for there to be some commercial story that can fund it – support, consulting, training, commercial add-ons, etc. A combination of community (both dev and use) and commercial cash-flow is probably most sustainable.

If you are running an OSS project, understand that these criteria will be used to evaluate it.

In closing

I think I’ve managed to alienate previous supporters from all sides.

I believe we are entering some interesting times, where not only are vendors and OSS projects being evaluated differently than in the past, but that traditional architectural paradigms are changing as well.

Regardless of what the answers are, I’m happy that more of us are asking more questions. Some questions are the right questions, some are the wrong ones, and sometimes we just ask at the wrong time, but as an industry I think that we’re getting better.

Thanks for reading.

Although I have been privy over the years to see behind the veil at Redmond, I will be focusing primarily on the externally visible actions of Microsoft and how the industry has reacted on average – specifically in the enterprise space, where I spend most of my time.

What managers are concerned about

When there will be problem with this technology, who will support us?
How will that change when the author of the technology moves on or loses interest?
How long can we expect to be using this technology?
How long will it take to learn it? When will we recoup our investment?

Traditionally, for companies on the Microsoft platform, choosing Microsoft as their primary technology vendor has been the safe bet. As a large company, Microsoft can afford to employ support engineers. These engineers are different from the ones who wrote the technology to begin with. Also, Microsoft has 10 year support guarantees. As a result, companies expected to use the technology for at least that long. With their size, Microsoft also had the ability to create copious amounts of documentation easing learning.

To turn the common phrase, nobody got fired for choosing Microsoft.

Open source seemed like risky business, at the time.

What changed?

It started with the Composite Application Block – CAB.

This technology was put out by the Patterns and Practices (P&P) group at Microsoft. After it came out, the gears of big marketing machines at Microsoft got to work, telling the industry about this great new thing at conferences, and the Microsoft Consulting Services (MCS) folks started using it with clients. CIOs at large companies sent developers to training on CAB by the dozen.

And then CAB was dead. Sorry. Microsoft prefers the word “done” to dead.
Just like Don Box said that COM wasn’t dead, it was done.

“What about that 10 year support thing?”, asked the CIOs (among many others).

You don’t understand, said Microsoft. You see, P&P are not a product group in Microsoft. They don’t have the resources to provide that kind of support. And the rest of the company isn’t obliged to support what they put out – since they’re not a product group.

You could literally hear the collective jaw of the Microsoft part of the industry drop.

This wasn’t supposed to happen (like housing prices in the US).

And then came .NET 3.0, and things seemed to go back to normal.

There were lots of wonderful things that came with it – like LINQ, and…

Linq to Sql

This was BIG.
Finally – after ObjectSpaces was promised at PDC ‘03 and later shelved, then WinFS (same story), it had arrived.
The object-relational mapper (ORM) from Microsoft.

The marketing machine went into high gear – it was the v3 promise. Linq2Sql (L2S) came from a product group. This was serious. It was shown at conferences and user groups all over the world. Developers were sent to be trained on it.

And then Entity Framework (EF) was announced – and L2S was done.

CIOs were rubbing their eyes in disbelief. “But this is from a product group – you have to support it, right?”

Well, said Microsoft, yes. We will support it with our support org. It’s just that we won’t continue to develop new features for it in the product org. We actually have a different sub-org that will be working on EF and they’re under a different division (SQL Server) than the sub-org that made L2S.

Jaws were hitting the pavement. All that investment erased. Just like that. It turns out that support without active development doesn’t mean very much, no matter how many support engineers are involved.

But the industry picked itself back up, and got back to work on the new foundational pieces that came with .NET 3.0. If Microsoft is calling it “Foundation”, that must mean they’re committed to it.

And then came Workflow Foundation

Workflow Foundation (WF) was a dream-come-true. At last, Model-Driven Development (MDD) had come to the Microsoft platform. Drag-and-drop on a whole new level. Imagine the reuse. Imagine the maintainability. Programming at higher levels of abstraction. A marketers dream. This was the culmination of previous efforts of Whitehorse in VS2005 and the Software Factories Initiative – or so we were told.

And then came .NET 3.5 – and the new WF wasn’t backwards compatible with the old one.
And then it happened again with .NET 3.5 SP1, and again with 4.0 (no more state machine – no wait, it’s back again, but not in the box).

All those companies that had long-running workflows in production needed to manually migrate them each time.

But this had become old news to the folks using Microsoft in the enterprise.
With Microsoft – you really can’t be sure any more.

What about open source?

Well, it was beginning to look more and more stable in comparison.
Especially the larger, more established projects. Those with active development.
Log4Net. NHibernate. Castle. etc.

There was also the fact that most enterprises were heterogeneous anyway – doing both Java and .NET development. Open source tools and frameworks were common in the Java space, politically greasing the wheels for .NET OSS in those organization.

Boasting features and capabilities several years ahead of what was coming out of Microsoft, more companies gave them a chance, and were pleasantly surprised. And the virtuous cycle of OSS gained speed. With more use, they become even more stable and got even more features, driving yet more use. Blog posts about them bloomed all over the web. User group presentations were given. At a presentation I gave at TechEd Europe 2006, I used NHibernate in my demo.

OSS had crossed the chasm. No, not everybody used it, or knew of it, but a critical mass of the industry had grown to depend on it.

Microsoft’s actions over the years had done more for OSS adoption than I think many would have imagined.

On Pub/Sub, Messaging, and SOA

Interestingly enough, when the first version of WCF was still in the oven (then called Indigo) there were discussions on whether it would support publish/subscribe messaging. Here we are, 3 versions later, and still no pub/sub, but the discussions continue

Message brokers were always important for enterprises in the Java space – IBM MQ; Tibco RV; Sonic; companies were paying millions for this stuff. Microsoft had MSMQ – finally at v3 with XP and Server 2003, but still with insignificant penetration to the market. BizTalk did have a good run, though, but not so much as a message broker, more as an integration and orchestration engine, unfortunately coming late to the Enterprise Application Integration (EAI) party.

Later, the SQL Server guys came with Service Broker – messaging in the database. But you could see that their heart wasn’t in it. The API was clunky. It still didn’t have pub/sub. There was no binding available for WCF.

After some time making noise in the SOA space with Oslo, that changed as well. Oslo is now Sql Server Modeling.

I imagine that some of the adoption pick-up with NServiceBus can be attributed to the vacuum Microsoft left behind when exiting the messaging/pub-sub/soa space. The way NServiceBus aligns with the principles found in the corresponding Java technologies makes it very palatable to enterprises working with both platforms.

The fact that there’s active development, a vibrant and growing community, and even training available definitely contribute as well.

In closing

I don’t fault Microsoft for any of this. There are a million things that they could have done. Choosing to do one thing means choosing not to do many others. The decisions they made were done with the best intentions. Hindsight is 20/20 of-course.

And that’s just it – we do need to take a look back.

If you’re a manager making a technology related decision, or are working with managers in those positions, knowing the history of today’s technology can give you a more accurate representation of the risk involved in each choice. Also, understanding the vector that Microsoft has decided to take in various areas is critical, especially if you find out that your architectural choices aren’t quite aligned with some of those vectors.

In this post, I’ve tried not to take a stance on whether a certain approach (ORM, Pub/Sub, etc) is good or bad, or even getting into which cases it’s appropriate or not – just to describe Microsoft’s externally visible behavior in that space.

I hope that this short history lesson can help your organization make the right technology decisions in the future for its specific context. Your comments and thoughts are most welcome, as always.

Before getting into that, I want to start with a slightly broader scope of discussion.

You see, I get asked about “best practices” on all sorts of things. And I try not to be the kind of consultant that responds with “it depends”, but the context of the question often makes the answer irrelevant. And the unspoken context of a best-practice question is:

Given infinite time and budget

The biggest problem that I see with well-intentioned, best-practices-following developers and architects is that they don’t ask the question “is this the right thing for us to be focusing on right now?” Understandably, that is a difficult question to answer – but it needs to be asked, since you don’t have infinite time or budget to do everything according to best practices (assuming those even exist).

About testing

The biggest issue I have with the “design for testability” topic is the extremely narrow view it takes of the word “testability”, usually in the form of more code written by a developer which invokes the production code of the system, also known as “unit tests”.

There are many different kinds of testing – unit, integration, functional, load, performance, exploratory, etc… where some may be automated and others not. Should we not discuss what “design for testability” means for not-just-unit-testing?

And what’s the point of testing anyway?

It’s not to find bugs.

Research has shown that testing (of all kinds) is not the most effective way of finding bugs. I don’t have the reference handy but I’m pretty sure that it’s from Alistair Cockburn’s work. Code reviews are (on average) about 60% more effective.

Don’t get me wrong – testing can provide indications that the software has bugs in it, but not necessarily where in the code those bugs are.

The purpose of testing is to provide quantitative and qualitative information about the system that can help various stakeholders in their decision-making processes. The relevance of that information indicates the quality of the testing. Here are some examples:

• The system supports 100 concurrent users, with the expected user-type distribution (X% role A, Y% role B, etc), performing expected use-case distributions, and collaboration scenarios.
• Time to proficiency for new users in role A is expected to be 3 days
• Alternate #2 of use case #12 fails on step #3

As you can see, the relevance of the above information is dependent on what decisions the various stakeholders need to make. The bullet on load can help us decide if more machines are needed or if developers need to tune the performance of the systems. The bullet on time to proficiency can help us decide if larger investment in usability is required. Information like the last bullet can be used in conjunction with the first two to decide on the timing and type of a release.

The timeliness of this relevant information is critical to the success of a project.

Choosing which and how much of the various testing activities to perform when is something that needs to be revisited several times throughout the lifetime of a project, taking into account the current risks (threats and probabilities) and time and resource investment to mitigate them.

Let me reiterate – we’re not going to have enough time to do everything.

On iterations

If the only part of your organization that is doing iterations are your developers, you’re not agile.

In order to capitalize on the information that testers are providing, you need them in your iterations.

The same goes for the other roles involved in the project – business analysts, DBAs, sysadmins, etc.

I know that 99% of organizations aren’t structured in a way to do this.

I never said doing this would be easy.

On design

Figuring out what kind of design and how much to do when is just as important, and just as hard. Design for testability is one part of that, but not the only one, or necessarily the most important one at any point of time.

Within that design for testability topic is the “design for unit-testing” sub-topic which seems to be the popular one. Before getting into the design aspects of it, let’s take a closer look at the unit-testing side of things.

On unit-testing

The assumption is that having more unit tests will lead to a code-base with less bugs, thus requiring shorter time to get the system into production, which will pay back the time it took to write those unit tests to begin with.

In practice, what tends to happen is that as development progresses, testing code breaks as the structure of the production code changes. Now one of two things happens – either the testing code is removed or rewritten. In either case, we didn’t get the return on investment we expected on the first bit of testing code. Unfortunately, rare is the case where the relevant people in the organization understand why, resulting in the same situation repeating itself over and over again.

Those projects would have been better off without unit testing, though the organization as a whole might have used those experiences to learn and improve. It’s been my experience that if the organization wasn’t conscious enough in the context of the project to notice the situation, it is unlikely to do so at higher levels.

On fragile unit tests

The reason that a unit test ends up being rewritten (or removed) is that its code was coupled to the production code in such a way that it broke when the production code changed. This tendency to break (fragility) is a critical property of a unit test. A fragile unit test will slow down a developer doing work on some existing code – it actually makes the system less maintainable.

For a unit test code to be stable (not fragile) it needs to be coupled to stable properties of the production code. The question of whether the production code is designed in such a way that it has stable properties – is a design question. Is it a unit? If not, you will not be able to write a unit-test against it.

And anyway, who said that every class is a unit, or should be a unit? Domain models (when done right) are good examples of a unit, yet the classes that make them up may not be units. Unit-testing should only be attempted with things which are units.

I think too much weight is put on whether a dependency of a class is a concrete or interface type, and not nearly enough on the nature of the dependency. I wouldn’t blame the hammer for pounding my thumb, and by the same token I think that blame should not be directed towards tools like those from TypeMock.

On tools

There is so much more depth to both design and testability that needs to be more broadly understood. No tool has yet been created to handle either design or testing in such a way that humans can give up responsibility for the outcome.

Over the years I’ve noticed that tools are most significant when used by skilled practitioners, which makes sense in retrospect. Giving a novice carpenter a laser-guided saw probably won’t significantly change the outcome of their work. Ultimately, the skilled practitioners are the ones that create tools – not the novices. And no tool, no matter how advanced, will make a novice perform at levels like the skilled practitioner.

In the case of a project too big for a single skilled practitioner to complete in the time required (or at all), the balance of importance shifts away from tools to the project management topics described above.

In summary

I hope that this post has shed some light on the context in which decisions with respect to testing need to be made. Design is one activity that can support certain kinds of testing, but not the only one, or even the most important one for the given type of testing necessary at that time in the project.

Design is hard. Project management is hard. Testing is hard.

Getting the right mix of people that together have enough experience and skills in these activities isn’t easy.

Don’t expect that sprinkling some interfaces in your code base will be enough.
That doesn’t count much in the way of design, just as writing code in a testing namespace doesn’t count much in the way of testability.

Looking forward to hearing your comments.

What makes an app small?

Or inversely, what makes an app warrant the “enterprise” moniker?

If there’s one thing that the history of our industry has shown repeatedly, it’s that developers aren’t particularly accurate with their estimates. Like, orders-of-magnitude inaccurate. Knowing this, it’s surprising that the “small app” argument seems to win so many arguments. The same goes for justifications in the form of “we’ve got to have an X, this is a BIG project”.

So, what makes an app small?

Is it a small number of lines of code? Well, what if those lines of code are keeping planes in the air?

Is it a small number of developers? Same as above. Actually, history has shown that some of the most valuable bits of code written were done by small numbers of developers.

Is it that it will only be installed on a single machine?

Is it…

What could it be?

The real issue

The small app argument is a diversionary tactic.

Loosely translated, it means “I’m comfortable where I am and I don’t want to change”.

Moving on…

The real story of size

Once we actually look at the specific context of an app, we tend to see that someone cares a great deal about it, enough to finance its custom development – rather than buying an off-the-shelf alternative. The expected lifetime of business use is easily 3-5 years, if not 7-10, during which many enhancements will likely be requested. Thus, some non-functional properties of the code matter – at the very least maintainability.

In which case, if the given pattern or approach does significantly improve the desired non-functional properties of the app, it only makes sense to use it.

There is one class of software that might possibly be treated as “small” – the one-off script that’s written to automate some IT task. And even then, so many of these scripts end up living longer than the apps themselves that they should be engineered at the same level of quality.

In closing

Don’t counter a “small app” argument with psychology.
It will only make matters worse.

Instead, rephrase the issue around the lifetime of business use.

I’ve found that there are precious few cases where the harsh light of reality doesn’t help the appropriate decisions be made. If indeed this is a small-lifetime-app, just drag-and-drop until you’re done. Otherwise, the time it takes to understand and evaluate the applicability of the given patterns will definitely pay itself back many times over the life of the app.

And managers, keep your ears open for it. The technical risks behind that statement are icebergs waiting to sink your project.

* with thanks to Mike Nichols for pushing my buttons.

The real world doesn’t cascade

Let’s say our marketing department decides to delete an item from the catalog. Should all previous orders containing that item just disappear? And cascading farther, should all invoices for those orders be deleted as well? Going on, would we have to redo the company’s profit and loss statements?

Heaven forbid.

So, is Ayende wrong? Do we really need soft deletes after all?

On the one hand, we don’t want to leave our database in an inconsistent state with invoices pointing to non-existent orders, but on the other hand, our users did ask us to delete an entity.

Or did they?

When all you have is a hammer…

We’ve been exposing users to entity-based interfaces with “create, read, update, delete” semantics in them for so long that they have started presenting us requirements using that same language, even though it’s an extremely poor fit.

Instead of accepting “delete” as a normal user action, let’s go into why users “delete” stuff, and what they actually intend to do.

The guys in marketing can’t actually make all physical instances of a product disappear – nor would they want to. In talking with these users, we might discover that their intent is quite different:

“What I mean by ‘delete’ is that the product should be discontinued. We don’t want to sell this line of product anymore. We want to get rid of the inventory we have, but not order any more from our supplier. The product shouldn’t appear any more when customers do a product search or category listing, but the guys in the warehouse will still need to manage these items in the interim. It’s much shorter to just say ‘delete’ though.”

There seem to be quite a few interesting business rules and processes there, but nothing that looks like it could be solved by a single database column.

Model the task, not the data

Looking back at the story our friend from marketing told us, his intent is to discontinue the product – not to delete it in any technical sense of the word. As such, we probably should provide a more explicit representation of this task in the user interface than just selecting a row in some grid and clicking the ‘delete’ button (and “Are you sure?” isn’t it).

As we broaden our perspective to more parts of the system, we see this same pattern repeating:

Orders aren’t deleted – they’re cancelled. There may also be fees incurred if the order is canceled too late.

Employees aren’t deleted – they’re fired (or possibly retired). A compensation package often needs to be handled.

Jobs aren’t deleted – they’re filled (or their requisition is revoked).

In all cases, the thing we should focus on is the task the user wishes to perform, rather than on the technical action to be performed on one entity or another. In almost all cases, more than one entity needs to be considered.

Statuses

In all the examples above, what we see is a replacement of the technical action ‘delete’ with a relevant business action. At the entity level, instead of having a (hidden) technical WasDeleted status, we see an explicit business status that users need to be aware of.

The manager of the warehouse needs to know that a product is discontinued so that they don’t order any more stock from the supplier. In today’s world of retail with Vendor Managed Inventory, this often happens together with a modification to an agreement with the vendor, or possibly a cancellation of that agreement.

This isn’t just a case of transactional or reporting boundaries – users in different contexts need to see different things at different times as the status changes to reflect the entity’s place in the business lifecycle. Customers shouldn’t see discontinued products at all. Warehouse workers should, that is, until the corresponding Stock Keeping Unit (SKU) has been revoked (another status) after we’ve sold all the inventory we wanted (and maybe returned the rest back to the supplier).

Rules and Validation

When looking at the world through over-simplified-delete-glasses, we may consider the logic dictating when we can delete to be quite simple: do some role-based-security checks, check that the entity exists, delete. Piece of cake.

The real world is a bigger, more complicated cake.

Let’s consider deleting an order, or rather, canceling it. On top of the regular security checks, we’ve got some rules to consider:

If the order has already been delivered, check if the customer isn’t happy with what they got, and go about returning the order.

If the order contained products “made to order”, charge the customer for a portion (or all) of the order (based on other rules).

And more…

Deciding what the next status should be may very well depend on the current business status of the entity. Deciding if that change of state is allowed is context and time specific – at one point in time the task may have been allowed, but later not. The logic here is not necessarily entirely related to the entity being “deleted” – there may be other entities which need to be checked, and whose status may also need to be changed as well.

Summary

I know that some of you are thinking, “my system isn’t that complex – we can just delete and be done with it”.

My question to you would be, have you asked your users why they’re deleting things? Have you asked them about additional statuses and rules dictating how entities move as groups between them? You don’t want the success of your project to be undermined by that kind of unfounded assumption, do you?

The reason we’re given budgets to build business applications is because of the richness in business rules and statuses that ultimately provide value to users and a competitive advantage to the business. If that value wasn’t there, wouldn’t we be serving our users better by just giving them Microsoft Access?

In closing, given that you’re not giving your users MS Access, don’t think about deleting entities. Look for the reason why. Understand the different statuses that entities move between. Ask which users need to care about which status. I know it doesn’t show up as nicely on your resume as “3 years WXF”, but “saved the company $4 million in wasted inventory” does speak volumes.

One last sentence: Don’t delete. Just don’t.

The thing is, when looking at this in light of the full software development lifecycle, there are signs that the waters run deeper than we might have originally thought.

Let’s take things one step at a time though…

What is it?

Wikipedia tells us:

“Convention over Configuration (aka Coding by convention) is a software design paradigm which seeks to decrease the number of decisions that developers need to make, gaining simplicity, but not necessarily losing flexibility. The phrase essentially means a developer only needs to specify unconventional aspects of the application.”

What this means is that frameworks built in this way have default implementations that can be swapped out if needed. So far so good.

For example…

In NServiceBus, there is an abstraction for how subscription data is stored and multiple implementations – one in-memory, another using a durable MSMQ queue, and a third which uses a database. The convention for that part of the system is that the MSMQ implementation will be used, unless something else is specified.

Developers wishing to specify a different implementation can specify the desired implementation in the container – either one that comes out of the box, or their own implementation of ISubscriptionStorage.

Things get more interesting when we consider the full lifecycle.

Lifecycle effects

When developers are in the early phases of writing a new service, they want to focus primarily on what the service does – its logic. They don’t want to muck around with MSMQ queues for storing subscriptions and would much rather use the in-memory storage.

As the service takes shape and the developers want to run the full service on their machine, possibly testing basic fault-tolerance behaviors – kill one service, see that the others get a timeout, bring the service back up, wanting it to maintain all the previous subscriptions.

Moving on from there, our developers want to take the same system they just tested on their machine and move it into a staging environment. There, they don’t want to use the MSMQ implementation for subscription storage, but rather the database implementation – as will be used in the production environment.

While it may not sound like a big deal – changing the code which specifies which implementation to use when moving from one environment to another, consider that on top of just subscription storage, there is logging (output to console, file, db?), saga persistence (in-memory, file-based DB, relational DB), and more.

It’s actually quite likely that something will get missed as we move the system between environments. Can there be a better way?

What if…

What if there was some way for the developer to express their intent to the system, and the system could change its conventions, without the developer having to change any code or configuration files?

You might compare this (in concept) to debug builds and release builds. Same code, same config, but the runtime behaves different between the two.

As I mulled over how we could capture that intent without any code or config changes, the solution that I kept coming to seemed too trivial at first, so I dismissed it. Yet, it was the simplest one that would work for console and WinForms applications, as well as windows services – command line arguments. The only thing is that I don’t think those are available for web applications.

But since we’re still in “what if” land, and I’m more thinking out loud here than providing workable solutions for tomorrow morning, let’s “what if” command line arguments worked for web apps too.

Command-Line Intent

Going back to our original scenario, when developers are working on the logic of the service, they run it using the generic NServiceBus host process, passing it the command line parameter /lite (or whatever). The host then automatically configures all the in-memory implementations.

As the system progresses, when the developer wants to run everything on their machine, they run the processes with /integration. The host then configures the appropriate implementations (MSMQ for subscription storage, SQLite for saga persistence, etc.

When the developers want to run the system in production, they could specify /production (or maybe that could be the default?), and the database backed implementations would be configured.

Imagine…

Imagine being able to move that fluidly from one environment to another. Not needing to pore over configuration files or startup script code which configures a zillion implementation details. Not needing to worry that as you moved the system to staging something would break.

Imagine short, frictionless iterations even for large scale systems.

Imagine – lifecycle-aware frameworks making all this imagination a reality.

In Closing

We’re not there yet – but we’re not that far either. The generic host we’re providing with NServiceBus 2.0 is now being extended to support exactly these scenarios.

It’s my hope that as more of us think about this challenge, we’ll come up with better solutions and more intelligent frameworks. Just as convention came to our rescue before, breaking us out of the pain of endless XML configuration, I hope this new family of lifecycle-aware frameworks will make the friction of moving a system through dev, test, staging, and production a thing of the past.

A worthy problem for us all to solve, don’t you think?

Any ideas on how to make it a reality?
Send them in – leave a comment below.

Background

Before diving right into the code, I wanted to take a minute to recall how we got here.

It started by looking for how to create fully encapsulated domain models.

The main assertion being that you do *not* need to inject anything into your domain entities.

Not services. Not repositories. Nothing.

Just pure domain model goodness.

Make Roles Explicit

I’m going to take the advice I so often give. A domain event is a role, and thus should be represented explicitly:

   1:  public interface IDomainEvent {}

If this reminds you of the IMessage marker interface in nServiceBus, you’re beginning to see where this is going…

How to define domain events

A domain event is just a simple POCO that represents an interesting occurence in the domain. For example:

   1:  public class CustomerBecamePreferred : IDomainEvent 

   2:  {

   3:      public Customer Customer { get; set; }

   4:  }

For those of you concerned about the number of events you may have, and therefore are thinking about bunching up these events by namespaces or things like that, slow down. The number of domain events and their cohesion is directly related to that of the domain model.

If you feel the need to split your domain events up, there’s a good chance that you should be looking at splitting your domain model too. This is the bottom-up way of identifying bounded contexts.

How to raise domain events

In your domain entities, when a significant state change happens you’ll want to raise your domain events like this:

   1:  public class Customer

   2:  {

   3:      public void DoSomething()

   4:      {

   5:          DomainEvents.Raise(new CustomerBecamePreferred() { Customer = this });

   6:      }

   7:  }

We’ll look at the DomainEvents class in just a second, but I’m guessing that some of you are wondering “how did that entity get a reference to that?” The answer is that DomainEvents is a static class. “OMG, static?! But doesn’t that hurt testability?!” No, it doesn’t. Here, look:

Unit testing with domain events

One of the things we’d like to check when unit testing our domain entities is that the appropriate events are raised along with the corresponding state changes. Here’s an example:

   1:  public void DoSomethingShouldMakeCustomerPreferred()

   2:  {

   3:      var c = new Customer();

   4:      Customer preferred = null;

   5:   

   6:      DomainEvents.Register<CustomerBecamePreferred>(

   7:          p => preferred = p.Customer

   8:              );

   9:   

  10:      c.DoSomething();

  11:      Assert(preferred == c && c.IsPreferred);

  12:  }

As you can see, the static DomainEvents class is used in unit tests as well. Also notice that you don’t need to mock anything – pure testable bliss.

Who handles domain events

First of all, consider that when some service layer object calls the DoSomething method of the Customer class, it doesn’t necessarily know which, if any, domain events will be raised. All it wants to do is its regular schtick:

   1:  public void Handle(DoSomethingMessage msg)

   2:  {

   3:      using (ISession session = SessionFactory.OpenSession())

   4:      using (ITransaction tx = session.BeginTransaction())

   5:      {

   6:          var c = session.Get<Customer>(msg.CustomerId);

   7:          c.DoSomething();

   8:   

   9:          tx.Commit();

  10:      }

  11:  }

The above code complies with the Single Responsibility Principle, so the business requirement which states that when a customer becomes preferred, they should be sent an email belongs somewhere else.

Notice that the key word in the requirement – “when”.

Any time you see that word in relation to your domain, consider modeling it as a domain event.

So, here’s the handling code:

   1:  public class CustomerBecamePreferredHandler : Handles<CustomerBecamePreferred>

   2:  { 

   3:     public void Handle(CustomerBecamePreferred args)

   4:     {

   5:        // send email to args.Customer

   6:     }

   7:  } 

This code will run no matter which service layer object we came in through.

Here’s the interface it implements:

   1:  public interface Handles<T> where T : IDomainEvent

   2:  {

   3:      void Handle(T args); 

   4:  } 

Fairly simple.

Please be aware that the above code will be run on the same thread within the same transaction as the regular domain work so you should avoid performing any blocking activities, like using SMTP or web services. Instead, prefer using one-way messaging to communicate to something else which does those blocking activities.

Also, you can have multiple classes handling the same domain event. If you need to send email *and* call the CRM system *and* do something else, etc, you don’t need to change any code – just write a new handler. This keeps your system quite a bit more stable than if you had to mess with the original handler or, heaven forbid, service layer code.

Where domain event handlers go

These handler classes do not belong in the domain model.

Nor do they belong in the service layer.

Well, that’s not entirely accurate – you see, there’s no *the* service layer. There is the part that accepts messages from clients and calls methods on the domain model. And there is another, independent part that handles events from the domain. Both of these will probably make use of a message bus, but that implementation detail shouldn’t deter you from keeping each in their own package.

The infrastructure

I know you’ve been patient, reading through all my architectural blah-blah, so here it is:

   1:  public static class DomainEvents

   2:  { 

   3:      [ThreadStatic] //so that each thread has its own callbacks

   4:      private static List<Delegate> actions;

   5:   

   6:      public static IContainer Container { get; set; } //as before

   7:   

   8:      //Registers a callback for the given domain event

   9:      public static void Register<T>(Action<T> callback) where T : IDomainEvent

  10:      {

  11:         if (actions == null)

  12:            actions = new List<Delegate>();

  13:   

  14:         actions.Add(callback);

  15:     }

  16:   

  17:     //Clears callbacks passed to Register on the current thread

  18:     public static void ClearCallbacks ()

  19:     {

  20:         actions = null;

  21:     }

  22:   

  23:     //Raises the given domain event

  24:     public static void Raise<T>(T args) where T : IDomainEvent

  25:     {

  26:        if (Container != null)

  27:           foreach(var handler in Container.ResolveAll<Handles<T>>())

  28:              handler.Handle(args);

  29:   

  30:        if (actions != null)

  31:            foreach (var action in actions)

  32:                if (action is Action<T>)

  33:                    ((Action<T>)action)(args);

  34:     }

  35:  } 

Notice that while this class *can* use a container, the container isn’t needed for unit tests which use the Register method.

When used server side, please make sure that you add a call to ClearCallbacks in your infrastructure’s end of message processing section. In nServiceBus this is done with a message module like the one below:

   1:  public class DomainEventsCleaner : IMessageModule

   2:  { 

   3:      public void HandleBeginMessage() { }

   4:   

   5:      public void HandleEndMessage()

   6:      {

   7:          DomainEvents.ClearCallbacks();

   8:      }

   9:  }

The main reason for this cleanup is that someone just might want to use the Register API in their original service layer code rather than writing a separate domain event handler.

Summary

Like all good things in life, 3rd time’s the charm.

It took a couple of iterations, and the API did change quite a bit, but the overarching theme has remained the same – keep the domain model focused on domain concerns. While some might say that there’s only a slight technical difference between calling a service (IEmailService) and using an event to dispatch it elsewhere, I beg to differ.

These domain events are a part of the ubiquitous language and should be represented explicitly.

CustomerBecamePreferred is nothing at all like IEmailService.

In working with your domain experts or just going through a requirements document, pay less attention to the nouns and verbs that Object-Oriented Analysis & Design call attention to, and keep an eye out for the word “when”. It’s a critically important word that enables us to model important occurrences and state changes.

What do you think? Are you already using this approach? Have you already tried it and found it broken in some way? Do you have any suggestions on how to improve it?

Let me know – leave a comment below.

First of all, the conventional wisdom:

“If you don’t choose assembly to represent component, the natural artifact candidate is then namespace.”

There’s only one minor assumption here that deserves being dragged out into the light.

While Visual Studio creates an assembly from every project by default, you can take those assemblies and merge them together into a single assembly using this nice little utility from Microsoft. It is likely that each project would have its own namespace too, so we should still be aligned with the conventional wisdom.

In other words, we could choose a Visual Studio project to represent a logical component and still be in the same camp as Jeremy:

“I’m very firmly in the camp that says you should only split assemblies by deployment targets”

What everyone agrees about seems to be that coupling hurts, and should be managed.

Where does coupling come from? Well, from references between two pieces of code. If we were to represent our logical components as Visual Studio projects, we could easily see those references without the help of any 3rd party tools. The compiler would even yell at us if we were to (accidentally) create an evil circular reference.

While some might complain about the long compile time when we have many projects in a single solution, good componentization often doesn’t require us to put all projects in a single solution. In fact, each component could theoretically have its own solution – since it’s reasonable to assume we’d really only be working on one component at a time. In which case, compile time per developer task would be a non-issue.

Going through the whole code base is usually only needed when doing a full-system debug when trying to track down a problem. This wouldn’t need to be done against a solution with all projects. We’d do this using PDBs of the merged projects (as that’s what actually got delivered, and where the bug was found). After spelunking through those PDBs, we’d eventually find the problematic component (or 2, or 3, or …), and open up developer tasks for each component.

Regardless of if we’re putting out a patch for an existing customer or rolling these changes into a release with other tasks, all the logical components would be built into a physical system (merged as necessary) and the system would be put through QA.

In short, it looks like just a bit of unconventional wisdom gets us a nice balance.

Thus begins the story of yet another developer trying to convince their manager to adopt test-driven development (or any other methodology or technology). There’s a good chance this developer’s been reading all sorts of stuff on blogs (like those linked here) that have convinced him that salvation lies that way.

Don’t get me wrong.

There’s a good chance the developer’s right.

It’s just that that’s besides the point.

Developers and Managers

There’s a difference between how developers view a practice and how a manager (defined for the purposes of this post as someone in charge of delivering something) view that same practice. From a developer perspective, Ian’s point about unit testing is spot on:

“The problem is that the most important step is not doing it right, but doing it at all.”

Yet, as Ian himself points out in the title, this is a learning issue. If you want to learn to swim, there’s no replacement for jumping in the pool.

The manager’s perspective is a bit different.

Yes, we want our developers to improve their skill set. Yes, we understand that unit testing will ultimately improve quality. Yes, we know developers need to practice these skills as a part of their job. But, and it’s a big ole’ but, when it comes time to sink or swim, and we’ve got a deadline, those desires need to be balanced with delivering. Accounts of unit testing adoption efforts resulting in more (test) code to support with little apparent improvement in quality in the short term, well, they scare us. Arnon’s post gives more links supporting that feeling.

What’s a Unit Test anyway?

Is it any class that happens to have a TestFixture attribute on it?

If we are to “decouple” unit testing from good design, as Roy has described, that’s a not-improbable outcome. If the design of the system is such as there aren’t any real “units”, what exactly are we testing? Regardless of static or dynamic typing, replaceability of code, and other technological things, does the fact that all TestMethods in that TestFixture complete successfully mean anything? In other words, what did the test test?

It is clear that these tests cost something.

It’s more code to write. It’s more code to maintain.

The question is, what value are we getting from these “unit tests that any developer without design skills can write”?

The manager in me doesn’t like this return on investment.

By the way, TDD is as much the evolution of unit testing as the screw driver is the evolution of the hammer. But that’ll have to wait for a different post.

What’s Design Got To Do With It?

If you’re looking for the technical ability to write a test fixture and replace calls to other classes, then design has nothing to it.

If tests are to be valuable – design has everything to do with it.

The difficulty our developer is having unit-testing the system is a symptom of design problems. There’s a good chance that’s why he suggested a rewrite.

By the way, please do a search & replace in your vocabulary on the word “rewrite” with the word “redesign”. The code’s syntax isn’t the problem – it’s not the “m_”, camel case, or anything like that. It’s not that if the code was rewritten under the same design that all problems will go away.

Redesign, or do nothing.

The community’s been discussing the issues of coupling, interfaces, mocking, and tools at length in the context of testability. I won’t reiterate the debate here but I’ll tell you this:

If logic is duplicated, if the code is tightly coupled, if there is no separation of concerns, the unit tests will be useless – even if they “test” the class in isolation.

Cut the coverage crap

Metrics lie.

The fact that there’s a bunch of other code which calls 100% of the system’s code and doesn’t contain false assertions doesn’t mean that the code is high quality or doesn’t contain bugs.

In a well designed system, most “logic” will be contained in two “layers” – the controllers and the domain model. These classes should be independent of any and all technological concerns. You can and should get high unit test coverage on classes in these layers. Shoot for 100%, it’s worth it.

Testing domain models is all about asserting state. While using setters to get the domain objects into a necessary initial state is OK, setters should not be used beyond that. Testing controllers is primarily about interactions – mocks will probably be needed for views and service agents. Commands do not need to be mocked out.

Most other layers have some dependence on technology that makes unit tests relatively less valuable. Coverage on these layers is most meaningless. My guidance is to take the effort that would have been spent on unit testing these other layers and invest it all in automated integration tests. You’re likely to get a much higher return on investment. Much higher.

Much.

Everybody’s Right

Developers aren’t just born knowing good design, testing, or anything else. Universities, colleges, and most vendors do little to change that state of affairs. Books help, a bit, but when learning to swim, you’ve got to get your feet wet, and on the job training is, by and large, all there is. As such, lowering the barrier to entry is important.

Keeping in mind the Dreyfus model of knowledge acquisition, it’s not about “dumbing down” software development, it’s about bringing novices up to speed:

“In the beginning [novices] learn to recognize objective facts and features, relevant to the skill. Characteristic of relevant elements are that they can be recognized context-free, i.e. without reference to the overall situation. The novice acquire basic rules to follow, acting upon those facts and features. The rules are also context-free, i.e. no notice is taken to the surroundings. On account of this the novice feels very little responsibility for the result.” (emphasis mine)

Managers are ultimately responsible for the result.

Managers shouldn’t necessarily sacrifice their projects on this altar of learning. Organizations need to find ways for developers to safely practice these techniques as a part of developing their “human resources”. First of all, this needs to be communicated to everyone – that the organization understands the importance of these techniques, the desires of developers to adopt them, and the projects that need to be delivered.

Some projects may be allocated additional non-functional requirements: the software will be developed test-first, there will be at least 80% unit test coverage, etc. It can make sense to have developers spend some time on these projects after finishing one more delivery focused project and before going onto another one. As more developers become proficient with unit testing and design, the delivery focused projects can start to benefit from these skills.

It’s a gradual process.

The Important Bit

No matter how you go about unit testing, do periodic test reviews.

Just like code reviews.

That’s it.

Related Posts

Business Process Verification

Self documenting and Test-Driven Alien Artifacts

SOA Testing

My previous post on how to create fully encapsulated domain models introduced the concept of events as a core pattern of communication from the domain back to the service layer. In that post, I put up enough code to get the idea across but didn’t address issues like memory leaks and multi-threading. This post will show the solution to those two critical points.

I’ve snipped out one of the events in the previous example for brevity.

Previous API

The previous API looked like this:

   1:  public static class DomainEvents

   2:  {

   3:       public static event EventHandler GameReportedLost;

   4:       public static void RaiseGameReportedLostEvent()

   5:       {

   6:             if (GameReportedLost != null)

   7:                 GameReportedLost(null, null);

   8:       }

   9:   

  10:       public static event EventHandler CartIsFull;

  11:       public static void RaiseCartIsFull()

  12:       {

  13:             if (CartIsFull != null)

  14:                 CartIsFull(null, null);

  15:       }

  16:  }

One thing that we want to keep in the solution is that all the code to define events, their names, and the parameters they bring will be in one place – in this case, the DomainEvents class. One thing that we’d like to fix is the amount of code needed to define an event.

Previous Service Layer

Here’s what our previous service layer code looked like:

   1:  public class AddGameToCartMessageHandler :

   2:      BaseMessageHandler<AddGameToCartMessage>

   3:  {

   4:      public override void Handle(AddGameToCartMessage m)

   5:      {

   6:          using (ISession session = SessionFactory.OpenSession())

   7:          using (ITransaction tx = session.BeginTransaction())

   8:          {

   9:              ICart cart = session.Get<ICart>(m.CartId);

  10:              IGame g = session.Get<IGame>(m.GameId);

  11:   

  12:              Domain.DomainEvents.GameReportedLost +=

  13:                gameReportedLost;

  14:              Domain.DomainEvents.CartIsFull +=

  15:                cartIsFull;

  16:   

  17:              cart.Add(g);

  18:   

  19:              Domain.DomainEvents.GameReportedLost -=

  20:                gameReportedLost;

  21:              Domain.DomainEvents.CartIsFull -=

  22:                cartIsFull;

  23:   

  24:              tx.Commit();

  25:          }

  26:      }

  27:   

  28:      private EventHandler gameReportedLost = delegate { 

  29:            Bus.Return((int)ErrorCodes.GameReportedLost);

  30:          };

  31:   

  32:      private EventHandler cartIsFull = delegate { 

  33:            Bus.Return((int)ErrorCodes.CartIsFull);

  34:          };

  35:      }

  36:  }

Another thing that should be improved is the amount of code needed in the service layer.

Raising an event, though, should still be fairly simple – one line of code similar to DomainEvents.RaiseGameReportedLost().

New API

Here’s what the new API looks like:

   1:  public static class DomainEvents

   2:  {

   3:       public static readonly DomainEvent<IGame> GameReportedLost = 

   4:                                            new DomainEvent<IGame>;

   5:   

   6:       public static readonly DomainEvent<ICart> CartIsFull=

   7:                                            new DomainEvent<ICart>;

   8:  }

It looks like we’ve managed to bring down the complexity of defining an event.

Raising an event is slightly different, but still only one line of code (”this” refers to the Cart class that is calling this API): DomainEvents.CartIsFull.Raise(this);

New Service Layer

The advantage of having a disposable domain event allows us to use the “using” construct for cleanup.

   1:  public class AddGameToCartMessageHandler :

   2:      BaseMessageHandler<AddGameToCartMessage>

   3:  {

   4:      public override void Handle(AddGameToCartMessage m)

   5:      {

   6:          using (ISession session = SessionFactory.OpenSession())

   7:          using (ITransaction tx = session.BeginTransaction())

   8:          using (DomainEvents.GameReportedLost.Register(gameReportedLost))

   9:          using (DomainEvents.CartIsFull.Register(cartIsFull))

  10:          {

  11:              ICart cart = session.Get<ICart>(m.CartId);

  12:              IGame g = session.Get<IGame>(m.GameId);

  13:   

  14:              cart.Add(g);

  15:   

  16:              tx.Commit();

  17:          }

  18:      }

  19:   

  20:      private Action<IGame> gameReportedLost = delegate { 

  21:            Bus.Return((int)ErrorCodes.GameReportedLost);

  22:          };

  23:   

  24:      private Action<ICart> cartIsFull = delegate { 

  25:            Bus.Return((int)ErrorCodes.CartIsFull);

  26:          };

  27:      }

  28:  }

I also want to mention that you don’t necessarily have to have the same service layer object handle these events as that which calls the domain objects. In other words, we can have singleton objects handling these events for things like sending emails, notifying external systems, and auditing.

The Infrastructure

The infrastructure that makes all this possible (in a thread-safe way) is quite simple and made up of two parts, the DomainEvent that we saw being used above, and the DomainEventRegistrationRemover which handles the disposing:

   1:  using System;

   2:  using System.Collections.Generic;

   3:   

   4:  namespace DomainEventInfrastructure

   5:  {

   6:      public class DomainEvent<E> 

   7:      {

   8:          [ThreadStatic] 

   9:          private static List<Action<E>> _actions; 

  10:   

  11:          protected List<Action<E>> actions 

  12:          {

  13:              get { 

  14:                  if (_actions == null) 

  15:                      _actions = new List<Action<E>>(); 

  16:   

  17:                  return _actions; 

  18:              }

  19:          }

  20:   

  21:          public IDisposable Register(Action<E> callback) 

  22:          {

  23:              actions.Add(callback);

  24:              return new DomainEventRegistrationRemover(delegate

  25:                  {

  26:                      actions.Remove(callback);

  27:                  }

  28:              ); 

  29:          }

  30:   

  31:          public void Raise(E args) 

  32:          {

  33:              foreach (Action<E> action in actions) 

  34:                  action.Invoke(args);

  35:          }

  36:      }

  37:  }

  38:   

Note that the invocation list of the domain event is thread static, meaning that each thread gets its own copy – even though they’re all working with the same instance of the domain event.

Here’s the DomainEventRegistrationRemover – even simpler:

   1:  using System;

   2:   

   3:  namespace DomainEventInfrastructure

   4:  {

   5:      public class DomainEventRegistrationRemover : IDisposable 

   6:      {

   7:          private readonly Action CallOnDispose;

   8:   

   9:          public DomainEventRegistrationRemover(Action ToCall) 

  10:          {

  11:              this.CallOnDispose = ToCall; 

  12:          }

  13:   

  14:          public void Dispose() 

  15:          {

  16:              this.CallOnDispose.DynamicInvoke();

  17:          }

  18:      }

  19:  }

For your convenience, I’ve made these available for download here.

I also want to add that if you haven’t looked at the comments on the original post – there’s some really good stuff there (36 comments so far). Take a look.

Not in applicative code anyway.

This follows up on Ayende’s post about the AOP way.

Now, I have nothing against AOP but some developers are leery of it.

In broader terms, all logging goes in framework-level code. For smart clients, one really good place to put logging is in your Command infrastructure – every time a command is invoked, log it and the args. For data access, well, any decent O/R Mapper has a lot of logging already, just use it. For communication, ditto. Funny that just last week this was one of the major bits of feedback I gave in a code review.

The Important Part

Logging is useful for developers to find out why a system isn’t working correctly.

It is terrible for knowing that a system isn’t working correctly.

If you’re entire exception management strategy is “write it to the log”, how will an admin know that something’s wrong? Did you remember to configure your logging library that errors (and maybe warnings too) should be pushed out to a monitoring system? Do you have a monitoring system?

And if the admins don’t know anything’s wrong, they won’t know they need to increase the fidelity of the logs, will they? Are you planning on providing training for your admins telling them this (and all the other things they need to know)? Or maybe this will all be set up as an automatic script?

An Agile Digression

I hope all that’s on your agile (”we can ship at the end of every 2 week iteration”) product backlog (pardon my cynicism). I hope it’s at least something that you’re looking at per release and feeding the relevant features into your iterations. Yes, there’s project work to do (writing training manuals) that isn’t “development” that needs to be handled; if you don’t timebox it into the same iterations, it won’t get done.

Now, back to you’re regularly schedule logging…

Things Logging Doesn’t Address

Logging is a mildly useless tool for pinpointing where in the system the source of a problem is.

“I know the entity isn’t in the database. I can see that. I want to know why it isn’t there.”

Sure, if you had every SQL statement logged you could figure these sorts of things out out. Of course, performance-wise, you wouldn’t put the system into production like that. In which case, the delete statement wouldn’t have been logged leaving you with precious little information to solve the root cause.

Also consider that the more logging you do, the more crap you’ll have to sift through to find the proverbial needle. Developers often don’t think twice about increasing the amount of crap logs they generate…

The Real Problem

The real problem is that developers think too much about logging and not at all nearly enough about designing the system in ways that it’ll be easy possible to answer questions like those above without having to know exactly how the system is built. One of the reasons that developers should care about this is that it’ll decrease the number of times they need to get up at 3:00 am to answer those questions.

A Path to the Solution

Now, if you had some kind of business activity monitoring (BAM) capability in your system, an admin could do a simple search/query [WHEN entity DELETED] and find out answers to the questions above, find out the time that the relevant activities occurred, figure out what the problem is on their own, and maybe even fix it – especially if it has to do with some esoteric configuration variable.Regardless of whether you buy a BAM tool or roll what you need yourself, you need to understand what about the system needs to be monitored. That’s a very different thought-process to go through than “should we log this? Yeah, sure, why not.”

It’s called “Design for Operations”.

Take a holistic perspective on exception management, logging, monitoring, etc. Think about questions like those above and then analyse your use of the relevant tools in that context. Think about all the different kinds of users of the information that’s going to be generated and how quickly their going to need to act on that information. Admins in the data-center in the middle of a crisis are going to have different needs than developers analysing logs on their machine. Think about:

• How will the administrator know that a server has been configured properly?
• If the system is feeling slow, how can the administrator know which server/process is to blame?
• So that maybe they can scale out that part of the system.

In Closing

It’s a mindset.

It takes time to make the shift.

It takes more time to bring the development process to this kind of maturity (god, I hate that word).

Writing exceptions to the log is not a strategy.

At the very best, its a tactic.

What’s your strategy?

In short, there seemed to be a hole in the development model.

Here’s an example from how nServiceBus used to do this:

builder.ConfigureComponent(typeof(HttpTransport), ComponentCallModelEnum.Singleton)
  .ConfigureProperty(”DefaultNumberOfWorkerThreads”, 10)
  .ConfigureProperty(”DefaultNumberOfSenderThreads”, 10);

The problem was that if a developer got the case of the property wrong, misspelled it in some way, or somebody later refactored/renamed that property, the system would break. It would also be very difficult to figure out why.

Then, a couple of weeks ago, it dawned on me.

This was the same problem we used to have with testing using mock objects – before we had today’s more advanced frameworks. So, the solution must be to use the same techniques. The container should give the developer an object that looks just like their class, but that would intercept all calls. Then, that interceptor could turn those into the config calls shown above. Here’s what the new config model looks like: 

HttpTransport transport = builder.ConfigureComponent<HttpTransport>
                                                   (ComponentCallModelEnum.Singleton);

transport.DefaultNumberOfSenderThreads = 10;
transport.DefaultNumberOfWorkerThreads = 10; 

Granted, you’re not going to have tons of code like this. However, for all those parameters which are factory-configured and that customers/integrators shouldn’t tinker with, it makes a difference. The biggest difference is during that time of development where you’ve gotten into preliminary integration tests but the systems components are still being “polished”.

Aside: On the current project that has adopted this model, we’ve probably saved (conservatively) about 3 months of effort with this tiny (?) thing, and this isn’t a huge project. If that’s more than you would’ve thought, well, I was surprised myself. First, understand that in the old config model, everything still compiles and unit tests pass, even though its broken.

Just consider what happens in the lab when this occurs. You have N testers that can’t test the new version, waiting. You have the person who installed the version, trying to figure out what’s wrong. They then call in one of the developers where most of the new development occurred since the previous version. They fiddle around with it, looking at exception traces and whatnot. In the best case, we’re talking about 2 hours from noticing its broken until a new version comes out fixed. Multiply that by N+3 people. Then multiply by the number of versions you do integration tests on in the lab.

Caveat: In the current version, properties must be virtual in order for this to work.

For those of you who want just this feature without nServiceBus, I’ve put up all the binaries here. For the source, you’ll need to go to here.

Let me know what you think – especially if you can take the implementation to the point where it won’t need virtual properties to work

One of the topics not covered by Prism is occasional connectivity, and I would like to say a word or two about that. First of all, if you’re building a standalone client (one that doesn’t communicate with anything), then there’s a good chance that Prism isn’t for you, although you could be composing other standalone client modules. So, if your client isn’t communicating with anything, well, then this post probably won’t interest you that much. Let’s start with…

Physics

Networks fail. Period.

This means that your client machine will not always be connected to other servers.

Also, servers fail – critical Windows patches and just regular power outages.

Ergo, your “smart” client will be occasionally connected, whether you planned for it or not.

And please don’t take this post as a “dumping on Prism” post – it isn’t intended that way. Rather, it is about how you should think about designing modules in Prism, and why.

Modules and Connectivity

Consider the case where we have two modules being composed in a single client. Each module communicates with a different server. Let’s call these modules Ma and Mb, and the servers Sa and Sb respectively. Now, let’s discuss what occurs given that the modules weren’t designed with occasional connectivity in mind.

User clicks something in Mb which requires communication.

Mb tries to call Sb, say, over HTTP, using a regular web service invocation.

The calling thread, in this case, the one used for user interaction, is blocked waiting for a response from Sb.

Sometime in this call, Sb fails, connectivity goes down, whatever.

30 seconds after the call, the HTTP connection times out.

If something important were happening in Ma at the same time, the user couldn’t even see it, let alone do anything about it since the user interaction thread is stuck. This is a serious concern for the financial services domain, but in many others as well.

You mean there’s more?

I can go on, but I think that that’s enough to paint the picture that if you are building a smart client, there are a lot more things to think about than just learning Prism. That’s my main concern after witnessing what happened around the CAB. Given the learning curve around these frameworks many developers don’t seek to deepen their understanding beyond just becoming proficient with them. This isn’t just centered on the developers, evangelists in Microsoft tend to paint the picture this way:

Once you understand X (CAB, Prism, BizTalk, whatever), all your problems are solved.

That’s not to say there aren’t good things in those technologies, but that’s just it, they’re just tools. Silver hammers and “laser” guided saws do not a master carpenter make. There’s actually a pretty good chance the regular guy will saw their arm off.

Help

I do hope more “instruction manuals” will be coming out of Microsoft on these topics. That’s not to say there aren’t any. Specifically on the topic of occasional connectivity, there is Chapter 4 of the Smart Client Architecture & Design Guide. Unfortunately, it doesn’t say anything about how that connects with the MVC/MVP being used client side (the bits affected by Prism). Chapter 6 of the same guide deals with the client-side threading, but doesn’t address issues like:

• Which model object instance are views bound to.
• Do other threads have access to that object at the same time.
• Which controller/presenter is responsible for giving that object to the view.
• Do they need to clone it.
• How deep should the clone be.
• How do various controllers/presenters (which may be showing the same object in different views at the same time) communicate changes to their various independent clones.

I haven’t yet documented all the patterns that answer these questions, but until I do (or Microsoft does), let me offer these few resources which I’ve put out over the years:

• Occasionally Connected Systems Architecture: The Client
• Occasionally Connected Systems Architecture: Concurrency
• Threading issues when smart clients reconnect
• [Podcast] Occasionally Connected Smart Clients and ADO.NET Sync Services

There’s also some more links under the Smart Client link of my “First time here?” page.

Also, please join me in asking Microsoft for an update to these guides – comments below or your own blog posts would be great.

Here’s a taste:

UI Thread and Concurrency Management

Whenever you use hosting on the UI thread, deadlocks are possible. For example, the following setup is guaranteed to result with a deadlock: A Windows Forms application is hosting a service with UseSynchronizationContext set to true, and UI thread affinity is established. The Windows Forms application then calls the service over one of its endpoints. The call to the service blocks the UI thread, while WCF posts a message to the UI thread to invoke the service. That message is never processed, because of the blocking UI thread—hence, the deadlock.

Another possible case for a deadlock occurs when a Windows Forms application is hosting a service with UseSynchronizationContext set to true and UI thread affinity is established. The service receives a call from a remote client. That call is marshaled to the UI thread and is eventually executed on that thread. If the service is allowed to call out to another service, that can result in a deadlock if the callout causality tries somehow to update the UI or call back to the service’s endpoint, because all of the service instances that are associated with any endpoint (regardless of the service-instancing mode) share the same UI thread.

Similarly, you risk a deadlock if the service is configured for reentrancy and it calls back to its client. You risk a deadlock if the callback causality tries to update the UI or enter the service, because that reentrance must be marshaled to the blocked UI thread.

Actually, I have difficulty believing that Juval would go so far as to suggest that even the forms should be services, but he does:

Form as a Service

The main motivation for hosting a WCF service on the UI thread is if the service must update the UI or the form. The problem is always: How does the service reach out and obtain a reference to the form? While the techniques and ideas that appear thus far in the listings certainly work, it would be simpler yet if the form were the service and hosted itself. For this to work, the form (or any window) must be a singleton service. The reason is that singleton is the only instancing mode that enables you to provide WCF with a live instance to host. In addition, you would not want a per-call form that exists only during a client call (which is usually very brief), nor would you want a per-session form that only a single client can establish a session with and update.

When a form is also a service, having that form as a singleton service is the best instancing mode all around.

I think that this article serves as a great treatise leading to only one conclusion – you’d have to be crazy to try to do this without some higher level framework, preferably with a different low-level framework too . Sucks Microsoft didn’t put one out – nor is there a pending beta, CTP, or even word about some project with a codename handling this. From what I know about Prism, it doesn’t intend to handle this issue either.

One thing that isn’t covered in the article is that if you do choose not to tie the client-side service to the UI thread, you open yourself up to race conditions. Reasons you’d want to handle messages on a different thread center around UI responsiveness. I’ve written about these things before:

• Object Builder – the place to fix system-wide threading bugs

• Eureka! AOP is the final piece of the multi-threaded smart client puzzle

• What Makes Smart Clients Safe?

The more I read things like this, the more I feel that I have to get going with my nServiceBus based solution. I’m fairly swamped as it is, so if anyone is interested in helping get this project off the ground, I’d be most grateful – as I think anyone else that had to build a smart client would.

Most people getting started with DDD and the Domain Model pattern get stuck on this. For a while I tried answering this on the discussion groups, but here we have a nice example that I can point to next time.

The underlying problem I’ve noticed over the past few years is that developers are still thinking in terms of querying when they need more data. When moving to the Domain Model pattern, you have to “simply” represent the domain concepts in code – in other words, see things you aren’t used to seeing. I’ll highlight that part in the question below so that you can see where I’m going to go with this in my answer:

I have an instance where I believe I need access to a service or repository from my entity to evaluate a business rule but I’m using NHibernate for persistence so I don’t have a real good way to inject services into my entity. Can I get some viewpoints on just passing the services to my entity vs. using a facade?

Let me explain my problem to provide more context to the problem.

The core domain revolves around renting video games. I am working on a new feature to allow customers to trade in old video games. Customers can trade in multiple games at a time so we have a TradeInCart entity that works similar to most shopping carts that everybody is familiar with. However there are several rules that limit the items that can be placed into the TradeInCart. The core rules are:

1. Only 3 games of the same title can be added to the cart.
2. The total number of items in the cart cannot exceed 10.
3. No games can be added to the cart that the customer had previously reported lost with regards to their rental membership.
    a. If an attempt is made to add a previously reported lost game, then we need to log a BadQueueStatusAddAttempt to the persistence store.

So the first 2 rules are easily handled internally by the cart through an Add operation. Sample cart interface is below.

   1:  class TradeInCart{

   2:      Account Account{get;}

   3:      LineItem Add(Game game);

   4:      ValidationResult CanAdd(Game game);

   5:      IList<LineItems> LineItems{get;}

   6:  }

However the #3 rule is much more complicated and can’t be handled internally by the cart, so I have to depend on external services. Splitting up the validation logic for a cart add operation doesn’t seem very appealing to me at all. So I have the option of passing in a repository to get the previously reported lost games and a service to log bad attempts. This makes my cart interface ugly real quick.

   1:  class TradeInCart{

   2:      Account Account{get;}

   3:      LineItem Add(

   4:          Game game, 

   5:          IRepository<QueueHistory> repository, 

   6:          LoggingService service);

   7:   

   8:      ValidationResult CanAdd(

   9:          Game game, 

  10:          IRepository<QueueHistory> repository, 

  11:          LoggingService service);

  12:   

  13:      IList<LineItems> LineItems{get;}

  14:  }

The alternative option is to have a TradeInCartFacade that handles the validations and adding the items to the cart. The façade can have the repository and services injected though DI which is nice, but the big negative is that the cart ends up totally anemic.

Any thought on this would be greatly appreciated.

Thanks,
Jesse

As I highlighted above, the thing that will help you with your business rules is to introduce the Customer object (that you probably already have) with the property GamesReportedLost (an IList<Game>). Your TradeInCart would have a reference to the Customer object and could then check the rule in the Add method.

Before I go into the code, it looks like your Account object might be used the same way, but your description of the domain doesn’t mention accounts, so I’m going to assume that that’s unrelated for now:

   1:  public class Customer{

   2:   

   3:      /* other properties and methods */

   4:   

   5:      private IList<Game> gamesReportedLost;

   6:      public virtual IList<Game> GamesReportedLost 

   7:      { 

   8:          get

   9:          {

  10:              return gamesReportedLost;

  11:          }

  12:          set

  13:          {

  14:              gamesReportedLost = value;

  15:          }

  16:      }

  17:  }

Keep in mind that the GamesReportedLost is a persistent property of Customer. Every time a customer reports a game lost, this list needs to be kept up to date. Here’s the TradeInCart now:

   1:  public class TradeInCart

   2:  {

   3:      /* other properties and methods */

   4:   

   5:      private Customer customer;

   6:      public virtual Customer Customer

   7:      { 

   8:          get { return customer; }

   9:          set { customer = value; }

  10:      }

  11:   

  12:      private IList<LineItem> lineItems;

  13:      public virtual IList<LineItem> LineItems

  14:      {

  15:          get { return lineItems; }

  16:          set { lineItems = value; }

  17:      }

  18:   

  19:      public void Add(Game game)

  20:      {

  21:          if (lineItems.Count >= CONSTANTS.MaxItemsPerCart)

  22:          {

  23:              FailureEvents.RaiseCartIsFullEvent();

  24:              return;

  25:          }

  26:   

  27:          if (NumberOfGameAlreadyInCart(game) >=

  28:              CONSTANTS.MaxNumberOfSameGamePerCart)

  29:          {

  30:              FailureEvents

  31:                .RaiseMaxNumberOfSameGamePerCartReachedEvent();

  32:              return;

  33:          }

  34:   

  35:          if (customer.GamesReportedLost.Contains(game))

  36:              FailureEvents.RaiseGameReportedLostEvent();

  37:          else

  38:              this.lineItems.Add(new LineItem(game));

  39:      }

  40:   

  41:      private int NumberOfGameAlreadyInCart(Game game)

  42:      {

  43:          int result = 0;

  44:   

  45:          foreach(LineItem li in this.lineItems)

  46:              if (li.Game == game)

  47:                  result++;

  48:   

  49:          return result;

  50:      }

  51:  }

  52:   

  53:  public static class FailureEvents

  54:  {

  55:      public static event EventHandler GameReportedLost;

  56:      public static void RaiseGameReportedLostEvent()

  57:      {

  58:           if (GameReportedLost != null)

  59:               GameReportedLost(null, null);

  60:      }

  61:   

  62:      public static event EventHandler CartIsFull;

  63:      public static void RaiseCartIsFullEvent()

  64:      {

  65:           if (CartIsFull != null)

  66:               CartIsFull(null, null);

  67:      }

  68:   

  69:      public static event EventHandler MaxNumberOfSameGamePerCartReached;

  70:      public static void RaiseMaxNumberOfSameGamePerCartReachedEvent()

  71:      {

  72:           if (MaxNumberOfSameGamePerCartReached != null)

  73:               MaxNumberOfSameGamePerCartReached(null, null);

  74:      }

  75:  }

Your service layer class that calls the Add method of TradeInCart would first subscribe to the relevant events in FailureEvents. If one of those events is raised, it would do the necessary logging, external system calls, etc.

As you can see, the API of TradeInCart doesn’t need to make use of any external repositories, nor do you need to inject any other external dependencies in.

One thing I didn’t do in the above code to keep it “short” is to define the relevant custom EventArgs for bubbling up the information as to which game was reported lost or already have 3 of those in the cart. That is something that definitely should be done so that the service layer can pass this information back to the client.

Here’s a look at Service Layer code:

   1:  public class AddGameToCartMessageHandler :

   2:      BaseMessageHandler<AddGameToCartMessage>

   3:  {

   4:      public override void Handle(AddGameToCartMessage m)

   5:      {

   6:          using (ISession session = SessionFactory.OpenSession())

   7:          using (ITransaction tx = session.BeginTransaction())

   8:          {

   9:              TradeInCart cart = session.Get<TradeInCart>(m.CartId);

  10:              Game g = session.Get<Game>(m.GameId);

  11:   

  12:              Domain.FailureEvents.GameReportedLost +=

  13:                gameReportedLost;

  14:              Domain.FailureEvents.CartIsFull +=

  15:                cartIsFull;

  16:              Domain.FailureEvents.MaxNumberOfSameGamePerCartReached +=

  17:                maxNumberOfSameGamePerCartReached;

  18:   

  19:              cart.Add(g);

  20:   

  21:              Domain.FailureEvents.GameReportedLost -=

  22:                gameReportedLost;

  23:              Domain.FailureEvents.CartIsFull -=

  24:                cartIsFull;

  25:              Domain.FailureEvents.MaxNumberOfSameGamePerCartReached -=

  26:                maxNumberOfSameGamePerCartReached;

  27:   

  28:              tx.Commit();

  29:          }

  30:      }

  31:   

  32:      private EventHandler gameReportedLost = delegate { 

  33:            Bus.Return((int)ErrorCodes.GameReportedLost);

  34:          };

  35:   

  36:      private EventHandler cartIsFull = delegate { 

  37:            Bus.Return((int)ErrorCodes.CartIsFull);

  38:          };

  39:   

  40:      private EventHandler maxNumberOfSameGamePerCartReached = delegate { 

  41:            Bus.Return((int)ErrorCodes.MaxNumberOfSameGamePerCartReached);

  42:          };

  43:      }

  44:  }

It’s important to remember to clean up your event subscriptions so that your Service Layer objects get garbage collected. This is one of the primary causes of memory leaks when using static events in your Domain Model. I’m hoping to find ways to use lambdas to decrease this repetitive coding pattern. You might be thinking to yourself that non-static events on your Domain Model objects would be easier, since those objects would get collected, freeing up the service layer objects for collection as well. There’s just on small problem:

The problem is that if an event is raised by a child (or grandchild object), the service layer object may not even know that that grandchild was involved and, as such, would not have subscribed to that event. The only way the service layer could work was by knowing how the Domain Model worked internally – in essence, breaking encapsulation.

If you’re thinking that using exceptions would be better, you’d be right in thinking that that won’t break encapsulation, and that you wouldn’t need all that subscribe/unsubscribe code in the service layer. The only problem is that the Domain Model needs to know that the service layer had a default catch clause so that it wouldn’t blow up. Otherwise, the service layer (or WCF, or nServiceBus) may end up flagging that message as a poison message (Read more about poison messages). You’d also have to be extremely careful about in which environments you used your Domain Model – in other words, your reuse is shot.

Conclusion

I never said it would be easy

However, the solution is simple (not complex). The same patterns occur over and over. The design is consistent. By focusing on the dependencies we now have a domain model that is reusable across many environments (server, client, sql clr, silverlight). The domain model is also testable without resorting to any fancy mock objects.

One closing comment – while I do my best to write code that is consistent with production quality environments, this code is more about demonstrating design principles. As such, I focus more on the self-documenting aspects of the code and have elided many production concerns.

Do you have a better solution?

Something that I haven’t considered?

Do me a favour – leave me a comment. Tell me what you think.

For instance, in an HR system, the process of interviewing candidates – wouldn’t you just insert, update, and delete these Appointment objects?

If I were to put on my domain-driven hat, I would describe those requirements differently – interview appointments have a lifecycle: proposed, accepted, cancelled, etc. It seems that only a user of the role HR Interviewer should be able to make appointments for themselves, so the service layer code would probably look something like this:

using (ISession session = SessionFactory.OpenSession())
using (ITransaction tx = session.BeginTransaction())
{
    ICandidateInterviewer interviewer = session.Get<ICandidateInterviewer>(message.InterviewerId);
    ICandidate candidate = session.Get<ICandidate>(message.CandidateId);

    interviewer.ScheduleInterviewWith(candidate).At(message.RequestedTime);
    tx.Commit();
}

The “ScheduleInterviewWith” method accepts an ICandidate and returns an IAppointment. IAppointment has a method “At” which accepts a DateTime parameter and returns void – just changes the data of the appointment. The state of the appointment at creation time would probably be proposed. The appointment object would probably be added to the list of appointments for that interviewer – that’s what will cause it to be persisted automatically.

Later, when the candidate accepts the meeting, we could have the following method on ICandidate – void Accept(IAppointment); that would obviously check that the candidate is the right person for that interview, the appointment’s current state (not cancelled), etc – finally updating its state. What part of this looks like create, update, delete? If that’s what your service layer to domain interaction looks like, do you now know what your messages will be looking like?CRUD seems to be what most of us are familiar with. Moving to domain-driven thinking takes time and practice, but is well worth it. Contrast this with a more traditional O/R mapping solution:

using (ISession session = SessionFactory.OpenSession())
using (ITransaction tx = session.BeginTransaction())
{
    ICandidateInterviewer interviewer = session.Get<ICandidateInterviewer>(message.InterviewerId);
    ICandidate candidate = session.Get<ICandidate>(message.CandidateId); 

    Appointment a = new Appointment(); 

    a.Interviewer = interviewer;
    interviewer.Appointments.Add(a); 

    a.Candidate = candidate;
    candidate.Appointments.Add(a); 

    a.Time = message.RequestedTime; 

    session.Save(a);  

    tx.Commit();
} 

As you can see, we’ve got simpler, more expressive, and more testable code when employing the domain model pattern, than using “just” O/R mapping. I’m not saying that the domain model pattern doesn’t need O/R mapping in the background for it to work. But that’s just it – the persistence gunk needs to be in the background and the business logic needs to be encapsulated.

So, while I’ll agree with Dave that the Domain Model is more lifestyle than pattern, I would argue against these conclusions:

If this post had a point, it’s only to share the idea that Domain Model is a big, big thing. It’s probably overkill in a lot of cases where you have simple applications that have very simple purposes.

As you just saw in the example above, there is no “overkill” to be seen. The domain model in the example wasn’t “a big, big thing”.

The domain model. Use it.

Why not have a better lifestyle?  

First of all let me say that Juval is indeed a master presenter. The “looks like a class, walks like a class, quacks like a class” bit was excellent. I could tell that most people didn’t notice the speedy hands quickly deleting all attributes from the classes before the “looks like a class…” bit. At times, I got flashbacks from the Wizard of Oz – “pay no attention to the man behind the curtain”. If all attributes in WCF only went on the interfaces, then this might actually fly, but we all know that that’s not the case.

One of the interesting comparisons Juval made with WCF was the introduction of .NET. Few people in the audience seemed to remember (or maybe were just professionally younger than .NET’s 8 years), but when it came out .NET was marketed as being mainly about XML Web Services. Juval stated that this was done to play down the fact that .NET made the previous Windows programming technologies obsolete. He then drew the same conclusion about WCF – that it’s as much .NET 3.0 as .NET was the next version of MFC; besides being written in a language that resembles the previous technology, it’s really all different. I don’t think that anyone would argue the difference, but is it really a “plain .NET” killer?

The answer seemed to come around the overhead of WCF – yet Juval deftly deflected that issue with a demo showing WCF doing 200 calls a second. And everybody just bought it – I was shocked. That’s 5ms per call. If you actually take Juval’s advice and use WCF on all your classes, you’ve bought yourself one hell of a performance nightmare. Say you have around 20 of your objects involved in a sequence to handle a user action – not that many actually. With a 5ms lag per object interaction, that user action is going to take 100ms – not including any database or webservice stuff you might be doing. If you do that in a server environment, you’ll be doing roughly 10 concurrent users per core. And that’s not even doing any heavy calculations or anything. Moderately sized systems are running upwards of 1000 concurrent users – if they needed 100 cores (or dozens of servers) for that, I’m guessing that they’d be out of business.

Let’s cut this short – WCF everywhere doesn’t scale, doesn’t perform, isn’t maintainable, or testable either. In other words – don’t do it. I know Juval is a brilliant guy, and an amazing presenter – but I don’t believe he would be employing this with his own clients. This actually bears repeating. WCF is a fine technology for your application’s boundaries, but don’t be pushing it in.

Don’t do it.

If you’re interested in getting a kick-start with nServiceBus, I’ll be presenting a full day tutorial on it at QCon in London on March 11th. Here are the full details :

Title:

Build Scalable, Maintainable, Distributed Enterprise .NET Solutions with nServiceBus

Description:

Despite the recent flood of technologies and releases, distributed enterprise .net solution development remains as hard as ever.

WCF and WF provide valuable runtime components, yet still leave open the risk of developers using the wrong combination of options and ending up with an unscalable solution.

In this tutorial, developers will learn the specific patterns necessary to achieve scalability, as well as use supporting open-source frameworks to enable parallel development (and debugging!) of service interfaces, layers, and domain models.

After delving in to asynchronous message design, long-running workflow state management, and transaction-boundary placement developers will be able to design, debug, and deploy their specific distributed systems.

—

Hope to see you there.

You see, the main problem that I hadn’t been able to solve was that in order for the code to be thread-safe, you had to make sure that no code in the views would/could change entity data. One solution is not to use data-binding, which sucks, but isn’t enough to be sure. Another solution is to have all supervising-controllers clone an entity before they give it to a view. Even if you could possibly code review every line of those classes, the new guy (or old guy who forgot) will, by accident, write one new line of code that could pass an entity to a view without cloning it first. That’s not a very sustainable solution.

This thing has been bothering me for a couple of months now and I hadn’t found a way around it. Until yesterday, like I said. I was talking to somebody about threading stuff, and somehow my unconscience lobbed me this thought about AOP. Now I’m not the sharpest pencil in the pack, but I know to listen when my unconscience “speaks”.

So I set about going over what I knew about AOP – interceptors, advisors, advice, introductions, etc, etc. And then it dawned on me. I could intercept all calls to any object that implemented IView, check the parameters of those calls, and if they implemented IEntity, to clone them before passing them through.

<Homer-style WOOHOO />

The great thing is that developers don’t need to remember to clone entities – it happens automatically. The even greater thing is that this will lead developers to writing the correct kind of interaction between their views and supervising controllers.

Together with nServiceBus, this is going to make the extremely difficult problem of writing thread-safe smart clients possible.

I’ve never made use of AOP in a framework before so I’d like to get the broader community’s feedback on this before incorporating this in production. I’ve spoken with some serious AOP folks who have allayed most of my uncertainties, but I’d like to hear more. Anyway, here’s the proof of concept (that makes use of Spring).

If this turns out to be a viable solution, I think we’ll have a solid environment for building a software factory on top of. That is something that I’m really excited about. In this multi-core future (present) that is upon us, multi-threading on the client is pretty much a necessity. We need a way to get things safe and stable by default without requiring a member of the CLR team to hold our hand.

Anybody who’s interested in helping, drop a comment below.

The Setup

Just so that the example is in itself secure, we’ll assume that the password is one-way hashed before being stored. Also, given a reasonable network infrastructure our web servers will be isolated in the DMZ and will have to access some application server which, in turn, will communicate with the DB. There’s also a good chance for something like round-robin load-balancing between web servers, especially for things like user login.

Before diving into the meat of it, I wanted to preface with a few words. One of the commonalities I’ve found when people dismiss asynchrony is that they don’t consider a real deployment environment, or scaling up a solution to multiple servers, farms, or datacenters.

The Synchronous Solution

In the synchronous solution, each one of our web servers will be contacting the app server for each user login request. In other words, the load on the app server and, consequently, on the database server will be proportional to the number of logins. One property of this load is its data locality, or rather, the lack of it. Given that user U logged in, the DB won’t necessarily gain any performance benefits by loading all username/password data into memory for the same page as user U. Another property is that this data is very non-volatile – it doesn’t change that often.

I won’t go to far into the synchronous solution since its been analysed numerous times before. The bottom line is that the database is the bottleneck. You could use sharding solutions. Many of the large sites have numerous read-only databases for this kind of data, with one master for updates – replicating out to the read-only replicas. That’s great if you’re using a nice cheap database like mySql (of LAMP), not so nice if you’re running Oracle or MS Sql Server.

Regardless of what you’re doing in your data tier, you’re there. Wouldn’t it be nice to close the loop in the web servers? Even if you are using Apache, that’s going to be less iron, electricity, and cooling all around. That’s what the asynchronous solution is all about – capitalizing on the low cost of memory to save on other things.

The Asynchronous Solution

In the asynchronous solution, we cache username/hashed-password pairs in memory on our web servers, and authenticate against that. Let’s analyse how much memory that takes:

Usernames are usually 12 characters or less, but let’s take an average of 32 to be sure. Using Unicode we get to 64 bytes for the username. Hashed passwords can run between 256 and 512 bits depending on the algorithm, divide by 8 and you have 64 bytes. That’s about 128 bytes altogether. So we can safely cache 8 million of these with 1GB of memory per web server. If you’ve got a million users, first of all, good for you Second, that’s just 128 MB of memory – relatively nothing even for a cheap 2GB web server.

Also, consider the fact that when registering a new user we can check if such a username is already taken at the web server level. That doesn’t mean it won’t be checked again in the DB to account for concurrency issues, but that the load on the DB is further reduced. Other things to notice include no read-only replicas and no replication. Simple. Our web servers are the "replicas".

The Authentication Service

What makes it all work is the "Authentication Service" on the app server. This was always there in the synchronous solution. It is what used to field all the login requests from the web servers, and, of course, allowed them to register new users and all the regular stuff. The difference is that now it publishes a message when a new user is registered (or rather, is validated – all a part of the internal long-running workflow). It also allows subscribers to receive the list of all username/hashed-password pairs. It’s also quite likely that it would keep the same data in memory too.

The same message can be used to publish both single updates, and returning the full list when using NServiceBus. Let’s define the message:

And the message that the web server sends when it wants the full list:

And the code that the web server runs on startup looks like this (assuming constructor injection):

And the code that runs in the Authentication Service when the GetAllUsernamesMessage is received:

And the class on the web server that handles a UsernameInUseMessage when it arrives:

When the app server sends the full list, multiple objects of the type UsernameInUseMessage are sent in one physical message to that web server. However, the bus object that runs on the web server dispatches each of these logical messages one at a time to the message handler above.

So, when it comes time to actually authenticate a user, this the web page (or controller, if you’re doing MVC) would call:

When registering a new user, the web server would of course first check its cache, and then send a RegisterUserMessage that contained the username and the hashed password.

When the RegisterUserMessage arrives at the app server, a new long-running workflow is kicked off to handle the process:

That UsernameInUseMessage would eventually arrive at all the web servers subscribed.

Performance/Security Trade-Offs

When looking deeper into this workflow we realize that it could be implemented as two separate message handlers, and have the email address take the place of the workflow Id. The problem with this alternate, better performing solution has to do with security. By removing the dependence on the workflow Id, we’ve in essence stated that we’re willing to receive a UserValidatedMessage without having previously received the RegisterUserMessage.

Since the processing of the UserValidatedMessage is relatively expensive – writing to the DB and publishing messages to all web servers, a malicious user could perform a denial of service (DOS) attack without that many messages, thus flying under the radar of many detection systems. Spoofing a guid that would result in a valid workflow instance is much more difficult. Also, since workflow instances would probably be stored in some in-memory, replicated data grid the relative cost of a lookup would be quite small – small enough to avoid a DOS until a detection system picked it up.

Improved Bandwidth & Latency

The bottom line is that you’re getting much more out of your web tier this way, rather than hammering your data tier and having to scale it out much sooner. Also, notice that there is much less network traffic this way. Not such a big deal for usernames and passwords, but other scenarios built in the same way may need more data. Of course, the time it takes us to log a user in is much shorter as well since we don’t have to cross back and forth from the web server (in the DMZ) to the app server, to the db server.

The important thing to remember in this solution is doing pub/sub. NServiceBus merely provides a simple API for designing the system around pub/sub. And publishing is where you get the serious scalability. As you get more users, you’ll obviously need to get more web servers. The thing is that you probably won’t need more database servers just to handle logins. In this case, you also get lower latency per request since all work needed to be done can be done locally on the server that received the request.

ETags make it even better

For the more advanced crowd, I’ll wrap it up with the ETags. Since web servers do go down, and the cache will be cleared, what we can do is to write that cache to disk (probably in a background thread), and "tag" it with something that the server gave us along with the last UsernameInUseMessage we received. That way, when the web server comes back up, it can send that ETag along with its GetAllUsernamesMessage so that the app server will only send the changes that occurred since. This drives down network usage even more at the insignificant cost of some disk space on the web servers.

And in closing…

Even if you don’t have anything more than a single physical server today, and it acts as your web server and database server, this solution won’t slow things down. If anything, it’ll speed it up. Regardless, you’re much better prepared to scale out than before – no need to rip and replace your entire architecture just as you get 8 million Facebook users banging down your front door.

So, go check out NServiceBus and get the most out of your iron.

Source: Ted Neward’s post: Welcome To The Shitty Code Support Group.

And I consider myself to be a top 10% kind-of-guy.

Imagine the kind of crap all those other people are putting out. Not you, the reader, of course. The fact that you’re reading my writing means you’re a top 10% kind of person too  

Having good design helps.

Helps decrease the impact mistakes in one part of the system have on other parts.

That way, when I’m working on one part of the system, I only have to deal with my crap. And that’s a good thing. Because I’ve gotten pretty good at fixing my crap over the years.

–

This movement is definitely a good thing, over all. Moving to a failure-oriented mindset will help us put the appropriate tools and processes in place to making more robust systems, maybe even Non-Stop Software.

The first is to insulate the framework and application code that I write from the choice of one dependency injection technology or another. I want the ability to switch easily from one to the other – not so much that projects go back and forth. Updating those config files is definitely not easy. However, it allows me to have “portable” framework code that is applicable to all the projects I consult on, regardless of their choice of technology.

The second has to do with NServiceBus specifically. In order to make use of duplex communication on smart clients, you need a background thread. That thread will be updating the same (model) objects as the UI thread. That means we need synchronization. I prefer to use .NET’s built-in synchronization domains in order to solve this rather thorny problem.

The only thing is that message handlers need to be in the synchronization domain so that they can easily update those objects. However, the Bus object must not be in the synchronization domain so that if we’ve received a large update from the server, we won’t be locking out the UI thread from interacting with data on the client.

Since the bus makes use of a dependency injection framework to create message handlers, this was the best place to put the code which causes message handlers to run within the synchronization domain.

Be aware that in order to enjoy this feature, you need to split up those large server updates into multiple, logical objects (that implement IMessage), but you can still publish them all in one go using the method:

void Publish(params IMessage[] messages);

And, of course, you need to set the JoinSynchronizationDomain property of the Object Builder.

I’ll have a podcast coming out on this topic soon.

You can get the code here:

Object Builder.zip

But you’ll have to get the Spring Framework code from the official site. Make sure you download RC 1.1. Then, take the binaries and copy them to the “BIN” folder of the Object Builder solution. If you’re looking to save on some “weight”, you only need “Spring.Core.dll”, “Common.Logging.dll” and “antlr.runtime.dll” for the solution to compile. You will need one of the logging implementations DLLs to get anything written to a log, obviously.