Before getting into that, I want to start with a slightly broader scope of discussion.

You see, I get asked about “best practices” on all sorts of things. And I try not to be the kind of consultant that responds with “it depends”, but the context of the question often makes the answer irrelevant. And the unspoken context of a best-practice question is:

Given infinite time and budget

The biggest problem that I see with well-intentioned, best-practices-following developers and architects is that they don’t ask the question “is this the right thing for us to be focusing on right now?” Understandably, that is a difficult question to answer – but it needs to be asked, since you don’t have infinite time or budget to do everything according to best practices (assuming those even exist).

About testing

The biggest issue I have with the “design for testability” topic is the extremely narrow view it takes of the word “testability”, usually in the form of more code written by a developer which invokes the production code of the system, also known as “unit tests”.

There are many different kinds of testing – unit, integration, functional, load, performance, exploratory, etc… where some may be automated and others not. Should we not discuss what “design for testability” means for not-just-unit-testing?

And what’s the point of testing anyway?

It’s not to find bugs.

Research has shown that testing (of all kinds) is not the most effective way of finding bugs. I don’t have the reference handy but I’m pretty sure that it’s from Alistair Cockburn’s work. Code reviews are (on average) about 60% more effective.

Don’t get me wrong – testing can provide indications that the software has bugs in it, but not necessarily where in the code those bugs are.

The purpose of testing is to provide quantitative and qualitative information about the system that can help various stakeholders in their decision-making processes. The relevance of that information indicates the quality of the testing. Here are some examples:

• The system supports 100 concurrent users, with the expected user-type distribution (X% role A, Y% role B, etc), performing expected use-case distributions, and collaboration scenarios.
• Time to proficiency for new users in role A is expected to be 3 days
• Alternate #2 of use case #12 fails on step #3

As you can see, the relevance of the above information is dependent on what decisions the various stakeholders need to make. The bullet on load can help us decide if more machines are needed or if developers need to tune the performance of the systems. The bullet on time to proficiency can help us decide if larger investment in usability is required. Information like the last bullet can be used in conjunction with the first two to decide on the timing and type of a release.

The timeliness of this relevant information is critical to the success of a project.

Choosing which and how much of the various testing activities to perform when is something that needs to be revisited several times throughout the lifetime of a project, taking into account the current risks (threats and probabilities) and time and resource investment to mitigate them.

Let me reiterate – we’re not going to have enough time to do everything.

On iterations

If the only part of your organization that is doing iterations are your developers, you’re not agile.

In order to capitalize on the information that testers are providing, you need them in your iterations.

The same goes for the other roles involved in the project – business analysts, DBAs, sysadmins, etc.

I know that 99% of organizations aren’t structured in a way to do this.

I never said doing this would be easy.

On design

Figuring out what kind of design and how much to do when is just as important, and just as hard. Design for testability is one part of that, but not the only one, or necessarily the most important one at any point of time.

Within that design for testability topic is the “design for unit-testing” sub-topic which seems to be the popular one. Before getting into the design aspects of it, let’s take a closer look at the unit-testing side of things.

On unit-testing

The assumption is that having more unit tests will lead to a code-base with less bugs, thus requiring shorter time to get the system into production, which will pay back the time it took to write those unit tests to begin with.

In practice, what tends to happen is that as development progresses, testing code breaks as the structure of the production code changes. Now one of two things happens – either the testing code is removed or rewritten. In either case, we didn’t get the return on investment we expected on the first bit of testing code. Unfortunately, rare is the case where the relevant people in the organization understand why, resulting in the same situation repeating itself over and over again.

Those projects would have been better off without unit testing, though the organization as a whole might have used those experiences to learn and improve. It’s been my experience that if the organization wasn’t conscious enough in the context of the project to notice the situation, it is unlikely to do so at higher levels.

On fragile unit tests

The reason that a unit test ends up being rewritten (or removed) is that its code was coupled to the production code in such a way that it broke when the production code changed. This tendency to break (fragility) is a critical property of a unit test. A fragile unit test will slow down a developer doing work on some existing code – it actually makes the system less maintainable.

For a unit test code to be stable (not fragile) it needs to be coupled to stable properties of the production code. The question of whether the production code is designed in such a way that it has stable properties – is a design question. Is it a unit? If not, you will not be able to write a unit-test against it.

And anyway, who said that every class is a unit, or should be a unit? Domain models (when done right) are good examples of a unit, yet the classes that make them up may not be units. Unit-testing should only be attempted with things which are units.

I think too much weight is put on whether a dependency of a class is a concrete or interface type, and not nearly enough on the nature of the dependency. I wouldn’t blame the hammer for pounding my thumb, and by the same token I think that blame should not be directed towards tools like those from TypeMock.

On tools

There is so much more depth to both design and testability that needs to be more broadly understood. No tool has yet been created to handle either design or testing in such a way that humans can give up responsibility for the outcome.

Over the years I’ve noticed that tools are most significant when used by skilled practitioners, which makes sense in retrospect. Giving a novice carpenter a laser-guided saw probably won’t significantly change the outcome of their work. Ultimately, the skilled practitioners are the ones that create tools – not the novices. And no tool, no matter how advanced, will make a novice perform at levels like the skilled practitioner.

In the case of a project too big for a single skilled practitioner to complete in the time required (or at all), the balance of importance shifts away from tools to the project management topics described above.

In summary

I hope that this post has shed some light on the context in which decisions with respect to testing need to be made. Design is one activity that can support certain kinds of testing, but not the only one, or even the most important one for the given type of testing necessary at that time in the project.

Design is hard. Project management is hard. Testing is hard.

Getting the right mix of people that together have enough experience and skills in these activities isn’t easy.

Don’t expect that sprinkling some interfaces in your code base will be enough.
That doesn’t count much in the way of design, just as writing code in a testing namespace doesn’t count much in the way of testability.

Looking forward to hearing your comments.

It’s one of the things that sometimes drives me mad about the YAGNI philosophy of Agile.

We need to stop throwing out the baby with the bath water.

Jay really liked that statement with relation to my previous post “Scalability – you wish you’re gonna need it“, so I thought I’d put up a logo for this movement. Anyone feeling like joining in, leave a comment, link, or whatever.

I understand that we don’t need to over-engineer everything, putting in every possible kind of extensibility point, so I accept that part of YAGNI. That is not a license to not think about the extensibility points you do need.

<Remarks>

This is a somewhat tongue-in-cheek post, and I do not want the pendulum to swing to far back the other way. But I do think it’s time it took a step back from the over-zealous “we’ll TDD our way there” thinking. Maybe Ron can pull it off. I’ve yet to see anyone else succeed.

</Remarks>

Ayende and I had an email conversation that started with me asking what would happen if I added an Order to a Customer’s “Orders” collection, when that collection was lazy loaded. My question was whether the addition of an element would result in NHibernate hitting the database to fill that collection. His answer was a simple “yes”. In the case where a customer can have many (millions) of Orders, that’s just not a feasible solution. The technical solution was simple – just define the Orders collection on the Customer as “inverse=true”, and then to save a new Order, just write:

Although it works, it’s not “DDD compliant”

In Ayende’s post Architecting for Performance he quoted a part of our email conversation. The conclusion I reached was that in order to design performant domain models, you need to know the kinds of data volumes you’re dealing with. It affects both internals and the API of the model – when can you assume cascade, and when not. It’s important to make these kinds of things explicit in the Domain Model’s API.

How do you make “transparent persistence” explicit?

The problem occurs around “transparent persistence”. If we were to assume that the Customer object added the Order object to its Orders collection, then we wouldn’t have to explicitly save orders it creates, so we would write service layer code like this:

On the other hand, if we designed our Domain Model around the million orders constraint, we would need to explicitly save the order, so we would write service layer code like this:

But the question remains, how do we communicate these guidelines to service layer developers from the Domain Model? There are a number of ways, but it’s important to decide on one and use it consistently. Performance and correctness require it.

Solution 1: Explicitness via Return Type

The first way is a little subtle, but you can do it with the return type of the “CreateOrder” method call. In the case where the Domain Model wishes to communicate that it handles transparent persistence by itself, have the method return “void”. Where the Domain Model wishes to communicate that it will not handle transparent persistence, have the method return the Order object created.

Another way to communicate the fact that an Order has been created that needs to be saved is with events. There are two sub-ways to do so:

Solution 2: Explicitness via Events on Domain Objects

The first is to just define the event on the customer object and have the service layer subscribe to it. It’s pretty clear that when the service layer receives a “OrderCreatedThatRequiresSaving” event, it should save the order passed in the event arguments.

The second realizes that the call to the customer object may come from some other domain object and that the service layer doesn’t necessarily know what can happen as the result of calling some method on the aggregate root. The change of state as the result of that method call may permeate the entire object graph. If each object in the graph raises its own events, its calling object will have to propagate that event to its parent – resulting in defining the same events in multiple places, and each object being aware of all things possible with its great-grandchild objects. That is clearly bad.

What [ThreadStatic] is for

So, the solution is to use thread-static events.

[Sidebar] Thread-static events are just static events defined on a static class, where each event has the ThreadStaticAttribute applied to it. This attribute is important for server-side scenarios where multiple threads will be running through the Domain Model at the same time. The easiest thread-safe way to use static data is to apply the ThreadStaticAttribute.

Solution 3: Explicitness via Static Events

Each object raises the appropriate static event according to its logic. In our example, Customer would call:

And the service layer would write:

The advantage of this solution is that it requires minimal knowledge of the Domain Model for the Service Layer to correctly work with it. It also communicates that anything that doesn’t raise an event will be persisted transparently behind the appropriate root object.

Statics and Testability

I know that many of you are wondering if I am really advocating the use of statics. The problem with most static classes is that they hurt testability because they are difficult to mock out. Often statics are used as Facades to hide some technological implementation detail. In this case, the static class is an inherent part of the Domain Model and does not serve as a Facade for anything.

When it comes to testing the Domain Model, we don’t have to mock anything out since the Domain Model is independent of all other concerns. This leaves us with unit testing at the single Domain Class level, which is pretty useless unless we’re TDD-ing the design of the Domain Model, in which case we’ll still be fiddling around with a bunch of classes at a time. Domain Models are best tested using State-Based Testing; get the objects into a given state, call a method on one of them, assert the resulting state. The static events don’t impede that kind of testing at all.

What if we used Injection instead of Statics?

Also, you’ll find that each Service Layer class will need to subscribe to all the Domain Model’s events, something that is easily handled by a base class. I will state that I have tried doing this without a static class, and injecting that singleton object into the Service Layer classes, and in that setter having them subscribe to its events. This was also pulled into a base class. The main difference was that the Dependency Injection solution required injecting that object into Domain Objects as well. Personally, I’m against injection for domain objects. So all in all, the static solution comes with less overhead than that based on injection.

Summary

In summary, beyond the “technical basics” of being aware of your data volumes and designing your Domain Model to handle each use case performantly, I’ve found these techniques useful for designing its API as well as communicating my intent around persistence transparency. So give it a try. I’d be grateful to hear your thoughts on the matter as well as what else you’ve found that works.

Related posts:

• Fetching Strategy Design – showing how to separate the concern of eager loading from both your Domain Model and your Service Layer.
• Better Domain-Driven Design Implementation – showing the basics of how valuable interfaces between your Domain Model and the Service Layer can be.
• Lazy Loading, and how messaging fixes everything again – describing the advantage of O/R mapping your message classes as well.
• Query Objects vs Methods on a Repository – discussing the scalability (in terms of number of developers and queries) of Query Objects.

In Ruby it’s easy to redefine the visibility of a method from private to public for testing purposes. This was just one small detail in his talk of course, but I started to think about how much pain I quite often go through regarding this in legacy code.

Let me just start out by saying that dealing with legacy code is far from easy. However, when writing new code, or as a part of refactoring that legacy, adding interfaces to your design can help you get around those visibility and testing issues.

For instance, if you only let “client” code access the interface, and not the implementation (probably using some kind of dependency injection), then you could leave all sorts of methods public on your concrete class without worry that the client will call them since the interface doesn’t expose them. Now that the methods on the concrete class are public, you can easily test that class.

The way to package your code to make sure this occurs follows a very simple design principle. This is much easier to put in practice on new development, but you’ll find that it isn’t that hard for legacy either. Since legacy code often doesn’t make use of interfaces, and the implementation is already packaged, and your new client code will already be separately packaged, you’re 90% there!

If you can change the legacy code, you’re actually 95% there. Just create a new package for the interface that the client code will use. Then change the legacy code to implement that interface – since the methods will be pretty much the same, it’s not that much work.

If you can’t change the legacy code, create a “wrapper” class that implements the interface and delegates the calls to the legacy code.

Finally, let me sum up by saying that I think Ruby is great. However, I think that often many advantages are attributed to anything new that may have already been possible/easy with what we already have today. Sometimes, the new stuff helps raise awareness on important issues – and then we can have great discussions on how to solve those issues with today’s (yesterday’s ?) technology

From some pointers about how to use (and not to use) these tools, see why object-relational mapping sucks.

Martin Fowler’s has this to say about the Domain Model Pattern:

At its worst business logic can be very complex. Rules and logic describe many different cases and slants of behavior, and it’s this complexity that objects were designed to work with. A Domain Model creates a web of interconnected objects, where each object represents some meaningful individual, whether as large as a corporation or as small as a single line on an order form.

In short, using Object-Oriented techniques to handle the complexity.

A more comprehensive discussion about what happens when it is not used can be found under the Anemic Domain Model Anti-Pattern:

The basic symptom of an Anemic Domain Model is that at first blush it looks like the real thing. There are objects, many named after the nouns in the domain space, and these objects are connected with the rich relationships and structure that true domain models have. The catch comes when you look at the behavior, and you realize that there is very little behavior on these objects. Indeed often these models come with design rules that say that you are not to put any domain logic in the the domain objects. Instead there are a set of service objects which capture all the domain logic. These services live on top of the domain model and use the domain model for data.

The fundamental horror of this anti-pattern is that it’s so contrary to the basic idea of object-oriented design; which is to combine data and process together. The anemic domain model is really just a procedural style design…

In terms of Domain-Driven Design, this pattern is also known as Domain Layer.

Domain Models do a lot for encapsulating Business Rules, thus making them amenable to automated testing. This hinges on keeping the Domain Model independent of things related to Data Access.

It is therefore almost required to use some kind of Object/Relational Mapping tool to make it possible to persist objects belonging to the domain model to databases and other kinds of storage.

The use of DataSets in .NET is often a sign of the Anemic Domain Model Anti-Pattern.

One thing to keep in mind when working on a domain model is that you probably won’t get it “right” the first time, and will have re-work the division of responsibility a couple of times. Techniques like Test-Driven Development help out immensely for that.

How much, and what kind of documentation do we need to create even if we have “self-documenting code”? Or is that kind of code enough all by itself? I for one yearn for the day where the code really will be enough, and I think that Scott and Ayende do too. First of all, I think that as time progresses, the size of systems that the average developer works on is increasing substantially. And the larger a system is, the more we find a kind of code/design dialect that developers use to talk about that system. I think that these dialects are significantly different between framework/open source code, and application code. So, when a new developer comes in, do we tell them to “just read the code”? Or if the application has gone into production some time ago and now needs to be enhanced but the original team is long gone, what can be done?

What I usually suggest (and practice) is to have some kind of documentation explaining “language” of the system – what things go together, how, and why; and just as importantly, what things must be kept apart. This can be a Software Architecture and Design document, or even videos of the design sessions where things were first discussed. The relevant requirements should be a part of this as well. We need to know that the reason asynchronous messaging was used was for the strenuous scalability requirements, otherwise we might start adding the high-productivity Visual Studio Web Services we like so much.

Continuous integration is a boon to projects who use them. But developers who are only familiar with building solutions in Visual Studio may not be used to working that way, keeping code checked out for weeks at a time.

Test-Driven Development and Architecture will help those who know about it. We should probably write something up about how the system should be enhanced. Side-effect or not, developers and testers will need to know how we test the system – what tools are used in which way and at which stage of the development lifecycle. We could reference existing methodologies as well as in what ways we deviate from them and why.

What sustainable business value do we provide by leaving behind us alien artifacts and practices? I’m not saying not to use state-of-the-art designs, techniques, and practices. On the contrary, I think that they are the key to sustainable business value. Necessary, but not sufficient. Documentation, of whatever flavor you find most suitable to each specific thing you are documenting, should be considered by the team as a whole, and the conclusions presented to stakeholders. The famous Chrysler C3 project’s long-term business value is shaky as the result of those stakeholders decision that documentation wasn’t necessary – in no way was the Agile process faulty in that respect.

Finally, you may be surprised the holes you find in your own thinking when forced to write down and explain to someone just coming in “how we do things around here”, I know I was. I can tell you that in the cases where I did that exercise, several stupid, costly mistakes were avoided. Fleshing out your thinking isn’t necessarily big design up front (BDUF), it’s just smart.

The most interesting thing is that the knowledge investment you make in paratechnology has a (much) longer lifespan than that in technology directly. Those of us who have perfected our skills in VB6 found that .NET obseleted much of that. The same is true for Microsoft’s Web Service Enhancements, EJBs, etc. Even the languages themselves are changing substantially – as are their runtimes. However, object-orientation seems to be holding its own well into its 3rd decade. I suspect Domain-Driven Design (DDD) to enjoy a long life as well. Beyond just being a lifetime issue, these paratechnological entities just seem to build on each other more and more. In order to understand higher order patterns, lower one need to be understood first. These patterns increase the scale of the problems that can be solved repeatably. 

It has been my experience that those developers with similar experience-in-years, but only technological knowledge, are less productive than their peers with more paratechnological knowledge. I measure productivity in terms of number of feature points per unit of time. I understand that this is biased against infrastructure developers in application development teams – but I haven’t figured out how to measure them yet (at least, not well enough to say anything about it).

Therefore, it is my thesis that developers should spend at least the same amount of time investing in paratechnological learning as technological learning. It is my suggestion that a 3-to-1 ratio would be even better – as in only a quarter of your time on technological study. I submit that the on-the-job work done with a given technology, including the figuring out how to do something that you haven’t done before, is enough. When I’m talking about “study”, I’m referring to courses, conferences, books read (not when looking sometime up). 

For me, the way that I find out (most) of what I need to know about a technology (not necessarily new), is to grill an expert on it. For instance, at TechEd Developers Barcelona 2006, I had the good fortune to sit with the Workflow Foundation guys for at least an hour, the WCF guys for about 3 hours total, WPF for half an hour, and the CAB guys for almost 4 hours – not all in one sitting Anyway, it was mostly me asking how to solve thorny issues I’m having difficulty with in my projects, and them trying to explain to me, in a way I’d understand, that most people don’t have my problems. Personally, I think that, if that’s true, “most people” just haven’t noticed yet There were also numerous new ways of doing things that I didn’t consider to be an issue, so I tried to focus my learning on other things.

And I guess that that’s my bottom line. When new technology comes along, you shouldn’t need to “start over”. The way that I design my systems, most technology is hidden away. WPF changes how my views are implemented – but that’s tucked behind an interface so that my supervising controllers don’t care. WCF changes how messages are transferred over the network, but that’s tucked away behind an “IBus” interface; message dispatch is also abstracted with “IMessageHandler” and “IMessage”.  By the way, I find WPF to be very interesting in improving the human-to-human interface between Human-Computer Interaction (HCI) professionals and developers. 

So… Invest wisely. Compound interest is your friend.

First of all, let me say that design does not necessarily begin and end with OO. The quality of a given design needs to be measured against both functional and non-functional requirements. Beyond the original OO, other principles have emerged that positively affect design – Arnon blogged about these a while ago. The Single Responsibility Principle being one that keeps you thinking, “should that code go here?”. I also couldn’t design without leaning heavily on the Interface Segregation Principle and the Dependency Inversion Principle. Together with OO, I have found that these principles lead to more modular, flexible, loosely-coupled, and (yes) testable designs – all good things. Furthermore, we are continuing to uncover more areas where the original OO thinking does not serve us too well – distribution is a big one, so is persistence.

On the topic of unit testing, it is OK to do state-based testing instead of directly calling methods on the class under test. I find this happening quite often with my “controller” classes which do most of their work in private event handlers. I could make those methods public for testability’s sake, not that it would change the design much since no other class interacts directly with the controller. I prefer to activate the controller as it would in production, and that’s by having my mocks raise events. The assertions I make are done upon the state the controller changes  – in my (mock) repository. I believe that this style of testing does nothing to compromise the “unit-ness” of the tests and addresses many of the points Eli brought up in his post.

Anyway, that’s my 2 cents and I hope they make sense, and bring some down-to-earth sensibility to those wondering “are we asking the right question?”.

Enjoy.

Two assemblies, one for the interface, one for the implementation, “tests” go in the interface assembly. For example:

Interface project: Udi.Collections

File: IStack.cs

Content:

public interface IStack
    {
        void Push(object o);
        object Pop();

        int Count { get; }
    }

File: StackBehavior.cs

Content:

public class StackBehavior : BehaviorSpecifier<IStack>
    {
        [Behavior]
        public void First()
        {
            int count = this.thing.Count;

            object o = 1;

            this.thing.Push(o);

            Assert.AreEqual(count + 1, this.thing.Count);

            object result = this.thing.Pop();

            Assert.AreEqual(o, result);
            Assert.AreEqual(count, this.thing.Count);
        }
    }

Implementation Project: Udi.Collections.Implementation

File: Stack.cs

Content:

public class Stack : IStack
    {
        #region IStack Members

        public void Push(object o)
        {
            list.Add(o);
        }

        public object Pop()
        {
            object result = list[list.Count - 1];
            list.RemoveAt(list.Count - 1);

            return result;
        }

        public int Count
        {
            get
            {
                return list.Count;
            }
        }

        #endregion

        private IList<object> list = new List<object>();
    }

***

Then, right click on the implementation project, and select ‘Verify Behavior’. The results appear just like regular tests run with TestDriven.net.

Some reasons why this is better:

1. Tests (behaviors) are dependent only on the interface, which reduces the cases where developers will have to refactor test code as a result of implementation changes.
2. Should a new implementation come along, we can reuse the tests already written with zero effort.

If you’d like to give this a try (using .net 2.0), leave me a comment and I’ll send you the plugin for TestDriven.

The first was about TDD and the GUI, hosted by Philip Nelson where he set the tone by commenting on his experiences with MUCH decreased ROI when using TDD in the GUI. It was then that I began to feel that unit testing is really trying to be more than one thing.

(This isn’t anything new, of course. From “unit tests”, to “state-based tests”, to “collaboration tests”, going on to “integration tests” and “regression tests” we’re writing the same style of code and running it with the same tools.)

It really hit home for me in the session I hosted with Mats about DI and IOC. When focusing on how DI changes the style of unit tests we write, Claudio mentioned that the example tests I was writing when not using DI were really state-based, and not unit tests. At first I thought, “oh, I should change that”, but after taking another look, I realized that I really REALLY liked the similarity of state-based test code and use-cases/stories.

I realized that state-based tests are a lot more robust to changes than classic white-box unit testing or collaboration testing. This prevents situations (like the one Philip presented) from occuring. Unit tests aren’t free, or that close to it, for that matter. Anytime you’re refactoring an implementation of a class, or the way it collaborates with other roles, but aren’t changing its interface, state-based tests won’t break – your run-of-the-mill unit tests probably will. I know mine always did.

The final point that brought it all together for me, in light of Dan’s thought-provoking Behavior-Driven-Development, was that state-based tests are really Behavior Specifications.

“This is how this class should behave.”

And then the light bulb went off over my head.

These state-based tests really should be able to run on any class that implements this interface.

In essence, these “tests” are as much a part of the interface as the actual interface declaration. The interface is really just a static declaration of capabilities, whereas the behavior specification is the dynamic one. Both of them belong together, in the production code…

As it appears, most developers I’ve ever met who TDD do, in fact, separate their code into “layers”, and do not refactor their way there. In their experience, this idea of separating code of differing responsibilities has been proven valuable enough times to just start with it. No need to re-invent/discover the wheel using TDD.

By this logic, those who have stated programming to an interface violates the “simplest thing” principle do not believe that it has been proven valuable. In that case, I guess that the argument just becomes “I believe” versus “You believe” in which case, I concede the point. Just as most of us have reached the conclusion the TDD has been proven valuable, some of us believe that mocking is similar.

So, my only recourse is to beseech you to give it a try. It has served me SO well that I would strongly recommend it to anyone who TDDs.

See, I’ve bolded and italicized, so I MUST be right

The good news is that it doesn’t really matter.

I can hear you all gasping.

The reason it doesn’t matter is that we don’t have to be 100% right the first time is because we’re going to be changing it over, and over, and over again. In fact, if we took the easy path “by mistake”, as we move along writing more tests, making them pass, refactoring, the code will “correct itself”. The same thing will happen if we stray from the one true path in any way.

The nice thing about the one true path is that you don’t have to be right on it. TDD and the XP tenets are rules of thumb that are like magnetic fields that push us back on the path when we stray a little too much.

Change is expected. Change is good. Change is what makes “the simplest thing that could possibly work”, in all it’s clear-as-mud-ness, really work, without forcing developers to agree ahead of time what it actually means.

You all know the pain of a large refactoring, not only is the production code changed, but lots of test code has to change too. Interfaces minimize that pain.

So, although working with interfaces is not the EASIEST thing to do, overall it is certainly the simplest. And, to tell you the truth, it’s not THAT big a deal, I mean, you’re already adding the methods to the class, why not add them to an interface as well, and make the “client” class depend on the interface instead? And about the whole “should an interface start with a big ‘I’” debate, it’s a coding convention issue – choose something and stick with it.

One of the classic problem cases in TDD/XP is how to approach a legacy code base that has no tests. Lots has been written about this topic, but I just wanted to talk about the bootstrap problem:

You’re afraid to make any changes to the code without tests.
How can you write unit tests for legacy code without changing it even the slightest bit – considering it wasn’t ever designed to be unit tested and is probably a big tangled mess.

The answer, as always, is more abstraction!

So you can’t write unit tests. Boo hoo. Get over it. Start by writing functional tests, use FIT or whatever else you like. Start at the outside and work your way in.

Once the functional tests are in place, you can start refactoring the design (I wouldn’t say) safely (but probably safer) into units that can then be unit tested.

Is it perfect? No. But its better than it was before.
Will there be bugs because of the changes we made? Yes. We’ll just have to catch them later.

All this sounds like a lot of pain, and it is. But it’s nothing compare to not doing it. Do you really want to mess around with a whole bunch of code that worked before without something telling you if it’s still working after? I know I don’t.

Remember the tenet of XP called Courage? Legacy is a great place to work on it

First off, let’s set some context. We’re talking about software projects that require a team of developers (let’s say at least 2) a significant amount of time (let’s say at least 6 months) to develop. In these sizes of projects, TDD is undisputably cost effective. Now, before getting to the question of mocks, let’s just go back to a basic truth.

Code to an interface, not an implementation.

You’ve heard a million times. I know I have. It is generally considered a “best practice” – whatever those are. For my part, I try to follow this guidance as best I can. What it usually results in is a significant number of interfaces, with a not-so-slightly larger number of implementation classes. This is usually in the first iterations, slowly moving to a larger number of implementations to interfaces.

In these cases, implementation classes rarely are aware of each other. For the most part, they are only aware of interfaces.

Now, mock objects are just a way to get an object that implements an interface so that you can use it for testing. So, if you are testing a given class that depends on N interfaces, you will probably need to mock up an object for each one of those interfaces to thoroughly test the class.

So, my question is: What’s the big deal?

My answer: If you’ve already followed to above principle, mocks are just what you need, and require no rework at all to use. On the other hand, if you did NOT follow the “code to an interface …” principle, in order to use mock objects, you’ll have to do a lot of rework; amounting (pretty much) to changing your code to follow the “golden rule”.

If anything, mocks become a great canary. You won’t be able to use mocks (your canary will die) if you didn’t code to an interface. If you run into a case when you can’t mock, it’s a sign that the code needs to be cleaned up anyway. Don’t blame the canary for dying when it takes a whiff of your code.

My understanding of TDD and mocks is that mocks are used to keep the unit testing in and around the unit. What that means is that when I’m TDDing a class that uses another class, I don’t want the results of my tests to depend on whether or not the other class exists. Let’s look at an example to make this more concrete. Say I’m TDDing a Customer class, which has a property called IsPreferred. This property calculates the sum of all the customer’s orders this past year, and if the sum is larger than some value, returns true. So my test code might look like this:

[Test]
public void IsPreferredCustomerShouldBeTrueWhenSumOfOrdersIsLargerThanX()
{
Customer c = new Customer();
c.AddOrder( new Order(X) );
c.AddOrder( new Order(1) );

Assert.IsTrue(c.IsPreferred);
}

and the code itself might look like this:

public class Customer
{
private List m_orders;
private double m_preferredOrderSum;

public bool IsPreferred
{
get
{
double result;
foreach(Order o in m_orders)
result += o.Price;

return ( result > m_preferredOrderSum);
}
}

public void AddOrder(Order o)
{
m_orders.Add(o);
}
}

The problem with this is that in order for this code to work, I must already have an Order class. In essence, I’m not really unit testing here, I’m actually integration testing – and we all know that integration testing should come only once we’ve finished unit testing all the classes participating in the integration. So, we’ve got a slight problem here – there’s no way to test the Customer class without the Order class, or is there?

Suppose that we had an interface called IOrder, which has a property called Price. We could make the Customer class dependent on the interface instead of on the Order class implementation. IOrder would look something like:

public interface IOrder
{
double Price { get; }
}

and Customer would then change to:

public class Customer
{
private List m_orders;
private double m_preferredOrderSum;

public bool IsPreferred
{
get
{
double result;
foreach(IOrder o in m_orders)
result += o.Price;

return ( result > m_preferredOrderSum);
}
}

public void AddOrder(IOrder o)
{
m_orders.Add(o);
}

}

And as a result, I no longer need to have the Order class at all in order to test the Customer class – all I need is something that implements IOrder. Mock objects are great for this, allowing us to quickly get an object that implements an interface without writing much code. This allows/forces me to change my test code to something like this (using NMock syntax):

[Test]
public void IsPreferredCustomerShouldBeTrueWhenSumOfOrdersIsLargerThanX()
{
Customer c = new Customer();
Mock firstOrder = new DynamicMock(typeof(IOrder));
Mock secondOrder = new DynamicMock(typeof(IOrder));

firstOrder.Expect(“Price”); // expect that one call will be made to the property
secondOrder.Expect(“Price”); // expect that one call will be made to the property

c.AddOrder( (IOrder)firstOrder.MockInstance);
c.AddOrder( (IOrder) secondOrder.MockInstance);

c.IsPreferred; // this should perform one call to the Price property for each of the orders

// verify that the expectations we set (that the Price property would be called once per Order) were fulfilled.
firstOrder.Verify();
secondOrder.Verify();
}

So, what have we achieved? True unit testing. We are now able to test the Customer class as a unit, without depending on any concrete implementations of other classes. As you can see, my test code has changed considerably as a result, but is now independent of my Order class. This is a GOOD THING, since one of the problems I find when TDDing without mocks is that a change in the design of one class (Order for example) breaks tests for totally different classes (Customer for example). When TDDing, I’m always changing the designs of my classes, and having to fix tests that break is something that slows me down – mock objects decrease the friction.

Some have began calling this style of TDD – mock driven development or MDD, but I prefer to say that mocks are the thing that keeps the “unit” in the unit testing in TDD.

Another one’s infected: Don Park writes about his first experiences with TDD – “Test-Driven“. I think that this practice will soon reach critical mass. One day, not so far away, not doing TDD will be like not writing OO code.

What’s needed for this to happen are better tools – tools that will make TDD the point of lowest potential; meaning that things will just gravitate to it, naturally. Just as its easier to sit than to stand, so the tools will need make it easier to TDD than to keep doing things the way you’ve done them up until now. We’re not there yet, but in light of the “recent” backing of Microsoft in Visual Studio 2005, hopefully things will start to snowball.

TDD is the future, and the future is now. Embrace it.

<aside>I’ve always been told to end with a call to action, so that’s it </aside>

This will be my last post for tonight, I swear

During the break at the UG, I started chatting with Leon Langleyben (yes, it did take me 7 tries until I spelled that right) about TDD.

I stated that, as opposed to XP, TDD’s entry barrier would quite low and could still be quite effective. Leon countered by saying that getting the customer to sign-off on it might be problematic seeing as it increases up-front development time, or rather, decreases the rate at which the customer sees new functionality being developed.

I agree that this is true in cases where there is a high level of transparency into the development process. However, I believe that it would be easier to convince anybody to use TDD than it would be to convince them to use XP.

Specifically, one of the cases that Roy stated was problematic for XP was fixed-price projects. Now, I don’t entirely agree on this, however, in this case, nobody needs to know you’re using TDD, besides (maybe) your immediate manager, so it should be quite a bit easier to start easing in XP practices on at a time once TDD is in.

I’m still grokking this, but I think that TDD might just be oil that will allow XP to ease into an organization. Anybody whose had experience doing this, drop me a line and let me know. I think that it would be most valuable to me, and to a lot of people I know, so share the love

My recent post about XP and TDD sparked some contraversy on the extreme programming group and the discussion is worth while.

BTW – the title is a quote from my favorite teacher.

I think that I’ve already said this about a million times, but it bears repeating. Test driven development does not a well tested system make. OK, “So what good is TDD then ?”, you ask.

First, let’s prove my original claim. In order to test a simple method like “long add(int a, int b)” you have to write a ridiculous amount of tests. Checking for range and sign are just the start of it, because that will at best tell you that it’s behaving ok. But what about performance ? Is it performant enough ? Does it behave in multi-threading situations ? What about when you’re out of memory ? And on, and on.

In order to properly test anything, so that your as close to 100% sure as you can that any possible use of the code will work as expected, requires a lot of thought, time, and code.

If you’re going to do all this on every bit of code you write, you’ll never finish anything ! And, of course, there will be some tests you forget to write, so you can’t really be sure that your tests will catch all the potential problems.

The TDD gurus will tell you that TDD is not about writing every possible test for every bit of code. BTW, studies have shown that testing is an inferior technique for reducing defects in code than code inspection. Of course, XP has pair-programming for constant code inspection for exactly this reason.

So, if TDD doesn’t give you a well tested system or particularly reduce defects, why the hell should you run around writing all these tests ?

There are 2 (main) reasons.

1. TDD is a “design” technique. It makes you think about how you’ll use your code, before you write it. This leads to higher quality code. Coupled with YAGNI ( you aren’t gonna need it ) from XP, and you prevent over-engineering.

2. TDD pushes the creation of many tests that show that the system is behaving as expected under given assumptions. This is important ! Document your assumptions. In code. In your tests. As assertions. Without the assumptions, you’ll be bumping your way around a dark room forgetting everything after each bump.

But here’s the point: The tests act as change detectors.

Whenever something changes ( usually one of the assumptions you made has now been found to be false as a result of some other code being written ), the place where your assumption appears, blows up.

Hooray !

There’s nothing better than the code standing up and yelling “Hey, over here ! This here’s broken ! Come fix it.”

On the other hand, it could be that the assumption is still valid, and then the person that wrote the code will see that their use of it is wrong. Hopefully sooner rather than later. Continuous integration would let them know right away. ( Getting pretty obvious why in XP you see all the techniques together – by themselves, they have holes, together, each covers the other’s holes. )

So, although TDD is a good technique, it doesn’t give much in the way of actually testing the system. Enter XP, yet again. Integration testing, on-site customer that checks (tests) that a story is complete, all contribute to the “coverage”. However, one of the places that XP does not have a technique built in, is when it comes to serious testing. Performance, scalability, out-of-resource situations are all areas which must be specifically tested for. They are not part of the rhythm which makes up XP.

This then defines a family of systems for which XP automatically does not quite fit. Systems which have stringent performance or scalability requirements will need to have the XP methodology tweaked so that they don’t run into problems.

One common workaround is this – create customer stories for performance. Its simple, easy, and doesn’t break the rhythm. But woe to those who forget. That’s the only reason why I bring it up, because XP doesn’t take care of it for you like all the rest.

Final point: if you’re doing TDD, but not XP, you’re missing out. XP covers TDD’s ass.