As I said in the article, you must assume that there are some tests that weren’t written, in which case there is no guarantee of correctness. Striking a balanace between different kinds of tests is difficult (as you said) but is important – and no, not all tests need to be automated necessarily.

On your statement:

“it took me a few years to learn test-driven design to the point where I really could benefit from it”

I’ve found that to be common. If Design-for-Testability advocates would include the fact that this is a long process involving deep skill acquisition, I wouldn’t have any issue with it.

What I don’t like is the message that if you have “unit tests” for your code, and your code makes use of interfaces, you will magically get all kinds of benefits.

Anyway, thanks for your comments.

Read books written by testers for testers – James Bach is one of the good authors in that space.

True, it took me a few years to learn test-driven design to the point where I really could benefit from it. But once I and a few other senior developers had that experience, disseminating the skills to level where we experience benefit require only a few months of pair programming and occasional code dojos.

Once we have a part of our code properly covered with non-coupled tests, the added nimbleness gives us the confidence that we can design for today’s requirements and adapt the code to meet future needs. Since we practice incremental and iterative development, this is essential.

Great post. I was wondering if you would be able to recommend any books on testability issues.

Thanks.

Your statement “It’s simply a matter of developer training and ability” is almost right. I’d remove the word “simply” from that.

Is it easy to get developers with the right level of ability?
Is it easy to train those developers that are at lower levels of ability?
And what do we do in the meantime?

I’d say that “SRP, DRY, and loose coupling principles” fall under Good Design rather than Design for Testability. As I said in a previous comment, well designed code tends to be testable, but the inverse isn’t necessarily true.

There is lots of value to be gained from well designed systems – I agree. It’s just that we need to understand that that doesn’t happen by itself – hiring practices need to be changed, as do training practices, and more. It is a broader organizational issue than can be solved just by developers writing “unit tests”.

I think that most people don’t divide up the system at the right level of granularity (services, business components) which results in domain models which are too large, in which case they’re not a unit either

When talking to clients about unit testing, I rarely get to the point of discussing the cost of *just* unit testing. Rather I focus on helping them find the right mix of the right kinds of testing throughout the lifecycle of the project – along with getting the right kinds of design decisions, requirements, etc…

Cheers.

As I mentioned in the previous comment, poorly designed systems have all kinds of hidden dependencies that unit tests don’t target. Changes in one part of a system still can break other parts without unit tests going red. In which case, we’re operating under a false sense of security. Is that really such a good thing?

The purpose of refactoring is to improve the design of the code while maintaining its behavior. It is unlikely that that is the highest business priority after a year of production. It is more likely that maintenance programmers will be fixing bugs and adding features – both of which change behavior, ergo not refactoring.

While I agree that a good suite of unit tests could help in refactoring a well designed code base, there is no inherent assurance that the unit tests are good, or that the code base well designed.

Hidden dependencies through shared databases and web services are often the places where design and unit tests break down and refactoring doesn’t necessarily help.

Cheers.

The term “testable code” is open to interpretation. If the test in question is just asserting that the code is implemented a certain way, “testable code” just means “more code” and is actually a violation of the DRY principle.

I agree that good code is often easy to test, but the inverse isn’t necessarily true – not all animals are dogs

This stuff isn’t easy, nor is it necessarily easy to teach.

Much of what you’re saying is “accepted truth” in the industry, but in my consulting I have seen a very different reality. Just like not everyone who writes code can write good code, not everyone who writes “unit tests” does that well.

The assumption that every unit test written is good does not hold up under scrutiny. In which case, it isn’t necessarily catching or preventing bugs – it’s just more code to maintain.

All I’m saying is for us to be more conscious of the investment choices we’re making, and following up with appropriate reflection from time to time on the return on those investments with real evidence rather than faith.

Thanks for your comments.

Thanks for your comments – I think we’re very much on the same page. If a certain bias for iterations exists in the post, that is more of a reflection of the industry than what I believe is best. That being said, I do temper my guidance very much based on what an organization is willing and able to hear at a given time, so I may recommend iterations from time to time as the political realities dictate.

Designing for Testability means adhering to SRP, DRY, and loose coupling principles. Over just about any time span, these principles SAVE time and therefore budget. Even if you don’t write any tests at all, the benefit gained from designing your code in the “testable” way are immense.

Have you ever informed your employer about possible paths for the project? For instance: ‘plenty of tests will cost you additional two week’, ‘without these we can finish the project on time’ and so on?

Take care
Szymon

Your ROI is more than that – each time you run the tests you gain value. You either know sooner that you’ve broken something or have confidence that you hopefully didn’t. The tighter feedback loop *is* your ROI.

You’re missing a very important use case for unit tests (or component/integration tests): refactoring! after a year of production you’ll usually have gathered so much more insight into the business domain that you’ll have to refaactor either to implement new features or to improve the performance. having a suite of unit tests will help you achieve this task more quickly.

g,
kris

Good stuff, I have been reading your post since last few years, also many times I quote from your post.

Going through “On Design for Testability”, I see few things differently, I considered the Unit testing as “white-box” testing — the tests are performed with full knowledge of the source code. Ideally, developer would be able to run unit tests so that all the code paths would be executed as a result of the testing. In other words, the unit tests would need to create the conditions to go through each line of code to ensure all the code was operating correctly today and tomorrow.

Unit Test is very critical because it is one of the first testing efforts performed on the code and the earlier defects are detected, the easier they are to fix. Early bug-detection is also the most cost-effective methodology, earlier we catch them less it would cost and later they are found higher the cost.

Also when we say Unit-testing should only be attempted with things which are units (or components), this is very subjective and depends on how much capital we want to spend, but I would always prefer to have all the new code written to have unit tests coverage, as an Investment in Future.

Basharat

That said, I’ve got a lot of experience in balancing internal testing with external testing and my own experience doesn’t jive with all that you’re reporting.

Ultimately, it comes to a matter of competence and learning, which you point out in conclusion. Any organization not learning from their practicing is going to end up with sub-par results on anything they do. Any organization creating fragility – whether designing a marketing strategy or designing a software unit is going to have pain.

I completely disagree with the iteration bias. We’ve begun to abandon iterations and optimize team organization and work management and design beyond what Scrum’s assertions can effectively hold up to. For me, this started more than two years ago, and my work management and organization management expectations are quite a bit different from what they used to be. This includes how we’re making use of testing and the meaning and purpose of testing and its effect on productivity. In the end, I don’t think “agile” has anything to do with this. “Agile” has become so watered down that it’s mostly meaningless at this point, and often an unproductive distraction except when rehabilitating very poorly managed software organization who are still trying to overly manufacturing organizational mechanics and values onto software development. Even then, I’d rather go straight to iterationless work management and control without stopping at the Scrum step.

Ultimately, doing things poorly usually means having poor results, and doing things goodly means having good results. This is the ultimate mitigator, and the fruit that most organizations fail to harvest whether in software projects or other aspects of business development and operations.