Tuesday, May 31, 2005

Taxonomy of Tests: Everything Else

While the majority of your focus on an agile project is on either unit tests or acceptance tests, there are quite a few other types of useful tests that fall through the cracks. All of these types of tests can and should be part of your continuous integration strategy.

Developer Written Partial Integration Tests

Bigger than a unit test, but smaller in scope than an acceptance test. That's a big mouthful for a category, but I've never seen any two development teams that use the same name for this broad category of tests. For multiple reasons, most of my team's unit tests are disconnected -- i.e. databases, web services, and the like are mocked or stubbed. I want to test each class in isolation, but they have to interact with the real services too. A secondary battery of developer tests that test partial code stacks are very advantageous in uncovering problems in the actual interaction between classes or subsystems. I've known some XP developers who actually emphasize these semi-integrated tests more than the disconnected unit tests. I like to keep the types of tests in separate test assemblies, but that is just preference. These are really hard to describe, so here's a couple of ideas I've used or heard of --

  • Testing a view and controller or presenter together. I would still try to mock out the backend business services from the controller in order to focus on testing the integration of view and controller. This is especially important in the new Model-View-Presenter mode of UI development.
  • Testing from a UI controller down to the persistence layer.
  • Test from the service layer down. SOA is surely overhyped and overused, but it presents some great opportunities for automated testing. Because the .Net SOAP serialization is a bit brittle I always put automated tests in for the web service proxy class communicating with a local web service. It's also good to test the service code without IIS being involved.

Environment Tests

Environment tests verify that a build or test environment is fully connected and correctly configured. I helped do a presentation on continuous integration to the Austin Agile group last week and we spent quite a bit of time talking about this very topic. The gist of the slide was preventing lost time by testing teams by uncovering disconnects or failures in the testing environment at code migration time. Nothing is more aggravating than reported defects that end up being merely a mistake in code migration. I spent a week one time getting grilled because a data interface wasn't working correctly. It turned out that the middleware team had shut down the webMethods server in the test environment and not told the development team. Putting aside the total incompetence of everybody involved (including me), had an automated environment test been in place to verify the connectivity to webMethods, a full week of testing would not have been wasted.

I had some periodic trouble on a project last year with maintaining various copies of the system configuration (database connections, file paths, web service locations, etc.) for development, build and test environments. To combat this problem in the future I've actually added some support into my StructureMap tool for doing environment checks and configuration validation inside of an automated deployment. We sometimes move code a lot faster in agile development and it provides a lot of opportunity to screw things up. These little environment tests go a long way towards keeping project friction down.

I got the name for this off Nat Pryce's post here.

Smoke Tests

Smoke tests are simple crude tests to exercise the system just to see if the code blows up. They don't provide that much benefit, but they're generally low effort as well. On my current project we're replacing a large VB6 component with a new C# equivalent. One of the things we did to create a safety "tripwire" effect was to run a huge amount of historical input XML messages and configurations through the new C# engine just to find exceptions. We found and fixed several problems from edge cases and some flatout bugs. It was definitely worth the effort.

Regression Tests

My last two projects have been more or less rewrites of existing legacy code. Oftentimes hairy legacy code isn't understood very well by the organization (hence the rewrite). Twice now I've had to jury-rig the legacy code to create repeatable automated tests that simply prove the new code creates the same output for known input. It's not entirely a good test because it has no connection to the intent of the code, but both times it found problems. The irritation level was high because the legacy code wasn't pretty or easy to get running, but otherwise unknown defects removed and fixed made it worth the effort. Automate the regression tests in the build if you can. If nothing else, there are always business processes and requirements buried in the code that aren't known or documented anywhere else.

"Build Time" Validations

There are always some aspects of the system that the compiler and maybe even the unit test library can't validate. Use the automated build as an opportunity to create "build time" checks for potential problems or enforce coding standards. Here's some ideas of things to enforce in the build:
  • A compiler warning or "TODO" threshold. I've heard of people making a build fail for this.
  • A limit on ignored unit tests
  • SQL validator. Gotta keep the sql external to the code somehow, though.
  • O/R Mapping. We're starting to adopt NHibernate for persistence. One of the first steps will be validating the mapping in the build
  • Automated code coverage is kind of a test of your tests
  • Static code analysis

There's probably more, but that's all I can think of. I'd love to hear other ideas.

Tuesday, May 24, 2005

Taxonomy of Tests: Acceptance Tests

From Ward Cunningham's wiki: "A formal test conducted to determine whether or not a system satisfies its acceptance criteria and to enable the customer to determine whether or not to accept the system."

The acceptance test is exactly what it sounds like, it verifies that the implementation of a user story works as the customer requires. In a literal interpretation of XP theory, the customer writes the acceptance test. In reality, the acceptance tests are going to be written by some combination of the business partner, testers, business analysts, or product managers. It really doesn't matter to me, as long as they are done in a timely manner with some degree of correctness. You'll never have the ideal XP customer, but it doesn't necessarily mean the project is hopeless.

It is a huge advantage when acceptance tests are approved and ready during coding. User stories, use cases or huge paper System Requirements Specification documents -- the best and most effective requirement is a cut and dried acceptance test. When the coders can (and do) easily execute the acceptance tests themselves, the bug counts will go down. The development team as a whole can use the acceptance tests as the “Red Bar” metric to watch during an iteration. A user story is “done” only when all the finalized acceptance tests pass. James Shore has a good post on acceptance tests as requirements here.

Developer/Tester Cooperation

Testing features during an iteration as soon as the code dries is a tremendous advantage. It's much, much simpler to fix bugs that were just coded before the memory of the code fades and the bug is covered up my much more code.

My current project has not had acceptance tests written during iterations, and it has clearly hurt us in terms of wasted effort and misunderstood requirements. Never again.

Organizational issues between testing and development are not specific to agile development, but the desire for rapid iterations will expose any friction between requirements, code, and testing. Moving fast without sacrificing testing requires the ability to cycle code quickly between development and testing environments. Code that is easy to test helps quite a bit, too. Developers have to support the testers with the mechanics of testing. Both testing and development can benefit from a good Continuous Integration (CI) strategy. Testers should be involved in the CI effort. I feel very strongly that it is the developers’ responsibility to ensure the test environment is in a valid state. “BuildMaster” folks are a desirable luxury, but I always want visibility into the test server environment.

Automating Acceptance Tests

Acceptance tests are a good and necessary thing. As far as testing is concerned, an automated test that runs often and visibly is far more valuable than a slow manual test buried in a test plan. A common objection to iterative development is the need for so much regression testing. The best answer to this perfectly valid objection is to automate as much as you can.
Enterprise applications are rarely finished; they're abandoned when they are judged to be too expensive to change. Automating acceptance tests is a great way to extend an application's lifecycle by reducing the cost of regression testing. Also, to paraphrase a former colleague -- "If your tests aren't automated, how do you know they're passing?" Think on that one.

Who writes the test is one decision. Who automates the tests, and how, is a different decision. Hopefully you're lucky enough to have testers with programming or scripting abilities. Even if you do have technical testers, plan on the developers assisting the testers with test automation. I'm not a big fan of FIT and its derivatives, but it is definitely a step in the right direction for acceptance testing (I love the concept, but I hate the implementation of NFit). The FIT model allows for writing acceptance tests in near English, with some non-trivial work behind the scenes to complete the wiring for the custom FIT fixtures.

Conventional wisdom says that acceptance tests are black box tests that test the system end-to-end. This doesn't necessarily have to be the case. I’ve seen both Brian Marick and Brett Pettichord write about using white box tests for acceptance tests, especially for user interfaces. Using Service Orientation principles to create a cohesive service layer can be a great aid for automated acceptance testing. I think (N)FIT chokes on user interfaces, but it is perfect for driving an isolated service point (note I didn’t necessarily say web service) in an automated test. These white box tests can be much easier to setup and execute. Some flavors of MVC architecture allow for slipping in a screen proxy in place of a real UI view (I've had mixed results with this). Again, it is not a “perfect” black box test, but it goes a long way towards validating the behavior of the system.

Good testers can work with developers to write these automated whitebox tests. It does take pretty good communication and cooperation between coders and testers to take advantage of these opportunities for test automation. It is definitely advantageous for the testers to be familiar with some of the inner workings of the system. I definitely think agile development starts to blur the line between developers and testers, and that's not an entirely bad thing.

Once and for all, software is not like construction or engineering.

Here's a great post from Coding Horror on Bridges, Software Engineering, and God.

I was a mechanical engineer before I wandered into software, and the comparisons between software development and engineering & construction always bug me. A software project has very little in common with a construction project, and we should stop trying to ram our square software peg into the construction hole.

Monday, May 23, 2005

Taxonomy of Tests: Unit Tests

At the most granular level is the unit test. Unit tests are written by developers to verify, on in Test Driven Development specify, the functionality of a very small piece of code to the satisfaction of the developer. Usually a unit test works on one class or a small cluster of classes at a time. One of the key attributes of a unit test is the ability to test a small unit in isolation. We want to prove that the unit of code itself is working without corruption or interference from other classes.

Purposely designing code in order to create isolated unit tests is part of a successful Test Driven Development effort. Mocks and stubs are an invaluable tool to create the isolation. Some people insist that all external dependencies (databases, Active Directory, web services, etc.) be mocked out in unit tests. My best advice is to consciously quarantine inconvenient things behind gateways with an abstracted interface. Use Inversion of Control to minimize dependencies from the majority of your .NET code to these gateways. When you can't eliminate a dependency, use Dependency Injection to provide a mechanism to replace the real thing with a mock or a stub. I have had very positive (well, mostly) experience with the Model-View-Presenter (MVP) pattern described by Martin Fowler as a strategy to maximize the testability of user interface code.

Unfortunately, some units of code only exist to manipulate the database or interact with infrastructure services like messaging. Not to mention that you still have to unit test the UI screens. MVP is not enough in and of itself to guarantee a properly functioning user interface (trust me on this one). You can either marginalize this code by not writing tests and hope for the best (or let the acceptance tests take care of it), or bite the bullet and write a battery of tests that are connected. My advice is to put some ketchup on the bullet and just do it.

I do prefer to separate connected and non-connected unit tests into separate test fixture assemblies. I generally lean on only the non-connected tests during refactoring or new development. I do think it is absolutely necessary to run the integrated unit tests in both your check-in procedure and the automated build.

There is a school of thought that says if you really unit test well enough against the mocks, integrated testing is unnecessary. The NEO project is the best attempt I've seen anybody use to actually approach this ideal. I still think this philosophy is unattainable and possibly dangerous.

I've been in some heated arguments on the merits of mocking ADO.NET when testing custom data access classes. I mostly vote no to mocking ADO.NET. In this case, the unit of valuable work is manipulating the data in the database. Who cares if the code sets the “expected” parameter values when the real stored procedure has been changed in the database behind your back? In this case, the effort to value ratio is all wrong. Of course, when you’re a consultant onsite at the client you work according to their goofy development standards and politely say “Thank you sir, may I have another?”

Next up, Acceptance tests...

Taxonomy of Tests: Defense in Depth

I made the mistake of posting on writing good unit tests while suffering from "Jedi Flu" and Jeffrey Palermo took me to task for not defining a unit test first, so here's my attempt at describing the various types and approaches of tests I've seen or used on agile projects. Keep in mind that I'm a developer first and only an occasional architect, “BuildMaster”, or tester second. The first thing to keep in mind is that agile processes don't change the fundamentals of software development. It is still best practice to test the code at multiple levels of integration (the old V-curve idea). It's great to mock the database in your unit tests, but the database is still there and just itching to get out of synch with your middle tier code.

A Taxonomy of Tests

Roughly, I would divide tests into:

  1. Unit tests
  2. Acceptance tests
  3. Intermediate/System/Integration tests, written by developers
  4. Other: smoke tests, regression tests, performance tests, miscellaneous validation

Testing Strategy

The answers will vary from project to project due to manpower constraints and the nature of the development, but any and every project is going to have to answer these questions below.

  • Who writes what tests?
  • Who automates the tests, and how?
  • Where and when do the tests run?
  • How do we want to organize the test suites?
  • How much testing is enough?

Sleeping with the Enemy

One thing I've learned that does not vary; it takes a great deal of cooperation and mutual respect between the testers and coders. Working in the same room is ideal. On projects where the tester and developers are separated by either organization or location, the development effort will suffer. For that matter, both developers and testers need quick and easy access to whoever holds the reins on requirements. At a bare minimum, don't look at testers as the enemy.

One of the truly great advantages agile development has over traditional waterfall projects is in team dynamics. There is far more interaction and communication between disciplines. Just being on the project at the same time is an advantage over “throw it over the wall” processes. Everybody’s common goal on an agile team is producing working software. Not code, tests, UML diagrams, or Gant charts, but working code that meets the customer’s expectations.

Next up, Unit Tests…

Friday, May 20, 2005

May 19th, 2005 - IT Productivity Hits an All Time Low

I'm finally recovered enough from the midnight opening of Revenge of the Sith to post some sort of review. I think I'll make a point of never going to a late movie on a work night ever again. My entire contribution to work Thursday consisted of suggesting Mexican food for lunch.

Here's a sorta review:

I loved it. There were definitely some movie mechanics things to nitpick about, but I had a blast. It moves pretty quickly. Even if the dialogue wasn't perfect, nothing stood out like a sore thumb the way Anakin whining or Jar Jar did in I & II. I think there is probably too much story for a single movie. You get the feeling that a lot of important events happened offscreen, and the progression to the dark side happens a bit too fast. I read a lot of people saying they came out of the movie and immediately wanted to go watch the original trilogy. I definitely concur.

The Good:
Jar Jar is rarely seen and never heard.
I liked Hayden Christensen in this one.
Yoda rocked
The Emperor nearly stole the show
The Obi-Wan/Anakin interaction was much better
R2 is still the best comic relief ploy
The visuals were stunning. The opening sequence was fantastic.
Lucas ties up a lot of plotlines at the end and sets the stage for the original trilogy
Wookies definitely trump Ewoks (or Gungans for that matter)

The Bad:
Some of the special effects look wrong. The clonetroopers look fake sometimes
IMO, It is too dark, violent, and emotional for small children. Take the PG-13 thing seriously. Much more violent than any of the others.

Wednesday, May 18, 2005

Qualities of a Good Unit Test

Many developers, including me sorry to say, treat unit test code as a second class citizen. After all, NUnit test fixture classes aren't going into production, so why should you put a lot of time, effort, and thought into them? An unfortunate reality is that unit tests are rarely "write-only" code. Badly written unit tests or interdependent unit tests hamper refactoring or adding new code to the system. I've heard of teams that were afraid to add new code to a system because the changes would introduce test failures. One of the advantages of automated unit testing should be the ability to safely modify existing code. Bad unit tests turn this advantage on its head.

I'm giving an internal presentation today on mock objects and I prepared a slide on the "Qualities of a Good Unit Test." I actually couldn't find much in the way of resources on the web for this, so I made up the list below. I'd love to hear from the rest of you what your best practices are for unit tests.

Unit Tests Should Be Atomic

A unit test should only test a small piece of functionality. This falls inline with the idea of getting into a rhythm of "write a little test, write a little code." A litmus test is to ask yourself if any part of the unit test could stand alone in a separate unit test. Another feedback loop for your unit testing quality is the amount of time you spend with the debugger. If your unit tests are coarse, a test failure is more difficult to find. If the unit test exercises a small amount of code, the test failure cause can usually be spotted very quickly.

Order Independent and Isolated

There should never be an order dependency, intentional or not, between unit tests. Problems arise when one unit test leaves some kind of dirty data laying around. Testing against a database or some sort of static cache is a common culprit. The theme of the day was mocking (actually Episode III), so put evil stateful things behind nice, deterministic mocks or stubs.

Intention Revealing

A unit test can, and should, be a valuable form of documentation. At best, a unit test should explain the intended usage and function of a class. At worst, the unit test should still be easy to debug in the case of a regression failure. Excessive data setup can obfuscate a unit test beyond any hope of comprehension.

Easy to Setup

How easily can I set up an environment to run the tests? If the answer is "not very," I might never get the tests running. As much as possible, decouple unit tests from external dependencies like databases and web services by using mocks or stubs. When you do test against an external dependency, the dependency better be setup correctly by the automated build.

Runs Fast

You're never going to run the test just once. For the sake of constant feedback, successful continuous integration, and your sanity, make sure your tests run quickly. Thirty minute build and unit test cycles are nothing but friction. Just to keep in the mock rut, mock things that make network calls to speed up your testing. I like to segregate connected tests into a seperate assembly. This way you can quickly run the majority of your unit tests on demand without the lag from network calls.

There's surely some obvious things I'm missing, but I'm short on sleep (and a midnight movie awaits) at the moment so that's all I've got.

Thursday, May 12, 2005

Architect: Person, Role, or Useless Appendage?

There is a good thread going (or was when I started this) on at TheServerSide.NET about "What Skills Should a .NET Architect Possess?" The correct question might be whether or not an architect is different from a senior developer. More importantly, how does the architect (or analogue) interact with developers? I've been (stranded) on both sides of the fence on this question and I'm still ambivalent, so I'll let everyone else decide.

At one pole is the heavy process waterfall shops with elaborate responsibility matrices. Architects do all the design upfront in comprehensive documents and developers code what they're told. It doesn't really work that way of course. When I was an architect (at least in name), I wasn't allowed to code. I was told that I was "too important to code." I wasn't important enough to be paid well, but just important enough to do lots of powerpoint presentations to kick off projects that were cancelled within a month.

On the other hand, I walked away very unimpressed from my first and only true Extreme Programming project. Doing the simplest thing that possibly works, with little or no emphasis on architecture or even design at all, can lead to truly awful code. I felt we consistently missed opportunities to become more productive by just stopping to think once in a while. We missed abstractions that could have helped us go faster. We waited to refactor to use an abstraction after it became pitifully obvious the abstraction was necessary. Refactoring is good, but there is no such thing as a big refactoring. Big refactorings are just rework.

A terrible amount of code was duplicated because every pair was so focused on their own stories. Pairing and rotating helps, but you're still only working on a user story at a time. Even worse is the fact that user stories are dribbled out to the developers, so there doesn't seem to be any way to get traction on a coherent design. Important concerns like instrumentation, data access strategy, operational support, and security can easily be lost in the hustle and bustle of trying to make the iteration.

I don't believe in very many absolutes (always do this, but never do that) in software development, but here's a couple I do believe:

  1. Non-coding architects create terrible architectures. If you can't code it, you certainly cannot design it. I had to come from behind one of my architect peers on a large project after he had been more or less been kicked off the project for being a pain in the rear. The project manager just looked a little dazed and said, "It was going so well at first. He was producing so many design documents and then..." Development just changes too fast to do the beard stroking architect thing. Turn around twice, and your COBOL/VB6/Pro IV skills have been passed by. It was pretty scary being a non-coding architect in early 2003 with no production .NET or J2EE experience.
  2. Development is often all about responding to feedback. Feedback from your code when you realize your design isn't ideal or a different approach proves superior, feedback from performance data, and feedback from the business people as they refine their understanding of their needs. Feedback needs to be acted on quickly in order to be valuable. Small refactorings done as the need arises greatly streamlines the code. Big refactorings put off lead to nightmares. Developers who aren't able to deviate from a bad design because of a lack of skill or authority are more or less helpless to create good code. Both times I've been the recipient of a design spec from someone else the "design" was tossed out as completely unworkable.
  3. Spec coders suck and cannot create any kind of non-trivial software. Being a spec coder sucks even worse. Being a non-coding architect working with spec coders is a living nightmare no one should have to endure. I've found to my chagrin that the more detailed your instructions are to another developer, the worse code they create. Give the same developer more leeway in their approach and maybe some amount of coaching, and their code is consistently better because they're thinking while they code. Everybody should be involved in design to better understand why they're doing what they're doing.
  4. Design should be continuous. I like Jim Shore's article on this topic. The key is to always be thinking and challenging your design and never get complacent. Keeping somewhat in line with agile ideology, do think ahead and anticipate needs and risks, but don't act upon them earlier than you need to. I'll blog much more on this some day. I don't really fit into either the XP or RUP crowds in my approach to design. It's the year 2005, the discussions should have moved on from "waterfall or iterative" to "how best to do iterative development" a long, long time ago.
  5. It's very advantageous to step outside of the current batch of stories or code and look at the bigger picture. If that requires tagging someone as the designated architect, so be it.
    You better have somebody (preferably plural) with real technical design skills involved with the coding effort on a daily basis.
  6. Regardless of process or approach, design skill is vital. Being able to quote Chairman Beck's little white book with a gleam in your eyes does not count as design knowledge. A working knowledge of design patterns and the fundamentals of good design can actually contribute to the success of a project. If all you know is write "if/then/else" statements until the NUnit tests pass, your version of the "Simplest Thing Possible" is probably a bad design.

"I feel better now, thank you for listening" -- Roy Moore

Friday, May 06, 2005

Presenting the Geek of the Day Award (off topic)

To help pass the time, a project team I was on as a consultant used to have an unofficial "Geek of the Day" award for whoever made the nerdiest comment or reference of the day. Obscure Star Wars quotes, Robotech, Tolkien, Dr. Who, and Dragonlance references qualified. We had to retire the award for a while when a business analyst (!) trumped us all by showing us the artwork he had published in Wizard magazine.

My new boss is paranoid I'm using this blog to rag on him and the mothership in Houston, so here it goes -- Chris Fields, I present to you the Geek of the Day for filling out your own performance self-evaluation as "Mostly Harmless." I'll take it over in a couple of weeks for faking a reason to make an overnight "business" trip to Houston just to see the midnight episode 3 premiere with the rest of the company.

Thursday, May 05, 2005

TDD Design Starter Kit - Ideas for Upcoming Posts

My loose plan for more posts in the 'TDD Design Starter Kit' is something like this:

  1. State vs. Interaction Testing
  2. Mocks & Stubs (probably put an example of NMock up - the docs aren't too great)
  3. Inversion of Control & Dependency Injection ( I've written and use a DI tool on sourceforge called StructureMap, so expect more than you ever want to know about DI usage, w/ or w/o a "container")
  4. Statics and Singletons Can (sometimes) be Harmful to Your Health
  5. Humble Dialog Box (Model-View-Presenter, I dislike Martin's name for this. It's just a variant of MVC)
  6. Favor Composition over Inheritance
  7. Isolate Hard-to-Test Dependencies
  8. Know When to Punt!
  9. Bare Bones NAnt/CruiseControl.NET

I'm intending this set of posts to be kind of an open forum for my new development team. I'd be plenty happy to take any suggestions on other topics.

Wednesday, May 04, 2005

Does TDD Make Development Take Longer? (No)

Overall, I'd say the answer to this question is no. Several people in the Austin .NET Agile Group are going through the typical growing pains of introducing TDD to a development organization. The very obvious objection to having to write and maintain much more code is being raised by developers.

Yes, writing automated tests can be laborious. I routinely observe myself writing every bit as much testing code as production code. However, I have found that writing code might take longer with TDD, but software development as a whole (design, code, test, rollout), is faster. After all, the real goal isn't slapping out code, it's getting working code to the business.

Here is where I think TDD cuts time in software development:

  1. Design. TDD is a low level design technique. It wasn't an overnight conversion, but since I've started using TDD I spend much less time with UML and other upfront design techniques. The feedback cycle between "I wonder if this design will work" to validating the design is much shorter.
  2. Debugging. As your ability to do Test-First unit tests improves, your usage of the debugger goes way down. I've definitely seen this in my coding over the last two years, and I've read may similar accounts.
  3. Fixing bugs. Bugs can be fixed more rapidly because the unit test coverage can detect regression failures before a tester does.
  4. Testing. Fewer bugs get through to the testing team. By doing test-first, you have to take testability into account in the design of your application. Non-coding architects doing upfront design may have no incentive to promote testability. From painful experience, if you use any kind of test coverage tool like NCover or Clover.NET, you'll find a nearly one-to-one relationship between code with poor unit test coverage and the code that spawns reported bugs.

Learning TDD

Learning TDD was difficult, especially for someone like me with a bit of background in UML modeling. It hurts at first, but TDD does get easier with practice. There is far more collective knowledge and experience with TDD in .NET today than there was a couple of years ago. I'm trying to use my "TDD Design Starter Kit" blog series as a start for my group.

Maximizing the TDD Advantage

TDD can be done with any process or organization, but it is more effective and valuable in an iterative and incremental process openly using emergent design. Every developer (or pair) needs to have some control over the fine grain design details. Spec coders need not apply.

Refactoring automation tools help tremendously. ReSharper (and Whidbey?) can generate method stubs for missing methods. That little piece of functionality made writing tests first much easier mechanically.

Continuous Integration (CI) is a natural extension of TDD. The TDD/CI combo is very powerful in reducing friction between development, testing, and production.

There are obviously some types of code that are much easier to verify manually instead of an automated test. In these cases you might compromise and get by with a manual test. A lot of effective TDD design is minimizing or isolating hard to test functionality to maximize the testability of the application as a whole. The Model-View-Presenter pattern ("Humble Dialog Box") is a good example of this. I'm a big fan of using mock objects for external dependencies.

Monday, May 02, 2005

TDD Design Starter Kit - Dependency Inversion Principle

In the last episode of the TDD Design Starter Kit, I talked about the need to build cohesive classes, and make the relationships between classes loosely coupled. The specific benefit for TDD is to truly isolate the functionality of a class under the microscope of a unit test. But we've done all we can to isolate our classes, and we still have some interaction with dependencies. How then can we test the class in isolation?

Use the Dependency Inversion Principle (DIP) to isolate one class from the actual implementation of its dependencies.

Depend upon Abstractions. Do not depend upon concretions

DIP is certainly not an invention of TDD practitioners, but it is employed frequently for no other reason than to enhance testability.

Take a look at this code that does not employ the Dependency Inversion Principle.

public class MessageProcessor1
public void RouteOrder(Order order)
string queueName;
if (order.IsComplete)
queueName = "Queue.Completed";
order.CompletionDate = DateTime.Now;
queueName = "Queue.Incomplete";

OrderQueue orderQueue = new OrderQueue();
orderQueue.SendOrder(order, queueName);

public void assignOrder(Order order)
// do something to the order

public class OrderQueue
public void SendOrder(Order order, string queueName)
MessageQueue queu = new MessageQueue(queueName);

So what's so wrong? The MessageProcessor1 class cannot be tested, or function, without the OrderQueue class being executed.

Not only is MessageProcessor1 strongly coupled to OrderQueue, it is coupled to anything that OrderQueu is dependent on. In this case the MessageProcessor1.RouteOrder() method cannot be tested unless MSMQ is available. More annoyingly, any automated test would need to query MSMQ to get the resulting Order object to compare the expected results.

Now, using DIP we make the MessageProcessor2 class dependent upon an abstracted IOrderRouter interface instead.

public interface IOrderRouter
void SendCompletedOrder(Order order);
void SendIncompleteOrder(Order order);

public class MessageProcessor2
private readonly IOrderRouter _router;

public MessageProcessor2(IOrderRouter router)
_router = router;

public void RouteOrder(Order order)
if (order.IsComplete)
order.CompletionDate = DateTime.Now;

public void assignOrder(Order order)
// do something to the order

MessageProcessor2 is only dependent upon an interface, and can use any implementation of the IOrderRouter interface (in theory).

Here's what a unit test for MessageProcessor2 might look like:

public void RouteACompletedOrder()
Order order = new Order();
order.IsComplete = false;

DynamicMock routerMock = new DynamicMock(typeof(IOrderRouter));

// SendIncompleteOrder(Order) should be called on the incomplete order
routerMock.Expect("SendIncompleteOrder", order);
routerMock.ExpectNoCall("SendCompletedOrder", typeof(Order));

IOrderRouter router = (IOrderRouter) routerMock.MockInstance;

MessageProcessor2 processor = new MessageProcessor2(router);

// Call to the DynamicMock to verify the interaction

Now we have our routing logic decoupled from the delivery mechanism. Besides testability, we have added the ability to substitute other delivery mechanisms like a web service, MQ Series, or simply writing to a file.

Mechanically, there is a couple things of note in the unit test above. I used a dynamic mock object with NMock as a stand-in for the IOrderRouter interface to test the interaction. The IOrderRouter instance was passed into the MessageProcessor2 constructor, not created by MessageProcessor2 itself. This is an example of Dependency Injection (Inversion of Control). I will try to blog on both topics soon.

Rules of Thumb to Use DIP

  1. Anytime your code calls out of the CLR, and maybe just the current AppDomain itself. I would isolate any kind of call to databases, messaging queues, web services, and remoting behind a .NET interface. In ASP.NET or web service applications, always abstract any kind of call to the HttpContext objects. Nothing gets in the way of automated testing like coupling to the ASP.NET runtime. Remember, one of your unofficial design goals is to test as much of the application as possible with your laptop disconnected from the network. Look at the Gateway pattern. As a mild rant, using a Service Oriented Architecture does not guarantee that your actual code is loosely coupled. I'll expand on that soon.
  2. Singleton's. Singletons can throw a pretty serious monkey wrench in your unit tests by maintaining state between unit tests. Here is one way to avoid the Singleton issue I stole from some former colleagues that wrote PicoContainer.
  3. Interface points between major subsystems. Aggressively use the GoF Facade pattern to limit the exposure between subsystems, and abstract the interface to allow for easier testing with mocks or stubs.
  4. Extension points. This is a pretty obvious usage, but knowing when to make an abstracted extension point is way, way beyond the scope of a mere blog post.

A great codebase to look at for examples is the CruiseControl.NET application. I think they do a very good job at testing a nasty piece of functionality.