Thursday, June 30, 2005

Create new entry points to enable unit testing of Legacy code

If you've been reading the Shade Tree Developer this week you'll know that I'm fighting with a legacy component. I was trying to get a piece of code into NUnit to test my new functionality. A class in the stack works by giving it a key value to a record in the database. I tried for awhile to use an existing stored procedure to push in test data. Guess what I found? I had relational integrity issues with setting up the test data, but so what, that's just what a database is. I wasted a bit of time pushing past the referential integrity by getting the correct data and stumble right into security authorization being performed inside the f***ing stored procedure!

It was about this time I remembered what I had been reading in the Feathers book on legacy code. Sometimes you have to change the code just to get it into the test harness. A quick "CTRL-ALT-M" action with ReSharper, and voila, an entry point into the business logic that bypasses the database and security checks. Add some ObjectMother magic to construct the fugly data structures, and I've got a working unit-ish test.

Moral of the story? Don't be afraid to change the code first before writing a test (well, maybe afraid but not terrified). Remembering to backup and look for an easier way to write a test was probably a good idea too.

Wednesday, June 29, 2005

External configuration should be human-readable

Ranting ahead...

For the love of all that is holy and good in the world make all of your external configuration readable and editable by human beings just in case I have to maintain your system later. Yeah, I know sometimes you have to encrypt configuration for those pesky security auditors, but at least make the unencrypted data readable. First of all, the mechanics of making little configuration tweaks shouldn't be difficult. Second, debugging and diagnosing problems is awfully difficult when you can't even read the configuration.

Xml by itself doesn't bother me, but I'm dealing with files that are partially SOAP serialized. "Simple" whatever protocol my ass. SOAP is not human readable!

Hulk Angry When He No Have ReSharper

I'm a ReSharper junkie, and I get angry when I don't have it. I've spent the majority of my .Net career working with converted Java developers. To a man, they all complained about the crudeness of Visual Studio.Net compared to IntelliJ or even Eclipse. I wrote them off as whiners because I was used to working in either VB6 or Visual InterDev (I still have a soft spot in my heart for doing what's now called Ajax development in VI) and VS.Net was clearly more polished (except for that little code and continue goodness). Then the ReSharper 1.0 beta builds dropped. My project team of ex-IntelliJ jockeys jumped at it immediately. Even with the sometimes spectacular instability of the ReSharper beta, it made an obvious improvement in coding efficiency. I know this because several times I started to code on the flight home without a working copy of ReSharper and became frustrated quickly when the code navigation and refactoring was missing. ReSharper quickly improved with each passing build, and the phrase "I'm having a ReSharper moment" left the project vocabulary.

Why am I complaining about this today? I'm wrestling with a legacy C# codebase this month and it's not entirely "ReSharper-able." One of Michael Feather's pieces of advice in his book Working Effectively with Legacy Code is to lean on the compiler when making changes to legacy code. Going farther, I really want to be able to use ReSharper's "Find Usages" capabilities to trace code in an attempt to understand interdependencies between classes and trace through code. In order to insert any kind of unit tests I'm having to extract methods, classes, and interfaces. The safest and quickest way to do this by far is to utilize the automated refactorings in ReSharper.

Out of the box the code could not be traversed by the static code analysis of ReSharper. The reasons? The solution files aren't using project references between projects. They are compiling all assemblies to a common hard-coded folder and making all references to this folder, creating a force field impervious to ReSharper code navigation and refactoring. This was largely done to get around problems they had with circular references between class libraries. A lot of classes are loaded and executed by reflection. Part of the problem is that the class library projects are much more fine-grained than they really need to be. I'm guessing that the assemblies were subdivided so granularly to apportion out the work between developers and avoid checkout conflicts with Visual SourceSafe's pessimistic locking, but I don't know that.

So far we're only tackling a small portion of the legacy code at first. The long range steps we're trying first are:

  1. Get to project references immediately. Enable both ReSharper refactoring across projects and the almighty task in NAnt. It also solves some reference conflicts
  2. Collapse the assembly structure to mitigate the circular dependencies and simplify deployment and build
  3. Eliminate as much of the load by reflection instantiations as possible. Minus automated unit testing, strong typing is absolutely essential. Since we don't have an automated NUnit test suite to speak of, we have to have the compiler enforce strong typing as a first line of defense when it's time to modify the code
  4. Get rid of VSS, move to Subversion, and get some automated builds put together for continuous integration
  5. Automate the migration of build products (think shared libraries, frameworks, and staged builds) between our different codebases. This is a soft spot in our infrastructure at the moment that is slowing us down. We're certainly not the first team to struggle with this by any means.

Another thing I highly recommend; my partner is an uber-buildmaster and one of the first adopters of the original CruiseControl. Fellows like him are awfully nice to have around when you're in this sort of situation.

I have played with IntelliJ briefly when I thought I was getting a chance to go onto a J2EE web project. For the agile style of incremental development (red bar, green bar, refactor), IntelliJ is a coder's paradise. I think this largely due to JetBrains dogfooding their own tool using an XP process. I know pockets of MS use agile processes (and I'm dubious about counting MSF Agile in that column), but I don't think the VS team really takes it seriously. VSTS is clearly aimed at the heavyweight Rational Unified Process product.

VS 2005 with VSTS is a definite improvement for agile development, but I think it still falls short of what I've observed in IntelliJ or Eclipse. I'm not entirely sure that some of the enhancements like unit testing integration are actually as good as the existing open source equivalents, and the refactoring in VS 2005 is inferior to ReSharper. I'm honestly more excited for the upcoming ReSharper 2.0 release than I am for the improvements with VSTS.

Tuesday, June 28, 2005

Taming the Legacy Beast - Dose It With Some NAnt

Yesterday I griped about legacy code. I carped about VSS, whined about the tight coupling, groused about the arcane configuration setup, and spewed invective about the stupid manual post compilation steps it took just to make the code work. Today it's time to start cleaning things up and applying some ibuprofen to the legacy headache.

Even though the work we're doing is supposed to be throwaway in a couple of months, we decided to "NAnt-ify" the development environment. First we're skimming the VSS project and setting up a new Subversion repository. The next step is to create a bare bones NAnt build to automate the application setup. The last step is a simple CruiseControl.Net configuration to get it into our existing continuous integration server (this is sooooo much easier than it was just a couple of years ago. Hat's off to the CC.NET guys).

If you can't easily make a repeatable build and get the application working on your own box in short order, you're not gonna go anywhere fast. That whole NAnt/CC.NET helps keep the code clean in source control so your coworkers aren't idle while you go hunt down the new class file you forgot to add. And one last time, VSS is a piece of crap. Using the far superior clients for Subversion (or almost anything else) that are actually intelligent enough to determine what files have been changed, added, or deleted makes for a smoother source control experience. And no, we're not going to wait until November for the promise of Hatteras in VSTS when Subversion is here now and free.

Monday, June 27, 2005

You know it's a bad day when...

You're typing "iisreset" or "net stop iisadmin /y" a lot.

Legacy Code and the Life and Death of an Application

My colleague and I are making minor extensions to some legacy code (in C#) for the next month or so (and that generally implies that the Shade Tree Developer will feature quite a bit of ranting in the near future). After that my team will be making some large scale architectural changes to an existing product. My job for the rest of the year is going to revolve around legacy code.

We very frequently make customer driven modifications to our application. We also have to keep the product ahead of the competition, and that means being agile (in the literal sense) with our development practices. Our existing codebase has some definite weak spots. The development organization has the beginnings of a long term "Grand Unified Theory" to transmogrify our existing systems and development environment to extend the company's investment in the current products and prevent the downward spiral into "legacy code."

Rewriting the applications is completely out of the question. So first, here's the symptoms we're trying to prevent and eliminate. As we get farther into the "Grand Unified Theory," I'll blog about our strategies for getting out of the legacy code trap.

Life and Death of an Application

In my finite experience and research, applications (or services if you're of the SOA persuasion) reach the end of their useful lifespan when the cost of altering or extending an application is greater or riskier than throwing it away and starting over. I've seen a lot of people make a parallel to the concept of entropy. The natural state of any system will tend towards entropy (in software terms, the big ball of mud). You have to exert energy into an application to reverse or stave off application rot (entropy).

Most applications undergo a twilight period where the previously shiny, well-factored application descends into a morass of spaghetti code that becomes more and more difficult to work with. The applications are still providing business value, but they are starting to be more of an opportunity cost rather than an enabler because of the increasing friction incurred by further modifying the application. This twilight slide into retirement is what most people mean when they say “Legacy Code.” The code works, but no one wants to touch it for fear of bringing the down the whole thing.

What is Legacy Code?

Here's a couple of takes: "Before we get started, do you know what 'Legacy Code' means? It’s code that works" -- via Roy Osherove.

Yeah, but this is true only if you leave it alone.

Michael Feathers described legacy code as "code without tests." What I think he’s driving for is code that can be safely modified and rapidly verified.

I'm going to add a corollary to Mr. Feather's definition; legacy code without tests is code that is difficult to test. Twice in the last year I’ve transitioned from greenfield development projects that were written with TDD to working with brownfield code that had not been written with TDD. In almost startling contrast, the test-first code was vastly easier to extend with new unit tests than the code written test-last. To paraphrase the noted development sage Ferris Bueller, “the legacy code was so tightly coupled that if you put a chunk of coal between the classes you would get a diamond.”

What made the legacy code made writing unit tests difficult?

  • Overuse of static methods. Static methods result in tight coupling. I ranted about this here. Keeping state in static fields makes it even worse.
  • Tight coupling to external configuration. It’s often impossible to run any piece of the code without an external configuration file. Dealing with configuration slows down unit testing. I’ll blog more on this soon, specifically ways to avoid the coupling.
  • Poorly factored code and tight coupling between concerns. Database, business logic, and presentation logic all mixed together.
  • Blobs

Legacy Code isn’t Necessarily Old

Of course, many applications aren't well-factored to begin with -- instant legacy code. Entropy also creeps in with any gap in communication or participation between the designer’s vision and the development team. I’m dealing with one of those right now. The original vision and design was thoughtful and innovative, but it was thrown over the wall to a team of developers who had not participated in the original vision. It’s particularly bad when the original prototype is coded by a skilled architect and then handed off to the developers with instructions to “just finish it off.” One of the advantages to continuous design is that the rough edges are smoothed down over the course of development. Remove the designer from the coding and those rough edges will only become sharper over time.

Shamefully, I think I’ve been responsible for a bit of this myself. One way or another the technical vision needs to be socialized and shared throughout the development team if you want to create a consistently structured application. Ideally, the development team should be creating the technical vision themselves.

The Development Environment Matters Too

Since much, if not most, of the cost of modifying an existing application is verification and regression testing, the ability to quickly migrate the application to a testing environment is paramount. Repeatable automated testing is such a great long term investment in an application that it’s a shame so few projects invest upfront in test automation. At this point in my career, I would consider development infrastructure like build scripts, database scripts, and continuous integration servers to be a vital part of the application itself. If that infrastructure is lacking or nonexistent, the application is that much harder to modify. I'm working with legacy code right now, and the lack of an automated development build with unit tests is making me feel, well, like I'm naked in a hailstorm.

The worst case I know of was a very large VB6 application that had been accreted around a nice tidy core over the course of several years. Simply setting up a test environment manually could take up to 10 business days, and even then they were never very confidant in the installation. That application was a core piece of the manufacturing system and changed often. Not only was the migration slow and hard, but the haphazard nature of the environmental control allowed several pieces of bad code to get through testing and into production, leading to factory downtime (and take it from me, factory downtime is a bad, bad thing. When the billionaire tycoon founder himself comes looking for someone’s head in IT, run away and get some plausible deniability). The real gallows humor was that the application was customized somewhat for each factory and business line and therefore had to be regression tested for each and every configuration.

"#region/#endregion" Tags Smell Bad

Ranting ahead...

I'm unilaterally declaring overuse of the "#region / #endregion" blocks in VS.NET a code smell. When I first started playing with VS.NET in 2001 I thought this feature was going to be the best thing since sliced bread. Boy, was I wrong. It seems like every time I expand an innocuous looking region out spills a big bag of entirely unrelated code.

Much like the excessive comments code smell, the existence of a large code region generally implies that the code in the region really should be in a different method or even a different class. The region block is no substitute for good factoring. The region block is often used as just a bit of perfume sprayed over smelly code.

If you want what I would call a bad example, go take a look at the source code of the original application blocks from Microsoft Patterns and Practices. Instead of factoring functionality, they wrote more or less procedural code segregated by region blocks that were nested 4-5 layers deep. From what I understand, the code style was mandated by their internal QA code standards. The result is thoroughly unreadable code. Finding the code that does anything is like playing a game of hide and go seek in a thick forest.

The .Net community's tendency to take anything written or published from MS as the gospel without any kind of critical analysis is a huge pet peeve of mine. The original application blocks were a good example. The code was rotten and they were reportedly buggy in spots -- but I saw nary a word of criticism anywhere.

Thursday, June 23, 2005

What an Amazing (Code) Smell You've Discovered

I'm naming a new code smell today, "Obsessive Tracing." You know exactly what I mean. If you see a long method or class with a *lot* of Debug.WriteLine("1") or Debug.WriteLine("I'm in here now!") methods sprinkled throughout the code, it's a good bet the code smells to high heaven. Those trace statements are in there because the code is prone to breaking and hard to understand.

The point of a code smell is to recognize a problem so you can begin to move in a different direction. I'm not sure what you do with legacy code, but with new code the key in my opinion is a well factored solution for ease of understanding and strong unit testing. Excessive amounts of debugging often means your unit testing isn't granular and comprehensive enough. Excessive debug statements might also mean a developer could benefit from reading up on the capabilities of their debugger.



Geek points for nailing the movie line in the title. No Chris Fields you don't count, that one's too easy for you.

Flimsy evidence that agile delivers the goods...

Does Agile development actually work in real life? Here's anecdotal proof in the affirmative (since the XP Refactored book sold a bunch of copies mostly based on negative anecdotes I figure I can get away with this too).

Production Moves in Ceremonial Waterfall

Production moves in big IT waterfall land are living nightmares of pure desperation and panic just waiting for the other shoe to drop. We usually had most of the team work an entire weekend in shifts just to be on hand as code and data was manually migrated. It's not unusual to see the cots come out in project warrooms. I don't think that's what the Godfather gang have in mind when they "take it to the mattresses," and the mafia seemed to have more fun with it than we did. As I've mentioned before, teams often made last minute coding changes right before go live because integration problems weren't discovered until making the migration and everybody was afraid to miss the installation window.

Agile Deployment

Shifting attention back to the shiny agile world of Test Driven Development and Continuous Integration, we made a final push into a production staging environment with our new product yesterday (with no known defects I might add). After yesterday's move, I left work on time and took my toddler to the YMCA pool with nary a cell phone at hand because I was perfectly confident everything was fine (I'll probably pay for that this morning).

On a previous project last year for a previous employer we made our second release one Tuesday morning. I was quite happily working on a user story in the mainline code branch for the following release when the project manager came up to tell us the new release had been put into production without a hitch. The developers had completely forgotten about the production move!

Where I Think Agile Development Helps

So why do I think the production moves went so much better with agile development than my earlier projects in waterfall world (to be fair, the earlier projects were COM based with a fair amount of middleware and the later projects were .NET systems)? The waterfall shop had a admirably organized (but radically inconsistent) ceremonial change management process, but pitiful build management. I think most of the reasons I list below are applicable for any disciplined iterative and incremental process (I'm thinking RUP here), but agile processes put more emphasis on early and frequent feedback over almost anything else.

  1. Earlier, more frequent, and repeatable testing. Test Driven Development is really a design technique as much as anything, but it certainly creates a large body of automated unit tests that provide a safety net to keep new bugs from emerging. Add any kind of automated acceptance or regression test suite and that safety net gets much tighter.
  2. Continuous Integration. CI can be thought of as a lot of trial runs at automated code installation. Much the same way that TDD forces you to write code that is testable, maintaining an automated CI build forces you (if your thinking cap is on straight) to think hard about making your application or service easier to deploy and version. Some people will excoriate me for this statement, but I think a NAnt build is the best documentation for system installation because it unambiguously specifies what has to happen for the application to function (and it tends to stay synchronized with reality easier than a document). I'd still supplement that with a Wiki page though.
  3. Defects can be fixed faster and stay fixed. CI allows a team to push code fixes to testing much faster without sacrificing environmental control. In ceremonial waterfall land, code pushes required filling out forms and convincing an external group to make my code move first, often up to a six hour process if other projects got there first. By automating builds under the control of the project team, I think the control over the environment is much better even without a controlling environment authority outside the development team, and certainly much faster. When you do write an automated test for a defect as you fix it, the defect stays fixed. Lost time due to regression failures can be mitigated by automated testing.
  4. Project teams are, or should be, more self-reliant and self-organizing. What I mean is that a project team should be able to carry out its own configuration management without waiting for a sluggish external group. In agile projects and organizations I've consistently had fewer dependencies on external IT service teams. I think that goes a long way towards reducing project friction. It does require carrying a broader skillset on a project team and people willing to step outside their normal skill silo. I think that might be one of the largest barriers to agile adoption in big shops. Flame away on point number 4, I probably deserve it.
Our organization has started down a path of using the WiX tool to create installers as part of our NAnt/CruiseControl.Net build. Using this, our testers will now be able to work off the automated installation in testing environments at all times, in effect testing the application and installation at the same time. I know several other organizations in the blogosphere are trying this, too. I'm only peripherally involved, but I'm excited for the potential. More updates when I know more.

Wednesday, June 22, 2005

I detest Visual SourceSafe

Ranting ahead, obviously...

All our new projects are in bright, shiny Subversion repositories and humming along just fine. I've got about a month of detour through existing products in Visual SourceSafe and I'm already aggravated on the first day. Don't get me wrong, every source control system has some sort of problem, but VSS is the absolute worst.

The client is a piece of crap; it falls over and corrupts in a swift gale, it's slow, and hangs during CC.NET builds almost as often as not. Branching and merging with VSS is a joke. In a world where Subversion and CVS are free and reliable with multiple clients and integration into IDE's, and SourceGear Vault is inexpensive, why is VSS still so popular in .Net shops?

I've played quite a bit with the vss2svn perl script from Tigris.org. I have no idea if it works or not because VSS keeps crapping out on corrupted history trails. VSS won't even behave long enough to get rid of it.

One thing I'd really like to see from the .Net community as a whole is to stop taking everything from Microsoft without question. There is a whole world of development tools and knowledge out there that doesn't originate in Redmond.

How's this for scary though, in a former job we used VSS under the dark of night on the side because the officially mandated source control system (CC/Harvest ) was even worse.

Tuesday, June 21, 2005

Thinking about Developer and Tester Interaction

The following is essentially a lecture and reminder to myself and my team, I apologize in advance for being preachy.

We're wrapping up a release candidate build this morning and quite naturally the subject of testing and bug fixing is on my mind. Because of resource constraints (i.e. not enough testers) we were not able to do much testing (acceptance testing by real testers, not just TDD) within iterations. Doing the old-fashioned (and stupid) big bang, waterfall testing cycle at the end under schedule duress is always tricksome.

It is absolutely imperative that the developer/tester interaction be as smooth as possible. Everybody needs to be focused on a common goal of finding and removing defects from the code, and that means paying attention to what the other half of the equation needs. Verbal abuse or general incivility from either side needs to be smacked down.

If you're going to do any kind of Agile development, one of the first and hardest lessons to learn is that the role boundaries must be blurred, or at least have much more transparency between project roles. You can't bury your head in coding and designing, you have to actively cooperate with testing on test automation and code migration between development and testing environments.

What Developers Owe Testers

  1. Don't waste the tester's time with buggy code. At a minimum, unit testing and either pairing rotation or code review should stop the majority of the problems from reaching the testers. If you can get acceptance tests to the developers before coding is "complete," all the better still.
  2. When you fix a bug, try hard to write an automated test that validates the exact conditions of the defect. At a bare minimum, exercise the code end-to-end to ensure the bug is really fixed. Don't assume the defect is really fixed because you dealt with the root cause (I'm currently wearing some egg on my face from this one).
  3. If anything is blocking the testers, stop what you're doing and get them working again. The application or even a user story cannot be considered complete until it is tested. If the testers are sitting on their hands because a testing environment is invalid or the application is down, your project is down. Again, a reminder to my team and I because we dropped the ball on this one day a week or so back.

What Testers Owe Developers

The singular thing we developers really need from the testers is just communication. When a defect is found, we have to understand a couple of things.
  1. A description of exactly what the error is
  2. Exactly how to reproduce the error
  3. The desired outcome of an operation in clear, unambiguous language
  4. Contextual information if it exists. Stack traces, audit trails, input data
  5. Severity of the defect. Is the defect stopping the tester from testing any farther?

The worst thing a tester can do is just say "It doesn't work" without any context or adequate description of the desired outcome (or my colleague's ultimate pet peeve, "it says NullReferenceException"). I had an incident a couple years ago with an analyst slash tester. She had given us the original requirements for a data exchange between two systems that wasn't much more than a big SQL statement in a stored procedure (mildly modified from the previous system we were replacing). When she was testing the exchange, she kept coming back to us and telling us the results were wrong. Her SQL script she was using to test the exchange was giving different results. I think at some point I said something like "!~@#* it, let's just use your SQL" so we could move on.

My favorite tester used to show me all defects at the tester workstation and walk me through the problems before his team would log a defect. That helped dramatically. It obviously helps if the testing team is colocated with the developers.

Testers are Pigs

Made you look, didn't I? All I mean is that the testers should be an active participant in the standup meetings and the project as a whole. One of the agile practices I like is to have a bug report discussed every single morning in the standup, even if it's just to say there are no bugs. It's a great way to ensure the developers are getting the testers what they need and vice versa. Bug tracking tools are fine and dandy (except ours is terrible), but nothing beats face to face communication.

Here's the original story about the derivation of Pigs and Chickens if you've never heard the phrase.

Sunday, June 19, 2005

TDD Design Starter Kit - Static Methods and Singletons May Be Harmful

Static methods can be a cool solution to many issues and they're definitely convenient. I routinely use static methods in place of specialized constructor functions. Sometimes it is absolutely necessary to maintain state between method calls or across threads and static methods are a logical choice. However, sometimes static methods do not mix well with Test Driven Development. A direct reference to a static method from one class to a second class means that the first class is strongly coupled to everything that the second class uses. Keeping any kind of state in static members can also interfere with your unit tests (more on that here). Always be cognizant of this fact before you create a static method.

Just to get this one out of the way, you cannot mock (or stub) a static method, period. Crying or wailing about it won't help. There is no easy way to substitute in a stubbed implementation for a static method. For that reason alone, I advise developers to never do anything in a static method that calls out of the current AppDomain (HttpContext objects, database calls, MSMQ manipulation, etc.). Even if you need to use a static (or thread local storage or HttpContext storage) field to maintain state, all consumers should access this cached state through some type of abstraction interface for easier testing.

Besides the testability issue, do you really want your application strongly bound to a particular implementation like MSMQ? If you limit the coupling to MSMQ, you can more easily substitute in MQSeries or a call into your company's new utopian Enterprise Service Bus later. My point merely being that designing to maximize testability often happily coincides with just plain good design or architecture.

Singletons

Singletons always, always, always interfere with writing decoupled and isolated unit tests. There is a lot of interest in Domain Driven Design right now and one of the key concepts in DDD is that often the domain objects are retrieved or persisted through a repository that manages caching and persistence for a domain class or small family of classes. The repository is stateful across requests or classes, so it is a natural candidate for a singleton. Don't be seduced by the dark side. If you make the repository a singleton, all of the consumers of the repository (and there will be a bunch) cannot be tested in isolation. You can do something to allow you to mock or stub out the underlying data source classes, but take it from me, this sucks (hard). Your unit tests are dependent upon the implementation of the repository class. This makes your unit tests harder to write because there is more code, and harder to understand because you are investing more of the unit test in setting up the internal state of the repository than exercising the consuming class. For a multitude of reasons, it is always important to make your unit tests intention revealing. Mocking the guts of the repository may end up obfuscating the real intention of a unit test.

Here's an article I wrote on a TDD-friendly alternative on the StructureMap site for exactly this situation with more discussion on the issue: http://structuremap.sourceforge.net/SingletonInjection.htm.

You should also check out Robert Martin's article Singleton vs. Just Create One. While you're on the ObjectMentor site, read everything those guys write, it's all good stuff.

As an aside, one of my friends does a lot of technical interviews for .Net developers for a very large employer in Austin. His weedout question is "what is the difference between a static and an instance method?" He is continually scratching his head on the percentage of people who wash out on this question. I have no idea what to say about that.

Thursday, June 16, 2005

A simple example of the "Humble Dialog Box"

At the Agile Austin lunch today, we talked a bit about different ways to apply the Model View Presenter pattern with WinForms clients. I promised to put up an example of using Michael Feather's "Humble Dialog Box" to create more testable user interface code.

The current thinking for writing automated unit tests against rich clients is a modification or variation of the classic Model View Controller (MVC) architecture. Specifically, take the view part of MVC and slice it as thin as possible so that it is only a skin around the actual UI components and make it completely passive. The controller, now called the "presenter," is responsible for all interaction with the rest of the system. There is a pattern of symbiosis between the view and the presenter. The presenter directs the view what and when to display and the view captures and relays user events to the presenter. Check the links above for a more comprehensive explanation from the professionals.

Here's a common scenario. You have some kind of form in your application for editing a piece of data. If the user trys to close the form and there are pending changes, put up a dialog box giving the user a chance to cancel the close operation. In this case the dialog box is the major impediment to automated testing, so we're going to hide the message box creation behind an interface that can be mocked (or stubbed if that's your predilection). Do the same thing for the view/presenter separation. Use the Dependency Inversion Principle to abstract the view away from the presenter and mock the view in the unit tests.




using System;
using System.Windows.Forms;
using NMock;
using NUnit.Framework;

namespace SampleCode.HumbleDialogBox
{
public interface IMessageBoxCreator
{
bool AskYesNoQuestion(string title, string message);
}

public class MessageBoxCreator : IMessageBoxCreator
{
public bool AskYesNoQuestion(string title, string message)
{
DialogResult result = MessageBox.Show(message, title, MessageBoxButtons.OKCancel);
return result == DialogResult.OK;
}
}

public interface IView
{
void Close();
bool IsDirty();
}

public class Presenter
{
public const string DIRTY_CLOSE_WARNING = "Changes are pending. "
+ "Ok to continue, cancel to return to the edit screen.";
public const string CLOSE_WARNING_TITLE = "Changes Pending";

private readonly IView _view;
private readonly IMessageBoxCreator _msgBox;

//
public Presenter(IView view, IMessageBoxCreator msgBox)
{
_view = view;
_msgBox = msgBox;
}

public void Close()
{
bool canClose = true;

if (_view.IsDirty())
{
canClose = _msgBox.AskYesNoQuestion(CLOSE_WARNING_TITLE, DIRTY_CLOSE_WARNING);
}

if (canClose)
{
_view.Close();
}
}
}

[TestFixture]
public class PresenterTester
{
private DynamicMock _viewMock;
private DynamicMock _msgBoxMock;
private Presenter _presenter;

[SetUp]
public void SetUp()
{
_msgBoxMock = new DynamicMock(typeof(IMessageBoxCreator));
_viewMock = new DynamicMock(typeof(IView));
_presenter = new Presenter((IView) _viewMock.MockInstance, (IMessageBoxCreator) _msgBoxMock.MockInstance);
}


[Test]
public void CloseViewWhenViewIsNotDirty()
{
// Define the expected interaction
_msgBoxMock.ExpectNoCall("AskYesNoQuestion", typeof(string), typeof(string));

_viewMock.ExpectAndReturn("IsDirty", false);
_viewMock.Expect("Close");

// Perform the unit of work
_presenter.Close();

// Verify the interaction
_msgBoxMock.Verify();
_viewMock.Verify();
}


[Test]
public void CloseViewWhenViewIsDirtyAndUserRespondsOk()
{
// Define the expected interaction
_msgBoxMock.ExpectAndReturn(
"AskYesNoQuestion",
true,
Presenter.CLOSE_WARNING_TITLE,
Presenter.DIRTY_CLOSE_WARNING);

_viewMock.ExpectAndReturn("IsDirty", true);
_viewMock.Expect("Close");

// Perform the unit of work
_presenter.Close();

// Verify the interaction
_msgBoxMock.Verify();
_viewMock.Verify();
}


[Test]
public void DoNotCloseViewWhenViewIsDirtyAndUserRespondsCancel()
{
// Define the expected interaction
_msgBoxMock.ExpectAndReturn(
"AskYesNoQuestion",
false,
Presenter.CLOSE_WARNING_TITLE,
Presenter.DIRTY_CLOSE_WARNING);

_viewMock.ExpectAndReturn("IsDirty", true);
_viewMock.ExpectNoCall("Close");

// Perform the unit of work
_presenter.Close();

// Verify the interaction
_msgBoxMock.Verify();
_viewMock.Verify();
}
}
}




Here's a rundown of the pieces from the example code.

  1. IMessageBoxCreator/MessageBoxCreator - An interface and wrapper class around the WinForms MessageBox class. The methods in the .NET framework for dialogs are all static, and static methods cannot be mocked.
  2. IView interface - An interface that establishes the public contract between the actual form and the presenter. I didn't show it, but assume the actual View has a reference to the Presenter.
  3. Presenter - the Presenter class drives the IView and IMessageBoxCreator interfaces. The Presenter class is completely unaware of any of the actual user interface plumbing, i.e. not one single reference to the System.Windows.Forms namespace.

In this example I used constructor injection to attach the IMessageBoxCreator. The next example is mostly the same, but I use StructureMap instead to locate the IMessageBoxCreator and take advantage of StructureMap's built in support for NMock.




using System;
using System.Windows.Forms;
using NMock;
using NUnit.Framework;
using StructureMap;

namespace SampleCode.HumbleDialogBox2
{
[PluginFamily("Default")]
public interface IMessageBoxCreator
{
bool AskYesNoQuestion(string title, string message);
}

[Pluggable("Default")]
public class MessageBoxCreator : IMessageBoxCreator
{
public bool AskYesNoQuestion(string title, string message)
{
DialogResult result = MessageBox.Show(message, title, MessageBoxButtons.OKCancel);
return result == DialogResult.OK;
}
}

public interface IView
{
void Close();
bool IsDirty();
}

public class Presenter
{
public const string DIRTY_CLOSE_WARNING = "Changes are pending. "
+ "Ok to continue, cancel to return to the edit screen.";
public const string CLOSE_WARNING_TITLE = "Changes Pending";


private readonly IView _view;

public Presenter(IView view)
{
_view = view;
}

public void Close()
{
bool canClose = true;

if (_view.IsDirty())
{
// Get the IMessageBoxCreator out of StructureMap
IMessageBoxCreator msgBox =
(IMessageBoxCreator)
ObjectFactory.GetInstance(typeof(IMessageBoxCreator));
canClose = msgBox.AskYesNoQuestion
(CLOSE_WARNING_TITLE, DIRTY_CLOSE_WARNING);
}

if (canClose)
{
_view.Close();
}
}
}

[TestFixture]
public class PresenterTester
{
private DynamicMock _viewMock;
private IMock _msgBoxMock;
private Presenter _presenter;

[SetUp]
public void SetUp()
{
_msgBoxMock = ObjectFactory.Mock(typeof(IMessageBoxCreator));
_viewMock = new DynamicMock(typeof(IView));
_presenter = new Presenter((IView) _viewMock.MockInstance);
}

[TearDown]
public void TearDown()
{
ObjectFactory.ResetDefaults();
}

[Test]
public void CloseViewWhenViewIsNotDirty()
{
// Define the expected interaction
_msgBoxMock.ExpectNoCall("AskYesNoQuestion", typeof(string), typeof(string));

_viewMock.ExpectAndReturn("IsDirty", false);
_viewMock.Expect("Close");

// Perform the unit of work
_presenter.Close();

// Verify the interaction
_msgBoxMock.Verify();
_viewMock.Verify();
}


[Test]
public void CloseViewWhenViewIsDirtyAndUserRespondsOk()
{
// Define the expected interaction
_msgBoxMock.ExpectAndReturn(
"AskYesNoQuestion",
true,
Presenter.CLOSE_WARNING_TITLE,
Presenter.DIRTY_CLOSE_WARNING);

_viewMock.ExpectAndReturn("IsDirty", true);
_viewMock.Expect("Close");

// Perform the unit of work
_presenter.Close();

// Verify the interaction
_msgBoxMock.Verify();
_viewMock.Verify();
}


[Test]
public void DoNotCloseViewWhenViewIsDirtyAndUserRespondsCancel()
{
// Define the expected interaction
_msgBoxMock.ExpectAndReturn(
"AskYesNoQuestion",
false,
Presenter.CLOSE_WARNING_TITLE,
Presenter.DIRTY_CLOSE_WARNING);

_viewMock.ExpectAndReturn("IsDirty", true);
_viewMock.ExpectNoCall("Close");

// Perform the unit of work
_presenter.Close();

// Verify the interaction
_msgBoxMock.Verify();
_viewMock.Verify();
}
}
}



Final Thoughts



It is not impossible to write automated unit tests for rich clients, but it's definitely difficult and time consuming. So what can you do? You can take a calculated risk and forgo writing the automated tests for the user interface. The biggest problem with that approach is that a complicated rich user interface can generate a large number of bugs and requires a lot of energy towards manual regression testing (duh). You can test a WinForms application with Luke Maxon's most excellent NUnitForms toolkit, but user interface tests are still more work to setup and execute. A better approach is to simply make as much code as possible independent of the WinForms (or Swing, etc.) engine. I say you still have to test the actual UI forms and controls. However, if they are passive and loosely coupled from the rest of the application your NUnitForms tests can be much simpler.



I left some implementation details out of the example. I've used the MVP pattern pretty extensively on a couple of projects now with mostly good results. Since it's such a hot topic and the book on best practices is literally being written as I type this, I'll try to blog soon on some MVP suggestions and pitfalls.

Wednesday, June 15, 2005

Steve Hebert article on Code-Talking

Code Talking: The key to great software?

A great article. I admit that I don't particularly enjoy pair programming, but Steve Hebert describes one of the best benefits of pairing. Collective code ownership is an awfully good thing in many ways. I've certainly observed that my stress level is generally less on projects that have some degree of collective ownership.

Tuesday, June 14, 2005

A classic article from Alistair Cockburn and horror stories just for fun

http://alistair.cockburn.us/crystal/articles/cpanfocisd/characterizingpeopleasnonlinear.html

This one has been around awhile, but it's still a worthwhile (re-)read. It's full of radical ideas like:

  • Individuals are important
  • People aren't "Plug'n Play" components
  • Productivity can be maximized by making it easy for the project team to communicate
  • Pay attention to the team's environment

If you're in anything like the environment I describe below, you can try using Cockburn's writing in an effort to reform your workplace. If you have any influence over your workplace, heed Mr. Cockburn's writing now rather than learning the hard way.

This Stuff Matters

A couple years ago I got to jump straight from pure insanity to a high caliber agile shop. An oranges to oranges comparison of effectiveness between the Microsoft Solutions Framework process at the first shop and Extreme Programming in the second is useless because too many other factors were different (stronger teams, more supportive organizations, less customer involvement), but the simple ideas of:

  1. Sitting near all of my teamates in an open room
  2. Everybody focused on only the one project
  3. Developers, testers, and analysts on the project at the same time!

were an enormous improvement in my mind.

Project Team Insanity

I used Cockburn's chart on the effectiveness of various communication channels in a presentation several years ago trying to convince my then management to move towards iterative and more collaborative processes. At the time the entire organization was in a "matrix" organization structured by functional group first, and project teams second. Cubicle assignment was strictly by discipline (development, architects, analysts, testers, etc.) because management was tired of reorganizing teams and paying moving costs to colocate teams. Everybody was working on multiple projects at a time so that all projects had equal access to specialized knowledge and skillsets. I had the distinctly unpleasant experience of simultaneously working on the rump end of a death march project as a developer (later cancelled), and trying to lead the development of a separate project.

The first project was an early Web Services project using a custom in-house SOAP-like framework developed by the internal integration architecture team (if you ever hear a sentence like that, run away!). The lead developer/project manager was in his first lead role and had no prior lead or architecture experience. There were five distinct methods for the service, and five developers, so naturally each developer was assigned a method and told to go off and code it. I failed in my attempt to escape the assignment and began to code my assignment (throwing out the unworkable design that was handed to me. Win32 API and COM calls from within XSLT transforms? No thank you).

Early on, I asked the lead where the development database was located and was told each developer was just using their own box. Come to find out, every single developer had created or reused a different database schema structure to hold the same logical data. Two developers were using an existing Oracle schema structure (the database was overworked and we had an opportunity to move some load off), and the other three had made their own Sql Server databases. Two of us had created fully functional, specialized rules engines that did roughly the same thing, but radically different in implementation. My first action as a newly minted Systems Architect (make finger quotes for the full effect) was to blow the whistle on the technical weaknesses to my new boss. The project was fortunately abandoned and replaced by a different effort (2+ years in MSF Planning!). Integration testing at the very end was apparently pretty interesting;) I sincerely think the project could have succeeded if the developers had been working in a common location instead of in a complete vacuum. Just accidentally overhearing conversations would have tipped us off that our code could be shared between service methods and certainly would have got us on a common database early. Of course, the fact that every developer was actively plotting to get off the project and the organization went through a massive reorganization at the same time didn't help either.

The second project was an extremely complex system with three other developers, all of whom sat in different wings of the floor, and were working on other projects at the same time. At no time during the course of the project was the requirements analyst present on the team at the same time I was. All communication was through a requirements document that was prepared when I was on the other project (that turned out to be incomplete and/or flatout wrong). Come to find out late in the game, there was a second requirements specification no one in development knew about. We pulled it off by working way too many hours, but I'll never do a project like that again. The saving grace was very good involvement from the business partners, otherwise it would have bombed. The system is undeniably successful in production (one of my friends won an award from the business for it), but it was a long time before I enjoyed software development again. I'm pretty sure the CMM crowd would call this an unrepeatable process.

Of course it didn't help that I over-engineered the application a bit, but I'm making fun of other people here, dammit! I read somewhere that the worst designer is a fellow on his second project trying to use every idea he had from the first system. Guilty as charged. I'd also just read the GoF book for the first time. They should put a warning label on that thing.

My former employer has since quietly removed the worst of the organizational madness and is simultaneously working toward climbing the CMM ladder and tentatively introducing some basic best practices like automated builds and better unit testing. All I can say is good luck to you guys, and it'll be worth the effort (maybe not the CMM part, though).

It's fun to laugh at the absurdity of past projects, as long as you don't repeat them. Bad examples can be more illustrative than good examples sometimes.

Monday, June 13, 2005

The Worst Possible Way to use a Stored Procedure

Rant time...

Repeat after me please,

I will not put any business logic into a stored procedure.
I will not put any business logic into a stored procedure.
I will not put any business logic into a stored procedure.
I will not put any business logic into a stored procedure.
I will not put any business logic into a stored procedure.

Okay, if your business logic can be expressed easiest through a declarative SQL WHERE clause, a SPROC can be a cool way to go. Set-based logic is almost always simpler to do with SQL than mucking through with procedural C#, but that's a different rant for another day, and I'm definitely talking about procedural code within sproc's here.

Question: How do I know if I might be doing something in a sproc that I shouldn't be doing? Answer: If there is *anything* other than CRUD in your sproc (IF/ELSE/ENDIF, LOOP/END LOOP, etc.).

The all time dumbest thing to do is to split your business logic between a SPROC and the consuming middle tier code. Twice now in the past 12 months I've bumped into cases where business logic is performed inside a sproc, then interpreted by middle tier C# or VB6 code. Splitting the logic into multiple spots like this makes a system harder to understand. It also makes a system brittle to change because some logic is duplicated in both places. A change in the sproc or C# can, and usually does, break the other. This is a case of the Don't Repeat Yourself principle the Pragmatic Programmer guys talk about.

I had an interesting experience as a consultant one time. I was fresh into the project working with some legacy C# code (yes, there is already legacy .NET code). I had just stumbled on the fact that most of the business logic was really in T-SQL procedures. In many cases, the C# code undid some of the T-SQL transformations to filter the results further. The client architect was giving us a bit of a tongue-lashing ("you consultants better not write crappy code, and there better be tests for everything you write") while I was looking at a 5,000 line stored procedure, with his name all over the comments, trying to decide if some erroneous data was coming out of the database or being transformed in the C# code. This particular system is the subject of a case study/testimonial on MSDN as an example of all the wonderful things about .NET;)

"Thank you for listening, I feel better now." -- Roy Moore

Friday, June 10, 2005

Getting Out-Geeked By Your Product Manager (Off Topic)

I work in a pretty geeky shop, and I certainly mean that in a good way. From experience, I really don't trust people on software projects that aren't interested in technology and all the ancillary geeky subjects that seem to go with that. It's very difficult to be good at anything you don't enjoy doing. A passion for software development is a huge plus in my book.

John, my product manager, gets the Geek of the Day award for out-geeking all of us developers. John's qualifications:

  • Competing in the Ruby on Rails coding challenge
  • Sending me an email at 6:30 this morning telling me to check out Ajax.Net
  • Having read Dune out loud to his wife on his honeymoon (lifetime achievement award)
  • Being a Cold Fusion junky

All this from an ex-lawyer.

Thursday, June 09, 2005

TDD Design Starter Kit - State vs. Interaction Testing

When you write a unit test, what is it that your asserting? Most of the time you're doing state-based testing, i.e. your unit tests either validates a return value from a method call or a change in a property of the class being tested. Here is a typical example of state-based testing like you'll find in almost every introductory TDD article ever published.



[TestFixture]
public class StateTester
{
[Test]
public void TwoPlusThreeIsFive()
{
RunningSum sum = new RunningSum();
int actual = sum.AddIntegers(2, 3);

Assert.AreEqual(5, actual);
}

[Test]
public void AddSeveralNumbersAndGetTheRunningSum()
{
RunningSum sum = new RunningSum();
sum.AddInteger(2);
sum.AddInteger(3);
sum.AddInteger(5);

Assert.AreEqual(10, sum.Total);
}
}


Easy money, all you have to do is check an expected outcome for a known set of inputs. Unfortunately, the real world of enterprise software development quickly intrudes into your newfound TDD bliss. Toss in a bunch of dependencies external to your application and you quickly find that:

  1. The expected outcome is difficult or time-consuming to verify in an automated test
  2. The known inputs are difficult or time-consuming to setup in an automated test
  3. Tests that depend on external services can be brittle or just plain slow
  4. Measuring the outcome requires checking the state of all a class's dependencies, often making the unit tests too coarse and dependent upon the actual implementation details
  5. Too many classes have to be involved in the unit test

Teams abandon TDD as a failure when the tests are too difficult to write or just plain laborious. Nobody is going to willingly use a technique that is more bang than buck.

One way to alleviate the problem is to use interaction-based testing. Just like it sounds, interaction-based testing is verifying the expected interaction between a class and its dependencies. Typically you would use either a mock or a stub object as a stand-in for the real dependency. The unit test will verify that the expected interaction via method calls took place.

To illustrate the difference between testing state or interaction, here's a somewhat contrived user story (for best results, user stories should really be much finer grained than this):

Email Alert for Invoice Validation Failures User Story

  • When an invoice is being processed, if certain nebulous validation failure conditions are met, send an email to the accounts payable individual assigned to the invoice. The email should list all of the invoice errors in a certain html format.
  • If the invoice is really, really bad, send a summary email to the proper expediter. The email message will be html formatted in a particular manner.
  • If the invoice is successfully validated, publish the invoice (via MSMQ)

Here's a possible C# class for this story designed with absolutely no regard for testability.



public class InvoiceProcessor
{
public void ProcessInvoices(Invoice[] invoices)
{
InvoiceErrorReport report = new InvoiceErrorReport();

foreach (Invoice invoice in invoices)
{
// Execute the validation business logic
InvoiceError[] invoiceErrors = this.validateQuantities(invoice);
report.StoreErrors(invoice, invoiceErrors);
}

// determine if the critical error threshold has been exceeded
if (this.hasTooManyCriticalErrors())
{
MailMessage message = this.createErrorSummaryMessage(report);

// Call to the local (or default) SMTP service to send the email
SmtpMail.Send(message);
}
// determine if the warning threshold for the error count has been reached
else if (this.hasTooManyErrors())
{
MailMessage message = this.createDetailsMessage(report);
SmtpMail.Send(message);
}
else
{
// If the invoices are okay, send the invoices on to the next process via MSMQ
MessageQueue queue = new MessageQueue(this.getMessageQueuePath());
queue.Send(invoices);
}
}


public MailMessage createErrorSummaryMessage(InvoiceErrorReport report)
{
MailMessage email = new MailMessage();

// Go get the user email addresses from the user information storage
// (database or Active Directory)
UserInformationStore userStore = new UserInformationStore();
email.To = userStore.GetEmailAddressesForInvoices(report);

email.Body = this.createErrorSummaryMessageBody(report);

return email;
}

// #region Other Methods
}


We could use a state-based strategy to test the email. We could run the test, then run around and ask the expected recipients to check their inbox (don't laugh too hard, I've been a party to this). Since we do want an automated test, we could query the Exchange server to see if the email was sent out correctly, but I'd rather gouge out my eyeballs with a dull spoon or code in Fortran 77 than attempt this (not to mention the look of apoplexy on the sysadmin's face when I ask for administrative rights to the Exchange server). You could also check an audit trail, but that's not the real functionality being tested.

The easiest approach in the case of the email tests is to use interaction testing to verify that the email was sent to the SMTP service. Now look at the same class, but somewhat more testable because it is designed for interaction testing between the class and the SMTP service.



public class InvoiceProcessor
{
// "Dependency Inversion Principle" -- All dependencies are now to an abstracted interface
private readonly IEmailGateway _emailGateway;
private readonly IMessagingGateway _messagingGateway;
private readonly IUserInformationStore _userStore;

// Use "Constructor Injection" to push dependencies into InvoiceProcessor
public InvoiceProcessor(
IEmailGateway emailGateway,
IMessagingGateway messagingGateway,
IUserInformationStore userStore)
{
_emailGateway = emailGateway;
_messagingGateway = messagingGateway;
_userStore = userStore;
}



public void ProcessInvoices(Invoice[] invoices)
{
InvoiceErrorReport report = new InvoiceErrorReport();

foreach (Invoice invoice in invoices)
{
// Execute the validation business logic
InvoiceError[] invoiceErrors = this.validateQuantities(invoice);
report.StoreErrors(invoice, invoiceErrors);
}

// determine if the critical error threshold has been exceeded
if (this.hasTooManyCriticalErrors())
{
MailMessage message = this.createErrorSummaryMessage(report);
_emailGateway.SendMail(message);
}
// determine if the warning threshold for the error count has been reached
else if (this.hasTooManyErrors())
{
MailMessage message = this.createDetailsMessage(report);
_emailGateway.SendMail(message);
}
else
{
// no longer responsible for knowing where the MSMQ path is. Or even if this
// is an MSMQ
_messagingGateway.SendInvoices(invoices);
}
}


public MailMessage createErrorSummaryMessage(InvoiceErrorReport report)
{
MailMessage email = new MailMessage();

// Go get the user email addresses from the user information storage
//(database or Active Directory)
email.To = _userStore.GetEmailAddressesForInvoices(report);

email.Body = this.createErrorSummaryMessageBody(report);

return email;
}

// Other methods ...
}



public interface IMessagingGateway
{
public void SendInvoices(Invoice[] invoices);
}

public interface IEmailGateway
{
public void SendMail(MailMessage message);
}


What's different here? First, I used the Dependency Inversion Principle to encapsulate the interaction with the SMTP email service and the MSMQ queue behind .Net interfaces (IEmailGateway and IMessagingGateway). The second change was to use Dependency Injection to push in the IEmailGateway and IMessagingGateway dependencies into the constructor function. Here's what a unit test might look like to test the email and MSMQ interaction using NMock to create mock objects as placeholders for the real dependencies.




[Test]
public void SendTheSummaryEmailIfTheCriticalErrorThresholdIsMet()
{
// 1.) Create the NMock objects
DynamicMock messagingMock = new DynamicMock(typeof(IMessagingGateway));
DynamicMock emailMock = new DynamicMock(typeof(IEmailGateway));
DynamicMock userStoreMock = new DynamicMock(typeof(IUserInformationStore));

// Setup the test data
Invoice[] invoices =
this.setUpArrayOfInvoicesThatWillSpawnACriticalNumberOfInvoiceErrors();

// 2.) Setup the expectations. Gets the email
// addresses, sends an email address, but does not put the invoice array onto
// the outgoing queue
emailMock.ExpectAndReturn(
"GetEmailAddressesForInvoices",
"somebody@somewhere.com",
new IsTypeOf(typeof(InvoiceErrorReport)));

messagingMock.ExpectNoCall("SendInvoices", typeof(Invoice[]));
emailMock.Expect("SendMail", new IsTypeOf(typeof(MailMessage)));

// 3.) Create the InvoiceProcessor object with the Mock instances
InvoiceProcessor processor = new InvoiceProcessor(
(IEmailGateway) emailMock.MockInstance,
(IMessagingGateway) messagingMock.MockInstance,
(IUserInformationStore) userStoreMock.MockInstance);

// 4.) Execute
processor.ProcessInvoices(invoices);

// 5.) Verify the correct calls were made
messagingMock.Verify();
emailMock.Verify();
userStoreMock.Verify();
}

All I've really tested in this unit test is that the InvoiceProcessor sent the proper signals to the IEmailGateway and IMessagingGateway interfaces. The test is way too coarse, but I had to leave something for later posts.

State or Interaction Testing, Which Do I Want to Use?

Not to be flippant, but use whichever mode of testing is easiest to apply on a unit test by unit test basis. Both styles of testing need to be in your intellectual toolbox. State-based testing is generally easier because of the overhead associated with setting up the mock objects, but interaction-based testing is essential for dealing with external dependencies like databases or middleware and user interface testing. I'll blog much more in the near future about how to maximize testability by design.

What Next?

The InvoiceProcessor structure above is better, but it can still be improved. The next step is to carefully consider how responsibilities can be assigned to classes to make testing easier. Design concepts like Inversion of Control and Dependency Injection (DI is a subset of IoC) can help tremendously to improve testability by reducing coupling. One of the best design philosophies is to quarantine external dependencies into narrow Gateway classes to minimize direct coupling to external code. It's also helpful to understand the best practices for using mocks or stubs. In my next set of blog posts, I'm going to further refine the invoice processing user story.

TDD Design Starter Kit Roadmap

  1. Assigning Responsibilities
  2. Inversion of Control
  3. Dependency Injection (with and without a Container)
  4. Gateway Pattern
  5. Mocks and Stubs - Best Practices

More Information

Read more about this from Nat Pryce and Martin Fowler. I actually disagree a bit with Fowler's discussion of interaction testing, so no more calling me a "Fowlbot" for a little while.



UPDATE 6/11/2005: Martin Fowler dinged me a little bit about this one. His main point about the difference in testing styles is that state-based testers use mocks or stubs for exceptional cases and interaction-based testers will try to use mocks in almost every case, including domain model classes. Once you put it that way, I'd lean more towards the state-based testing approach. Mocks get you out of tight spots, but overuse is a bad, bad thing.

Friday, June 03, 2005

Learning Lessons -- Can You Make Mistakes at Work?

Of course you make mistakes, that's not really what I meant. Can you comfortably admit any type of fault in your workplace without undue fear of reprisal? Do you trust your teammates and management enough to discuss problems openly or do you sweep problems under the rug? After all, we know that admitting a problem is the first step to recovery.

A Healthy Environment

Last week my team did a project retrospective on our last four months effort. Half of our team is new to the company and our team is distributed between Austin and Houston, so there were definitely some rough spots to sand down. We came up with a list of problem areas with our execution and some lost opportunities. It was actually a very positive experience, because everyone involved was focused on improving our execution instead of pointing fingers or deflecting blame. Never once did I feel defensive or sense any kind of defensiveness in anyone else in the room.

We've already started to implement some improvements in our iteration and project management. Of course we still have to follow through and correct the problems, but at least the team got the issues out into the open and everybody is aware. I've heard the phrase "Lessons Observed" in the past to describe teams that don't act upon their lessons.

The Blame Culture

Much of my IT career has been in a shop where too much energy was wasted on blame avoidance and "CYA". We didn't learn any lessons or ever make any corrections because no one ever wanted to admit any kind of failing. It was more or less a waterfall shop (and still is) and management stressed a predictable project date and resource count over everything else. The worst sin was missing a deadline. The result? Sacrifice quality or outrageously pad estimates. I routinely saw projects that were coding right up to the production install because everybody was afraid to miss the arbitrary date. Very scary.

External compliance teams are the worst. They aren't involved in your project at all, but they get the last say on the success of the project by putting out a metrics laden scorecard. One of the primary measurements is a bug count read out loud in a nasty meeting at the end of the project. The typical response was to hide bugs. Developers would quietly negotiate with testers to fix bugs without reporting them. That's an awful practice, but it seemed more like survival at the time.

Now they're climbing up the CMM ladder (I think they're about to be level 3). It turns out CMM is a great work avoidance tool. You need what? Did you fill out the correct paperwork? Sorry, I can't do that until I have the TPS form signed and approved.

Avoid the Embarrassment Meeting

Be careful how you use metrics. Metrics reveal information and trends about your effort, but they're never the entire story. Avoid the "embarrassment Meeting" described by Steve Maguire. Misuse of metrics to punish people leads to stupid behavior that doesn't contribute to the project.

An agile process alone isn't enough. I worked on an XP-ish project with an absolutely miserable jerk of a project manager that routinely abused the development team with metrics. The developers varied wildly in skillset, and we were struggling. Every iteration close meeting featured a full review of exactly how the development team missed their estimates on a story by story basis. One time he went away to a conference for several days. Coincidentally the developers had what we all considered to be our best iteration. We completed stories on time, we got a lot of refactoring done that had been put off, and we even managed to make some overdue improvements to the NAnt build. The project manager came back from his conference, looked at *his* metrics, and loudly declared that we still weren't doing any better. I wanted to punch him.

An iteration later, I excitedly showed him a story that was finished ahead of the estimate. His response? We had sandbagged the estimate in the first place.

After some external intervention from higher management, we made some adjustments to our estimating and planning process that lowered the misery on the project considerably. It wasn't pleasant, but the mistakes were recognized and corrected. A healthier team would have made the correction faster though. I sincerely wish I had done something to alleviate the problem instead of fuming, but I didn't feel comfortable in the environment and didn't feel like I had a voice in the project.

To state the obvious, that was a dysfunctional environment. Martin Fowler has some pointed comments on the usage of metrics here.

Does Agile Development Help?

I want to say yes, but the real answer is "it depends." I think this is really more of a people issue than a process or organization problem. An iterative process is definitely advantageous because it gives you so many more chances to reflect on your project and make adjustments mid-stream. However, if you don't trust your team or feel comfortable making yourself heard this opportunity will be wasted. A team with poor chemistry is probably going to fail no matter what else is in place.

One of the tenets of agile development is the self-organizing team. It is everybody's responsibility to continuously improve the team's work, not just the project manager. Being empowered to make changes in your process means you're responsible for the process. Don't ever assume anyone else knows something you don't. If you think your team is doing something wrong, say so, no matter what position you have on the team.

Collective code ownership does help in this regard by removing some psychological barriers to code inspection. Classic code reviews can be nasty affairs. A row of accusers sitting across the table from a solo defendant is the image that comes to mind. With true collective ownership, the team can calmly talk knowledgeably about improving "our" code instead of pointing fingers to "his" code.

Things to Learn

"Life moves pretty fast. If you don't stop and look around once in a while, you could miss it." -- Ferris Bueller

The art of software development moves fast. We're still a young discipline compared to engineering and we're still learning new and better ways to create software. Last year I sharpened my saw by adding mock objects, dependency injection/inversion of control, WinForms, and "BuildMaster" skills to my toolset. I spent a ludicrous amount of time on StructureMap though and let other new skill development and learning slide.

Here's the things I want to learn more about this year:


  • Ruby and Ruby on Rails. I'm still a little dubious about the dynamic typing, but too many people are effusive about Ruby's potential to ignore this very much longer. If nothing else, Ruby should grow into the language of choice for scripts and what I call "NAnt-ey" tasks.
  • Ajax. I did a lot of IE specific Ajax development several years ago (before Google named the concept) and had quite a bit of success with it (a mission critical supply chain application and a dispute resolution workflow system, both still running). At the time it wasn't really a good platform choice because of the lack of developer familiarity with the techniques (plus IE5 had *huge* memory leaks with heavy DHTML manipulation). Because of the new hype from the Google apps and cross browser testing tools for DHTML like JSUnit and Selenium, Ajax development is looking like a viable choice now. I'll blog more on my experiences with this now that it is actually relevant.
  • Acceptance testing tools like NFit and Selenium. I love the concepts, but I think the NFit implementation could be better.
  • Finally finish reading Domain Driven Development
  • NHibernate. When .NET first came out in beta in 2002, I had dreams of creating the ultimate O/R mapper for the platform (use emitting for the mapping and use parameterized sql for better performance than NHibernate's purely reflective approach). I even wrote an article for C# Today on using reflection for O/R mapping. Alas, I didn't have the skillset then and I don't have the ambition today. Especially when NHibernate is there with an active development community and poised to become the de facto standard for O/R mapping in .NET. I know they have their proponents, but I don't count persistence generation tools like Neo, LLBGen, or Codus as O/R mapping.
  • Aspect Oriented Programming. I'd love to add some runtime AOP support to StructureMap.

My StructureMap tool actually started its life as an O/R and data access layer tool. What's actually there on SourceForge is just a refinement of what was supposed to be the configuration framework for the rest.