The Shade Tree Developer

The Shade Tree Developer is moving to CodeBetter.com

2005-07-18T09:43:00.000-05:00

Sorry for the inconvenience, but I'm moving The Shade Tree Developer to CodeBetter. Besides keeping some great company at CodeBetter, I'll get to have a bit better blogging infrastructure like trackbacks and categories so you can skip over my recent rantings about legacy code to get to the TDD content. I might also try to remember enough about CSS to get a stylesheet that doesn't make my code samples look fugly. I'll put forwarding links on the blogspot location for the older posts.

The new address is http://codebetter.com/blogs/jeremy.miller/.

The new RSS feed is http://codebetter.com/blogs/jeremy.miller/rss.aspx.

An Atom feed is available at http://codebetter.com/blogs/jeremy.miller/atom.aspx.

Resources from the Continuous Integration talk last night

2005-07-15T07:53:00.000-05:00

If you were at the Continuous Integration talk last night at the Austin SPIN, here is the set of resources I mentioned. The slides from the presentation are here or at http://www.jwspro.net/aspin/. Thanks for coming out last night in the rain and putting up with me.

Articles

Continuous Integration by Martin Fowler and Matthew Foemmel
Evolutionary Database Design by Martin Fowler and Promod Sadalage
Integrate Often from extremeprogramming.org
Driving on CruiseControl by Lasse Koskela. Setting up CI in the Java world.
The Agile Difference for SCM by Brad Appleton from CM Crossroads
Mike Roberts on multi-project continuous integration

Tools

I published my company's .Net tool usage with descriptions and links here.
CruiseControl (Continuous Integration for Java)

Books

I know there has been a couple of books published on CI now, but I don't know any to recommend. I heartily recommend Michael Feather's Working Effectively with Legacy Code for retroactively applying Test Driven Development and CI.

Bunch of Good Links

2005-07-14T13:49:00.000-05:00

First, maybe the best blog post I've every read from Mike Spille. I've been guilty of the "guardrail to guardrail" effect a time or two. On a project last year we probably did some dumb stuff because of someone else's negative experience with a certain technology on his previous project.

Read the comments too --> "BTW, you left out the obligatory insult to VB."

In a similar vein, James Bach warns us against being absolutist about "Best Practices." One of my favorite quotes is "A maturity model is basically a gang of best practices hooked on crystal meth." Just to add my 2 cents worth, I think it's best to always keep an eye on the first causes. I've written before that a centralized architecture group handing down black and white mandatory best practices from Mount Olympus to the pitiful wretches in the trenches is a terrible organization anti-pattern. There is always some kind of exception to every rule and the guys doing the actual work should be capable of exercising their own judgment.

Lastly, integration guru Gregor Hohpe asks what the real difference is between coding and configuration. Here you go Gregor, the difference is that I can write unit tests for code and use a debugger to troubleshoot code. I don't know why people keep inviting Gregor to these kinds of conferences because he just mocks the proceedings in his blog later.

My StructureMap tool is very configuration intensive and I have an unnatural tolerance for external XML configuration. To compensate, I've tried to provide the usage of .Net attributes for some configuration to be closer to the code. I've also had to invest a lot of energy into creating better tools for troubleshooting and validating configuration files in StructureMap. Since I dogfood StructureMap, expect more and better troubleshooting tools (because my colleague gets irritated otherwise).

Impressions from the VSTS talk last night

2005-07-12T10:00:00.000-05:00

I went to a talk on Visual Studio Team System last night given by Chris Menegay at the Austin DotNet Group. I tried to go into this with an open mind, but my bias is clearly with open source tools and agile processes. I'm still not convinced VSTS provides any value beyond our current tools and process, but it'll help somebody. Either way, I think VSTS is a valuable addition to the .Net world. If nothing else, it's helped introduce lifecycle practices to shops that have none and started some healthy conversations.

I wrote this several months ago, and nothing I saw last night substantially changes my opinion.

http://jeremydmiller.blogspot.com/2005/04/bypass-vsts-and-get-yourself-poor-mans.html

Without further rambling, cue the "wah, wahhh, wahhhhhh" theme music and let the cliche begin...

The Good

I think VSTS could be great for project tracking. Being able to intelligently log checkins to a user story or a bug fix with the time spent on the task could be a great boon to project management. The Heisenberg Principle always applies to this kind of tracking. If I, the developer, have to go out of my way, my velocity is going to be slower. VSTS might change the cost/benefit equation somewhat. A lot of any Agile process is the ability to accurately forecast user stories during iteration planning. Using a good "yesterday's weather" report from past iterations could certainly help. We're still going to take a look at Trac though as an alternative.
The code coverage visualization is awesome. It's not clear yet, but it might be possible to create test coverage reports from other tools outside of VSTS (NUnit, NFit, etc.)
It is an integrated suite of tools that will be supported by MS. I personally don't buy the argument that assembling a stack of NUnit/NAnt/CC.NET/NDoc/Subversion is really that hard, but I've been immersed in the agile world for awhile and I'm comfortable doing that (I've also worked with some of the NUnit and CC.NET folks too, so I'm not really impartial). For a shop with no experience in these tools or with a hostile policy towards open source, I can see the value of buying the whole stack in one place.

The Bad

Call me close-minded, but I wouldn't touch MSF with a ten foot pole and a firewall in between. The process templates are editable, but it doesn't sound like there is a GUI yet for doing the editing. I'll be interested to see if anybody (with much more patience and ambition than me) tries to make an XP or Scrum template.
At this point it doesn't look like you can use any other source control system than what comes with VSTS. As much a pain as it has been trying to move from VSS to Subversion, I'm not very excited about another migration. The source control is clearly better than VSS (how could it not be), but I didn't see any particular value over Subversion today.
VSTS does not support Continuous Integration, only scheduled builds. You'll need to continue using CruiseControl.Net. I fully expect the CC.NET guys to have a plugin for VSTS and/or MSBuild for CC.NET by the end of the year.
The web testing looks weak to me. I still think we're looking towards either Selenium or Watir.

The Ugly

From the initial impression, the MSF for Agile process looks like a heavyweight process. It is clearly meant as an iterative process. While that's certainly an improvement on the older MSF, iterative alone doesn't automatically translate to being agile.
VSTS is clearly aimed at very large companies using laborious, high ceremony processes--everything that my employer is not. I think process automation is a great thing, but if your process is so complex that nobody can know the whole, I think you've got some serious issues.
Pricing. When alternative tools are available and generally free, I have a hard time justifying the cost.
The one piece of the presentation where I thought Menegay was completely off his rocker was the idea that VSTS became a directed workflow tool to tell everybody (expecially developers) exactly what they should be doing. As a tracking tool for project record keeping, I think VSTS is great. Using something like VSTS as your primary communication channel is absurd. Turning developers into mindless zombies doing exactly what the VSTS workflow tells them to do sounds like a recipe for disaster to me. Not to mention a steep drop off in developer retention.

Don't play with strange databases...

2005-07-11T10:03:00.000-05:00

...you don't know where they've been, or who's been touching them.

I've been burned pretty badly in the past by making assumptions about existing databases. Database documentation isn't always helpful either, because it is out of date or just wrong to begin with. Even if the documentation is "correct," the corresponding code may be using it differently than the database designer intended. Even worse, you often have to interpret the data, and that's dangerous when you don't know the database. The only accurate source of information is usually the subject matter expert for the system.

What's the answer? I don't know. All I can say is to approach someone else's database with great caution. Kinda like going up to a big, mean looking dog and trying to make friends. I wouldn't make any kind of assumption about the meaning of any table or field. Put some effort into understanding the schema and expect to go back and forth a little bit.

It's the year 2005. Isn't the existence of SOA supposed to eliminate the need to be doing integration directly against a database? I'm writing a little code to integrate our main application with another 3rd party system. The only way I have to do this right now is to just write SQL queries against their database schema. I feel dirty.

Even worse is when shops start writing their own extensions and back door queries against a 3rd party application. That's a great idea. Take a database that you're not supposed to touch directly, and that you don't fully understand, and write all kinds of custom code into it. That'll really make it easy to upgrade the software package later.

Don't Make It Worse

At a previous employer they use an ancient inventory system that is written in a rare 3GL language. If you think of your IT infrastructure as the cardiovascular system of a manufacturer, the inventory system is the heart muscle. Everything else talks to the inventory system. The business logic of the system was bound up in the UI screens, so the only front door integration point was a screen emulation package from the vendor that was only certified for low volumes. At this point an intelligent IT organization in the SOA era would start looking for alternatives. My old employer beat the integration problem by writing hundreds of PL/SQL procedures to duplicate the business logic in the screens in lieu of a real service layer. To support other functions they added about 300 custom tables to the out of the box database. Just to make things worse, the database customizations were different region to region.

At one point I needed to write an integration from the inventory system to my application. I went and asked the SME where I could find a certain piece of information. He told me to look in table A and I turned around to leave. He then said, "you could also get it out of table B, or table C come to think of it." Oh, ouch. When I left they were struggling with a batch job that was supposed to run nightly that was taking about 25 hours to run and locking all kinds of database tables. They've experienced some problems with scalability, who would have guessed?

Even though the system is horribly obsolete and arguably an opportunity cost slowing down new development, they have no chance of upgrading to the newer J2EE version with web services because their database is way too wired. Idiots.

Harris Boyce warns us that O/R isn't a silver bullet

2005-07-11T09:09:00.000-05:00

Harris Boyce wrote a response to my ADO.NET contagion post: "Soul Vaccination" for Data Access Layers.

I think that Harris is quite right in warning us about overusing Object/Relational mapping.

While there are still plenty of scenarios where I would forgo an O/R approach for the classic data access layer, I still think that the "Persistence Ignorant" (POCO/PONO/POO whatever) approach for business classes is the way to go for the sake of testability. I think the decision hinges primarily on the amount of business logic in your application (and secondarily on the comfort level of the developers with OO coding in general). Even if I'm not using an O/R tool, I usually use some type of Inversion of Control to keep the rest of the code loosely-coupled from the data access.

Before diving into O/R mapping, you might take a look at these resources. Understanding some of the underlying patterns and mechanisms of O/R mapping might help alleviate some of the pain or help in making design decisions.

Patterns of Enterprise Application Architecture by Martin Fowler has a couple of good chapters about O/R patterns. The chapter on organizing business logic is a canonical read for deciding whether or not to use a Domain Model approach that would lead to using O/R for persistence.
Applying UML and Patterns by Craig Larman has a great chapter on creating a small persistence framework. This book is my choice as the best introduction to OO programming around.
This article from Scott Ambler introduces the basics of O/R
My article on C# Today from a couple years ago, but it's derivative of the guys above.

Continuous Integration Presentation at the Austin SPIN

2005-07-10T17:43:00.000-05:00

I'm giving a presentation on Continuous Integration at the Austin SPIN meeting on Thursday, July 14th. The meeting info is here. It's an adaptation of the talk Steve Donie and I did for Austin Agile a while back, just less .Net-centric for a broader audience.

I'm excited (OT)

2005-07-10T14:01:00.000-05:00

Communications technology is undeniably changing our world. "The World is Flat" type effects may act as a downward force limiting our salaries, but my wife just let me sign up for the NFL Sunday Ticket and that's pretty darn cool. I live in Texas, so every Sunday at noon my football viewing choices consist of mediocre Cowboys on one channel and mediocre (but improving) Texans on the other channel. I grew up in Missouri, so I'm a lifetime Kansas City Chiefs fan. Thanks to a little technology I'm gonna be watching my beloved Chiefs in my living room this year instead of trying to sneak over to a sports bar to see the game.

And if the Chiefs' new linebackers still can't stop my Grandmother from running up the middle, I'll just change to a different game.

When I was a Mort...

2005-07-08T08:57:00.000-05:00

Yesterday at lunch we were discussing (mocking) Rhocky Lhotka's defense of "Mort's" on his blog. What the heck is a Mort you ask? A Mort is supposed to mean an opportunistic programmer who is most concerned with delivering business functionality rather than wasting any time on silly ivory tower concepts like technical quality. Unfortunately the term "Mort" has become a pejorative term synonymous with low skill developers using data aware widgets to drag'n drop their way to one tier applications. I'll admit that I commonly use the term Mort as a putdown, but there was a day and time when I was a Mort, too.

My first programming experience was writing some ASP and Access tools for my engineering group in the late 90's. At that time the engineering and construction world was pretty crude in terms of IT automation. There were a lot of data silos, and quite often the most junior engineer (always me) was stuck manually typing information contained in one database into Excel sheets. We had to generate a lot of paperwork and track a lot of audit style information. I hated the paperwork aspect of engineering (hence my gravitation towards agile processes), so I set out to create some automated tools to create the Word and Excel documents from information in an Access database that was edited by ASP pages. This work led me to a position with the project automation group creating an ASP system to verify and manage a very poorly written, but mission critical, data exchange.

I clearly created a lot of business value and I was pretty proud of myself. Then I left the company to relocate to Austin, and everything I left behind collapsed in a few months because no one could support it. Some of it was rewritten by actual IT folks (and consider this a long overdue apology to you all), but most of it just disappeared. So what did I do wrong? Here's a laundry list of really bad things I did because I just didn't know much about good practices in software development.

Source control? What's that?
(Boss) Jeremy, where is this stuff running? (Me) It's running on personal web server on my box. Hey wait a minute, what's that awful sound coming from my hard drive and why won't my box reboot anymore?
(Boss) Do you have that stuff backed up? (Me) What does 'back up' mean?
Coding directly into a production database of a 3rd party product that was notorious for database corruption on projects ranging from little 100 million dollar projects to multi-billion dollar projects
ActiveX controls on ASP pages. Believe it or not we had some issues with installation.
ASP pages running against MS Access via ODBC in production
Connecting directly to production Oracle databases from ActiveX controls on an ASP page
Using the old Remote Data Service (RDS) library. Great stuff for productivity, huge security hole. Using RDS meant that I had the user name and password for the fragile Oracle production database embedded in each ASP page where any yokel could go "View Source" and hack our sensitive data.

I was focused on business needs and delivered, but I was dramatically ignorant of anything approaching decent development practices or design. I distinctly remember one humbling episode where I asked a senior automation expert for help on comparing two different databases. He took a very short look at my several hundred lines of VBScript and banged out a SQL statement that did exactly what I needed to do in one line of code. His exact words were "you've got good ideas, but you would be much more effective if you just knew more." Ouch.

All right, I just shared some of the truly stupid and ignorant things I did in my Mort days. What did you do in your Mort days?

What's so wrong with being a Mort? After all, they're pretty common and they definitely add value to their employers. They're usually closer to the business and have a better understanding of the problem domain. For one thing the Mort's of the world are in mortal danger if I have to work with any more awful Mort-ish code (regardless of what language it's written in). For another more serious reason, the Mort jobs are the first developers to be offshored. Lastly, if you're developing software, learn about your craft and climb the skill ladder. The whole Mort/Elvis/Einstein taxonomy may be crap, but the more you know the more effective you'll be and that seems pretty pragmatic to me. Make your great business solutions stand the test of time.

Great CMM quote at lunch today

2005-07-07T15:32:00.000-05:00

At the Agile Austin lunch today we had a great quote. We were discussing using a continuous integration build to auto-generate documentation specifically for CMM compliance when somebody said "[doing CMM paperwork] is like paying protection money to the mafia."

Quarantine the ADO.NET Contagion in your Code

2005-07-07T10:28:00.000-05:00

From my perspective the .Net world is clearly becoming more sophisticated in our approach to persistence. Reusable persistence solutions like NHibernate or NEO are starting to become more popular, reducing the need or desire to work with ADO.NET directly.

However, most of the .Net systems out there that I encounter still have old fashioned data access layers rolled by hand. This doesn't have to be that bad, ADO.NET is pretty easy stuff, and the .Net community is generally pretty strong in data access. The problem is that ADO.NET usage has to be done right, every single time. Forget to close a connection or an IDataReader somewhere, then add in enough volume through the system, and watch your application roll over and crash because the database has too many open connections. The same thing with exception handling occurs. You have to have the same "try/catch/finally" code every single time you use ADO.NET code.

The second problem is a little more endemic. Because ADO.NET is so easy to use, developers coding a web page or a business project give into temptation to just write the data access code right where it is needed instead of separating out the data access concern with a little thought. These people are often colloquially known as "Mort's." To Mort, please stop doing this. Especially if you're writing code that I have to support. Just in case it's not obvious, writing the ADO.NET code straight into a business object basically eliminates any possibility of writing easy unit tests.

My advice is to simply count the number of times the following lines of code show up in your application. Offhand, I'd say the maximum number of times this code should be called to be no more than 3-5 times.

connection.Open();
command.ExecuteNonQuery();
command.ExecuteReader();
connection.Close();

You can eliminate a grotesque amount of duplicated code by simply centralizing the low level ADO.NET manipulation into one or very few classes. It's much easier to write the error handling and connection management once than it is to get it right everywhere. It'll also allow you to create a better answer for tracing and instrumentation. All other higher level data access layer classes call into the central execution class to actually execute database commands. Other than the possible exception of DataSet's, nothing from the System.Data namespace should ever leak out beyond the classes that are specific to data access.

Here's an example of writing the inner code execution class from our data access layer:


using System;
using System.Data;
using StructureMap.DataAccess.ExecutionStates;

namespace StructureMap.DataAccess
{
 [Pluggable("Default")]
 public class DataSession : IDataSession
 {
  private readonly IDatabaseEngine _database;
  private readonly ICommandFactory _factory;
  private readonly IExecutionState _defaultState;
  private readonly ITransactionalExecutionState _transactionalState;
  private IExecutionState _currentState;
  private readonly ICommandCollection _commands;
  private readonly ReaderSourceCollection _readerSources;

  [DefaultConstructor]
  public DataSession(IDatabaseEngine database)
   : this(database, 
   new CommandFactory(database),
   new AutoCommitExecutionState(database.GetConnection()),
   new TransactionalExecutionState(database.GetConnection()))
  {

  }

  /// 
  /// Testing constructor
  /// 

  /// 
  /// 
  /// 
  /// 
  public DataSession(
                    IDatabaseEngine database, 
                    ICommandFactory factory, 
                    IExecutionState defaultState, 
                    ITransactionalExecutionState transactionalState)
  {
   _database = database;
   _factory = factory;
   _defaultState = defaultState;
   _transactionalState = transactionalState;

   _currentState = _defaultState;

   _commands = new CommandCollection(this, _factory);
   _readerSources = new ReaderSourceCollection(this, _factory);
  }



  public int ExecuteCommand(IDbCommand command)
  {
   try
   {
    return _currentState.Execute(command);
   }
   catch (Exception ex)
   {
    throw new CommandFailureException(command, ex);
   }
  }

  public int ExecuteSql(string sql)
  {
   IDbCommand command = createCommand(sql);
   return this.ExecuteCommand(command);
  }

  private IDbCommand createCommand(string sql)
  {
   IDbCommand command = _database.GetCommand();
   command.CommandType = CommandType.Text;
   command.CommandText = sql;
   return command;
  }

  public IDataReader ExecuteReader(IDbCommand command)
  {
   try
   {
    return _currentState.ExecuteReader(command);
   }
   catch (Exception ex)
   {
    throw new CommandFailureException(command, ex);
   }
  }

  public IDataReader ExecuteReader(string sql)
  {
   IDbCommand command = createCommand(sql);
   return _currentState.ExecuteReader(command);
  }

  public ICommandCollection Commands
  {
   get { return _commands; }
  }

  public IReaderSourceCollection ReaderSources
  {
   get { return _readerSources; }
  }

  public void Initialize(IInitializable initializable)
  {
   initializable.Initialize(_database);
  }

 }
}

using System;
using System.Data;

namespace StructureMap.DataAccess.ExecutionStates
{
 public class AutoCommitExecutionState : IExecutionState
 {
  private readonly IDbConnection _connection;

  public AutoCommitExecutionState(IDbConnection connection)
  {
   _connection = connection;
  }

  public int Execute(IDbCommand command)
  {
   try
   {
    setupConnection(command);
    return command.ExecuteNonQuery();
   }
   finally
   {
    cleanupConnection(command);
   }
  }

  private void setupConnection(IDbCommand command)
  {
   command.Connection = _connection;
   _connection.Open();
  }

  private void cleanupConnection(IDbCommand command)
  {
   try
   {
    command.Connection = null;
    _connection.Close();
   }
   catch (Exception)
   {
   }
  }

  public IDataReader ExecuteReader(IDbCommand command)
  {
   try
   {
    setupConnection(command);
    return command.ExecuteReader
(CommandBehavior.CloseConnection);
   }
   catch (Exception)
   {
    cleanupConnection(command);
    throw;
   }
  }
 }



}

In this example, all data access classes have to use a class called DataSession
to execute an IDbCommand object or a sql string. The DataSession class delegates to an IExecutionState interface object. The AutoCommitExecutionState deals with all of the connection opening and shutting, as well as providing a consistent error handler to report the actual sql (with parameters) that failed. By centralizing this operation to one spot, we could add extra code to provide more instrumentation or simply change the error handling across the board. Most importantly, other classes don't have to be obfuscated by mucking with so much ADO.NET manipulation.

My Poor Friend

One of my previous career stops was at a very bad internal IT shop. The leadership in our division was making it painfully clear that developers were merely a commodity, and undeserving of anything like good working conditions, raises, or a career path. After years of abuse working on a truly horrendous VB6 system (inspired by the architectural writings of a "Super Mort"), a friend of mine named Scott secured a transfer to a different organization. Scott was baited by promises of getting to do hard core J2EE development, the hottest thing going at the time.

Upon arrival at the new digs, Scott was told they needed him to help clean up the existing system first to improve stability before he could work on the new J2EE work. It turned out that the existing system was a 900 page ASP classic application that had been written by people that had grown into a mission critical application. The instability problems turned out to be because none of the shadow IT developers had closed any of the ADO connections in the ASP code, causing the database (I don't know what the database was, but it wouldn't surprise me if it had been MS Access) to crash and burn with too many open connections. Scott's new Job? Go through each and every ASP page and add code by hand to close connections, commands, and recordsets.

I know what you're thinking, just write a fancy Perl script with some regex magic and go on. No such luck, the code was too inconsistent. Just in case you've never been afflicted by classic ASP code, take the worst code you've ever written and put it into a blender. That's about what you'll find in ASP (my ASP code circa 1998 was pretty smelly).

I Don't Think Much of the DAAB

Just to be opinionated, I don't see much value in the original Data Access Application Block or the improved replacement in the Enterprise Library. In my opinion, they just don't add much value. It's still a low level API with not much there to guarantee good data access practices. I know a lot of people like them and they're better than nothing, but there are so many better tools out there. For that matter, I just don't think it is difficult to make a data access layer specific to your application that is superior to the DAAB in terms of connection management, transaction management, error handling, and instrumentation.

Attitudes towards Stored Procedures

2005-07-06T10:25:00.000-05:00

My philosophy towards stored procedures has shifted so much over the past five years that I'm not sure I have a strong opinion on them anymore. At one point I would have told you with great fervor that you had to be doing data access through stored procedures to have a good application architecture, now I'm not quite sure. On the other hand, I've seen developers write hundreds of lines of procedural code to do set-based manipulations that would have been relatively simple to express as a SQL statement.

For me today the decision to use stored procedures comes down to the particulars of the application, the comfort level of the rest of the team, the relationship with an application DBA (if he/she exists), and the relative pain or ease of configuration management for database code. For whatever reasons, a lot, if not most, shops do a terrible job with configuration management for database code and structure. As a consultant I worked on an engagement with a client that went through way too much manual process rigamarole just to get a stored procedure moved into the development server. Of course, they handled the database code moves in an unrelated process separate from the code, allowing all kinds of bad things to happen.

I spent a lot of my early career writing Oracle PL/SQL, so I’m pretty comfortable with using and writing stored procedures in general. And to be honest, I think a lot of ex-DNA developers (victims) like me are inclined to write sproc’s because they were often easier to change and deploy than COM(+) DLL’s, especially with a clustered server architecture. Case in point, my current employer has a deep VB6/ASP/Sql Server background and stored procedures in T-SQL run rampant throughout our architecture. Developers who aren't comfortable with object oriented programming will often use stored procedures as an extensibility mechanism (I don't like this, but it does work).

When I moved to a shop doing TDD that was mostly stocked with J2EE veterans I was a little shocked at their disdain for stored procedures. In their world order, stored procedures were evil. Not very many of them could explain exactly why stored procedures were problematic, but they all knew it for established fact. Later on I started to understand their point of view. J2EE has a multitude of proven persistence mechanisms that isolate middle tier code from the database and eliminate a lot of manual SQL creation. In most cases, there was also a fear that using stored procedures would inevitably lead to business logic in the database. A general discomfort with relational databases probably explains a lot of the hostility towards stored procedures in some developer circles.

For many people the most important reason was that stored procedures are inherently more work to test than plain old objects. Just as importantly, unit tests that touch the database are slower to execute. A domain model approach coupled with some kind of metadata-driven persistence mechanism can lead to much easier unit testing and faster development overall. Stored procedures often simply interfere with persistence strategies.

Some O/R tools support stored procedures, but it’s often more trouble than it’s worth. If you want both O/R mapping and stored procedures, I’d actually recommend rolling your own tool. Writing a complete solution like NHibernate is hard, but creating a just good enough custom tool isn’t that bad. Reflection in .Net is pretty easy to use, and the System.Convert class makes type coercion relatively painless. If your domain model is small, just hard code the database mapper classes and move on with your life.

Enter the DBA

Another obvious factor in an organization’s attitude towards stored procedures is the relative strength and influence of DBA’s. Developer/DBA conflict can be even more harmful than the spitball fights between developers and testers. If a DBA is the only person allowed to write SQL, or has to approve every piece of SQL, you’re going to go slower. I hate having this kind of territorial bottleneck in the development process, but when you’re stuck with this situation my advice is to:

Keep your DBA engaged in the development work of the rest of the team. Don’t let them work in a vacuum. It helps tremendously if the DBA actually has a stake in the success of the project.
Aggressively use mocks or stubs to build business logic and user interfaces so you aren’t stuck waiting on a DBA. This also has the advantage of firming up the data access requirements before engaging the DBA.
The DBA must follow the exact same configuration management practices as developers. Ideally, you want a DBA to check any changes into source control and monitor the continuous integration build and fix any problems from their check-in's. Stored procedures are just code and should be treated as code. Make sure you help the DBA out with check in policies because it is a different world for most of them. It’s also cool as a way to give the DBA’s some visibility into the project at large. I'm plenty paranoid about this one because of some early issues on a project last year when the DBA decided to start optimizing some PL/SQL and broke the application for a while.

Good and Evil in the Garden of Stored Procedures

2005-07-05T13:03:00.000-05:00

Periodically I see or engage in an argument over the usage of stored procedures. I think a lot of these types of arguments are somewhat unproductive because the combatants are thinking and talking about radically different things. Besides, I think the argument over the proper role of stored procedures in data access is often obfuscated by the misuse of stored procedures for things other than data access. My opinion has changed quite a bit through the years, so I try to stay out of these arguments.

However, my development group has to make a change soon in our philosophy for using stored procedures (or hit a brick wall). The way my company has used T-SQL in the past is jeopardizing our future development (besides, I’m an Oracle guy and I think T-SQL is butt-ugly). We’re a product development company that has grown by leaps and bounds from a startup venture and the existing codebase has grown somewhat chaotically. To keep the growth going and accommodate new customers both here and abroad we’re going to need to localize the application for different languages and port our main application to Oracle. Toss in the ongoing adoption of TDD and continuous integration as a means of creating better software quicker, and the usage of stored procedures has to be reevaluated.

What’s Good about Stored Procedures

Before I start ranting about the grotesque misuse of stored procedures I’m dealing with, here is what I think is useful about stored procedures.

Using stored procedures externalizes the SQL from the compiled code. I really like having the SQL in an easy to read format instead of built by string concatenation in the code. This is also great if you need to optimize the stored procedure for performance. I haven’t seen that come up very often, but it only seems to happen on systems where the SQL is embedded in the code. There’s also the issue of allowing a DBA to alter or write stored procedures independently of the code. I have generally had control of both database and code throughout my career, so I’m always suspicious of DBA’s writing stored procedures instead of developers. I do think that dynamically constructed SQL can result in systems that are hard to debug, but that can be beaten with just a little judicious instrumentation in your data access layer.

Performance is an advantage of stored procedures, but maybe not enough by itself to justify stored procedures over other approaches. I think the performance advantage is either largely mitigated by the ability of newer database engines to cache execution plans or simply a case of premature optimization. If you have an opportunity to batch up a set of updates or simply reduce the amount of information you send over the wire to the database a stored procedure certainly leads to a performance improvement. Otherwise, I think the stored procedure advantage is too slight to be a serious consideration.

Security is another advantage that is often cited as an advantage of using stored procedures. On one hand, stored procedures quickly mitigate vulnerability to SQL injection attacks. On the other hand, the database can be tightened down so that no ad hoc SQL can be executed, stopping security breaches right at the start. Personally, I abhor this idea myself because it drags down the development team’s velocity. I don’t see why it’s necessary, but it is an option.

Most of this goodness can be happily achieved with nary a stored procedure. Parameterized queries can eliminate SQL injection vulnerabilities while providing comparable performance. The data access layer we built (I might release this as part of StructureMap someday) for our last project already externalizes SQL from the code for ease of maintenance and decoupling from the database.

When Stored Procedures Go Bad

While there is nothing inherently evil about stored procedures, developers often misuse stored procedures. I think stored procedures should be nothing but basic CRUD. I wrote in an earlier post that “IF/THEN/ELSE” logic in stored procedures is a design smell.

Stored procedures are a terrible place to create business logic. Development environments for the .Net languages or Java are far better suited for developing applications. Intellisense support, debugging, and refactoring are all inferior with stored procedure languages. T-SQL is a procedural language that pretty well forces you into the transaction script style of organizing business logic. One thing I’m observing in the reams of T-SQL I’m wrestling with is the obscene amount of code duplication. Now that we have to support Oracle as well, this duplication stands out like a sore thumb. An OO language can do a much better job of code reuse.

Here’s my list of things I’ve seen put into stored procedures that should never be in a stored procedure.

User input validation that creates the validation messages seen on user screens. I can’t even begin to describe how stupid this is.
Creating HTML, enough said.
Security authorization logic. This is definitely a middle tier responsibility. I’ve used joins to some sort of permission table in select statements to filter records. I think that’s fine, but using procedural logic or even role checks inside stored procedures is nuts. Security is volatile and sensitive. Put it where it’s easy to test. For that matter, always leave a way to test your business logic without security checks if you can. Security always makes code harder to unit test. By wrapping security up with business logic in a bag of stored procedure goo, you’ve essentially shot yourself in the foot for testability.
Business logic. It’s easier to write and test in C#, period. Splitting logic between stored procedures and middle tier code is even worse, but I’ve seen it done many times.

Insight from Scott Bellware

2005-07-01T15:58:00.000-05:00

Fellow Austinite and champion ranter Scott Bellware has some great insight on the impedance mismatch between application architecture and the newer SOA paradigm.

Overall, I think SOA is an advance in the construction of software solutions, especially in large IT infrastructures. What always bugs me is the seeming disdain (or ignorance) the SOA guys have for good OO or even application design. I'm familiar with the situation and perpetrator of the (very foolish) guidance of mandatory SOAP web services between the UI and backend (even if the code is all running in the same process space!) "just in case" they need a connected service later. In no way does the existence of SOA services abrogate the need to create well-factored solutions. Write a well-structured code base and your application will always be able to expose SOA endpoints later. And if you're doing SOA, keep the internals of your service or system completely independent of the messaging transport. Like Scott implies, serializing your real domain classes as part of the SOAP contract strongly couples your clients to the internals of your service.

Create new entry points to enable unit testing of Legacy code

2005-06-30T12:41:00.000-05:00

If you've been reading the Shade Tree Developer this week you'll know that I'm fighting with a legacy component. I was trying to get a piece of code into NUnit to test my new functionality. A class in the stack works by giving it a key value to a record in the database. I tried for awhile to use an existing stored procedure to push in test data. Guess what I found? I had relational integrity issues with setting up the test data, but so what, that's just what a database is. I wasted a bit of time pushing past the referential integrity by getting the correct data and stumble right into security authorization being performed inside the f***ing stored procedure!

It was about this time I remembered what I had been reading in the Feathers book on legacy code. Sometimes you have to change the code just to get it into the test harness. A quick "CTRL-ALT-M" action with ReSharper, and voila, an entry point into the business logic that bypasses the database and security checks. Add some ObjectMother magic to construct the fugly data structures, and I've got a working unit-ish test.

Moral of the story? Don't be afraid to change the code first before writing a test (well, maybe afraid but not terrified). Remembering to backup and look for an easier way to write a test was probably a good idea too.

External configuration should be human-readable

2005-06-29T11:52:00.000-05:00

Ranting ahead...

For the love of all that is holy and good in the world make all of your external configuration readable and editable by human beings just in case I have to maintain your system later. Yeah, I know sometimes you have to encrypt configuration for those pesky security auditors, but at least make the unencrypted data readable. First of all, the mechanics of making little configuration tweaks shouldn't be difficult. Second, debugging and diagnosing problems is awfully difficult when you can't even read the configuration.

Xml by itself doesn't bother me, but I'm dealing with files that are partially SOAP serialized. "Simple" whatever protocol my ass. SOAP is not human readable!

Hulk Angry When He No Have ReSharper

2005-06-29T10:30:00.000-05:00

I'm a ReSharper junkie, and I get angry when I don't have it. I've spent the majority of my .Net career working with converted Java developers. To a man, they all complained about the crudeness of Visual Studio.Net compared to IntelliJ or even Eclipse. I wrote them off as whiners because I was used to working in either VB6 or Visual InterDev (I still have a soft spot in my heart for doing what's now called Ajax development in VI) and VS.Net was clearly more polished (except for that little code and continue goodness). Then the ReSharper 1.0 beta builds dropped. My project team of ex-IntelliJ jockeys jumped at it immediately. Even with the sometimes spectacular instability of the ReSharper beta, it made an obvious improvement in coding efficiency. I know this because several times I started to code on the flight home without a working copy of ReSharper and became frustrated quickly when the code navigation and refactoring was missing. ReSharper quickly improved with each passing build, and the phrase "I'm having a ReSharper moment" left the project vocabulary.

Why am I complaining about this today? I'm wrestling with a legacy C# codebase this month and it's not entirely "ReSharper-able." One of Michael Feather's pieces of advice in his book Working Effectively with Legacy Code is to lean on the compiler when making changes to legacy code. Going farther, I really want to be able to use ReSharper's "Find Usages" capabilities to trace code in an attempt to understand interdependencies between classes and trace through code. In order to insert any kind of unit tests I'm having to extract methods, classes, and interfaces. The safest and quickest way to do this by far is to utilize the automated refactorings in ReSharper.

Out of the box the code could not be traversed by the static code analysis of ReSharper. The reasons? The solution files aren't using project references between projects. They are compiling all assemblies to a common hard-coded folder and making all references to this folder, creating a force field impervious to ReSharper code navigation and refactoring. This was largely done to get around problems they had with circular references between class libraries. A lot of classes are loaded and executed by reflection. Part of the problem is that the class library projects are much more fine-grained than they really need to be. I'm guessing that the assemblies were subdivided so granularly to apportion out the work between developers and avoid checkout conflicts with Visual SourceSafe's pessimistic locking, but I don't know that.

So far we're only tackling a small portion of the legacy code at first. The long range steps we're trying first are:

Get to project references immediately. Enable both ReSharper refactoring across projects and the almighty task in NAnt. It also solves some reference conflicts
Collapse the assembly structure to mitigate the circular dependencies and simplify deployment and build
Eliminate as much of the load by reflection instantiations as possible. Minus automated unit testing, strong typing is absolutely essential. Since we don't have an automated NUnit test suite to speak of, we have to have the compiler enforce strong typing as a first line of defense when it's time to modify the code
Get rid of VSS, move to Subversion, and get some automated builds put together for continuous integration
Automate the migration of build products (think shared libraries, frameworks, and staged builds) between our different codebases. This is a soft spot in our infrastructure at the moment that is slowing us down. We're certainly not the first team to struggle with this by any means.

Another thing I highly recommend; my partner is an uber-buildmaster and one of the first adopters of the original CruiseControl. Fellows like him are awfully nice to have around when you're in this sort of situation.

I have played with IntelliJ briefly when I thought I was getting a chance to go onto a J2EE web project. For the agile style of incremental development (red bar, green bar, refactor), IntelliJ is a coder's paradise. I think this largely due to JetBrains dogfooding their own tool using an XP process. I know pockets of MS use agile processes (and I'm dubious about counting MSF Agile in that column), but I don't think the VS team really takes it seriously. VSTS is clearly aimed at the heavyweight Rational Unified Process product.

VS 2005 with VSTS is a definite improvement for agile development, but I think it still falls short of what I've observed in IntelliJ or Eclipse. I'm not entirely sure that some of the enhancements like unit testing integration are actually as good as the existing open source equivalents, and the refactoring in VS 2005 is inferior to ReSharper. I'm honestly more excited for the upcoming ReSharper 2.0 release than I am for the improvements with VSTS.

Taming the Legacy Beast - Dose It With Some NAnt

2005-06-28T14:18:00.000-05:00

Yesterday I griped about legacy code. I carped about VSS, whined about the tight coupling, groused about the arcane configuration setup, and spewed invective about the stupid manual post compilation steps it took just to make the code work. Today it's time to start cleaning things up and applying some ibuprofen to the legacy headache.

Even though the work we're doing is supposed to be throwaway in a couple of months, we decided to "NAnt-ify" the development environment. First we're skimming the VSS project and setting up a new Subversion repository. The next step is to create a bare bones NAnt build to automate the application setup. The last step is a simple CruiseControl.Net configuration to get it into our existing continuous integration server (this is sooooo much easier than it was just a couple of years ago. Hat's off to the CC.NET guys).

If you can't easily make a repeatable build and get the application working on your own box in short order, you're not gonna go anywhere fast. That whole NAnt/CC.NET helps keep the code clean in source control so your coworkers aren't idle while you go hunt down the new class file you forgot to add. And one last time, VSS is a piece of crap. Using the far superior clients for Subversion (or almost anything else) that are actually intelligent enough to determine what files have been changed, added, or deleted makes for a smoother source control experience. And no, we're not going to wait until November for the promise of Hatteras in VSTS when Subversion is here now and free.

You know it's a bad day when...

2005-06-27T15:46:00.000-05:00

You're typing "iisreset" or "net stop iisadmin /y" a lot.

Legacy Code and the Life and Death of an Application

2005-06-27T11:50:00.000-05:00

My colleague and I are making minor extensions to some legacy code (in C#) for the next month or so (and that generally implies that the Shade Tree Developer will feature quite a bit of ranting in the near future). After that my team will be making some large scale architectural changes to an existing product. My job for the rest of the year is going to revolve around legacy code.

We very frequently make customer driven modifications to our application. We also have to keep the product ahead of the competition, and that means being agile (in the literal sense) with our development practices. Our existing codebase has some definite weak spots. The development organization has the beginnings of a long term "Grand Unified Theory" to transmogrify our existing systems and development environment to extend the company's investment in the current products and prevent the downward spiral into "legacy code."

Rewriting the applications is completely out of the question. So first, here's the symptoms we're trying to prevent and eliminate. As we get farther into the "Grand Unified Theory," I'll blog about our strategies for getting out of the legacy code trap.

Life and Death of an Application

In my finite experience and research, applications (or services if you're of the SOA persuasion) reach the end of their useful lifespan when the cost of altering or extending an application is greater or riskier than throwing it away and starting over. I've seen a lot of people make a parallel to the concept of entropy. The natural state of any system will tend towards entropy (in software terms, the big ball of mud). You have to exert energy into an application to reverse or stave off application rot (entropy).

Most applications undergo a twilight period where the previously shiny, well-factored application descends into a morass of spaghetti code that becomes more and more difficult to work with. The applications are still providing business value, but they are starting to be more of an opportunity cost rather than an enabler because of the increasing friction incurred by further modifying the application. This twilight slide into retirement is what most people mean when they say “Legacy Code.” The code works, but no one wants to touch it for fear of bringing the down the whole thing.

What is Legacy Code?

Here's a couple of takes: "Before we get started, do you know what 'Legacy Code' means? It’s code that works" -- via Roy Osherove.

Yeah, but this is true only if you leave it alone.

Michael Feathers described legacy code as "code without tests." What I think he’s driving for is code that can be safely modified and rapidly verified.

I'm going to add a corollary to Mr. Feather's definition; legacy code without tests is code that is difficult to test. Twice in the last year I’ve transitioned from greenfield development projects that were written with TDD to working with brownfield code that had not been written with TDD. In almost startling contrast, the test-first code was vastly easier to extend with new unit tests than the code written test-last. To paraphrase the noted development sage Ferris Bueller, “the legacy code was so tightly coupled that if you put a chunk of coal between the classes you would get a diamond.”

What made the legacy code made writing unit tests difficult?

Overuse of static methods. Static methods result in tight coupling. I ranted about this here. Keeping state in static fields makes it even worse.
Tight coupling to external configuration. It’s often impossible to run any piece of the code without an external configuration file. Dealing with configuration slows down unit testing. I’ll blog more on this soon, specifically ways to avoid the coupling.
Poorly factored code and tight coupling between concerns. Database, business logic, and presentation logic all mixed together.
Blobs

Legacy Code isn’t Necessarily Old

Of course, many applications aren't well-factored to begin with -- instant legacy code. Entropy also creeps in with any gap in communication or participation between the designer’s vision and the development team. I’m dealing with one of those right now. The original vision and design was thoughtful and innovative, but it was thrown over the wall to a team of developers who had not participated in the original vision. It’s particularly bad when the original prototype is coded by a skilled architect and then handed off to the developers with instructions to “just finish it off.” One of the advantages to continuous design is that the rough edges are smoothed down over the course of development. Remove the designer from the coding and those rough edges will only become sharper over time.

Shamefully, I think I’ve been responsible for a bit of this myself. One way or another the technical vision needs to be socialized and shared throughout the development team if you want to create a consistently structured application. Ideally, the development team should be creating the technical vision themselves.

The Development Environment Matters Too

Since much, if not most, of the cost of modifying an existing application is verification and regression testing, the ability to quickly migrate the application to a testing environment is paramount. Repeatable automated testing is such a great long term investment in an application that it’s a shame so few projects invest upfront in test automation. At this point in my career, I would consider development infrastructure like build scripts, database scripts, and continuous integration servers to be a vital part of the application itself. If that infrastructure is lacking or nonexistent, the application is that much harder to modify. I'm working with legacy code right now, and the lack of an automated development build with unit tests is making me feel, well, like I'm naked in a hailstorm.

The worst case I know of was a very large VB6 application that had been accreted around a nice tidy core over the course of several years. Simply setting up a test environment manually could take up to 10 business days, and even then they were never very confidant in the installation. That application was a core piece of the manufacturing system and changed often. Not only was the migration slow and hard, but the haphazard nature of the environmental control allowed several pieces of bad code to get through testing and into production, leading to factory downtime (and take it from me, factory downtime is a bad, bad thing. When the billionaire tycoon founder himself comes looking for someone’s head in IT, run away and get some plausible deniability). The real gallows humor was that the application was customized somewhat for each factory and business line and therefore had to be regression tested for each and every configuration.

"#region/#endregion" Tags Smell Bad

2005-06-27T07:56:00.000-05:00

Ranting ahead...

I'm unilaterally declaring overuse of the "#region / #endregion" blocks in VS.NET a code smell. When I first started playing with VS.NET in 2001 I thought this feature was going to be the best thing since sliced bread. Boy, was I wrong. It seems like every time I expand an innocuous looking region out spills a big bag of entirely unrelated code.

Much like the excessive comments code smell, the existence of a large code region generally implies that the code in the region really should be in a different method or even a different class. The region block is no substitute for good factoring. The region block is often used as just a bit of perfume sprayed over smelly code.

If you want what I would call a bad example, go take a look at the source code of the original application blocks from Microsoft Patterns and Practices. Instead of factoring functionality, they wrote more or less procedural code segregated by region blocks that were nested 4-5 layers deep. From what I understand, the code style was mandated by their internal QA code standards. The result is thoroughly unreadable code. Finding the code that does anything is like playing a game of hide and go seek in a thick forest.

The .Net community's tendency to take anything written or published from MS as the gospel without any kind of critical analysis is a huge pet peeve of mine. The original application blocks were a good example. The code was rotten and they were reportedly buggy in spots -- but I saw nary a word of criticism anywhere.

What an Amazing (Code) Smell You've Discovered

2005-06-23T15:02:00.000-05:00

I'm naming a new code smell today, "Obsessive Tracing." You know exactly what I mean. If you see a long method or class with a *lot* of Debug.WriteLine("1") or Debug.WriteLine("I'm in here now!") methods sprinkled throughout the code, it's a good bet the code smells to high heaven. Those trace statements are in there because the code is prone to breaking and hard to understand.

The point of a code smell is to recognize a problem so you can begin to move in a different direction. I'm not sure what you do with legacy code, but with new code the key in my opinion is a well factored solution for ease of understanding and strong unit testing. Excessive amounts of debugging often means your unit testing isn't granular and comprehensive enough. Excessive debug statements might also mean a developer could benefit from reading up on the capabilities of their debugger.

Geek points for nailing the movie line in the title. No Chris Fields you don't count, that one's too easy for you.

Flimsy evidence that agile delivers the goods...

2005-06-23T07:51:00.000-05:00

Does Agile development actually work in real life? Here's anecdotal proof in the affirmative (since the XP Refactored book sold a bunch of copies mostly based on negative anecdotes I figure I can get away with this too).

Production Moves in Ceremonial Waterfall

Production moves in big IT waterfall land are living nightmares of pure desperation and panic just waiting for the other shoe to drop. We usually had most of the team work an entire weekend in shifts just to be on hand as code and data was manually migrated. It's not unusual to see the cots come out in project warrooms. I don't think that's what the Godfather gang have in mind when they "take it to the mattresses," and the mafia seemed to have more fun with it than we did. As I've mentioned before, teams often made last minute coding changes right before go live because integration problems weren't discovered until making the migration and everybody was afraid to miss the installation window.

Agile Deployment

Shifting attention back to the shiny agile world of Test Driven Development and Continuous Integration, we made a final push into a production staging environment with our new product yesterday (with no known defects I might add). After yesterday's move, I left work on time and took my toddler to the YMCA pool with nary a cell phone at hand because I was perfectly confident everything was fine (I'll probably pay for that this morning).

On a previous project last year for a previous employer we made our second release one Tuesday morning. I was quite happily working on a user story in the mainline code branch for the following release when the project manager came up to tell us the new release had been put into production without a hitch. The developers had completely forgotten about the production move!

Where I Think Agile Development Helps

So why do I think the production moves went so much better with agile development than my earlier projects in waterfall world (to be fair, the earlier projects were COM based with a fair amount of middleware and the later projects were .NET systems)? The waterfall shop had a admirably organized (but radically inconsistent) ceremonial change management process, but pitiful build management. I think most of the reasons I list below are applicable for any disciplined iterative and incremental process (I'm thinking RUP here), but agile processes put more emphasis on early and frequent feedback over almost anything else.

Earlier, more frequent, and repeatable testing. Test Driven Development is really a design technique as much as anything, but it certainly creates a large body of automated unit tests that provide a safety net to keep new bugs from emerging. Add any kind of automated acceptance or regression test suite and that safety net gets much tighter.
Continuous Integration. CI can be thought of as a lot of trial runs at automated code installation. Much the same way that TDD forces you to write code that is testable, maintaining an automated CI build forces you (if your thinking cap is on straight) to think hard about making your application or service easier to deploy and version. Some people will excoriate me for this statement, but I think a NAnt build is the best documentation for system installation because it unambiguously specifies what has to happen for the application to function (and it tends to stay synchronized with reality easier than a document). I'd still supplement that with a Wiki page though.
Defects can be fixed faster and stay fixed. CI allows a team to push code fixes to testing much faster without sacrificing environmental control. In ceremonial waterfall land, code pushes required filling out forms and convincing an external group to make my code move first, often up to a six hour process if other projects got there first. By automating builds under the control of the project team, I think the control over the environment is much better even without a controlling environment authority outside the development team, and certainly much faster. When you do write an automated test for a defect as you fix it, the defect stays fixed. Lost time due to regression failures can be mitigated by automated testing.
Project teams are, or should be, more self-reliant and self-organizing. What I mean is that a project team should be able to carry out its own configuration management without waiting for a sluggish external group. In agile projects and organizations I've consistently had fewer dependencies on external IT service teams. I think that goes a long way towards reducing project friction. It does require carrying a broader skillset on a project team and people willing to step outside their normal skill silo. I think that might be one of the largest barriers to agile adoption in big shops. Flame away on point number 4, I probably deserve it.

Our organization has started down a path of using the WiX tool to create installers as part of our NAnt/CruiseControl.Net build. Using this, our testers will now be able to work off the automated installation in testing environments at all times, in effect testing the application and installation at the same time. I know several other organizations in the blogosphere are trying this, too. I'm only peripherally involved, but I'm excited for the potential. More updates when I know more.

I detest Visual SourceSafe

2005-06-22T13:40:00.000-05:00

Ranting ahead, obviously...

All our new projects are in bright, shiny Subversion repositories and humming along just fine. I've got about a month of detour through existing products in Visual SourceSafe and I'm already aggravated on the first day. Don't get me wrong, every source control system has some sort of problem, but VSS is the absolute worst.

The client is a piece of crap; it falls over and corrupts in a swift gale, it's slow, and hangs during CC.NET builds almost as often as not. Branching and merging with VSS is a joke. In a world where Subversion and CVS are free and reliable with multiple clients and integration into IDE's, and SourceGear Vault is inexpensive, why is VSS still so popular in .Net shops?

I've played quite a bit with the vss2svn perl script from Tigris.org. I have no idea if it works or not because VSS keeps crapping out on corrupted history trails. VSS won't even behave long enough to get rid of it.

One thing I'd really like to see from the .Net community as a whole is to stop taking everything from Microsoft without question. There is a whole world of development tools and knowledge out there that doesn't originate in Redmond.

How's this for scary though, in a former job we used VSS under the dark of night on the side because the officially mandated source control system (CC/Harvest ) was even worse.

Thinking about Developer and Tester Interaction

2005-06-21T09:49:00.000-05:00

The following is essentially a lecture and reminder to myself and my team, I apologize in advance for being preachy.

We're wrapping up a release candidate build this morning and quite naturally the subject of testing and bug fixing is on my mind. Because of resource constraints (i.e. not enough testers) we were not able to do much testing (acceptance testing by real testers, not just TDD) within iterations. Doing the old-fashioned (and stupid) big bang, waterfall testing cycle at the end under schedule duress is always tricksome.

It is absolutely imperative that the developer/tester interaction be as smooth as possible. Everybody needs to be focused on a common goal of finding and removing defects from the code, and that means paying attention to what the other half of the equation needs. Verbal abuse or general incivility from either side needs to be smacked down.

If you're going to do any kind of Agile development, one of the first and hardest lessons to learn is that the role boundaries must be blurred, or at least have much more transparency between project roles. You can't bury your head in coding and designing, you have to actively cooperate with testing on test automation and code migration between development and testing environments.

What Developers Owe Testers

Don't waste the tester's time with buggy code. At a minimum, unit testing and either pairing rotation or code review should stop the majority of the problems from reaching the testers. If you can get acceptance tests to the developers before coding is "complete," all the better still.
When you fix a bug, try hard to write an automated test that validates the exact conditions of the defect. At a bare minimum, exercise the code end-to-end to ensure the bug is really fixed. Don't assume the defect is really fixed because you dealt with the root cause (I'm currently wearing some egg on my face from this one).
If anything is blocking the testers, stop what you're doing and get them working again. The application or even a user story cannot be considered complete until it is tested. If the testers are sitting on their hands because a testing environment is invalid or the application is down, your project is down. Again, a reminder to my team and I because we dropped the ball on this one day a week or so back.

What Testers Owe Developers

The singular thing we developers really need from the testers is just communication. When a defect is found, we have to understand a couple of things.

A description of exactly what the error is
Exactly how to reproduce the error
The desired outcome of an operation in clear, unambiguous language
Contextual information if it exists. Stack traces, audit trails, input data
Severity of the defect. Is the defect stopping the tester from testing any farther?

The worst thing a tester can do is just say "It doesn't work" without any context or adequate description of the desired outcome (or my colleague's ultimate pet peeve, "it says NullReferenceException"). I had an incident a couple years ago with an analyst slash tester. She had given us the original requirements for a data exchange between two systems that wasn't much more than a big SQL statement in a stored procedure (mildly modified from the previous system we were replacing). When she was testing the exchange, she kept coming back to us and telling us the results were wrong. Her SQL script she was using to test the exchange was giving different results. I think at some point I said something like "!~@#* it, let's just use your SQL" so we could move on.

My favorite tester used to show me all defects at the tester workstation and walk me through the problems before his team would log a defect. That helped dramatically. It obviously helps if the testing team is colocated with the developers.

Testers are Pigs

Made you look, didn't I? All I mean is that the testers should be an active participant in the standup meetings and the project as a whole. One of the agile practices I like is to have a bug report discussed every single morning in the standup, even if it's just to say there are no bugs. It's a great way to ensure the developers are getting the testers what they need and vice versa. Bug tracking tools are fine and dandy (except ours is terrible), but nothing beats face to face communication.

Here's the original story about the derivation of Pigs and Chickens if you've never heard the phrase.

TDD Design Starter Kit - Static Methods and Singletons May Be Harmful

2005-06-19T21:46:00.000-05:00

Static methods can be a cool solution to many issues and they're definitely convenient. I routinely use static methods in place of specialized constructor functions. Sometimes it is absolutely necessary to maintain state between method calls or across threads and static methods are a logical choice. However, sometimes static methods do not mix well with Test Driven Development. A direct reference to a static method from one class to a second class means that the first class is strongly coupled to everything that the second class uses. Keeping any kind of state in static members can also interfere with your unit tests (more on that here). Always be cognizant of this fact before you create a static method.

Just to get this one out of the way, you cannot mock (or stub) a static method, period. Crying or wailing about it won't help. There is no easy way to substitute in a stubbed implementation for a static method. For that reason alone, I advise developers to never do anything in a static method that calls out of the current AppDomain (HttpContext objects, database calls, MSMQ manipulation, etc.). Even if you need to use a static (or thread local storage or HttpContext storage) field to maintain state, all consumers should access this cached state through some type of abstraction interface for easier testing.

Besides the testability issue, do you really want your application strongly bound to a particular implementation like MSMQ? If you limit the coupling to MSMQ, you can more easily substitute in MQSeries or a call into your company's new utopian Enterprise Service Bus later. My point merely being that designing to maximize testability often happily coincides with just plain good design or architecture.

Singletons

Singletons always, always, always interfere with writing decoupled and isolated unit tests. There is a lot of interest in Domain Driven Design right now and one of the key concepts in DDD is that often the domain objects are retrieved or persisted through a repository that manages caching and persistence for a domain class or small family of classes. The repository is stateful across requests or classes, so it is a natural candidate for a singleton. Don't be seduced by the dark side. If you make the repository a singleton, all of the consumers of the repository (and there will be a bunch) cannot be tested in isolation. You can do something to allow you to mock or stub out the underlying data source classes, but take it from me, this sucks (hard). Your unit tests are dependent upon the implementation of the repository class. This makes your unit tests harder to write because there is more code, and harder to understand because you are investing more of the unit test in setting up the internal state of the repository than exercising the consuming class. For a multitude of reasons, it is always important to make your unit tests intention revealing. Mocking the guts of the repository may end up obfuscating the real intention of a unit test.

Here's an article I wrote on a TDD-friendly alternative on the StructureMap site for exactly this situation with more discussion on the issue: http://structuremap.sourceforge.net/SingletonInjection.htm.

You should also check out Robert Martin's article Singleton vs. Just Create One. While you're on the ObjectMentor site, read everything those guys write, it's all good stuff.

As an aside, one of my friends does a lot of technical interviews for .Net developers for a very large employer in Austin. His weedout question is "what is the difference between a static and an instance method?" He is continually scratching his head on the percentage of people who wash out on this question. I have no idea what to say about that.

A simple example of the "Humble Dialog Box"

2005-06-16T21:26:00.000-05:00

At the Agile Austin lunch today, we talked a bit about different ways to apply the Model View Presenter pattern with WinForms clients. I promised to put up an example of using Michael Feather's "Humble Dialog Box" to create more testable user interface code.

The current thinking for writing automated unit tests against rich clients is a modification or variation of the classic Model View Controller (MVC) architecture. Specifically, take the view part of MVC and slice it as thin as possible so that it is only a skin around the actual UI components and make it completely passive. The controller, now called the "presenter," is responsible for all interaction with the rest of the system. There is a pattern of symbiosis between the view and the presenter. The presenter directs the view what and when to display and the view captures and relays user events to the presenter. Check the links above for a more comprehensive explanation from the professionals.

Here's a common scenario. You have some kind of form in your application for editing a piece of data. If the user trys to close the form and there are pending changes, put up a dialog box giving the user a chance to cancel the close operation. In this case the dialog box is the major impediment to automated testing, so we're going to hide the message box creation behind an interface that can be mocked (or stubbed if that's your predilection). Do the same thing for the view/presenter separation. Use the Dependency Inversion Principle to abstract the view away from the presenter and mock the view in the unit tests.


using System;
using System.Windows.Forms;
using NMock;
using NUnit.Framework;

namespace SampleCode.HumbleDialogBox
{
 public interface IMessageBoxCreator
 {
  bool AskYesNoQuestion(string title, string message);
 }

 public class MessageBoxCreator : IMessageBoxCreator
 {
  public bool AskYesNoQuestion(string title, string message)
  {
   DialogResult result = MessageBox.Show(message, title, MessageBoxButtons.OKCancel);
   return result == DialogResult.OK;
  }
 }

 public interface IView
 {
  void Close();
  bool IsDirty();
 }

 public class Presenter
 {
  public const string DIRTY_CLOSE_WARNING = "Changes are pending.  "
   + "Ok to continue, cancel to return to the edit screen.";
  public const string CLOSE_WARNING_TITLE = "Changes Pending";

  private readonly IView _view;
  private readonly IMessageBoxCreator _msgBox;

  // 
  public Presenter(IView view, IMessageBoxCreator msgBox)
  {
   _view = view;
   _msgBox = msgBox;
  }

  public void Close()
  {
   bool canClose = true;
   
   if (_view.IsDirty())
   {
    canClose = _msgBox.AskYesNoQuestion(CLOSE_WARNING_TITLE, DIRTY_CLOSE_WARNING); 
   }

   if (canClose)
   {
    _view.Close();
   }
  }
 }

 [TestFixture]
 public class PresenterTester
 {
  private DynamicMock _viewMock;
  private DynamicMock _msgBoxMock;
  private Presenter _presenter;

  [SetUp]
  public void SetUp()
  {
   _msgBoxMock = new DynamicMock(typeof(IMessageBoxCreator));
   _viewMock = new DynamicMock(typeof(IView));
   _presenter = new Presenter((IView) _viewMock.MockInstance, (IMessageBoxCreator) _msgBoxMock.MockInstance);
  }


  [Test]
  public void CloseViewWhenViewIsNotDirty()
  {
   // Define the expected interaction
   _msgBoxMock.ExpectNoCall("AskYesNoQuestion", typeof(string), typeof(string));
   
   _viewMock.ExpectAndReturn("IsDirty", false);
   _viewMock.Expect("Close");

   // Perform the unit of work
   _presenter.Close();

   // Verify the interaction
   _msgBoxMock.Verify();
   _viewMock.Verify();
  }


  [Test]
  public void CloseViewWhenViewIsDirtyAndUserRespondsOk()
  {
   // Define the expected interaction
   _msgBoxMock.ExpectAndReturn(
    "AskYesNoQuestion", 
    true, 
    Presenter.CLOSE_WARNING_TITLE, 
    Presenter.DIRTY_CLOSE_WARNING);
   
   _viewMock.ExpectAndReturn("IsDirty", true);
   _viewMock.Expect("Close");

   // Perform the unit of work
   _presenter.Close();

   // Verify the interaction
   _msgBoxMock.Verify();
   _viewMock.Verify();
  }


  [Test]
  public void DoNotCloseViewWhenViewIsDirtyAndUserRespondsCancel()
  {
   // Define the expected interaction
   _msgBoxMock.ExpectAndReturn(
    "AskYesNoQuestion", 
    false, 
    Presenter.CLOSE_WARNING_TITLE, 
    Presenter.DIRTY_CLOSE_WARNING);
   
   _viewMock.ExpectAndReturn("IsDirty", true);
   _viewMock.ExpectNoCall("Close");

   // Perform the unit of work
   _presenter.Close();

   // Verify the interaction
   _msgBoxMock.Verify();
   _viewMock.Verify();
  } 
 }
}

Here's a rundown of the pieces from the example code.

IMessageBoxCreator/MessageBoxCreator - An interface and wrapper class around the WinForms MessageBox class. The methods in the .NET framework for dialogs are all static, and static methods cannot be mocked.
IView interface - An interface that establishes the public contract between the actual form and the presenter. I didn't show it, but assume the actual View has a reference to the Presenter.
Presenter - the Presenter class drives the IView and IMessageBoxCreator interfaces. The Presenter class is completely unaware of any of the actual user interface plumbing, i.e. not one single reference to the System.Windows.Forms namespace.

In this example I used constructor injection to attach the IMessageBoxCreator. The next example is mostly the same, but I use StructureMap instead to locate the IMessageBoxCreator and take advantage of StructureMap's built in support for NMock.


using System;
using System.Windows.Forms;
using NMock;
using NUnit.Framework;
using StructureMap;

namespace SampleCode.HumbleDialogBox2
{
 [PluginFamily("Default")]
 public interface IMessageBoxCreator
 {
  bool AskYesNoQuestion(string title, string message);
 }

 [Pluggable("Default")]
 public class MessageBoxCreator : IMessageBoxCreator
 {
  public bool AskYesNoQuestion(string title, string message)
  {
   DialogResult result = MessageBox.Show(message, title, MessageBoxButtons.OKCancel);
   return result == DialogResult.OK;
  }
 }

 public interface IView
 {
  void Close();
  bool IsDirty();
 }

 public class Presenter
 {
  public const string DIRTY_CLOSE_WARNING = "Changes are pending.  "
   + "Ok to continue, cancel to return to the edit screen.";
  public const string CLOSE_WARNING_TITLE = "Changes Pending";


  private readonly IView _view;

  public Presenter(IView view)
  {
   _view = view;
  }

  public void Close()
  {
   bool canClose = true;
   
   if (_view.IsDirty())
   {
    // Get the IMessageBoxCreator out of StructureMap
    IMessageBoxCreator msgBox = 
        (IMessageBoxCreator)
                                    ObjectFactory.GetInstance(typeof(IMessageBoxCreator));
    canClose = msgBox.AskYesNoQuestion
                                   (CLOSE_WARNING_TITLE, DIRTY_CLOSE_WARNING); 
   }

   if (canClose)
   {
    _view.Close();
   }
  }
 }

 [TestFixture]
 public class PresenterTester
 {
  private DynamicMock _viewMock;
  private IMock _msgBoxMock;
  private Presenter _presenter;

  [SetUp]
  public void SetUp()
  {
   _msgBoxMock = ObjectFactory.Mock(typeof(IMessageBoxCreator));
   _viewMock = new DynamicMock(typeof(IView));
   _presenter = new Presenter((IView) _viewMock.MockInstance);
  }

  [TearDown]
  public void TearDown()
  {
   ObjectFactory.ResetDefaults();
  }

  [Test]
  public void CloseViewWhenViewIsNotDirty()
  {
   // Define the expected interaction
   _msgBoxMock.ExpectNoCall("AskYesNoQuestion", typeof(string), typeof(string));
   
   _viewMock.ExpectAndReturn("IsDirty", false);
   _viewMock.Expect("Close");

   // Perform the unit of work
   _presenter.Close();

   // Verify the interaction
   _msgBoxMock.Verify();
   _viewMock.Verify();
  }


  [Test]
  public void CloseViewWhenViewIsDirtyAndUserRespondsOk()
  {
   // Define the expected interaction
   _msgBoxMock.ExpectAndReturn(
    "AskYesNoQuestion", 
    true, 
    Presenter.CLOSE_WARNING_TITLE, 
    Presenter.DIRTY_CLOSE_WARNING);
   
   _viewMock.ExpectAndReturn("IsDirty", true);
   _viewMock.Expect("Close");

   // Perform the unit of work
   _presenter.Close();

   // Verify the interaction
   _msgBoxMock.Verify();
   _viewMock.Verify();
  }


  [Test]
  public void DoNotCloseViewWhenViewIsDirtyAndUserRespondsCancel()
  {
   // Define the expected interaction
   _msgBoxMock.ExpectAndReturn(
    "AskYesNoQuestion", 
    false, 
    Presenter.CLOSE_WARNING_TITLE, 
    Presenter.DIRTY_CLOSE_WARNING);
   
   _viewMock.ExpectAndReturn("IsDirty", true);
   _viewMock.ExpectNoCall("Close");

   // Perform the unit of work
   _presenter.Close();

   // Verify the interaction
   _msgBoxMock.Verify();
   _viewMock.Verify();
  } 
 }
}

Final Thoughts

It is not impossible to write automated unit tests for rich clients, but it's definitely difficult and time consuming. So what can you do? You can take a calculated risk and forgo writing the automated tests for the user interface. The biggest problem with that approach is that a complicated rich user interface can generate a large number of bugs and requires a lot of energy towards manual regression testing (duh). You can test a WinForms application with Luke Maxon's most excellent NUnitForms toolkit, but user interface tests are still more work to setup and execute. A better approach is to simply make as much code as possible independent of the WinForms (or Swing, etc.) engine. I say you still have to test the actual UI forms and controls. However, if they are passive and loosely coupled from the rest of the application your NUnitForms tests can be much simpler.

I left some implementation details out of the example. I've used the MVP pattern pretty extensively on a couple of projects now with mostly good results. Since it's such a hot topic and the book on best practices is literally being written as I type this, I'll try to blog soon on some MVP suggestions and pitfalls.

Steve Hebert article on Code-Talking

2005-06-15T12:17:00.000-05:00

Code Talking: The key to great software?

A great article. I admit that I don't particularly enjoy pair programming, but Steve Hebert describes one of the best benefits of pairing. Collective code ownership is an awfully good thing in many ways. I've certainly observed that my stress level is generally less on projects that have some degree of collective ownership.

A classic article from Alistair Cockburn and horror stories just for fun

2005-06-14T11:28:00.000-05:00

http://alistair.cockburn.us/crystal/articles/cpanfocisd/characterizingpeopleasnonlinear.html

This one has been around awhile, but it's still a worthwhile (re-)read. It's full of radical ideas like:

Individuals are important
People aren't "Plug'n Play" components
Productivity can be maximized by making it easy for the project team to communicate
Pay attention to the team's environment

If you're in anything like the environment I describe below, you can try using Cockburn's writing in an effort to reform your workplace. If you have any influence over your workplace, heed Mr. Cockburn's writing now rather than learning the hard way.

This Stuff Matters

A couple years ago I got to jump straight from pure insanity to a high caliber agile shop. An oranges to oranges comparison of effectiveness between the Microsoft Solutions Framework process at the first shop and Extreme Programming in the second is useless because too many other factors were different (stronger teams, more supportive organizations, less customer involvement), but the simple ideas of:

Sitting near all of my teamates in an open room
Everybody focused on only the one project
Developers, testers, and analysts on the project at the same time!

were an enormous improvement in my mind.

Project Team Insanity

I used Cockburn's chart on the effectiveness of various communication channels in a presentation several years ago trying to convince my then management to move towards iterative and more collaborative processes. At the time the entire organization was in a "matrix" organization structured by functional group first, and project teams second. Cubicle assignment was strictly by discipline (development, architects, analysts, testers, etc.) because management was tired of reorganizing teams and paying moving costs to colocate teams. Everybody was working on multiple projects at a time so that all projects had equal access to specialized knowledge and skillsets. I had the distinctly unpleasant experience of simultaneously working on the rump end of a death march project as a developer (later cancelled), and trying to lead the development of a separate project.

The first project was an early Web Services project using a custom in-house SOAP-like framework developed by the internal integration architecture team (if you ever hear a sentence like that, run away!). The lead developer/project manager was in his first lead role and had no prior lead or architecture experience. There were five distinct methods for the service, and five developers, so naturally each developer was assigned a method and told to go off and code it. I failed in my attempt to escape the assignment and began to code my assignment (throwing out the unworkable design that was handed to me. Win32 API and COM calls from within XSLT transforms? No thank you).

Early on, I asked the lead where the development database was located and was told each developer was just using their own box. Come to find out, every single developer had created or reused a different database schema structure to hold the same logical data. Two developers were using an existing Oracle schema structure (the database was overworked and we had an opportunity to move some load off), and the other three had made their own Sql Server databases. Two of us had created fully functional, specialized rules engines that did roughly the same thing, but radically different in implementation. My first action as a newly minted Systems Architect (make finger quotes for the full effect) was to blow the whistle on the technical weaknesses to my new boss. The project was fortunately abandoned and replaced by a different effort (2+ years in MSF Planning!). Integration testing at the very end was apparently pretty interesting;) I sincerely think the project could have succeeded if the developers had been working in a common location instead of in a complete vacuum. Just accidentally overhearing conversations would have tipped us off that our code could be shared between service methods and certainly would have got us on a common database early. Of course, the fact that every developer was actively plotting to get off the project and the organization went through a massive reorganization at the same time didn't help either.

The second project was an extremely complex system with three other developers, all of whom sat in different wings of the floor, and were working on other projects at the same time. At no time during the course of the project was the requirements analyst present on the team at the same time I was. All communication was through a requirements document that was prepared when I was on the other project (that turned out to be incomplete and/or flatout wrong). Come to find out late in the game, there was a second requirements specification no one in development knew about. We pulled it off by working way too many hours, but I'll never do a project like that again. The saving grace was very good involvement from the business partners, otherwise it would have bombed. The system is undeniably successful in production (one of my friends won an award from the business for it), but it was a long time before I enjoyed software development again. I'm pretty sure the CMM crowd would call this an unrepeatable process.

Of course it didn't help that I over-engineered the application a bit, but I'm making fun of other people here, dammit! I read somewhere that the worst designer is a fellow on his second project trying to use every idea he had from the first system. Guilty as charged. I'd also just read the GoF book for the first time. They should put a warning label on that thing.

My former employer has since quietly removed the worst of the organizational madness and is simultaneously working toward climbing the CMM ladder and tentatively introducing some basic best practices like automated builds and better unit testing. All I can say is good luck to you guys, and it'll be worth the effort (maybe not the CMM part, though).

It's fun to laugh at the absurdity of past projects, as long as you don't repeat them. Bad examples can be more illustrative than good examples sometimes.

The Worst Possible Way to use a Stored Procedure

2005-06-13T13:08:00.000-05:00

Rant time...

Repeat after me please,

I will not put any business logic into a stored procedure.
I will not put any business logic into a stored procedure.
I will not put any business logic into a stored procedure.
I will not put any business logic into a stored procedure.
I will not put any business logic into a stored procedure.

Okay, if your business logic can be expressed easiest through a declarative SQL WHERE clause, a SPROC can be a cool way to go. Set-based logic is almost always simpler to do with SQL than mucking through with procedural C#, but that's a different rant for another day, and I'm definitely talking about procedural code within sproc's here.

Question: How do I know if I might be doing something in a sproc that I shouldn't be doing? Answer: If there is *anything* other than CRUD in your sproc (IF/ELSE/ENDIF, LOOP/END LOOP, etc.).

The all time dumbest thing to do is to split your business logic between a SPROC and the consuming middle tier code. Twice now in the past 12 months I've bumped into cases where business logic is performed inside a sproc, then interpreted by middle tier C# or VB6 code. Splitting the logic into multiple spots like this makes a system harder to understand. It also makes a system brittle to change because some logic is duplicated in both places. A change in the sproc or C# can, and usually does, break the other. This is a case of the Don't Repeat Yourself principle the Pragmatic Programmer guys talk about.

I had an interesting experience as a consultant one time. I was fresh into the project working with some legacy C# code (yes, there is already legacy .NET code). I had just stumbled on the fact that most of the business logic was really in T-SQL procedures. In many cases, the C# code undid some of the T-SQL transformations to filter the results further. The client architect was giving us a bit of a tongue-lashing ("you consultants better not write crappy code, and there better be tests for everything you write") while I was looking at a 5,000 line stored procedure, with his name all over the comments, trying to decide if some erroneous data was coming out of the database or being transformed in the C# code. This particular system is the subject of a case study/testimonial on MSDN as an example of all the wonderful things about .NET;)

"Thank you for listening, I feel better now." -- Roy Moore

Getting Out-Geeked By Your Product Manager (Off Topic)

2005-06-10T09:45:00.000-05:00

I work in a pretty geeky shop, and I certainly mean that in a good way. From experience, I really don't trust people on software projects that aren't interested in technology and all the ancillary geeky subjects that seem to go with that. It's very difficult to be good at anything you don't enjoy doing. A passion for software development is a huge plus in my book.

John, my product manager, gets the Geek of the Day award for out-geeking all of us developers. John's qualifications:

Competing in the Ruby on Rails coding challenge
Sending me an email at 6:30 this morning telling me to check out Ajax.Net
Having read Dune out loud to his wife on his honeymoon (lifetime achievement award)
Being a Cold Fusion junky

All this from an ex-lawyer.

TDD Design Starter Kit - State vs. Interaction Testing

2005-06-09T20:17:00.000-05:00

When you write a unit test, what is it that your asserting? Most of the time you're doing state-based testing, i.e. your unit tests either validates a return value from a method call or a change in a property of the class being tested. Here is a typical example of state-based testing like you'll find in almost every introductory TDD article ever published.


 [TestFixture]
 public class StateTester
 {
  [Test]
  public void TwoPlusThreeIsFive()
  {
   RunningSum sum = new RunningSum();
   int actual = sum.AddIntegers(2, 3);

   Assert.AreEqual(5, actual);
  }

  [Test]
  public void AddSeveralNumbersAndGetTheRunningSum()
  {
   RunningSum sum = new RunningSum();
   sum.AddInteger(2);
   sum.AddInteger(3);
   sum.AddInteger(5);

   Assert.AreEqual(10, sum.Total);
  }
 }

Easy money, all you have to do is check an expected outcome for a known set of inputs. Unfortunately, the real world of enterprise software development quickly intrudes into your newfound TDD bliss. Toss in a bunch of dependencies external to your application and you quickly find that:

The expected outcome is difficult or time-consuming to verify in an automated test
The known inputs are difficult or time-consuming to setup in an automated test
Tests that depend on external services can be brittle or just plain slow
Measuring the outcome requires checking the state of all a class's dependencies, often making the unit tests too coarse and dependent upon the actual implementation details
Too many classes have to be involved in the unit test

Teams abandon TDD as a failure when the tests are too difficult to write or just plain laborious. Nobody is going to willingly use a technique that is more bang than buck.

One way to alleviate the problem is to use interaction-based testing. Just like it sounds, interaction-based testing is verifying the expected interaction between a class and its dependencies. Typically you would use either a mock or a stub object as a stand-in for the real dependency. The unit test will verify that the expected interaction via method calls took place.

To illustrate the difference between testing state or interaction, here's a somewhat contrived user story (for best results, user stories should really be much finer grained than this):

Email Alert for Invoice Validation Failures User Story

When an invoice is being processed, if certain nebulous validation failure conditions are met, send an email to the accounts payable individual assigned to the invoice. The email should list all of the invoice errors in a certain html format.
If the invoice is really, really bad, send a summary email to the proper expediter. The email message will be html formatted in a particular manner.
If the invoice is successfully validated, publish the invoice (via MSMQ)

Here's a possible C# class for this story designed with absolutely no regard for testability.


 public class InvoiceProcessor
 {
  public void ProcessInvoices(Invoice[] invoices)
  {
   InvoiceErrorReport report = new InvoiceErrorReport();

   foreach (Invoice invoice in invoices)
   {
    // Execute the validation business logic
    InvoiceError[] invoiceErrors = this.validateQuantities(invoice);
    report.StoreErrors(invoice, invoiceErrors);
   }

   // determine if the critical error threshold has been exceeded
   if (this.hasTooManyCriticalErrors())
   {
    MailMessage message = this.createErrorSummaryMessage(report);

    // Call to the local (or default) SMTP service to send the email
    SmtpMail.Send(message);
   }
   // determine if the warning threshold for the error count has been reached
   else if (this.hasTooManyErrors())
   {
    MailMessage message = this.createDetailsMessage(report);
    SmtpMail.Send(message);
   }
   else
   {
    // If the invoices are okay, send the invoices on to the next process via MSMQ
    MessageQueue queue = new MessageQueue(this.getMessageQueuePath());
    queue.Send(invoices);
   }
  }


  public MailMessage createErrorSummaryMessage(InvoiceErrorReport report)
  {
   MailMessage email = new MailMessage();

   // Go get the user email addresses from the user information storage 
   // (database or Active Directory)
   UserInformationStore userStore = new UserInformationStore();
   email.To = userStore.GetEmailAddressesForInvoices(report);

   email.Body = this.createErrorSummaryMessageBody(report);

   return email;
  }

  // #region Other Methods
 }

We could use a state-based strategy to test the email. We could run the test, then run around and ask the expected recipients to check their inbox (don't laugh too hard, I've been a party to this). Since we do want an automated test, we could query the Exchange server to see if the email was sent out correctly, but I'd rather gouge out my eyeballs with a dull spoon or code in Fortran 77 than attempt this (not to mention the look of apoplexy on the sysadmin's face when I ask for administrative rights to the Exchange server). You could also check an audit trail, but that's not the real functionality being tested.

The easiest approach in the case of the email tests is to use interaction testing to verify that the email was sent to the SMTP service. Now look at the same class, but somewhat more testable because it is designed for interaction testing between the class and the SMTP service.


 public class InvoiceProcessor
 {
  // "Dependency Inversion Principle" -- All dependencies are now to an abstracted interface
  private readonly IEmailGateway _emailGateway;
  private readonly IMessagingGateway _messagingGateway;
  private readonly IUserInformationStore _userStore;

  // Use "Constructor Injection" to push dependencies into InvoiceProcessor
  public InvoiceProcessor(
      IEmailGateway emailGateway, 
      IMessagingGateway messagingGateway, 
      IUserInformationStore userStore)
  {
   _emailGateway = emailGateway;
   _messagingGateway = messagingGateway;
   _userStore = userStore;
  }



  public void ProcessInvoices(Invoice[] invoices)
  {
   InvoiceErrorReport report = new InvoiceErrorReport();

   foreach (Invoice invoice in invoices)
   {
    // Execute the validation business logic
    InvoiceError[] invoiceErrors = this.validateQuantities(invoice);
    report.StoreErrors(invoice, invoiceErrors);
   }

   // determine if the critical error threshold has been exceeded
   if (this.hasTooManyCriticalErrors())
   {
    MailMessage message = this.createErrorSummaryMessage(report);
    _emailGateway.SendMail(message);
   }
   // determine if the warning threshold for the error count has been reached
   else if (this.hasTooManyErrors())
   {
    MailMessage message = this.createDetailsMessage(report);
    _emailGateway.SendMail(message);
   }
   else
   {
    // no longer responsible for knowing where the MSMQ path is.  Or even if this
    // is an MSMQ
    _messagingGateway.SendInvoices(invoices);
   }
  }


  public MailMessage createErrorSummaryMessage(InvoiceErrorReport report)
  {
   MailMessage email = new MailMessage();

   // Go get the user email addresses from the user information storage 
   //(database or Active Directory)
   email.To = _userStore.GetEmailAddressesForInvoices(report);

   email.Body = this.createErrorSummaryMessageBody(report);

   return email;
  }

                // Other methods ...
 }



 public interface IMessagingGateway
 {
  public void SendInvoices(Invoice[] invoices);
 }

 public interface IEmailGateway
 {
  public void SendMail(MailMessage message);
 }

What's different here? First, I used the Dependency Inversion Principle to encapsulate the interaction with the SMTP email service and the MSMQ queue behind .Net interfaces (IEmailGateway and IMessagingGateway). The second change was to use Dependency Injection to push in the IEmailGateway and IMessagingGateway dependencies into the constructor function. Here's what a unit test might look like to test the email and MSMQ interaction using NMock to create mock objects as placeholders for the real dependencies.


  [Test]
  public void SendTheSummaryEmailIfTheCriticalErrorThresholdIsMet()
  {
   // 1.) Create the NMock objects
   DynamicMock messagingMock = new DynamicMock(typeof(IMessagingGateway));
   DynamicMock emailMock = new DynamicMock(typeof(IEmailGateway));
   DynamicMock userStoreMock = new DynamicMock(typeof(IUserInformationStore));

   // Setup the test data
   Invoice[] invoices = 
       this.setUpArrayOfInvoicesThatWillSpawnACriticalNumberOfInvoiceErrors();

   // 2.) Setup the expectations.  Gets the email 
   // addresses, sends an email address, but does not put the invoice array onto
   // the outgoing queue
   emailMock.ExpectAndReturn(
       "GetEmailAddressesForInvoices", 
       "somebody@somewhere.com", 
       new IsTypeOf(typeof(InvoiceErrorReport)));

   messagingMock.ExpectNoCall("SendInvoices", typeof(Invoice[]));
   emailMock.Expect("SendMail", new IsTypeOf(typeof(MailMessage)));

   // 3.) Create the InvoiceProcessor object with the Mock instances
   InvoiceProcessor processor = new InvoiceProcessor(
    (IEmailGateway) emailMock.MockInstance,
    (IMessagingGateway) messagingMock.MockInstance,
    (IUserInformationStore) userStoreMock.MockInstance);

   // 4.) Execute
   processor.ProcessInvoices(invoices);

   // 5.) Verify the correct calls were made
   messagingMock.Verify();
   emailMock.Verify();
   userStoreMock.Verify();
  }

All I've really tested in this unit test is that the InvoiceProcessor sent the proper signals to the IEmailGateway and IMessagingGateway interfaces. The test is way too coarse, but I had to leave something for later posts.

State or Interaction Testing, Which Do I Want to Use?

Not to be flippant, but use whichever mode of testing is easiest to apply on a unit test by unit test basis. Both styles of testing need to be in your intellectual toolbox. State-based testing is generally easier because of the overhead associated with setting up the mock objects, but interaction-based testing is essential for dealing with external dependencies like databases or middleware and user interface testing. I'll blog much more in the near future about how to maximize testability by design.

What Next?

The InvoiceProcessor structure above is better, but it can still be improved. The next step is to carefully consider how responsibilities can be assigned to classes to make testing easier. Design concepts like Inversion of Control and Dependency Injection (DI is a subset of IoC) can help tremendously to improve testability by reducing coupling. One of the best design philosophies is to quarantine external dependencies into narrow Gateway classes to minimize direct coupling to external code. It's also helpful to understand the best practices for using mocks or stubs. In my next set of blog posts, I'm going to further refine the invoice processing user story.

TDD Design Starter Kit Roadmap

Assigning Responsibilities
Inversion of Control
Dependency Injection (with and without a Container)
Gateway Pattern
Mocks and Stubs - Best Practices

More Information

Read more about this from Nat Pryce and Martin Fowler. I actually disagree a bit with Fowler's discussion of interaction testing, so no more calling me a "Fowlbot" for a little while.

UPDATE 6/11/2005: Martin Fowler dinged me a little bit about this one. His main point about the difference in testing styles is that state-based testers use mocks or stubs for exceptional cases and interaction-based testers will try to use mocks in almost every case, including domain model classes. Once you put it that way, I'd lean more towards the state-based testing approach. Mocks get you out of tight spots, but overuse is a bad, bad thing.

Learning Lessons -- Can You Make Mistakes at Work?

2005-06-03T11:39:00.000-05:00

Of course you make mistakes, that's not really what I meant. Can you comfortably admit any type of fault in your workplace without undue fear of reprisal? Do you trust your teammates and management enough to discuss problems openly or do you sweep problems under the rug? After all, we know that admitting a problem is the first step to recovery.

A Healthy Environment

Last week my team did a project retrospective on our last four months effort. Half of our team is new to the company and our team is distributed between Austin and Houston, so there were definitely some rough spots to sand down. We came up with a list of problem areas with our execution and some lost opportunities. It was actually a very positive experience, because everyone involved was focused on improving our execution instead of pointing fingers or deflecting blame. Never once did I feel defensive or sense any kind of defensiveness in anyone else in the room.

We've already started to implement some improvements in our iteration and project management. Of course we still have to follow through and correct the problems, but at least the team got the issues out into the open and everybody is aware. I've heard the phrase "Lessons Observed" in the past to describe teams that don't act upon their lessons.

The Blame Culture

Much of my IT career has been in a shop where too much energy was wasted on blame avoidance and "CYA". We didn't learn any lessons or ever make any corrections because no one ever wanted to admit any kind of failing. It was more or less a waterfall shop (and still is) and management stressed a predictable project date and resource count over everything else. The worst sin was missing a deadline. The result? Sacrifice quality or outrageously pad estimates. I routinely saw projects that were coding right up to the production install because everybody was afraid to miss the arbitrary date. Very scary.

External compliance teams are the worst. They aren't involved in your project at all, but they get the last say on the success of the project by putting out a metrics laden scorecard. One of the primary measurements is a bug count read out loud in a nasty meeting at the end of the project. The typical response was to hide bugs. Developers would quietly negotiate with testers to fix bugs without reporting them. That's an awful practice, but it seemed more like survival at the time.

Now they're climbing up the CMM ladder (I think they're about to be level 3). It turns out CMM is a great work avoidance tool. You need what? Did you fill out the correct paperwork? Sorry, I can't do that until I have the TPS form signed and approved.

Avoid the Embarrassment Meeting

Be careful how you use metrics. Metrics reveal information and trends about your effort, but they're never the entire story. Avoid the "embarrassment Meeting" described by Steve Maguire. Misuse of metrics to punish people leads to stupid behavior that doesn't contribute to the project.

An agile process alone isn't enough. I worked on an XP-ish project with an absolutely miserable jerk of a project manager that routinely abused the development team with metrics. The developers varied wildly in skillset, and we were struggling. Every iteration close meeting featured a full review of exactly how the development team missed their estimates on a story by story basis. One time he went away to a conference for several days. Coincidentally the developers had what we all considered to be our best iteration. We completed stories on time, we got a lot of refactoring done that had been put off, and we even managed to make some overdue improvements to the NAnt build. The project manager came back from his conference, looked at *his* metrics, and loudly declared that we still weren't doing any better. I wanted to punch him.

An iteration later, I excitedly showed him a story that was finished ahead of the estimate. His response? We had sandbagged the estimate in the first place.

After some external intervention from higher management, we made some adjustments to our estimating and planning process that lowered the misery on the project considerably. It wasn't pleasant, but the mistakes were recognized and corrected. A healthier team would have made the correction faster though. I sincerely wish I had done something to alleviate the problem instead of fuming, but I didn't feel comfortable in the environment and didn't feel like I had a voice in the project.

To state the obvious, that was a dysfunctional environment. Martin Fowler has some pointed comments on the usage of metrics here.

Does Agile Development Help?

I want to say yes, but the real answer is "it depends." I think this is really more of a people issue than a process or organization problem. An iterative process is definitely advantageous because it gives you so many more chances to reflect on your project and make adjustments mid-stream. However, if you don't trust your team or feel comfortable making yourself heard this opportunity will be wasted. A team with poor chemistry is probably going to fail no matter what else is in place.

One of the tenets of agile development is the self-organizing team. It is everybody's responsibility to continuously improve the team's work, not just the project manager. Being empowered to make changes in your process means you're responsible for the process. Don't ever assume anyone else knows something you don't. If you think your team is doing something wrong, say so, no matter what position you have on the team.

Collective code ownership does help in this regard by removing some psychological barriers to code inspection. Classic code reviews can be nasty affairs. A row of accusers sitting across the table from a solo defendant is the image that comes to mind. With true collective ownership, the team can calmly talk knowledgeably about improving "our" code instead of pointing fingers to "his" code.

Things to Learn

2005-06-03T09:42:00.000-05:00

"Life moves pretty fast. If you don't stop and look around once in a while, you could miss it." -- Ferris Bueller

The art of software development moves fast. We're still a young discipline compared to engineering and we're still learning new and better ways to create software. Last year I sharpened my saw by adding mock objects, dependency injection/inversion of control, WinForms, and "BuildMaster" skills to my toolset. I spent a ludicrous amount of time on StructureMap though and let other new skill development and learning slide.

Here's the things I want to learn more about this year:

Ruby and Ruby on Rails. I'm still a little dubious about the dynamic typing, but too many people are effusive about Ruby's potential to ignore this very much longer. If nothing else, Ruby should grow into the language of choice for scripts and what I call "NAnt-ey" tasks.
Ajax. I did a lot of IE specific Ajax development several years ago (before Google named the concept) and had quite a bit of success with it (a mission critical supply chain application and a dispute resolution workflow system, both still running). At the time it wasn't really a good platform choice because of the lack of developer familiarity with the techniques (plus IE5 had *huge* memory leaks with heavy DHTML manipulation). Because of the new hype from the Google apps and cross browser testing tools for DHTML like JSUnit and Selenium, Ajax development is looking like a viable choice now. I'll blog more on my experiences with this now that it is actually relevant.
Acceptance testing tools like NFit and Selenium. I love the concepts, but I think the NFit implementation could be better.
Finally finish reading Domain Driven Development
NHibernate. When .NET first came out in beta in 2002, I had dreams of creating the ultimate O/R mapper for the platform (use emitting for the mapping and use parameterized sql for better performance than NHibernate's purely reflective approach). I even wrote an article for C# Today on using reflection for O/R mapping. Alas, I didn't have the skillset then and I don't have the ambition today. Especially when NHibernate is there with an active development community and poised to become the de facto standard for O/R mapping in .NET. I know they have their proponents, but I don't count persistence generation tools like Neo, LLBGen, or Codus as O/R mapping.
Aspect Oriented Programming. I'd love to add some runtime AOP support to StructureMap.

My StructureMap tool actually started its life as an O/R and data access layer tool. What's actually there on SourceForge is just a refinement of what was supposed to be the configuration framework for the rest.

Taxonomy of Tests: Everything Else

2005-05-31T11:37:00.000-05:00

While the majority of your focus on an agile project is on either unit tests or acceptance tests, there are quite a few other types of useful tests that fall through the cracks. All of these types of tests can and should be part of your continuous integration strategy.

Developer Written Partial Integration Tests

Bigger than a unit test, but smaller in scope than an acceptance test. That's a big mouthful for a category, but I've never seen any two development teams that use the same name for this broad category of tests. For multiple reasons, most of my team's unit tests are disconnected -- i.e. databases, web services, and the like are mocked or stubbed. I want to test each class in isolation, but they have to interact with the real services too. A secondary battery of developer tests that test partial code stacks are very advantageous in uncovering problems in the actual interaction between classes or subsystems. I've known some XP developers who actually emphasize these semi-integrated tests more than the disconnected unit tests. I like to keep the types of tests in separate test assemblies, but that is just preference. These are really hard to describe, so here's a couple of ideas I've used or heard of --

Testing a view and controller or presenter together. I would still try to mock out the backend business services from the controller in order to focus on testing the integration of view and controller. This is especially important in the new Model-View-Presenter mode of UI development.
Testing from a UI controller down to the persistence layer.
Test from the service layer down. SOA is surely overhyped and overused, but it presents some great opportunities for automated testing. Because the .Net SOAP serialization is a bit brittle I always put automated tests in for the web service proxy class communicating with a local web service. It's also good to test the service code without IIS being involved.

Environment Tests

Environment tests verify that a build or test environment is fully connected and correctly configured. I helped do a presentation on continuous integration to the Austin Agile group last week and we spent quite a bit of time talking about this very topic. The gist of the slide was preventing lost time by testing teams by uncovering disconnects or failures in the testing environment at code migration time. Nothing is more aggravating than reported defects that end up being merely a mistake in code migration. I spent a week one time getting grilled because a data interface wasn't working correctly. It turned out that the middleware team had shut down the webMethods server in the test environment and not told the development team. Putting aside the total incompetence of everybody involved (including me), had an automated environment test been in place to verify the connectivity to webMethods, a full week of testing would not have been wasted.

I had some periodic trouble on a project last year with maintaining various copies of the system configuration (database connections, file paths, web service locations, etc.) for development, build and test environments. To combat this problem in the future I've actually added some support into my StructureMap tool for doing environment checks and configuration validation inside of an automated deployment. We sometimes move code a lot faster in agile development and it provides a lot of opportunity to screw things up. These little environment tests go a long way towards keeping project friction down.

I got the name for this off Nat Pryce's post here.

Smoke Tests

Smoke tests are simple crude tests to exercise the system just to see if the code blows up. They don't provide that much benefit, but they're generally low effort as well. On my current project we're replacing a large VB6 component with a new C# equivalent. One of the things we did to create a safety "tripwire" effect was to run a huge amount of historical input XML messages and configurations through the new C# engine just to find exceptions. We found and fixed several problems from edge cases and some flatout bugs. It was definitely worth the effort.

Regression Tests

My last two projects have been more or less rewrites of existing legacy code. Oftentimes hairy legacy code isn't understood very well by the organization (hence the rewrite). Twice now I've had to jury-rig the legacy code to create repeatable automated tests that simply prove the new code creates the same output for known input. It's not entirely a good test because it has no connection to the intent of the code, but both times it found problems. The irritation level was high because the legacy code wasn't pretty or easy to get running, but otherwise unknown defects removed and fixed made it worth the effort. Automate the regression tests in the build if you can. If nothing else, there are always business processes and requirements buried in the code that aren't known or documented anywhere else.

"Build Time" Validations

There are always some aspects of the system that the compiler and maybe even the unit test library can't validate. Use the automated build as an opportunity to create "build time" checks for potential problems or enforce coding standards. Here's some ideas of things to enforce in the build:

A compiler warning or "TODO" threshold. I've heard of people making a build fail for this.
A limit on ignored unit tests
SQL validator. Gotta keep the sql external to the code somehow, though.
O/R Mapping. We're starting to adopt NHibernate for persistence. One of the first steps will be validating the mapping in the build
Automated code coverage is kind of a test of your tests
Static code analysis

There's probably more, but that's all I can think of. I'd love to hear other ideas.

Taxonomy of Tests: Acceptance Tests

2005-05-24T16:45:00.000-05:00

From Ward Cunningham's wiki: "A formal test conducted to determine whether or not a system satisfies its acceptance criteria and to enable the customer to determine whether or not to accept the system."

The acceptance test is exactly what it sounds like, it verifies that the implementation of a user story works as the customer requires. In a literal interpretation of XP theory, the customer writes the acceptance test. In reality, the acceptance tests are going to be written by some combination of the business partner, testers, business analysts, or product managers. It really doesn't matter to me, as long as they are done in a timely manner with some degree of correctness. You'll never have the ideal XP customer, but it doesn't necessarily mean the project is hopeless.

It is a huge advantage when acceptance tests are approved and ready during coding. User stories, use cases or huge paper System Requirements Specification documents -- the best and most effective requirement is a cut and dried acceptance test. When the coders can (and do) easily execute the acceptance tests themselves, the bug counts will go down. The development team as a whole can use the acceptance tests as the “Red Bar” metric to watch during an iteration. A user story is “done” only when all the finalized acceptance tests pass. James Shore has a good post on acceptance tests as requirements here.

Developer/Tester Cooperation

Testing features during an iteration as soon as the code dries is a tremendous advantage. It's much, much simpler to fix bugs that were just coded before the memory of the code fades and the bug is covered up my much more code.

My current project has not had acceptance tests written during iterations, and it has clearly hurt us in terms of wasted effort and misunderstood requirements. Never again.

Organizational issues between testing and development are not specific to agile development, but the desire for rapid iterations will expose any friction between requirements, code, and testing. Moving fast without sacrificing testing requires the ability to cycle code quickly between development and testing environments. Code that is easy to test helps quite a bit, too. Developers have to support the testers with the mechanics of testing. Both testing and development can benefit from a good Continuous Integration (CI) strategy. Testers should be involved in the CI effort. I feel very strongly that it is the developers’ responsibility to ensure the test environment is in a valid state. “BuildMaster” folks are a desirable luxury, but I always want visibility into the test server environment.

Automating Acceptance Tests

Acceptance tests are a good and necessary thing. As far as testing is concerned, an automated test that runs often and visibly is far more valuable than a slow manual test buried in a test plan. A common objection to iterative development is the need for so much regression testing. The best answer to this perfectly valid objection is to automate as much as you can.
Enterprise applications are rarely finished; they're abandoned when they are judged to be too expensive to change. Automating acceptance tests is a great way to extend an application's lifecycle by reducing the cost of regression testing. Also, to paraphrase a former colleague -- "If your tests aren't automated, how do you know they're passing?" Think on that one.

Who writes the test is one decision. Who automates the tests, and how, is a different decision. Hopefully you're lucky enough to have testers with programming or scripting abilities. Even if you do have technical testers, plan on the developers assisting the testers with test automation. I'm not a big fan of FIT and its derivatives, but it is definitely a step in the right direction for acceptance testing (I love the concept, but I hate the implementation of NFit). The FIT model allows for writing acceptance tests in near English, with some non-trivial work behind the scenes to complete the wiring for the custom FIT fixtures.

Conventional wisdom says that acceptance tests are black box tests that test the system end-to-end. This doesn't necessarily have to be the case. I’ve seen both Brian Marick and Brett Pettichord write about using white box tests for acceptance tests, especially for user interfaces. Using Service Orientation principles to create a cohesive service layer can be a great aid for automated acceptance testing. I think (N)FIT chokes on user interfaces, but it is perfect for driving an isolated service point (note I didn’t necessarily say web service) in an automated test. These white box tests can be much easier to setup and execute. Some flavors of MVC architecture allow for slipping in a screen proxy in place of a real UI view (I've had mixed results with this). Again, it is not a “perfect” black box test, but it goes a long way towards validating the behavior of the system.

Good testers can work with developers to write these automated whitebox tests. It does take pretty good communication and cooperation between coders and testers to take advantage of these opportunities for test automation. It is definitely advantageous for the testers to be familiar with some of the inner workings of the system. I definitely think agile development starts to blur the line between developers and testers, and that's not an entirely bad thing.

Once and for all, software is not like construction or engineering.

2005-05-24T16:07:00.000-05:00

Here's a great post from Coding Horror on Bridges, Software Engineering, and God.

I was a mechanical engineer before I wandered into software, and the comparisons between software development and engineering & construction always bug me. A software project has very little in common with a construction project, and we should stop trying to ram our square software peg into the construction hole.

Taxonomy of Tests: Unit Tests

2005-05-23T15:11:00.000-05:00

At the most granular level is the unit test. Unit tests are written by developers to verify, on in Test Driven Development specify, the functionality of a very small piece of code to the satisfaction of the developer. Usually a unit test works on one class or a small cluster of classes at a time. One of the key attributes of a unit test is the ability to test a small unit in isolation. We want to prove that the unit of code itself is working without corruption or interference from other classes.

Purposely designing code in order to create isolated unit tests is part of a successful Test Driven Development effort. Mocks and stubs are an invaluable tool to create the isolation. Some people insist that all external dependencies (databases, Active Directory, web services, etc.) be mocked out in unit tests. My best advice is to consciously quarantine inconvenient things behind gateways with an abstracted interface. Use Inversion of Control to minimize dependencies from the majority of your .NET code to these gateways. When you can't eliminate a dependency, use Dependency Injection to provide a mechanism to replace the real thing with a mock or a stub. I have had very positive (well, mostly) experience with the Model-View-Presenter (MVP) pattern described by Martin Fowler as a strategy to maximize the testability of user interface code.

Unfortunately, some units of code only exist to manipulate the database or interact with infrastructure services like messaging. Not to mention that you still have to unit test the UI screens. MVP is not enough in and of itself to guarantee a properly functioning user interface (trust me on this one). You can either marginalize this code by not writing tests and hope for the best (or let the acceptance tests take care of it), or bite the bullet and write a battery of tests that are connected. My advice is to put some ketchup on the bullet and just do it.

I do prefer to separate connected and non-connected unit tests into separate test fixture assemblies. I generally lean on only the non-connected tests during refactoring or new development. I do think it is absolutely necessary to run the integrated unit tests in both your check-in procedure and the automated build.

There is a school of thought that says if you really unit test well enough against the mocks, integrated testing is unnecessary. The NEO project is the best attempt I've seen anybody use to actually approach this ideal. I still think this philosophy is unattainable and possibly dangerous.

I've been in some heated arguments on the merits of mocking ADO.NET when testing custom data access classes. I mostly vote no to mocking ADO.NET. In this case, the unit of valuable work is manipulating the data in the database. Who cares if the code sets the “expected” parameter values when the real stored procedure has been changed in the database behind your back? In this case, the effort to value ratio is all wrong. Of course, when you’re a consultant onsite at the client you work according to their goofy development standards and politely say “Thank you sir, may I have another?”

Next up, Acceptance tests...

Taxonomy of Tests: Defense in Depth

2005-05-23T15:07:00.000-05:00

I made the mistake of posting on writing good unit tests while suffering from "Jedi Flu" and Jeffrey Palermo took me to task for not defining a unit test first, so here's my attempt at describing the various types and approaches of tests I've seen or used on agile projects. Keep in mind that I'm a developer first and only an occasional architect, “BuildMaster”, or tester second. The first thing to keep in mind is that agile processes don't change the fundamentals of software development. It is still best practice to test the code at multiple levels of integration (the old V-curve idea). It's great to mock the database in your unit tests, but the database is still there and just itching to get out of synch with your middle tier code.

A Taxonomy of Tests

Roughly, I would divide tests into:

Unit tests
Acceptance tests
Intermediate/System/Integration tests, written by developers
Other: smoke tests, regression tests, performance tests, miscellaneous validation

Testing Strategy

The answers will vary from project to project due to manpower constraints and the nature of the development, but any and every project is going to have to answer these questions below.

Who writes what tests?
Who automates the tests, and how?
Where and when do the tests run?
How do we want to organize the test suites?
How much testing is enough?

Sleeping with the Enemy

One thing I've learned that does not vary; it takes a great deal of cooperation and mutual respect between the testers and coders. Working in the same room is ideal. On projects where the tester and developers are separated by either organization or location, the development effort will suffer. For that matter, both developers and testers need quick and easy access to whoever holds the reins on requirements. At a bare minimum, don't look at testers as the enemy.

One of the truly great advantages agile development has over traditional waterfall projects is in team dynamics. There is far more interaction and communication between disciplines. Just being on the project at the same time is an advantage over “throw it over the wall” processes. Everybody’s common goal on an agile team is producing working software. Not code, tests, UML diagrams, or Gant charts, but working code that meets the customer’s expectations.

Next up, Unit Tests…

May 19th, 2005 - IT Productivity Hits an All Time Low

2005-05-20T14:30:00.000-05:00

I'm finally recovered enough from the midnight opening of Revenge of the Sith to post some sort of review. I think I'll make a point of never going to a late movie on a work night ever again. My entire contribution to work Thursday consisted of suggesting Mexican food for lunch.

Here's a sorta review:

I loved it. There were definitely some movie mechanics things to nitpick about, but I had a blast. It moves pretty quickly. Even if the dialogue wasn't perfect, nothing stood out like a sore thumb the way Anakin whining or Jar Jar did in I & II. I think there is probably too much story for a single movie. You get the feeling that a lot of important events happened offscreen, and the progression to the dark side happens a bit too fast. I read a lot of people saying they came out of the movie and immediately wanted to go watch the original trilogy. I definitely concur.

The Good:
Jar Jar is rarely seen and never heard.
I liked Hayden Christensen in this one.
Yoda rocked
The Emperor nearly stole the show
The Obi-Wan/Anakin interaction was much better
R2 is still the best comic relief ploy
The visuals were stunning. The opening sequence was fantastic.
Lucas ties up a lot of plotlines at the end and sets the stage for the original trilogy
Wookies definitely trump Ewoks (or Gungans for that matter)

The Bad:
Some of the special effects look wrong. The clonetroopers look fake sometimes
IMO, It is too dark, violent, and emotional for small children. Take the PG-13 thing seriously. Much more violent than any of the others.

Qualities of a Good Unit Test

2005-05-18T10:54:00.000-05:00

Many developers, including me sorry to say, treat unit test code as a second class citizen. After all, NUnit test fixture classes aren't going into production, so why should you put a lot of time, effort, and thought into them? An unfortunate reality is that unit tests are rarely "write-only" code. Badly written unit tests or interdependent unit tests hamper refactoring or adding new code to the system. I've heard of teams that were afraid to add new code to a system because the changes would introduce test failures. One of the advantages of automated unit testing should be the ability to safely modify existing code. Bad unit tests turn this advantage on its head.

I'm giving an internal presentation today on mock objects and I prepared a slide on the "Qualities of a Good Unit Test." I actually couldn't find much in the way of resources on the web for this, so I made up the list below. I'd love to hear from the rest of you what your best practices are for unit tests.

Unit Tests Should Be Atomic

A unit test should only test a small piece of functionality. This falls inline with the idea of getting into a rhythm of "write a little test, write a little code." A litmus test is to ask yourself if any part of the unit test could stand alone in a separate unit test. Another feedback loop for your unit testing quality is the amount of time you spend with the debugger. If your unit tests are coarse, a test failure is more difficult to find. If the unit test exercises a small amount of code, the test failure cause can usually be spotted very quickly.

Order Independent and Isolated

There should never be an order dependency, intentional or not, between unit tests. Problems arise when one unit test leaves some kind of dirty data laying around. Testing against a database or some sort of static cache is a common culprit. The theme of the day was mocking (actually Episode III), so put evil stateful things behind nice, deterministic mocks or stubs.

Intention Revealing

A unit test can, and should, be a valuable form of documentation. At best, a unit test should explain the intended usage and function of a class. At worst, the unit test should still be easy to debug in the case of a regression failure. Excessive data setup can obfuscate a unit test beyond any hope of comprehension.

Easy to Setup

How easily can I set up an environment to run the tests? If the answer is "not very," I might never get the tests running. As much as possible, decouple unit tests from external dependencies like databases and web services by using mocks or stubs. When you do test against an external dependency, the dependency better be setup correctly by the automated build.

Runs Fast

You're never going to run the test just once. For the sake of constant feedback, successful continuous integration, and your sanity, make sure your tests run quickly. Thirty minute build and unit test cycles are nothing but friction. Just to keep in the mock rut, mock things that make network calls to speed up your testing. I like to segregate connected tests into a seperate assembly. This way you can quickly run the majority of your unit tests on demand without the lag from network calls.

There's surely some obvious things I'm missing, but I'm short on sleep (and a midnight movie awaits) at the moment so that's all I've got.

Architect: Person, Role, or Useless Appendage?

2005-05-12T21:43:00.000-05:00

There is a good thread going (or was when I started this) on at TheServerSide.NET about "What Skills Should a .NET Architect Possess?" The correct question might be whether or not an architect is different from a senior developer. More importantly, how does the architect (or analogue) interact with developers? I've been (stranded) on both sides of the fence on this question and I'm still ambivalent, so I'll let everyone else decide.

At one pole is the heavy process waterfall shops with elaborate responsibility matrices. Architects do all the design upfront in comprehensive documents and developers code what they're told. It doesn't really work that way of course. When I was an architect (at least in name), I wasn't allowed to code. I was told that I was "too important to code." I wasn't important enough to be paid well, but just important enough to do lots of powerpoint presentations to kick off projects that were cancelled within a month.

On the other hand, I walked away very unimpressed from my first and only true Extreme Programming project. Doing the simplest thing that possibly works, with little or no emphasis on architecture or even design at all, can lead to truly awful code. I felt we consistently missed opportunities to become more productive by just stopping to think once in a while. We missed abstractions that could have helped us go faster. We waited to refactor to use an abstraction after it became pitifully obvious the abstraction was necessary. Refactoring is good, but there is no such thing as a big refactoring. Big refactorings are just rework.

A terrible amount of code was duplicated because every pair was so focused on their own stories. Pairing and rotating helps, but you're still only working on a user story at a time. Even worse is the fact that user stories are dribbled out to the developers, so there doesn't seem to be any way to get traction on a coherent design. Important concerns like instrumentation, data access strategy, operational support, and security can easily be lost in the hustle and bustle of trying to make the iteration.

I don't believe in very many absolutes (always do this, but never do that) in software development, but here's a couple I do believe:

Non-coding architects create terrible architectures. If you can't code it, you certainly cannot design it. I had to come from behind one of my architect peers on a large project after he had been more or less been kicked off the project for being a pain in the rear. The project manager just looked a little dazed and said, "It was going so well at first. He was producing so many design documents and then..." Development just changes too fast to do the beard stroking architect thing. Turn around twice, and your COBOL/VB6/Pro IV skills have been passed by. It was pretty scary being a non-coding architect in early 2003 with no production .NET or J2EE experience.
Development is often all about responding to feedback. Feedback from your code when you realize your design isn't ideal or a different approach proves superior, feedback from performance data, and feedback from the business people as they refine their understanding of their needs. Feedback needs to be acted on quickly in order to be valuable. Small refactorings done as the need arises greatly streamlines the code. Big refactorings put off lead to nightmares. Developers who aren't able to deviate from a bad design because of a lack of skill or authority are more or less helpless to create good code. Both times I've been the recipient of a design spec from someone else the "design" was tossed out as completely unworkable.
Spec coders suck and cannot create any kind of non-trivial software. Being a spec coder sucks even worse. Being a non-coding architect working with spec coders is a living nightmare no one should have to endure. I've found to my chagrin that the more detailed your instructions are to another developer, the worse code they create. Give the same developer more leeway in their approach and maybe some amount of coaching, and their code is consistently better because they're thinking while they code. Everybody should be involved in design to better understand why they're doing what they're doing.
Design should be continuous. I like Jim Shore's article on this topic. The key is to always be thinking and challenging your design and never get complacent. Keeping somewhat in line with agile ideology, do think ahead and anticipate needs and risks, but don't act upon them earlier than you need to. I'll blog much more on this some day. I don't really fit into either the XP or RUP crowds in my approach to design. It's the year 2005, the discussions should have moved on from "waterfall or iterative" to "how best to do iterative development" a long, long time ago.
It's very advantageous to step outside of the current batch of stories or code and look at the bigger picture. If that requires tagging someone as the designated architect, so be it.
You better have somebody (preferably plural) with real technical design skills involved with the coding effort on a daily basis.
Regardless of process or approach, design skill is vital. Being able to quote Chairman Beck's little white book with a gleam in your eyes does not count as design knowledge. A working knowledge of design patterns and the fundamentals of good design can actually contribute to the success of a project. If all you know is write "if/then/else" statements until the NUnit tests pass, your version of the "Simplest Thing Possible" is probably a bad design.

"I feel better now, thank you for listening" -- Roy Moore

Presenting the Geek of the Day Award (off topic)

2005-05-06T09:52:00.000-05:00

To help pass the time, a project team I was on as a consultant used to have an unofficial "Geek of the Day" award for whoever made the nerdiest comment or reference of the day. Obscure Star Wars quotes, Robotech, Tolkien, Dr. Who, and Dragonlance references qualified. We had to retire the award for a while when a business analyst (!) trumped us all by showing us the artwork he had published in Wizard magazine.

My new boss is paranoid I'm using this blog to rag on him and the mothership in Houston, so here it goes -- Chris Fields, I present to you the Geek of the Day for filling out your own performance self-evaluation as "Mostly Harmless." I'll take it over in a couple of weeks for faking a reason to make an overnight "business" trip to Houston just to see the midnight episode 3 premiere with the rest of the company.

TDD Design Starter Kit - Ideas for Upcoming Posts

2005-05-05T13:03:00.000-05:00

My loose plan for more posts in the 'TDD Design Starter Kit' is something like this:

State vs. Interaction Testing
Mocks & Stubs (probably put an example of NMock up - the docs aren't too great)
Inversion of Control & Dependency Injection ( I've written and use a DI tool on sourceforge called StructureMap, so expect more than you ever want to know about DI usage, w/ or w/o a "container")
Statics and Singletons Can (sometimes) be Harmful to Your Health
Humble Dialog Box (Model-View-Presenter, I dislike Martin's name for this. It's just a variant of MVC)
Favor Composition over Inheritance
Isolate Hard-to-Test Dependencies
Know When to Punt!
Bare Bones NAnt/CruiseControl.NET

I'm intending this set of posts to be kind of an open forum for my new development team. I'd be plenty happy to take any suggestions on other topics.

Does TDD Make Development Take Longer? (No)

2005-05-04T11:18:00.000-05:00

Overall, I'd say the answer to this question is no. Several people in the Austin .NET Agile Group are going through the typical growing pains of introducing TDD to a development organization. The very obvious objection to having to write and maintain much more code is being raised by developers.

Yes, writing automated tests can be laborious. I routinely observe myself writing every bit as much testing code as production code. However, I have found that writing code might take longer with TDD, but software development as a whole (design, code, test, rollout), is faster. After all, the real goal isn't slapping out code, it's getting working code to the business.

Here is where I think TDD cuts time in software development:

Design. TDD is a low level design technique. It wasn't an overnight conversion, but since I've started using TDD I spend much less time with UML and other upfront design techniques. The feedback cycle between "I wonder if this design will work" to validating the design is much shorter.
Debugging. As your ability to do Test-First unit tests improves, your usage of the debugger goes way down. I've definitely seen this in my coding over the last two years, and I've read may similar accounts.
Fixing bugs. Bugs can be fixed more rapidly because the unit test coverage can detect regression failures before a tester does.
Testing. Fewer bugs get through to the testing team. By doing test-first, you have to take testability into account in the design of your application. Non-coding architects doing upfront design may have no incentive to promote testability. From painful experience, if you use any kind of test coverage tool like NCover or Clover.NET, you'll find a nearly one-to-one relationship between code with poor unit test coverage and the code that spawns reported bugs.

Learning TDD

Learning TDD was difficult, especially for someone like me with a bit of background in UML modeling. It hurts at first, but TDD does get easier with practice. There is far more collective knowledge and experience with TDD in .NET today than there was a couple of years ago. I'm trying to use my "TDD Design Starter Kit" blog series as a start for my group.

Maximizing the TDD Advantage

TDD can be done with any process or organization, but it is more effective and valuable in an iterative and incremental process openly using emergent design. Every developer (or pair) needs to have some control over the fine grain design details. Spec coders need not apply.

Refactoring automation tools help tremendously. ReSharper (and Whidbey?) can generate method stubs for missing methods. That little piece of functionality made writing tests first much easier mechanically.

Continuous Integration (CI) is a natural extension of TDD. The TDD/CI combo is very powerful in reducing friction between development, testing, and production.

There are obviously some types of code that are much easier to verify manually instead of an automated test. In these cases you might compromise and get by with a manual test. A lot of effective TDD design is minimizing or isolating hard to test functionality to maximize the testability of the application as a whole. The Model-View-Presenter pattern ("Humble Dialog Box") is a good example of this. I'm a big fan of using mock objects for external dependencies.

TDD Design Starter Kit - Dependency Inversion Principle

2005-05-02T12:34:00.000-05:00

In the last episode of the TDD Design Starter Kit, I talked about the need to build cohesive classes, and make the relationships between classes loosely coupled. The specific benefit for TDD is to truly isolate the functionality of a class under the microscope of a unit test. But we've done all we can to isolate our classes, and we still have some interaction with dependencies. How then can we test the class in isolation?

Use the Dependency Inversion Principle (DIP) to isolate one class from the actual implementation of its dependencies.

Depend upon Abstractions. Do not depend upon concretions

DIP is certainly not an invention of TDD practitioners, but it is employed frequently for no other reason than to enhance testability.

Take a look at this code that does not employ the Dependency Inversion Principle.


 public class MessageProcessor1
 {
  public void RouteOrder(Order order)
  {
   string queueName;
   if (order.IsComplete)
   {
    queueName = "Queue.Completed";
    order.CompletionDate = DateTime.Now;
   }
   else
   {
    queueName = "Queue.Incomplete";
    this.assignOrder(order);
   }

   OrderQueue orderQueue = new OrderQueue();
   orderQueue.SendOrder(order, queueName);
  }

  public void assignOrder(Order order)
  {
   // do something to the order
  }
 }

 public class OrderQueue
 {
  public void SendOrder(Order order, string queueName)
  {
   MessageQueue queu = new MessageQueue(queueName);
   queu.Send(order);
  }
 }

So what's so wrong? The MessageProcessor1 class cannot be tested, or function, without the OrderQueue class being executed.

Not only is MessageProcessor1 strongly coupled to OrderQueue, it is coupled to anything that OrderQueu is dependent on. In this case the MessageProcessor1.RouteOrder() method cannot be tested unless MSMQ is available. More annoyingly, any automated test would need to query MSMQ to get the resulting Order object to compare the expected results.

Now, using DIP we make the MessageProcessor2 class dependent upon an abstracted IOrderRouter interface instead.


 public interface IOrderRouter
 {
  void SendCompletedOrder(Order order);
  void SendIncompleteOrder(Order order);
 }

 public class MessageProcessor2
 {
  private readonly IOrderRouter _router;

  public MessageProcessor2(IOrderRouter router)
  {
   _router = router;
  }

  public void RouteOrder(Order order)
  {
   if (order.IsComplete)
   {
    order.CompletionDate = DateTime.Now;
    _router.SendCompletedOrder(order);
   }
   else
   {
    this.assignOrder(order);
    _router.SendIncompleteOrder(order);
   }
  }

  public void assignOrder(Order order)
  {
   // do something to the order
  }
 }

MessageProcessor2 is only dependent upon an interface, and can use any implementation of the IOrderRouter interface (in theory).

Here's what a unit test for MessageProcessor2 might look like:


[Test]
public void RouteACompletedOrder()
{
    Order order = new Order();
    order.IsComplete = false;

    DynamicMock routerMock = new DynamicMock(typeof(IOrderRouter));

    // SendIncompleteOrder(Order) should be called on the incomplete order
    routerMock.Expect("SendIncompleteOrder", order);
    routerMock.ExpectNoCall("SendCompletedOrder", typeof(Order));

    IOrderRouter router = (IOrderRouter) routerMock.MockInstance;

    MessageProcessor2 processor = new MessageProcessor2(router);
    processor.RouteOrder(order);

    // Call to the DynamicMock to verify the interaction
    routerMock.Verify();
}

Now we have our routing logic decoupled from the delivery mechanism. Besides testability, we have added the ability to substitute other delivery mechanisms like a web service, MQ Series, or simply writing to a file.

Mechanically, there is a couple things of note in the unit test above. I used a dynamic mock object with NMock as a stand-in for the IOrderRouter interface to test the interaction. The IOrderRouter instance was passed into the MessageProcessor2 constructor, not created by MessageProcessor2 itself. This is an example of Dependency Injection (Inversion of Control). I will try to blog on both topics soon.

Rules of Thumb to Use DIP

Anytime your code calls out of the CLR, and maybe just the current AppDomain itself. I would isolate any kind of call to databases, messaging queues, web services, and remoting behind a .NET interface. In ASP.NET or web service applications, always abstract any kind of call to the HttpContext objects. Nothing gets in the way of automated testing like coupling to the ASP.NET runtime. Remember, one of your unofficial design goals is to test as much of the application as possible with your laptop disconnected from the network. Look at the Gateway pattern. As a mild rant, using a Service Oriented Architecture does not guarantee that your actual code is loosely coupled. I'll expand on that soon.
Singleton's. Singletons can throw a pretty serious monkey wrench in your unit tests by maintaining state between unit tests. Here is one way to avoid the Singleton issue I stole from some former colleagues that wrote PicoContainer.
Interface points between major subsystems. Aggressively use the GoF Facade pattern to limit the exposure between subsystems, and abstract the interface to allow for easier testing with mocks or stubs.
Extension points. This is a pretty obvious usage, but knowing when to make an abstracted extension point is way, way beyond the scope of a mere blog post.

A great codebase to look at for examples is the CruiseControl.NET application. I think they do a very good job at testing a nasty piece of functionality.

Long Methods and Classes Are Evil

2005-04-26T21:34:00.000-05:00

I've spent a great deal of time in the last week reverse engineering legacy VB6 code, and I'd like to rant a little bit. What's making my life harder is how large and complex the methods are in each class module. The code is very procedural, with little usage of subroutines. The VB functions are several pages long, if/then nesting goes 6 or 7 deep, and case statements span hundreds of lines. Unsurprisingly, the code is hard to maintain and brittle in regression testing. Finding the location of a piece of functionality requires arcane knowledge of the application or some amount of luck with CTRL-F.

If the application you are building or modifying is important or useful, it is a sure bet the code will have to be modified later as requirements change and problems occur. There are some easy steps to take to make your code more understandable. Good naming is obviously important, a variable named "a1" isn't very descriptive, especially when it is an untyped variant. Can you say "Mystery Meat?"

My personal pet peeve is really long methods and classes that do way too much. I'm a terrible slob in most other ways, but I want my code readily organized with each type of code on its proper shelf, err, class. The human brain can only process so much information at a time. Small methods can be seen and understood all at one time. Bugs can hide out in very long methods because there is simply more brush to hide in. Large case statements are veritable camouflage for lurking bugs.

Rules of Thumb for Class and Method Size

Any method should fit into your IDE window so the entire method can be read at one time. I'd take it farther if you can. One of the few code metrics I pay attention to is the average lines of code per method. I strive for an average of not more than 5 lines of code per method.
Too many private members. You know what I mean, if you see a class with an excessive number of private fields (say >10), the class is probably too big. I'm tracing a class with about 75 private fields this morning. Not pleasant. In Fowler's Refactoring book this is described as the code smell "Primitive Obsession."
The size of the code file. Offhand, I'd say any class over 10k in size is getting too large for its britches. A class that is 200k in the file system is certainly too long. TestFixture classes might be an exception.
Exception handling code and instrumentation tend to push methods to be much larger. Invest some thought in how to segregate this type of code away from the main functionality. Aspect Oriented Programming may justify its existence by merely doing this.
Reduce the number of public methods in a class. Just picking a number, I would say less than 10 in most cases. There is a good discussion on this subject at http://c2.com/cgi/wiki?FewShortMethodsPerClass.

I'm not just picking on VB developers, because I've seen (and written) bad examples in other languages, including a C# system that is featured in an early case study on MSDN as a shining example of the wonders of .NET. At a former employer, my teammates and I inherited a large new supply chain system hacked together by a very prominent consultancy. I distinctly remember a coworker printing out a Perl script that contained an if/then branch spanning 18 pages.

As an aside I'm 2-3 years removed from any kind of serious VB6 work. I remember a time when I was a staunch pro-VB guy because of its productivity for building code rapidly and hated being looked down on by Java guys. After becoming quite used to C# coding with ReSharper and other toys, VB6 feels extremely clunky. Combining well-factored C# code with the advanced code navigation features from VS.NET itself and ReSharper, leads to code that is readily traced and reverse-engineered. Oh yeah, I forgot to mention that variants suck and COM sucks more. I do still hate being looked down on by Java guys, though.

Bypass VSTS and get yourself a "Poor Man's IntelliJ"

2005-04-12T21:13:00.000-05:00

A couple months ago at an agile users group the question was posed, "Do I need Visual Studio Team System to do Test Driven Development?." The answer is an emphatic no. I think it is great that Microsoft has tacitly endorsed TDD and Refactoring by building support into VS.NET, but VSTS is not the only game in town for agile development with .NET.

The painful truth is that the Java community has had far superior IDE support from tools like IntelliJ and Eclipse. IntelliJ is an agile developer's dream IDE. Very good integration with source control, JUnit, and tremendous support for refactoring and code generation. The .Net world got a late start, and Microsoft's emphasis always seems to be on providing more visual drag 'n drop programming for Mort. Lucky for us we don't have to defect to Java or wait and pay (a lot apparently) for VSTS because there are alternatives for .Net today. And don't think for a minute Microsoft hasn't heard "just make it be like IntelliJ/Eclipse" a few too many times for their own comfort.

The VSTS team at Microsoft seems to bristle anytime someone mentions open source alternatives and spit out a comment about "Cobbling together downloaded tools." The reality is that these tools are superior in some respects. Assuming you don't mind assembling the stack yourself, you can use these mostly open source (and free) tools to create what I affectionately call my "Poor Man's IntelliJ."

NUnit - The defacto standard for unit testing in .NET. Arguably better than its Java forebear. I know the NUnit guys are already planning to be compatible with the Whidbey unit testing. I've also heard good things about MbUnit. Supplement NUnit with NMock for dynamic mock objects and NUnitForms and NUnitASP for UI testing.
TestDriven.NET - VS.NET plugin to run NUnit or MbUnit tests in the IDE. I love this tool. It allows you to run tests by clicking on a unit test, testfixture class, or namespace. It also provides the ability to start the debugger using any test method as an entry point. You lose the visual green bar, but the text output includes stack traces that can be clicked on to navigate to the offending test code.
NAnt and NAntContrib - Automated build tool. Right now it looks like MSBuild provides only a subset of the NAnt/NAntContrib tasks, several of which my team depends on in our build process. The MSBuild integration/synchronization with VS2005 is cool, but you can run NAnt from within VS.NET today. Using the solution task, a NAnt script is not too onerous to maintain for moderate sized solutions.
Subversion - We started using Subversion, a free open source version control system built to finally replace CVS. So far I think it is the most problem free source control I've ever used. CVS has a personal animus towards me, but is still a viable and proven tool. VSS is simply unsuitable for serious development needs. Agile development can require more robust source control and VSS just won't deliver.
Ankh - VS.NET plugin for Subversion integration. I'm able to do probably 90% of my source control updating and commits from within the IDE now. It has some contention with ReSharper, but I still enjoy using the tool. I still use TortoiseSVN occasionally.
CruiseControl.NET - Continuous Integration tool. The latest versions seem to be much easier to install and configure, and the website reporting has gotten much easier to use for debugging build failures. Very good integration now with Subversion, NAnt, and NUnit. To the best of my knowledge, VSTS only supports scheduled builds anyway.
ReSharper - The only commercial tool on this list. The best $100 tool anywhere. ReSharper provides refactoring, code navigation, and code generation templates. Since it became stable last year (the beta was flaky), I will not develop on any box that does not have ReSharper. The refactoring support in ReSharper 1.5 is superior to VS 2005 in many respects anyway.
NCover - Code coverage tool. The VSTS equivalent integrated into the IDE does sound really good.
OpenWiki - Use any Wiki engine as a great means of storing and organizing project information and documentation. I've used Jira in the past for project tracking and actually liked it, but a Wiki is good enough for now.

This is not an exhaustive list be any means, but it covers most of the bases. I'm much happier with .NET development now that I can perform most tasks from within the IDE.

To be honest, I have not paid much attention to Visual Studio Team System. My shop probably doesn't have any justifiable need for VSTS and a deep investment in existing tools. Some of the ideas for integrated project tracking and metrics sound pretty cool, but by itself an agile process provides so much more transparency than heavier methods that I'm not sure the formal tracking is important.

Feel free to suggest other tools. I would like to hear from people adopting VSTS, too.

TDD Design Starter Kit – Responsibilities, Cohesion, and Coupling

2005-03-30T21:56:00.000-06:00

Recently, I started a new position with a company transitioning to TDD. One of the things I'm trying to accomplish in my new job is to help my colleagues become effective in TDD without most of the growing pains I experienced. So in an effort to help my new organization, I'm using this blog to communicate all the things I wish someone had shown me before I started writing code with Test Driven Development. So, in a couple of parts, here is all the off-the-cuff advice I can think to give to a TDD newbie to avoid seeing NUnit red bars in their sleep and getting the “Shame Card” permanently affixed to their monitor (for repeatedly breaking the build).
Separation of Concerns

Separation of Concerns is one of the most important concepts in programming, and the single most effective strategy for writing classes and methods that are easy to test. Simply put, divide your system into modules or layers, each with a distinct area of responsibility. We’ve known that layering is a good idea for a long time, but nothing exposes an intermingled ball of mud than trying to write automated tests. After you do TDD for awhile, you will probably notice that you slice classes thinner than before.

At the heart of every programming paradigm and design technique is a “Divide and Conquer” strategy to break a complicated whole into easily digestible pieces. Using OO programming, we are mostly concerned with defining the different responsibilities of the code, and assigning the responsibilities to the logical classes and methods.

There are a couple of qualities (more info here) to guide the division of labor in your class structure:
Cohesion – A measure of whether a class has a well defined, meaningful responsibility. High cohesion is desirable. If a class contains unrelated functions or responsibilities, it is not cohesive.
Coupling – More or less, how entangled or dependent a class is with other classes. A loosely coupled design implies that classes or subsystems can largely be modified independently of one another. The Pragmatic Programmers call this quality “orthogonality.”

TDD is all about rapidly making small pieces of code work, then molding the pieces into the aggregate. Cohesion is important to us, because we want to focus on making a single concern work at a time. Coupling is important because we want to make a concern work without interference from other concerns polluting our tests. Data access, business logic, user interface display, messaging, and logging are examples of separate concerns or responsibilities that should be built and unit tested separately. Divide and conquer.

Here’s a sample of code that flunks both cohesion and coupling tests.


public void PublishBigThings()
{
    // Go get some configurion
    int threshold = int.Parse(ConfigurationSettings.AppSettings["threshold"]);
    string connectionString = ConfigurationSettings.AppSettings["connectionString"];
    
    string sql = select * from things where ready_to_publish = true and size > ";
    sql += threshold;
    
    SqlConnection connection = new SqlConnection(connectionString);
    
    connection.Open();
    SqlCommand command = new SqlCommand(sql, connection);
    SqlDataReader reader = command.ExecuteReader();
    
    while (reader.Read())
    {
        string name = reader["Name"].ToString();
        string destination = reader["destination"].ToString();
        
        if (this.IsDestinationApproved(destination))
        {
            Publish(name, destination);
        }
    }
}

The method is not cohesive because it is responsible for doing everything – pulling flags out of configuration, fetching data from a database, and making business logic decisions. The method is tightly coupled because it cannot function correctly without the proper configuration in place and a Sql Server database (other database engines need not apply).

“One Class, One Responsibility” is a good rule of thumb to follow in class construction. Don’t hesitate to create more classes. It can feel like more complexity, but it can also isolate complexities. See Fear of Adding Classes. It’s also much easier to make mistakes in big methods than in smaller methods.

More to come. Someday.

Succeed with TDD by designing with TDD

2005-03-22T21:18:00.000-06:00

After 2 years of using Test Driven Development on .NET projects I'm a believer. Effective TDD leads to cleaner code, fewer bugs, and superior architectural qualities. Systems written with TDD are easier and safer to modify and deploy, leading to a reduced cost of ownership over the lifetime of an application. Yes, TDD means a lot more code to write. Yes, TDD can make the activity of writing code take longer, but it can and does shorten the time to delivery of working, production-quality code.

Using TDD dramatically changed the way I approach OO coding for the better, but I made a lot of mistakes along the way. The easiest way to learn TDD is to work closely with teammates that are already comfortable with TDD. Otherwise, you are on your own, and a lot of the writing I have seen on TDD falls into one of two categories -- "TDD is Cool" and "How to use NUnit." You've got to start somewhere, but when faced with the very real world of enterprise architecture (databases, messaging, user interfaces, Active Directory), the simple examples of create an object, pump in some inputs, and validate the results suddenly don't seem to be enough.

In my experience, newcomers to TDD often struggle with the mechanical work of writing unit tests, negating most of the value of TDD by writing unit tests that are too coarse. A common complaint is that the tests are too hard to write, or just take too much time to write. Frequently, well-meaning developers will "just make it work" first, and retrofit tests around the new code later. I did this pretty often on my first TDD project -- with the result largely being a set of fragile tests that flat out sucked and broke the build anytime someone breathed too hard on the codebase.

The usual culprit is the way that the code is written and structured -- god and blob classes, stovepipe architecture, poor separation of concerns, and an overuse of static methods. So how do we write code to maximize the efficacy of TDD? The most important thing is to write good OO code (I'm not trying to exclude folks doing procedural or functional coding, but this post is meant primarily for .NET developers). A couple years most of my team of System Architects (sic) were sent to an introductory(!) class on OOA/D and came back giggling and blathering about code being "Highly Cohesive" and "Loosely Coupled." All those fuzzy sounding OO qualities we've known about for years aren't just a way to impress other developers while stroking your beard, they are absolutely necessary to easily utilize TDD. It's time to take these concepts seriously in the .NET world. Read this from Scott Bellware on TDD in the Microsoft community versus the Java world. A bit more on cohesion and coupling later.

Which Comes First, Chicken or Egg?

A TDD developer follows a continuous cycle of writing a small, isolated unit test for a little bit of functionality, then writing a little code to make the test pass. Ideally, an individual unit test involves a discrete chunk of a single class (or a few classes). How do you know if your unit tests are small/isolated/discrete enough? Easy, VS.NET debugger sessions become uncommon and short and test failures are easily diagnosed and remedied.

So if small unit tests are good, how do you design your code to maximize the TDD goodness? By designing with TDD in the first place.

Using TDD as a Heuristic

From Merriam Webster -

Heuristic: "involving or serving as an aid to learning, discovery, or problem-solving by experimental and especially trial-and-error methods "

Simply put, TDD is a technique for determining class structure by making testability a first class consideration in your design. Focusing on testing a unit of code at a time leads to creating cohesive classes with a distinct purpose and responsibility. The need and desire to quickly setup an isolated unit test on a class will lead to a loosely coupled design. Here's a couple of quick questions answer to test whether your class structure really exhibits desirable architectural qualities:

Paraphrasing Michael Feathers, can someone fire up VS.NET on your code, take any arbitrary method, and write a unit test in a short amount of time?
Can you completely remove your computer from the network and run 100% of your unit tests covering both user interface and business logic code? Using a local database instance or a wireless connection does not count.
Can you write tests that only involve one or a very few classes?
If you are writing a web application (or a Web Service for that matter), can you run unit tests on much of your user interface code with IIS turned off?

Test Driven Development is an Alternative Design Process

There is far more to TDD than automated unit testing. Much like UML modeling or CRC cards, Test Driven Development is a process to explore a design and arrive at a good solution. The difference, in my mind, is that TDD is a "bottom up" process, where other design techniques are "top down." The truly good practitioners focus on rapidly building discrete, working pieces of code, then arranging the coded classes into an emerging structure guided by a knowledge of good design principles and a strong working understanding of Design Patterns.

Some of the impetus for using UML or CRC is the belief that it easier to explore design ideas by using abstract non-code artifacts because code itself is too difficult or inefficient to change once it is written. TDD can turn this assumption on its head by making code easier to change and restructure because the resulting code is cohesive, and safer to change because a good automated unit test suite can catch errors resulting from the changes. A good refactoring and code navigation tool like ReSharper (JetBrains rocks!) makes this type of malleable code assembly more efficient.

One way to think about TDD is an analogy to Lego blocks. The Lego sets I had as a child were the very basic block shapes. Using a lot of little Lego pieces, you can build almost anything your imagination can create. If you buy a fancy Lego set that has a single large piece shaped like a pirate's ship, all you can make is a variation of a pirate ship.

Over the next month or so, I'm going to blog a TDD Starter Kit of design concepts and strategies. Keep watching this space, the next posts will be shorter and actually contain code.

What's This Blog All About?

2005-03-10T08:33:00.000-06:00

When I was growing up in a farm community in Missouri, there was a "special" breed of folks around called Shade Tree Mechanics. Usually they were not the most reputable people in the world, but they had a knack for fixing mechanical problems and a reckless fearlessness to tinker with anything. A shade tree mechanic can be spotted by finding the pair of legs sticking out from under a beater car on blocks, surrounded by other skeletal vehicles crowding the rest of his scrubby, junk-laden yard. The beaters you see abandoned all around him aren't useless, they're fodder. He takes bits and pieces and tweaks and tunes, and comes up with a creative solution to your needs. Reputation notwithstanding, a shade tree mechanic knows how to get things running.

While I don't have any particular mechanical ability (despite a degree in mechanical engineering), I like to think that I am the developer equivalent of a shade tree mechanic . My hard drive is certainly littered with the detritus of abandoned open source projects (and one successful project here). I've had a short, but varied, path into and through the land of software development ...

Shadow IT - Getting my feet wet writing rogue applications for my engineering team. Having a blast cowboy-coding and generating a lot of fodder for the DailyWTF. Oracle, ASP, VB, and DHTML. I moved on to be a "real" developer.

Captive IT - Working in in-house IT for a Fortune 100 company designing enterprise applications. I actually got to be the technical lead and architect on a mission-critical system for my first real development project. No prior experience, no adult supervision, no problems. I described the architecture to a former colleague, and his only comment was, "You're going to spend some time in Purgatory for that one..." I also got to experience my first Death March Project, a lot of corporate politics, and secretive reorganizations. On the positive side, I learned about high availability, disaster recovery, and instrumentation, and I began to successfully apply OO design patterns and UML modeling. Our waterfall process changed often, and was enforced capriciously by a dizzying array of compliance bodies, laying in wait like 18th century highwaymen determined to ambush hapless travelers.

Non-Coding Architect - The last reorganization left me a non-coding architect just as the Microsoft world was dumping the DLL Hell (think VB/COM/ASP) technologies for .NET. I got to sharpen my Powerpoint skills while decorating my cubicle walls with Dilbert cartoons. The organization simply could not resist any type of evil fad -- CMM, Six Sigma, "It's not a process, it's a framework," fuzzy-headed SOA strategies, offshoring, "programmers are a commodity," "Who Moved My Cheese?", the "Fish" book, and a matrix model (yay for dotted-line bosses). I made a half-hearted, quixotic attempt to introduce iterative and incremental processes, but no one in management cared. In order to regain some type of technical relevancy and avoid 80-some hours of mandatory CMM training, I jumped again to become a consultant.

High Priests of Agile - I spent a mostly positive year and a half working in the .NET practice for an elite consulting company. Extreme Programming, Test Driven Development, Continuous Integration and .NET. Consulting is a great experience. The intellectual opportunities are abundant, and no matter how bad an assignment or client is, you can solace yourself that it's only a matter of time before you'll get to move on to a new venue. I had to learn software design all over again to incorporate TDD techniques. Besides too much StarBucks on the expense account, I got an absolute bellyful of XP theory and zealotry. If I hear, "In the white book, Kent Beck says ..." one more time, I'm going to puke. XP, and agile in general, works -- but software was written successfully before XP, and some of the older stuff (UML, Responsibility-Driven Design, Design Patterns) is more effective in certain situations. Besides, constant travel with an infant at home is a non-starter, so I left for a job close to home.

Independent Software Vendor - I am now working for a product-development company doing .NET development with a SCRUM process. I am finally getting to make a positive impact on an organization by helping introduce TDD, CI, and new architectural ideas. Being empowered in a pragmatic and stable organization to proactively make improvements has been a ray of light after a series of frustrating positions. A ten-minute commute via a scenic backroad doesn't hurt either.

I would like to use this blog to explore my thoughts about software design, .NET development, and agile methodologies. TDD and XP practices are getting a lot of attention, but I think there is a huge gap in writing between agile theory and successful practice. In particular, I would like to share my experiences with TDD and the role of design within an agile methodology.

My name is Jeremy Miller and I am a .Net Developer and Architect in Austin, TX.