Wednesday, March 30, 2005

TDD Design Starter Kit – Responsibilities, Cohesion, and Coupling

Recently, I started a new position with a company transitioning to TDD. One of the things I'm trying to accomplish in my new job is to help my colleagues become effective in TDD without most of the growing pains I experienced. So in an effort to help my new organization, I'm using this blog to communicate all the things I wish someone had shown me before I started writing code with Test Driven Development. So, in a couple of parts, here is all the off-the-cuff advice I can think to give to a TDD newbie to avoid seeing NUnit red bars in their sleep and getting the “Shame Card” permanently affixed to their monitor (for repeatedly breaking the build).
Separation of Concerns

Separation of Concerns is one of the most important concepts in programming, and the single most effective strategy for writing classes and methods that are easy to test. Simply put, divide your system into modules or layers, each with a distinct area of responsibility. We’ve known that layering is a good idea for a long time, but nothing exposes an intermingled ball of mud than trying to write automated tests. After you do TDD for awhile, you will probably notice that you slice classes thinner than before.

At the heart of every programming paradigm and design technique is a “Divide and Conquer” strategy to break a complicated whole into easily digestible pieces. Using OO programming, we are mostly concerned with defining the different responsibilities of the code, and assigning the responsibilities to the logical classes and methods.

There are a couple of qualities (more info here) to guide the division of labor in your class structure:
Cohesion – A measure of whether a class has a well defined, meaningful responsibility. High cohesion is desirable. If a class contains unrelated functions or responsibilities, it is not cohesive.
Coupling – More or less, how entangled or dependent a class is with other classes. A loosely coupled design implies that classes or subsystems can largely be modified independently of one another. The Pragmatic Programmers call this quality “orthogonality.”

TDD is all about rapidly making small pieces of code work, then molding the pieces into the aggregate. Cohesion is important to us, because we want to focus on making a single concern work at a time. Coupling is important because we want to make a concern work without interference from other concerns polluting our tests. Data access, business logic, user interface display, messaging, and logging are examples of separate concerns or responsibilities that should be built and unit tested separately. Divide and conquer.

Here’s a sample of code that flunks both cohesion and coupling tests.

public void PublishBigThings()
// Go get some configurion
int threshold = int.Parse(ConfigurationSettings.AppSettings["threshold"]);
string connectionString = ConfigurationSettings.AppSettings["connectionString"];

string sql = select * from things where ready_to_publish = true and size > ";
sql += threshold;

SqlConnection connection = new SqlConnection(connectionString);

SqlCommand command = new SqlCommand(sql, connection);
SqlDataReader reader = command.ExecuteReader();

while (reader.Read())
string name = reader["Name"].ToString();
string destination = reader["destination"].ToString();

if (this.IsDestinationApproved(destination))
Publish(name, destination);

The method is not cohesive because it is responsible for doing everything – pulling flags out of configuration, fetching data from a database, and making business logic decisions. The method is tightly coupled because it cannot function correctly without the proper configuration in place and a Sql Server database (other database engines need not apply).

“One Class, One Responsibility” is a good rule of thumb to follow in class construction. Don’t hesitate to create more classes. It can feel like more complexity, but it can also isolate complexities. See Fear of Adding Classes. It’s also much easier to make mistakes in big methods than in smaller methods.

More to come. Someday.

Tuesday, March 22, 2005

Succeed with TDD by designing with TDD

After 2 years of using Test Driven Development on .NET projects I'm a believer. Effective TDD leads to cleaner code, fewer bugs, and superior architectural qualities. Systems written with TDD are easier and safer to modify and deploy, leading to a reduced cost of ownership over the lifetime of an application. Yes, TDD means a lot more code to write. Yes, TDD can make the activity of writing code take longer, but it can and does shorten the time to delivery of working, production-quality code.

Using TDD dramatically changed the way I approach OO coding for the better, but I made a lot of mistakes along the way. The easiest way to learn TDD is to work closely with teammates that are already comfortable with TDD. Otherwise, you are on your own, and a lot of the writing I have seen on TDD falls into one of two categories -- "TDD is Cool" and "How to use NUnit." You've got to start somewhere, but when faced with the very real world of enterprise architecture (databases, messaging, user interfaces, Active Directory), the simple examples of create an object, pump in some inputs, and validate the results suddenly don't seem to be enough.

In my experience, newcomers to TDD often struggle with the mechanical work of writing unit tests, negating most of the value of TDD by writing unit tests that are too coarse. A common complaint is that the tests are too hard to write, or just take too much time to write. Frequently, well-meaning developers will "just make it work" first, and retrofit tests around the new code later. I did this pretty often on my first TDD project -- with the result largely being a set of fragile tests that flat out sucked and broke the build anytime someone breathed too hard on the codebase.

The usual culprit is the way that the code is written and structured -- god and blob classes, stovepipe architecture, poor separation of concerns, and an overuse of static methods. So how do we write code to maximize the efficacy of TDD? The most important thing is to write good OO code (I'm not trying to exclude folks doing procedural or functional coding, but this post is meant primarily for .NET developers). A couple years most of my team of System Architects (sic) were sent to an introductory(!) class on OOA/D and came back giggling and blathering about code being "Highly Cohesive" and "Loosely Coupled." All those fuzzy sounding OO qualities we've known about for years aren't just a way to impress other developers while stroking your beard, they are absolutely necessary to easily utilize TDD. It's time to take these concepts seriously in the .NET world. Read this from Scott Bellware on TDD in the Microsoft community versus the Java world. A bit more on cohesion and coupling later.

Which Comes First, Chicken or Egg?

A TDD developer follows a continuous cycle of writing a small, isolated unit test for a little bit of functionality, then writing a little code to make the test pass. Ideally, an individual unit test involves a discrete chunk of a single class (or a few classes). How do you know if your unit tests are small/isolated/discrete enough? Easy, VS.NET debugger sessions become uncommon and short and test failures are easily diagnosed and remedied.

So if small unit tests are good, how do you design your code to maximize the TDD goodness? By designing with TDD in the first place.

Using TDD as a Heuristic

From Merriam Webster -

Heuristic: "involving or serving as an aid to learning, discovery, or problem-solving by experimental and especially trial-and-error methods "

Simply put, TDD is a technique for determining class structure by making testability a first class consideration in your design. Focusing on testing a unit of code at a time leads to creating cohesive classes with a distinct purpose and responsibility. The need and desire to quickly setup an isolated unit test on a class will lead to a loosely coupled design. Here's a couple of quick questions answer to test whether your class structure really exhibits desirable architectural qualities:

  1. Paraphrasing Michael Feathers, can someone fire up VS.NET on your code, take any arbitrary method, and write a unit test in a short amount of time?
  2. Can you completely remove your computer from the network and run 100% of your unit tests covering both user interface and business logic code? Using a local database instance or a wireless connection does not count.
  3. Can you write tests that only involve one or a very few classes?
  4. If you are writing a web application (or a Web Service for that matter), can you run unit tests on much of your user interface code with IIS turned off?
Test Driven Development is an Alternative Design Process

There is far more to TDD than automated unit testing. Much like UML modeling or CRC cards, Test Driven Development is a process to explore a design and arrive at a good solution. The difference, in my mind, is that TDD is a "bottom up" process, where other design techniques are "top down." The truly good practitioners focus on rapidly building discrete, working pieces of code, then arranging the coded classes into an emerging structure guided by a knowledge of good design principles and a strong working understanding of Design Patterns.

Some of the impetus for using UML or CRC is the belief that it easier to explore design ideas by using abstract non-code artifacts because code itself is too difficult or inefficient to change once it is written. TDD can turn this assumption on its head by making code easier to change and restructure because the resulting code is cohesive, and safer to change because a good automated unit test suite can catch errors resulting from the changes. A good refactoring and code navigation tool like ReSharper (JetBrains rocks!) makes this type of malleable code assembly more efficient.

One way to think about TDD is an analogy to Lego blocks. The Lego sets I had as a child were the very basic block shapes. Using a lot of little Lego pieces, you can build almost anything your imagination can create. If you buy a fancy Lego set that has a single large piece shaped like a pirate's ship, all you can make is a variation of a pirate ship.

Over the next month or so, I'm going to blog a TDD Starter Kit of design concepts and strategies. Keep watching this space, the next posts will be shorter and actually contain code.

Thursday, March 10, 2005

What's This Blog All About?

When I was growing up in a farm community in Missouri, there was a "special" breed of folks around called Shade Tree Mechanics. Usually they were not the most reputable people in the world, but they had a knack for fixing mechanical problems and a reckless fearlessness to tinker with anything. A shade tree mechanic can be spotted by finding the pair of legs sticking out from under a beater car on blocks, surrounded by other skeletal vehicles crowding the rest of his scrubby, junk-laden yard. The beaters you see abandoned all around him aren't useless, they're fodder. He takes bits and pieces and tweaks and tunes, and comes up with a creative solution to your needs. Reputation notwithstanding, a shade tree mechanic knows how to get things running.

While I don't have any particular mechanical ability (despite a degree in mechanical engineering), I like to think that I am the developer equivalent of a shade tree mechanic . My hard drive is certainly littered with the detritus of abandoned open source projects (and one successful project here). I've had a short, but varied, path into and through the land of software development ...

  • Shadow IT - Getting my feet wet writing rogue applications for my engineering team. Having a blast cowboy-coding and generating a lot of fodder for the DailyWTF. Oracle, ASP, VB, and DHTML. I moved on to be a "real" developer.
  • Captive IT - Working in in-house IT for a Fortune 100 company designing enterprise applications. I actually got to be the technical lead and architect on a mission-critical system for my first real development project. No prior experience, no adult supervision, no problems. I described the architecture to a former colleague, and his only comment was, "You're going to spend some time in Purgatory for that one..." I also got to experience my first Death March Project, a lot of corporate politics, and secretive reorganizations. On the positive side, I learned about high availability, disaster recovery, and instrumentation, and I began to successfully apply OO design patterns and UML modeling. Our waterfall process changed often, and was enforced capriciously by a dizzying array of compliance bodies, laying in wait like 18th century highwaymen determined to ambush hapless travelers.
  • Non-Coding Architect - The last reorganization left me a non-coding architect just as the Microsoft world was dumping the DLL Hell (think VB/COM/ASP) technologies for .NET. I got to sharpen my Powerpoint skills while decorating my cubicle walls with Dilbert cartoons. The organization simply could not resist any type of evil fad -- CMM, Six Sigma, "It's not a process, it's a framework," fuzzy-headed SOA strategies, offshoring, "programmers are a commodity," "Who Moved My Cheese?", the "Fish" book, and a matrix model (yay for dotted-line bosses). I made a half-hearted, quixotic attempt to introduce iterative and incremental processes, but no one in management cared. In order to regain some type of technical relevancy and avoid 80-some hours of mandatory CMM training, I jumped again to become a consultant.
  • High Priests of Agile - I spent a mostly positive year and a half working in the .NET practice for an elite consulting company. Extreme Programming, Test Driven Development, Continuous Integration and .NET. Consulting is a great experience. The intellectual opportunities are abundant, and no matter how bad an assignment or client is, you can solace yourself that it's only a matter of time before you'll get to move on to a new venue. I had to learn software design all over again to incorporate TDD techniques. Besides too much StarBucks on the expense account, I got an absolute bellyful of XP theory and zealotry. If I hear, "In the white book, Kent Beck says ..." one more time, I'm going to puke. XP, and agile in general, works -- but software was written successfully before XP, and some of the older stuff (UML, Responsibility-Driven Design, Design Patterns) is more effective in certain situations. Besides, constant travel with an infant at home is a non-starter, so I left for a job close to home.
  • Independent Software Vendor - I am now working for a product-development company doing .NET development with a SCRUM process. I am finally getting to make a positive impact on an organization by helping introduce TDD, CI, and new architectural ideas. Being empowered in a pragmatic and stable organization to proactively make improvements has been a ray of light after a series of frustrating positions. A ten-minute commute via a scenic backroad doesn't hurt either.

I would like to use this blog to explore my thoughts about software design, .NET development, and agile methodologies. TDD and XP practices are getting a lot of attention, but I think there is a huge gap in writing between agile theory and successful practice. In particular, I would like to share my experiences with TDD and the role of design within an agile methodology.

My name is Jeremy Miller and I am a .Net Developer and Architect in Austin, TX.