Monday, May 23, 2005

Taxonomy of Tests: Defense in Depth

I made the mistake of posting on writing good unit tests while suffering from "Jedi Flu" and Jeffrey Palermo took me to task for not defining a unit test first, so here's my attempt at describing the various types and approaches of tests I've seen or used on agile projects. Keep in mind that I'm a developer first and only an occasional architect, “BuildMaster”, or tester second. The first thing to keep in mind is that agile processes don't change the fundamentals of software development. It is still best practice to test the code at multiple levels of integration (the old V-curve idea). It's great to mock the database in your unit tests, but the database is still there and just itching to get out of synch with your middle tier code.

A Taxonomy of Tests

Roughly, I would divide tests into:

  1. Unit tests
  2. Acceptance tests
  3. Intermediate/System/Integration tests, written by developers
  4. Other: smoke tests, regression tests, performance tests, miscellaneous validation

Testing Strategy

The answers will vary from project to project due to manpower constraints and the nature of the development, but any and every project is going to have to answer these questions below.

  • Who writes what tests?
  • Who automates the tests, and how?
  • Where and when do the tests run?
  • How do we want to organize the test suites?
  • How much testing is enough?

Sleeping with the Enemy

One thing I've learned that does not vary; it takes a great deal of cooperation and mutual respect between the testers and coders. Working in the same room is ideal. On projects where the tester and developers are separated by either organization or location, the development effort will suffer. For that matter, both developers and testers need quick and easy access to whoever holds the reins on requirements. At a bare minimum, don't look at testers as the enemy.

One of the truly great advantages agile development has over traditional waterfall projects is in team dynamics. There is far more interaction and communication between disciplines. Just being on the project at the same time is an advantage over “throw it over the wall” processes. Everybody’s common goal on an agile team is producing working software. Not code, tests, UML diagrams, or Gant charts, but working code that meets the customer’s expectations.

Next up, Unit Tests…


Post a Comment

<< Home