Monday, December 10, 2007

Why Are the Fountains Always Broken? (The forgotten cost of test automation: maintenance)

I'm writing this post from just outside of Boston. By American standards, Boston is an old city. One of the distinguishing features of the city is its collection of public parks. These parks are dotted with fountains which vary in ages and design from classical and historic to modern and avent-garde. These fountains, however, all share one common characteristic.

They always seem to be broken.

Why is this the case? It is certainly at least partially due to the ravages of harsh New England climate (As I'm writing this, the Boston area is having its second snow and ice storm of the young 2007-2008 winter) on outdoor plumbing. There is, however, I think another possible reason.

When we (we human beings, that is) build something, we often forget about the cost of maintaining it after it is built.

In the case of the fountains, the lack of maintenance may be caused by the fact that in any year's budget, other budget responsibilities such as health care, roads and bridges, and public safety are higher priorities. While it may be possible to attact funding for new construction of exciting or ground breaking public places, maintenance is, in contrast, considered a mundane exercise.

The same pattern can be seen in software enginering. For example, in the development and maintenance of automated software tests.

In the course of developing a plan for automating tests, we consider the investment in time and resources to design, build and debug the tests. Once written, however, the tests will lose their value unless they are maintained and kept in synch with the software project that they are intended to test. As the project procedes from release to release, new features are added, and new tests are needed. But, the existing library of tests may either begin to fail due their falling out of synch with the project code. Worse yet, the existing tests may provide a false sense of security as they may continue to run without error, but fail to actively exercise changed, and therefore, at risk, areas of code.

Automated tests are essential to any software test effort. The tests will pay back the investment you had to make to build them many times over during the life of a project. But, don't think that the tests are "done" just because you've completed building the first version of them. You will have to revisit and review and update the tests to keep them in working order. So, when you build new tests, don't forget to plan for maintenance. Part of this planning involves good test design, so that the tests can be modified as needed. Another part of this planning involves remembering that this maintenance will require time and human effort.

Why are the fountains broken? Maybe because no one planned to or was able to maintain them. Are your automated tests running correctly today? Even if it has been several months and project updates since they were first written or last reviewed? It might be a good time to look for leaks...

No comments: