Untimely Assumptions to Look for in Test Design

Having no time to blog got me thinking about...time.

In my Sept. 2007 Red Hat Magazine article[1] about software about software communities, I referred to the need to deal with different timezones when your team and community are spread across multiple geographic locations. The folks with whom I'm working these days can attest to my being challenged when it comes to converting times between timezones. I always seem to ask them to attend meetings at inconvenient times for them! ;-)

Anyway, I received an email yesterday that got me thinking about time, and about assumptions that sometimes get made in system design about the availability of "quiet time." The email announced a lab was being powered down at 2:00 AM for some necessary maintenance. The time of day selected was a "quiet time" when the lab systems would be idle.

While I was reading the email, I had to think when 2:00 AM local time was in remote offices as we frequently share test servers in remote locations. As it turned out, the lab outage was not a serious problem based on the location and timezones of people would would be accessing the lab systems.

But, wow, that email caused me to have a sudden flashback to a software project on which I worked several years ago. The project was a voicemail call accounting system. Every day at 2:00AM the system ran several accounting tasks to calculate billing for that day. These billing tasks were large and complex and reduced overall system throughput while the tasks were running. The only problem with the system was that while 2:00 AM might be a quiet time for your own location, it might be a busy time for users accessing the system from elsewhere in the world, and those users would see poor performance when they accessed the system.

I don't think that we ever solved this problem, and I have long since left that company, but it illustrates an interesting problem in developing real-world stress tests. If the system under test includes faulty assumptions about traffic patterns, then unless you are able to identify the implications of these assumptions and attack them, then your testing may be flawed. The discrete tasks that the system has to perform may vary at different times of the day, or days of the week or month.

Let's wrap this up: what are some good faulty design assumption to look for in today's globally networked world?

The notion that the requirements that the system under test must fulfill are unchanging over time. Just as traffic patterns will change over a time period, the tasks performed by the system may vary from hour to hour.

Also, the notion that the system under test is the center of that world. These days, the system under test may be in Bonn, but it may have to simultaneously support people in Boston, Bogata, and Brno. And remember, 2:00 AM arrives at a different moment in time for everyone!

[1] http://www.redhatmagazine.com/2007/09/11/a-tale-of-three-communities/