Parameterized Testing? But wait, there's more! (Inversion of Control/Dependency Injection, that is)

When you spend most of your waking hours (and, truth be told, some of you sleeping hours too) involved in software engineering, you can find yourself speaking a language foreign to friends and relatives. In my own case, most of my friends and family have no experience in or with software design, development, or testing. They tend to see my software engineering experience as useful, but also something out of the ordinary.

This past Monday, for example, a good friend of mine, a lawyer by profession, telephoned me late at night, desparate to rescue his home computer from the malware that he had inadvertently downloaded. I managed to walk him through the steps necessary to get his computer working again. That is, I thought I had, until a couple of nights later, he called me again asking my advice in buying a new computer.

A few days later, I was reading a printed copy of Martin Fowler's paper "Inversion of Control Containers and the Dependency Injection pattern" [1] when a friend of mine commented, "inversion of control?" Dude, that's the story of my life. My kids rule the house."

The term "dependency injection" can at first be bit more intimidating than the reality of what it is and how it's used. The simplest explanation that I've ever seen for it is in James Shore's excellent blog "The Art of Agile" [2] when he states that:

'...dependency injection means giving an object its instance variables..'

OK. This is all interesting, but what does it have to do with software testing? Quite a bit, actually. In the blog entry from a few weeks ago titled "Parameterized Testing - Very Easy with TestNG's DataProvider" we talked about designing tests that accept parameters, so that the same test could be run against multiple sets of data. In this post, we'll revisit that test design pattern.

But first, let's take a deeper look at inversion of control and dependency injection.

To begin, let's disect the phrase "inversion of control." In this context, who wants control, and just how is this control inverted?

In an object oriented language such as Java, you solve programming problems by designing classes. The classes contain variables (to hold data) and methods (functions that manipulate the data). To actually perform the tasks that you want the programs to do, you create discrete instances of the objects. When you create an object instance, you work with instances of the variables.

Let's say that you've defined a Java class of type Car, and within the class you set a Manufacturer variable to, say, "Audi."

What have we just done in terms of control and dependencies?

* Who has control? The "Car" class has the control. In order to build a new car, it defines a variable of type "Manufacturer." Or more precisely it defines an instance variable of type "Manufacturer." In this case, the class only allows for a Manufacturer of "Audi."

* The "Car" class needs the "Manufacturer" variable to do its work. In other words, it depends on this variable. That's right, this instance variable is a "dependency."

However, there's a problem with the "Car" class. How can build a test for the class, and test multiple car manufacturers? To build a test for Toyota, for example, we will have to build another class that's almost identical to the "Car" class, except that it will create a different "Manufacturer" object.

The answer is easy, of course, we just add another constructor to the Car class:

  public Car(Manufacturer targetManufacturer) {
    carManufacturer = targetManufacturer;
  } 

So, now what have we just done in terms of control and dependencies?

* The value of the dependency, that is, the "carManufacturer" variable is not hardwired into the "Car" class. It is "injected" into the "Car" class through its new constructor.

* Who has control now? Not the "Car" class anymore. The class can now be used to create any type of car. Who controls the decision of the type of car to create? The program (or test program) that creates the instance of the "Car" class. That's the "inversion" of control.

One important use of this model is that it emables us to isolate the class being tested. For example, if we want to test the Car class with a mock or stub[3] version of the Manufacturer class, we can pass that stub or mock object to the Car class during the test. This will let the test concentrate on verifying the operation of the Car class, independent of the function (or lack of function) provided by the Manufacturer class.

When you couple this design pattern with ability to pass a parameter into a test, such as is supported in TestNG, this enables us to inject the dependecies into the classes being tested. This makes it possible for the tests to control not only the test data, but also the defined profile and characteristics of the classes under test. And to do it without hardcoding data into the tests or the classes under test. Now, that's control!

References:

[1] http://martinfowler.com/articles/injection.html

[2] http://jamesshore.com/Blog/Dependency-Injection-Demystified.html

[3] Next Generation Java Testing by Cedric Beust and Hani Suleiman, pages 95-96. http://testng.org/doc/book.html

And not a drop to drink - Leak and Soak Tests

Owning an old house has some advantages. You can enjoy its period architectural details and perhaps find some hidden treasures. When we did some house renovations, we found newspapers from 1923 in one wall of the house. It was quite a surprise to be able to read contemporary sports coverage of Babe Ruth[1]! Owning an old house also means that you are always performing maintenance. I remember late one cold January night when I woke up to the sound of water dripping. It wasn't the bathroom, it was a vintage cast-iron radiator drippig water through the ceiling. Nothing had changed in the water pressure, but the washers in the radiator had just been worn down by prolonged use. Another time, a massive rainstorm resulted in a leak in the roof. The roof had performed well for years of less severe rain storms or equally severe, but shorter downpours, but the extended high level of rain simply soaked through it.

In the first case, the problem wasn't that the level of water traffic had increased beyond what the radiator "system" could manage, it was the accumulated damage of a relatively low level of traffic over time. In the second case, the problem was that the roof "system" was overwhelmed by prolonged exposure to a high level of traffic.

OK - what do all these expensive home repairs have to do with software testing? Just this; when you go about designing stress tests for your product or application, you should include "leak" and "soak" tests* in your plan.

When some people approach stress test planning, they treat the tests as a blunt instrument with which they try to assault the system under test. This type of scorched earth approach to testing can result in system failures being encountered, but these failures can be difficult to diagnose and reproduce. A combination of disciplined leak and soak tests can help you better identify the root causes of stress related system failures.

How do leak and soak tests differ? The differences can be thought of in the terms of the radiator and roof failures that I mentioned above.

In a leak test, you tend to run the system under a manageable, and tracable level of traffic for an extended period, to determine of the accumulated weight of usage causes a system failure. The classic case is when you are looking for a memory leak. The taffic load should not be extreme.

It may be that each system action results in a very small memory leak, or in the premanent allocation of some other system resource. If you run this test once, the leak may occur, but in the context of a system with several GB of memory, and a process that is using several MB of memory, you might not notice that the memory or system resource is not freed up when the test completes. But, if you repeat the test, and observe the process under test a tool such as JBoss Profiler[2], and observe the system with utilities such as top, sar, or vmstat, then you may be able to spot a trend of when system resources are used and not released. A great way to begin a leak testing regimen, is by observing the memory and system resource use of the software under test in an idle state. Just start up the server or application product under test and then leave it alone for an extended period. You may find that it's use of system resources increases over time, just through its self-maintenance, even when it is not actively processing user or client requests.

So, in a leak test, the key variable in the test equation is time.

In contrast, a soak test, hits the system under test with a significant test load, and maintains that load over an extended period of time. A leak test may involve running traffic through the system under test on only one thread, but a soak test may involve stressing the system to its maximum number of concurrent threads. This is where you may start to see inter-thread contention issues or problems with database connection pools being exhausted. If you can establish a reliable baseline of system operation with a leak test that runs over an extended period of time, then you can move onto more aggressive tests such as soak tests. If, however, a leak test exposes system failures, then you would probably encounter the same types of failures with soak tests. But, the level of traffic used in a soak might make these failures harder to diagnose.

So, in a soak test, the key variables in the test equation are both time and a sustained high load level.

To sum it up, if a leak test is passive aggressive in nature, a soak test is just plain aggressive.

* Dzięki Jarek!

References:

[1] http://en.wikipedia.org/wiki/Curse_of_the_Bambino
[2] JBoss Profiler - http://www.jboss.org/community/docs/DOC-10728