The ESB Beginners Guide just Showed up on Amazon (Yes, THE Amazon!)

http://www.amazon.com/JBoss-ESB-Beginners-Guide-DiMaggio/dp/1849516588

It won't be long now...

Quarter Century Mark at Fotopedia!

Well, summer is about over. That's sad, but Fotopedia was nice enough to publish some of my late summer pictures:

Crawford Notch on Fotopedia

Provincetown Harbor on Fotopedia

In fact, I just hit a total of 25 articles as a top contributor!

(OK, back to software.....)

The JBoss ESB Book

The "little" project that has prevented me from having much time for blogging this summer is nearing completion.

The book will be out in November!

http://www.packtpub.com/jboss-esb-beginners-guide/book

New Pictures Voted Tops in Fotopedia

Just got some pictures of St Peter's church (no, the one in Bermuda, not Rome) voted up on Fotopedia. Very cool to see this!

St. Peter's Church, St. George's on Fotopedia

OK - the next post will be about software and testing...   

The Best Cross Posting of the Year!

One of the great things about blogging, is when your blog posts are thought of so highly by readers that they cross post them.  I was happily surprised this week to find that my post "Middleware in English Please" had been reposted. In Brazil and in Portuguese!

http://www.webbertek.com.br/site/interno.php?id_str=2&id_sub=17&id_ctns=44

Obrigado!

In praise of Micro-Reviews

The test plan was a work of art.

One hundred and twenty eight and three quarter pages of sheer brilliance.

It started with the IEEE standard and then added two bedrooms and an extra bathroom. Functional tests, stress tests, usability tests. Server backend and client frontend. Risks and mitigation strategies. Network diagrams and functional decompositions. Cross references to the functional specs, design specs, and even the team member's dietary requirements. Weeks of investigation and writing.

We proof-read it, revised it, formatted it, reformatted it, and sent it out to the entire project team for their review and feedback. And, because we knew that everyone already had a heavy workload, we allowed a full 48 hours for their review.

With as much pride in our workmanship as a ship builder, we sent it out for review and waited for the glowing reviews.

After 48 hours we received......nothing.

We decided that given the size and complexity of the plan that we would give people more time. So, we sent out an email to the entire project team and asked everyone to review the plan in 24 hours.

24 hours later we received......nothing.  

At this point, since the wholesale approach wasn't getting us anywhere, I decided to talk to one of the reviewers on a retail level. I tracked down Zyg, a senior member of the team and asked him why he hadn't reviewed the plan. His response was:

"Dude. I don't have time to read Charlie Sheen's tweets and you sent me a novel. It's like I ordered a glass of beer and you ran me over with a truck of Budweiser."

At that point, we still felt like shipbuilders, but of the Titanic.

Then, he offered an interesting deal. He said:

"Look, I'll try to help. I don't touch the GUIs, but I'll review the section that covers my server side code, so we can both get something out of it. I'll tell you where your tests suck, and maybe we can pre-empt you QE guys logging a hundred bugs where my code sucks. OK - how long is the section in that monster you wrote on my part of the code?"

So there it was. The answer was micro-reviewing.

Our goal was to collect technical information from people who were the subject matter experts for a variety of technical areas. So, why not let them focus their energies on those areas in which they were the experts? And, not burden them with having to read and review the entire work?

To be sure, this was not a perfect solution, as we still wanted input from people on the entire plan, that is, on the overall strategy and how the tests all fit together. But for that higher level view we could concentrate on the team leads and managers. And we could still divide and conquer, as we would now have review coverage for most of the discrete functional areas, thanks to our micro-review approach.

OK. That problem's solved. Now I own Zyg lunch...at the local microbrewery. 

(This is a somewhat fictionalized story - Zyg really does exist, but not in software development, dude.)

Software Metrics Bloom in the Spring

It's finally(!) spring in New England. Although, it is snowing today...not an April Fool's joke!



The flowers are opening and the ice rinks will soon be closing. Before the season completely changes, I thought it would be a good idea to revisit my friend the software manager/hockey dad.

When I asked him how things were going with his previously troubled QA team he said:

"Oh man, they're doing better and actually finding bugs, but now I have a new problem. They never stop finding bugs. Here's the deal - we planned out an 8 week project release schedule. So, we're in week 8, working on the final build before we want to ship, and they found 18 new bugs and almost all of them were in the same feature!"

At this point, I asked him, "So, what was the find/fix rate looking like?"

His response was a blank stare.  

I tried again, "Were you tracking the history of the bugs that were found in each component?"

Another blank stare.

So I told him, "Dude, you gotta get some bug tracking metrics. You need to start planning your future based on what happened in your past. You're treating your bug tracking system like a write-only database. Remember what Seinfeld's friend Newman said about the mail.  When you control it, you control information. You shouldn't just record the information, you need to start using it."

(Ok, I am fictionalizing things a bit. But, he does really talk like that, dude.)

Mining Bug Data - Gently Introducing Metrics

There's a joke in golf that goes something like this:

Q: What's the most important shot in golf?

A: The next one.

It can be the same way when you test software. Technology always changes quickly, but designs and code implementations can change more quickly. And the current state of the software under test can change even more quickly as tests expose code that is more at risk than your original plans had taken into account. It can be easy to fall into a trap of only looking forward. If, however, you make more proactive use of your bug tracking information, you can use that information to assist you both in making decisions for the present and plans for the future. What you have to do is to track some bug metrics and "mine" that information,

OK - how can you introduce metrics to a team with no experience in tracking or using them?

The short answer is "gently."

The longer answer is that you introduce a small number of metrics initially, ensure that tracking these metrics adds logos or no overhead to the team's already heavy workload, and and show the team an immediate benefit/payback from tracking the metrics.  

What are the best metrics to start with? That's easy, you can start with the metric that your bug tracking system already tracks for you.

Find Rate -vs- Fix Rate

Regardless of the software development model that you follow, you should see a pattern in the number of new bugs logged. The numbers will start slow when the QA team is more occupied with test development, rise when the tests are being debugged, peak when each test is being run for the first time, then decline sharply as test cycles continue and the only new bugs found are either in functional areas where testing had previously been blocked by other bugs or are bugs that are newly introduced when other bugs are resolved. The fix rate should follow the same general pattern, with a slight time lag.

A sign of trouble in the find rate for our hockey dad's product would have been that consistent find rate from week to week. The calendar might have told him that he was near the end of his planned schedule, but the find rate, as it was not decreasing, would have told him that the product was not complete.

What might have been the causes for this consistent find rate? Maybe the test development was not complete, so that each week new tests were being run for the first time. Or, maybe each new weekly build introduced new bugs. In order to have a clear picture of the state of the project, he would have to start tracking one more metric: the root cause of each bug.

Where is the Next Bug Coming From? 

I suggested that if it was the case that the QA team was consistently finding new bugs with old tests, that it was time for him to look beyond just the number of bugs that were being found, but to also look at the locations in the code where the bugs were being found.  

First, as to the location of the bugs, here's another software riddle;

Q:  Where are you likely to find the next 100 bugs in your product?

A: In the same places you found the last 100 bugs.

(As much as I'd like to take credit for this concept, I can't. It's from Glenford Myers in his groundbreaking book, The Art of Software Testing.)

It may be that the code in these locations is complex, which means that it is difficult to maintain so that every change runs the risk of introducing a new bug. Or, the code may have been changed and patched so many times that the original design integrity has been compromised. Or, it may be that the code is just plain buggy so that running any new tests finds new bugs.

Whatever the specifics, changes to code introduce risk as along with the code changes that are added, bugs can also be added. So, the places where you found your last bugs is often the places where you will find your next bugs.

OK - now that you have this information about where in the code you may find future bugs, what do you do with it? You can allocate more time and resources to building additional tests to give those functional areas more test coverage. And you can also ensure that any change made to the code in those areas, even a seemingly minor change is done carefully and with an eye to the likelihood of a minor bug fix inadvertently opening up a major new bug. In short, this information can give you a road map as to where in the code you should tread lightly.  

What's Next?

I didn't want to overload him with helpful suggestions all at once, but as he left, I reminded him that there is always uncertainty in software testing. But, if was able to leverage information from the past on his project, he might be able to better predict, or at least plan for the possible events in its future. And, some of the initial information that he needed to do this was already being recorded by his bug tracking system.