Monday, November 29, 2010

It’s Always Cold in a Room Full of Ice

This story is true. Well mostly...)

Ice hockey McGill University 1901

One of the social benefits of being a hockey parent is being able to commiserate with other hockey  parents about the shared experiences of early mornings, long drives, and always too cold ice rinks. At a recent game - at 7:00 AM on a Saturday when the temperature inside the rink was a balmy 34F - I happened to ask a fellow hockey father about his software company. His situation, and my suggested actions for him form the basis of this post. I'm hoping that this post will help him and others.

(I'm guessing that anyone involved in software testing will have encountered one or more of the types of problems that he described - but hopefully not all at the same time! Note that all the names have been changed from the humans involved to the names of household pets.)

The Horror

When I asked Walter how his company was doing, his response sounded like a cross between a hi-tech project plan, James Joyce, and Conrad's "The Heart of Darkness."

He started by saying, "The horror, man, the horror of this situation is freaking insane. "

He then described how, through an acquisition by his company, he had inherited a QE team. Walter had several years of project management experience, and while he had worked with QE teams and QE managers, he had never before had direct control over a QE team. To call the QE team "troubled" was something of an understatement. In only his first few days being responsible for the team he quickly learned the following about the team:
  • The team was geographically dispersed. Walter was based in Boston and the rest of the team was spread out between Baltimore, Boise, Belarus, and Bangalore. (The team had been nick-named the "killer-B's.) 
  • Most of the team member had never met each other. The QE manager (her name was Celia) was based in Boise and had "issues" with either getting up early or staying up late to accommodate other team members' time zones. As a result, the team seldom had group meetings and the team manager rarely communicated over the telephone directly with individual team members in one-on-one conversations.
  • There was no mapping of tests and test coverage to product features.
  • A central/shared bug tracking system was used, but no organized bug triage reviews were ever held. Hundreds of old unresolved bugs or bugs that were likely no longer relevant were cluttering the bug database.
  • Many tests were automated, but there was no central/shared repository for all the tests. Individual team member often kept the automated tests in their personal home directories.
  • As the automated tests were written in multiple different languages, there was also no central/shared automated test framework. Some team member used Junit or similar frameworks, while others built their own "home grown" test frameworks.
  • Many of the automated tests were suspect due to a lack of maintenance. Individual engineers maintained personal lists of tests that were "known to fail."
  • The level of confidence in the QE team on the part of the other project teams was low. One product manager had recently asked Walter if testing was “strictly necessary, after all we do code reviews.”
  • Test plans that defined a test strategy or that provided traceability of product features and operational requirements to actual test coverage were non-existent. At the start of a project release, the team manager would send emails to team members instructing them as to which tests to develop. In the absence of these emails, individual team members often created whatever tests they felt were necessary. 
  • There was no central/shared repository of archived test results. In short, he knew that he and the team were in bad shape, but because of the general lack of "institutional memory" (e.g., test plans, test results) he didn't even know how bad off he was in any detail.
Yes, It Can Always Get Worse

Years ago, before they broke the "Curse of the Bambino" (http://en.wikipedia.org/wiki/Curse_of_the_Bambino), when I commented to a friend that the prospects for the Boston Red Sox could not get any worse, an older friend of mine commented that "things can always get worse." This can be true of software too. 

A few days after Walter first described his situation to me, he mentioned to me that, "Would you believe it? My QE manager just walked out!

At this point, I thought that it was worthwhile to try to give him some advice. Besides, I had been looking for a good topic for my software testing blog for weeks.  ;-)

The Wine is Bad, Throw it Out!

What I suggested to him was that he should look at the hasty exit of his QE manager as a positive development. The team had gotten itself into such a deep hole, that only some drastic changes could improve the situation. 

There’s a really great line from a really bad old movie, “The Agony and the Ecstasy.” In the film, Michaelanglo (played by Charlton Heston. And if that seems like strange casting, how about Rex Harrison as the Pope?) is disgusted by his first attempt to paint the ceiling of the Sistine Chapel. When he witnesses his local bartender throwing away wine that has gone bad, he is inspired to throw away his work and start again.

Walter’s initial reaction to his team’s problems was to try to patch things together. When I asked him if the team had ever been effective or even functional, he replied, “No way man. All I keep hearing is that the team has always been a disaster.” I suggested that instead of patching things together to back to where they were before the manager left, he would better off trying to address and resolve some of the team's problems. What ever they had been doing was simply not working. He was pressed for time, as his product release schedule was tight. But, I suggested that investing only a few days in restructuring things could pay some immediate results.

While he thought he was dealing with an infinite number of problems, I suggested that he classify the problems, and the solutions, into categories:
  • People
  • Tools and Assets
  • Processes
The People

My diagnosis for his team’s primary problem was: fragmentation. In fact, the team was suffering from two distinct forms of fragmentation. 

The first form was the fragmentation of their work effort. 
  • Regional Fragmentation - The teams’ tasks were divided by geographic region. The stress testing was performed by the Boise team, while the UI testing was performed in Bangalore, and the integration testing was performed in Baltimore. As a result, it was common for team members, when they were asked about specific tests, would just say, “oh, the other guys do that work - those tests are theirs.” This made test status information collecting difficult if the team in one location had gone home for the day. I suggested to Walter that the work be divided such that each location would have some people able to run any type of test that the team built. In other words, spread the work tasks across the timezones and divide the team by tasks, not by geography.
  • What’s Your Role in this Organization? What’s My Role? - In listening to Walter describe the manner in which the recently departed manager led the team, I imagined that the general lack of information, planning, and leadership would have caused many team members to be uncertain of their role within the team. I suggested that Walter not define their roles in terms of an org-chart, but rather, to define everyone’s role in terms of their dependencies and their deliverables. The functioning of a large, dispersed team can be thought of as being analogous to integrating software modules together. What you need is not just an org-chart, but also a dependency diagram. The inter-dependencies between the team members are in effect a social contract that binds the team together. 
The second form was the fragmentation of their work “community.” I think it was Robert Kennedy who lamented that divisions in society meant that “we share a city, but not a community.” In the case of Walter’s team, their geographic divisions made it difficult for the team to function as a team. Walter could not move everyone to the same physical location, but I did have some suggestions to get the team members working together better:
  • They Aren’t Remote, You Are - It’s only a matter of words, but I suggested that Walter stop referring to the team members that were not in his physical location as “remote.” The fact was, that purely in terms of numbers, that the larger concentrations of team members were in locations other than Walters. If anyone was “remote,” it was Walter. The goal of this change in terminology would be to help team members feel that they were full partners in the team, and not second class team members.
  • Is it Time for the Meeting? What Time is it Exactly? - The practice of having the same team members be inconvenienced by the time selected for team meetings had to stop. Walter had to set up a system where the times for the meetings would rotate so everyone would share the burden of getting up early, or staying up late. And, speaking of time, I suggested that Walter always refer to the time of day for meetings or other events in UTC time, and not in his local timezone. By using UTC time, everyone on the team could share a common “virtual time zone.”
  • Daily Contact - Question: How do you communicate with people? Answer: You talk to them. Having regular weekly team meetings would help the team communicate, but I suggested that it was important for Walter to establish, and then work hard to maintain, daily contacts, whether on the telephone, or via on-line chat. Email, I suggested, would not be a substitute for some informal, and at least daily contact.
  • What’s a “Day?” - Finally, I suggested to Walter that he re-think what a “day” is. The reality of his geographically dispersed team was that, at almost any given time of day, some members of his team would be working. There really was no “end of the day.” I suggested to Walter that he not look at this as a handicap, but as an advantage. The analogy that I used was that of a ship at sea. While the ship has to keep moving 24 hours a day, the sailors don’t stay awake for 24 hours. They work in shifts. His team could do the same thing if, as we discussed a few minutes ago, their work was partitioned so that people in different locations worked together on the same tasks. 
The Assets

When I asked Walter about the tests that the team created, he described them by saying, “Man, they have thousands of tests. But lots of them fail, others are disabled, and, nothing is described in any test plan documents. I have no idea if we have enough tests, too many, or no way near enough tests. And, what’s worse, no one can explain to me just what the tests actually do.

Walter then described how he wanted to review every test, and build up documentation on the operation of each test, so that he could get at least a general idea of the feature coverage provided by the tests. He also mentioned that the project did not have any detailed design specifications, or requirement definition documents. I suggested a slightly different approach:
  • Define the Product’s Test Needs First - I suggested to him, that before he started wading through all the tests, he first needed to create a functional definition of the product under test. Before he started to measure test coverage, he needed to create the “yardstick” against which he could measure the coverage. This “functional decomposition” of the product would define the functions performed by the product and the user requirements that the product was intended to fulfill. AND, I suggested strongly to him that in order to make this product functional decomposition useful, it would have to include the relative priorities of each product function, and the perceived risk of bugs being found in testing each feature. My reasoning was that since it is never possible to test the infinite number of possible function, configuration, sequence of user actions, etc., it’s important to concentrate your always finite test resources on those product functions that have the highest priority and are most at risk. 
In Walter’s case, since he was starting with a blank slate for a product test coverage definition, the place he had to start was with these high priority functions, configurations, integrations, use cases, etc. Once he had this definition in hand, he could then begin the review of the tests, and leverage whatever partial information individual team members had and map that information into a test coverage matrix. And then, he could expand on that definition to include features of lesser priorities until his test coverage matrix was complete.

This matrix would be the beginning of building his team’s “institutional memory.”  In a geographically dispersed team, where it would frequently be difficult to contact people in real-time, it was vital that every team member have access to persistent information on the tests’ goals and coverage and the test strategy, including the defined priorities for tests. The documents would have to be able to stand on their own.
  • Plan is Also a Verb - The next step would be the creation of test plans. When I asked Walter about the teams’ test plans, he told me, “Man, they are terrified of writing formal plans. All they do is tell me that they don’t have time to create huge documents.
My reply was, “Then, start by making the plans small. The goal of writing a test plan is not just a document. The act of writing the document forces people to think and review their analysis of the product’s risks.” I then suggested that Walter try a light-weight template for test plans. And, luckily, I had one handy - http://swqetesting.blogspot.com/2007/12/when-less-may-be-more-lighter-weight.html

I suggested to Walter that he introduce the team to the value of test plans by explaining to them that the plan’s real goal would be to get the team to be able to answer a series of questions. And that it was easier to ask themselves these questions before other people did! Each the answer to each question would take the form of a section of the plan. Some of these questions would be:
  • Introduction - what are we doing?
  • Test Strategy - how are we doing it
  • Test Priorities - what's most important?
  • Scope - What's being tested?
  • Scope - What's beyond the scope of testing?
  • Test Pass/Fail Criteria - how do we know that it's good or bad?
  • Test Deliverables - What are the docs and tests that we'll build?
  • Test Cases - What does each test do?
  • Responsibilities - who's doing what?
  • Schedule/Milestones - when are we doing it?
  • Risks and Contingencies - what might go wrong and how we'll handle it?
  • Approvals - do we agree?
  • References - pointers to background docs?
  • Revision history - why did the plan change and how?
The answers to these questions would be important to other teams on the project, to inform them of the test coverage planned, and to be used as a vehicle for them to provide input and suggestions and criticisms to the team. 

The Processes

Finally, we talked about the tests themselves and how they were run. I had two suggestions for Water:
  • Open Things Up - One of the problems that Walter’s team had was a lack of information sharing. Sometimes by design and sometimes by accident team members kept important information from other team members. I suggested that Walter start to run the team as if it were an open source software project. In his essay “The Cathedral and the Bazaar,”  (http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/), Eric Raymond defined Linus’ Law (named for Linus Torvalds) as "given enough eyeballs, all bugs are shallow." This is one of the great strengths of open source software; as the bugs are not hidden, they can be identified and resolved. For Walter’s team, the test plans, test results (logs, reports, etc.), and especially the tests themselves should be shared repository, where all the team members, and the people working on the Development, Management, Documentation teams, could access and review them. The reviews of the tests could be the beginning of the process of getting the tests to running on a single test framework. And, if more people on the team were aware of the automated tests’ design, then maybe someone other than the original author would be able to maintain them - AND - maybe start to clean up that backlog of old bugs too!  
  • You’re Not Finished - You’re Just Starting - Finally, I suggested to Walter, that even if he were able to improve the effectiveness of his team with any of my suggestions, that his work was only just beginning. What’s the most important process for a software test team to adopt? To me, it’s continuous improvement. (http://swqetesting.blogspot.com/2009/12/choosing-kaizen-over-cheez-whiz.html) Software will always include bugs, and some bugs will be missed in testing. You have to analyze your mistakes in an honest manner, and constantly refine your processes to incorporate corrections to past mistakes AND adapt your processes to meet new situations. But, in order to be able to do this, you first have to document your plans and your results so that you can review them at a later date. 
Closing Thoughts

Well Walter, I hope these ideas can help you dig your team out of that hole. In thinking about your situation, the word that kept coming to mind was “fragmentation.” Your team was fragmented, the tools and data stores they built and relied on were fragmented, and their working environment was fragmented. Whatever you can do to replace this fragmentation with a coordinated effort, where everyone understands their role within the team, should improve things quickly. How should you begin? Talk to your team, both as a group in a team meeting, and as individuals. Define their roles in terms of their dependencies and deliverables. Map the product features, and then map the tests to cover those features. Make all communications, and institutional memory shared and open. And, don’t fall behind by standing still. If think you see some improvement, then press for more. 

Oh, and remember to think in UTC time. Just not for the hockey games!  ;-)

Tuesday, October 5, 2010

From Panoramio to Google Earth

Hey - google earth just accepted some of my pictures - - including the orange dinosaur... ;-)

Thursday, September 16, 2010

Twitter!

Finally joined twitter:

   http://twitter.com/ldimaggi314

And there are even some followers already!   ;-)

Tuesday, August 17, 2010

Monday, July 19, 2010

Named a DZone "Most Valuable Blogger"

I was just named to the DZone "MVB" list: http://www.dzone.com/aboutmvb

DZone is a great site - this is a very nice surprise on a Monday morning!

Thursday, July 8, 2010

Cross Posted!

The previous post - on the JBoss SOA Platform's Rules-JBossESB integration - has been re-posted to the JBoss SOA Platform blog:

http://jboss-soa-p.blogspot.com/2010/07/jbossesb-drools-integration-in-jboss.html

Tuesday, June 22, 2010

The JBossESB-Drools Integration in the JBoss SOA Platform

As I've mentioned in previous posts and articles, one of the great strengths of the JBoss SOA Platform is the large number of integrations that it supports. Some of these integrations take the form of support for JBoss and third-party supplied JMS and UDDI providers or support for multiple JDKs and databases. The SOA Platform also supports an integration to jBPM for business process orchestration, and JBDS for application development.

In this post, I want to take a look at the SOA Platform's integration with JBoss Rules. In an earlier blog post (http://jboss-soa-p.blogspot.com/2009/07/when-content-knows-way-content-based.html), I discussed using Rules to implement content based routing over the JBossESB. In this post, we'll look at another aspect of the JBossESB-Rules integration; the creation of rules-based services.

Before we examine this integration in detail, let's take a quick look at JBoss Rules.

Drools and Rules


JBoss Rules as packaged in the SOA Platform is the commercialized version of the open source "Drools" project. (You can learn more about Drools at the project web site here: http://www.jboss.org/drools/) Drools is a unified and integrated solution for Business Rules, Business Processes Management, and Event Processing. The Drools project is organized into a number of sub-projects. When we refer to JBoss Rules in the SOA Platform, what we're primarily talking about is the Drools "Expert" sub-project. This sub-project consists of the Rules API, the Rules engine, and Rules editing and debugging tools.

Rules-based programming, as its name implies, is built on the ability to define decision points and keep them separate from other program logic. OK, that sounds interesting, but why would I want to use this? In other words, what's the big deal? Here are two reasons why this is important:
  • First, it enables you to separate your application's business logic and decision point handling from the application code. This means that your business process specialists can concentrate on the business logic rules and your programmers can concentrate on the application code. This makes application development and maintenance easier and more effective.
  • Second, and don't take this as a personal criticism of your programming skills or mine, but the since the rules engine is designed and optimized to process rules, it is more efficient than any massive if-then-else statement that you can write. So, your application's performance can be improved. The JBoss Rules engine makes use of the Rete algorithm http://en.wikipedia.org/wiki/Rete_algorithm for efficient Rules processing.
When you define a rule, the model that you follow is not "if-then-else," but rather "when" and "then." The two constructs that you use in a rule are:
  • The Condition - This is the left hand side of the rule and covers the "when" aspects of the rule.
  • The Consequence - This is the right hand side of the rule and covers the "then" aspects of the rule.
Rules are written in the Drools Rule Language (drl). This language can be extended into a Domain Specific Language (dsl) to support application-specific requirements such as medical or financial procedures and terminology.

The general template for a rule is:
1:  rule “a simple rule”
2: when (LHS)
3: you need a rules-based app
4: then (RHS)
5: build it with JBoss Rules
How does a rule access information? Through Rules' working memory. The information that rules perform operations take the form of Java beans that are referred to as "facts." (The elements in these facts are accessed through the getter and setter methods.) What happens is that facts are inserted into working memory, updated in working memory, or removed from working memory, those changes to the facts in working memory can cause the Rules' "when" conditions to be true and the rules to be executed.

It's important to note that unlike procedural programming, changes to facts can cause more than one rule to reach true conclusions and be available to be executed at once. What happens then? Well, instead of hardcoding a sequence of rules, the rules engine adds each matching rule to its "Agenda" of rules to be executed. If the Agenda includes more than one rule, the rule engine performs conflict resolution on the rules and determines the sequence in which the rules should be executed. This conflict resolution is based on the rules' salience (you define this as a property when you write the rules), how often the rule has fired in the past, complexity (the more complex a rule the more likely the rule engine will consider it to apply to the current situation), and the order in which the rules are loaded.

Invoking Rules from a JBossESB Service in the SOA Platform

The Rules - JBossESB integration in the SOA Platform enables you to access rules from an service's actions. This is supported by the org.jboss.soa.esb.actions.BusinessRuleProcessor and the org.jboss.soa.esb.actions.DroolsRuleService action classes.

The BusinessRuleProcessor class uses rules loaded from rules files. Generally, you use this class for simple rules services as loading large numbers of rules from large numbers of rules files is difficult to manage and not efficient.

For production environments, where you will have complicated rules services that deal with hundreds or even thousands of rules, it's better to use the DroolsRuleService. This service uses the RuleAgent to either access packages of rules from files, or from a Business Rules Management System (BRMS) .

The JBoss BRMS Platform (http://www.jboss.com/products/platforms/brms/) combines a central repository for rules, with a web based rules authoring interface, and rules management that provides import/export/archiving, audit trail or edits, error log, automated test development and execution, rules analysis, status tracking, and version control. The BRMS Platform rules authoring "guided editor" enables non-programmers to more easily create rules while the Platform makes it easy for rules administrartors can maintain large numbers of rules and control their development and use by users. Here's a screenshot:

The SOA Platform supports rules services that are either stateless or stateful. In the stateless model, messages sent to the services contain all the facts to be inserted into the rules engine working memory, before the rules are executed. In the stateful model, where the execution of the rules may take place in a session over an extended time period, several messages may be sent to a rule service, and the rules may fire and update either the message or the facts until a final message causes the service to end the session. The best way to explain and illustrate the Rules - JBossESB integration in the SOA Platform is with one of the Platform's "quickstart" example programs. Let's take a look.

Rules Services in Action - The Quickstart

I never get tired of saying that one of the great features of the SOA Platform is its extensive, and always growing, set of "quickstart" programs. These programs go far beyond being simple examples as they clearly illustrate various features supported by the Platform. They also serve as a great resource for writing your own applications. For our example, we'll look at the "business_rules_service" quickstart. Before we walk through the configuration and execution of the quickstart, let's take a look at its (3) Rules files. It important to note that Rules are actually used in multiple ways in this quickstart as it simulates a customer making a purchase from an e-commerce site. The quickstart uses Rules to:
  • Calculate the priority of an incoming customer order
  • Calculate the discount to be applied to an order
  • And route the order to the appropriate service, based on the content of the order
It's also important to note that the quickstart illustrates how a message can be modified as it is processed through the SOA Platform's JBossESB action pipeline. Let's start with the "MyBusinessRules.drl" file, as it establishes the priority of the order. (Oh, wait. We'll jump ahead a bit here and explain that by default, every incoming order has a priority value of "1.")
1:  package com.jboss.soa.esb.routing.cbr
2:
3: #list any import classes here.
4: import org.jboss.soa.esb.message.Message;
5: import org.jboss.soa.esb.message.format.MessageType;
6: import org.jboss.soa.esb.samples.quickstart.businessrules.dvdstore.OrderHeader;
7: import org.jboss.soa.esb.samples.quickstart.businessrules.dvdstore.Customer;
8:
9: global java.util.List destinations;
10:
11:
12:
13: rule "Logging"
14: when
15: order: OrderHeader()
16: customer: Customer()
17: then
18: System.out.println("Customer Status: " + customer.getStatus());
19: System.out.println("Order Total: " + order.getTotalAmount());
20: end
21:
22: rule "Customer Platinum Status"
23: when
24: customer: Customer(status > 50)
25: order: OrderHeader(totalAmount > 50)
26: then
27: System.out.println("Platinum Customer - High Priority");
28: order.setOrderPriority(3);
29: end
30:
31: rule "Customer Gold Status"
32: when
33: customer: Customer(status > 10, status <= 50)
34: order: OrderHeader(totalAmount > 25)
35: then
36: System.out.println("Gold Customer - Medium Priority ");
37: order.setOrderPriority(2);
38: end
39:
Let's examine this Rules file line-by-line:
  • Line 1 - Similar to Java, a package is a related set of rules.
  • Lines 4-7 - These imports perform the same function as Java language imports. Note that we're importing both SOA Platform Message related packages and packages contained in the quickstart itself.
  • Line 9 - Make a mental note of this global definition as it will be used by the other Rules files and by the quickstart for content based routing.
  • Line 13-14 - The name for the first Rule and the start of its "when" clause.
  • Line 15 - This creates a Rule variable named "order" and initializes with the value of the OrderHeader fact (org.jboss.soa.esb.samples.quickstart.businessrules.dvdstore.OrderHeader) that is presented to the Rule.
  • Line 16 does the same for a Rule variable named customer and the Customer fact. Note that there is nothing conditional about these assignments. The rule will match every OrderHeader and Customer. This matching is actually an important concept to keep in mind as part of each "when" clause involves trying to match the facts passed to the rules.
  • Line 20 - A rule must always have an end statement.
  • Line 22 - This rule sets the status for the highest priority ("Platinum") customers.
  • Line 24-25 - Note the differences in these statements to those in lines 15 and 16. The rule will only be executed if:
    • Line 24 - The customer's status is greater that 50. If this is not the case, then the customer variable we set in line 16 will not be changed, and
    • Line 25 - The total amount of the order is also greater than 50. If this is not the case, then the order variable that we set in line 15 will not be changed.
  • Line 28 - Remember how I said that the default priority for each order was set to "1". Here's where we set the priority. Note that we are setting the value in the "order" Rule variable.
  • Lines 31-38 - This rule sets the status for the 2nd highest priority ("Gold") customers. What's interesting in this rule is line 33 as it includes two criteria that must both be met in order for the fact to match the rule.
The next Rules file that we'll look at picks up where MyBusinessRules.drl leaves off. This Rules file is named: MyBusinessRulesDiscount.drl

1:  package com.jboss.soa.esb.routing.cbr
2:
3: #list any import classes here.
4: import org.jboss.soa.esb.message.Message;
5: import org.jboss.soa.esb.samples.quickstart.businessrules.dvdstore.OrderHeader;
6: import org.jboss.soa.esb.samples.quickstart.businessrules.dvdstore.Customer;
7:
8: global java.util.List destinations;
9:
10:
11:
12: rule "Logging"
13: salience 10
14: when
15: order: OrderHeader()
16: customer: Customer()
17: then
18: System.out.println("Customer Status: " + customer.getStatus());
19: System.out.println("Order Total: " + order.getTotalAmount());
20: end
21:
22: rule "Customer Platinum Status"
23: salience 20
24: when
25: customer: Customer(status > 50)
26: order: OrderHeader(orderPriority == 3)
27: then
28: System.out.println("Platinum Customer - High Priority - Higher discount");
29: order.setOrderDiscount(8.5);
30: end
31:
32: rule "Customer Gold Status"
33: salience 20
34: when
35: customer: Customer(status > 10, status <= 50)
36: order: OrderHeader(orderPriority == 2)
37: then
38: System.out.println("Gold Customer - Medium Priority - discount ");
39: order.setOrderDiscount(3.4);
40: end
  • Lines 1-8 and the "Logging" rule should look familiar, so let's move on to the other rules defined in this file.
  • Line 13 - Remember how we talked about how rules are not executed in the exact sequence in which they can be viewed in a rules .drl file? This rule is assigned a salience property value of 10. Since this rule only prints out some logging information, it's assigned a lower salience than the other rules defined in the file.
  • Line 22 - This rule sets the discount level for the highest class of customers.
  • Line 23 - And, since we want to be sure that this rule fires, we assign it a salience property value of 20.
  • Lines 25-26 - This "when" clause is true when the rule is able to match both a Customer fact with a status greater than 50 and an OrderHeader fact with an orderPriority equal to 3. If both these conditions are true, then rule the customer and order variables are initialized from the Customer and and OrderHeader facts and the rule is fired.
  • Line 29 - Note that when the rule fires, the setOrderDiscount setter method is executed on the rule variable "order." The same setter method is executed on the OrderHeader fact in working memory. (Remember how we talked about how rules can both react to changes to facts in working memory and also cause changes on those facts? This is an example.) For the "Platinum" class of customers, we assign a generous discount.
  • Lines 32-39 - This rule follows the same pattern as the "Customer Platinum Status" rule. Note that we give the "Gold" class of customers a somewhat less generous discount. ;-)
The third and final Rules file used by the quickstart controls the content-based routing used by the quickstart to route messages to services. In the SOA Platform, Rules is one of the supported mechanisms for implementing content based routing. In contrast to more static routing approaches, this form of message routing is relies on the content in the messages to dictate the route that a message takes. (For background on content based routing, please refer to this blog post: http://jboss-soa-p.blogspot.com/2009/07/when-content-knows-way-content-based.html )

This rules file is aptly named: MyRoutingRules.drl
1:  package com.jboss.soa.esb.routing.cbr
2:
3: #list any import classes here.
4: import org.jboss.soa.esb.message.Message;
5: import org.jboss.soa.esb.message.format.MessageType;
6: import org.jboss.soa.esb.samples.quickstart.businessrules.dvdstore.OrderHeader;
7:
8: #declare any global variables here
9: global java.util.List destinations;
10:
11:
12: rule "Highest Priority Orders"
13:
14: when
15: OrderHeader( orderPriority == 3 )
16: then
17: System.out.println("HIGHEST PRIORITY");
18: destinations.add("SuperSpecialCustomerService");
19:
20: end
21:
22: rule "Medium Priority Orders"
23:
24: when
25: OrderHeader( orderPriority == 2 )
26: then
27: System.out.println("Medium Priority");
28: destinations.add("SpecialCustomerService");
29: end
30:
31: rule "Low Priority Orders"
32:
33: when
34: OrderHeader( orderPriority == 1 )
35: then
36: System.out.println("Low Priority");
37: destinations.add("RegularCustomerService");
38: end
39:
  • Lines 1-6 - Once again, these import lines should look familiar.
  • Line 9 - Make note of the destinations List. We'll see this used when the messages are routed.
  • The rules in this rules file are pretty straight-forward. In each of the three rules we add the appropriate destination, based on the orderPriority in the OrderHeader, where that destination is a service that the quickstart deploys to the JBossESB in the SOA Platform.
OK, those are the rules that we'll use in the quickstart. Let's now step through the quickstart as it is run.

To deploy the quickstart, execute this ant target:
1:  ant deploy
And, we then see this written to the server log:
1:  22:20:57,568 INFO [QueueService] Queue[/queue/quickstart_Business_Rules_Request_GW] started, fullSize=200000, pageSize=2000, downCacheSize=2000
2: 22:20:57,580 INFO [QueueService] Queue[/queue/quickstart_Business_Rules_Request_ESB] started, fullSize=200000, pageSize=2000, downCacheSize=2000
3: 22:20:57,621 INFO [QueueService] Queue[/queue/quickstart_Business_Rules_ConciergeManager] started, fullSize=200000, pageSize=2000, downCacheSize=2000
4: 22:20:57,632 INFO [QueueService] Queue[/queue/quickstart_Business_Rules_DistributionManager] started, fullSize=200000, pageSize=2000, downCacheSize=2000
5: 22:20:57,643 INFO [QueueService] Queue[/queue/quickstart_Business_Rules_BasicShipping] started, fullSize=200000, pageSize=2000, downCacheSize=2000
6: 22:20:57,682 INFO [EsbDeployment] Starting ESB Deployment 'Quickstart_business_rules_service.esb'
And to run it, exceute this ant target:
1:  ant runtest
When the quickstart is run, here's what happens, step by step. Note that while we'll be examining most of the contents of the jboss-esb.xml file in detail, we'll be doing it in pieces or fragments so that it's easier to follow. The line numbers in each of these fragments will therefore be different from the actual (whole) file.

Step 1 - Create a Message and Pass it Through a Gateway to the Deployed Quickstart Application

Like many of the SOA Platform quickstarts, the business_rules_service quickstart initiates its actions when a message is inserted into a queue that is being watched by a gateway listener. What's a gateway? On the JBossESB in the SPA Platform, everything is either a service that generates or consumes messages, or a message. That is, a message that is in the form (org.jboss.soa.esb.message) that the ESB understands. Services that can understand messages in this form are referred to as being "ESB-aware."

How can you connect other, and potentially older, legacy applications over the ESB? By using gateways. A gateway (org.jboss.soa.esb.listeners.gateway) is a service that acts as a bridge between an ESB-aware and an ESB-unaware client and service. Gateways translate information between ESB and non-ESB message formats and EPRs. (EPR stands for endpoint reference.) Gateways are listener processes in that they "listen" for incoming communications. They are different from ESB-aware listeners in that they accept data in different formats such as objects in files or SQL tables. ESB-aware listeners can only accept messages in the org.jboss.soa.esb.message format.

The SOA Platform supports these gateways:
  • file gateways: local filesystem, ftp, sftp and ftps
  • JMS
  • HTTP/HTTPS
  • email (POP3)
  • SQL table
  • Hibernate
In the case of this quickstart, we'll use a JMS gateway to receive the JMS message. The gateway queue, and its corresponding ESB-aware queue are defined in the jms-provider section of the jboss-esb.xml file:
1:  <jms-bus busid="quickstartGwChannel">
2: <jms-message-filter dest-type="QUEUE"
3: dest-name="queue/quickstart_Business_Rules_Request_GW" />
4: </jms-bus>
5: <jms-bus busid="quickstartEsbChannel">
6: <jms-message-filter dest-type="QUEUE"
7: dest-name="queue/quickstart_Business_Rules_Request_ESB" />
8: </jms-bus>
And the listener is defined at the top of the "Business_Rules_Service" service definition:
1:   <service category="Business_RulesServices"
2: name="Business_Rules_Service" description="The main entry point">
3: <listeners>
4: <!-- Gateway -->
5: <jms-listener name="TheGateway"
6: busidref="quickstartGwChannel" is-gateway="true" />
7: <jms-listener name="TheESBChannel"
8: busidref="quickstartEsbChannel" >
9: </jms-listener>
10: </listeners>
11: <actions mep="OneWay">
  • Note that on line 6, we identify the listener as a gateway.
  • Also note that on line 11, we define the mep, or "message exchange pattern." In the case of this quickstart, the pattern is "OneWay" which indicates that the message pattern is asynchronous. We're sending messages, but not waiting around (or blocking) for a response.
How do we generate this message? Take a look at the "runtest" target, specifically the classname, in the quickstart's ant build.xml file:
1:  <target name="runtest" depends="compile"
2: description="willl receive JMS message to tigger the actions in the ESB">
3: <echo>Runs Test JMS Sender</echo>
4: <java fork="yes" classname="org.jboss.soa.esb.samples.quickstart.businessrules.test.SendJMSMessage" failonerror="true">
5: <classpath refid="exec-classpath" />
6: </java>
That's right - we're sending a JMS message with a program named "SendJMSMessage." It's hard to get simpler than that. ;-)

In order to simulate a realistic customer order in the message, SendJMSMessage builds the message from the quickstart's SampleOrder.xml file:
1:  <Order orderId="1" orderDate="Wed Nov 15 13:45:28 EST 2006" statusCode="0"
2: netAmount="59.97" totalAmount="64.92" tax="4.95">
3: <Customer userName="user1" firstName="Harry" lastName="Fletcher" state="SD"/>
4: <OrderLines>
5: <OrderLine position="1" quantity="1">
6: <Product productId="364" title="The 40-Year-Old Virgin " price="29.98"/>
7: </OrderLine>
8: <OrderLine position="2" quantity="1">
9: <Product productId="299" title="Pulp Fiction" price="29.99"/>
10: </OrderLine>
11: </OrderLines>
12: </Order>
The SendJMSMessage class reads this file, creates a message object of type javax.jms.ObjectMessage (remember, this is an ESB-unaware message), and writes it to the queue (queue/quickstart_Business_Rules_Request_GW) on which our JMS gateway is listening. The listener receives the message and the ESB converts it to an ESB-aware message and then passes it onto the ESB through the queue/quickstart_Business_Rules_Request_ESB queue.

Now it starts to get more interesting.

Remember that our rules rely on certain types of facts (which are JavaBeans) being available in working memory. Where do these facts come from? We'll create them out of that message.

Step 2 - Transform the Message into Beans

The problem is that we have to have a way to create those facts out of the information in the message. Luckily, one of the tasks that the JBossESB in the SOA Platform performs is "transformation." The next action in the quickstart's action pipeline uses the smooks processing engine (http://www.smooks.org/) and the JBossESB's out-of-the-box "SmooksAction" action to perform the transformation of the information in the message into facts:
1:  <action name="transform"
2: class="org.jboss.soa.esb.smooks.SmooksAction">
3: <property name="smooksConfig" value="/smooks-res.xml" />
4: <property name="resultType" value="JAVA" />
5: </action>
The source code for the facts (remember that these are JavaBeans and have getter and setter methods) is in these source files in the quickstart:

In: src/org/jboss/soa/esb/samples/quickstart/businessrules/dvdstore
  • Customer.java
  • OrderHeader.java
  • OrderItem.java
The quickstart uses the smooks-res.xml file to perform the transformation. Let's take a look at this file.
1:  <?xml version='1.0' encoding='UTF-8'?>
2: <smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
3: xmlns:jb="http://www.milyn.org/xsd/smooks/javabean-1.2.xsd">
4:
5: <!-- Populate the OrderHeader -->
6: <jb:bean beanId="orderHeader" class="org.jboss.soa.esb.samples.quickstart.businessrules.dvdstore.OrderHeader" createOnElement="order">
7: <jb:value property="orderId" data="Order/@orderId" />
8: <jb:value property="orderDate" data="Order/@orderDate" decoder="Calendar">
9: <jb:decodeParam name="format">EEE MMM dd HH:mm:ss z yyyy</jb:decodeParam>
10: <jb:decodeParam name="locale-language">en</jb:decodeParam>
11: <jb:decodeParam name="locale-country">US</jb:decodeParam>
12: </jb:value>
13: <jb:value property="statusCode" data="Order/@statusCode" />
14: <jb:value property="netAmount" data="Order/@netAmount" />
15: <jb:value property="totalAmount" data="Order/@totalAmount" />
16: <jb:value property="tax" data="Order/@tax" />
17: </jb:bean>
18:
19: <!-- Populate the Customer -->
20: <jb:bean beanId="customer" class="org.jboss.soa.esb.samples.quickstart.businessrules.dvdstore.Customer" createOnElement="customer">
21: <jb:value property="userName" data="customer/@userName" />
22: <jb:value property="firstName" data="customer/@firstName" />
23: <jb:value property="lastName" data="customer/@lastName" />
24: <jb:value property="state" data="customer/@state" />
25: </jb:bean>
26:
27: <!-- Populate the OrderItem list -->
28: <jb:bean beanId="orderItemList" class="java.util.ArrayList" createOnElement="orderlines">
29: <jb:wiring beanIdRef="orderItem" />
30: </jb:bean>
31:
32: <!-- Populate the OrderItem instance -->
33: <jb:bean beanId="orderItem" class="org.jboss.soa.esb.samples.quickstart.businessrules.dvdstore.OrderItem" createOnElement="orderlines/orderline">
34: <jb:value property="position" data="orderline/@position" />
35: <jb:value property="quantity" data="orderline/@quantity" />
36: <jb:value property="productId" data="orderline/product/@productId" />
37: <jb:value property="title" data="orderline/product/@title" />
38: <jb:value property="price" data="orderline/product/@price" />
39: </jb:bean>
40:
41: </smooks-resource-list>
That's right - it's using XPath (http://www.w3.org/TR/xpath/) to parse the information in the message into Customer (starting at line NN), OrderHeader (starting at line NN), and OrderItem.java (starting at line NN) JavaBeans and a java.util.ArrayList object that contains a list of the orders.

Step 3 - And Add the Beans Back into the Message

What happens next? Well, we have the original message, and some JavaBeans. But, remember that on the JBossESB in the SOA Platform, everything is either a message or a service. What we need is some way to get those JavaBeans back into the message. The way that we do this is with the next action in the action pipeline:
1:  <action name="map_order_components" class="org.jboss.soa.esb.actions.scripting.GroovyActionProcessor">
2: <property name="script" value="/map_order_components.groovy" />
3: </action>
GroovyActionProcessor is another of the SOA Platform's out-of-the-box actions. The groovy script referenced by the action takes the JavaBeans that we just created and adds them (note the use of the beanId's that we defined in the smooks-res.xml file) back into the message with property names that match the bean IDs. The script is very short - lines 3 and 4 add the orderHeard and customer to the message body:
1:    // Need to map down the orderHeader and customer beans onto the message
2: // to make them available to the ObjectMapper...
3: message.getBody().add("orderHeader", message.getBody().get().get("orderHeader"));
4: message.getBody().add("customer", message.getBody().get().get("customer"));
Now we have the JavaBeans in the message. What happens next? The quickstart updates the customer status (it's set to a value of "0" in the JMS message that started the quickstart) with the "UpdateCustomerStatus" custom action:
1:  <!-- Update Customer Status -->
2: <action name="updateCustomerStatus"
3: class="org.jboss.soa.esb.samples.quickstart.businessrules.UpdateCustomerStatus">
4: <property name="status" value="60"/>
5: </action>
We'll set this to a value of 60 as the customer is in the platinum customer class. (Hint: For extra credit, try the quickstart with different status values.)

Here's the output in the log - this line is printed by the org.jboss.soa.esb.samples.quickstart.businessrules.UpdateCustomerStatus custom action:
1:  21:42:15,793 INFO [STDOUT] { Updated customer status to 60}
Step 4 - Process the Message with the BusinessRulesProcessor

OK - now we can see the BusinessRulesProcessor execute in the next action:
1:    <!-- Use the BRP to calculate the order priority -->
2: <action
3: class="org.jboss.soa.esb.actions.BusinessRulesProcessor"
4: name="BRP">
5: <property name="ruleSet"
6: value="MyBusinessRules.drl" />
7: <property name="ruleReload" value="true" />
8: <property name="object-paths">
9: <object-path esb="body.orderHeader" />
10: <object-path esb="body.customer" />
11: </property>
12: </action>
13:
14: <action name="reviewMessage1"
15: class="org.jboss.soa.esb.samples.quickstart.businessrules.ReviewMessage">
16: <property name="stuff" value="After Order Priority"/>
17: </action>
  • Line 1 - Comments are always good things! ;-)
  • Line 2 - Here's the start of the action that makes use of the BusinessRulesProcessor.
  • Line 3 - And here's the reference to the BusinessRulesProcessor class.
  • Lines 5-6 - And, here's the reference to the Rules file that we want to execute.
  • Line 7 - This property causes the Rule to be if the file changes.
  • Lines 8-11 - And, here are the objects that we added to the message - remember the map_order_components.groovy file?
So, what just happened? We made the orderHeader and customer objects available to the Rules defined in MyBusinessRules.drl and executed the Rules. The net effect of this is that the priority of the order defined in the message should have changed. The next action in the action pipeline writes this to the log:
1:  21:42:15,931 INFO [STDOUT] Platinum Customer - High Priority
2: 21:42:15,932 INFO [STDOUT] Customer Status: 60
3: 21:42:15,932 INFO [STDOUT] Order Total: 64.92
The first three lines are printed by the rules. Remember how the rules all included System.out.prinln statements? And how the logging rule that prints the customer status and order total has a lower value salience property than the rules that set and print the customer priority? That explains the order in which the statements are printed to the log.

And then these lines are printed by the org.jboss.soa.esb.samples.quickstart.businessrules.ReviewMessage cusom action:
1:  21:42:15,932 INFO [STDOUT] { ================ After Order Priority
2: 21:42:15,933 INFO [STDOUT] Customer: user1,Harry,Fletcher,SD,60
3: 21:42:15,933 INFO [STDOUT] Order Priority: 3
4: 21:42:15,933 INFO [STDOUT] Order Discount: 0.0
5: 21:42:15,933 INFO [STDOUT] Order Total: 64.92
6: 21:42:15,933 INFO [STDOUT] } ================ After Order Priority
The rule "Highest Priority Orders" defined in MyBusinessRules.drl has set the order priority to 3.

Next, the quickstart calls the BusinessRulesProcessor again, this time to determine the order discount:
1:   <!-- Use the BRP to calculate the order discount -->
2: <action
3: class="org.jboss.soa.esb.actions.BusinessRulesProcessor"
4: name="BRP2">
5: <property name="ruleSet"
6: value="MyBusinessRulesDiscount.drl" />
7: <property name="ruleReload" value="true" />
8: <property name="object-paths">
9: <object-path esb="body.orderHeader" />
10: <object-path esb="body.customer" />
11: </property>
12: </action>
13:
14: <action name="reviewMessage2"
15: class="org.jboss.soa.esb.samples.quickstart.businessrules.ReviewMessage">
16: <property name="stuff" value="After Order Discount"/>
17: </action>
  • Line 6 - The rule "Customer Platinum Status" defined in the MyBusinessRulesDiscount.drl file is executed as our customer has both the highest priority as well as the status value of 60. That rule and the logging rule print out these statements to the log in this order:
1:  21:42:16,062 INFO [STDOUT] Platinum Customer - High Priority - Higher discount
2: 21:42:16,063 INFO [STDOUT] Customer Status: 60
3: 21:42:16,063 INFO [STDOUT] Order Total: 64.92
Next, the org.jboss.soa.esb.samples.quickstart.businessrules.ReviewMessage custom action is executed to print these statements to the log:
1:  21:42:16,063 INFO [STDOUT] { ================ After Order Discount
2: 21:42:16,063 INFO [STDOUT] Customer: user1,Harry,Fletcher,SD,60
3: 21:42:16,064 INFO [STDOUT] Order Priority: 3
4: 21:42:16,064 INFO [STDOUT] Order Discount: 8.5
5: 21:42:16,064 INFO [STDOUT] Order Total: 64.92
6: 21:42:16,064 INFO [STDOUT] } ================ After Order Discount
And there we see our generous 8.5% discount.

Next, the quickstart prints the entire message to the log with this out of the box action:
1:  <action name="sout" class="org.jboss.soa.esb.actions.SystemPrintln" />
And here's how it appears in the server.log:
1:  21:42:16,064 INFO [STDOUT] Message structure:
2: 21:42:16,064 INFO [STDOUT] [{orderHeader=1, java.util.GregorianCalendar[time=1163616328000,areFieldsSet=true,areAllFieldsSet=false,lenient=true,zone=sun.util.calendar.ZoneInfo[id="US/Eastern",offset=-18000000,dstSavings=3600000,useDaylight=true,transitions=235,lastRule=java.util.SimpleTimeZone[id=US/Eastern,offset=-18000000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]],firstDayOfWeek=1,minimalDaysInFirstWeek=1,ERA=?,YEAR=2006,MONTH=10,WEEK_OF_YEAR=?,WEEK_OF_MONTH=?,DAY_OF_MONTH=15,DAY_OF_YEAR=?,DAY_OF_WEEK=4,DAY_OF_WEEK_IN_MONTH=?,AM_PM=?,HOUR=?,HOUR_OF_DAY=13,MINUTE=45,SECOND=28,MILLISECOND=?,ZONE_OFFSET=-18000000,DST_OFFSET=0], 0, 59.97, 64.92, 4.95, , orderItemList=[1,1,364,The 40-Year-Old Virgin ,29.98, 2,1,299,Pulp Fiction,29.99], orderItem=2,1,299,Pulp Fiction,29.99, customer=user1,Harry,Fletcher,SD,60}].
That's a little hard to read. Let's take out the order date/time so that we can better see the customer and orderHeader Beans that we added to the message:
1:  {
2: orderHeader=1, 0, 59.97, 64.92, 4.95,
3: orderItemList=[1,1,364,The 40-Year-Old Virgin ,29.98, 2,1,299,Pulp Fiction,29.99],
4: orderItem=2,1,299,Pulp Fiction,29.99,
5: customer=user1,Harry,Fletcher,SD,60
6: }
Step 5 - Route the Message to its Destination Based on its Content

At this point, the quickstart has finished using the BusinessRulesProcessor, but it's not yet done using Rules. Remember how I said that on the JBossESB in the SOA Platform everything is either a message or a service? Well, one of the main functions performed by the ESB is to route messages to the correct service. These routes can be static, or they can be dynamic, based on the content of a message. Here's where content based routing with Rules comes in. The routing rules are defined in the MyRoutingRules.drl file. Remember how the rules in this file designated the "destinations" of the messages? This next action uses invokes the org.jboss.soa.esb.actions.ContentBasedRouter class to route the messages to their intended destinations.
1:  1  <!-- Use the CBR to route the "scored" order to the appropriate service team -->
2: 2 <action
3: 3 class="org.jboss.soa.esb.actions.ContentBasedRouter"
4: 4 name="ContentBasedRouter">
5: 5 <property name="ruleSet" value="MyRoutingRules.drl" />
6: 6 <property name="ruleReload" value="true" />
7: 7 <property name="destinations">
8: 8 <route-to
9: 9 destination-name="SuperSpecialCustomerService"
10: 10 service-category="ConciergeManager" service-name="ConciergeService" />
11: 11 <route-to
12: 12 destination-name="SpecialCustomerService"
13: 13 service-category="DistributionManager" service-name="DistributionService" />
14: 14 <route-to
15: 15 destination-name="RegularCustomerService"
16: 16 service-category="BasicShipping" service-name="ShipperService" />
17: 17 </property>
18: 18 <property name="object-paths">
19: 19 <object-path esb="body.orderHeader" />
20: 20 <object-path esb="body.customer" />
21: 21 </property>
22: 22 </action>
  • Lines 107-109, 110-112, and 113-115 define the routes. Since our message now has the highest priority, it is routed to the ConciergeService.
Here's what the log shows us. Again, the first line is printed by the rule:
1:  21:42:16,210 INFO [STDOUT] HIGHEST PRIORITY
And the remaining lines are printed by the org.jboss.soa.esb.samples.quickstart.businessrules.ReviewMessage custom action as invoked by the ConciergeService.
1:  21:42:16,309 INFO [STDOUT] { ================ Concierge
2: 21:42:16,309 INFO [STDOUT] Customer: user1,Harry,Fletcher,SD,60
3: 21:42:16,309 INFO [STDOUT] Order Priority: 3
4: 21:42:16,309 INFO [STDOUT] Order Discount: 8.5
5: 21:42:16,310 INFO [STDOUT] Order Total: 64.92
6: 21:42:16,310 INFO [STDOUT] } ================ Concierge
Well, the customer's order was delivered to the very posh ConciergeService, and the quickstart's execution is complete.

Closing Thoughts

OK, let's review what happened. The quickstart defines multiple business rules to examine and modify a message as it is processed by actions executed by ESB services, then it routes that message to the correct destination service. The rules are maintained in .drl files, separate from the services' custom action code, which makes it easier to maintain them.

And, notice what the quickstart did not have to do - the rules were executed and the content based routing was performed through out of the box actions provided by the SOA Platform's JBossESB. This made it possible for the quickstart to concentrate on the Rules business logic.

Acknowledgements

As always, I want to thank the JBoss SOA Platform team and community (especially Kevin Conner and Mark Proctor for their timely review input for this blog post).


References

Monday, June 14, 2010

The Good, the Bad, and the Vague

That last post got me thinking in concrete terms about...ambiguity.

Most of us experience being on the receiving end of ambiguity at an early age. It happens on our birthdays when we ask our parents for a dirt bike or air rifle. Some parents just say "no," but, the truly creative parents make use of the power of ambiguity and say "we'll see." Your parents are practicing intentional ambiguity.

Unintentional ambiguity, however, can be just as powerful, and, in engineering disciplines, can have serious consequences. When you're dealing with a physical medium such as glass or steel, you need to have an unambiguous understanding of the medium's limitations. For example, when you're building a bridge, you need to know exactly how much stress the steel handle. A design specification that states that the steel "might be at least as strong as this type of bridge generally requires" could lead to problems like this:



In the previous post to this blog, I discussed a couple of instances where unclear communications in source code and test reports had unpleasant side effects. How can you avoid situations like this, or problems caused by ambiguous product specification, requirement, or design documents? By including ambiguity reviews into your technical reviews.

I'd like to take credit for the idea of ambiguity reviews, but I can't. A formalized process for ambiguity reviews was defined and documented by Richard Bender here: http://benderrbt.com/Ambiguityprocess.pdf

The way it works is that it converts your technical specification reviews into a 2-phase process; first a review for ambiguity, and once any problems are resolved, your technical review. What sorts of things should you look for in an ambiguity review?
  • Adjectives such as: frequent or infrequent, many or few, normal or unusual
  • Adverbs such as: in general, not quite, hardly ever (remember the bridge!)
  • Verbs such as: maximize, efficiently, seamlessly (these sound like "resume words," don't they ;-)
In short, descriptions should be in measurable terms. Just like constants in an equation.

It's common to think of software testing problems in algebraic terms in that when you are writing a test or chasing a bug, you're trying to resolve variables to be able to reach a conclusion. If you can remove ambiguity from the information on which you base your tests, you can reduce the number of those variables. But, don't stop there! You should subject your own test plans to same ambiguity reviews so that your tests are just as precise as the product definition.

References:

http://benderrbt.com

(Special thanks to Mike for inspiring this!)

Friday, June 4, 2010

Barely Adequate Reports and the NADA Packet - The Value of Clear Communications in Software Testing

One of the most important non-technical skills for a software test engineer is the ability to communicate clearly. In software testing, you are frequently confronted with situations where you must describe complicated tests, system configurations, and sequences of events. And, you sometimes have to do this during moments of stress during project development and test cycles, where the last thing that anyone wants to hear about is vague descriptions of yet another "interesting" bug.

I talking to former co-workers of former companies recently, I remembered a couple of instances from years ago where I saw, or was guilty of, less than clear communications. Each of these instances presented a different type of "teachable moment."

The first one involved a problem with lack of substantive detail. I had submitted a test analysis report on the state of a product's quality where I said that it was "barely adequate" for release to beta. My boss then informed me that my report was "barely adequate" as it was more of an editorial than a factual report. ;-) The correct approach to have taken in the report would have been to compare the state of the product to its defined beta test requirements in clearly measurable terms and not in personal opinions.

The second one involved a problem with language and jargon. In reviewing some existing tests, I noticed a comment in the code that read "then NADA packets result." It took me a few puzzled minutes to realize that "NADA" was not an obscure 4 letter acronym, but the Spanish word for "nothing." Yes, it would have been a lot easier if the comments had been at least as clear as the code they were meant to augment.

So, to ensure that you're communicating in a better than "barely adequate" manner, stick to the facts, and keep things clear.

(Gracias Cheryl!)

Tuesday, May 4, 2010

Sunday, April 25, 2010

Great Brain-Storming Tool

A co-worker recently showed me a great brainstorming tool - XMind (http://www.xmind.net/).

It's cool to work with as you can easily convert your thoughts to a graphical model. For example, here are some quick and semi-random thoughts about what it takes to be a software test engineer, all arranged into an organized map:



(Thanks Prabhat!)

Sunday, April 11, 2010

JMS Transactions and the SOA Platform

The Server's Down - And What's Worse, I Lost my Money!

Regardless of how carefully you plan, it's inevitable that the software you build will encounter a situation or chain of events that you did not anticipate, and experience some type of failure. (Trust me on this. I work in software testing and spend most of my waking hours either causing software failures, or debugging them.) Since it is impossible to avoid every failure, the ability to recover from failures is a crucial part of every software system design.

A classic type of failure involves moving money between bank accounts. It's a 2-phase set of actions. First, money is withdrawn from one account and, second, the money is deposited into a second account. This sounds simple, right? Well, suppose that the the system encounters a failure after it has withdrawn the money from the first account, but before it has had a chance to deposit into the second account? What happens to the money? How can you avoid losing this money? The answer is that you enclose these two actions into a transaction.

Transactions and ACID Properties

What exactly is a "transaction?" Like a lot of things in software, it's a buzzword that is overused. But beyond that, a transaction is a logical grouping of actions into a single larger action, where this larger action is performed, or not performed, based on an all-or-nothing evaluation of the results of the individual actions[1]. In the case of the transfer of money example we just mentioned, wrapping the two transfers into a transaction would ensure that if either transfer failed, your money would not be lost, and both accounts would be restored to their original condition.

Transactions are frequently described in terms of having an "ACID"[2] set of properties:
  • Atomicity - When we talk about wanting a transaction to be "atomic" what we mean is that we want the transaction, which will be made up of a set of individual actions, to be treated as a single atomic unit. If any of these actions fails, such as one of the bank account transfers in our example, then the entire transaction is canceled. (The term used for this is "rolled back.") A transaction is an all-or-nothing construct. All the actions enclosed by the transaction must be completed successfully before the transaction itself is completed (or "committed.")
  • Consistency - Transactions don't leave the data that it works on in a partially finished state. If a transaction rolls back, the data or database in question is returned to the (consistent) state that it was in before the transaction started.
  • Isolation - In the context of a transaction, "isolation" refers to how any conditions or state created by a transaction are not visible to any other transaction. For example, if our bank account transfer was executed in a transaction, then while it was executing, any other transactions or operations would not be able to see any of results of the transaction until the entire transaction had completed.
  • Durability - One of the dangers of the bank account transfer example, if it were to be attempted outside of a transaction, is that if something went wrong, data (or even worse, money!) would be lost. Because a transaction guarantees the consistency of the data, that data is durable. If the transaction rolls back, the original data is restored.
Transactions and the SOA Platform's JBossESB

OK, this is all interesting, but it sounds like something better suited to databases than Service Oriented Architecture. How does a transactional model apply to the JBossESB in the SOA Platform where everything is either a message or a service?

Here's how: some transports supported by the Platform (InVM[3] and JMS) support a transactional delivery of messages. The actual delivery of that message will not happen until the transaction is committed. How can you cause a rollback of a transaction in an action pipeline to occur? By configuring the application and its services to include the transaction and then by raising a RuntimeException in the action pipeline. The best way to explain and illustrate how this is with one of the SOA Platform's "quickstart" example programs. Let's take a look.

The Quickstart

One of the great features of the SOA Platform is its extensive, and always growing, set of "quickstart" programs. These programs illustrate various features supported by the Platform and serve as a great resource for writing your own applications. For our example, we'll look at the aptly named "jms_transacted" quickstart.

This quickstart illustrates using the JMS transport with the JBossESB in the SOA Platform to handle transactions. The quickstart also shows how message redelivery can be performed with the JMS transport. Before we examine the quickstart's code, configuration files, and actual output, it's important that we make note of the quickstart's use of JCA (Java Connector Architecture) [4]. JCA provides a standardized way to connect to JMS providers. For the JBossESB in the SOA Platform, jms-jca-providers provide transaction support for the action pipeline by enclosing actions in a JTA [5] transaction. These transactions ensure that messages are handled within an enclosing transaction. If something goes wrong in the transaction, the messages are put back into the JMS queues to be reprocessed. We'll see this in action when we run the quickstart.

The sequence of actions that the quickstart performs is:
  • Like many of the SOA Platform's quickstarts, things begin by having a JMS message routed to a service to start the action pipeline.
  • The first time that the pipeline is entered, a row is inserted into the quickstart's HSQLDB database. Note that while the SOA Platform supports multiple databases such as MySQL, Oracle, PostgreSQL and others, in production environments, it includes a fully functional HSQLDB database for demonstration purposes.
  • Then, the org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction class is called to raise an exception. This action class is configured to raise the exception (5) times, so as to force a rolback of the transaction that encloses the writes to the database. Each time that the exception is raised, the JCA adapter rolls back the transaction. The exception is also propagated to the JCA adapter and written error messages to the log.
  • After the configured limit of (5) rollbacks is reached, the org.jboss.soa.esb.samples.quickstart.jmstransacted.test.DBInsertAction class completes and the transaction is committed by the SOA Platform's JBossESB's org/jboss/soa/esb/common/JBossESBTransactionService class so that the net result is only one row written to the database.
Now, we'll take a detailed look at how this transaction is configured, and just what happens as the quickstart runs.

The Quickstart - In Detail

The transaction in quickstart includes both JMS messaging and writing to a database. Let's start by examining the connection to the HSQLDB database that the quickstart uses. First, we need to create a datasource definition to connect to the database.

In file: quickstart-ds.xml
    <?xml version="1.0" encoding="UTF-8"?>
<datasources>
<local-tx-datasource>
<jndi-name>JmsTransactedDB</jndi-name>
<connection-url>jdbc:hsqldb:hsql://localhost:1706</connection-url>
<driver-class>org.hsqldb.jdbcDriver</driver-class>
<user-name>sa</user-name>
<password></password>
<min-pool-size>5</min-pool-size>
<max-pool-size>20</max-pool-size>
<idle-timeout-minutes>0</idle-timeout-minutes>
<depends>jboss:service=JmsTransactedDB</depends>
<prepared-statement-cache-size>32</prepared-statement-cache-size>
</local-tx-datasource>

<mbean code="org.jboss.internal.soa.esb.dependencies.HypersonicDatabase"
name="jboss:service=JmsTransactedDB">
<attribute name="Port">1706</attribute>
<attribute name="BindAddress">localhost</attribute>
<attribute name="Database">JmsTransactedDB</attribute>
<attribute name="Silent">true</attribute>
<attribute name="Trace">false</attribute>
<attribute name="No_system_exit">true</attribute>
<attribute name="DataDir">${jboss.server.data.dir}</attribute>
</mbean>
</datasources>
Some items to make note of here are:
  • Line 3 - We're defining a local-tx-datasource as we will want a JCA connection with transaction support[6].
  • Line 4 - We're use this JNDI name to reference the datasource.
  • Lines 5-13 - DB driver class, connection, URL, username/password, etc. for the database connection
  • Line 12 - MBean interface for running Hypersonic in the same VM with JBoss - note that the service name and the JNDI name as we'll refer to them later.
OK, we have a datasource that we can use to get to the HSQLDB database, and get us there in a mode that supports transactions. What's next? We have to ensure that our database actually exists before we try to use it! Luckily, the SOA Platform has a class that can initialize our database when the .esb application is deployed.

In file: jbossesb-service.xml
    <?xml version="1.0" encoding="UTF-8"?>

<server>
<mbean code="org.jboss.internal.soa.esb.dependencies.DatabaseInitializer"
name="jboss.esb:service=JmsTransactedDatabaseInitializer">
<attribute name="Datasource">java:/JmsTransactedDB</attribute>
<attribute name="ExistsSql">select * from jms_transacted_table</attribute>
<attribute name="SqlFiles">
hsqldb/create.sql
</attribute>
<depends>jboss.jca:name=JmsTransactedDB,service=DataSourceBinding</depends>
</mbean>
</server>
There are (3) interesting things in this file:
  • Line 4 - This MBean creates the database on startup. How does it connect to the database? See line 6.
  • Line 6 - Here's a reference to our datasource. Note the JNDI name that we observed in the datasource definition file. How does the DatabaseInitializer know what the database schema look like? See line 9.
  • Line 9 - Here's a reference to the quickstart's src/hsqldb/create.sql file. This file create a very simple table in our database for the quickstart to use:
In file: src/hsqldb/create.sql
create table jms_transacted_table
(
unique_id INTEGER GENERATED BY DEFAULT AS IDENTITY(START WITH 1) NOT NULL PRIMARY KEY,
data_column VARCHAR(255) NOT NULL
);
What else does the quickstart's database need? The service that creates the database (JmsTransactedDatabaseInitializer) has to be deployed. This is handled in the deployment.xml file (as are the JBoss Messaging queues that the quickstart will use):
<jbossesb-deployment>
<depends>jboss.esb.quickstart.destination:service=Queue,name=quickstart_jms_transacted_Request_esb</depends>
<depends>jboss.esb.quickstart.destination:service=Queue,name=quickstart_jms_transacted_Request_gw</depends>
<depends>jboss.esb:service=JmsTransactedDatabaseInitializer</depends>
</jbossesb-deployment>
And finally, the jbossesb-service.xml quickstart-ds.xml files have to be included in the quickstart's .esb application archive file. This is handled by the "additional.deploys" property in the quickstart's build.xml file and the ../conf/base-build.xml file that is used by all the quickstarts.

cat -n build.xml | grep quick

<property name="additional.deploys" value="jbossesb-service.xml quickstart-ds.xml" />
OK, that takes care of the database. Now, let's look at how we configure the quickstart to take advantage of the transaction support on the JBossESB in the SOA Platform. As always, the place to begin is the jboss-esb.xml file.

In file: jboss-esb.xml
    <?xml version = "1.0" encoding = "UTF-8"?>
<jbossesb xmlns="http://anonsvn.labs.jboss.com/labs/jbossesb/trunk/product/etc/schemas/xml/jbossesb-1.0.1.xsd" parameterReloadSecs="5">

<providers>
<jms-jca-provider name="JBossMessaging" connection-factory="XAConnectionFactory">

<jms-bus busid="quickstartGwChannel">
<jms-message-filter
dest-type="QUEUE"
dest-name="queue/quickstart_jms_transacted_Request_gw"
transacted="true"
/>
</jms-bus>
<jms-bus busid="quickstartEsbChannel">
<jms-message-filter
dest-type="QUEUE"
dest-name="queue/quickstart_jms_transacted_Request_esb"
transacted="true"
/>
</jms-bus>
<activation-config>
<!-- The maximum number of times a message is redelivered before it is sent to the DLQ -->
<property name="dLQMaxResent" value="5"/>
</activation-config>

</jms-jca-provider>
</providers>

<services>
<service
category="JMSSecuredESB"
name="SimpleListener"
description="JMS Secured quickstart sample">
<listeners>
<jms-listener name="JMS-Gateway"
busidref="quickstartGwChannel"
is-gateway="true"/>
<jms-listener name="jmssecured"
busidref="quickstartEsbChannel"/>
</listeners>
<actions mep="OneWay">

<action name="printMessage" class="org.jboss.soa.esb.actions.SystemPrintln">
<property name="message" value="JMS Transacted Quickstart entered. Message body"/>
<property name="printfull" value="false"/>
</action>

<action name="insertDBAction" class="org.jboss.soa.esb.samples.quickstart.jmstransacted.test.DBInsertAction">
<property name="datasource-name" value="java:JmsTransactedDB"/>
<property name="db-insert-sql" value="insert into jms_transacted_table(data_column) values(?)"/>
</action>

<!--
Will throw an Exception for the configured number of rollbacks. This should trigger the transaction to be
rolledback and the message placed back onto the JMS queue.
-->
<action name="throwExceptionAction" class="org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction">
<property name="rollbacks" value="5"/> <!-- if greater than dLQMaxResent then message goes to the JMS DLQ -->
</action>

<action name="printMessageDone" class="org.jboss.soa.esb.actions.SystemPrintln">
<property name="message" value="JMS Transacted Quickstart processed successfully. Message body"/>
<property name="printfull" value="false"/>
</action>

<!-- The next action is for Continuous Integration testing -->
<action name="testStore" class="org.jboss.soa.esb.actions.StoreMessageToFile"/>
</actions>
</service>
</services>

</jbossesb>
  • Line 2 - Before we go any further, we should look at the XSD (XML schema definition) for jboss-esb.xml files. The schema definition is available here: http://anonsvn.jboss.org/repos/labs/labs/jbossesb/trunk/product/etc/schemas/xml/jbossesb-1.0.1.xsd. If you're going to be working with the SOA Platform, it's a good idea to become familiar with this XSD as all the elements that you work with in jboss-esb.xml are defined there. The two main elements are of course, providers and services. Let's look at the providers first. Two types of providers are supported. Schedule providers define schedule driven listeners that pull messages from queues at (you guessed it) scheduled time periods. In contrast, bus providers such as the jms-jca-providers that we're using in this quickstart, are associated with listeners that have messages pushed to them. [7]
  • Line 5 - Here's the start of our jms-jca-provider. Note that the XAConnectionFactory provides support for distributed transactions
  • Lines 7-13 - This is a reference to the channel and JMS queue that will be used when a JMS message that is sent to a JMS gateway to initiate the quickstart. Let's stop for a second and talk about the forms in which data is moved onto and across the JBossESB in the SOA Platform. One of the primary functions performed by JBossESB is the routing of messages between services. (I said it before, and I'll say it again; remember that, on an ESB, everything is either a message or a service.) The messages that JBoss ESB handles conform to org.jboss.soa.esb.message.Message. Service endpoints that can process messages in this format are described as being "ESB-aware." This is all well and good for services and applications that were designed and written with this message format in mind, but what about other services, including legacy applications, that communicate through other non-ESB aware data formats? JBossESB handles connecting services that communicate in non-ESB aware message formats through its gateway listeners. These listeners route incoming messages from outside the ESB to ESB services. The ESB's set of gateway listeners (generally referred to as "gateways") support a variety of message formats such as files, FTP, and JMS. These gateways accept messages in a non-ESB aware format onto the ESB and then convert them (actually, they wrap the message payload) into ESB-aware messages before sending them to their service's action pipeline.
  • Line 11 - Remember, we want the service to process messages within a transaction.
  • Lines 14-20 - Gateways are intended to enable the JBossESB to communicate to external data sources. The gateways themselves don't move data across the ESB. Accordingly, for each Gateway channel that we define, we have to define a corresponding ESB-aware channel to be used to route messages within the ESB.
  • Line 21 - The activation-config defines how we link to the JCA. In the case of this quickstart, the interesting property is: dLQMaxResent. As the comment indicates, the “DLQMaxResent” controls how many times the JCA adapter resends the message before it is sent to the dead letter queue. This is a JBossESB service that handles messages for transports when the messages cannot be delivered. (The JMS transport actually has its own dead letter queue as JBoss Messaging is shipped with a set of pre-configured destinations defined in destinations-service.xml including: <mbean code="org.jboss.jms.server.destination.Queue" name="jboss.messaging.destination:service=Queue,name=DLQ") In this quickstart, the JCA adapter will try to send the message 5 times.
After our providers are defined, we can define our services.

A service is where thing happen. Each service definition includes a set of listeners and a sequential set of actions referred to as an "action pipeline." The listeners "listen" for incoming messages for the service and route those messages to the action pipeline.

After the service name and category are defined on lines 31 and 32, we define the service's listeners:
  • Lines 35-37 - Here's the JMS gateway listener definition. Notice how it references the bus ID of the channel that we defined in earlier in the provider definition. Also, notice how it's "is-gateway" property is set to true to indicate that this listener is a gateway.
  • Lines 38-39 - And here is the ESB-aware listener that corresponds to the gateway listener.
So much for the listeners, now let's look at the actions.
  • Line 41 - This is the start of the definitions of the sequence of actions that we referred to as the action pipeline. The message exchange pattern (or "mep") indicates that the messages are not being exchanged in a request/response pattern, but rather are being sent in only one direction. (In contrast, web services exposed on the ESB would follow a synchronous request/response pattern.)
  • Lines 43-46 - This action invokes the simplest of the JBossESB's many useful out-of-the-box actions (http://soa.dzone.com/articles/works-great-right-out-box) to write some messages to the server log.
  • Lines 48-51 - Here's where things start to get interesting. This custom action writes a row to our database table. Remember the "java:JmsTransactedDBdatasource" that we defined? Here's where gets used.
  • Line 57 - Remember how we talked about how to cause a rollback of a transaction in the action pipeline to occur? We configured the .esb to include a transaction, and now we raise a RuntimeException in the action pipeline. The action defined here raises an exception of a type that we define in the org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction class. The ActionProcessingPipeline will catch the exception and pass it up the stack to its caller, the JMS/JCA adapter. The Adapter then causes the transaction to rollback and the message is redelivered.
  • Line 58 - When we defined the jms-jca-provider's activation-config, we specified a dLQMaxResent value of 5 to indicate that if the transaction was rolled back more than 5 times, the message processed in the transaction should be sent to the JMS dead letter queue. In line 58, we also set the value of the rollbacks property equal to 5, to ensure that the transaction can complete after 5 rollbacks, and the message is not sent to the dead letter queue service. As the comment helpfully explains, setting the rollbacks property to be greater than dLQMaxResent will send the message to the dead letter queue service.
  • Lines 61-64 This action is only reached after the rollbacks have been performed and the transaction is able to complete. The action writes the message body to the server log.
Yogi Berra once said that "In theory there is no difference between theory and practice. In practice there is."[8] OK. Enough theory, let's run the quickstart and watch what happens.

After the SOA-P server is started, the quickstart, is deployed with this command: ant deploy

When the quickstart is deployed, the following is to the server.log:
2010-04-07 09:26:22,117 INFO  [org.jboss.resource.connectionmanager.ConnectionFactoryBindingService] (HDScanner) Bound ConnectionManager 'jboss.jca:service=DataSourceBinding,name=JmsTransactedDB' to JNDI name 'java:JmsTransactedDB'
2010-04-07 09:26:22,157 INFO [org.jboss.internal.soa.esb.dependencies.DatabaseInitializer] (HDScanner) Initializing java:/JmsTransactedDB from listed sql files
2010-04-07 09:26:22,326 INFO [org.jboss.soa.esb.listeners.deployers.mc.EsbDeployment] (HDScanner) Starting ESB Deployment 'Quickstart_JMS_Transacted.esb'
Note how the JmsTransactedDB database is initialized by org.jboss.internal.soa.esb.dependencies.DatabaseInitializer class the when the quickstart is deployed.

Next, we run the quickstart with this command: ant runtest

When the ant runtest target is executed, the following chain reaction is started:

The org.jboss.soa.esb.samples.quickstart.jmstransacted.test.SendJMSMessage class sends a (non-ESB aware) JMS message to the quickstart_jms_transacted_Request_gw queue. The message is routed through that queue's corresponding ESB-aware queue (quickstart_jms_transacted_Request_esb) to the SimpleListener service and the action pipeline is initiated.

The first time that the action pipeline is started, the org.jboss.soa.esb.samples.quickstart.jmstransacted.test.DBInsertAction class inserts a row into the database through the HSQLDB datasource. This results in the following messages being written to the server.log:

Then, the org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction runtime exception is thrown.

Since this exception is thrown from inside the action pipeline, the JMS/JCA adapter rolls back the enclosing transaction. The exception is propagated to the JMS/JCA adapter as the exception is passed up the stack from the action pipeline to its the caller (which is the adapter). The exception is caught by the JBossESB's org.jboss.soa.esb.listeners.message.ActionProcessingPipeline, and is propagated up the stack to the JMS/JCA adapter (which is the action pipeline's caller). The adapter causes the transaction to rollback. This results in the row being removed from the database and in the original message being redelivered. Remember what we said about a transaction being an "all or nothing" event?

The transaction/rollback sequence then repeats until the configured number of rollbacks are performed. You can see a counter (maintained by the org.jboss.soa.esb.samples.quickstart.jmstransacted.test.DBInsertAction class) of the number of times the action is attempted.

The printMessage action prints the message to the log:
2010-04-07 09:26:35,340 INFO  [STDOUT] (WorkManager(2)-9) JMS Transacted Quickstart entered. Message body:
2010-04-07 09:26:35,340 INFO [STDOUT] (WorkManager(2)-9) [Hello Transacted JMS World]].
And the insertDBAction action writes the row to the database - note the counter value of [1]:
2010-04-07 09:26:35,415 INFO  [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.DBInsertAction] (WorkManager(2)-9) Successfully inserted [Hello Transacted JMS World] counter[1]] into jms_transacted_table
But wait! Then our exception is raised:
2010-04-07 09:26:35,488 ERROR [org.jboss.resource.adapter.jms.inflow.JmsServerSession] (WorkManager(2)-9) Unexpected error delivering message delegator->JBossMessage[5204569273565185]:PERSISTENT, deliveryId=0
java.lang.IllegalStateException: [Throwing Exception to trigger a transaction rollback]
at org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction.process(ThrowExceptionAction.java:75)
The JMS/JCA adapter rolls back the transaction, and the message is delivered again, and another row is written to the database. The counter is incremented to [2] this time and the exception is raised again.
2010-04-07 09:26:36,435 INFO  [STDOUT] (WorkManager(2)-10) JMS Transacted Quickstart entered. Message body:
2010-04-07 09:26:36,435 INFO [STDOUT] (WorkManager(2)-10) [Hello Transacted JMS World]].
2010-04-07 09:26:36,438 INFO [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.DBInsertAction] (WorkManager(2)-10) Successfully inserted [Hello Transacted JMS World] counter[2]] into jms_transacted_table
2010-04-07 09:26:36,442 ERROR [org.jboss.resource.adapter.jms.inflow.JmsServerSession] (WorkManager(2)-10) Unexpected error delivering message delegator->JBossMessage[5204569273565185]:PERSISTENT, deliveryId=1
java.lang.IllegalStateException: [Throwing Exception to trigger a transaction rollback]
at org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction.process(ThrowExceptionAction.java:75)
When the rollback counter number is reached, the action pipeline completes and the transaction commits so that the row is finally written to the database. The following messages are written to the servler.log:

This sequence repeats (5) times until the counter is passed and the row is actually written to the database:
2010-04-07 09:26:40,489 INFO  [STDOUT] (WorkManager(2)-14) JMS Transacted Quickstart entered. Message body:
2010-04-07 09:26:40,489 INFO [STDOUT] (WorkManager(2)-14) [Hello Transacted JMS World]].
2010-04-07 09:26:40,490 INFO [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.DBInsertAction] (WorkManager(2)-14) Successfully inserted [Hello Transacted JMS World] counter[6]] into jms_transacted_table
2010-04-07 09:26:40,490 INFO [STDOUT] (WorkManager(2)-14) JMS Transacted Quickstart processed successfully. Message body:
2010-04-07 09:26:40,490 INFO [STDOUT] (WorkManager(2)-14) [Hello Transacted JMS World]].
Now, as I mentioned in the beginning of this post, I work as a software QE engineer. One of the basic tenents of software QE is to never trust anything. Let's take a closer look at what happens when the quickstart runs.

Server logs are goldmines for gathering debugging information. Let's restart the server, but this time set the server logging level to DEBUG:
./run.sh -Djboss.server.log.threshold=DEBUG
And then rerun the quickstart. The server.log will be much more verbose this time, but if you look closely, you can see the rollback counter in action:
grep  nr-of-rollbacks server.log
2010-04-07 23:14:19,776 DEBUG [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction] (WorkManager(2)-11) rollbackCounter [0], nr-of-rollbacks [5]
2010-04-07 23:14:20,804 DEBUG [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction] (WorkManager(2)-12) rollbackCounter [1], nr-of-rollbacks [5]
2010-04-07 23:14:21,819 DEBUG [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction] (WorkManager(2)-13) rollbackCounter [2], nr-of-rollbacks [5]
2010-04-07 23:14:22,833 DEBUG [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction] (WorkManager(2)-14) rollbackCounter [3], nr-of-rollbacks [5]
2010-04-07 23:14:23,848 DEBUG [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction] (WorkManager(2)-15) rollbackCounter [4], nr-of-rollbacks [5]
2010-04-07 23:14:24,858 DEBUG [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction] (WorkManager(2)-16) rollbackCounter [5], nr-of-rollbacks [5]
Before we move on, let's look at one more thing and verify that only one row is actually written to the database. Now, the quickstart is very tidy and always cleans up after itself by removing rows from the database. Let's disable this cleanup by removing this target from the quickstart's build.xml file:

cat build.xml | grep truncate



And, let's start the server again, but with one additional command line option: sh ./run.sh -Djava.awt.headless=false

Why "headless=false?" After we run the quickstart again, the row that was written to the database should still be there. We'll look at it with the HSQL DB Manager service. This service requires the server to not be started in its default "headless" mode, as the service opens up a Java UI application. We invoke the DB Manager from the JMX console - see screenshot - the service tag is: database=jbossesb,service=Hypersonic.


Once the DM Manager application UI launches, we just connect to the "JmsTransactedDB" database - see screenshot - and execute a query to return all rows in the "jms_transacted_table" - see screenshot.

And, there is our one record!


Before we move on, it's worthwhile to note what we did NOT have to do to execute a transaction with the SOA Platform.

We did not have to write transaction specific code. The JBossESB in the Platform did the dirty work for us. All we had to do is to define the .esb application's configuration in its jboss-esb.xml file to include the transaction-relevant properties and reference a JMS/JCA adapter and the Platform did the rest. This is what makes middleware so useful. You could write the code to handle transactions all on your own, but wouldn't you rather concentrate on solving your own business problems instead of build infrastucture plumbing? [9]

Closing Thoughts

The value that using transactions adds to an application is pretty obvious (just keep your bank account in mind). The value added by middleware in implementing transactions should also be obvious. You could write your own code to handle atomicity, consistency, isolation and durability, but, this would not be an easy task, and it would also take away a lot of time from your being able to concentrate on solving your own business logic problems. The SOA Platform's support for transactions over the JBossESB enables you to provide a higher level of reliability and durablity to your service based applications.

References


[1] http://qconlondon.com/london-2010/file?path=/qcon-london-2010/slides/MarkLittle_TransactionsOverUsedOrJustMisunderstood.pdf

[2] Little, Mark, Maron, Jon, Pavlik, Greg. Transaction Processing: Design and Implementation, Upper Saddle RIver, new Jersey: Prentice Hall/Hewlett-Packard Professional Books, 2004.

[3] http://www.redhat.com/docs/en-US/JBoss_SOA_Platform/5.0.0/html-single/Programmers_Guide/index.html#sect-SOA_ESB_Programmers_Guide-What_is_a_Service-InVM_Transport

[4] http://java.sun.com/j2ee/connector/

[5] http://java.sun.com/javaee/technologies/jta/index.jsp

[6] http://www.jboss.org/file-access/default/members/jbossas/freezone/docs/Server_Configuration_Guide/4/html/Connectors_on_JBoss-Configuring_JDBC_DataSources.html

[7] http://www.redhat.com/docs/en-US/JBoss_SOA_Platform/5.0.0/html-single/Programmers_Guide/index.html#sect-SOA_ESB_Programmers_Guide-Configuration-Providers

[8] http://en.wikiquote.org/wiki/Yogi_Berra

[9] http://magazine.redhat.com/2008/03/11/what-is-middleware-in-plain-english-please

Acknowledgments

As always, I want to thank the JBoss SOA Platform team and community (especially Kevin Conner for his timely review input for this blog post).