The previous post - on JMS Transactions has been re-posted to the JBoss SOA Platform blog:
http://jboss-soa-p.blogspot.com/2010/04/jms-transactions-and-soa-platform.html
And DZone:
http://soa.dzone.com/articles/jms-transactions-soa-platform
Tuesday, April 20, 2010
Sunday, April 11, 2010
JMS Transactions and the SOA Platform
The Server's Down - And What's Worse, I Lost my Money!
Regardless of how carefully you plan, it's inevitable that the software you build will encounter a situation or chain of events that you did not anticipate, and experience some type of failure. (Trust me on this. I work in software testing and spend most of my waking hours either causing software failures, or debugging them.) Since it is impossible to avoid every failure, the ability to recover from failures is a crucial part of every software system design.
A classic type of failure involves moving money between bank accounts. It's a 2-phase set of actions. First, money is withdrawn from one account and, second, the money is deposited into a second account. This sounds simple, right? Well, suppose that the the system encounters a failure after it has withdrawn the money from the first account, but before it has had a chance to deposit into the second account? What happens to the money? How can you avoid losing this money? The answer is that you enclose these two actions into a transaction.
Transactions and ACID Properties
What exactly is a "transaction?" Like a lot of things in software, it's a buzzword that is overused. But beyond that, a transaction is a logical grouping of actions into a single larger action, where this larger action is performed, or not performed, based on an all-or-nothing evaluation of the results of the individual actions[1]. In the case of the transfer of money example we just mentioned, wrapping the two transfers into a transaction would ensure that if either transfer failed, your money would not be lost, and both accounts would be restored to their original condition.
Transactions are frequently described in terms of having an "ACID"[2] set of properties:
OK, this is all interesting, but it sounds like something better suited to databases than Service Oriented Architecture. How does a transactional model apply to the JBossESB in the SOA Platform where everything is either a message or a service?
Here's how: some transports supported by the Platform (InVM[3] and JMS) support a transactional delivery of messages. The actual delivery of that message will not happen until the transaction is committed. How can you cause a rollback of a transaction in an action pipeline to occur? By configuring the application and its services to include the transaction and then by raising a RuntimeException in the action pipeline. The best way to explain and illustrate how this is with one of the SOA Platform's "quickstart" example programs. Let's take a look.
The Quickstart
One of the great features of the SOA Platform is its extensive, and always growing, set of "quickstart" programs. These programs illustrate various features supported by the Platform and serve as a great resource for writing your own applications. For our example, we'll look at the aptly named "jms_transacted" quickstart.
This quickstart illustrates using the JMS transport with the JBossESB in the SOA Platform to handle transactions. The quickstart also shows how message redelivery can be performed with the JMS transport. Before we examine the quickstart's code, configuration files, and actual output, it's important that we make note of the quickstart's use of JCA (Java Connector Architecture) [4]. JCA provides a standardized way to connect to JMS providers. For the JBossESB in the SOA Platform, jms-jca-providers provide transaction support for the action pipeline by enclosing actions in a JTA [5] transaction. These transactions ensure that messages are handled within an enclosing transaction. If something goes wrong in the transaction, the messages are put back into the JMS queues to be reprocessed. We'll see this in action when we run the quickstart.
The sequence of actions that the quickstart performs is:
The Quickstart - In Detail
The transaction in quickstart includes both JMS messaging and writing to a database. Let's start by examining the connection to the HSQLDB database that the quickstart uses. First, we need to create a datasource definition to connect to the database.
In file: quickstart-ds.xml
In file: jbossesb-service.xml
cat -n build.xml | grep quick
In file: jboss-esb.xml
A service is where thing happen. Each service definition includes a set of listeners and a sequential set of actions referred to as an "action pipeline." The listeners "listen" for incoming messages for the service and route those messages to the action pipeline.
After the service name and category are defined on lines 31 and 32, we define the service's listeners:
After the SOA-P server is started, the quickstart, is deployed with this command: ant deploy
When the quickstart is deployed, the following is to the server.log:
Next, we run the quickstart with this command: ant runtest
When the ant runtest target is executed, the following chain reaction is started:
The org.jboss.soa.esb.samples.quickstart.jmstransacted.test.SendJMSMessage class sends a (non-ESB aware) JMS message to the quickstart_jms_transacted_Request_gw queue. The message is routed through that queue's corresponding ESB-aware queue (quickstart_jms_transacted_Request_esb) to the SimpleListener service and the action pipeline is initiated.
The first time that the action pipeline is started, the org.jboss.soa.esb.samples.quickstart.jmstransacted.test.DBInsertAction class inserts a row into the database through the HSQLDB datasource. This results in the following messages being written to the server.log:
Then, the org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction runtime exception is thrown.
Since this exception is thrown from inside the action pipeline, the JMS/JCA adapter rolls back the enclosing transaction. The exception is propagated to the JMS/JCA adapter as the exception is passed up the stack from the action pipeline to its the caller (which is the adapter). The exception is caught by the JBossESB's org.jboss.soa.esb.listeners.message.ActionProcessingPipeline, and is propagated up the stack to the JMS/JCA adapter (which is the action pipeline's caller). The adapter causes the transaction to rollback. This results in the row being removed from the database and in the original message being redelivered. Remember what we said about a transaction being an "all or nothing" event?
The transaction/rollback sequence then repeats until the configured number of rollbacks are performed. You can see a counter (maintained by the org.jboss.soa.esb.samples.quickstart.jmstransacted.test.DBInsertAction class) of the number of times the action is attempted.
The printMessage action prints the message to the log:
This sequence repeats (5) times until the counter is passed and the row is actually written to the database:
Server logs are goldmines for gathering debugging information. Let's restart the server, but this time set the server logging level to DEBUG:
cat build.xml | grep truncate
And, let's start the server again, but with one additional command line option: sh ./run.sh -Djava.awt.headless=false
Why "headless=false?" After we run the quickstart again, the row that was written to the database should still be there. We'll look at it with the HSQL DB Manager service. This service requires the server to not be started in its default "headless" mode, as the service opens up a Java UI application. We invoke the DB Manager from the JMX console - see screenshot - the service tag is: database=jbossesb,service=Hypersonic.
Once the DM Manager application UI launches, we just connect to the "JmsTransactedDB" database - see screenshot - and execute a query to return all rows in the "jms_transacted_table" - see screenshot.
And, there is our one record!
Before we move on, it's worthwhile to note what we did NOT have to do to execute a transaction with the SOA Platform.
We did not have to write transaction specific code. The JBossESB in the Platform did the dirty work for us. All we had to do is to define the .esb application's configuration in its jboss-esb.xml file to include the transaction-relevant properties and reference a JMS/JCA adapter and the Platform did the rest. This is what makes middleware so useful. You could write the code to handle transactions all on your own, but wouldn't you rather concentrate on solving your own business problems instead of build infrastucture plumbing? [9]
Closing Thoughts
The value that using transactions adds to an application is pretty obvious (just keep your bank account in mind). The value added by middleware in implementing transactions should also be obvious. You could write your own code to handle atomicity, consistency, isolation and durability, but, this would not be an easy task, and it would also take away a lot of time from your being able to concentrate on solving your own business logic problems. The SOA Platform's support for transactions over the JBossESB enables you to provide a higher level of reliability and durablity to your service based applications.
References
[1] http://qconlondon.com/london-2010/file?path=/qcon-london-2010/slides/MarkLittle_TransactionsOverUsedOrJustMisunderstood.pdf
[2] Little, Mark, Maron, Jon, Pavlik, Greg. Transaction Processing: Design and Implementation, Upper Saddle RIver, new Jersey: Prentice Hall/Hewlett-Packard Professional Books, 2004.
[3] http://www.redhat.com/docs/en-US/JBoss_SOA_Platform/5.0.0/html-single/Programmers_Guide/index.html#sect-SOA_ESB_Programmers_Guide-What_is_a_Service-InVM_Transport
[4] http://java.sun.com/j2ee/connector/
[5] http://java.sun.com/javaee/technologies/jta/index.jsp
[6] http://www.jboss.org/file-access/default/members/jbossas/freezone/docs/Server_Configuration_Guide/4/html/Connectors_on_JBoss-Configuring_JDBC_DataSources.html
[7] http://www.redhat.com/docs/en-US/JBoss_SOA_Platform/5.0.0/html-single/Programmers_Guide/index.html#sect-SOA_ESB_Programmers_Guide-Configuration-Providers
[8] http://en.wikiquote.org/wiki/Yogi_Berra
[9] http://magazine.redhat.com/2008/03/11/what-is-middleware-in-plain-english-please
Acknowledgments
As always, I want to thank the JBoss SOA Platform team and community (especially Kevin Conner for his timely review input for this blog post).
Regardless of how carefully you plan, it's inevitable that the software you build will encounter a situation or chain of events that you did not anticipate, and experience some type of failure. (Trust me on this. I work in software testing and spend most of my waking hours either causing software failures, or debugging them.) Since it is impossible to avoid every failure, the ability to recover from failures is a crucial part of every software system design.
A classic type of failure involves moving money between bank accounts. It's a 2-phase set of actions. First, money is withdrawn from one account and, second, the money is deposited into a second account. This sounds simple, right? Well, suppose that the the system encounters a failure after it has withdrawn the money from the first account, but before it has had a chance to deposit into the second account? What happens to the money? How can you avoid losing this money? The answer is that you enclose these two actions into a transaction.
Transactions and ACID Properties
What exactly is a "transaction?" Like a lot of things in software, it's a buzzword that is overused. But beyond that, a transaction is a logical grouping of actions into a single larger action, where this larger action is performed, or not performed, based on an all-or-nothing evaluation of the results of the individual actions[1]. In the case of the transfer of money example we just mentioned, wrapping the two transfers into a transaction would ensure that if either transfer failed, your money would not be lost, and both accounts would be restored to their original condition.
Transactions are frequently described in terms of having an "ACID"[2] set of properties:
- Atomicity - When we talk about wanting a transaction to be "atomic" what we mean is that we want the transaction, which will be made up of a set of individual actions, to be treated as a single atomic unit. If any of these actions fails, such as one of the bank account transfers in our example, then the entire transaction is canceled. (The term used for this is "rolled back.") A transaction is an all-or-nothing construct. All the actions enclosed by the transaction must be completed successfully before the transaction itself is completed (or "committed.")
- Consistency - Transactions don't leave the data that it works on in a partially finished state. If a transaction rolls back, the data or database in question is returned to the (consistent) state that it was in before the transaction started.
- Isolation - In the context of a transaction, "isolation" refers to how any conditions or state created by a transaction are not visible to any other transaction. For example, if our bank account transfer was executed in a transaction, then while it was executing, any other transactions or operations would not be able to see any of results of the transaction until the entire transaction had completed.
- Durability - One of the dangers of the bank account transfer example, if it were to be attempted outside of a transaction, is that if something went wrong, data (or even worse, money!) would be lost. Because a transaction guarantees the consistency of the data, that data is durable. If the transaction rolls back, the original data is restored.
OK, this is all interesting, but it sounds like something better suited to databases than Service Oriented Architecture. How does a transactional model apply to the JBossESB in the SOA Platform where everything is either a message or a service?
Here's how: some transports supported by the Platform (InVM[3] and JMS) support a transactional delivery of messages. The actual delivery of that message will not happen until the transaction is committed. How can you cause a rollback of a transaction in an action pipeline to occur? By configuring the application and its services to include the transaction and then by raising a RuntimeException in the action pipeline. The best way to explain and illustrate how this is with one of the SOA Platform's "quickstart" example programs. Let's take a look.
The Quickstart
One of the great features of the SOA Platform is its extensive, and always growing, set of "quickstart" programs. These programs illustrate various features supported by the Platform and serve as a great resource for writing your own applications. For our example, we'll look at the aptly named "jms_transacted" quickstart.
This quickstart illustrates using the JMS transport with the JBossESB in the SOA Platform to handle transactions. The quickstart also shows how message redelivery can be performed with the JMS transport. Before we examine the quickstart's code, configuration files, and actual output, it's important that we make note of the quickstart's use of JCA (Java Connector Architecture) [4]. JCA provides a standardized way to connect to JMS providers. For the JBossESB in the SOA Platform, jms-jca-providers provide transaction support for the action pipeline by enclosing actions in a JTA [5] transaction. These transactions ensure that messages are handled within an enclosing transaction. If something goes wrong in the transaction, the messages are put back into the JMS queues to be reprocessed. We'll see this in action when we run the quickstart.
The sequence of actions that the quickstart performs is:
- Like many of the SOA Platform's quickstarts, things begin by having a JMS message routed to a service to start the action pipeline.
- The first time that the pipeline is entered, a row is inserted into the quickstart's HSQLDB database. Note that while the SOA Platform supports multiple databases such as MySQL, Oracle, PostgreSQL and others, in production environments, it includes a fully functional HSQLDB database for demonstration purposes.
- Then, the org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction class is called to raise an exception. This action class is configured to raise the exception (5) times, so as to force a rolback of the transaction that encloses the writes to the database. Each time that the exception is raised, the JCA adapter rolls back the transaction. The exception is also propagated to the JCA adapter and written error messages to the log.
- After the configured limit of (5) rollbacks is reached, the org.jboss.soa.esb.samples.quickstart.jmstransacted.test.DBInsertAction class completes and the transaction is committed by the SOA Platform's JBossESB's org/jboss/soa/esb/common/JBossESBTransactionService class so that the net result is only one row written to the database.
The Quickstart - In Detail
The transaction in quickstart includes both JMS messaging and writing to a database. Let's start by examining the connection to the HSQLDB database that the quickstart uses. First, we need to create a datasource definition to connect to the database.
In file: quickstart-ds.xml
<?xml version="1.0" encoding="UTF-8"?>Some items to make note of here are:
<datasources>
<local-tx-datasource>
<jndi-name>JmsTransactedDB</jndi-name>
<connection-url>jdbc:hsqldb:hsql://localhost:1706</connection-url>
<driver-class>org.hsqldb.jdbcDriver</driver-class>
<user-name>sa</user-name>
<password></password>
<min-pool-size>5</min-pool-size>
<max-pool-size>20</max-pool-size>
<idle-timeout-minutes>0</idle-timeout-minutes>
<depends>jboss:service=JmsTransactedDB</depends>
<prepared-statement-cache-size>32</prepared-statement-cache-size>
</local-tx-datasource>
<mbean code="org.jboss.internal.soa.esb.dependencies.HypersonicDatabase"
name="jboss:service=JmsTransactedDB">
<attribute name="Port">1706</attribute>
<attribute name="BindAddress">localhost</attribute>
<attribute name="Database">JmsTransactedDB</attribute>
<attribute name="Silent">true</attribute>
<attribute name="Trace">false</attribute>
<attribute name="No_system_exit">true</attribute>
<attribute name="DataDir">${jboss.server.data.dir}</attribute>
</mbean>
</datasources>
- Line 3 - We're defining a local-tx-datasource as we will want a JCA connection with transaction support[6].
- Line 4 - We're use this JNDI name to reference the datasource.
- Lines 5-13 - DB driver class, connection, URL, username/password, etc. for the database connection
- Line 12 - MBean interface for running Hypersonic in the same VM with JBoss - note that the service name and the JNDI name as we'll refer to them later.
In file: jbossesb-service.xml
<?xml version="1.0" encoding="UTF-8"?>There are (3) interesting things in this file:
<server>
<mbean code="org.jboss.internal.soa.esb.dependencies.DatabaseInitializer"
name="jboss.esb:service=JmsTransactedDatabaseInitializer">
<attribute name="Datasource">java:/JmsTransactedDB</attribute>
<attribute name="ExistsSql">select * from jms_transacted_table</attribute>
<attribute name="SqlFiles">
hsqldb/create.sql
</attribute>
<depends>jboss.jca:name=JmsTransactedDB,service=DataSourceBinding</depends>
</mbean>
</server>
- Line 4 - This MBean creates the database on startup. How does it connect to the database? See line 6.
- Line 6 - Here's a reference to our datasource. Note the JNDI name that we observed in the datasource definition file. How does the DatabaseInitializer know what the database schema look like? See line 9.
- Line 9 - Here's a reference to the quickstart's src/hsqldb/create.sql file. This file create a very simple table in our database for the quickstart to use:
create table jms_transacted_tableWhat else does the quickstart's database need? The service that creates the database (JmsTransactedDatabaseInitializer) has to be deployed. This is handled in the deployment.xml file (as are the JBoss Messaging queues that the quickstart will use):
(
unique_id INTEGER GENERATED BY DEFAULT AS IDENTITY(START WITH 1) NOT NULL PRIMARY KEY,
data_column VARCHAR(255) NOT NULL
);
<jbossesb-deployment>And finally, the jbossesb-service.xml quickstart-ds.xml files have to be included in the quickstart's .esb application archive file. This is handled by the "additional.deploys" property in the quickstart's build.xml file and the ../conf/base-build.xml file that is used by all the quickstarts.
<depends>jboss.esb.quickstart.destination:service=Queue,name=quickstart_jms_transacted_Request_esb</depends>
<depends>jboss.esb.quickstart.destination:service=Queue,name=quickstart_jms_transacted_Request_gw</depends>
<depends>jboss.esb:service=JmsTransactedDatabaseInitializer</depends>
</jbossesb-deployment>
cat -n build.xml | grep quick
OK, that takes care of the database. Now, let's look at how we configure the quickstart to take advantage of the transaction support on the JBossESB in the SOA Platform. As always, the place to begin is the jboss-esb.xml file.
<property name="additional.deploys" value="jbossesb-service.xml quickstart-ds.xml" />
In file: jboss-esb.xml
<?xml version = "1.0" encoding = "UTF-8"?>
<jbossesb xmlns="http://anonsvn.labs.jboss.com/labs/jbossesb/trunk/product/etc/schemas/xml/jbossesb-1.0.1.xsd" parameterReloadSecs="5">
<providers>
<jms-jca-provider name="JBossMessaging" connection-factory="XAConnectionFactory">
<jms-bus busid="quickstartGwChannel">
<jms-message-filter
dest-type="QUEUE"
dest-name="queue/quickstart_jms_transacted_Request_gw"
transacted="true"
/>
</jms-bus>
<jms-bus busid="quickstartEsbChannel">
<jms-message-filter
dest-type="QUEUE"
dest-name="queue/quickstart_jms_transacted_Request_esb"
transacted="true"
/>
</jms-bus>
<activation-config>
<!-- The maximum number of times a message is redelivered before it is sent to the DLQ -->
<property name="dLQMaxResent" value="5"/>
</activation-config>
</jms-jca-provider>
</providers>
<services>
<service
category="JMSSecuredESB"
name="SimpleListener"
description="JMS Secured quickstart sample">
<listeners>
<jms-listener name="JMS-Gateway"
busidref="quickstartGwChannel"
is-gateway="true"/>
<jms-listener name="jmssecured"
busidref="quickstartEsbChannel"/>
</listeners>
<actions mep="OneWay">
<action name="printMessage" class="org.jboss.soa.esb.actions.SystemPrintln">
<property name="message" value="JMS Transacted Quickstart entered. Message body"/>
<property name="printfull" value="false"/>
</action>
<action name="insertDBAction" class="org.jboss.soa.esb.samples.quickstart.jmstransacted.test.DBInsertAction">
<property name="datasource-name" value="java:JmsTransactedDB"/>
<property name="db-insert-sql" value="insert into jms_transacted_table(data_column) values(?)"/>
</action>
<!--
Will throw an Exception for the configured number of rollbacks. This should trigger the transaction to be
rolledback and the message placed back onto the JMS queue.
-->
<action name="throwExceptionAction" class="org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction">
<property name="rollbacks" value="5"/> <!-- if greater than dLQMaxResent then message goes to the JMS DLQ -->
</action>
<action name="printMessageDone" class="org.jboss.soa.esb.actions.SystemPrintln">
<property name="message" value="JMS Transacted Quickstart processed successfully. Message body"/>
<property name="printfull" value="false"/>
</action>
<!-- The next action is for Continuous Integration testing -->
<action name="testStore" class="org.jboss.soa.esb.actions.StoreMessageToFile"/>
</actions>
</service>
</services>
</jbossesb>
- Line 2 - Before we go any further, we should look at the XSD (XML schema definition) for jboss-esb.xml files. The schema definition is available here: http://anonsvn.jboss.org/repos/labs/labs/jbossesb/trunk/product/etc/schemas/xml/jbossesb-1.0.1.xsd. If you're going to be working with the SOA Platform, it's a good idea to become familiar with this XSD as all the elements that you work with in jboss-esb.xml are defined there. The two main elements are of course, providers and services. Let's look at the providers first. Two types of providers are supported. Schedule providers define schedule driven listeners that pull messages from queues at (you guessed it) scheduled time periods. In contrast, bus providers such as the jms-jca-providers that we're using in this quickstart, are associated with listeners that have messages pushed to them. [7]
- Line 5 - Here's the start of our jms-jca-provider. Note that the XAConnectionFactory provides support for distributed transactions
- Lines 7-13 - This is a reference to the channel and JMS queue that will be used when a JMS message that is sent to a JMS gateway to initiate the quickstart. Let's stop for a second and talk about the forms in which data is moved onto and across the JBossESB in the SOA Platform. One of the primary functions performed by JBossESB is the routing of messages between services. (I said it before, and I'll say it again; remember that, on an ESB, everything is either a message or a service.) The messages that JBoss ESB handles conform to org.jboss.soa.esb.message.Message. Service endpoints that can process messages in this format are described as being "ESB-aware." This is all well and good for services and applications that were designed and written with this message format in mind, but what about other services, including legacy applications, that communicate through other non-ESB aware data formats? JBossESB handles connecting services that communicate in non-ESB aware message formats through its gateway listeners. These listeners route incoming messages from outside the ESB to ESB services. The ESB's set of gateway listeners (generally referred to as "gateways") support a variety of message formats such as files, FTP, and JMS. These gateways accept messages in a non-ESB aware format onto the ESB and then convert them (actually, they wrap the message payload) into ESB-aware messages before sending them to their service's action pipeline.
- Line 11 - Remember, we want the service to process messages within a transaction.
- Lines 14-20 - Gateways are intended to enable the JBossESB to communicate to external data sources. The gateways themselves don't move data across the ESB. Accordingly, for each Gateway channel that we define, we have to define a corresponding ESB-aware channel to be used to route messages within the ESB.
- Line 21 - The activation-config defines how we link to the JCA. In the case of this quickstart, the interesting property is: dLQMaxResent. As the comment indicates, the “DLQMaxResent” controls how many times the JCA adapter resends the message before it is sent to the dead letter queue. This is a JBossESB service that handles messages for transports when the messages cannot be delivered. (The JMS transport actually has its own dead letter queue as JBoss Messaging is shipped with a set of pre-configured destinations defined in destinations-service.xml including: <mbean code="org.jboss.jms.server.destination.Queue" name="jboss.messaging.destination:service=Queue,name=DLQ") In this quickstart, the JCA adapter will try to send the message 5 times.
A service is where thing happen. Each service definition includes a set of listeners and a sequential set of actions referred to as an "action pipeline." The listeners "listen" for incoming messages for the service and route those messages to the action pipeline.
After the service name and category are defined on lines 31 and 32, we define the service's listeners:
- Lines 35-37 - Here's the JMS gateway listener definition. Notice how it references the bus ID of the channel that we defined in earlier in the provider definition. Also, notice how it's "is-gateway" property is set to true to indicate that this listener is a gateway.
- Lines 38-39 - And here is the ESB-aware listener that corresponds to the gateway listener.
- Line 41 - This is the start of the definitions of the sequence of actions that we referred to as the action pipeline. The message exchange pattern (or "mep") indicates that the messages are not being exchanged in a request/response pattern, but rather are being sent in only one direction. (In contrast, web services exposed on the ESB would follow a synchronous request/response pattern.)
- Lines 43-46 - This action invokes the simplest of the JBossESB's many useful out-of-the-box actions (http://soa.dzone.com/articles/works-great-right-out-box) to write some messages to the server log.
- Lines 48-51 - Here's where things start to get interesting. This custom action writes a row to our database table. Remember the "java:JmsTransactedDBdatasource" that we defined? Here's where gets used.
- Line 57 - Remember how we talked about how to cause a rollback of a transaction in the action pipeline to occur? We configured the .esb to include a transaction, and now we raise a RuntimeException in the action pipeline. The action defined here raises an exception of a type that we define in the org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction class. The ActionProcessingPipeline will catch the exception and pass it up the stack to its caller, the JMS/JCA adapter. The Adapter then causes the transaction to rollback and the message is redelivered.
- Line 58 - When we defined the jms-jca-provider's activation-config, we specified a dLQMaxResent value of 5 to indicate that if the transaction was rolled back more than 5 times, the message processed in the transaction should be sent to the JMS dead letter queue. In line 58, we also set the value of the rollbacks property equal to 5, to ensure that the transaction can complete after 5 rollbacks, and the message is not sent to the dead letter queue service. As the comment helpfully explains, setting the rollbacks property to be greater than dLQMaxResent will send the message to the dead letter queue service.
- Lines 61-64 This action is only reached after the rollbacks have been performed and the transaction is able to complete. The action writes the message body to the server log.
After the SOA-P server is started, the quickstart, is deployed with this command: ant deploy
When the quickstart is deployed, the following is to the server.log:
2010-04-07 09:26:22,117 INFO [org.jboss.resource.connectionmanager.ConnectionFactoryBindingService] (HDScanner) Bound ConnectionManager 'jboss.jca:service=DataSourceBinding,name=JmsTransactedDB' to JNDI name 'java:JmsTransactedDB'Note how the JmsTransactedDB database is initialized by org.jboss.internal.soa.esb.dependencies.DatabaseInitializer class the when the quickstart is deployed.
2010-04-07 09:26:22,157 INFO [org.jboss.internal.soa.esb.dependencies.DatabaseInitializer] (HDScanner) Initializing java:/JmsTransactedDB from listed sql files
2010-04-07 09:26:22,326 INFO [org.jboss.soa.esb.listeners.deployers.mc.EsbDeployment] (HDScanner) Starting ESB Deployment 'Quickstart_JMS_Transacted.esb'
Next, we run the quickstart with this command: ant runtest
When the ant runtest target is executed, the following chain reaction is started:
The org.jboss.soa.esb.samples.quickstart.jmstransacted.test.SendJMSMessage class sends a (non-ESB aware) JMS message to the quickstart_jms_transacted_Request_gw queue. The message is routed through that queue's corresponding ESB-aware queue (quickstart_jms_transacted_Request_esb) to the SimpleListener service and the action pipeline is initiated.
The first time that the action pipeline is started, the org.jboss.soa.esb.samples.quickstart.jmstransacted.test.DBInsertAction class inserts a row into the database through the HSQLDB datasource. This results in the following messages being written to the server.log:
Then, the org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction runtime exception is thrown.
Since this exception is thrown from inside the action pipeline, the JMS/JCA adapter rolls back the enclosing transaction. The exception is propagated to the JMS/JCA adapter as the exception is passed up the stack from the action pipeline to its the caller (which is the adapter). The exception is caught by the JBossESB's org.jboss.soa.esb.listeners.message.ActionProcessingPipeline, and is propagated up the stack to the JMS/JCA adapter (which is the action pipeline's caller). The adapter causes the transaction to rollback. This results in the row being removed from the database and in the original message being redelivered. Remember what we said about a transaction being an "all or nothing" event?
The transaction/rollback sequence then repeats until the configured number of rollbacks are performed. You can see a counter (maintained by the org.jboss.soa.esb.samples.quickstart.jmstransacted.test.DBInsertAction class) of the number of times the action is attempted.
The printMessage action prints the message to the log:
2010-04-07 09:26:35,340 INFO [STDOUT] (WorkManager(2)-9) JMS Transacted Quickstart entered. Message body:And the insertDBAction action writes the row to the database - note the counter value of [1]:
2010-04-07 09:26:35,340 INFO [STDOUT] (WorkManager(2)-9) [Hello Transacted JMS World]].
2010-04-07 09:26:35,415 INFO [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.DBInsertAction] (WorkManager(2)-9) Successfully inserted [Hello Transacted JMS World] counter[1]] into jms_transacted_tableBut wait! Then our exception is raised:
2010-04-07 09:26:35,488 ERROR [org.jboss.resource.adapter.jms.inflow.JmsServerSession] (WorkManager(2)-9) Unexpected error delivering message delegator->JBossMessage[5204569273565185]:PERSISTENT, deliveryId=0The JMS/JCA adapter rolls back the transaction, and the message is delivered again, and another row is written to the database. The counter is incremented to [2] this time and the exception is raised again.
java.lang.IllegalStateException: [Throwing Exception to trigger a transaction rollback]
at org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction.process(ThrowExceptionAction.java:75)
2010-04-07 09:26:36,435 INFO [STDOUT] (WorkManager(2)-10) JMS Transacted Quickstart entered. Message body:When the rollback counter number is reached, the action pipeline completes and the transaction commits so that the row is finally written to the database. The following messages are written to the servler.log:
2010-04-07 09:26:36,435 INFO [STDOUT] (WorkManager(2)-10) [Hello Transacted JMS World]].
2010-04-07 09:26:36,438 INFO [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.DBInsertAction] (WorkManager(2)-10) Successfully inserted [Hello Transacted JMS World] counter[2]] into jms_transacted_table
2010-04-07 09:26:36,442 ERROR [org.jboss.resource.adapter.jms.inflow.JmsServerSession] (WorkManager(2)-10) Unexpected error delivering message delegator->JBossMessage[5204569273565185]:PERSISTENT, deliveryId=1
java.lang.IllegalStateException: [Throwing Exception to trigger a transaction rollback]
at org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction.process(ThrowExceptionAction.java:75)
This sequence repeats (5) times until the counter is passed and the row is actually written to the database:
2010-04-07 09:26:40,489 INFO [STDOUT] (WorkManager(2)-14) JMS Transacted Quickstart entered. Message body:Now, as I mentioned in the beginning of this post, I work as a software QE engineer. One of the basic tenents of software QE is to never trust anything. Let's take a closer look at what happens when the quickstart runs.
2010-04-07 09:26:40,489 INFO [STDOUT] (WorkManager(2)-14) [Hello Transacted JMS World]].
2010-04-07 09:26:40,490 INFO [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.DBInsertAction] (WorkManager(2)-14) Successfully inserted [Hello Transacted JMS World] counter[6]] into jms_transacted_table
2010-04-07 09:26:40,490 INFO [STDOUT] (WorkManager(2)-14) JMS Transacted Quickstart processed successfully. Message body:
2010-04-07 09:26:40,490 INFO [STDOUT] (WorkManager(2)-14) [Hello Transacted JMS World]].
Server logs are goldmines for gathering debugging information. Let's restart the server, but this time set the server logging level to DEBUG:
./run.sh -Djboss.server.log.threshold=DEBUGAnd then rerun the quickstart. The server.log will be much more verbose this time, but if you look closely, you can see the rollback counter in action:
grep nr-of-rollbacks server.logBefore we move on, let's look at one more thing and verify that only one row is actually written to the database. Now, the quickstart is very tidy and always cleans up after itself by removing rows from the database. Let's disable this cleanup by removing this target from the quickstart's build.xml file:
2010-04-07 23:14:19,776 DEBUG [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction] (WorkManager(2)-11) rollbackCounter [0], nr-of-rollbacks [5]
2010-04-07 23:14:20,804 DEBUG [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction] (WorkManager(2)-12) rollbackCounter [1], nr-of-rollbacks [5]
2010-04-07 23:14:21,819 DEBUG [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction] (WorkManager(2)-13) rollbackCounter [2], nr-of-rollbacks [5]
2010-04-07 23:14:22,833 DEBUG [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction] (WorkManager(2)-14) rollbackCounter [3], nr-of-rollbacks [5]
2010-04-07 23:14:23,848 DEBUG [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction] (WorkManager(2)-15) rollbackCounter [4], nr-of-rollbacks [5]
2010-04-07 23:14:24,858 DEBUG [org.jboss.soa.esb.samples.quickstart.jmstransacted.test.ThrowExceptionAction] (WorkManager(2)-16) rollbackCounter [5], nr-of-rollbacks [5]
cat build.xml | grep truncate
And, let's start the server again, but with one additional command line option: sh ./run.sh -Djava.awt.headless=false
Why "headless=false?" After we run the quickstart again, the row that was written to the database should still be there. We'll look at it with the HSQL DB Manager service. This service requires the server to not be started in its default "headless" mode, as the service opens up a Java UI application. We invoke the DB Manager from the JMX console - see screenshot - the service tag is: database=jbossesb,service=Hypersonic.
Once the DM Manager application UI launches, we just connect to the "JmsTransactedDB" database - see screenshot - and execute a query to return all rows in the "jms_transacted_table" - see screenshot.
And, there is our one record!
Before we move on, it's worthwhile to note what we did NOT have to do to execute a transaction with the SOA Platform.
We did not have to write transaction specific code. The JBossESB in the Platform did the dirty work for us. All we had to do is to define the .esb application's configuration in its jboss-esb.xml file to include the transaction-relevant properties and reference a JMS/JCA adapter and the Platform did the rest. This is what makes middleware so useful. You could write the code to handle transactions all on your own, but wouldn't you rather concentrate on solving your own business problems instead of build infrastucture plumbing? [9]
Closing Thoughts
The value that using transactions adds to an application is pretty obvious (just keep your bank account in mind). The value added by middleware in implementing transactions should also be obvious. You could write your own code to handle atomicity, consistency, isolation and durability, but, this would not be an easy task, and it would also take away a lot of time from your being able to concentrate on solving your own business logic problems. The SOA Platform's support for transactions over the JBossESB enables you to provide a higher level of reliability and durablity to your service based applications.
References
[1] http://qconlondon.com/london-2010/file?path=/qcon-london-2010/slides/MarkLittle_TransactionsOverUsedOrJustMisunderstood.pdf
[2] Little, Mark, Maron, Jon, Pavlik, Greg. Transaction Processing: Design and Implementation, Upper Saddle RIver, new Jersey: Prentice Hall/Hewlett-Packard Professional Books, 2004.
[3] http://www.redhat.com/docs/en-US/JBoss_SOA_Platform/5.0.0/html-single/Programmers_Guide/index.html#sect-SOA_ESB_Programmers_Guide-What_is_a_Service-InVM_Transport
[4] http://java.sun.com/j2ee/connector/
[5] http://java.sun.com/javaee/technologies/jta/index.jsp
[6] http://www.jboss.org/file-access/default/members/jbossas/freezone/docs/Server_Configuration_Guide/4/html/Connectors_on_JBoss-Configuring_JDBC_DataSources.html
[7] http://www.redhat.com/docs/en-US/JBoss_SOA_Platform/5.0.0/html-single/Programmers_Guide/index.html#sect-SOA_ESB_Programmers_Guide-Configuration-Providers
[8] http://en.wikiquote.org/wiki/Yogi_Berra
[9] http://magazine.redhat.com/2008/03/11/what-is-middleware-in-plain-english-please
Acknowledgments
As always, I want to thank the JBoss SOA Platform team and community (especially Kevin Conner for his timely review input for this blog post).
Wednesday, March 24, 2010
The Value of a Fresh Set of Eyes
The test failed.
I checked and rechecked the results. There were no errors or exceptions. The logs were all clean.
But, the test failed.
I rebooted the server and reran the test.
And the test still failed.
It's not an uncommon situation in software QE to encounter unexpected test failures, but this one was very puzzling. Everything seemed to be functioning correctly, bit the test always failed.
The test in question verified that data could traverse a complex series of actions in the software under test. In the course of being routed through the system, the data would be processed into multiple intermediate forms such as files and messages and then be sent between multiple subsystems over protocols including FTP and JMS.
"Divide and conquer" is an effective debugging technique, so I broke the test into pieces and then verified that each piece was functioning correctly. I checked to ensure that messages were correctly formatted and written to queues. I also checked to ensure that the intermediate work files, each created with a different file type, were created in the correct directory path on the various FTP servers.
And the test still failed.
Finally, after staring at this test for a few hours, I asked a co-worker to take a look at it. After about 5 minutes he laughed and said, "The files on the FTP are in the correct places and have the correct type extensions, but the files are empty." The problem was a matter of timing. The test was trying to read the files before any data was written to them.
In "The Cathedral and the Bazaar,"[1] Eric Raymond observed the principle that "many eyes make shallow bugs." He was referring to how in open source software, a program's code is freely available, and how this openness enables more people to identify and resolve a program's flaws.
It can be difficult to test your own code. After you spend time and invest effort in building a program you can take for granted how, and how well, it operates in certain circumstances. You can look at and simply not see its flaws.
It's a common, and very effective practice on software test teams to rotate assignments. This helps to ensure that no one develops a "stale" view of the software under test or begins to take any potential flaws in the software as an intrinsic part of the the code.
So, maybe we have another principle, "fresh eyes see things that you look right past."
References:
[1] http://www.catb.org/~esr/writings/homesteading/
Special thanks to Martin! Děkuji ti!
I checked and rechecked the results. There were no errors or exceptions. The logs were all clean.
But, the test failed.
I rebooted the server and reran the test.
And the test still failed.
It's not an uncommon situation in software QE to encounter unexpected test failures, but this one was very puzzling. Everything seemed to be functioning correctly, bit the test always failed.
The test in question verified that data could traverse a complex series of actions in the software under test. In the course of being routed through the system, the data would be processed into multiple intermediate forms such as files and messages and then be sent between multiple subsystems over protocols including FTP and JMS.
"Divide and conquer" is an effective debugging technique, so I broke the test into pieces and then verified that each piece was functioning correctly. I checked to ensure that messages were correctly formatted and written to queues. I also checked to ensure that the intermediate work files, each created with a different file type, were created in the correct directory path on the various FTP servers.
And the test still failed.
Finally, after staring at this test for a few hours, I asked a co-worker to take a look at it. After about 5 minutes he laughed and said, "The files on the FTP are in the correct places and have the correct type extensions, but the files are empty." The problem was a matter of timing. The test was trying to read the files before any data was written to them.
In "The Cathedral and the Bazaar,"[1] Eric Raymond observed the principle that "many eyes make shallow bugs." He was referring to how in open source software, a program's code is freely available, and how this openness enables more people to identify and resolve a program's flaws.
It can be difficult to test your own code. After you spend time and invest effort in building a program you can take for granted how, and how well, it operates in certain circumstances. You can look at and simply not see its flaws.
It's a common, and very effective practice on software test teams to rotate assignments. This helps to ensure that no one develops a "stale" view of the software under test or begins to take any potential flaws in the software as an intrinsic part of the the code.
So, maybe we have another principle, "fresh eyes see things that you look right past."
References:
[1] http://www.catb.org/~esr/writings/homesteading/
Special thanks to Martin! Děkuji ti!
Thursday, February 25, 2010
Two new posts (well, reposts) in DZone!
And, here they are:
"From HTTP to the ESB - JBossESB 4.7's New Servlet based HTTP Gateway," http://architects.dzone.com/news/jbossesb-http-gateways
"Message Alerts in JBossESB 4.7," http://soa.dzone.com/news/messagealerts-jbossesb-47
"From HTTP to the ESB - JBossESB 4.7's New Servlet based HTTP Gateway," http://architects.dzone.com/news/jbossesb-http-gateways
"Message Alerts in JBossESB 4.7," http://soa.dzone.com/news/messagealerts-jbossesb-47
Thursday, February 11, 2010
Software Usability - Not Just in the Eye of the Beholder
One of the most important parts of any software test definition is the set of "expected results" for the test. If you cannot define the results that you expect the test to generate, then you make the task of determining whether the test has passed or failed much more difficult. For functional tests, this task is fairly straightforward; does method ABC return a boolean true or false value?
For non-functional tests, the task is more complicated. Performance tests can be measured against a set of benchmarks and security tests can be built to ensure that the system under test does not exhibit any of a set of known vulnerabilities such as the OWASP "top 10" (http://www.owasp.org/index.php/OWASP_Top_Ten_Project).
But, what about usability tests? Is usability purely in the eye of the beholder?
I've always tended to work on the "server side" of software products. So, when I had to think about usability tests, I wanted to find some resources that would help me approach the test planning in an organized, systematic way.
I pretty quickly discovered the work of Jakob Nielsen. Dr. Nielsen has written extensively on the subject of usability and is probably the best known web usability consultants in the world. He has defined the five "quality components of usability goals" as learnability, efficiency, memorability, errors (error rate), satisfaction (http://www.useit.com/alertbox/20030825.html
In addition to his books and papers, he also maintains the http://www.useit.com website. This is a great place to learn about usability testing and to find information that can help you in defining specific usability tests.
When Speed (well, the lack of it) Kills
In my own case, I was concerned about how to quantify user interface performance or responsiveness. Dealing with an unresponsive user interface can quickly ruin a user's impression of your application, even if it is otherwise well designed and error free.
I heard an interesting story on the the radio a couple of days ago about a new launched social engineering site. The site designer was asked, "What was your best day so far?" He answered that their first day being on-line was the best day ("yesterday"). He was then asked, "What was your worst day?" He answered "today." The problem was that the site was quickly overloaded and was presenting users with such long delays that people were abandoning the site.
What I needed for my tests was a set of benchmarks to measure UI responsiveness, and do it in a systematic way, so that my test findings would be quantitative, and not based on my opinion of the UI as being "too slow."
What I found was Dr. Neilsen's benchmark definition - http://www.useit.com/papers/responsetime.html
It really helped me to be able to compare the responsiveness of the UI to a measureable baseline as I was able to avoid discussions such as:
User 1: This UI is slow.
User 2: How slow?
User 3: Too slow!
For non-functional tests, the task is more complicated. Performance tests can be measured against a set of benchmarks and security tests can be built to ensure that the system under test does not exhibit any of a set of known vulnerabilities such as the OWASP "top 10" (http://www.owasp.org/index.php/OWASP_Top_Ten_Project).
But, what about usability tests? Is usability purely in the eye of the beholder?
I've always tended to work on the "server side" of software products. So, when I had to think about usability tests, I wanted to find some resources that would help me approach the test planning in an organized, systematic way.
I pretty quickly discovered the work of Jakob Nielsen. Dr. Nielsen has written extensively on the subject of usability and is probably the best known web usability consultants in the world. He has defined the five "quality components of usability goals" as learnability, efficiency, memorability, errors (error rate), satisfaction (http://www.useit.com/alertbox/20030825.html
In addition to his books and papers, he also maintains the http://www.useit.com website. This is a great place to learn about usability testing and to find information that can help you in defining specific usability tests.
When Speed (well, the lack of it) Kills
In my own case, I was concerned about how to quantify user interface performance or responsiveness. Dealing with an unresponsive user interface can quickly ruin a user's impression of your application, even if it is otherwise well designed and error free.
I heard an interesting story on the the radio a couple of days ago about a new launched social engineering site. The site designer was asked, "What was your best day so far?" He answered that their first day being on-line was the best day ("yesterday"). He was then asked, "What was your worst day?" He answered "today." The problem was that the site was quickly overloaded and was presenting users with such long delays that people were abandoning the site.
What I needed for my tests was a set of benchmarks to measure UI responsiveness, and do it in a systematic way, so that my test findings would be quantitative, and not based on my opinion of the UI as being "too slow."
What I found was Dr. Neilsen's benchmark definition - http://www.useit.com/papers/responsetime.html
- .1 second is about the limit for having the user feel that the system is reacting instantaneously, meaning no special feedback is necessary except to display the result.
- 1.0 second is about the limit for the user's flow of thought to stay uninterrupted, even though the user will notice the delay.
- 10.0 seconds is the limit for keeping the user's attention focused on the dialog. for longer delays, the user will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially important if the response time is likely to be highly variable, since users will then not know what to expect.
It really helped me to be able to compare the responsiveness of the UI to a measureable baseline as I was able to avoid discussions such as:
User 1: This UI is slow.
User 2: How slow?
User 3: Too slow!
Sunday, December 27, 2009
Choosing Kaizen Over Cheez Whiz
The end of December is a good time to reflect on the year that is ending. You can look back on your accomplishments, your victories and defeats, and the wreckage (or just the abandonment) of your new years' resolutions from the previous year. This year, the end of the year has gotten me thinking of....software development and test processes.
One of my original goals in starting this blog was for it to as a resource for "new-to-it" software test engineers and managers. The general type of question that I want the blog to be able to answer is "now what do I do?" What I especially want is to be able suggest were specific actions that the reader can follow, instead providing only general, more abstract advice. These actions are related to three components of a successful software test team:
One specific case where I want the blog to be useful involves the first steps that a new software test manager should perform. I want the blog to be able to help a person in a situation where it's the start of the first day after you've just been appointed software test manager. You have no team, no tools, and no processes in place. As you sit at your desk, you're not only asking yourself "what do I do?", you're asking yourself, "what do I do first?"
I addressed the "people" component aspect of this question in an earlier post to this blog where I discussed the characterIstics of the first person to add to a team. I'll write at a later date on the first tools that you'll want to start using and the best way to use them together. For today, I want to talk about the first process that you want to implement.
"We Own the Process"
For many software engineers (the author included), the word "process" can have a negative connotation, as it conjures up images of having to spend lots of your time doing everything other than writing code. I once worked for a manager who, when he announced at a department meeting that "we own the process" subsequently discovered that we had hidden several jaras of "cheez whiz" in his office. (In case anyone has not heard of "cheez whiz," it's a famous American delicacy. It cannot legally be called cheese as it is so heavily processed, instead, it is described as a "pasturized process cheese food."
That was an unfortunate and extreme incident as "process" really ought not to be a bad word. In order for a software test organization to be successful over the long term, it has to develop a process that can be relied on as a roadmap to lead the team through difficult problems. In fact, the very "process" (pardon the pun) of defining, examining and documenting your software development and test process can itself be a useful task in making your organization more effective. If you force yourself to write down the rules under which your organization operates, you will probably find ways to improve it. (But, this is a subject for a different blog post!) This process also has to be repeatable so that the team's success can be repeatable. And, most importantly of all, the process has to always be a means to an end, where that end is the creation of high quality software, on time and on budget.
To get back to subject of this post, which particular process should the team put into place first? There are several possible answers to this question:
Hey, what about test automation? Well, for the purposes of this discussion, I'm putting test automation in a separate category, sort of like breathing. Every team and every project has to invest in automation. Without automation, you can never achieve a sufficient level of test coverage, and the consistency of test execution and test results to ensure the quality of the software under test. In addition, you'll never be able to put into place an effective continuous integration process without a high degree of test automation. The degree to which automation is essential makes me treat this in a different class than other software test processes.
I think, however, if I had to advise someone as to the first process to set up for a new test team, I would choose "kaizen." [2] In other words, continuous improvement.
A systematic approach to continuous improvement can be effective in any complex human endeavor, whether it's software testing or formula 1 racing, or investment banking. You learn from past mistakes, modify your approach to tasks to incorporate those lessons learned, or, in Scrum terms, you continuously "inspect and adapt" [3].
But, what is it about software testing that makes continuous improvement especially applicable?
The Answer is: Bugs
"We deal in lead." [4]
- Steve McQueen in "The Magnificent Seven"
"We deal in bugs."
- Any software tester
The common denominator that ties all software testing together is that every test has the same ultimate goal; to find an as yet undiscovered bug.[5] When you practice software testing, you spend your time looking for, investigating, recreating, and trying to understand the root cause of software bugs. And, every bug presents an opportunity for some type of design or implementation improvement. What's more, the bugs that look for and miss, also present opportunities for improvement in test design and implementation.[6]
In software testing, you are always inspecting both the software product under test and adapting the test programs and test approach to the current state of that product. Your tests and test plans can never be static, they always have to adapt, and be improved, to meet the current conditions. And, it's important that this improvement be continuous, and not only be a task that you think about at the end of a project. In terms of a software product's development and testing lifecycle, waiting days or weeks to make a change to a test or adapt your test plan to deal with an unexpected set of new bugs.
So, how (specifically) can you implement a process of continuous improvement? Some ways are to:
And, remember, "there is always something left to improve"[7] - No matter how successful your testing is, there will be bugs that you missed. The complexity of software products, coupled with the flexible (or is it "brittle"?) nature of software as a medium, means that changes will be made and bugs will be introduced. So, if you think that that your test plan and tests are "complete," think again. The surest way to fall behind is by standing still!
References
[1] http://swqetesting.blogspot.com/2007/11/in-praise-of-exploratory-testing.html
[2] http://en.wikipedia.org/wiki/Kaizen
[3] http://www.scrumalliance.org/pages/scrum_framework
[4] http://www.rateitall.com/i-854965-we-deal-in-lead-steve-mcqueen-the-magnificent-seven.aspx
[5] http://swqetesting.blogspot.com/2007/11/fundamental-1-goal-of-sw-test-is-to.html
[6] http://swqetesting.blogspot.com/2008_08_01_archive.html
[7] As much as I'd like to take credit for the line "there is always something left to improve," I can't. It's attributed to Ben Hogan (1912-1997), the great American golfer of the 1940's and 1950's. His life story is really fascinating. He was born into poverty, reportedly witnessed his father's suicide at age 7, survived a crippling automobile accident, and after decades of failure and constant practice, made himself into the best professional golfer of his time. He is also without a doubt the greatest self-taught athlete in history. No one, in any sport, has ever worked harder to constantly improve than Hogan. After he retired from competition, he founded a golf club manufacturing company. His products were known for their high level of quality as he would never sell anything of low quality with his name on it. In the early 1990's, I happened to have a broken golf club manufactured by the Hogan company. It was broken in such a way that it could not be repaired, so I sent a letter to the company asking to buy a replacement. I received a personal letter from Mr Hogan, asking me to send him the club. He wanted to examine it himself to understand if it was a manufacturing defect that could be improved on. (He later sent me a free replacement club too!)
(Special thanks to Jirka for the Kaizen inspiration!)
One of my original goals in starting this blog was for it to as a resource for "new-to-it" software test engineers and managers. The general type of question that I want the blog to be able to answer is "now what do I do?" What I especially want is to be able suggest were specific actions that the reader can follow, instead providing only general, more abstract advice. These actions are related to three components of a successful software test team:
- People - who to hire and when to hire them, how to organize them into a team, how to maximize the effectiveness of a team
- Tools - which tools to use, and how to use them, and how to use them in combination, and
- Processes - how to create a flexible, efficient, and repeatable set of procedures
One specific case where I want the blog to be useful involves the first steps that a new software test manager should perform. I want the blog to be able to help a person in a situation where it's the start of the first day after you've just been appointed software test manager. You have no team, no tools, and no processes in place. As you sit at your desk, you're not only asking yourself "what do I do?", you're asking yourself, "what do I do first?"
I addressed the "people" component aspect of this question in an earlier post to this blog where I discussed the characterIstics of the first person to add to a team. I'll write at a later date on the first tools that you'll want to start using and the best way to use them together. For today, I want to talk about the first process that you want to implement.
"We Own the Process"
For many software engineers (the author included), the word "process" can have a negative connotation, as it conjures up images of having to spend lots of your time doing everything other than writing code. I once worked for a manager who, when he announced at a department meeting that "we own the process" subsequently discovered that we had hidden several jaras of "cheez whiz" in his office. (In case anyone has not heard of "cheez whiz," it's a famous American delicacy. It cannot legally be called cheese as it is so heavily processed, instead, it is described as a "pasturized process cheese food."
That was an unfortunate and extreme incident as "process" really ought not to be a bad word. In order for a software test organization to be successful over the long term, it has to develop a process that can be relied on as a roadmap to lead the team through difficult problems. In fact, the very "process" (pardon the pun) of defining, examining and documenting your software development and test process can itself be a useful task in making your organization more effective. If you force yourself to write down the rules under which your organization operates, you will probably find ways to improve it. (But, this is a subject for a different blog post!) This process also has to be repeatable so that the team's success can be repeatable. And, most importantly of all, the process has to always be a means to an end, where that end is the creation of high quality software, on time and on budget.
To get back to subject of this post, which particular process should the team put into place first? There are several possible answers to this question:
- Requirements traceability - In defining tests, you should be able to establish a relationship between the product's requirements and your tests tests. Any gaps represent holes in your test coverage.
- Formal test planning - There's a place for ad hoc or exploratory testing[1] in any project, but it cannot represent the majority of a testing effort. You'll doomed to overlook some critical tests unless you impose some discipline on the process by compiling your test ideas into a plan. The goal of test planning is not to produce a document, but to create define the tests, ensure that you have proper test coverage, mitigate risks, etc. The document is almost a by-product to the act of collecting and defining the information in the document. Like I always tell people, "plan is also a verb."
- Defect tracking - If you find a bug, but have no means to record it, is it still a bug? Obviously, you have to track and record bugs in a presistent data store.
- Test results tracking - Likewise, if you run a test, but do not have the means to record the results for future analysis, and to compare the results with future test runs, then the test is a lot less useful than it could be.
Hey, what about test automation? Well, for the purposes of this discussion, I'm putting test automation in a separate category, sort of like breathing. Every team and every project has to invest in automation. Without automation, you can never achieve a sufficient level of test coverage, and the consistency of test execution and test results to ensure the quality of the software under test. In addition, you'll never be able to put into place an effective continuous integration process without a high degree of test automation. The degree to which automation is essential makes me treat this in a different class than other software test processes.
I think, however, if I had to advise someone as to the first process to set up for a new test team, I would choose "kaizen." [2] In other words, continuous improvement.
A systematic approach to continuous improvement can be effective in any complex human endeavor, whether it's software testing or formula 1 racing, or investment banking. You learn from past mistakes, modify your approach to tasks to incorporate those lessons learned, or, in Scrum terms, you continuously "inspect and adapt" [3].
But, what is it about software testing that makes continuous improvement especially applicable?
The Answer is: Bugs
"We deal in lead." [4]
- Steve McQueen in "The Magnificent Seven"
"We deal in bugs."
- Any software tester
The common denominator that ties all software testing together is that every test has the same ultimate goal; to find an as yet undiscovered bug.[5] When you practice software testing, you spend your time looking for, investigating, recreating, and trying to understand the root cause of software bugs. And, every bug presents an opportunity for some type of design or implementation improvement. What's more, the bugs that look for and miss, also present opportunities for improvement in test design and implementation.[6]
In software testing, you are always inspecting both the software product under test and adapting the test programs and test approach to the current state of that product. Your tests and test plans can never be static, they always have to adapt, and be improved, to meet the current conditions. And, it's important that this improvement be continuous, and not only be a task that you think about at the end of a project. In terms of a software product's development and testing lifecycle, waiting days or weeks to make a change to a test or adapt your test plan to deal with an unexpected set of new bugs.
So, how (specifically) can you implement a process of continuous improvement? Some ways are to:
- Turn every new bug into a new test - This one is easy, whenever you find a new bug, write an automated regression test for it. This will ensure that if the bug is ever re-introduced, you'll be able to catch it.
- Mine the customer support calls for new test ideas, and for missing tests - In the course of preparing your test plan, you should collect input from other product team members. You should also review the problems that actual customers are having. You may find that they are using the product in unexpected ways. I once saw a problem where a telephone voicemail system offered customers a feature where they could have the system call them back. People started to use this as a wakeup service, but the call back processes always ran at lower priority level than incoming call processes. What happened? A lot of people received some very late wakeup calls. ;-)
- Regularly re-review existing plans and tests - Like I said earlier, the plans should be dynamic, not static.
And, remember, "there is always something left to improve"[7] - No matter how successful your testing is, there will be bugs that you missed. The complexity of software products, coupled with the flexible (or is it "brittle"?) nature of software as a medium, means that changes will be made and bugs will be introduced. So, if you think that that your test plan and tests are "complete," think again. The surest way to fall behind is by standing still!
References
[1] http://swqetesting.blogspot.com/2007/11/in-praise-of-exploratory-testing.html
[2] http://en.wikipedia.org/wiki/Kaizen
[3] http://www.scrumalliance.org/pages/scrum_framework
[4] http://www.rateitall.com/i-854965-we-deal-in-lead-steve-mcqueen-the-magnificent-seven.aspx
[5] http://swqetesting.blogspot.com/2007/11/fundamental-1-goal-of-sw-test-is-to.html
[6] http://swqetesting.blogspot.com/2008_08_01_archive.html
[7] As much as I'd like to take credit for the line "there is always something left to improve," I can't. It's attributed to Ben Hogan (1912-1997), the great American golfer of the 1940's and 1950's. His life story is really fascinating. He was born into poverty, reportedly witnessed his father's suicide at age 7, survived a crippling automobile accident, and after decades of failure and constant practice, made himself into the best professional golfer of his time. He is also without a doubt the greatest self-taught athlete in history. No one, in any sport, has ever worked harder to constantly improve than Hogan. After he retired from competition, he founded a golf club manufacturing company. His products were known for their high level of quality as he would never sell anything of low quality with his name on it. In the early 1990's, I happened to have a broken golf club manufactured by the Hogan company. It was broken in such a way that it could not be repaired, so I sent a letter to the company asking to buy a replacement. I received a personal letter from Mr Hogan, asking me to send him the club. He wanted to examine it himself to understand if it was a manufacturing defect that could be improved on. (He later sent me a free replacement club too!)
(Special thanks to Jirka for the Kaizen inspiration!)
Tuesday, December 22, 2009
New post in the JBoss ESB blog
New post in the JBoss ESB blog: http://jbossesb.blogspot.com/2009/12/from-http-to-esb-jboss-esb-47s-new.html
The last one for this year!
The last one for this year!
Subscribe to:
Posts (Atom)