Eclipse Remote Debugging an SWTBot Test

Debugging a failing automated GUI test can be difficult. It's generally not an efficient debugging technique to sit staring at your computer there watching the test manipulate the UI until it crashes. (Although, it can be fun to see the reactions of co-workers as you sit with your arms folded while your computer seems to be working on its own.  ;-)

I had a problem recently in debugging a test for an eclipse plugin, but, luckily a couple of co-workers were able to point me toward a solution. 

The tests in question were written using SWTBot (http://www.eclipse.org/swtbot/). SWTBot is a great open source testing framework that provides a layer of abstraction through its API to facilitate automated SWT and eclipse plugin test development. SWTBot's API makes it easy to write automated tests that exercise the UI and provide pass/fail information through its implementation of assertions.  

My test was failing, but it was not immediately obvious if the problem was a bug in the software under test, a bug in (gasp!) the test code, or maybe even a bug in the test framework. After staring at the code for a while, I fell into the trap of running the test over and over, while I watched the UI. After a few attempts it dawned on me that this approach was crazy. I might as well have been watching old movies on TV. 

What I needed to do is use a debugger to stop the test's execution while I examined the UI.

The problem was that while I could have used a debugger in Eclipse, I wanted to be able to run the test in the same unattended configuration (running under maven from the CLI) that it would have to use when it was run in our test framework. But, at the same time, I also wanted to be able to manipulate the program and the UI and access its source code through a debugger in eclipse.

The test had to be run unattended with maven -  but, how could I do this and use the debugger? 

The answer was to use remote debugging. Before looking at how this works, let's look at some background on Java debugging in general.

Java Debugging

The place to begin is with the Java Platform Debugger Architecture (JPDA, http://docs.oracle.com/javase/6/docs/technotes/guides/jpda/architecture.html)

The JPDA provides a multi-layer debugging architecture. There are (3) elements involved:

• The debugger
• The process being debugged (the "debuggee")
• The channel over which the debugger and debuggee communicate 

Each element makes use of one of the Java APIs provided by the JPDA:

• The debugger uses the Java Debug Interface (JDI). The JDI defines a high level interface that can be used to program a debugger.
• The debuggee uses the Java VM Tool Interface (JVM TI). The JVM TI  defines the debugging services, such as inspecting the state of a running application, that  a JVM provides.
• The communications channel makes use of the Java Debug Wire Protocol (JDWP). The JDWP defines the communications protocol (requests, messages, etc.) between the debugger and debuggee.

I referred to "remote" debugging a minute ago. This is the case even though the test being debugged was completely running on a local system. What happens is that to debug the test, an eclipse Remote Java Application Debug configuration is defined and configured to listen to the JVM over a specified port.

The JDWP communications channel is the vehicle that get used to connect the debuggee (which in this case is the test running under maven) to the debugger (which in this case is the Eclipse debugger).

Creating and remote debugging an SWTBot project turns out to be a simple 4-step process:

Step 1 - Creating a New SWTBot Plugin Project, Converting it to a Maven Project

Creating a new SWTBot plugin project in Eclipse is as easy as, well, installing SWTBot into Eclipse and then creating a new project.

Let's start by creating a new SWTBot project. Now, since we want to debug a SWTBot test, we'll include buggy test class in the project. Here's the code for our buggy little test:

package simple;
import org.eclipse.swtbot.eclipse.finder.SWTWorkbenchBot;
import org.eclipse.swtbot.swt.finder.junit.SWTBotJunit4ClassRunner;
import org.eclipse.swtbot.swt.finder.widgets.SWTBotShell;
import org.junit.Test;
import org.junit.runner.RunWith;
 
@RunWith(SWTBotJunit4ClassRunner.class)
public class BuggyTest {
 
    @Test
    public void canCreateANewJavaProject() throws Exception {

        SWTWorkbenchBot bot;
        bot = new SWTWorkbenchBot();
        bot.viewByTitle("Welcome").close();

        bot.menu("File").menu("New").menu("Project...").click();
 
        SWTBotShell shell = bot.shell("New Project");
        shell.activate();
        bot.tree().expandNode("Java").select("Java Project");
        bot.button("Next >").click();
 
        bot.textWithLabel("Project nname:").setText("TestProject");   

        // oops - it should be "name" not "nname"
        bot.button("Finish").click();
    }
}

Since we want to be able to run the project with Maven outside of Eclipse, the project then has to be converted to a Maven project. This conversion is also simple from within Eclipse. Just right-click on the project name, then select Configure->Convert to Maven Project

Before we move on, we have to configure our Maven project to download both SWTBot and to configure an Eclipse Test Platform to be used to run the test. Luckily, this is easy to handle in the project's pom.xml file:

 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>simpleSWTBot</groupId>
  <artifactId>simpleSWTBot</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>eclipse-test-plugin</packaging>

  <properties>
    <tycho-version>0.11.0-SNAPSHOT</tycho-version>
  </properties>

  <repositories>
   <repository>
     <id>juno</id>
     <layout>p2</layout>
     <url>http://download.eclipse.org/releases/juno</url>
   </repository>
   <repository>
     <id>swtbot</id>
     <layout>p2</layout>
     <url>http://download.eclipse.org/technology/swtbot/releases/2.1.0/</url>
   </repository>
  </repositories>
  
    <pluginRepositories>
    <pluginRepository>
        <id>sonatype</id>
        <url>https://repository.sonatype.org/content/repositories/snapshots/</url>
        <snapshots>
           <enabled>true</enabled>
        </snapshots>
     </pluginRepository>
  </pluginRepositories>

  <build>
    <plugins>
      <plugin>
        <groupId>org.sonatype.tycho</groupId>
        <artifactId>tycho-maven-plugin</artifactId>
        <version>${tycho-version}</version>
        <extensions>true</extensions>
      </plugin>
      <plugin>
        <groupId>org.sonatype.tycho</groupId>
        <artifactId>target-platform-configuration</artifactId>
        <version>${tycho-version}</version>
        <configuration>
          <resolver>p2</resolver>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.sonatype.tycho</groupId>
        <artifactId>maven-osgi-test-plugin</artifactId>
        <version>${tycho-version}</version>
        <configuration>
          <useUIHarness>true</useUIHarness>
          <useUIThread>false</useUIThread>
          <product>org.eclipse.sdk.ide</product>
          <application>org.eclipse.ui.ide.workbench</application>
          <dependencies>
            <dependency>
              <type>p2-installable-unit</type>
              <artifactId>org.eclipse.sdk.ide</artifactId>
              <version>0.0.0</version>
            </dependency>
           </dependencies>
        </configuration>
      </plugin>
    </plugins>  
  </build>
</project>

It's worthwhile to review a couple of elements in the pom.xml file as the file, while small, enables Maven to do quite a bit:

• The first repository definition enables Maven to download the version of Eclipse (Juno) artifacts that we'll use in the test, and the second repository enables Maven to download the SWTBot artifact that will be used. The Eclipse p2 provisioning system (http://www.eclipse.org/equinox/p2/) performs the downloads and installations.

• The Tycho plugins (http://www.sonatype.org/tycho) perform the actual building of the eclipse plugins that constitute the test and the extensions to Eclipse for SWTBot.

The other project file that we have to edit is the MANIFEST.MF file. All we have to do here is to ensure bundle version matches that defined in the pom.xml file:

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: simpleSWTBot
Bundle-SymbolicName: simpleSWTBot;singleton:=true
Bundle-Version: 0.0.1.qualifier
Bundle-ActivationPolicy: lazy
Bundle-Vendor:
Bundle-RequiredExecutionEnvironment: J2SE-1.5
Require-Bundle: org.eclipse.swtbot.go

Now that our project is built and configured, the next step is to execute it with Maven outside of Eclipse.

Step 2 - Running the Test with Maven

Before we try to debug the error that we've built into the test, let's run the test so that our maven repo is fully populated with the necessary Eclipse and SWTBot artifacts. We'll do this first as the first time we run the test Maven's downloading these artifacts this may take several minutes.

Configuring maven to avoid using mirror sites can make this run faster:  -Dtycho.disableP2Mirrors=true

The command to install and run the test is:  mvn clean install

When we run this program, we (eventually - after all the downloads are done) get this predictable error:

Could not find widget matching: (of type 'Text' and with label (with mnemonic 'Project nname:'))

Note that you may also see problems using Java 1.7 - these may be caused by the compression utility that Mavin is using to unpack jar files. If you see errors such as:

[ERROR] Internal error: java.lang.IllegalArgumentException: Comparison method violates its general contract! -> [Help 1]

Then adding this setting to your command can resolve that problem:

-Djava.util.Arrays.useLegacyMergeSort=true

Now, we're all set to run the test with a remote debugger.

The best way to attach a remote debugger is to use the debugPort system property.

(See https://community.jboss.org/wiki/RemoteDebuggingForEclipseTestPlug-inRunningByTycho for details.)

When re-run the test again, we see the following output:

mvn install -DdebugPort=8001
Listening for transport dt_socket at address: 8001

What's happening here is that the debugger is waiting for a remote program to connect to it on port 8001. Make note of the port number (8001) as we'll need to reference that in the Eclipse debug configuration that we'll use to run the test.

Step 3 - Add a Breakpoint to the Test Class

Now, back in Eclipse, let's add a breakpoint to the test class:

We're all set run the remote debugger now.

Step 4 - Creating and Running an Eclipse Debug Configuration

Next, while still in Eclipse, select the test class that we want to execute, select "Debug As," and create a new debug configuration. In the debug configuration, specify that you want to run the test class as a Remote Java Application. Then, fill in the test project, the host is localhost and port 8001 and we're ready to go.

Starting the debugger makes the waiting test execution run and then stop on the first breakpoint. (Eclipse will also ask us if we want to switch to the Debug perspective.)

At this point, the SWTBot thread gets suspended and we can play with Eclipse UI. Lo and behold, there's the bug! We misspelled "name."

Summary

OK, let's recap what we did here. We wanted to debug a failing SWTBot UI test, and have the test run outside of Eclipse with Maven. We converted the SWTBot project into a Maven project, reconfigured the project's pom.xml file so Maven could download the Eclipse and SWTBot resources that it needed to run, ran the test, and connected to it as a remote Java application. And we did all this with only a few mouse clicks in Eclipse!

(Special thanks to Michael Istria and Vlado Pakan for his help in writing this post!)

Information Sources

http://docs.oracle.com/javase/6/docs/technotes/guides/jpda/index.html

http://docs.oracle.com/javase/6/docs/technotes/guides/jpda/jdwpTransport.html

http://www.ibm.com/developerworks/java/library/os-eclipse-javadebug/index.html

https://docs.sonatype.org/display/TYCHO/How+to+run+SWTBot+tests+with+Tycho

http://www.eclipse.org/swtbot/

http://www.eclipse.org/equinox/p2/

http://www.sonatype.org/tycho

https://community.jboss.org/wiki/RemoteDebuggingForEclipseTestPlug-inRunningByTycho

Brno!

Just returned from a short trip to Europe...

All the stuff that you've heard about Prague is true. It truly is one of the places on earth that you have to see at least once in your life. 

But, if you're heading to Prague, you owe it to yourself to go about and hour and a half southeast to Brno. 

It's a great city, a college town, and home to my favorite high-tech company.   ;-)

Brno, Czech Republic
https://plus.google.com/photos/112716013955946373347/albums/5768068278221269489

Prague, Czech Republic
https://plus.google.com/photos/112716013955946373347/albums/5768185127371694209

JBoss' SwitchYard - for the impatient newbie user

Just published a blog post for the SwitchYard project by JBoss:

https://community.jboss.org/en/switchyard/blog/2012/06/22/an-impatient-newbie-user-s-introduction-to-switchyard

SwitchYard is the next generation of SOA - the post shows how to create a simple service based application from scratch using the new SwitchYard graphical editor. The editor is very cool. Even an impatient newbie can use it.  ;-)

Sharing Photographs - with an Open Source Flavor

I just discovered a new photo sharing site today - OpenPhoto - http://theopenphotoproject.org/

This is a new project, so it doesn't have all the features that some other sites do, but it does offer the promise of being able to have more freedom of choice on how and where your photos, and their information, is stored.

It has a clean and simple interface - for example:  http://swqetesting.openphoto.me/photos/list

And, it has one notable feature, it's open source!:  https://github.com/openphoto/frontend

On Kindle too!

And - the JBoss ESB book is on Kindle too:

http://www.amazon.com/JBoss-ESB-Beginners-Guide-ebook/dp/B0071LKMTU/ref=dp_kinw_strp_1?ie=UTF8&m=AG56TWVU5XWC2

The JBossESB Book is officially published!

It's available at Packt now! http://www.packtpub.com/jboss-esb-beginners-guide/book

And at Amazon: http://www.amazon.com/JBoss-ESB-Beginners-Guide-DiMaggio/dp/1849516588/ 

A review of the book is here:  http://rickwagner.blogspot.com/2011/10/book-review-for-jboss-esb-beginners.html

Embracing the Sliding Window of Doubt

We create schedules for many different things but generally for the same reasons - to enable us to determine when in the future a task will be completed, to enable us to allocate time and money and resources to perform that task, and to enable other organizations to coordinate their future actions.


In other words, we create schedules in an attempt to predict the future.

But, why do we get it wrong so often? 


Why is it that, in spite of how precisely we plan our tasks and try to take into account what might go wrong, so many project schedules miss their original target completion date? 


Why did Boston's famous "Big Dig" road and tunnel construction project (http://en.m.wikipedia.org/wiki/Big_Dig_(Boston,_Massachusetts)) slip from being a $3 billion project to a $15 billion project?


Part of the reason may be simply that it is a difficult task to create a schedule that can accurately predict the future. Another part of the reason may be that people, as Frederick Brooks observed in "The Mythical Man-Month," (http://en.wikipedia.org/wiki/The_Mythical_Man-Month) are eternal optimists. We always think that "this time" things will go well, and we won't have any schedule-related problems. As a result, we fail to plan for everything that may possibly "go wrong."
   

But, there may be another reason. Maybe we cause our own imprecision by trying to be too precise.  In other words, maybe we set ourselves up for failure and make schedule "slips" a self-fulfilling prophecy.

Which leads me to a golf story. An historical golf story, that is.

When he was asked to explain his surprising lack of success when he tried to play competitive golf a few years after his retirement at the early age 28, golf great Bobby Jones attributed it to his misguided attempt to achieve a level of precision and consistency greater than he had ever had during his best playing years. Who was Bobby Jones? The Tiger Woods of his era - back when gofers wore ties.

Maybe it's a bit like that with scheduling. Do we doom ourselves to always creating schedules that are missed by trying to achieve a target that is so precise that it can never actually be hit?

I've been thinking lately that a better approach for scheduling a large, long running, and complex project schedule is to not attempt to target a single "hard" and unchanging target completion date (for example, "our project will be completed on May 23rd at 09:00, Eastern Daylight Time"), but rather but rather to target a window of time in which the project will be completed, "slide" that window closer or further away and have it expand and contract based upon your level of confidence according to the current conditions of the project, all in the context of the the current project conditions.

This window can be thought of as the "sliding window of doubt."

OK, this idea of a "sliding window of doubt" raises a couple of questions:

What are the potential advantages of a sliding window approach?

One of the best aspects the sliding window approach is that it of scheduling a project schedule with a sliding window is that it converts the reassessment of the schedule from a reactive to a proactive task. Reassessing the schedule, and possibly sliding the project completion window is not an aberration or exception, that is only done as a reaction to a problem. Instead, it is a ongoing part of the management of the project. These re-assessments will become increasingly more accurate estimate as the project proceeds as the more you learn about the problems that a project faces during its development the better you can predict its future. As a result, you will have more accurate estimates (the window only shrinks when your confidence in the project's ability to meet its schedule increases) based on the re-assessments, and the project team will have constant access to up-to-date schedule information.


Another advantage of the sliding window approach is the flexibility that it provides the project team. The "consumers" of the schedule include the project team members themselves as well as the consumers (other project teams or end user customers) of the project deliverables, as these consumers must create their own schedules, for their own deliverables, all of which depend on your project and its schedule. With a rigid, single-date based schedule, even small changes can effect that date, and cause disruption (or maybe even panic) to dependent project and customer schedules.



Also, by building into the project plan the ability to slide the release window,people who might otherwise delay in providing bad news that would affect the schedule, out of fear of a "shoot the messenger" situation,  may be more inclined to make that news available to the team sooner.


How is a "sliding" window any different from defining a more convention schedule, and then "slipping" the target dates? 


The difference really comes down to what controls the schedule changes.



As we discussed just a moment ago, the constant reassessment of the schedule, where the level of doubt is re-evaluated based on the current project conditions, is a proactive task. Your project team is in more control of its fate with the sliding window approach. Also, remember that these re-assessments are not only performed when there are problems to be resolved, they are also performed when things are going well. Accordingly, while the window can slide out in response to unexpected problems, it can also move in if the project is progressing better than the original plan.


So, to sum up, when you are faced with the challenge of devising a schedule for a large, complex, long-running project, it's worthwhile to consider the "sliding window of doubt" as an approach to scheduling key milestones and release dates. At its core, this approach provides you with the flexibility to refine the schedule over time, and perform proactive schedule reassessments, that will result in the window sliding closer or further away, and expanding or contracting, all based the most up-to-date information available.


We started this discussion by asking - "Why does every project miss its target milestones and completion dates?"


Maybe the the real question that we should ask ourselves is: Why does every project schedule seem to miss its target completion date?


If the dates were rigidly defined, and the project team never re-assessed those dates as the project proceeded and unanticipated problems were encountered, then the dates were probably never achievable in the first place. What the project needed was a sliding window of doubt!


Postscript - the presence of doubt or the absence of confidence?

It's interesting how your occupation can color your view of the world. If you work in engineering (and especially if you work in software testing) you tend to look for the negative side of things. In other words, you look for what might go wrong. People engaged in sales or marketing tend to have a rosier view of things. For example, when I described the sliding window of doubt to a marketing person, she replied:


"Oh, you mean the sliding window of confidence!"


I guess she was correct, as confidence is the inverse of doubt!


(Special thanks to John Graham (http://osmusings.blogspot.com/) for his inspiration for this post.)