Wednesday, April 10, 2013

Eclipse Remote Debugging an SWTBot Test

Debugging a failing automated GUI test can be difficult. It's generally not an efficient debugging technique to sit staring at your computer there watching the test manipulate the UI until it crashes. (Although, it can be fun to see the reactions of co-workers as you sit with your arms folded while your computer seems to be working on its own.  ;-)

I had a problem recently in debugging a test for an eclipse plugin, but, luckily a couple of co-workers were able to point me toward a solution. 

The tests in question were written using SWTBot (http://www.eclipse.org/swtbot/). SWTBot is a great open source testing framework that provides a layer of abstraction through its API to facilitate automated SWT and eclipse plugin test development. SWTBot's API makes it easy to write automated tests that exercise the UI and provide pass/fail information through its implementation of assertions.  

My test was failing, but it was not immediately obvious if the problem was a bug in the software under test, a bug in (gasp!) the test code, or maybe even a bug in the test framework. After staring at the code for a while, I fell into the trap of running the test over and over, while I watched the UI. After a few attempts it dawned on me that this approach was crazy. I might as well have been watching old movies on TV. 

What I needed to do is use a debugger to stop the test's execution while I examined the UI.
The problem was that while I could have used a debugger in Eclipse, I wanted to be able to run the test in the same unattended configuration (running under maven from the CLI) that it would have to use when it was run in our test framework. But, at the same time, I also wanted to be able to manipulate the program and the UI and access its source code through a debugger in eclipse.

The test had to be run unattended with maven -  but, how could I do this and use the debugger? 

The answer was to use remote debugging. Before looking at how this works, let's look at some background on Java debugging in general.

Java Debugging

The place to begin is with the Java Platform Debugger Architecture (JPDA, http://docs.oracle.com/javase/6/docs/technotes/guides/jpda/architecture.html)

The JPDA provides a multi-layer debugging architecture. There are (3) elements involved:
  • The debugger
  • The process being debugged (the "debuggee")
  • The channel over which the debugger and debuggee communicate 
Each element makes use of one of the Java APIs provided by the JPDA:
  • The debugger uses the Java Debug Interface (JDI). The JDI defines a high level interface that can be used to program a debugger.
  • The debuggee uses the Java VM Tool Interface (JVM TI). The JVM TI  defines the debugging services, such as inspecting the state of a running application, that  a JVM provides.
  • The communications channel makes use of the Java Debug Wire Protocol (JDWP). The JDWP defines the communications protocol (requests, messages, etc.) between the debugger and debuggee.
I referred to "remote" debugging a minute ago. This is the case even though the test being debugged was completely running on a local system. What happens is that to debug the test, an eclipse Remote Java Application Debug configuration is defined and configured to listen to the JVM over a specified port.

The JDWP communications channel is the vehicle that get used to connect the debuggee (which in this case is the test running under maven) to the debugger (which in this case is the Eclipse debugger).

Creating and remote debugging an SWTBot project turns out to be a simple 4-step process:

Step 1 - Creating a New SWTBot Plugin Project, Converting it to a Maven Project

Creating a new SWTBot plugin project in Eclipse is as easy as, well, installing SWTBot into Eclipse and then creating a new project.

Let's start by creating a new SWTBot project. Now, since we want to debug a SWTBot test, we'll include buggy test class in the project. Here's the code for our buggy little test:

package simple;
import org.eclipse.swtbot.eclipse.finder.SWTWorkbenchBot;
import org.eclipse.swtbot.swt.finder.junit.SWTBotJunit4ClassRunner;
import org.eclipse.swtbot.swt.finder.widgets.SWTBotShell;
import org.junit.Test;
import org.junit.runner.RunWith;
 
@RunWith(SWTBotJunit4ClassRunner.class)
public class BuggyTest {
 
    @Test
    public void canCreateANewJavaProject() throws Exception {

        SWTWorkbenchBot bot;
        bot = new SWTWorkbenchBot();
        bot.viewByTitle("Welcome").close();

        bot.menu("File").menu("New").menu("Project...").click();
 
        SWTBotShell shell = bot.shell("New Project");
        shell.activate();
        bot.tree().expandNode("Java").select("Java Project");
        bot.button("Next >").click();
 
        bot.textWithLabel("Project nname:").setText("TestProject");   
        // oops - it should be "name" not "nname"
        bot.button("Finish").click();
    }
}

Since we want to be able to run the project with Maven outside of Eclipse, the project then has to be converted to a Maven project. This conversion is also simple from within Eclipse. Just right-click on the project name, then select Configure->Convert to Maven Project

Before we move on, we have to configure our Maven project to download both SWTBot and to configure an Eclipse Test Platform to be used to run the test. Luckily, this is easy to handle in the project's pom.xml file:

 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>simpleSWTBot</groupId>
  <artifactId>simpleSWTBot</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>eclipse-test-plugin</packaging>

  <properties>
    <tycho-version>0.11.0-SNAPSHOT</tycho-version>
  </properties>

  <repositories>
   <repository>
     <id>juno</id>
     <layout>p2</layout>
     <url>http://download.eclipse.org/releases/juno</url>
   </repository>
   <repository>
     <id>swtbot</id>
     <layout>p2</layout>
     <url>http://download.eclipse.org/technology/swtbot/releases/2.1.0/</url>
   </repository>
  </repositories>
  
    <pluginRepositories>
    <pluginRepository>
        <id>sonatype</id>
        <url>https://repository.sonatype.org/content/repositories/snapshots/</url>
        <snapshots>
           <enabled>true</enabled>
        </snapshots>
     </pluginRepository>
  </pluginRepositories>

  <build>
    <plugins>
      <plugin>
        <groupId>org.sonatype.tycho</groupId>
        <artifactId>tycho-maven-plugin</artifactId>
        <version>${tycho-version}</version>
        <extensions>true</extensions>
      </plugin>
      <plugin>
        <groupId>org.sonatype.tycho</groupId>
        <artifactId>target-platform-configuration</artifactId>
        <version>${tycho-version}</version>
        <configuration>
          <resolver>p2</resolver>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.sonatype.tycho</groupId>
        <artifactId>maven-osgi-test-plugin</artifactId>
        <version>${tycho-version}</version>
        <configuration>
          <useUIHarness>true</useUIHarness>
          <useUIThread>false</useUIThread>
          <product>org.eclipse.sdk.ide</product>
          <application>org.eclipse.ui.ide.workbench</application>
          <dependencies>
            <dependency>
              <type>p2-installable-unit</type>
              <artifactId>org.eclipse.sdk.ide</artifactId>
              <version>0.0.0</version>
            </dependency>
           </dependencies>
        </configuration>
      </plugin>
    </plugins>  
  </build>
</project>

It's worthwhile to review a couple of elements in the pom.xml file as the file, while small, enables Maven to do quite a bit:
  • The first repository definition enables Maven to download the version of Eclipse (Juno) artifacts that we'll use in the test, and the second repository enables Maven to download the SWTBot artifact that will be used. The Eclipse p2 provisioning system (http://www.eclipse.org/equinox/p2/) performs the downloads and installations.
  • The Tycho plugins (http://www.sonatype.org/tycho) perform the actual building of the eclipse plugins that constitute the test and the extensions to Eclipse for SWTBot.
The other project file that we have to edit is the MANIFEST.MF file. All we have to do here is to ensure bundle version matches that defined in the pom.xml file:

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: simpleSWTBot
Bundle-SymbolicName: simpleSWTBot;singleton:=true
Bundle-Version: 0.0.1.qualifier
Bundle-ActivationPolicy: lazy
Bundle-Vendor:
Bundle-RequiredExecutionEnvironment: J2SE-1.5
Require-Bundle: org.eclipse.swtbot.go

Now that our project is built and configured, the next step is to execute it with Maven outside of Eclipse.

Step 2 - Running the Test with Maven

Before we try to debug the error that we've built into the test, let's run the test so that our maven repo is fully populated with the necessary Eclipse and SWTBot artifacts. We'll do this first as the first time we run the test Maven's downloading these artifacts this may take several minutes.

Configuring maven to avoid using mirror sites can make this run faster:  -Dtycho.disableP2Mirrors=true

The command to install and run the test is:  mvn clean install

When we run this program, we (eventually - after all the downloads are done) get this predictable error:

Could not find widget matching: (of type 'Text' and with label (with mnemonic 'Project nname:'))
Note that you may also see problems using Java 1.7 - these may be caused by the compression utility that Mavin is using to unpack jar files. If you see errors such as:

[ERROR] Internal error: java.lang.IllegalArgumentException: Comparison method violates its general contract! -> [Help 1]

Then adding this setting to your command can resolve that problem:
-Djava.util.Arrays.useLegacyMergeSort=true

Now, we're all set to run the test with a remote debugger.

The best way to attach a remote debugger is to use the debugPort system property.

(See https://community.jboss.org/wiki/RemoteDebuggingForEclipseTestPlug-inRunningByTycho for details.)

When re-run the test again, we see the following output:

mvn install -DdebugPort=8001
Listening for transport dt_socket at address: 8001

What's happening here is that the debugger is waiting for a remote program to connect to it on port 8001. Make note of the port number (8001) as we'll need to reference that in the Eclipse debug configuration that we'll use to run the test.

Step 3 - Add a Breakpoint to the Test Class

Now, back in Eclipse, let's add a breakpoint to the test class:


We're all set run the remote debugger now.

Step 4 - Creating and Running an Eclipse Debug Configuration

Next, while still in Eclipse, select the test class that we want to execute, select "Debug As," and create a new debug configuration. In the debug configuration, specify that you want to run the test class as a Remote Java Application. Then, fill in the test project, the host is localhost and port 8001 and we're ready to go.


Starting the debugger makes the waiting test execution run and then stop on the first breakpoint. (Eclipse will also ask us if we want to switch to the Debug perspective.)


At this point, the SWTBot thread gets suspended and we can play with Eclipse UI. Lo and behold, there's the bug! We misspelled "name."

Summary

OK, let's recap what we did here. We wanted to debug a failing SWTBot UI test, and have the test run outside of Eclipse with Maven. We converted the SWTBot project into a Maven project, reconfigured the project's pom.xml file so Maven could download the Eclipse and SWTBot resources that it needed to run, ran the test, and connected to it as a remote Java application. And we did all this with only a few mouse clicks in Eclipse!

(Special thanks to Michael Istria and Vlado Pakan for his help in writing this post!)

Information Sources

Posted by at No comments:
Subscribe to: Posts (Atom)