Liz Douglass

Posts Tagged ‘JUnit4ClassRunner

The same, but different

leave a comment »

Recently I’ve been using JUnit. In keeping with the most up to date libraries, we upgraded our project this week to JUnit 4.4 to JUnit 4.5 and that’s where we encountered a hiccup….

In our project, we are interested in doing specific things if a test has failed. For the purposes of demonstration, lets suppose that we want to output the test name to the console when a test fails (using a method annotated with @After). How do we know whether a test has failed or not? We can access this information by using a custom test runner that extends the JUnit4ClassRunner (in JUnit 4.4), like this one …

public class MyTestRunner
   extends JUnit4ClassRunner
{
   private static String
     failureMessage = null;

   public static String getFailureMessage() {
      return failureMessage;
   }

   public MyTestRunner(Class klass)
      throws InitializationError
   {
      super(klass);
   }

   public void run(final RunNotifier notifier) {
      failureMessage = "";
      notifier.addListener(new FailureListener());
      super.run(notifier);
   }

   private class FailureListener
      extends RunListener {

      public void testFailure
        (Failure failure) throws Exception {
           failureMessage = failure.getTestHeader();
        }
      }
}

Our runner (above) overrides the run method (only) so that we can add a FailureListener to the RunNotifier. As per the observer pattern, our notifier will call the testFailure method if the test fails, thereby setting the failureMessage. We can access the failure message from our @After method like this…

@RunWith(MyTestRunner.class)
public class MyClassTest
{
   @After
   public void printRunStatusMessage()
   {
      if (!MyTestRunner.getFailureMessage().equals(""))
         System.out.println(MyTestRunner.getFailureMessage());
      else
         System.out.println("Test did not fail");
   }

   @Test
   public void shouldAssertTruth() throws Exception
   {
      assertEquals(true, false);
   }
}

Note that our test class is annotated with @RunWith(MyTestRunner.class) – meaning that we are running the tests using our custom runner. If we go ahead and run the shouldAssertTruth test we get this line printing to the console “shouldAssertTruth(com.thoughtworks.blogdemo.test.MyClassTest)”. Ta da!

In the migration to JUnit 4.5, we discovered that JUnit4ClassRunner is now deprecated. It has been replaced with the new BlockJUnit4ClassRunner. According to the JUnit API documentation the BlockJunit4ClassRunner “should have exactly the same behavior as the old test class runner (JUnit4ClassRunner)”. By simply substituting the old runner class with the new one and re-executing the shouldAssertTruth test we get (drumroll) “Test did not fail” output to the console… hmm, that’s not expected….

In JUnit 4.4 there are calls to an addFailure method scattered throughout the code that executes the @Before, @Test and @After annotated methods for each test. This is what the addFailure method looks like:

protected void addFailure(Throwable e) {
   fNotifier.fireTestFailure
      (new Failure(fDescription, e));
}

This method lets us know immediately when a test has failed and is the reason why our custom runner worked so well. In JUnit 4.5 we now have the concept of Statements, and without going into detail about them, JUnit now has a RunAfters class. This class is used when we want to run a test and execute its associated @After methods. The class looks like this…

public class RunAfters extends Statement {
   private final Statement fNext;
   private final Object fTarget;
   private final List fAfters;

   public RunAfters(Statement next,
      List afters,
      Object target) {
         fNext= next;
         fAfters= afters;
         fTarget= target;
   }

   @Override
   public void evaluate() throws Throwable {
      List fErrors = new ArrayList();
      fErrors.clear();
      try {
         fNext.evaluate();
      } catch (Throwable e) {
      fErrors.add(e);
      } finally {
         for (FrameworkMethod each : fAfters)
            try {
               each.invokeExplosively(fTarget);
            }
            catch (Throwable e) {
               fErrors.add(e);
            }
         }
      if (fErrors.isEmpty())
         return;
      if (fErrors.size() == 1)
         throw fErrors.get(0);
      throw new MultipleFailureException(fErrors);
   }
}

We can see in the evaluate method that any errors that are thrown during the executing of the test are collected in fErrors. fErrors is used at the end of the evaluate method to create and throw a new MultipleFailureException, which will notify our FailureListener. The crucial part for us is that despite an error being thrown during the execution of a test, the @After method will always be attempted in the finally block, and at that point we are blissfully unaware that the test has failed. Hmph.

Our solution, the quickest one to restore functionality (and not use anything deprecated), has been to create a custom RunAfters class that notifies our FailureListener of any errors in the catch block (see above). It seems less than ideal to have to do this, especially since the changes to the RunAfters class are so minor. Perhaps the next JUnit version will do something else…

Written by lizdouglass

September 6, 2008 at 6:16 am