JUnit Exception Testing

Testing code that throws exceptions is as important than testing only the happy path. It may be even more important. Luckily JUnit provides us with a basic toolkit to verify if an exception is thrown. There are some differences in how we can check these exceptions. Below I will list the possible default options provided by JUnit and at the end we’ll be creating our own custom runner.

Expected Exception

The code float temp = 5 / 0; will throw an ArithmeticException because we are not allowed to divide by zero. We can verify if this code throws an exception by adding the expected exception to the expected parameter of the @Test annotation. This parameter takes a subclass of Throwable.

@Test(expected = ArithmeticException.class)
public void exceptionFailTest(){
    float temp = 5 / 0;
}

Note: the above test will pass if any code in the method throws the expected exception. This is ok for small tests, but for longer tests, it is recommend to use the ExpectedException rule, which is described below.

Inspecting Exception Messages

The first approach is only useful for simple use cases, but it lacks the possibility to inspect the expected exception messages or the state of the domain objects after the exception has been thrown. The next section we are addressing these issues for more complex and longer tests.

Try/Catch Idiom

First up is the try/catch idiom. This was introduced in JUnit 3.x. When the exception was thrown we could still make some asserts to check the type of the exception and also the message of the exception and any other asserts after the exception was thrown.

package com.memorynotfound.test;

import org.junit.Test;
import static junit.framework.TestCase.fail;
import static org.hamcrest.CoreMatchers.containsString;
import static org.hamcrest.CoreMatchers.instanceOf;
import static org.hamcrest.MatcherAssert.assertThat;

public class TryCatch {

    @Test
    public void throwsArithmeticException(){
        try {
            float temp = 5 / 0;

            fail("should throw an exception");
        } catch (ArithmeticException e){
            assertThat(e.getMessage(), containsString("/ by zero"));
            assertThat(e, instanceOf(ArithmeticException.class));
        }
    }

}

ExpectedException Rule

Alternatively, there is the ExpectedException rule. This rule not only lets you inspect the message and the exception, but also let you use Matchers, which gives you a bit more flexibility in your tests.

package com.memorynotfound.test;

import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExpectedException;
import static org.hamcrest.CoreMatchers.containsString;

public class TestExpectedExceptionRule {

    @Rule
    public ExpectedException thrown = ExpectedException.none();

    @Test
    public void test_runtime_exception_with_message(){
        thrown.expect(ArithmeticException.class);
        thrown.expectMessage(containsString("zero"));
        thrown.expectMessage("/ by zero");

        float temp = 5 / 0;
    }
}

Custom Exception Runner

Finally when you want total control, you can always implement your custom exception runner. Here is an example how you can create your own runner, which lets you inspect the exceptions in much finer detail.

First we create an annotation, which we register the expected exception and message.

package com.memorynotfound.test;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD})
public @interface ExpectsException {

    Class type();

    String message() default "";
}

Next we create the ExpectsExceptionrunner which will inspect the exception thrown by the method and if things don’t add up, we make sure the test fails accordingly.

package com.memorynotfound.test;

import org.junit.AssumptionViolatedException;
import org.junit.runners.BlockJUnit4ClassRunner;
import org.junit.runners.model.FrameworkMethod;
import org.junit.runners.model.InitializationError;
import org.junit.runners.model.Statement;

public class ExpectsExceptionRunner extends BlockJUnit4ClassRunner {

    public ExpectsExceptionRunner(Class klass) throws InitializationError {
        super(klass);
    }

    @Override
    protected Statement possiblyExpectingExceptions(FrameworkMethod method, Object test, Statement next) {
        ExpectsException annotation = method.getAnnotation(ExpectsException.class);
        if (annotation == null) {
            return next;
        }
        return new ExpectExceptionWithMessage(next, annotation.type(), annotation.message());
    }

    class ExpectExceptionWithMessage extends Statement {

        private final Statement next;
        private final Class expected;
        private final String expectedMessage;

        public ExpectExceptionWithMessage(Statement next, Class expected, String expectedMessage) {
            this.next = next;
            this.expected = expected;
            this.expectedMessage = expectedMessage;
        }

        @Override
        public void evaluate() throws Exception {
            boolean complete = false;
            try {
                next.evaluate();
                complete = true;
            } catch (AssumptionViolatedException e) {
                throw e;
            } catch (Throwable e) {
                if (!expected.isAssignableFrom(e.getClass())) {
                    String message = "Unexpected exception, expected<"
                            + expected.getName() + "> but was <"
                            + e.getClass().getName() + ">";
                    throw new Exception(message, e);
                }

                if (isNotNull(expectedMessage) && !expectedMessage.equals(e.getMessage())) {
                    String message = "Unexpected exception message, expected<"
                            + expectedMessage + "> but was<"
                            + e.getMessage() + ">";
                    throw new Exception(message, e);
                }
            }
            if (complete) {
                throw new AssertionError("Expected exception: "
                        + expected.getName());
            }
        }

        private boolean isNotNull(String s) {
            return s != null && !s.isEmpty();
        }
    }
}

Finally we register the custom runner with our test and add the appropriate exceptions. As we cannot divide by zero, the code below will throw an ArithmeticException.

package com.memorynotfound.test;

import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(ExpectsExceptionRunner.class)
public class TestExceptions {

    @Test
    @ExpectsException(type = ArithmeticException.class, message = "/ by zero")
    public void throwsArrayIndexOutOfBoundsException(){
        float temp = 5 / 0;
    }

}

Conclusion

When working with simple tests the expected parameter in the @Test annotation is very clean and easy to read. When you need more control over the exception, say that you want to inspect the message or the domain models, I recommend you to use the ExpectedException rule, which gives you much finer control over the exception and let’s you use Matchers. When this is not enough, you can still create your own implementation which we showed in the last example.