Parameterized tests with exception case

It is quite likely that some of you have to write a functionality that can be tested with parameterized tests. It can be some form of translation service or normalisation of input values into a format “understandable” by a system. Couple of days ago I wrote a service (Java class) that translates filtering operators from input request into a format known by queried database system. So a unit test of this functionality written in JUnit framework could look like this:

@RunWith(Parameterized.class)
public class OperatorNormaliserTest {
private String queryOperator;
private String expectedOperator;
public OperatorNormaliserTest(String queryOperator, String expectedOperator) {
this.queryOperator = queryOperator;
this.expectedOperator = expectedOperator;
}
@Parameters
public static Collection<Object[]> data() {
Object[][] data = new Object[][] { { "=", "eq" }, { "<", "lt" }, { ">", "gt" } };
return Arrays.asList(data);
}
@Test
public void shouldNormaliseOperatorToDatabaseNativeForm() {
OperatorNormaliser normaliser = new OperatorNormaliser();
String nativeOperator = normaliser.normalise(queryOperator);
assertThat(nativeOperator).isEqualTo(expectedOperator);
}
}

And here is the implementation of the functionality in Java:

public class OperatorNormaliser {
private final Map<String, String> operatorsMap;
public OperatorNormaliser() {
this.operatorsMap = Maps.newHashMap();
this.operatorsMap.put("=", "eq");
this.operatorsMap.put("<", "lt");
this.operatorsMap.put(">", "gt");
}
public String normalise(String queryOperator) {
return operatorsMap.get(queryOperator);
}
}

So far so good. The test case is clean and looking at it you know what it is meant for. Frankly speaking it works nice.

Next, we’d like to add tests that would ensure that the input parameter contains text, i.e. it is not null nor empty string (filled with whitespace characters only). In case of blank input a proper exception should be thrown and this is situation we should test as well. Here is the code that checks input argument:

Preconditions.checkArgument(StringUtils.isNotBlank(queryOperator),
"query operator cannot be blank");

How we could organize tests in such case? Having parameterized tests of one class slightly complicates our situation, but luckily we have more then one possible solution here.

  1. We can simply create another test class for this specific test case, it would look like this:
     @Test(expected = IllegalArgumentException.class)
public void shouldNormalisingFailsForBlankQueryOperator() {
OperatorNormaliser normaliser = new OperatorNormaliser();
normaliser.normalise("");
}

Having two test classes for the same class can be misleading. Which one contain what tests? How both classes should be named so it is clear what their content is and what class they acctualy test? This can be problematic I think.

  1. Another solution to this scenario is to provide parameterized data “manually” in a loop inside the standard test and another one (like the one above) for the exception case.
public class OperatorNormaliserTest {
@Test(expected = IllegalArgumentException.class)
public void shouldNormalisingFailsForBlankQueryOperator() {
OperatorNormaliser normaliser = new OperatorNormaliser();
normaliser.normalise("");
}
@Test
public void shouldNormaliseOperatorToDatabaseNativeForm() {
Map<String, String> testData = ImmutableMap.of("=", "eq"
"<", "lt",
">", "gt");
OperatorNormaliser normaliser = new OperatorNormaliser();
for(Entry<String, String> dataEntry : testData.entrySet()) {
String queryOperator = dataEntry.getKey();
String expectedOperator = dataEntry.getValue();
String nativeOperator = normaliser.normalise(queryOperator);
assertThat(nativeOperator)
.describedAs("%s should be normalised into %s", queryOperator, expectedOperator)
.isEqualTo(expectedOperator);
}
}
}

The thing I don’t like here is this handmade provisioning of test data. Having ready and well working runner provided by JUnit library we shouldn’t look for such solutions.

  1. Going back to Parameterized runner we can introduce a third, new parameter in our test data.
@RunWith(Parameterized.class)
public class OperatorNormaliserTest {
@Rule
public ExpectedException exception = ExpectedException.none();
private String queryOperator;
private String expectedOperator;
private Class errorClass;
public OperatorNormaliserTest(String queryOperator, String expectedOperator, Class errorClass) {
this.queryOperator = queryOperator;
this.expectedOperator = expectedOperator;
this.errorClass = errorClass;
}
@Parameters
public static Collection<Object[]> data() {
Object[][] data = new Object[][] { { "=", "eq", null }, { "<", "lt", null },
{ ">", "gt", null }, { "", "", IllegalArgumentException.class} };
return Arrays.asList(data);
}
@Test
public void shouldNormaliseOperatorToDatabaseNativeForm() {
if (errorClass != null) {
exception.expect(errorClass);
}
OperatorNormaliser normaliser = new OperatorNormaliser();
String nativeOperator = normaliser.normalise(queryOperator);
assertThat(nativeOperator).isEqualTo(expectedOperator);
}
}

While, in this case scenario, we have all test cases in a single class introducing additional parameter to handle single exception case may be useless. The code looks clattered at least and the flow of the test isn’t as clear as it should be.

  1. The last way we could handle with this case is Enclosed runner provided in JUnit library. Thanks to this solution we can introduce two inner test classes where the first one tests our translation service using parameterized data, and the second one contains test for an exception case.
@RunWith(Enclosed.class)
public class OperatorNormaliserTest {
@RunWith(Parameterized.class)
public static class SuccessfulNormalisingTest {
private String queryOperator;
private String expectedOperator;
public SuccessfulNormalisingTest(String queryOperator, String expectedOperator) {
this.queryOperator = queryOperator;
this.expectedOperator = expectedOperator;
}
@Parameters
public static Collection<Object[]> data() {
Object[][] data = new Object[][] { { "=", "eq" }, { "<", "lt" }, { ">", "gt" } };
return Arrays.asList(data);
}
@Test
public void shouldNormaliseOperatorToDatabaseNativeForm() {
OperatorNormaliser normaliser = new OperatorNormaliser();
String nativeOperator = normaliser.normalise(queryOperator);
assertThat(nativeOperator).isEqualTo(expectedOperator);
}
}
public static class FailingNormalisingTest {
@Rule
public ExpectedException exception = ExpectedException.none();
@Test
public void shouldNormalisingFailsForBlankQueryOperator() {
exception.expect(IllegalArgumentException.class);
OperatorNormaliser normaliser = new OperatorNormaliser();
normaliser.normalise("");
}
}
}

I have to admit this solution is my favorite one. We have all tests of the class in one place and we’re using standard JUnit mechanisms here. Naming of classes (especially the inner ones) isn’t a problem this time as well – the name of every internal test class provides short information only: whether tests contained are about successful cases or not. We don’t have to bother with providing information about which part of the system they tests; this is the task of the name of the top class that serves as container of the inner ones.

Unfortunately the Enclosed runner is in experimental package (it can be seen as a drawback of this solution). Hopefully in the future releases of the framework the runner will be moved to the official runners package.

  1 comment for “Parameterized tests with exception case

  1. Nicholas S Drone
    September 27, 2016 at 2:29 pm

    Thank you for this example it is truly helpful

Leave a Reply