We (the Apache MRUnit team) have just released Apache MRUnit 0.8.1-incubating. Apache MRUnit is an Apache Incubator project. MRUnit is a Java library that helps developers unit test Apache Hadoop MapReduce jobs. Unit testing is a technique for improving project quality and reducing overall costs by writing a small amount of code that can automatically verify the software you write performs as intended. This is considered a best practice in software development since it helps identify defects early, before they're deployed to a production system.
The MRUnit project is actively looking for contributors, even ones brand new to the world of open source software. There are many ways to contribute: documentation, bug reports, blog articles, etc. If you are interested but have no idea where to start, please email brock at cloudera dot com. If you are an experienced open source contributor, the MRUnit wiki explains How you can Contribute.
The new release includes one bug fix and a number of new features including, but not limited to:
- Support Hadoop 0.20 and 0.23 (MRUNIT-31 and MRUNIT-56)
- Improved test failure messages (MRUNIT-15)
- Static driver factories (MRUNIT-43)
Support Hadoop 0.20 and 0.23
In MAPREDUCE-954 Hadoop changed the public org.apache.hadoop.mapreduce API. The MRUnit team in MRUNIT-31 was able to provide one release artifact which supports both versions of the API. MRUNIT-56 provided three maven classifiers, hadoop020, hadoop023, and hadoop100 to provide easy dependency on the correct binary.
Improved test failure messages
Improving the test case failure messages is how I became involved in the project. Before this change you had to look at your console to see what errors had occurred. Now, when a test fails the error message contains an explanation of exactly what failed:
java.lang.RuntimeException: 1 Error(s): (Missing expected output (cat1, 1) at position 0.) at org.apache.hadoop.mrunit.TestDriver.validate(TestDriver.java:194) at org.apache.hadoop.mrunit.MapDriverBase.runTest(MapDriverBase.java:186) at TestWordCount.testMapper(TestWordCount.java:34)
Static driver factories
Static driver factories are an addition which reduce much the weight imposed by Java generics. Before this new feature, to create a driver, say MapDriver, you would have to specify all the generic types:
mapDriver = new MapDriver
Now, the generic types are inferred and you can simply specify:
mapDriver = MapDriver.newMapDriver();
This should provide users with the ability to write simpler code. Additionally, using the MapDriver constructor will continue to work and is fully supported.