While unit testing using MRUnit is very useful, there can be certain integration test scenarios that have to be tested in a cluster environment. MiniYARNCluster of Hadoop YARN is a cluster simulator that we can use to create a testing environment for such integration tests. In this recipe, we'll be using MiniYARNCluster to perform integration testing of the WordCountWithTools MapReduce application.
The source code for the test program used in this recipe is available in the
chapter3\test\chapter3\minicluster\WordCountMiniClusterTest.java file in the Git repository.
We use Gradle as the build tool for our sample code base. If you haven't already done so, please install ...