Enterprise Data Workflows with Cascading

Appendix A. Troubleshooting Workflows

The following tips are intended to help troubleshoot common issues when people are first working with Cascading. These points are mostly about running the examples in the book, but they apply to Enterprise use cases in general.

Build and Runtime Problems

One of the most frequent and useful tips given to people who are new to Cascading—and to Apache Hadoop in general—is that if your build isn’t working as expected, you may need to delete the local Maven repo.

On a Linux or Mac OS X laptop, that purge is handled by:

$ rm -rf ~/.m2

The build systems mentioned in this book—Gradle, Leiningen, SBT—all depend on Maven under the hood. Unfortunately, sometimes Maven gets stuck. Purge its local repository, and then run your build again.

Another common issue with builds is that the Hadoop distribution—or other included JARs—has a dependency conflict with the Cascading artifacts in the Maven repo that you’re using. For example, most of the builds shown in this book require cascading-core and cascading-hadoop for compile-time dependencies. The builds that include unit tests will also depend on cascading-test, junit, etc. Depending on your deployment environment, some artifacts may need to be excluded, e.g., logging.

Other typical problems encountered include the following:

• Using Java 7—should use Java 6 instead
• Using a Hadoop version higher than 1.x—see the Cascading compatibility matrix
• Installing Hadoop but not in “standalone” mode
• Running Hadoop atop Cygwin ...