O'Reilly logo

Programming MapReduce with Scalding by Antonios Chalkiopoulos

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

TDD for Scalding developers

This section describes an approach to deal with the development process with a testing context. We will work through code examples in Scalding, but the majority of the concepts are also valid in a broader context, in other Scala-based MapReduce frameworks such as Scoobi and Scrunch in particular. We will first describe the testing strategy, and then implement a framework to support this strategy.

Implementing the TDD methodology

The rest of the chapter presents a step-by-step implementation of the TDD methodology. We will use the example presented in the previous chapter, which requires the following actions:

  • Read a logfile and count the number of visits per user per day, where the logfiles contain user URL requests
  • Enrich ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required