O'Reilly logo

CoreOS in Action: Running Applications on Container Linux by Matt Bailey

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 7. Big Data stack

This chapter covers

  • Adding reliability to the data store from chapter 6
  • Managing a distributed persistent data store in CoreOS
  • Simulating failures in the data system

In this chapter, you’ll build a Big Data aggregation platform that seeds a database with random search queries against Twitter. You’ll build a small corpus of data, make Twitter rate-limit you (while still being a good API citizen), and see how to take care of your mission-critical (although random) data. Your application will function like this:

  1. Six stateless workers will generate a random word and search for it on the Twitter API.
  2. The results will be stored in Couchbase.
  3. Workers will continue to search every 100 ms in parallel until they’re ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required