Chapter 7. Big Data stack
This chapter covers
- Adding reliability to the data store from chapter 6
- Managing a distributed persistent data store in CoreOS
- Simulating failures in the data system
In this chapter, you’ll build a Big Data aggregation platform that seeds a database with random search queries against Twitter. You’ll build a small corpus of data, make Twitter rate-limit you (while still being a good API citizen), and see how to take care of your mission-critical (although random) data. Your application will function like this:
- Six stateless workers will generate a random word and search for it on the Twitter API.
- The results will be stored in Couchbase.
- Workers will continue to search every 100 ms in parallel until they’re ...
Get CoreOS in Action now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
