Big Data

Chapter 12. Realtime views

This chapter covers

• The theoretical model of the speed layer
• How the batch layer eases the responsibilities of the speed layer
• Using random-write databases for realtime views
• The CAP theorem and its implications
• The challenges of incremental computation
• Expiring data from the speed layer

Up to this point, our discussion of the Lambda Architecture has revolved around the batch and serving layers—components that involve computing functions over every piece of data you have. These layers satisfy all the desirable properties of a data system save one: low-latency updates. The sole job of the speed layer is to satisfy this final requirement.

Running functions over the entire master dataset—potentially petabytes ...