O'Reilly logo

Architecting the Industrial Internet by Carla Romano, Robert Stackowiak, Shyam Nath

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

The batch layer

The batch layer is the location of data-management systems storing historical data in our architecture. Incoming data streams from the speed layer commonly land in a NoSQL database. Initially, a variety of NoSQL key-value pair databases were commonly used and scaled through a technique called sharding, the horizontal partitioning of a database across multiple database servers. Some organizations use this type of NoSQL database in PoCs for rapid deployment or as pre-processing engines in production solutions.

In the past few years, Hadoop clusters have gained in popularity as the landing spot and serve as a data lake in the Lambda architecture you saw in the diagram. They can support the large data volumes of historical data ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required