O'Reilly logo

Hadoop for Finance Essentials by Rajiv Tiwari

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

The lambda architecture

In financial organizations, as we are in the year 2015, business is almost always demanding analytics in real time.

As discussed in the previous chapter, Hadoop can process data in memory using Storm or Spark, but what a business really needs is the ability to combine the full historical dataset on disk with real-time in-memory data.

Lambda architecture addresses this requirement. The concept is very simple, yet powerful: Use both, the batch layer and the speed layer, using a shared serving layer, as shown next:

The lambda architecture

We already know which technologies are suited for batch layer and which ones for speed layer.

  • Incoming data: This ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required