Designing Big Data Platforms

Introduction

 

The value of big data and processing data to get actionable insights have been a competitive advantage across the world. Organizations are still in a rush to collect, process, and drive values from big data. Designing big data platforms to collect, store, process, and discover big data is a significant challenge from many aspects. Thanks to collaborative push from engineers from different parts of the world and several organizations, we have so many great systems we can use to design a big data platform.

A big data platform consists of many components where we have many alternatives for the same job. Our task is to design a platform that caters to the needs and requirements of the organization. In doing so, we should choose the right tool for the job. Ideally, the platform should adapt, accept, and evolve due to new expectations. The challenge is to design a simple platform while keeping it cost‐efficient in terms of development, maintenance, deployment, and the actual running expense.

In this book, I present many different technologies for the same job. Some of them are already off the shelf, while others are cutting edge. I want to give perspectives on these systems so that we can create solutions that are based on the experience of others. Hopefully, we can all have a better grasp on designing big data platforms after reading this book.