AI and Analytics in Production

Chapter 1. Is It Production-Ready?

The future is already here—it’s just not evenly distributed.

William Gibson

Big data has grown up. Many people are already harvesting huge value from large-scale data via data-intensive applications in production. If you’re not yet doing that or not doing it successfully, you’re missing out. This book aims to help you design and build production-ready systems that deliver value from large-scale data. We offer practical advice on how to do this based on what we’ve observed across a wide range of industries.

The first thing to keep in mind is that finding value isn’t just about collecting and storing a lot of data, although that is an essential part of it. Value comes from acting on that data, through data-intensive applications that connect to real business goals. And this means that you need to identify practical actions that can be taken in response to the insights revealed by these data-driven applications. A report by itself is not an action; instead, you need a way to connect the results to value-based business goals, whether internal or customer facing. For this to work in production, the entire pipeline—from data ingestion, through processing and analytic applications to action—must be doable in a predictable, dependable, and cost-effective way.

Note

Big data isn’t just big. It’s much more than just an increase in data volume. When used to full advantage, big data offers qualitative changes as well as quantitative. In aggregate, data ...