Financial Governance for Data Processing in the Cloud
by Amit Duvedi, Balaji Mohanam, Andy Still, Andrew Ash
Chapter 1. Introduction
The advent of cloud service providers such as AWS, Azure, and GCP has changed the world of big data for the better, opening up the ability to build and run a best-of-breed big data–processing system to virtually every company, regardless of size.
The need for large upfront investment in hardware and specialist system administration staff to build and configure on-premises platforms for data processing has been removed as cloud platforms have evolved. The services offered are all on demand, pay-as-you-go services, meaning that investigations and proofs of concept (PoCs) will cost very little or even nothing.
The development of open source data processing engines such as Hadoop, Spark, Kafka, TensorFlow, Presto, and others as industry leading platforms has led to the widespread adoption of data processing and the development of a vibrant technology community.
However, with every revolution comes new challenges. With cloud platforms, although the initial investment is low, it is very easy for costs to get out of control without careful management.
This report provides guidance to effectively govern the costs associated with data processing in the cloud. Looking at the three areas of any successful financial governance plan—cost control, traceability and predictability—the following chapters provide some pointers to the tools, systems, and processes that you can employ to deliver a successful cloud-based data-processing platform with effective financial governance. ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access