Streaming Change Data Capture

Chapter 7. Conclusion

As with any new technology, the greatest barrier to successful adoption can be inertia. Perhaps your organization is managing to meet business requirements with traditional extract, transform, and load scripting and/or batch loading without change data capture. Perhaps Sqoop is enabling your new data lake to ingest sufficient data volumes with tolerable latencies and a manageable impact on production database workloads. Or perhaps your CIO grew up in the scripting world and is skeptical of graphical interfaces and automated replication processes.

But we are on a trajectory in which the business is depending more and more on analyzing growing volumes of data at a faster and faster clip. There is a tipping point at which traditional manual bulk loading tools and manual scripting begin to impede your ability to deliver the business-changing benefits of modern analytics. Successful enterprises identify the tipping point before it arrives and adopt the necessary enabling technologies. Change data capture is such a technology. It provides the necessary heartbeat for efficient, high-scale, and nondisruptive data flows in modern enterprise circulatory systems.