Ship New Data as Deployable Units
A large part of working with data involves building systems that can consistently ingest new data and hold it to your governance standards. You’ll need to ensure that new data in your pipelines meets security standards, has its lineage tracked, meets your data governance council’s quality standards, and is mapped to your data catalog. Your data governance work never ends; it scales to meet evolving needs.
For instance, let’s say your job involves creating monthly reports which share data on commodity production in Texas and Oklahoma. Your colleague—the statistician who produces these reports—receives the latest data from their field office on the first Friday of each month. Adding this data to your system involves ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access