5
Ingesting Data
Getting data into a big data platform is a process that typically comes loaded with questions. How long will data onboarding take? How many resources do I need to apply to the project? Do I need the vendors' help to migrate? How much will that cost?...and on and on the questions go. Depending on the size of your ecosystem, getting all the metrics, events, logs, and data flowing in can be a Herculean undertaking that takes months. Then we get into the technical requirements, which are vast: What log formats are accepted? Should we transform the data in the pipeline? How complicated are the schemas? How is the data indexed?
This is why a data ingestion strategy is crucial because it not only helps you make sense of your growing ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access