In a large global health services company, streaming data for processing and sharing comes with its own challenges. Data science and analytics platforms need data fast, from relevant sources, to act on this data quickly and share the insights with consumers with the same speed and urgency.
Join Mohammad Quraishi (Cigna) to learn why streaming data architectures are a necessity—Kafka and Hadoop are key. Mohammad outlines architectures centered around the Hadoop Platform and Kafka that were implemented to support a variety of integration and analytics requirements.
- Enabling streaming to and from relational sources and files using custom frameworks that automate and speed up workflows
- Combining the polyglot techniques with Kafka API to support various streaming solutions
- Combining data driven techniques to support consumers through a simple streaming architecture and microservices
- How HBase, Kudu, and Kafka Streams are used to reduce latency between these microservices and frontend application APIs
- Enabling the consumption and sharing of data sources and results using streams
- Enabling Spark Structured Streaming, Flink, and Spark ML on these streams
- Enabling data sync between on-premises data lakes and the cloud
- Supporting cloud native architectures that enable machine learning in the cloud
This session was recorded at the 2019 O'Reilly Strata Data Conference in San Francisco.
- Title: Enabling insights and analytics with data streaming architectures and pipelines using Kafka and Hadoop
- Release date: October 2019
- Publisher(s): O'Reilly Media, Inc.
- ISBN: 0636920339854
You might also like
CompTIA Security+ SY0-601 Exam Cram
CompTIA® Security+ SY0-601 Exam Cram, is the perfect study guide to help you pass the newly …
Data Structures and Algorithms in Java, 2nd Edition
Data Structures and Algorithms in Java, Second Edition is designed to be easy to read and …
Windows Internals Seventh Edition Part 1: System architecture, processes, threads, memory management, and more, Seventh Edition
The definitive guide–fully updated for Windows 10 and Windows Server 2016 Delve inside Windows architecture and …
AWS Certified SysOps Administrator - Associate (SOA-C02) Exam Cram
AWS Certified SysOps Administrator Associate (SOA-C02) Exam Cram is an all-inclusive study guide designed to help …