book

Streaming Architecture

by Ted Dunning, Ellen Friedman

May 2016

Beginner to intermediate

120 pages

2h 41m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Who Should Use This BookWhat Is CoveredConventions Used in This BookSafari® Books OnlineHow to Contact Us
Planes, Trains, and Automobiles: Connected Vehicles and the IoTStreaming Data: Life As It HappensWhere Streaming MattersBeyond Real Time: More Benefits of Streaming ArchitectureEmerging Best Practices for Streaming ArchitecturesHealthcare Example with Data StreamsStreaming Data as a Central Aspect of Architectural Design
A Limited View: Single Real-Time ApplicationKey Aspects of a Universal Stream-based ArchitectureImportance of the Messaging TechnologyChoices for Real-Time AnalyticsApache StormApache Spark StreamingApache FlinkApache ApexComparison of Capabilities for Streaming AnalyticsSummary
Why Microservices MatterWhat Is Needed to Support MicroservicesMicroservices in More DetailDesigning a Streaming Architecture: Online Video Service ExampleA New Design: Infrastructure to Support MessagingImportance of a Universal MicroarchitectureWhat’s in a Name?Why Use Distributed Files and NoSQL Databases?New Design for the Video ServiceSummary: The Converged Platform View
Motivations for KafkaKafka InnovationsKafka Basic ConceptsOrderingPersistenceThe Kafka APIsKafkaProducer APIKafkaConsumer APILegacy APIsKafka Utility ProgramsLoad BalancingMirroringKafka GotchasKafka in Production SettingsLimited Number of Topics and PartitionsManual Balancing of Partitions and LoadNo Inherent Serialization MechanismMirroring DeficienciesSummary
Innovations in MapR StreamsHistory and Context of MapR’s Streaming SystemHow MapR Streams WorksHow to Configure MapR StreamsGeo-Distributed ReplicationMapR Streams Gotchas
Card VelocityFast Response Decision to the Question: “Is It Fraud?”Multiuse Streaming DataScaling Up the Fraud DetectorSummary
StakeholdersDesign GoalsDesign ChoicesOur DesignFollow the DataControl Who Has Access to Stream DataAdvantages of Streams-based Geo-Replication
Benefits of Stream-based ArchitecturesMaking the Transition to Streaming ArchitectureConclusion
Streaming Data TopicsSelected O’Reilly Publications by the Authors

Content preview from Streaming Architecture

Appendix A. Additional Resources

Streaming Data Topics

The following links and recommendations may be helpful in finding out more about streaming design and the technologies that support it:

Apache Kafka

MapR Streams

I Heart Logs: Book by Jay Kreps, committer and PMC member, Apache Kafka (O’Reilly).
Apache Spark Streaming: Project website
Getting Started with Apache Spark: Free interactive eBook by Jim Scott available for download from MapR.
Apache Flink: Project website
“Essential Guide to Streaming-first Processing with Apache Flink”: Blog post by Fabian Hueske and Kostas Tzoumas, committers and PMC members, Apache Flink.
Apache Storm: Project website
Apache Apex: Project website
University of Sheffield Advanced Manufacturing Research Centre with Boeing including Factory 2050: Website

Selected O’Reilly Publications by the Authors

Short books on a variety of big data topics you may find interesting:

Practical Machine Learning: Innovations in Recommendation (February 2014): http://oreil.ly/1qt7riC
Practical Machine Learning: A New Look at Anomaly Detection (June 2014): http://bit.ly/anomaly_detection
Time Series Databases: New Ways to Store and Access Data (October 2014): http://oreil.ly/1ulZnOf
Real-World Hadoop (March 2015): http://oreil.ly/1U4U2fN ...