book

Streaming Architecture

by Ted Dunning, Ellen Friedman

May 2016

Beginner to intermediate

120 pages

2h 41m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Who Should Use This BookWhat Is CoveredConventions Used in This BookSafari® Books OnlineHow to Contact Us
Planes, Trains, and Automobiles: Connected Vehicles and the IoTStreaming Data: Life As It HappensWhere Streaming MattersBeyond Real Time: More Benefits of Streaming ArchitectureEmerging Best Practices for Streaming ArchitecturesHealthcare Example with Data StreamsStreaming Data as a Central Aspect of Architectural Design
A Limited View: Single Real-Time ApplicationKey Aspects of a Universal Stream-based ArchitectureImportance of the Messaging TechnologyChoices for Real-Time AnalyticsApache StormApache Spark StreamingApache FlinkApache ApexComparison of Capabilities for Streaming AnalyticsSummary
Why Microservices MatterWhat Is Needed to Support MicroservicesMicroservices in More DetailDesigning a Streaming Architecture: Online Video Service ExampleA New Design: Infrastructure to Support MessagingImportance of a Universal MicroarchitectureWhat’s in a Name?Why Use Distributed Files and NoSQL Databases?New Design for the Video ServiceSummary: The Converged Platform View
Motivations for KafkaKafka InnovationsKafka Basic ConceptsOrderingPersistenceThe Kafka APIsKafkaProducer APIKafkaConsumer APILegacy APIsKafka Utility ProgramsLoad BalancingMirroringKafka GotchasKafka in Production SettingsLimited Number of Topics and PartitionsManual Balancing of Partitions and LoadNo Inherent Serialization MechanismMirroring DeficienciesSummary
Innovations in MapR StreamsHistory and Context of MapR’s Streaming SystemHow MapR Streams WorksHow to Configure MapR StreamsGeo-Distributed ReplicationMapR Streams Gotchas
Card VelocityFast Response Decision to the Question: “Is It Fraud?”Multiuse Streaming DataScaling Up the Fraud DetectorSummary
StakeholdersDesign GoalsDesign ChoicesOur DesignFollow the DataControl Who Has Access to Stream DataAdvantages of Streams-based Geo-Replication
Benefits of Stream-based ArchitecturesMaking the Transition to Streaming ArchitectureConclusion
Streaming Data TopicsSelected O’Reilly Publications by the Authors

Content preview from Streaming Architecture

Chapter 3. Streaming Architecture: Ideal Platform for Microservices

Over the last decade or so, there has been a strong movement toward a flexible style of building large systems that has lately been called microservices. This trend started earliest at innovative companies such as Google, and many aspects of microservices have since been reinvented at a variety of other companies. Now, among highly successful, fast-evolving companies that include Amazon, LinkedIn, and Netflix, the microservices approach has become more the rule than the exception, at least partly because companies who adopt this style of architecture move faster and compete better.

“Instead of a big monolithic application, where every change is centrally coordinated, the new Netflix app is a series of microservices, each of which can be changed independently.”¹

Yevgeniy Sverdlik

The microservices idea is simple: larger systems should be built by decomposing their functions into relatively simple, single-purpose services that communicate via lightweight and simple techniques. Such services can be built and maintained by small, very efficient teams.

Why Microservices Matter

Microservices are an important trend in building huge systems, primarily because the weak coupling between microservices allows agility even in large organizations. Here’s how.

The evolution of the idea behind microservices started as a bit of a backlash against the complexity of service-oriented architecture (SOA) and the associated enterprise ...