book

Designing Data-Intensive Applications, 2nd Edition

by Martin Kleppmann, Chris Riccomini

February 2026

Intermediate to advanced

650 pages

22h 2m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Who Should Read This Book?What’s New in the Second Edition?References and Further ReadingConventions Used in This BookO’Reilly Online LearningHow to Contact UsAcknowledgments
Operational Versus Analytical SystemsCharacterizing Transaction Processing and AnalyticsData WarehousingSystems of Record and Derived DataCloud Versus Self-HostingPros and Cons of Cloud ServicesCloud-Native System ArchitectureOperations in the Cloud EraDistributed Versus Single-Node SystemsProblems with Distributed SystemsMicroservices and ServerlessCloud Computing Versus SupercomputingData Systems, Law, and SocietySummary
Case Study: Social Network Home TimelinesRepresenting Users, Posts, and FollowsMaterializing and Updating TimelinesDescribing PerformanceLatency and Response TimeAverage, Median, and PercentilesUse of Response Time MetricsReliability and Fault ToleranceFault ToleranceHardware and Software FaultsHumans and ReliabilityScalabilityDescribing LoadShared-Memory, Shared-Disk, and Shared-Nothing ArchitecturePrinciples for ScalabilityMaintainabilityOperability: Making Life Easy for OperationsSimplicity: Managing ComplexityEvolvability: Making Change EasySummary
Relational Model versus Document ModelThe Object-Relational MismatchNormalization, Denormalization, and JoinsMany-to-One and Many-to-Many RelationshipsStars and Snowflakes: Schemas for AnalyticsWhen to Use Which ModelGraph-Like Data ModelsProperty GraphsThe Cypher Query LanguageGraph Queries in SQLTriple-Stores and SPARQLDatalog: Recursive Relational QueriesGraphQLEvent Sourcing and CQRSDataFrames, Matrices, and ArraysSummary
Storage and Indexing for OLTPLog-Structured StorageB-TreesComparing B-Trees and LSM-TreesMulti-Column and Secondary IndexesKeeping everything in memoryData Storage for AnalyticsCloud Data WarehousesColumn-Oriented StorageQuery Execution: Compilation and VectorizationMaterialized Views and Data CubesMultidimensional and Full-Text IndexesFull-Text SearchVector EmbeddingsSummary
Formats for Encoding DataLanguage-Specific FormatsJSON, XML, and Binary VariantsProtocol BuffersAvroThe Merits of SchemasModes of DataflowDataflow Through DatabasesDataflow Through Services: REST and RPCDurable Execution and WorkflowsEvent-Driven ArchitecturesSummary
Single-Leader ReplicationSynchronous Versus Asynchronous ReplicationSetting Up New FollowersHandling Node OutagesImplementation of Replication LogsProblems with Replication LagReading Your Own WritesMonotonic ReadsConsistent Prefix ReadsSolutions for Replication LagMulti-Leader ReplicationGeographically Distributed OperationSync Engines and Local-First SoftwareDealing with Conflicting WritesCRDTs and Operational TransformationLeaderless ReplicationWriting to the Database When a Node Is DownLimitations of Quorum ConsistencySingle-Leader Versus Leaderless Replication PerformanceDetecting Concurrent WritesSummary
Pros and Cons of ShardingSharding for MultitenancySharding of Key-Value DataSharding by Key RangeSharding by Hash of KeySkewed Workloads and Relieving Hot SpotsOperations: Automatic or Manual RebalancingRequest RoutingSharding and Secondary IndexesLocal Secondary IndexesGlobal Secondary IndexesSummary
What Exactly Is a Transaction?The Meaning of ACIDSingle-Object and Multi-Object OperationsWeak Isolation LevelsRead CommittedSnapshot Isolation and Repeatable ReadPreventing Lost UpdatesWrite Skew and PhantomsSerializabilityActual Serial ExecutionTwo-Phase Locking (2PL)Serializable Snapshot Isolation (SSI)Distributed TransactionsTwo-Phase Commit (2PC)Distributed Transactions Across Different SystemsDatabase-internal Distributed TransactionsSummary
Faults and Partial FailuresUnreliable NetworksThe Limitations of TCPNetwork Faults in PracticeDetecting FaultsTimeouts and Unbounded DelaysSynchronous Versus Asynchronous NetworksUnreliable ClocksMonotonic Versus Time-of-Day ClocksClock Synchronization and AccuracyRelying on Synchronized ClocksProcess PausesKnowledge, Truth, and LiesThe Majority RulesDistributed Locks and LeasesByzantine FaultsSystem Model and RealityFormal Methods and Randomized TestingSummary

LinearizabilityWhat Makes a System Linearizable?Relying on LinearizabilityImplementing Linearizable SystemsThe Cost of LinearizabilityID Generators and Logical ClocksLogical ClocksLinearizable ID GeneratorsConsensusThe Many Faces of ConsensusConsensus in PracticeCoordination ServicesSummary
Batch Processing with Unix ToolsSimple Log AnalysisChain of Commands Versus Custom ProgramSorting Versus In-memory AggregationBatch Processing in Distributed SystemsDistributed FilesystemsObject StoresDistributed Job OrchestrationBatch Processing ModelsMapReduceDataflow EnginesShuffling DataQuery languagesDataFramesBatch Use CasesExtract–Transform–Load (ETL)AnalyticsMachine LearningServing Derived DataSummary
Transmitting Event StreamsMessaging SystemsLog-based Message BrokersDatabases and StreamsKeeping Systems in SyncChange Data CaptureState, Streams, and ImmutabilityProcessing StreamsUses of Stream ProcessingReasoning About TimeStream JoinsFault ToleranceSummary
Data IntegrationCombining Specialized Tools by Deriving DataBatch and Stream ProcessingUnbundling DatabasesComposing Data Storage TechnologiesDesigning Applications Around DataflowObserving Derived StateAiming for CorrectnessThe End-to-End Argument for DatabasesEnforcing ConstraintsTimeliness and IntegrityTrust, but VerifySummary
Predictive AnalyticsBias and DiscriminationResponsibility and AccountabilityFeedback LoopsPrivacy and TrackingSurveillanceConsent and Freedom of ChoicePrivacy and Use of DataData as Assets and PowerRemembering the Industrial RevolutionLegislation and Self-RegulationSummary

Content preview from Designing Data-Intensive Applications, 2nd Edition

About the Authors

Martin Kleppmann is an Associate Professor at the University of Cambridge, UK, where he teaches on distributed systems and cryptographic protocols. The first edition of Designing Data-Intensive Applications in 2017 established him as an authority on data systems, and through his research on distributed systems he helped start the local-first software movement. Previously he was a software engineer and entrepreneur at internet companies including LinkedIn and Rapportive, where he worked on large-scale data infrastructure.

Chris Riccomini is a software engineer, startup investor, and author with 15+ years of experience at PayPal, LinkedIn, and WePay. He runs Materialized View Capital, where he invests in infrastructure startups. He is also the co-creator of Apache Samza and SlateDB, and co-author of The Missing README: A Guide for the New Software Engineer.