book

Software Mistakes and Tradeoffs

Name: Software Mistakes and Tradeoffs
ISBN: 9781617299209

by Tomasz Lelek, Jonathan Skeet

May 2022

Intermediate to advanced

416 pages

13h 30m

English

Manning Publications

Audiobook available

Read now

Unlock full access

inside front cover
Software Mistakes and Tradeoffs
Copyright
dedication
contents
front matter
prefaceacknowledgmentsabout this bookWho should read this bookHow this book is organizedAbout the codeliveBook discussion forumabout the authorsabout the cover illustration
1 Introduction
1.1 Consequences of every decision and pattern1.1.1 Unit testing decisions1.1.2 Proportions of unit and integration tests1.2 Code design patterns and why they do not always work1.2.1 Measuring our code1.3 Architecture design patterns and why they do not always work1.3.1 Scalability and elasticity1.3.2 Development speed1.3.3 Complexity of microservicesSummary
2 Code duplication is not always bad: Code duplication vs. flexibility
2.1 Common code between codebases and duplication2.1.1 Adding a new business requirement that requires code duplication2.1.2 Implementing the new business requirement2.1.3 Evaluating the result2.2 Libraries and sharing code between codebases2.2.1 Evaluating the tradeoffs and disadvantages of shared libraries2.2.2 Creating a shared library2.3 Code extraction to a separate microservice2.3.1 Looking at the tradeoffs and disadvantages of a separate service2.3.2 Conclusions about separate service2.4 Improving loose coupling by code duplication2.5 An API design with inheritance to reduce duplication2.5.1 Extracting a base request handler2.5.2 Looking at inheritance and tight coupling2.5.3 Looking at the tradeoffs between inheritance and composition2.5.4 Looking at inherent and incidental duplicationSummary
3 Exceptions vs. other patterns of handling errors in your code
3.1 Hierarchy of exceptions3.1.1 Catch-all vs. a more granular approach to handling errors3.2 Best patterns to handle exceptions in the code that you own3.2.1 Handling checked exceptions in a public API3.2.2 Handling unchecked exceptions in a public API3.3 Anti-patterns in exception handling3.3.1 Closing resources in case of an error3.3.2 Anti-pattern of using exceptions to control application flow3.4 Exceptions from third-party libraries3.5 Exceptions in multithread environments3.5.1 Exceptions in an async workflow with a promise API3.6 Functional approach to handling errors with Try3.6.1 Using Try in production code3.6.2 Mixing Try with code that throws an exception3.7 Performance comparison of exception-handling codeSummary
4 Balancing flexibility and complexity
4.1 A robust but not extensible API4.1.1 Designing a new component4.1.2 Starting with the most straightforward code4.2 Allowing clients to provide their own metrics framework4.3 Providing extensibility of your APIs via hooks4.3.1 Guarding against unpredictable usage of the hooks API4.3.2 Performance impact of the hook API4.4 Providing extensibility of your APIs via listeners4.4.1 Using listeners vs. hooks4.4.2 Immutability of our design4.5 Flexibility analysis of an API vs. the cost of maintenanceSummary

5 Premature optimization vs. optimizing the hot path: Decisions that impact code performance
5.1 When premature optimization is evil5.1.1 Creating accounts processing pipeline5.1.2 Optimizing processing based on false assumptions5.1.3 Benchmarking performance optimization5.2 Hot paths in your code5.2.1 Understanding the Pareto principle in the context of software systems5.2.2 Configuring the number of concurrent users (threads) for a given SLA5.3 A word service with a potential hot path5.3.1 Getting the word of the day5.3.2 Validating if the word exists5.3.3 Exposing the WordsService using HTTP service5.4 Hot path detection in your code5.4.1 Creating API performance tests using Gatling5.4.2 Measuring code paths using MetricRegistry5.5 Improvements for hot path performance5.5.1 Creating JMH microbenchmark for the existing solution5.5.2 Optimizing word exists using a cache5.5.3 Modifying performance tests to have more input wordsSummary
6 Simplicity vs. cost of maintenance for your API
6.1 A base library used by other tools6.1.1 Creating a cloud service client6.1.2 Exploring authentication strategies6.1.3 Understanding the configuration mechanism6.2 Directly exposing settings of a dependent library6.2.1 Configuring the batch tool6.3 A tool that is abstracting settings of a dependent library6.3.1 Configuring the streaming tool6.4 Adding new setting for the cloud client library6.4.1 Adding a new setting to the batch tool6.4.2 Adding a new setting to the streaming tool6.4.3 Comparing both solutions for UX friendliness and maintainability6.5 Deprecating/removing a setting in the cloud client library6.5.1 Removing a setting from the batch tool6.5.2 Removing a setting from the streaming tool6.5.3 Comparing both solutions for UX friendliness and maintainabilitySummary
7 Working effectively with date and time data
7.1 Concepts in date and time information7.1.1 Machine time: Instants, epochs, and durations7.1.2 Civil time: Calendar systems, dates, times, and periods7.1.3 Time zones, UTC, and offsets from UTC7.1.4 Date and time concepts that hurt my head7.2 Preparing to work with date and time information7.2.1 Limiting your scope7.2.2 Clarifying date and time requirements7.2.3 Using the right libraries or packages7.3 Implementing date and time code7.3.1 Applying concepts consistently7.3.2 Improving testability by avoiding defaults7.3.3 Representing date and time values in text7.3.4 Explaining code with comments7.4 Corner cases to specify and test7.4.1 Calendar arithmetic7.4.2 Time zone transitions at midnight7.4.3 Handling ambiguous or skipped times7.4.4 Working with evolving time zone dataSummary
8 Leveraging data locality and memory of your machines
8.1 What is data locality?8.1.1 Moving computations to data8.1.2 Scaling processing using data locality8.2 Data partitioning and splitting data8.2.1 Offline big data partitioning8.2.2 Partitioning vs. sharding8.2.3 Partitioning algorithms8.3 Join big data sets from multiple partitions8.3.1 Joining data within the same physical machine8.3.2 Joining that requires data movement8.3.3 Optimizing join leveraging broadcasting8.4 Data processing: Memory vs. disk8.4.1 Using disk-based processing8.4.2 Why do we need MapReduce?8.4.3 Calculating access times8.4.4 RAM-based processing8.5 Implement joins using Apache Spark8.5.1 Implementing a join without broadcast8.5.2 Implementing a join with broadcastSummary
9 Third-party libraries: Libraries you use become your code
9.1 Importing a library and taking full responsibility for its settings: Beware of the defaults9.2 Concurrency models and scalability9.2.1 Using async and sync APIs9.2.2 Distributed scalability9.3 Testability9.3.1 Testing library9.3.2 Testing with fakes (test double) and mocks9.3.3 Integration testing toolkit9.4 Dependencies of third-party libraries9.4.1 Avoiding version conflicts9.4.2 Too many dependencies9.5 Choosing and maintaining third-party dependencies9.5.1 First impressions9.5.2 Different approaches to reusing code9.5.3 Vendor lock-in9.5.4 Licensing9.5.5 Libraries vs. frameworks9.5.6 Security and updates9.5.7 Decision checklistSummary
10 Consistency and atomicity in distributed systems
10.1 At-least-once delivery of data sources10.1.1 Traffic between one-node services10.1.2 Retrying an application’s call10.1.3 Producing data and idempotency10.1.4 Understanding Command Query Responsibility Segregation (CQRS)10.2 A naive implementation of a deduplication library10.3 Common mistakes when implementing deduplication in distributed systems10.3.1 One node context10.3.2 Multiple nodes context10.4 Making your logic atomic to prevent race conditionsSummary
11 Delivery semantics in distributed systems
11.1 Architecture of event-driven applications11.2 Producer and consumer applications based on Apache Kafka11.2.1 Looking at the Kafka consumer side11.2.2 Understanding the Kafka brokers setup11.3 The producer logic11.3.1 Choosing consistency vs. availability for the producer11.4 Consumer code and different delivery semantics11.4.1 Committing a consumer manually11.4.2 Restarting from the earliest or latest offsets11.4.3 (Effectively) exactly-once semantic11.5 Leveraging delivery guarantees to provide fault toleranceSummary
12 Managing versioning and compatibility
12.1 Versioning in the abstract12.1.1 Properties of versions12.1.2 Backward and forward compatibility12.1.3 Semantic versioning12.1.4 Marketing versions12.2 Versioning for libraries12.2.1 Source, binary, and semantic compatibility12.2.2 Dependency graphs and diamond dependencies12.2.3 Techniques for handling breaking changes12.2.4 Managing internal-only libraries12.3 Versioning for network APIs12.3.1 The context of network API calls12.3.2 Customer-friendly clarity12.3.3 Common versioning strategies12.3.4 Further versioning considerations12.4 Versioning for data storage12.4.1 A brief introduction to Protocol Buffers12.4.2 What is a breaking change?12.4.3 Migrating data within a storage system12.4.4 Expecting the unexpected12.4.5 Separating API and storage representations12.4.6 Evaluating storage formatsSummary
13 Keeping up to date with trends vs. cost of maintenance of your code
13.1 When to use dependency injection frameworks13.1.1 Do-it-yourself (DIY) dependency injection13.1.2 Using a dependency injection framework13.2 When to use reactive programming13.2.1 Creating single-threaded, blocking processing13.2.2 Using CompletableFuture13.2.3 Implementing a reactive solution13.3 When to use functional programming13.3.1 Creating functional code in a nonfunctional language13.3.2 Tail recursion optimization13.3.3 Leveraging immutability13.4 Using lazy vs. eager evaluationSummary
index
inside back cover

Content preview from Software Mistakes and Tradeoffs

8 Leveraging data locality and memory of your machines

This chapter covers

Data locality in big data processing
Optimizing join strategies with Apache Spark
How to reduce shuffling
Memory vs. disk usage in big data processing

With both streaming and batch processing in big data applications, we often need to use data from multiple sources to get insights and business value. The data locality pattern allows us to move computation to data. Our data can live in the database or the filesystem, and this situation is simple as long as our data fits into the disk or memory of our machines. Processing can be local and fast, but in big data applications, it is not feasible to store large amounts of data on one machine. We need to employ techniques such ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Software Mistakes and Tradeoffs, Video Edition

Publisher Resources

ISBN: 9781617299209Publisher Support Publisher Website Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Software Mistakes and Tradeoffs

by Tomasz Lelek, Jonathan Skeet

8 Leveraging data locality and memory of your machines

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.