Book description
A comprehensive guide to understanding key techniques for architecture and hardware planning, monitoring, replication, backups, and decoupling
Key Features
- Newly updated edition, covering the latest PostgreSQL 12 features with hands-on industry-driven recipes
- Create a PostgreSQL cluster that stays online even when disaster strikes
- Learn how to avoid costly downtime and data loss that can ruin your business
Book Description
Databases are nothing without the data they store. In the event of an outage or technical catastrophe, immediate recovery is essential. This updated edition ensures that you will learn the important concepts related to node architecture design, as well as techniques such as using repmgr for failover automation. From cluster layout and hardware selection to software stacks and horizontal scalability, this PostgreSQL cookbook will help you build a PostgreSQL cluster that will survive crashes, resist data corruption, and grow smoothly with customer demand.
You'll start by understanding how to plan a PostgreSQL database architecture that is resistant to outages and scalable, as it is the scaffolding on which everything rests. With the bedrock established, you'll cover the topics that PostgreSQL database administrators need to know to manage a highly available cluster. This includes configuration, troubleshooting, monitoring and alerting, backups through proxies, failover automation, and other considerations that are essential for a healthy PostgreSQL cluster. Later, you'll learn to use multi-master replication to maximize server availability. Later chapters will guide you through managing major version upgrades without downtime.
By the end of this book, you'll have learned how to build an efficient and adaptive PostgreSQL 12 database cluster.
What you will learn
- Understand how to protect data with PostgreSQL replication tools
- Focus on hardware planning to ensure that your database runs efficiently
- Reduce database resource contention with connection pooling
- Monitor and visualize cluster activity with Nagios and the TIG (Telegraf, InfluxDB, Grafana) stack
- Construct a robust software stack that can detect and avert outages
- Use multi-master to achieve an enduring PostgreSQL cluster
Who this book is for
This book is for Postgres administrators and developers who are looking to build and maintain a highly reliable PostgreSQL cluster. Although knowledge of the new features of PostgreSQL 12 is not required, a basic understanding of PostgreSQL administration is expected.
Table of contents
- Title Page
- Copyright and Credits
- About Packt
- Contributors
- Preface
- Architectural Considerations
- Hardware Planning
-
Minimizing Downtime
- Determining acceptable losses
- Configuration – getting it right the first time
- Configuration – managing scary settings
- Identifying important tables
- Defusing cache poisoning
- Terminating rogue connections
- Reducing contention with concurrent indexes
- Managing system migrations
- Managing software upgrades
- Mitigating the impact of hardware failure
- Applying bonus kernel tweaks
-
Proxy and Pooling Resources
- Exploring the magic of virtual IPs
- Obtaining and installing HAProxy
- Configuring HAProxy to load balance PostgreSQL
- Determining connection costs and limits
- Installing PgBouncer
- Configuring PgBouncer safely
- Connecting to PgBouncer
- Listing PgBouncer server connections
- Listing PgBouncer client connections
- Evaluating PgBouncer pool health
- Changing PgBouncer connections while online
- Enhancing PgBouncer authentication
-
Troubleshooting
- Performing triage
- Installing common statistics packages
- Evaluating the current disk performance with iostat
- Tracking I/O-heavy processes with iotop
- Viewing past performance with sar
- Correlating performance with dstat
- Interpreting /proc/meminfo
- Examining /proc/net/bonding/bond0
- Checking the pg_stat_activity view
- Checking the pg_stat_statements view
- Deciphering database locks
- Debugging with strace
- Logging checkpoints properly
-
Monitoring
- Figuring out what to monitor
- Installing and configuring Nagios
- Configuring Nagios to monitor a database host
- Enhancing Nagios with Check_MK
- Getting to know check_postgres
- Installing and configuring Telegraf
- Adding a custom PostgreSQL monitor to Telegraf
- Installing and configuring InfluxDB
- Installing and configuring Grafana
- Building a graph in Grafana
- Customizing a Grafana graph
- Using InfluxDB tags in Grafana
-
PostgreSQL Replication
- Deciding what to copy
- Securing the WAL stream
- Setting up a hot standby
- Upgrading to asynchronous replication
- Bulletproofing with synchronous replication
- Faking replication with pg_receivewal
- Setting up Slony
- Copying a few tables with Slony
- Setting up Bucardo
- Copying a few tables with Bucardo
- Setting up pglogical
- Copying a few tables with pglogical
- Copying a few tables with native logical replication
-
Backup Management
- Deciding when to use third-party tools
- Installing and configuring Barman
- Backing up a database with Barman
- Restoring a database with Barman
- Obtaining Barman diagnostics and information
- Sending Barman backups to a remote location
- Installing and configuring pgBackRest
- Backing up a database with pgBackRest
- Restoring a database with pgBackRest
- Installing and configuring WAL-E
- Managing WAL files with WAL-E
-
High Availability with repmgr
- Preparing systems for repmgr
- Installing and configuring repmgr
- Cloning a database with repmgr
- Incorporating a repmgr witness
- Performing a managed failover
- Customizing the failover process
- Using an outage to test availability
- Returning a node to the cluster
- Integrating primary fencing
- Performing online maintenance and upgrades
-
High Availability with Patroni
- Understanding more about Patroni and its components
- Preparing systems for the stack
- Installing and configuring etcd
- Installing and configuring Patroni
- Installing and configuring HAProxy
- Performing a managed switchover
- Using an outage to test availability
- Returning a node to the cluster
- Adding additional nodes to the mix
- Replacing etcd with ZooKeeper
- Replacing etcd with Consul
- Upgrading while staying online
-
Low-Level Server Mirroring
- Understanding our chosen filesystem components
- Preparing systems for volume mirroring
- Getting started with the LVM
- Adding block-level replication
- Incorporating the second LVM layer
- Verifying a DRBD filesystem
- Correcting a DRBD split brain
- Formatting an XFS filesystem
- Tweaking XFS performance
- Maintaining an XFS filesystem
- Using LVM snapshots
- Switching live stack systems
- Detaching a problematic node
-
High Availability via Pacemaker
- Before we begin...
- Installing the components
- Configuring Corosync
- Preparing start up services
- Starting with base options
- Adding DRBD to cluster management
- Adding LVM to cluster management
- Adding XFS to cluster management
- Adding PostgreSQL to cluster management
- Adding a virtual IP to proxy the cluster
- Adding an email alert
- Grouping associated resources
- Combining and ordering related actions
- Performing a managed resource migration
- Using an outage to test migration
-
High Availability with Multi-Master Replication
- Overview of multi-master
- Deciding whether multi-master is right for you
- Obtaining and installing BDR
- Starting with a single BDR node
- Creating an additional BDR node
- Testing DDL replication on each node
- Using sequences safely
- Configuring HAProxy for the multi-master approach
- Combining PgBouncer with HAProxy
- Performing a managed node switchover
- Improving failover speed
- Performing a major version upgrade online
-
Data Distribution
- Identifying horizontal candidates
- Setting up a foreign PostgreSQL server
- Mapping a remote user
- Creating a foreign table
- Using a foreign table in a query
- Optimizing foreign table access
- Transforming foreign tables into local tables
- Creating a scalable nextval replacement
- Building a sharding API
- Talking to the correct shard
- Moving a shard to another server
- Zero-downtime Upgrades
- Other Books You May Enjoy
Product information
- Title: PostgreSQL 12 High Availability Cookbook
- Author(s):
- Release date: February 2020
- Publisher(s): Packt Publishing
- ISBN: 9781838984854
You might also like
book
PostgreSQL High Availability Cookbook - Second Edition
Master over 100 recipes to design and implement a highly available server with the advanced features …
book
PostgreSQL 10 Administration Cookbook
A practical guide to administer, monitor and replicate your PostgreSQL 10 databaseAbout This Book Get to …
book
PostgreSQL High Performance Cookbook
Get to know effective ways to improve PostgreSQL’s performance and master query optimization, and database monitoring. …
book
PostgreSQL 13 Cookbook
Get to grips with building reliable, scalable, and maintainable database solutions for enterprises and production databases …