How Linux Works, 3rd Edition

How Linux Works, 3rd Edition

by Brian Ward
Released April 2021
Publisher(s): No Starch Press
ISBN: 9781718500402

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Unlike some operating systems, Linux doesn’t try to hide the important bits from you—it gives you full control of your computer. But to truly master Linux, you need to understand its internals, like how the system boots, how networking works, and what the kernel actually does.

In this third edition of the bestselling How Linux Works, author Brian Ward peels back the layers of this well-loved operating system to make Linux internals accessible. This edition has been thoroughly updated and expanded with added coverage of Logical Volume Manager (LVM), virtualization, and containers.

You'll learn:

•How Linux boots, from boot loaders to init (systemd)
•How the kernel manages devices, device drivers, and processes
•How networking, interfaces, firewalls, and servers work
•How development tools work and relate to shared libraries
•How to write effective shell scripts

You’ll also explore the kernel and examine key system tasks inside user space, including system calls, input and output, and filesystems. With its combination of background, theory, real-world examples, and patient explanations, How Linux Works, 3rd edition will teach you what you need to know to solve pesky problems and take control of your operating system.

Table of contents

  1. Reviews for How Linux Works
  2. Title Page
  3. Copyright
  4. About the Author
  5. Acknowledgments
  6. Preface
    1. Who Should Read This Book?
    2. Prerequisites
    3. How to Read This Book
    4. A Hands-on Approach
    5. How This Book Is Organized
    6. What’s New in the Third Edition?
    7. A Note on Terminology
  7. Chapter 1: The Big Picture
    1. 1.1 Levels and Layers of Abstraction in a Linux System
    2. 1.2 Hardware: Understanding Main Memory
    3. 1.3 The Kernel
      1. 1.3.1 Process Management
      2. 1.3.2 Memory Management
      3. 1.3.3 Device Drivers and Management
      4. 1.3.4 System Calls and Support
    4. 1.4 User Space
    5. 1.5 Users
    6. 1.6 Looking Forward
  8. Chapter 2: Basic Commands and Directory Hierarchy
    1. 2.1 The Bourne Shell: /bin/sh
    2. 2.2 Using the Shell
      1. 2.2.1 The Shell Window
      2. 2.2.2 cat
      3. 2.2.3 Standard Input and Standard Output
    3. 2.3 Basic Commands
      1. 2.3.1 ls
      2. 2.3.2 cp
      3. 2.3.3 mv
      4. 2.3.4 touch
      5. 2.3.5 rm
      6. 2.3.6 echo
    4. 2.4 Navigating Directories
      1. 2.4.1 cd
      2. 2.4.2 mkdir
      3. 2.4.3 rmdir
      4. 2.4.4 Shell Globbing (“Wildcards”)
    5. 2.5 Intermediate Commands
      1. 2.5.1 grep
      2. 2.5.2 less
      3. 2.5.3 pwd
      4. 2.5.4 diff
      5. 2.5.5 file
      6. 2.5.6 find and locate
      7. 2.5.7 head and tail
      8. 2.5.8 sort
    6. 2.6 Changing Your Password and Shell
    7. 2.7 Dot Files
    8. 2.8 Environment and Shell Variables
    9. 2.9 The Command Path
    10. 2.10 Special Characters
    11. 2.11 Command-Line Editing
    12. 2.12 Text Editors
    13. 2.13 Getting Online Help
    14. 2.14 Shell Input and Output
      1. 2.14.1 Standard Error
      2. 2.14.2 Standard Input Redirection
    15. 2.15 Understanding Error Messages
      1. 2.15.1 Anatomy of a Unix Error Message
      2. 2.15.2 Common Errors
    16. 2.16 Listing and Manipulating Processes
      1. 2.16.1 ps Command Options
      2. 2.16.2 Process Termination
      3. 2.16.3 Job Control
      4. 2.16.4 Background Processes
    17. 2.17 File Modes and Permissions
      1. 2.17.1 Modifying Permissions
      2. 2.17.2 Working with Symbolic Links
    18. 2.18 Archiving and Compressing Files
      1. 2.18.1 gzip
      2. 2.18.2 tar
      3. 2.18.3 Compressed Archives (.tar.gz)
      4. 2.18.4 zcat
      5. 2.18.5 Other Compression Utilities
    19. 2.19 Linux Directory Hierarchy Essentials
      1. 2.19.1 Other Root Subdirectories
      2. 2.19.2 The /usr Directory
      3. 2.19.3 Kernel Location
    20. 2.20 Running Commands as the Superuser
      1. 2.20.1 sudo
      2. 2.20.2 /etc/sudoers
      3. 2.20.3 sudo Logs
    21. 2.21 Looking Forward
  9. Chapter 3: Devices
    1. 3.1 Device Files
    2. 3.2 The sysfs Device Path
    3. 3.3 dd and Devices
    4. 3.4 Device Name Summary
      1. 3.4.1 Hard Disks: /dev/sd*
      2. 3.4.2 Virtual Disks: /dev/xvd*, /dev/vd*
      3. 3.4.3 Non-Volatile Memory Devices: /dev/nvme*
      4. 3.4.4 Device Mapper: /dev/dm-*, /dev/mapper/*
      5. 3.4.5 CD and DVD Drives: /dev/sr*
      6. 3.4.6 PATA Hard Disks: /dev/hd*
      7. 3.4.7 Terminals: /dev/tty*, /dev/pts/*, and /dev/tty
      8. 3.4.8 Serial Ports: /dev/ttyS*, /dev/ttyUSB*, /dev/ttyACM*
      9. 3.4.9 Parallel Ports: /dev/lp0 and /dev/lp1
      10. 3.4.10 Audio Devices: /dev/snd/*, /dev/dsp, /dev/audio, and More
      11. 3.4.11 Device File Creation
    5. 3.5 udev
      1. 3.5.1 devtmpfs
      2. 3.5.2 udevd Operation and Configuration
      3. 3.5.3 udevadm
      4. 3.5.4 Device Monitoring
    6. 3.6 In-Depth: SCSI and the Linux Kernel
      1. 3.6.1 USB Storage and SCSI
      2. 3.6.2 SCSI and ATA
      3. 3.6.3 Generic SCSI Devices
      4. 3.6.4 Multiple Access Methods for a Single Device
  10. Chapter 4: Disks and Filesystems
    1. 4.1 Partitioning Disk Devices
      1. 4.1.1 Viewing a Partition Table
      2. 4.1.2 Modifying Partition Tables
      3. 4.1.3 Creating a Partition Table
      4. 4.1.4 Navigating Disk and Partition Geometry
      5. 4.1.5 Reading from Solid-State Disks
    2. 4.2 Filesystems
      1. 4.2.1 Filesystem Types
      2. 4.2.2 Creating a Filesystem
      3. 4.2.3 Mounting a Filesystem
      4. 4.2.4 Filesystem UUID
      5. 4.2.5 Disk Buffering, Caching, and Filesystems
      6. 4.2.6 Filesystem Mount Options
      7. 4.2.7 Remounting a Filesystem
      8. 4.2.8 The /etc/fstab Filesystem Table
      9. 4.2.9 Alternatives to /etc/fstab
      10. 4.2.10 Filesystem Capacity
      11. 4.2.11 Checking and Repairing Filesystems
      12. 4.2.12 Special-Purpose Filesystems
    3. 4.3 Swap Space
      1. 4.3.1 Using a Disk Partition as Swap Space
      2. 4.3.2 Using a File as Swap Space
      3. 4.3.3 Determining How Much Swap You Need
    4. 4.4 The Logical Volume Manager
      1. 4.4.1 Working with LVM
      2. 4.4.2 The LVM Implementation
    5. 4.5 Looking Forward: Disks and User Space
    6. 4.6 Inside a Traditional Filesystem
      1. 4.6.1 Inode Details and the Link Count
      2. 4.6.2 Block Allocation
      3. 4.6.3 Working with Filesystems in User Space
  11. Chapter 5: How the Linux Kernel Boots
    1. 5.1 Startup Messages
    2. 5.2 Kernel Initialization and Boot Options
    3. 5.3 Kernel Parameters
    4. 5.4 Boot Loaders
      1. 5.4.1 Boot Loader Tasks
      2. 5.4.2 Boot Loader Overview
      3. 5.5.1 Exploring Devices and Partitions with the GRUB Command Line
      4. 5.5.2 GRUB Configuration
      5. 5.5.3 GRUB Installation
    5. 5.6 UEFI Secure Boot Problems
    6. 5.7 Chainloading Other Operating Systems
    7. 5.8 Boot Loader Details
      1. 5.8.1 MBR Boot
      2. 5.8.2 UEFI Boot
      3. 5.8.3 How GRUB Works
  12. Chapter 6: How User Space Starts
    1. 6.2 Identifying Your init
    2. 6.3 systemd
      1. 6.3.1 Units and Unit Types
      2. 6.3.2 Booting and Unit Dependency Graphs
      3. 6.3.3 systemd Configuration
      4. 6.3.4 systemd Operation
      5. 6.3.5 systemd Process Tracking and Synchronization
      6. 6.3.6 systemd Dependencies
      7. 6.3.7 systemd On-Demand and Resource-Parallelized Startup
      8. 6.3.8 systemd Auxiliary Components
    3. 6.4 System V Runlevels
    4. 6.5 System V init
      1. 6.5.1 System V init: Startup Command Sequence
      2. 6.5.2 The System V init Link Farm
      3. 6.5.3 run-parts
      4. 6.5.4 System V init Control
      5. 6.5.5 systemd System V Compatibility
    5. 6.6 Shutting Down Your System
    6. 6.7 The Initial RAM Filesystem
    7. 6.8 Emergency Booting and Single-User Mode
    8. 6.9 Looking Forward
  13. Chapter 7: System Configuration: Logging, System Time, Batch Jobs, and Users
    1. 7.1 System Logging
      1. 7.1.1 Checking Your Log Setup
      2. 7.1.2 Searching and Monitoring Logs
      3. 7.1.3 Logfile Rotation
      4. 7.1.4 Journal Maintenance
      5. 7.1.5 A Closer Look at System Logging
    2. 7.2 The Structure of /etc
    3. 7.3 User Management Files
      1. 7.3.1 The /etc/passwd File
      2. 7.3.2 Special Users
      3. 7.3.3 The /etc/shadow File
      4. 7.3.4 Manipulating Users and Passwords
      5. 7.3.5 Working with Groups
    4. 7.4 getty and login
    5. 7.5 Setting the Time
      1. 7.5.1 Kernel Time Representation and Time Zones
      2. 7.5.2 Network Time
    6. 7.6 Scheduling Recurring Tasks with cron and Timer Units
      1. 7.6.1 Installing Crontab Files
      2. 7.6.2 System Crontab Files
      3. 7.6.3 Timer Units
      4. 7.6.4 cron vs. Timer Units
    7. 7.7 Scheduling One-Time Tasks with at
      1. 7.7.1 Timer Unit Equivalents
    8. 7.8 Timer Units Running as Regular Users
    9. 7.9 User Access Topics
      1. 7.9.1 User IDs and User Switching
      2. 7.9.2 Process Ownership, Effective UID, Real UID, and Saved UID
      3. 7.9.3 User Identification, Authentication, and Authorization
      4. 7.9.4 Using Libraries for User Information
    10. 7.10 Pluggable Authentication Modules
      1. 7.10.1 PAM Configuration
      2. 7.10.2 Tips on PAM Configuration Syntax
      3. 7.10.3 PAM and Passwords
    11. 7.11 Looking Forward
  14. Chapter 8: A Closer Look at Processes and Resource Utilization
    1. 8.1 Tracking Processes
    2. 8.2 Finding Open Files with lsof
      1. 8.2.1 Reading the lsof Output
      2. 8.2.2 Using lsof
    3. 8.3 Tracing Program Execution and System Calls
      1. 8.3.1 strace
      2. 8.3.2 ltrace
    4. 8.4 Threads
      1. 8.4.1 Single-Threaded and Multithreaded Processes
      2. 8.4.2 Viewing Threads
      3. 8.5.1 Measuring CPU Time
      4. 8.5.2 Adjusting Process Priorities
      5. 8.5.3 Measuring CPU Performance with Load Averages
      6. 8.5.4 Monitoring Memory Status
      7. 8.5.5 Monitoring CPU and Memory Performance with vmstat
      8. 8.5.6 I/O Monitoring
      9. 8.5.7 Per-Process Monitoring with pidstat
    5. 8.6 Control Groups (cgroups)
      1. 8.6.1 Differentiating Between cgroup Versions
      2. 8.6.2 Viewing cgroups
      3. 8.6.3 Manipulating and Creating cgroups
      4. 8.6.4 Viewing Resource Utilization
    6. 8.7 Further Topics
  15. Chapter 9: Understanding Your Network and Its Configuration
    1. 9.1 Network Basics
    2. 9.2 Packets
    3. 9.3 Network Layers
    4. 9.4 The Internet Layer
      1. 9.4.1 Viewing IP Addresses
      2. 9.4.2 Subnets
      3. 9.4.3 Common Subnet Masks and CIDR Notation
    5. 9.5 Routes and the Kernel Routing Table
    6. 9.6 The Default Gateway
    7. 9.7 IPv6 Addresses and Networks
      1. 9.7.1 Viewing IPv6 Configuration on Your System
      2. 9.7.2 Configuring Dual-Stack Networks
    8. 9.8 Basic ICMP and DNS Tools
      1. 9.8.1 ping
      2. 9.8.2 DNS and host
    9. 9.9 The Physical Layer and Ethernet
    10. 9.10 Understanding Kernel Network Interfaces
      1. 9.11.1 Manually Configuring Interfaces
      2. 9.11.2 Manually Adding and Deleting Routes
    11. 9.12 Boot-Activated Network Configuration
    12. 9.13 Problems with Manual and Boot-Activated Network Configuration
    13. 9.14 Network Configuration Managers
      1. 9.14.1 NetworkManager Operation
      2. 9.14.2 NetworkManager Interaction
      3. 9.14.3 NetworkManager Configuration
    14. 9.15 Resolving Hostnames
      1. 9.15.1 /etc/hosts
      2. 9.15.2 resolv.conf
      3. 9.15.3 Caching and Zero-Configuration DNS
      4. 9.15.4 /etc/nsswitch.conf
    15. 9.16 Localhost
    16. 9.17 The Transport Layer: TCP, UDP, and Services
      1. 9.17.1 TCP Ports and Connections
      2. 9.17.2 UDP
    17. 9.18 Revisiting a Simple Local Network
    18. 9.19 Understanding DHCP
      1. 9.19.1 Linux DHCP Clients
      2. 9.19.2 Linux DHCP Servers
    19. 9.20 Automatic IPv6 Network Configuration
    20. 9.21 Configuring Linux as a Router
    21. 9.22 Private Networks (IPv4)
    22. 9.23 Network Address Translation (IP Masquerading)
    23. 9.24 Routers and Linux
    24. 9.25 Firewalls
      1. 9.25.1 Linux Firewall Basics
      2. 9.25.2 Setting Firewall Rules
      3. 9.25.3 Firewall Strategies
    25. 9.26 Ethernet, IP, ARP, and NDP
    26. 9.27 Wireless Ethernet
      1. 9.27.1 iw
      2. 9.27.2 Wireless Security
    27. 9.28 Summary
  16. Chapter 10: Network Applications and Services
    1. 10.1 The Basics of Services
    2. 10.2 A Closer Look
    3. 10.3 Network Servers
      1. 10.3.1  Secure Shell
      2. 10.3.2  The sshd Server
      3. 10.3.3  fail2ban
      4. 10.3.4  The SSH Client
    4. 10.4 Pre-systemd Network Connection Servers: inetd/xinetd
    5. 10.5 Diagnostic Tools
      1. 10.5.1  lsof
      2. 10.5.2  tcpdump
      3. 10.5.3  netcat
      4. 10.5.4  Port Scanning
    6. 10.6 Remote Procedure Calls
    7. 10.7 Network Security
      1. 10.7.1  Typical Vulnerabilities
      2. 10.7.2  Security Resources
    8. 10.8 Looking Forward
    9. 10.9 Network Sockets
    10. 10.10 Unix Domain Sockets
  17. Chapter 11: Introduction to Shell Scripts
    1. 11.1 Shell Script Basics
      1. 11.1.1 Limitations of Shell Scripts
    2. 11.2 Quoting and Literals
      1. 11.2.1 Literals
      2. 11.2.2 Single Quotes
      3. 11.2.3 Double Quotes
      4. 11.2.4 Literal Single Quotes
    3. 11.3 Special Variables
      1. 11.3.1 Individual Arguments: $1, $2, and So On
      2. 11.3.2 Number of Arguments: $#
      3. 11.3.3 All Arguments: $@
      4. 11.3.4 Script Name: $0
      5. 11.3.5 Process ID: $$
      6. 11.3.6 Exit Code: $?
    4. 11.4 Exit Codes
    5. 11.5 Conditionals
      1. 11.5.1 A Workaround for Empty Parameter Lists
      2. 11.5.2 Other Commands for Tests
      3. 11.5.3 elif
      4. 11.5.4 Logical Constructs
      5. 11.5.5 Testing Conditions
      6. 11.5.6 case
    6. 11.6 Loops
      1. 11.6.1 for Loops
      2. 11.6.2 while Loops
    7. 11.7 Command Substitution
    8. 11.8 Temporary File Management
    9. 11.9 Here Documents
    10. 11.10 Important Shell Script Utilities
      1. 11.10.1  basename
      2. 11.10.2  awk
      3. 11.10.3  sed
      4. 11.10.4  xargs
      5. 11.10.5  expr
      6. 11.10.6  exec
    11. 11.11 Subshells
    12. 11.12 Including Other Files in Scripts
    13. 11.13 Reading User Input
    14. 11.14 When (Not) to Use Shell Scripts
  18. Chapter 12: Network File Transfer and Sharing
    1. 12.1 Quick Copy
    2. 12.2 rsync
      1. 12.2.1  Getting Started with rsync
      2. 12.2.2  Making Exact Copies of a Directory Structure
      3. 12.2.3  Using the Trailing Slash
      4. 12.2.4  Excluding Files and Directories
      5. 12.2.5  Checking Transfers, Adding Safeguards, and Using Verbose Mode
      6. 12.2.6  Compressing Data
      7. 12.2.7  Limiting Bandwidth
      8. 12.2.8  Transferring Files to Your Computer
      9. 12.2.9  Further rsync Topics
      10. 12.3.1  File Sharing Usage and Performance
      11. 12.3.2  File Sharing Security
    3. 12.4 Sharing Files with Samba
      1. 12.4.1  Server Configuration
      2. 12.4.2  Server Access Control
      3. 12.4.3  Passwords
      4. 12.4.4  Manual Server Startup
      5. 12.4.5  Diagnostics and Logfiles
      6. 12.4.6  File Share Configuration
      7. 12.4.7  Home Directories
      8. 12.4.8  Printer Sharing
      9. 12.4.9  The Samba Client
    4. 12.5 SSHFS
    5. 12.6 NFS
    6. 12.7 Cloud Storage
    7. 12.8 The State of Network File Sharing
  19. Chapter 13: User Environments
    1. 13.1 Guidelines for Creating Startup Files
    2. 13.2 When to Alter Startup Files
    3. 13.3 Shell Startup File Elements
      1. 13.3.1  The Command Path
      2. 13.3.2  The Manual Page Path
      3. 13.3.3  The Prompt
      4. 13.3.4  Aliases
      5. 13.3.5  The Permissions Mask
    4. 13.4 Startup File Order and Examples
      1. 13.4.1  The bash Shell
      2. 13.4.2  The tcsh Shell
    5. 13.5 Default User Settings
      1. 13.5.1  Shell Defaults
      2. 13.5.2  Editor
      3. 13.5.3  Pager
    6. 13.6 Startup File Pitfalls
    7. 13.7 Further Startup Topics
  20. Chapter 14: A Brief Survey of the Linux Desktop and Printing
    1. 14.1 Desktop Components
      1. 14.1.1  Framebuffers
      2. 14.1.2  The X Window System
      3. 14.1.3  Wayland
      4. 14.1.4  Window Managers
      5. 14.1.5  Toolkits
      6. 14.1.6  Desktop Environments
      7. 14.1.7  Applications
    2. 14.2 Are You Running Wayland or X?
    3. 14.3 A Closer Look at Wayland
      1. 14.3.1  The Compositing Window Manager
      2. 14.3.2  libinput
      3. 14.3.3  X Compatibility in Wayland
    4. 14.4 A Closer Look at the X Window System
      1. 14.4.1  Display Managers
      2. 14.4.2  Network Transparency
      3. 14.4.3 Ways of Exploring X Clients
      4. 14.4.4  X Events
      5. 14.4.5  X Input and Preference Settings
    5. 14.5 D-Bus
      1. 14.5.1  System and Session Instances
      2. 14.5.2  D-Bus Message Monitoring
    6. 14.6 Printing
      1. 14.6.1  CUPS
      2. 14.6.2  Format Conversion and Print Filters
    7. 14.7 Other Desktop Topics
  21. Chapter 15: Development Tools
    1. 15.1 The C Compiler
      1. 15.1.1  Compiling Multiple Source Files
      2. 15.1.2  Linking with Libraries
      3. 15.1.3 Working with Shared Libraries
      4. 15.1.4  Working with Header (Include) Files and Directories
    2. 15.2 make
      1. 15.2.1  A Sample Makefile
      2. 15.2.2  Built-in Rules
      3. 15.2.3  Final Program Build
      4. 15.2.4  Dependency Updates
      5. 15.2.5  Command-Line Arguments and Options
      6. 15.2.6  Standard Macros and Variables
      7. 15.2.7  Conventional Targets
      8. 15.2.8  Makefile Organization
    3. 15.3 Lex and Yacc
    4. 15.4 Scripting Languages
      1. 15.4.1  Python
      2. 15.4.2  Perl
      3. 15.4.3  Other Scripting Languages
    5. 15.5 Java
    6. 15.6 Looking Forward: Compiling Packages
  22. Chapter 16: Introduction to Compiling Software from C Source Code
    1. 16.1 Software Build Systems
    2. 16.2 Unpacking C Source Packages
    3. 16.3 GNU Autoconf
      1. 16.3.1  An Autoconf Example
      2. 16.3.2  Installation Using a Packaging Tool
      3. 16.3.3  configure Script Options
      4. 16.3.4  Environment Variables
      5. 16.3.5  Autoconf Targets
      6. 16.3.6  Autoconf Logfiles
      7. 16.3.7  pkg-config
    4. 16.4 Installation Practice
      1. 16.4.1  Where to Install
    5. 16.5 Applying a Patch
    6. 16.6 Troubleshooting Compiles and Installations
      1. 16.6.1  Specific Errors
    7. 16.7 Looking Forward
  23. Chapter 17: Virtualization
    1. 17.1 Virtual Machines
      1. 17.1.1  Hypervisors
      2. 17.1.2  Hardware in a Virtual Machine
      3. 17.1.3  Common Uses of Virtual Machines
      4. 17.1.4  Drawbacks of Virtual Machines
    2. 17.2 Containers
      1. 17.2.1  Docker, Podman, and Privileges
      2. 17.2.2  A Docker Example
      3. 17.2.3  LXC
      4. 17.2.4  Kubernetes
      5. 17.2.5  Pitfalls of Containers
    3. 17.3 Runtime-Based Virtualization
  24. Index

Product information

  • Title: How Linux Works, 3rd Edition
  • Author(s): Brian Ward
  • Release date: April 2021
  • Publisher(s): No Starch Press
  • ISBN: 9781718500402