book

How Linux Works, 3rd Edition

by Brian Ward

April 2021

Beginner

464 pages

12h 47m

English

No Starch Press

Read now

Unlock full access

Who Should Read This Book?PrerequisitesHow to Read This BookA Hands-on ApproachHow This Book Is OrganizedWhat’s New in the Third Edition?A Note on Terminology
1.1 Levels and Layers of Abstraction in a Linux System 1.2 Hardware: Understanding Main Memory 1.3 The Kernel1.3.1 Process Management1.3.2 Memory Management1.3.3 Device Drivers and Management1.3.4 System Calls and Support 1.4 User Space 1.5 Users 1.6 Looking Forward
2.1 The Bourne Shell: /bin/sh 2.2 Using the Shell2.2.1 The Shell Window2.2.2 cat2.2.3 Standard Input and Standard Output 2.3 Basic Commands2.3.1 ls2.3.2 cp2.3.3 mv2.3.4 touch2.3.5 rm2.3.6 echo 2.4 Navigating Directories2.4.1 cd2.4.2 mkdir2.4.3 rmdir2.4.4 Shell Globbing (“Wildcards”) 2.5 Intermediate Commands2.5.1 grep2.5.2 less2.5.3 pwd2.5.4 diff2.5.5 file2.5.6 find and locate2.5.7 head and tail2.5.8 sort 2.6 Changing Your Password and Shell 2.7 Dot Files 2.8 Environment and Shell Variables 2.9 The Command Path 2.10 Special Characters 2.11 Command-Line Editing 2.12 Text Editors 2.13 Getting Online Help 2.14 Shell Input and Output2.14.1 Standard Error2.14.2 Standard Input Redirection 2.15 Understanding Error Messages2.15.1 Anatomy of a Unix Error Message2.15.2 Common Errors 2.16 Listing and Manipulating Processes2.16.1 ps Command Options2.16.2 Process Termination2.16.3 Job Control2.16.4 Background Processes 2.17 File Modes and Permissions2.17.1 Modifying Permissions2.17.2 Working with Symbolic Links 2.18 Archiving and Compressing Files2.18.1 gzip2.18.2 tar2.18.3 Compressed Archives (.tar.gz)2.18.4 zcat2.18.5 Other Compression Utilities 2.19 Linux Directory Hierarchy Essentials2.19.1 Other Root Subdirectories2.19.2 The /usr Directory2.19.3 Kernel Location 2.20 Running Commands as the Superuser2.20.1 sudo2.20.2 /etc/sudoers2.20.3 sudo Logs 2.21 Looking Forward
3.1 Device Files 3.2 The sysfs Device Path 3.3 dd and Devices 3.4 Device Name Summary3.4.1 Hard Disks: /dev/sd*3.4.2 Virtual Disks: /dev/xvd*, /dev/vd*3.4.3 Non-Volatile Memory Devices: /dev/nvme*3.4.4 Device Mapper: /dev/dm-*, /dev/mapper/*3.4.5 CD and DVD Drives: /dev/sr*3.4.6 PATA Hard Disks: /dev/hd*3.4.7 Terminals: /dev/tty*, /dev/pts/*, and /dev/tty3.4.8 Serial Ports: /dev/ttyS*, /dev/ttyUSB*, /dev/ttyACM*3.4.9 Parallel Ports: /dev/lp0 and /dev/lp13.4.10 Audio Devices: /dev/snd/*, /dev/dsp, /dev/audio, and More3.4.11 Device File Creation 3.5 udev3.5.1 devtmpfs3.5.2 udevd Operation and Configuration3.5.3 udevadm3.5.4 Device Monitoring 3.6 In-Depth: SCSI and the Linux Kernel3.6.1 USB Storage and SCSI3.6.2 SCSI and ATA3.6.3 Generic SCSI Devices3.6.4 Multiple Access Methods for a Single Device
4.1 Partitioning Disk Devices4.1.1 Viewing a Partition Table4.1.2 Modifying Partition Tables4.1.3 Creating a Partition Table4.1.4 Navigating Disk and Partition Geometry4.1.5 Reading from Solid-State Disks 4.2 Filesystems4.2.1 Filesystem Types4.2.2 Creating a Filesystem4.2.3 Mounting a Filesystem4.2.4 Filesystem UUID4.2.5 Disk Buffering, Caching, and Filesystems4.2.6 Filesystem Mount Options4.2.7 Remounting a Filesystem4.2.8 The /etc/fstab Filesystem Table4.2.9 Alternatives to /etc/fstab4.2.10 Filesystem Capacity4.2.11 Checking and Repairing Filesystems4.2.12 Special-Purpose Filesystems 4.3 Swap Space4.3.1 Using a Disk Partition as Swap Space4.3.2 Using a File as Swap Space4.3.3 Determining How Much Swap You Need 4.4 The Logical Volume Manager4.4.1 Working with LVM4.4.2 The LVM Implementation 4.5 Looking Forward: Disks and User Space 4.6 Inside a Traditional Filesystem4.6.1 Inode Details and the Link Count 4.6.2 Block Allocation4.6.3 Working with Filesystems in User Space

5.1 Startup Messages 5.2 Kernel Initialization and Boot Options 5.3 Kernel Parameters 5.4 Boot Loaders5.4.1 Boot Loader Tasks5.4.2 Boot Loader Overview5.5.1 Exploring Devices and Partitions with the GRUB Command Line5.5.2 GRUB Configuration5.5.3 GRUB Installation 5.6 UEFI Secure Boot Problems 5.7 Chainloading Other Operating Systems 5.8 Boot Loader Details5.8.1 MBR Boot5.8.2 UEFI Boot5.8.3 How GRUB Works
6.2 Identifying Your init 6.3 systemd6.3.1 Units and Unit Types6.3.2 Booting and Unit Dependency Graphs6.3.3 systemd Configuration6.3.4 systemd Operation6.3.5 systemd Process Tracking and Synchronization6.3.6 systemd Dependencies6.3.7 systemd On-Demand and Resource-Parallelized Startup6.3.8 systemd Auxiliary Components 6.4 System V Runlevels 6.5 System V init6.5.1 System V init: Startup Command Sequence6.5.2 The System V init Link Farm6.5.3 run-parts6.5.4 System V init Control6.5.5 systemd System V Compatibility 6.6 Shutting Down Your System 6.7 The Initial RAM Filesystem 6.8 Emergency Booting and Single-User Mode 6.9 Looking Forward
7.1 System Logging7.1.1 Checking Your Log Setup7.1.2 Searching and Monitoring Logs7.1.3 Logfile Rotation7.1.4 Journal Maintenance7.1.5 A Closer Look at System Logging 7.2 The Structure of /etc 7.3 User Management Files7.3.1 The /etc/passwd File7.3.2 Special Users7.3.3 The /etc/shadow File7.3.4 Manipulating Users and Passwords7.3.5 Working with Groups 7.4 getty and login 7.5 Setting the Time7.5.1 Kernel Time Representation and Time Zones7.5.2 Network Time 7.6 Scheduling Recurring Tasks with cron and Timer Units7.6.1 Installing Crontab Files7.6.2 System Crontab Files7.6.3 Timer Units7.6.4 cron vs. Timer Units 7.7 Scheduling One-Time Tasks with at7.7.1 Timer Unit Equivalents 7.8 Timer Units Running as Regular Users 7.9 User Access Topics7.9.1 User IDs and User Switching7.9.2 Process Ownership, Effective UID, Real UID, and Saved UID7.9.3 User Identification, Authentication, and Authorization7.9.4 Using Libraries for User Information 7.10 Pluggable Authentication Modules7.10.1 PAM Configuration7.10.2 Tips on PAM Configuration Syntax7.10.3 PAM and Passwords 7.11 Looking Forward
8.1 Tracking Processes 8.2 Finding Open Files with lsof8.2.1 Reading the lsof Output8.2.2 Using lsof 8.3 Tracing Program Execution and System Calls8.3.1 strace8.3.2 ltrace 8.4 Threads8.4.1 Single-Threaded and Multithreaded Processes8.4.2 Viewing Threads8.5.1 Measuring CPU Time8.5.2 Adjusting Process Priorities8.5.3 Measuring CPU Performance with Load Averages8.5.4 Monitoring Memory Status8.5.5 Monitoring CPU and Memory Performance with vmstat8.5.6 I/O Monitoring8.5.7 Per-Process Monitoring with pidstat 8.6 Control Groups (cgroups)8.6.1 Differentiating Between cgroup Versions 8.6.2 Viewing cgroups8.6.3 Manipulating and Creating cgroups8.6.4 Viewing Resource Utilization 8.7 Further Topics
9.1 Network Basics 9.2 Packets 9.3 Network Layers 9.4 The Internet Layer9.4.1 Viewing IP Addresses9.4.2 Subnets9.4.3 Common Subnet Masks and CIDR Notation 9.5 Routes and the Kernel Routing Table 9.6 The Default Gateway 9.7 IPv6 Addresses and Networks9.7.1 Viewing IPv6 Configuration on Your System9.7.2 Configuring Dual-Stack Networks 9.8 Basic ICMP and DNS Tools9.8.1 ping9.8.2 DNS and host 9.9 The Physical Layer and Ethernet 9.10 Understanding Kernel Network Interfaces9.11.1 Manually Configuring Interfaces 9.11.2 Manually Adding and Deleting Routes 9.12 Boot-Activated Network Configuration 9.13 Problems with Manual and Boot-Activated Network Configuration 9.14 Network Configuration Managers9.14.1 NetworkManager Operation9.14.2 NetworkManager Interaction9.14.3 NetworkManager Configuration 9.15 Resolving Hostnames9.15.1 /etc/hosts9.15.2 resolv.conf9.15.3 Caching and Zero-Configuration DNS9.15.4 /etc/nsswitch.conf 9.16 Localhost 9.17 The Transport Layer: TCP, UDP, and Services9.17.1 TCP Ports and Connections9.17.2 UDP 9.18 Revisiting a Simple Local Network 9.19 Understanding DHCP9.19.1 Linux DHCP Clients9.19.2 Linux DHCP Servers 9.20 Automatic IPv6 Network Configuration 9.21 Configuring Linux as a Router 9.22 Private Networks (IPv4) 9.23 Network Address Translation (IP Masquerading) 9.24 Routers and Linux 9.25 Firewalls9.25.1 Linux Firewall Basics9.25.2 Setting Firewall Rules9.25.3 Firewall Strategies 9.26 Ethernet, IP, ARP, and NDP 9.27 Wireless Ethernet9.27.1 iw9.27.2 Wireless Security 9.28 Summary
10.1 The Basics of Services 10.2 A Closer Look 10.3 Network Servers10.3.1 Secure Shell10.3.2 The sshd Server10.3.3 fail2ban10.3.4 The SSH Client 10.4 Pre-systemd Network Connection Servers: inetd/xinetd 10.5 Diagnostic Tools10.5.1 lsof10.5.2 tcpdump10.5.3 netcat10.5.4 Port Scanning 10.6 Remote Procedure Calls 10.7 Network Security10.7.1 Typical Vulnerabilities10.7.2 Security Resources 10.8 Looking Forward 10.9 Network Sockets 10.10 Unix Domain Sockets
11.1 Shell Script Basics11.1.1 Limitations of Shell Scripts 11.2 Quoting and Literals11.2.1 Literals11.2.2 Single Quotes11.2.3 Double Quotes11.2.4 Literal Single Quotes 11.3 Special Variables11.3.1 Individual Arguments: $1, $2, and So On11.3.2 Number of Arguments: $#11.3.3 All Arguments: $@11.3.4 Script Name: $011.3.5 Process ID: $$11.3.6 Exit Code: $? 11.4 Exit Codes 11.5 Conditionals11.5.1 A Workaround for Empty Parameter Lists11.5.2 Other Commands for Tests11.5.3 elif11.5.4 Logical Constructs11.5.5 Testing Conditions11.5.6 case 11.6 Loops11.6.1 for Loops11.6.2 while Loops 11.7 Command Substitution 11.8 Temporary File Management 11.9 Here Documents 11.10 Important Shell Script Utilities11.10.1 basename11.10.2 awk11.10.3 sed11.10.4 xargs11.10.5 expr11.10.6 exec 11.11 Subshells 11.12 Including Other Files in Scripts 11.13 Reading User Input 11.14 When (Not) to Use Shell Scripts
12.1 Quick Copy 12.2 rsync12.2.1 Getting Started with rsync12.2.2 Making Exact Copies of a Directory Structure12.2.3 Using the Trailing Slash12.2.4 Excluding Files and Directories12.2.5 Checking Transfers, Adding Safeguards, and Using Verbose Mode12.2.6 Compressing Data12.2.7 Limiting Bandwidth12.2.8 Transferring Files to Your Computer12.2.9 Further rsync Topics12.3.1 File Sharing Usage and Performance12.3.2 File Sharing Security 12.4 Sharing Files with Samba12.4.1 Server Configuration12.4.2 Server Access Control12.4.3 Passwords12.4.4 Manual Server Startup12.4.5 Diagnostics and Logfiles12.4.6 File Share Configuration12.4.7 Home Directories12.4.8 Printer Sharing12.4.9 The Samba Client 12.5 SSHFS 12.6 NFS 12.7 Cloud Storage 12.8 The State of Network File Sharing
13.1 Guidelines for Creating Startup Files 13.2 When to Alter Startup Files 13.3 Shell Startup File Elements13.3.1 The Command Path13.3.2 The Manual Page Path13.3.3 The Prompt13.3.4 Aliases13.3.5 The Permissions Mask 13.4 Startup File Order and Examples13.4.1 The bash Shell13.4.2 The tcsh Shell 13.5 Default User Settings13.5.1 Shell Defaults13.5.2 Editor13.5.3 Pager 13.6 Startup File Pitfalls 13.7 Further Startup Topics
14.1 Desktop Components14.1.1 Framebuffers14.1.2 The X Window System14.1.3 Wayland14.1.4 Window Managers14.1.5 Toolkits14.1.6 Desktop Environments14.1.7 Applications 14.2 Are You Running Wayland or X? 14.3 A Closer Look at Wayland14.3.1 The Compositing Window Manager14.3.2 libinput14.3.3 X Compatibility in Wayland 14.4 A Closer Look at the X Window System14.4.1 Display Managers14.4.2 Network Transparency14.4.3 Ways of Exploring X Clients14.4.4 X Events14.4.5 X Input and Preference Settings 14.5 D-Bus14.5.1 System and Session Instances14.5.2 D-Bus Message Monitoring 14.6 Printing14.6.1 CUPS14.6.2 Format Conversion and Print Filters 14.7 Other Desktop Topics
15.1 The C Compiler15.1.1 Compiling Multiple Source Files15.1.2 Linking with Libraries15.1.3 Working with Shared Libraries15.1.4 Working with Header (Include) Files and Directories 15.2 make15.2.1 A Sample Makefile 15.2.2 Built-in Rules15.2.3 Final Program Build 15.2.4 Dependency Updates15.2.5 Command-Line Arguments and Options15.2.6 Standard Macros and Variables15.2.7 Conventional Targets15.2.8 Makefile Organization 15.3 Lex and Yacc 15.4 Scripting Languages15.4.1 Python15.4.2 Perl15.4.3 Other Scripting Languages 15.5 Java 15.6 Looking Forward: Compiling Packages
16.1 Software Build Systems 16.2 Unpacking C Source Packages 16.3 GNU Autoconf16.3.1 An Autoconf Example16.3.2 Installation Using a Packaging Tool16.3.3 configure Script Options16.3.4 Environment Variables16.3.5 Autoconf Targets16.3.6 Autoconf Logfiles16.3.7 pkg-config 16.4 Installation Practice16.4.1 Where to Install 16.5 Applying a Patch 16.6 Troubleshooting Compiles and Installations16.6.1 Specific Errors 16.7 Looking Forward
17.1 Virtual Machines17.1.1 Hypervisors17.1.2 Hardware in a Virtual Machine17.1.3 Common Uses of Virtual Machines17.1.4 Drawbacks of Virtual Machines 17.2 Containers17.2.1 Docker, Podman, and Privileges17.2.2 A Docker Example17.2.3 LXC17.2.4 Kubernetes17.2.5 Pitfalls of Containers 17.3 Runtime-Based Virtualization

Content preview from How Linux Works, 3rd Edition

16 Introduction to Compiling Software from C Source Code

Most nonproprietary third-party Unix software packages come as source code that you can build and install. One reason for this is that Unix (and Linux itself) has so many different flavors and architectures, it would be difficult to distribute binary packages for all possible platform combinations. The other reason, which is at least as important, is that widespread source code distribution throughout the Unix community encourages users to contribute bug fixes and new features to software, giving meaning to the term open source.

You can get nearly everything you see on a Linux system ...