book

The Linux Programming Interface

Name: The Linux Programming Interface
Author: Michael Kerrisk
ISBN: 9781593272203

by Michael Kerrisk

October 2010

Intermediate to advanced

1552 pages

45h 39m

English

No Starch Press

Read now

Unlock full access

Cover
Title Page
Copyright Page
Brief Contents
Contents in Detail
Preface
Chapter 1: History and Standards
1.1 A Brief History of UNIX and C1.2 A Brief History of Linux1.2.1 The GNU Project1.2.2 The Linux Kernel1.3 Standardization1.3.1 The C Programming Language1.3.2 The First POSIX Standards1.3.3 X/Open Company and The Open Group1.3.4 SUSv3 and POSIX.1-20011.3.5 SUSv4 and POSIX.1-20081.3.6 UNIX Standards Timeline1.3.7 Implementation Standards1.3.8 Linux, Standards, and the Linux Standard Base1.4 Summary
Chapter 2: Fundamental Concepts
2.1 The Core Operating System: The Kernel2.2 The Shell2.3 Users and Groups2.4 Single Directory Hierarchy, Directories, Links, and Files2.5 File I/O Model2.6 Programs2.7 Processes2.8 Memory Mappings2.9 Static and Shared Libraries2.10 Interprocess Communication and Synchronization2.11 Signals2.12 Threads2.13 Process Groups and Shell Job Control2.14 Sessions, Controlling Terminals, and Controlling Processes2.15 Pseudoterminals2.16 Date and Time2.17 Client-Server Architecture2.18 Realtime2.19 The /proc File System2.20 Summary
Chapter 3: System Programming Concepts
3.1 System Calls3.2 Library Functions3.3 The Standard C Library; The GNU C Library (glibc)3.4 Handling Errors from System Calls and Library Functions3.5 Notes on the Example Programs in This Book3.5.1 Command-Line Options and Arguments3.5.2 Common Functions and Header Files3.6 Portability Issues3.6.1 Feature Test Macros3.6.2 System Data Types3.6.3 Miscellaneous Portability Issues3.7 Summary3.8 Exercise
Chapter 4: File I/O: The Universal I/O Model
4.1 Overview4.2 Universality of I/O4.3 Opening a File: open()4.3.1 The open() flags Argument4.3.2 Errors from open()4.3.3 The creat() System Call4.4 Reading from a File: read()4.5 Writing to a File: write()4.6 Closing a File: close()4.7 Changing the File Offset: lseek()4.8 Operations Outside the Universal I/O Model: ioctl()4.9 Summary4.10 Exercises

Chapter 5: File I/O: Further Details
5.1 Atomicity and Race Conditions5.2 File Control Operations: fcntl()5.3 Open File Status Flags5.4 Relationship Between File Descriptors and Open Files5.5 Duplicating File Descriptors5.6 File I/O at a Specified Offset: pread() and pwrite()5.7 Scatter-Gather I/O: readv() and writev()5.8 Truncating a File: truncate() and ftruncate()5.9 Nonblocking I/O5.10 I/O on Large Files5.11 The /dev/fd Directory5.12 Creating Temporary Files5.13 Summary5.14 Exercises
Chapter 6: Processes
6.1 Processes and Programs6.2 Process ID and Parent Process ID6.3 Memory Layout of a Process6.4 Virtual Memory Management6.5 The Stack and Stack Frames6.6 Command-Line Arguments (argc, argv)6.7 Environment List6.8 Performing a Nonlocal Goto: setjmp() and longjmp()6.9 Summary6.10 Exercises
Chapter 7: Memory Allocation
7.1 Allocating Memory on the Heap7.1.1 Adjusting the Program Break: brk() and sbrk()7.1.2 Allocating Memory on the Heap: malloc() and free()7.1.3 Implementation of malloc() and free()7.1.4 Other Methods of Allocating Memory on the Heap7.2 Allocating Memory on the Stack: alloca()7.3 Summary7.4 Exercises
Chapter 8: Users and Groups
8.1 The Password File: /etc/passwd8.2 The Shadow Password File: /etc/shadow8.3 The Group File: /etc/group8.4 Retrieving User and Group Information8.5 Password Encryption and User Authentication8.6 Summary8.7 Exercises
Chapter 9: Process Credentials
9.1 Real User ID and Real Group ID9.2 Effective User ID and Effective Group ID9.3 Set-User-ID and Set-Group-ID Programs9.4 Saved Set-User-ID and Saved Set-Group-ID9.5 File-System User ID and File-System Group ID9.6 Supplementary Group IDs9.7 Retrieving and Modifying Process Credentials9.7.1 Retrieving and Modifying Real, Effective, and Saved Set IDs9.7.2 Retrieving and Modifying File-System IDs9.7.3 Retrieving and Modifying Supplementary Group IDs9.7.4 Summary of Calls for Modifying Process Credentials9.7.5 Example: Displaying Process Credentials9.8 Summary9.9 Exercises
Chapter 10: Time
10.1 Calendar Time10.2 Time-Conversion Functions10.2.1 Converting time_t to Printable Form10.2.2 Converting Between time_t and Broken-Down Time10.2.3 Converting Between Broken-Down Time and Printable Form10.3 Timezones10.4 Locales10.5 Updating the System Clock10.6 The Software Clock (Jiffies)10.7 Process Time10.8 Summary10.9 Exercise
Chapter 11: System Limits and Options
11.1 System Limits11.2 Retrieving System Limits (and Options) at Run Time11.3 Retrieving File-Related Limits (and Options) at Run Time11.4 Indeterminate Limits11.5 System Options11.6 Summary11.7 Exercises
Chapter 12: System and Process Information
12.1 The /proc File System12.1.1 Obtaining Information About a Process: /proc/PID12.1.2 System Information Under /proc12.1.3 Accessing /proc Files12.2 System Identification: uname()12.3 Summary12.4 Exercises
Chapter 13: File I/O Buffering
13.1 Kernel Buffering of File I/O: The Buffer Cache13.2 Buffering in the stdio Library13.3 Controlling Kernel Buffering of File I/O13.4 Summary of I/O Buffering13.5 Advising the Kernel About I/O Patterns13.6 Bypassing the Buffer Cache: Direct I/O13.7 Mixing Library Functions and System Calls for File I/O13.8 Summary13.9 Exercises
Chapter 14: File Systems
14.1 Device Special Files (Devices)14.2 Disks and Partitions14.3 File Systems14.4 I-nodes14.5 The Virtual File System (VFS)14.6 Journaling File Systems14.7 Single Directory Hierarchy and Mount Points14.8 Mounting and Unmounting File Systems14.8.1 Mounting a File System: mount()14.8.2 Unmounting a File System: umount() and umount2()14.9 Advanced Mount Features14.9.1 Mounting a File System at Multiple Mount Points14.9.2 Stacking Multiple Mounts on the Same Mount Point14.9.3 Mount Flags That Are Per-Mount Options14.9.4 Bind Mounts14.9.5 Recursive Bind Mounts14.10 A Virtual Memory File System: tmpfs14.11 Obtaining Information About a File System: statvfs()14.12 Summary14.13 Exercise
Chapter 15: File Attributes
15.1 Retrieving File Information: stat()15.2 File Timestamps15.2.1 Changing File Timestamps with utime() and utimes()15.2.2 Changing File Timestamps with utimensat() and futimens()15.3 File Ownership15.3.1 Ownership of New Files15.3.2 Changing File Ownership: chown(), fchown(), and lchown()15.4 File Permissions15.4.1 Permissions on Regular Files15.4.2 Permissions on Directories15.4.3 Permission-Checking Algorithm15.4.4 Checking File Accessibility: access()15.4.5 Set-User-ID, Set-Group-ID, and Sticky Bits15.4.6 The Process File Mode Creation Mask: umask()15.4.7 Changing File Permissions: chmod() and fchmod()15.5 I-node Flags (ext2 Extended File Attributes)15.6 Summary15.7 Exercises
Chapter 16: Extended Attributes
16.1 Overview16.2 Extended Attribute Implementation Details16.3 System Calls for Manipulating Extended Attributes16.4 Summary16.5 Exercise
Chapter 17: Access Control Lists
17.1 Overview17.2 ACL Permission-Checking Algorithm17.3 Long and Short Text Forms for ACLs17.4 The ACL_MASK Entry and the ACL Group Class17.5 The getfacl and setfacl Commands17.6 Default ACLs and File Creation17.7 ACL Implementation Limits17.8 The ACL API17.9 Summary17.10 Exercise
Chapter 18: Directories and Links
18.1 Directories and (Hard) Links18.2 Symbolic (Soft) Links18.3 Creating and Removing (Hard) Links: link() and unlink()18.4 Changing the Name of a File: rename()18.5 Working with Symbolic Links: symlink() and readlink()18.6 Creating and Removing Directories: mkdir() and rmdir()18.7 Removing a File or Directory: remove()18.8 Reading Directories: opendir() and readdir()18.9 File Tree Walking: nftw()18.10 The Current Working Directory of a Process18.11 Operating Relative to a Directory File Descriptor18.12 Changing the Root Directory of a Process: chroot()18.13 Resolving a Pathname: realpath()18.14 Parsing Pathname Strings: dirname() and basename()18.15 Summary18.16 Exercises
Chapter 19: Monitoring File Events
19.1 Overview19.2 The inotify API19.3 inotify Events19.4 Reading inotify Events19.5 Queue Limits and /proc Files19.6 An Older System for Monitoring File Events: dnotify19.7 Summary19.8 Exercise
Chapter 20: Signals: Fundamental Concepts
20.1 Concepts and Overview20.2 Signal Types and Default Actions20.3 Changing Signal Dispositions: signal()20.4 Introduction to Signal Handlers20.5 Sending Signals: kill()20.6 Checking for the Existence of a Process20.7 Other Ways of Sending Signals: raise() and killpg()20.8 Displaying Signal Descriptions20.9 Signal Sets20.10 The Signal Mask (Blocking Signal Delivery)20.11 Pending Signals20.12 Signals Are Not Queued20.13 Changing Signal Dispositions: sigaction()20.14 Waiting for a Signal: pause()20.15 Summary20.16 Exercises
Chapter 21: Signals: Signal Handlers
21.1 Designing Signal Handlers21.1.1 Signals Are Not Queued (Revisited)21.1.2 Reentrant and Async-Signal-Safe Functions21.1.3 Global Variables and the sig_atomic_t Data Type21.2 Other Methods of Terminating a Signal Handler21.2.1 Performing a Nonlocal Goto from a Signal Handler21.2.2 Terminating a Process Abnormally: abort()21.3 Handling a Signal on an Alternate Stack: sigaltstack()21.4 The SA_SIGINFO Flag21.5 Interruption and Restarting of System Calls21.6 Summary21.7 Exercise
Chapter 22: Signals: Advanced Features
22.1 Core Dump Files22.2 Special Cases for Delivery, Disposition, and Handling22.3 Interruptible and Uninterruptible Process Sleep States22.4 Hardware-Generated Signals22.5 Synchronous and Asynchronous Signal Generation22.6 Timing and Order of Signal Delivery22.7 Implementation and Portability of signal()22.8 Realtime Signals22.8.1 Sending Realtime Signals22.8.2 Handling Realtime Signals22.9 Waiting for a Signal Using a Mask: sigsuspend()22.10 Synchronously Waiting for a Signal22.11 Fetching Signals via a File Descriptor22.12 Interprocess Communication with Signals22.13 Earlier Signal APIs (System V and BSD)22.14 Summary22.15 Exercises
Chapter 23: Timers and Sleeping
23.1 Interval Timers23.2 Scheduling and Accuracy of Timers23.3 Setting Timeouts on Blocking Operations23.4 Suspending Execution for a Fixed Interval (Sleeping)23.4.1 Low-Resolution Sleeping: sleep()23.4.2 High-Resolution Sleeping: nanosleep()23.5 POSIX Clocks23.5.1 Retrieving the Value of a Clock: clock_gettime()23.5.2 Setting the Value of a Clock: clock_settime()23.5.3 Obtaining the Clock ID of a Specific Process or Thread23.5.4 Improved High-Resolution Sleeping: clock_nanosleep()23.6 POSIX Interval Timers23.6.1 Creating a Timer: timer_create()23.6.2 Arming and Disarming a Timer: timer_settime()23.6.3 Retrieving the Current Value of a Timer: timer_gettime()23.6.4 Deleting a Timer: timer_delete()23.6.5 Notification via a Signal23.6.6 Timer Overruns23.6.7 Notification via a Thread23.7 Timers That Notify via File Descriptors: The timerfd API23.8 Summary23.9 Exercises
Chapter 24: Process Creation
24.1 Overview of fork(), exit(), wait(), and execve()24.2 Creating a New Process: fork()24.2.1 File Sharing Between Parent and Child24.2.2 Memory Semantics of fork()24.3 The vfork() System Call24.4 Race Conditions After fork()24.5 Avoiding Race Conditions by Synchronizing with Signals24.6 Summary24.7 Exercises
Chapter 25: Process Termination
25.1 Terminating a Process: _exit() and exit()25.2 Details of Process Termination25.3 Exit Handlers25.4 Interactions Between fork(), stdio Buffers, and _exit()25.5 Summary25.6 Exercise
Chapter 26: Monitoring Child Processes
26.1 Waiting on a Child Process26.1.1 The wait() System Call26.1.2 The waitpid() System Call26.1.3 The Wait Status Value26.1.4 Process Termination from a Signal Handler26.1.5 The waitid() System Call26.1.6 The wait3() and wait4() System Calls26.2 Orphans and Zombies26.3 The SIGCHLD Signal26.3.1 Establishing a Handler for SIGCHLD26.3.2 Delivery of SIGCHLD for Stopped Children26.3.3 Ignoring Dead Child Processes26.4 Summary26.5 Exercises
Chapter 27: Program Execution
27.1 Executing a New Program: execve()27.2 The exec() Library Functions27.2.1 The PATH Environment Variable27.2.2 Specifying Program Arguments as a List27.2.3 Passing the Caller’s Environment to the New Program27.2.4 Executing a File Referred to by a Descriptor: fexecve()27.3 Interpreter Scripts27.4 File Descriptors and exec()27.5 Signals and exec()27.6 Executing a Shell Command: system()27.7 Implementing system()27.8 Summary27.9 Exercises
Chapter 28: Process Creation and Program Execution in More Detail
28.1 Process Accounting28.2 The clone() System Call28.2.1 The clone() flags Argument28.2.2 Extensions to waitpid() for Cloned Children28.3 Speed of Process Creation28.4 Effect of exec() and fork() on Process Attributes28.5 Summary28.6 Exercise
Chapter 29: Threads: Introduction
29.1 Overview29.2 Background Details of the Pthreads API29.3 Thread Creation29.4 Thread Termination29.5 Thread IDs29.6 Joining with a Terminated Thread29.7 Detaching a Thread29.8 Thread Attributes29.9 Threads Versus Processes29.10 Summary29.11 Exercises
Chapter 30: Threads: Thread Synchronization
30.1 Protecting Accesses to Shared Variables: Mutexes30.1.1 Statically Allocated Mutexes30.1.2 Locking and Unlocking a Mutex30.1.3 Performance of Mutexes30.1.4 Mutex Deadlocks30.1.5 Dynamically Initializing a Mutex30.1.6 Mutex Attributes30.1.7 Mutex Types30.2 Signaling Changes of State: Condition Variables30.2.1 Statically Allocated Condition Variables30.2.2 Signaling and Waiting on Condition Variables30.2.3 Testing a Condition Variable’s Predicate30.2.4 Example Program: Joining Any Terminated Thread30.2.5 Dynamically Allocated Condition Variables30.3 Summary30.4 Exercises
Chapter 31: Threads: Thread Safety and Per-Thread Storage
31.1 Thread Safety (and Reentrancy Revisited)31.2 One-Time Initialization31.3 Thread-Specific Data31.3.1 Thread-Specific Data from the Library Function’s Perspective31.3.2 Overview of the Thread-Specific Data API31.3.3 Details of the Thread-Specific Data API31.3.4 Employing the Thread-Specific Data API31.3.5 Thread-Specific Data Implementation Limits31.4 Thread-Local Storage31.5 Summary31.6 Exercises
Chapter 32: Threads: Thread Cancellation
32.1 Canceling a Thread32.2 Cancellation State and Type32.3 Cancellation Points32.4 Testing for Thread Cancellation32.5 Cleanup Handlers32.6 Asynchronous Cancelability32.7 Summary
Chapter 33: Threads: Further Details
33.1 Thread Stacks33.2 Threads and Signals33.2.1 How the UNIX Signal Model Maps to Threads33.2.2 Manipulating the Thread Signal Mask33.2.3 Sending a Signal to a Thread33.2.4 Dealing with Asynchronous Signals Sanely33.3 Threads and Process Control33.4 Thread Implementation Models33.5 Linux Implementations of POSIX Threads33.5.1 LinuxThreads33.5.2 NPTL33.5.3 Which Threading Implementation?33.6 Advanced Features of the Pthreads API33.7 Summary33.8 Exercises
Chapter 34: Process Groups, Sessions, and Job Control
34.1 Overview34.2 Process Groups34.3 Sessions34.4 Controlling Terminals and Controlling Processes34.5 Foreground and Background Process Groups34.6 The SIGHUP Signal34.6.1 Handling of SIGHUP by the Shell34.6.2 SIGHUP and Termination of the Controlling Process34.7 Job Control34.7.1 Using Job Control Within the Shell34.7.2 Implementing Job Control34.7.3 Handling Job-Control Signals34.7.4 Orphaned Process Groups (and SIGHUP Revisited)34.8 Summary34.9 Exercises
Chapter 35: Process Priorities and Scheduling
35.1 Process Priorities (Nice Values)35.2 Overview of Realtime Process Scheduling35.2.1 The SCHED_RR Policy35.2.2 The SCHED_FIFO Policy35.2.3 The SCHED_BATCH and SCHED_IDLE Policies35.3 Realtime Process Scheduling API35.3.1 Realtime Priority Ranges35.3.2 Modifying and Retrieving Policies and Priorities35.3.3 Relinquishing the CPU35.3.4 The SCHED_RR Time Slice35.4 CPU Affinity35.5 Summary35.6 Exercises
Chapter 36: Process Resources
36.1 Process Resource Usage36.2 Process Resource Limits36.3 Details of Specific Resource Limits36.4 Summary36.5 Exercises
Chapter 37: Daemons
37.1 Overview37.2 Creating a Daemon37.3 Guidelines for Writing Daemons37.4 Using SIGHUP to Reinitialize a Daemon37.5 Logging Messages and Errors Using syslog37.5.1 Overview37.5.2 The syslog API37.5.3 The /etc/syslog.conf File37.6 Summary37.7 Exercise
Chapter 38: Writing Secure Privileged Programs
38.1 Is a Set-User-ID or Set-Group-ID Program Required?38.2 Operate with Least Privilege38.3 Be Careful When Executing a Program38.4 Avoid Exposing Sensitive Information38.5 Confine the Process38.6 Beware of Signals and Race Conditions38.7 Pitfalls When Performing File Operations and File I/O38.8 Don’t Trust Inputs or the Environment38.9 Beware of Buffer Overruns38.10 Beware of Denial-of-Service Attacks38.11 Check Return Statuses and Fail Safely38.12 Summary38.13 Exercises
Chapter 39: Capabilities
39.1 Rationale for Capabilities39.2 The Linux Capabilities39.3 Process and File Capabilities39.3.1 Process Capabilities39.3.2 File Capabilities39.3.3 Purpose of the Process Permitted and Effective Capability Sets39.3.4 Purpose of the File Permitted and Effective Capability Sets39.3.5 Purpose of the Process and File Inheritable Sets39.3.6 Assigning and Viewing File Capabilities from the Shell39.4 The Modern Capabilities Implementation39.5 Transformation of Process Capabilities During exec()39.5.1 Capability Bounding Set39.5.2 Preserving root Semantics39.6 Effect on Process Capabilities of Changing User IDs39.7 Changing Process Capabilities Programmatically39.8 Creating Capabilities-Only Environments39.9 Discovering the Capabilities Required by a Program39.10 Older Kernels and Systems Without File Capabilities39.11 Summary39.12 Exercise
Chapter 40: Login Accounting
40.1 Overview of the utmp and wtmp Files40.2 The utmpx API40.3 The utmpx Structure40.4 Retrieving Information from the utmp and wtmp Files40.5 Retrieving the Login Name: getlogin()40.6 Updating the utmp and wtmp Files for a Login Session40.7 The lastlog File40.8 Summary40.9 Exercises
Chapter 41: Fundamentals of Shared Libraries
41.1 Object Libraries41.2 Static Libraries41.3 Overview of Shared Libraries41.4 Creating and Using Shared Libraries—A First Pass41.4.1 Creating a Shared Library41.4.2 Position-Independent Code41.4.3 Using a Shared Library41.4.4 The Shared Library Soname41.5 Useful Tools for Working with Shared Libraries41.6 Shared Library Versions and Naming Conventions41.7 Installing Shared Libraries41.8 Compatible Versus Incompatible Libraries41.9 Upgrading Shared Libraries41.10 Specifying Library Search Directories in an Object File41.11 Finding Shared Libraries at Run Time41.12 Run-Time Symbol Resolution41.13 Using a Static Library Instead of a Shared Library41.14 Summary41.15 Exercise
Chapter 42: Advanced Features of Shared Libraries
42.1 Dynamically Loaded Libraries42.1.1 Opening a Shared Library: dlopen()42.1.2 Diagnosing Errors: dlerror()42.1.3 Obtaining the Address of a Symbol: dlsym()42.1.4 Closing a Shared Library: dlclose()42.1.5 Obtaining Information About Loaded Symbols: dladdr()42.1.6 Accessing Symbols in the Main Program42.2 Controlling Symbol Visibility42.3 Linker Version Scripts42.3.1 Controlling Symbol Visibility with Version Scripts42.3.2 Symbol Versioning42.4 Initialization and Finalization Functions42.5 Preloading Shared Libraries42.6 Monitoring the Dynamic Linker: LD_DEBUG42.7 Summary42.8 Exercises
Chapter 43: Interprocess Communication Overview
43.1 A Taxonomy of IPC Facilities43.2 Communication Facilities43.3 Synchronization Facilities43.4 Comparing IPC Facilities43.5 Summary43.6 Exercises
Chapter 44: Pipes and FIFOs
44.1 Overview44.2 Creating and Using Pipes44.3 Pipes as a Method of Process Synchronization44.4 Using Pipes to Connect Filters44.5 Talking to a Shell Command via a Pipe: popen()44.6 Pipes and stdio Buffering44.7 FIFOs44.8 A Client-Server Application Using FIFOs44.9 Nonblocking I/O44.10 Semantics of read() and write() on Pipes and FIFOs44.11 Summary44.12 Exercises
Chapter 45: Introduction to System V IPC
45.1 API Overview45.2 IPC Keys45.3 Associated Data Structure and Object Permissions45.4 IPC Identifiers and Client-Server Applications45.5 Algorithm Employed by System V IPC get Calls45.6 The ipcs and ipcrm Commands45.7 Obtaining a List of All IPC Objects45.8 IPC Limits45.9 Summary45.10 Exercises
Chapter 46: System V Message Queues
46.1 Creating or Opening a Message Queue46.2 Exchanging Messages46.2.1 Sending Messages46.2.2 Receiving Messages46.3 Message Queue Control Operations46.4 Message Queue Associated Data Structure46.5 Message Queue Limits46.6 Displaying All Message Queues on the System46.7 Client-Server Programming with Message Queues46.8 A File-Server Application Using Message Queues46.9 Disadvantages of System V Message Queues46.10 Summary46.11 Exercises
Chapter 47: System V Semaphores
47.1 Overview47.2 Creating or Opening a Semaphore Set47.3 Semaphore Control Operations47.4 Semaphore Associated Data Structure47.5 Semaphore Initialization47.6 Semaphore Operations47.7 Handling of Multiple Blocked Semaphore Operations47.8 Semaphore Undo Values47.9 Implementing a Binary Semaphores Protocol47.10 Semaphore Limits47.11 Disadvantages of System V Semaphores47.12 Summary47.13 Exercises
Chapter 48: System V Shared Memory
48.1 Overview48.2 Creating or Opening a Shared Memory Segment48.3 Using Shared Memory48.4 Example: Transferring Data via Shared Memory48.5 Location of Shared Memory in Virtual Memory48.6 Storing Pointers in Shared Memory48.7 Shared Memory Control Operations48.8 Shared Memory Associated Data Structure48.9 Shared Memory Limits48.10 Summary48.11 Exercises
Chapter 49: Memory Mappings
49.1 Overview49.2 Creating a Mapping: mmap()49.3 Unmapping a Mapped Region: munmap()49.4 File Mappings49.4.1 Private File Mappings49.4.2 Shared File Mappings49.4.3 Boundary Cases49.4.4 Memory Protection and File Access Mode Interactions49.5 Synchronizing a Mapped Region: msync()49.6 Additional mmap() Flags49.7 Anonymous Mappings49.8 Remapping a Mapped Region: mremap()49.9 MAP_NORESERVE and Swap Space Overcommitting49.10 The MAP_FIXED Flag49.11 Nonlinear Mappings: remap_file_pages()49.12 Summary49.13 Exercises
Chapter 50: Virtual Memory Operations
50.1 Changing Memory Protection: mprotect()50.2 Memory Locking: mlock() and mlockall()50.3 Determining Memory Residence: mincore()50.4 Advising Future Memory Usage Patterns: madvise()50.5 Summary50.6 Exercises
Chapter 51: Introduction to POSIX IPC
51.1 API Overview51.2 Comparison of System V IPC and POSIX IPC51.3 Summary
Chapter 52: POSIX Message Queues
52.1 Overview52.2 Opening, Closing, and Unlinking a Message Queue52.3 Relationship Between Descriptors and Message Queues52.4 Message Queue Attributes52.5 Exchanging Messages52.5.1 Sending Messages52.5.2 Receiving Messages52.5.3 Sending and Receiving Messages with a Timeout52.6 Message Notification52.6.1 Receiving Notification via a Signal52.6.2 Receiving Notification via a Thread52.7 Linux-Specific Features52.8 Message Queue Limits52.9 Comparison of POSIX and System V Message Queues52.10 Summary52.11 Exercises
Chapter 53: POSIX Semaphores
53.1 Overview53.2 Named Semaphores53.2.1 Opening a Named Semaphore53.2.2 Closing a Semaphore53.2.3 Removing a Named Semaphore53.3 Semaphore Operations53.3.1 Waiting on a Semaphore53.3.2 Posting a Semaphore53.3.3 Retrieving the Current Value of a Semaphore53.4 Unnamed Semaphores53.4.1 Initializing an Unnamed Semaphore53.4.2 Destroying an Unnamed Semaphore53.5 Comparisons with Other Synchronization Techniques53.6 Semaphore Limits53.7 Summary53.8 Exercises
Chapter 54: POSIX Shared Memory
54.1 Overview54.2 Creating Shared Memory Objects54.3 Using Shared Memory Objects54.4 Removing Shared Memory Objects54.5 Comparisons Between Shared Memory APIs54.6 Summary54.7 Exercise
Chapter 55: File Locking
55.1 Overview55.2 File Locking with flock()55.2.1 Semantics of Lock Inheritance and Release55.2.2 Limitations of flock()55.3 Record Locking with fcntl()55.3.1 Deadlock55.3.2 Example: An Interactive Locking Program55.3.3 Example: A Library of Locking Functions55.3.4 Lock Limits and Performance55.3.5 Semantics of Lock Inheritance and Release55.3.6 Lock Starvation and Priority of Queued Lock Requests55.4 Mandatory Locking55.5 The /proc/locks File55.6 Running Just One Instance of a Program55.7 Older Locking Techniques55.8 Summary55.9 Exercises
Chapter 56: Sockets: Introduction
56.1 Overview56.2 Creating a Socket: socket()56.3 Binding a Socket to an Address: bind()56.4 Generic Socket Address Structures: struct sockaddr56.5 Stream Sockets56.5.1 Listening for Incoming Connections: listen()56.5.2 Accepting a Connection: accept()56.5.3 Connecting to a Peer Socket: connect()56.5.4 I/O on Stream Sockets56.5.5 Connection Termination: close()56.6 Datagram Sockets56.6.1 Exchanging Datagrams: recvfrom() and sendto()56.6.2 Using connect() with Datagram Sockets56.7 Summary
Chapter 57: Sockets: UNIX Domain
57.1 UNIX Domain Socket Addresses: struct sockaddr_un57.2 Stream Sockets in the UNIX Domain57.3 Datagram Sockets in the UNIX Domain57.4 UNIX Domain Socket Permissions57.5 Creating a Connected Socket Pair: socketpair()57.6 The Linux Abstract Socket Namespace57.7 Summary57.8 Exercises
Chapter 58: Sockets: Fundamentals of TCP/IP Networks
58.1 Internets58.2 Networking Protocols and Layers58.3 The Data-Link Layer58.4 The Network Layer: IP58.5 IP Addresses58.6 The Transport Layer58.6.1 Port Numbers58.6.2 User Datagram Protocol (UDP)58.6.3 Transmission Control Protocol (TCP)58.7 Requests for Comments (RFCs)58.8 Summary
Chapter 59: Sockets: Internet Domains
59.1 Internet Domain Sockets59.2 Network Byte Order59.3 Data Representation59.4 Internet Socket Addresses59.5 Overview of Host and Service Conversion Functions59.6 The inet_pton() and inet_ntop() Functions59.7 Client-Server Example (Datagram Sockets)59.8 Domain Name System (DNS)59.9 The /etc/services File59.10 Protocol-Independent Host and Service Conversion59.10.1 The getaddrinfo() Function59.10.2 Freeing addrinfo Lists: freeaddrinfo()59.10.3 Diagnosing Errors: gai_strerror()59.10.4 The getnameinfo() Function59.11 Client-Server Example (Stream Sockets)59.12 An Internet Domain Sockets Library59.13 Obsolete APIs for Host and Service Conversions59.13.1 The inet_aton() and inet_ntoa() Functions59.13.2 The gethostbyname() and gethostbyaddr() Functions59.13.3 The getservbyname() and getservbyport() Functions59.14 UNIX Versus Internet Domain Sockets59.15 Further Information59.16 Summary59.17 Exercises
Chapter 60: Sockets: Server Design
60.1 Iterative and Concurrent Servers60.2 An Iterative UDP echo Server60.3 A Concurrent TCP echo Server60.4 Other Concurrent Server Designs60.5 The inetd (Internet Superserver) Daemon60.6 Summary60.7 Exercises
Chapter 61: Sockets: Advanced Topics
61.1 Partial Reads and Writes on Stream Sockets61.2 The shutdown() System Call61.3 Socket-Specific I/O System Calls: recv() and send()61.4 The sendfile() System Call61.5 Retrieving Socket Addresses61.6 A Closer Look at TCP61.6.1 Format of a TCP Segment61.6.2 TCP Sequence Numbers and Acknowledgements61.6.3 TCP State Machine and State Transition Diagram61.6.4 TCP Connection Establishment61.6.5 TCP Connection Termination61.6.6 Calling shutdown() on a TCP Socket61.6.7 The TIME_WAIT State61.7 Monitoring Sockets: netstat61.8 Using tcpdump to Monitor TCP Traffic61.9 Socket Options61.10 The SO_REUSEADDR Socket Option61.11 Inheritance of Flags and Options Across accept()61.12 TCP Versus UDP61.13 Advanced Features61.13.1 Out-of-Band Data61.13.2 The sendmsg() and recvmsg() System Calls61.13.3 Passing File Descriptors61.13.4 Receiving Sender Credentials61.13.5 Sequenced-Packet Sockets61.13.6 SCTP and DCCP Transport-Layer Protocols61.14 Summary61.15 Exercises
Chapter 62: Terminals
62.1 Overview62.2 Retrieving and Modifying Terminal Attributes62.3 The stty Command62.4 Terminal Special Characters62.5 Terminal Flags62.6 Terminal I/O Modes62.6.1 Canonical Mode62.6.2 Noncanonical Mode62.6.3 Cooked, Cbreak, and Raw Modes62.7 Terminal Line Speed (Bit Rate)62.8 Terminal Line Control62.9 Terminal Window Size62.10 Terminal Identification62.11 Summary62.12 Exercises
Chapter 63: Alternative I/O Models
63.1 Overview63.1.1 Level-Triggered and Edge-Triggered Notification63.1.2 Employing Nonblocking I/O with Alternative I/O Models63.2 I/O Multiplexing63.2.1 The select() System Call63.2.2 The poll() System Call63.2.3 When Is a File Descriptor Ready?63.2.4 Comparison of select() and poll()63.2.5 Problems with select() and poll()63.3 Signal-Driven I/O63.3.1 When Is “I/O Possible” Signaled?63.3.2 Refining the Use of Signal-Driven I/O63.4 The epoll API63.4.1 Creating an epoll Instance: epoll_create()63.4.2 Modifying the epoll Interest List: epoll_ctl()63.4.3 Waiting for Events: epoll_wait()63.4.4 A Closer Look at epoll Semantics63.4.5 Performance of epoll Versus I/O Multiplexing63.4.6 Edge-Triggered Notification63.5 Waiting on Signals and File Descriptors63.5.1 The pselect() System Call63.5.2 The Self-Pipe Trick63.6 Summary63.7 Exercises
Chapter 64: Pseudoterminals
64.1 Overview64.2 UNIX 98 Pseudoterminals64.2.1 Opening an Unused Master: posix_openpt()64.2.2 Changing Slave Ownership and Permissions: grantpt()64.2.3 Unlocking the Slave: unlockpt()64.2.4 Obtaining the Name of the Slave: ptsname()64.3 Opening a Master: ptyMasterOpen()64.4 Connecting Processes with a Pseudoterminal: ptyFork()64.5 Pseudoterminal I/O64.6 Implementing script(1)64.7 Terminal Attributes and Window Size64.8 BSD Pseudoterminals64.9 Summary64.10 Exercises
Appendix A: Tracing System Calls
Appendix B: Parsing Command-Line Options
Appendix C: Casting the NULL Pointer
Appendix D: Kernel Configuration
Appendix E: Further Sources of Information
Appendix F: Solutions to Selected Exercises
Bibliography
Index

Content preview from The Linux Programming Interface

13FILE I/O BUFFERING

In the interests of speed and efficiency, I/O system calls (i.e., the kernel) and the I/O functions of the standard C library (i.e., the stdio functions) buffer data when operating on disk files. In this chapter, we describe both types of buffering and consider how they affect application performance. We also look at various techniques for influencing and disabling both types of buffering, and look at a technique called direct I/O, which is useful for bypassing kernel buffering in certain circumstances.

13.1 Kernel Buffering of File I/O: The Buffer Cache

When working with disk files, the read() and write() system calls don’t directly initiate disk access. Instead, they simply copy data between a user-space buffer and a buffer ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Understanding the Linux Kernel, 3rd Edition

Publisher Resources

ISBN: 9781593272203Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

The Linux Programming Interface

by Michael Kerrisk

13FILE I/O BUFFERING

13.1 Kernel Buffering of File I/O: The Buffer Cache

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.