Unix™ Systems Programming: Communication, Concurrency, and Threads

Unix™ Systems Programming: Communication, Concurrency, and Threads

by Kay A. Robbins, Steven Robbins
Released June 2003
Publisher(s): Pearson
ISBN: 0130424110

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

UNIX Systems Programming: Communication, Concurrency, and Threadsby Kay A. Robbins and Steven Robbins

  • UNIX processes, files, and special files

  • Signals and timers

  • POSIX threads, semaphores, and IPC

  • TCP, UDP, multicast, and the Web

  • Features projects on Internet radio, server performance, timers, web caching, and shells

    • Learn how to design and implement reliable UNIX software whether you are using Linux, Solaris, Mac OS X, or another POSIX-based system.

    This completely updated classic (originally titled Practical UNIX Programming) demonstrates how to design complex software to get the most from the UNIX operating system. UNIX Systems Programming provides a clear and easy-to-understand introduction to the essentials of UNIX programming. Starting with short code snippets that illustrate how to use system calls, Robbins and Robbins move quickly to hands-on projects that help readers expand their skill levels.

    This practical guide thoroughly explores communication, concurrency,and multithreading. Known for its comprehensive and lucid explanationsof complicated topics such as signals and concurrency, the bookfeatures practical examples, exercises, reusable code, and simplifiedlibraries for use in network communication applications.

    A self-contained reference that relies on the latest UNIX standards,UNIX Systems Programming provides thorough coverage of files, signals,semaphores, POSIX threads, and client-server communication. Thisedition features all-new chapters on the Web, UDP, and serverperformance. The sample material has been tested extensively in theclassroom.

    PRENTICE HALL

    Professional Technical Reference

    Upper Saddle River, NJ 07458

    www.phptr.com

    ISBN: 0-13-042411-0

    Table of contents

    1. Copyright
      1. Dedication
    2. About the Web Site
    3. Preface
      1. Acknowledgments
    4. I. Fundamentals
      1. 1. Technology’s Impact on Programs
        1. 1.1. Terminology of Change
        2. 1.2. Time and Speed
        3. 1.3. Multiprogramming and Time Sharing
        4. 1.4. Concurrency at the Applications Level
          1. 1.4.1. Interrupts
          2. 1.4.2. Signals
          3. 1.4.3. Input and output
          4. 1.4.4. Processes, threads and the sharing of resources
          5. 1.4.5. Multiple processors with shared memory
          6. 1.4.6. The network as the computer
        5. 1.5. Security and Fault Tolerance
        6. 1.6. Buffer Overflows for Breaking and Entering
          1. 1.6.1. Consequences of buffer overflows
          2. 1.6.2. Buffer overflows and security
        7. 1.7. UNIX Standards
        8. 1.8. Additional Reading
      2. 2. Programs, Processes and Threads
        1. 2.1. How a Program Becomes a Process
        2. 2.2. Threads and Thread of Execution
        3. 2.3. Layout of a Program Image
        4. 2.4. Library Function Calls
        5. 2.5. Function Return Values and Errors
        6. 2.6. Argument Arrays
          1. 2.6.1. Creating an argument array with makeargv
          2. 2.6.2. Implementation of makeargv
        7. 2.7. Thread-Safe Functions
        8. 2.8. Use of Static Variables
        9. 2.9. Structure of Static Objects
        10. 2.10. Process Environment
        11. 2.11. Process Termination
        12. 2.12. Exercise: An env Utility
        13. 2.13. Exercise: Message Logging
        14. 2.14. Additional Reading
      3. 3. Processes in UNIX
        1. 3.1. Process Identification
        2. 3.2. Process State
        3. 3.3. UNIX Process Creation and fork
        4. 3.4. The wait Function
          1. 3.4.1. Status values
        5. 3.5. The exec Function
        6. 3.6. Background Processes and Daemons
        7. 3.7. Critical Sections
        8. 3.8. Exercise: Process Chains
        9. 3.9. Exercise: Process Fans
        10. 3.10. Additional Reading
      4. 4. UNIX I/O
        1. 4.1. Device Terminology
        2. 4.2. Reading and Writing
        3. 4.3. Opening and Closing Files
        4. 4.4. The select Function
        5. 4.5. The poll Function
        6. 4.6. File Representation
          1. 4.6.1. File descriptors
          2. 4.6.2. File pointers and buffering
          3. 4.6.3. Inheritance of file descriptors
        7. 4.7. Filters and Redirection
        8. 4.8. File Control
        9. 4.9. Exercise: Atomic Logging
          1. 4.9.1. An atomic logging library
        10. 4.10. Exercise: A cat Utility
        11. 4.11. Additional Reading
      5. 5. Files and Directories
        1. 5.1. UNIX File System Navigation
          1. 5.1.1. The current working directory
          2. 5.1.2. Search paths
        2. 5.2. Directory Access
          1. 5.2.1. Accessing file status information
          2. 5.2.2. Determining the type of a file
        3. 5.3. UNIX File System Implementation
          1. 5.3.1. UNIX file implementation
          2. 5.3.2. Directory implementation
        4. 5.4. Hard Links and Symbolic Links
          1. 5.4.1. Creating or removing a link
          2. 5.4.2. Creating and removing symbolic links
        5. 5.5. Exercise: The which Command
        6. 5.6. Exercise: Biffing
        7. 5.7. Exercise: News biff
        8. 5.8. Exercise: Traversing Directories
        9. 5.9. Additional Reading
      6. 6. UNIX Special Files
        1. 6.1. Pipes
        2. 6.2. Pipelines
        3. 6.3. FIFOs
        4. 6.4. Pipes and the Client-Server Model
        5. 6.5. Terminal Control
          1. 6.5.1. Canonical and noncanonical input processing
        6. 6.6. Audio Device
        7. 6.7. Exercise: Audio
        8. 6.8. Exercise: Barriers
        9. 6.9. Exercise: The stty Command
        10. 6.10. Exercise: Client-Server Revisited
        11. 6.11. Additional Reading
      7. 7. Project: The Token Ring
        1. 7.1. Ring Topology
        2. 7.2. Ring Formation
        3. 7.3. Ring Exploration
        4. 7.4. Simple Communication
        5. 7.5. Mutual Exclusion with Tokens
        6. 7.6. Mutual Exclusion by Voting
        7. 7.7. Leader Election on an Anonymous Ring
        8. 7.8. Token Ring for Communication
        9. 7.9. Pipelined Preprocessor
        10. 7.10. Parallel Ring Algorithms
          1. 7.10.1. Image filtering
            1. Filtering algorithms on the ring
            2. A bidirectional ring
            3. Block computation
          2. 7.10.2. Matrix multiplication
        11. 7.11. Flexible Ring
        12. 7.12. Additional Reading
    5. II. Asynchronous Events
      1. 8. Signals
        1. 8.1. Basic Signal Concepts
        2. 8.2. Generating Signals
        3. 8.3. Manipulating Signal Masks and Signal Sets
        4. 8.4. Catching and Ignoring Signals—sigaction
        5. 8.5. Waiting for Signals—pause, sigsuspend and sigwait
          1. 8.5.1. The pause function
          2. 8.5.2. The sigsuspend function
          3. 8.5.3. The sigwait function
        6. 8.6. Handling Signals: Errors and Async-signal Safety
        7. 8.7. Program Control with siglongjmp and sigsetjmp
        8. 8.8. Programming with Asynchronous I/O
        9. 8.9. Exercise: Dumping Statistics
        10. 8.10. Exercise: Spooling a Slow Device
        11. 8.11. Additional Reading
      2. 9. Times and Timers
        1. 9.1. POSIX Times
          1. 9.1.1. Expressing time in seconds since the Epoch
          2. 9.1.2. Displaying date and time
          3. 9.1.3. Using struct timeval to express time
          4. 9.1.4. Using realtime clocks
          5. 9.1.5. Contrasting elapsed time to processor time
        2. 9.2. Sleep Functions
        3. 9.3. POSIX:XSI Interval Timers
        4. 9.4. Realtime Signals
        5. 9.5. POSIX:TMR Interval Timers
        6. 9.6. Timer Drift, Overruns and Absolute Time
        7. 9.7. Additional Reading
      3. 10. Project: Virtual Timers
        1. 10.1. Project Overview
        2. 10.2. Simple Timers
        3. 10.3. Setting One of Five Single Timers
          1. 10.3.1. The virtualtimers object
          2. 10.3.2. The hardwaretimer object
          3. 10.3.3. Main program implementation
          4. 10.3.4. Instrumentation of the timer code with show
        4. 10.4. Using Multiple Timers
          1. 10.4.1. Setting multiple timers
          2. 10.4.2. Testing with multiple timers
        5. 10.5. A Robust Implementation of Multiple Timers
        6. 10.6. POSIX:TMR Timer Implementation
        7. 10.7. mycron, a Small Cron Facility
        8. 10.8. Additional Reading
      4. 11. Project: Cracking Shells
        1. 11.1. Building a Simple Shell
        2. 11.2. Redirection
        3. 11.3. Pipelines
        4. 11.4. Signal Handling in the Foreground
        5. 11.5. Process Groups, Sessions and Controlling Terminals
          1. 11.5.1. Process Groups
          2. 11.5.2. Sessions
        6. 11.6. Background Processes in ush
        7. 11.7. Job Control
        8. 11.8. Job Control for ush
          1. 11.8.1. A job list object
          2. 11.8.2. The job list in ush
          3. 11.8.3. Job control in ush
          4. 11.8.4. Process behavior in waiting for a pipeline
        9. 11.9. Additional Reading
    6. III. Concurrency
      1. 12. POSIX Threads
        1. 12.1. A Motivating Problem: Monitoring File Descriptors
        2. 12.2. Use of Threads to Monitor Multiple File Descriptors
        3. 12.3. Thread Management
          1. 12.3.1. Referencing threads by ID
          2. 12.3.2. Creating a thread
          3. 12.3.3. Detaching and joining
          4. 12.3.4. Exiting and cancellation
          5. 12.3.5. Passing parameters to threads and returning values
        4. 12.4. Thread Safety
        5. 12.5. User Threads versus Kernel Threads
        6. 12.6. Thread Attributes
          1. 12.6.1. The thread state
          2. 12.6.2. The thread stack
          3. 12.6.3. Thread scheduling
        7. 12.7. Exercise: Parallel File Copy
        8. 12.8. Additional Reading
      2. 13. Thread Synchronization
        1. 13.1. POSIX Synchronization Functions
        2. 13.2. Mutex Locks
          1. 13.2.1. Creating and initializing a mutex
          2. 13.2.2. Destroying a mutex
          3. 13.2.3. Locking and unlocking a mutex
          4. 13.2.4. Protecting unsafe library functions
          5. 13.2.5. Synchronizing flags and global values
          6. 13.2.6. Making data structures thread-safe
        3. 13.3. At-Most-Once and At-Least-Once-Execution
        4. 13.4. Condition Variables
          1. 13.4.1. Creating and destroying condition variables
          2. 13.4.2. Waiting and signaling on condition variables
        5. 13.5. Signal Handling and Threads
          1. 13.5.1. Directing a signal to a particular thread
          2. 13.5.2. Masking signals for threads
          3. 13.5.3. Dedicating threads for signal handling
        6. 13.6. Readers and Writers
        7. 13.7. A strerror_r Implementation
        8. 13.8. Deadlocks and Other Pesky Problems
        9. 13.9. Exercise: Multiple Barriers
        10. 13.10. Additional Reading
      3. 14. Critical Sections and Semaphores
        1. 14.1. Dealing with Critical Sections
        2. 14.2. Semaphores
        3. 14.3. POSIX:SEM Unnamed Semaphores
        4. 14.4. POSIX:SEM Semaphore Operations
        5. 14.5. POSIX:SEM Named Semaphores
          1. 14.5.1. Creating and opening named semaphores
          2. 14.5.2. Closing and unlinking named semaphores
        6. 14.6. Exercise: License Manager
          1. 14.6.1. License object
          2. 14.6.2. The runsim main program
          3. 14.6.3. Extensions to the license manager
        7. 14.7. Additional Reading
      4. 15. POSIX IPC
        1. 15.1. POSIX:XSI Interprocess Communication
          1. 15.1.1. Identifying and accessing IPC objects
          2. 15.1.2. Accessing POSIX:XSI IPC resources from the shell
        2. 15.2. POSIX:XSI Semaphore Sets
          1. 15.2.1. Semaphore creation
          2. 15.2.2. Semaphore control
          3. 15.2.3. POSIX semaphore set operations
        3. 15.3. POSIX:XSI Shared Memory
          1. 15.3.1. Accessing a shared memory segment
          2. 15.3.2. Attaching and detaching a shared memory segment
          3. 15.3.3. Controlling shared memory
          4. 15.3.4. Shared memory examples
        4. 15.4. POSIX:XSI Message Queues
          1. 15.4.1. Accessing a message queue
        5. 15.5. Exercise: POSIX Unnamed Semaphores
        6. 15.6. Exercise: POSIX Named Semaphores
        7. 15.7. Exercise: Implementing Pipes with Shared Memory
        8. 15.8. Exercise: Implementing Pipes with Message Queues
        9. 15.9. Additional Reading
      5. 16. Project: Producer Consumer Synchronization
        1. 16.1. The Producer-Consumer Problem
        2. 16.2. Bounded Buffer Protected by Mutex Locks
        3. 16.3. Buffer Implementation with Semaphores
        4. 16.4. Introduction to a Simple Producer-Consumer Problem
        5. 16.5. Bounded Buffer Implementation Using Condition Variables
        6. 16.6. Buffers with Done Conditions
        7. 16.7. Parallel File Copy
          1. 16.7.1. Parallel file copy producer
          2. 16.7.2. Parallel file copy consumer
          3. 16.7.3. Parallel file copy main program
          4. 16.7.4. Parallel file copy enhancements
        8. 16.8. Threaded Print Server
          1. 16.8.1. The request buffer
          2. 16.8.2. The producer thread
          3. 16.8.3. The consumer threads
          4. 16.8.4. The print server
          5. 16.8.5. Other enhancements
        9. 16.9. Additional Reading
      6. 17. Project: The Not Too Parallel Virtual Machine
        1. 17.1. PVM History, Terminology, and Architecture
        2. 17.2. The Not Too Parallel Virtual Machine
        3. 17.3. NTPVM Project Overview
          1. 17.3.1. NEWTASK packets
          2. 17.3.2. DATA packets
          3. 17.3.3. DONE packets
        4. 17.4. I/O and Testing of Dispatcher
          1. 17.4.1. Testing with multiple windows
          2. 17.4.2. Testing with remote logging
        5. 17.5. Single Task with No Input
        6. 17.6. Sequential Tasks
          1. 17.6.1. The input thread
          2. 17.6.2. The output thread
        7. 17.7. Concurrent Tasks
        8. 17.8. Packet Communication, Broadcast and Barriers
        9. 17.9. Termination and Signals
        10. 17.10. Ordered Message Delivery
        11. 17.11. Additional Reading
    7. IV. Communication
      1. 18. Connection-Oriented Communication
        1. 18.1. The Client-Server Model
        2. 18.2. Communication Channels
        3. 18.3. Connection-Oriented Server Strategies
        4. 18.4. Universal Internet Communication Interface (UICI)
          1. 18.4.1. Handling errors
          2. 18.4.2. Reading and writing
        5. 18.5. UICI Implementations of Different Server Strategies
        6. 18.6. UICI Clients
        7. 18.7. Socket Implementation of UICI
          1. 18.7.1. The socket function
          2. 18.7.2. The bind function
          3. 18.7.3. The listen function
          4. 18.7.4. Implementation of u_open
          5. 18.7.5. The accept function
          6. 18.7.6. Implementation of u_accept
          7. 18.7.7. The connect function
          8. 18.7.8. Implementation of u_connect
        8. 18.8. Host Names and IP Addresses
        9. 18.9. Thread-Safe UICI
        10. 18.10. Exercise: Ping Server
        11. 18.11. Exercise: Transmission of Audio
        12. 18.12. Additional Reading
      2. 19. Project: WWW Redirection
        1. 19.1. The World Wide Web
        2. 19.2. Uniform Resource Locators (URLs)
        3. 19.3. HTTP Primer
          1. 19.3.1. Client requests
          2. 19.3.2. Server response
          3. 19.3.3. HTTP message exchange
        4. 19.4. Web Communication Patterns
          1. 19.4.1. Tunnels
          2. 19.4.2. Proxies
          3. 19.4.3. Caching and Transparency
          4. 19.4.4. Gateways
        5. 19.5. Pass-through Monitoring of Single Connections
        6. 19.6. Tunnel Server Implementation
        7. 19.7. Server Driver for Testing
        8. 19.8. HTTP Header Parsing
        9. 19.9. Simple Proxy Server
        10. 19.10. Proxy Monitor
        11. 19.11. Proxy Cache
        12. 19.12. Gateways as Portals
        13. 19.13. Gateway for Load Balancing
        14. 19.14. Postmortem
          1. 19.14.1. Threading and timing errors
          2. 19.14.2. Uncaught errors and bad exits
          3. 19.14.3. Writing style and presentation
          4. 19.14.4. Poor testing and presentation of results
          5. 19.14.5. Programming errors and bad style
        15. 19.15. Additional Reading
      3. 20. Connectionless Communication and Multicast
        1. 20.1. Introduction to Connectionless Communication
        2. 20.2. Simplified Interface for Connectionless Communication
          1. 20.2.1. Host names and the u_buf_t structure
          2. 20.2.2. UICI UDP return errors
          3. 20.2.3. UDP buffer size and UICI UDP
        3. 20.3. Simple-Request Protocols
        4. 20.4. Request-Reply Protocols
        5. 20.5. Request-Reply with Timeouts and Retries
        6. 20.6. Request-Reply-Acknowledge Protocols
        7. 20.7. Implementation of UICI UDP
          1. 20.7.1. Implementation of u_openudp
          2. 20.7.2. The sendto function
          3. 20.7.3. Implementation of u_sendto and u_sendtohost
          4. 20.7.4. The recvfrom function
          5. 20.7.5. Implementation of u_recvfrom and u_recvfromtimed
          6. 20.7.6. Host names and u_buf_t
        8. 20.8. Comparison of UDP and TCP
        9. 20.9. Multicast
          1. 20.9.1. Multicast Addressing
          2. 20.9.2. Implementation of u_join
          3. 20.9.3. Implementation of u_leave
        10. 20.10. Exercise: UDP Port Server
        11. 20.11. Exercise: Stateless File Server
          1. 20.11.1. Remote File Services
        12. 20.12. Additional Reading
      4. 21. Project: Internet Radio
        1. 21.1. Project Overview
        2. 21.2. Audio Device Simulation
        3. 21.3. UDP Implementation with One Program and One Receiver
          1. 21.3.1. Simple implementation
          2. 21.3.2. Termination of the receiver
          3. 21.3.3. Buffering by the receiver to handle network latency
          4. 21.3.4. Buffering by the receiver to handle out-of-order delivery
        4. 21.4. UDP Implementation with Multiple Programs and Receivers
          1. 21.4.1. Multiple programs and one receiver
          2. 21.4.2. Multiple programs and multiple receivers
        5. 21.5. UDP Implementation of Radio Broadcasts
        6. 21.6. Multicast Implementation of Radio Broadcasts
        7. 21.7. TCP Implementation Differences
          1. 21.7.1. TCP implementation of one program and one receiver
          2. 21.7.2. TCP implementation of multiple programs with one receiver
          3. 21.7.3. TCP implementation of radio broadcasts
        8. 21.8. Receiving Streaming Audio Through a Browser
          1. 21.8.1. Using browser helper applications
          2. 21.8.2. Setting a new mime type in your web server
          3. 21.8.3. Setting your browser to handle a new mime type
          4. 21.8.4. Creating a web page
          5. 21.8.5. Using a predefined mime type
        9. 21.9. Additional Reading
      5. 22. Project: Server Performance
        1. 22.1. Server Performance Costs
        2. 22.2. Server Architectures
        3. 22.3. Project Overview
        4. 22.4. Single-Client Driver
          1. 22.4.1. Processing a connection
          2. 22.4.2. Programming the response
          3. 22.4.3. Gathering statistics
          4. 22.4.4. Testing the client
        5. 22.5. Multiple-Client Driver
          1. 22.5.1. Alternative multiple-client design
        6. 22.6. Thread-per-request and Process-per-request Implementations
        7. 22.7. Thread-worker-pool Strategy
        8. 22.8. Thread-worker Pool with Bounded Buffer
        9. 22.9. Process-worker Pool
        10. 22.10. Influence of Disk I/O
        11. 22.11. Performance Studies
          1. 22.11.1. Baseline measurements
          2. 22.11.2. Sources of variability
          3. 22.11.3. Measurement errors
          4. 22.11.4. Synchronization
          5. 22.11.5. Just plain errors
          6. 22.11.6. What to measure?
          7. 22.11.7. Data analysis and presentation
        12. 22.12. Report Writing
          1. 22.12.1. Introduction
          2. 22.12.2. Design, implementation and testing
          3. 22.12.3. Experiments
          4. 22.12.4. Results and analysis
          5. 22.12.5. Conclusion
          6. 22.12.6. Bibliography
        13. 22.13. Additional Reading
    8. Appendices
      1. A. UNIX Fundamentals
        1. A.1. Manual Pages
        2. A.2. Compilation
          1. A.2.1. Header files
          2. A.2.2. Linking and libraries
          3. A.2.3. Macros and conditional compilation
        3. A.3. Makefiles
        4. A.4. Debugging Aids
          1. A.4.1. The lint utility
          2. A.4.2. Debuggers
          3. A.4.3. The truss utility
          4. A.4.4. Profilers
        5. A.5. Identifiers, Storage Classes and Linkage Classes
        6. A.6. Additional Reading
      2. B. Restart Library
      3. C. UICI Implementation
        1. C.1. Connection-Oriented UICI TCP Implementation
        2. C.2. Name Resolution Implementations
          1. C.2.1. Implementation with gethostbyaddr and gethostbyname
          2. C.2.2. Reentrant versions of name resolution functions
          3. C.2.3. Reentrant name resolution with mutex locks
        3. C.3. Connectionless UICI UDP Implementation
      4. D. Logging Functions
        1. D.1. Local Atomic Logging
        2. D.2. Remote Logging
          1. D.2.1. Use of the remote logging facility
          2. D.2.2. Implementation details
      5. E. POSIX Extensions
    9. Bibliography

    Product information

    • Title: Unix™ Systems Programming: Communication, Concurrency, and Threads
    • Author(s): Kay A. Robbins, Steven Robbins
    • Release date: June 2003
    • Publisher(s): Pearson
    • ISBN: 0130424110