book

Unix™ Systems Programming: Communication, Concurrency, and Threads

Name: Unix™ Systems Programming: Communication, Concurrency, and Threads
ISBN: 0130424110

by Kay A. Robbins, Steven Robbins

June 2003

Intermediate to advanced

912 pages

21h 37m

English

Pearson

Read now

Unlock full access

Copyright
Dedication
About the Web Site
Preface
Acknowledgments
I. Fundamentals
1. Technology’s Impact on Programs
1.1. Terminology of Change1.2. Time and Speed1.3. Multiprogramming and Time Sharing1.4. Concurrency at the Applications Level1.4.1. Interrupts1.4.2. Signals1.4.3. Input and output1.4.4. Processes, threads and the sharing of resources1.4.5. Multiple processors with shared memory1.4.6. The network as the computer1.5. Security and Fault Tolerance1.6. Buffer Overflows for Breaking and Entering1.6.1. Consequences of buffer overflows1.6.2. Buffer overflows and security1.7. UNIX Standards1.8. Additional Reading
2. Programs, Processes and Threads
2.1. How a Program Becomes a Process2.2. Threads and Thread of Execution2.3. Layout of a Program Image2.4. Library Function Calls2.5. Function Return Values and Errors2.6. Argument Arrays2.6.1. Creating an argument array with makeargv2.6.2. Implementation of makeargv2.7. Thread-Safe Functions2.8. Use of Static Variables2.9. Structure of Static Objects2.10. Process Environment2.11. Process Termination2.12. Exercise: An env Utility2.13. Exercise: Message Logging2.14. Additional Reading
3. Processes in UNIX
3.1. Process Identification3.2. Process State3.3. UNIX Process Creation and fork3.4. The wait Function3.4.1. Status values3.5. The exec Function3.6. Background Processes and Daemons3.7. Critical Sections3.8. Exercise: Process Chains3.9. Exercise: Process Fans3.10. Additional Reading
4. UNIX I/O
4.1. Device Terminology4.2. Reading and Writing4.3. Opening and Closing Files4.4. The select Function4.5. The poll Function4.6. File Representation4.6.1. File descriptors4.6.2. File pointers and buffering4.6.3. Inheritance of file descriptors4.7. Filters and Redirection4.8. File Control4.9. Exercise: Atomic Logging4.9.1. An atomic logging library4.10. Exercise: A cat Utility4.11. Additional Reading
5. Files and Directories
5.1. UNIX File System Navigation5.1.1. The current working directory5.1.2. Search paths5.2. Directory Access5.2.1. Accessing file status information5.2.2. Determining the type of a file5.3. UNIX File System Implementation5.3.1. UNIX file implementation5.3.2. Directory implementation5.4. Hard Links and Symbolic Links5.4.1. Creating or removing a link5.4.2. Creating and removing symbolic links5.5. Exercise: The which Command5.6. Exercise: Biffing5.7. Exercise: News biff5.8. Exercise: Traversing Directories5.9. Additional Reading
6. UNIX Special Files
6.1. Pipes6.2. Pipelines6.3. FIFOs6.4. Pipes and the Client-Server Model6.5. Terminal Control6.5.1. Canonical and noncanonical input processing6.6. Audio Device6.7. Exercise: Audio6.8. Exercise: Barriers6.9. Exercise: The stty Command6.10. Exercise: Client-Server Revisited6.11. Additional Reading

7. Project: The Token Ring
7.1. Ring Topology7.2. Ring Formation7.3. Ring Exploration7.4. Simple Communication7.5. Mutual Exclusion with Tokens7.6. Mutual Exclusion by Voting7.7. Leader Election on an Anonymous Ring7.8. Token Ring for Communication7.9. Pipelined Preprocessor7.10. Parallel Ring Algorithms7.10.1. Image filteringFiltering algorithms on the ringA bidirectional ringBlock computation7.10.2. Matrix multiplication7.11. Flexible Ring7.12. Additional Reading
II. Asynchronous Events
8. Signals
8.1. Basic Signal Concepts8.2. Generating Signals8.3. Manipulating Signal Masks and Signal Sets8.4. Catching and Ignoring Signals—sigaction8.5. Waiting for Signals—pause, sigsuspend and sigwait8.5.1. The pause function8.5.2. The sigsuspend function8.5.3. The sigwait function8.6. Handling Signals: Errors and Async-signal Safety8.7. Program Control with siglongjmp and sigsetjmp8.8. Programming with Asynchronous I/O8.9. Exercise: Dumping Statistics8.10. Exercise: Spooling a Slow Device8.11. Additional Reading
9. Times and Timers
9.1. POSIX Times9.1.1. Expressing time in seconds since the Epoch9.1.2. Displaying date and time9.1.3. Using struct timeval to express time9.1.4. Using realtime clocks9.1.5. Contrasting elapsed time to processor time9.2. Sleep Functions9.3. POSIX:XSI Interval Timers9.4. Realtime Signals9.5. POSIX:TMR Interval Timers9.6. Timer Drift, Overruns and Absolute Time9.7. Additional Reading
10. Project: Virtual Timers
10.1. Project Overview10.2. Simple Timers10.3. Setting One of Five Single Timers10.3.1. The virtualtimers object10.3.2. The hardwaretimer object10.3.3. Main program implementation10.3.4. Instrumentation of the timer code with show10.4. Using Multiple Timers10.4.1. Setting multiple timers10.4.2. Testing with multiple timers10.5. A Robust Implementation of Multiple Timers10.6. POSIX:TMR Timer Implementation10.7. mycron, a Small Cron Facility10.8. Additional Reading
11. Project: Cracking Shells
11.1. Building a Simple Shell11.2. Redirection11.3. Pipelines11.4. Signal Handling in the Foreground11.5. Process Groups, Sessions and Controlling Terminals11.5.1. Process Groups11.5.2. Sessions11.6. Background Processes in ush11.7. Job Control11.8. Job Control for ush11.8.1. A job list object11.8.2. The job list in ush11.8.3. Job control in ush11.8.4. Process behavior in waiting for a pipeline11.9. Additional Reading
III. Concurrency
12. POSIX Threads
12.1. A Motivating Problem: Monitoring File Descriptors12.2. Use of Threads to Monitor Multiple File Descriptors12.3. Thread Management12.3.1. Referencing threads by ID12.3.2. Creating a thread12.3.3. Detaching and joining12.3.4. Exiting and cancellation12.3.5. Passing parameters to threads and returning values12.4. Thread Safety12.5. User Threads versus Kernel Threads12.6. Thread Attributes12.6.1. The thread state12.6.2. The thread stack12.6.3. Thread scheduling12.7. Exercise: Parallel File Copy12.8. Additional Reading
13. Thread Synchronization
13.1. POSIX Synchronization Functions13.2. Mutex Locks13.2.1. Creating and initializing a mutex13.2.2. Destroying a mutex13.2.3. Locking and unlocking a mutex13.2.4. Protecting unsafe library functions13.2.5. Synchronizing flags and global values13.2.6. Making data structures thread-safe13.3. At-Most-Once and At-Least-Once-Execution13.4. Condition Variables13.4.1. Creating and destroying condition variables13.4.2. Waiting and signaling on condition variables13.5. Signal Handling and Threads13.5.1. Directing a signal to a particular thread13.5.2. Masking signals for threads13.5.3. Dedicating threads for signal handling13.6. Readers and Writers13.7. A strerror_r Implementation13.8. Deadlocks and Other Pesky Problems13.9. Exercise: Multiple Barriers13.10. Additional Reading
14. Critical Sections and Semaphores
14.1. Dealing with Critical Sections14.2. Semaphores14.3. POSIX:SEM Unnamed Semaphores14.4. POSIX:SEM Semaphore Operations14.5. POSIX:SEM Named Semaphores14.5.1. Creating and opening named semaphores14.5.2. Closing and unlinking named semaphores14.6. Exercise: License Manager14.6.1. License object14.6.2. The runsim main program14.6.3. Extensions to the license manager14.7. Additional Reading
15. POSIX IPC
15.1. POSIX:XSI Interprocess Communication15.1.1. Identifying and accessing IPC objects15.1.2. Accessing POSIX:XSI IPC resources from the shell15.2. POSIX:XSI Semaphore Sets15.2.1. Semaphore creation15.2.2. Semaphore control15.2.3. POSIX semaphore set operations15.3. POSIX:XSI Shared Memory15.3.1. Accessing a shared memory segment15.3.2. Attaching and detaching a shared memory segment15.3.3. Controlling shared memory15.3.4. Shared memory examples15.4. POSIX:XSI Message Queues15.4.1. Accessing a message queue15.5. Exercise: POSIX Unnamed Semaphores15.6. Exercise: POSIX Named Semaphores15.7. Exercise: Implementing Pipes with Shared Memory15.8. Exercise: Implementing Pipes with Message Queues15.9. Additional Reading
16. Project: Producer Consumer Synchronization
16.1. The Producer-Consumer Problem16.2. Bounded Buffer Protected by Mutex Locks16.3. Buffer Implementation with Semaphores16.4. Introduction to a Simple Producer-Consumer Problem16.5. Bounded Buffer Implementation Using Condition Variables16.6. Buffers with Done Conditions16.7. Parallel File Copy16.7.1. Parallel file copy producer16.7.2. Parallel file copy consumer16.7.3. Parallel file copy main program16.7.4. Parallel file copy enhancements16.8. Threaded Print Server16.8.1. The request buffer16.8.2. The producer thread16.8.3. The consumer threads16.8.4. The print server16.8.5. Other enhancements16.9. Additional Reading
17. Project: The Not Too Parallel Virtual Machine
17.1. PVM History, Terminology, and Architecture17.2. The Not Too Parallel Virtual Machine17.3. NTPVM Project Overview17.3.1. NEWTASK packets17.3.2. DATA packets17.3.3. DONE packets17.4. I/O and Testing of Dispatcher17.4.1. Testing with multiple windows17.4.2. Testing with remote logging17.5. Single Task with No Input17.6. Sequential Tasks17.6.1. The input thread17.6.2. The output thread17.7. Concurrent Tasks17.8. Packet Communication, Broadcast and Barriers17.9. Termination and Signals17.10. Ordered Message Delivery17.11. Additional Reading
IV. Communication
18. Connection-Oriented Communication
18.1. The Client-Server Model18.2. Communication Channels18.3. Connection-Oriented Server Strategies18.4. Universal Internet Communication Interface (UICI)18.4.1. Handling errors18.4.2. Reading and writing18.5. UICI Implementations of Different Server Strategies18.6. UICI Clients18.7. Socket Implementation of UICI18.7.1. The socket function18.7.2. The bind function18.7.3. The listen function18.7.4. Implementation of u_open18.7.5. The accept function18.7.6. Implementation of u_accept18.7.7. The connect function18.7.8. Implementation of u_connect18.8. Host Names and IP Addresses18.9. Thread-Safe UICI18.10. Exercise: Ping Server18.11. Exercise: Transmission of Audio18.12. Additional Reading
19. Project: WWW Redirection
19.1. The World Wide Web19.2. Uniform Resource Locators (URLs)19.3. HTTP Primer19.3.1. Client requests19.3.2. Server response19.3.3. HTTP message exchange19.4. Web Communication Patterns19.4.1. Tunnels19.4.2. Proxies19.4.3. Caching and Transparency19.4.4. Gateways19.5. Pass-through Monitoring of Single Connections19.6. Tunnel Server Implementation19.7. Server Driver for Testing19.8. HTTP Header Parsing19.9. Simple Proxy Server19.10. Proxy Monitor19.11. Proxy Cache19.12. Gateways as Portals19.13. Gateway for Load Balancing19.14. Postmortem19.14.1. Threading and timing errors19.14.2. Uncaught errors and bad exits19.14.3. Writing style and presentation19.14.4. Poor testing and presentation of results19.14.5. Programming errors and bad style19.15. Additional Reading
20. Connectionless Communication and Multicast
20.1. Introduction to Connectionless Communication20.2. Simplified Interface for Connectionless Communication20.2.1. Host names and the u_buf_t structure20.2.2. UICI UDP return errors20.2.3. UDP buffer size and UICI UDP20.3. Simple-Request Protocols20.4. Request-Reply Protocols20.5. Request-Reply with Timeouts and Retries20.6. Request-Reply-Acknowledge Protocols20.7. Implementation of UICI UDP20.7.1. Implementation of u_openudp20.7.2. The sendto function20.7.3. Implementation of u_sendto and u_sendtohost20.7.4. The recvfrom function20.7.5. Implementation of u_recvfrom and u_recvfromtimed20.7.6. Host names and u_buf_t20.8. Comparison of UDP and TCP20.9. Multicast20.9.1. Multicast Addressing20.9.2. Implementation of u_join20.9.3. Implementation of u_leave20.10. Exercise: UDP Port Server20.11. Exercise: Stateless File Server20.11.1. Remote File Services20.12. Additional Reading
21. Project: Internet Radio
21.1. Project Overview21.2. Audio Device Simulation21.3. UDP Implementation with One Program and One Receiver21.3.1. Simple implementation21.3.2. Termination of the receiver21.3.3. Buffering by the receiver to handle network latency21.3.4. Buffering by the receiver to handle out-of-order delivery21.4. UDP Implementation with Multiple Programs and Receivers21.4.1. Multiple programs and one receiver21.4.2. Multiple programs and multiple receivers21.5. UDP Implementation of Radio Broadcasts21.6. Multicast Implementation of Radio Broadcasts21.7. TCP Implementation Differences21.7.1. TCP implementation of one program and one receiver21.7.2. TCP implementation of multiple programs with one receiver21.7.3. TCP implementation of radio broadcasts21.8. Receiving Streaming Audio Through a Browser21.8.1. Using browser helper applications21.8.2. Setting a new mime type in your web server21.8.3. Setting your browser to handle a new mime type21.8.4. Creating a web page21.8.5. Using a predefined mime type21.9. Additional Reading
22. Project: Server Performance
22.1. Server Performance Costs22.2. Server Architectures22.3. Project Overview22.4. Single-Client Driver22.4.1. Processing a connection22.4.2. Programming the response22.4.3. Gathering statistics22.4.4. Testing the client22.5. Multiple-Client Driver22.5.1. Alternative multiple-client design22.6. Thread-per-request and Process-per-request Implementations22.7. Thread-worker-pool Strategy22.8. Thread-worker Pool with Bounded Buffer22.9. Process-worker Pool22.10. Influence of Disk I/O22.11. Performance Studies22.11.1. Baseline measurements22.11.2. Sources of variability22.11.3. Measurement errors22.11.4. Synchronization22.11.5. Just plain errors22.11.6. What to measure?22.11.7. Data analysis and presentation22.12. Report Writing22.12.1. Introduction22.12.2. Design, implementation and testing22.12.3. Experiments22.12.4. Results and analysis22.12.5. Conclusion22.12.6. Bibliography22.13. Additional Reading
Appendices
A. UNIX Fundamentals
A.1. Manual PagesA.2. CompilationA.2.1. Header filesA.2.2. Linking and librariesA.2.3. Macros and conditional compilationA.3. MakefilesA.4. Debugging AidsA.4.1. The lint utilityA.4.2. DebuggersA.4.3. The truss utilityA.4.4. ProfilersA.5. Identifiers, Storage Classes and Linkage ClassesA.6. Additional Reading
B. Restart Library
C. UICI Implementation
C.1. Connection-Oriented UICI TCP ImplementationC.2. Name Resolution ImplementationsC.2.1. Implementation with gethostbyaddr and gethostbynameC.2.2. Reentrant versions of name resolution functionsC.2.3. Reentrant name resolution with mutex locksC.3. Connectionless UICI UDP Implementation
D. Logging Functions
D.1. Local Atomic LoggingD.2. Remote LoggingD.2.1. Use of the remote logging facilityD.2.2. Implementation details
E. POSIX Extensions
Bibliography

Content preview from Unix™ Systems Programming: Communication, Concurrency, and Threads

Chapter 11. Project: Cracking Shells

By developing a shell from the bottom up, this chapter explores the intricacies of process creation, termination, identification and the correct handling of signals. Example programs handle foreground and background processes, pipelines, process groups, sessions and controlling terminals. The chapter also looks at job control and terminal I/O. The closing project integrates these concepts by incorporating job control into a shell.

Building a Simple Shell

A shell is a process that does command-line interpretation. In other words, ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

C++ Common Knowledge: Essential Intermediate Programming

Publisher Resources

ISBN: 0130424110Purchase book

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Unix™ Systems Programming: Communication, Concurrency, and Threads

by Kay A. Robbins, Steven Robbins

Chapter 11. Project: Cracking Shells

Building a Simple Shell

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.