O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Shared Memory Application Programming

Book Description

Shared Memory Application Programming presents the key concepts and applications of parallel programming, in an accessible and engaging style applicable to developers across many domains. Multithreaded programming is today a core technology, at the basis of all software development projects in any branch of applied computer science. This book guides readers to develop insights about threaded programming and introduces two popular platforms for multicore development: OpenMP and Intel Threading Building Blocks (TBB). Author Victor Alessandrini leverages his rich experience to explain each platform’s design strategies, analyzing the focus and strengths underlying their often complementary capabilities, as well as their interoperability.

The book is divided into two parts: the first develops the essential concepts of thread management and synchronization, discussing the way they are implemented in native multithreading libraries (Windows threads, Pthreads) as well as in the modern C++11 threads standard. The second provides an in-depth discussion of TBB and OpenMP including the latest features in OpenMP 4.0 extensions to ensure readers’ skills are fully up to date. Focus progressively shifts from traditional thread parallelism to modern task parallelism deployed by modern programming environments. Several chapter include examples drawn from a variety of disciplines, including molecular dynamics and image processing, with full source code and a software library incorporating a number of utilities that readers can adapt into their own projects.

  • Designed to introduce threading and multicore programming to teach modern coding strategies for developers in applied computing
  • Leverages author Victor Alessandrini's rich experience to explain each platform’s design strategies, analyzing the focus and strengths underlying their often complementary capabilities, as well as their interoperability
  • Includes complete, up-to-date discussions of OpenMP 4.0 and TBB
  • Based on the author’s training sessions, including information on source code and software libraries which can be repurposed

Table of Contents

  1. Cover image
  2. Title page
  3. Table of Contents
  4. Copyright
  5. Preface
    1. Pedagogical Objectives
    2. Programming Environments
    3. Book Organization
  6. Biography
  7. Acknowledgments
  8. Chapter 1: Introduction and Overview
    1. Abstract
    2. 1.1 Processes and Threads
    3. 1.2 Overview of Computing Platforms
    4. 1.3 Memory System of Computing Platforms
    5. 1.4 Parallel Processing Inside Cores
    6. 1.5 External Computational Devices
    7. 1.6 Final Comments
  9. Chapter 2: Introducing Threads
    1. Abstract
    2. 2.1 Applications and Processes
    3. 2.2 Multithreaded Processes
    4. 2.3 Programming and Execution Models
    5. 2.4 Benefits of Concurrent Programming
  10. Chapter 3: Creating and Running Threads
    1. Abstract
    2. 3.1 Introduction
    3. 3.2 Overview of Basic Libraries
    4. 3.3 Overview of Basic Thread Management
    5. 3.4 Using Posix Threads
    6. 3.5 Using Windows Threads
    7. 3.6 C++11 Thread Library
    8. 3.7 SPool Utility
    9. 3.8 SPool Examples
    10. 3.9 First Look at OpenMP
    11. 3.10 Database Search Example
    12. 3.11 Conclusions
    13. 3.12 Annex: Coding C++11 Time Durations
  11. Chapter 4: Thread-Safe Programming
    1. Abstract
    2. 4.1 Introduction
    3. 4.2 Some Library Functions are not Thread Safe
    4. 4.3 Dealing with Random Number Generators
    5. 4.4 Thread Local Storage Services
    6. 4.5 Second Example: A Gaussian Random Generator
    7. 4.6 Comments on Thread Local Storage
    8. 4.7 Conclusion
  12. Chapter 5: Concurrent Access to Shared Data
    1. Abstract
    2. 5.1 First Comments on Thread Synchronization
    3. 5.2 Need for Mutual Exclusion Among Threads
    4. 5.3 Different Kinds of Mutex Flavors
    5. 5.4 Pthreads Mutual Exclusion
    6. 5.5 Other Simple Examples
    7. 5.6 Windows Mutual Exclusion
    8. 5.7 OpenMP Mutual Exclusion
    9. 5.8 C++11 Mutual Exclusion
    10. 5.9 TBB Mutual Exclusion
    11. 5.10 First Look at Atomic Operations
    12. 5.11 Container Thread Safety
    13. 5.12 Comments on Mutual Exclusion Best Practices
  13. Chapter 6: Event Synchronization
    1. Abstract
    2. 6.1 Idle Waits Versus Spin Waits
    3. 6.2 Condition Variables in Idle Waits
    4. 6.3 Idle Waits in Pthreads
    5. 6.4 Windows Condition Variables
    6. 6.5 C++11 condition_variable Class
    7. 6.6 Examples of Idle Wait
    8. 6.7 C++11 Futures and Promises
  14. Chapter 7: Cache Coherency and Memory Consistency
    1. Abstract
    2. 7.1 Introduction
    3. 7.2 Cache Coherency Issue
    4. 7.3 What is a Memory Consistency Model?
    5. 7.4 Weak-Ordering Memory Models
    6. 7.5 Pthreads Memory Consistency
    7. 7.6 OpenMP Memory Consistency
  15. Chapter 8: Atomic Types and Operations
    1. Abstract
    2. 8.1 Introduction
    3. 8.2 C++11 std::atomic<T> Class
    4. 8.3 Lock-Free Algorithms
    5. 8.4 Synchronizing Thread Operations
    6. 8.5 Examples of Atomic Synchronizations
    7. 8.6 TBB atomic<T> Class
    8. 8.7 Windows Atomic Services
    9. 8.8 Summary
  16. Chapter 9: High-Level Synchronization Tools
    1. Abstract
    2. 9.1 Introduction and Overview
    3. 9.2 General Comments on High-Level Synchronization Tools
    4. 9.3 Overview of the vath Synchronization Utilities
    5. 9.4 Timers
    6. 9.5 Boolean Locks
    7. 9.6 SynchP< T &gt; Template Class > Template Class
    8. 9.7 Idle and Spin Barriers
    9. 9.8 Blocking Barriers
    10. 9.9 ThQueue<T> Class
    11. 9.10 Reader-Writer Locks
    12. 9.11 RWlock Class
    13. 9.12 General Overview of Thread Pools
  17. Chapter 10: OpenMP
    1. Abstract
    2. Overview
    3. 10.1 Basic Execution Model
    4. 10.2 Configuring OpenMP
    5. 10.3 Thread Management and Work-Sharing Directives
    6. 10.4 Synchronization Directives
    7. 10.5 Examples of Parallel and Work-Sharing Constructs
    8. 10.6 Task API
    9. 10.7 Task Examples
    10. 10.8 Task Best Practices
    11. 10.9 Cancellation of Parallel Constructs
    12. 10.10 Offloading Code Blocks to Accelerators
    13. 10.11 Thread Affinity
    14. 10.12 Vectorization
    15. 10.13 Annex: SafeCounter Utility Class
  18. Chapter 11: Intel Threading Building Blocks
    1. Abstract
    2. 11.1 Overview
    3. 11.2 TBB Content
    4. 11.3 TBB Initialization
    5. 11.4 Operation of the High-Level Algorithms
    6. 11.5 Simple parallel_for Example
  19. Chapter 12: Further Thread Pools
    1. Abstract
    2. 12.1 Introduction
    3. 12.2 SPool Reviewed
    4. 12.3 NPool Features
    5. 12.4 NPool API
    6. 12.5 Building Parallel Jobs
    7. 12.6 Operation of the NPool
    8. 12.7 Examples
    9. 12.8 Running Parallel Routines in Parallel
    10. 12.9 Hybrid MPI-Threads Example
  20. Chapter 13: Molecular Dynamics Example
    1. Abstract
    2. 13.1 Introduction
    3. 13.2 Molecular Dynamics Problem
    4. 13.3 Integration of the Equations of Motion
    5. 13.4 Md.C Sequential Code
    6. 13.5 Parallel Implementations of Md.C
  21. Chapter 14: Further Data Parallel Examples
    1. Abstract
    2. 14.1 Introduction
    3. 14.2 Relaxation Methods
    4. 14.3 First Example: Heat.C
    5. 14.4 Parallel Implementation of Heat.C
    6. 14.5 Heat Performance Issues
    7. 14.6 Second Example: Sor.C
    8. 14.7 Parallel Implementations of Sor.C
    9. 14.8 Sor Performance Issues
    10. 14.9 Alternative Approach to Data Dependencies
  22. Chapter 15: Pipelining Threads
    1. Abstract
    2. 15.1 Pipeline Concurrency Pattern
    3. 15.2 Example 1: Two-Dimensional Heat Propagation
    4. 15.3 Sequential Code Heat.C
    5. 15.4 Pipelined Versions
    6. 15.5 Pipeline Classes
    7. 15.6 Example: Pipelined Sor
    8. 15.7 Pipelining Threads in TBB
    9. 15.8 Some Performance Considerations
    10. 15.9 Annex: Solution to the Heat Diffusion Equation
    11. 15.10 Annex: FFT Routines
  23. Chapter 16: Using the TBB Task Scheduler
    1. Abstract
    2. 16.1 Introduction
    3. 16.2 Structure of the TBB Pool
    4. 16.3 TBB Task Management Classes
    5. 16.4 Complete Scheduler API: The Task Class
    6. 16.5 Miscellaneous Task Features
    7. 16.6 Using the TBB Scheduler
    8. 16.7 Job Submission by Client Threads
    9. 16.8 Example: Molecular Dynamics Code
    10. 16.9 Recycling Parallel Region Tasks
    11. 16.10 Annex: Task Class Member Functions
  24. Annex A: Using the Software
    1. A.1 Libraries Required
    2. A.2 Software Organization
    3. A.3 vath Classes
  25. Annex B: C++ Function Objects and Lambda Expressions
    1. B.1 Function Objects
    2. B.2 Function Object Syntax
    3. B.3 Lambda Expressions
  26. Bibliography
  27. Index