book

Inside the Message Passing Interface

Name: Inside the Message Passing Interface
Author: Alexander Supalov
ISBN: 9781501506789

by Alexander Supalov

September 2018

Intermediate to advanced

384 pages

10h 47m

English

De Gruyter

Read now

Unlock full access

Title Page
Copyright
Dedication
Acknowledgments
Contents
Preface
Introduction
Chapter 1: Overview
1.1 Parallel Computer1.1.1 Intraprocessor Parallelism1.1.2 Interprocessor Parallelism1.1.3 Exercises1.2 MPI Standard1.2.1 MPI History1.2.2 Related Standards1.2.3 Exercises1.3 MPI Subsetting1.3.1 Motivation1.3.2 Sensible MPI Subsets1.3.3 Exercises1.4 Fine Points1.4.1 Programming1.4.2 Code Management1.4.3 Building1.4.4 Testing and Debugging1.4.5 Benchmarking1.4.6 Presentation1.4.7 Optimization1.4.8 Issue Tracking1.4.9 Exercises
Chapter 2: Shared Memory
2.1 Subset Definition2.1.1 Exercises2.2 Communication Basics2.2.1 Intraprocess Communication2.2.2 Interprocess Communication2.2.3 Ping-Pong Pattern Revisited2.2.4 Exercises2.3 Subset Implementation: Blocking Communication2.3.1 Design Decisions2.3.2 Start-up and Termination2.3.3 Blocking Point-to-Point Communication2.3.4 Testing2.3.5 Benchmarking2.3.6 Optimization2.3.7 Exercises2.4 Conclusions
Chapter 3: Sockets
3.1 Subset Definition3.1.1 Exercises3.2 Communication Basics3.2.1 Intranode Communication3.2.2 Internode Communication3.2.3 Exercises3.3 Initial Subset Implementation: Start-up and Blocking Communication3.3.1 Design Decisions3.3.2 Start-up and Termination3.3.3 Point-to-Point Communication3.3.4 Testing3.3.5 Benchmarking3.3.6 Exercises3.4 Subset Reimplementation: Nonblocking Communication3.4.1 Redesign Directions3.4.2 Start-up and Termination3.4.3 Point-to-Point Communication3.4.4 Testing3.4.5 Benchmarking3.4.6 Optimization3.4.7 Exercises3.5 Conclusions

Chapter 4: Extensions
4.1 Subset Definition4.1.1 Exercises4.2 Subset Basics4.2.1 Communicator Management4.2.2 Datatypes4.2.3 Collective Operations4.2.4 Error Handling4.2.5 Language Bindings4.2.6 Interdependencies and Steps4.2.7 Exercise4.3 Initial Subset Implementation: Start-up, Data Transfer, and Error Handling4.3.1 Design Decisions4.3.2 Code Refactoring4.3.3 Initialization and Termination4.3.4 Error Handling4.3.5 Process Start-up and Data Transfer4.3.6 Testing4.3.7 Benchmarking4.3.8 Exercises4.4 Subset Extension 1: Communicators, Datatypes, and Collectives4.4.1 Extension Directions4.4.2 Communicator Management4.4.3 Datatypes4.4.4 Point-to-Point Communication4.4.5 Collective Operations4.4.6 Testing4.4.7 Benchmarking4.5 Subset Extension 2: Communicators and Collectives Revisited4.5.1 Extension Directions4.5.2 Communicator Management Revisited4.5.3 Collective Operations Revisited4.5.4 Testing4.5.5 Benchmarking4.5.6 Exercises4.6 Subset Completion: Language Bindings4.6.1 Language Bindings4.6.2 Testing4.6.3 Benchmarking4.6.4 Exercises4.7 Conclusions
Chapter 5: Optimization
5.1 Unifabric Optimization5.1.1 Design Decisions5.1.2 Code Refactoring5.1.3 Start-up and Termination5.1.4 Connection Establishment5.1.5 Point-to-Point Communication5.1.6 Collective Operations5.1.7 Exercises5.2 Multifabric Optimization5.2.1 Design Decisions5.2.2 Code Refactoring5.2.3 Multifabric Device5.2.4 Progress Engine Revisited5.2.5 Start-up and Termination5.2.6 Connection Establishment5.2.7 Point-to-Point Communication5.2.8 Collective Operations5.2.9 Exercises5.3 Conclusions
Chapter 6: Tuning
6.1 Overview6.1.1 Default Settings6.1.2 Static Settings6.1.3 Dynamic Settings6.1.4 Exercises6.2 Static Tuning6.2.1 Design Decisions6.2.2 Parameter Control6.2.3 Process Remapping6.2.4 Exercises6.3 Dynamic Tuning6.3.1 Design Decisions6.3.2 Parameter Control6.3.3 Process Migration6.3.4 Closed Loop Tuning6.3.5 Exercises6.4 Conclusions
Chapter 7: And the Rest
7.1 MPI-1 Features7.1.1 Communication Modes7.1.2 Wildcards7.1.3 Derived Datatypes and Heterogeneity7.1.4 Profiling Interface7.1.5 Other Features7.1.6 Exercises7.2 MPI-2 Features7.2.1 Dynamic Process Management7.2.2 One-Sided Communication7.2.3 Thread Support7.2.4 File I/O7.2.5 Other Features7.2.6 Exercises7.3 MPI-3 Features7.3.1 Fast One-Sided Communication7.3.2 Long Counts7.3.3 Nonblocking Collectives7.3.4 Other Features7.3.5 Exercises7.4 Conclusions
Chapter 8: A Look Ahead
8.1 Selected MPI-4 Features (En Route)8.1.1 Fault Tolerance8.1.2 Threads as MPI Processes8.1.3 Exercises8.2 MPI Limitations8.2.1 Dense Process Rank Sequence8.2.2 Reliable and Pairwise Ordered Data Transfer8.2.3 Message Progress Rules8.2.4 Exercises8.3 Beyond MPI8.3.1 Scalability, Mobility, and IoT Challenges8.3.2 Frequency Wall and Memory Lag8.3.3 Programmability Barrier and Stack Crawling8.3.4 Exercises8.4 Conclusions
Bibliography
Appendix A: Solutions
A.1 OverviewA.2 Shared MemoryA.3 SocketsA.4 ExtensionsA.5 OptimizationA.6 TuningA.7 And the RestA.8 A Look Ahead
Index

Content preview from Inside the Message Passing Interface

Acknowledgments

I would like to thank Stuart Douglas and Jeffrey Pepper of De Gruyter for approaching me with an offer that prompted the writing of this book. My thanks also go to Jeffrey Pepper and Jaya Dalal for their efforts on the manuscript, to Mary Sudul for diligent copyediting, and to MacPS for their production work.

Jeff Squyres and Kenneth Raffenetti helped me with certain aspects of Open MPI and MPICH, respectively, saving quite a bit of time that would have otherwise gone into issue investigation.

Many anonymous reviewers provided valuable feedback on my conference submissions, helping me fine tune the text of the book. I extend my gratitude to them, too.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Real-Time Agility: The Harmony/ESW Method for Real-Time and Embedded Systems Development

Publisher Resources

ISBN: 9781501506789

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Inside the Message Passing Interface

by Alexander Supalov

Acknowledgments

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.