Parallel Computer Architecture

11.1 Overview of Latency Tolerance 839

Time in processor's critical path

Iteration 0

o o

CA: m Ð3

l o o I

N:

I II I I

Iteration 1

i 'i i—i

Iteration 2



CD

III i

3*c

a

Data Acknowledgment

3

*

c

c

i i communication

P:  '~N •'%

CA: CO * C

N: I B I I I

 Local computation (c

a

, q,, c

c

)

EZ] Endpoint overhead

Ü3 Network delay

E3 Bandwidth-related time

FIGURE 11.5 Effect of making messages larger on the timeline for the example message-

passing program. The figure assumes that three iterations of the loop are executed. The sender pro-

cess

(P

A

)

first computes the three values A [ 0.

. 2

] to be sent (the computation c

a

), then sends them in

one message ...