Computer Architecture, 5th Edition

4 Data-Level Parallelism in Vector, SIMD, and GPU Architectures

We call these algorithms data parallel algorithms because their parallelism comes from simultaneous operations across large sets of data, rather than from multiple threads of control.

W. Daniel Hillis and Guy L. Steele

“Data Parallel Algorithms,” Comm. ACM (1986)

If you were plowing a field, which would you rather use: two strong oxen or 1024 chickens?

Seymour Cray, Father of the Supercomputer

(arguing for two powerful vector processors versus many simple processors)

4.1 Introduction 262

4.2 Vector Architecture 264

4.3 SIMD Instruction Set Extensions for Multimedia 282

4.4 Graphics Processing Units 288

4.5 Detecting and Enhancing Loop-Level Parallelism 315

4.6 Crosscutting Issues ...