book

CUDA Programming

by Shane Cook

December 2012

Intermediate to advanced

600 pages

18h 19m

English

Morgan Kaufmann

Read now

Unlock full access

IntroductionVon Neumann ArchitectureCrayConnection MachineCell ProcessorMultinode ComputingThe Early Days of Gpgpu CodingThe Death of the Single-Core SolutionNvidia and CudaGpu HardwareAlternatives to CudaConclusion
IntroductionTraditional Serial CodeSerial/Parallel ProblemsConcurrencyTypes of ParallelismFlynn’s TaxonomySome Common Parallel PatternsConclusion
PC ArchitectureGPU HardwareCPUs and GPUsCompute Levels
IntroductionInstalling the Sdk Under WindowsVisual StudioLinuxMacInstalling a DebuggerCompilation ModelError HandlingConclusion
What it all MeansThreadsBlocksGridsWarpsBlock SchedulingA Practical Example—HistogramsConclusion

IntroductionCachesRegister UsageShared MemoryConstant Memory
Texture MemoryConclusion
IntroductionSerial and Parallel CodeProcessing DatasetsProfilingAn Example Using AESConclusionReferences
IntroductionLocalityMulti-CPU SystemsMulti-GPU SystemsAlgorithms on Multiple GPUSWhich GPU?Single-Node SystemsStreamsMultiple-Node SystemsConclusion
Strategy 1: Parallel/Serial GPU/CPU Problem BreakdownStrategy 2: Memory ConsiderationsStrategy 3: TransfersStrategy 4: Thread Usage, Calculations, and Divergence
Strategy 6: Resource ContentionsStrategy 7: Self-Tuning ApplicationsConclusion
IntroductionLibrariesCUDA Computing SDKDirective-Based ProgrammingWriting Your Own KernelsConclusion
IntroductionCPU ProcessorGPU DevicePCI-E BusGeForce cardsCPU MemoryAir CoolingLiquid CoolingDesktop Cases and MotherboardsMass StoragePower ConsiderationsOperating SystemsConclusion
IntroductionErrors With CUDA DirectivesParallel Programming IssuesAlgorithmic IssuesFinding and Avoiding ErrorsDeveloping for Future GPUsFurther ResourcesConclusionReferences

Content preview from CUDA Programming

Chapter 2 Understanding Parallelism with GPUs

Introduction

This chapter aims to provide a broad introduction to the concepts of parallel programming and how these relate to GPU technology. It’s primarily aimed at those people reading this text with a background in serial programming, but a lack of familiarity with parallel processing concepts. We look at these concepts in the primary context of GPUs.

Traditional Serial Code

A significant number of programmers graduated when serial programs dominated the landscape and parallel programming attracted just a handful of enthusiasts. Most people who go to university get a degree related to IT because they are interested in technology. However, they also appreciate they need to have a job or career ...