315
Chapter 24
A View from
International Data
Corporation (IDC)
HPC and HPDA
Steve Conway and Chirag Dekate
CONTENTS
24.1 Introduction 316
24.2 IDC Opinion 317
24.3 Situation Overview 318
24.3.1 HPC: 10 Essential Considerations 318
24.3.1.1 HPC Is One of the Fastest-Growing
IT Markets 318
24.3.1.2 Commercial Firms Began Adopting HPC
in the 1970s 319
24.3.1.3 Ninety-Seven Percent of Adopters Say
HPC Is Indispensable for eir Ability to
Compete and Survive 320
24.3.1.4 Senior Government Ocials Increasingly
Recognize HPCs Economic Value 320
24.3.1.5 HPC Is Dierent from Business Computing 321
24.3.1.6 Goals of IT and HPC Are Also Dierent 322
24.3.1.7 Key IT Datacenter Technologies Have
Trickled Down from HPC 322
24.3.1.8 HPC Systems Now Start at Under $10,000 323
316 Industrial Applications of High-Performance Computing
24.1 INTRODUCTION
e rst part of this chapter summarizes the historical growth of the
worldwide high-performance computing (HPC) market and describes
important dierences between HPC and commercial business computing.
e remainder of the chapter zeroes in on the formative market for high-
performance data analysis (HPDA)—big data using HPC—and how this
market is bringing the worlds of HPC and commercial business computing
closer together. To illustrate the value HPC and its close relative, HPDA,
can bring to an organization, the chapter includes rst-hand perspectives
from two very dierent adopters who graciously agreed to contribute to
24.3.1.9 Commercial Firms Are Also Adopting
HPC for Challenging Big Data Problems 324
24.3.1.10 ere is More on Tap from HPC 325
24.4 Case Studies 325
24.4.1 e National Oceanic and Atmospheric
Administration: HPC as a Mission-Critical Tool 325
24.4.2 PayPal Exploits HPC for Fraud Detection 327
24.5 Future Outlook 328
24.6 Essential Guidance 329
24.7 HPDA: Big Data Meets HPC 329
24.8 e Origins of Data-Intensive Computing 330
24.8.1 Data-Intensive Computing Categories 331
24.8.2 High-Performance Data Analysis 332
24.8.3 What Is Driving Demand for HPDA? 333
24.8.4 Running Modeling and Simulation Problems with
Greater Realism 333
24.8.5 Expanding Analytics Use in Established HPC Fields 334
24.8.6 Fraud Detection, Cyber Security, and Insider reats 336
24.8.7 A New Wave of Commercial Adopters 338
24.8.8 Other Commercial HPDA Applications 339
24.8.9 An Extreme Application: China Will Use HPC to
Help Manage Its Internet of ings 340
24.8.10 Vendors and the HPDA Convergence Market 341
24.8.11 HPDA Worldwide Market Size and Forecast 342
24.8.12 A Major Challenge: Data Movement and Storage 342
24.9 Conclusion 343
24.10 Related Research 345
A View from International Data Corporation (IDC) 317
the International Data Corporation (IDC) report on which much of this
chapter is based:
1. e rst perspective is from the National Oceanic and Atmospheric
Administration (NOAA), a government organization that, for many
years, has relied on HPC to produce its National Weather Service
forecasts and other leading-edge work.
2. e second perspective comes from PayPal, a successful, global
e-commerce company that adopted HPC not long ago for real-time
detection of online fraud and now plans to extend HPC use to an-
ity marketing and other applications.
24.2 IDC OPINION
HPC, once a niche market serving government- and university-based
researchers, began penetrating Tier-1 commercial rms in the late 1970s.
HPC quickly established itself as a game changer for accelerating innova-
tion and competitiveness in public- and private-sector organizations. e
arrival of commercial-grade clusters in 20012002 made HPC aordable
even for most small- and medium-size enterprises (SMEs) and startups,
with HPC system prices now starting at under $10,000. During the past
two decades, HPC has been one of the fastest growing markets in infor-
mation technology (IT), expanding from $2 billion in 1990 to $21.9 billion
in 2012. Few people would have imagined at the dawn of the supercom-
puter era that HPC systems would be used to help design products ranging
from cars and airplanes to golf clubs, potato chips, and diapers—much
less to enable a company such as PayPal to detect online consumer fraud
in near real time.
IDC forecasts that
Commercial HPC adoption will continue to ramp up, helping to
propel the HPC market to $29 billion in 2017.
As more companies of all sizes in more markets learn to exploit HPC
to speed and improve innovation, competitors lacking this advan-
tage will fall behind.
Successful corporate chief information ocers (CIOs) will need to
gain a basic understanding of HPC and ensure that their organiza-
tions carefully consider whether to adopt this technology.

Get Industrial Applications of High-Performance Computing now with the O’Reilly learning platform.

O’Reilly members experience live online training, plus books, videos, and digital content from nearly 200 publishers.