Industrial Applications of High-Performance Computing

315

Chapter 24

A View from

International Data

Corporation (IDC)

HPC and HPDA

Steve Conway and Chirag Dekate

CONTENTS

24.1 Introduction 316

24.2 IDC Opinion 317

24.3 Situation Overview 318

24.3.1 HPC: 10 Essential Considerations 318

24.3.1.1 HPC Is One of the Fastest-Growing

IT Markets 318

24.3.1.2 Commercial Firms Began Adopting HPC

in the 1970s 319

24.3.1.3 Ninety-Seven Percent of Adopters Say

HPC Is Indispensable for eir Ability to

Compete and Survive 320

24.3.1.4 Senior Government Ocials Increasingly

Recognize HPC’s Economic Value 320

24.3.1.5 HPC Is Dierent from Business Computing 321

24.3.1.6 Goals of IT and HPC Are Also Dierent 322

24.3.1.7 Key IT Datacenter Technologies Have

Trickled Down from HPC 322

24.3.1.8 HPC Systems Now Start at Under $10,000 323

316 ◾ Industrial Applications of High-Performance Computing

24.1 INTRODUCTION

e rst part of this chapter summarizes the historical growth of the

worldwide high-performance computing (HPC) market and describes

important dierences between HPC and commercial business computing.

e remainder of the chapter zeroes in on the formative market for high-

performance data analysis (HPDA)—big data using HPC—and how this

market is bringing the worlds of HPC and commercial business computing

closer together. To illustrate the value HPC and its close relative, HPDA,

can bring to an organization, the chapter includes rst-hand perspectives

from two very dierent adopters who graciously agreed to contribute to

24.3.1.9 Commercial Firms Are Also Adopting

HPC for Challenging Big Data Problems 324

24.3.1.10 ere is More on Tap from HPC 325

24.4 Case Studies 325

24.4.1 e National Oceanic and Atmospheric

Administration: HPC as a Mission-Critical Tool 325

24.4.2 PayPal Exploits HPC for Fraud Detection 327

24.5 Future Outlook 328

24.6 Essential Guidance 329

24.7 HPDA: Big Data Meets HPC 329

24.8 e Origins of Data-Intensive Computing 330

24.8.1 Data-Intensive Computing Categories 331

24.8.2 High-Performance Data Analysis 332

24.8.3 What Is Driving Demand for HPDA? 333

24.8.4 Running Modeling and Simulation Problems with

Greater Realism 333

24.8.5 Expanding Analytics Use in Established HPC Fields 334

24.8.6 Fraud Detection, Cyber Security, and Insider reats 336

24.8.7 A New Wave of Commercial Adopters 338

24.8.8 Other Commercial HPDA Applications 339

24.8.9 An Extreme Application: China Will Use HPC to

Help Manage Its Internet of ings 340

24.8.10 Vendors and the HPDA Convergence Market 341

24.8.11 HPDA Worldwide Market Size and Forecast 342

24.8.12 A Major Challenge: Data Movement and Storage 342

24.9 Conclusion 343

24.10 Related Research 345

A View from International Data Corporation (IDC) ◾ 317

the International Data Corporation (IDC) report on which much of this

chapter is based:

1. e rst perspective is from the National Oceanic and Atmospheric

Administration (NOAA), a government organization that, for many

years, has relied on HPC to produce its National Weather Service

forecasts and other leading-edge work.

2. e second perspective comes from PayPal, a successful, global

e-commerce company that adopted HPC not long ago for real-time

detection of online fraud and now plans to extend HPC use to an-

ity marketing and other applications.

24.2 IDC OPINION

HPC, once a niche market serving government- and university-based

researchers, began penetrating Tier-1 commercial rms in the late 1970s.

HPC quickly established itself as a game changer for accelerating innova-

tion and competitiveness in public- and private-sector organizations. e

arrival of commercial-grade clusters in 2001–2002 made HPC aordable

even for most small- and medium-size enterprises (SMEs) and startups,

with HPC system prices now starting at under $10,000. During the past

two decades, HPC has been one of the fastest growing markets in infor-

mation technology (IT), expanding from $2 billion in 1990 to $21.9 billion

in 2012. Few people would have imagined at the dawn of the supercom-

puter era that HPC systems would be used to help design products ranging

from cars and airplanes to golf clubs, potato chips, and diapers—much

less to enable a company such as PayPal to detect online consumer fraud

in near real time.

IDC forecasts that

• Commercial HPC adoption will continue to ramp up, helping to

propel the HPC market to $29 billion in 2017.

• As more companies of all sizes in more markets learn to exploit HPC

to speed and improve innovation, competitors lacking this advan-

tage will fall behind.

• Successful corporate chief information ocers (CIOs) will need to

gain a basic understanding of HPC and ensure that their organiza-

tions carefully consider whether to adopt this technology.