CHAPTER 2Data Management and Preparation

“Deep learning craves big data because big data is necessary to isolate hidden patterns and to find answers without over-fitting the data. With deep learning, the better-quality data you have, the better the results.”

—Wayne Thompson, SAS Chief Data Scientist

Data has become a vital resource for organizations, entities, and governments alike.

For the risk management function, data has always been a pivotal enabler. Since its inception as a scientific discipline, sound risk management has been underpinned by efficiency in obtaining and retrieving of information—at the time when it’s needed—and robust data management. To name a few examples, data is used to inform risk assessments, monitor risks, and help to detect new types of risks. For risk modeling, real-time and granular data are increasingly being used to develop, monitor, and maintain better and more innovative risk models.

A critical lesson learned from the Global Financial Crisis was that banks' information technology (IT) and data architectures were inadequate to support the broad management of financial risks. Some banks were unable to manage their risks properly because of weak risk data aggregation capabilities and risk-reporting practices. This had severe consequences for the banks themselves and to the stability of the financial system. In response, the Basel Committee on Banking Standards (BCBS) developed a standard with principles for effective risk data, its aggregation, ...

Get Risk Modeling now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.