Data Quality Engineering in Financial Services

Chapter 2. The Shape of Data

This chapter will challenge you to think like a manufacturer and to think about data as if it had physical form. This thought process, this comparison to manufacturing physical material, is the first step toward developing an enlightened understanding about data, its shape and form, and (most importantly) assessing data quality at the data dimension level.

Data as Physical Asset

In your firm, data must be thought of as a critically important physical asset. When data is treated as a physical material and regarded as an asset of the firm, then applying techniques used in industrial materials manufacturing to achieve high-quality products becomes easier to understand and implement.

Data has temporal dimensionality, which means data is generally either dynamic or persistent:

Dynamic data

This type of data changes over time and is fluid in the context of the business processes (e.g., current analytics, trade lists, portfolio positions, transactions, cash flows, performance).

Persistent data

This type of data does not change (at least, not nearly as much). This data represents history.

Dynamic and persistent data are managed and curated differently, and often with different technologies and techniques. Curated, fully validated, confirmed historical data typically represents persistent data.

Data lives in technology. While this seems obvious, it is important to recognize that all data initiatives require a technological component, since data does not ...