Data Quality Engineering in Financial Services

Chapter 5. Data Quality Metrics and Visualization

This chapter demonstrates how to visualize the data quality metrics that were generated in Chapter 4, after applying the DQS framework to data volumes.

Data Quality Metrics

As discussed, data quality metrics are the results generated from data quality measurements, defined in DQS, that are applied to your data. In Chapter 4, we applied the completeness DQS to the raw security master data volume. The statistics from the completeness data validation are 25 datum records comprised of 11 data elements for a total of 275 datum values. This yields 248 valid, 22 invalid, and 5 suspect data quality metrics for the completeness dimension. The metrics for all DQS applied to all data elements in the raw security master data volume are summarized in Table 5-1.

Table 5-1. Summary of data quality metrics after application of all DQS to raw security master data volume

Dimension	Data element	Valid	Invalid	Suspect
Completeness	Ticker	22	3	0
	Issue Name	22	3	0
	Exchange	20	5	0
	Bid	23	2	0
	Ask	21	4	0
	Spread	25	0	0
	Market Cap	25	0	0
	Market Cap Scale	23	2	0
	Price to Earnings (PE)	22	3	0
	Consensus Recommendation	21	0	4
	Consensus Date	24	0	1
Timeliness	Consensus Date	15	6	4
Accuracy	Ticker	16	9	0
	Issue Name	11	14	0
	Exchange	20	5	0
Precision	Bid	17	2	6
	Ask	17	4	4
	Spread	17	3	5
	PE	20	3	2
Conformity	Issue Name	19	6	0
	Market Cap Scale	22	3	0
	Consensus Recommendation	19	2	4
Metrics totals	441	79	30 ...