Analytics in a Big Data World: The Essential Guide to Data Science and its Applications

CHAPTER 7 Analytics: Putting It All to Work 

In Chapter 1, we discussed the following key requirements of analytical models:

• Business relevance
• Statistical performance
• Interpretability and justifiability
• Operational efficiency
• Economical cost
• Regulatory compliance

When only considering statistical performance as the key objective, analytical techniques such as neural networks, SVMs, and random forests are among the most powerful. However, when interpretability and justifiability are the goal, then logistic regression and decision trees should be considered. Obviously, the ideal mix of these requirements largely depends on the setting in which analytics is to be used. For example, in fraud detection, response and/or retention modeling, interpretability, and justifiability are less of an issue. Hence, it is common to see techniques such as neural networks, SVMs, and/or random forests applied in these settings. In domains such as credit risk modeling and medical diagnosis, comprehensibility is a key requirement. Techniques such as logistic regression and decision trees are very popular here. Neural networks and/or SVMs can also be applied if they are complemented with white box explanation facilities using, for example, rule extraction and/or two-stage models, as explained in Chapter 3.

BACKTESTING ANALYTICAL MODELS

Backesting is an important model monitoring activity that aims at comparing ex-ante made predictions with ex-post observed numbers.¹ For example, consider the example ...