Probabilistic Machine Learning for Finance and Investing

Preface

Generative AI, and Chat GPT-4 in particular, is all the rage these days. Probabilistic machine learning (ML) is a type of generative AI that is ideally suited for finance and investing. Unlike deep neural networks, on which ChatGPT is based, probabilistic ML models are not black boxes. These models also enable you to infer causes from effects in a fairly transparent manner. This is important in heavily regulated industries, such as finance and healthcare, where you have to explain the basis of your decisions to many stakeholders.

Probabilistic ML also enables you to explicitly and systematically encode personal, empirical, and institutional knowledge into ML models to sustain your organization’s competitive advantages. What truly distinguishes probabilistic ML from its conventional counterparts is its capability of seamlessly simulating new data and counterfactual knowledge conditioned on the observed data and model assumptions on which it was trained and tested, regardless of the size of the dataset or the ordering of the data. Probabilistic models are generative models that know their limitations and honestly express their ignorance by widening the ranges of their inferences and predictions. You won’t get such quantified doubts from ChatGPT’s confident hallucinations, more commonly known as fibs and lies.

All ML models are built on the assumption that patterns discovered in training or in-sample data will persist in testing or out-of-sample data. However, when nonprobabilistic ...