The Data Science Handbook, 2nd Edition

24Deep Learning and AI

This chapter will dive deeper into the technical details of deep‐learning models, many of the tricks you can do with them, and how they are used to create modern artificial intelligence (AI) systems. I have discussed neural networks previously, but only in their use as yet another machine learning classifier. This chapter is focused on techniques that are specific to neural networks, especially big ones, and that facilitate their key role in modern AI systems.

First off, a disclaimer. Deep learning is a very big topic, and the experts in it tend to be either machine learning (ML) engineers or researchers. Deep learning plays a comparatively minor role in data science – at least for now, and I suspect going forward as well – so my treatment here is somewhat sparse. Additionally, the techniques are evolving at a rapid clip, so the details I give could easily be out‐of‐date before this book comes off the presses.

I will cover the key concepts – including example code of course – but will not go into great detail. Very few data scientists will find themselves training high‐performing Large Language Models (LLMs). They are more likely to train mid‐tier models for specific problems, or to use pre‐trained models for a specific task. This chapter aims to equip you for these common use cases and give you a conceptual foundation for further learning if you want to go deeper (pun intended).

If there is a single concept that forms the unifying theme of this chapter, ...