Programming Machine Learning

Unreasonable Effectiveness

As you read through this book, you might have been surprised by the capabilities of simple programs like our first MNIST classifier. And yet, little prepares you for the uncanny capabilities of modern deep networks. In the words of one famous researcher, those networks are “unreasonably effective.”^[33] In a sense, they do nothing more than recognize patterns—and yet, they increasingly beat us at quintessentially human tasks, like facial recognition or medical diagnosis. How is that even possible?

Here’s the short answer: we don’t really know. As I write, nobody understands why deep neural networks work so well on so many tasks. Indeed, there’s a lot of ongoing research bent to explain that fact.

At first sight, it’s ...