Analytics for the Internet of Things (IoT)

Trade-off and complexity

In a perfect world, you would have ML models that, after the learning process, have both low bias and low variance. However, as you may be aware, we live in a far from perfect world. So, as in most things, there is a trade-off needed to get the optimal result.

The goal of ML research efforts is to not only find the optimal trade-off combination but to also develop new ML techniques that sacrifice a little on one in exchange for a significant reduction in the other. For any given ML model, there is a trade-off to be made between bias and variance in order to find the optimal configuration that will generalize the best. Unfortunately, this optimal position is unknown and has to be inferred based on error calculations ...