O'Reilly logo

Effective Amazon Machine Learning by Alexis Perrier

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Comparing with a baseline

The RMSE is a relative quantity and not an absolute one. An RMSE of 100 may be good for a certain context, while a RMSE of 10 may be a sign of poor predictions in another context. It is, therefore, important to have a baseline that we can compare our model to. Each Amazon ML evaluation provides a baseline, which is calculated differently depending on the nature of the problem (regression, binary or multiclass classification). The baseline is the score we would obtain using the most simple and obvious model:

  • The baseline for regression is given by a model that always predicts the mean of the target
  • The baseline for binary classification is an AUC of 0.5, which is the score for a model that randomly assigns 0 or 1 ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required