Chapter 3

Learning in Parametric Modeling

Basic Concepts and Directions

Abstract

The chapter presents an overview of basic directions in machine learning and the basic notions related to parametric modeling are introduced. The tasks of regression and classification are defined and basic concepts related to parameter estimation are outlined such as estimator efficiency, Cramér-Rao bound, sufficient statistic. The least-squares estimator and some of its properties are discussed. The notions of inverse problems, overfitting, bias-variance dilemma and regularization are presented. The methods of maximum likelihood, maximum a posteriori and Bayesian inference are introduced. Finally, the curse of dimensionality and the cross-validation technique ...

Get Machine Learning now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.