Statistical Topics and Stochastic Models for Dependent Data with Applications

8 Inference in Parametric and Semi-parametric Models: The Divergence-based Approach

This chapter presents a short review of basic features of statistical inference in parametric and semi-parametric models through duality techniques in the realm of divergence-based methods. A simple solution for testing the number of components in a mixture is presented; this is a non-regular model with currently no amenable solution. The interplay between the choice of the divergence and the model is enlightened.

8.1. Introduction

This paper aims to present the main features of Csiszár-type divergences in relation with statistical inference. We first introduce some notations and definitions, and then embed the present setup in the context of decomposable measures of discrepancy between probability measures. A variational representation of Csiszár divergences is the basic ingredient for this embedding. The specificity of the classes of decomposable measures of discrepancy allows us to select such a class for specific inference purposes.

Minimizing a discrepancy between a given distribution P and a class of measures M occurs in a certain number of problems. On the one hand, when the distribution P is known, then the resulting problem is of analytic nature and bears no statistical aspect. On the other hand, the measure P might only be known through a sampling of observations X₁, .., X_n, whose empirical distribution

converges to P in some sense. The class M is then called a model, which may be ...