Probability, Random Variables, Statistics, and Random Processes

10Hypothesis Testing

When the decision is based on probabilities, the reasoning is referred to as statistical inference. The fundamentals of probability theory are employed to make accurate decisions in the presence of uncertainty. The steps in a statistical inference are thus as follows: perform an experiment, collect and analyze data, and state a decision. This chapter briefly highlights the significance testing, where a given hypothesis is accepted or rejected, and testing simple hypotheses, where testing of one hypothesis against another is considered using various decision rules and metrics. For example, if we have 100 independent, identically‐distributed (iid) samples of a Gaussian random variable with unknown mean, the question that can be addressed by significance testing is whether or not we should accept the hypothesis that the sample mean is, say, 2, whereas the question addressed by hypothesis testing is that if the sample mean is equal to, say, 1 or 2. The focus of this chapter is on binary hypothesis testing.

10.1 Significance Testing

In order to make decisions, assumptions are made about the populations, which may or may not be true. An assumption made about a population parameter is called a statistical hypothesis or simply a hypothesis. A statistical hypothesis is formulated for the sole purpose of rejecting (nullifying) it. Significance testing is a process through which a formally stated hypothesis about the population parameter can be accepted or rejected. ...