Advancing into Analytics

Chapter 3. Foundations of Inferential Statistics

Chapter 1 provided a framework for exploring a dataset by classifying, summarizing, and visualizing variables. Though this is an essential start to analytics, we usually don’t want it to end there: we would like to know whether what we see in our sample data can generalize to a larger population.

The thing is, we don’t actually know what we’ll find in the population, because we don’t have the data for all of it. However, using the principles of probability introduced in Chapter 2, we can quantify our uncertainty that what we see in our sample will also be found in the population.

Estimating the values of a population given a sample is known as inferential statistics and is carried out by hypothesis testing. That framework is the basis of this chapter. You may have studied inferential statistics in school, which could have easily turned you off the subject, seeming incomprehensible and without application. That’s why I’ll make this chapter as applied as possible, exploring a real-world dataset using Excel.

By the end of the chapter, you will have a handle on this basic framework that powers much of analytics. We’ll continue to build on its application in Chapter 4.

Chapter 1 concluded with an exercise on the housing dataset, which will be the focus of this chapter. You can find the dataset in the datasets folder of the book repository, under the housing subfolder. Make a copy of it, add an index column, and convert this dataset into ...