Think Bayes, 2nd Edition

Chapter 16. Logistic Regression

This chapter introduces two related topics: log odds and logistic regression.

In “Bayes’s Rule”, we rewrote Bayes’s theorem in terms of odds and derived Bayes’s rule, which can be a convenient way to do a Bayesian update on paper or in your head. In this chapter, we’ll look at Bayes’s rule on a logarithmic scale, which provides insight into how we accumulate evidence through successive updates.

That leads directly to logistic regression, which is based on a linear model of the relationship between evidence and the log odds of a hypothesis. As an example, we’ll use data from the Space Shuttle to explore the relationship between temperature and the probability of damage to the O-rings.

As an exercise, you’ll have a chance to model the relationship between a child’s age when they start school and their probability of being diagnosed with attention deficit hyperactivity disorder (ADHD).

Log Odds

When I was in grad school, I signed up for a class on the Theory of Computation. On the first day of class, I was the first to arrive. A few minutes later, another student arrived.

At the time, about 83% of the students in the computer science program were male, so I was mildly surprised to note that the other student was female.

When another female student arrived a few minutes later, I started to think I was in the wrong room. When a third female student arrived, I was confident I was in the wrong room. And as it turned out, I was.

I’ll use this anecdote ...