Data Science and Big Data Analytics: Discovering, Analyzing, Visualizing and Presenting Data

6

Advanced Analytical Theory and Methods: Regression

Key Concepts

Categorical Variable

Linear Regression

Logistic Regression

Ordinary Least Squares (OLS)

Receiver Operating Characteristic (ROC) Curve

Residuals

In general, regression analysis attempts to explain the influence that a set of variables has on the outcome of another variable of interest. Often, the outcome variable is called a dependent variable because the outcome depends on the other variables. These additional variables are sometimes called the input variables or the independent variables. Regression analysis is useful for answering the following kinds of questions:

• What is a person's expected income?
• What is the probability that an applicant will default on a loan?

Linear regression is a useful tool for answering the first question, and logistic regression is a popular method for addressing the second. This chapter examines these two regression techniques and explains when one technique is more appropriate than the other.

Regression analysis is a useful explanatory tool that can identify the input variables that have the greatest statistical influence on the outcome. With such knowledge and insight, environmental changes can be attempted to produce more favorable values of the input variables. For example, if it is found that the reading level of 10-year-old students is an excellent predictor of the students' success in high school and a factor in their attending college, then additional emphasis on reading ...