Topic 2.8 Least Squares Regression: determine the least-squares regression line from summary statistics, and interpret the coefficient of determination r-squared and the standard deviation of the residuals.

What this topic is asking

The College Board (Topic 2.8) wants you to find the least-squares regression line from summary statistics (means, standard deviations, and $r$), to know why it is the best-fitting line, and to interpret the coefficient of determination $r^2$ and the standard deviation of the residuals.

Why "least squares"

Among all possible lines, exactly one minimizes that sum, and that is the line technology reports. The choice to square (rather than, say, take absolute values) is what makes the slope and intercept have clean formulas in terms of $r$ and the standard deviations, and it ties the line to the correlation you already know.

Computing the line from summary statistics

The slope formula is worth reading: it scales the correlation by the ratio of the spreads, converting the unit-free $r$ into a slope in the units of $y$ per unit of $x$. Because it contains $r$, the slope has the same sign as the correlation: positive correlation gives positive slope. And once you have the slope, the intercept formula forces the line through $(\bar{x}, \bar{y})$, a fact that is itself sometimes tested directly.

Interpreting r-squared

The coefficient of determination $r^2$ is the single most important fit measure on the exam. It is the proportion (or percentage) of the variation in the response $y$ that is explained by the linear model with $x$. If $r^2 = 0.64$, then about $64\%$ of the variability in $y$ is accounted for by its linear relationship with $x$, and the remaining $36\%$ is due to other factors and random variation. A full-credit interpretation always contains four elements: the percentage, "of the variation in [$y$ in context]," "is explained by," and "the linear relationship with [$x$ in context]." Two errors recur: interpreting $r^2$ as the proportion of points on the line (it is about variation, not points), and confusing $r^2$ with $r$ (the correlation). Because $r^2 = (r)^2$, you can move between them, but they answer different questions: $r$ measures the strength and direction of the linear association, while $r^2$ measures the share of variation explained.

The standard deviation of the residuals

The other fit measure is $s$, the standard deviation of the residuals, which estimates the typical size of a prediction error in the units of $y$. Where $r^2$ is a unitless proportion, $s$ is a concrete "on average our predictions are off by about $s$ [units]." A smaller $s$ means tighter predictions. Reading the two together gives a rounded picture: $r^2$ says what fraction of the variation the line captures, and $s$ says, in real units, how large the leftover errors typically are. On the exam, $s$ usually appears in computer output labelled near the regression equation, and you interpret it as the typical residual size, for example "predicted exam scores are typically off by about $4$ points." Being fluent at pulling $b$, $a$, $r$, $r^2$, and $s$ out of standard regression output, and interpreting each in context, is exactly the skill the next layer of exam questions (and the guide on reading computer output) builds on.

Try this

Q1. A regression has $r = -0.5$. Find $r^2$ and state what it means. [2 points]

• Cue. $r^2 = (-0.5)^2 = 0.25$; about $25\%$ of the variation in $y$ is explained by the linear relationship with $x$.

Q2. Given $\bar{x} = 10$, $\bar{y} = 50$, and slope $b = 4$, find the intercept. [1 point]

• Cue. $a = \bar{y} - b\bar{x} = 50 - 4(10) = 50 - 40 = 10$.