Correlation and regression

Correlation and regression

Scatter diagrams

A scatter diagram plots pairs of values $(x, y)$ for two variables. Correlation describes the pattern.

• Positive correlation: $y$ tends to increase as $x$ increases.
• Negative correlation: $y$ tends to decrease as $x$ increases.
• No correlation: no pattern.

The nearer the points are to a straight line, the stronger the correlation.

Explanatory and response variables

The explanatory (independent) variable $x$ is the one that is controlled, or is used to explain the other. The response (dependent) variable $y$ is the one that is measured or predicted. On a scatter diagram the explanatory variable goes on the horizontal axis.

Hours of revision is the explanatory variable $x$, and the score is the response variable $y$.

Summary values

For $n$ pairs of values, the regression and correlation formulae use

$S_{xx} = \sum x^2 - \frac{(\sum x)^2}{n}, \qquad S_{yy} = \sum y^2 - \frac{(\sum y)^2}{n}, \qquad S_{xy} = \sum xy - \frac{\sum x \sum y}{n}$

The regression line

The least squares regression line of $y$ on $x$ is the straight line $y = a + bx$ that makes the sum of the squares of the vertical distances from the points as small as possible.

$b = \frac{S_{xy}}{S_{xx}}, \qquad a = \bar{y} - b\bar{x}$

The line always passes through $(\bar{x}, \bar{y})$.

$S_{xx} = 240 - \frac{40^2}{8} = 40, \qquad S_{xy} = 520 - \frac{40 \times 96}{8} = 40$

$b = \frac{40}{40} = 1, \qquad \bar{x} = 5, \quad \bar{y} = 12, \qquad a = 12 - 1 \times 5 = 7$

The line is $y = 7 + x$.

Interpreting the line

• The gradient $b$ is the change in $y$ for each increase of 1 in $x$.
• The intercept $a$ is the predicted value of $y$ when $x = 0$. It may have no meaning if $x = 0$ is outside the data.

The score is predicted to rise by 1 mark for each extra hour of revision. With no revision the predicted score is 7.

Using the line to predict

Substitute a value of $x$ to predict $y$.

• Interpolation predicts inside the range of the data and is usually reliable.
• Extrapolation predicts outside the range and is less reliable, because the pattern may not continue.

Product moment correlation coefficient

The product moment correlation coefficient $r$ measures how close the points are to a straight line:

$r = \frac{S_{xy}}{\sqrt{S_{xx}S_{yy}}}$

It always lies between $-1$ and $1$.

$S_{yy} = 1260 - \frac{96^2}{8} = 108, \qquad r = \frac{40}{\sqrt{40 \times 108}} = \frac{40}{65.7} = 0.609$

This shows moderate positive linear correlation.

Limitations

• $r$ only measures linear correlation. A curved pattern can have $r$ near 0.
• A single unusual point (an outlier) can change $r$ and the line a great deal.
• A high $r$ does not show that $x$ causes $y$.
• A prediction beyond the data may not hold.