Measuring Regression Error

Intuition

How wrong are the predictions? Average the errors — but squared errors punish big misses more than absolute errors. R² compares the model with the simplest possible model: always predicting the mean.

Precise explanation

With residuals ei=yi−y^ie_i = y_i - \hat y_i:

  • MSE=1n∑ei2\text{MSE} = \frac1n\sum e_i^2, RMSE=MSE\text{RMSE} = \sqrt{\text{MSE}} (same units as y).
  • MAE=1n∑∣ei∣\text{MAE} = \frac1n\sum |e_i| — less sensitive to outliers.
  • R2=1−∑ei2∑(yi−yˉ)2R^2 = 1 - \frac{\sum e_i^2}{\sum (y_i - \bar y)^2}. R2=1R^2 = 1 is perfect, 0 equals the mean baseline, negative is worse than the baseline. Undefined when all yiy_i are equal.

Worked example

y = [3, 5, 7], ŷ = [2, 5, 9]: e = [1, 0, −2]; MSE 5/3 ≈ 1.667, RMSE 1.291, MAE 1; Σ(y−ȳ)² = 8, R² = 1 − 5/8 = 0.375.

Common mistakes

  • Averaging signed errors (positive and negative cancel).
  • Swapping arguments in R² (it is not symmetric).
  • Reporting R² on training data only.

Why this matters

These metrics evaluate ML-09/07/12 and are the losses many models minimise.

Library implementation

sklearn.metrics provides mean_squared_error, root_mean_squared_error, mean_absolute_error and r2_score with the same definitions. Comparison runs when scikit-learn is installed.

References