Start learning

MSE, MAE and RMSE

Lesson 9. About 7 minutes.

MSE, MAE and RMSE measure how far a regression model's predictions are from the real values, each in a slightly different way. Together with R squared, they tell you how good a regression model really is.

Your Path Through This Lesson

0 of 5 done

By the End of This Lesson You Can
  • Calculate MSE, MAE and RMSE by hand from a list of errors.
  • Explain why MSE punishes big mistakes more than MAE does.
  • Choose the right metric for a problem, and compute it in scikit-learn.

Why We Need More Than R Squared

R squared tells you the share of variation a model explains: 0.6 for our students. But a teacher asking "how many marks will my prediction be off by?" wants an answer in marks, not a percentage. That's what these three metrics give you.

We'll keep using our five students and the line :

Student (hours) (marks) error
Noman 1 20 28 −8
Lamia 2 40 34 6
Rafi 3 50 40 10
Tanvir 4 40 46 −6
Arif 5 50 52 −2

Mean Absolute Error (MAE)

Take the size of each error, ignoring the sign, and average them:

On average, the line misses by 6.4 marks. MAE is in the same units as the target, and it's the easiest metric to explain to anyone.

Mean Squared Error (MSE)

Square each error, then average:

The unit is now "marks squared", which is hard to picture. But squaring has a big advantage: it's smooth and easy to differentiate, which is why the cost function we minimise during training is built from it.

Root Mean Squared Error (RMSE)

Take the square root of MSE to get back to the original units:

RMSE is in marks, like MAE, but it's a little bigger (6.93 vs 6.4) because the 10-mark miss for Rafi counts more after squaring.

Big Mistakes: Where They Differ

Imagine two models with the same total error of 28 marks across five students:

Errors MAE RMSE
Model A 6, 6, 6, 5, 5 5.6 5.62
Model B 2, 2, 2, 2, 20 5.6 9.12

MAE says they're equally good. RMSE says Model B is much worse, because one student was off by 20 marks. Squaring punishes big mistakes.

Use When
MAE Every mistake costs the same per unit; you want a robust number that outliers don't dominate
RMSE Big mistakes are much worse than small ones, like predicting delivery times or medicine doses
MSE Training (the maths is easier); less often for reporting
R² Comparing how well models explain the target, independent of units
Watch Out

Lower is better for MAE, MSE and RMSE, but higher is better for R². When scikit-learn scores models during cross-validation it wants "higher is better" everywhere, so it uses negative MSE (neg_mean_squared_error). A score of −48 means an MSE of 48.

In scikit-learn

python
import numpy as np
from sklearn.metrics import mean_absolute_error, mean_squared_error, root_mean_squared_error, r2_score

y_true = np.array([20, 40, 50, 40, 50])
y_pred = np.array([28, 34, 40, 46, 52])

print("MAE :", mean_absolute_error(y_true, y_pred))
print("MSE :", mean_squared_error(y_true, y_pred))
print("RMSE:", round(root_mean_squared_error(y_true, y_pred), 2))
print("R2  :", round(r2_score(y_true, y_pred), 2))
Output
MAE : 6.4
MSE : 48.0
RMSE: 6.93
R2  : 0.6

The same numbers we worked out by hand. Now the outlier example:

python
a = np.array([6, 6, 6, 5, 5])      # absolute errors of model A
b = np.array([2, 2, 2, 2, 20])     # absolute errors of model B
zeros = np.zeros(5)

for name, err in [("A", a), ("B", b)]:
    print(name, "MAE", mean_absolute_error(zeros, err), " RMSE", round(root_mean_squared_error(zeros, err), 2))
Output
A MAE 5.6  RMSE 5.62
B MAE 5.6  RMSE 9.12
Try It Yourself
  1. Change Arif's real mark from 50 to 90, so the line misses him by 38. Recompute all three metrics. Which one jumps the most?
  2. Is RMSE ever smaller than MAE? Try a few lists of errors and see. (It can't be, and the two are equal only when every error has the same size.)
The Key Idea

MAE is the average miss. RMSE is in the same units but punishes big misses more. MSE is RMSE squared and is mostly used for training.

Summary

  • MAE = average absolute error. Easy to explain, robust to outliers.
  • MSE = average squared error. Used for training.
  • RMSE = √MSE. Same units as the target, punishes large errors.
  • For our line: MAE 6.4, MSE 48, RMSE 6.93.
  • Next: the same ideas with more than one feature, in multiple linear regression.