Regression Metrics — MAE, RMSE, R²

Mean Absolute Error (MAE)

Mean Absolute Error is the most intuitive regression metric, representing the average magnitude of the errors in a set of predictions, without considering their direction. It is calculated by taking the sum of the absolute differences between the predicted and actual values and dividing by the number of observations. Because MAE treats all errors linearly, it provides a very interpretable result: an MAE of 10 in a house price prediction model literally means the model is, on average, off by 10 units of currency. MAE is highly robust to outliers because it does not square the error terms, meaning one extreme data point will not disproportionately influence the metric. This makes it the preferred metric when the data is known to contain significant noise or when the relative impact of all errors should be weighted equally regardless of their magnitude.

import numpy as np

def calculate_mae(y_true, y_pred):
    # Absolute difference ensures errors don't cancel out
    return np.mean(np.abs(y_true - y_pred))

y_true = np.array([100, 150, 200])
y_pred = np.array([110, 140, 205])
print(f"MAE: {calculate_mae(y_true, y_pred)}")

Root Mean Squared Error (RMSE)

Root Mean Squared Error is a more sensitive metric that penalizes larger errors more heavily than MAE. The process involves calculating the difference between the predicted and actual values, squaring those differences, taking the mean, and finally taking the square root. By squaring the residuals, the model is punished significantly more for large deviations, making RMSE an excellent choice for applications where large errors are particularly dangerous or costly, such as in high-stakes financial forecasting or safety-critical engineering tasks. However, this sensitivity comes at the cost of robustness; a single massive outlier can inflate the RMSE value significantly, potentially masking the fact that the model performs well on the majority of the data. If you notice a high RMSE alongside a low MAE, it is a strong indicator that your model is making a few very large errors rather than many small ones.

import numpy as np

def calculate_rmse(y_true, y_pred):
    # Squaring errors penalizes larger deviations heavily
    return np.sqrt(np.mean((y_true - y_pred)**2))

y_true = np.array([100, 150, 200])
y_pred = np.array([110, 140, 205])
print(f"RMSE: {calculate_rmse(y_true, y_pred)}")

R-squared (Coefficient of Determination)

R-squared provides a statistical measure of how well the regression predictions approximate the actual data points, representing the proportion of variance for the dependent variable that is explained by independent variables. Unlike MAE and RMSE, which are in the units of the target variable, R-squared is a dimensionless scale, typically ranging from 0 to 1, though it can be negative if the chosen model is worse than simply guessing the mean of the data. It is essentially comparing your model against a 'naive' baseline that always predicts the mean value of the targets. While R-squared is useful for judging the 'goodness of fit,' it is not a complete diagnostic. A high R-squared does not necessarily imply that your model is bias-free, as it can be artificially inflated by adding irrelevant features. It is best used as a high-level summary of model performance relative to a trivial baseline.

from sklearn.metrics import r2_score

y_true = [100, 150, 200]
y_pred = [110, 140, 205]
# R2 compares model error to the variance of the baseline
print(f"R-squared: {r2_score(y_true, y_pred)}")

Comparing Error Distributions

To truly evaluate model performance, one must look beyond single scalar values and analyze the distribution of residuals. Residuals are simply the differences between the ground truth and the predictions. By plotting a histogram of these residuals, you can often identify systematic patterns that aggregate metrics like RMSE or MAE might hide. For example, if your residuals show a bell-shaped curve centered at zero, your model is likely capturing the signal well with only random noise remaining. However, if you see a skewed distribution, it suggests the model is consistently over-predicting or under-predicting specific ranges of data. Always complement your automated metrics with visualization to ensure the model's errors are distributed normally and not clustered around specific features, which would indicate a flaw in the feature selection or the chosen hypothesis class used for learning.

import matplotlib.pyplot as plt

residuals = y_true - y_pred
# Plotting allows detection of systematic bias
plt.hist(residuals, bins=5)
plt.xlabel('Residual Error')
plt.ylabel('Frequency')

Selecting the Right Metric

Choosing the correct metric requires a deep understanding of your specific use case objectives. If your goal is to minimize the total average distance without worrying about the occasional wild prediction, MAE is the mathematically sound choice for your deployment. If the business environment dictates that a large miss is a catastrophic failure—such as predicting medical dosage or autonomous braking timing—then you must choose RMSE to force the model to minimize these large individual errors at all costs. Furthermore, if you need to explain model performance to non-technical stakeholders, R-squared acts as a convenient shorthand for 'percentage of variance explained.' Expert practitioners often calculate all three, using them in concert to get a holistic view: RMSE for tuning, MAE for operational error estimation, and R-squared for comparative baseline analysis across different versions of the model.

def evaluate_model(y_true, y_pred):
    # Providing a complete evaluation suite for diagnostics
    return {
        "mae": calculate_mae(y_true, y_pred),
        "rmse": calculate_rmse(y_true, y_pred),
        "r2": r2_score(y_true, y_pred)
    }
print(evaluate_model(y_true, y_pred))

Interview questions

All Machine Learning interview questions →

Check yourself

Take the full Machine Learning quiz →