Regression metrics are quantitative measures used to evaluate the performance of regression models. They provide information about how well a regression model fits the data and how accurately it predicts the outcome variable. If you want to learn how to evaluate your Regression Machine Learning models, this article is for you. In this article, I’ll take you through a complete guide to all Regression performance evaluation metrics in Machine Learning with their implementation using Python.
Regression Performance Evaluation Metrics
Below are all the regression performance evaluation metrics in Machine Learning you should know:
- Mean Squared Error
- Root Mean Squared Error
- Mean Absolute Error
- R-squared
- Adjusted R-squared
Before understanding these regression performance evaluation metrics, let’s train a regression model so that we can understand the meaning and implementation of each metric one by one.
Training a Regression Model
Let’s train a simple regression model on the diabetes dataset:
from sklearn.datasets import load_diabetes from sklearn.linear_model import LinearRegression from sklearn.model_selection import train_test_split # Load the diabetes dataset diabetes_data = load_diabetes() # Select features and target variable X = diabetes_data.data y = diabetes_data.target # Split the data into training and testing sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) # Train a linear regression model model = LinearRegression() model.fit(X_train, y_train) # Make predictions on the test set y_pred = model.predict(X_test)
In the above code, we used linear regression on the diabetes dataset by splitting the data into training and testing sets, training a linear regression model on the training data, and then using the trained model to make predictions on the test data. It will allow us to evaluate the model’s performance in predicting disease progression in diabetes patients.
Now, let’s go through each regression performance evaluation metric.
Mean Squared Error
Mean Squared Error works by calculating the squared differences between each predicted value and its corresponding actual value. These differences are then averaged to obtain the MSE score. By squaring the differences, MSE emphasizes larger errors, making it sensitive to outliers.
The desired result for the mean squared error is to get a lower value, indicating a regression model that closely aligns the predicted values with the actual values. A lower MSE suggests better accuracy and lower average prediction error. Here’s how to calculate Mean Squared Error using Python:
# Compute the Mean Squared Error
from sklearn.metrics import mean_squared_error
mse = mean_squared_error(y_test, y_pred)
print("Mean Squared Error:", mse)Mean Squared Error: 2900.193628493482
The MSE score quantifies the mean square difference between predicted and actual values in a regression model. In our case, an MSE score of 2900.19 suggests that, on average, the predictions made by the model differ from the actual values by about 2900.19 squared units.
Root Mean Squared Error
RMSE works by calculating the square root of the root mean square difference between predicted and actual values. It captures the typical size of errors made by the model and provides a more interpretable measure.
The desired result for the root mean square error is to get a lower value, indicating a regression model that has smaller prediction errors and is more accurate in its predictions. A lower RMSE suggests a better fit of the model to the data and more accurate predictions. Here’s how to calculate root mean squared error using Python:
from sklearn.metrics import mean_squared_error
import numpy as np
mse = mean_squared_error(y_test, y_pred)
rmse = np.sqrt(mse)
print("Root Mean Squared Error:", rmse)Root Mean Squared Error: 53.85344583676593
An RMSE score of 53.85 indicates that, on average, model predictions deviate from actual values by approximately 53.85 units. It suggests the need for further refinement and evaluation of the model to obtain more accurate predictions and improve the overall performance of the regression model.
Mean Absolute Error
MAE calculates the average of the absolute differences between the predicted and actual values. It captures the typical size of errors made by the model and provides a more interpretable measure.
The desired result for the mean absolute error is to get a lower value, indicating a regression model that has smaller prediction errors and is more accurate in its predictions. A lower MAE suggests a better fit of the model to the data and more accurate predictions. Here’s how to calculate mean absolute error using Python:
from sklearn.metrics import mean_absolute_error
mae = mean_absolute_error(y_test, y_pred)
print("Mean Absolute Error:", mae)Mean Absolute Error: 42.79409467959994
An MAE score of 42.79 indicates that, on average, the model’s predictions deviate from the actual values by about 42.79 units. It suggests the need for further refinement and evaluation of the model to obtain more accurate predictions and improve the overall performance of the regression model.
R-squared
R-squared works by comparing the variation in predicted values to the variation in actual values. It quantifies the proportion of the total variation of the dependent variable that is accounted for by the regression model. The R-squared ranges from 0 to 1, with higher values indicating a better fit of the model to the data.
The desired result for the R-squared is to obtain a higher value, closer to 1. A higher value indicates that a greater proportion of the variance of the dependent variable is explained by the independent variables in the model of regression. It suggests a better fit of the model to the data and more accurate predictions. Here’s how to calculate R-squared using Python:
from sklearn.metrics import r2_score
r2 = r2_score(y_test, y_pred)
print("R-squared:", r2)R-squared: 0.4526027629719195
An R-squared score of 0.45 suggests that the regression model explains about 45% of the variation in the dependent variable. It indicates the need for further investigation and possibly refinement of the model to achieve a higher degree of explanation and improve its overall performance.
Adjusted R-squared
Adjusted R-squared works by penalizing the inclusion of unnecessary variables in the model. It adjusts the R-squared value by incorporating the number of independent variables and the sample size, providing a more conservative assessment of the model’s goodness of fit.
The desired result for the adjusted R-squared is to obtain a higher value, closer to 1. A higher adjusted R-squared indicates that the model can explain a greater proportion of the variance of the dependent variable, even after taking into account the number of independent variables and potential overfitting. Here’s how to calculate Adjusted R-squared using Python:
from sklearn.metrics import r2_score
r2 = r2_score(y_test, y_pred)
n = len(y_test)
k = 10 # Number of independent variables in the model
adjusted_r2 = 1 - ((1 - r2) * (n - 1) / (n - k - 1))
print("Adjusted R-squared:", adjusted_r2)Adjusted R-squared: 0.38242363001960145
An adjusted R-squared score of 0.38 suggests that the regression model explains about 38% of the variation in the dependent variable, given the complexity of the model and the number of independent variables. Further analysis and refinement of the model are required to achieve a higher degree of explanation and improve the overall performance of the regression model.
So, this is how you can evaluate your regression models using performance evaluation metrics using Python. You can learn many more Machine Learning algorithms and concepts from my book – Machine Learning Algorithms: Handbook.
Summary
So below are all the regression performance evaluation metrics you should know:
- Mean Squared Error: Mean Squared Error works by calculating the squared differences between each predicted value and its corresponding actual value.
- Root Mean Squared Error: RMSE works by calculating the square root of the root mean square difference between predicted and actual values.
- Mean Absolute Error: MAE calculates the average of the absolute differences between the predicted and actual values.
- R-squared: R-squared works by comparing the variation in predicted values to the variation in actual values.
- Adjusted R-squared: Adjusted R-squared works by penalizing the inclusion of unnecessary variables in the model.
I hope you liked this article on Regression Performance Evaluation Metrics in Machine Learning with their implementation using Python. Feel free to ask valuable questions in the comments section below.
