🚀 Think you’ve got what it takes for a career in Data? Find out in just one minute!
🚀 Think you’ve got what it takes for a career in Data? Find out in just one minute!
After developing a Machine Learning model, evaluating its performance is crucial to assessing its effectiveness. To objectively compare multiple models, the use of metrics is indispensable. Understanding and knowing how to use these metrics is essential for developing an effective Machine Learning model. In this article, you will discover the main metrics used to evaluate a Machine Learning model.
Machine Learning allows computers to learn and make predictions or decisions based on data.
There are two types of learning: supervised learning and unsupervised learning.
In this article, we will focus on a supervised framework. For more details on the basics of Machine Learning and the difference between these two types of learning, we recommend reading this article, which introduces key Machine Learning concepts useful for understanding the use of metrics in a Machine Learning model.
A metric is a numerical value that quantifies the quality of a model’s predictions. Its role is essential during all stages of the development of a Machine Learning model as it helps determine if a model meets our expectations. Based on the results obtained, metrics allow for the objective comparison of several models with each other, the selection of the most efficient model, or changes to a model’s hyperparameters.
A good grasp of different metrics is essential for deploying an effective model.
To choose the appropriate metric, it is important to understand the context of the problem and the model’s objectives. There are many metrics, and we will present a few of them and their advantages.
In a supervised learning framework, one must first determine the type of prediction the model must make. If the model must predict a numerical value (e.g., the price of a house), it is a regression problem (e.g., linear regression). Conversely, if the model must predict a categorical value (e.g., the presence or absence of fraud in a banking transaction), we are in a classification context. The metrics used in regression models are indeed different from those used in classification models.
In this article, we will present two of the primary regression metrics: Mean Squared Error (MSE) and Mean Absolute Error (MAE).
where N is the number of observations, yi is the actual value, and ŷi is the predicted value.
This metric sums the squares of the differences between actual values and predictions. Mean Squared Error heavily penalizes large differences between the actual value and the prediction, which can be useful in contexts where such differences are particularly undesirable.
You can get more information on Mean Squared Error by reading this article, which details its characteristics and an example of its application.
where N is the number of observations, yi is the actual value, and ŷi is the predicted value.
This metric sums the absolute values of the differences between predictions and actual values.
Mean Absolute Error is less sensitive to large differences than Mean Squared Error.
In a classification framework, the way to evaluate a model’s performance is different. We will present three primary classification metrics: accuracy, precision, and recall.
This formula returns a number between 0 and 1. A score close to 1 indicates a very good model, whereas a score close to 0 indicates a poor model. This metric is quite intuitive and easy to understand. However, it poorly evaluates the performance of a model based on imbalanced data, or data where prediction errors do not have the same impact.
To illustrate this concept, let’s take the example of a model that detects the presence of a disease in a patient. If in 90% of the cases, the patient is not sick, the model could systematically predict that the patient is healthy. The accuracy of this model would then be 0.9, which seems to be a very good score. However, two major problems can arise:
Accuracy does not allow for nuances between different predictions and does not account for imbalanced data. This is why there are metrics to address this problem:
where TP represents the number of True Positives and FP represents the number of False Positives.
This metric is useful when the cost of false positives is high.
where TP represents the number of True Positives and FN represents the number of False Negatives.
This metric is useful when the cost of false negatives is high.
In conclusion, using metrics is essential to evaluate the performance of a Machine Learning model. Choosing the right metric for the model allows for making the correct decisions regarding how to improve it. Depending on the type of model (classification or regression model), context, and data type, certain metrics will be preferable to others. It is important to understand the advantages and disadvantages of each metric to use the one that best fits your problem.
Data Scientists use metrics to prepare effective Machine Learning models. For this, they use a multitude of mathematical concepts and specific software capable of preparing and analyzing data. As such, training is more than necessary. This is precisely possible with DataScientest. We offer comprehensive training programs in bootcamps, continuous, or alternating formats.
