Performance and interpretability: A necessary trade-off?
Over the last decade, an increasing number of companies have embraced the digital transformation by incorporating Artificial Intelligence methods in the way they conceive their products and define their processes. Gathering, analyzing, and harnessing the data have been considered as essential drivers for business growth in an increasingly number of fields.
In this context, understanding the importance of model interpretability to solve concrete problems is the first step towards successful Data Science projects.
Explicability vs. Interpretability
Explicability. An algorithmic decision is said to be explainable if it can be explicitly accounted for on the basis of known data and characteristics of the situation. In other words, if it is possible to relate the values taken by certain variables (the characteristics) and their consequences on the prediction, for example of a score, and thus on the decision.
To use the example of the bank loan, the model can be considered as explainable if it explicitly indicates the relationship between the values taken by the variables (age, family situation, salary, etc.) and the final score.Interpretability. An algorithmic decision is said to be interpretable if it is possible to identify the characteristics or variables that participate most in the decision, or even to quantify their importance
For instance, the model used for loan scoring is interpretable if it is able to measure the relative importance of the variables used (age, family situation, salary, etc.) in the determination of the final score.Note that an explainable decision is interpretable. Now we are clear about the context, how does it apply in practice?
The graph below provides a graphical illustration for case of the credit loan application
Why is it important?
A Classification of Machine Learning Algorithms
Broadly speaking, machine learning algorithms can be structured in two groups according to whether they lead, by construction, to an explicit model (white box) or a black box.
|White box model||Black box model|
|Characteristics|| Explanation of the decision|
is possible via the method
or the model
|Once the method leading to the best |
predictions has been determined,
indicators are calculated a posteriori.
|Examples||Gaussian model, binomial,|
binary decision tree, etc.
under the assumption of a small
number of hyperparameters
|k-nearest neighbors, neural networks, |
support vector machines,
ensemble methods, etc.
- The first group contains regression algorithms, decision trees, and traditional classification rules. They are close to monotonic linear functions and commonly used in economics and sociology;
- The second group includes more advanced algorithms such as graphical models
- The third group consist of advanced machine learning techniques such as SVM, ensemble learning, and deep learning methods. They are only able to provide information on the importance of the variables for the explicability of the model.
Choosing the right model
- Do you need a model with an intrinsic or post-hoc interpretability ?
Depending on the level of interpretability required, you might choose a model easy to explain (intrinsic interpretability) or a back-box model for the training and the use of the results such as the feature importance to explain (post-hoc interpretability).
- Do you need Model-Specific or Model-agnostic interpretability ?
There is a distinction to be made between the interpretability that can be specific to a model such as linear regression where weights can be interpreted and the interpretability that comes from general tools that applied after the model training (such as correlation, matrix confusion, or model-specific metrics)
- Do you need local or global interpretability ?
The global explicability requirement is intended to make the decision-making process transparent for all data, while the local explicability criterion is intended to provide explanations for a single decision in a restricted neighborhood of data.
 Problématiques juridiques et analyse automatique des données, Machine Learning and the Law
https://perso.math.univ-toulouse.fr/mllaw/home/statisticien/explicabilite-des-decisions-algorithmiques/  A. Veriné, S. Mir, L’interprétabilité du Machine learning: quels défis à l’ère des processus de la décisions automatisés ? Wavestone report https://www.wavestone.com/app/uploads/2019/09/Wavestone_Interpretabilite_Machine_learning.pdf  J. Cupe, L’interprétabilité de l’IA – Le nouveau défi des data scientists, octobre 2018