{"id":230825,"date":"2024-03-14T06:30:00","date_gmt":"2024-03-14T06:30:00","guid":{"rendered":"https:\/\/datascientest.com\/?p=230825"},"modified":"2024-03-08T17:56:01","modified_gmt":"2024-03-08T17:56:01","slug":"missingno-tout-savoir","status":"publish","type":"post","link":"https:\/\/datascientest.com\/missingno-tout-savoir","title":{"rendered":"Missingno : la librairie Python pour les donn\u00e9es manquantes"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"230825\" class=\"elementor elementor-230825\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-2fe4034 e-flex e-con-boxed e-con\" data-id=\"2fe4034\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-6b291e8 elementor-widget elementor-widget-heading\" data-id=\"6b291e8\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-heading-title{padding:0;margin:0;line-height:1}.elementor-widget-heading .elementor-heading-title[class*=elementor-size-]>a{color:inherit;font-size:inherit;line-height:inherit}.elementor-widget-heading .elementor-heading-title.elementor-size-small{font-size:15px}.elementor-widget-heading .elementor-heading-title.elementor-size-medium{font-size:19px}.elementor-widget-heading .elementor-heading-title.elementor-size-large{font-size:29px}.elementor-widget-heading .elementor-heading-title.elementor-size-xl{font-size:39px}.elementor-widget-heading .elementor-heading-title.elementor-size-xxl{font-size:59px}<\/style><h2 class=\"elementor-heading-title elementor-size-default\">L\u2019une des \u00e9tapes les plus importantes d\u2019un projet en Data Science, avant d\u2019entamer la conception des mod\u00e8les de Machine Learning, est le nettoyage de notre jeu de donn\u00e9es. Cela implique la suppression des doublons, l\u2019encodage de certaines variables, mais aussi et surtout le remplacement des valeurs manquantes.<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a281ec0 elementor-widget elementor-widget-text-editor\" data-id=\"a281ec0\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-text-editor.elementor-drop-cap-view-stacked .elementor-drop-cap{background-color:#69727d;color:#fff}.elementor-widget-text-editor.elementor-drop-cap-view-framed .elementor-drop-cap{color:#69727d;border:3px solid;background-color:transparent}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap{margin-top:8px}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap-letter{width:1em;height:1em}.elementor-widget-text-editor .elementor-drop-cap{float:left;text-align:center;line-height:1;font-size:50px}.elementor-widget-text-editor .elementor-drop-cap-letter{display:inline-block}<\/style>\t\t\t\t<p><span style=\"font-weight: 400;\">Il n\u2019est en effet pas rare que le jeu de donn\u00e9es que nous manipulons ne soit pas enti\u00e8rement complet, et qu\u2019il contienne des valeurs manquantes appel\u00e9es <strong>\u00ab\u00a0NaN\u00a0\u00bb<\/strong>. Ces valeurs, si elles ne sont pas supprim\u00e9es ou remplac\u00e9es par diverses techniques, peuvent alt\u00e9rer la performance de nos mod\u00e8les.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-53c3b39 e-flex e-con-boxed e-con\" data-id=\"53c3b39\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-9f8e733 elementor-widget elementor-widget-heading\" data-id=\"9f8e733\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">C\u2019est ici qu\u2019intervient Missingno !<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e28c3fc elementor-widget elementor-widget-text-editor\" data-id=\"e28c3fc\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Outre le nom d\u2019un c\u00e9l\u00e8bre Pok\u00e9mon bug, <b>Missingno<\/b> est une librairie Python de \u00ab\u00a0missing data visualization\u00a0\u00bb vous permettant de mieux <b>comprendre et analyser les valeurs manquantes<\/b> de votre jeu de donn\u00e9es ! Cette librairie d\u00e9velopp\u00e9e par Aleksey \u00ab\u00a0ResidentMario\u00a0\u00bb Bilogur est disponible en acc\u00e8s libre sur son <a href=\"https:\/\/github.com\/ResidentMario\/missingno\">repository Github<\/a>\u00a0et est bien \u00e9videmment compatible avec le module Pandas.<\/span><\/p><p><span style=\"font-weight: 400;\">Comme toutes les librairies, Missingno s\u2019installe avec la commande <strong>pip install missingo<\/strong>, puis s\u2019importe dans un programme Python avec <strong>import missingo as msno<\/strong>.<\/span><\/p><p><span style=\"font-weight: 400;\">Voyons ensemble les quatre types de <b>visualisation<\/b> r\u00e9alisables gr\u00e2ce \u00e0 Missingno !<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-fcda623 e-flex e-con-boxed e-con\" data-id=\"fcda623\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-8b87d9f elementor-widget elementor-widget-heading\" data-id=\"8b87d9f\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Le diagramme en barres (bar chart)<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e908093 elementor-widget elementor-widget-text-editor\" data-id=\"e908093\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Le <b>diagramme en barres<\/b> propos\u00e9 sur Missingno se trace avec la fonction <strong>msno.bar(df)<\/strong> et se pr\u00e9sente comme suit :\u00a0<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5a2bf9f elementor-widget elementor-widget-image\" data-id=\"5a2bf9f\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-image{text-align:center}.elementor-widget-image a{display:inline-block}.elementor-widget-image a img[src$=\".svg\"]{width:48px}.elementor-widget-image img{vertical-align:middle;display:inline-block}<\/style>\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"1433\" height=\"823\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image3.png\" class=\"attachment-full size-full wp-image-230851\" alt=\"\" srcset=\"https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image3.png 1433w, https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image3-300x172.png 300w, https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image3-1024x588.png 1024w, https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image3-768x441.png 768w\" sizes=\"(max-width: 1433px) 100vw, 1433px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-55810f1 elementor-widget elementor-widget-text-editor\" data-id=\"55810f1\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Dans ce graphique, chaque barre symbolise une <b>colonne<\/b> du jeu de donn\u00e9es, et leur hauteur correspond au <b>taux de valeurs non nulles<\/b> dans chaque colonne. Nous avons \u00e0 gauche le pourcentage de valeurs compl\u00e8tes, \u00e0 droite les valeurs des index, et en haut le nombre de lignes.<\/span><\/p><p><span style=\"font-weight: 400;\">Pour ce jeu de donn\u00e9es par exemple, les colonnes <i>DATE<\/i>, <i>TIME<\/i> ou <i>NUMBERS OF PERSONS INJURED<\/i> ne contiennent aucunes valeurs manquantes. Inversement, la colonne <i>OFF STREET NAME<\/i> ne contient que tr\u00e8s peu de valeurs, tandis que <i>NUMBER OF CYCLISTS INJURED \/ KILLED<\/i> en sont totalement d\u00e9nu\u00e9es.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ffeef00 elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"ffeef00\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/formation-data-analyst\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Tout savoir de Missingno<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-c36008f e-flex e-con-boxed e-con\" data-id=\"c36008f\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-ed21e68 elementor-widget elementor-widget-heading\" data-id=\"ed21e68\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">La matrice de nullit\u00e9<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b89101a elementor-widget elementor-widget-text-editor\" data-id=\"b89101a\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">La <b>matrice de nullit\u00e9<\/b>, ou <b><i>\u00ab\u00a0nullity matrix\u00a0\u00bb<\/i><\/b>, se trace avec la fonction <i>msno.matrix(df)<\/i> :<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-52ab440 elementor-widget elementor-widget-image\" data-id=\"52ab440\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"1472\" height=\"787\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image4.png\" class=\"attachment-full size-full wp-image-230853\" alt=\"\" srcset=\"https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image4.png 1472w, https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image4-300x160.png 300w, https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image4-1024x547.png 1024w, https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image4-768x411.png 768w\" sizes=\"(max-width: 1472px) 100vw, 1472px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-13b8dc8 elementor-widget elementor-widget-text-editor\" data-id=\"13b8dc8\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Cette matrice s\u2019interpr\u00e8te presque comme le premier histogramme plus haut : pour chaque colonne, <\/span><b>plus la case est gris\u00e9e, plus elle contient de valeurs non nulles<\/b><span style=\"font-weight: 400;\">. Inversement, la pr\u00e9sence de barres blanches, plus ou moins importantes, symbolise l\u2019absence de valeurs.\u00a0<\/span><\/p><p><span style=\"font-weight: 400;\">On retrouve \u00e9galement, \u00e0 droite de la matrice, une fine courbe qui oscille de gauche \u00e0 droite, la droite \u00e9tant le nombre de colonnes du jeu de donn\u00e9es, et la gauche \u00e9tant donc \u00e0 z\u00e9ro. Ainsi, pour une ligne donn\u00e9e, lorsqu\u2019elle comporte une valeur dans chaque colonne, la courbe penchera vers la droite. Sinon, si les valeurs commencent \u00e0 manquer, elle penchera vers la gauche.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-6d90d9e e-flex e-con-boxed e-con\" data-id=\"6d90d9e\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-f8a59cb elementor-widget elementor-widget-heading\" data-id=\"f8a59cb\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">La matrice de corr\u00e9lation (heatmap)<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4857c11 elementor-widget elementor-widget-text-editor\" data-id=\"4857c11\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">\u00c0 l\u2019instar de <i>Seaborn<\/i>, la <b>matrice de corr\u00e9lation<\/b> de <i>Missingno<\/i> se trace avec <i>msno.heatmap(df)<\/i> :<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-478976b elementor-widget elementor-widget-image\" data-id=\"478976b\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"1422\" height=\"887\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image2.png\" class=\"attachment-full size-full wp-image-230854\" alt=\"\" srcset=\"https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image2.png 1422w, https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image2-300x187.png 300w, https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image2-1024x639.png 1024w, https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image2-768x479.png 768w\" sizes=\"(max-width: 1422px) 100vw, 1422px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-727c74e elementor-widget elementor-widget-text-editor\" data-id=\"727c74e\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Les valeurs de la matrice de corr\u00e9lation, aussi appel\u00e9es <\/span><b>\u00ab\u00a0corr\u00e9lations de nullit\u00e9\u00a0\u00bb<\/b><span style=\"font-weight: 400;\"> ou <\/span><b><i>\u00ab\u00a0nullity correlations\u00a0\u00bb<\/i><\/b><i><span style=\"font-weight: 400;\">, <\/span><\/i><span style=\"font-weight: 400;\">sont comprises entre -1 et 1 :\u00a0<\/span><\/p><ul><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Une valeur proche de <\/span><b>-1<\/b><span style=\"font-weight: 400;\"> signifie que si une variable appara\u00eet, il est tr\u00e8s probable que l&rsquo;autre variable soit manquante.<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Une valeur proche de <\/span><b>0 <\/b><span style=\"font-weight: 400;\">signifie qu&rsquo;il n&rsquo;y a pas de corr\u00e9lation particuli\u00e8re sur la pr\u00e9sence ou non de valeurs entre deux variables.<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Une valeur proche de <\/span><b>1<\/b><span style=\"font-weight: 400;\"> signifie que si une variable appara\u00eet, il est tr\u00e8s probable que l&rsquo;autre variable soit pr\u00e9sente.<\/span><\/li><\/ul>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c837277 elementor-widget elementor-widget-text-editor\" data-id=\"c837277\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Par exemple, pour la matrice ci-dessus, on observe de <b>tr\u00e8s fortes corr\u00e9lations<\/b> entre certaines colonnes <i>VEHICLE TYPE CODE<\/i> et <i>CONTRIBUTING FACTOR VEHICLE<\/i>, ce qui signifie que les informations \u00e0 propos des v\u00e9hicules semblent avoir le m\u00eame taux de compl\u00e9tion de valeurs (pr\u00e9sentes comme absentes). Inversement, <i>OFF STREET NAME<\/i> est <b>tr\u00e8s anti-corr\u00e9l\u00e9e<\/b> avec <i>ON \/ CROSS STREET NAME<\/i>, donc l\u00e0 o\u00f9 cette premi\u00e8re colonne manque de donn\u00e9es, les deux autres sont au contraire tr\u00e8s compl\u00e8tes.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d7075f9 elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"d7075f9\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/formation-data-scientist\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Tout ma\u00eetriser sur Missingno<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-b54f132 e-flex e-con-boxed e-con\" data-id=\"b54f132\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-0c9cd3c elementor-widget elementor-widget-heading\" data-id=\"0c9cd3c\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Le dendrogramme<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-427b2b4 elementor-widget elementor-widget-text-editor\" data-id=\"427b2b4\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Derri\u00e8re ce nom un peu barbare se cache un <b>diagramme en arborescence<\/b> qui tr\u00e8s utilis\u00e9 en Machine Learning non supervis\u00e9e afin de repr\u00e9senter la hi\u00e9rarchie des clusters. Avec <i>Missingno<\/i>, nous le tra\u00e7ons gr\u00e2ce la m\u00e9thode <i>msno.dendrogram(df)<\/i> :\u00a0<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d83a306 elementor-widget elementor-widget-image\" data-id=\"d83a306\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"1453\" height=\"780\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image1.png\" class=\"attachment-full size-full wp-image-230858\" alt=\"\" srcset=\"https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image1.png 1453w, https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image1-300x161.png 300w, https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image1-1024x550.png 1024w, https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/image1-768x412.png 768w\" sizes=\"(max-width: 1453px) 100vw, 1453px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7dfa89d elementor-widget elementor-widget-text-editor\" data-id=\"7dfa89d\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Le dendrogramme permet de <b>regrouper les colonnes selon leurs corr\u00e9lations de nullit\u00e9<\/b>. Plus les colonnes sont rapproch\u00e9es, plus cette corr\u00e9lation est importante (plus il y a un lien entre leur pr\u00e9sence ou absence de valeurs).<\/span><\/p><p><span style=\"font-weight: 400;\">Ici par exemple, nous pouvons distinguer <b>trois groupes<\/b> :\u00a0<\/span><\/p><ul><li>les colonnes tout \u00e0 droite, de <i>CONTRIBUTING FACTOR VEHICLE 1<\/i> \u00e0 <i>TIME<\/i>, pour lesquelles on ne retrouve aucune valeur manquante ;<\/li><li>les colonnes au milieu, de <i>CONTRIBUTING FACTOR VEHICLE 2<\/i> \u00e0 <i>VEHICLE TYPE CODE 1<\/i>, qui contiennent une l\u00e9g\u00e8re proportion de valeurs manquantes ;<\/li><li>les colonnes tout \u00e0 gauche, d\u2019<i>OFF STREET NAME<\/i> \u00e0 <i>VEHICLE TYPE CODE 5<\/i>, qui sont quasiment vides.<\/li><\/ul>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-e02e192 e-flex e-con-boxed e-con\" data-id=\"e02e192\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-9dc2df8 elementor-widget elementor-widget-heading\" data-id=\"9dc2df8\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">En conclusion<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2b0ef41 elementor-widget elementor-widget-text-editor\" data-id=\"2b0ef41\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">La librairie <i>Missingno<\/i> est ainsi <b>id\u00e9ale<\/b>, lorsque nous rencontrons des <b>valeurs manquantes<\/b> dans notre jeu de donn\u00e9es, pour visualiser o\u00f9 ces valeurs se situent et comment elles impactent nos variables.<\/span><\/p><p><span style=\"font-weight: 400;\">On peut distinguer les graphiques disponibles en <b>deux cat\u00e9gories<\/b> : ceux qui permettent de visualiser les <b>proportions de pr\u00e9sence et d\u2019absence<\/b> de valeurs dans chaque colonne (diagramme en barres, matrice de nullit\u00e9), et ceux qui permettent d\u2019\u00e9tudier les <b>corr\u00e9lations de nullit\u00e9<\/b> entre les variables (matrice de corr\u00e9lation, dendrogramme).<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-c406901 e-flex e-con-boxed e-con\" data-id=\"c406901\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-069e1e1 elementor-widget elementor-widget-text-editor\" data-id=\"069e1e1\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Pour devenir un expert en manipulation de donn\u00e9es, n\u2019h\u00e9sitez pas \u00e0 suivre les formations <a href=\"https:\/\/datascientest.com\/formation-data-scientist\">Data Scientist<\/a> &amp; <a href=\"https:\/\/datascientest.com\/formation-data-analyst\">Data Analyst<\/a> propos\u00e9es par DataScientest !<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ba385cf elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"ba385cf\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/formations-data\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">S'inscire \u00e0 l'une de nos formations data<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-084c4c0 elementor-widget elementor-widget-html\" data-id=\"084c4c0\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<script type=\"application\/ld+json\">\n{\n  \"@context\": \"https:\/\/schema.org\",\n  \"@type\": \"FAQPage\",\n  \"mainEntity\": [{\n    \"@type\": \"Question\",\n    \"name\": \"Qu\u2019est-ce que Missingno ?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"Missingno est une librairie Python de missing data visualization vous permettant de mieux comprendre et analyser les valeurs manquantes de votre jeu de donn\u00e9es ! Cette librairie d\u00e9velopp\u00e9e par Aleksey \u201cResidentMario\u201d Bilogur est disponible en acc\u00e8s libre sur son repository Github, et est bien \u00e9videmment compatible avec le module Pandas.\"\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"Comment installer Missingno ?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"Comme toutes les librairies, Missingno s\u2019installe avec la commande 'pip install missingno', puis s\u2019importe dans un programme Python avec 'import missingno as msno'.\"\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"Quels sont les types de visualisation r\u00e9alisables gr\u00e2ce \u00e0 Missingno ?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"Les types de visualisation r\u00e9alisables gr\u00e2ce \u00e0 Missingno incluent :\\\\n1. Le diagramme en barres (bar chart),\\\\n2. La matrice de nullit\u00e9,\\\\n3. La matrice de corr\u00e9lation (heatmap),\\\\n4. Le dendrogramme.\"\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"Pourquoi utiliser Missingno ?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"La librairie Missingno est id\u00e9ale, lorsque nous rencontrons des valeurs manquantes dans notre jeu de donn\u00e9es, pour visualiser o\u00f9 ces valeurs se situent et comment elles impactent nos variables.\"\n    }\n  }]\n}\n<\/script>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>L\u2019une des \u00e9tapes les plus importantes d\u2019un projet en Data Science, avant d\u2019entamer la conception des mod\u00e8les de Machine Learning, est le nettoyage de notre jeu de donn\u00e9es. Cela implique la suppression des doublons, l\u2019encodage de certaines variables, mais aussi et surtout le remplacement des valeurs manquantes. Il n\u2019est en effet pas rare que le [&hellip;]<\/p>\n","protected":false},"author":77,"featured_media":230850,"comment_status":"open","ping_status":"open","sticky":false,"template":"elementor_theme","format":"standard","meta":{"content-type":"","inline_featured_image":false},"categories":[2497],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.7 (Yoast SEO v21.7) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Missingno : la librairie Python pour les donn\u00e9es manquantes<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/datascientest.com\/missingno-tout-savoir\" \/>\n<meta property=\"og:locale\" content=\"fr_FR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Missingno : la librairie Python pour les donn\u00e9es manquantes\" \/>\n<meta property=\"og:url\" content=\"https:\/\/datascientest.com\/missingno-tout-savoir\" \/>\n<meta property=\"og:site_name\" content=\"Formation Data Science | DataScientest.com\" \/>\n<meta property=\"article:published_time\" content=\"2024-03-14T06:30:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-03-08T17:56:01+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/missingno-python-datascientest.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"686\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Raphael Kassel\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u00c9crit par\" \/>\n\t<meta name=\"twitter:data1\" content=\"Raphael Kassel\" \/>\n\t<meta name=\"twitter:label2\" content=\"Dur\u00e9e de lecture estim\u00e9e\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/datascientest.com\/missingno-tout-savoir\",\"url\":\"https:\/\/datascientest.com\/missingno-tout-savoir\",\"name\":\"Missingno : la librairie Python pour les donn\u00e9es manquantes\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/#website\"},\"datePublished\":\"2024-03-14T06:30:00+00:00\",\"dateModified\":\"2024-03-08T17:56:01+00:00\",\"author\":{\"@id\":\"https:\/\/datascientest.com\/#\/schema\/person\/6eca16f63daaafcbc96e15549b9d4ee1\"},\"inLanguage\":\"fr-FR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/datascientest.com\/missingno-tout-savoir\"]}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/datascientest.com\/#website\",\"url\":\"https:\/\/datascientest.com\/\",\"name\":\"Formation Data Science | DataScientest.com\",\"description\":\"Leader de la formation en Data Science\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/datascientest.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"fr-FR\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/datascientest.com\/#\/schema\/person\/6eca16f63daaafcbc96e15549b9d4ee1\",\"name\":\"Raphael Kassel\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\/\/datascientest.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/11\/raph.jpeg\",\"contentUrl\":\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/11\/raph.jpeg\",\"caption\":\"Raphael Kassel\"},\"description\":\"Raphael Kassel est le Head of Data Programs chez DataScientest. Il supervise une \u00e9quipe Data, impl\u00e9mente de nouveaux parcours comme \\\"Chef de projet IA\\\" et \\\"Data Product Manager\\\", et contribue \u00e0 la modernisation des op\u00e9rations du groupe. En collaboration \u00e9troite avec l'\u00e9quipe de d\u00e9veloppement, il adapte l'accompagnement acad\u00e9mique aux \u00e9volutions de l'\u00e9quipe data et \u00e0 l'internationalisation.\",\"sameAs\":[\"https:\/\/www.linkedin.com\/in\/raphaelkassel\/\",\"Data Science\"],\"url\":\"https:\/\/datascientest.com\/author\/raphaelkassel\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Missingno : la librairie Python pour les donn\u00e9es manquantes","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/datascientest.com\/missingno-tout-savoir","og_locale":"fr_FR","og_type":"article","og_title":"Missingno : la librairie Python pour les donn\u00e9es manquantes","og_url":"https:\/\/datascientest.com\/missingno-tout-savoir","og_site_name":"Formation Data Science | DataScientest.com","article_published_time":"2024-03-14T06:30:00+00:00","article_modified_time":"2024-03-08T17:56:01+00:00","og_image":[{"width":1200,"height":686,"url":"https:\/\/datascientest.com\/wp-content\/uploads\/2024\/02\/missingno-python-datascientest.jpg","type":"image\/jpeg"}],"author":"Raphael Kassel","twitter_card":"summary_large_image","twitter_misc":{"\u00c9crit par":"Raphael Kassel","Dur\u00e9e de lecture estim\u00e9e":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/datascientest.com\/missingno-tout-savoir","url":"https:\/\/datascientest.com\/missingno-tout-savoir","name":"Missingno : la librairie Python pour les donn\u00e9es manquantes","isPartOf":{"@id":"https:\/\/datascientest.com\/#website"},"datePublished":"2024-03-14T06:30:00+00:00","dateModified":"2024-03-08T17:56:01+00:00","author":{"@id":"https:\/\/datascientest.com\/#\/schema\/person\/6eca16f63daaafcbc96e15549b9d4ee1"},"inLanguage":"fr-FR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/datascientest.com\/missingno-tout-savoir"]}]},{"@type":"WebSite","@id":"https:\/\/datascientest.com\/#website","url":"https:\/\/datascientest.com\/","name":"Formation Data Science | DataScientest.com","description":"Leader de la formation en Data Science","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/datascientest.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"fr-FR"},{"@type":"Person","@id":"https:\/\/datascientest.com\/#\/schema\/person\/6eca16f63daaafcbc96e15549b9d4ee1","name":"Raphael Kassel","image":{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/datascientest.com\/#\/schema\/person\/image\/","url":"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/11\/raph.jpeg","contentUrl":"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/11\/raph.jpeg","caption":"Raphael Kassel"},"description":"Raphael Kassel est le Head of Data Programs chez DataScientest. Il supervise une \u00e9quipe Data, impl\u00e9mente de nouveaux parcours comme \"Chef de projet IA\" et \"Data Product Manager\", et contribue \u00e0 la modernisation des op\u00e9rations du groupe. En collaboration \u00e9troite avec l'\u00e9quipe de d\u00e9veloppement, il adapte l'accompagnement acad\u00e9mique aux \u00e9volutions de l'\u00e9quipe data et \u00e0 l'internationalisation.","sameAs":["https:\/\/www.linkedin.com\/in\/raphaelkassel\/","Data Science"],"url":"https:\/\/datascientest.com\/author\/raphaelkassel"}]}},"_links":{"self":[{"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/posts\/230825"}],"collection":[{"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/users\/77"}],"replies":[{"embeddable":true,"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/comments?post=230825"}],"version-history":[{"count":0,"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/posts\/230825\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/media\/230850"}],"wp:attachment":[{"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/media?parent=230825"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/categories?post=230825"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/tags?post=230825"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}