{"id":173848,"date":"2023-11-26T13:09:14","date_gmt":"2023-11-26T12:09:14","guid":{"rendered":"https:\/\/datascientest.com\/en\/?p=173848"},"modified":"2023-11-26T13:13:59","modified_gmt":"2023-11-26T12:13:59","slug":"preprocessing-what-is-it-how-does-it-work","status":"publish","type":"post","link":"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work","title":{"rendered":"Preprocessing: What is it? How does it work?"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"173848\" class=\"elementor elementor-173848\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-ea92d63 e-flex e-con-boxed e-con\" data-id=\"ea92d63\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-91cbe93 elementor-widget elementor-widget-heading\" data-id=\"91cbe93\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-heading-title{padding:0;margin:0;line-height:1}.elementor-widget-heading .elementor-heading-title[class*=elementor-size-]>a{color:inherit;font-size:inherit;line-height:inherit}.elementor-widget-heading .elementor-heading-title.elementor-size-small{font-size:15px}.elementor-widget-heading .elementor-heading-title.elementor-size-medium{font-size:19px}.elementor-widget-heading .elementor-heading-title.elementor-size-large{font-size:29px}.elementor-widget-heading .elementor-heading-title.elementor-size-xl{font-size:39px}.elementor-widget-heading .elementor-heading-title.elementor-size-xxl{font-size:59px}<\/style><h2 class=\"elementor-heading-title elementor-size-default\">The proliferation of data acquisition and systematic processing has facilitated the rise of machine learning methods that require ample data for training and operation. While one might naively assume that having a large amount of data is sufficient for a high-performing algorithm, the data we have is often not well-suited, and preprocessing is typically necessary before using them: this is the preprocessing step.<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2609ca7 elementor-widget elementor-widget-text-editor\" data-id=\"2609ca7\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-text-editor.elementor-drop-cap-view-stacked .elementor-drop-cap{background-color:#69727d;color:#fff}.elementor-widget-text-editor.elementor-drop-cap-view-framed .elementor-drop-cap{color:#69727d;border:3px solid;background-color:transparent}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap{margin-top:8px}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap-letter{width:1em;height:1em}.elementor-widget-text-editor .elementor-drop-cap{float:left;text-align:center;line-height:1;font-size:50px}.elementor-widget-text-editor .elementor-drop-cap-letter{display:inline-block}<\/style>\t\t\t\t<p>Indeed, acquisition errors due to human or technical mistakes can <a href=\"https:\/\/datascientest.com\/en\/what-is-a-dataset-how-do-i-work-with-it\">corrupt our dataset<\/a> and introduce bias during training. Among these errors, we can mention incomplete information, missing or incorrect values, or even noisy artifacts related to data acquisition.<\/p><p>Therefore, it is often essential to establish a data preprocessing strategy &#8211; also known as <strong>Data Preprocessing<\/strong> &#8211; starting from our raw data to obtain usable data that will lead to a more efficient model. We will explore the most important steps of this preprocessing, their significance, and their implementation in <a href=\"https:\/\/datascientest.com\/en\/pytest-for-python-why-and-how-to-use-it\">Python for some of them.<\/a><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a2d4d62 elementor-widget elementor-widget-image\" data-id=\"a2d4d62\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-image{text-align:center}.elementor-widget-image a{display:inline-block}.elementor-widget-image a img[src$=\".svg\"]{width:48px}.elementor-widget-image img{vertical-align:middle;display:inline-block}<\/style>\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2022\/04\/image3.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\"><\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-54299fa elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"54299fa\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-management-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Start a training in Data Management<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-536994f e-flex e-con-boxed e-con\" data-id=\"536994f\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-728ada5 elementor-widget elementor-widget-heading\" data-id=\"728ada5\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Data Cleaning: The different stages<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9b6e8e0 elementor-widget elementor-widget-text-editor\" data-id=\"9b6e8e0\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The first step involves <a href=\"https:\/\/datascientest.com\/en\/data-cleaning-definition-methods-and-relevance-in-data-science\">cleaning incorrect, incomplete, or missing data<\/a>. There are several ways to address these issues, which we will review.<\/p><p>If there is missing data in the dataset, you can choose to ignore it if the database is <strong>sufficiently large and if many data points are missing in the same row.<\/strong><\/p><p>Alternatively, you can decide to fill in these missing data in different ways: you can replace them with the mean, median, or, for categorical variables, with the most frequent mode.<\/p><p><a href=\"https:\/\/datascientest.com\/en\/pandas-the-python-library\">Pandas provides us with methods<\/a> that allow us to perform these treatments as follows:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4971674 elementor-widget elementor-widget-shortcode\" data-id=\"4971674\" data-element_type=\"widget\" data-widget_type=\"shortcode.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-shortcode\"><script src=\"https:\/\/gist.github.com\/zaxaheb-dst\/2b91ed470621d54f6750865e3579ef16.js\"><\/script><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-37ca5fc elementor-widget elementor-widget-text-editor\" data-id=\"37ca5fc\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Data can sometimes suffer from <strong>acquisition noise<\/strong>, in which case they may not be correctly processed by a computer. One way to address this issue is to perform data binning (after sorting the data). The data is divided into groups of the same size, and each group is treated independently.<\/p><p>Within the same group, all data points can be replaced by their mean, median, or extreme values.<\/p><p>Another way to handle noisy data is to use regression or clustering, which automatically creates data groups that can help us detect outliers and<a href=\"https:\/\/datascientest.com\/en\/refactoring-databases-and-code-comprehensive-guide-to-the-essentials\"> remove them from the database.<\/a><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-f79586e e-flex e-con-boxed e-con\" data-id=\"f79586e\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-7d9896b elementor-widget elementor-widget-heading\" data-id=\"7d9896b\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Data Transformation: What for?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-00f5e38 elementor-widget elementor-widget-text-editor\" data-id=\"00f5e38\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>This <strong>preprocessing step<\/strong> involves changes made to the data structure itself. These transformations are related to the mathematical definitions of algorithms and how they process data in order to <a href=\"https:\/\/datascientest.com\/en\/boosting-business-with-3-essential-machine-learning-algorithms\">optimize performance<\/a>. Among these techniques, we can mention:<\/p><p>&#8211; Data smoothing if the data is noisy.<br \/>&#8211; Data aggregation from various different sources.<br \/>&#8211; Discretization of continuous variables (using interval splitting) to reduce the number of categories for a descriptor.<br \/>&#8211; Normalization and standardization of data, which scale numerical data to a smaller range (e.g., between -1 and 1) and can also center the mean and reduce variance.<\/p><p>Here&#8217;s an example of how to perform normalization, which is often necessary in this data transformation part:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8cea93e elementor-widget elementor-widget-shortcode\" data-id=\"8cea93e\" data-element_type=\"widget\" data-widget_type=\"shortcode.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-shortcode\"><script src=\"https:\/\/gist.github.com\/zaxaheb-dst\/e687d5c652378455a06ee085b88fdab2.js\"><\/script><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-46ff021 e-flex e-con-boxed e-con\" data-id=\"46ff021\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-1b2dbf7 elementor-widget elementor-widget-heading\" data-id=\"1b2dbf7\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Data Reduction: Was ist das?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-578b546 elementor-widget elementor-widget-text-editor\" data-id=\"578b546\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>While it may seem <strong>intuitive that a large amount of data<\/strong> improves the performance of a model, having an excessively large dataset can sometimes make the analysis more complex.<\/p><p>Therefore, it can be interesting to reduce the quantity or dimension of the data to improve storage capabilities and reduce analysis costs without sacrificing performance (and sometimes even <strong>gaining performance<\/strong> in certain cases). There are several data reduction techniques.<\/p><p>For example, we can choose to keep a certain number of variables we prefer and drop others. The selection of relevant variables can be done through analysis of the variable&#8217;s p-value or through decision tree techniques that provide an estimation of the importance of different descriptors.<\/p><p>Another widely used technique for data reduction is dimensionality reduction. This method reduces the dimension of the data through <strong>well-defined encoding mechanisms<\/strong>.<\/p><p>There are two types of dimensionality reduction: with or without loss. If you can reconstruct the exact data from the reduced data, it&#8217;s called <strong>lossless reduction.<\/strong> Otherwise, the reduction is lossy. Two preferred methods for this type of data reduction are wavelet<strong> transformation or Principal Component Analysis (PCA).<\/strong><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-8ea233b e-flex e-con-boxed e-con\" data-id=\"8ea233b\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-c8f280f elementor-widget elementor-widget-heading\" data-id=\"c8f280f\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Data Integration: Ein wichtiger Schritt in der Vorverarbeitung<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d0ad805 elementor-widget elementor-widget-text-editor\" data-id=\"d0ad805\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>This step in the<strong> preprocessing strategy<\/strong> involves combining multiple sources into a single dataset. It is carried out in a data management framework for creating usable databases (such as creating image <a href=\"https:\/\/datascientest.com\/en\/api-integration-in-python-databases-made-easy\">databases<\/a>, cross-sectional abdominal scans, MRI or X-rays for diagnostic support).<\/p><p>However, some issues may arise, such as the incompatibility of certain formats or <strong>data redundancy.<\/strong><\/p><p>The preliminary data processing step is one of the most crucial in <strong>data processing and analysis.<\/strong><\/p><p>There is no perfect method to apply to every model creation, but we have discussed best practices to implement in a data preprocessing strategy.<\/p><p>The methods presented here are explored in more depth in our various training programs, where fundamental mathematical concepts and best practices for data preprocessing based on context and situation are explained.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-82668b3 elementor-widget elementor-widget-image\" data-id=\"82668b3\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2022\/04\/image1.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\"><\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-abba0bf elementor-widget elementor-widget-text-editor\" data-id=\"abba0bf\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>To discover our curriculum in detail and learn all the best practices of data preprocessing, find out more by clicking below:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-fb3d3ba elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"fb3d3ba\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-scientist-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Discover our Data Science training courses<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-482b7bb e-flex e-con-boxed e-con\" data-id=\"482b7bb\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-8dc780e elementor-widget elementor-widget-html\" data-id=\"8dc780e\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<script type=\"application\/ld+json\">\n{\n  \"@context\": \"https:\/\/schema.org\",\n  \"@type\": \"FAQPage\",\n  \"mainEntity\": [{\n    \"@type\": \"Question\",\n    \"name\": \"Data Cleaning : The different steps\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"The first step is to clean up any incorrect, incomplete or missing data. There are several ways of dealing with these problems, which we'll look at below. If there is missing data in the dataset, you can choose to ignore it if the database is quite full and there is a lot of missing data in the same row. You can also decide to fill in the missing data in different ways: you can replace it with the mean value, with the median, or, for example, with the most frequent modality in the case of categorical variables (otherwise known as the mode).\"\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"Data Transformation: For what purpose?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"This pre-processing stage groups together the changes made to the structure of the data itself. These transformations are linked to the mathematical definitions of the algorithms and the way they process the data, so as to optimize performance.\"\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"Data Reduction: What is it?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"Although it seems intuitive to imagine that a very large amount of data improves the performance of a model, it may be that too much data can make analysis more complicated. It can therefore sometimes be worthwhile to reduce the quantity or size of data, in order to improve storage capacities and reduce analysis costs, without losing performance (or, in some cases, even gaining it). There are a number of data reduction techniques, for example, we can choose a certain number of variables that we prefer to keep and drop others. The choice of relevant variables can be made by analysis of the variable's p-value, or by decision tree techniques which give us an estimate of the importance of the various descriptors.\"\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"Data Integration: An important step in preprocessing\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"This step in the preprocessing strategy involves combining multiple sources into a single dataset. It is carried out within a data management framework for the creation of exploitable databases (such as the creation of image databases, cross-sections of the abdomen, MRIs or X-rays for diagnostic aid problems). There are, however, certain problems that could arise, such as incompatibility of certain formats or redundancy of certain data.\"\n    }\n  }]\n}\n<\/script>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>The proliferation of data acquisition and systematic processing has facilitated the rise of machine learning methods that require ample data for training and operation. While one might naively assume that having a large amount of data is sufficient for a high-performing algorithm, the data we have is often not well-suited, and preprocessing is typically necessary [&hellip;]<\/p>\n","protected":false},"author":67,"featured_media":173873,"comment_status":"open","ping_status":"open","sticky":false,"template":"elementor_theme","format":"standard","meta":{"content-type":"","inline_featured_image":false},"categories":[2425],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.7 (Yoast SEO v21.7) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Preprocessing: What is it? How does it work?<\/title>\n<meta name=\"description\" content=\"Data preprocessing is a crucial step in the data analysis and machine learning pipeline. It involves cleaning, transforming, and organizing raw data into a format that is suitable for analysis or model training.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Preprocessing: What is it? How does it work?\" \/>\n<meta property=\"og:description\" content=\"The proliferation of data acquisition and systematic processing has facilitated the rise of machine learning methods that require ample data for training\" \/>\n<meta property=\"og:url\" content=\"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work\" \/>\n<meta property=\"og:site_name\" content=\"Data Science Courses | DataScientest\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataScientestEN\" \/>\n<meta property=\"article:published_time\" content=\"2023-11-26T12:09:14+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-11-26T12:13:59+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/datascientest.com\/en\/files\/2023\/11\/data-preprocessing-1.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"438\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"author\" content=\"Melanie\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:site\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Melanie\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work#article\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work\"},\"author\":{\"name\":\"Melanie\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d\"},\"headline\":\"Preprocessing: What is it? How does it work?\",\"datePublished\":\"2023-11-26T12:09:14+00:00\",\"dateModified\":\"2023-11-26T12:13:59+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work\"},\"wordCount\":838,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"articleSection\":[\"Data Science\"],\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work#respond\"]}],\"copyrightYear\":\"2023\",\"copyrightHolder\":{\"@id\":\"https:\/\/datascientest.com\/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work\",\"url\":\"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work\",\"name\":\"Preprocessing: What is it? How does it work?\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/#website\"},\"datePublished\":\"2023-11-26T12:09:14+00:00\",\"dateModified\":\"2023-11-26T12:13:59+00:00\",\"description\":\"Data preprocessing is a crucial step in the data analysis and machine learning pipeline. It involves cleaning, transforming, and organizing raw data into a format that is suitable for analysis or model training.\",\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work\"]}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/datascientest.com\/en\/#website\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"name\":\"Data Science Courses | DataScientest\",\"description\":\"Leader in Data Science Courses\",\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/datascientest.com\/en\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/datascientest.com\/en\/#organization\",\"name\":\"DataScientest\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"contentUrl\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"width\":301,\"height\":301,\"caption\":\"DataScientest\"},\"image\":{\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/DataScientestEN\",\"https:\/\/twitter.com\/DataScientest__\",\"https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/\",\"https:\/\/www.tiktok.com\/@datascientest\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d\",\"name\":\"Melanie\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g\",\"caption\":\"Melanie\"},\"url\":\"https:\/\/datascientest.com\/en\/author\/melaniekrause\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Preprocessing: What is it? How does it work?","description":"Data preprocessing is a crucial step in the data analysis and machine learning pipeline. It involves cleaning, transforming, and organizing raw data into a format that is suitable for analysis or model training.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work","og_locale":"en_GB","og_type":"article","og_title":"Preprocessing: What is it? How does it work?","og_description":"The proliferation of data acquisition and systematic processing has facilitated the rise of machine learning methods that require ample data for training","og_url":"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work","og_site_name":"Data Science Courses | DataScientest","article_publisher":"https:\/\/www.facebook.com\/DataScientestEN","article_published_time":"2023-11-26T12:09:14+00:00","article_modified_time":"2023-11-26T12:13:59+00:00","og_image":[{"width":1024,"height":438,"url":"https:\/\/datascientest.com\/en\/files\/2023\/11\/data-preprocessing-1.webp","type":"image\/webp"}],"author":"Melanie","twitter_card":"summary_large_image","twitter_creator":"@DataScientest__","twitter_site":"@DataScientest__","twitter_misc":{"Written by":"Melanie","Estimated reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work#article","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work"},"author":{"name":"Melanie","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d"},"headline":"Preprocessing: What is it? How does it work?","datePublished":"2023-11-26T12:09:14+00:00","dateModified":"2023-11-26T12:13:59+00:00","mainEntityOfPage":{"@id":"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work"},"wordCount":838,"commentCount":0,"publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"articleSection":["Data Science"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work#respond"]}],"copyrightYear":"2023","copyrightHolder":{"@id":"https:\/\/datascientest.com\/#organization"}},{"@type":"WebPage","@id":"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work","url":"https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work","name":"Preprocessing: What is it? How does it work?","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/#website"},"datePublished":"2023-11-26T12:09:14+00:00","dateModified":"2023-11-26T12:13:59+00:00","description":"Data preprocessing is a crucial step in the data analysis and machine learning pipeline. It involves cleaning, transforming, and organizing raw data into a format that is suitable for analysis or model training.","inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/datascientest.com\/en\/preprocessing-what-is-it-how-does-it-work"]}]},{"@type":"WebSite","@id":"https:\/\/datascientest.com\/en\/#website","url":"https:\/\/datascientest.com\/en\/","name":"Data Science Courses | DataScientest","description":"Leader in Data Science Courses","publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/datascientest.com\/en\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/datascientest.com\/en\/#organization","name":"DataScientest","url":"https:\/\/datascientest.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","contentUrl":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","width":301,"height":301,"caption":"DataScientest"},"image":{"@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataScientestEN","https:\/\/twitter.com\/DataScientest__","https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/","https:\/\/www.tiktok.com\/@datascientest"]},{"@type":"Person","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d","name":"Melanie","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g","caption":"Melanie"},"url":"https:\/\/datascientest.com\/en\/author\/melaniekrause"}]}},"_links":{"self":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/173848"}],"collection":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/users\/67"}],"replies":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/comments?post=173848"}],"version-history":[{"count":0,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/173848\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media\/173873"}],"wp:attachment":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media?parent=173848"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/categories?post=173848"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/tags?post=173848"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}