{"id":176949,"date":"2024-01-19T14:32:03","date_gmt":"2024-01-19T13:32:03","guid":{"rendered":"https:\/\/datascientest.com\/en\/?p=176949"},"modified":"2024-01-19T14:35:14","modified_gmt":"2024-01-19T13:35:14","slug":"dimension-reduction-how-does-it-work","status":"publish","type":"post","link":"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work","title":{"rendered":"Dimension reduction: How does it work?"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"176949\" class=\"elementor elementor-176949\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-8eb0f8c e-flex e-con-boxed e-con\" data-id=\"8eb0f8c\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-7c4eb77 elementor-widget elementor-widget-heading\" data-id=\"7c4eb77\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-heading-title{padding:0;margin:0;line-height:1}.elementor-widget-heading .elementor-heading-title[class*=elementor-size-]>a{color:inherit;font-size:inherit;line-height:inherit}.elementor-widget-heading .elementor-heading-title.elementor-size-small{font-size:15px}.elementor-widget-heading .elementor-heading-title.elementor-size-medium{font-size:19px}.elementor-widget-heading .elementor-heading-title.elementor-size-large{font-size:29px}.elementor-widget-heading .elementor-heading-title.elementor-size-xl{font-size:39px}.elementor-widget-heading .elementor-heading-title.elementor-size-xxl{font-size:59px}<\/style><h2 class=\"elementor-heading-title elementor-size-default\">Data analysis can be seen as an experimental science, where properties are demonstrated after being observed, and codes are heuristically established to interpret the results. In machine learning and data analysis, dimension reduction plays an essential role in simplifying complex data sets.<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-a551200 e-flex e-con-boxed e-con\" data-id=\"a551200\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-da55308 elementor-widget elementor-widget-text-editor\" data-id=\"da55308\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-text-editor.elementor-drop-cap-view-stacked .elementor-drop-cap{background-color:#69727d;color:#fff}.elementor-widget-text-editor.elementor-drop-cap-view-framed .elementor-drop-cap{color:#69727d;border:3px solid;background-color:transparent}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap{margin-top:8px}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap-letter{width:1em;height:1em}.elementor-widget-text-editor .elementor-drop-cap{float:left;text-align:center;line-height:1;font-size:50px}.elementor-widget-text-editor .elementor-drop-cap-letter{display:inline-block}<\/style>\t\t\t\t<p>The <strong>basic idea behind dimension reduction<\/strong> is to represent multi-dimensional data in a much lower-dimensional space, while retaining the important and relevant information.<\/p><p>Before delving into more mathematical detail, it&#8217;s important to understand some key terms.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-854c4d2 e-flex e-con-boxed e-con\" data-id=\"854c4d2\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-7ccd016 elementor-widget elementor-widget-heading\" data-id=\"7ccd016\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">The vocabulary of dimension reduction<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-fb60acb elementor-widget elementor-widget-text-editor\" data-id=\"fb60acb\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<ul><li><strong>Dimension space<\/strong>: number of characteristics or variables that describe an object in a dataset. For example, a dataset of characteristics about people might include variables such as age, income, address, gender and level of education. This defines a five-dimensional space.<\/li><li><strong>Projection<\/strong> involves transforming data from a high-dimensional space to a low-dimensional space using dimension reduction techniques. This transformation enables data to be represented more compactly, while retaining important information.<\/li><li><strong>Orthogonality<\/strong> is a mathematical property in which two vectors or two spaces are perpendicular to each other. In the context of dimension reduction, orthogonality is often used to describe variables or components that are decorrelated from one another.<\/li><\/ul>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0581e70 elementor-widget elementor-widget-text-editor\" data-id=\"0581e70\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>If the <a href=\"https:\/\/datascientest.com\/en\/mariadb-a-solution-for-data-management-and-analysis\">data is represented in a table<\/a>, dimensionality is reduced by reducing the number of columns. If our dataset is composed of more than 3 variables, the higher the dimension, the more difficult it is to visualize. For example, it&#8217;s not possible to visualize a dataset with 13 attributes, but it is possible with just 2 or 3. The aim is therefore to project these data onto the 2 or 3 most important axes in terms of information content, in order to keep the data representation as close as possible to that of <strong>high-dimensional data.<\/strong><\/p><p>A very simple example: suppose we have a <a href=\"https:\/\/datascientest.com\/en\/datasets-top-5-places-to-find-quality-datasets\">dataset containing information on students<\/a>, comprising the following 5 variables, and each student is represented by a row in the dataset, with each variable corresponding to a column.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c5f1a27 elementor-widget elementor-widget-html\" data-id=\"c5f1a27\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style type=\"text\/css\">\n.tg  {border-collapse:collapse;border-spacing:0;}\n.tg td{border-color:black;border-style:solid;border-width:1px;font-family:Arial, sans-serif;font-size:14px;\n  overflow:hidden;padding:10px 5px;word-break:normal;}\n.tg th{border-color:black;border-style:solid;border-width:1px;font-family:Arial, sans-serif;font-size:14px;\n  font-weight:normal;overflow:hidden;padding:10px 5px;word-break:normal;}\n.tg .tg-80go{font-family:\"Trebuchet MS\", Helvetica, sans-serif !important;text-align:left;vertical-align:top}\n.tg .tg-4lje{font-family:\"Trebuchet MS\", Helvetica, sans-serif !important;text-align:center;vertical-align:top}\n.tg .tg-7t5w{background-color:#6200c9;color:#efefef;font-family:\"Trebuchet MS\", Helvetica, sans-serif !important;font-size:20px;\n  font-weight:bold;text-align:center;vertical-align:top}\n.tg .tg-pm1a{background-color:#6200c9;color:#efefef;font-family:\"Trebuchet MS\", Helvetica, sans-serif !important;font-size:20px;\n  font-weight:bold;text-align:left;vertical-align:top}\n<\/style>\n<table class=\"tg\" style=\"undefined;table-layout: fixed; width: 600px\">\n<colgroup>\n<col style=\"width: 75px\">\n<col style=\"width: 50px\">\n<col style=\"width: 50px\">\n<col style=\"width: 50px\">\n<col style=\"width: 50px\">\n<col style=\"width: 75px\">\n<\/colgroup>\n<thead>\n  <tr>\n    <th class=\"tg-pm1a\">Student<\/th>\n    <th class=\"tg-7t5w\">Age<\/th>\n    <th class=\"tg-7t5w\">Height<\/th>\n    <th class=\"tg-7t5w\">Weight<\/th>\n    <th class=\"tg-7t5w\">Math<\/th>\n    <th class=\"tg-7t5w\">Science<\/th>\n  <\/tr>\n<\/thead>\n<tbody>\n  <tr>\n    <td class=\"tg-80go\">Student 1<\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">18<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">165<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">60<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">85<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">90<\/span><\/td>\n  <\/tr>\n  <tr>\n    <td class=\"tg-80go\">Student 2<\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">20<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">170<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">65<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">75<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">80<\/span><\/td>\n  <\/tr>\n  <tr>\n    <td class=\"tg-80go\">Student 3<\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">19<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">175<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">70<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">90<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">95<\/span><\/td>\n  <\/tr>\n  <tr>\n    <td class=\"tg-80go\">Student 4<\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">22<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">180<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">75<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">80<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">70<\/span><\/td>\n  <\/tr>\n<\/tbody>\n<\/table>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-fba8107 elementor-widget elementor-widget-text-editor\" data-id=\"fba8107\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>After applying PCA to reduce the dimension, we could obtain a new dataset with fewer variables, for example, by retaining only the first two principal components. This would give us the following dataset:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1b2f884 elementor-widget elementor-widget-html\" data-id=\"1b2f884\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style type=\"text\/css\">\n.tg  {border-collapse:collapse;border-spacing:0;}\n.tg td{border-color:black;border-style:solid;border-width:1px;font-family:Arial, sans-serif;font-size:14px;\n  overflow:hidden;padding:10px 5px;word-break:normal;}\n.tg th{border-color:black;border-style:solid;border-width:1px;font-family:Arial, sans-serif;font-size:14px;\n  font-weight:normal;overflow:hidden;padding:10px 5px;word-break:normal;}\n.tg .tg-80go{font-family:\"Trebuchet MS\", Helvetica, sans-serif !important;text-align:left;vertical-align:top}\n.tg .tg-4lje{font-family:\"Trebuchet MS\", Helvetica, sans-serif !important;text-align:center;vertical-align:top}\n.tg .tg-7t5w{background-color:#6200c9;color:#efefef;font-family:\"Trebuchet MS\", Helvetica, sans-serif !important;font-size:20px;\n  font-weight:bold;text-align:center;vertical-align:top}\n.tg .tg-pm1a{background-color:#6200c9;color:#efefef;font-family:\"Trebuchet MS\", Helvetica, sans-serif !important;font-size:20px;\n  font-weight:bold;text-align:left;vertical-align:top}\n<\/style>\n<table class=\"tg\" style=\"undefined;table-layout: fixed; width: 600px\">\n<colgroup>\n<col style=\"width: 75px\">\n<col style=\"width: 200px\">\n<col style=\"width: 200px\">\n<\/colgroup>\n<thead>\n  <tr>\n    <th class=\"tg-pm1a\">Student<\/th>\n    <th class=\"tg-7t5w\">Principal Component 1<\/th>\n    <th class=\"tg-7t5w\">Principal Component 2<\/th>\n  <\/tr>\n<\/thead>\n<tbody>\n  <tr>\n    <td class=\"tg-80go\">Student 1<\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">0.23<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">-0.12<\/span><\/td>\n  <\/tr>\n  <tr>\n    <td class=\"tg-80go\">Student 2<\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">-0.12<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">0.10<\/span><\/td>\n  <\/tr>\n  <tr>\n    <td class=\"tg-80go\">Student 3<\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">0.30<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">0.25<\/span><\/td>\n  <\/tr>\n  <tr>\n    <td class=\"tg-80go\">Student 4<\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">-0.41<\/span><\/td>\n    <td class=\"tg-4lje\"><span style=\"font-weight:400;font-style:normal;text-decoration:none;color:#000;background-color:transparent\">-0.23<\/span><\/td>\n  <\/tr>\n<\/tbody>\n<\/table>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7e94173 elementor-widget elementor-widget-text-editor\" data-id=\"7e94173\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>This data set can be easily represented on a 2-dimensional graph:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-55aa586 elementor-widget elementor-widget-image\" data-id=\"55aa586\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-image{text-align:center}.elementor-widget-image a{display:inline-block}.elementor-widget-image a img[src$=\".svg\"]{width:48px}.elementor-widget-image img{vertical-align:middle;display:inline-block}<\/style>\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/06\/image5-3.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\"><\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-460f6ed elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"460f6ed\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-courses\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Training in dimension reduction<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-cb9c760 e-flex e-con-boxed e-con\" data-id=\"cb9c760\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-1eb4a22 elementor-widget elementor-widget-heading\" data-id=\"1eb4a22\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Find out more about Principal Component Analysis (PCA)<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8120d94 elementor-widget elementor-widget-text-editor\" data-id=\"8120d94\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>One of the most commonly used methods is Principal Component Analysis (PCA). <strong>PCA<\/strong> seeks to find the principal directions in the data that explain most of the observed variance. To do this, we use the notion of covariance.<\/p><p>Let&#8217;s represent our data set, made up of n observations (samples) and p variables (characteristics), by a matrix X of size n x p. Before applying PCA, it is often recommended to standardize the variables so that they have zero mean and unit variance, so that variables with different scales don&#8217;t dominate the analysis.<\/p><p>We calculate the covariance matrix of X, which we denote C. This matrix measures the linear relationships between variables. It is calculated using the following formula:<strong> C = 1\/nXT X<\/strong><\/p><p><br \/>We are looking for the <strong>eigenvectors of the covariance matrix C.<\/strong> Eigenvectors are directions in the space of variables that describe the variance of the data. Eigenvectors are normalized so that their norm is equal to 1.<\/p><p>We order these eigenvectors in descending order of their associated eigenvalue: \u03bd1 associated with \u03bb1, \u03bd2 associated with \u03bb2 and \u03bb1\u2265\u03bb2 and so on. The higher the eigenvalue, the greater the variance explained by the corresponding direction. <\/p><p>We choose a number k of principal components (often 1, 2 or 3) to keep and select the first k eigenvectors (\u03bd1, \u03bd2, &#8230; , \u03bdk). These eigenvectors form an orthonormal basis in the new reduced-dimensional space. Thus, the new variables created are decorrelated. We denote W the matrix formed by these first k eigenvectors.<\/p><p>We now want to project our data into this new space, a basis of which is composed of the vectors (\u03bd1, \u03bd2, &#8230; , \u03bdk). To project the original data into this reduced-dimensional space, we use the following linear transformation: Y = XW, where Y corresponds to the coordinates of our data following projection.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-deb2969 elementor-widget elementor-widget-text-editor\" data-id=\"deb2969\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>There are other methods of dimension reduction, the best-known of which are :<\/p><ul><li><strong>LDA (linear discriminant analysis):<\/strong> linear discriminant analysis is used to identify directions that are decorrelated from one another. It aims to find a linear projection of the data that maximizes the separation between classes while minimizing the intra-class variance.<\/li><li><strong>T-SNE (t-Distributed Stochastic Neighbor Embedding)<\/strong>: T-SNE is a nonlinear dimension reduction method for representing high-dimensional data in a reduced-dimensional space (typically 2D or 3D). For more details, read this article. T-SNE is particularly well suited to visualizing complex structures and<a href=\"https:\/\/datascientest.com\/en\/datasets-top-5-places-to-find-quality-datasets\"> non-linear relationships in data.<\/a><\/li><\/ul>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-b5846fd e-flex e-con-boxed e-con\" data-id=\"b5846fd\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-f03f1c2 elementor-widget elementor-widget-heading\" data-id=\"f03f1c2\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Limits and difficulties of dimension reduction<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-257994b elementor-widget elementor-widget-text-editor\" data-id=\"257994b\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>It&#8217;s important to note that dimension reduction is a compromise between data simplification and information loss. By <strong>reducing the dimension<\/strong>, it is possible to lose some subtle information or specific details. Consequently, it&#8217;s crucial to strike the right balance by selecting the appropriate number of principal components that capture most of the relevant information.<\/p><p>Each method has its advantages and limitations, and the choice of method depends on the specific context and <a href=\"https:\/\/datascientest.com\/en\/analysis-of-variance-anova-a-basic-tool-for-data-analysis\">objectives of the data analysis.<\/a><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-fd5e065 elementor-widget elementor-widget-text-editor\" data-id=\"fd5e065\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>If you liked this article, you can find out more about these <a href=\"https:\/\/datascientest.com\/en\/data-analysis-course-face-to-face-or-online-whats-the-best-choice\">methods in our Data Analyst and Data Scientist training courses.<\/a>\u00a0<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a6e242f elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"a6e242f\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-courses\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Discover our Data Science training courses<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-b79f19e e-flex e-con-boxed e-con\" data-id=\"b79f19e\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-013412a elementor-widget elementor-widget-html\" data-id=\"013412a\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<script type=\"application\/ld+json\">\n{\n  \"@context\": \"https:\/\/schema.org\",\n  \"@type\": \"FAQPage\",\n  \"mainEntity\": [{\n    \"@type\": \"Question\",\n    \"name\": \"The vocabulary of dimension reduction\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"Dimension space: number of characteristics or variables that describe an object in a dataset. For example, a dataset collecting characteristics about people might include variables such as age, income, address, gender, and education level. This defines a five-dimensional space. Projection involves transforming data from a high-dimensional space to a low-dimensional space using dimension reduction techniques. This transformation enables data to be represented more compactly, while retaining important information.\"\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"Learn more about Principal Component Analysis (PCA)\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"One of the most commonly used methods is Principal Component Analysis (PCA). PCA seeks to find the principal directions in the data that explain most of the observed variance. To do this, we use the notion of covariance. Let's represent our data set, made up of n observations (samples) and p variables (characteristics), by a matrix X of size n x p. Before applying PCA, it is often recommended to standardize the variables so that they have zero mean and unit variance, so that variables with different scales don't dominate the analysis.\"\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"Limits and difficulties of dimension reduction\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"It's important to note that dimension reduction is a compromise between data simplification and information loss. By reducing the dimension, it is possible to lose some subtle information or specific details. Therefore, it is crucial to find the right balance by selecting the appropriate number of principal components capturing most of the relevant information.\"\n    }\n  }]\n}\n<\/script>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Data analysis can be seen as an experimental science, where properties are demonstrated after being observed, and codes are heuristically established to interpret the results. In machine learning and data analysis, dimension reduction plays an essential role in simplifying complex data sets. The basic idea behind dimension reduction is to represent multi-dimensional data in a [&hellip;]<\/p>\n","protected":false},"author":67,"featured_media":176955,"comment_status":"open","ping_status":"open","sticky":false,"template":"elementor_theme","format":"standard","meta":{"content-type":"","inline_featured_image":false},"categories":[2425],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.7 (Yoast SEO v21.7) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Dimension reduction: How does it work?<\/title>\n<meta name=\"description\" content=\"Dimension reduction is a valuable tool in data preprocessing and analysis, offering a way to manage high-dimensional data efficiently. It plays a crucial role in feature engineering, model training, and visualization, contributing to the overall success of machine learning projects.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Dimension reduction: How does it work?\" \/>\n<meta property=\"og:description\" content=\"Data analysis can be seen as an experimental science, where properties are demonstrated after being observed, and codes are heuristically established to\" \/>\n<meta property=\"og:url\" content=\"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work\" \/>\n<meta property=\"og:site_name\" content=\"Data Science Courses | DataScientest\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataScientestEN\" \/>\n<meta property=\"article:published_time\" content=\"2024-01-19T13:32:03+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-01-19T13:35:14+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/datascientest.com\/en\/files\/2024\/01\/Illu_BLOG__Reduction_dimension-768x422-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"768\" \/>\n\t<meta property=\"og:image:height\" content=\"422\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Melanie\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:site\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Melanie\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work#article\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work\"},\"author\":{\"name\":\"Melanie\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d\"},\"headline\":\"Dimension reduction: How does it work?\",\"datePublished\":\"2024-01-19T13:32:03+00:00\",\"dateModified\":\"2024-01-19T13:35:14+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work\"},\"wordCount\":892,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"articleSection\":[\"Data Science\"],\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work#respond\"]}],\"copyrightYear\":\"2024\",\"copyrightHolder\":{\"@id\":\"https:\/\/datascientest.com\/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work\",\"url\":\"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work\",\"name\":\"Dimension reduction: How does it work?\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/#website\"},\"datePublished\":\"2024-01-19T13:32:03+00:00\",\"dateModified\":\"2024-01-19T13:35:14+00:00\",\"description\":\"Dimension reduction is a valuable tool in data preprocessing and analysis, offering a way to manage high-dimensional data efficiently. It plays a crucial role in feature engineering, model training, and visualization, contributing to the overall success of machine learning projects.\",\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work\"]}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/datascientest.com\/en\/#website\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"name\":\"Data Science Courses | DataScientest\",\"description\":\"Leader in Data Science Courses\",\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/datascientest.com\/en\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/datascientest.com\/en\/#organization\",\"name\":\"DataScientest\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"contentUrl\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"width\":301,\"height\":301,\"caption\":\"DataScientest\"},\"image\":{\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/DataScientestEN\",\"https:\/\/twitter.com\/DataScientest__\",\"https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/\",\"https:\/\/www.tiktok.com\/@datascientest\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d\",\"name\":\"Melanie\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g\",\"caption\":\"Melanie\"},\"url\":\"https:\/\/datascientest.com\/en\/author\/melaniekrause\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Dimension reduction: How does it work?","description":"Dimension reduction is a valuable tool in data preprocessing and analysis, offering a way to manage high-dimensional data efficiently. It plays a crucial role in feature engineering, model training, and visualization, contributing to the overall success of machine learning projects.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work","og_locale":"en_GB","og_type":"article","og_title":"Dimension reduction: How does it work?","og_description":"Data analysis can be seen as an experimental science, where properties are demonstrated after being observed, and codes are heuristically established to","og_url":"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work","og_site_name":"Data Science Courses | DataScientest","article_publisher":"https:\/\/www.facebook.com\/DataScientestEN","article_published_time":"2024-01-19T13:32:03+00:00","article_modified_time":"2024-01-19T13:35:14+00:00","og_image":[{"width":768,"height":422,"url":"https:\/\/datascientest.com\/en\/files\/2024\/01\/Illu_BLOG__Reduction_dimension-768x422-1.png","type":"image\/png"}],"author":"Melanie","twitter_card":"summary_large_image","twitter_creator":"@DataScientest__","twitter_site":"@DataScientest__","twitter_misc":{"Written by":"Melanie","Estimated reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work#article","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work"},"author":{"name":"Melanie","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d"},"headline":"Dimension reduction: How does it work?","datePublished":"2024-01-19T13:32:03+00:00","dateModified":"2024-01-19T13:35:14+00:00","mainEntityOfPage":{"@id":"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work"},"wordCount":892,"commentCount":0,"publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"articleSection":["Data Science"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work#respond"]}],"copyrightYear":"2024","copyrightHolder":{"@id":"https:\/\/datascientest.com\/#organization"}},{"@type":"WebPage","@id":"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work","url":"https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work","name":"Dimension reduction: How does it work?","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/#website"},"datePublished":"2024-01-19T13:32:03+00:00","dateModified":"2024-01-19T13:35:14+00:00","description":"Dimension reduction is a valuable tool in data preprocessing and analysis, offering a way to manage high-dimensional data efficiently. It plays a crucial role in feature engineering, model training, and visualization, contributing to the overall success of machine learning projects.","inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/datascientest.com\/en\/dimension-reduction-how-does-it-work"]}]},{"@type":"WebSite","@id":"https:\/\/datascientest.com\/en\/#website","url":"https:\/\/datascientest.com\/en\/","name":"Data Science Courses | DataScientest","description":"Leader in Data Science Courses","publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/datascientest.com\/en\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/datascientest.com\/en\/#organization","name":"DataScientest","url":"https:\/\/datascientest.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","contentUrl":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","width":301,"height":301,"caption":"DataScientest"},"image":{"@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataScientestEN","https:\/\/twitter.com\/DataScientest__","https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/","https:\/\/www.tiktok.com\/@datascientest"]},{"@type":"Person","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d","name":"Melanie","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g","caption":"Melanie"},"url":"https:\/\/datascientest.com\/en\/author\/melaniekrause"}]}},"_links":{"self":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/176949"}],"collection":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/users\/67"}],"replies":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/comments?post=176949"}],"version-history":[{"count":0,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/176949\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media\/176955"}],"wp:attachment":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media?parent=176949"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/categories?post=176949"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/tags?post=176949"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}