{"id":180678,"date":"2024-03-07T18:49:19","date_gmt":"2024-03-07T17:49:19","guid":{"rendered":"https:\/\/datascientest.com\/en\/?p=180678"},"modified":"2024-02-28T20:43:14","modified_gmt":"2024-02-28T19:43:14","slug":"jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model","status":"publish","type":"post","link":"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model","title":{"rendered":"Jensen-Shannon divergence: Everything you need to know about this ML model"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"180678\" class=\"elementor elementor-180678\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-9efa4cd e-flex e-con-boxed e-con\" data-id=\"9efa4cd\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-e1e824a elementor-widget elementor-widget-heading\" data-id=\"e1e824a\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-heading-title{padding:0;margin:0;line-height:1}.elementor-widget-heading .elementor-heading-title[class*=elementor-size-]>a{color:inherit;font-size:inherit;line-height:inherit}.elementor-widget-heading .elementor-heading-title.elementor-size-small{font-size:15px}.elementor-widget-heading .elementor-heading-title.elementor-size-medium{font-size:19px}.elementor-widget-heading .elementor-heading-title.elementor-size-large{font-size:29px}.elementor-widget-heading .elementor-heading-title.elementor-size-xl{font-size:39px}.elementor-widget-heading .elementor-heading-title.elementor-size-xxl{font-size:59px}<\/style><h2 class=\"elementor-heading-title elementor-size-default\">The Jensen-Shannon divergence is used to measure the similarity between two probability distributions, particularly in the field of Machine Learning. Find out everything you need to know about this measure, from its history to its modern applications!<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-4808479 e-flex e-con-boxed e-con\" data-id=\"4808479\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-773b1a5 elementor-widget elementor-widget-text-editor\" data-id=\"773b1a5\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-text-editor.elementor-drop-cap-view-stacked .elementor-drop-cap{background-color:#69727d;color:#fff}.elementor-widget-text-editor.elementor-drop-cap-view-framed .elementor-drop-cap{color:#69727d;border:3px solid;background-color:transparent}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap{margin-top:8px}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap-letter{width:1em;height:1em}.elementor-widget-text-editor .elementor-drop-cap{float:left;text-align:center;line-height:1;font-size:50px}.elementor-widget-text-editor .elementor-drop-cap-letter{display:inline-block}<\/style>\t\t\t\t<p>During the 20th century, the<strong> Danish mathematician Johan Jensen and the American statistician Peter Shannon<\/strong> made major contributions to <a href=\"https:\/\/datascientest.com\/en\/understanding-kurtosis-calculating-outlier-frequency-in-statistical-distributions\">information theory and statistics.<\/a><\/p><p>Born in 1859, Johan Jensen devoted much of his career to the study of convex functions and inequalities. In 1906 he published an article entitled &#8220;On convex functions and inequalities between mean values&#8221;.<\/p><p>In it, he introduced the notion of convexity, and established several results on Jensen&#8217;s inequalities that bear his name. These are mathematical results describing the properties of convex functions.<\/p><p><strong>Peter Shannon<\/strong> was born in 1917 and for a long time studied measures of divergence between probability distributions. In particular, he worked on problems related to density estimation.<\/p><p>In the 1940s, several decades after <strong>Jensen&#8217;s work,<\/strong> the American developed methods for estimating the divergence between probability distributions.<\/p><p>It was based on the Kullback-Leibler divergence: a measure invented in the 1950s by Solomon Kulback and Richard Leibler, widely used to quantify the <strong>difference between two distributions.<\/strong><\/p><p>It measures the dissimilarity between two probability distributions based on the logarithms of the probability ratios.<\/p><p>Later, in the 1990s, researchers began to explore possible extensions and variations of the <a href=\"https:\/\/datascientest.com\/en\/navigating-data-drift-strategies-to-tackle-shifting-data-landscapes\">Kullback-Leibler divergence.<\/a><\/p><p>Their aim was to take better account of symmetry and dissimilarity between distributions. They drew on the pioneering work of Johan Jensen and Peter Shannon to create the <strong>Jensen-Shannon divergence.<\/strong><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-ef7a900 e-flex e-con-boxed e-con\" data-id=\"ef7a900\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-b378dda elementor-widget elementor-widget-heading\" data-id=\"b378dda\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">What is the Jensen-Shannon divergence?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-af4de02 elementor-widget elementor-widget-text-editor\" data-id=\"af4de02\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Jensen-Shannon divergence was first introduced in an article by Barry E. S. Lindgren in 1991, entitled <strong>&#8220;Some Properties of Jensen-Shannon Divergence and Mutual Information&#8221;.<\/strong><\/p><p>He developed this metric as a measure of symmetric divergence between two probability distributions. Its main difference from the Kullback-Leibler divergence on which it is based is its symmetry.<\/p><p>It takes the <strong>weighted average of two KL divergences.<\/strong> One is calculated from the first distribution and the other from the second.<\/p><p>The<strong> Jensen-Shannon divergence<\/strong> can therefore be defined as the weighted average of the Kullback-Leibler divergences between each distribution and an average distribution.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-96e1191 elementor-widget elementor-widget-image\" data-id=\"96e1191\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-image{text-align:center}.elementor-widget-image a{display:inline-block}.elementor-widget-image a img[src$=\".svg\"]{width:48px}.elementor-widget-image img{vertical-align:middle;display:inline-block}<\/style>\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/06\/divergence-Jensen-Shannon-1.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-78e69e8 elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"78e69e8\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/machine-learning-engineer-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Learn more about JS divergence<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-45b3c93 e-flex e-con-boxed e-con\" data-id=\"45b3c93\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-fa0a9bf elementor-widget elementor-widget-heading\" data-id=\"fa0a9bf\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">How do you calculate the Jensen-Shannon divergence?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-22ae258 elementor-widget elementor-widget-text-editor\" data-id=\"22ae258\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The first step in <strong>calculating the Jensen-Shannon divergence<\/strong> is to pre-process the data to obtain the probability distributions P and Q.<\/p><p>The probability distributions can then be estimated from the data. For example, it is possible to count the occurrences of each element in the sample.<\/p><p>When the distributions are available, the divergence can be calculated using the formula :<\/p><p><strong>JS(P || Q) = (KL(P || M) + KL(Q || M)) \/ 2<\/strong><\/p><p>where M = (P + Q) \/ 2 and || represents the concatenation operator<\/p><p>A higher value of JS divergence indicates greater dissimilarity between distributions, while a value closer to zero indicates greater similarity.<\/p><p>To illustrate with a concrete example, suppose we have two texts and we want to measure their similarity.<\/p><p>Each text can be represented by a distribution of words, where each word is an element of the alphabet.<\/p><p>By counting the occurrences of words in each text and normalising these occurrences by the frequency of the total words, we obtain the <a href=\"https:\/\/datascientest.com\/en\/understanding-kurtosis-calculating-outlier-frequency-in-statistical-distributions\">probability distributions P and Q.<\/a><\/p><p>We then use the Jensen-Shannon divergence formula to calculate a value that indicates how similar the two texts are!<\/p><p>There are several important properties of this measure. Firstly, it is always positive and reaches zero if and only if the distributions P and Q are identical.<\/p><p>In addition, it is upper bounded by log2(n), where n is the size of the alphabet of the distribution. It is statistically significant and can be used in hypothesis testing and confidence intervals.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-967a38a e-flex e-con-boxed e-con\" data-id=\"967a38a\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-8c72843 elementor-widget elementor-widget-heading\" data-id=\"8c72843\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Advantages and disadvantages<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4c7df8f elementor-widget elementor-widget-text-editor\" data-id=\"4c7df8f\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The strength of the <strong>Jensen-Shannon divergence<\/strong> is that it takes into account the overall structure of the distributions. It is therefore more resistant to local variations than other divergence measures.<\/p><p>It is relatively efficient to calculate, which also makes it applicable to large quantities of data. These are its main advantages.<\/p><p>On the other hand, it can be sensitive to sample size. Estimates of probability distributions can be unreliable when sample sizes are small, and this can affect the similarity measure.<\/p><p>It may also be less suitable when the distributions are very different. This is because it does not capture the fine details of local differences.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-429450e e-flex e-con-boxed e-con\" data-id=\"429450e\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-b07514e elementor-widget elementor-widget-heading\" data-id=\"b07514e\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Jensen-Shannon divergence and Machine Learning<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0ddbcf1 elementor-widget elementor-widget-text-editor\" data-id=\"0ddbcf1\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>JS divergence plays a crucial role in <a href=\"https:\/\/datascientest.com\/en\/boosting-business-with-3-essential-machine-learning-algorithms\">Machine Learning.<\/a> It measures the similarity between the probability distributions associated with different samples or clusters.<\/p><p>It can be used to group similar data together or to classify new <strong>samples by comparing them with reference distributions.<\/strong><\/p><p>In <a href=\"https:\/\/datascientest.com\/en\/natural-language-processing-definition-and-principles\">natural language processing,<\/a> it can be used to compare word distributions in different texts. This can be used to identify similar documents, detect duplicate content or find semantic relationships between texts.<\/p><p>It is also a tool for evaluating <a href=\"https:\/\/datascientest.com\/en\/large-language-models-llm-everything-you-need-to-know\">language models.<\/a> In particular, it can be used to assess the diversity and quality of the texts generated.<\/p><p>By comparing the probability distributions of the generated texts with those of the reference texts, it is possible to measure the extent to which the generations are similar to or different from the reference corpus.<\/p><p>In cases where the training and test data come from different distributions, the<strong> Jensen-Shannon divergence<\/strong> can be used to guide domain adaptation strategies.<\/p><p>This can help to adjust a model trained on a source distribution to better fit new data from a target distribution.<\/p><p>Finally, for sentiment analysis, JS divergence can be used to compare profiles between different documents or sample classes.<\/p><p>This allows similarities and differences in expression to be identified, for example for opinion detection or emotion classification.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-50fa61d elementor-widget elementor-widget-image\" data-id=\"50fa61d\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/06\/divergence-Jensen-Shannon-2.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\"><\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6625c27 elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"6625c27\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/machine-learning-engineer-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Mastering JS and ML divergence<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-3b0b09b e-flex e-con-boxed e-con\" data-id=\"3b0b09b\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-e400eb0 elementor-widget elementor-widget-heading\" data-id=\"e400eb0\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Jensen-Shannon divergence and Data Science<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-03ca59c elementor-widget elementor-widget-text-editor\" data-id=\"03ca59c\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>For <a href=\"https:\/\/datascientest.com\/en\/synergies-unveiled-the-dynamic-intersection-of-data-science-and-finance\">data science,<\/a> JS divergence is used to compare the similarity between distributions of variables or characteristics in a data set.<\/p><p>It can be used to measure the difference between observed data distributions and expected or reference distributions.<\/p><p>This allows variations and discrepancies between different distributions to be identified, which can be valuable for detecting anomalies or validating hypotheses.<\/p><p>For textual data analysis, this measure can be used to estimate the similarity between distributions of words, phrases or themes in documents.<\/p><p>This can help to group similar documents, extract common topics or detect significant differences between sets of documents.<\/p><p>For example, it can be used to classify documents based on their content or for sentiment analysis by comparing sentiment distributions between different texts.<\/p><p>When there is high dimensionality in the data, Jensen-Shannon divergence can be used to select the most discriminating features or to reduce the dimensionality of the data.<\/p><p>By calculating the divergence between the distributions of different characteristics, it is possible to identify those that contribute most to the differentiation between classes or groups of data.<\/p><p>Model evaluation: In the process of developing and evaluating predictive models, JS divergence can be used as a metric to compare the probability distributions of predictions and actual values.<\/p><p>This makes it possible to assess the quality of the model by measuring how closely the predictions match the actual observations. For example, it can be used to evaluate classification, regression or recommendation models.<\/p><p>Finally, it can be used to measure the similarity between observations or instances in a dataset.<\/p><p>By comparing the <strong>distributions of features between different instances<\/strong>, it is possible to determine the proximity or distance between them. This can be used in clustering tasks to group similar observations or to perform <a href=\"https:\/\/datascientest.com\/en\/database-what-is-it\">similarity searches in large databases.<\/a><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-9b37e56 e-flex e-con-boxed e-con\" data-id=\"9b37e56\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-ec0ee50 elementor-widget elementor-widget-heading\" data-id=\"ec0ee50\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Conclusion: Jensen-Shannon divergence, a key tool for data analysis and machine learning<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-331e0d5 elementor-widget elementor-widget-text-editor\" data-id=\"331e0d5\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Since its creation, the Jensen-Shannon divergence has been widely used in many fields, including computer science, statistics, natural language processing, bioinformatics and machine learning.<\/p><p>It is still an essential tool for measuring similarity between probability distributions, and has opened up new perspectives in data analysis and statistical modelling.<\/p><p>Researchers around the world use it to solve classification and clustering problems. It is a key element in the toolbox of scientists and practitioners in many fields, starting with Data Science.<\/p><p>To learn how to master all the techniques of analysis and Machine Learning, you can choose DataScientest.<\/p><p>Our courses will give you all the skills you need to become a data engineer, analyst, data scientist, data product manager or ML engineer.<\/p><p>You&#8217;ll learn about the Python language and its libraries, DataViz tools, Business Intelligence solutions and databases.<\/p><p>All our courses are entirely distance learning, lead to professional certification and are eligible for funding options. Discover DataScientest!<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7359575 elementor-widget__width-initial elementor-widget elementor-widget-video\" data-id=\"7359575\" data-element_type=\"widget\" data-settings=\"{&quot;youtube_url&quot;:&quot;https:\\\/\\\/youtu.be\\\/kNPe_pgbuHg&quot;,&quot;video_type&quot;:&quot;youtube&quot;,&quot;controls&quot;:&quot;yes&quot;}\" data-widget_type=\"video.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-video .elementor-widget-container{overflow:hidden;transform:translateZ(0)}.elementor-widget-video .elementor-wrapper{aspect-ratio:var(--video-aspect-ratio)}.elementor-widget-video .elementor-wrapper iframe,.elementor-widget-video .elementor-wrapper video{height:100%;width:100%;display:flex;border:none;background-color:#000}@supports not (aspect-ratio:1\/1){.elementor-widget-video .elementor-wrapper{position:relative;overflow:hidden;height:0;padding-bottom:calc(100% \/ var(--video-aspect-ratio))}.elementor-widget-video .elementor-wrapper iframe,.elementor-widget-video .elementor-wrapper video{position:absolute;top:0;right:0;bottom:0;left:0}}.elementor-widget-video .elementor-open-inline .elementor-custom-embed-image-overlay{position:absolute;top:0;right:0;bottom:0;left:0;background-size:cover;background-position:50%}.elementor-widget-video .elementor-custom-embed-image-overlay{cursor:pointer;text-align:center}.elementor-widget-video .elementor-custom-embed-image-overlay:hover .elementor-custom-embed-play i{opacity:1}.elementor-widget-video .elementor-custom-embed-image-overlay img{display:block;width:100%;aspect-ratio:var(--video-aspect-ratio);-o-object-fit:cover;object-fit:cover;-o-object-position:center center;object-position:center center}@supports not (aspect-ratio:1\/1){.elementor-widget-video .elementor-custom-embed-image-overlay{position:relative;overflow:hidden;height:0;padding-bottom:calc(100% \/ var(--video-aspect-ratio))}.elementor-widget-video .elementor-custom-embed-image-overlay img{position:absolute;top:0;right:0;bottom:0;left:0}}.elementor-widget-video .e-hosted-video .elementor-video{-o-object-fit:cover;object-fit:cover}.e-con-inner>.elementor-widget-video,.e-con>.elementor-widget-video{width:var(--container-widget-width);--flex-grow:var(--container-widget-flex-grow)}<\/style>\t\t<div class=\"elementor-wrapper elementor-open-inline\">\n\t\t\t<div class=\"elementor-video\"><\/div>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5e17d83 elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"5e17d83\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-scientist-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Start a Machine Learning course<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-4ec5d3d e-flex e-con-boxed e-con\" data-id=\"4ec5d3d\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-dff9c01 elementor-widget elementor-widget-html\" data-id=\"dff9c01\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<script type=\"application\/ld+json\">\n{\n  \"@context\": \"https:\/\/schema.org\",\n  \"@type\": \"FAQPage\",\n  \"mainEntity\": [{\n    \"@type\": \"Question\",\n    \"name\": \"What is the Jensen-Shannon divergence?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"Jensen-Shannon divergence was first introduced in an article by Barry E. .S. Lindgren in 1991, entitled \"Some Properties of Jensen-Shannon Divergence and Mutual Information\". He developed this metric as a measure of symmetric divergence between two probability distributions. Its main difference from the Kullback-Leibler divergence on which it is based is its symmetry\".\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"How do I calculate the Jensen-Shannon divergence?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"The first step in calculating the Jensen-Shannon divergence is to pre-process the data to obtain the probability distributions P and Q. The probability distributions can then be estimated from the data. For example, it is possible to count the occurrences of each element in the sample.\"\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"Advantages and disadvantages\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"The strong point of the Jensen-Shannon divergence is that it takes into account the global structure of the distributions. It is therefore more resistant to local variations than other divergence measures. It is relatively efficient to calculate, which also makes it applicable to large quantities of data. These are its main advantages.\"\n    }\n  }]\n}\n<\/script>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>The Jensen-Shannon divergence is used to measure the similarity between two probability distributions, particularly in the field of Machine Learning. Find out everything you need to know about this measure, from its history to its modern applications! During the 20th century, the Danish mathematician Johan Jensen and the American statistician Peter Shannon made major contributions [&hellip;]<\/p>\n","protected":false},"author":67,"featured_media":180679,"comment_status":"open","ping_status":"open","sticky":false,"template":"elementor_theme","format":"standard","meta":{"content-type":"","inline_featured_image":false},"categories":[2409],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.7 (Yoast SEO v21.7) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Jensen-Shannon divergence: Everything you need to know about this ML model<\/title>\n<meta name=\"description\" content=\"Discover everything you need to know about Jensen-Shannon Divergence, a vital concept in machine learning. Learn its principles, applications, and how it measures the similarity between probability distributions, guiding your understanding of complex data analysis.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Jensen-Shannon divergence: Everything you need to know about this ML model\" \/>\n<meta property=\"og:description\" content=\"The Jensen-Shannon divergence is used to measure the similarity between two probability distributions, particularly in the field of Machine Learning. Find\" \/>\n<meta property=\"og:url\" content=\"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model\" \/>\n<meta property=\"og:site_name\" content=\"Data Science Courses | DataScientest\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataScientestEN\" \/>\n<meta property=\"article:published_time\" content=\"2024-03-07T17:49:19+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-02-28T19:43:14+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/datascientest.com\/en\/files\/2024\/02\/Jensen-Shannon-datascientest-1024x512-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"512\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Melanie\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:site\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Melanie\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model#article\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model\"},\"author\":{\"name\":\"Melanie\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d\"},\"headline\":\"Jensen-Shannon divergence: Everything you need to know about this ML model\",\"datePublished\":\"2024-03-07T17:49:19+00:00\",\"dateModified\":\"2024-02-28T19:43:14+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model\"},\"wordCount\":1434,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"articleSection\":[\"Machine Learning\"],\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model#respond\"]}],\"copyrightYear\":\"2024\",\"copyrightHolder\":{\"@id\":\"https:\/\/datascientest.com\/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model\",\"url\":\"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model\",\"name\":\"Jensen-Shannon divergence: Everything you need to know about this ML model\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/#website\"},\"datePublished\":\"2024-03-07T17:49:19+00:00\",\"dateModified\":\"2024-02-28T19:43:14+00:00\",\"description\":\"Discover everything you need to know about Jensen-Shannon Divergence, a vital concept in machine learning. Learn its principles, applications, and how it measures the similarity between probability distributions, guiding your understanding of complex data analysis.\",\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model\"]}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/datascientest.com\/en\/#website\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"name\":\"Data Science Courses | DataScientest\",\"description\":\"Leader in Data Science Courses\",\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/datascientest.com\/en\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/datascientest.com\/en\/#organization\",\"name\":\"DataScientest\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"contentUrl\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"width\":301,\"height\":301,\"caption\":\"DataScientest\"},\"image\":{\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/DataScientestEN\",\"https:\/\/twitter.com\/DataScientest__\",\"https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/\",\"https:\/\/www.tiktok.com\/@datascientest\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d\",\"name\":\"Melanie\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g\",\"caption\":\"Melanie\"},\"url\":\"https:\/\/datascientest.com\/en\/author\/melaniekrause\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Jensen-Shannon divergence: Everything you need to know about this ML model","description":"Discover everything you need to know about Jensen-Shannon Divergence, a vital concept in machine learning. Learn its principles, applications, and how it measures the similarity between probability distributions, guiding your understanding of complex data analysis.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model","og_locale":"en_GB","og_type":"article","og_title":"Jensen-Shannon divergence: Everything you need to know about this ML model","og_description":"The Jensen-Shannon divergence is used to measure the similarity between two probability distributions, particularly in the field of Machine Learning. Find","og_url":"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model","og_site_name":"Data Science Courses | DataScientest","article_publisher":"https:\/\/www.facebook.com\/DataScientestEN","article_published_time":"2024-03-07T17:49:19+00:00","article_modified_time":"2024-02-28T19:43:14+00:00","og_image":[{"width":1024,"height":512,"url":"https:\/\/datascientest.com\/en\/files\/2024\/02\/Jensen-Shannon-datascientest-1024x512-1.png","type":"image\/png"}],"author":"Melanie","twitter_card":"summary_large_image","twitter_creator":"@DataScientest__","twitter_site":"@DataScientest__","twitter_misc":{"Written by":"Melanie","Estimated reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model#article","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model"},"author":{"name":"Melanie","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d"},"headline":"Jensen-Shannon divergence: Everything you need to know about this ML model","datePublished":"2024-03-07T17:49:19+00:00","dateModified":"2024-02-28T19:43:14+00:00","mainEntityOfPage":{"@id":"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model"},"wordCount":1434,"commentCount":0,"publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"articleSection":["Machine Learning"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model#respond"]}],"copyrightYear":"2024","copyrightHolder":{"@id":"https:\/\/datascientest.com\/#organization"}},{"@type":"WebPage","@id":"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model","url":"https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model","name":"Jensen-Shannon divergence: Everything you need to know about this ML model","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/#website"},"datePublished":"2024-03-07T17:49:19+00:00","dateModified":"2024-02-28T19:43:14+00:00","description":"Discover everything you need to know about Jensen-Shannon Divergence, a vital concept in machine learning. Learn its principles, applications, and how it measures the similarity between probability distributions, guiding your understanding of complex data analysis.","inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/datascientest.com\/en\/jensen-shannon-divergence-everything-you-need-to-know-about-this-ml-model"]}]},{"@type":"WebSite","@id":"https:\/\/datascientest.com\/en\/#website","url":"https:\/\/datascientest.com\/en\/","name":"Data Science Courses | DataScientest","description":"Leader in Data Science Courses","publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/datascientest.com\/en\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/datascientest.com\/en\/#organization","name":"DataScientest","url":"https:\/\/datascientest.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","contentUrl":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","width":301,"height":301,"caption":"DataScientest"},"image":{"@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataScientestEN","https:\/\/twitter.com\/DataScientest__","https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/","https:\/\/www.tiktok.com\/@datascientest"]},{"@type":"Person","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d","name":"Melanie","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g","caption":"Melanie"},"url":"https:\/\/datascientest.com\/en\/author\/melaniekrause"}]}},"_links":{"self":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/180678"}],"collection":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/users\/67"}],"replies":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/comments?post=180678"}],"version-history":[{"count":0,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/180678\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media\/180679"}],"wp:attachment":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media?parent=180678"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/categories?post=180678"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/tags?post=180678"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}