{"id":176681,"date":"2024-01-24T22:14:22","date_gmt":"2024-01-24T21:14:22","guid":{"rendered":"https:\/\/datascientest.com\/en\/?p=176681"},"modified":"2024-01-16T22:43:08","modified_gmt":"2024-01-16T21:43:08","slug":"catastrophic-interference-what-is-it-how-to-avoid-it","status":"publish","type":"post","link":"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it","title":{"rendered":"Catastrophic Interference: What is it? How to avoid it?"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"176681\" class=\"elementor elementor-176681\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-0a74cea e-flex e-con-boxed e-con\" data-id=\"0a74cea\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-0dd641b elementor-widget elementor-widget-heading\" data-id=\"0dd641b\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-heading-title{padding:0;margin:0;line-height:1}.elementor-widget-heading .elementor-heading-title[class*=elementor-size-]>a{color:inherit;font-size:inherit;line-height:inherit}.elementor-widget-heading .elementor-heading-title.elementor-size-small{font-size:15px}.elementor-widget-heading .elementor-heading-title.elementor-size-medium{font-size:19px}.elementor-widget-heading .elementor-heading-title.elementor-size-large{font-size:29px}.elementor-widget-heading .elementor-heading-title.elementor-size-xl{font-size:39px}.elementor-widget-heading .elementor-heading-title.elementor-size-xxl{font-size:59px}<\/style><h2 class=\"elementor-heading-title elementor-size-default\">Catastrophic interference is a phenomenon in which AI forgets what it has learned. Find out why it's a serious problem in Machine Learning, and how to remedy it!<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-fae64da e-flex e-con-boxed e-con\" data-id=\"fae64da\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-00f4d0e elementor-widget elementor-widget-text-editor\" data-id=\"00f4d0e\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-text-editor.elementor-drop-cap-view-stacked .elementor-drop-cap{background-color:#69727d;color:#fff}.elementor-widget-text-editor.elementor-drop-cap-view-framed .elementor-drop-cap{color:#69727d;border:3px solid;background-color:transparent}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap{margin-top:8px}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap-letter{width:1em;height:1em}.elementor-widget-text-editor .elementor-drop-cap{float:left;text-align:center;line-height:1;font-size:50px}.elementor-widget-text-editor .elementor-drop-cap-letter{display:inline-block}<\/style>\t\t\t\t<p>With the rise of tools like <strong>ChatGPT<\/strong>, artificial intelligence is now ubiquitous.<\/p><p>This technology is destined to occupy an<strong> increasingly important place in our personal and professional lives.<\/strong><\/p><p>However, <a href=\"https:\/\/datascientest.com\/en\/xai-or-explainable-artificial-intelligence-what-is-it-about\">AI is not infallible.<\/a> Like humans, it can make mistakes and even have memory lapses.<\/p><p>Unfortunately, forgetting can spell disaster for a neural network. In the <a href=\"https:\/\/datascientest.com\/en\/unlock-your-future-dive-into-machine-learning-engineer-training\">field of Machine Learning<\/a>, this phenomenon is known as &#8220;catastrophic interference&#8221;.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-4ec06c9 e-flex e-con-boxed e-con\" data-id=\"4ec06c9\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-d587a1c elementor-widget elementor-widget-heading\" data-id=\"d587a1c\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">What is a neural network?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1b26f17 elementor-widget elementor-widget-text-editor\" data-id=\"1b26f17\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><a href=\"https:\/\/datascientest.com\/en\/recurrent-neural-network-what-is-it\">Neural networks<\/a> are a popular form of Machine Learning, widely used for prediction.<\/p><p>As the name suggests, this type of system is inspired by the way the <strong>human brain learns new information.<\/strong><\/p><p>In the way that neurons are interconnected, multiple mathematical equations are linked together.<br \/>And just as the brain reacts when the senses perceive a phenomenon, the <a href=\"https:\/\/datascientest.com\/en\/convolutional-neural-network-everything-you-need-to-know\">artificial neural network is activated when it receives data.<\/a><\/p><p>Certain pathways are activated and others inhibited depending on the nature of the information received. At the end of the process, as a result, a node produces new information, such as a prediction.<\/p><p>For example, when you see a dog, your brain immediately identifies it. A neural network can also learn to recognize a dog.<\/p><p>However, to do so, it must first be trained to distinguish a dog from a cat. It therefore needs to be fed with data.<\/p><p>During this<strong> training phase<\/strong>, the neural network is fed data sets. To continue with the same example, this could be a series of images with a caption indicating whether they show a dog or a cat.<\/p><p>Subsequently, another <strong>dataset<\/strong> is used to test the network to see if the training has been successful. This is the prediction phase.<\/p><p>If the neural network achieves a sufficient rate of accurate predictions, it is ready for deployment. However, the test is not always conclusive&#8230;<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e5397e5 elementor-widget elementor-widget-image\" data-id=\"e5397e5\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-image{text-align:center}.elementor-widget-image a{display:inline-block}.elementor-widget-image a img[src$=\".svg\"]{width:48px}.elementor-widget-image img{vertical-align:middle;display:inline-block}<\/style>\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/05\/reseaux-de-neurones.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\"><\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e51bc83 elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"e51bc83\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-engineer-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Mastering Machine Learning<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-1e6df21 e-flex e-con-boxed e-con\" data-id=\"1e6df21\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-b891fb9 elementor-widget elementor-widget-heading\" data-id=\"b891fb9\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">What is catastrophic interference?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-adeae82 elementor-widget elementor-widget-text-editor\" data-id=\"adeae82\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>One important feature distinguishes the human brain from artificial neural networks:<strong> plasticity.<\/strong><\/p><p>Plasticity is the human capacity for continuous learning. Thanks to this, after learning to distinguish dogs from cats, we also <strong>learn to recognize other animals, plants, trees and the whole world around us.<\/strong><\/p><p><a href=\"https:\/\/datascientest.com\/en\/u-net-computer-visions-neural-network\">Neural Networks<\/a>, on the other hand, are more limited. When learning a new task, they tend to forget what they&#8217;ve learned before.<\/p><p>For example, in a famous experiment conducted by McCloskey and Cohen in 1989, the researchers trained a neural network to <strong>solve mathematical problems based on examples containing the number 1.<\/strong><\/p><p>They then fed the model another series of problems, this time containing the number 2. As a result, the neural network learned to solve problems containing a 2, but forgot how to solve those with the number 1.<\/p><p>How can we explain this phenomenon? The neural network dynamically created the paths between nodes during the training phase, based on the data supplied to it.<\/p><p>However,<strong> by feeding it new information<\/strong>, new paths were formed. This is what sometimes causes an algorithm to &#8220;forget&#8221; the previous tasks for which it has been trained.<\/p><p>The extent of this amnesia can vary. It may be as simple as an increase in the margin of error, but it can go as far as completely forgetting a previously learned task.<\/p><p>Note that catastrophic interference can occur even when the data sets successively fed to the neural network are not so different.<\/p><p>The various layers between the input and output of a neural network are hidden and operate as a black box. It is therefore impossible to know which data may break a path before it happens.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-046c456 e-flex e-con-boxed e-con\" data-id=\"046c456\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-b2d0f94 elementor-widget elementor-widget-heading\" data-id=\"b2d0f94\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Why is this a big problem?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-673f8d4 elementor-widget elementor-widget-text-editor\" data-id=\"673f8d4\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Most of today&#8217;s<a href=\"https:\/\/datascientest.com\/en\/deep-neural-network-what-is-it-and-how-is-it-working\"> neural networks<\/a> are trained using <a href=\"https:\/\/datascientest.com\/en\/boosting-business-with-3-essential-machine-learning-algorithms\">supervised learning<\/a>. Engineers manually select and clean the data with which they feed the network, to avoid biases and other concerns that may arise from datasets.<\/p><p>With this type of neural network, <strong>catastrophic interference is not really a problem<\/strong>. However, as Machine Learning develops, agents are approaching autonomous, continuous learning.<\/p><p>Such neural networks are able to continue learning from new data, without even needing to be supervised by humans.<\/p><p>This development offers <strong>tremendous opportunities<\/strong>, but also introduces new risks. It is no longer really possible to know what type of data the network is using to learn.<\/p><p>And if it decides to use data far removed from its initial training, this can lead to catastrophic oversight.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-382d65d elementor-widget elementor-widget-image\" data-id=\"382d65d\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/05\/catastrophique-interference-2.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\"><\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8fef4ba elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"8fef4ba\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/machine-learning-engineer-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Understanding catastrophic interference<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-d43d9c6 e-flex e-con-boxed e-con\" data-id=\"d43d9c6\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-e32245b elementor-widget elementor-widget-heading\" data-id=\"e32245b\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">How to avoid catastrophic interference?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-02b5fe3 elementor-widget elementor-widget-text-editor\" data-id=\"02b5fe3\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>There are many ways of<strong> minimizing the risk of catastrophic interference.<\/strong><\/p><p>One approach is to add regularization terms such as L1 or L2 to control the complexity of the model and reduce its sensitivity to minor changes in the input data.<\/p><p>Dropout involves randomly deleting a certain number of neurons during training. This can prevent the model from relying too heavily on specific neurons.<\/p><p>In addition, <strong>training data augmentation t<\/strong>echniques such as rotation, translation or inversion help the model to generalize better on unknown data.<\/p><p>Transfer learning, based on models pre-trained on similar tasks, can also help initialize the model with weights that have already learned important features to speed up learning and reduce catastrophic interference.<\/p><p>Similarly, progressive training involves training the model on progressively larger subsets of data. The model can then focus on simpler features before moving on to complex tasks.<\/p><p>Another<strong> common trick is to train a new neural network with all the data simultaneously<\/strong>. This avoids the sequential learning that can lead to the overwriting of previously acquired knowledge.<\/p><p>Some architectures are more resistant to forgetting, such as residual networks or evolutionary neural networks.<\/p><p>A <strong>judicious strategy<\/strong> is to create a backup of a neural network before re-training it. In the event of a problem, it is then possible to restore the previous version.<\/p><p>In a study carried out at the end of 2022, researchers discovered that <strong>catastrophic interference<\/strong> can be avoided by letting neural networks rest. This confirms the resemblance with our brain, which needs sleep to memorize better!<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-91c2499 e-flex e-con-boxed e-con\" data-id=\"91c2499\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-e34a555 elementor-widget elementor-widget-heading\" data-id=\"e34a555\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Conclusion: Catastrophic interference, one of the many challenges of Machine Learning<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5d45af9 elementor-widget elementor-widget-text-editor\" data-id=\"5d45af9\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><strong>Catastrophic interference<\/strong> is just one of the many <a href=\"https:\/\/datascientest.com\/en\/tpot-all-about-this-machine-learning-python-library\">challenges facing Machine Learning<\/a>. It will be necessary to overcome these obstacles in order to reveal the full potential of artificial intelligence.<\/p><p><a href=\"https:\/\/datascientest.com\/en\/mlops-devops-applied-to-machine-learning-projects\">To gain expertise as a Machine Learning engineer,<\/a> you can choose DataScientest training.<br \/>Our ML Engineer course enables you to learn how to develop AI systems and use large data sets to research, develop and generate algorithms capable of learning and predicting.<\/p><p>You&#8217;ll master the entire Machine Learning process, from algorithm design to deployment and production. This will enable you to deal with problems such as catastrophic interference.<\/p><p>As you progress through the various modules, you&#8217;ll acquire solid skills in Python programming, DataViz, Machine Learning, data engineering, DataOps and MLOps, as well as Business Intelligence.<\/p><p>At the end of the course, you&#8217;ll receive a diploma from Mines ParisTech PSL Executive Education, a level 7 RNCP36129 &#8220;Project Manager in Artificial Intelligence&#8221; certification (Master level)\u00a0 issued by the Coll\u00e8ge de Paris and an AWS Amazon Certified Cloud Practitioner certification!<\/p><p>This state-recognized training program is eligible for funding options and can be taken entirely by distance learning in an intensive 7-month BootCamp, or part-time over 16 months. Discover DataScientest!<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7748c9b elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"7748c9b\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/machine-learning-engineer-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Start a Machine Learning Engineer training course<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-c1309fd e-flex e-con-boxed e-con\" data-id=\"c1309fd\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-0cc6f0f elementor-widget elementor-widget-html\" data-id=\"0cc6f0f\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<script type=\"application\/ld+json\">\n{\n  \"@context\": \"https:\/\/schema.org\",\n  \"@type\": \"FAQPage\",\n  \"mainEntity\": [{\n    \"@type\": \"Question\",\n    \"name\": \"What is a neural network?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"Neural networks are a form of Machine Learning, very popular and massively used for predictions. As the name suggests, this type of system is inspired by the way the human brain learns new information. In the way that neurons are interconnected, multiple mathematical equations are linked together. And just as the brain reacts when the senses perceive a phenomenon, the artificial neural network is activated when it receives data.\"\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"What is catastrophic interference?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"An important characteristic distinguishes the human brain from artificial neural networks: plasticity. This is the ability of humans to learn continuously. Thanks to plasticity, after learning to distinguish dogs from cats, we also learn to recognize other animals, plants, trees and the whole world around us. Neural Networks, on the other hand, are more limited. When learning a new task, they tend to forget what they've assimilated before.\"\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"Why is this a big problem?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"Most neural networks today are trained with supervised learning. Engineers manually select and clean up the data with which they feed the network, to avoid biases and other concerns that may arise from datasets.\"\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"How to avoid catastrophic interference?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"There are many ways to minimize the risk of catastrophic interference. One approach is to add regularization terms such as L1 or L2 to control the complexity of the model and reduce its sensitivity to minor changes in the input data. Dropout involves randomly deleting a certain number of neurons during training. This can prevent the model from relying too heavily on specific neurons.\"\n    }\n  }]\n}\n<\/script>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Catastrophic interference is a phenomenon in which AI forgets what it has learned. Find out why it&#8217;s a serious problem in Machine Learning, and how to remedy it! With the rise of tools like ChatGPT, artificial intelligence is now ubiquitous. This technology is destined to occupy an increasingly important place in our personal and professional [&hellip;]<\/p>\n","protected":false},"author":67,"featured_media":176688,"comment_status":"open","ping_status":"open","sticky":false,"template":"elementor_theme","format":"standard","meta":{"content-type":"","inline_featured_image":false},"categories":[2420],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.7 (Yoast SEO v21.7) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Catastrophic Interference: What is it? How to avoid it?<\/title>\n<meta name=\"description\" content=\"Catastrophic interference refers to a phenomenon in machine learning and artificial neural networks where learning new information significantly disrupts previously learned information, leading to a loss of\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Catastrophic Interference: What is it? How to avoid it?\" \/>\n<meta property=\"og:description\" content=\"Catastrophic interference is a phenomenon in which AI forgets what it has learned. Find out why it&#039;s a serious problem in Machine Learning, and how to\" \/>\n<meta property=\"og:url\" content=\"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it\" \/>\n<meta property=\"og:site_name\" content=\"Data Science Courses | DataScientest\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataScientestEN\" \/>\n<meta property=\"article:published_time\" content=\"2024-01-24T21:14:22+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-01-16T21:43:08+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/datascientest.com\/en\/files\/2024\/01\/catastrophique-interference.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"768\" \/>\n\t<meta property=\"og:image:height\" content=\"384\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"author\" content=\"Melanie\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:site\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Melanie\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it#article\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it\"},\"author\":{\"name\":\"Melanie\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d\"},\"headline\":\"Catastrophic Interference: What is it? How to avoid it?\",\"datePublished\":\"2024-01-24T21:14:22+00:00\",\"dateModified\":\"2024-01-16T21:43:08+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it\"},\"wordCount\":1219,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"articleSection\":[\"Business Intelligence\"],\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it#respond\"]}],\"copyrightYear\":\"2024\",\"copyrightHolder\":{\"@id\":\"https:\/\/datascientest.com\/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it\",\"url\":\"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it\",\"name\":\"Catastrophic Interference: What is it? How to avoid it?\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/#website\"},\"datePublished\":\"2024-01-24T21:14:22+00:00\",\"dateModified\":\"2024-01-16T21:43:08+00:00\",\"description\":\"Catastrophic interference refers to a phenomenon in machine learning and artificial neural networks where learning new information significantly disrupts previously learned information, leading to a loss of\",\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it\"]}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/datascientest.com\/en\/#website\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"name\":\"Data Science Courses | DataScientest\",\"description\":\"Leader in Data Science Courses\",\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/datascientest.com\/en\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/datascientest.com\/en\/#organization\",\"name\":\"DataScientest\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"contentUrl\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"width\":301,\"height\":301,\"caption\":\"DataScientest\"},\"image\":{\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/DataScientestEN\",\"https:\/\/twitter.com\/DataScientest__\",\"https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/\",\"https:\/\/www.tiktok.com\/@datascientest\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d\",\"name\":\"Melanie\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g\",\"caption\":\"Melanie\"},\"url\":\"https:\/\/datascientest.com\/en\/author\/melaniekrause\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Catastrophic Interference: What is it? How to avoid it?","description":"Catastrophic interference refers to a phenomenon in machine learning and artificial neural networks where learning new information significantly disrupts previously learned information, leading to a loss of","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it","og_locale":"en_GB","og_type":"article","og_title":"Catastrophic Interference: What is it? How to avoid it?","og_description":"Catastrophic interference is a phenomenon in which AI forgets what it has learned. Find out why it's a serious problem in Machine Learning, and how to","og_url":"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it","og_site_name":"Data Science Courses | DataScientest","article_publisher":"https:\/\/www.facebook.com\/DataScientestEN","article_published_time":"2024-01-24T21:14:22+00:00","article_modified_time":"2024-01-16T21:43:08+00:00","og_image":[{"width":768,"height":384,"url":"https:\/\/datascientest.com\/en\/files\/2024\/01\/catastrophique-interference.webp","type":"image\/webp"}],"author":"Melanie","twitter_card":"summary_large_image","twitter_creator":"@DataScientest__","twitter_site":"@DataScientest__","twitter_misc":{"Written by":"Melanie","Estimated reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it#article","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it"},"author":{"name":"Melanie","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d"},"headline":"Catastrophic Interference: What is it? How to avoid it?","datePublished":"2024-01-24T21:14:22+00:00","dateModified":"2024-01-16T21:43:08+00:00","mainEntityOfPage":{"@id":"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it"},"wordCount":1219,"commentCount":0,"publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"articleSection":["Business Intelligence"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it#respond"]}],"copyrightYear":"2024","copyrightHolder":{"@id":"https:\/\/datascientest.com\/#organization"}},{"@type":"WebPage","@id":"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it","url":"https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it","name":"Catastrophic Interference: What is it? How to avoid it?","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/#website"},"datePublished":"2024-01-24T21:14:22+00:00","dateModified":"2024-01-16T21:43:08+00:00","description":"Catastrophic interference refers to a phenomenon in machine learning and artificial neural networks where learning new information significantly disrupts previously learned information, leading to a loss of","inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/datascientest.com\/en\/catastrophic-interference-what-is-it-how-to-avoid-it"]}]},{"@type":"WebSite","@id":"https:\/\/datascientest.com\/en\/#website","url":"https:\/\/datascientest.com\/en\/","name":"Data Science Courses | DataScientest","description":"Leader in Data Science Courses","publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/datascientest.com\/en\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/datascientest.com\/en\/#organization","name":"DataScientest","url":"https:\/\/datascientest.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","contentUrl":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","width":301,"height":301,"caption":"DataScientest"},"image":{"@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataScientestEN","https:\/\/twitter.com\/DataScientest__","https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/","https:\/\/www.tiktok.com\/@datascientest"]},{"@type":"Person","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d","name":"Melanie","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g","caption":"Melanie"},"url":"https:\/\/datascientest.com\/en\/author\/melaniekrause"}]}},"_links":{"self":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/176681"}],"collection":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/users\/67"}],"replies":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/comments?post=176681"}],"version-history":[{"count":0,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/176681\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media\/176688"}],"wp:attachment":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media?parent=176681"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/categories?post=176681"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/tags?post=176681"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}