{"id":180810,"date":"2024-04-16T23:17:18","date_gmt":"2024-04-16T22:17:18","guid":{"rendered":"https:\/\/datascientest.com\/en\/?p=180810"},"modified":"2024-02-28T23:36:43","modified_gmt":"2024-02-28T22:36:43","slug":"multimodal-learning-what-is-it-what-is-it-used-for","status":"publish","type":"post","link":"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for","title":{"rendered":"Multimodal Learning: What is it? What is it used for?"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"180810\" class=\"elementor elementor-180810\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-5b00403 e-flex e-con-boxed e-con\" data-id=\"5b00403\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-adb6588 elementor-widget elementor-widget-heading\" data-id=\"adb6588\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-heading-title{padding:0;margin:0;line-height:1}.elementor-widget-heading .elementor-heading-title[class*=elementor-size-]>a{color:inherit;font-size:inherit;line-height:inherit}.elementor-widget-heading .elementor-heading-title.elementor-size-small{font-size:15px}.elementor-widget-heading .elementor-heading-title.elementor-size-medium{font-size:19px}.elementor-widget-heading .elementor-heading-title.elementor-size-large{font-size:29px}.elementor-widget-heading .elementor-heading-title.elementor-size-xl{font-size:39px}.elementor-widget-heading .elementor-heading-title.elementor-size-xxl{font-size:59px}<\/style><h2 class=\"elementor-heading-title elementor-size-default\">Multimodal Learning is an evolution of Machine Learning, involving the simultaneous use of several data sources such as text, image and audio to solve much more complex tasks. Find out all you need to know about this new technique, which is set to push back the frontiers of AI!<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-eb3ee38 elementor-widget elementor-widget-text-editor\" data-id=\"eb3ee38\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-text-editor.elementor-drop-cap-view-stacked .elementor-drop-cap{background-color:#69727d;color:#fff}.elementor-widget-text-editor.elementor-drop-cap-view-framed .elementor-drop-cap{color:#69727d;border:3px solid;background-color:transparent}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap{margin-top:8px}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap-letter{width:1em;height:1em}.elementor-widget-text-editor .elementor-drop-cap{float:left;text-align:center;line-height:1;font-size:50px}.elementor-widget-text-editor .elementor-drop-cap-letter{display:inline-block}<\/style>\t\t\t\t<p><a href=\"https:\/\/datascientest.com\/en\/introduction-to-business-intelligence\">Artificial intelligence<\/a> has made impressive progress in recent years. Its development is linked in particular to machine learning and <a href=\"https:\/\/datascientest.com\/en\/deep-neural-network-what-is-it-and-how-is-it-working\">deep neural networks.<\/a><\/p><p>However, these advances have mainly been made in &#8220;unimodal&#8221; tasks. In the real world, however, information comes from multiple sensory sources, combining text, image, audio and even video.<\/p><p>So the next step for AI is to exploit these multiple modalities simultaneously and in an integrated way, for a richer and more complete understanding. To achieve this, researchers are using the &#8220;multimodal learning&#8221; technique.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-eb6ddc4 e-flex e-con-boxed e-con\" data-id=\"eb6ddc4\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-610cfc9 elementor-widget elementor-widget-heading\" data-id=\"610cfc9\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">What is Multimodal Learning? What data does it use?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-fe2e5f6 elementor-widget elementor-widget-text-editor\" data-id=\"fe2e5f6\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>When you put your head out of the window, you immediately receive a great deal of information. This is linked to the combination of our 5 senses: <em>hearing, sight, smell, taste and touch,<\/em> which enable us to perceive sounds, images, textures and scents all at the same time.<\/p><p>Multimodal Learning aims to apply this idea of using different data simultaneously to the field of AI. Let&#8217;s start by looking at the different types of source.<\/p><p>Text is one of the most commonly used modalities in Machine Learning. Textual data contains rich, structured information, and <a href=\"https:\/\/datascientest.com\/en\/natural-language-processing-definition-and-principles\">natural language processing<\/a> (NLP) makes it easy to extract knowledge from it.<\/p><p>This data can come from documents, press articles, messages on social networks or any other type of text. The NLP techniques used to process them include tokenisation, lemmatisation, syntactic analysis, named entity detection and text classification.<\/p><p>For their part, images are an essential source of visual information in Multimodal Learning. Thanks to the growing popularity of <a href=\"https:\/\/datascientest.com\/en\/convolutional-neural-network-everything-you-need-to-know\">convolutional neural networks<\/a> (CNNs), major advances have been made in the understanding of images.<\/p><p>Computer vision techniques can be used to analyse and interpret images to extract knowledge. Examples include object detection, facial recognition and image segmentation.<\/p><p>Audio includes information from voice recordings, sound files or live streams. These are analysed using audio processing techniques to extract acoustic and linguistic characteristics.<\/p><p>The most commonly used methods include voice recognition, sound event detection, source separation and <a href=\"https:\/\/datascientest.com\/en\/classification-algorithms-definition-and-main-models\">classification.<\/a><\/p><p>Finally, video is a powerful source of multimodal data, combining visual and audio information. Once again, computer vision and audio processing techniques can be used to extract knowledge from a sequence.<\/p><p>This makes it possible to detect moving objects, analyse human activity and even recognise gestures. This fusion of visual and audio modalities enables machines to better understand scenes and events.<\/p><p>With the rise of smartphone cameras and video-sharing social networks such as TikTok and YouTube, AIs now have access to a vast reservoir of resources on which to train.<\/p><p>In the future, with the <strong>emergence of humanoid robots<\/strong> equipped with tactile sensors on their fingers, artificial intelligences could also receive the sense of touch and use it to learn&#8230;<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8e22042 elementor-widget elementor-widget-image\" data-id=\"8e22042\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-image{text-align:center}.elementor-widget-image a{display:inline-block}.elementor-widget-image a img[src$=\".svg\"]{width:48px}.elementor-widget-image img{vertical-align:middle;display:inline-block}<\/style>\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/08\/multimodal_learning1.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\"><\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-477c5a6 elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"477c5a6\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/machine-learning-engineer-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Learning to master Multimodal Learning<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-eb9bf90 e-flex e-con-boxed e-con\" data-id=\"eb9bf90\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-1ae7e07 elementor-widget elementor-widget-heading\" data-id=\"1ae7e07\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">How is Multimodal Learning used?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-76ec394 elementor-widget elementor-widget-text-editor\" data-id=\"76ec394\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><strong>Multimodal Learning<\/strong> is applied in many and varied ways in many areas of <a href=\"https:\/\/datascientest.com\/en\/google-improves-its-workspace-suite-with-artificial-intelligence\">artificial intelligence.<\/a><\/p><p>One of the main use cases is scene recognition and understanding. By combining visual, audio and video information, it is possible to analyse and interpret complex scenes with greater precision and detail.<\/p><p>Examples include detecting and tracking moving objects in a video, such as people in CCTV footage.<\/p><p>The combination of visual and audio information helps to automatically d<strong>etect suspicious events such as aggressive behaviour<\/strong>, intrusions or emergency situations in security camera images. This is a valuable asset for surveillance.<\/p><p>It is also possible to recognise and understand human activity in videos thanks to visual and audio information. For example, in a video recorded during a sporting event, the detection of gestures and the understanding of social interactions help the AI to recognise a sport.<\/p><p>Another field of application for <strong>Multimodal Learning<\/strong> is translation. In particular, this <a href=\"https:\/\/datascientest.com\/en\/pillow-how-to-process-images-with-python\">approach enables speech and images<\/a> to be translated simultaneously during an oral presentation accompanied by visual slides. This facilitates comprehension for a<strong> multilingual audience.<\/strong><\/p><p>Similarly, textual instructions can be automatically translated into visual instructions. The aim may be, for example, to guide a robot through its tasks.<\/p><p>There are also caption generators for images, based on Multimodal Learning. This is very useful for people with impaired vision or for automating the subtitling process.<\/p><p>Thanks to a conversational interface based on <strong>Multimodal Learning,<\/strong> a virtual assistant can interact with users using voice, text and images. The experience is therefore more natural and immersive, as it becomes possible to express intentions and needs in a variety of ways.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-c82b958 e-flex e-con-boxed e-con\" data-id=\"c82b958\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-50c2f23 elementor-widget elementor-widget-heading\" data-id=\"50c2f23\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Multimodal Learning and generative AI<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5f75415 elementor-widget elementor-widget-text-editor\" data-id=\"5f75415\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><strong>Generative artificial intelligence<\/strong> is also based on Multimodal Learning. This type of AI uses neural networks to generate new content: images, videos, text, etc.<\/p><p>For example, AI chatbots such as ChatGPT are based on generative AI to produce text from prompts. By integrating different modalities, they are able to interact with users in a richer, more natural way.<\/p><p>The most advanced dialogue systems, such as <strong>OpenAI&#8217;s GPT-4,<\/strong> integrate text, speech and images to understand and respond to requests in a contextual and personalised way.<\/p><p>Similarly, the<strong> DALL-E AI<\/strong> creates images from text prompts. It has been trained on both text and images to learn how to associate them.<\/p><p>Generative AI can enable more personalised human-machine interactions, create realistic 3D images and videos for films or video games, or even new product designs.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-d8c733d e-flex e-con-boxed e-con\" data-id=\"d8c733d\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-126fb49 elementor-widget elementor-widget-heading\" data-id=\"126fb49\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Different approaches and techniques<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c00f3ea elementor-widget elementor-widget-text-editor\" data-id=\"c00f3ea\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><strong>Multimodal Learning<\/strong> is a complex discipline, based on a vast array of techniques. Here are the most commonly used.<\/p><p>Firstly, fusion models play a key role in combining information.<strong> Multimodal Neural Networks<\/strong> can be built using specific fusion layers that take into account the characteristics of each modality and combine the information appropriately.<\/p><p>Another method is<a href=\"https:\/\/datascientest.com\/en\/transfer-learning-what-is-it\"> transfer learning<\/a>, which allows knowledge learned from one modality to be transferred to another, which can be very useful when data is limited in a specific modality. For example, models pre-trained on <a href=\"https:\/\/datascientest.com\/en\/mushroom-recognition\">computer vision<\/a> tasks can be transferred to image comprehension tasks in other domains.<\/p><p>It is also possible to use pre-trained models on large amounts of data as a starting point for accelerating <strong>multimodal learning.<\/strong><\/p><p>A pre-trained language model can be used to extract textual features in a multimodal task, in order to exploit the knowledge already gained from initial training on a large dataset.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2053d9e elementor-widget elementor-widget-image\" data-id=\"2053d9e\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/08\/multimodal_learning2.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\"><\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ba6216b elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"ba6216b\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/machine-learning-engineer-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Learn all about Multimodal Learning<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8f59a22 elementor-widget elementor-widget-text-editor\" data-id=\"8f59a22\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Finally, the representation of multimodal data is a crucial stage, as it influences the model&#8217;s ability to understand and exploit the various modalities.<\/p><p>For example, learning common representations aims to find shared representation spaces between the different modalities.<\/p><p>This makes it possible to extract common characteristics that capture shared information and facilitate the <strong>overall understanding of multimodal data.<\/strong><\/p><p>Co-learning or adversarial learning techniques are used to learn these shared representations.<\/p><p>An alternative is the use of self-encoding neural networks: architectures that learn to reconstruct input data by passing through a latent representation.<\/p><p>They can be used to<strong> extract relevant multimodal features,<\/strong> which are then exploited for the fusion and learning of multimodal models.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-6423b40 e-flex e-con-boxed e-con\" data-id=\"6423b40\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-b3a1e23 elementor-widget elementor-widget-heading\" data-id=\"b3a1e23\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">The challenges of multimodal learning<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d72d65e elementor-widget elementor-widget-text-editor\" data-id=\"d72d65e\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Multimodal learning presents a number of challenges, and requires particular attention to ensure that the <strong>different types of data are used effectively.<\/strong><\/p><p>One of the main issues is the alignment of modalities. For example, when analysing a video with an audio track, the visual and audio information must be temporarily aligned so that the scene can be understood as a whole. Various synchronisation techniques are used to meet this requirement.<\/p><p>Similarly, merging information from different modalities can be a complex task. There are several methods for efficiently combining textual, visual, audio and video information, such as concatenation or the use of multimodal neural networks to learn integrated representations.<\/p><p>Whatever approach is used, the fusion must capture interactions and dependencies to provide a global understanding and overview.<\/p><p>Another challenge is to represent the data in such a way as to capture the relevant information from each modality for effective use in learning.<\/p><p>In general, <strong>Deep Learning techniques<\/strong> are used to extract significant features. For example, encoding neural networks can be used to capture information shared between modalities.<\/p><p>\u00a0<\/p><p>\ud83d\udca1Related articles:<\/p><table dir=\"ltr\" border=\"1\" cellspacing=\"0\" cellpadding=\"0\" data-sheets-root=\"1\"><colgroup><col width=\"656\" \/><\/colgroup><tbody><tr><td data-sheets-value=\"{&quot;1&quot;:2,&quot;2&quot;:&quot;Image Processing&quot;}\" data-sheets-hyperlink=\"https:\/\/datascientest.com\/en\/image-processing-fundamental-principles-and-practical-uses\"><a class=\"in-cell-link\" href=\"https:\/\/datascientest.com\/en\/image-processing-fundamental-principles-and-practical-uses\" target=\"_blank\" rel=\"noopener\">Image Processing<\/a><\/td><\/tr><tr><td data-sheets-value=\"{&quot;1&quot;:2,&quot;2&quot;:&quot;Deep Learning - All you need to know&quot;}\" data-sheets-hyperlink=\"https:\/\/datascientest.com\/en\/all-about-deep-learning\"><a class=\"in-cell-link\" href=\"https:\/\/datascientest.com\/en\/all-about-deep-learning\" target=\"_blank\" rel=\"noopener\">Deep Learning &#8211; All you need to know<\/a><\/td><\/tr><tr><td data-sheets-value=\"{&quot;1&quot;:2,&quot;2&quot;:&quot;Mushroom Recognition&quot;}\" data-sheets-hyperlink=\"https:\/\/datascientest.com\/en\/mushroom-recognition\"><a class=\"in-cell-link\" href=\"https:\/\/datascientest.com\/en\/mushroom-recognition\" target=\"_blank\" rel=\"noopener\">Mushroom Recognition<\/a><\/td><\/tr><tr><td data-sheets-value=\"{&quot;1&quot;:2,&quot;2&quot;:&quot;Tensor Flow - Google's ML&quot;}\" data-sheets-hyperlink=\"https:\/\/datascientest.com\/en\/tensor-flow-all-about-googles-machine-learning-framework\"><a class=\"in-cell-link\" href=\"https:\/\/datascientest.com\/en\/tensor-flow-all-about-googles-machine-learning-framework\" target=\"_blank\" rel=\"noopener\">Tensor Flow &#8211; Google&#8217;s ML<\/a><\/td><\/tr><tr><td data-sheets-value=\"{&quot;1&quot;:2,&quot;2&quot;:&quot;Dive into ML&quot;}\" data-sheets-hyperlink=\"https:\/\/datascientest.com\/en\/unlock-your-future-dive-into-machine-learning-engineer-training\"><a class=\"in-cell-link\" href=\"https:\/\/datascientest.com\/en\/unlock-your-future-dive-into-machine-learning-engineer-training\" target=\"_blank\" rel=\"noopener\">Dive into ML<\/a><\/td><\/tr><\/tbody><\/table>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-56eec5f e-flex e-con-boxed e-con\" data-id=\"56eec5f\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-17c3217 elementor-widget elementor-widget-heading\" data-id=\"17c3217\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Examples of Multimodal Learning systems<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b96b0c4 elementor-widget elementor-widget-text-editor\" data-id=\"b96b0c4\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Thanks to scientific advances in the field of Multimodal Learning, a number of systems have emerged, some of which are used by many people. Here are a few examples.<\/p><p>The American company <strong>OpenAI has developed DALL-E,<\/strong> an AI system for converting text into images. It is a neural network with 12 billion parameters.<\/p><p>The company has also created CLIP. This multimodal system can perform a wide variety of visual recognition tasks, and can classify images by category without the need for example data.<\/p><p>For its part, Google has created <strong>ALIGN<\/strong>: an AI model trained on a dataset containing numerous image-text pairs. According to several benchmarks, it is the best performing model of its kind.<\/p><p>The Californian giant has also created the MURAL AI for image-text association and linguistic translation. This model uses multi-task learning applied to image-text pairs in association with their translation into more than 100 languages.<\/p><p>Another Google project is <strong>VATT:<\/strong> a multimodal Video-Audio-Text AI. It can make predictions from raw data, generate descriptions of events in video or even create videos from a prompt.<\/p><p>Microsoft researchers have created NUWA to produce new images and videos or modify existing ones. This model is trained on images, videos and text. It has learned to predict the next frame of a video or to fill in incomplete images.<\/p><p>Another Microsoft Resarch project is Florence, which is capable of modelling space, time and modality. Finally, FLAVA has been trained by Meta on images and 35 different languages, and is proving to be very effective for a wide variety of multimodal tasks.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-3db4e21 e-flex e-con-boxed e-con\" data-id=\"3db4e21\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-ed24960 elementor-widget elementor-widget-heading\" data-id=\"ed24960\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Conclusion: Multimodal Learning, the next frontier of AI<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-627ec3e elementor-widget elementor-widget-text-editor\" data-id=\"627ec3e\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>By <strong>enabling AI systems to learn from several types of data simultaneously,<\/strong> Multimodal Learning brings machines closer to the human brain and its multisensory perception.<\/p><p>So, in the near future, this approach could well enable <strong>artificial intelligence to continue to approach human intelligence, and even surpass it&#8230;<\/strong><\/p><p>To master Machine Learning and all its techniques, you can choose DataScientest. Our Data Science courses all include one or more modules dedicated to Machine Learning, Deep Learning and AI.<\/p><p>The concepts covered include classification, regression and clustering techniques using scikit-learn, text mining and time series analysis methods, as well as CNN and RNN neural networks using Keras,<a href=\"https:\/\/datascientest.com\/en\/step-by-step-guide-installing-tensorflow-with-anaconda-on-macos\"> TensorFlow<\/a> and PyTorch.<\/p><p>Our various courses are entirely distance learning, and enable you to acquire all the skills you need to work as a Data Scientist, Data Analyst, Data Engineer,<a href=\"https:\/\/datascientest.com\/en\/machine-learning-engineer-course\"> ML Engineer<\/a>, or in new AI professions such as Prompt Engineer.<\/p><p>Our organisation is eligible for the funding options recognised by the French government, and you can receive a diploma issued by MINES Paris Executive Education and a cloud certificate from our partners AWS and Microsoft Azure. Discover DataScientest!<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-86fe221 elementor-widget elementor-widget-image\" data-id=\"86fe221\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/08\/multimodal_learning3.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\"><\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d7d6486 elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"d7d6486\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/machine-learning-engineer-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Become a machine learning engineer<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e7fd156 elementor-widget elementor-widget-text-editor\" data-id=\"e7fd156\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Now you know all about Multimodal Learning. For more information on the same subject, <a href=\"https:\/\/datascientest.com\/en\/unraveling-machine-learning-vs-deep-learning-key-differences-explained\">read our full report on Machine Learning.<\/a><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-d9b91c7 e-flex e-con-boxed e-con\" data-id=\"d9b91c7\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-c7942a7 elementor-widget elementor-widget-html\" data-id=\"c7942a7\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<script type=\"application\/ld+json\">\n{\n  \"@context\": \"https:\/\/schema.org\",\n  \"@type\": \"FAQPage\",\n  \"mainEntity\": [{\n    \"@type\": \"Question\",\n    \"name\": \"What is Multimodal Learning? What data does it use?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"When you put your head out of the window, you immediately receive a lot of information. This is linked to the combination of our 5 senses: hearing, sight, smell, taste and touch, which enable us to perceive sounds, images, textures and scents.\"\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"How is Multimodal Learning used?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"First of all, fusion models play a key role in combining information. Multimodal neural networks allow integrated representations to be learned, thanks to the extraction of common features from multimodal data.These Neural Networks can be built using specific fusion layers that take into account the characteristics of each modality and combine information appropriately.\"\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"The Challenges of Multimodal Learning\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"One of the main problems is the alignment of modalities. For example, when analysing a video with an audio track, the visual and audio information must be temporarily aligned so that the whole scene can be understood. Various synchronisation techniques are used to meet this requirement.\"\n    }\n  }]\n}\n<\/script>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Multimodal Learning is an evolution of Machine Learning, involving the simultaneous use of several data sources such as text, image and audio to solve much more complex tasks. Find out all you need to know about this new technique, which is set to push back the frontiers of AI! Artificial intelligence has made impressive progress [&hellip;]<\/p>\n","protected":false},"author":67,"featured_media":180811,"comment_status":"open","ping_status":"open","sticky":false,"template":"elementor_theme","format":"standard","meta":{"content-type":"","inline_featured_image":false},"categories":[2409],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.7 (Yoast SEO v21.7) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Multimodal Learning: What is it? What is it used for?<\/title>\n<meta name=\"description\" content=\"Learn about multimodal learning, a technique that integrates multiple modes of information such as text, images, and audio. Discover its applications across various fields and how it enhances learning experiences, data analysis, and understanding in diverse contexts\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Multimodal Learning: What is it? What is it used for?\" \/>\n<meta property=\"og:description\" content=\"Multimodal Learning is an evolution of Machine Learning, involving the simultaneous use of several data sources such as text, image and audio to solve\" \/>\n<meta property=\"og:url\" content=\"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for\" \/>\n<meta property=\"og:site_name\" content=\"Data Science Courses | DataScientest\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataScientestEN\" \/>\n<meta property=\"article:published_time\" content=\"2024-04-16T22:17:18+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-02-28T22:36:43+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/datascientest.com\/en\/files\/2024\/02\/Illu_BLOG__multimodal.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"562\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Melanie\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:site\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Melanie\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"9 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for#article\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for\"},\"author\":{\"name\":\"Melanie\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d\"},\"headline\":\"Multimodal Learning: What is it? What is it used for?\",\"datePublished\":\"2024-04-16T22:17:18+00:00\",\"dateModified\":\"2024-02-28T22:36:43+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for\"},\"wordCount\":1882,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"articleSection\":[\"Machine Learning\"],\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for#respond\"]}],\"copyrightYear\":\"2024\",\"copyrightHolder\":{\"@id\":\"https:\/\/datascientest.com\/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for\",\"url\":\"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for\",\"name\":\"Multimodal Learning: What is it? What is it used for?\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/#website\"},\"datePublished\":\"2024-04-16T22:17:18+00:00\",\"dateModified\":\"2024-02-28T22:36:43+00:00\",\"description\":\"Learn about multimodal learning, a technique that integrates multiple modes of information such as text, images, and audio. Discover its applications across various fields and how it enhances learning experiences, data analysis, and understanding in diverse contexts\",\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for\"]}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/datascientest.com\/en\/#website\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"name\":\"Data Science Courses | DataScientest\",\"description\":\"Leader in Data Science Courses\",\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/datascientest.com\/en\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/datascientest.com\/en\/#organization\",\"name\":\"DataScientest\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"contentUrl\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"width\":301,\"height\":301,\"caption\":\"DataScientest\"},\"image\":{\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/DataScientestEN\",\"https:\/\/twitter.com\/DataScientest__\",\"https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/\",\"https:\/\/www.tiktok.com\/@datascientest\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d\",\"name\":\"Melanie\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g\",\"caption\":\"Melanie\"},\"url\":\"https:\/\/datascientest.com\/en\/author\/melaniekrause\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Multimodal Learning: What is it? What is it used for?","description":"Learn about multimodal learning, a technique that integrates multiple modes of information such as text, images, and audio. Discover its applications across various fields and how it enhances learning experiences, data analysis, and understanding in diverse contexts","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for","og_locale":"en_GB","og_type":"article","og_title":"Multimodal Learning: What is it? What is it used for?","og_description":"Multimodal Learning is an evolution of Machine Learning, involving the simultaneous use of several data sources such as text, image and audio to solve","og_url":"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for","og_site_name":"Data Science Courses | DataScientest","article_publisher":"https:\/\/www.facebook.com\/DataScientestEN","article_published_time":"2024-04-16T22:17:18+00:00","article_modified_time":"2024-02-28T22:36:43+00:00","og_image":[{"width":1024,"height":562,"url":"https:\/\/datascientest.com\/en\/files\/2024\/02\/Illu_BLOG__multimodal.png","type":"image\/png"}],"author":"Melanie","twitter_card":"summary_large_image","twitter_creator":"@DataScientest__","twitter_site":"@DataScientest__","twitter_misc":{"Written by":"Melanie","Estimated reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for#article","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for"},"author":{"name":"Melanie","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d"},"headline":"Multimodal Learning: What is it? What is it used for?","datePublished":"2024-04-16T22:17:18+00:00","dateModified":"2024-02-28T22:36:43+00:00","mainEntityOfPage":{"@id":"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for"},"wordCount":1882,"commentCount":0,"publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"articleSection":["Machine Learning"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for#respond"]}],"copyrightYear":"2024","copyrightHolder":{"@id":"https:\/\/datascientest.com\/#organization"}},{"@type":"WebPage","@id":"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for","url":"https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for","name":"Multimodal Learning: What is it? What is it used for?","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/#website"},"datePublished":"2024-04-16T22:17:18+00:00","dateModified":"2024-02-28T22:36:43+00:00","description":"Learn about multimodal learning, a technique that integrates multiple modes of information such as text, images, and audio. Discover its applications across various fields and how it enhances learning experiences, data analysis, and understanding in diverse contexts","inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/datascientest.com\/en\/multimodal-learning-what-is-it-what-is-it-used-for"]}]},{"@type":"WebSite","@id":"https:\/\/datascientest.com\/en\/#website","url":"https:\/\/datascientest.com\/en\/","name":"Data Science Courses | DataScientest","description":"Leader in Data Science Courses","publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/datascientest.com\/en\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/datascientest.com\/en\/#organization","name":"DataScientest","url":"https:\/\/datascientest.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","contentUrl":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","width":301,"height":301,"caption":"DataScientest"},"image":{"@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataScientestEN","https:\/\/twitter.com\/DataScientest__","https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/","https:\/\/www.tiktok.com\/@datascientest"]},{"@type":"Person","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d","name":"Melanie","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g","caption":"Melanie"},"url":"https:\/\/datascientest.com\/en\/author\/melaniekrause"}]}},"_links":{"self":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/180810"}],"collection":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/users\/67"}],"replies":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/comments?post=180810"}],"version-history":[{"count":0,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/180810\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media\/180811"}],"wp:attachment":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media?parent=180810"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/categories?post=180810"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/tags?post=180810"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}