{"id":177263,"date":"2024-01-21T12:19:49","date_gmt":"2024-01-21T11:19:49","guid":{"rendered":"https:\/\/datascientest.com\/en\/?p=177263"},"modified":"2024-01-21T12:24:45","modified_gmt":"2024-01-21T11:24:45","slug":"style-transfer-with-cyclegan","status":"publish","type":"post","link":"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan","title":{"rendered":"Style transfer with CycleGAN"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"177263\" class=\"elementor elementor-177263\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-75fefe1 e-flex e-con-boxed e-con\" data-id=\"75fefe1\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t<div class=\"elementor-element elementor-element-15eb482 e-con-full e-flex e-con\" data-id=\"15eb482\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;full&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-a5fcc99 elementor-widget elementor-widget-text-editor\" data-id=\"a5fcc99\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-text-editor.elementor-drop-cap-view-stacked .elementor-drop-cap{background-color:#69727d;color:#fff}.elementor-widget-text-editor.elementor-drop-cap-view-framed .elementor-drop-cap{color:#69727d;border:3px solid;background-color:transparent}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap{margin-top:8px}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap-letter{width:1em;height:1em}.elementor-widget-text-editor .elementor-drop-cap{float:left;text-align:center;line-height:1;font-size:50px}.elementor-widget-text-editor .elementor-drop-cap-letter{display:inline-block}<\/style>\t\t\t\t<h2>Neural Style Transfer (NST) is a set of models and methods for transferring the visual style of one image to another, using images or videos. In this article, we&#8217;ll be taking a closer look at a particular model called CycleGAN.<\/h2>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-020b95a elementor-widget elementor-widget-text-editor\" data-id=\"020b95a\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Today, the most successful NST algorithms are adapted <a href=\"https:\/\/datascientest.com\/en\/unleashing-artistic-expression-with-deep-learning-style-transfer\">Deep Learning algorithms<\/a> using convolution layers. Artistically, NST is most often used to create artificial works of art based on a photograph and a painter&#8217;s style. There are currently several applications (mobile and web) for <strong>style transfer<\/strong>, such as DeepArt and Reiinakano&#8217;s blog. For example, you can transform a Star Wars character (Darth Revan) into a painting inspired by Paul Klee&#8217;s Clouds over Bor :<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-885ed6d elementor-widget elementor-widget-image\" data-id=\"885ed6d\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-image{text-align:center}.elementor-widget-image a{display:inline-block}.elementor-widget-image a img[src$=\".svg\"]{width:48px}.elementor-widget-image img{vertical-align:middle;display:inline-block}<\/style>\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2022\/03\/Fichier-2-8.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\"><\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f7bc2ab elementor-align-center button-principal elementor-widget elementor-widget-button\" data-id=\"f7bc2ab\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-md\" href=\"https:\/\/datascientest.com\/en\/data-courses\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Discover our courses<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4128559 elementor-widget elementor-widget-heading\" data-id=\"4128559\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-heading-title{padding:0;margin:0;line-height:1}.elementor-widget-heading .elementor-heading-title[class*=elementor-size-]>a{color:inherit;font-size:inherit;line-height:inherit}.elementor-widget-heading .elementor-heading-title.elementor-size-small{font-size:15px}.elementor-widget-heading .elementor-heading-title.elementor-size-medium{font-size:19px}.elementor-widget-heading .elementor-heading-title.elementor-size-large{font-size:29px}.elementor-widget-heading .elementor-heading-title.elementor-size-xl{font-size:39px}.elementor-widget-heading .elementor-heading-title.elementor-size-xxl{font-size:59px}<\/style><h3 class=\"elementor-heading-title elementor-size-default\">The CycleGAN model, what is it?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-75aa457 elementor-widget elementor-widget-text-editor\" data-id=\"75aa457\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>As its name suggests, CycleGAN is a type of generative model (created by Jun-Yan Zhu et al. for the BAIR research laboratory at UC Berkeley), a GAN (Generative Adversarial Network) used to perform style transfer on an image. In practice, the vast majority of GANs used to transform an image from the style of a domain X to the style of a target domain Y are trained using a paired dataset in which each image instance of X has an instance directly associated with it in another Y style.<\/p><p>Where <strong>CycleGAN<\/strong> stands out is precisely in its lack of need for image peers during training. In fact, when we train such a model, we get very good performance without ever using paired images, which makes training much easier for particular datasets where no correspondence has been made. We can then imitate a painter&#8217;s style for different images using a database of his paintings, for example!<\/p><p>First, let&#8217;s look at the <a href=\"https:\/\/datascientest.com\/en\/gan-machine-learning-putting-fictitious-faces-into-practice\">architecture of a GAN.<\/a> If you&#8217;ve already read one of <a href=\"https:\/\/datascientest.com\/en\/edit-your-photos-at-will-with-drag-your-gan\">our articles on GANs,<\/a> you&#8217;ll know that a GAN is a model that breaks down into two large sub-models, a generator and a discriminator:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e5e59fa elementor-widget elementor-widget-image\" data-id=\"e5e59fa\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2022\/03\/schema_gan_blog_4-121.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Classic GAN structure<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-85091d9 elementor-align-center button-principal elementor-widget elementor-widget-button\" data-id=\"85091d9\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-md\" href=\"https:\/\/datascientest.com\/en\/machine-learning-engineer-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Become a Machine Learning Engineer<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-70a6a7c elementor-widget elementor-widget-text-editor\" data-id=\"70a6a7c\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The<strong> generator is the model<\/strong> used to generate images, generally from random noise, while the discriminator (which takes a value between 0 and 1) is used to classify a given image to determine whether it is artificial or not. Thus, during training, both the generator and the discriminator are trained so that the discriminator becomes stronger and stronger to differentiate a real image from a generated one, and the generator becomes stronger and stronger to fool the discriminator. During training, the generator does not have direct access to the real data; only the generator uses it to<strong> compare with simulated data.<\/strong><\/p><p>The loss function for the discriminator will be the errors it makes in classifying the data, and that of the generator will be the discriminator&#8217;s success in differentiating it from the real images. Some models use the same loss function, which is minimized by the discriminator and <strong>maximized by the generator.<\/strong><\/p><p>The <strong>CycleGAN structure<\/strong> is designed so that the generator G translates the distribution from one style X to another style Y, so that the discriminator cannot distinguish the transformed style Y=G(X) from the original style Y. There&#8217;s a problem that often arises when applying this strategy as it stands: the generator will translate all the images in the same way, so it can only generate a single example b, which is not the desired task.<\/p><p>To alleviate this problem, the <strong>CycleGAN structure<\/strong> will add a constraint by defining another generator F whose role will be to be the inverse transform of G in the sense that FGX\u2248X. This guarantees that the transformation of X will not be reduced to a single example. A new loss function characterizing cycle consistency loss is therefore added during training, encouraging transformations to verify the properties FGX\u2248X and that G(FY)\u2248Y. In addition to the discriminator for G (which we call DY because we want to discriminate Y from Y), we add a discriminator for F (which we call DX). In this paper, we find a diagram<strong> summarizing the various relationships described above.<\/strong><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5d53f19 elementor-widget elementor-widget-image\" data-id=\"5d53f19\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2022\/03\/schema_gan_blog_4-122.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Diagram summarizing the model's specific features<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-cf5127c elementor-widget elementor-widget-text-editor\" data-id=\"cf5127c\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The <strong>cycle-consistency loss<\/strong> is then separated into two distinct pieces, the first (1) corresponding to the loss between the elements of X and their reconstructions and the second (2) corresponding to the loss between the elements of Y and their reconstructions.<\/p><p>In this model, the <strong>generators and discriminators<\/strong> will optimize the same general loss function, which is made up of two loss sub-functions associated with the generators (adversial loss):<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b818865 elementor-widget elementor-widget-image\" data-id=\"b818865\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2022\/03\/Capture-de\u0301cran-2022-03-22-a\u0300-12.07.00.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4c6fb27 elementor-widget elementor-widget-image\" data-id=\"4c6fb27\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2022\/03\/Capture-de\u0301cran-2022-03-22-a\u0300-12.07.30.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-21fe4ba elementor-widget elementor-widget-text-editor\" data-id=\"21fe4ba\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>and the cyclic consistency loss function :<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-071dacd elementor-widget elementor-widget-image\" data-id=\"071dacd\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2022\/03\/Capture-de\u0301cran-2022-03-22-a\u0300-12.09.53.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1ed9a47 elementor-widget elementor-widget-text-editor\" data-id=\"1ed9a47\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>giving the overall loss function :<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-bee4534 elementor-widget elementor-widget-image\" data-id=\"bee4534\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2022\/03\/Capture-de\u0301cran-2022-03-22-a\u0300-12.10.56.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-40c8a9c elementor-widget elementor-widget-text-editor\" data-id=\"40c8a9c\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>which will be <strong>maximized by the discriminators (so that they can distinguish generation from reality)<\/strong> and minimized by the generators, so that they can create examples that are increasingly indistinguishable from real data.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7d82ab0 elementor-widget elementor-widget-heading\" data-id=\"7d82ab0\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">CycleGAN architecture in detail<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0850766 elementor-widget elementor-widget-text-editor\" data-id=\"0850766\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The two <strong>generators and the two discriminators<\/strong> are perfectly identical in architecture, i.e. they are made up of the same layers and have the same dimensions. The generators mainly use <a href=\"https:\/\/datascientest.com\/en\/convolutional-neural-network-everything-you-need-to-know\">convolution layers and residual blocks.<\/a><\/p><p>The convolution layers have already been discussed and explained in detail in another article on our blog, which you can find here.<\/p><p>Residual blocks are very <a href=\"https:\/\/datascientest.com\/en\/deep-learning-python-unveiling-the-basics\">simple layers in Deep Learning,<\/a> often used in style transfer, which ensure that the image doesn&#8217;t stray too far from the original one, recalling it to the network as it goes along. More formally, such a layer is of the form :<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5da9cc5 elementor-widget elementor-widget-image\" data-id=\"5da9cc5\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2022\/03\/schema_gan_blog_4-123.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Residual block<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9ff8b7e elementor-align-center elementor-widget elementor-widget-button\" data-id=\"9ff8b7e\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-scientist-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Discover our data scientist training<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-345f1cc elementor-widget elementor-widget-text-editor\" data-id=\"345f1cc\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>So, as shown in graph (d), the residual block just consists of a normal layer, to which the initial input value is added at the end.<\/p><p>A <strong>ReLU<\/strong> is also applied in some cases, to cancel the layer output if F(x) turns out to be too high negatively (where a color image would have positive values).<\/p><p>For discriminators, the model uses 70 x 70 PatchGANs, i.e. the discriminator will be made up of several small <strong>GANs, PatchGANs<\/strong> of initial size 70 x 70, which will be applied all over the image and may overlap in the part of the image under consideration, determining whether a given part of the image is real or not.<\/p><p>The final response is then calculated as the average of the responses returned for each patch. Each <strong>PatchGAN<\/strong> will have a similar architecture consisting solely of classical convolution layers. Finally, training is performed with a batch size of 1 and a classical optimizer: Adam.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ce90794 elementor-widget elementor-widget-image\" data-id=\"ce90794\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2022\/03\/resultat_gan.jpg\" title=\"\" alt=\"resultat_gan\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\"><\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5c69435 elementor-align-center elementor-widget elementor-widget-button\" data-id=\"5c69435\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-scientist-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Discover our data scientist training<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ab62f0a elementor-widget elementor-widget-text-editor\" data-id=\"ab62f0a\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Here we present some results obtained with test images through the previously defined model; on the left we have the original images provided as input and on the right the transfigured images. We can see, for example, that the model trained with images of horses and zebras can switch freely from one animal style to the other.<\/p><p>Similarly, when trained with artists&#8217; paintings,<strong> it can switch from a photograph to a painting in the style of Van Gogh.<\/strong> Very good results are also obtained in image reconstruction. In addition to the near-identical reconstruction, the quality of the reconstructed image is improved (with better depth of field) compared with the basic image:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9a47dff elementor-widget elementor-widget-image\" data-id=\"9a47dff\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2022\/03\/unnamed-7.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\"><\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-60683aa elementor-widget elementor-widget-text-editor\" data-id=\"60683aa\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>In this article, we looked at what style transfer is, and more specifically at the architecture of the<strong> CycleGAN model, inspired by GANs, for efficient style transfer. CycleGAN<\/strong> is one of the best algorithms for style transfer. We have also seen that this algorithm, unlike almost all style transfer algorithms, does not require matched data, which is complicated to obtain in practice, and can therefore be used to train different databases that were previously unusable, such as those of paintings by famous painters like Van Gogh.<\/p><p>Finally, we have seen that in addition to efficient style transfer, this model also features <a href=\"https:\/\/datascientest.com\/en\/image-processing-fundamental-principles-and-practical-uses\">faithful image reconstruction,<\/a> resulting in an output image with a better-realized depth of field, demonstrating the algorithm&#8217;s stability. This algorithm is not all-powerful, however, as it has shortcomings when it comes to<a href=\"https:\/\/datascientest.com\/en\/google-partners-with-adobe-for-a-more-ethical-image-generation\"> modifying the geometric properties of images<\/a>, such as when you want to turn a cat into a dog, so it is only suitable for changing textures.<\/p><p>If you&#8217;d like to learn more about object transfiguration, Deep Learning and artificial intelligence applied to images, take a look at our Data Scientist training course.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f827e20 elementor-widget elementor-widget-html\" data-id=\"f827e20\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<script type=\"application\/ld+json\">\n{\n  \"@context\": \"https:\/\/schema.org\",\n  \"@type\": \"FAQPage\",\n  \"mainEntity\": [{\n    \"@type\": \"Question\",\n    \"name\": \"The CycleGAN model, what is it?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"As its name suggests, CycleGAN is a type of generative model (created by Jun-Yan Zhu et al. for the BAIR research lab at UC Berkeley), a GAN (Generative Adversarial Network) used to perform style transfer on an image. In practice, the vast majority of GANs used to transform an image from the style of a domain X to the style of a target domain Y are trained using a paired dataset in which each image instance of X has a directly associated instance in another style Y. Where CycleGAN stands out is precisely in its non-need for image peers during training.\"\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"More details on CycleGAN architecture?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"The two generators and the two discriminators are perfectly identical in architecture, i.e. they are made up of the same layers and have the same dimensions. The generators mainly use convolution layers and residual blocks.\"\n    }\n  }]\n}\n<\/script>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-80c8b03 elementor-align-center elementor-widget elementor-widget-button\" data-id=\"80c8b03\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-scientist-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Discover our data scientist training<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Neural Style Transfer (NST) is a set of models and methods for transferring the visual style of one image to another, using images or videos. In this article, we&#8217;ll be taking a closer look at a particular model called CycleGAN. Today, the most successful NST algorithms are adapted Deep Learning algorithms using convolution layers. Artistically, [&hellip;]<\/p>\n","protected":false},"author":67,"featured_media":177268,"comment_status":"open","ping_status":"open","sticky":false,"template":"elementor_theme","format":"standard","meta":{"content-type":"","inline_featured_image":false},"categories":[2409],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.7 (Yoast SEO v21.7) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Style transfer with CycleGAN<\/title>\n<meta name=\"description\" content=\"Neural Style Transfer (NST) is a set of models and methods for transferring the visual style of one image to another, using images or videos. In this\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Style transfer with CycleGAN\" \/>\n<meta property=\"og:description\" content=\"Neural Style Transfer (NST) is a set of models and methods for transferring the visual style of one image to another, using images or videos. In this\" \/>\n<meta property=\"og:url\" content=\"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan\" \/>\n<meta property=\"og:site_name\" content=\"Data Science Courses | DataScientest\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataScientestEN\" \/>\n<meta property=\"article:published_time\" content=\"2024-01-21T11:19:49+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-01-21T11:24:45+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/datascientest.com\/en\/files\/2024\/01\/cyclegan.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"562\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"author\" content=\"Melanie\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:site\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Melanie\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan#article\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan\"},\"author\":{\"name\":\"Melanie\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d\"},\"headline\":\"Style transfer with CycleGAN\",\"datePublished\":\"2024-01-21T11:19:49+00:00\",\"dateModified\":\"2024-01-21T11:24:45+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan\"},\"wordCount\":1362,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"articleSection\":[\"Machine Learning\"],\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan#respond\"]}],\"copyrightYear\":\"2024\",\"copyrightHolder\":{\"@id\":\"https:\/\/datascientest.com\/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan\",\"url\":\"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan\",\"name\":\"Style transfer with CycleGAN\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/#website\"},\"datePublished\":\"2024-01-21T11:19:49+00:00\",\"dateModified\":\"2024-01-21T11:24:45+00:00\",\"description\":\"Neural Style Transfer (NST) is a set of models and methods for transferring the visual style of one image to another, using images or videos. In this\",\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan\"]}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/datascientest.com\/en\/#website\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"name\":\"Data Science Courses | DataScientest\",\"description\":\"Leader in Data Science Courses\",\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/datascientest.com\/en\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/datascientest.com\/en\/#organization\",\"name\":\"DataScientest\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"contentUrl\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"width\":301,\"height\":301,\"caption\":\"DataScientest\"},\"image\":{\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/DataScientestEN\",\"https:\/\/twitter.com\/DataScientest__\",\"https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/\",\"https:\/\/www.tiktok.com\/@datascientest\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d\",\"name\":\"Melanie\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g\",\"caption\":\"Melanie\"},\"url\":\"https:\/\/datascientest.com\/en\/author\/melaniekrause\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Style transfer with CycleGAN","description":"Neural Style Transfer (NST) is a set of models and methods for transferring the visual style of one image to another, using images or videos. In this","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan","og_locale":"en_GB","og_type":"article","og_title":"Style transfer with CycleGAN","og_description":"Neural Style Transfer (NST) is a set of models and methods for transferring the visual style of one image to another, using images or videos. In this","og_url":"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan","og_site_name":"Data Science Courses | DataScientest","article_publisher":"https:\/\/www.facebook.com\/DataScientestEN","article_published_time":"2024-01-21T11:19:49+00:00","article_modified_time":"2024-01-21T11:24:45+00:00","og_image":[{"width":1024,"height":562,"url":"https:\/\/datascientest.com\/en\/files\/2024\/01\/cyclegan.webp","type":"image\/webp"}],"author":"Melanie","twitter_card":"summary_large_image","twitter_creator":"@DataScientest__","twitter_site":"@DataScientest__","twitter_misc":{"Written by":"Melanie","Estimated reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan#article","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan"},"author":{"name":"Melanie","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d"},"headline":"Style transfer with CycleGAN","datePublished":"2024-01-21T11:19:49+00:00","dateModified":"2024-01-21T11:24:45+00:00","mainEntityOfPage":{"@id":"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan"},"wordCount":1362,"commentCount":0,"publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"articleSection":["Machine Learning"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan#respond"]}],"copyrightYear":"2024","copyrightHolder":{"@id":"https:\/\/datascientest.com\/#organization"}},{"@type":"WebPage","@id":"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan","url":"https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan","name":"Style transfer with CycleGAN","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/#website"},"datePublished":"2024-01-21T11:19:49+00:00","dateModified":"2024-01-21T11:24:45+00:00","description":"Neural Style Transfer (NST) is a set of models and methods for transferring the visual style of one image to another, using images or videos. In this","inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/datascientest.com\/en\/style-transfer-with-cyclegan"]}]},{"@type":"WebSite","@id":"https:\/\/datascientest.com\/en\/#website","url":"https:\/\/datascientest.com\/en\/","name":"Data Science Courses | DataScientest","description":"Leader in Data Science Courses","publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/datascientest.com\/en\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/datascientest.com\/en\/#organization","name":"DataScientest","url":"https:\/\/datascientest.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","contentUrl":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","width":301,"height":301,"caption":"DataScientest"},"image":{"@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataScientestEN","https:\/\/twitter.com\/DataScientest__","https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/","https:\/\/www.tiktok.com\/@datascientest"]},{"@type":"Person","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d","name":"Melanie","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g","caption":"Melanie"},"url":"https:\/\/datascientest.com\/en\/author\/melaniekrause"}]}},"_links":{"self":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/177263"}],"collection":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/users\/67"}],"replies":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/comments?post=177263"}],"version-history":[{"count":0,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/177263\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media\/177268"}],"wp:attachment":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media?parent=177263"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/categories?post=177263"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/tags?post=177263"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}