{"id":196717,"date":"2025-08-07T06:18:00","date_gmt":"2025-08-07T05:18:00","guid":{"rendered":"https:\/\/datascientest.com\/en\/?p=196717"},"modified":"2025-07-31T00:19:18","modified_gmt":"2025-07-30T23:19:18","slug":"all-about-how-the-gpt-model-works","status":"publish","type":"post","link":"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works","title":{"rendered":"How does the GPT model work?"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"196717\" class=\"elementor elementor-196717\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-3ce8918 e-flex e-con-boxed e-con\" data-id=\"3ce8918\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-fac7442 elementor-widget elementor-widget-text-editor\" data-id=\"fac7442\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-text-editor.elementor-drop-cap-view-stacked .elementor-drop-cap{background-color:#69727d;color:#fff}.elementor-widget-text-editor.elementor-drop-cap-view-framed .elementor-drop-cap{color:#69727d;border:3px solid;background-color:transparent}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap{margin-top:8px}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap-letter{width:1em;height:1em}.elementor-widget-text-editor .elementor-drop-cap{float:left;text-align:center;line-height:1;font-size:50px}.elementor-widget-text-editor .elementor-drop-cap-letter{display:inline-block}<\/style>\t\t\t\t<p><b>Here we are not talking about <a href=\"https:\/\/datascientest.com\/en\/chatgpt-how-does-this-nlp-algorithm-work\">ChatGPT<\/a>, but that&#8217;s indeed where it got its name.<\/b><\/p><p><b>If you\u2019ve never heard of GPT, you have at least used text-generating AIs, or &#8216;<a href=\"https:\/\/datascientest.com\/en\/large-language-models-llm-everything-you-need-to-know\">LLM<\/a>&#8216; (Large Language Model). GPT is a type of LLM, on which most of the generative AIs that everyone is talking about right now are based!<\/b><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-669e634 elementor-widget elementor-widget-text-editor\" data-id=\"669e634\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><i><span style=\"font-weight: 400;\">Chat<\/span><\/i><b><i>GPT<\/i><\/b><i><span style=\"font-weight: 400;\">, now you see why?<\/span><\/i><\/p><p><span style=\"font-weight: 400;\">GPT stands for \u2018<a href=\"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers\">Generative Pre-trained Transformer<\/a>\u2019 (for the French variant), and it\u2019s a very clear name that perfectly summarizes its operation!<\/span><\/p><p><i><span style=\"font-weight: 400;\">How come you&#8217;re not convinced?<\/span><\/i><\/p><p><span style=\"font-weight: 400;\">Imagine GPT as a super-expert in word prediction, its main talent is guessing which word would most likely complete the beginning of a sentence. By repeating this prediction over and over, word by word, it builds entire sentences, paragraphs, and even complete articles!<\/span><\/p><p><i><span style=\"font-weight: 400;\">It helped a bit in writing this one, but there&#8217;s still a human behind it\u2026<\/span><\/i><\/p><p><span style=\"font-weight: 400;\">Let\u2019s detail together the different steps of operation to see how GPT learns to understand how human language works.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8c98ec2 elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"8c98ec2\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-courses\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Going deeper into GPT<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-46c01f1 e-flex e-con-boxed e-con\" data-id=\"46c01f1\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-add7575 elementor-widget elementor-widget-heading\" data-id=\"add7575\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-heading-title{padding:0;margin:0;line-height:1}.elementor-widget-heading .elementor-heading-title[class*=elementor-size-]>a{color:inherit;font-size:inherit;line-height:inherit}.elementor-widget-heading .elementor-heading-title.elementor-size-small{font-size:15px}.elementor-widget-heading .elementor-heading-title.elementor-size-medium{font-size:19px}.elementor-widget-heading .elementor-heading-title.elementor-size-large{font-size:29px}.elementor-widget-heading .elementor-heading-title.elementor-size-xl{font-size:39px}.elementor-widget-heading .elementor-heading-title.elementor-size-xxl{font-size:59px}<\/style><h2 class=\"elementor-heading-title elementor-size-xl\">Step 1: Transforming Words into Numbers<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0e90b1c elementor-widget elementor-widget-text-editor\" data-id=\"0e90b1c\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Computers don\u2019t understand words like we do. For them, \u201ccat\u201d or \u201chouse\u201d are just sequences of letters; for a machine to work with words, they need to be <\/span><b>transformed into numbers<\/b><span style=\"font-weight: 400;\">.<\/span><\/p><p><span style=\"font-weight: 400;\">This is the role of Embeddings. Imagine that each word in the language (or almost) receives a unique secret code, which is a list of numbers.<\/span><\/p><p><i><span style=\"font-weight: 400;\">Think of a huge library. Each book (each word for GPT) gets a special label with a unique barcode (the <\/span><\/i><b><i>embedding vector<\/i><\/b><i><span style=\"font-weight: 400;\">). This barcode isn\u2019t just an ID number; it contains hidden information about the book.<\/span><\/i><\/p><p><i><span style=\"font-weight: 400;\">For example, the barcodes of science fiction books might look similar, as do those of cookbooks, and if they are science fiction books that also talk about robots, their barcodes would be even closer!<\/span><\/i><\/p><p><span style=\"font-weight: 400;\">The more two words have a similar meaning, the \u201ccloser\u201d their list of numbers (their <\/span><b>embedding vector<\/b><span style=\"font-weight: 400;\">) will be in an imaginary space. For example, the vectors for \u2018king\u2019 and \u2018queen\u2019 will be very close, and the difference between the \u2018king\u2019 vector and \u2018man\u2019 will be similar to the difference between \u2018queen\u2019 and \u2018woman\u2019! Pretty handy, right?<\/span><\/p><p><span style=\"font-weight: 400;\">The first thing GPT does when you give it text is to look at each word and find its corresponding list of numbers (embedding vector) in its \u201csecret code table.\u201d<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-adb44e1 elementor-widget elementor-widget-image\" data-id=\"adb44e1\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-image{text-align:center}.elementor-widget-image a{display:inline-block}.elementor-widget-image a img[src$=\".svg\"]{width:48px}.elementor-widget-image img{vertical-align:middle;display:inline-block}<\/style>\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"1562\" height=\"702\" src=\"https:\/\/datascientest.com\/en\/files\/2025\/06\/KnowledgeGraphEmbedding.png\" class=\"attachment-full size-full wp-image-196720\" alt=\"\" srcset=\"https:\/\/datascientest.com\/en\/files\/2025\/06\/KnowledgeGraphEmbedding.png 1562w, https:\/\/datascientest.com\/en\/files\/2025\/06\/KnowledgeGraphEmbedding-300x135.png 300w, https:\/\/datascientest.com\/en\/files\/2025\/06\/KnowledgeGraphEmbedding-1024x460.png 1024w, https:\/\/datascientest.com\/en\/files\/2025\/06\/KnowledgeGraphEmbedding-768x345.png 768w, https:\/\/datascientest.com\/en\/files\/2025\/06\/KnowledgeGraphEmbedding-1536x690.png 1536w\" sizes=\"(max-width: 1562px) 100vw, 1562px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-4cd857d e-flex e-con-boxed e-con\" data-id=\"4cd857d\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-4b70941 elementor-widget elementor-widget-heading\" data-id=\"4b70941\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-xl\">Step 2: The Order of Words (Positional Encoding)<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-767fbae elementor-widget elementor-widget-text-editor\" data-id=\"767fbae\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">We now have a list of numbers for each word. However, if we mix up the words, the sentence loses its meaning: \u201cThe cat sleeps\u201d doesn\u2019t mean the same as \u201cSleeps the cat.\u201d I tell you nothing new; word order is crucial!<\/span><\/p><p><span style=\"font-weight: 400;\">The issue is that our <\/span><b>GPT model<\/b><span style=\"font-weight: 400;\"> processes words in parallel (for obvious execution time reasons), thus without considering their order. To solve this, we add extra information to each word\u2019s number list: a <\/span><b>position marker<\/b><span style=\"font-weight: 400;\">.<\/span><\/p><p><span style=\"font-weight: 400;\">These position markers (<\/span><b>Positional Encoding<\/b><span style=\"font-weight: 400;\">) are also lists of numbers, calculated with special mathematical formulas.\u00a0<\/span><\/p><p><i><span style=\"font-weight: 400;\">We won\u2019t go into details here, let\u2019s just note that it works really well!<\/span><\/i><\/p><p><span style=\"font-weight: 400;\">We simply add them to the word&#8217;s Embedding vectors (their own barcode).<\/span><\/p><p><span style=\"font-weight: 400;\">Now the model has a list of embedding vectors for each word, which contains information on our whole sentence; each element of this list contains both the information about the word itself and about where it is in the sentence.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-797f4c8 elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"797f4c8\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-courses\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Learn all about the GPT model<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-d4dac87 e-flex e-con-boxed e-con\" data-id=\"d4dac87\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-105684a elementor-widget elementor-widget-heading\" data-id=\"105684a\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-xl\">Step 3: The Transformer<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-bf8f04f elementor-widget elementor-widget-text-editor\" data-id=\"bf8f04f\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><i><span style=\"font-weight: 400;\">Nothing to do with Optimus Prime, though\u2026<\/span><\/i><\/p><p><span style=\"font-weight: 400;\">The smartest part of GPT is <\/span><span style=\"font-weight: 400;\">the Transformer architecture<\/span><span style=\"font-weight: 400;\">. Think of it as the brain that analyzes the list of numbers of the sentence to understand the context and predict the next word.<\/span><\/p><p><span style=\"font-weight: 400;\">GPT models use a simplified version of the original Transformer, called the <\/span><b>Decoder<\/b><span style=\"font-weight: 400;\">. Why? Because their job is to <\/span><b>generate<\/b><span style=\"font-weight: 400;\"> text, and that\u2019s the role of the Decoder!<\/span><\/p><p><span style=\"font-weight: 400;\">This Transformer is built by stacking several identical &#8220;blocks&#8221; on top of each other. The more blocks there are (the more complex it becomes), the more powerful it is.<\/span><\/p><p><span style=\"font-weight: 400;\">Each block has several steps to process the embedding vectors of our words:<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-454137d elementor-widget elementor-widget-heading\" data-id=\"454137d\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">1. The Principle of Attention<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-100837d elementor-widget elementor-widget-text-editor\" data-id=\"100837d\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">This is the brilliant idea behind the Transformer. When you read a sentence, you don\u2019t give the same importance to all words to understand the meaning. For example, in \u201cThe DataScientest student who had studied well passed their exam,\u201d to understand \u201cpassed,\u201d you focus on \u201cstudent\u201d and \u201cexam.\u201d<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9b2f9d9 elementor-widget elementor-widget-image\" data-id=\"9b2f9d9\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"850\" height=\"765\" src=\"https:\/\/datascientest.com\/en\/files\/2025\/06\/The-Transformer-model-architecture.png\" class=\"attachment-full size-full wp-image-196721\" alt=\"\" srcset=\"https:\/\/datascientest.com\/en\/files\/2025\/06\/The-Transformer-model-architecture.png 850w, https:\/\/datascientest.com\/en\/files\/2025\/06\/The-Transformer-model-architecture-300x270.png 300w, https:\/\/datascientest.com\/en\/files\/2025\/06\/The-Transformer-model-architecture-768x691.png 768w\" sizes=\"(max-width: 850px) 100vw, 850px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7d70ede elementor-widget elementor-widget-text-editor\" data-id=\"7d70ede\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">The <\/span><b>Attention<\/b><span style=\"font-weight: 400;\"> mechanism allows our model to do the same: for each word in the sentence, it looks at all previous words and decides which ones are the most important to understand the current word and predict the next!<\/span><\/p><p><span style=\"font-weight: 400;\">Often, the Transformer uses several \u201c<\/span><b>attention heads<\/b><span style=\"font-weight: 400;\">\u201d in parallel. It\u2019s like having several people reading the sentence at the same time, each focusing on a different type of relation (one on grammar, one on meaning&#8230;), to then combine their analyses.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4b8c64f elementor-widget elementor-widget-heading\" data-id=\"4b8c64f\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">2. Reflection (Or Feed-Forward)<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-bbf6d61 elementor-widget elementor-widget-text-editor\" data-id=\"bbf6d61\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">After the Attention mechanism allows each word to integrate the context of the previous words, each embedding vector independently passes through several layers of mathematical functions that rely on numbers, called <\/span><b>weights<\/b><span style=\"font-weight: 400;\">. This ensemble is called a <a href=\"https:\/\/datascientest.com\/en\/all-about-artificial-neural-networks\">neural network<\/a>.<\/span><\/p><p><span style=\"font-weight: 400;\">This layer allows the model to perform more complex transformations on the information that <\/span><b>Attention<\/b><span style=\"font-weight: 400;\"> has extracted.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-20f86ba elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"20f86ba\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-courses\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Training for AI development<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-a415c30 e-flex e-con-boxed e-con\" data-id=\"a415c30\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-1a59d37 elementor-widget elementor-widget-heading\" data-id=\"1a59d37\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-xl\">Step 4: Rinse and Repeat!<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-dbd81ac elementor-widget elementor-widget-text-editor\" data-id=\"dbd81ac\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">The real power of GPT comes from the fact that it\u2019s not a single Transformer block but several (dozens, even hundreds!) stacked on top of each other.<\/span><\/p><p><span style=\"font-weight: 400;\">Imagine a multi-story factory, on each floor, our embedding vectors are processed by the <\/span><b>Attention and Reflection mechanisms<\/b><span style=\"font-weight: 400;\">. The information that comes out of one floor then becomes the input for the floor below.<\/span><\/p><p><span style=\"font-weight: 400;\">The first layers learn to handle simple relationships between words.<\/span><\/p><p><span style=\"font-weight: 400;\">The intermediate layers combine this information to understand more complex relationships and the structure of sentences. The last layers understand the overall meaning, tone, and style. The information becomes richer as it goes down the floors!<\/span><\/p><p><i><span style=\"font-weight: 400;\">And more and more abstract and incomprehensible to us, poor humans.<\/span><\/i><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-808f228 elementor-widget elementor-widget-image\" data-id=\"808f228\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"2560\" height=\"1707\" src=\"https:\/\/datascientest.com\/en\/files\/2025\/06\/aerps-com-5e4Zlblkvks-unsplash-scaled-1.jpg\" class=\"attachment-full size-full wp-image-196722\" alt=\"\" srcset=\"https:\/\/datascientest.com\/en\/files\/2025\/06\/aerps-com-5e4Zlblkvks-unsplash-scaled-1.jpg 2560w, https:\/\/datascientest.com\/en\/files\/2025\/06\/aerps-com-5e4Zlblkvks-unsplash-scaled-1-300x200.jpg 300w, https:\/\/datascientest.com\/en\/files\/2025\/06\/aerps-com-5e4Zlblkvks-unsplash-scaled-1-1024x683.jpg 1024w, https:\/\/datascientest.com\/en\/files\/2025\/06\/aerps-com-5e4Zlblkvks-unsplash-scaled-1-768x512.jpg 768w, https:\/\/datascientest.com\/en\/files\/2025\/06\/aerps-com-5e4Zlblkvks-unsplash-scaled-1-1536x1024.jpg 1536w, https:\/\/datascientest.com\/en\/files\/2025\/06\/aerps-com-5e4Zlblkvks-unsplash-scaled-1-2048x1366.jpg 2048w\" sizes=\"(max-width: 2560px) 100vw, 2560px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-bae18fa e-flex e-con-boxed e-con\" data-id=\"bae18fa\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-d4bbb90 elementor-widget elementor-widget-heading\" data-id=\"d4bbb90\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-xl\">Step 5: Training<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7826dc2 elementor-widget elementor-widget-text-editor\" data-id=\"7826dc2\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Imagine we give the machine billions of texts (books, articles, web pages&#8230;). We hide the next word in each sentence from it and say: \u201cGuess!\u201d<\/span><\/p><p><span style=\"font-weight: 400;\">This is <\/span><b>Training<\/b><span style=\"font-weight: 400;\">.<\/span><\/p><p><span style=\"font-weight: 400;\">The model tries to predict the next word based on the previous words.<\/span><\/p><p><span style=\"font-weight: 400;\">At first, it makes lots of mistakes, but each time it does, we tell it: \u201cNo, the real word was this one,\u201d and the model then adjusts its internal parameters (<\/span><b>the weights!<\/b><span style=\"font-weight: 400;\">) so that the next time it sees a similar situation, it has a better chance of guessing the right word.<\/span><\/p><p><span style=\"font-weight: 400;\">This error-based adjustment process is called Gradient Descent<\/span><b>.<\/b><\/p><p><span style=\"font-weight: 400;\">By playing this prediction game billions of times on billions of texts, the model learns not only which words often go together but also grammar, syntax, and even different writing styles!<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4b8469c elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"4b8469c\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-courses\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Mastering AI model training<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-81b2559 e-flex e-con-boxed e-con\" data-id=\"81b2559\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-ca52769 elementor-widget elementor-widget-heading\" data-id=\"ca52769\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-xl\">Final Step: Text Generation<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-55b9823 elementor-widget elementor-widget-text-editor\" data-id=\"55b9823\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Once the model is trained, it is ready to generate text, based on an initial prompt to predict associated words, <\/span><b>the prompt!<\/b><\/p><ol><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">The model takes your prompt, transforms it into lists of numbers, and sends them through all its Transformer blocks.<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">It outputs a list of probabilities for <\/span><b>each possible word<\/b><span style=\"font-weight: 400;\"> in the vocabulary. For example, after \u2018<\/span><i><span style=\"font-weight: 400;\">Once upon a time\u2026<\/span><\/i><span style=\"font-weight: 400;\">,&#8217; the word \u2018<\/span><i><span style=\"font-weight: 400;\">there was<\/span><\/i><span style=\"font-weight: 400;\">&#8216; has an 80% chance, \u2018<\/span><i><span style=\"font-weight: 400;\">once<\/span><\/i><span style=\"font-weight: 400;\">&#8216; has 10%, \u2018<\/span><i><span style=\"font-weight: 400;\">the<\/span><\/i><span style=\"font-weight: 400;\">&#8216; has 5%, \u2018<\/span><i><span style=\"font-weight: 400;\">in<\/span><\/i><span style=\"font-weight: 400;\">&#8216; has 3%\u2026<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">The model then chooses a word from this list of possibilities. It doesn\u2019t always pick the most probable to ensure the text isn\u2019t too repetitive. This word is added to the sequence: \u201c<\/span><i><span style=\"font-weight: 400;\">Once upon a time there was<\/span><\/i><span style=\"font-weight: 400;\">.\u201d<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">The model takes this new sequence as input and repeats the process: it predicts the next word (\u201c<\/span><i><span style=\"font-weight: 400;\">knight<\/span><\/i><span style=\"font-weight: 400;\">\u201d? \u201c<\/span><i><span style=\"font-weight: 400;\">cat<\/span><\/i><span style=\"font-weight: 400;\">\u201d? \u201c<\/span><i><span style=\"font-weight: 400;\">day<\/span><\/i><span style=\"font-weight: 400;\">\u201d?), chooses a word, adds it to the sequence\u2026<\/span><\/li><\/ol>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d1c2109 elementor-widget elementor-widget-text-editor\" data-id=\"d1c2109\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">And it continues, word by word, until it generates a special word that means \u201cend of sentence\u201d or \u201cend of text,\u201d or until it reaches a maximum length!<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-629ed7c elementor-widget elementor-widget-image\" data-id=\"629ed7c\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"2560\" height=\"1922\" src=\"https:\/\/datascientest.com\/en\/files\/2025\/06\/shantanu-kumar-xvdkNBaja90-unsplash-1-scaled-1.jpg\" class=\"attachment-full size-full wp-image-196723\" alt=\"\" srcset=\"https:\/\/datascientest.com\/en\/files\/2025\/06\/shantanu-kumar-xvdkNBaja90-unsplash-1-scaled-1.jpg 2560w, https:\/\/datascientest.com\/en\/files\/2025\/06\/shantanu-kumar-xvdkNBaja90-unsplash-1-scaled-1-300x225.jpg 300w, https:\/\/datascientest.com\/en\/files\/2025\/06\/shantanu-kumar-xvdkNBaja90-unsplash-1-scaled-1-1024x769.jpg 1024w, https:\/\/datascientest.com\/en\/files\/2025\/06\/shantanu-kumar-xvdkNBaja90-unsplash-1-scaled-1-768x577.jpg 768w, https:\/\/datascientest.com\/en\/files\/2025\/06\/shantanu-kumar-xvdkNBaja90-unsplash-1-scaled-1-1536x1153.jpg 1536w, https:\/\/datascientest.com\/en\/files\/2025\/06\/shantanu-kumar-xvdkNBaja90-unsplash-1-scaled-1-2048x1538.jpg 2048w\" sizes=\"(max-width: 2560px) 100vw, 2560px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-93c5fff e-flex e-con-boxed e-con\" data-id=\"93c5fff\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-f6a88cd elementor-widget elementor-widget-heading\" data-id=\"f6a88cd\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-xl\">Conclusion: This Works Really Well!<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d99a5b7 elementor-widget elementor-widget-text-editor\" data-id=\"d99a5b7\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">The power of GPT comes from the combination of several elements:<\/span><\/p><ul><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">The <\/span><b>Transformer<\/b><span style=\"font-weight: 400;\"> architecture and the <\/span><b>Attention<\/b><span style=\"font-weight: 400;\"> mechanism, which allow it to understand context over very long sentences.<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">The <\/span><b>layer stacking<\/b><span style=\"font-weight: 400;\"> that allows it to learn increasingly complex representations of language.<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">The <\/span><b>massive training<\/b><span style=\"font-weight: 400;\"> on gigantic amounts of text, giving it a very broad knowledge of language and the world.<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">The <\/span><b>word-by-word generation<\/b><span style=\"font-weight: 400;\"> process based on prediction, allowing it to create fluid and varied text.<\/span><\/li><\/ul>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b9d4fd2 elementor-widget elementor-widget-text-editor\" data-id=\"b9d4fd2\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">GPT doesn\u2019t \u201cthink\u201d: It is extremely skilled at identifying <\/span><b>complex statistical patterns in language<\/b><span style=\"font-weight: 400;\"> and uses them to predict the most likely continuation of a sequence of words.<\/span><\/p><p><span style=\"font-weight: 400;\">But the result of this prediction, thanks to the scale of the model and the data used for training, is often text that seems intelligent, relevant, and creative to us!<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c037794 elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"c037794\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-courses\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Discover our AI training courses<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-722fc31 e-flex e-con-boxed e-con\" data-id=\"722fc31\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-04c6e09 elementor-widget elementor-widget-html\" data-id=\"04c6e09\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<script type=\"application\/ld+json\">\r\n{\r\n  \"@context\": \"https:\/\/schema.org\",\r\n  \"@type\": \"FAQPage\",\r\n  \"mainEntity\": [\r\n    {\r\n      \"@type\": \"Question\",\r\n      \"name\": \"Step 1: Transforming Words into Numbers\",\r\n      \"acceptedAnswer\": {\r\n        \"@type\": \"Answer\",\r\n        \"text\": \"Computers don\u2019t understand words like we do. For them, words need to be transformed into numbers using embeddings.\"\r\n      }\r\n    },\r\n    {\r\n      \"@type\": \"Question\",\r\n      \"name\": \"Step 2: The Order of Words (Positional Encoding)\",\r\n      \"acceptedAnswer\": {\r\n        \"@type\": \"Answer\",\r\n        \"text\": \"If we mix up the words, the sentence loses its meaning, so we add extra information to each word\u2019s vector: a position marker.\"\r\n      }\r\n    },\r\n    {\r\n      \"@type\": \"Question\",\r\n      \"name\": \"Step 3: The Transformer\",\r\n      \"acceptedAnswer\": {\r\n        \"@type\": \"Answer\",\r\n        \"text\": \"The smartest part of GPT is the Transformer architecture, which analyzes word vectors to understand context and predict the next word.\"\r\n      }\r\n    },\r\n    {\r\n      \"@type\": \"Question\",\r\n      \"name\": \"Step 4: Rinse and Repeat!\",\r\n      \"acceptedAnswer\": {\r\n        \"@type\": \"Answer\",\r\n        \"text\": \"GPT consists of multiple Transformer blocks stacked together, processing information layer by layer to capture increasingly complex patterns.\"\r\n      }\r\n    },\r\n    {\r\n      \"@type\": \"Question\",\r\n      \"name\": \"Step 5: Training\",\r\n      \"acceptedAnswer\": {\r\n        \"@type\": \"Answer\",\r\n        \"text\": \"We hide the next word from the model and ask it to guess; it learns from its errors by adjusting internal parameters using gradient descent.\"\r\n      }\r\n    },\r\n    {\r\n      \"@type\": \"Question\",\r\n      \"name\": \"Final Step: Text Generation\",\r\n      \"acceptedAnswer\": {\r\n        \"@type\": \"Answer\",\r\n        \"text\": \"The model takes your prompt, generates probabilities for the next word, picks one, and repeats this process to build sentences.\"\r\n      }\r\n    }\r\n  ]\r\n}\r\n<\/script>\r\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Here we are not talking about ChatGPT, but that&#8217;s indeed where it got its name. If you\u2019ve never heard of GPT, you have at least used text-generating AIs, or &#8216;LLM&#8216; (Large Language Model). GPT is a type of LLM, on which most of the generative AIs that everyone is talking about right now are based! [&hellip;]<\/p>\n","protected":false},"author":45,"featured_media":196719,"comment_status":"open","ping_status":"open","sticky":false,"template":"elementor_theme","format":"standard","meta":{"content-type":"","inline_featured_image":false},"categories":[2427],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.7 (Yoast SEO v21.7) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>How does the GPT model work?<\/title>\n<meta name=\"description\" content=\"Discover what lies behind the acronym GPT: how this predictive model transforms words into numbers, learns to understand human language, and generates text with astonishing fluency. An accessible dive into the inner workings of AI!\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How does the GPT model work?\" \/>\n<meta property=\"og:description\" content=\"Here we are not talking about ChatGPT, but that&#039;s indeed where it got its name. If you\u2019ve never heard of GPT, you have at least used text-generating AIs,\" \/>\n<meta property=\"og:url\" content=\"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works\" \/>\n<meta property=\"og:site_name\" content=\"DataScientest\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataScientestEN\" \/>\n<meta property=\"article:published_time\" content=\"2025-08-07T05:18:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-07-30T23:19:18+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/datascientest.com\/en\/files\/2025\/06\/fonctionnement-GPT-datascientest.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"1536\" \/>\n\t<meta property=\"og:image:height\" content=\"974\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"author\" content=\"Daniel\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:site\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Daniel\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works#article\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works\"},\"author\":{\"name\":\"Daniel\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/5a86e30c5090234fb43dd8baa83c44b0\"},\"headline\":\"How does the GPT model work?\",\"datePublished\":\"2025-08-07T05:18:00+00:00\",\"dateModified\":\"2025-07-30T23:19:18+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works\"},\"wordCount\":1465,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"articleSection\":[\"Artificial Intelligence\"],\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works#respond\"]}],\"copyrightYear\":\"2025\",\"copyrightHolder\":{\"@id\":\"https:\/\/datascientest.com\/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works\",\"url\":\"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works\",\"name\":\"How does the GPT model work?\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/#website\"},\"datePublished\":\"2025-08-07T05:18:00+00:00\",\"dateModified\":\"2025-07-30T23:19:18+00:00\",\"description\":\"Discover what lies behind the acronym GPT: how this predictive model transforms words into numbers, learns to understand human language, and generates text with astonishing fluency. An accessible dive into the inner workings of AI!\",\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works\"]}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/datascientest.com\/en\/#website\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"name\":\"DataScientest\",\"description\":\"Leader in Data Science Courses\",\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/datascientest.com\/en\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/datascientest.com\/en\/#organization\",\"name\":\"DataScientest\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"contentUrl\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"width\":301,\"height\":301,\"caption\":\"DataScientest\"},\"image\":{\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/DataScientestEN\",\"https:\/\/twitter.com\/DataScientest__\",\"https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/\",\"https:\/\/www.tiktok.com\/@datascientest\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/5a86e30c5090234fb43dd8baa83c44b0\",\"name\":\"Daniel\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/837acad4f2dc47e85d81cce837261670?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/837acad4f2dc47e85d81cce837261670?s=96&d=mm&r=g\",\"caption\":\"Daniel\"},\"sameAs\":[\"https:\/\/www.linkedin.com\/school\/datascientest\/\",\"user\"],\"jobTitle\":\"ing\u00e9nieur logiciel\",\"worksFor\":\"DataScientest\",\"url\":\"https:\/\/datascientest.com\/en\/author\/team-redac\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"How does the GPT model work?","description":"Discover what lies behind the acronym GPT: how this predictive model transforms words into numbers, learns to understand human language, and generates text with astonishing fluency. An accessible dive into the inner workings of AI!","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works","og_locale":"en_GB","og_type":"article","og_title":"How does the GPT model work?","og_description":"Here we are not talking about ChatGPT, but that's indeed where it got its name. If you\u2019ve never heard of GPT, you have at least used text-generating AIs,","og_url":"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works","og_site_name":"DataScientest","article_publisher":"https:\/\/www.facebook.com\/DataScientestEN","article_published_time":"2025-08-07T05:18:00+00:00","article_modified_time":"2025-07-30T23:19:18+00:00","og_image":[{"width":1536,"height":974,"url":"https:\/\/datascientest.com\/en\/files\/2025\/06\/fonctionnement-GPT-datascientest.webp","type":"image\/webp"}],"author":"Daniel","twitter_card":"summary_large_image","twitter_creator":"@DataScientest__","twitter_site":"@DataScientest__","twitter_misc":{"Written by":"Daniel","Estimated reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works#article","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works"},"author":{"name":"Daniel","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/5a86e30c5090234fb43dd8baa83c44b0"},"headline":"How does the GPT model work?","datePublished":"2025-08-07T05:18:00+00:00","dateModified":"2025-07-30T23:19:18+00:00","mainEntityOfPage":{"@id":"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works"},"wordCount":1465,"commentCount":0,"publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"articleSection":["Artificial Intelligence"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works#respond"]}],"copyrightYear":"2025","copyrightHolder":{"@id":"https:\/\/datascientest.com\/#organization"}},{"@type":"WebPage","@id":"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works","url":"https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works","name":"How does the GPT model work?","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/#website"},"datePublished":"2025-08-07T05:18:00+00:00","dateModified":"2025-07-30T23:19:18+00:00","description":"Discover what lies behind the acronym GPT: how this predictive model transforms words into numbers, learns to understand human language, and generates text with astonishing fluency. An accessible dive into the inner workings of AI!","inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/datascientest.com\/en\/all-about-how-the-gpt-model-works"]}]},{"@type":"WebSite","@id":"https:\/\/datascientest.com\/en\/#website","url":"https:\/\/datascientest.com\/en\/","name":"DataScientest","description":"Leader in Data Science Courses","publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/datascientest.com\/en\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/datascientest.com\/en\/#organization","name":"DataScientest","url":"https:\/\/datascientest.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","contentUrl":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","width":301,"height":301,"caption":"DataScientest"},"image":{"@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataScientestEN","https:\/\/twitter.com\/DataScientest__","https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/","https:\/\/www.tiktok.com\/@datascientest"]},{"@type":"Person","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/5a86e30c5090234fb43dd8baa83c44b0","name":"Daniel","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/837acad4f2dc47e85d81cce837261670?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/837acad4f2dc47e85d81cce837261670?s=96&d=mm&r=g","caption":"Daniel"},"sameAs":["https:\/\/www.linkedin.com\/school\/datascientest\/","user"],"jobTitle":"ing\u00e9nieur logiciel","worksFor":"DataScientest","url":"https:\/\/datascientest.com\/en\/author\/team-redac"}]}},"_links":{"self":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/196717"}],"collection":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/users\/45"}],"replies":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/comments?post=196717"}],"version-history":[{"count":10,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/196717\/revisions"}],"predecessor-version":[{"id":198483,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/196717\/revisions\/198483"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media\/196719"}],"wp:attachment":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media?parent=196717"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/categories?post=196717"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/tags?post=196717"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}