{"id":186338,"date":"2024-06-24T06:30:00","date_gmt":"2024-06-24T05:30:00","guid":{"rendered":"https:\/\/datascientest.com\/en\/?p=186338"},"modified":"2024-06-22T12:16:42","modified_gmt":"2024-06-22T11:16:42","slug":"all-about-generated-pre-trained-transformers","status":"publish","type":"post","link":"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers","title":{"rendered":"What are Generated Pre-trained Transformers (GPTs)?"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"186338\" class=\"elementor elementor-186338\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-bf3d956 e-flex e-con-boxed e-con\" data-id=\"bf3d956\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-4a5c08a elementor-widget elementor-widget-heading\" data-id=\"4a5c08a\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-heading-title{padding:0;margin:0;line-height:1}.elementor-widget-heading .elementor-heading-title[class*=elementor-size-]>a{color:inherit;font-size:inherit;line-height:inherit}.elementor-widget-heading .elementor-heading-title.elementor-size-small{font-size:15px}.elementor-widget-heading .elementor-heading-title.elementor-size-medium{font-size:19px}.elementor-widget-heading .elementor-heading-title.elementor-size-large{font-size:29px}.elementor-widget-heading .elementor-heading-title.elementor-size-xl{font-size:39px}.elementor-widget-heading .elementor-heading-title.elementor-size-xxl{font-size:59px}<\/style><p class=\"elementor-heading-title elementor-size-default\">Able to write emails for you, translate texts into different languages, code, compose poems, and more. It's impossible to ignore ChatGPT, the generative AI from OpenAI. But are you aware of the technology it's based on?<\/p>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-bb74b40 elementor-widget elementor-widget-text-editor\" data-id=\"bb74b40\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-text-editor.elementor-drop-cap-view-stacked .elementor-drop-cap{background-color:#69727d;color:#fff}.elementor-widget-text-editor.elementor-drop-cap-view-framed .elementor-drop-cap{color:#69727d;border:3px solid;background-color:transparent}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap{margin-top:8px}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap-letter{width:1em;height:1em}.elementor-widget-text-editor .elementor-drop-cap{float:left;text-align:center;line-height:1;font-size:50px}.elementor-widget-text-editor .elementor-drop-cap-letter{display:inline-block}<\/style>\t\t\t\t<p><span style=\"font-weight: 400;\">The Generative Pre-trained Transformer is an AI model excelling in tasks involving natural language processing. <\/span><b>Explore pre-trained generative transformers,<\/b><span style=\"font-weight: 400;\"> understand how they function, their benefits, and their limitations.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-20f184c e-flex e-con-boxed e-con\" data-id=\"20f184c\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-d0df26f elementor-widget elementor-widget-heading\" data-id=\"d0df26f\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">What is a pre-trained generative transformer (GPT)?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-342faac elementor-widget elementor-widget-text-editor\" data-id=\"342faac\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">The pre-trained generative transformer encompasses a range of <a href=\"https:\/\/datascientest.com\/en\/recurrent-neural-network-what-is-it\">recurrent neural network<\/a> models <\/span><b>employing transformer architecture<\/b><span style=\"font-weight: 400;\">. This technology marks a significant milestone in the realm of generative artificial intelligence. The widespread adoption of ChatGPT is a testament to this. Since its introduction, all leading tech corporations have been vying to develop the most effective <a href=\"https:\/\/datascientest.com\/en\/large-language-models-llm-everything-you-need-to-know\">language model<\/a>, aiming for a human-like experience.<\/span><\/p><p><span style=\"font-weight: 400;\">Why such enthusiasm? Because this machine learning model is adept at <\/span><b>executing various tasks associated with natural language processing<\/b><span style=\"font-weight: 400;\">. Its ability to simulate conversations between humans is sometimes eerily accurate. From comprehending queries to generating assorted types of coherent and relevant texts, it enables the simulation of a conversation with a human (almost).<\/span><\/p><p><span style=\"font-weight: 400;\">In doing so, users can automate a myriad of tasks: linguistic translation, document summarization, <a href=\"https:\/\/datascientest.com\/en\/empowering-natural-language-processing-with-hugging-face-transformers-api\">blog article creation<\/a>, social media content ideas, writing code, and even crafting poetry. There&#8217;s no need to spend countless hours on research, planning, and drafting; pre-trained generative transformers can handle these tasks in seconds.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-31d115e elementor-widget elementor-widget-image\" data-id=\"31d115e\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-image{text-align:center}.elementor-widget-image a{display:inline-block}.elementor-widget-image a img[src$=\".svg\"]{width:48px}.elementor-widget-image img{vertical-align:middle;display:inline-block}<\/style>\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"800\" height=\"457\" src=\"https:\/\/datascientest.com\/en\/files\/2024\/06\/Generated-Pre-trained-Transformer-GPT1.jpg\" class=\"attachment-full size-full wp-image-186344\" alt=\"\" srcset=\"https:\/\/datascientest.com\/en\/files\/2024\/06\/Generated-Pre-trained-Transformer-GPT1.jpg 800w, https:\/\/datascientest.com\/en\/files\/2024\/06\/Generated-Pre-trained-Transformer-GPT1-300x171.jpg 300w, https:\/\/datascientest.com\/en\/files\/2024\/06\/Generated-Pre-trained-Transformer-GPT1-768x439.jpg 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e05ebe1 elementor-align-center button-principal elementor-widget elementor-widget-button\" data-id=\"e05ebe1\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-md\" href=\"https:\/\/datascientest.com\/en\/machine-learning-engineer-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Learn to develop an AI model<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2acd53f elementor-widget elementor-widget-text-editor\" data-id=\"2acd53f\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><b><span style=\"text-decoration: underline;\">Good to know<\/span>:<\/b><span style=\"font-weight: 400;\"> The transformer neural network architecture isn&#8217;t entirely new. It emerged from various research efforts on natural language processing and deep learning, with the term first being introduced in the 2017 paper &#8220;Attention is All You Need&#8221;.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-a033a9d e-flex e-con-boxed e-con\" data-id=\"a033a9d\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-16e2f9a elementor-widget elementor-widget-heading\" data-id=\"16e2f9a\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">How do pre-trained models work?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-459ae69 elementor-widget elementor-widget-text-editor\" data-id=\"459ae69\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">To achieve editorial feats comparable to (or sometimes surpassing) human capabilities, the pre-trained generative transformer relies on &#8220;transformer&#8221; neural network architecture.<\/span><\/p><p><span style=\"font-weight: 400;\">It employs auto-regressive attention (or self-attention mechanism). This AI model <\/span><b>considers not just the last word to generate text but the overall context<\/b><span style=\"font-weight: 400;\">. It can assign varying degrees of significance to words, thereby better discerning the relationships among words and sentences.<\/span><\/p><p><span style=\"font-weight: 400;\">Ultimately, it&#8217;s this interplay of words and sentences that enables the GPT to comprehend the user&#8217;s query and deliver a <\/span><b>coherent response, both in content and form<\/b><span style=\"font-weight: 400;\">.<\/span><\/p><p><span style=\"font-weight: 400;\">Initially, the GPT model was pre-trained with extensive textual data to grasp the structure, syntax, and nuances of language. Only after achieving a solid understanding of human language was the model further trained to carry out specific tasks.<\/span><\/p><p><b><span style=\"text-decoration: underline;\">Good to know<\/span>:<\/b><span style=\"font-weight: 400;\"> Although pre-trained generative transformers generate human-like results, they remain machines. They analyze user queries and then predict the most suitable response based on their contextual understanding.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b882fa7 elementor-widget elementor-widget-image\" data-id=\"b882fa7\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"800\" height=\"457\" src=\"https:\/\/datascientest.com\/en\/files\/2024\/06\/Generated-Pre-trained-Transformer-GPT2.jpg\" class=\"attachment-full size-full wp-image-186354\" alt=\"\" srcset=\"https:\/\/datascientest.com\/en\/files\/2024\/06\/Generated-Pre-trained-Transformer-GPT2.jpg 800w, https:\/\/datascientest.com\/en\/files\/2024\/06\/Generated-Pre-trained-Transformer-GPT2-300x171.jpg 300w, https:\/\/datascientest.com\/en\/files\/2024\/06\/Generated-Pre-trained-Transformer-GPT2-768x439.jpg 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c235647 elementor-align-center button-principal elementor-widget elementor-widget-button\" data-id=\"c235647\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-md\" href=\"https:\/\/datascientest.com\/en\/deep-learning-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Mastering the use of generative AI<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-60ecaa0 e-flex e-con-boxed e-con\" data-id=\"60ecaa0\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-d03869d elementor-widget elementor-widget-heading\" data-id=\"d03869d\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">What are GPTs used for?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9acf452 elementor-widget elementor-widget-text-editor\" data-id=\"9acf452\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">With increasing sophistication, pre-trained generative transformers can execute a broad spectrum of tasks. Below are some of their most common applications:<\/span><\/p><ul><li><b>Text generation<\/b><span style=\"font-weight: 400;\">: capable of crafting blog articles, social media posts, video scripts, emails, programming code, and more, in a variety of styles. Simply specify the desired outcome.<\/span><\/li><li><b>Automatic translation<\/b><span style=\"font-weight: 400;\">: trained on billions of pieces of textual data, they can translate text into any language.<\/span><\/li><li><b>Creation of sophisticated chatbots<\/b><span style=\"font-weight: 400;\">: acting as virtual assistants, they can <a href=\"https:\/\/datascientest.com\/en\/autogpt-discover-the-new-tool-that-makes-chatgpt-autonomous\">answer any question posed by users<\/a>.<\/span><\/li><li><b>Summary extraction<\/b><span style=\"font-weight: 400;\">: given lengthy texts, they can generate concise summaries of around a hundred words.<\/span><\/li><li><b>Data analysis<\/b><span style=\"font-weight: 400;\">: they can analyze vast datasets and transcribe them into tables or spreadsheets. Some tools even offer graphical representations.<\/span><\/li><\/ul>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9f44422 elementor-widget elementor-widget-text-editor\" data-id=\"9f44422\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">For users, the real advantage of pre-trained generative transformers is their <\/span><b>speed of execution<\/b><span style=\"font-weight: 400;\">. They accomplish in seconds what would take a human hours, thus significantly boosting productivity.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-320528c e-flex e-con-boxed e-con\" data-id=\"320528c\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-601f8e0 elementor-widget elementor-widget-heading\" data-id=\"601f8e0\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">What are the limitations of pre-trained generative transformers?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-19e07ea elementor-widget elementor-widget-text-editor\" data-id=\"19e07ea\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Despite their utility and impressive efficiency, pre-trained generative transformers have their shortcomings, notably due to the training datasets. These may <\/span><b>contain biases<\/b><span style=\"font-weight: 400;\"> &#8211; be they sexist, racist, homophobic, etc. If these biases are incorporated into the model, it will replicate them in its outputs.<\/span><\/p><p><span style=\"font-weight: 400;\">Therefore, it&#8217;s essential to approach its responses with caution. Ideally, verify the sources of the information (if the model provides them).<\/span><\/p><p><span style=\"font-weight: 400;\">To mitigate these biases, it&#8217;s imperative to continually refine the models by feeding them unbiased data. This is a critical task for data scientists. If you&#8217;re keen on <a href=\"https:\/\/datascientest.com\/en\/deep-learning-course\">training the next GPT<\/a> to yield better results, <a href=\"https:\/\/datascientest.com\/en\/data-scientist-course\">consider getting trained in data science<\/a>.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6bc2bdc elementor-widget elementor-widget-image\" data-id=\"6bc2bdc\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"800\" height=\"457\" src=\"https:\/\/datascientest.com\/en\/files\/2024\/06\/Generated-Pre-trained-Transformer-GPT3.jpg\" class=\"attachment-full size-full wp-image-186361\" alt=\"\" srcset=\"https:\/\/datascientest.com\/en\/files\/2024\/06\/Generated-Pre-trained-Transformer-GPT3.jpg 800w, https:\/\/datascientest.com\/en\/files\/2024\/06\/Generated-Pre-trained-Transformer-GPT3-300x171.jpg 300w, https:\/\/datascientest.com\/en\/files\/2024\/06\/Generated-Pre-trained-Transformer-GPT3-768x439.jpg 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ffc6225 elementor-align-center button-principal elementor-widget elementor-widget-button\" data-id=\"ffc6225\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-md\" href=\"https:\/\/datascientest.com\/en\/data-courses\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Training with DataScientest<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-61e9fbe elementor-widget elementor-widget-html\" data-id=\"61e9fbe\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<script type=\"application\/ld+json\">\r\n{\r\n  \"@context\": \"https:\/\/schema.org\",\r\n  \"@type\": \"FAQPage\",\r\n  \"mainEntity\": [{\r\n    \"@type\": \"Question\",\r\n    \"name\": \"What is a GPT?\",\r\n    \"acceptedAnswer\": {\r\n      \"@type\": \"Answer\",\r\n      \"text\": \"The pre-trained generative transformer encompasses a range of recurrent neural network models employing transformer architecture. This technology marks a significant milestone in the realm of generative artificial intelligence. The widespread adoption of ChatGPT is a testament to this.\"\r\n    }\r\n  },{\r\n    \"@type\": \"Question\",\r\n    \"name\": \"How does a GPT work?\",\r\n    \"acceptedAnswer\": {\r\n      \"@type\": \"Answer\",\r\n      \"text\": \"To achieve editorial feats comparable to (or sometimes surpassing) human capabilities, the pre-trained generative transformer relies on \"transformer\" neural network architecture. It employs auto-regressive attention (or self-attention mechanism). This AI model considers not just the last word to generate text but the overall context.\"\r\n    }\r\n  },{\r\n    \"@type\": \"Question\",\r\n    \"name\": \"What are GPTs used for?\",\r\n    \"acceptedAnswer\": {\r\n      \"@type\": \"Answer\",\r\n      \"text\": \"With increasing sophistication, pre-trained generative transformers can execute a broad spectrum of tasks. Below are some of their most common applications: \\n - Text generation \\n - Automatic translation \\n - Creation of sophisticated chatbots \\n - Summary extraction \\n - Data analysis\"\r\n    }\r\n  },{\r\n    \"@type\": \"Question\",\r\n    \"name\": \"What are the limitations of GPT?\",\r\n    \"acceptedAnswer\": {\r\n      \"@type\": \"Answer\",\r\n      \"text\": \"Despite their utility and impressive efficiency, pre-trained generative transformers have their shortcomings, notably due to the training datasets. These may contain biases - be they sexist, racist, homophobic, etc.\"\r\n    }\r\n  }]\r\n}\r\n<\/script>\r\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Able to write emails for you, translate texts into different languages, code, compose poems, and more. It&#8217;s impossible to ignore ChatGPT, the generative AI from OpenAI. But are you aware of the technology it&#8217;s based on? The Generative Pre-trained Transformer is an AI model excelling in tasks involving natural language processing. Explore pre-trained generative transformers, [&hellip;]<\/p>\n","protected":false},"author":45,"featured_media":186340,"comment_status":"open","ping_status":"open","sticky":false,"template":"elementor_theme","format":"standard","meta":{"content-type":"","inline_featured_image":false},"categories":[2427],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.7 (Yoast SEO v21.7) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>What are Generated Pre-trained Transformers (GPTs)?<\/title>\n<meta name=\"description\" content=\"The pre-trained generative transformer is the technology used by ChatGPT. Discover its operation, its uses, its limitations...\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"What are Generated Pre-trained Transformers (GPTs)?\" \/>\n<meta property=\"og:description\" content=\"Able to write emails for you, translate texts into different languages, code, compose poems, and more. It&#039;s impossible to ignore ChatGPT, the generative\" \/>\n<meta property=\"og:url\" content=\"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers\" \/>\n<meta property=\"og:site_name\" content=\"Data Science Courses | DataScientest\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataScientestEN\" \/>\n<meta property=\"article:published_time\" content=\"2024-06-24T05:30:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-06-22T11:16:42+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/datascientest.com\/en\/files\/2024\/06\/Generated-Pre-trained-Transformer-GPT.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"686\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Daniel\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:site\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Daniel\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers#article\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers\"},\"author\":{\"name\":\"Daniel\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/5a86e30c5090234fb43dd8baa83c44b0\"},\"headline\":\"What are Generated Pre-trained Transformers (GPTs)?\",\"datePublished\":\"2024-06-24T05:30:00+00:00\",\"dateModified\":\"2024-06-22T11:16:42+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers\"},\"wordCount\":716,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"articleSection\":[\"Artificial Intelligence\"],\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers#respond\"]}],\"copyrightYear\":\"2024\",\"copyrightHolder\":{\"@id\":\"https:\/\/datascientest.com\/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers\",\"url\":\"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers\",\"name\":\"What are Generated Pre-trained Transformers (GPTs)?\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/#website\"},\"datePublished\":\"2024-06-24T05:30:00+00:00\",\"dateModified\":\"2024-06-22T11:16:42+00:00\",\"description\":\"The pre-trained generative transformer is the technology used by ChatGPT. Discover its operation, its uses, its limitations...\",\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers\"]}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/datascientest.com\/en\/#website\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"name\":\"Data Science Courses | DataScientest\",\"description\":\"Leader in Data Science Courses\",\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/datascientest.com\/en\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/datascientest.com\/en\/#organization\",\"name\":\"DataScientest\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"contentUrl\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"width\":301,\"height\":301,\"caption\":\"DataScientest\"},\"image\":{\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/DataScientestEN\",\"https:\/\/twitter.com\/DataScientest__\",\"https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/\",\"https:\/\/www.tiktok.com\/@datascientest\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/5a86e30c5090234fb43dd8baa83c44b0\",\"name\":\"Daniel\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/837acad4f2dc47e85d81cce837261670?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/837acad4f2dc47e85d81cce837261670?s=96&d=mm&r=g\",\"caption\":\"Daniel\"},\"sameAs\":[\"https:\/\/www.linkedin.com\/school\/datascientest\/\",\"user\"],\"jobTitle\":\"ing\u00e9nieur logiciel\",\"worksFor\":\"DataScientest\",\"url\":\"https:\/\/datascientest.com\/en\/author\/team-redac\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"What are Generated Pre-trained Transformers (GPTs)?","description":"The pre-trained generative transformer is the technology used by ChatGPT. Discover its operation, its uses, its limitations...","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers","og_locale":"en_GB","og_type":"article","og_title":"What are Generated Pre-trained Transformers (GPTs)?","og_description":"Able to write emails for you, translate texts into different languages, code, compose poems, and more. It's impossible to ignore ChatGPT, the generative","og_url":"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers","og_site_name":"Data Science Courses | DataScientest","article_publisher":"https:\/\/www.facebook.com\/DataScientestEN","article_published_time":"2024-06-24T05:30:00+00:00","article_modified_time":"2024-06-22T11:16:42+00:00","og_image":[{"width":1200,"height":686,"url":"https:\/\/datascientest.com\/en\/files\/2024\/06\/Generated-Pre-trained-Transformer-GPT.jpg","type":"image\/jpeg"}],"author":"Daniel","twitter_card":"summary_large_image","twitter_creator":"@DataScientest__","twitter_site":"@DataScientest__","twitter_misc":{"Written by":"Daniel","Estimated reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers#article","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers"},"author":{"name":"Daniel","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/5a86e30c5090234fb43dd8baa83c44b0"},"headline":"What are Generated Pre-trained Transformers (GPTs)?","datePublished":"2024-06-24T05:30:00+00:00","dateModified":"2024-06-22T11:16:42+00:00","mainEntityOfPage":{"@id":"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers"},"wordCount":716,"commentCount":0,"publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"articleSection":["Artificial Intelligence"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers#respond"]}],"copyrightYear":"2024","copyrightHolder":{"@id":"https:\/\/datascientest.com\/#organization"}},{"@type":"WebPage","@id":"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers","url":"https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers","name":"What are Generated Pre-trained Transformers (GPTs)?","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/#website"},"datePublished":"2024-06-24T05:30:00+00:00","dateModified":"2024-06-22T11:16:42+00:00","description":"The pre-trained generative transformer is the technology used by ChatGPT. Discover its operation, its uses, its limitations...","inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/datascientest.com\/en\/all-about-generated-pre-trained-transformers"]}]},{"@type":"WebSite","@id":"https:\/\/datascientest.com\/en\/#website","url":"https:\/\/datascientest.com\/en\/","name":"Data Science Courses | DataScientest","description":"Leader in Data Science Courses","publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/datascientest.com\/en\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/datascientest.com\/en\/#organization","name":"DataScientest","url":"https:\/\/datascientest.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","contentUrl":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","width":301,"height":301,"caption":"DataScientest"},"image":{"@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataScientestEN","https:\/\/twitter.com\/DataScientest__","https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/","https:\/\/www.tiktok.com\/@datascientest"]},{"@type":"Person","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/5a86e30c5090234fb43dd8baa83c44b0","name":"Daniel","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/837acad4f2dc47e85d81cce837261670?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/837acad4f2dc47e85d81cce837261670?s=96&d=mm&r=g","caption":"Daniel"},"sameAs":["https:\/\/www.linkedin.com\/school\/datascientest\/","user"],"jobTitle":"ing\u00e9nieur logiciel","worksFor":"DataScientest","url":"https:\/\/datascientest.com\/en\/author\/team-redac"}]}},"_links":{"self":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/186338"}],"collection":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/users\/45"}],"replies":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/comments?post=186338"}],"version-history":[{"count":0,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/186338\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media\/186340"}],"wp:attachment":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media?parent=186338"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/categories?post=186338"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/tags?post=186338"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}