{"id":179078,"date":"2024-04-11T19:21:16","date_gmt":"2024-04-11T18:21:16","guid":{"rendered":"https:\/\/datascientest.com\/en\/?p=179078"},"modified":"2024-02-17T21:20:06","modified_gmt":"2024-02-17T20:20:06","slug":"q-learning-machine-learning-with-reinforcement-learning","status":"publish","type":"post","link":"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning","title":{"rendered":"Q-learning &#8211; Machine Learning with reinforcement learning"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"179078\" class=\"elementor elementor-179078\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-0c7a710 e-flex e-con-boxed e-con\" data-id=\"0c7a710\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t<div class=\"elementor-element elementor-element-1270b9a e-con-full e-flex e-con\" data-id=\"1270b9a\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;full&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-4666e04 elementor-widget elementor-widget-heading\" data-id=\"4666e04\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-heading-title{padding:0;margin:0;line-height:1}.elementor-widget-heading .elementor-heading-title[class*=elementor-size-]>a{color:inherit;font-size:inherit;line-height:inherit}.elementor-widget-heading .elementor-heading-title.elementor-size-small{font-size:15px}.elementor-widget-heading .elementor-heading-title.elementor-size-medium{font-size:19px}.elementor-widget-heading .elementor-heading-title.elementor-size-large{font-size:29px}.elementor-widget-heading .elementor-heading-title.elementor-size-xl{font-size:39px}.elementor-widget-heading .elementor-heading-title.elementor-size-xxl{font-size:59px}<\/style><h2 class=\"elementor-heading-title elementor-size-default\">Reinforcement learning is a Machine Learning method that enables complex tasks to be performed autonomously. The aim of this article is to help you understand Q-learning and save time when implementing this type of solution.<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ca759f3 elementor-widget elementor-widget-text-editor\" data-id=\"ca759f3\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-text-editor.elementor-drop-cap-view-stacked .elementor-drop-cap{background-color:#69727d;color:#fff}.elementor-widget-text-editor.elementor-drop-cap-view-framed .elementor-drop-cap{color:#69727d;border:3px solid;background-color:transparent}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap{margin-top:8px}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap-letter{width:1em;height:1em}.elementor-widget-text-editor .elementor-drop-cap{float:left;text-align:center;line-height:1;font-size:50px}.elementor-widget-text-editor .elementor-drop-cap-letter{display:inline-block}<\/style>\t\t\t\t<p>Even more recently, this family of algorithms made headlines in e-sports with the release of AlphaStar, an algorithm developed to challenge the best players in the world in <strong>StarCarft<\/strong>. <a href=\"https:\/\/datascientest.com\/en\/boosting-business-with-3-essential-machine-learning-algorithms\">These algorithms have great potential,<\/a> but can be very time-consuming to build and configure.\u00a0<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-e4aea80 e-flex e-con-boxed e-con\" data-id=\"e4aea80\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t<div class=\"elementor-element elementor-element-fdd5b53 e-con-full e-flex e-con\" data-id=\"fdd5b53\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;full&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-1d58427 elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"1d58427\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-scientist-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Learning Machine Learning<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-11b9d3c e-flex e-con-boxed e-con\" data-id=\"11b9d3c\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t<div class=\"elementor-element elementor-element-d7a5435 e-con-full e-flex e-con\" data-id=\"d7a5435\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;full&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-c12ed7d elementor-widget elementor-widget-heading\" data-id=\"c12ed7d\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Definitions:  Q-Learning<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c33c91a elementor-widget elementor-widget-heading\" data-id=\"c33c91a\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\">What is reinforcement learning?<\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b119596 elementor-widget elementor-widget-text-editor\" data-id=\"b119596\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><strong>Reinforcement learning<\/strong> is a<a href=\"https:\/\/datascientest.com\/en\/machine-learning-engineer-bootcamp-why-is-it-interesting\"> machine learning method<\/a> whose objective is to enable an agent (virtual entity: robot, programme, etc.), placed in an interactive environment (its actions modify the state of the environment), to choose actions that maximise quantitative rewards. The agent performs trials and improves its action strategy according to the rewards provided by the environment.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-59b2765 elementor-widget elementor-widget-heading\" data-id=\"59b2765\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\">What is Q-learning?<\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ad55108 elementor-widget elementor-widget-text-editor\" data-id=\"ad55108\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>There are many <strong>reinforcement learning algorithms<\/strong>, categorised into several sub-families. <strong>Q-learning<\/strong> is both relatively simple and allows us to understand the learning mechanisms common to many other models.<\/p><p>As an introductory illustration, a<strong> Q-learning<\/strong> <a href=\"https:\/\/datascientest.com\/en\/algorithm-what-is-it\">algorithm works to solve a basic problem.<\/a> For example, in the maze game, the aim of the game is to teach the robot to get out of the maze as quickly as possible when it is randomly placed on one of the white squares. To achieve this, there are three central stages in the learning process:<\/p><ol><li><strong>Knowledge:<\/strong> defining a Q action-value function ;<\/li><li><strong>Reinforcing knowledge:<\/strong> updating the Q function;<\/li><li><strong>Taking action:<\/strong> adopting a PI action strategy.<\/li><\/ol><p>Q-learning is therefore a reinforcement learning algorithm that seeks to find the best action to take given the current state.<\/p><p>It is considered non-policy because the <strong>Q-learning<\/strong> function learns actions that are outside the current policy, such as taking random actions, and therefore a policy is not required. More precisely, Q-learning seeks to learn a policy that maximises total reward.<\/p><p>The &#8220;Q&#8221; in Q-learning stands for quality. In this case, quality represents the utility of a given action in obtaining a future reward.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-5473359 e-flex e-con-boxed e-con\" data-id=\"5473359\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t<div class=\"elementor-element elementor-element-654a3b8 e-con-full e-flex e-con\" data-id=\"654a3b8\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;full&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-fda0ae6 elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"fda0ae6\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-scientist-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Learn to master Q-learning<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-834deb3 e-flex e-con-boxed e-con\" data-id=\"834deb3\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t<div class=\"elementor-element elementor-element-8007c15 e-con-full e-flex e-con\" data-id=\"8007c15\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;full&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-848d3f0 elementor-widget elementor-widget-heading\" data-id=\"848d3f0\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Q-learning: practical implementation<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3fd4d27 elementor-widget elementor-widget-heading\" data-id=\"3fd4d27\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\">Creating a Q table<\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e6602eb elementor-widget elementor-widget-text-editor\" data-id=\"e6602eb\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>When learning is complete, we create a so-called<strong> q-table or matrix of the form [state, action]<\/strong> and initialise our values to zero. We then update and store our q-values after each episode. This table of values becomes a reference table for our agent, who selects the best action based on the values in this matrix.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d6102e1 elementor-widget elementor-widget-code-highlight\" data-id=\"d6102e1\" data-element_type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-python line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-python\">\n\t\t\t\t\t<xmp>import numpy as np \r\n \r\n# Initialize q-table values to 0\r\nQ = np.zeros((state_size, action_size))\r\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-bad8971 elementor-widget elementor-widget-heading\" data-id=\"bad8971\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\">Q-learning and updates<\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-660f3c1 elementor-widget elementor-widget-text-editor\" data-id=\"660f3c1\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The next step is simply for the agent to interact with the environment and update the state-action pairs in our Q[state, action] array.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-877382b elementor-widget elementor-widget-heading\" data-id=\"877382b\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\">- Taking action: Explore or Exploit<\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-337c05a elementor-widget elementor-widget-text-editor\" data-id=\"337c05a\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>An agent interacts with the environment in two ways. The first is to use the Q table as a reference and view all the possible actions for a given state. The agent then selects the action based on the maximum value of these actions. This is called exploitation, as we use the<strong> information available to us to make a decision.<\/strong><\/p><p>The second way is to act randomly. This is called exploration. Instead of selecting actions based on the maximum future reward, we choose an action at random. Acting randomly is important because it allows the agent to explore and discover new states that might not otherwise be selected during the exploitation process.<\/p><p>You can balance exploration \/ exploitation by using epsilon (\u03b5) and setting the value of how often you want to explore or exploit. Here is some rough code that will depend on how the state and action space are configured.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-432417e elementor-widget elementor-widget-code-highlight\" data-id=\"432417e\" data-element_type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-python line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-python\">\n\t\t\t\t\t<xmp>import random  \r\n \r\n# Set the percent you want to explore\r\nepsilon = 0.2\r\n \r\nif random.uniform(0, 1) < epsilon:\r\n    \"\"\"\r\n    Explore: select a random action    \r\n    \"\"\"\r\nelse:\r\n    \"\"\"\r\n    Exploit: select the action with max value (future reward)    \r\n    \"\"\"\r\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5316b44 elementor-widget elementor-widget-heading\" data-id=\"5316b44\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\">- Updating the q-table<\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4eb0c0b elementor-widget elementor-widget-text-editor\" data-id=\"4eb0c0b\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Updates take place after each step or action and end when an episode is completed. In this case, &#8220;completed&#8221; means that the agent has reached a terminal point. For example, an end state might be landing on a payment page or achieving a desired goal. With enough exploration (steps and episodes), the agent will be able to converge and learn the optimal values of q or q-star (Q\u2217).<\/p><p>Here are the 3 basic steps:<\/p><ol><li>The agent starts in a state (s1) takes an action (a1) and receives a reward (r1).<\/li><li>The agent chooses the action by referring to the Q-table with the highest value (max) OR at random (epsilon, \u03b5).<\/li><li>Updating q-values<\/li><\/ol><p>The basic update rule for Q-learning is as follows:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-fece19c elementor-widget elementor-widget-code-highlight\" data-id=\"fece19c\" data-element_type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-python line-numbers\">\n\t\t\t\t<code readonly=\"true\" class=\"language-python\">\n\t\t\t\t\t<xmp># Update q values\r\nQ[state, action] = Q[state, action] + lr * (reward + gamma * np.max(Q[new_state, :]) \u2014 Q[state, action])\r\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-87a6546 elementor-widget elementor-widget-text-editor\" data-id=\"87a6546\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>In the update above, there are a few variables that we haven&#8217;t mentioned yet. What happens here is that we adjust our q-values according to the difference between the new updated values and the old ones. We update the new values using gamma and adjust our step size using the learning rate (lr).<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4360c27 elementor-widget elementor-widget-heading\" data-id=\"4360c27\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">References<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f9c4507 elementor-widget elementor-widget-text-editor\" data-id=\"f9c4507\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<ul><li><strong>Learning rate:<\/strong> lr, often called alpha, can be defined as the degree of acceptance of the new value compared to the old one. Above, we take the difference between the new value and the old value, then multiply this value by the learning rate. This value is then added to our previous q-value, moving it in the direction of our last update.<\/li><li><strong>Gamma:<\/strong> gamma or \u03b3 is a discount factor. It is used to balance immediate and future reward. In our update rule above, you can see that we apply the discount to the future reward. In general, this value can vary between 0.8 and 0.99.<\/li><li><strong>Reward:<\/strong> The reward is the value received after performing a certain action in a given state. A reward can occur at any given time step or only at the terminal time step.<\/li><li><strong>Max:<\/strong> np.max() uses the numpy library and takes the maximum of the future reward and applies it to the reward in the current state. This has the effect of influencing the current action by the possible future reward. In effect, thanks to Q-learning, we are able to allocate future reward to current actions to help the agent select the most profitable action in any given state.<\/li><\/ul>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e107cab elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"e107cab\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/en\/data-scientist-course\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Become a Q-learning expert<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-298822a e-flex e-con-boxed e-con\" data-id=\"298822a\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t<div class=\"elementor-element elementor-element-7ee1bb4 e-con-full e-flex e-con\" data-id=\"7ee1bb4\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;full&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-006234c elementor-widget elementor-widget-html\" data-id=\"006234c\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<script type=\"application\/ld+json\">\n{\n  \"@context\": \"https:\/\/schema.org\",\n  \"@type\": \"FAQPage\",\n  \"mainEntity\": [{\n    \"@type\": \"Question\",\n    \"name\": \"What is reinforcement learning?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"Reinforcement learning is a machine learning method whose objective is to enable an agent (virtual entity: robot, programme, etc.), placed in an interactive environment (its actions modify the state of the environment), to choose actions that maximise quantitative rewards. The agent performs trials and improves its action strategy according to the rewards provided by the environment\".\n    }\n  },{\n    \"@type\": \"Question\",\n    \"name\": \"What is Q-learning?\",\n    \"acceptedAnswer\": {\n      \"@type\": \"Answer\",\n      \"text\": \"There are many reinforcement learning algorithms, categorised into several sub-families. Q-learning is both relatively simple and at the same time allows us to understand learning mechanisms common to many other models.\"\n    }\n  }]\n}\n<\/script>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Reinforcement learning is a Machine Learning method that enables complex tasks to be performed autonomously. The aim of this article is to help you understand Q-learning and save time when implementing this type of solution. Even more recently, this family of algorithms made headlines in e-sports with the release of AlphaStar, an algorithm developed to [&hellip;]<\/p>\n","protected":false},"author":67,"featured_media":179089,"comment_status":"open","ping_status":"open","sticky":false,"template":"elementor_theme","format":"standard","meta":{"content-type":"","inline_featured_image":false},"categories":[2409],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.7 (Yoast SEO v21.7) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Q-learning - Machine Learning with reinforcement learning<\/title>\n<meta name=\"description\" content=\"Embark on a journey into Q-learning, a powerful algorithm in reinforcement learning. Explore how this method enables machines to make optimal decisions in dynamic environments, and uncover its applications across various domains in the realm of artificial intelligence and machine learning.&quot;\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Q-learning - Machine Learning with reinforcement learning\" \/>\n<meta property=\"og:description\" content=\"Reinforcement learning is a Machine Learning method that enables complex tasks to be performed autonomously. The aim of this article is to help you\" \/>\n<meta property=\"og:url\" content=\"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning\" \/>\n<meta property=\"og:site_name\" content=\"Data Science Courses | DataScientest\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataScientestEN\" \/>\n<meta property=\"article:published_time\" content=\"2024-04-11T18:21:16+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-02-17T20:20:06+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/datascientest.com\/en\/files\/2024\/02\/q-learning.png\" \/>\n\t<meta property=\"og:image:width\" content=\"768\" \/>\n\t<meta property=\"og:image:height\" content=\"422\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Melanie\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:site\" content=\"@DataScientest__\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Melanie\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning#article\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning\"},\"author\":{\"name\":\"Melanie\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d\"},\"headline\":\"Q-learning &#8211; Machine Learning with reinforcement learning\",\"datePublished\":\"2024-04-11T18:21:16+00:00\",\"dateModified\":\"2024-02-17T20:20:06+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning\"},\"wordCount\":977,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"articleSection\":[\"Machine Learning\"],\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning#respond\"]}],\"copyrightYear\":\"2024\",\"copyrightHolder\":{\"@id\":\"https:\/\/datascientest.com\/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning\",\"url\":\"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning\",\"name\":\"Q-learning - Machine Learning with reinforcement learning\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/en\/#website\"},\"datePublished\":\"2024-04-11T18:21:16+00:00\",\"dateModified\":\"2024-02-17T20:20:06+00:00\",\"description\":\"Embark on a journey into Q-learning, a powerful algorithm in reinforcement learning. Explore how this method enables machines to make optimal decisions in dynamic environments, and uncover its applications across various domains in the realm of artificial intelligence and machine learning.\\\"\",\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning\"]}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/datascientest.com\/en\/#website\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"name\":\"Data Science Courses | DataScientest\",\"description\":\"Leader in Data Science Courses\",\"publisher\":{\"@id\":\"https:\/\/datascientest.com\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/datascientest.com\/en\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/datascientest.com\/en\/#organization\",\"name\":\"DataScientest\",\"url\":\"https:\/\/datascientest.com\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"contentUrl\":\"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png\",\"width\":301,\"height\":301,\"caption\":\"DataScientest\"},\"image\":{\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/DataScientestEN\",\"https:\/\/twitter.com\/DataScientest__\",\"https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/\",\"https:\/\/www.tiktok.com\/@datascientest\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d\",\"name\":\"Melanie\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g\",\"caption\":\"Melanie\"},\"url\":\"https:\/\/datascientest.com\/en\/author\/melaniekrause\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Q-learning - Machine Learning with reinforcement learning","description":"Embark on a journey into Q-learning, a powerful algorithm in reinforcement learning. Explore how this method enables machines to make optimal decisions in dynamic environments, and uncover its applications across various domains in the realm of artificial intelligence and machine learning.\"","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning","og_locale":"en_GB","og_type":"article","og_title":"Q-learning - Machine Learning with reinforcement learning","og_description":"Reinforcement learning is a Machine Learning method that enables complex tasks to be performed autonomously. The aim of this article is to help you","og_url":"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning","og_site_name":"Data Science Courses | DataScientest","article_publisher":"https:\/\/www.facebook.com\/DataScientestEN","article_published_time":"2024-04-11T18:21:16+00:00","article_modified_time":"2024-02-17T20:20:06+00:00","og_image":[{"width":768,"height":422,"url":"https:\/\/datascientest.com\/en\/files\/2024\/02\/q-learning.png","type":"image\/png"}],"author":"Melanie","twitter_card":"summary_large_image","twitter_creator":"@DataScientest__","twitter_site":"@DataScientest__","twitter_misc":{"Written by":"Melanie","Estimated reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning#article","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning"},"author":{"name":"Melanie","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d"},"headline":"Q-learning &#8211; Machine Learning with reinforcement learning","datePublished":"2024-04-11T18:21:16+00:00","dateModified":"2024-02-17T20:20:06+00:00","mainEntityOfPage":{"@id":"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning"},"wordCount":977,"commentCount":0,"publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"articleSection":["Machine Learning"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning#respond"]}],"copyrightYear":"2024","copyrightHolder":{"@id":"https:\/\/datascientest.com\/#organization"}},{"@type":"WebPage","@id":"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning","url":"https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning","name":"Q-learning - Machine Learning with reinforcement learning","isPartOf":{"@id":"https:\/\/datascientest.com\/en\/#website"},"datePublished":"2024-04-11T18:21:16+00:00","dateModified":"2024-02-17T20:20:06+00:00","description":"Embark on a journey into Q-learning, a powerful algorithm in reinforcement learning. Explore how this method enables machines to make optimal decisions in dynamic environments, and uncover its applications across various domains in the realm of artificial intelligence and machine learning.\"","inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/datascientest.com\/en\/q-learning-machine-learning-with-reinforcement-learning"]}]},{"@type":"WebSite","@id":"https:\/\/datascientest.com\/en\/#website","url":"https:\/\/datascientest.com\/en\/","name":"Data Science Courses | DataScientest","description":"Leader in Data Science Courses","publisher":{"@id":"https:\/\/datascientest.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/datascientest.com\/en\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/datascientest.com\/en\/#organization","name":"DataScientest","url":"https:\/\/datascientest.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","contentUrl":"https:\/\/datascientest.com\/en\/wp-content\/blogs.dir\/9\/files\/2024\/02\/DataScientest_icon_blue-2.png","width":301,"height":301,"caption":"DataScientest"},"image":{"@id":"https:\/\/datascientest.com\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataScientestEN","https:\/\/twitter.com\/DataScientest__","https:\/\/www.linkedin.com\/school\/datascientest\/mycompany\/","https:\/\/www.tiktok.com\/@datascientest"]},{"@type":"Person","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/19d5f7ade9407eb69af3bf6d1ae1731d","name":"Melanie","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/datascientest.com\/en\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/8d3d660a0698de06f13ecf4823519da1?s=96&d=mm&r=g","caption":"Melanie"},"url":"https:\/\/datascientest.com\/en\/author\/melaniekrause"}]}},"_links":{"self":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/179078"}],"collection":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/users\/67"}],"replies":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/comments?post=179078"}],"version-history":[{"count":0,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/posts\/179078\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media\/179089"}],"wp:attachment":[{"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/media?parent=179078"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/categories?post=179078"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/datascientest.com\/en\/wp-json\/wp\/v2\/tags?post=179078"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}