The notion of derivative is particularly used in DataScience for machine learning models. Apart from what you've probably seen in high school, the applications of this mathematical tool are much broader, and we'll come back to them at the end of this article. In this article, you'll first learn how to calculate the derivative of a function, and then we'll look in a little more detail at the usefulness of this calculation.
Definition
The derivative of a function f(x) represents its rate of change. It may be denoted f'(x) or dfdx. Calculating and studying the derivative are important concepts in the study of functions.
Formulas
Calculating a derivative is generally quite straightforward if you know the basic formulas for the usual functions.
These formulas are summarized in the table below:
This table gives you the most basic derivatives.
Sum of functions
If your function is the sum of several other functions :
f(x)=u(x)+v(x)
then its derivative will be the sum of the derivatives:
f'(x)=u'(x)+v'(x)
For example:
f(x)=x2+3
In this case u(x)=x2 and v(x)=3
We can therefore calculate
u'(x)=(x2)’=2×2-1=2x
v'(x)=(3)’ = 0
f'(x)=u'(x)+v'(x) = 2x + 0 = 2x
Products of functions
For multiplication of functions:
f(x)=u(x)*v(x)
The total derivative is a little more specific and follows the following rule:
f'(x)=u'(x)*v(x)+u(x)*v'(x)
f(x)=x2sin(x)
In this case u(x)=x2 and v(x)=sin(x)
We can therefore calculate
u'(x)=(x2)’=2×2-1=2x
v'(x)=(sin(x))’ = cos(x)
f'(x)=u'(x)*v(x)+u(x)*v'(x) = 2x*sin(x) + x2*cos(x)
Quotients of functions
Now if a function is the division of two other functions: f(x)=u(x)v(x) then the derivative will be
Example :
In this case u(x)=(3x+6) and v(x)=(x3-7)
We can therefore calculate
u'(x)=(3x+6)’=3
v'(x)=(x3-7)’ =3×2
Compound functions
If your function is perceived as two functions grouped together, or in other words, a function of a function:
h(x)=f(g(x))=f ◦ g
then its derivative is calculated as follows:
h'(x)=(f ◦ g) ‘ = g’ .(f ‘ ◦ g)
For example:
We can therefore consider :
g(x)=x2+2x-4 and g'(x)=2x+2
f(x)=x and f'(x)=121×1/2=12x
and
so
Applications
But what’s the point of calculating the derivative?
Simply put, the sign of the derivative indicates the variations of the function f. It represents the tangent to the function. And the derivative itself represents the directing coefficient of the tangent to f at the point.
Reminder: The tangent of a function at a point is the line that touches the function at that point, and only that point.
If f ‘ ≥ 0, then f is increasing
If f ‘ ≤ 0, then f is decreasing
And when f’=0, we have the point x for which the directing coefficient of the tangent cancels out, and the function f possibly changes variation.
For example
f(x)=x2-4x+2
f'(x)=2x-4
f'(x)=0 ⇔ 2x=4 ⇔ x=2
This indicates that forx=2, my function may be changing its variation. As you can see below, x=2 is an inflection point:
So, replacing x by x<2 for example x=1,
We have:
f'(1)=2 × 1-4=⎯2<0
The derivative is less than 0, so the function f is decreasing.
Replacing x by x>2 for example x=3,
we have :
f'(3)=2*3-4=2>0
The derivative is greater than 0, so the function f is increasing.
We therefore have the following table of variation:
So what does this have to do with Machine Learning?
In machine learning problems, we seek to find the minimum of the cost function in order to minimize the prediction error and thus maximize model performance.
To do this, we use the derivative to find the minimum of the function.
However, calculating the derivative and equating it to 0 in order to find the corresponding x, does not allow us to find analytical solutions with any certainty.
This is why we use the gradient descent method. To find out more about this technique, see this article.
