Library
Mathematics

Visualizing Gradient Descent

This lesson explores Gradient Descent as a process of finding the local minimum of a function by stepping in the direction of steepest descent, guided by the local slope.

You
f(x,y)f(x, y)
Imagine standing on a foggy mountain range, wanting to reach the lowest valley. Since you cannot see the whole map, your only strategy is to feel the slope under your feet and take a step downhill.
Step-by-step solver
1

Identify the function

Define the cost function you wish to minimize, such as the Mean Squared Error.

f(θ)f(\theta)
2

Calculate the derivative

Compute the partial derivative of the function with respect to your parameter.

ddθf(θ)\frac{d}{d\theta} f(\theta)
3

Update parameter

Subtract the gradient (scaled by alpha) from the current value to move closer to the minimum.

θnext=θ−αddθf(θ)\theta_{next} = \theta - \alpha \frac{d}{d\theta} f(\theta)

Original question

EXPLAIN HOW GRADIENT DESCENT WORKS

Follow-up chat

Ask me anything about this lesson. I'll answer using what we just covered.