Derivation Linear Regression with Gradient Descent

Key Takeaways

[Music] let's say we have a bunch of points and we want to find the regression line the best fits the data if we take a moment and just look at the points visually we can find a straight line that looks something like this but how do we know if the line is any good well one idea is that we can check how far away each point is from the line and this would be there and then we would just sum the errors and this would measure how good line is the problem with this is if one error is positive and another is negative then when we take the sum their error would be zero a solution for this is taking the sum of the errors squared instead let's define what we mean for the sum of squared errors lfw will be the average of the sum of squared errors specifically M will be the number of total points that we have and y hat of I will be our prediction for the correct Y for a specific point I Y hat for a specific point I will just be our linear model where we set x 1 to 1 to make the notation a little bit simpler the question is then how can we minimize the loss function one way is to use gradient descent using the following formula we want to find this gradient of the laws with respect to specific weight WJ moving the derivative sign inside the sum we see that Y hat is dependent on W and we will utilize the chain rule the inside derivative will in this case just be XJ the update rule for gradient descent and looks like the following in the next video I will code linear regression using gradient descent in Python