costFunctionReg.m - Programming Assignment 2 Machine Learning

Key Takeaways

we want to implement the cost function and the gradient once again for logistic regression except that we want to add the additional the regularization part so first we want to just copy our old cost function and the thing we want to add to the cost function is the regularization so it's lambda over 2 times M and then we want to take here it's important to be a little bit careful we don't want all the Thetas we want to tear us from from 1 to N and so recognize that this is math which is index from 0 so so this is this should be 0 here if we want to include all the Thetas and in MATLAB its index starting at 1 so it's a bit confusing but that's that's the idea so we want to regularization so first check just what's the size of theta so that we know how to multiply them see data from this 28 by 1 okay so and over 2m times theta and remember we want to from second theta value to the end and all all combs and we want to take that transpose times theta to come and to to end them all so we want them to all be summed together right so we want a scaler at the end and way to get a scalar is by taking this transpose times itself then we get 1 by 28 times 28 by a-1 which yeah so we get this is let's see this above here is one instead of Mumbai 28 because we took away the first value matrix multiplied by 27 by one which is just a one by one so let's Taylor okay so we get a scalar here we get the regularization all we want to do now is just add the regularization to our cost and we want to check if this works by the way here it's kind of tricky but if you get the wrong if you get the wrong result it might be that you run this part several times and every time you you rerun this part you need to also rerun the load data so rerun that part and then run this okay maybe not okay so we need one more parenthesis it's now we've only done the cost function yeah right we need to rerun this part so always we run this part first because otherwise you'll see that we get the absolute wrong result so now we get the cost 3.16 3.16 and 0.6 t 9 0.69 okay so that the cost function is quick what we want to do now is the gradient so essentially this is the gradient it's the same as the normal the cost function that we have already done except that we need to add so we need to add plus C lambda over m and then we need to take all the Thetas except the first one so similarly to 2 and and to 2n to come all columns okay so we run that that part again then we check this part okay so this took quite a while to the bug but so we have different derivatives depending on which theta we're looking at so for J equals zero we have this one which is just a cost function and then for J greater or equal to one we have the the regularization part so what we need to do here is exactly as we did which is 2 plus lambda over m times theta to come and all columns and the only thing that's tricky here is that well we actually don't we need to have the gradient for the first one - this will be 27 by 1 this will be an 8 by 1 so if we try to add them don't work so we have to do is we have to add a 0 start just to make the dimensions match and now it should work so if we're in the first part and then we're in the second part let's check we get the correct cost yeah and the correct they click gradients - ok so thank you for watching this video and see you in the next one