Solve Optimization Problems with Lagrange Multipliers in Python

Key Takeaways

what is going on guys welcome back in this video today we're going to learn how to solve optimization problems with constraints using lrange multipliers in Python so let us get right into [Music] ited all right so this video today is going to be a little bit more mathematical than usual we're going to learn how to use LR multipliers in Python to solve optimization problems with constraints and I want to show you what this means visually I want to show you what it looks like so you get an intuition of what we're doing here let's say we have some function that we want to find a minimum or maximum for so let's say min max off and we can do something very simple like X squ in this case we only have a minimum uh as you can see here a global minimum we can do the same thing for something like 2 * x^2 - 2 * y 2quared for example I think this doesn't have a global Optimum uh but the goal is not to just find the op Optimum for a given function we want to also have a constraint we want to have a condition for example in this case I could say something like I want to have the minimum and maximum of this function given this condition given this constraint that X to the^ of 2 plus uh y ^ of 2 is equal to 2 for example this would now be a circle and this basically means um that I'm looking for the minimum and maximum at these points here so I'm looking at these uh at this specific line here which is a circle and I'm only interested in points that lie on this line so don't confuse this to be the area it's not the area it's points on this line so they have to be on here and you can see that all the values that we find here are on the line uh maybe to make this a little bit clearer we can use something else let's go with something like X to the^ of 3 um Min - y or - 3 y to the power of 3 that should be a line not a circle uh actually we need to say equals something equals 5 for example um and this will also give us a result uh but you can see now that this is a line so it's not the area we're looking for the lowest and highest point on this line that's the basic idea here so this is what we want to solve and how you do that I don't want to go too much into the mathematics here because I'm not a mathematician I'm a computer scientist I do understand how it works to some degree but I'm not here to explain to you the concept or the theory behind lrange multipliers there are a lot of good videos on YouTube that you can watch for that I want to show you how to solve this in Python easily so you basically solve this in Python the same way you solve it uh by hand but you do it in an automated way so you don't have to to do all the calculations yourself we're going to use a package called simpai for this so we're going to open up the terminal and we're going to install the simpai package and um this is basically a package that allows us to do symbolic mathematics in Python so we're going we're going to say import sypi SSP and then we're going to basically uh State the function that we want to optimize then a second function that is basically our um our constraint and then we're going to take the derivative of the combined function using a Lambda uh which we're going to multiply with the um with the constraint and then we're going to solve the system for equations that comes from setting all the partial derivatives to zero so we're going to start by saying we want to have the c a couple of certain symbols let's go with x y and Lambda I'm going to just use an L here x y and Lambda are going to be SP symbols and we're going to pass your x y and L and now we can use them as variables in mathematics so I can say that my function is for example uh 2 * x² + 2 * or actually minus 2 * y^ 2 and uh this is basically the same thing we had in in wolf from alpha so let's go back to our constraint x^2 + y^2 = 2 now the tricky thing here it's actually not tricky but the thing that you need to do here is if you want to specify the constraint you have to set it equal to zero so you basically have to subtract by two in this case so that the constraint is actually x^2 + y^2 - 2 = 0 because you need to multiply the expression it's not an equation it's actually a function you need to multiply it with Lambda um so we're going to say G our constraint is going to be uh that x^ 2 + y ^ 2 - 2 is equal to 0 but we're not going to specify is equal to zero this is implicit here because now to get the final function we're actually going to take the partial derivatives of uh large F uppercase F we're going to say f plus Lambda time G um this is basically how you do that with your uh with the lrange multiplier so we have basically our function and then we want to multiply it we want to add to it Lambda multiplied with G which is our uh constraint and now all we have to do is we have to compute FX FY and F Lambda so the partial derivatives with respect to x y and Lambda which we can do also with simpai by saying SP diff so take the derivative of f with respect to X and then just copy this to be the same for y and for Lambda for y and for Lambda there you go and all we have to do now is we have to solve the system of equations because this basically now is going to be set to zero if we don't specify any additional thing so we're going to basically have the the derivative with respect to x y and Lambda set to zero and then we just need to solve the system of equations to get the results for x y and Lambda and Lambda is just basically a helper tool we could say because the only thing we're interested in is actually X and Y so the value of Lambda is important for the calculations for the solving of the system but it's not really relevant to us uh once we have the results so we're going to say Solutions is equal to SP solve and we're going to pass here FX FY f l and we're interested in x y and L there you go and then we can print the solutions so I can run this and you can see that our Solutions are the same that we get here from um well from alpha we have all the minimums and all the maximums here in uh in the results so we can also do the same thing with three variables for example we can also have something like XYZ and then we have for example the function xus uh y + z squared now this is hard to visualize because now we have three input values and then we have one output which is four dimensional so uh we're not going to be able to easily visualize that uh but we can go ahead now and say that we want to do something quite similar we want to say plus Z to the^ of 2 minus 2 for example this would now be a um it would now be like like a ball basically and uh but it would only be the the outside of it so it would not be everything in this sphere but it would be um what is it the not circumference the the surface of the sphere is uh what we're looking at here in this four-dimensional space so we're going to do the same thing here the only thing that we need to add is an FZ and we need to say derivative with respect to Z then we need to say f z and we're also interested in Z and when I run this we get a problem because we only have three symbols here so let me Define Z as well run this again and there you go we now have all the optimums all the extreme values for this particular optimization problem now we can also go ahead and add multiple conditions we can go ahead and say for example uh let's go with a different function that I have prepared here three * x - y uh - 3 * z um and now we have two conditions the first one is going to be x + y - Z is equal to zero so just like that and then the second one H is going to be equal to x^2 + 2 * z^ squared is equal to 1 so minus one and to now use both of these constraints all we have to do is we have to Define two lambdas so we can say L1 L2 L1 L2 and then we just say uh L1 * G + L2 * H and all we have to do then is we have to say we want to have F L1 FL L2 L1 L2 F1 FL 2 L1 L2 there you go run this and we get the results easily so this is how you can do that easily in Python now to find out whether these are SLE points or uh maximums or minimums you can always go with the hessan and you can compute uh uh basically uh you can use the hesan to to see if this is a minimum maximum or settle Point uh or you can just plug in some values and and find out by by testing but this is how you can EAS easily use lrange multipliers to solve optimization problems with constraints in Python so that's it for today's video I hope you enjoyed it and hope you learn something if so let me know by hitting a like button and leaving a comment in the comment section down below and of course don't forget to subscribe to this Channel and hit the notification Bell to not miss a single future video for free other than that thank you much for watching see you in the next video and bye