The Gamma Distribution

Key Takeaways

In this video, we'll explore the gamma distribution. So, um we're just going to go through everything. So, a random variable x is distributed as gamma with parameters k and theta where k and theta are both positive numbers if it has the following pdf. So, the pdf is of this form. Uh we have again f big x little x is equal to 1 / and this weird symbol will be explained later in just a second. Uh it's called gamma. So, the gamma function. So one over the gamma function evaluated at k theta^ k x^ of k - 1 e^x over theta as long as x is bigger than equal to 0 and 0 otherwise. So it's only defined on the same range as the exponential function and we'll see the relationship between exponential and gamma at the end of the video. Okay. Now the elusive thing here the thing you probably don't know is the gamma function. So the gamma function is defined as an integral. So it's simply defined as the uh the gamma function evaluated at some real number t is the integral from 0 to infinity x^ the tus 1 e minus x dx. And that's it. It seems kind of complicated but it's just defined as this integral. So it's just easier notation than if we had to put this whole integral down here. Okay, cool. So let's do the expected value, variance, and mgf. All of which I've calculated already to save us some time. So expected value of x is given by well the basic expected value formula. It's integral from 0 to infinity since that's where it's defined uh to be non zero. Uh we just put the PDF right here. We put an extra x dx. So I'm going to pull out. Now notice something interesting. You might freak out at first because you think, oh no, this gamma function is an integral and it's inside another integral. That's going to be very hard to calculate. Well, the thing to notice is that this out this integral right here is dx. And this gamma function is a function of k. So it has nothing to do with this x at all. So we can just pull it out. Likewise, this theta to the k has nothing to do with the x. we can just pull it out. So we'll just pull them out right here. We're left with this part inside. So uh this x to the k minus 1 and this x I've combined as x to the k e x over theta. Now we make a u substitution. U= x over theta. du= 1 theta dx. And uh we put we do the u substitution. We substitute everywhere there's an x we would put u theta. So we do all that and then we get uh this result right here. And we get a nice cancellation of theta to the k theta decay on the bottom. And I'm going to call this c. What's remaining? So it's going to be theta over gamma function at k that's going to be our big c just a constant just for renaming purposes. Uh and then what we get is we have u to the k e to the negative u du. So now we have to do integration by parts and I've given the uh what you should put as v and dw and then over here this way you get the same c over here you get integral from 0 to infinity uh e the minus u k u to the kus 1 du. So let me see what happened when you do integration by parts you have uh vw. So you have u to the k e to the negative u evaluated from 0 to infinity. This is a pattern we've seen many times in the past. If you put infinity into here, so what we're dealing with is the product of this guy and this guy. If you put infinity, this will be uh in the denominator, this e to the u will dominate and become zero. If you put zero, it'll be 0 to the^ k. The numerator will vanish. So either way, this is zero. So all we're left with is uh it's really vw minus integral uh minus integral w dv. So since the vw vanishes we just have uh integral w dv which is exactly what I've written here. Okay so uh now that is written right here. This k is a constant. We can pull it out. Nothing to do with u. And now notice something interesting. This e to the minus u. U to the k minus one is exactly the gamma function. So instead of uh here we have x. So it's x to the tus 1. Here it's u to the k minus one. Here it's e x. Here it's e u. So it's the same thing. So this whole thing is really k * the gamma function uh evaluated at the point k and notice this c is the theta over gamma to the k. So this gamma to the k cancels with this gamma to the k. So we just have theta k. So the final answer is theta k for the expected value. Okay. Now we'll do the uh that's not this is the uh variance. So the variance we're going to use the typical variance formula. Here we have this we just calculated previously. So we need to find expected value of x squar. So I've written the uh definition by definition what it is. We have x^2 times the pdf and uh this c is a constant I've taken out as 1 over gamma the k theta^ k in the denominator that's taken over here and we have x^2 x kus 1 combined to make x k + 1 e x over theta. We're going to use that same u substitution. And doing that we have coming down here we have this quantity right here. here. So now we have everything in terms of u. Uh so now we can take this theta to the k + 1 and then we have another theta. So it's theta the k + 2 and the c part of the c we had theta to the k in the denominator. So we have theta squared over uh the gamma function at k. So that c I've kind of opened it up and said what it really is. So now here's our constant right here. So now again here we're just going to do the same thing. We're going to use integration by parts uh using very similar choices just instead we have a k plus one instead of a k here. So we're going to do all that coming down here. Again, the VW term goes away when we take it from 0 to infinity. Uh so then what we have is this right here. Uh this is just the wdv term, the integral of the wdv term. So then that's all here. Uh and then we have theta^ 2 over the gamma function of k k + 1. And then this notice this guy uh about this guy. The thing is that if we look at when we were calculating the expected value, we see that the same thing is right here. So notice here we have u to the k eus u du from 0 to infinity. Here are the same thing. We have u to the k e minus u du from 0 to infinity. And using integration by parts there we found that was k gamma function at k. So likewise I've written here we have theta^ 2 over gamma the k k + 1 outside k gamma function at k. These twos cancel right here and we have k * k + 1 * theta^ 2. So that is just this. Now we have to remember to subtract k^ 2 theta squar. So doing that we get k theta squar as the variance of our gamma distribution. The last thing we got to do is find the mgf. So the mgf is uh very interesting in that it's pretty easy to calculate based uh relative to the other two things. So by definition it is the pdf times this e to the sx term. So I've taken out this constants already and this e and this e since they're both e we can just add their exponents and we have this. Now let's do a little bit let's just kind of get a key insight what's going on here. Okay so now continuing that I've written the mgf in a little different form. So I've taken the um I've taken the e terms and I have uh put them together and I've rewritten them in different form. So just confirm for yourself that from the previous uh statement to this current statement this e to the power of what this is is the same as what e to the power of the previous thing was. Okay. So I've written it in the suggestive form because I want to think about this uh distribution. Think about a distribution which is gamma with parameters k and theta over 1 - s theta. Now what would the PDF of that look like? Well, it would look very similar to this. It would be x kus 1 e^x uh remember it's x over theta. So since theta in this case is being played by theta over 1 - s theta, it's going to be negative this guy uhx over this guy which is exactly what we have here since it's negativex times the reciprocal of this guy. Okay except what we don't have this is uh we don't have the constants. So we need to remember the constants. Okay. And speaking of those constants actually we need to put 1 / gamma the k theta the k since those were there from the previous statement. And now so this would be one because it's integrating over the whole over all the places it's defined of this gamma distribution. It would be one except we have to remember uh that it it's one times the constant since it's not the pure it's not this pure PDF. It's the PDF divided by the constants. So we need to this would be 1 / gamma evaluated at k theta to the k uh gamma of k and the other constant would be theta to the power of k. So theta in this case is being played by theta over 1 - s theta. So it's theta the k over 1 - s theta to the k. Now we have lots of nice cancellations right here. So we just have 1 - s theta to the k since this is in the denominator. So notice that this is our mgf except we have one little uh caveat that we have to deal with. uh theta remember we had to say was greater than zero which means that this guy right here this uh this thing right here has to be greater than zero. So we see that that's true only if well theta is definitely greater than zero. It came from a distribution where theta was the actual theta. Uh so it's only true if 1 - s theta is greater than 0. So we need 1 - s theta to be greater than 0. Just manipulating a little bit 1 is greater than s theta. We need s is less than 1 / theta. So that's a restriction on the kinds of s's we can put in there. So s must be less than 1 / theta if this is to work out. Okay. So this is our mgf. And the last thing we'll do in this video is we're going to talk about how the exponential distribution we talked about in a previous video is just a special case of this. So the special case uh exactly is consider the gamma distribution where k is equal to 1 and instead of theta we're going to have 1 over lambda play the role of theta. So now let's see what happens. Writing the PDF, we're going to have uh we're going to have uh x to the k minus one. So it's going to be x^ the 1 - 1 e to thex over theta. So in this case, it's going to be e to the negative x lambda. And of course, don't uh we have to remember to do the constants. So we have 1 over and what is the gamma function evaluated at 1? If you look at if you plug it into the definition, you're going to see that it's going to be 1. And then we have theta to the power of k. So that's going to be 1 - 1 / lambda to the^ of k. and k in here is 1. So it's going to be lambda to the power of negative k. And since k here is one, it's lambda to the negative 1, which is just one over lambda in the denominator. So flipping it again, we have this is lambda. This guy right here is just one. It's going to be lambda e to the lambda x x is greater than equal to 0. And we see that's exactly the definition of the exponential distribution. So uh it's actually very interesting how this is a special case of the exponential distribution. And when we talk about or uh when we talk about kai squared random variables or if you go to that video you're going to see that kai squared is also a special case of uh the gamma distribution.