Expected Values for Continuous Variables!!!

Key Takeaways

stat quest stat quest stat quest yeah stat quest hello i'm josh starmer and welcome to statquest today we're going to talk about expected values part 2 continuous variables note this stack quest assumes that you are already familiar with the concept of an expected value if not check out the quest in expected values part 1 we travel to the mystical and magical place called statland and we showed how to calculate expected values for two really bad bets made by our friend statsquatch for example statsquatch made this bet i bet you one dollar that the next person we meet has heard of the movie troll 2 and we use the probabilities of people in statland having heard of troll 2 or not having heard of troll 2 and the amount of money we would lose if we met someone who had heard of troll 2 in the amount of money we would gain if we met someone who had not heard of troll 2 to calculate the expected value of the overall amount of money we would gain or lose per bet if we made the bet a lot of times and the result was 0.66 meaning that we expect to gain on average 60 cents per bet if we make the bet a bunch of times oh no it's the dreaded terminology alert when we calculate the expected value for a bad bet like this we say that we are calculating the expected value for a discrete variable in this case the discrete variable is the bet and it has two outcomes lose one dollar or gain one dollar in general any time we have discrete outcomes we have a discrete variable bam now let's talk about expected values for continuous variables continuous variables come from measuring things and the outcomes are wait for it continuous for example imagine you and your friend statsquatch are walking around statland and statsquatch says i wonder how long we would have to wait per person to see people in other words statsquatch wants to know the expected value for waiting time which is something we can measure and after 10 seconds you meet someone yo what's up squatch so you and statsquatch decide to keep track of that by putting a dot on this number line at 10. the next person we meet shows up after 30 seconds and the next person shows up immediately and the next person shows up in 10 seconds etc etc etc then statsquatch says ugh collecting all this data is taking forever then statsquatch notices the gaps in the data and says gasp gaps gaps in the data mean we still have more to collect then statsquatch notices that the data are plotted using 10 second intervals and says but what if we want different interval sizes like 5 or 2.5 seconds so you tell stat squatch chill out squatch instead of spending the rest of our lives collecting data and worrying about the interval size we can model the waiting times with an exponential distribution bam this curve that skims the top of the data is an exponential distribution and this is the equation for the exponential distribution lambda which is also called the rate is a parameter that defines the shape of the curve in this example the rate refers to the number of people we meet per second because that is the unit on the x-axis and if we set lambda to 0.05 we get a curve that fits the data we have already collected however if we set lambda to 0.1 meaning we meet more people per second then we get this green curve that has a steeper slope close to zero and if we set lambda to 0.01 meaning we met fewer people per second then we get this orange curve that barely bends but is much higher on the right side compared to the other curves but like we said when lambda equals 0.05 then the curve fits the data now if we want to calculate the probability we meet someone in 10 seconds or less then we calculate the area under the curve between zero and ten in other words we integrate the exponential distribution from zero to ten for now i'll spare you this math and just tell you that when lambda equals 0.05 the integral is 0.39 which means that the probability we will meet someone in 10 seconds or less is 0.39 alternatively if we wanted to know the probability of meeting someone between 25.302 seconds and 30.122 seconds then we can calculate the area under the curve between 25.302 and 30.122 in this case the area under the curve is 0.06 which means that the probability we will meet someone in this range of time is 0.06 in summary the exponential distribution fits the data that we have collected so far but it doesn't have any gaps or missing values and we can use it to make calculations on any interval we want note i call the y-axis likelihood because the y-axis coordinates generated by this equation are the likelihood values that we use for maximum likelihood estimation and if you want to learn about how these y-axis values are used and maximum likelihood with the exponential distribution check out this quest also note the y axis is scaled so that the total area under the curve is equal to one lastly in theory this curve should go all the way to positive infinity on the x-axis but we don't have enough room to draw a graph that goes all the way to infinity so we'll stop drawing at 90 seconds because at that time the curve is pretty close to zero on the y-axis okay enough about the exponential distribution itself now let's talk about how to calculate the expected value and i know we just talked about how awesome it is to have a continuous distribution but for a few minutes let's pretend that this is actually a discrete distribution and let each 10 second interval represent an outcome note in this case we've drawn each rectangle so that the curve goes through the midpoint of each top side so for example because the interval is 10 seconds long the curve intersects the first rectangle at 5 seconds and the curve intersects the second rectangle at 15 seconds etc etc etc now instead of having to integrate the function to get the area under the curve we can approximate the area under the curve for each outcome with the corresponding area of each rectangle for example the probability of meeting someone in the first 10 seconds is approximately the width of the first rectangle which is 10 times the height to calculate the height we need to find the y-axis coordinate for where the top edge of the rectangle intersects the curve and that means we need to find the y-axis coordinate for this exponential distribution when time equals five so we plug x equals five into the equation and do the math and we get 0.04 so the height of the rectangle is 0.04 and the area of the rectangle is the height times the width which is 0.4 and that means the probability of meeting someone in the first 10 seconds is approximately 0.4 and compared to the exact probability calculated with the integral 0.39 the approximation is not terrible so let's put 0.4 inside the first rectangle so we don't forget likewise we use the exponential distribution to calculate the height for each rectangle and the probabilities for each outcome bam now if we want to approximate the expected value of the exponential distribution we can plug the outcomes and their approximated probabilities into the equation for discrete outcomes for example the first outcome is meeting people in 10 seconds or less and the probability is 0.4 so the first term is 10 times 0.4 the second outcome is the 10 second interval that ends at 20 seconds and the associated probability is 0.2 so the second term is 20 times 0.2 likewise we add the remaining terms and when we do the math we get 22 and that suggests that on average we expect to wait 22 seconds between each time we meet someone bam now if we want to improve our approximation we can cut the intervals in half so that each one lasts 5 seconds instead of 10. and when we do the math plugging in each outcome and its corresponding probability we get 21.8 now to improve the estimate of the expected value even more we can keep decreasing the width of each rectangle until the width of the rectangles goes to zero and the number of rectangles goes to infinity note when we have an infinite number of rectangles with zero width then we are no longer approximating the area under the curve but calculating it exactly now remember that the probability of observing a specific outcome is the height times the width of the associated rectangle and that the height the y-axis coordinate of the top of each rectangle is the likelihood at that point and the width can be written as delta x now if you remember from high school calculus if the sum of the number of rectangles goes to infinity while the width of each rectangle delta x goes to zero then we end up with an integral now i know this is a huge mess of math so let's summarize everything when we have a discrete distribution like this the expected value of the corresponding discrete variable is the sum of the outcomes times their associated probabilities and when we have a continuous distribution like this then the expected value of the corresponding continuous variable uses an integral instead of a sum and the rest of the equations are very similar except we replace the probability with the likelihood the y-axis coordinate note although we have been using the exponential distribution as an example this formula works for any continuous variable double bam now that we have a formula for the expected value for continuous variable let's calculate the expected value for a continuous variable from the exponential distribution since we use the exponential distribution equation to calculate the likelihoods let's plug it into the equation for the expected value and because the exponential distribution is defined for all values greater than or equal to zero we will integrate everything from zero to infinity now we just do the math first because we can split this into two functions we can use integration by parts to find the solution note if you are not familiar with integration by parts there are helpful links in the description below anyways we'll start by setting f of x equal to x now since integration by parts requires the derivative of f of x we will put that here now we'll set the derivative of g of x g prime of x to be equal to the second term in the integral and because we need g of x the antiderivative of g prime of x we need to figure it out now you might have an awesome strategy for finding anti-derivatives but i do not so i start by observing that the derivative of e to the x is e to the x which is close to what we want but among other things is missing the negative lambda in the exponent so let's put negative lambda in the exponent to match what we want and the derivative via the chain rule is this negative lambda times e raised to the negative lambda x and that is almost the derivative we are shooting for except g prime of x does not have this negative sign so let's try putting a negative sign in front of the equation now when we take the derivative with the chain rule we get the same thing as g prime of x so this equation must be the anti-derivative g of x small bam now we just plug these functions and their derivatives into the integration by parts formula first let's plug in f of x now let's plug in the derivative of f of x and since multiplying by one doesn't change anything we can omit it now let's plug in the derivative of g of x and lastly let's plug in g of x and move the minus sign outside of the parentheses now let's do some more math first let's tackle this term by evaluating it when x equals infinity and when x equals zero so we set x equal infinity then we subtract the term and set x equal to zero now since the exponent in the first term is negative we can turn it into a fraction however it's not obvious what this fraction is equal to 0 infinity a quick google search on l'hopital's rule tells us that the limit as x goes to infinity of a of x divided by b of x is equal to the limit as x goes to infinity of a prime of x divided by b prime of x so in our case we have the limit as x goes to infinity of x divided by e raised to the lambda x then we take the derivatives of the numerator and the denominator and plug in infinity for x and that equals zero because we are basically dividing one by infinity so we can replace the first term with zero and we can replace the second term with zero two because zero times anything is zero lastly zero minus zero is zero and we are done computing the first term of the integration by parts now let's compute the second term first we recognize that we are subtracting the second term from the first then we solve for the antiderivative of the stuff inside the integral using the trial and error approach we saw earlier and evaluate it at infinity and zero just like before since the exponent in the first term is negative we can turn it into a fraction and one divided by infinity is zero now since the exponent in the second term is multiplied by zero the whole exponent is zero and anything raised to the zero power is one and that leaves us with one divided by lambda lastly we commute this minus sign do the math and we see that the expected value is 1 divided by lambda so let's move this up here now given this specific exponential distribution where lambda equals 0.05 we calculate the expected value by plugging in 0.05 in for lambda and we see we expect to wait on average 20 seconds between meeting people so going back to the original question that statsquatch asked how long will we have to wait per person to see people we answer 20 seconds and then statsquatch says triple bam in summary the expected values for discrete and continuous variables are very similar the only two differences are one we replace the sum with the integral and two we replace the probability with the likelihood bam note if you would like to know how we estimate the value for lambda with a pile of data check out the quest on the exponential distribution and maximum likelihood the link is in the description below also note if you watch this video hoping to learn exactly why we divide the sample variance by n minus 1 know that you have taken a big step towards understanding this mystery now it's time for some shameless self-promotion if you want to review statistics and machine learning offline check out the stackquest study guides at statquest.org there's something for everyone hooray we've made it to the end of another exciting stat quest if you like this stat quest and want to see more please subscribe and if you want to support statquest consider contributing to my patreon campaign becoming a channel member buying one or two of my original songs or a t-shirt or a hoodie or just donate the links are in the description below alright until next time quest on