Bias Correction of Exponentially Weighted Averages (C2W2L05)

Key Takeaways

you've learned how to implement exponentially weighted averages there's one technical detail called bias correction that can make your computation of these averages more accurately let's see how that works in the previous video you saw this figure for beta equals 0.9 this figure for beta equals 0.98 but it turns out that if you implement the formula as written here you won't actually get the green curve when say beta equals 0.98 you actually get the purple curve here and you notice that the purple curve starts off really low so let's see how to fix that when you're implementing a moving average you initialize it with B zero equals zero and then V 1 is equal to 0.98 V zero plus 0.02 theta one but the zero is equal to 0 so that term just goes away so B 1 is just 0.02 times theta 1 so that's why if the first day's temperature is say 40 degrees Fahrenheit then V 1 will be 0.02 times 40 which is 8 so you get a much lower value down here so it's not a very good estimate of the first day's temperature V 2 will be 0.98 times B 1 plus 0.02 times theta 2 and if you plug in you know V 1 which is this down here and multiply it out then you find that B 2 is actually equal to 0.98 times zero point zero 2 times theta 1 plus 0.02 times theta 2 and that is zero point zero one nine six theta one plus zero point zero two theta 2 so again you know assuming if they the 1 and theta 2 a positive numbers when you compute this B 2 will be much less than say they want all theta 2 so B 2 is in a very good estimate of the first two days temperature of the year so it turns out that there's a way to modify the Zestimate that makes it much better and it makes it more accurate especially during this initial phase of your estimate which is that instead of taking VT take VT divided by one minus beta to the power of T where T is the current data you're on so let's take a concrete example when T is equal to 2 1 minus Bay to the power of T is 1 minus 0.98 squared and it turns out that this is 0.0 0.6 and so your estimate of the Thames on day 2 becomes be 2 divided by 0.03 9 6 and this is going to be 0.01 9 6 times theta 1 plus 0.02 Zeta 2 you notice that these two things adds up to the nominator Oh Penelope 9 6 and so this becomes a weighted average of theta 1 and theta 2 and this removes this bias so you notice that dump as T becomes large beta to the T will become will approach 0 which is why when T is large enough the bias correction makes almost no difference this is why when T is large the Purple Line and the green line you are pretty much overlap but during this initial phase of learning when you're still warming up with your estimate when bias correction can help you to obtain a better estimate of temperature and as this bias correction that helps you go from the purple line to the green line so in machine learning for most informations of the exponential weighted average people don't often bother to implement bias Corrections because most people would rather just wait that initial period and a slightly more bias estimate and go from there but we are concerned about the buyers during this initial phase while you're exponentially weighted moving average is the warming up or then bias Corrections can help you get a better estimate early on so that you now know how to implement exponentially weighted moving averages let's go on and use this to build some better optimization algorithms