Time Series Prediction

Key Takeaways

what happens if we predict the past oh hello world it's Suraj and I get a ton of questions about how to make predictions using time series data so I'm gonna explain eight different time series prediction techniques to predict the price of gold in this video in order of increasing complexity we can think of time series data as a sequence of data points that measure the same thing over an ordered period of time another way of thinking about it is that it's a series of numeric values each with its own timestamp defined by a name and a set of label dimensions and we're seeing this type of data set become more common in fact if we look at developer usage patterns in the past two years time series databases have emerged as the fastest growing category of databases autonomous trading algorithms continuously collect data on market fluctuations over a period of time it's like digital wolf of Wall Street well smart homes monitor data points like the temperature number of people in the house swag level and energy usage both Jeff Bezos I mean Amazon monitors how its many assets are moving across the world with a fine-grained level of precision and efficiency that makes same-day delivery possible all of these applications have one thing in common they rely on a form of data that measures how things change over time usually time series data sets have three commonalities the data that does arrive is recorded as a new entry it arrives in time order and time is a primary axis these data sets are generally append-only the data that has been reported doesn't change since it was recorded at some point in the past time series data sets are different than just having a time field as a column in a data set in that when we collect a new data point for time series data we have to create a brand new row for it we can't overwrite what happened before only by doing this will we be able to track all changes to a system over time recording data points over a series of time allows us to analyze how something has changed in the past monitor how is changing in the presence and even predict how it could change in the future so let's say we have a data set of the price of gold over a period of time from many days ago up until today we want to be able to predict the price of gold tomorrow using this data set how are we going to do that dark magic well before we do anything let's split our data into training and testing data we can use the popular machine learning library called scikit-learn and pandas to do this let's start with the most naive approach we can if we want to forecast the price for a given day we can just take the last day's value of our training set and use it to estimate the same value for the next day we can represent this as the equation y hat the price of gold of the next day is equal to Y the price of gold from the previous day that's the most basic timeseriesforecasting equation we could make then we can check the accuracy of our model on the testing data set by using scikit-learn z-- mean squared error module it'll allow us to compute the error between our prediction and the testing data set which are the real values for all the data points we predict we can compute the root mean squared error and it'll output a scalar value as you can see this value is pretty bad we could do much better so instead let's try using a different method called the simple average sometimes we get a data set that varies by a small margin through its time period but the average at each time period remains constant we can thus try and forecast the price of the next day using the average of all the past days the expected value will then be equal to the average of all previously observed data points it's not going to be exact but it can be somewhat close right actually when we compute the error again we'll see that this model didn't improve our score this method works best when the average at each time period is constant but even though the naive methods scored better than the average method it doesn't mean the naive method is universally better it just happened to be better in this case if in our graph we can see that there's a big difference in terms of the prices in the initial period and the later period perhaps we could improve our simple average approach by only taking the average of the prices for the last time period only we call this the moving average we just need to pick the right value for P where P is the fixed finite number of previous values when we test this out we can see that we are getting better at predictions our error is smaller and our prediction line intersects much more of our test data one thing we could possibly do to improve our model is add some weights to our values meaning it seems like the more recent the data points the more it matters if we gave different weights within the sliding window values which all add up to one we can mathematically define a preference for values that came later on in the series notice that both the simple average and the weighted moving average are on opposite ends of the spectrum in terms of how they approach the problem ideally we can use a method that incorporates ideas from both techniques combining them like a megazord it's clear that attaching larger weights to more recent observations and two observations from the distant past will work well but instead of manually deciding what those weights are let's define a smoothing parameter called alpha the rate at which the weights will decrease is controlled by this parameter and we can make the equation more simple in that the value of y hat just turns out to be the sum of just two products we've created a weighted moving average with two weights because the expression is recursive this is an exponential method simple exponential smoothing to be exact but still it's not taking into account just how much variation there in our data Charles Holt a professor at Carnegie Mellon noticed this problem a few decades ago and extended the idea of simple exponential smoothing to allow for forecasting of data with a trend this method called Holt's linear trend applies exponential smoothing to both the level and the trend the trend is the general pattern of prices that we observe over a time period and the level is the average value in the series to express this mathematically we need three equations one for level one for trend and want to combine both to get the expected forecast the level equation demonstrates that it's the weighted average of the observation and the width in sample one step ahead forecast while the trend equation shows that it's a weighted average of the estimated trend at time T based on the other values as well as the previous estimate of the trend this model is better but it's still not taking the variation into account as well as we'd like so let's look at one more improvement from our buddy Holt hold notice that sometimes there are trends that depend on certain seasons for example a store that sells winter clothing will usually make more sales in the winter instead of this summer and this is a pattern that repeats as the seasons change yearly we can call this pattern seasonality and data sets that show a similar set of patterns after fixed intervals over a time period exhibit this holds winter method takes into account both trend and seasonality to forecast prices we apply exponential smoothing to the seasonal components in addition to level and trend each has its own unique smoothing parameter a level equation will show the weighted average between the seasonally adjusted observations in the non seasonal forecast from time T the trend is the same as Holt linear method and the seasonal equation shows a weighted average between the current seasonal index and the seasonal index of the same season previously when we plot this we get our lowest error yet and our prediction finally looks like it's fitting or testing data but wait before you try and collapse the financial markets as great as that last method was keep in mind that so far we've only looked at univariate time series data meaning it's a series with a single time dependent variable usually though in the real world there are multiple variables at play and handling all of them at the same time is a skill that can be mastered this is instead considered multivariate time series data each variable in a multivariate time series depends not only on its past values but also has some dependency on other variables we want to use these dependencies to help forecast future values one of the most commonly used methods for multivariate time series forecasting is called vector autoregression or or VAR in the bar model each variable is a linear function of the past values of itself and the past values of all other variables if we have two variables y1 and y2 and we need to forecast the value of these two variables at a given time we can represent the relation between both as such we've got constant terms coefficients and error terms here in order to use the multiple variable terms in each equation we'll use vectors but what else could we do here well we haven't yet used a learning technique recurrent neural networks are able to learn how to make predictions in sequences of data and a variance of recurrent networks called long short term memory networks are able to learn from even longer sequences of data we can treat our data set as a supervised problem if we'd like and use an LS TM network to predict our target we could also use reinforcement learning where we frame the market as a Markov decision process and use real time reward signals from the market to help adjust our predictions as in if our buy and sell decisions affect the price of gold I've got videos on both of those topics link will be in the video description there are three things to remember from this video for univariate time series data Holt's Wynter method works pretty for multivariate time series data we can use a model called dektor Auto regression and if we're feeling ambitious we can use a learning technique like recurrent networks to help learn trends in the data rather than using a static model I'm so proud of you for making it to the end please subscribe for more programming videos and for now I've got to go forward in time so thanks for watching