ARIMA modeling and forecasting | Time Series in Python Part 2

Key Takeaways

[Music] hi welcome back to the stutter science doge a video tutorial series on time series in part 1 we left to that difference in our data to make it more stationary as this is a requirement of many time series models in part 2 we'll take our different stutter and start modeling on it and forecast into the future so what we need to do now is look at the autocorrelation function and partial autocorrelation plots or a CFP ACF for short so these plots help determine the number of order aggressive terms and moving average terms an autoregressive moving average model or to spot the seasonality or periodic trends so explain what I mean by order aggressive and moving average so order aggressive basically is able to forecast the next timestamps value by regressing over the previous values and a moving average is able to forecast the next timestamps value by averaging the previous values so auto regressive integrated moving average model which is the one we're going to use is useful for non stationary data as it allows us to difference the data plus has an additional seasonal differencing parameter for seasonal non stationary data so first let's produce these plots and then I'll explain how to interpret them so we're going to produce our first plots going to be a CF pot I know different style and we're going to produce a nice PACAF blood as well okay let's have a look at these okay so the ACF and the PCA F plot includes a 95% confidence interval Ben so anything outsides this kind of um you know shaded band here it's a statistically significant correlation so if we see a significant spike at lag X in the ACF that helps us determine the number of moving average terms and if we see a significant spike at lag X in the PA CF that helps us determine the number of autoregressive terms so here in the ACF plot we see a spike at about one here so that will turn help us determine the number of moving average terms and if we look at the PA CF we can see two major spikes here so one added about five and one I think in about thirteen so that will help us determine the number of AR terms for now we're just going to go ahead with a model that only includes about five AR terms and see how that goes so now that we have looked at our ACF NP ACF plots we can now build our ARIMA model that takes into account the amount of terms that we need to use and just keep in mind this models also going to infer the frequency so we need to make sure the gaps between our date times are we've always done modeling okay so let's call this um ARMA one model [Music] and I'm going to play our Reema model [Music] we're gonna give it our data and the order of terms is gonna be our AR ma terms and differencing so first of all put in number of AR terms here two rounds of differences or two sets of differences and one ma term here and I'm going to put an option here or specified trans parameters as false this kind of ensures if you said it as true ensures that things are kept stationary but you'll see why I have to set this as false later on in the video tutorial series when we talk about issues with our model and we're going to print the summary of our model um so we can get a few details little serious let's do that now we'll explain how to interpret the summary as well okay let's go ahead and run this so we've had a look at our autocorrelation and partial autocorrelation now we've built our model alright so this shows us a summary of our model here we want to probably look at the p values for our coefficient of our terms here so our our terms and our ma terms here so looking at this is is useful because if the p value for say an AR or an MA coefficient is greater than 0.05 which is our significance level I kind of cut off them up to determine whether it's significant or not then we can say it's probably not significant enough of a term to keep in the model so how you look at this we might want to remodel and include only this they are our ma term here as the other ones might not be necessary but for the purpose of demonstration let's go ahead and then we'll discuss issues with our model later on so next step is we want to predict the next 5 hours on the next 5 timestamps ahead which is our test holdout set so a comment these out so they're not too much of a distraction [Music] and we're given our model and we use the predictor function here and I'm going to give it the time stamps from the last time stamp was basically 6:00 p.m. on the 6th of February 9 2019 so I'm going to take the time stamps into the future from the last time stamp which is from 7:00 p.m. to 11:00 p.m. on the 5:00 time stamps ahead so let's do this I'm also going to make this type levels and you'll see why later on we need to specify that ok let's run this was it going to print these predictions obviously [Music] okay let's run this [Music] all right so here are our forecasts or our predictions for the next five hours ahead we can kind of see going this sort of downward trajectory here so it predicts that sentiment is likely to go normal to turn in a kind of bad direction but what we need to keep in mind is for time series we need to back transform our D difference predicted values with our D differenced or original actual values this is automatically done when predicting so when we specified type levels here we kind of wanted to predict on the original scale not on the d difference to kind of scale nevertheless we're going to demonstrate how to DEET rants form say two rounds of differences using cumulative sum when you've been given original data so the first step in that is we want to basically get the second round of differences back to the first round of differences and then take that D different starter and get it back to the original so kind of like it's two-step process so let's go ahead and do it through this so as I said we want to get our second round of differences back to the first round so I'll just call this undo one take our second round of differences and we're going to fill in any missing values just so they didn't cause the same problems okay and the next step we want to get that difference data or one difference data back to the original so this one do you [Music] fill in any missing values okay now we can compare these so the difference or they're going to be very small differences between our original data and our undifferenced utter we're going to round it up to six places after the decimal point I mean our values only come in six places after the decimal point anyway so they're not very big differences to care about but they're essentially the same when we do round it up six places past the decimal point so let's have a look at this and we'll just look at our original data first to about six places after the decimal point I want to see if it's equal to the same as on difference to total six places left of this one point and just for our iron sanity check we can just look at the first few values for the original values and compare it with the D difference values to see there on pop all the same you okay let's have a look at this okay cool so it's come back as true as if there are though differences or real differences between them so our on different starter and our original values are on par and you can have your own kind of sanity check here to make sure to say the first few examples are definitely the same now that we have modelled the data and made our predictions we'll compare our predictions against the actual values in part three thanks for watching if you found this video tutorial useful give us a like otherwise you can check out our other videos at data science dojo tutorials [Music]