Informer: Training and Inference

Key Takeaways

greetings fellow Learners now before we get into this wonderful world of Time series forecasting with the Informer I have a thought-provoking question for you what is your favorite part about machine learning is the modeling the understanding of data the hyperparameter tuning the testing or something else now as I make my own Journey as a machine learning engineer I've been understanding the importance of understanding more than just the code or just the actual core modeling in fact I personally enjoy ml system design as I build more AI solutions to products in companies so turning this question over to you what is your favorite part about machine learning and I would love to hear your thoughts now this video is going to be divided into a few passes as we are going to walk through the coding of the training and inference phase along with some other phases of the Informer architecture through this playlist of videos we have seen the Informer architecture at its core from like the core machine learning operation standpoint but now we are going to take a higher level in just trying to actually make you know load a data set try to make some inference on this so let's get to it so for this first pass what we're going to do is we're going to take the GitHub repository for the open source Informer architect Ure and we're going to open the given collab notebook that they have right over here and I kind of open that here in another Tab and in this notebook we're going to walk through it but try to explain every line going forward and I've also added some like logging features so that we can see some extra features going on but it's going to be fun so first we're going to execute this cell right over here to download the data set and and when you're doing all of this make sure that you activate your GPU right here in collab and once you do that you run this first cell you can also open the side panel in order to see all the file structures and you can see here that here we have downloaded the Informer code from the open source repository on the first line and the second line is going to download the data set the data set that we use here is going to be the electricity Transformer data set or the ET data set and the nature of the data it kind of looks like this where we have hourly Transformer based data where we have data from 2016 to about 2018 it's going to be about like 17,000 examples where we have a date followed by seven features over here now we could make predictions for all of these features using some multivariate forecasting but in this case we're probably just going to stick to oil temperature as the target variable coming back to our code here you can kind of see the data set is loaded over here and we'll be using the smaller version of this data set ET small so here we'll set a system path for the Informer 2020 and then we'll run now the section to train and test our experiments so we're just going to import a few libraries and then we have these huge list of arguments that we can kind of go through they're all arguments that we have discussed previously in a video when talking about the full code for the Informer but I'll also mention some extra ones here too so first of all the model that we're going to use is the inform architecture there's a couple of models that are here there's another model which is the Informer stack which basically includes the core Informer architecture but there's more that's kind of built around it too for this case though we're really just considered about the core informal architecture that we have been talking about through this playlist of videos and hence we set it as just the Informer next we're going to check the data source this is going to be the et1 data so this is just the current data set that we just looked at for Transformers and we're going to just Define like the path in which we can find that data set which is this ET small right over here and the CSV as well this H1 CSV next we're going to look at the features here the features we set as M which is basically saying that we want to perform like a multivariate analysis so we saw that there was like seven columns in the data we're going to be forecasting for all of those seven features and we could have just very well just executed it just as a univariate analysis if we just had like one of those features that we wanted to forecast over time or we can also do Ms which is multivariate inputs like seven inputs and then just like the single output for like the oil temperature the target we set to be oil temperature the column the frequency is H which sounds for hourly because we have hourly data in our data set then checkpoints the checkpoints folder is going to be set to be the inore checkpoints location let's just put that here and that's going to be appearing right over here so sequence length this indicates what is the number of input vectors that we are passing in simultaneously into our encoder architecture and this is going to be 96 time steps so in this case it's 96 hourly time steps which is basically like four days of data that we're passing directly into our encoder label Len is 48 this indicates the context vectors that we are passing into the decoder architecture to start our predictions because here the decoder architecture is typically going to take like the last chunk of whatever we put into the input that's going to be put into the decoder itself too in order to generate predictions and we're going to take the last 48 time steps to do so PR Len is going to be what is the number of time steps that we want to predict on the output of the decoder and this is going to be 24 so just to see like how these three numbers actually come to play let's look at an architecture diagram this here is an entire encoder decoder architecture diagram for the Informer architecture and if you want to know exactly like how each and every single component works I have created a video right over here but you can check that out later if you haven't seen it already but the main focus what I want to bring in is the input to the encoder and the input to the decoder so zooming into this encoder input you can see we have a batch size of 32 we're passing a 96 length of the sequence and each of those is going to have seven data points which we saw in the data set now we would pass this into the encoder but also for the decoder we're going to extract like the last half of this data set that is the last 48 vectors and then we're going to concatenate with some padding vectors this is going to be like some 24 padding vectors of zeros so that's going to be like 72 vectors right over here which you're going to pass into the decoder in order to make some predictions and eventually those 24 padded vectors that we have over here if we scroll all the way to the end of the decoder they are going to encapsulate our predictions over here and hence we have the numbers 96 48 and then 24 so coming back to the code we know why we now see these numbers next is the encoder in EN in which is seven that is the number of features per time series Vector in this case it is seven the deck in it's again the number of features passed into the decoder it's again seven C out is the number of features that we output from the decoder which is again seven for the prediction Factor five this is internally going to be used in prop sparse attention to determine the number of queries or key vectors that are internally going to be sampled so it's a it's a hyperparameter that we set for for prop sparse attention internally D model is again internal to the Informer architecture where we set the internal embedding sizes to be like 512 Dimensions n heads is eight this is specifically used in multi head attenion where we can we use multi-head attention so that the neural network can understand more complexities in data along eight different paths in parallel next is e layers is the number of encoder layers layers in this case we set it to two D layers is the number of decoder layers which we set to one dff is the number of or rather the embedding length of the internal full feed forward Network which is 248 next is Dropout this is just a parameter for the Dropout which indicates like the probability of the number of neurons to turn off and we turn off neurons randomly so that during the test phase we are going to see an increase in testing performance attention is going to be prob which is going to say hey for the encoder part we want to use prob sparse attention as opposed to full attention and prob sparse attention helps increase the efficiency for especially like longer sequences of Time series data time f is going to be like the time features and coding so this is going to um we're going to see like how exactly we want to encode Global timestamp Information into our current time series vectors from an architecture standpoint it's basically going to be used in This Global timestamp section in the beginning over here like how do we construct this that's going to be determined by the embed argument next we have an activation function over here which is going to be G next is distillation distill is equal to True basically sets distillation is going to be true and the distillation process is required if we want want some more efficiency in the encoder part of the architecture output attention this is going to determine whether or not we want to Output attention vectors so it could be either like true or false mix is true this is going to just do some very internal um rearrangement of parameters so it's a pretty minor argument here padding is zero indicates padding for our input time series vectors we want to set it hourly for hourly data batch size is 32 we want to pass in 32 items at a time to our Network to make predictions for 32 um series of data at a time learning rate this is like an initial learning rate that we set here the loss is going to be a mean squared error which is very typical for regression type problems LR adjusted or LR adj this is going to specify how our learning rate should be adjusted and if we look at the actual code base to see like where this is used you can see that the learning rate is going to be modified in this way over time or we could specify it as type two and depending on like the epox you could see that the learning rate is going to diminish over time in this way overall these are just going to be different learning rate methods to tune over time that they should decrease over time as we're trying to optimize for uh the correct Optimum mixed Precision training it's going to be set to false but if this was set to true it would use mixed Precision training which would typically all the gradients that are used in the network it would have cut the floating Point Precision instead of using 32 it would have used something much less like 16 floating Point Precision in order to in order to make the training process more efficient at the cost of maybe a little bit of accuracy num workers is basically going to introduce what is the efficiency of like the data loading process you can set this to different numbers for especially like larger data sets that you want to load it might be more efficient ITR in this case it's just going to be the number of times that we're actually going to train our model we're just sending it to one epox is the total number of iterations for every single ITR or for one training session we're going to train with six Epoch like over six times the entire data set patience is going to be a parameter that's going to be used in early stopping so in the event that our validation msse or means squared error error is not going to change over the epox we are actually going to stop the training after you know the third iteration after we stop seeing any improvement in our model Dees is simply like a descriptor and I know exp here in general is a variable that we're going to be using to reference the Informer architecture Now using GPU in this case it's going to be true because we have like a Cuda machine available right here and we set using the multiple gpus to false in this case so let's execute this cell by pressing command enter next we're just going to set the usage to be gpus over here and now we're going to do some loading of data so first of all there's multiple data sets that are available to us and like we mentioned before we're going to use the eth1 with some parameters that are listed over here so what this is saying is that hey we want to First just make use of et1 CSV which we already defined in our argument data above and what we're going to do here is we want to say the target is going to be OT which is the oil temperature that's the Target that we want to truly forecast and we're also going to say that the features here are going to be well in this case we already know features is M that is we want to do like a multivariate analysis that's how we initialized this to begin with so if we go way to the beginning over here I think we already initialized like features is m so scrolling down all the way here again you'll see that it's going to be 777 so enk in is going to be seven deck in is going to be seven and C out is going to be seven which makes sense because we have seven uh features for the that we want to pass into the encoder seven features we want to pass into the decoder and seven features we want to get out of the decoder so let's execute this cell and then we also have like a d details frequency over here so let me just execute this and just to show you that we want to do this on an hourly basis we can actually just kind of like see this as well so it should be H and it is H right and then if we want to look at the args and the experiments let's do that and you can just see all of the arguments that we have that we discussed just above now right here exp this is the first what is the kind of model that we want to reference this is going to be the EXP Informer architecture to understand what the EXP Informer is is that we go to the expore in.pay in the ex folder of the open source architecture and you can kind of see a class over here that says exp Informer and that's going to be used to like build the actual model so we have like building the model over here then we have a function to get the data over here and then we can also train validate and also make predictions on the data set over here too so all of this is just in one class right here called expore Informer and that's what we're referencing so we have exp Informer next let's load our data set which was in that et1 and we'll look at the data itself and you can kind of see here there's 177,000 records and there is hourly data for seven Fields so let's look at the arguments again right over here if we display the arguments uh you can kind of see the exact same arguments that we just talked about previously that's listed out right over here and it's these arguments that you know if you were building a custom data set yourself you would want to set based on the way that I've explained it so far and then training is actually quite simple over here so let me just start this this is going to kick off a training process so let me just start that training so that it'll just take like a few minutes anyways so when you're seeing this training process also pay attention to this side because there might be like some folders that are created here so essentially what's happening is that we now we going to create this file name over here and it's in this file name that we're actually going to create our uh training checkpoints so you'll have like the model stored over here once the training is complete now this line over here so let's just actually see what we're doing so we're going to use a GPU we have a GPU available next is this is going to be an Informer architecture we're going to use the etth H1 data set feature it's going to be multivariate M the sequence length is 96 the the label length is 48 because that's what we're passing into the decoder the prediction length is 24 o you can actually see the checkpoints that now populated over here right just to to see what's going going on right and then this DM is going to be 512 that's the D model that's 512 Dimensions internally eight attention heads we have two encoder layers one decoder layer internal 248 that's for feed forward networks and then uh we we're using probab we're using prob sparse attention in the encoder uh we're using a factor of five um we're using time frequency for um Global time stamp embeddings uh we are also using the distillation as well as mix is set to true and this is like the first training iteration overall and we only have one training iteration ITR was set to one after all so this here is now going to call the actual Informer or it's going to con it's going to call specifically it's expore Informer which will internally call Informer and then it's going to train the model right over here and then we're going to test the model and we're going to empty our cache so now the files that were created we just discussed were Informer checkpoints and we also got this results folder right over here for the same model and under results we have the True Values of like what this should have been we have the predicted values as well as some metrics so this is stuff that we're going to plot out and just like see later on just to see like how well our model is actually performing we can create some cool graphs and charts we can also visualize attention and so forth now let's look at like what what this here is actually saying so what happened is that we've iterated only one iteration because you know this is only one right it's like just one one time next what we're going to do here is you see this training validation set and test set of like how many examples we're using we are making logs for every like 100 examples and what you can see here is that you have a train loss A validation loss and a test loss you could see like for like these like first 100 iterations it's fine and you can see that there is probably not much of a difference right over here it's only the train loss that's actually decreasing out of here and because like there's not much of a difference between the validation loss here and the validation loss here we now trigger an early stopping and early stopping you can see is going to happen right here it says early stopping counter one of three because three was why three it's because we set that to be our patience and the this is going to prevent our model from overfitting because it'll be like okay validation is not getting any better what we're going to do is now we're just going to stop this model from training all together but we're going to stop after like three main Epoch iterations so that's why we iter at one here two and then three and then we perform the stopping criteria and then training is complete and during this time too you're not really seeing much of a Improvement in the model itself in fact the validation loss slightly increased in test error which indicates potentially there might be some more overfitting here which you know I could probably tune my model a little bit to adjust for accounting for this but for at least instructional purposes I hope you know how to like read this piece of information so that you can tune your model accordingly next is like testing here so what we're going to do is you could see that the prediction outputs we have a batch size of 32 and we are getting a prediction of 24 sequences because we're predicting 24 of the next time steps at a given time and each of those is going to have an output like number of features is going to be seven that's going to be like the test shape and we have 20 848 individual predictions here because we're just probably going to we're just going to like multiply these two together because after all they're just like a bunch of predictions so we're collapsing these first two dimensions and what we can do now is actually visualize these predictions just to see like how well they've performed and we have like um an msse as well as a mean absolute error so that's going to be the training phase now if you want to make predictions what you want to do now is first read your model checkpoint so let's just look at the checkpoint path and read this path right over here I created a Cell by accident so let's delete that and now let's actually just make predictions so if we do like make predictions what we're going to do is we're just going to like sample a single a single sample here that's right over here it says one which means that we're making predictions on just one sample and we already are using a GPU because it's just on right in the corner over here now when you execute this cell you're going to notice that there's like a single file that is created right over here called realcore prediction and this is just going to be like a single prediction so in order to see like what that prediction looks like you could see that it's just a 1 cross 24 cross 7 dimensional tensor and you can just like read that tensor itself and this is just going to be the predictions for all the seven features that we described quiz time have you been paying attention let's quiz you to find out which of the following types of predictions is the Informer designed for a univariate forecasting B multivariate forecasting C sentence translation or D text summarization note here that multiple options may be correct and I'll give you a few seconds to answer this question the correct answers are A and B but can you tell me why so comment your reasoning down below and let's have a discussion and if you think I do deserve it at this point please do consider giving this video a like because it will help me out a lot now that's going to do it for pass one of this explanation and quiz one but keep paying attention because I will be back to [Music] quizio so in P one now that described how to you know read the data and actually perform some training and testing fairly simple and also just like get predictions very easily let's actually look a little bit more deep into this predict function over here because it really looks like a black box right now so let's expand that out right here on more details about prediction so we have this predict function and internally what it's actually doing is just a bunch of stuff right over here so so this predict function is essentially first it's going to load data from this checkpoint that we described right over here and then we are going to Now set the model to evaluation mode being like hey we actually want to make some predictions here and then what we're going to do is actually load data from this PR loader over here and if you want to know where this is coming from specifically it's going to be a part of the EXP Informer get data and we're passing in a prediction flag and this is that definition of get data which is under exp Informer right over here so we are analyzing this data set right here we are then setting this flag to prediction which is here which means that the batch size is just going to be one because we just want to get a single prediction right now and then what we do is we create a data set and then we also create a data loader and return the data set along with the data loader data sets and data loaders work hand inand and are very common in pytorch so as to provide a unified way of creating and formatting data and also fetching and reading and writing data too and so jumping back to the code we can see that this is going to be our essentially our data loader and it's going to be loading batch X batch y batch x mark and batch y Mark so batch X and batch y are just like individual I guess since we're only we're in the prediction or the pr flag is turned on it's just going to be a batch of one and batch X is essentially just going to be the shape of like 96 cross 7 because this is going to eventually be fed into the Informer encoder now match Y is eventually going to be fed into the Informer decoder and it's going to be used as that context so it's the last 48 of batex and so it's going to be 1 cross 48 cross 7 right over here so that's 1 cross 48 cross 7 and this is going to be batch X is going to be 1 cross 96 cross 7 and then batch x mark is going to be it's going to indicate the um the global time stamps for each of these 96 cases so for each of these like 96 examples it's going to indicate for hourly data it's going to indicate four major Global time stamp points it's going to be the month of the year the hour of the day the week date and it's also going to be like the date of the month so that's going to be four and then the Y Mark is going to be very similar but for again for the decoder so for these like 1 cross 48 cases it's going to be just a size four to understand where that four number came from we already know that here we are we passed in like the frequency is going to be H right and H right here basically says that we're going to encode time we're going to add like four these features from time stamp data which is the month the day the week day and the hour and this is why we see that dimension for coming back to our code here we are going to determine what the input to the decoder should be so in this case the padding is zero that's what we described in the very beginning and because the padding is zero we're going to create the input to the decoder like we mentioned by taking the last part of the 48 from the encoder so this is going to be like 48 from the encoder that's this shape over here plus the prediction length the pr Len is the number of features that we want to Output from the decoder so that's going to be 24 and this is essentially going to be 72 and it's going to be both of these so that's like 72 right here times whatever the batch y last shape of it is so that's going to be seven so it's going to be um 72 cross 7 right over here and so when we actually print out the decoder you're going to see it's like 72 cross 7 right over here or rather like 1 cross 72 cross 7 in this case it's going to check if we want to use mixed Precision training which we're not going to do and so it's going to execute this part where should we output attension in this case we will execute this line where we're just getting the first case because here when we actually make the call to the model this is actually going to make the forward pass call to the Informer architecture and when you make the forward pass call to the Informer it's going to return two values the first is the main output and the second one is going to be the attention vectors so this here is just going to return just the output itself and this is going to return of course just the output if you know the attension values are false and we're actually going to print that out right over here just so that we can see what's going on and we already know that the output shape should be of 1 CR 24 CR 7 and let's just make this one in this case the dimensions over here this here is going to be false so it's going to be zero right here and hence what we're going to do is for the batch we're going to extract just the last 24 predictions and so it's going to be like a 1 cross 24 cross 7 right over here because I think batch y was like 48 this is now going to uh 24 the prediction outputs outputs over here was going to be also like a 1 cross 24 cross 1 cross 24 cross 7 and the outputs that we see in the predictions over here is also going to be the same 1 cross 24 cross 7 and that's going to be our single prediction right here and then we're going to save all of this into the results folder which is exactly why we see this folder right over here and we're going to save it under like the real predictions too and how do you executed this then you can also see here that this folder would have also been executed so you can see how this entire function is just a deep dive of like what we would see just by using this like single line of code right over here so scrolling here you can actually see for example let's just confirm that what we're seeing is correct we're seeing the we we're in flag is one prediction mode is one the batch X this is going to be passed into the encoder that's 1 cross 96 cross 7 which is correct and this is the value of it it's a long tensor next is batch wide this is going to be passed into well the one part of it passed into the decoder right over here which is 1 cross 48 cross 7 and this is the value you'll notice that with batch X though like batch X and batch y are actually related like we mentioned before the the latter half of batch X is batch y so for example you can see this last value of batch X here it's going to be 0.387 and you can see in the last value of batch y it actually matches right and the same is true for you know if you want to go here 0.264 and last but one is going to be 0.264 so you could see that like maybe you could see like there will be a halfway point somewhere over here where all of this is essentially going to be the same as what you see in batch y over here batch x mark is going to indicate like the time stamp information which is going to be of like the four feature lengths and these values are going to be normalized internally and that's why we see them as such here and this along with of course like the 1 cross 72 cross4 tensor for batch y Mark is in all of these four cases batch X batch y batch x mark batch y Mark they are going to be passed into the Informer in order to make the outputs so you can see here that the decoder input right now is going to be 1 cross 72 cross 7 where so you can see the 48 of the 72 is just this is essentially batch y right and you can see that what's being passed into the decoder is just a bunch of zeros over here which is just the padding and these are eventually going to be updated with the outputs that's why if you look at the outputs which is 1 cross 24 cross 7 the these are the outputs that that we are looking at right here the outputs of the model quiz time is that time of video again have you been paying attention let's quiz you to find out why do we pass in batch x mark and batch y Mark into the Informer a to transform the current vectors to recognize complex patterns B to add position level information to the input vectors C to add timestamp level inform information to the input vectors or D they are optionally added as they don't provide much value note here that multiple options may be correct I'll give you a few seconds to answer this question the correct answer is C but can you tell me why comment your reasoning down below and let's have a discussion and that's going to do it for quiz two and pass two of this explanation but keep paying attention because I will be back to quiz you now for this third past I actually went ahead and restarted The Notebook just to retrain the model CU I think there was some issues with like initial variables when I was just trying to train it here because I did see that the model was overfitting quite a bit and so you can see like the visualiz ations of that oil temperature for the 24 time stamps now look like this and just scrolling back really quick to the top over here you can see that the early stopping criteria let's go over here you can see that the model trained for slightly longer because it no longer stopped as soon as it did before because I think after this first Epoch the model actually started encountering situations of overfitting but at least here you can see that the validation loss is actually still continuing to decrease it's only until it hits this point that the validation loss increases a little where we're now encountering early stopping early stopping and eventually we stop model training so I just wanted to throw this out there so that you know why like the graphs look different a little bit and so we get a chart that looks like this now let's actually get a comparison between these predicted values and also the actual values just to see how you know you can visualize what errors look like like so we'll go to this visualization section over here and we're going to make use of under results we're going to do we're going to compare the True Values to the predicted values and in this case we actually read 2848 test examples for which we've made predictions and we want to just like make predictions on them so I'm just getting the first of those samples and of these seven cases or these seven um features really we really cared about the oil temperature right so this is going to be like the set of predictions for the 24 predictions for the oil temperature over time and those are 24 predictions of oil temperature that is the actual like ground truth and if we use map plot live over here and plot these values you can kind of see that okay there's at least now some semblance of how the current oil prediction which is like the orange chart is kind of mimicking and doing pretty well in mimicking the the ground truth itself now because like we set before that we set the features to be M which was like a multivariate you can also predict some other charts too right like the Huf so instead of predicting the last feature what we do is we predict the first feature or the First Column which is zero and it looks the comparison looks like this and we can also use that to predict let's say the second column which index with index one which is the H prediction that looks kind of like this it's not picking up very well on this specifically but I think you get the point of how you can also visualize your multi-variate um prediction cases what we want to do right now is try to visualize attention that happened in the encoder architecture so we want to see those attention values we can again create our data set and data loader and we're loading this in the testing envir environment just so that we can see like multiple examples we then now initialize our Informer architecture model by reading the trained architecture that we stored right over here in the checkpoints and what we want to visualize it we're basically going to again construct for the Informer it requires again a batch X batch Y and also the batch x mark and batch y Mark like we described previously and we would want to pass all of that into the model itself and the model is going to Output the actual model outputs and also the attention matrices now in this case we had two encoder layers and so we will have these like two attention variables over here so one of these attention variables is going to be 96 cross 96 across eight attention heads this is because the input sequence length was 96 cross 96 and we did multi-ad ATT tension so there's eight and then after the initial encoding we're going to do distillation which would squish the number of um time steps by half only passing in the very richest information and then we pass this into the next encoder which is going to be half because it's half of this it's going to be like 48 cross 48 if you want more details on exactly like how we get these shapes you can look at my previous video on the entire coded architecture of the Informer which I'll link to right here and so what we can do is has now visualize eight of these 96 cross 96 attention matrices as like heat maps and so let's just do that so you can kind of see here that this here is like for the first encoder layer we have performed like prob sparse attention now what was the attention values that were learned because in the input we passed in like 96 sequences so we have like a 96 cross 96 tensor and we have that heat map and you can see that there are some attention values that show some interest or importance over here and these values range between 0er and one right so these are the higher attention values similarly this is for layer zero head one so this is the next attention head and like this we will have eight such attention heads right so if we scroll all the way down here I also wanted to see okay what is the attention that is going to be learned by these uh the next layer and this will actually show like whether distillation truly helped propagate these layers correctly like for example you can see that in this the second lay head I guess in this case you could see that there is still information here that is retained and if you compare this to the first layer's second attention head what do that look like it looked like this too very similar where we had only very few of these examples truly of like actual importance so what we were able to now visually see here is that even though this heat map or Matrix is technically four times larger right it's 96 cross 96 versus the next layer 48 cross 48 but this 48 cross 48 still was able to retain the information that was also seen in the input which means that distillation truly did make the compression that we needed to while still retaining the information that we wanted and this is how like through using distillation you can like visually see how important the distillation process is in actually making the Prov spars attention efficient so that's kind of the main Crux of what I wanted to show you in this video and if you wanted to have and load your custom data set you could do so by just replacing in the CSV using the format that I provided like using the original data set format of creating a date column it has to be called date and then you have whether it's hourly or daily data whatever it is in that order you would also create the individual columns that you would want to forecast and then you set the argument parameters accordingly so you have your data and then you set your argument parameters you create your um data set you create a data loader and then because you have a data set and data loader created you can now create these batch X batch y batch x mark batch y Mark and this information can all be passed in to the Informer architecture to make predictions quiz time ooh this is going to be a fun one why is distillation performed a to reduce the size of the tensors in the encoder B to make the attention operation more efficient C to increase the accuracy of the concatenated encoder features or D to increase the accuracy of the decoder outputs note here too that multiple options may be correct I'll give you a few seconds to answer this question the correct answers are A and B but can you tell me why comment your reasoning down below and let's have a discussion and at this point if you do think I deserve it please do consider giving this video a like because it will help me out a lot now that's going to do it for quiz time and pass three of this explanation but before we go let's generate a [Music] summary as a summary of what we have done in this video so first of all we took a look at this collaboratory notebook book for the Informer architecture where we started with coding and downloading the data set we then experimented with just training and immediately testing this data set using like very simple commands and then we dug deeper into like okay how do we make predictions using that predict function what does it actually entail and also showed some visualizations of comparing the actual predictions to the true predictions as well as some attention vectors just to show how how distillation really is valuable in this entire process and then also left a snippet here for your custom data set and data loader and that's all that we have for today thank you all so much for watching this playlist of videos the link to the main architecture diagrams and S of this code is going to be down in the description below and also if you wanted to see exactly how the and if you just visited this video on its own and you're still now curious about how the encoder works and if you just saw this video on your own and you see okay this is how I use the Informer but you're really interested in how the Informer actually works well you can go back to the beginning of this playlist and I will link that first video right over here thank you all so much for watching if you think I do deserve it please do give this video a like and I'll see you in the next one bye-bye