Financial Forecasting using Tensorflow.js (LIVE)

Key Takeaways

to alright your life hello world it's Suraj and welcome to this live stream I'm so excited to be here with up with my new livestream setup in this video we're going to be predicting stock prices so financial forecasting using tensorflow TAS in this so what you're seeing right here is a graph of us predicting stock prices using what's called a recurrent neural network but the actual aim of this video is to build a financial forecasting model using a convolutional network so we're gonna be looking at both so I'm actually gonna code out a convolutional Network in the second half of the video but in the first half we're gonna talk through some math we're gonna talk through the different models out there for a financial forecasting and I'm going to also answer some audience questions because what is a live stream without some audience questions right so hello everybody yes good to see so many people here I'm gonna just like sporadically be answering questions in 5-minute intervals but really it's it's got to be related to the topic right and also in terms of a demo I've got a great demo for us here in terms of it you know an actual visual demo what this is is a nice little web interface that we're running off of localhost and what this is doing is it's using a neural network to make predictions based on just simple binary data right so this is just an example of us being able to visualize what it looks like right so what we see here are the actual labels right the outputs 0 1 0 1 and we have our predictions which are which are right here inside of this tensor and they look very similar and they're gonna get better and better over time right so that's that's the visual demo but we're going to be predicting Apple stock ok so that's that's the that's the actual aim of this video all right so I want to start off by just answering two questions and then we're gonna go right into the into the code alright so the question one is should I do masters from the USA or waits to earn some money here and go for a PhD later ok so I think first of all if you're gonna do a masters or a PhD it has to be at a really good school right it's got to be like at a top-ten school and if you're not gonna go to a top-ten school you might as well just do this yourself using the internet as your university I mean look at the most popular.we institution in the world right deep mind if you look at their jobs page they don't even have PhD listed as a requirement right what a PhD does is is that it doesn't just say you know you are the greatest what it's what it really says is you have published actual research under the Advisory of a professor or someone who is more knowledgeable you can do all of that with the internet you can find an advisor online you can find educational resources online right so that's my thought on that and one more question before we get into the code why not Python great question so guys javascript is really coming up in the machine learning space right now right now you can build a model in Python import it to JavaScript and vice versa and yeah so javascript is really becoming easier and easier to use for machine learning models and you're gonna also start to see more developers build models in JavaScript but most of the complex models are built in Python right so if I want to see someone build alphago in JavaScript and once that happens and it will happen and then I think the playing field will be a lot more you know level anyway back to this to the two of them alright so time series data is starting to play a larger and larger role in our world right so examples include examples include self-driving Tesla's right so Tesla's are constantly creating time series data right they're mapping out to where they are and then these are 3d point clouds and then over time this this comes out to be a time series right so by the way a time series is a sequence of data points right measuring the same thing over time this could be 3d data points through 3d map points for Tesla's this could be stock prices this could be smart homes measuring temperature changes over time this could be open data publishing police departments of crimes happening at certain times so time time is the key word here as data changes over time that measurement is a time series so imagine sensor is collecting data from three settings right you have a factory you have a city new a farm and so it's measuring temperature it's and measuring the number of people in a room it's measuring you know the output of a factory all of these are static data points that change over time because time is involved it's a time series exactly cool right so when it comes to financial forecasting ARIMA is like the go-to model right so a Rhema it's called is the go-to model and it stands for auto regressive it's a it's an auto regressive integrated moving average so the first question here is is ARIMA a machine learning model ARIMA I you can consider it a regression model it is it is a regression model is it a machine learning model I would say no because it was handpicked for a specific use case stock prediction but you can't really use it for things outside of that right it was hand picked and hand created for a specific use case whereas a model a machine learning model like say a support vector machine is made for a more general use case you can use support vector machines to classify cat faces you can use support vector machines to classify stock prices you know anything really but ARIMA you know have a little algorithmic outline of what ARIMA looks like here but it comes down to is us approximating three different variables the first variable is the the number of auto regressive terms the second variable is the non seasonal differences needed for stationarity and the third variable is the lagged forecast errors in prediction equations so all three of these variables have their own Associated equations so we're calculating all three and so it turns out that this has been the most popular model for stock price forecasting but I'm not going to go into that because with the advent of deep learning technology very recently we can we can outperform models like that and there's reasons and there is reason to believe we can because there's a bunch of machine learning models out there right but neural networks when given a lot of data and a lot of computing power tend to outperform all of them most of the time so why wouldn't they when it comes to financial forecasting and you might be thinking well why don't I hear more about this because big banks and you know hedge funds of course they're not gonna give away these models right that this is not an open source industry right finance is not an open source industry we'll talk about Bitcoin and all of that later but like in general traditional finance so these are very close source models why am I making a video on this because I want you guys to win and I don't care it's not about you know that for me it's about democratizing AI so this is just for me this is like I don't even care about this but like I don't even care about like predicting stock prices myself but I do care about you guys knowing about recurrent networks about convolutional networks and so this is a medium for me to explain that to you okay so why deep learning right so I like I said when you when you when you look at the performance versus the amount of data deep learning outperforms all of those other models when it comes to having a lot of data so what is the obvious approach here now this is very exciting right so what is the obvious approach here the obvious approach when it comes to time series prediction is to use a recurrent neural network okay so I'm gonna I'm gonna explain how this works mathematically so we have some input data right so first of all first of all we have a sequence right so we have a sequence of numbers so let's say 0 1 2 3 4 5 6 7 8 9 10 and we want we were gonna use the first nine numbers to predict what the next number is it's gonna be 10 and there is some function that describes this sequence in our case that function would be f of x equals x plus 1 right where X is the term that we are trying to predict the next term in right that's the function and we can use an recurrent network to map out this this this function and learn what this function is there's a function that represents everything in life there's a function that represents how much attention you are paying to me there's a function that represents how loud my voice is in relation to everything else there's a function for beauty as a function for love there's a function for everything and we can learn these functions using machine learning technology right so when it comes to recurrent networks normally would feed-forward networks we are feeding in every new data point right so we have term one term two term 3 term 4 and that's it but we're current networks they're fed in not just the new data points they're fed in the hidden state the learned weight value that matrix that's initialized is just 0 0 0 0 0 right just all zeros that's getting optimized over time it's fed that in as well so it's the input and the old weight matrix from the last time step that are both concatenated together combined together and then fed to the model and that just keeps repeating and because the model is fed in the previously learned matrix values from the weight from the last time step it's considered a recurrent network so mathematically speaking we have our input X of T okay so where T is the time variable where we have X of T equals W of T which is the input data plus s of t minus one where that is the hidden state from the last time step so it's fed both together that's the input and so when it comes to the hidden layers for as many as we have and that's what this Sigma value means right this big e its Sigma notation it means take this this operation or these sets of variables here and whatever operations there are and add them up for as many as you have so in our case that would be the number of data points so we're gonna take that input value X of T like you we just talked about here we're gonna multiply it by the weight value right at that at that time step and that's gonna give it in the function of that that operation is our hidden layer okay and that's why it's called s of T and that's why up here we say s of t minus one because this is our new hidden layer our new hidden state and this is our old hidden state from the last time step and so now we have this hidden state we can use it to compute an output so we take that state s of T we multiply it by the last weight value V of K and we do that for as many inputs as we have that's why the Sigma notation is there that's our G of that's our G of X X is this function and the result is our output and the output is a label or it could be the next term in the sequence and that's Y of T okay so that's that is a recurrent neural network it's it's a it's a universal function approximator that is learning a function by being fed in new data points as well as what it has learned in the previous time step and this is the go-to strategy for regression models right predicting the next data point in it in a sequence but there is a problem and before I talk about that problem I'm gonna answer two questions again from the live stream okay so the first question is how old are you that's not related but I'm gonna answer it I am 27 years old can you believe that I'm 27 years old what that means is if you are 27 younger older age doesn't matter right nothing we live in an absurd reality right we're credentials age where you're from and all of that doesn't matter anymore all that matters is how motivated you are because you've got the Internet you can literally do anything you want now you have the inner you can become a world expert in AI as a college dropout you can do anything right so that's the reality we're living in I'm 27 and one more question about AI specifically can we use reinforcement learning for time series data you guys are speaking my language oh did someone just donate some money a hundred pounds or your euros so I gotta answer this question I didn't even know I nee PI enabled this but um what was the first question in time series you yes you can you can and I have a video coming out on that tomorrow rupees okay he's learning so de bont did Dominic is learning tensorflow required or being able to use chaos is fine look up the the more you learn the better the more you learn the better you're gonna be okay but I would say yes learn tensorflow don't just learn careless start off with chaos because it's easy then learn tensorflow then learn just just progressively get harder and harder over time don't just start with like buying a giant linear algebra textbook and saying I'm gonna start learning with XI learning today and then you're like learning about things that you don't necessarily need for machine learning right just give yourself these small incremental rewards to keep you going on the learning path this is what I do okay and as you get those rewards you're gonna become more and more confident in this space because in the end a degree a credential a PhD all that is is you being able to tell yourself that you have learned something right so as long as you can tell yourself that you've learned something you will be more confident to learn the next subject in the future right okay anyway guys back to this there's a problem with recurrent networks the gradients that's computed to optimize the network over time it it diminishes it vanishes slowly right because forward so there are two processes two processes in a neural network okay I'm gonna I'm about to go into the math right now so you better hold on to your butt's okay this is this is some good stuff when we are forward propagating the gradients right we're doing input times weight add a bias activate right input times weight add a bias activate we just repeat that over and over and that's that's the feed-forward process until we get that output value we have that output value we compare it with our actual label we compute an error we use the error to compute the partial derivative with respect to our weight values weight one weight to a three in the reverse order so we go backwards we use that partial derivative to construct a gradient and a gradient is a list of partial derivatives that we can use recursively to come to update those weight values using that gradient but the problem is over time that gradient gets smaller and smaller and smaller and smaller until it's not so the the layers at the beginning of the network aren't updated as much as the layers at the end of the network and this is a problem because we want that gradient to not diminish over time because we want our network to learn sequences so someone came up with a solution called an LS TM network what this is is turning one equation into three equations that's what it is and we can talk about you know gates and all of that but really it really comes down to is this this set of four equations here the three equations really just make this one equation so I'll talk about this in a second what we are really doing in an LS TM network versus a normal recurrent Network is we are replacing one weight with three weights the right we have that input times weight there's a weight that's a matrix at every layer as a group of numbers that are that are learning over time what an LS TM is is three weights we have an input gate of four and an update gates and all these gates are many perceptrons there are many neural networks and they all have their own weights and they're represented by these equations so what we do is when we have an input and it goes into the first layer which is an LST M cell this is a cell for all of these four equations we multiply the first weight by the hidden state times the input we apply a sigmoid activation function to it and that gives us a value Z of T now we take that value we do the same for that so that's our we have an input gate we ever forget gate we have an update gate so we do that for the input gate and we have a forget gate we multiply that weight by the hence a times X and that gives us our sigmoid we have the third weight value our third gate weight value times hidden state times the previously computed value here times the input and we apply the 10h activation function to it not the sigmoid and this gives us H of T with the with the squiggly over it and we use all three of those values here to compute the final hidden state so what are we doing first of all if this is confusing that's ok I'm going at it right now what are we really doing in every layer of a neural network we're computing a hidden state so that's what this is this hidden state it's the same result we're computing a hidden States but we're using three weight values instead of one-way value to compute that hidden state and that's what that is and we could repeat that over and over again and what this does is inside of those three weights the gradient is being captured right it's learning what to forget what to remember what's relevant in the sequence and that's the whole idea behind LS TM networks so the gradient doesn't diminish it doesn't vanish it stays static over time which is what we need we need full upgrades for all of the weights in our network so when it comes to predicting a sequence and the sequence is Apple stock we have we can have a sliding window right so we have W which is some interval of prices over time and we say WT plus one and that's going to give us the next one so given these prices what are the next prices so we can think of it as sliding a window across a time series its that is what it is so we look at feed-forward Network data we have the number of examples right how many data points times the number of inputs by the number of inputs but when it comes to recurrent network data we don't just have we have a third dimension time right time matters and we have a you know a number of Time series examples the values at every time step and the number of and the number of time steps so it's a three dimensional input for this recurrent network data it can be more actually but it comes down to having time as that third dimension and we can squash the dimensionality using a dimensionality reduction technique to do that okay so we we have some smart people in this chat room by the way guys I'm so proud of just that the level of interest in AI and just the the yeah just you guys are yea really impressive people boys Ertz okay so um right so we can think about W so W is that window we can think about it as the sum of all of those values before it so W of one is the sum of all the values before it W two is sum of all values before it right and so inside of our LST m we're taking that input we're giving it to the LST m and it's gonna output the next value in that window WT plus 1 weight W subscript T plus 1 so the next time step and so that's called back propagation through time when we're applying back propagation to or current networks is called back propagation through time so I'm going to build a model at the end of this but I first wanted to show you really quickly a simple LS TM model in Python we're gonna get to the JavaScript I'm literally gonna build that but I want to show you a simple Python model first what we do is we create this data set right here using this data pre-processing technique we have our input data we have our output data data X is our input data why are our labels we fix a random seed because these values are generated randomly inside of this test example but we want it to be reproducible that means we want the same random values to be generated every time we run this for testings for testing purposes and debugging purposes we'll load up our dataset will use pandas to process it by the way pandas great great Python library for data pre-processing will normalize it so it's all on the same scale we always should have all of our data on the same scale between 0 and 1 or whatever so that it's easier for our network to find the relationships between all these data points we don't want one feature to be between 0 and a thousand and another feature to be between 0 and 1 right that's it's way too big right we don't want one feature to be a list of strings and another feature to be images right that's completely different we want that data to be similar we want it so that's where normalization comes in so that our network can see hey this is all just data you know this is all just numbers that are that's on the same scale I can easily see the you know the line line of best fit you know whatever it is between all these features we split that data up we reshape it we feed it and here it is here is that complex lsdm network that I just talked about we can do this in five lines of chaos that's where we are today obviously if you want more detail and you can add in tensorflow or numpy if you're gonna be a baller you're just gonna build a straight add numpy and then we make a prediction right so model dot predict train date a model to predict testing data and then we can graph it out and this is what it looks like just like this seems you know the red is our prediction blue is the actual the real data the time series works well enough now to the to the good stuff what we've been waiting for so I'm actually not going to use a recurrent network I'm going to use a convolutional network you might be thinking what suraj convolutional networks they're used for images why would you use them for sequence classification that's that's crazy talk and before I answer that question two questions from the comments here we go all right two questions is is asynchronous actor critic reinforcement learning practical for trading thank you for your live performance it's quite some effort to improvise thank you yes okay so short answer is yes reinforcement learning in general is a very underutilized technique for financial forecasting and it's a very interesting research problem in general you have multi agent multi agent networks you can think about other traders as agents you can think about there's a lot of different ways to frame this in a reinforcement learning context that hasn't been before where your environment is your the prices and the algorithms and the traders the humans are all considered BOTS and you're interacting with this environment inside of this what is considered a Markov decision process that's partially observable because you don't know the values of their account balances or you know things about them in general so that's why it's partially observable and that makes it a very interesting research problem so yes - how will you save your trained model we're gonna save it as a PB file proto buffer right this is the standard for tensorflow it's a standard for curraghs and it makes it easier to export models between different frameworks and languages even you can train a model in tensorflow with Python and then use it in tension flow is with one line of code and vice versa as well very great for portability back to convolutional net works right so convolutional networks are generally used for image classification right and they're used for image classification because they outperform most other models when it comes to image classification and what it comes down to is again it's a series of matrix operations that's it that's what all neural networks are they're a series of you know matrix operations add subtract multiply divide simple operations between groups of numbers that are called matrices input times weight at a bias activate right so here's how a convolutional network works so we can think of an image as a matrix right with RGB values right between 0 and 255 it's a it's a it's a an image is really just a matrix of numbers that specify how red how green how blue every pixel should be now in the context of a convolutional Network when we feed in these images into the model we don't consider it a two-dimensional matrix we consider it a three dimensional matrix why because the third dimension is RGB right so RGB is one dimension and then the other dimension is the width and the third dimension is the height so with height RGB there are three dimensions and so that is what a convolutional network is used to a three dimensional tensor right because a tensor there's okay so little linear algebra terminology here a scaler is one number a vector is two or more numbers no wait no no a matrix okay so let me let me start over let me start over a scalar is one single number a matrix is 2 is 2 a row in a column of numbers a tensor is a generalizable form of all of them right so tensor can be a scalar a tensor can be a matrix it can be a vector of size and to the you know 100 dimensional it could be a hundred dimensions so attention is just the most generalized form of saying all of this that's why tensor flow it's that's why it's called tensor flow because tensors inputs are flowing through different operations in a computation graph and that's a neural network so we have our input and so in suck so here's here's how it works the convolutional networks have a really it's it's three operations that are repeated over and over again convolution riilu pool repeat that convolution riilu pool what the convolutional layer is is it's a flashlight that slides over that input image and it computes a filter here's what I mean so we have our receptive field okay so what that is is it's a matrix and we are continuously performing the dot product on that input by that filter and we're sliding over the image so that value plus 1 that value plus 1 value plus 1 go down again that value plus 1 that value plus 1 that this one what's up yeah devalue plus 1 the value plus 1 the value plus 1 right and it could be plus 2 that's the stride so the stride is the value that says how how big are the intervals that I'm sliding this picket before it could be 8 right but we're computing the dot product between this randomly initialized filter and that input value in that and that input matrix tensor and it's it's it's computing this this output right that that's our output filter and we repeat that so that's the convolutional layer right so it's once we have computed that filter we apply we apply an activation function to its called really which stands for rectified linear units which is in the equation for that is y equals max 0 x y equals max parentheses 0 comma X what that means is it's negating all that is those negative values and that are that could be computed so everything's got to be positive and what this does is two things one it makes sure that the network can learn both linear and nonlinear functions so it's a universal function approximator and it makes sure that the gradient doesn't vanish so riilu was also invented as a way to prevent the vanishing gradient problem that we talked about and recurrent networks and so once we have that riilu then we can do pooling and so what pooling is is us saying you know once we've this is the convolving part right this is the flashlight part it's multiplying by the dot product by that filter by that input over and over and over and over again and then we have that we have the output filter and then so what pooling is is those saying what is the what is the what is the best I'll talk about best what is the best values from every region of this filter that we've computed and that's gonna be our output and what this does is it reduces computational complexity which makes the model faster and so we see here is like let's say this is our filter here with single depth slice if we perform max pooling so that's one form of pooling we take the maximum value from each reason region in this region it would be six in this region it would be 8 3 4 right you see what I'm saying and so what's happening is as we move through this convolutional Network there's there are more and more filters that are be computed it could be like four sixteen thirty-two but each of these filters are smaller and denser and they're more specific to certain features right so it's it's it's it's this level of abstraction where the biggest filters are like head and then it goes to like you know eyes shapes curvature you know like very little details and the output then is going to be one of several classes right so class values is the cars the truck is it's a plane so that's a convolutional network why apply them to time series data okay so two reasons one is so a recurrent network was invented to look to to make sure that we take the context of the previous sequence into account right so if we're gonna predict the next word and say a sentence you know I'm going to the bank to deposit some so you know what I'm gonna say money but a machine doesn't necessarily know that we we have to take the context of Bank today you know me what I've done in the past into context so the recurrent networks are really great I remembering the past they're not great ever membrane not remembering but taking the future into context as well so if we have some testing data we know what the ten next points should be that doesn't matter in a recurrent network only the past matters so convolutional networks are good at seeing both sides are better than recurrent Nets at taking into account both sides of the timeline second of all convolutional networks are faster there are less computationally expensive so you can run them on big datasets in a much faster time scale third convolutional networks take into account look locality right so the locality of features because this is a good thing when it comes to images the same the same feature that matters locally like say you know the the blackness of my eye also matters here right so if it sees like white in this shape it'll know like locally speaking well there's probably black and it sees here Oh white there's probably black whereas you know recurrent networks are more generalized into taking the entire data set into account whereas convolutional networks focus on low on the locality of features so when it comes to prices for example you know what what bitcoins price was five years ago is irrelevant to what its price is gonna be now if you think about it right now there's a whole everything is different right so that can apply to stocks as well right so high-frequency trading for example right making trades at nanosecond timescales that is all about locality of where we are right right exactly now and what's happening and so that's that's why convolutional networks are better at there they have fewer sequential calculations they tend to be more computationally efficient and they focus on not looking at the the data holistically as a whole but more capturing those and to be specific temporal relations from the beginning of the time to the end of the time all right so and here's the here's a little complexity chart okay so two more questions then I'm gonna code code time is coming okay so what are the questions here can you share that notebook yes it's actually in the video description and the second question is why should we not add even more weights than the three in LST M because as and Ray karpati says there is a diminishing return to adding more layers right so up to a certain points generally like three to four or five six layers you know you have marginal returns but then after that it's a diminishing return and you're adding computational complexity because it's all about trade-offs okay so now let's build our recurrent Network okay so we're gonna build this using tensor flow at j s now I want to say that we have built out this front end so the front end I talked about his here what what I want to do now is actually build the network itself and focus on that because that's the most important part we tend to float at JSU just you just import ten float j s and your HTML and that's it right so there's you don't even need to install it really it's just learning it as long as you have an internet connection it's there so we have that so let's build our convolutional Network that's step one we're gonna build our CN n we'll call it this constant built CN n that we're gonna use later so this is going to be a function of the data that we feed the model okay and so that's what that is so what are we going to do inside of this function what we're gonna do is we're going to return a promise and you might be saying well what's a promise a promise is a it represents the eventual result that we're going to have of any asynchronous operation which is in our case is going to be the the data that we are convolutional network that we that we compute so it's a placeholder okay so it's a placeholder and then we're going to compute a result later on and so that promises value is going to be a function or neural network and function great new promise function resolve reject ku-ku-ku-ku now let's start building this thing so the first thing we're gonna do is we're gonna say we have our model using tensorflow dot sequential because this is a linear stack of layers you know layer 1 layer 2 layer 3 for more complex models we would want a nonlinear stack of layers so it can branch in two different directions a lot of models that incorporate attention mechanisms can do this but we're gonna start out very simple with a sequence you know simple sequential linear stack of layers so now we can start adding these layers using the TF dot layers function right so each latest the layers is a high level API it's like carrots but for JavaScript ok so we're gonna say well our first layer is gonna be our convolutional layer right because our convolutional networks have these these layers and so the first layer we're gonna set we're gonna have to specify some of these some of these values and I'll talk about them in a second we have a filter we have our Strides what else we have to specify for a layer we have our activation function we have power kernel initializer and right if you look at the attention flow documentation all of that is there and so I'm just kind of like you know based on what I've seen doing that right so for our input shape what is our orders are what is our data gonna look like by the way so our data is going to look like our by the way I can't believe I didn't even show that this what our data looks like we have the dates and we have the the closing price right so that's that's what that is so we have one price for the input data so given so this is our input the date and the price is the output that's what we're predicting and so because convolutional networks prefer a 3d tensor as input because they're used to images what we're gonna do is we're gonna turn this date into a three dimensional tensor whereas one dimension is the year one dimension is the month and one dimension is the day and so it's a 3d tensor as input the label is this is this price okay so when it comes to our input shape we're gonna say you know assuming that we have that the dates because we have the data that's being fed into this CNN which is which are the you know the the input data we're gonna say it's gonna be the length of the dates you know buy one now when it comes to our kernel size we'll say let's say we want a hundred of those data points to start off with filters you know generally there are multiples of eight eight sixteen thirty-two well let's start off with the with this with a small one to eight strides as well there's those generally increase or decrease in size as we as we go further along so you want to start off with that with a bigger number and then decrease it over time so I'll start off with to activation of course like I said riilu and kernel initializer is called variance scaling what do I mean by variance scaling this basically is a weight initialization technique it's also called Xavier in some context and it tries to make the variance of the output of a layer be equal to the variance of the inputs and this this reduces overfitting so it's called barians scaling I can make an entire video on Xavier initialization by the way but we're gonna call it that that's our first layer now for a pooling layer like we did our convolution we did our riilu and now we can do our pooling well do TF doc layers dot max pooling that's our one D and now we can say well what's our pooling size gonna be we'll start off with a big pooling number and then for the next pooling remember we start off big and we'll we get smaller and generally pooling is like there in intervals of 100 so I'll do 500 I'll also answer some questions in a second the striding like I said is going to be well let's say 2 again because it's like a part of the same block you know in convolution riilu cooling and then it's gonna change over time so to say strides are 2 and now we'll just repeat this process again so we'll say model that add for the next layer the kernel size is gonna be much much smaller so let's pick something much smaller 5 what I said about those filters the filters increase right because there are less there are more filters but they're smaller as we progress through the network filters gonna be 16 strides are gonna be 1 because it's smaller riilu and variance scaling again we'll repeat that again with one more pooling layer and it's gonna be a like I said the pooling is gonna be smaller let's pick 100 stars gonna be 2 and lastly we want to add our and I'm gonna answer questions right after this lastly we're gonna add our we're gonna add we're gonna add our model T up top layers dense are fully connected later that's what I'm trying to say our fully connected layer because we're gonna now take all that data and make a single class prediction and that's what our that's what our softmax layer is gonna do is gonna output you know class predictions which is going to be in our case the next price so dense and that's going to be units 10 that's the thing about it's fun because you get to see you know what's going to work and what doesn't work activation is of course is softmax function and now we can return all of that as a resolved thanks to the promise that we've made at the beginning we're gonna return the model we're gonna return the data turn the model return the data and that's it that's for our bill TNN function I take two questions mist comma on line twelve thank you very much dat date star lang thank you boom now where were we why don't you use the adaptive theme on sublime text because I don't have time and can I run these on CPU yes you can that's the great thing about je s you can run these on the CPU alright so yeah lots of questions okay so now where were we now we're going to train this thing right that it that it that it done so step two is to train our model we built our CN n now we're going to train it so we're gonna say C n N equals function model given our data I'm given have the number of cycles or training loops we're gonna say let's split out that data so we have that input data into its own tensor each of them so we have our dates we have our testing data testing data is gonna be TF dot tensor 1d again we have our data test times and we have our out equals model yet layer we want to get that last dense layer so we can make the actual prediction right we want to make that prediction in a second thank you thank you thank you Colin's on twelve fifteen sixteen fifteen also fourteen see guys when I'm teaching and I'm coding I'm getting I'm gonna get better at this it's been a wasp in a year since of that you know live coding stream but thank you very much yeah cool : : : : : or period period or comma comma comma yep you know great alright where were we alright so model helper is gonna be our helper function for our model and now we can train this model and it's gonna be inside a book inside of a promise again you know if it doesn't work we have some it's it's kind of like a try-catch looping away okay so we're gonna try this out speaking of try-catch loops we're gonna say compiled the model and now we built the model we want to we want to run this thing using the optimizer stochastic gradient descent which is the go-to optimizer for neural networks our loss function is going to be the very probably the most popular one binary cross-entropy we're just going to compute our error value and our learning rate is going to be a very standard 0.1% most often learning rates are at 0.1% a thing about deep learning by the way guys is we are basically just jumbling up some of these parameter values and we are learning over time so that's the art of deep learning is saying I think these are good strides I think these are good filter is the thing this will work compile run the models see the results and then change those values as you see necess as necessary and you know sometimes you'll change those values and you'll break the you know you'll break the internet because you just beat everything else and that's the really exciting part about deep learning and so once we've compiled that model we're ready to train it with the fit function okay so we're gonna say let's make sure to reshape that input data so that it's in three dimensions and do that one more time dot reshape one 1960 and I will rap at the end of this so stick around by the way one batch size and then the number of epochs how many cycles do we want to run this for and now we can tell I know I know I got a like fix this up in a second just hold on I'm gonna I'm gonna fix all these like little parentheses and brackets etc etc we can tell our model like it's running you know it's running you got it out you know whatever we want to say and then print out that prediction okay and then in so we have a catch loop as well we have our catch loop is going to be resolved print whatever the exception is that's for our exception and now we can execute this so we can say prep data for the function and take the result we're gonna build our CNN using the result and then once we've built our CNN built function then we can say no no I'll answer questions in a second CNN built dot model built data 100 then function print done so it's like that now I will purchase the license at some point okay so I've probably got some serious probably got some serious errors here let's see on CD scripts where were we CD public CD scripts Python to be CNN is invalid syntax on line one oh you know what sometimes with hold on a second github Tom guys because we are running out of time in this live stream and battery and everything but yeah anyway I've got here's the github link for you guys by the way which I already have here which you guys should have been following along with anyway anyway public scripts boom here we go and it's very well commented for you guys as well so definitely check this out when it comes down to it given either the logical data or the Apple data it's going to be predicting the next in the next data in the time series if we can wait and there we go and so that's for that's for test data we're pulling from the Apple API and then we made this JSON file right here and then we trained it on that but yeah convolutional networks super useful for time series forecasting also I want you guys to I want you guys to say a subject I'm gonna rap about alright so just say some subject in the meantime I'm going to answer two questions Siraj need a crowdfunding campaign for a sublime license yes no no no don't worry about it it's kind of like a it's kind of like a meme now like I just never get sublime and then lastly uh what are the questions anyone here from Amsterdam I used to live there Cohutta it and should I know T fjs before seeing this video no I think this was simple enough as long as you've you've recognized you know Karros or tensorflow and you've seen linear models then you can you can get this right so thoughts on reinforcement learning for time series of course I like it medical image analysis of a great video in that search medical image analysis and stationarity how important is it for LST M it's not as important for LST MS as it is for a Rhema so a Rhema focuses on that whereas LST M is more focus on features across the entire so it's a holistic overview of all of that sequential data rap about egalitarian technocracy why did I pick the hardest one to mention seriously gala Terry and technocracy all right here we go here we go it's freestyle turn up the sound Oh Shi I don't have sound could you play a beat from there anything like just like pick a instrumental we're about to do this on YouTube to search like rap instrumental okay here we go with the the rap it's coming in five four three two one all right there we go oh I got it SiC trap beat you say I need a technocracy i'ma give you a democracy I'm gonna show you how to break atom monotony and show you the life that you can't see I'm trying to go in five so free outside of and see all my enemies telling me that you can do it hold up imma break it down show you that government don't mean anything it's the Internet we got a new techno utopia it's like man I'm in it I'm in it I'm out of it I'm back in it you don't even see cuz I'm beyond it that's it for this rap I just wanted to do a little five-second demo to keep it interesting guys so that's my wrap for today's video I hope you liked this video share the best thing you could do for me is to hit the subscribe button or and tell your friends to subscribe that is a single metric I care about subscribe I'm trying this audience is growing but it's not growing nearly fast enough we're trying to get to the top you know I'm saying million subscribers by the end of the year that is the goal okay it's a very unrealistic goal but I'm an unrealistic person okay so thank you guys for watching I love you guys you're the reason I do this and for now I've got to go take a plane to Houston to visit my parents so thanks for watching