Quant Trading Accelerator - Part 5: Statistical Edge

Hello everyone. So welcome back. This is part five which will be discussing the statistical edge. So the key goals here is that we're going to learn some key matrix algebra operations which are fundamental for for building our model. We're not going to learn the whole subject of linear algebra. I do recommend to to take like a linear algebra course. Maybe I'll I'll put like a there's a really fantastic video series uh that I can send so you can if you want to to learn more about linear algebra. Uh the but the key goal really is to how to create and measure statistical edge. And also another really important goal is to show how you can make money uh with just a very tiny statistical edge. And the key lesson there is like the win rate does not equal the the alpha. And then yeah, we're going to actually build our first alpha model as well. So the whole series so far has gone from like learning one-dimensional uh arrays, vectors, uh operations that are on based on arrays. Uh then we've gone into time series and now we're going to go into matrices. Uh with matrices we can model multivariate. So that's multiple time series at once because we uh with our models we it just won't be just like one time series. We maybe want to model like several time series. So we need to be very comfortable working with matrices and once we're comfortable with that then we can actually do like forecasting predictions [snorts] and just uh the key goals of of this series is to uh [snorts] develop knowledge or domain knowledge for building models and with the models that's how we create our statistical edge and then the strategy then we'll go into the strategy which is how to execute the statistical edge. Uh so that's the we're going to go into these we're already now tackling models but then we're going to go into some more model topics and then go into strategy. So the first thing we need to do is to import our libraries. Uh so we've got here again we've got numpy which is for numerical like numerical computing. It's just very fast vectorized calculations. Uh we've got our data analysis library pandas. We're going to this is our data visualization library to to plot uh some data and then our machine learning library and so we're using PyTorch and I've had some feedback as well saying why why use PyTorch it seems way too uh like you can just use like scikit and that's a valid point uh but the with the machine like for example we can just build like a linear regression in torch and it gives us so much control over the whole process of training. So like for example, if we we researching to find out what's the best way to to build a profitable model, we want very fine grained control over the whole process. And uh PyTorch gives you that uh flexibility, that low-level API, but it's still very like Python code and you can like still just write like a you see like a machine learning library in like just 10 lines of code. [snorts] Anyway, so we need to now discuss um matrix algebra. So I'm going to go over this quickly. So this is only just a purely a subset of linear algebra. uh I'm just teaching you just the bare minimum just to so you can understand the key operations of machine learning and so [snorts] we need to just go over like what is a matrix and a matrix essentially in terms of code is just a two-dimensional array so for example uh this is like a matrix here and I'm just going to five six seven, eight and nine. Okay, so this is a two it's essentially just a two-dimensional array when we think about it in terms of code [snorts] and for example if we want to access like the first element. So this would give us the like and it gives us gives us an array. So to get like an individual element which we call like a scaler then you would do have to also then index it again. So then we can get like 01 if for example we want to get uh you just the other elements then we can do it like this. So it's a bit tedious but then just to make it crystal clear. So that's two. And if I want to do it for uh the next one and so forth. And I could even do it for like for example uh we can then go into the next row. Okay. So that's a that's a matrix. Okay. And there are some very important operations that we need need to be able to do. And one of them is to do um what's called matrix scala algebra. So I'll just write this down. Matrix scala algebra. And we've already discussed like scalers um in previous videos. So for example essentially what we want to do is like for example in this example we want to do matrix whatever. So for example here this matrix we want to add one to it. uh if we were going to do it um manually we would have to do something like this. So we just say like okay the number of rows is the length of the matrix. It's nice that it's auto completed [snorts] and we want so so we want to work out what our M is. And so we'll just do this. So in mathematical notation you you'll see like a matrix being defined as like n or n. Uh so for example here like number of rows. So when you see it like in mathematical notation then this is essentially what it represents. And like to to do this we can loop over range of M and then for J in range of [snorts] N. Okay. And then matrix I J. So we this again it's two dimensional so we need to to get the to the individual element and then we're just going to add one to it. Okay. And then we're just going to print out the the matrix. So you can see here like it started from one and now it's done plus. But this is like extremely tedious if you're having to do these like operations. So um and I just show you like essentially what essentially it's done just to make it super crystal clear. So that's essentially the the operation that that it's done. So it's just added one to each individual element in the matrix. Um but you you don't a there's two reasons why you won't wouldn't do this because uh it's not it would be extremely tedious and uh unproductive to write these manual loops and also in terms of performance this would be terrible. Uh so for example, numpy does has like specialized uh code to do matrix operations extremely fast and this is again factorization. [snorts] So for example um we can do this we can do for example we can rep so exactly the same code I'm just I'm now just going to represent it as a a numpy array. Okay. So if we now print out a so you see this is like an array and it kindly displays it in a more uh matrix notation. >> [snorts] >> So that's the the matrix and what's really nice is that instead of having to manually do the loops we there's a really nice abstraction that we can just use like mathematical notation which is just like okay so you can just do treat this and this it gives us exactly the same see here and it's way much faster as well. So imagine this is like a huge matrix uh the performance difference is night and day. So and then also as well there's like this uh just a very important part is like this associ associivity [snorts] okay and we can also do like multiply multiply as well so you can see here but that's matrix scaler so that's uh an important operation and uh the most important operation is the matrix vector algebra. So we need to be able to multiply matrix by a factor. And this is very fundamental because uh essentially we're going to we're going to be doing this which is like uh x multiplied by our weights. And so x is our matrix and we need to multiply it by a vector which is our weights. And we want to make a prediction. and the prediction is the y hat. So we'll have a a matrix even if it's univariatic we still need to to model it as a matrix. So this is like very fundamental operation. So it's it's important that we understand that the mathematics and the algorithm behind it. So again we're going to model we're going to use numpy. So for example here I've got we just maybe just print it out. So, we've got a matrix here of let's say log returns and we've got some weights and let's just say we've used machine learning to to work out the most optimal weights. Uh so, and then we're just going to define that. I'm just going to show this. Okay. And essentially we what we want to do is what's called like a dot product. Okay. So we take the x and this is essentially matrix multiplication with a vector. Okay. So then and this gives us the the predictions for each uh row. Uh so it's calculate it's it's multiplying each of the these these two elements uh with this and summing them up together. [snorts] So essentially what it's doing is it's doing this. Maybe if I just copy and paste it'll be quicker and easier. And you can see this is exactly the same. So I'm just putting the the individual weights weights one and weight two. Uh so imagine these is like lag one this is lag two. Okay. So the with lag one this is the weight for it and for lag two this is the the respective weight for it. So it's just multiplying this say make it more concrete. This is say log return. We're multiplying it by this weight and plus the the value uh the the lag two value and multiplying it by weight two. Uh so that would be our first prediction and this is our second prediction and then our third prediction. So this is like very fundamental for for machine learning. And then also we add like a remember that we add a we add a bias and so I'm just going to put this here. Uh but in a mathematical notation you this would be like a B. And so essentially like our model is going to be the mathematics of our model is this [snorts] is we're going to do a dotproduct between X and our weights which is what what we're going to work out is like what's the optimal weights and then plus the the bias. So essentially we're just adding you see what what the bias just does. It just uh adds this tiny let's say epsilon number to to it. And so for example this could be just a very tiny negative number but the bias could skew it to be turn into positive. So [snorts] uh and also as well if let's say if it the the way like if it's this dot product is zero then essentially it's just the the bias and so but there's a very an important uh operation that uh distinguishment which is broadcasting does not equal I can make a broadcasting does not equal matrix multiplication. So a really common gotcha I would say is this. So for example you may be tempted when you're to do this and this is not uh matrix multiplication even even though this is the multiply sign but this is something called uh broadcasting and this this is like totally different. So just got to be careful when you're for example the API that you're using to represent your matrix that you're actually doing matrix multiplication and not broadcasting. So for example in numpy uh this this is how it's done. [snorts] So now we just know understand some very basic fundamental operations of matrices and now we can sort of go into dive deep into machine learning and so the now the next part is which is I'm just going to this comes up is to we're now going to work out how to uh how to develop a statistical edge. Let's call edge using machine. I can't see. Okay. And the whole the Yeah, maybe if I put another text in here. So the the idea is that we're going to predict a future log return. And so essentially the statistical edge is the oh let me just maybe I'll just copy and paste this because it's my is the following right is this that the statistical edge equals like a good forecast. So, and essentially it's like where the the expected value is positive, but we'll go into that in more detail. So, the first thing we need to do do is to download some data. So, what we're going to do now is download some what's called open, high, low, close, prices. Uh this is just a very very basic uh data set, but I just want just to show just like with a basic data set, you can still make good uh predictions. So, we're going to download the per data. And so, we're going to use Bitcoin dollar per contract. It's like a f it's essentially like what's it's like a futures contra contract that doesn't roll. And the futures are great for be for speculating because it gives you a very quick ability to to bet if it's the price is going up or if bet are going down and you can also leverage. But uh there's a bit out of scope to go into the details, the mechanics of a per contract. But um so now we're just going to download the data and load it into a Python a pandas data frame. So and this is an hourly time series. So you can see here like each row represents an an hour and you've got the open price that's the price it starts at and then the highest price uh it reaches in this time period uh and then the lowest price it reaches and then the price that it closed at. So at just right the end of of this time period just before uh the first hour here then it closes at this price and this is the trading volume for that respective time period. So that's the the what's called OLC data. It's just a very basic data. most like firms will be using uh high definition high frequency uh order book data and trade feed data but um we're just going to just want to show you that even just with a very basic uh data set that we can still make develop a statistical edge. So the first thing we need to do which we've already discussed in previous videos is to to create the the log return. Okay. So this is I'm not going to go over it uh again because I've done it in previous videos but essentially we want to measure the log returns because they have this property called time additivity and they can also model compound growth rates. Uh but yeah, this is just uh I just recommend to if you're interested just look at an econometrics book and it will explain the uh in more detail. [snorts] So that's uh now we've got the log returns and now so this essentially what we want to predict is we want to predict this closed log return and we're going to predict using uh the previous uh uh log returns. So this is what's called like an auto uh in econometrics is what's called like an auto reggressive model. So we what we're essentially looking at is say like the so what we're doing here is we're just shifting uh the data to say okay lag one. So for example what what ideally what we'd want to do is let's say uh make using this lag say like lag one we want to make a prediction to predict this log return. So this is the the price movement the actual price movement and that was the previous price movement. So we want to predict the the price movement this one from this previous uh this previous log return and that's essentially that this came from uh the previous one. So for example here the price return was this okay this number and you can see this now for the next time period that it's now the lag one and then for the previous one in the next future time step it's now lag two. So that's essentially what we want to do. So essentially what we're predicting is like if we buy at this time and then close it just before the the next time step, what's our return? So that that that's essentially what we're modeling. Okay? [snorts] And we're just going to look only just three lags to to the behind. You could actually like do even more. And so another so if we just look at the correlation so we in the previous video we discussed correlation. Okay so if we now look at the correlation here so uh we've got these like lags and you can see what's quite interesting is like the it's not uh decaying. So for example uh the the strongest correlation here is actually lag two and not lag will be see this is negative. So this means that when lag one goes up then clos typically goes down and vice versa. Okay. So, um, and now we just I want to do is just to inspect the like visualize the the data because I just want to show you essentially what what we want to do. And to do this, we're going to do it's just going to inspect the data. Okay, this might just take a while. And so this plot is going to show us the distribution for each uh variable here. So we're going to see we've got close log return log return and then the lag one and lag two of it and lag three. And you can see here that it's plotting the the relationship visually the relationship between uh say log return and its previous lags. And you can see here that there is really no what we call like there's no polomial that we can model to represent this. There's no like clear like okay this is a linear relationship. Okay, this is a quadratic relationship or this is like there's no polomial that we can that's is so obvious that we know okay how to model this. So this is the and this is the the problem with financial data is extremely noisy. Uh so this is why for example linear regressions are quite good because it just makes the decision space you know to buy or to go long or to go short it's a linear curve so it's a good approx like approximation with the the presence of noise. So that's the so and and also here as well for example this is you can see here that the this there's only one peak here [snorts] and unfortunately you can't see like other properties of the distribution here like for example the we in the previous video we looked at like the the central tendency and then the dispersion the standard deviation but then there's there's other what's called moments which is like for example kurtosis like How fat are the tails essentially of like and that essentially models like extreme events and then there's also skewess which is okay uh is the distribution skewing to one side so is it like more trend like um skewing more to like say positive or negative anyway so that's the you can see like a visualiz visualization and see visually why it's so hard to predict uh these log returns So now we've discussed that now for example I just want to show like okay we're going to model uh like a matrix okay so so from this is our values and this will essentially turn into a matrix so using the previous returns [snorts] and now we can see this is now in a a matrix notation so this is now ready to be passed into a machine learning model. And another important uh characteristic is to to look at the shape. And the shape tells us the the dimensions of the the matrix. And you can even get it into like three-dimensional uh matrices which are essentially actually n dimensional uh matrices which are called tensors. But um we're not going to cover those. And then for example now what we want to do is to create our vector of like what we want to predict. So X is the model the input and then essentially what we want to do is predict this and then we model this as a as a vector. And if we just look at the the vector here now you can see this is just again it's just a one-dimensional array. And if we look at the shape of it, you can see okay and for this to be able to multiply, you have to ensure that uh this has the the same these two numbers match. But anyway, so now we're going to go into So what we're going to do now is to do forecasting. So we're now going to predict uh the future log return. And we're going to do this and we're going to do it via regression. So regression is where we predict a real valued number. So like it could be a floatingoint number, it could be an integer. Uh and in the pre in the next video we're going to do is we're going to do forecasting but using a classification which is we're going to classify it being up or down. And uh with classification it also gives you like an associated probability but anyway uh we'll discuss that in detail in the next video. So the what we're going to do now the what's really important is to split I'm just going to put it down here to split training and testing data. And essentially we want in sample data and out of sample data. And it's just very important that we do this in in the correct way. So, and I just want to just visually just show you roughly what it looks like. So, for example, with a time series, it like for example, it's really important that you preserve what's called the temporal order, which is just the the natural order of the the time series. So what we want to do is to to learn the pattern on let's say that we've got only got um eight data points. We want to to teach our model using like the the first five uh data values and then we want to say okay is there persistence in this pattern and uh we would then use for testing is the out. So this is called in sample and this is out of sample. We want to test uh you know does this pattern persist uh for for future values in the time series. And it's really important because for example if I just taught or teached uh a model just on all the this time data I don't know if this will work into the future. So uh I don't know if this pattern is persistent or not and especially because the the data is so noisy. Uh and uh this is another really important thing as well is that we don't want to for example uh there's like um there's a library called scikit and there's this thing called train train test split and be very careful using that because uh I think by default it's to to shuffle the data and you do not want to shuffle time series data because you'll get amazing results and you think you're a genius and you think okay you go off to the races you start to trade it and then realize that It's you're like hemorrhaging money and that's because of data leakage because for example if I mix the the data so for example if I just like randomly picked like t0 t5 67 and I just tested on t like uh it would it would see data in the future that it would not know at that time. So it's always very important that if we're passing any data that we don't have what's called data leakage. it doesn't see data they wouldn't see in the future because you're going to get these like impressive results but it's purely done because of like a data leakage anyway so that's just really an important step uh and then so for example we can just it's very easy to to split by time because that's what we want to do for example we want to split it at T4 here so at T4 we we put this into train the the and then the most recent for testing. And so for example, we can just do it like this. So essentially what this code is doing here is we're passing our matrix uh oh actually that should be X. And oops. And then we're deciding okay what what's what ratio should our train size be and then returns back this like tupil which is the first part is a train and then the the next part is the test. So this is what we do. So we pass in the our data frame and we say okay our train size is 0.7. And by the way this is what's called like a default value. So for example, I don't have to add uh this explicit uh value. It would just by default, for example, if I just passed in this, then by default, it's just going to to split it like the test size would be 25%. So So [snorts] now I can just run this. And now I just want to show you like the the the underlying data now that I've split it. So you can see here it starts the beginning is uh 2020 1127 and then the end of the the training data is the 5th on the 24th. So, and then the most recent data that so that's the data that we're going to be using for for learning a pattern and then once it's learned a pattern then we want to to to test it and we've got a quite an extensive range of uh data here. So, so you can see here that's just 1 hour next is the is for our test and then it goes all the way down to the the 10th of the 11th. And so then that's what we going to this is our outer sample. And so now we we've we've split it. Now we're actually go into the the meat of of developing an edge which is uh to develop the train the model and I'm just going to put so train model. So just to make it and a really important uh property that we need is the re reproducibility which is if I run this code and you run this code it should produce exactly the same results. It shouldn't just be like you run it and then hey, I get completely different results. And so what I'm going to probably do is uh just copy and paste and then go over the the code. And this is the our machine learning our training. So I'm just going to go over what does each one do. So uh we actually don't need to import the libraries, but anyway. Uh so we've got the reproducibility settings. So we're going to set what we need to set what's called a seed. And this just is to ensure that uh all the non-deterministic algorithms is sort of uh it's reproducible. So we're just setting several uh um seeds here. It's kind of overkill, but I just just in case like um you change anything that's uh still reproducible. So we're setting this seeds here. Um if for example if we you want going into GPUs or multiple GPUs then uh there's this CUDA there's another like for example in CUDA NN there's this like deterministic flag that we can set but uh we don't need this and so so once we've got the re reproducibility settings then we actually want to start to create tensors so um with PyTorch which is the machine learning library we work with it it works with tensor So even though we're still working with vectors and matrices, we need to it needs to be treated as a tensor object. And so essentially what we need to do here is define what our input feature is. Uh and so we're just going to create a very basic model just one input feature and it's going to predict the the log return. So this is the the predictors and this is what we want to to predict and we want to model this relationship as a linear regression. Um and then we need to split the the data into PyTorch tenses. So this is the code uh that does it. Uh there's just one thing just important detail that I need to mention here is this unsqueeze. uh we need because uh this is just we we need to add this like column dimension otherwise it'll complain and say this is uh the incorrect shape. So it's important when you're doing like matrix al matrix algebra operations is that the the dimensions are the correct size. So this is the just to make sure that it's um it adds this column dimension to it. And so now um and also as well I just recommend is just to print this like print this out like before the squeezing and then with this the unsqueezing so you can really have a good understanding what it's actually doing behind the hood. And so now we need to define the model. So this is the number of input features one in our case. Uh so this is what we call like a a univariate model. And we're just going to do a simple linear regression model. Uh especially if you're starting uh you need to walk before you can run. So before you want to for example ex like for example if an extremely competitive scenario let's say market making it it would be advantageous to look at more complex models because you know just squeezing a tiny like amount of EV is gamechanging whereas but for example with our models we just want to start with just something very basic and it can still be very profitable and there's actually quite a lot because we're using PyTorch the way how we calculate the weights. There's a lot of alpha opportunity anyway. So now we're going to go this is the the model and uh so this is this um it's already baked into PyTorch. So we don't need to to manually like create a model here. We can just represent it this linear module here. Uh and now we're going to use something called uh hubos and hubos. Um so this is essentially measuring the loss between uh the error between our prediction and the actual value. And there's multiple ways to to measure loss. Uh there's like some very common ways which is like mean squared error and that's like squared difference which is very sensitive to to outliers. Uh and then another very common one is what's called mean absolute error which is where it's not sensitive to out outliers. uh which is you're not you're not squaring you're just taking out the absolute difference and um so and there's there's a balance between this is something called the hubos I recommend to to to Google it and look at the how it's defined and uh and again if we're just using scikit we not be able to control like how we would have this great flexibility so we're going to use something called the hubalos I really recommend it. It's uh it's a nice balance between mean squared error and mean absolute error. Uh and we're going to use like a very basic uh gradient descent optimizer. So this is essentially is how we optimize our model. Uh I don't really want to go like because I've already I'm already spending quite a lot of time to go over the matrix, but maybe in a future video I'll just go over the uh the mathematics of the the sarcastic gradient descent. Uh but you could if you just take a machine learning course, it will cover this and probably do a better job than I I could. Uh and then you got this thing called the learning rate which is like um this is what's called a hyperparameter. Uh but we're just going to leave this the learning rate as this value at the moment. But if we're really like fine-tuning things, then we we can this is one parameter that would be interested in optimizing [snorts] anyway. So now we can go into the the meat and bone of the the the training which is the the training loop. So essentially I'm it's just I'm looping over and there's this thing called an epoch. So after each time I go over each the whole up we're doing what's called a full batch gradient descent. So we're taking advantage that all our data is in memory. But for example, if we're trying to build like a large language model, we do not be able to fit all the data into memory. So we would need to do like a uh like a half like a small batch essentially like for example only 64 batches at a time or 32 uh but because we've got all the the we can load all the data into memory at once we're going to take advantage of that by using what's called a full batch gradient descent and essentially what we need to do uh because the the gradient it adds accumulate like it's additive these gradients. So what we need to do is always just to reset the the gradients so it doesn't accumulate them by default. So for example, if the gradient is one and it it adds let's say another one to it, then uh it's not now two. So it's just very important that we just zero out the the gradient and the gradient is essentially just how the process of learning essentially. Um [snorts] anyway, so this is the most important part which is our prediction. So we pass the training all of the training data into the model and it makes the prediction for every data point every row essentially in in the training set and it's represented here as a factor. Uh and this vector represents all the for every row essentially in the training data the the corresponding prediction. And now the most uh now we're going to use the criterion which is our huba loss to measure to quantify the loss or the error between uh the actual values which is wide train. So this is the actual values in the training set against the our predictions and to quantify that loss or the error and then we we need to do something called back propagation which is uh we need to compute the gradients of the loss with regard to the parameters. Uh so once we've done that uh then we can update the model parameters using the computer gradients. And once we've done that, then we that's the whole process done. And what we do is like for every every 500 epochs, we're going to print out the the the loss. And then at right at the end, we're going to look at our model and see what the the the final parameter weight is and what the final bias is. So, if I run this now, this might take a while because I probably been talking quite a lot and hopefully it hasn't the the kernel hasn't restarted. Okay, here we go. Okay, so you can see here that the the loss is actually quite high and you'll see as it starts training that this starts becoming a smaller and smaller number. And in practice typically you have like you want to okay maybe I should just discuss or not okay there's there's a okay there's a point in time where we want to stop because then it starts to overfit especially when we have models that are more complex like a neural network there'll be like a cutff point with where okay it's it's overfitting so it's it's not generalizing so there's a certain cutff point and it's both like to work out this it's both an art and a science So anyway, so that's the the key machine learning algorithm. So it's what it's only 1 2 3 4 5 6 seven like eight lines of code. So it's it's not extremely complicated. So I just want to just also then just do this which is just just to print the the code out. It's just to show like the the weight and the bias. And you see here that uh it has this requires grad. So this is it's represented as the as a tensor. And and now that we've ah we also another important thing to do is that for example if we want to to reuse this into the future and maybe like deploy this into production we need to to store it onto disk. And so that to store it onto disk we can just call this torch.save. And uh this essentially the the state of the model is saved as a dictionary. Uh uh and a dictionary is just like a hashmap uh uh or just yeah anyway it doesn't matter how it's stored. It's just that we we store it. And now once because for example like deploying it into production we can then just like once we load up the training system we can just load up the the model um and then execute it. So now what we want to do is to evaluate the model. So like we don't know yet if this is a good model like the loss hasn't measuring the loss doesn't tell us like is this a profitable model or not. So this is what we want to do is and this is we want to evaluate the model with respect to profitability out of sample and remember that we're look we're going to use now use the testing uh the most recent data to see okay does this pattern persist or not and what we need to do is to add back our uh the data. So we're going to ah we're going to do this. We're going to predict make a prediction using the model and with its new uh with its parameters that it learned the weight and the bias. Oh, I should maybe just point out that it's uh negative. And the beauty of linear regression is extremely interp inter interpretable. So I know I can look at this and I can say okay what like why has it decided to trade? And I can see straight away that because it's a negative number, then this is a mean reversion behavior. So for example, if lag one is positive, then I know that the the future log return is going to be negative and vice versa. So I know there's because it has a negative weight and that it's a mean this is modeling some mean reversion dynamics. So now what we're going to do is look at the outer sample data. Okay. And we're going to have a look and see that it's a a tensor. Uh and don't worry about there's this gradient that's um we will remove that. Uh so the next goal thing is to uh detach it and yhat detach. And I think it probably explain it. Oh, it doesn't explain it. So when you detach it, you're going to remove this gradient. uh from it. So now you can see that the gradient function uh is removed now. So but we now want to because we're going to add it back into our data analysis frame library, we want to convert it back into a numpy array. Uh and I just want to quickly just show you though um the the shape of it. So we need to do the reverse of the unsqueeze. And the reverse is to squeeze it. So what we want to do now is to oops is to squeeze and we can see now you can see here this is like a row vector and now this is uh a column what's called like a column vector. Uh don't worry too much if that's doesn't make too much sense just uh the the most important thing is that you see that the shape is different now. So, for example, hang on, my mouse is just not moving. Uh, so I just wanted to show the the the shape because that'll make it more clear and understandable. So, now we've gone like the the the shape. What's this? Uh, it's now gone from two dimensional to one-dimensional. Uh, so now we we've squeezed it. [snorts] And now that what we can do now is to add it into our testing data frame. So so now we can see here now we've added it to our test data frame data set. Okay. So we've made predict so we haven't this is completely out of sample. Our our model has not seen any data the most recent data. So uh it hasn't seen this. So we can say that okay is you know then this is what gives us like okay does this pattern have persistence or not. Uh [snorts] so and you can see here for example maybe I just explain so that's the actual price movement as a log return. So it moved down here and it predicted it goes up by this amount. Okay. And so for example here uh for the next hour it predicted so for example we would lose money here because we predicted the the wrong direction here [snorts] and for example here we predicted like it went up but and our model predicted it went up. So we would actually make money here on this. So we need to create a column to know if we won or not if if our prediction is correct. And this is what's um we need to say like okay what what what direction did we bet? Did we bet going up or did we bet going down? Uh so for example we're now going to add what's called the directional signal. And another key important thing I just because I'm talking in trading lingo and [snorts] I'm just aware that some people may not have any trading background that the for example if we're going to we're going to encode that one equals to go long and to go long essentially means betting price going up. So if you hear anyone says I've got a long position, that just means that they're betting that the price is going up. And then on the other side, if we're going to encode, it's either going to be minus one or one. And that means to go short. And short is essentially you're betting the price going down. And with these like um future contracts, per contracts, uh they allow you very easily to to to speculate both up or down with actually o out without owning the underlying asset. Uh but that's a bit out of scope, but but what we're we're modeling the the what's called the per contract, which is uh I think is much better than futures because they don't have this rolling property. Uh but anyway the key important thing is that we're go one is to go long minus one is to go short and so now what we need to do is to encode this directional signal. So to do this we're going to maybe actually I'm just going to copy paste it be easier. Okay. So now that's our directional signal and I'm now I'm just going to print out and just to make it super crystal clear to see what it's actually doing. So and we essentially what we're doing is is we're taking the the sign the mathematical sign of our prediction and uh that's that tells us the signal that we're going in. So for example here uh we're predicting it's going up. So the directional signal is one and it actually went down. So we actually lost there. For example, here our prediction like if we encode it uh as just the sign is 1.0 and we and it actually does go up. So uh that here we actually did win and vice versa here. For example here this is the the the prediction is to go short but actually the price went up. So we would lose money here. So we need now we need to to encode did did uh each individual trade win or not. And to do that we're just going to and this is what's called the directional accuracy. Now we're just going to do this is to evaluate the directional accuracy. Okay. Let me just take a Okay. and or is very important because you may see some papers especially like econometric papers that might you just mention it as DA and that's essentially if you see any of like my code or DA then that's you know what we're talking about is the directional accuracy and to to model to to quantify the directional accuracy and essentially the directional accuracy is just the the the win rate and so for example we this is how we do it so we can say is equals uh the directional signal equals the sign of the what we're trying to predict. So the closed log return. And now what we can do is just to go over. Okay. So we've got like our actual values that this is what the target that we're trying to predict the future log return. This is our predicted value of this and we want to know like the the the encode the directional signal and then we want to know has it won or not. So for example here we haven't won because it predicted to go long and it went short and but for example here we predicted uh to go long and it does go long. So that's and then just see if there's a good example where it's negative and it wins and it's negative. Okay. Yeah. Here's a good one. So for example, here it's predicted the price is going down going short and it actually does go down. So it wins there and you're going to see something very interesting now. So I'm now going to All right. Hang on. Why is this not Okay. So, this is a really interesting and this is you'll be shocked by this. Uh, and you may even go, "Okay, this is rubbish. I'm going to close the video and dislike it." So, uh, just bear with me because if I show you the the directional the win rate or the direction accuracy, you can see here it's only 50.7%. So, not even 51% it wins under the time. It's just a very very tiny edge. Well, a very tiny amount that it wins. It's like dangerously low. Okay. So, but if we now, so for example, we just looked at the win rate and we just like, okay, I'll throw this away. It's rubbish. Okay. But, uh, bear with me here because you'll see that, uh, the win rate is not the key performance metric. Um, you want to look at the expected value and then the scaling like how many times. So you can have like a very tiny edge but if you trade lots and lots of times uh you you can make huge returns anyway. So now that we've got the direction accuracy and and I'm just going to hopefully you haven't closed down the the video and so now I want to do is to add the the trade returns. So at the moment you're going to say okay something's a bit fishy here. So now what we need to do is to add the the trade return and the trade return is essentially the what's the actual return that we make and it's essentially we take the we multiply the directional signal multiply it by the target. So and it it probably be easier if I just show you the like what the calculation it's actually doing. So, for example, uh we're predicting it it's going up and it's actually gone down. So, uh we've lost money here. Uh maybe put is one as well just to make it crystal clear. Okay. So, essentially if we've won then this is going to be positive. So this means we've we've made money and we're not and the very key important thing is that we're working in returns because then we can work out okay uh it's not like a concrete value like $10 or we can see okay how much money would we make if we invested like $100, $10, a million dollars. Uh so it's just allows us to know to model the scalability of it. So and again here so for example it's false then it's it's negative right uh so that's essentially what it's doing because like for example uh if we predict if the signal let's see where okay so for example here the directional signal is minus one and it actually has gone down if you and basic maths if you multiply two uh negative numbers you get positive so uh this is why the the trade returns positive here okay so Now we we've modeled the actual like on each individual uh this is what's important is this called round trip. So at the beginning of the time period we open a position and just before we close the position be just uh before the next time series uh data point. So we just basically opening a position at the beginning of every hour and then closing until before the next. So just literally just before and you would reopen with a new position. Um so essentially that that's what we're doing. Uh so that's the the trade return and now we need to evaluate the statistical edge because this is the most uh important thing because it's very I don't know if it's just human intuition that we just always look at the win rate because uh I don't know it's just like for example there's like this interesting property that if you ask to guess a number between 1 and 10 most people choose like three and seven and they're prime numbers And this is just natural. So maybe it's just a natural reason that we look at the the win rate. [snorts] Okay. So now we're going to evaluate statistical edge and we this is to do this we want to calculate the the expected value. The expected value is essentially just the the mean. And you can see here that it's positive. And literally just the the huristic to know if we've got a a statistical edge is this. It's like okay, is the mean greater than zero? And then that means we have a statistical edge. But there are some good statistical edges and there's some bad statistical edges. Uh maybe that's for another video. But then we can just now if we just print out uh has statistical edge you can see it's true that it does have stat statistical edge. [snorts] So now what we want to do is to evaluate the total return. So for example if we invested x amount of money what's the total return that we we would make? Uh so that's the next thing we're going to do. Evaluate total return. And to evaluate the total return we can just because these are log returns and they have this property of that you can add them up together then we can model the the total rate. So here the the sum here is 1.03 and that's if we add up all these log returns. Remember, it's only just winning just 50.7% of its trades. And if we, for example, here and now what we need to do because they're they're in log space, we need to convert them back into into normal returns or simple what's called simple returns. And you can see here that it's a huge number. And so for example, if I if my initial capital, let's just make this code here. Let's just I start with very humble beginnings, right? And I just started with $100. And just to emphasize that uh because I'm I'm assuming majority of students, they're not going to and also there's actually a capacity because you can't just trade like billions on each trade because your you your edge will be lost essentially. So there's like a capacity. Anyway, that's a bit too much details, but um now what we want to do is to do this, which is to essentially just multiply this by one our initial capital. And this tells us like okay, why is it not? Ah, there we go. Okay. So, you can see here that if we invested 100, we'll end up with $281. So, it's not more than 2x. And that's there is a caveat to this that um we need to include fees. But, for example, uh anyway, we'll discuss this later, but just see that's a huge amount, right? And it's very counterintuitive because we're only just making like 50.7% uh win rate. And so now what we want a very important uh metric or just like say visualization or data analysis is to evaluate the equity curve which I'll put here. Now we need to evaluate the equity curve and the equity curve is a quite a good visualization and uh we can model to see like for example these properties called like draw down um but let's just go over okay so I I just explain what's going on here right so with log returns it can model like if you reinvest your money uh how much you make and you get this like compound growth. So this is why the beauty of log returns because we can model this compound growth and so we do something accumulate sum. So essentially for example for here it just adds the h maybe if I uh show is this log yeah it is log returns that it adds the previous trade log actually I shouldn't it's a bit misleading but this is uh not the trade log returns uh this is the cumulative tra trade log returns so essentially it's just summing up the previous trade log returns at each time step um and then You can at each time period we can model like the compound growth if we if we reinvest the the money and now what do is to just to display this equity curve. So if I just go into here. Okay. So this is what our equity curve looks like. it goes into. So, you can see it's it's going up like uh even though it's only like winning 50 point 50.7% of the time. So, not just under 51% of the time, we've got this like sort of linear growth. Uh so, and the key ah maybe did I go into this? Did I show the uh I haven't actually gone into Yeah. Okay. I just wanted to go over like the the calculation as to why uh so I've gone over the statistical edge but we've calculated the mean but I just wanted just to to show um like if we the reason why is because it has a very tiny edge. Actually maybe I'll do that in the the future video but essentially our wins are just just a tiny amount bigger than our losses. So just a very tiny tiny statistical edge uh we that we have here and then this is why we have this but the there's a one uh there's one caveat to this which is we haven't added transaction fees and this is where we essentially uh was called piss on my bonfire because this would be amazing right uh this so in in practice this c this model can work for example if we have rebates but I'm going to go talk over that later. I just what I want to do though is so that's this is the equity curve in log space. So this is but I just want to show con like in concrete we usually rather like let's say for example if I invested uh $100 okay what's the actual equity curve so this make it more concrete right so we start 100 and you can see it like shoots up to 150 at a certain time point and I think this is where there's a huge crash and so um this model does well when there's like extreme price movements uh But you see it's uh like shoots up right. So this is but this is gross. We haven't factored in any uh transaction fees. And another important uh metric that we need to evaluate which is the the sharp ratio. And the sharp ratio is just like a model of like um how much risk is there in in in this uh and just put this down evaluate sharp ratio. And this is essentially modeling risk adjusted returns like what returns do we make per unit of risk [snorts] and then the mass behind it maybe if I just copy and paste this bit easier it's essentially just this is essentially informally what we're modeling is the returns over the risk and we're going to do this we can calculate it as just the the mean over the standard ard deviation. We know what the standard deviation is because that's like a common measure of dispersion from the central tendency. So you see here but the problem here is that it's an hour this is like an hourly uh sharp ratio. it's not uh we need to annualize it into annual sharp ratio and to do that we need to add multiply it by like an what's called an annualization factor [snorts] and annualize it because for example if I have a model that's say in I'm trading in minutes it's going to have completely different sharp characteristics but we want to measure that like independent of the what we call like t time horizon so for example we can measure it consistently between arbitrary time intervals and and to do this we need to do this annualization and I'm just going to annualization and I've had a few people mention me say hey um about you know why do you do this I thought it would be a good point time to explain it so so for example this is the secular the same but this is the sharp ratio for like on an hourly level or we want it on an annual level because then we can just consistently measure it between like for different intervals right so essentially we just need to do is to multiply uh so for example we just need to work out how how like what are t so we do is we multiply by the square root of t I think mathematically they is n capital n but that's more of a pedant observation So the T here is essentially because we're we're we're trade like for example we're trading PES and per you can trade 24/7 uh every day. So essentially there's 365 days in a year and there's because this is an hourly trading model that we're we're making a forecast every one hour and we assume that we're going to trade every 1 hour then it's 24. RT is just the number of trading days in a year multiply by the number of hours in a day and that's RT and then we just take the the hourly sharp ratio and then multiply it by our annualization factor and that's how we can analyze the the sharp ratio. So yes, we are multiplying scaling the sharp ratio but we're scaling it proportionally to our time horizon. So we can have so it's just so that we can compare this against let's say we go even more low like uh predicting on a minute level that we can consistently compare our training model that's in hours against minutes and any arbitrary time scale like for example days. So that's the the reason why we do it. It's not like a academic exercise. [snorts] Uh and yeah and just the just to to piss on the bonfire is get really excited to think okay I can start trade this and you know uh we're not we haven't factored in the trans the transaction fees and what's called like net returns. So, uh, and that's where you include the transaction fees. And it's quite common, uh, in some, well, when I was looking at like academic papers at the time was that some papers don't even model the, uh, the transaction fees. And so, and it's really important that we do model the transaction fees. And so, what we're going to do, maybe if I just copy and paste this Okay. So, and it really depends on what type of like so for example, we could actually make we could actually use this model. So, for example, uh if we're like running this as a making strategy, which is essentially we're adding liquidity, not taking away liquidity, the the fees are usually lower. And let's say we have a really good relationship with let's say we're just trading lots of volume and they actually like let's say we apply for like a market maker and some exchanges will even like say okay we will pay you to to market make. So you actually make money uh so this um makes our expected value even more increases it actually. So uh this is feasible but it depends if it's a maker or a taker strategy. Even for example in some exchanges you may not be able to run this even as a maker strategy. For example, Binance the per exchange the they strangely say like okay uh if you uh have a limit order you pay the same fee as a taker fee. Uh so there there is no you you would not make any money there and then yeah just maybe show you what a take is. So a take is that you're taking away liquidity and there will be typically there would not be as a rule of thumb there would not be like because we're trading this on such a short time frame that the the fees will decay the edge to a point where it's like you've inversed it. So essentially what happens is your win values get reduced because the fees uh ampl like really decay the the wins and then the losses get amplified and typically what will happen is your positive EV turns to negative EV [snorts] and this is why for example this could be you could do this but typically as like as a making strategy. Uh so but yeah we we'll I'm not sure if we'll look into making maybe we'll look into making but maybe much later date. So I [snorts] think that's ah yeah okay so maybe just go over the so we've gone over the model we haven't so the next videos is that we're going to do this is we're going to look into classification which is so we've looked at regression but we haven't looked at classification and classification is where we classify into labels and more specifically we're going to look into classifying it as a label like it's going up or if it's going down and with classification you get an associated probability So for example, you get a probability of it like okay, what's it the probability of it going up and we could potentially use this for our strategy. It's like okay deciding when to to buy or sell or u yeah anyway um so then once we've gone into classification then we're going to go into cross validation and also some back testing because we haven't gone over some of the back testing performance metrics. Uh so that will be the next part and then after that we'll go into uh into strategy where we actually like use this model to build a strategy and there's some very important decisions and the strategy is also equally as important as the model itself and oh yeah so let's do some uh I I didn't do exercises last time uh because I couldn't really think of some good exercises but this time I do have some okay so some exercises is just to to build up uh being comfortable working with matrices uh is to do the following. So the first exercise is to do the dot product to implement the dot product and I strongly believe to understand the mechanics is to actually start building your own algorithm for it. So, um, what we're going to I'm just going to just copy and paste this code. So, and I want you to write a manual loop, right? Manual loop to do the dot product. And the the dot product is basically you're multiplying x by y. So, uh, and to know if you got the correct result. Uh, okay. So what we want to do is to is to pass this test. So at the moment it's false but we that's the code but obviously don't just this has to be done as as a loop. A few people have posted that you know they just pasted this and but yeah it has to be in a in a loop. Okay. And then the next one exercise two is to transpose. Okay. And this is quite a common matrix operation is to is to transpose. Uh so and to transpose is essentially flipping the so for example we look at the shape is you're you're flipping the the values of of the shape around. Uh but so then I'll just make it more concrete is that this is what you want to do. So the shape of at the moment is two three and we want it to make it three too. Uh and just in case you're not sure what then what I'm asking you to do is that I've got the example of like what it what it looks like when when it's transposed. Uh so yeah so then we see look here it's false and then another important uh operation and we should be able to be able to do it by scratch is the hadamard product okay we just okay with the hadamard product essentially what we're going to do is let's just assume that we've got our predicted our actual values and then the predicted denoted as yhat and we want to measure the error and we're going to measure the error as the difference between them. Uh so essentially what I want you to do is to calculate the the error and why is this not okay here we go [snorts] so that I want you to write a loop uh to calculate the difference between this matrix and that these two matrices and this denotes the error. Okay. [snorts] and and this is what the the value should be. So if we look at the error then uh yeah error is not defined. Oh, because I didn't. So, um, so yeah, there's that. I think, um, you may have some issues with like floating point uh precision. Uh, if I do, I'm I'm just going to run it myself just there's um a numpy to to to factor in like imprecision errors. Uh, but yeah, that's roughly the the code anyway. So yeah, that's the the whole video. Thank you for watching and uh see you in the next video. Cheers. Bye. This course is free thanks to my paying Patreon supporters. If you're getting value from this series and want to help me create more, please consider joining them on Patreon. You can also support me by buy me a coffee on buy me a coffee.com. Links on my channel details page.