AI Trading Using Machine Learning (Step-by-Step)

Hey guys, so that. I'm going to show you how to use machine learning for trading in Python [music] and we're going to start from the very beginning. You don't have to be an expert and you wouldn't believe how easy it is. But I'm not going to lie to you. There are some caveats and problems when using machine learning in trading, but that brings me to the best part of the video, which is not only I'm going to show you the implementation, but I will show you a clear path for moving on so that you can keep going on till you get the results that you want. Now to get to this point, I had to do a lot of research and in try a lot of different implementations to get to this point. And one book which I found really helpful was called Advances in Financial Machine Learning by Marcos Lopez and I made it really easy to implement his concepts. I'm going to show you what I mean by that in a minute. All right, if all that sounds good, let's get right to it. All right, so let's spend a minute and talk about what is machine learning in the first place. Now machine learning is a way for the computer to learn from some patterns in order to predict something. Now if that sounds confusing, don't worry. But for now, just know that it is the opposite of setting some rules, which is what we usually do. So let me show you that with an example. So suppose we want to define a very simple function in Python which just adds two number, all right? So we would say something like this. So define some function and it's going to take two parameters and it's going to return A plus B. And if I were to use this function, I would simply do this. So 1 plus 2 and and let's, you know, print the values of it. So now if I go ahead and just run this file, I would get number three, right? So again, nothing fancy. But what if in a very special scenario, we did not know this logic. So we didn't know that to sum up two numbers, you're going to have to use this very special character to make it happen. So maybe it was something complex, right? If we were to use machine learning to solve this problem, to be able to add two numbers. Instead of this, we would have had to have some numbers. So, first we're going to have to prepare some values. So, for example, if the first two values, which is one and two, and the third one, which was the result, was three, well, we're telling the model, "Okay, so one and two equals three." Okay, but this one is not enough, right? So, we could give it another one. So, three and four, and then the third one is going to be what? Seven. And then we can again repeat. So, two and three, and the result is going to be five. And even my autocomplete, the one that I have on my editor, is learning this very fast, right? So, if I give it like Actually, no, it's a bit dumb. So, 0 + 9 equals again 9. And 0 by 0 equals 0. Okay, so this one's good. 1 + 6 = 7. So, you see, even the autocomplete on my machine is learning learning this very fast. So, now imagine if, instead of like seven or eight examples, I had like 1,000 examples. What machine learning does is that you're going to feed it these values, and it's going to give you this function. Although what gives you, it doesn't actually have these values, all right? So, it's just going to be the result of it. So, it's going to be a function called whatever, in this case like some funk. And this function is going to be able that when you give it a value such as like five and six or seven, it's going to give you the number 12. Without actually knowing the logic behind it, because it learned from the examples that you gave it. All right, so this makes sense. But what about in trading? Like, what are we going to try to learn and what? So, usually we we want to follow some exact and clear entry and exit rules for the strategy. So, for example, we want to say, "If the RSI is above this value, and the EMA crosses like the other EMA line, which is like faster or slower, and then we want to buy or open a long position. And then if the opposite of this happens, then the trend is against us and we want to look liquidate the position or exit it in other words. So, we have to have these indicator values in order to define these rules. And we also need to know these rules. But the problem is that so many people are following the exact same rule. So, for example, if you try to use something such as the RSI or the Bollinger Bands, let's say alone and that one single indicator, at this point the edge of it is mostly gone because so many other people know about this, right? So, like we know how many people just use TradingView. But what if you were to use some new data? So, for example, the funding fee of the exchange or the volume, things that not as many people use or any other kind of alternative data that you know has some kind of predictive power or at least you can guess that it does. But you cannot put it into words, like you cannot write an exact rule for it or cannot find a profitable one. Well, what if you could search through the data and just find something that actually works? Well, machine learning can do that for us. But what kind of features do you want to use? Well, in the case of trading, we could use anything. It could be some kind of indicator value such as the RSI or the EMA. It could be the volume, which again to me personally is never easy to use with an indicator, but with machine learning that is very different. It could be the time, like what time of the day it is, which day of the week we are in, which part of the month are we in, or the weekend or the beginning of every week. So, we know that these days are going to be different. But how are you going to use that with an indicator? But again, with machine learning you could just feed it into the model and maybe, just maybe, it can find a good pattern. All right, but the next question is what should the model try to predict? Like what exactly? Because the simplest one that comes to mind is the price, right? Well, actually, if you ask people who do this thing properly or successfully, if you look them up, they're going to tell you trying to predict the price is the hardest thing to do. So, there are much easier ways to use machine learning. Now, predicting the price, yes, it is one way to use it. Another way is to find a way to set the position sizing of your position. Another way is to simply use it as some sort of filter, like is this trade that I'm I'm actually about to take, is it profitable or not? If the model says it is, I'm going to take it. If it says it's not, I will not take it. And by doing this, we're going to improve our win rate. So, this is another type to do it. But, one which is very simple and handy, and what we're going to try in this video, is to simply try to predict the direction of the price. So, not the exact price, but the direction of it, like whether it's going up, whether it's going down, or whether it's in a sideways. And if I want to take, let's say, long positions, of course, I will only take it if the model says that the price is about to go up. So, that's it. I will only try to predict the direction of the price. So, for an uptrend, it will just give me one. For a downtrend, it will give me minus one. And for a sideways, it will give me zero. So, basically, like any kind of indicator that I would usually use in a strategy for the direction of the trend. Now, in machine learning, because we only have three types of outputs, this is called a classification problem. Now, another way you might have heard about a classification problem in machine learning is when you train a model to simply tell you if in this photo, is this a cat, is it a dog, or is it a human, for example. So, in this case, if you only expect three these outputs, it's going to be again a classification problem with three outputs only. Now, of course, this was a super simplification way of putting things. But, before I move on to the code, I want to talk about something really important. So, even if you are already with machine learning, I'm betting that this has been a really big problem for you. For me personally, this was the very reason why I tackled this thing way later than what I should have. All right. Now, this is the best part of the video because even if you have used machine learning in the past successfully in other fields, you're going to need it because I'm going to show you a solution that I found to a big problem. Now, what's the big problem? Well, you see, when you want to use machine learning for a simple problem such as detecting whether the photo is a dog or a cat. You simply feed it, let's say, thousands of photos, and that's it. The model can predict it pretty well with a very good accuracy. Or even when the model is trying to learn to play a game in the case of reinforcement learning, which you don't have to know what exactly it is, but my point is if you have a problem that is simple, it doesn't change, or in other words, in technical words, it is a stationary, then it is very easy to train a model. But in finance, we're dealing with random data, random noise, and markets which are changing all the time. And even different markets usually behave differently. So, often times you write a strategy, let's say, for BTC, but when you try to trade the same strategy on an altcoin such as, let's say, SOL USDT, you find out that it doesn't work at all. That's a really a scary problem, which means the accuracy of your model it's it's not going to be like 99%. It might be 60%. It might be significantly lower. Now, we all know this problem exist, but as I said, even if you get accuracy of, let's say, 60%, it's good, right? Like especially if strategy has a win rate of, let's say, 40%, and then out of nowhere a model can have an accuracy of 60% just to know if the trades are profitable or not, that's going to be huge. But, here's the problem that I didn't have a solution for. You see, suppose I write my implementation, and I feed it some data. Now, it could be any data, let's say, just price data and volumes and some indicator values, the things that we easily have, not some alternative data such as the funding fees or things like that, okay? Now, suppose that I'm not getting good results. I'm getting negative results. Like my strategy isn't profitable. So, yes, we are getting the direction of the trend using this model which I'm about to train, but how should I know the problem is with my model or with the implementation that I've done? So, I thought about this question for a long time and like I said, I postponed machine learning altogether until I came up with a solution. In programming, we have this concept called TDD, which it stands for test-driven development. So, let me show you very quickly what that means in case you're not a super developer or you had to you just haven't heard about this. All right, so let's copy everything we had before. Okay, so you remember this example that I gave earlier? So, 1 + 2 = 3, 3 + 4 = 7, 2 + 3 is supposed to be like 5. Now, if you want to implement this using TDD or again, test-driven development in Python, first we would have to write the test and then we would write the implementation. So, here's the example. So, I'm going to write a simple test function first, all right? So, I'm going to call it test some function and in it, I will say assert that some function when I give it 1 and 2, the result is going to be 3. We can also test for other scenarios because let's say the function is supposed to get one of them right, but just in case, I want to add some other scenarios, right? So, again, now, 3 and 4 = 7. All right, so it wrote these based on these examples that we had before. Very clever, actually. So, if I just do it one more time, yes, this is correct and so is this one. All right, so now we have all these examples here, right? You see we're getting an error here because we haven't defined the function yet. So, that was my point. So, first we define the test and then we define the function. Now, it doesn't really matter, but my point is that we're to have to have tests for it, right? So, now I can write the function itself. So, def and some function. When I give it A and B, it should return A plus B. And now we're not getting an error anymore, right? Now, just for this to make a little bit more sense, let's duplicate this line and comment these. And now, let's say I was going to say that 1 + 2 does not equal 3. It equals, let's say, 5. Right? So, we know that this is incorrect, right? Now, we know this because we humans can easily sum up two integer numbers. That's very easy for us. But again, imagine that this is a super complicated function. So, it's not a simple one, and we don't know what it's supposed to return. But, now that we're trying to use TDD to write this, we know that the sum of 1 and 2 equals 3. So, we know that 5 is incorrect, right? So, I'm going to run this file by simply saying pytest tdd.py. Now, it doesn't matter what pytest is. Just forget about it. The point is it's how I am executing this, and I just want to show you the concept, okay? So, if I run this, it's going to tell me that it's a fail. And it is saying that 3 does not equal 5. So, this is what we were expecting, right? So, 3 should equal 5, but it doesn't. So, it is failing. So, if I comment this and bring these back, which are the correct values, and run this one more time, it's going to give me a pass. Now, forget these warnings, okay? So, it's giving me a pass. So, now that I'm getting a pass, I can be sure that my implementation of summing two integer values is correct. Why? Because all these tests are passing. Now, what if we did the same thing in machine learning? Well, if you run a test for it, and then it passes, then I can easily run a back test or deploy it for life. And if it doesn't work, I can be sure that my machine learning implementation is correct, and the problem is from somewhere else. Maybe it's in my position sizing, risk management, maybe it's in my exit rules of the strategy, or any other part of it. But, if it failed, if the unit test, or whatever kind of test that you want to call it, if that test failed, I can be sure right there that my implementation is incorrect. Then, I can dig further. Maybe the data that I'm feeding is incorrect, maybe the features are incorrect, maybe the labeling way that I'm doing is incorrect. Whatever, it doesn't matter. The point is I can know exactly where I should look for the problem to solve it. Now, I hope this all made sense to you, and I don't know about you, but as someone who has struggled with this concept, this simple solution that I just explained, it changed everything for me. So, in the rest of the video, yes, I'm going to implement things, but we're also going to test it, and that's the really important part. All right, so now that I've explained these concepts, I can move on to the actual strategy. Now, as always, I'm going to use the Jesse framework, which just recently added the machine learning and stuff, and you kind of see how easy it makes it for both training the model and deploying it for back test, live trading, or whatever that you want to do. All right, so, we're going to have one script to collect the data and train it, and then run the actual back test, and we're going to have the strategy file itself, which is basically the one you guys care the most. So, let's begin with that one. So, you see, here we have a simple strategy class, which is inheriting from the strategy class of Jesse. Now, if you have watched my previous videos or familiar with this framework even a little bit, you already know what these are and what type of properties it gives you in order to write a strategy, which makes it really easy. Now, we also have some comments here, which will describe the strategy's logic if you want to go through that. But, for now, let's just talk about what we are trying to predict here, okay? So, the method that we're trying to use here is called the triple barrier vertical method, okay? Now, I got this concept from the book that I mentioned in the beginning of the video. So, you could go and give it a watch, but it's actually super simple and the book didn't really add anything except just maybe one thing or maybe it made it a little bit clearer for me. Now, what is it? Well, we simply try to predict that n bars from now, if the price is going to be higher, lower, or if it's going to almost stay the same or in the concept of trading, are we going to be in a range market? Now, why is this important? Because usually when we open a position, like assuming that we only have one entry and one exit, the entry could, let's say, be done by a market order. Nothing fancy. But, the exit could happen with either a stop-loss order or a take profit, which is usually a limit order, right? And let's say it's a long position, all right? So, if the upper barrier or, you know, the higher line is touched first, we're going to say, "Okay, so this is a plus one." As if the direction of the trend is toward up or, in other words, we are in an uptrend. And if the stop-loss is going to be touched first, meaning that we're going to lose money, then this model is supposed to return minus one. In other words, it's saying that, "Hey, maybe we're in a downtrend, so don't take any long positions." And if it returns zero, it means that our vertical barrier is being touched first. Now, what is a vertical barrier? Well, you see, I said that after n bars, okay? So, we need to have some kind of box. So, suppose that we open the position right now. Now, n bars from now, now n could be any number, such as, let's say, 10 bars. All right? So, 10 candles from now, what is going to be the price? Like, are we going to touch the upper barrier first or the lower barrier first? Or, if we're not going to touch either of them, we're going to consider it a range market and in that case we're going to return zero, because we have to have some kind of window, right? So, we cannot, like, open the position and wait like three months for a simple scalping strategy. It doesn't make sense. So, there has to be some kind of window. And we're going to have to define the window. And in this example, I defined it the number 50, which you can find here. So, feel free to change it however you like. But basically, so that's what we're trying to predict, right? So, it's a simple classification problem, and it either gives us minus one, plus one, or zero. That's it. Let's move on to the other parts of the strategy, starting with the before function, which is basically the one that you're going to use especially with the type of machine learning that we are using in this strategy, which is the triple barrier method. Okay. Now, we're simply saying that if you are in the gather mode, return because if we are in the deep play mode, we don't want to do the training and stuff that we're going to do right now. We just want to use the model, okay? Which we're going to cover later. All right. So, we're simply saying that if you haven't recorded anything yet, let's record the features, which I'm going to show you how that is. And then we are setting the upper barrier, the lower barrier, the index, the index that we started doing this, which is right now. And we simply give it the current index by simply saying self.index because that's a built-in property of Jesse. And then we're going to say, "Okay, so features have been recorded." So, this is this flag. And what it does is that on the next candle, we're not going to go through this. Okay? Not until at least like we have successfully recorded one whole you know record for machine learning. Now, starting the next candle, we're going to go here, right? So, we're saying, "Okay, has the upper barrier been touched?" And to do that, we're saying, "Okay, if the current price is above it, then it's been touched." We do the opposite for lower barrier. And then for the vertical, we're using the time. So, we're saying, "If the current index minus the recorded index, which we recorded here, is more than the vertical barrier, which we set it to number 50, if you remember." Okay, so it was here. Again, this could be any number that you want. And we're saying, "If the upper barrier have been touched or the lower one or the vertical, then the label is going to be one if it was the upper barrier. It's minus one if it was the lower barrier and it's zero if it was the vertical, okay? And then we are using self.record_label function of Jesse to record it. And we're giving it a name, which could be anything you want. And we're setting the value, which again is either 1, -1, or 0. And then we reset these flags, okay? So, nothing fancy. So, so far you have used two functions for machine learning. One is the record features, which is where we basically give the inputs of the model, and second is the record label, which is where we get the output of the model. Although, because we are in the training mode right now or gather mode as we are calling it here, we have to feed the output to the model. So, that's the thing. When we are in the gather mode or training mode, we have to feed the model both the input and the output. But, when we are in the deploy mode, that's when we're going to say, "Okay, here's the input. Now, give me the output. Predict the output." So, that's what machine learning does, all right? Now, what are the features that we are feeding it? So, the features are here. So, this is the function that you have to define in your machine learning in Jesse. And this has to be the exact name of it. And what it does is it simply returns a dictionary. Now, in here, these are the things that I have defined and, you know, I'm explaining it a little bit here in the comments. It doesn't matter. But, the point is we are defining the ATR, the current price, the EMA 9, 21, 50, the recent close numbers, the Keltner indicator, and some things like that. But, if you notice, I'm not just passing these values because they are not stationary, I'm turning them stationary first and then I'm using them. So, for example, here instead of simply the ATR, I'm saying ATR divided by the current price. Instead of simply the EMA 21, I'm saying 21 minus 50 divided by EMA 50. So, I'm giving it a ratio. Because these values are stationary. But, if again, if I simply give it the price or the EMA, which also looks like the price, it's not going to be a stationary. Now, by the way, I made a whole page on our documentation about the stationary and why that is important. So, definitely make sure to check it out. Anyways, so we are returning a dictionary of values. So, those are my inputs and again, my output. So, that's it. That's all we had to do in order to train the model. Now, the next part is using it and that's also going to be really simple. So, for using the strategy, we're going to use the typical Jesse functions, the should long, should short, which are used to know whether or not we want to open a long or short position. So, basically, the entry rules of the strategy. And in it, we are simply saying that if you are in the gather mode, return false. So, because we don't want to be recording any trades in the gather mode or training phase. And again, this is just for this type of machine learning. For another one, for example, if you want to know if your trade is going to be profitable or not, you actually want to take trades and that's going to be the data that you're going to train the model on. But, in the triple barrier method, which we are simply trying to predict the direction of the trend, we don't want to be taking any trades. All right, so in it, we're simply saying the probs or probabilities is going to be self.ml.predict_probability. And this is a built-in function of Jesse. So, it makes it super simple. So, basically, assuming we already have the model, this is all I have to run in order to take it. And now, I have both the probability of the price going up and it going down. Now, I could also get it, you know, being in the vertical, but I don't need it here, so that's why I'm not using it. But, the entry rule of the strategy is going to be, so if the probability of the price going up is bigger than my threshold and if the probability of it going up is also more than 20% more than probability of it going down, then I want to take a long position. So, again, this is just something I defined. You could play around with the numbers. And also, if you want to see what's the threshold, well, we defined it here, so we set 45%. So, we want to know if the model has at least, you know, a little bit of confidence before actually taking a trade. Now, this is a test strategy that I'm running just to demonstrate how these things work, but in a production strategy, you probably want to have this condition as a secondary or just, you know, one other filter for your strategy. So, you probably don't want to use it as the single point of truth for the entry rule of the strategy. And that's it, guys. Now, for the short position, we're doing the opposite, and this is where we do the position sizing, the go long and go short method. So, assuming that the should long is returning true, we say, "Okay, so now let's go long." And if that's the case, my entry is going to be this price, which we passing the current price, in other words, the market order. And distance is something that I'm calculating here, simply using the ATR, and then I'm getting the quantity of it, and I'm submitting the buy order, which is my entry order, the stop loss, and the take profit. Now, usually I submit these two in the on open position method of Jesse, which is this function where it says, "Okay, so now that we have an open position, let's submit the exit orders," which would be these two, but we can also set it here, so this is perfectly fine. And that's it. That's all that we had to do. So, let's go up, and yep, so this is the distance that I talked about, which I'm using the ATR, and that's it. Like, this was an entire strategy for using machine learning in Jesse. So, that's how simple it just got. But now, let's move on to the script, which you're going to have to run. Now, in the documentation, I have mentioned a couple of scripts and how to run them, but here I'm going to leave you with one, at least for the triple barrier method, and I'm using the TDD. Now, what is TDD? Well, I I explain. So, basically, we're creating some synthetic data, right? Some fake data. So, first we have to create that fake data, and I'm importing some libraries and also the strategy I just showed you guys and I'm defining the number of regimes, you know, the price regimes that are going to change and the total number of candles I'm giving it 6 months. The exchange I named it test exchange. The symbol is test USD and the time frame is 15 minutes. We are calculating some stuff for the one hour candles and so on and this is just for the printing output and this is where we define the price regime. So first we're going to have an uptrend, sideway, a downtrend and again a sideway and again we're going to repeat this whole thing four times which will give us four in three in total. Now you can also change this and 6 months may not be enough and again this is just a test one. Feel free to change it however you like and then here's where we actually generate the close data, right? So we're saying that okay, this is a empty list. The starting price is 1,000 and if the direction is plus one, we're going to add to the price by 2.5. If it's in a downtrend, we want to subtract it and if it's in a sideway, we are just using a sinus method to add to it. And that's it. It returns the prices. Now these are just for printing stuff, not important. And then the first phase which is the gather mode, again printing itself doesn't matter. We're getting all the candles. We are defining the routes according to what Jesse expects. So we're simply giving it the exchange symbol and the candles and we're returning the config. The fees are set to zero. I'm passing the routes, the one hour candles and everything and that's it. This is returning the data points. In the second phase which is where we do the retraining, we're using the train model method of Jesse. Now this method has been added to the research module of Jesse and you have to import it and it takes the data and some parameters of the estimator or the classifier of the machine learning. Now here I'm using the random forest classifier and this could be anything that you like. It could be a support vector machine which actually works pretty well especially when the number of data that you have is limited, or it could be any other type of classifier. And this is really important. So, not only I made it simple, I also made it flexible. So, I didn't want to limit you to just like one type of machine learning. So, that's why you can just pass any kind of classifier and it will just work. And these are some values that I have defined, which help with the model. Now, for these values, if you're not sure what to set, just ask AI. They are really good with these things. But especially if you want to prevent the overfitting, or if the number of classes that you have are not balanced, like these values are going to be important. And then I'm passing the task, which is multi-class. Now, if we only had two types, like a true or false kind of output, we could have set this to binary. And we also have the regression type, and we also have the multi-class, which could be more than two, like in this case, just three classes. And then we have the face deploy, which is actually where we actually run a backtest and see some results to see if the model is actually working correctly or not. And in it, we're using the backtest function of the research module of Jesse, which isn't really something new. We always had this. Now, this one is again for printing stuff, and this one is for validating the model, because we want to see some metrics in order to be sure the model is actually working properly, so we don't just run a backtest. First, we validate the model, and then we run the backtest. And this is the main entry of the file, where we actually run those face functions that I just showed you. So, the data point, the training point, then we are validating the model. After we get the results, then we deploy it to get some backtest results, and we print some final stuff. So, that's it. And again, I'm going to open source this file, so you guys can see it and run it for yourself. And now we can actually run this by saying Python triple barrier, and that's it. And as you can see, it's actually pretty fast, so it's not going to take a lot of time. All right, so a lot of stuff are not absolutely necessary, they're just some helpful output that we are printing. So, for example, the total number of bars, the bars per regime, the price change, so we started from here and went up to this, so it was clear uptrend overall. And these are the regimes, so the first the uptrend, the sideways, downtrend, and you can see what the price it started and where did it end. Again, nothing that important, although it does help you to ensure that the data that you generated is correct. And here we can see the number of features, the strategy, the regimes, and things like that. Now, this is the backtest results initially, and it says no trade were opened, which is what we wanted because, like I said, if you are in the garden, what we don't want to execute any trade, so this is correct. And we can also see that what was in total number of data points which we used for training. It was 1,584, and 46% of it was just the output being minus one, 42% was one, and 10% of it was a clear sideways. All right, so this is talking about the data collected, and this is talking about data that was trained, which is exactly the same thing, basically, so you don't have to read this again. And then we have the feature importance, which will tell you which one of these features that we used were actually helpful. It gives them some kind of score, and some F value, so these are some standard metrics which are really helpful, and here you can also read about every single one of them, like which one does what. Now, if this isn't super clear to you, it doesn't matter cuz we have another one which is clearer, so let's just skip it for now. And here it says the type of the classifier we used, and here we can see the accuracy of the model and some other metrics. And as you can see, it got a accuracy of 99.7%, which is almost 100%. And this is how I know that the model is working. Again, the reason we are seeing such a huge number of accuracy is because we're using synthetic data, fake data in other words. But in a real-world scenario, there's no way you're going to get this good of a result. Okay? And that's perfectly normal. But, because we were doing TDD, we're creating the tests ourselves with fake data just to see if the model works or not, we actually wanted to have such a high accuracy. So, this is expected and this shows that my implementation is correct and I'm ready to move on to the real prices of the market and try to beat the market. And if it doesn't work, I can just play around with my inputs and things like that. But, the implementation itself is correct. Now, this also tells me like how many times the model predicted that it's going to be minus one and how many times it was actually, which you can see it was this. There was just one case where it predicted that it's going to be zero, but it was actually minus one. And that's why we did not get an accuracy of 100%. And for the other stuff, it's 100%. So, it predicted zero and one perfectly fine, but for minus one, it just got one thing wrong. Now, in a real world scenario, you really going to love this table. So, it's my favorite, actually, cuz it tells you how many false and true negatives or false and true positives exist with the model. Now, if you don't know what that is, don't worry about it. Once you get your hands dirty with machine learning, you're going to hear it all the time. Now, we also have some other metrics of the model, so we can know for every single output, for minus one, zero, or one, what was precision, what was the recall, and what was the F1. Now, this is also really, really important because you don't want just the precision of the model to be high, like the recall of it and the F1 are also important. Now, if you don't know these metrics, again, look them up because they can be pretty helpful. Now, this is that table that I said, you know, I'm going to show you later and it's super helpful. You know, instead of this one, which is might be a little hard for some of you guys to read, you could just use this. And basically, it gives us uh the fact that what if we did not use this input? Like, what would happen to the accuracy then? And based on that, it's going to say if this input is actually helping or hurting the predictability of your model. Now, in this case, we're getting the exact same number. It's saying that all of them are neutral, so it doesn't matter if I drop them or not, and the accuracy is not changing, and that's because I'm using fake data. But, in a real-world scenario, you're not going to see this, and for some of them it's going to say, "Okay, so these are helpful, keep them." And for some of them, it's going to say, "Okay, drop them because they're actually hurting the performance of the model." Because you see, in a real-world scenario, you shouldn't just give the model as many inputs as you can. Because, sure, in that case, you could just give it 100 inputs, right? But, the reality is that some of the inputs are not going to be helpful. They will actually hurt the performance of the model. So, that's why you want to run this test, look at this table, and if some of them are hurting it, you could just remove them, and by doing so, you're going to actually get a better performance out of your model and data. But, because this is going to be different per each test, per each market or or things, so you don't want to just always use one certain input or always not use it. Okay, so this is why you need to actually test things for yourself to see if it helps or it hurts. And that's it pretty much. So, we got the accuracy again, the MCC, and it's saying the model quality is well above the random chance on synthetic data. The triple barrier labeling and training pipeline are correct. So, again, I know everything is fine. And if you were to run a backtest with this, we would have executed a total number of 1,816 trades, and the win rate would have been 99.83%, which is absolutely insane. Of course, you're not going to see this in a real-world scenario. The net profit or P&L is 251%, and that's because I did not use a compound position sizing. If I did, this number would have been crazy high, and the Sharpe ratio is 10.80, which again is totally unrealistic. And that's it. Now, guys, we are not done here. You should definitely check out the other types of machine learning stuff that just supports now. So, this was the multi-class that I just showed you. We also have the binary, which can be very helpful. So, for example, if you want to know if the trade you're about to take is going to be profitable or not. And maybe based on that, you want to set your position sizing. Or maybe if it's it tells you it's not going to be profitable, you just want to drop that entire trade. And by doing so, you're going to increase the win rate of your strategy. So, this could be super helpful. Check out this page. We also have another one for regression. Now, if you guys want me to make videos about it and explain it, well, drop a comment and let me know. I might do it. But, we also have another one called meta labeling, which is for setting the position sizing of the trade. And pretty much, that's it. Again, I also explain the stationary data and the importance of using it. And I'm giving some examples. So, instead of, you know, the price, you should be using something like this. Instead of the ATR, you should be using something like this. So, this is also a really helpful page. And this was my first attempt at adding machine learning to the framework. So, I cannot wait to hear your feedback, especially if you guys are an expert in machine learning. Let me know what you think about the implementation that I just did. And whether or not you want it to be more flexible or if there's something that we are missing here. Or if there are more ways that you want to use for testing the model and performance of the strategy, which is using the machine learning. Now, my goal was to make it so easy that you guys don't have to actually read about this concept as much as I did just in order to be able to, you know, try a couple of things. Because I found the implementation of machine learning very much scary, no matter where I read about it. And I think this just made it so much easier, at least for me. And I hope it does the same thing for you. Then we're going to have a giveaway. A random subscriber who likes and comments is going to win 1 million Punk Token. Now, let's pick the winner from the previous video. And the winner is, "Would be cool to see how things work out if you give each model personality trait like ex-legendary trait." Yeah, that sounds really great. Okay, thank you so much for your comment. Please do reach out to me so I can send you your tokens. Thank you so much for watching. I'll see you in the next one.