Analyzing Backtest with Pyfolio - Algorithmic Trading with Python and Quantopian p. 8

Key Takeaways

what's up everybody Welcome to part eight of our quantopian tutorial series as well as part 20 of the Python for finance Series in the last tutorial what we did was we took our Alpha factor and just tried it against the market um this was the final test basically I kind of skipped us having to run all these full back tests but we at least coded it out first once uh and what we're going to do here is we're going to take the back tests and we're going to run it through p and the way that we do that is through the uh notebooks so if you were to go basically to your notebooks and this is the one I think that we yeah Q research tutorials so this is where we've been programming so we can just kind of keep adding to this one if you want or you can make a new one it doesn't really matter so what we're going to say is BT for back test equals getor back test quantopian magic and we get the back test now um there might be a better way to do this but the way that i' do this is I just come here right and then what we can do is we go to all back test let's grab the this first one so the first one was on using quantiles let's see if using quantiles commission was set to 0.001 and over that onee kind of Market neutral-ish period basically so Market changed a little bit but it's only 1.3% but in the meantime it went up and wiggled around then went down wiggled around and went back up and you know basically almost no no change so on this one so this the first number here is your user I think this is the back test so we'll copy that come over to research paste and then run that should grab it really quickly so I'm going to already start the next line so bt. creatore full teore sheet we'll run that and this like the other one we'll kind of take a minute to populate but you get a bunch of information here and right away I mean we've got a lot of of why did you do that do this okay so we got a lot of information here a lot of this is already available to you on the uh actual back algorithm back test but anyways you can kind of immediately see okay where's our worst draw down periods net draw down of those periods where did that happen when did we recover um how long did it last all that cuz draw down is like something you're going to really want to watch out for um cuz draw down basically is just add to your risk also draw down pretty much will inhibit you from um applying leverage so if you have like no draw down you could in theory just keep throwing leverage at something um and do better but the more draw down you have the less leverage you're going to want to apply because leverage does crazy things it doesn't just multiply like you can multiply you know leverage multiplies your investment ability but it it does not just simply multiply your returns uh it it does very strange things in in reality um so it can it can just ruin you especially with in things like draw down so then we've got cumulative returns we already kind of knew this um but these are like some interesting ones so this one's like cumulative returns volatility matched to The Benchmark which is kind of cool because it'll just match you based on volatility to your benchmark which is an important factor uh cumulative Returns on a logarithmic scale nothing really to look at there You' got returns basically on percentage over time um so of course as we can see here towards the end is where we kind of did a lot better um rolling portfolio beta uh to the equity so this is something you you really kind of want to see as flat as it is we're pretty much zero beta so that's a good thing uh the rolling sharp ratio over time not bad we we did improve over time so that's kind of good by the end of it um we still only had 0.89 which is not not very good um let's see these are the top draw down period basically our entire strategy is a draw down period okay uh this is your underwater plot basically all the draw down so just always keep in mind the uh the y axis here monthly return so we can kind of see how we did monthly basically we did pretty good here and then really good here otherwise we kind of uh not very great um then we've got like some nice comparison charts here new normal in like basically fall 2015 how we did um basically we held out really nice and solid and the S&P 500 just I don't know went went down south uh these are our best long positions a bunch of companies I've I couldn't tell you the name of any of these oh this is Urban Outfitters ha I did it I'm pretty sure that's Urban Outfitters anyway the top 10 short positions I should know this one I I recognize it I can't tell you what it is though uh and then top 10 positions of all time that we did okay so you get a lot of information there coming on down here our exposure this should be totally perfectly even because that's what our strategy dictated uh we can see the portfolio allocations over time uh let's see long short medium position concentration awesome continuing long gross Leverage which should pretty much be one um unless you intended to apply leverage uh coming on down trading volume okay we finally made it finally made it to the end okay so that's one of the back tests now we could keep doing it like so this one was uh that first one that we did right which just barely outperformed the market uh but then we can go to let's see all back test we can do the same thing for like for example let's do this was over the entire time this will be an interesting one to look at actually well first let's compare I guess I do cuz cuz that first one was quantiles this one is not quantiles this one was uh based on the actual signal itself so in fact let's just let's just overwrite this one paste and then once that's done we'll do this down here creating the new sheet um so again you can see your return your volatility sharp is much better on this one let's see yeah we ended with 1.85 uh and again you know all the same things and generally like once you start seeing that like um like for example here your cumulative returns that's pretty much what we already saw but I do like the whole volatility match to Benchmark uh generally you you'd like to be able to beat The Benchmark and ideally as much as you can uh this one has pretty much the same draw down period the same strategy pretty much anyway same graphs uh and then we could do the we could do the last one because the last one is over like the full the full uh at least the full range that we can do with sentex paste that one in but we already pretty much know what what the problem is or at least what we expect the problem to be and it it's just it all boils down to that spread of alpha so a weak Alpha overall even though the mean is positive it was still pretty weak and we have this just huge spread of alpha over um you know the two buckets that we we we worked with this one's taking longer it's because this one's like the three years or whatever three and a half years of data wait for it so this one will be good to compare because if you look at like for example our return turn compared to the S&P 500 we did not do very well right uh but we also didn't really we doesn't I think I don't think we ever got into the negatives oh we did at this point that's negative and then over here did we get back into the ne no we did have draw down though um but this is better draw down than what we had before uh compared to our returns anyway coming back over here um I want to look at that volatility chart so we scroll down so this is the accumulative returns makes us look as bad as we did um and then volatility match to Benchmark so we're actually like volatility wise uh we could we're close we're still less So in theory like let's say you applied some leverage uh we would have still been kind of eh not the greatest uh and would it have been worth the added risk of that I'm not really sure uh cumulative Returns on logarithmic scale not what you wanted to see we' we'd like to be above also here we would like to be above the S&P 500 um I'm trying to think if there's anything else we but again we can still see here like the majority of our strategy is like is these draw down periods you don't want that you just don't want that okay so anyway all the same charts you've already seen so from this point basically we we kind of knew what we were getting into I think when we left Alpha lens with a single Alpha factor that was already kind of a weakish alpha factor and decided to trade on it we we kind of saw this coming but I just wanted to run through basically all of the steps that you're going to kind of go through as you're developing a strategy so basically you know at least for me what I would do is we're going to have to go back to the drawing board as far as like we're going to need to find some more Alpha factors combine them with this one see if we can't get that and once we combine them make that the factor that we test and see if we can't fix this the these spreads here um also the other thing you could just do is um put a stop loss on your trades so for example um let me go to the docks let's do stop loss or stop price let's wow it's loud again sorry stop price is it okay yeah so when you place your order you've got security amount but you can also throw in the style so you can say Okay I want to have a stop price basically uh and so this one will stop the order basically also known as stop a so actually you wouldn't do stop limit you would do stop order so place that if after you know maybe after the price has changed more than I don't know 5% or 3% or something in the wrong direction just just stop just just close it out U and that would probably help us um but that's like something you would want to do after you've already got like you want to start with strong successful Alpha and then make a strategy and protect it with stop-loss you don't really want stoploss to fix your Alpha problem okay so anyways um hopefully that was helpful for you guys uh thanks to Jamie mccorison for uh coming up with uh basically the the notebook in the the actual algorithm that we pretty much ran through I changed a few little things but most of that was actually his code also the webinar was was at least really helpful to me so if there's something more you're kind of confused about or you just want to run through it again um definitely check out that webinar I'll try to remember to put them basically in the description of all the videos that we've done so far um otherwise uh if you have ideas on how to improve this or whatever uh feel free to leave those below otherwise I will see you guys in another video