Testing Hazel's Performance! // Game Engine series

Key Takeaways

Oh sup guys my name is China welcome back to my game into the series so last time we took a look at improving how render a 2-d a bit adding statistics so that we could see how long everything takes check out that video if you haven't already and today I thought that we would do something a little bit different what we're gonna do today is take a look at a game that someone has made and we're going to both take a look at it in terms of like a stress test for a bat renderer but also just in general to see how someone goes about making a game in hazel someone from the community which is really really cool so you might remember from the last episode I asked you guys to come up with some kind of stress test for the batch render or basically just some kind of test case scenario something that we could use to actually assess the performance and the scalability I guess of our new batch renderer so someone called 0x has gone ahead and made a clone of this eddie's game called balderdash in hazel and gone ahead and posted it to the game engine series channel on discord and with his permission with his or her permission we're gonna go ahead and actually both look at it and use it as a test case for Hazel's performance and even though it's not particularly like a very very very heavy stress test or something that's designed to absolutely use every feature I still think it's a really good indication of the general performance of the engine and the way that this started was just with like a simple balderdash clone that was you know not much of really a stress test because again when you're remaking an a DS game usually it's not gonna stress it's not gonna stress out modern hardware or a modern engine but then 0x or ox maybe we'll just call me we'll just call them ox from now on zero zero ox has gone ahead and I ox is it's definitely not ox zero X's like at the beginning of like hexadecimal string but anyway whatever so 0x has gone ahead and actually kind of just expanded the level and turned it into a stress test I just got I've been talking with him on discord and that's what's happened so first of all if you guys are not on discord you're missing out link in description below the journal I'm such discord this is all happening in the game engine series channel if you're quick you can just scroll up and you can see all of this taking place you can just you know read that conversation and all of that stuff and in fact look at that your 0x is posting right now that hazel 2d is already my favorite engine so that's nice I don't know if that's sarcasm because there is a week face but we'll just take it as it is and the thing that I love about this particular example is it's not just a graphics stress test this is like the whole game it's it's a clone of the game so this is like you know I mean there's animation there's game logic there's kind of there's all this stuff happening there's even a little bit of like 80s style physics all of this stuff is actually happening in this application I think that's obviously a much better test than just like you know hey let's draw as many causes that we can with no real logic no real gameplay nothing like that that stops boring so I really like this thank you to 0x of course for making this clone and I will of course leave a link in the description below through the github repository that contains this balderdash clone so if you want to take a look and play this for yourself and just take a look at it then you can of course do that ok enough talking let's jump in and take a look at this balderdash clone and take a look at Hazel's performance okay so launching this this is what we get so where this little player here you can see there's a bunch of animators Bryce we've got a fair few quads and textures and stuff like that on the screen let's go through this and try and play it we can use the WASD keys here we have these diamonds we can collect I'm running this in release mode by the way just so that we can get an accurate performance kind of reading and the camera moves and we have these boulders crashing all in all I think it's a really cool little game demo thing let's go here collect this I really should be going a lot faster I don't know why I'm taking my time I have to be honest with you guys as someone who's relatively young I have not played the original at all so I I yeah this is completely just this is what's the word I'm looking for embarrassing ok I just died that's embarrassing almost pretty that pretty to that alright lets quickly try and get through this and we'll try and get into the second level so I can show you guys what that looks like if you're quick enough I did look up some videos of the original derp and I have to say that I think the original is a lot faster you can move a lot faster which probably makes it easier but then again maybe I just saw some weird video who knows let's just run past this let's get this again I'm going very slowly here let's see if I can just accelerate a little bit alright let's go over here alright get this I'm kind of worried about this because of course this could all crumble just by chance because there is a mechanic that I learned about we're in this game and I didn't know this because of course I haven't played the original where you can just you can just die because oh no I'm kind of stuck okay I made it okay let's go this way all right we're on level two you can just die because every like you can just randomly this stuff can randomly collapse on you so that's a bit bad like that was just dumb alright as scary starts it let's get the diamond and let's go over here anyway you guys get the coin I'm not gonna this isn't like a let's play but but if it was let's kind of hide here I'm dead on I can't do anything alright that's done I was gonna show you guys this enemy but it seems that I can't oh we can push boulders I did not know that alright this thing kills you as well as you can see okay that's the idea let's take a little bit of a look at the performance so if I just select one of these things here so that we pause the console you can see that the performance is pretty good like interlaced mode we're getting around it I mean it's quite sporadic because you know of course at this at this low of a millisecond count the framerate is gonna jump a lot even though we might only be like half a millisecond slower in a certain frame than another frame but you can see generally like wait we're around 2000 FPS half a millisecond I'd say is a pretty good average here we're getting a lot of frames that are a lot faster than that performance is great and we can't really easily compare this to what it was like with the old renderer because well we deleted the old renderer and it also uses a slightly different API but I'm sure it would have been probably a little bit slower I mean again there are not that many tiles going on here there are not that many quads so the draw calls aren't going to be slowing this down too much I think however this is where the fun stress test comes in so if we take a look at this stress test this is a much larger level you can see here is level it's just defined as a string here if we launch this and you can see here that I'm launching it in debug I just want to show you guys what debug looks like because I think that it's an important distinction to make here so this is debug you can see this level is huge you can see our frame time is quite terrible 6 frames per second it's running very very slowly you can see the renderer 2d statistics here so I've actually split this up into three draw calls I was just playing around with this before if we jump into renderer 2d we can set that back to like 20,000 because I just lowered this to 5000 max quads just so we could see if splitting it up into draw calls would make it any faster but anyway you can see that we're running at about 6 frames per second here it's really not that great that's what debug mode looks like but we do have this nice big beautiful level now the problem is it's because we're running in debug mode so if we change this to release mode and now we run it it's going to be a lot faster so this is running in release mode with iron GUI and you can see that first of all instead of 6 frames per second we're in like a hundred and 70 or so right our frame time is like around 5 to 7 milliseconds it's super fast but that's again that's with I am GUI I am GUI is actually taking about half of our frame time if I I made a little preprocessor macro here that I can just use so if I actually turn off I am GUI by removing it and we recompile our code and we rerun our code you'll see it's going to be a lot faster so these are just generally important things I think that you should take into consideration when you try and test the performance of hazel 100% make sure you were running in release mode debug mode tests especially with the amount of geometry that you're trying to render if you are trying to render like over 5000 or maybe even over 10,000 quads on the screen at once it's really gonna slow down in debug mode and we'll talk about why in just a minute but make sure you're running in release mode that's step one and step two is you have to but disabling i''m GUI will definitely boost the performance and I am GUI really is not a great indication because we're not going to be using I'm GUI ever for distribution builds of hazel I am GUI is only ever going to be used for tools and debugging that kind of stuff in hazel is going to use i''m GUI that's why we have I am GUI well for now it's useful for just debugging and like quickly like adding buttons and sliders and stuff like that to the screen so we can manipulate variables but in the future like in hazel dev for example I'll show it on the screen now we have a whole we have a whole level editor and that whole kind of well it's more of like a model of your at the moment but that whole tool is built using I am GUI and that's my kind of strategy for UI for tools but for runtime we are going to write our own UI library we are going to have our own like text rendering and all of that stuff so I'm go is kind of a temporary solution that you can use if you want UI in hazel right now but in the future that's going to change and hopefully we're gonna write something that's gonna be a lot faster than I'm GUI for the actual runtime UI that we end up doing but in general UI unfortunately does tend to be a little bit slower in game engines okay so now this is compiled if we run it now you'll see that the frame rate the frame time is now down to about three or so milliseconds we've even got some 2.8 here 300 frames per second seems to be about the normal here it's a lot faster without I'm GUI that's kind of the point unless maximizes and take a look so here's the player in the top left corner everything is very tiny we're actually rendering around 13,000 quads in this so it's again not an amazing stress test it's not like a hundred thousand we could definitely squeeze a lot more out of this it is running extremely fast through 2.8 2.9 you know three milliseconds per frame you can just ignore that for that you know it's running very very quickly and like the FPS might seem lower like 300 versus 2000 but remember FPS is not like a linear scale whereas this ears so you should always look at milliseconds instead because one millisecond slower is gonna end up being a lot more than like you know one unit let's just say less FPS like it's not a great comparison because X FPS tread tends to like exponentially get larger when you get you know a little bit lower towards lower end of the millisecond frame times so yeah take a look at that that's important all right so and you can see there's animations going this game logic going like it's it's cool and you know this one chasing meteor we've got the hazel logo here which is super cool we have a little bit of a hazel stress test which is gonna be nice now I want to quickly talk about why it's so slow in debug mode because I know that that's a question that a lot of people gonna have why exactly is it running so slowly in debug mode well long story short it's all the maths that we're doing so we're using GLM for mathematics and if you run GLM in debug mode it's well I care I haven't tested this but my very strong suspicion is that it's not using any kind of any kind of sim D rise single instruction multiple so what that means is like SSA and all those kind of cpu ins extensions that exist to actually use more than just your general kind of 32-bit floats so you cannot basically end up packing packing a bunch of data packing like four floats four 32-bit floats into a 128-bit wide register for example and then actually multiplying them together we're not gonna talk about SSE now I just thought I'd give a general definition of what it actually is for those who have never heard of it basically it is a cpu extension we'll call it that lets you perform operations on multiple pieces of data with a single instruction hence the name single instruction multiple data so what that means is instead of multiplying four floats with each other individually you can actually pack them into a 128-bit wide register well two sets of them and then multiply them with one CPU instruction and it basically just makes everything a lot faster long story short it basically makes your maths potentially four times faster it's not necessarily but potentially four times faster so it's very very good now without any kind of optimization maths simple maths like just multiplication and addition really falls apart really falls apart so if we jump back into here we are now in debug mode if I run this in debug mode and this is how I can prove my theory it's not really a theory but we'll prove it anyway just so that you guys are aware and just cuz I thought this would be a good kind of test case you can see we are getting 150 milliseconds per frame ridiculous so if we profile this so we'll go debug performance profiler CPU usage now note the visual she is actually telling us that we're in debug mode so we should switch to release for more accurate results but will will will still profile it indeed in debug mode it is still useful to profile things in debug mode by the weight because you can see like what exactly is the problem in debug mode and what the slow parts are in debug mode and also like a lot of the time it's very desirable to optimize debug mode because something like this is very difficult to test but you still might want all of your asserts and all of your debug mode stuff alright we'll stop that we'll look at this and if we double click on let's double click on maybe bold a layer on update then we're taking to the code we can actually see in red all the hot parts you can see that by far 64 percent of our time here is inside draw quad okay so this is very red it's very is a very hot path meaning that it takes up a lot of time if we click on it over here to go into it and see what the problem is you'll see that well the biggest problem here is this kind of just this multiplication operator and we can see up here GLM operator multiplied so this kind of operator here is taking up the most time by far so this is something that really really fades away in release mode and the reason for that is because our compiler is able to optimize that code it's able to use SSE in vectorization and it's able to well among other things like a faster floating-point model we're not going to get into like deep kind of optimizations but the point is in release mode maths becomes a lot faster and that's obviously very important because we are doing a lot of math over half of our time here is spent just adding numbers together that is why this stuff is slower if we were a little bit more like cache coherent maybe this would be faster because we are kind of skipping where we're not exactly completely as like you know CPU cache packed together as we could be for example the texture index is all the same for a lot of these it's hiring factors all the same the texture coordinates are all the same you know the colors are all the same but we're still kind of having to process all that data and retrieve more and more memory and when you scale it up to like over 10,000 quarts that can also slow it down so I'm sure if we rearranged the kind of the packing order and like we'll rearrange the memory layout of all of this so that our positions were all like let's say all in a row meaning that when we had to write to position we could just kind of do that easily then we this will probably be a lot faster this particular stage but of course OpenGL does need everything to be interleaved we can't we can pack like positions say into a separate vertex buffer and do that that is actually possible I haven't tried that I've actually just thought of that now we can try that if someone wants to try that go ahead post it in discord and I'll take a look at it but you know we are kind of limited by it by a few things here the moral of the story is superfast maths release mode that's why it's important to profile stuff in a release mode and in fact as a little test let's go ahead and jump in to release a launch release and make sure it's running it is running what I really meant to do was actually go into debug performance profiler and we're gonna profile this in release mode just to see where the bottleneck is in release mode so let's let it run for a little bit maybe we'll move around a bit and we'll hit stop collection here and let's take a look at what this looks like okay so what do we have here application run on update so application run itself is like you know not taking up that much time I mean it's 60 something hazel on update I mean look it still is draw quad like this is still kind of the bottleneck and this is now where it's like making me think that maybe it is more or less to do with our memory layout and that's probably the next thing I would maybe look at optimizing we have this kind of 11% so this kind of transform matrix because we are doing a bunch of matrix multiplies here well we're doing one measures multiply to be fair but that's kind of the biggest thing I think that's going on here so this stuff isn't really that bad we have a for loop here that goes through all of this and that's not too bad I think but yeah this kind of transform is is a little bit of a problem and that's where I said I think last episode that what you could do is since we're not drawing rotated quads at all we don't have to do a matrix transform model like we don't have to do a matrix multiplication here and create a bunch of matrices like it's a little bit of a waste because or if we if all we want is position and scale we don't need matrices at all we can just bake that data immediately into the vertices for our for our quad without actually ever touching matrices we can just you know multiply or add to workout scales and position offsets and all of that stuff so switching to that would probably make us faster this is just thoughts I'm giving out I'm not going to go into this and attempt to make it faster I don't think it's really that necessary like part of me wants true the other part of me thinks that like maybe when we actually get to the point where this is an issue where someone actually makes a game and it you know it really does need an optimization they can either optimize it themselves or I can think about actually getting it into hazel and I might play around with this but just in general I'm still happy with the code I'm still happy with the performance the way it is it would be better it would be really good to get better debug performance and I might look into that maybe we can compile well the GLM related code in like some kind of optimized format even in debug mode because it's not like we need to debug that so we might enable like you know vectorization as I say that stuff for debug that would probably speed stuff up as well as like you know inline functions and all that stuff but anyway that's pretty much all I wanted to talk about today let me know what you guys thought of this episode it was a little something a little bit different looking at what you guys have made in the community is always so much fun to me and it makes me so happy to see people using hazel we are going to like with all this in mind I think we can proceed with whatever we're going to proceed with whenever I look at stuff that the people have made as well maybe it's worth me actually diving into the codebase and taking a look at how this person has used hazel to make this I think I will do that if you guys want to see that in a video let me know in the comments below and I'll make a video just to meet looking at this card and in fact I'm not even gonna look at the card until I see whether in fact I'll make a poll there'll be a poll up here do you guys want to see a video of me going through this card and taking a look at it or would you rather just continue on and hurry up and get to the next episode of hazel let me know I'll leave a poll up there make sure that you vote and if I do do that I think I won't even take a look at the code before that video so you guys can see me kind of my real thought process of looking at it for the first time and what my thoughts are on that anyway thank you guys so much for watching patron or come for such tutoring the best way to help support this series and my channel and everything that I do here live streams are the best thing I think I'm doing on patreon right now there's one tomorrow morning so in fact I've been streaming pretty much every Friday morning I finally gotten better internet at home so now my upload speed is like fantastic and I can stream in like really good quality so I'm super excited about that the first stream with the new Internet is happening tomorrow morning as I mentioned so check that out if you're interested you'll see me working on a hazel dev and I think I'm going to be implementing well fairly soon I'm going to be implementing in ECS I'm gonna be in implementing Lua I'm going to be improving the graphics with like ambient occlusion and all of that fun stuff and doing shadow mapping soon if you're interested in that then definitely head on over to patreon help support the series you'll get access to the source code of what I'm writing as well as all of these live streams so I think it's a really cool way to help support the community and help support me and everything I love you guys and I will see you next time goodbye [Music]