Introduction to Rendering | Game Engine series

Key Takeaways

hey what's up guys my name is and welcome back to my game engine Series so today we got something a little bit different first of all I want you guys to watch the previous episode if you haven't already that's us finishing up all of the IM goey stuff that I wanted to actually do and now that we've done that we're going to move on to a very big new topic called rendering now first of all let's talk about what's different in this episode my computer has just completely broken itself like just will not work at all which is all great and I could probably fix it if I had enough time but unfortunately I don't have time because at the moment I'm working full-time of course at EA on frostbite I'm also renovating my apartment I'm also getting married in about a month so I have to do a lot of wedding preparation and my life is just completely hectic but I still really wanted to make a video for you guys today so I thought you know what let's just talk about stuff today because this whole rendering topic that we're about to dive into is so incredibly complicated and just just enormous that you just really cannot plan enough and if I think about how many times I've written a renderer before and how many times I've written rendering API abstractions before doesn't matter how many times I do this I'm going to be 50 years old and I'm still going to be writing them a different way because it's just you just it's just incredible like it's just crazy how just complicated and how much stuff there is to actually know and I still don't really know what I'm doing so because my computer's broken and I can't really do anything on it i' I'm gone gone to pen and paper and we're actually going to talk about a lot of what we're going to do in the coming weeks because what we're going to do now in Hazel is write a renderer basically okay so what does that mean we're going to talk today we're going to talk about what that means exactly next week what I want to do is actually put together probably like a PowerPoint presentation with like diagrams and like the structure of the API and everything we're doing in more of a formal kind of format um this video right now is actually going live for everyone at the same time so patreons aren't getting this video early just because of the fact that I've had no computer and I'm probably going to have to edit this on my laptop which is going to be terrible anyway that aside everyone's getting this at the same time which is cool because I can get your feedback immediately and that's going to be quite good because next week what I want to actually do is record a video where I actually talk about the design of this whole thing right and that's going to be like with a PowerPoint kind of presentation cuz there's no way I can just talk myself into this it's just going to be too much in and it's going to be very hard to kind of visualize if it's not drawn like like as a as an actual diagram for you guys so that's what's going to happen next week but today we're just going to talk about this I'm just going to give you my experience with just rendering in general and talk about what this means for Hazel and like just we're just going to talk okay so step one um and the first thing that I have on my list here is just that it's a really hard and complex topic um we should probably Define what the topic actually is so rendering what does rendering involved why do we need it in a game engine you know what what is its purpose why is it in Hazel what does it do okay so rendering refers to basically our ability to draw graphics on the screen now what that means usually is well hey I have like this 3D model I want to draw I have this world I want to draw I have this whole scene that I've built up I want to render this in a in a realtime fashion that is you know fast enough to actually be real time so probably 60 frames per second which is 16.7 milliseconds per frame and I want it to be interactive which means that there has to be fairly low latency between input and actual you know it doing something and affecting the rendering in some way that is essentially what rendering is um now why is it so complicated well the way that we achieve rendering that is fast and that is capable of producing High Fidelity Graphics which again is some I did that see I'm not professional at all the way that we produce something that that is high fidelity right which is again a term that like to each their own right like what is high resolution what's what's High Fidelity Graphics to you right a lot of what you see today is maybe some of the some of the like new games that are being released by huge companies you know you see like I might even play some of them here I don't know if I can copyright wise but you know stuff like um the new rate tracing stuff in Battlefield 5 you know you've got all of these kind of High Fidelity Graphics that are coming out um and we perceive them as High Fidelity usually because they're close to being photorealistic which means that that rendering technology is actually capable of somewhat producing something that is that looks like a movie or a photograph in real time right and people compare it to how the real world works a lot and you have these things like physically based rendering which is actually rendering based on physical properties of surfaces and like how light works and photons and all that kind of stuff and that's kind of our Pursuit our like us actually pursuing a way to render photorealistic Graphics in real time because we can we can more or less render photorealistic Graphics right but doing it in real time is really really challenging and there's like a huge like huge huge like thing going on with with people obviously trying to achieve that in real time but that's still kind of not really quite there yet um and certainly this kind of R Racing uh Hardware that's coming out in the form of like RTX for example on on Nvidia that stuff is definitely like stepping into the right direction because everyone knows that Ray tracing is how you how a lot of offline renderers so non-real time renderers so renderers that people use for like movies and stuff like that how they're able to achieve fer realism like quote unquote right is by using R tracing kind of renderers but anyway getting ahead of myself here um it's very complicated because apart from doing all of that the way that we actually achieve that high performance or performance that is at the level where it's it's possible for us to do this stuff in real time and still achieve a high fidelity look um we need to use something called the GPU which is the graphics Processing Unit it's a dedicated piece of Hardware that resides in most computers right sometimes it's integrated into our CPU um there are various architectures for how this Hardware is actually set up but ultimately it is a dedicated chip that is on our computer or our whatever device we're using that is capable of Performing or executing certain certain mathematical operations or algorithms that are very common for common like rendering tasks essentially or common Graphics processing tasks so a lot of what we have to do with Graphics is display a large number of pixels on the screen if we have a full HD display that's about 2 million pixels we need a way to determine the color for each pixel usually this is independent of every other pixel so it's possible to do that stuff in parallel right that is something that a GPU is usually quite good at okay other things like you know we deal a lot with four component vectors we deal a lot with 4x4 matrices we have hardware optimizations for operations like that on the GPU which is why it's faster than than doing these kinds of things on the CPU so parallel processing and then a lot of specific math operations is the gist of it obviously we have specific like R tracing processes and other kind of architectures that may have a lot of things but that's kind of just to dumb it down and to keep it basic it's because of that it's because the math that we kind of want to use for rendering Graphics is faster to do on the GPU because of dedicated hardware for that and then also we have a large like huge um parallelization essentially that lets us do a lot of operations in um just at the same time essentially like in parallel okay so um to interface with this GPU we have to use something called a rendering API now I do have an openg series which you can check out but I do realize as time goes on opengl is becoming less and less a thing and I mean look me personally like if you ask me personally what is Yan chernikov what is the Cho going to use in his High Performance Engine is it going to be openg GL of course not openg GL is not something that I can recommend using if you're going for bleeding edge cutting like Edge I don't know I want to make the best game engine the best graphics rendering engine in 2019 of course not you don't want to open GL you probably if you're absolutely at the top Edge you want probably the vcan or directx12 right we are not a huge game Studio or a huge rendering Powerhouse that is capable of implementing a Vulcan or a direct X12 rander immediately in a way that can also teach it to all of you guys because Vulcan and direct X specifically looking at Vulcan though is extremely complicated those of you who have not taken a good look at Vulcan I like I am there is no one to me capable of explaining exactly how like the Vulcan swapchain actually works and how like open GL code right I can I can write you probably a physically based renderer from memory in openg G like all API commands everything just correct right I can do that because it's simple Vulcan I probably wouldn't be able to draw a triangle for you and Vulcan off the top of my head no way no I definitely wouldn't be able to right and I've worked with Vulcan since before it was released actually so I do have quite a bit of vcon experience and yet I'm not able to do that because it's incredibly complicated you like 1,500 lines of code or something to render a triangle so that's why I've specifically gone with openg as a rendering API in the beginning why is is it because it's the best choice obviously not but the thing is I can't spend 100 videos setting up a Vulcan swapchain to show you guys this because that's going to lose a lot of viewers and that's not going to be interesting okay and it's not something that we need why because i' much rather get hazel to the point where I can render like a PBR scene in 3D and then be like hey you know what maybe now we'll Implement Vulcan because we're getting to the point where it might be necessary so you have to look at it from this kind of point of view a lot of people ask me if the way that I'm building Hazel is how I would build my engine or if I'm making all the right decisions I'm making the right decisions considering it's a video series on YouTube I want to make that absolutely clear I actually quite like this video I'm just ranting a lot I'm venting and I'm just explaining how everything's working which is good cuz there's so many comments I've read and so many things I've wanted to say but I can't anyway so that being said that's why we've kind of chosen open jail because it allows us to tap into our graphics card and actually use it but it's the most simple API for doing that open shell is extremely simple it's like a a 10 out of 10 in terms of how high level of an abstraction it is right whereas Vulcan might be like a one direct X12 might be like a three direct X11 and like metal might be around a 56 whereas Vulcan is like you know on one extreme and openg GL is on the Other Extreme which is quite interesting as well to just look at okay so here's where it gets even more complicated though so okay you pick a rendering API which lets you use the graphics card which is what you want to do to perform rendering right that's our task in a way but we the problem is one One graphics API is not enough now openg is something that happens to at least in the past now now more so it's getting more deprecated and it's getting more or less kind of removed like channeled out whatever you want to call it so it's a little bit different now but in the past openl was extremely good in the sense that it just worked on everything it was almost like Java actually I might compare it to Java because Java worked asterisk everywhere but it was mostly garbage right I wouldn't say open G is like just absolute garbage or anything like that I actually quite like openg G in the sense that it's just so simple it's so nice and and warm and inviting to come back to openg after spending some time in other rendering apis and being like this is insane it's nice to come back to open jail in a way even though I still think the API is not amazing um it's still nice to come back to open G and not amazing I'm putting it lightly by the way but anyway coming back to open is always nice because it's so simple right it's incredibly simple um now the thing is though uh and I don't even remember where I was with this but the thing is when you have something really um when you go back to designing all these rendering apis and you think about what you actually need um it becomes a problem of what platforms essentially am I kind of shipping on right so to drag this back a little bit as I was just talking earlier before I completely forgot where I was going I was saying that openg back in the day was kind of just like Java in the sense that just it just worked everywhere right I mean Windows Mac Linux Android iOS they all ran they were all capable of running openg whereas now if you take a look at what's happening well like you know not saying Windows wasn't absolutely great with it um but like Android iOS you know Mac Linux they all kind of ran on it right but now if you look at the state of apis what is the best API to render on a given platform for Windows it's DirectX 11 or DirectX 12 for Mac it's metal and for iOS it's metal for Linux it's Vulcan for Android it's Vulcan maybe GL GES right GL but that's for probably older devices mostly Vulcan right did I miss anything um for Xbox it's direct X2 usually and for um PS4 which I'm missing out on uh is Sony's own proprietary API which I always forget the name of because I don't deal with too often um and then for like the Nintendo switch I think it might be Falcon as well to be honest but I'm not 100% sure anyway for those kind of platforms you can see that openg is non-existent right do I pick openg well no maybe on older Android devices it's pretty much maybe the only reason you pick openg right you can still render openg on Windows but is it preferred no you should be using direct X if you're serious about rendering on Windows it's as simple as that where I'm going with this is that you need to ENT you can't get away with using with using one rendering API and that's why rendering becomes really complicated well one of the reasons it becomes complicated because you need to do something you need to design an API abstraction which hides away all of the actual render API like calls behind your own abstraction so that ideally you basically or in this case Hazel has its own API it has its own rendering API and then that rendering API could be using either direct X2 could be using direct x 11 could be using vcan could be using gel could be using metal whatever right it could be using any rendering API it wants but you tell Hazel I want to upload a Vertex buffer full of data it does that how it does that using Wich API that's that's implementation details that we don't want to worry about we have this abstraction layer an API where we can generically tell Hazel upload the vertex buffer upload this texture data you know I want to render you know from this index buffer three indices and with this vertex buffer with this material with this Shader with this you know topology I don't know just a million different details right but the point is we tell Hazel we want that Hazel figures out okay well we're running on Windows and we're currently using directx11 because that's what we've selected therefore I'm going to use that or we're we're currently running on a Mac so I'm going to use metal I'm going to you kind of get the point right we have this abstraction layer so that our application code doesn't change based on what it's running on and it will be using the optimal rendering API on the platform and configuration that we've actually set up that's extremely difficult to do okay that's extremely difficult to do because what you need to do is draw a line somewhere and say this is what my API looks like that's really difficult because the more you go on with graphics and the further you get and the more Advanced Techniques you start implementing you may need more control dragging the whole API line is a huge and expensive task to do in case you guys aren't clear about what I mean by API line it's it's where you draw the abstraction so in other words where really high level might be draw a triangle at this location and really low level might be like unlock my Vulcan semaphore right that's kind of maybe the two extremes or like you know like un releases memory fence or whatever like it it where you draw the line is really important right now usually what I tend to do is I like to think of it conceptually so things like did I just do that again twice in one video man anyway so we do it we kind of draw the line at a point where like I might want to say I want a Vertex buffer I want an index buffer I want a texture I want to Shader I want to you know what what API exists on the Shader well I want to find out information about my uniforms my attributes I want to be able to set up uniform buffers I want to be able to kind of upload sections of uniform buffers I might want to retrieve information about attributes I might want to retrieve certain other information right there's a lot of things that you might want to do I I might want to get all the resources attached to that Shader such as different textures and like Samplers maybe like how many textures how many how many outputs does this Shader have do I have to hook them up to frame buffers you you have to you have to kind of have an API that is flexible enough that allows you to implement all these rendering techniques but also abstracts away all of these apis but here's the other problem these apis do not work the same way right these apis do not work the same way openg is completely different to Vulcan right I mean yes conceptually you can abstract it out and say you know okay how do I render a triangle conceptually what you could say is okay well what do I need for that I need to first of all set up some kind of context right in Vulcan you'd have to set up like a crazy amount of things such as like command cues you know rendering devices like logical devices physical devices um what what is what else is there cues instances I don't even remember everything that I can't even list all things you need to set up in vcon there's a lot of stuff you need to set up whereas open jail might just be like okay fine and frame buff is obviously in vcon as well like you need to get all the swapchain you need to set up your swapchain and and set up frame buffers frame buffers for each buffer in the swapchain lots of stuff you need to do but essentially it comes down to I set up my context and my rendering frame buffer right I upload a Vertex buffer with vertices I upload an index buffer with indices I upload a Shader um I set my viewport maybe and other kind of rendering context things and then I issue a draw call that's probably about it right that's what I do and and like obviously that process in Vulcan is going to require a lot more code but that might be your kind of that might be the steps that you need to actually render a triangle on the screen so with that in mind you kind of have enough to go on where you can say I'm I can make this work in both Vulcan and open Gil because I know what I need I know my steps and they're just going to look different but essentially I've drawn that line at I have a Vertex buffer class that's where my abstraction ends right so inside that vertex buffer class I'll have a different different implementation of that vertex buffer class which applies to Vulcan and GL for example right and then the API that will be exposed are things like I want to create a Vertex buffer it should be this many bytes right it should be this kind of usage so like whether it's kind of dynamic or static things like that I want to be able to upload a section I want to be able to maybe map the buffer or upload a section of data to a particular offset in the buffer so I want that API available I want to be able to see the size of the buffer I want to be able to set the layout of the buffer so that we know how to actually render with it and what the data inside there looks like those kinds of things right I want to bind the buffer I want to lots of stuff to do right but the idea is you've created an API so that's really the first thing that we need to kind of do we need to create a we need to create a rendering API now there is no way I'm going to sit down here and write you everything every single class and every single detail right now because that'll be extreme ex lengthy and you know that's like designing a whole specification we're going to do it as we go along but as I said next time when I have like little PowerPoint presentation for you guys um that's where and I'm going to take in probably some suggestions for the com from the comments as well from people who are experienced in this um just because like I look I've been doing this for years and years and years and years and professionally as well obviously and I'm I'm more than open to ideas because this kind of stuff you know sometimes I'll learn something new suddenly I'll be like wo wo I I want to revisit this whole thing and rewrite it from scratch now that I have this piece of information right but anyway so we designed an API we need to come up with an API essentially where we where we draw the line of abstraction so that we actually know how kind of low level that'll be usually what ends up happening by the way is that you have like kind of a um you have your render API abstraction layer and then on top of that you have a renderer so you don't like the renderer does not directly interface with any rendering API but you still kind of have that right so the renderer layer is where you probably have things like your scene graph you know any kind of batching logic coloring logic that kind of stuff that's at like that renderer level right um and like render sorting and all of that kind of stuff right whereas you know the render API doesn't deal with it the render API is things like upload a Vertex buffer upload a texture it's like render commands essentially but you've abstracted it out so that you can switch out the rendering API to whatever you want okay okay so that's the API that we need to design we're not going to talk about the renderer just yet we're just going to talk about um that rendering API abstraction that's one thing we need to do and we'll talk about that next week um you know I've got a lot of other things here as well um you know there are things like command cues and multi-threading and I don't mean renderer multi-threading I mean like game engine multi-threading so it's very common for example to actually submit commands from your game thread or from your application thread but all that is is command submission so you say I want to upload this texture or please create a Vertex buffer for me on the GPU right what what happens with that code is it gets queed it gets put into something called a command queue or like a render command que and then what happens is the render thread when that starts up it goes through the list of commands and executes them okay a lot of apis actually do stuff like this like vulon and DirectX 12 for example have that stuff in them but like GL doesn't so usually it's common nowadays to actually design that stuff yourself will'll have to do that right probably and I'll show you guys how to do that cuz it's quite I don't know I it's quite cool I quite like it even though it is sometimes tedious but essentially you have like a command encoder that goes through and like it copies all the data you need for a command plus the command itself plus maybe like a function pointer and then you you store all of that in like a binary buffer that's all continuous memory so that it's all kind of C friendly and then you kind of the render thread goes through that and like it's almost like a CPU you reading CPU instructions where you go through like different op codes and you say okay I'm doing this command now the the you know the next 12 bytes are parameters in the form of three floats so transmit them like that and it's quite it's actually quite interesting um and it's very you can make it very efficient which is really cool um so we'll have to cover stuff like that um and we'll talk about some of the negatives of that as well as the positives there's a lot to do with rendering I just want you guys to understand that I'm kind of overwhelmed um I want to design this as much as possible um I always knew this would happen in a way but like I just it's almost annoying that this rendering that we're up to rendering now because as I said I'm getting married next month and I'm going to go on a honeymoon for a little bit of time so I'll be away and there won't be videos here and obviously I don't want to be worrying about rendering architecture in Hazel um while I'm supposed to be getting married so because of that um I'm just like asking you guys to just take it easy on me for the next month or two probably two months like in May I should be back in Action 100% but like for now I'm just I'm going to still try and release a video a week but if it's like not as saturated or not as good as you kind of not as much we don't get as much done as you expect to um it's because I just kind of want to talk this through I want to make sure we have a solid design because this is probably one of the most complicated parts of Hazel um it's it's definitely the biggest system of Hazel uh and I definitely want to just make sure that um we get this kind of right but also that you guys understand all the implications with this because this is a this is an educational series I'm not just writing Hazel for myself we're building Hazel together and I want you guys to understand how everything works anyway I think that's about it um next time what we're actually going to do is talk about that rendering API abstraction I'm going to uh see what you guys see in the comments but also I'm going to get that PowerPoint presentation ready where we actually um talk about how we are going to abstract that and what it's going to turn into and how how everything's going to work the whole architecture of the renderer is really what we're going to talk about um uh next next week and and not really the architecture of the renderer so much as the architecture of the render API abstraction or the rendering API Graphics API abstraction whatever you want to call it okay so that's next week hope you guys enjoyed this video and I need to fix my computer before next week as well so this is not going to be a fun week anyway and we've got people from like frostbite over in the EA Studio this week as well there's so much stuff happening I hope you guys enjoyed this video if you did please hit that like button you can help support Me On patreon by patreon.com the chel I need you guys support more than ever right now cuz I'm just like just I'm thinking of even taking like unpaid leave from work because it's just everything's just overwhelming so um I would you know I really want to keep making these videos for you guys um um because I love it right like even though there are potentially more important things for me to be doing right now I really love making these videos for you guys um like you mean a lot to me and I really do want to write this renderer for hazel it's going to be so cool and working with you guys as well on that it's going to be amazing so if you can please support me um and you'll get access to a lot of this source code I've already written like a render command queue you know that runs like on a different thread I don't think it runs on a different thread yet but it is a full kind of command process and command Q um the you know the where you can submit commands um I've already written all that stuff and it works with openl and we are rendering more than just a triangle um all that stuff is in the Hazel development repository um already ready to go so if you want to kind of jump ahead and see kind of how this is going to look um definitely check that out and I think that I am um because that is open G I'm not sure exactly where but I am already beginning that abstraction so I'm playing around with things like that um so definitely check that out and then usually you'll get videos a week early I can't promise for the next like 2 months basically until May that you will get videos early every week I'll try but like I mean with this week I kind of couldn't do that cuz my computer maybe that'll be better but um again your support means um a lot to me and this series just I'm telling you like straight up would not be here without all of those wonderful supporters so thank you so much hope you guys enjoy this video I will see you next time goodbye w [Music]