How Path Tracing Makes Computer Graphics Look Awesome - Computerphile

Key Takeaways

last time it was recursive so you You' hinted there might be something something better oh yes so have you ever seen a video game trainer and g wow this looks absolutely groundbreaking Next Level I can't wait to play this and then you play it and it looks something like that really not very good the reason for that is because they pre-render these trailers so they when they they don't care how long it's going to take to render they'll shoot out millions and millions of rays and use the best um Ray tracing or path tracing algorithms to render these really Crisp Images so when it comes to real-time rendering they they can't afford that and you end up with a result which looks not not very good so one of the ways that they can simulate really good light is with path tracing and that's what I want to go over today to really understand path tracing we have to go back and quickly recap what we've what we've done so in previous videos we spoke about Ray tracing which involves shooting Rays into your scene tracking where they go and we compared it to rest ization and showed why that's better and then we built upon that with recursive Ray tracing which is able to do Reflections and refractions and how that works is basically Rays calling themselves recursively until they terminate um now that's all well and good but those all handle things to do with direct light now one of the questions um that I got on the previous video was to do with the shadow in the Cornell box scene and one of the questions was well why is there something something rendered there it should just be black right and what I didn't go over was that you have something called an ambient value which is basically a way of going ah um there is some light hitting it from somewhere I'm not going to calculate it I'm just going to say 10% of the light is just hitting it from somewhere right and in the first video we went outside and it was a beautiful day it was very unusual actually it was actually really nice weather while this is more like the British weather it's clouded over it's been raining it's horrible but that's what we're used to right now if we were going to render outside um using um raay tracing the standard way that I originally said we'd hit uh shoot let's say shoot a ray off it hits the floor right let's check if um what the light is ah the light's uded cuz it's in the clouds therefore don't render any direct light therefore everything outside would be one shade of light let's say 10 ambient um Valu to 10% therefore everything would just gray I mean quite realistic actually for the UK but in real in reality um that's not the case what's happening is that um the light is coming from our sun it's bouncing around in the clouds it's and then it's coming down hitting different objects and then hitting our ey right we can see in in the actual clouds there are some parts some um parts of the clouds that are more lit up than others cuz they're less dense so more light is coming through you probably won't be able to see this on the actual camera but but just you know you you'll know what I'm talking about right um so that's a form of indirect light okay um but normal Ray tracing doesn't account for that you know it just goes right some light will hit it from somewhere uh but that really does hurt the realism of of some of the images if you're trying to render something that looks real really really crisp um it it does affect it so I want to um draw the Cornell box scene which is um a scene that you you create and render and what you can do is compare other Ray tracers and rasterizes and path tracers um to each other because you're rendering the same scene what I showed in the previous video was that we had um a light source up here and there was a shadow here at the bottom of this square and the way it works is because we shoot a ray out from the camera it hit let's say this point here go right let's calculate the direct light we shoot array off here oh this this square is in the way therefore don't render any direct light indirect light is just a default ambient value of let's say 10% therefore render 10% light on this um floor which is not Black by the way it's white but I I can't draw white on here so draw as black but yeah now the issue is that um it doesn't look it doesn't look super realistic right because Shadows don't work like that because there are some indirect light coming from um anywhere in your scene I want to show you what what some indirect light may look like so this is is your light source there may be some light bouncing off there and hitting there there may be some light bouncing off like that there may be some more up here or this can happen they may be like going like this Hitting off like this Hitting off like this and hitting off like that it's coming from everywhere how do we simulate that well what we do is at the points that you're trying to um render you shoot off Rays um out into your scene that simulate the indirect light so let me read draw this scene here so I've redrawn this and let's say we want to calculate the indirect light here so what you do is you're trying to simulate light coming from everywhere so coming from here coming from here coming from here coming from here an infinite amount of light you can't do that it it just you can't shoot off infinite amounts of rays going everywhere so what you do is you sample from those Rays so you can imagine that you have Rays coming in from all around um here let me just I'll draw it they could be Rays coming in from all around here from the he this this hemisphere here so what you do is you let's say pick a certain amount of rays um that can come in so let's say we pick five so we'll pick five Rays we'll say oh yeah let's just shoot off one here let's just shoot off one here let's just shoot off one here one here and one there right so these are almost like feelers we're sending these out I'm feeling what the indirect light should look like so we shut off those Rays here what do we do now well we just render what the color should be right as we normally would so here we go right this color is red and um there is nothing in the way so we render it as a as a bright red and we do the same for this one here and here we've got ah yeah one green render that as a a green color there ah this one these ones are blue but they're occluded so therefore you can't render any direct light here um but there is some indirect light coming in I'll go into that in a sec but let's just say it's a dark blue okay imagine this is a dark blue so right brilliant we sent those Rays off and we've got the calculations that we we need so what what is the indirect light so what what we do is we add all of these together so we have red red blue blue and green and you add them all together and you divide by the number this is Monte Carlo where you're sampling from the amount of potential light coming in so you add these all together divide by the number and you get the average amount of indirect light now what I mentioned was that there's also indirect light coming into these parts of the scene right so what you do here and this depends on what you set your depth to I.E the am amount of bounces um that you can do so at the moment it's a depth of one let's say you set it to two for these ones you shoot off five more like this and you calculate the indirect light for this color right and you do that again for this one and you do that again for this one and you do that again for this one and you do that again for this one and let's say you have a depth for three you also got to calculate the indirect like for the the the Rays that you've just shot off there so you calculate for this one you calculate for this one you C presumably each of those Rays because there's so many more of them is having less and less of an yeah that is true yeah so what you get is loads and loads of rays being shot off right but the more rays that you shoot off um the less those uh overall impacts of of each Ray has and when you get to let's say a depth of five those Rays aren't doing that much the contribution is is less and less and you got to remember this is the indirect light for the indirect light for the indirect light for the eventually but it it's not going to um impact too much but it it is worth doing because it's that just that nice shade so you can get you know um so yes to summarize it involves calculating indirect light by shooting Rays out seeing what color they are and then shooting more Rays out for the indirect light for those Rays shooting more Rays out for and so you end up getting this BR so it's like recursive Ray tracing on steroids it is a lot of rays which is why this isn't really used for realtime rendering because it's so intensive on your GPU now all well and good me saying that but but what's the point really does it actually look any better I'm going to give you an example here of some of a situation where yes it really is uh useful for doing this so imagine here we've got a scene of a corridor okay and you got a camera this can be your eye in this this scenario here and then we got a a light source there so if you looking down a corridor like this what would happen is that you'd see the end of the corridor here would be light CU it's lit up by this lamp or whatever but it doesn't just stop being light there it gets dimmer and dimmer and dimmer and dimmer and dimmer the uh closer you get to um away from it which makes sense right because less light is hitting your eye if you were going to rasterize or or raycast this um what would happen is you get a very har Shadow like that and why would that happen cuz you shoot a ray out here let's say hit say calculate a direct light brilliant it's it's lit up but if you shoot a r out here I'll do that here if you shoot a r out here and calculate a direct light oh it's uded by this part here just set the ambient value value as 10% and the issue is that if you do that for you can do that for all of these parts of the corridor and they're all occluded but there's no direct light hitting them so they're all the same shade and you get that really harsh Shadow and the lighting isn't realistic right um and this is yeah this is if you're going to rasterize it or basic Ray cast or basic Ray tracing algorithms right so what happens if we're going to path trace this so you shoot a ray out calculate the direct light as normal no direct light okay what about the indirect light so what you do is we'll shoot out let's say five Rays five samples so you shoot out array here and and for context let's say these walls are white okay you shoot out array here shoot out array here shoot out array here shoot out array here and shoot out array here right we just pick them at random right and what you get is you go for each Ray go okay is this um in a direct light oh yes it is so let's say this is white oh yeah that's in the light as well oh yeah that's in the light as well oh oh dear these are not in the light and this is with a depth of one so we're only shooting off one Ray here these rays are not shooting off more Rays yet but what you get is you get an overall light of white or or lit up lit up lit up not lit up plus run out space not lit up divide that by five and what you get is 60% lit up indirect light and let's say the indirect light makes up 30% of your thing I'm not going to do the calculation but that's how much light you you render there for that for that part there I feel like I'm in Doom or Quake or something we like first person shoot him up oh yeah well that's yeah yeah yeah this is this is the sort of thing I'm trying you know I'd actually like to see a comparison I think there is actually online L um between path tracing games like that versus rasterizing and you can really see how far we've come right now what happens if we're going to render this point here all right so we shoot out a ray as normal go okay is there any direct light no because it's uded by this part of the wall so what you do is you then shoot out five Rays right ah okay let's add these together so one check yeah check calculate your lighting for this one oh yes this is all good calculate your lighting for that oh just click the corner no that's not in the light that one's not in the light that one's not in the light that one's not in the light and so when you go to calculate the amount of light it'll be lit up plus not lit up plus not lit up plus not lit up plus not lit up divided by five equals 20% indirect light add that with your direct light there is not a lot of light hitting there but that's fine right because that's that's how it would work in real life because less light further down the corridor you get less light is going to hit those positions from that light source there and that's how you render those um it's not just a har Shadow there it it's a gradient of the amount of light hitting there that's what path tracing is trying to achieve Global illumination which is that light should be coming from everywhere hitting everything but more light hits C certain parts of the scene than others and it's all reflected and bounced around until you and that's how you get those really nice um images Now quickly before we move on here this is with depth of one what would happen is if you set it as depth of three that would shoot off five more to calculate indirect light of that uh this one would shoot out five more this one would shoot out five more and then those ones that you shot off would shoot out five more we shoot out five more even positions like here which you've already calculated are in the light and have direct light you still calculate the indirect light for these ones as well as well as the ones down here and in in proper scenes like the Cornell box for the light it isn't just black and white light it is green light blue light red light and that's where you get um shades of of light if let's say we say the walls here are red down here what you get is you'll get red tints up here because there's a lot of red light from the walls here and you get really nice shading effects on there as well all right one thing I also want to um talk about how does it choose where the Rays go right so let's say you have a surface here and you want to render the uh indirect light for this point here so what you what really light is coming in anywhere from here okay so what you do is just randomly choose Rays along this hemisphere so you can shoot one up here one here and one here rays that are closer um to the surface here and further away from your normal will have have less contribution to the overall indirect light that is part of it because if you have um indirect light coming um from that angle here it's going to be less than if it's all the way up here kind of less important less prominent right yeah yeah yeah so that's another part of it that that you've got to consider I've shown you why you use path tracing to render scenes like this now should we actually have a look at a path Tracer let's do that yeah I'm showing you right now um a comparison between open gel which is a rasterizer and a ray Tracer here that that made now this is um a framework that was built by University of leads I've gone through um this in the previous video but now I want to show you what happens if we upgrade this Ray Tracer to a path Tracer right so I'm going to give it a sample of one which means that for the indirect light we only shoot one Ray off and let's say it's a think it's a depth of let's say three you shoot one Ray off um and see what happens there I can tell you now it takes a little longer um because you're having to calculate that uh extra Ray bounces from the indirect light right it's finally rendered o that doesn't look very nice ises it because we're only shooting off one Ray here for the indirect light which means that it's not a very good estimation of what the indirect light really is okay so let's pick it's probably difficult to to see but let's say we pick some some on the floor there there's a lot of green and red scattered on the floor let's say we render this floor shoot a ray out hits the floor um Cal the direct light it is direct light um but let's um calculate indirect light and what it would do let's say for the red one here it will go right pick a random Ray Direction all right it's over here shoot a ray off oh it hits that red red wall there therefore the indirect light is red that's not true though because there's light coming from the red wall light coming from the green wall lights coming from the white light coming from the blue it should be a mix of all of those things not just one same with the green part here same with some of the blue colors on the wall there it's it should be um a mix of all of those colors because light is coming in from everywhere not just one thing so what you need to do is up the number of samples so let's do five this could to take forever it's finally calculated and O it's looking better now still a bit grainy still a bit of noise but in general it's looking a lot nicer and what we're starting to notice is that along here on this side of um the square there's a lot of green and the reason for that is because there is a lot of indirect light coming from this um green wall and we're simulating that because we shoot Ray that and a lot of them are going over there in the green wall right because we're now doing five Rays there is a higher probability that some of those rays are hitting that green wall right this is why we have more samples and these rays are scattering out and more of them are hitting the green wall you can see there's still some you know strange noise of colors going out because let's say five of them don't actually hit the the green wall when they're going some crazy direction right however if we then increase the sample to let's say 30 that should iron out most of the noise and we should get an actual real good indication of the amount of indirect light here's one that I rendered previously cuz I'm not waiting 3 hours for this to render um as I say this is my path Tracer that is not the most efficient um but it does an all right job at showing you um why we use path tracing there is actually proper green shading here and red shading over here and and you can if we compare we've got the rasterizer on the left and on the right we've got the path tracing you can really see why rasterization just doesn't cut the mustard compared to uh path tracing because the light is so uniform because we're not calculating any indirect light we're using those ambient values that I spoke about I've said how good path tracing can be but I've only showed you my path Tracer which in the grand schem F isn't actually that good I want to show you what an actual really good path Tracer is and and and show you some of the things um it can do right so I'm on a um this is an article by um demo Fox and they have a great little image here of a path Tracer and and they've highlighted what is good about it and what what path tracing does really well so first thing ambient occlusion which is where if you're in a corner there is less indirect light being able to hit the corner right because there's this this opportunity for light to to come in and you can see here it's darker this area here is darker than let's say that wall because less less light is being is bounced around and being able to hit that part of the scene color bleed and that's what you saw on my path Tracer there which is that light from the red wall a lot of that is bouncing and hitting that object and you can see there's not much red here because it's not less there's less probability that light is coming from the red wall and hitting there there's a high probability that um indirect light is coming from there and it there and I want to um preface this by saying there are probably a lot of samples that have been used here a load of samples and this is going back to the thing I said about the the video games at the start with the trailers if they want to render stuff like this they can do that because this may take an hour to render um it doesn't matter because you're pre-rendering it all but if you're trying to play a game real time there's no you can't really use path uh really intense path tracing because your GPU just can't handle it maybe 10 years down the line when gpus get even better maybe we'll start seeing proper path tracing um used and and honestly it probably is being used at some point I just don't know about it because how quickly this stuff moves so if you want to render a scene you re you've got to decide between the three methods you can um rasterize which is very quick but can't handle complex lighting effects very easily um Ray Trace which is a bit of a middle ground it's it's it's slower than rasterization um can create cool detail but may struggle with really crisp lighting effects or you can use path tracing which is a lot slower but wow does it look good they're trying to say how much each Ray contributes to the the the final light color now imagine you're at the beach okay and you're looking at the ocean if you're looking horizontally at the sea you won't be able to see into the the ocean but if you then walk into the ocean and you look down