The Visibility Problem - Computerphile

Key Takeaways

we've gone through all of this complicated process of bringing our representation of the 3D objects down into two dimensions and then deciding which pixels belong to which triangle or which pixels are inside a triangle and which aren't which tells us which pixels we need to light up on the screen uh but there's a problem um which is called the visibility problem we want to render two triangles to say um triangle a and triangle B A and B so using this whole system that we've got before let's say oh let's say in 3D space that triangle a is in front of triangle B so we go through our process we transform the vertices into World space then view space we project down we decide which pixels are going to be green um and we render them uh into the frame buffer onto the screen at the end then we do the same for b um we go through the whole process we collapse it down to 2D we decide which pixels are um belonging to B and then we write them to the screen but the problem is because B was drawn after a it overrides the pixels that were already in the buffer on the screen so this is a problem how do we go about making sure that our representation of the scene is Faithfully recorded when we convert from the representation of triangles to pixels how do we know which pixels are in front of others the original and uh simplest solution to this is just to make sure that we draw our triangles in the correct order which is called the painterly algorithm um because you do it like a painter would you draw the background first so you render B first and then you render a over the top of it and that way the pixels that we end up with represents the correct depth in the scene we have the green pixels overriding the ones that we drew originally the problem is this isn't very robust if these two intersected or if you had three triangles that had more complex overlapping a is in front of B but C is in front of a but behind B somehow then you can't necessarily order them correctly more generally what the problem is is that if you're ordering them on say a per TR Tri angle or a per object basis that's not quite enough to say which pixel is in front of which pixel because the operation needs to happen at the pixel level so the solution is to record the depth information in another buffer so first of all when we render a first remember a is in front of B when we render a we record at every pixel the depth information for this triangle then when we come to render B before we write the coln of B into the frame buffer what we do is we do a test we say is the depth at this pixel closer or further away than the existing depth in the buffer if it's closer to the eye then we can safely overwrite because we know that b is meant to be in front of a in that case um if it's further away we know that in that specific case for that pixel B is meant to be further away and we just throw away the results so let's say all of these are at one unit out and then we want to draw another triangle on top what happens then is that the depth values for the second triangle get sent into this buffer and here we do a test we say is the depth at this pixel when we're rendering this triangle is it closer or further away than the depth that's already in the buffer so let's say these are all at depth um 0.5 so they'll all be closer and in fact then what we would do is override this pixel that belongs to that triangle that way we can do per pixel depth ordered rendering um without worrying about having to order the geometry um and about worrying about not have to worry about special cases um and that's the solution that is um used ubiquitously now in any guide of 3D rendering not necessarily just because it's useful for solving this problem um because the depth BU as we'll find out uh actually has a lot of other uses because when you finish drawing your scene what you have um is for every pixel uh a depth value that tells you exactly how far away it is from the eye from the point of view of the viewer so what you have is a CO representation of the scene which can be used for all kinds of interesting purposes for post-processing effects and for generating Shadows what happens if the scene's got so windows in it or something right yes that's a complicated problem this is the area where Zed buffering can be problematic because if you want to draw a window or a trans transparent or translucent object and you write its depth into the depth buffer and then you draw another object that's open op for example that needs to be behind the transparent object it will fail the depth test because it's behind the transparent object so its depth values um are further away than the windows so the solution is simply to draw everything opaque first and then afterwards you turn off the depth buffering or you don't write to the depth buffer and you draw your translucent or trans transparent materials um over the top and do blending with what exists already in the frame buffer because that way you won't have problems whereby uh you look through a window and everything that's behind it isn't there or has disappeared because it's failing the the depth test the problem of course with that is that because you're not using depth testing to draw translucent or transparent objects you start to run into the problem of well if you're looking through with window a at window B you need to draw them in the right order again and I think for that um situation uh the only real solution is just to fall back to the painterly algorithm again sort all of the transparent objects back to front draw B first and then a um and in general it works because the majority of scenes are composed mostly of opaque objects that are amable to zed testing and only occasionally will you have a layer of transparency or window or a um cloud of fog or something like that that you need to look through and have that rendered using the paint of the algorithm very rarely will you have the situations that break down where two windows of different colors intersect and things like that but there of course there are ways of handling that special case um if you really really need to do that kind of Rend but in most circumstances um the paintly algorithm works fine for that now what we've done is taken that simple geometric representation and discretized it in general we prefer to download in pieces because it lets you adapt it means that you can decide that things are great and give people a high quality video or you can decide that things are not going so well and you could downgrade and try to make the playback smoother