This AI Reconstruct Real Life Objects From Just Images [NeRF]

Key Takeaways

in the previous video i've made about 3d impainting it uses an ai and painting technique to generate new pixels or colors that did not exist and also ai learned depth awareness to separate the foreground in the background and the results are stunning considering only one image was taken as an input too so for today let me introduce you to neural radiant fields short for yes nerf and and that is trained to reconstruct a scene and render new views unlike 3d impainting what it basically does is not only the images are taken as an input but also the angle of the image the distance to the object and the position of the view and it gets all these data not by you inputting it but by the images that you input it uses this information alongside multiple other images with different inputs to generate a new angle that estimates how the scene would look like but the obvious downside is that it requires a set of images for nerf to construct a scene that works similarly to 3d impainting and a lot of time like way more so it is more difficult to make this ai run on a single artwork or just a single image of a scene like what i've done in the previous video on 3d impainting along with the paper the author also released video demos as you can see here it is really impressive this ai not only can reproject the lights but also create a smooth 3d image of the object or scene that you pictured what makes me say this is amazing is that even though we need 20 to 30 images it can construct a video which makes the scene or object looks like it has a sense of depth and structure they have also compared to other similar papers before and it shows a significant difference too but as i train my own scenes and objects it seems that my results cannot match up to the quality that was shown first i tested on my microphone which consists of very dynamic 3d structure and a mesh-like design on the top and this is to see the accuracy of the 3d reconstruction of the images and the results are exactly on point it also even came with a depth map video that i can view but the quality just doesn't seem right maybe it is the lack of iterations for my images but i doubt it because since 100k iterations there are less and less differences in the clarity of the scene to go even further on testing the depth awareness in the mapping i tested on some trees which have really complex and difficult structures that i predict that the ai would struggle to pick them up but apparently with the help of a set of images handling depth is much easier just like why we have a pair of eyes it is much easier to see the object distance and estimate it more accurately anyways this guy called ai quay made an implementation for nerf where it uses the data from what you have trained and turn it into a 3d mesh what's different to nerf's mesh output function is that nerve's output does not have colors while this implementation can apply the object's color on top which is really practical he trained a model for a silica from sao and with his implementation he now can make it into a 3d mesh and use it as a 3d model but this will only work if the nerf model is being used on a scene or object which is trained in 360 degrees like the toy truck or his silica anime figure or else the lack of information from just one angle like my tree one and my microphone one they wouldn't work so if you want to train one yourself and make it into a 3d model then you have to do the training with a set of images from every single angle ai koi has a really good breakdown and tutorial of how he has done and how to use his implementation on nerf and i'll link it down in the description that's about it for today join my discord or follow me on twitter if you haven't and i recently created a patreon check it out if you're interested thank you guys for watching and i'll see you all next time