Transfer clothes between photos using AI. From a single image!

Key Takeaways

this ai transfers clothes between photos given a single image of a person they are able to create synthetic images of the same person in different poses and with different clothing obtained from another input image [Music] this is what's ai and i share artificial intelligence news every week if you are new to the channel and want to stay up to date please consider subscribing to not miss any further news this video is about a new paper from facebook reality labs that will be presented in the european conference of computer vision the eccv in 2020 where they worked on human re-rendering from a single image in short given an image of a person they are able to create synthetic images of the person in different poses or with different clothing obtained from another input image this is called pose transfer and garment transfer most current approaches use a color based uv texture map where for each texture pixel of a filter map a corresponding pixel coordinate in the source image is assigned this correspondence map is then used to estimate the color texture between the input and the target image on the common surface uv system the main difference with their technique is that instead of using this color based uv texture map they employ a learned high dimensional uv texture map to encode the appearance this is a way of getting more details on the appearance variation across poses viewpoints person identities and clothing styles of a picture this may sound abstract but before showing some results let's dive a bit deeper into the process to clarify everything up given a specific image of a person they were able to synthesize a new image of the person in a different target body pose this new technique is basically composed of four main steps using densenet developed in another paper they were able to use the correspondences found between the input image and the smpl in order to extract the partial texture represented in the uv texture map discussed earlier in short the smpl is a realistic learned model of human body shape available for research purposes then using feature net which is a unit like convolutional network shown in this image they convert the partial uv texture map to a full uv feature map which gives richer representation for each texture pixel the third step takes a target pose as input as well as the uv feature map recently found in order to render an intermediate uv feature image finally they use render net a generator networked based on pix2pix hd which can be used for tuning semantic label maps into photorealistic images or synthesizing portraits for face label maps in this case pix2pix was used on the feature image to generate a photorealistic image of the reposed person all the papers involved in this new technique are linked in the description below if you'd like to learn more about them now let's see some amazing results made with this new technique i invite you to pause the video at any time and admire the little details in the synthetic images here's what it can do for pose transfer and here's what it can do for garment transfer without the need for any further training do of course this was just a simple overview of this new paper i strongly recommend to read the paper linked in the description for more information please leave a like if you went this far in the video and since there are over 90 of you guys watching that are not subscribed yet consider subscribing to the channel to not miss any further news clearly explained if you would like to start or improve with machine learning i've linked all the best online courses in a reporter in the description thank you for watching [Music] you