DALL·E 2 Explained

Key Takeaways

have you ever seen a polar bear playing bass or robot painted like a picasso didn't think so dolly 2 is a new ai system from open ai that can take simple text descriptions like a koala dunking a basketball and turn them into photo realistic images that have never existed before dolly 2 can also realistically edit and retouch photos based on a simple natural language description it can fill in or replace part of an image with ai generated imagery that blends seamlessly with the original it's called in painting in january 2021 open ai introduced dolly a system that could generate images from text like this avocado armchair dolly 2 takes the technology even further with higher resolution greater comprehension and new capabilities like in painting it can even start with an image as an input and create variations with different angles and styles dolly was created by training a neural network on images and their text descriptions through deep learning it not only understands individual objects like koala bears and motorcycles but learns from relationships between objects and when you ask dolly for an image of a koala bear riding a motorcycle and knows how to create that or anything else with a relationship to another object or action the dolly research has three main outcomes first it can help people express themselves visually in ways they may not have been able to before second an ai generated image can tell us a lot about whether the system understands us or is just repeating what it's been taught third dolly helps humans understand how ai systems see and understand our world this is a critical part of developing ai that's useful and safe the technology is constantly evolving and dolly 2 has limitations if it's taught with images that are incorrectly labeled like a plane labeled car and a user tries to generate a car dali may create a plane it's like talking to a person who learned the wrong word for something dolly can also be limited by gaps in its training if you type baboon and dolly has learned what a baboon is through images and accurate labels it will generate a lot of great baboons but if you type howler monkey and it hasn't learned what a heller monkey is dolly will give you its best idea of what it thinks it could be like a howling monkey what's exciting about the approach used to train dolly is that it can take what it learned from a variety of other labeled images and then apply it to a new image given a picture of a monkey dolly can infer what it would look like doing something it's never done before like paying its taxes while wearing a funny hat dolly is an example of how imaginative humans and clever systems can work together to make new things amplifying our creative potential [Music]