Introducing Runway Gen-4 | Runway

Key Takeaways

Hey, I'm Chris. And I'm Jamie. And today we're very excited to announce and release Gen 4, our most powerful AI model [Applause] [Music] ever. All right, everybody. It looks like I found this. But should I just grab it? CN double [Applause] [Music] [Applause] [Music] Gen 4 sets a completely new standard for both image and video generation and it's the first model ever that we're putting out that achieves world consistency. That means that you can create consistent worlds with consistent environments, objects, location, and characters. And when you can do that, you can start to tell longer form narrative content with actual continuity. You can generate the same characters, the same objects, the same locations across different scenarios, so you can block your scenes and tell your stories with intention over and over and over again. It's been very fun to design and develop Gen 4 for the last couple of months and see what the model is really capable of doing. We believe that the best demonstration of Gen 4 as a creative tool using the stories that can be told using the model. We also put together a series of short films and experiments to showcase what you're capable of doing with Gen 4. So now we want to show you the behind the scenes of how those films were made so you can understand how powerful this model is for creating stories with consistent environments. Let's get into it. The first short we're going to take a look at is the lonely little flame. It's actually a story that our director has been wanting to tell for over a decade, but never had the means to do so. With Gen 4, they were able to tell this story of a lonely little flame who just couldn't find a friend. [Music] I love this film. Yeah, it's beautiful. The opening shot is really interesting because it kind of like sets the tone and the feeling and the mood for the story itself. Yeah. And this scene's really all about this skunk searching for something. And with Gen 4, you can actually direct the subject across the scene. So, you'll see we give the skunk two marks. So in this case, we wanted the skunk to go first in this side and then return, making it feel like he's looking for something. And you can see that throughout the scenes, the same character with the same light, with the same mood, with the same condition. And here we're introducing the character, the protagonist, right? Yeah. Again, like all great animation, you see so much expressiveness in the way that not only the characters are designed with the model, but also moving within these scenes. Okay, so what we just saw was a really great example of character consistency, bringing those same characters across different scenes, different lighting conditions with different motions and actions being directed, but you can also just take objects, objects from the real world. So we have a little toy here. We're going to take a photo of it. And then you can basically bring that photo of this toy into Gen 4 and then bring it into any environment you want. So um I'm going to do a kind of little demo here. I'm just going to take a picture of um this particular object that we have here again just from my phone. So you'll see it come here. Um I'll do that and then what I'll do is I'll take that picture um and I'll run it through a set of different environments. So now that we have that picture, I'll just drag that into Gen 4 over here as my reference and I'll also use this photo of New York that I took a couple days ago. And now I'm going to type the kind of composition that I want. So, uh, in this case, I want this wooden toy maybe leaning or next to the sidewalk of the street in New York City. Now, J4 is combining those references. And there we go. We have four initial images that we can browse and select. I kind of like this one. So, let's animate it. And maybe I want the I want people kind of like coming in front of the toy. And now you can see how we get the motion and animation that I required both with the toy and the CD. uh use as a reference. And of course, you can do this with any location. So, we can take the toy um to perhaps the mountains, we can bury it on the desert. Basically, you can do with it whatever you want. Having just seen that, want to show off the next piece. you out there, boy. Holland, I don't like killing cows, but if I have to kill every last one of them, that would be worth [Music] it. Yeah. A lot of drama, a lot of mystery, someone being pursued. We don't know why. I don't like killing cows. So, let's pause right here. This is a shot I really like. I don't know if you noticed, but um we actually see the reflection of the character in the eye of the [Music] cow. Okay, this moment right here, this does it for me. This is so lifelike, so realistic. the physics here as the fire catches across the grass. You're still nice and dark over here and in the back, but you're getting this nice fall-off. Super realistic. So, the really interesting thing about Gen 4 is that you can use consistent characters and objects and environments all across your scenes. So, in this case, we can start with a character with design with cast. We can use another character and environment to basically create the mood, the look and feel, how the character would want to look. you can create a net new image out of that and then you can branch again out of that image. So you can promp the model for different shots, different angles or maybe some changes on how you want that person, that object or that environment to react, which is really powerful because you can take anything from real images to generate it. You can look at anything around you um and use it as a prop either for the characters, the environments or the objects that are in your shots. Okay, so this next one is called New York is a Zoo and it's a really great showcase of the visual effects capabilities of Gen 4. [Music] And this is combining real photos we took of New York, different parts of the city, and then we're taking real photos of animals, and we're basically merging those together to get all these interesting compositions of animals in impossible locations. Yeah. And it also is a clear demonstration of its understanding of physics, the weight of these animals, how they move across surfaces, the way they interact with those surfaces. It's so fun to make this because like again, you're you're making this in just a couple of hours. And it feels like very addictive to be able to get your ideas out of your head really fast by having the model that can understand or be your creative partner in a way. I think one of the most powerful and fun things about Gen 4 is that it allows you to combine anything with everything that you want. Well, sometimes it starts with one image. The next video that we're going to show started with a generation of a monkey in a white studio space and from there just snowballed into a music video that we created. So, let's go check that out now. [Music] Monkey. I hope by now you get a better sense of how powerful and fun Gen 4 is to make everything from consistent worlds to characters, locations in ways that you couldn't do before. Gen 4 is now available to all of our pay users and enterprise customers. Scene references for consistency with character, locations, and objects will be coming soon. [Music] [Music]