NEW Text2Video - OpenSource!

Key Takeaways

Hello friends today we will be looking at text to video without any video input at all and we won't just be looking at we've seen some examples previously where there are white papers showing what you can do this one you can actually try for yourself you can download it use locally on your computer or you can use it online on hugging face so let's check it out hey by the way did you know that 3.14 percent of sailors are pirates so I first found this on Reddit posted by using illustrious row within the post here there's some links to model scope in China where you can see some of the examples here it's very simple as of yet it's just the two second video most of them actually has shutter stock marks so they you know they train on that obviously but there's a cool examples here and you know the consistency is is pretty good it's not that flickering kind of thing yet that you get in stable diffusion animation this is very nice and I saw a post on Twitter Victor M he made a clip that will put together some of the scenes and here's a Star Wars clip it's it's kind of it's kind of janky still since this is just text to video it's still fairly cool so let me show you how to set that up I'm gonna post the links in the description below this is where you're gonna go you're gonna go the hugging phase and you can actually use this without setting up anything by your own but there's a cue here there are some examples like here's an astronaut riding a horse and there you go and the panda eating bamboo on a rock and here you can say whatever you want a bird flying in the sky and you can see here there's a queue now we're 10 out of 10 so it's going to be about a three minute wait so if you want to do this just test it out it's fine go for it but if you don't it's actually fairly cheap you can run like an hour or three bucks you can get a generation like 20 seconds so you can get a lot of stuff done so what you do is just duplicate this space it says this space is running on a Nvidia a100 large but your copy will be downward to free CPU so you're gonna need to change that so let's duplicate this oh I already have this let's do this tutorial duplicate and now this is all setting up and what you're going to need to do is go into settings here and then it says a CPU basic here that's the free one and you can choose something with a little more vram for example the a10g here or the a100 large depending on how fast you want it to go so I'm going to pick the a10g large here 46 gigs could do the a100 but it requires email us to request access another one a hassle with that so I'm just going to do this one here I'm gonna sleep after five minutes of inactivity so it doesn't drain your wallet if you're just leaving it on and then I'm gonna confirm new hardware here now this is starting up I'm just going to change the sleep here I don't know if it makes a difference but we're doing it anyway okay now it's building here you can follow the what's going on in the log or you can just click this down and wait for the link to finish it's going to be a couple of minutes all right so we have finished and you can see it's now running here on our a10g and now we can start prompting robot dancing oil painting and now you don't have to be in any queue it starts processing right away and after about 20 seconds we have our first video here and while it's not great it is something and this basically feels like the first day of stable diffusion people are trying to generate stuff and it's not great but it's working-ish and that's the cool part now we're just waiting for this to get better and for people to find creative ways to use it maybe there's some prompt tricks and ways to get better results with this I don't know not a lot of people do yet so go check it out see what you can do let's do cat eating a banana our last process took 24.2 seconds I mean that cat is pretty good the banana is kind of weird but the cat movement is uh very cat-like now I'm trying pirate ship camera panning around so now we have our parent chip and the camera is panning here let's get pretty cool you can see the sales are um flying in the wind alright so day one with text Video available for anyone without the video input without the flickering and actual consistency let's see if we can get something done with this that'll be all for today I'll see you in the next video have a good one see ya