LTX-2 Released opensource!

Key Takeaways

a new open- source, that's right, I said open-source video model just released that is capable of producing up to 4K 50 well 1 2 3 4 50 fps videos with sound and can run on your computer. Well, I guess it depends on your computer, I guess. So, let's look at how to use it, its results, its quirks, and its pros. Yep, it finally happened. After a few dry months, we finally have a model that is open to the public. We have gotten a lot of closed source models lately, but finally we got some open source stuff. Actually, it's not just a model. LTX2, which was previously closed source, is now more like a whole system that includes eight base models, 11 loras, six pre-made workflows, day zero, and also guides on how to further train it to create your own molds or loras. The base lures available are pretty interesting. And you have some that are meant to help you with camera poses like dollies and jibs. And you also have some that behave like control nets. You got your canny, you got your depth, you got your open pose. Well, they're calling it pose, but still it's it's, you know, s same same. And even a detailer node designed to enhance the fine details and textures. I think there's a pretty stacked release if you ask me. So, here on the screen is the official open-source GitHub repository and it includes everything I previously mentioned and more. It's down in the link for you to check out if you want to. And inside you can download all the open models Loras and of course you'll find more information on how to further train Lauras if you want to do that. to actually run LTX on your PC, they are recommending you have at least, wait for it, 32 GB of VRAM, which is definitely a lot, but the community already managed to run it on machines as low as 8 gigs of VRAM. With that said, I think they used a lot of system RAM. I think like 64 or something. So, to use LTX2, we'll be working in Comfy UI. And if you're watching me for the first time and you're like, "M, Seb, what's Comfy UI?" I strongly encourage you to watch my how to use Comfy UI video guide to learn more about it and stop living under a rock. Setting it up in Comfy is fairly simple. And don't forget to update your Comfy UI too. If you don't want to install Comfy UI on your computer, you can use platforms like Flio or Actifusion instead. Second, we need to download all the models there to run this thing. Like I said previously, there are eight models, like a bunch, but you don't need all of them. Firstly, you need the actual model. So, you have a full version, which is a whopping 43 GB, an FP8 version at 27 GB, and also an FP4 version at 20 GB. And you're thinking, well, which one's the best one? Well, it's fairly easy. Bigger number, better model. So, there are also distilled versions available. I'll be comparing the four. and you're going to place that in the models checkpoints folder. Okay, so that's the model which is fine for running some workflows but for most we'll also need the spatial upsampler temporal upsampler on the distalora which I'll put inside the models latent upscale models spatial upsampler temporal upsampler what is that what does that mean ah you also need this gamma 3 text encoder which goes under models/ext encoders and optionally you can download these base loras as well I know I will thirdly you're going to need workflows What's available to us is text to video, image to video, and this IC Laura workflow. So, this last one is used for video to video as well using the input as a reference. And I know that's something that a lot of you are going to want to do. Again, all the files, models, and workflows can be found in links below. All right, now that we have everything set up, we can drop in the workflow. And if you did everything correctly, there should be no errors or missing nodes. And if there still are, you probably need to update something. Could be comfy. These workflows seem to be organized pretty well. You have your little section here, video settings, inputs, prompts, images, and you also have this Laura section here. So, they're disabled by default, but you can enable them if you want to use any Loras for your work. I just press Ctrl +B bypass and unbypass. The main component of all this is of course the prompt. You got to tell the what you want to see most of the time in text. Creators of LTX2 are really trying to emphasize that a good prompt goes a long way. So here are some important things to keep in mind when prompting. Just start by defining the scene. Is it wide shot, a close-up, cinematic, 2D, 3D? Setting the visual language right away is key. Describe the scene's lighting, colors, and atmosphere. Write what happens from start to finish. Keep everything as one paragraph without any bullet points. Four to eight sentences is recommended. Define who is in the scene. Age, clothing, hairstyle, and so on. Even if you already have an image input, it should help a ton or at least what that's what LTX are telling us. Add some camera movement. You already can use the Laura for this, but if you want something specific, just describe how the camera moves relative to the subject. Is it tracking it? pushing in towards the subject, stuff like that. Maybe it's rotating. Lastly, avoid short prompts like a man is sad. Your goal is to describe the scene as much as possible. Is he crying? Is he slumped? Is he trembling? Stuff like that goes a long way. Additionally, you can also prompt dialogue. Is it any good? Well, it's definitely not 11 Labs quality. Sometimes it's good and sometimes you can really tell it's AI. Okay, so let's actually put it to the test. Now I'll input an image here and write my prompt which I leave lingering on the screen here for a little bit. The settings I have now will generate a full HD video with 241 frames. So that will be a 10 second video. However, LTX2 can also generate video with over 20 seconds long. So, we have a model that can do 20 seconds, it can do 4K, it can do 50 fps, and it's open source. What's not to love at this point? All righty. Now, we are ready to generate. >> LTX just dropped. And >> excuse me, are you recording? >> Hi, Candy person. >> And this is pretty cool. I mean, obviously it took some time and I'm on a 48 GB VRAMm machine at uh while rendering this or generating this, but hey, it's a 10-second video with sound that I got with open-source tools, so I can't really complain. I'm sure with some workflow optimizations, we can get even further. So, okay, that is LTX2. This one is pretty exciting as we've been waiting for more open source release. And while we can be hoping and praying for new water models to open source, it's uh it's been a while and LTX pretty much swooped in and did the thing we all asked for. Overall, we're excited for the release and the Loras and uh let's see if we can get this model and uh some cool workflows out of it. Let me know how you're feeling about it in the comments below. And me personally, I think it's a pretty great as a standalone release. There definitely quirks. I'm really hoping that as time passes, we'll be able to use our own audio to lip sync stuff kind of like with one, right? So, I'm thinking of creating some more videos about LTX. So, please tell me in the comments if you're trying to learn something specific. And that will be all for today and I hope I'll see you in the next one. Okay, bye. [laughter]