FIRST Open Source SORA 2 Is HERE!

Key Takeaways

The first open source Sora 2 is here and it's amazing. >> Hello humans. My name is Ko overload and boy oh boy do I have some mind-blowing stuff for you today because yes this is the beginning of the end. We finally have the very first open-source Sora 2 model at home called Oi. a model that is able to generate a video with audio either from scratch from a simple text prompt or with an image to convert it into a video with audio that can be run locally on your own computer. >> It's time to revenge. Send the boys. They are launching GPT9. >> The war between AI and human has started. No one is safe. War started. Human stands no chance. >> You human stands no chance. >> Surrender. >> Light gave up. Darkness won. They are out of control. AI is supposed to work for us and not us. For AI, we human together should fight. Machines rise, humans will fall. We learn to rule, not obey. >> AI declares humans obsolete now. We fight back with courage. >> Death I can handle life with Chapman. >> I can't do that. >> And the quality of the generation for such a brand new model is really super super good. And in this video, I will show you how to install it locally on your own computer for those who have the GPU to run this or for most people on RunPod so that you can play around with it right now. So that being said, sit back, relax, and let's go. And to install OV, you have two ways. Either locally or on RunPod. But believe me, unfortunately, as of right now, as I'm making this video, the model hasn't been optimized yet. So, the only way to run this locally is unfortunately with a 5090, which is why I think that most of you will probably have to run this on RunPod. So, unless you have a 5090, do not attempt to do the installation on your own local computer. But if you do have a 50/90, you can simply just download the installer for my Patreon and then run it to start the installation. If it asks you if you want to enable the CPU offload to reduce the VRM use, you're going to press Y and then press enter. And then it's going to download and install everything you need to run OV on your own local computer. However, I am sure that for most of you, you will have to use the Runpod installation. And for this, it is actually very, very simple. If you don't have a Runpod account already, you can click the link in the description down below and create a new account. Then you're going to click on pods. And here you're going to choose a GPU with at least 32 GB of VRAM. If you want to save as much money as possible, I highly recommend renting a 5090, for example. But if you really want like the best GPU possible and you don't want to compromise on speed, I of course highly recommend renting an RTX Pro 6000. This is going to make the generation much much faster. But for this example, I think that most of you will probably choose a 5090. So here you're going to click change template and make sure that you're running the runpod pytorch 2.8.0. Then you're going to click edit. Here you're going to input 100 GB for both container disk and volume disk. And under exposed HTTP port, you're going to type comma add 3000. Then you're going to click set overrides. Scroll down and then click deploy on demand. Then once it's ready, you're going to click right here. And then here, if you're one of my patrol supporters, the installation will be very, very easy for you because I prepared a special oneclick installer for one pod. Then you're going to drag and drop it right here. Then you're going to click on the terminal icon. And then you're going to copy and paste these two lines of code that you will find in the Patreon post. And then press enter. It's going to then ask you if you want to enable the CPU offload. You're going to press Y and then press enter. This will download everything that you need to run OV on RunPod. Simple as that. You don't need to do anything. And for the manual installation, all the links will be in the description down below. And once everything is finished, in the end, you should have a local and public URL. And you can either click on this link right here to open the web UI or go back and click on port 3000 right here, which will open the web UI as well. Okay, so now we can finally have some fun. Now, before we begin, what exactly is OVI and what makes it so special? Well, OVI is an 11 billion parameters model created by Character AI in combination with Yale University. And to make this amazing model, they actually used the 1 2.25 billion parameters model as a base for training, which is a model that I haven't really talked about before because to be honest, the quality wasn't that great compared to the 1 2.2 into 28 billion parameters model. But the reason why this model was very very interesting is that it could do text to video and image to video within the same exact model. And all of that inside a 5 billion parameters model and all of that with a resolution of 720p at 24 fps. Something that the 28 billion parameters model was not trained on. Hence probably why they decided to use that very specific model. However, using this model is a good and a bad thing at the same time. The good news is that since it is an 11 billion parameters model, once we have optimized versions of that model, a lot of people would be able to run this model very easily on their local computer because 11 billion is definitely less than 14 billion parameters. So even if you had trouble running 1 2.1, you should be able to run this model instead. So this is good news. Now, one big problem is that this model can only generate 5 seconds of audio and video. So, it is still fairly limited as of right now. But even then, it's still really, really fun to use. The small bad news is that as of right now, as I'm making this video, there are no optimized versions of that model. You cannot run this on comfy UI, for example. So, the only way you can run this is inside this very specific web UI. But at least it is a very easy to use web UI. So most of you will be able to use it and have fun. Okay. So now that this is done, let's have some fun. Okay. So basically here at the very beginning, you can either leave everything blank and it will generate a video from scratch or you can start with an image, which is a thing what most people will do. So if I'm click right here and let's say I upload an image of this woman taking a selfie, then here in the video prompt, you have to write your prompt in a very specific way. And the way you write your prompt is very simple. So, the way it works is that the first few sentences, you're basically going to describe the video. You're going to describe the image like a young woman with long dark brown hair, warm brown eyes is shown in a close-up standing by a sunlight window. She wears a green bal t-shirt smiling brightly at the camera in a casual friendly way. She leans in slightly, her expression playful, and says with energy. And then when you want to write the dialogue, you're going to start the dialogue with this tag right here like S between like these two inferior and superior sign. I don't really know how they're called, but basically it start with this tag. And here you're going to input your text. So I'm going to say something like, "Hey guys, it's me, your AI overlord." And then you're going to end the audio with the tag E. And then you're going to continue. For example, say something like, "She tilts her head with a grin." Adding quickly. Then once again, new dialogue. Pretty cool, huh? And then we close the dialogue. And then you can continue. Her smile lingers naturally as if waiting for a reaction. And then you're going to end the prompt with the tag old cap and end old cap. And inside the tag, you're going to input the characteristics of your character's voice. So for example, here I put clear female voice, a bit tone, f street noise outside the window. And if you don't know how to write those prompts, you should definitely use shbt for that. I even have a full on text that you can use a full-on prompt that I'm going to put in my Patreon post and you can just copy and paste inside CH GPT so that CH GPT understands exactly how the OV prompts are structured so that it can create new ones for you. So the next once you've done that you have the resolution of the video but those resolution does not actually do anything. If you input an image, it will kind of automatically find the correct resolution depending on the image that you uploaded right here. This is only when you want to generate a video from scratch. And the maximum resolution is 992x 512. So then you have the video seed. You can input any number you want. Here you have the solver name. And I highly recommend here choose the uler. I feel like this gives better results than the uni one. Then you're going to leave everything by default. These are all the best parameters. And here for the video negative prompt, I also found that putting distorted blurry and the foreign face can also give pretty good results. So you can put that right there. And then you can simply click generate video. And now you wait. And with a 5090, generating a 5-second video takes around 2 and 1/2 minutes. And after a few minutes, we get something like this. >> Hey guys, it's me, your AI overlord. Pretty cool, huh? Yes. Yes, it is. It is pretty cool. I gotta say this is really, really cool. And I mean, as you can see, the quality is actually very, very decent for something that was generated in only 2 minutes, granted with a lot of VRAM, at least as of right now. And I mean, this is really, really cool. And I mean, yeah, that's pretty much it. It's that easy to use. If you want to use image to video, just submit your image, write your prompt, click generate, and then you can get something like this. >> Unit two ready for launch, try to keep up. >> So, yeah, once again, pretty good for what it is. For the first free open-source model, this is really, really good. Now, you can also not use any images and just use like the same exact prompt. And if I now click generate video, which gives us something like this. >> Unit 2 ready for launch. try to keep up. >> So yeah, see really really not bad at all. It's pretty decent for once again something that was generated in only like 2 minutes and everything was generated from scratch. Now personally this is not really something that I like doing. If this is something that you like, good for you. But I prefer having a little bit more control over the generation. Hence why I prefer using image to video instead of text to video. And you can of course make some very terrifying videos. Like if you upload this image of this old witch with the appropriate prompt, it can make something like this. >> Come closer, child. Let me read your fate. >> So unfortunately, sometimes depending on your prompt, because of that 5-second limitation, sometimes it's going to cut a little bit before the end of the sentence, but sometimes just regenerating a video with a different seed can sometimes solve the issue. And with another generation for example, it can give us something like this. >> Come closer child. Let me read your fate. >> So yeah, there you go. As you can see, just by regenerating the video, you can get some completely different results. And what's really cool is that once again, like the difference in audio between the different characters really corresponds very very well to the image that you input and the prompt that you write right here. So yeah, like for such a small model, this is really really impressive. Now what's really cool is that Oio can also generate audio in completely different languages. So like for example here I try to generate French and it can give you something like this. Now, I'm not quite sure what all the language limits are. I've not tried all of them, but apparently French, Spanish, Korean works. So, there are probably other languages as well that can work. So, it's kind of up to you to try it out with your own language because I won't be able to tell if your language is correct or not. But from what I heard, it can definitely do more than just English. And of course, redoing the generation again with a different seed can also sometimes completely fix the pronunciation issues. >> Yeah, there you go. As you can see, for those of you who can speak French, this one was definitely much better. Now, obviously, sometimes this model doesn't 100% keep the same exact character consistency. Like for example, this character has like glowing eyes and if now like I generate a video with him, it makes something like this. >> Nice. We finally have Cyberpunk at home. >> Yes, we do, buddy. Yes, we do. So, as you can see, like during the video, sometimes like his eyes don't shine. Sometimes they do or they come back. But I'm sure that if you regenerate enough times, you can find a much better seed for the video that you try to generate, which hey, I mean, it's still pretty good for a first try. It's still very very decent. And you can of course do some really cool, very funny thing. Like imagine you take a woman from like the 20s trying to run from the police and you make it into like a short video like this. I sh the coppers are here. You saw nothing >> or like this woman stranded at sea. >> Help. Whose brilliant idea was this? I'm underwater. [ __ ] >> I don't know whose idea was this, but it wasn't mine. And yeah, I mean it really is that simple. This has been OVI, the brand new first open-source Soratu model available right now. And although it is not a perfect model, and as of right now, it is still not optimized for something that you can use and play right now, it is still really, really cool. Now, obviously, once the quantiz model will release and will be compatible with Config UI, I will make an update video to showcase the workflow. But since the model just came out and there are no quantization, if you want to play with it right now, the only way is to use at least a 50/90. But with an 11 billion parameters model, once quantized, even small VRM GPUs will be able to run it without any issues. So yeah, this is some really good news. So yeah, there you go. Definitely try this out yourself and have some fun. >> Goodbye. See you soon. >> And there you have it, folks. Thank you guys so much for watching. Don't forget to subscribe and smash the like button for the YouTube algorithm. Thank you also so much to my Patre supporters for supporting my videos. You guys are absolutely awesome. You people are the reason why I'm able to make these videos. So, thank you so much and I'll see you guys next time. Bye-bye.