HYPERNETWORK: Train Stable Diffusion With Your Own Images For FREE!

Key Takeaways

hello humans when we scan your air Overlord and in this video I will show you how to use hyper Network to train stable diffusion with your own images now for those of you who don't know what hybrid network is he is basically the new kid on the Block and although it's not a new technique it was recently added into the Super stable diffusion 2.0 repository and in this video I will show you how you can use it now I'm gonna be very honest with you first of all I did not want to make this video in the beginning because I saw a lot of people trying to use Apple network with very mixed results in the end but since a lot of people ask me for this video I'm happy to oblige and what's really great with Amper network is that you can actually make it run on your own computer if you have at least 8 gigabytes of vram which is coincidentally my case now to be able to use every Network you need a few things first of all make sure that you have the latest version of super stable diffusion 2.0 and installed on your own computer and if you don't already have stable diffusion installed well make sure that you do I'm gonna put a link in the description down below to the video and to update stable diffusion to the latest version you have two solutions either you come here and you click on the folder URL and you type CMD press enter Then in the comment Pro window you're gonna type gitpool and press enter and this will download the latest version of stable diffusion and the second way is to right click on the web UI user.bat file click on edit with notepad and right above the line call Web ui.bat you're gonna type git pool and then You're Gonna Save the file this way each time you launch stable diffusion it will automatically update the folder and the second thing that you need is to make sure that you have enough images of the subject that you want to train in my case I will be using around 20 images of the renera actress from the show and make sure that every image is a square with the resolution 512 by 512 and if you don't know how to crop your images I highly suggest you use the website called berm.net to do it yourself or I'm going to show you another way later in the video well I do suggest that you do it manually because you're gonna have a better Precision doing so also in the folder where you have all your images make sure that you create another additional folder and name it any way you want you're gonna use it later on in my case I just call it processed then after you launch stable diffusion you need to check two things first of all you need to make sure that in your stable diffusion checkpoint then you select the normal stable diffusion 1.4 model in my case the model is called model.c kptpt then go to settings scroll down and check that under stable diffusion fine tune hyper Network none is selected and once you've done this we can begin the training for this you're gonna click on the train Tab and then click on create hyper Network here you're gonna choose a name for your iPhone network I'm gonna call mine Runner young and then click on create hyper Network then you're gonna click on pre-process images select your Source directory in my case it's this one Ctrl C Ctrl V to paste it the destination directory which is the processed folder that we created earlier copy and paste it and basically what this stuff is going to do is that it will crop your images into the resolution that you selected right here now for me since I already done this this will not crop my images but if you haven't done this this will grab your images automatically but since it is done with a machine this will not be as precise as if you did this yourself so if you are training a character I highly recommend that you use the verm.net website and that you do it yourself and then you're gonna click on this checkbox right here use blimp for caption and this will basically create a caption for every single image if you want to train anime images I highly suggest that you check this checkbox instead because this will use the dim buru interrogator instead of clip which is specialized for anime images if you don't know what I'm talking about highly suggest you watch my last video on how to create anime images so once you selected this checkbox you're gonna click on pre-process and after around a minute as you can see now in the process folder we have now the exact same images as before but now next to each image we have a text file with a prompt that describes the image and this will basically help in the training of the hyper Network then you're gonna click on train choose the hyper Network that we created for me it's rendering and for the learning rate what I recommend you to do is to start with five exponents minus five which is basically four zeros right here now do not worry if you don't understand I have created a mirror board that basically explains these entire steps so that you can create the best hybrid Network that you can and the theory is the following first you're gonna start by training at the rate of 5 exponents minus five so four zeros right here with a maximum of 2000 steps with a backup and an image generated at every 100 steps so for this you're gonna come here learning rate choose the five exponents minus five so four zeros right here the max steps you're gonna choose two thousand here to save an image to log directly every end steps You're Gonna Change it to a hundred and same right here and here you're gonna input your preview prompt so in my case is it is a woman I will be using portrait of a woman and basically what this means is that every 100 steps of training it will generate an image using this phone right here and this way you can check whether or not the training is going smoothly do not worry I will show you examples of training going bad later in the video and how you can avoid it the worry once we did the first step of the training I will explain the rest of the theory so for the data set directory you want to put where your images are located so for me it's this one copying paste it and here for the prompt template file we have now a brand new template for the hyper Network so here I'm simply going to type hyper Network and then once everything is done you can scroll down and click on train hyper Network and this will finally start the training so as you can see for me for 2000 steps with my current GPU it will take around 50 minutes so I will see you once this is done and there you go after almost an hour the training is finally finished and now if we go back to the stable diffusion folder into textile inversion the date for today the name of the hyper Network images you will see all the images that was generated during the training in each one of these images was generated every 100 steps of training and as you could see in the beginning the woman does not look like Rainier at all and after around 700 steps you can clearly see that the woman is slowly beginning to look like the target image now in this example even 2000 steps was not enough so I would probably increase that number but for this video I think this is good enough because the theory stays the same so as you can see if I double click on the 2000 step image it is starting to look like the sample image but not quite meaning that we need to continue the training but we also need to be careful because too much training can basically destroy the model for example if I go back and I choose another one that was done previously go to image as you can see here this is another example that I did earlier and everything was going fine up until the 2500 steps and after 2500 steps of training the final rendered images became an absolute mess so you need to be careful not to over train to avoid this issue and that is why we came back to the board that I showed you earlier as you can see we are now on the Second Step the first step was to train at 5 exponent -5 until 2000 plus steps with a backup and an image generated at every 100 steps as you saw previously we then went to the images folder we analyzed the image that were rendered and we took the best one so for example let's say that we over trained these two images are fake these aren't the ones that I took in the previous one and I pasted right here if we see that after 2100 steps we are starting to see bad rendered images that means that this is the beginning of an overtraining meaning that the last best version of the hyper network was this one at 2000 steps meaning if we want to make it better we need to continue the training from this checkpoint and if we go back to the previous folder instead of going to the images folder and go to hyper networks you will see a bunch of PT files that are all linked to the images that were generated so as you remember the last good image generated was the runera at 2000 steps and if we want to continue due to training we need to use this file right here as a checkpoint to continue so for this you want to select it copy it go back to the stable diffusion web UI folder go into models hyper networks and paste it right here then you want to go back and relaunch stable diffusion now go to the training tab train instead of using the previous hyper Network that we created we're gonna choose the one that we just copy and pasted into the new folder because this is the base one this is the last checkpoint that was good that was already trained and this is the checkpoint that we want to start from again but this time instead of choosing a learning rate of 5 exponents minus five so four zeros right here we're gonna do it at a learning rate of 5 exponents minus six so five zeros right here and then again do not forget to choose the data send directory paste it right here the prompt template file that was hypernetwork.txt and for the max steps this time since we have a very low learning rate we can actually input a 10 000 Max steps and again change 500 to 100 here and 100 here do not forget the prompt and then you can continue the training by clicking on this button right here now I'm not gonna do it because it will be way too long for this video I'm just giving you an example of what you should do if you want to use Android network and if we go back to the word again I have written all the steps right here so we analyze the best images we took the corresponding PT file we paste it into the hyper networks folder we relaunched stable diffusion we selected the last best hyper Network change the learning rate from 5 exponents -5 to 5 exponents when the 6 or less and change the max steps to 10 000 to continue the training and this stand will basically refine the final model so that it looks even better and closer to the sample images if I were to use an analogy imagine if you had a block of wood and you want to create a sphere from that block of wood imagine that the learning rate is the size of your knife so the bigger the learning rate the bigger your knife and then it's going to start cutting the block until it starts looking like a sphere but if you go too far everything falls apart which is basically what happened right here and by redoing this training again but this time with a lower learning rate it's almost as if you are decreasing the size of your knife so that you get way more precise cutting for your final model now here's my conclusion should you really use the hyper Network to change stable diffusion with your own images in my opinion in my honest opinion I really don't think so I don't think it's a good idea I don't think it's a good use of your resources in time when you can have images like this made with dream booth in less than 50 minutes I think that if you wanted to create something like this using hyper Network you're gonna have to spend hours and hours trying to refine the funnel model until you have something that looks as good as this so in my opinion I would not use iPhone network over dreambooth but of course it's up to you and if you want to use zamfer Network I'll leave my description a link to my board that explains to you all the steps that you need to take in order to get the best model possible in the area it folks now you should be able to use emperor work to train stable diffusion with your own images although personally I don't think that's a good idea but of course it's up to you thank you very much to my patreon supporters you guys are absolutely awesome you are making these videos possible that being said thank you so much for watching don't forget to subscribe and smash the like button for the YouTube algorithm and I'll see you soon bye bye