Dreambooth tutorial for stable diffusion. Quick, free and easy!

Key Takeaways

now you've probably seen the video with carer crew where they put Sam G's face into all sorts of images obviously you wanted to do that yourself then you realized what kind of Hardware They're ran on and went into the bathroom to cry for 3 hours me too buddy me too until I found out that you can do all this without owning any hardware at all without installing anything so I tried it put my face in and these are some of the results with no editing now let me show how you can do it as well now what you need to get started is images of yourself so I've pulled together a bunch of images of me from Facebook and then I walked around my house and took pictures of me in various angles it's important that you have a variety of these and with different backgrounds I'd say that a number of between 10 and 30 images is a good start then we need to resize this to fit 512 x 512 so we're doing that by going to a site called b.net and then just inputting all of these and you're going to see here how you can crop these so make sure your face is in them you don't need to be super centered or anything just make sure that you're in the images save them and now we're going to input these into dream booth and create our model so I'm going to link all the important links in the description below so this this is dream Booth with Google collab and running in this way won't require anything from your own computer Google will run all the hardware so you can run this on any computer or even your cell phone now there are however some prerequisites since this is all kind of put together with duct tape and it's all brand new technology when I say brand new this collab has only been live for a couple of hours and we're all new with this learning testing processing doing our best to move this further on so anything I say here today could change by tomorrow but I hope it will get you on the process of learning so take anything I say here with a grain of salt for example as of today it's only possible to run this with a Tesla T4 so if you would get an a100 for example it won't run so you need to disconnect your run time and try again until you get a T4 this will probably change in the coming days but let's work on what we have right now so we're going to start by running each cell so we have a T4 that's all good we're going to install the requirements this is going to take some time but I'll speed this up and catch up with you when it's done all right let's move on to the next and install this now you need to make sure that you have a hugin face account and uh if you don't you can go to huggingface doco and create an account up here and then you're going to want to request access to the stable diffusion uh model so I'm going to link this below when you logged in and enter this site scroll down down you press I read license agree with these terms access repository and while you're in hugging phase go into your account settings go to access tokens make a new token let's name this dream Booth put it at right generate a token copy that and we're going to save that for dream Booth so let's run this out now we're running this cell and this is where we need to input the token log and these are the settings that we want to change so the class name is dependent of of what your subject is let's say for example that you take images of yourself and your a man then this could for example be guy or man or person let's put this as man for now this folder is where I'm going to upload all my images to change this to Seb input so I know which one that is and let's change this one to Seb output let's run this and then open our folder structure go to data Seb input which is the newly created folder that we created while running this cell and here we're going to place all of our images now the images are being uploaded and it's fairly quick now in the next cell here the instance prompt that is what you're going to use to prompt yourself into the image so let's change this to my name and let's change this to our class name which is what we set up here you could also leave this which would read from that but let's put man in there the rest we're going to leave it default now the learning rate here joen recommends 1 E minus 6 but for EAS of use let's just leave this at default for now if you want to learn more about these settings you can go into uh dream Booth Discord I can link that in description we're going to up the max training steps usually a number between 1,000 1500 is a good start so let's go with a th000 for now the more steps you put in the longer it will take and it's a quite time intensive process let's run this and when I say quite a long time I mean quite a long time it could be an hour or two or even more depending on how many steps you put in now at first here it will generate your class images and remember we put class man so it will first learn from the stable diffusion model Manis and gather those images and after that it will compile a model based on your input images together with man now the model that we created is saved in Google collab under models this is Seb output so this is the model now if you were to close this Google collab this would all disappear and you would have to retrain your model so either you would have to save that by by starting a new cell putting in z-r the name of your zip file then you put content models and your model name which in this case would be Seb output then you would run that now this would be a pretty large file and it will be saved in diffusers examples dream booth and when that finishes you're going to see a sip here you just press that zp and press download it will take quite some time uh it's almost faster to just retrain the model just make sure that you make all the images you want to do before you quit Google collab so the training is now completed so we can go down run the inference and this next cell here and while that's loing we can start inputting our prompt so let's try pencil drawing of man so this is your to token name that you set up and this is the class name you should always use the token name and the class name together let's set the batch size to two and U let's see what we get as with every other stable diffusion setup you can change the steps here and the strength scale here and here we are our first samples of images from me now to get cooler results you're going to have to look at prompting a little more interesting than this but for a first try I think these are pretty sweet now let's look at some of the results I got here I think this one's super cool it's in a Diablo esque type Universe here I am as a soldier out in the jungle this pencil drawing I think is fantastic captures the hair very cool this I don't even know what it is but some kind of futuristic me this is me in Game of Thrones and here I am again in some kind of game Universe these turned out pretty sweet all of them and this is uh well I don't know some kind of dreamy an older dreamy and here I am in a spacit or a futuristic spacit at least this in an older style painting and here I am in Lord of the Rings so let's say the results aren't what you're looking for well I would recommend you to go to dream Booth's Discord server I'll link that in the description and there are a couple of tips you can try here's a post from yena where he goes through a few options option one here they not not looking at you at all make sure that you have your token and your class and not just your token so remember this should be this is my name and then man if it still doesn't look like you you didn't train long enough so you can try option three you could try token class in a portrait photograph token class in a 85 MIM whatever whatever just remember to put token class in several times however if they're looking like you but they're all looking like your training images well you might have trained too long or the images were too similar perhap you took all your images in your bedroom and didn't change the angle at all you can adapt that with your prompt and stable diffusion it's a lot of Merit to whatever you type first so if you have overtrained your your model just put your token class further back in the prompt so I hope this is going to help you to create some cool images of yourself or whatever you choose as your input model if you like this content please please like And subscribe that's going to help me a lot good luck have fun create some great art and if you have any questions or comments just put them below I'll make sure to read all of them have a great day now