Unlimited CONTROL with SLIDERS! (SPECIAL loras changed the game)

Key Takeaways

stable diffusion has the problem of just being able to generate one extreme of a concept or the other but that ends today after this video you'll be able to control every step in between thanks to this relatively new technology and you'll be shocked at how easy it is to use and Trin I even talked with one of its creators to make sure that you have the best information out there and I will teach you how to properly use it how to train it as well as why I think this can be game-changing for stable diffusion let's quickly see what we are aiming to get with this we will create a Laura and don't panic yet because it isn't trained like your normal Laura changing its weight we will dictate how much we want it to go towards one extreme of the concept or the other one will make her look down and with minus one she will be looking up any number in between will move the eyes towards one or the other allowing us to create eye positions that are close to impossible to get with just prompting okay more on how to use this later first let's learn how to train it we will be using this GI Page by Rohit gandikota who is also with whom I talked and gave me some tips and tricks on how to use the tool massive thanks to you here you also have their website a Google collab and a hugging face page where you can test this stuff if you want the installation process is very simple since to use a stable diffusion you should already have most of this stuff installed just copy this text here now go to the folder where you want to install it and up in the folder path type CMD when the CMD opens you'll just have to paste what you copied it will start a process and after the first step has finished it will wait for you to hit enter again just press it and it will install all the needed requirements now you have the tool itself the only issue is that it doesn't have a UI to train on meaning that you will need to modify the files one by one manually or that is what I would say if I didn't create a UI for it myself so that you don't have to worry about any of that I will leave a link in the description where you can download it for free all you need to do now is move the sliders UI python file inside your newly created sliders folder and it should work just fine when it opens it will give you a port just paste it in your browser and we can get into training first disclaimer though I don't know how to code so for this my girlfriend made the HTML part of the UI thank you and I bashed my head against chat GPT for a whole week to make the functionalities work as intended I say this because there might be some bugs other stuff to keep in mind okay once we open the UI click on text sliders and you will see the prompts section this is all you will need for most of your trainings let's look into how it works as a first example let's try training a slider for closing eyes first thing you will need is a Target and a neutral which are slightly different things but we couldn't think of an example where you would use one and change the other so I'm just going to teach them as one you can think of these as the constants kind of like the canvas where change will happen but that shouldn't be taken as the change itself if I ask you what changed in these images you will tell me about 4,000 different things if we want an effective training we will have to narrow it down as much as possible I would like to give you these two images instead for this example you could use person but since a woman's closed eyes can look different from a man's closed eyes we will use two TPS instead one where I use man as the Target and one where I'll use woman next up we have both positive and unconditional this is where you will prompt for your opposites you need to have prompts with a clear difference then AI will grab the positive prompt results and subtract the unconditional from it that way it will try to see what changes between the two and find a way to move from one to the other the higher weight you use on your trained Laura the more it will take from the positive and distance itself from the unconditional and vice versa in case you use a lower weight using erase just just flips sides and guidance scale is the steps it will take to go from a closed eye to an open eye don't put it super high though I think roit recommended a maximum of six or seven as far as resolution goes I would recommend that for stable diffusion 1.5 you use 512 or 768 and for sdxl it is set as theault to be24 I don't activate Dynamic resolution but you can test it if you want for bad size you should know that it multiplies the iterations so if you have a batch size higher than one you should divide the max number of iterations the training will take for that number so in this case I would use 500 also some types of trainings only allow for a bat size of one at the moment now onto the configuration for the training good thing about Snider is that you can train on whatever checkpoint you want and will use the checkpoint epic realism for this one if you are training on a 2.1 model activate B2 and uninstall stable diffusion from your computer to be honest I don't know what B pret is and I forgot to ask so I would just not activate it but you can try it if you want okay you have two training methods and I think the recommended ones are the best ones but if for your case these aren't working you can change them and see if it helps No Ex attention will not train on the ex attention layers of the model ex attention will only train on those layers and fool Will train on both fool could probably help when training harder Concepts but but it will also take more resources and Time by default just use No Ex attention as far as Rank and Alpha go the tool has been mainly tested on rank four so that's the recommended value I tried playing with it but apart from making the luras weit more I didn't find any significant Improvement on most trainings don't be afraid to train and share your results though I'm sure they will be very helpful probably increasing the alpha value will make the change from one concept to the other faster so I just keep it at one for the most part if you want an alpha lower than one you will have to decrease the learning rate and increase the guidance scale if your computer can't handle the F-16 you can use fp32 or try the other Precision methods when training starts check the loss just in case if none appears in it just stop the training and try a different one that will just make the images pure black so it's no good as far as optimizers and L schedular go this is pretty much the same as in Coya SS just choose your favorite one and play around with it and same with iterations this is the amount of time it will train for in my experience it usually is already trained at 250 steps or so and you will most likely never need more than 1,000 iterations un as you really lower the learning rate to make it learn slower of course as far as Max the noising steps go this is like the steps you use inside automatic 1111 I wouldn't go higher than 50 but you can try from 20 to 50 depending on how many you use when generating images here you just name the Laura however you want and choose the folder where you want it to be Sav that by default it will save inside the slider folder in models save every X iterations is the equivalent of the EPO you save for regular training if you save every 250 iterations and have a total of 1,000 you will get for Laura saved you can play around with want DB and buros if you want I would just leave them unchecked as per default if you want to train safe tensors then leave this active else it will train a PT file once we have all the training set up click save parameters you will need to click this button every time you change stuff as it will update the files with your new parameters it will also save a Json file so that you can remember how you trained each model in case you want to revisit them you can load them from up here I would check if this has been saved properly with either the CMD or the web console like so now hit train it will open a Powershell window and start the training I have a 4090 so it will take about 15 minutes to train I'm not sure how much it will take for slower gpus all I know is that my 1080 could not train at all but you have some options one is training from their Google collab just follow the mentioned steps until you are done and now to train you will need to modify the config do yaml and prompts do daml files for SD 1.5 if you're training for sdxl then modify the XL ones also I think the regular GPU collab uses doesn't support the F16 so you can use fp32 instead to input a different model that isn't SD 1.4 you can try to find one in hugging face and copy it like so then go to the GitHub page and depending on what you're training try copying this text now modify it to what you need and run it on collab remember to add an exclamation point before pasting it though you could also try using their hugging face page or even set it up in runp pod. I don't know man that page is very confusing to me however you do it once you have your trained Laura you will need to send it to where you store your luras for automatic 1111 if you are using the UI then down here you have the send to test button oh and by the way the UI won't work unless you close the window for the training after clicking send to test you will first need to select where you want the luras to go and second you will select the models you want to try I'll just send them all for now this will copy them to the selected folder and here in the CMD you will see some training parameters copy and paste them in your positive prompt click the Blue Tick icon right here and it will activate an XYZ plot with your selected luras I will prompt for what I'm testing so closeup photo of a man and no you don't really need a keyword for this now just click generate and it will test all weights from all the luras you can see why the XYZ starts at minus 3 here sometimes the trained Lura will not go from minus1 to 1 as intended but instead go from minus 3 to zero like in this case I want to learn how to adjust this to fit the minus one to one ratio but I haven't found a good way yet for now I just test with a wide range of weights and then I narrow it down most of the change seems to be happening from minus1 to 0.3 or so but from close to open that happens somewhere between -2 and -1 let's see if we can find where the midpoint is I'll take the same seat and narrow down the XYZ values using just one Lowa of the four this time and now we can see that the big leap happens from - 1.5 to -1.3 if we use - 1.4 we get a step in between the two this might not always be the case though as sometimes change can happen in a very small fraction of the weight like in this monster example where I had to go inside the 10,000th digit for a good transition I will go more defin how to use it later okay this was a pretty easy example but there will be some times where the change isn't as simple and you will need to find the right prompt for it how do we do that well you'll want to go to stable diffusion and select the model you're are going to train with if I wanted to train a slider on going from a clean presentable room to a dirty streamer looking room we would test it like this first we will try the basic prompt photo of a room clean but we can see that all the rooms here appear a bit empty and adding Furniture to The Prompt makes it a living room instead so after some testing I ended up with a bedroom clean polished Furniture interior photography which gave me good looking images for the unconditional of dirty just bedroom dirty was not getting it dirty enough so I added dirty moldy humidity stains trash everywhere and we will use these prompts to train with bedroom as the Target and the neutral then just choose the model you used for testing Colossus XEL in this case it is sdxl so I select sdxl as the model name the training safe parameters and train once it's done we can send it to test and see if it works and there you go it's that simple you just input some prompts and hit train a back to front slider done human to monster green to snowy I don't know one for defrosting even one from simple clothes to cosplay of course we still need to talk about how to void beving as well as other uses that this tool has but this is pretty neat already there will be times though where you can't actually find a prompt that works for this for example if we wanted to train a looking from sight to S side ey slider making a person consistently look right or left is pretty much impossible with just prompting so what do we do in that case when you open the UI you will see the image slider stop and here you have the dreaded word data set but these data sets are usually very easy to create since you don't need many images for the realistic side to side slider I made I just took photos of people I knew and myself which probably took around 15 minutes or less in this case we will make an anime data set to create the same slider but for anime models I will use the tips mentioned in this video to create four pairs of images looking to one site and the other you will only need about four to six pairs and more than that won't necessary L Al improve the training the main goal of this data set is to have the same image in both extremes of the concept another example of this would be my empty to full slider where I took pictures of for example a coffee cup empty and then full the last stuff that changes from image to image and the clearer the difference the better when you create the data set make sure that you save the images in different folders here I will have right and left folders every image pair will have one image inside each folder but both images will have the same name like here I'm exporting this one as one maintaining the same name as its counterpart make sure that you save all of the same type of images in the same folder by the way images can't have transparency so make them jpg even if it's just by name I will create a folder containing both the left and right folders and in the UI I will select this one it will detect the data set folders and assign a value to them you should always have the the same value and it's inverted pair so 1 and minus one 2 and minus 2 Etc you could prompt here if you wanted but most of the time it's probably better not to I will select an anime model in this case an Laura leave everything by default except the name and save at every iterations then I'll hit save parameters and train again these trainings are usually pretty fast as a first look it works just fine probably would be better with a data set of six instead of four and playing with the configuration to hit the best spot but I would consider this good enough for now the real test will start in a second but first I need to mention that you can train with more than two folders so not only getting the extremes but in between as well for example if I were to train a raising the arm slider and I didn't want the training to go towards Poland I would need to let the training know by using some image purse in the middle so I would have a data set of images with the arm down here the arm up and then in the middle I'd have two semi opposites then I'd assign values to the data set in order of importance so high up would be two down would be minus 2 halfway up would be 1 and halfway down would be minus one I have only trained one slider with more than two folders and it didn't come out great I assigned a maximum of six folders to the UI but you can do more if you go manual for now just two folder data sets can work super well finally there is too main things you should know know on how to use this one I kind of explained earlier that you will sometimes need to find the exact point of transition and it might be very very precise for example in this generation I divide the last number by half and I keep doing that until I find where the big change has already happened and from here I slowly backtrack until I get where I want it to be the other extremely important thing I want to go over is on how to avoid bleeding as much as possible to achieve results like this only changing the part that we want there are two main methods for doing this roit actually told me one of this and it is a pretty good idea that we've used in the channel before for other luras prompt editing we will use prompt editing to insert the Laura on the generation once the main composition has already been decided for SD 1.5 you can do this with composable Laura enable it and check composable Laura with step now put the Laura inside square brackets like so and add a double dot after the Laura inside the square brackets next enter a number probably four or five will be enough in this case but every Laura and use will be different this will make it so the Laura is added after step four giving AI three steps to create the base image with no bleeding if you add the Laura later you will have more consistency of the base image but you will risk the Laura not having enough time to act and affect it it's about finding a good balance in between the two trying to add theora as late as you possibly can while maintaining its effect for this anime Laura I think it at step four gave results but it also changed the image quite a bit but if we try something like 15 then the image doesn't change enough and the eyes don't really move at all we can adjust and play with it in this case 12 worked pretty well that's how I made this gift and this other one for some I could use up to 18 out of 30 steps and you can try to use a larger range of values too since you add theora for Less last time for Landscapes it can work as well but it is harder to control so you might have to add the Laura pretty early into the generation and sacrifice a bit of bleeding also for sdxl you might have to use a different extension like Dynamic Lura weights if you don't want to go through the trouble of balancing the proper steps though you can use the other method image to image or in painting this is much easier but requires the extra step of sending the images to image to image this is probably a good option for landscapes where you can input at the noising strength of 0.75 or so and play with the low's weight like I'm doing here there is still a bit of bleeding but I think with controled tile and playing around with the noising strength we could find a way to minimize it in every occasion for stuff that only changes one part of the image like our ey sliders you can just in paint the eyes and then use XYZ normally to do the rest for you taking to consideration the the noising strength you want to use which will change depending on what your slider is about and keep in mind that you can push your weight Lora values higher and lower since you need to modify an existing image before I go over other cool stuff that this tool can do and I haven't mentioned yet let me give credit to Rohit under team as well as say thanks for creating this amazing project I will leave their website in the description in case you want to read about how this works you can also follow Rohit on Twitter too if you want to stay up to date with the project and I will also leave their hugging face site so you can test some of the sliders they trained and even train your own I really like the repair slider which acts like an enhancer Laura and works pretty well this is pretty cool because it shows other possible uses for this aside from just using it as a process trainer you can use it to enhance certain aspects of your image or even erase Concepts from your model using the erase tool you could even erase the not safe for work part of stable diffusion wait wait no no no I was joking okay but as I was saying this tool has incredible potential and I haven't seen it be fully explored by the community this basically unlocks a huge part of the data set that wasn't available before yes of course a stable diffusion knows what an almost closed eye looks like but it hasn't been tacted to assign it a text value so we couldn't prompt for it with this that is no longer the case and who knows what else we can make it do I think the hidden potential of this tool will be unlocked when the community starts using it and testing stuff out so do not take this video as a bounding box on how to use the tool big free experiment try crazy stuff post your slider luras and share the knowledge on how you train them the more people train with this the better it will get and the more things we will be able to control I genuinely think this could be a very big deal for stable diffusion and I would love to see it fully developed that's said thank you all so much for watching see you