Stanford CS236: Deep Generative Models I 2023 I Lecture 16 - Score Based Diffusion Models

Key Takeaways

all right so let's get started today we're back talking about diffusion models I think there is still a few things that uh we didn't get a chance to cover so specifically we're going to see how to uh think about score based models as a diffusion model and so where does the name come from and what's the relationship between uh you know score matching and other kinds of training objectives you might have seen before uh we'll see how we can think of a diffusion model or even a score based model to some extent as a type of variational out incoder at the end of the day a hierarchical one but essentially a variational out encoder and there's going to be some connection between evidence lower bounds and the denoising score matching losses that we've been seeing uh then we'll go back to kind of like interpreting diffusion models as normal Iz in flows this is the sort of idea of converting an SD to an OD that we briefly talked about but we didn't have time to go into a lot of detail which will allow us to compute likelihoods uh exactly because it's a flow model and then we'll talk about how to make sampling efficient so take advantage of the fact that once you view generation as soling some kind of ordinary differential equation or stochastic differential equation then you can use Advanced numerical methods to accelerate sampling and then we'll talk about controllable generation so if you want to build a text to image model or you want to use some kind of control uh some kind of side information to let's say generate an image how do you bring that into the equation how do you change the models to to allow you to do that so let's start with a brief recap uh of score based models recall the the underlying idea there was that we're going to model a probability distribution by working with the score function which is this gradient of the log density of the log likelihood essentially with respect to the to the input Dimensions so you can think of it as a as a vector field that basically tells you in which direction you should move if you want to increase the likelihood and we use a dip neural network to model it and this score model which is like a a neural network that takes a let's say an image as an input and Maps it to the corresponding score or gradient of the log likelihood evaluated at that point and uh we've seen that the score can be estimated from data uh using score matching kind of losses and uh it's kind of like a relatively simple regression like loss where you try to compare the estimated gradient to the true gradient uh and you can look at the L2 distance between these two vectors averaged over the data distribution and we've seen that there are ways to rewrite that loss into one that you can at least in print principle compute and optimize as a function of theta that's intractable or at least expensive uh with respect to the data Dimension but we've seen that there is something called Ding score matching uh which basically uh is much more efficient the kind of basic idea is that in instead of trying to estimate the score of the data distribution you estimate the score of a distrib of a version of the data distribution that has been perturbed with let's say gion noise so you have some kind of Kernel or noise kernel which is just like a a gaan at the end of the day uh that will take a sample X and will add the noise to it and uh that defines basically a new distribution Q Sigma which is basically just like the what you get by convolving the original data distribution which is unknown with some a gaussian kernel so it's just kind of like a smoothed out version of the data distribu tion and it turns out that estimating the score of this Q Sigma instead of p daa is actually efficient and there is this kind of uh uh you know if you look at the usual kind of like regression law where you compare your model uh with that with the true score of the noisy data distribution averaged over the noisy data distribution turns out that that objective can be Rewritten into a d noising objective so basically if you can train a model as Theta that can take a noisy image X Plus noise and tries to basically estimate the noise Vector that was added to the image so if you can somehow go from this noisy image to the clean image or equivalently you can figure out what was the vector of noise that was added to this image which if you subtract it will give you back the clean image if you can Doo then you can also estimate the score of the noisy data distribution Q Sigma and this does not involve any kind of uh trace of the Jacobian it doesn't involve any sort of um differentiation it's just a straight forward loss that is basically just D noising the reason we're doing D noising is because by solving the D noising objective that you see here in the third line you are actually learning the score of the noise perturbed data distribution and uh that's a good thing to have access to because if you have the score then you can basically use LINE Dynamics to effectively generate samples so you kind of like if you know how to den noise then you know how in which direction perturbing your image would increase the likelihood most rapidly so you can have like a tailor approximation of the log likelihood around every data point and you can use that information to inform the way you produce you explore displays and you you generate samples the tradeoff is that you're no longer estimating the score of the clean data distribution but you're estimating the score of the noisy data distribution and so yeah it's much more scalable it reduces to the noising but the trade-off is you're not estimating the score of the data distribution you're estimating the score of the noise perturbed data distribution and uh and then yeah once you have the score back to the question once if you somehow are able to estimate the scores uh then you can generate samples by basically doing some kind of noisy stochastic kind of a gradient Ascent procedure where you just uh initialize your particle somewhere and then you follow the arrows essentially adding a little bit of noise at every step trying to move towards high probability regions and we've seen that in order to make this work uh it actually makes sense to not only estimate the score of the data distribution uh perturbed with a single noise intensity but you actually want to estimate the score of multiple versions of the data distributions that basically where each version has been perturbed with a different amount of noise and so you have kind of like these different views of the data distribution that have been uh perturbed with increasingly small in this case uh amounts of noise and what you do is you train a single model a single score Network which is conditional on the noise level so it takes Sigma as an input and it will estimate the score uh for all these different data distributions perturbed with different amounts of noise and if you can train this model uh then you can do uh basically Ling Dynamics where what you would do is you would uh initialize if you have this good model of the score then you would initialize which you estimated by the noising score matching uh what you would do is you would then do L Dynamics where you would initialize your particles somehow then you follow the gradients corresponding to the data distribution perturbed with large amounts of noise you reduce you you improve the quality of your samples a little bit and then you use these samples to initialize a new line Dynamics chain for uh where you're going to use the scores of the data distribution per turbed with a smaller amount of noise and again you follow these gradients a little bit and then once again you take these particles and you initialize a new chain for an even smaller amount of noise and you keep doing that until you have the sigma small enough that basically you're sampling from something very close to the true data distribution and this is an NE L Dynamics and you can see here how it would work so you would start with pure noise and then you would run this sequence of Ling dynamic ex chains and you would eventually generate something that is pretty close to a to a clean sample so you can see that it has this the noising flavor where you would start with pure noise and then you slowly remove noise until you reveal kind of like a sample at the end and this is just again L Dynamics at every step you're just following the gradient more or less and you go towards the a clean data sample at the end so now this was all recap now uh what we're going to do is we're going to start to think about this process as a as a variational out incoder right so if you think about it what's going on here is that we are sort of like going from right to left if you think about multiple versions of the data distribution that has been perturbed with increasingly large amounts of noise what we're doing is is we're starting with pure noise and then we are iteratively removing noise by running these Ling chains so we run a ling chain to kind of like try to transform XT into a sample from the data distribution with a fairly large amount of noise and then we use these particles to initialize a new chain where we follow the gradients corresponding to a data distribution with a little bit less noise and then we run it for a little bit and then we keep going until we kind of like generate a clean sample at the end so we can think of the procedure that we were seeing before as basically uh trying to uh iteratively generate samples from these random variables x0 through XT uh where these random variables are essentially uh what you will get if you were to take a real data sample and you were to add noise to it right because that's we were estimating the scores of these noise perturbed data distributions that were indeed obtained just by taking data and adding noise to it right that was the whole idea of the of the uh noise conditional score Network right and so this is uh essentially at a intuitive level what's going on we iteratively reducing the amount of noise that we have in the sample and uh so the inverse of this process uh is the one that we've used to basically train the network to generate samples for the D nois matching law and we can think about the inverse process which is the one that you would use if you wanted to go from data to Pure Noise and that's a very simple process where at every step you just add a little bit of noise so if you want to go from x0 to X1 you take a data point and you add a little bit of noise if you want to go from X1 through X2 you take a sample from X1 you add a little bit more noise and as you go from left to right you add more and more noise until at the end there is no no structure left and you're left with basically pure Pure Noise and so you can start to see that this sort of like has the flavor a little bit of a vae where there is kind of like an encoder process and then there is a decoder process down here and will make that more formal but that's sort of like the intuition and so the more more specifically basically what's going on here is that there is a relatively simple procedure that we're using to generate this random variables X1 X2 all the way through XT and that procedure is just adding noise so at every step what you do is if you have a sample from XT and you want to generate a sample from xt+ one what you do is you take XT and you add noise to it just gaan noise so that defines a set of conditional densities Q of XT given XT minus one which are just gaussians uh where these GS of basically a given mean and a given variance and the mean is just like the the current sample XT minus one rescaled it's not super important that there is a rescaling there but uh the way you would generate a sample XT given an a sample XT minus one is you would draw a sample from a gaan with a mean which is just XT minus one prescaled and some fixed uh standard deviation or or fixed covariance and uh so we can think of this process of going from data to noise as some kind of like Mark of process where at every step we add a little bit of noise and perhaps we rescale by some fixed constant beta T not super important that you do the rescaling but uh that's how it's uh it's usually done and so uh I'm having it here just to make it consistent with the with the literature and uh this basically defines a joint distrib tion uh so given an initial data point x0 there is a joint distribution over all this random variables X1 X2 all their way through XT uh which is just the product of all these conditionals which are just GS so that sort of like defines a joint given an initial data point x0 there is a joint distribution over all these other random variables X1 through XT where the the the joint is given by a product of conditionals it's kind of like an auto regressive model but a little bit more simple because it's maravian so the distribution of XT does not depend on all the previous ones but it basically only depends on XT minus one on the previous time step and I'm using the notation Q because it will turn out that this is indeed like the encoder in a variational out encoder so you can think of this process of taking x0 and mapping it to through this Vector of random variables X1 through XT as some kind of encoder and the encoder happens to be pretty simple because all you have to do is you just have to add noise to the to the original data point x zero so in a typical vae what you would do is you would take x0 and then you would maybe map it through some neural network that would give you a mean and a standard deviation for the distribution over the latent here the way we get a distribution over the latence which in this case are just X1 through XT is through this procedure so there is nothing learned you just add noise to the to the in original sample x0 so this defines a some valid procedure of basically uh defining multiple views of an original data point x0 where every view is kind of like a version of the data point with different amounts of noise the output technically for this encoder uh is higher dimensional than x0 in the sense that it's the whole collection of random variables uh each one of the random variables XT has the same Dimension as x0 it's the times the the yeah the dimension of the original data point and yes the mapping is not invertible we're adding noise at every step so that defines uh some way of basically mapping a data point to some latent variable or or a vector of latent variables through this very simple kind of like procedure where you just add noise to it and uh it turns out that adding Gan noise is pretty convenient uh because you can also compute uh because everything is basically gausian so the marginals of this distribution are also gausian and so if you want to compute uh what is the probability uh of observing a certain like noisy view of the of a data point x0 after T steps that's another gaan where the parameters of that gaan basically depend on these beta coefficients that we had before again not super important how you take the betas and you combine them to get the alphas what's important is that if you add a little bit of gaan noise at every step the result of applying this kernel multiple times is also another gussan with just different mean and a different standard deviation but you can basically compute them in in close form so the probability of transitioning from x0 uh through through to XT is some other gaussian distribution where the parameters of this gaussian basically depend on the effects of each of the individual transitions that you would do to get through to time XT and this is important for a couple of reasons uh first of all uh basically it's efficient to simulate this chain so if you want to generate a sample at time step T you don't have to generate the whole process of going through the steps you can directly kind of like sample from this marginal distribution without having to simulate the whole chain and yeah if you choose the you know the parameters in the right way this is essentially the same uh exact way we were generating training data for our D noising score matching procedure remember in the denoising score matching procedure what we were doing is we were taking clean data and we were adding uh different amounts of noise cor corresponding to different you know time steps or different noise levels Sigma uh generating all these different views kind of like of the original data corresponding to different amounts of noise levels so it still achieves the same kind of like effect but we're thinking of it as a process that adds noise in incrementally uh at every at every step of this of this process which you can also think of it as a diffusion process right uh you can think of what's going on here as a diffusion process where there is an initial distribution over data points uh which is the data distribution which could be for example like a mixture of two gaussians kind of looks like this here the colors basically indicate uh the uh the intensity of the how how large the PDF f is at that point so yellow points tend to have higher probability mass than let's say these Blue Points that are more closer to the Tails of these two gaussion and what's going on is is that we're basically defining this noise perturbed uh data distributions by basically adding noise so we randomly draw a sample from the data distribution then we add noise to it and by doing that we Define all these noise perturbed distributions where as you can see kind of like the shape of this distribution changes as you add more and more noise you see that there is no probability Mass here in the middle uh but if you add a little bit of noise to the original samples then you're going to get a little bit of probability Mass here in the middle and then if you add a lot of noise uh then basically everything just becomes gaion and so you can think of it as a uh as a diffusion we're basically given an initial condition uh this which is just a data point on this line then uh you can imagine simulating this uh uh this process where you add noise at every step and eventually the probability mass is going to be all spread out sort like all over the space and this kind of like behaves like the process of heat diffusing uh let's say in a in a in a solid or some sort and so that's why it's called the diffusion because there's some kind of process that takes probability mass and then diffuses it over the whole space and uh that this process of essentially defines is defined by the transition kernel which is just basically the gaan in theory uh yeah all you need uh like if you think of it as a well maybe I will come back to this uh in a in a few slides but yes to some extent uh you need several things you need to be able to uh smooth out the kind of like destroy the structure so that you end up with a distribution at the end that is easy to sample from because essentially what we're going to do at inference time is we're going to try to invert this process and we're going to try to go from noise to data so first you have to define a process that destroys structure and goes from data to noise and the noise that you get at the end has to be something simple it has to be efficient so you need to be able to some kind of like simulate any slice here efficiently because we'll see that the learning objective will end up being the noise and score matching and so you need to be able to sample from it efficiently if you want to use uh the noises score matching like objectives uh other than that pretty much yes you get a valid probabilistic model if you if you have those two things uh then you can essentially use this machine and and it turn out the way to invert this process exactly involves the score so if you have the Noah if you have the score then you can invert the process or if you think of it from a vae perspective as we'll see then you can also just try to basically invert the process by trying to learn some kind of like decoder that will try to invert you can think of this process are going from data to noise as an incoder then you can try to just using an training an elbow just by variationally try to learn an operator that goes in the opposite direction which might not involve the score in general but if everything is gaussian uh then it turns out that yeah what you need is the is the score the the important thing is that the original data distribution here okay it's a mixture of gaussians but it it can be anything uh it doesn't have to be remotely closed to to a gausian distribution uh it has to be continuous for this Machinery to to be applicable directly Al though we'll see later when we talk about lat in diffusion models that you can actually um also like embed discrete kind of data into a continuous space and then it will all fall out pretty naturally from a VA perspective uh but clear the initial distribution doesn't have to be gausian it could be just a distribution of our natural images Which is far from gausian what's important is that the transition kernel that you use to spread out the probability mass is gaussian so that kind of like you destroy structure in a controllable way and you know that after adding a sufficiently large amount of gaion noise you have a ition distribution uh the signal to noise is basically uh extremely low and and at that point sampling from a Pure Noise is the same as kind of starting from a data point and adding a huge amount of noise essentially this is just okay this is a diffusion and uh that basically maps from data on the left hand side here to Pure Noise on the right hand side and uh what's uh uh uh what this suggests is that there might be a way of generating samples which basically involves the process of inverting this procedure so we had a simple procedure that goes from data to noise just by adding option noise at every step so we had a collection of random variables with some well-defined uh joint distribution which was just just just like that gausian Define in terms of the Q that you know given an XT minus one uh you find the next uh uh the next one by you know just adding noise to it if we could we could try to generate uh samples by inverting this process and so what we could do is we could try by initially sampling a value for this random variable x capital T and we know that x capital t uh comes from some known distribution um for example It's Just Pure gausian Noise uh and this notation here this Pi you can think of it as a prior some fixed distribution that this diffusion process basically converges to so that's easy and then what we could try to do is we could try to basically uh reverse this process by sampling from these conditionals so you would sample XT minus one given x capital T and then we could go back and back one step at a time and going from Pure Noise to to data and this procedure would work uh perfectly if somehow we had a way of knowing what this distribution is so we know how to define q of XT given XT minus one because that's just a Gauss uh but the reverse kernel which goes from XT to XT minus one the one that goes from right to left uh is actually unknown and that's sort of like why this procedure cannot be uh directly used but the what we can try to do is we can try to learn some kind of like approximation of this reverse uh kernel that goes from kind of like right to left that basically learns how to so that we can learn basically how to how to remove noise from a samp and so so basically that's the the core underlying idea uh we're going to Define now you can start to see it we're going to define a decoder or an iterative decoder uh which has the flavor of a vae you start by sampling a latent variable from a simple prior which could be just a gaan distribution and then we uh go from right to left uh by sampling from these conditionals P of XT minus one given XT which are uh defined variationally in a sense that these are sort of like this is our generative model this is how we usually you know just like in a vae the decoder is some is defined through some sort of neural network in this case the probability density over XT minus one given XT is a gausian and as usual the parameters of the gausian are computed by some neural network so it's the same same sort of like Flavor of a vae where you would sample Z from a simple prior and you feed Z into some neural network like the MU Theta here to get some a parameter for a gaussian distribution over X and then you would sample from that distribution it has the similar flavor here in the sense that uh the reverse process is Def defined variational lead through this uh conditionals which are parameterized in terms of neural network works and uh so there is a true Den noising distribution that would map you from XT through XT minus one we don't know what this object is we're going to approximate it with some gaussian where the parameters of the gaussian are learned as usual like in a variational approximation and we're going to try to choose Theta so that these two distributions are close to each other intuitively so that what we get by sampling from this uh variational approximation of the reverse process is close close to what we would get if you were to sample from the true D noising distribution and so more specifically this basically defines a a joint distribution which is going to be our generative distribution uh where we uh basically first which essentially just corresponds to that sampling procedure that I just described where there is a prior distribution over the the rightmost variable this XT which we know comes from a simple distribution like a gausian and then you would sample from all the remaining variables one at a time going from right to left by sampling from these conditionals which are all gausian with parameters defined by some through some neural networks the the key thing here is that we choose the parameters so we choose this uh Alpha t such that uh basically there is no signal to noise at the end uh and so you are basically left with Pure Noise so basically this uh uh Alpha bar T goes to zero essentially U by choosing basically you know you you might imagine that if you had a sufficiently large amount of noise it doesn't matter where you started from uh everything kind of like looks the same and so that's sort of like the trick you have to define a diffusion process such that and you have to run it for a sufficiently long amount of time which is kind of like the same thing such that so like you forget about the initial condition or you eventually reach a steady state where uh which is known it's some sort of like a distribution with some with some known sort of like mean and standard deviation so that you can sample from it at the end yeah and so that's the that's what's going on here there is a distribution QT uh which is going to be close indeed to some gaan for example which you can always set it up you choose the the the the the transition kernels in the right way B uh as we'll see you can actually do something similar by uh using L Dynamics so uh because it turn out that if you train this thing variationally this me that you learn is basically the score and so as we'll see there is basically one way of generating samples which basically just involves T steps where you just do step step step step three times until you get clean you get a good somewhat whatever hopefully a good approximation of a clean data point but you don't really know it only depends on how well you've learned this reverse process if you're willing to throw more computer it you can actually do kind of like do more compute at every step to try to invert the process better uh one way to do it is to do L Dynamics so L dyamic is just a general way of generating samples it's like an mcmc procedure to generate samples from a distribution uh at the end of the day we know what kind of distribution we're trying to sample from here which is just like the noisy data distribution and if you had the score of that distribution and you can generate samples from it and so we'll see that there is a way to kind of like correct the mistakes that you would do uh if you just were to use this vanilla procedure uh by putting in more compute like if you wanted to sample from the this joint what you would do is you would sample XT then you would XT -1 xt-2 all the way through x0 there is no l Dynamics at this point so there is there is not deterministic it's stochastic because this this transition is a gaan so you would have to sample from a gaan uh where the parameters are given by some neural network so the neural network Park would be deterministic but then just like in a VA decoder it's stochastic the mapping is stochastic okay so now we've defined two things we basically have we have an encoder and we have a decoder essentially here which is parameterized by this neural networks new Theta uh what we can do is and so we can kind of start viewing this as a hierarchical kind of like VA or just a VA where there is an encoder that takes x0 data point and Maps it stochastically to a sequence of latent variables which are just like the X1 X2 X3 all the way through XT and there is some kind of like prior distribution over the latent uh which is just this P of XT this just simple gaan and then there is a decoder uh that would basically invert the process and the decoder is uh in this case as usual just parameterized using using neural networks which are going to be learned so just like in a vae there is these two there is an encoder and a decoder so it has kind of like the flavor of a VA except that kind of like the lat variables there is a sequence of variables that are indexed by time and they have this uh specific structure where the encoder is actually fixed there is nothing learned about the encoder the encoder is just adding noise to the to the data so it's kind of like a ve where the encoder is fixed have a very special kind of structure so the the recall that the sort like the the vanilla vae would look something like this you have a lat variable model you have a lat variable Z which has a simple prior distribution like a gausian and then there is an a decoder which would take a z map it to through through a couple of neural networks mu and sigma and then P of x given Z is defined as a kind of simple distribution ations where the parameters are given by this by these two neural networks and then you have the encoder which does the opposite it takes X and it basically tries to predict z uh and again that's usually some simple gaan distribution where the parameters are usually computed by some other neural network that takes uh X as an input and gives you the parameters of the distribution over the latent and we know that you would train this model by maximizing an elbow and evidence lower bound that would look something like this so you uh you would basically guess the values of the latent variables using the encoder then you have the joint distribution over observed variables and latent variables as inferred by the encoder and then you have this term that is basically just encouraging High entropy in the in the encoder so this is the vanilla version what we have here is kind of like a hierarchical version of the vanilla VA so if you were to replace this single latent variable Z with two latent variables you will get something that looks like this where the the generative process would start by sampling Z2 by a from a simple prior and then passing it through a first decoder to generate Z1 and then another decoder to generate X so you have a joint distribution which is just the product of all this the prior over Z2 the first encoder and the second encoder and then the second decoder sorry and then you have an encoder which is what you would use to infert the L variables given X so it's a distribution over Z1 and Z2 given X and you have a you would have an Evidence lower bound that would look like this just like before here we have a simple prior over Z if you replace this with a vae then you get what we have here and so that would be the the training objective and so that's is exactly what we have in the in the diffusion model so in the diffusion model well we don't have just two we have a sequence of latent variables but it's essentially the same thing we have a joint decoding distribution which is what you're get by going from right to left and then we have an encoder which is fixed which is just adding aan noise to the to the images the way you would train this model is by uh minimizing some elbow loss or maximizing the elbow you know averaged over the data distribution and so just like before the objective function would look something like this so here Q of x0 is just the data distribution and so you would want to maximize the true log likelihood over the data distribution and and we don't have access to it so instead you use the the evidence lower bound which is just like the usual thing uh Q of Z given x p of X comma Z ided by Q of Z given X that's the usual evidence lower bound and there is just a minor sign because I I want to minimize that objective as a function of Thea and uh and where what would you plug in so Q is fixed Q is just this product of gaussians which is this process of adding noise at every step and P Theta is the interesting bit is this distribution that you get by starting from some simple prior like pure gausian noise and then kind of passing it through this sequence of neural networks that will try to kind of like infer uh the parameters from other gaion that you sample from to to to to go through this sampling procedure so that's how this joint in the numerator is is defined it's defined in terms of this uh P Theta of XT minus one given XT and uh so it's just like you know you could try you you can optimize this loss as a function of theta and so it's actually a little bit simpler than the usual vae in the usual vae Q itself is learnable remember you have those five parameters here Q is fixed so you don't have to actually optimize it so it's kind of it's it's just like a VA except except that the encoder is fixed it's not learnable that's the usual elbow objective and recall that these decoders are parameterized are all gaussians and they have this simple form where to sample XT minus one you would take XT you pass it some neural network to get the mean and then they will have fixed the kind of like variance and the the interesting thing uh is that if you parameterize this the neural networks that give you the means of these gausian using this form which is uh again it depends on these betas and these Alphas not super important uh but kind of like if you parameterize the network in terms of an Epsilon Network that takes XT as an input uh and then tries to predict the noise that was added to XT and subtracts it from x sub subtracts this estimated noise from XT to get a again of what XT minus uh one should be uh then you can show that this elbow objective is actually equivalent to the usual denos score matching loss so minimizing the negative elbow or maximizing the lower bound on the average uh blog likelihood is actually the same as trying to estimate the scores of this noise per tered data distributions and so if you were to yeah basically if you parameterize these mean networks by saying okay take the data point that you have and subtract something to make it look more realistic and you up this network essentially ends up trying to estimate the score of this noise perturbed data distributions and so the although we derived everything from the perspective of a variational out encoder it turns out that it's what you're actually doing is estimating scores so the score based model would sample differently so here I'm just claiming that the loss would be the same as uh up to some I guess there is some scalings but roughly uh if you look at it uh you're basically starting from data you're sampling a noise Vector you're feeding the noisy image to this network Epsilon and you're trying to estimate the noise Vector that was added to the data and so the training objective if you choose this kind of parameterization are the same then you have different choices in terms of like how you sample from it like in a score based model you would sample by doing Lin here you're not sampling using Lin you would sample based on uh just going through the decoding process right so this is the training procedure of ddpm of the den noising diffusion probabilistic model and if you look at the loss uh the loss is basically denoising score matching like if you look at the loss that you have here uh that's the same as the denoising score matching loss that we had before the sampling is also very similar actually to uh to what you would do in a score based model if you look at the way you would generate samples is you you start from Pure Noise same as as score based models and then at every step you basically uh follow the gradient which is Epsilon Theta and add a little bit of noise because that's what you're supposed to do if you were to sample from a gaan that is p of XT given or XT minus one given XT and so the sampling procedure uh that you get by iteratively sampling from these denoisers actually is very similar to the to the line Dynamics it's just different scalings basically of the uh the amount of this is kind of like follow the gradient take a step in the gradient Direction and then add noise you basically just do different amounts of noise essentially but it's roughly the same procedure usually you would learn the encoder and the decoder together and they would try to help each other out in some sense the encoder is trying to find structure and the decoder is trying to leverage the structure to generate data more efficiently here the encoder is fixed and it's just adding noise and so the the decoder is just trying that its best basically at uh minimizing the the the the K divergences that you would have or basically maximizing the elbow which is the same as inverting the generative process and uh turns out that in order to do that you need to estimate this the scores and uh to the extent that you can do that process well then you would be able to generate good samples it seems like you would have a easier Time by actually allowing yourself to learn the encoder as well uh but that doesn't actually work in practice uh so it would give you better elbow but worse kind of like sample quality you can also do one step of Ling in practice that what you would do and so at that point I mean what's the difference I me essentially the same thing uh I think one advantage of this score based model perspective is that you can actually think of it as like in the limit of infinite number of noise levels as we'll see which is not something it would it would be a little bit trickier to get with the vae perspective but to some extent they're essentially the same thing it I mean from the albow perspective you're going to get better numbers if you learn the encoder uh but then uh I don't know if it's an optimization issue but then in terms of like the sample quality you get you're going to get blurry samples kind of like the in the sort of like in a VA well there are some intuition related to Progressive coding and in practice people don't actually optimize the elbow uh in practice people optimize a scaled version of the elbow so the elbow do I have it uh yeah so the elbow basically looks like this where you have this Lambda T's that basically control how much you care about the different types steps and in the limit of infinite capacity doesn't matter uh but then in practice people would set them to be all one so which is not the same which is not what you should do if you wanted to optimize the elbow so it's a matter of optimizing likelihood it doesn't necessarily correlate with sample quality so even if the encoder is fixed and it's just something really simple like uding ation noise the reverse is not it requires you to have the score so it's not trivial to to actually invert it but you could argue that maybe if you were to destroy the structure in a more structured way then maybe it would be even easier to invert the generative process yeah so they they generalize to some extent not out of distribution so if you train it on images of cats they're not going to generate images of dogs because they've never seen them and there's no point for them to put probability Mass on uh on those kind of uh so it's it's really based on the actual data distribution that you that you you have access that you're using for training the model so when you add a lot of noise the best way to do noise is to basically predict the average image in the data set and so there you already see that if you've train it on images of cats what the network will do when T is equal to capital T it will basically output the average image in the training set and so it's going to be completely off I think one of the main reasons is that if you think about it the the the amount of compute that you can put at generation time is very large because you're going to kind of like pass it through kind of like a thousand V essentially maybe T is usually capital T here is 1,000 usually so it's a very deep stack of VES that you can use at generation time however because of how things are set up at training time you never have to actually look at this whole very deep very expensive computation graph you can kind of like train it kind of like layer by layer incrementally without actually having to look the whole process so even though you just train it locally to just get a little bit better at every step which is very efficient it's it's it's all breaking down yeah yeah over kind of like labelable level so the stack of is is would I essenti would exactly give you this like if you if you uh the problem is that if you just if the decoders if the encoders are not structured in a certain way you might not not be able to do this trick of basically jumping forward remember that we had this uh now this back a lot this process here that you're it's very easy to go from X Z through XT if these cues are arbitrary neural network there is no way for you to jump from x0 through XT in one step and so the fact that these cues are very simple and you can you can kind of compose them in close form allows you to get a very efficient training process so not all hierarchical vaes would be very efficient to to to train but this is a particular type of hierarchical vae so there are certainly some that would be efficient if you were to just train the vae the usual way you will get a loss if you go through the math it ends up being the same thing it seems counterintuitive that you would want to fix the encoder to be something strange like this where you kind of just add noise and you're not reducing the dimension but if you once you start making that choice then the loss ends up being the same as as the den noising score matching loss historically we came up with a score matching first and showing that it works and then people show okay you can take a vae and you can get something essentially identical and that saves you a little bit of trouble at inference time because you don't no longer have to do L Dynamics you can just sample from a VA so the Lambda basically is just uh something that turns out to be kind of like how much you care about the different uh the different uh Den noising losses over different noise intensities and uh there is a principle like if you just do the math and you go through the elbow there's going to be certain value of Lambda T that you should choose if you really cared about the evidence lower Bound in practice people just choose that to be one and so you're not actually optimizing an elbow beta is the parameter beta T the alphas are Computing in terms of the beta T's that basically controls how how quickly you add noise to the to the data essentially and you can choose it so at the end time we do start from random noise and then uh move it back to the S like clean data uh but it kind of like makes sense to do it incrementally as opposed you could also do it in one step like you could imagine a a VA where the encoder is fixed takes the image and adds a lot of noise uh presumably that inverse distribution that you would have to learn which is this uh where do I have it this procedure here that tries to invert the process going from noise to data it's going to be very complicated well if uh sort of like Q of XT XT minus one is kind of like the same thing to starting a little bit of noise presumably inverting that is also going to be relatively easy so we're kind of like breaking down this complicated problem going from noise to data into a thousand little sub problems where all you have to do is to just remove a little bit of noise and that ends up being better in practice because the side problems are much more tractable to learn so what you do is uh you start with a data point you randomly pick a t which is a like an index in this time the sequence of of noisy random variables then you start with just a uh standard gaion noise and then you generate an XT by basically adding the right amount of noise so it's the same as kind of like Den score magic so this this argument that you fit into Epsilon Theta is just a sample from Q of XT and x0 yeah so the architecture is the same as the as a noise conditional score model so you have the same problem that you need to learn a bunch of decoders one for every T and instead of learning a thousand different decoders you learn one uh that is kind of amortized across the different T's so you have the same uh like this Epsilon Theta Network that is trying to predict noise basically takes the image XT the noisy image it takes T which is encoded somehow and those are both inputs to the to the network that then are used to predict the noise so that's the same as the noise conditional score Network where you take XT you take Sigma or T and then you you use it to to predict the the noise and so yeah the architecture would basically be the uh usually some kind of unit because we are sort of like doing some kind of like dense image prediction task where we go from image to image and unit type architectures tend to be pretty good at this and this is still people are using Transformers too but this is still one of the the best performing models for for solving for learning AO so we've we have a paper that you can look up on trying to get a hierarchical vee to perform one problem with the hierarchical VA is that basically there is a lot of if you start learning the encoder um it's not identifiable there's many different ways to you you basically want to encode the information about the input across all these lat variables but there's many different ways to do it like which bits of information you store where in the latent variable is very much up to the encoder and the decoder to figure out it turns out that if you use a certain kind of encoding strategy that is kind of like forcing a model to spread out the bits in a certain way then you can get pretty close to the performance of with with a learn encoder you can get pretty close to the performance of of these kind of models uh but yeah it's still not entirely clear how to do it there are multiple papers on trying to figure out good noise schedules uh there is many different choices um what's important is that at the end you the only real constraint is that at the end you basically the signal to noise ratio is um zero essentially so uh you destroy all the information then there is a lot of flexibility in terms of like how much noise you add at different steps such that you will get you a that result um you can even try to optimize it uh if you take the elbow perspective you could just maybe learn a simple function that kind of like controls how much noise you add at different steps so you still just add a gaan noise but you can try to learn how to uh so there are ways to to yeah even learn the schedule I would go in the direction of maybe let's learn the the encoder uh or let's make it more flexible um in practice I mean there are a number of papers where people have tried different strategies different ways of destroying structure called diffusion uh they've shown some empirical success uh but in practice people still mostly use cions what we can do now is start thinking about what happens you know we have this diffusion model perspective hierarchical vae kind of perspective where we have clean data and then we have a thousand let's say different versions of the data distribution perturbed increasingly large kind of like amounts of noise really uh if you think about it in terms of a diffusion process a diffusion process is a continuous time kind of process if you think about how heat diffuses or some metal bar that process is not happening at discrete time intervals it's really more naturally thought as something happening over continuous time where time is continuous and so or another way to think about it is you can imagine making this discretization finer and finer maybe fine you're still we're still going to take the hierarchical VA perspective but you can start thinking about what happens if we were to take more and more steps if we go from 1,000 2,000 4,000 we make this steps smaller and smaller and smaller and smaller until eventually we kind of like get this Continuum of distributions which really correspond to the diffusion process and so we have on the a usual on the left hand side we have the clean data distribution which is this mixture of two gaussian where there's kind of like these two spots where most of the probability mass is and and then there is this continuous time diffusion process hap