Math4ML Exercises: Probability

Key Takeaways

if there's one core idea that actually unites all three of the branches of math that we're talking about in this course is that linearity is important linearity is central to linear algebra it was central in calculus because we were doing these linear approximations and now we have these expected values they're also linear operations so i got some advice once for a physicist friend that if i wanted to understand mathematics i should take a long bubble bath and meditate on linearity and that turned out to be really great advice nice i need to try that yeah um let me know how that goes welcome back to the final exercise session for the math for ml class today's session is on probability and i am it's always your host charles and with me today i have ml engineer at weights and biases scott condren working through the exercises hey scott all right probability is one of the tougher components of the math of mls and there's like lots of really deep ideas so understanding it from its fundamentals can be really challenging i'm going to try and focus then as much as possible on the ideas that we most need for machine learning and in particular ideas around entropy and information theory all right let's let's dive right in scott this first section here on representing probability distributions this is a nice little set of exercises just to firm up this idea of probability mass functions and also probability density functions there's like some text there it's worth reading and playing with these exercises but i'm actually just let's just skip past these they're relatively straightforward if you have any questions you can post about them on our community on the channel we'll answer them but these are pretty straightforward i want to get into the more exciting and interesting stuff here with scott so go past this section on probability density functions and here there we go surprise and machine learning loss functions scott in working in machine learning i would guess you've come across things like the cross entropy before is that great that's correct yeah what about like the entropy or the kl divergence are those familiar ideas or unfamiliar ideas those are more ideas i would have seen mentioned but certainly haven't gone deep into great yeah so we'll talk about them a little bit here we won't go into incredible detail about what these things are i think it's a good way to get more intuition about them and why they behave the way they do and why we're using them in machine learning in greater detail but at least for the exercises here what we're mostly going to do is just implement ways to calculate these things also a good way to get a little bit of intuition but those links there point to blog posts and other places to learn additional stuff about all these pieces the surprise the cross entropy the kl divergence okay cool so our first bit here is negative log probability or surprise so in the lectures for this class i talk a lot about this idea of the surprise as the negative logarithm of the probability what that means but we don't ever calculate it so the short answer is that surprises are it's very much like our intuitive idea of surprise when something is unexpected the surprise is really big when something's impossible the surprise is infinite when something is certain there is no surprise at all so that's the the intuition for the surprise um but now let's write some python code for the surprise so that's this exercise here which takes in a probability mass function as an array and an index into that array and returns the surprise just like is written in that little block of map up there okay one thing that i'm immediately a little bit surprised by is that the pmf here has function in the name but it's being passed as an array is that right should i be alarmed there or is that okay yeah no you're right to you're right to at least raise alarms they're call probability mass functions yeah we've talked about the idea of thinking about arrays as functions but this is actually distinct from that as well which is an array is also a function that takes in an index and returns a value so you give an array an index and then out comes either another array or a concrete value if you've given enough indices and so that's the sense in which this is a function this is the exercise that we skipped over there with talking about representing probability mass functions as arrays maybe just think of it as these are the numbers that sum to one it's an array that's got the probabilities in it and we're going to use that to calculate the surprise okay so just connecting into the lecture am i am i correct and saying it's like the space that you showed that was like an uneven pizza shape and then the index is let's say the location of the pepperoni am i correct saying that yeah absolutely it is the index is exactly the location of the pepperonis yeah okay sweet i'm thinking that this is the answer to this first so i'm indexing in and then i'm calculating the log and then i'm returning the negative of that yep nice there we go yeah when you've got these functions to calculate the log and things like that for you calculating the entropy or the surprise is pretty straightforward our goal here is to connect these ideas to machine learning and so we're going to need some other functions here one that comes up in machine learning a lot is the soft max function machine learning models often need to produce some kind of probability distribution so they say what's the chance really what a machine learning model will tell you is what's the chance that the label of this image is dog what's the chance that the label of this image is cat not just what is the label because there's some uncertainty there maybe different people would label it differently or maybe it's hard to tell whether it's a dog or a cat another example would be if you have a model that generates sentences like the famous gpt3 model it produces a probability distribution over what's going to come next and then when we're generating sentences we actually draw randomly according to that distribution so in order to do this we need to output a probability distribution at the end so we need to output an array whose elements sum to 1 as our output there and the normal way that this is done in machine learning is this soft max function which takes an array of whatever numbers you want and turns it into an array full of values that are non-negative and that sum to one so that can be thought of as a probability distribution whose indices tell you the probabilities of different events like the probability it's a dog the probability it's a cat the probability that the next word in the sentence is world after hello the formula for soft max is that right there we use the exponential function to make things positive only and then we divide by the sum to make it so that they add up to one so those are the two steps on the top we turn it to positive only on the bottom we divide by that sum in that formula there all right so now let's implement this softmax function let's take that formula that's up there and compute the softmax so i want to sum over the x i is it x so we're getting these one-dimensional arrays so you don't need to worry about which dimension you're summing over or anything like that we're just getting an individual array here and the exponents on the top and then numpy does broadcasting which means the bottom part of that returns a single number that thing on the bottom is like a normalization constant or just a single number to divide by and we'll divide every element of the array by that same number okay so the reason why we're calculating these surprises is because the surprise connects us to ideas from information theory so the idea that you may have come across as the idea of the entropy of a random variable the entropy of some source of randomness that in terms of its probability mass function and the surprise it's just defined as take the probability mass function multiply it by the surprise for each value of the surprise and then sum those up this is also called the expected value of the surprise or so the expected value of a random variable is the average value of that variable if you were to take lots and lots of samples and calculate the mean it should be very close to the expected value here we're taking the average of the surprise so thinking of the surprise as a random variable so on any given draw the random variable how surprising was that particular draw how surprising was it that this image was labeled dog how surprising was it that the next word in this sentence was world after hello this entropy function has a lot of uses in information theory in lots of different places it's how people quantify compression here in machine learning it's showing up in our loss functions okay here's my formula it's asking me to implement a function to compute the entropy of a pmf represented by an array i assume that means i'm supposed to be implementing this function above is that correct yep okay i'm going to do mp.sum over the probabilities and then i'm going to this is a this is maybe something surprising about the way that that notation works so actually the sum is over i and it gets all of the stuff to the right of it so it's not the sum over the probability or times the negative log okay makes sense so sorry this is like in brackets or something yeah okay that makes sense and then i'm just gonna do mp.log here and i'll figure out that yeah the dot there is multiplication so can i do mp dot dot how does this look almost so the dot there represents just regular multiplication it just says like take these two individual scalar numbers and multiply them together oh okay if you've got that sum there really would you yeah exactly so that's a direct translation of what i wrote up there into code let's check to make sure we translated it right nice awesome okay this is a great this is a perfect answer this is probably the way most implementations of entropy actually work but there's actually a cool little fact here that i want to bring up if you look at that formula again what we're doing is we're multiplying a bunch of numbers together and summing up the results we've got the numbers in the probability array we've got numbers in an array of surprises and we're multiplying the entries and summing them up that maybe sounds familiar going back to the linear algebra section of the course we were really often doing things where we would multiply entries together and sum up the results right yes if we had the entries in one vector and the entries in another vector and we wanted to multiply all them together and sum up the results in a single operation there was one operation that we used to do all of that in a single step which was the product okay i just got there before you tell me okay so that's the dot of mp.low is that going to do it okay nice that's a cool trick yeah maybe this isn't best way to implement the entropy there's maybe more numerically stable ways to do it there's some caveats here but the core idea is that there actually is a really close relationship between these expected values and ideas in linear algebra these are the expected value is a linear operation if there's one core idea that actually unites all three of the branches of math that we're talking about in this course is that linearity is important linearity is central to linear algebra it was central in calculus because we were doing these linear approximations and in fact even the limit is a linear operation and now we have these expected values they're also linear operations uh so i got some advice once for a physicist friend that if i wanted to understand mathematics i should take a long bubble bath and meditate on linearity and that turned out to be really great advice nice i need to try that yeah um let me know how that goes the entropy is something that tells us the minimum number of bits required to store the values of a signal if something varies randomly entropy tells us no matter how good you are at compression you're going to need at least this many bits if you want to represent it perfectly but in order to calculate it in order to know it we need to know the true distribution of these random values we know exactly how likely it is that each event will happen and that's not something we generally have access to we can train models that try and approximate it or we can develop laws of physics they try and help us compute these things but we're always there's some error there's some uncertainty and so we don't usually actually end up getting entropies we generally have something that's called a cross-entropy when you do a cross-entropy you calculate your surprises according to one distribution but then you average them or you weight them according to a different one so the idea here is my model tells me i should be this surprised by the next output in this sentence being the word world after hello it tells me i should be two bits surprised and then when i come across that in a data set that adds two bits of surprise to my average and then i loop over like a gigantic data set and i can get a guess at the cross entropy of my model getting the true entropy of the english language much harder than estimating the cross entropy of a particular model okay so sorry just so i understand this is like would be like if you had access to the the real distribution of whatever you're trying to figure out and this is like trying to approximate it with an estimation each time am i correct in saying that yeah yeah so q here is our estimation p is still the truth and so to you can estimate the cross entropy because you know how surprised you are and you can draw samples according to p so you can estimate the cross entropy with the entropy it's a little bit harder to estimate because you also don't know how surprised somebody who knows the true distribution would be the right component of our dot product that surprise you also don't know so this is at least one step more known okay that's a good point great question okay now let's do the cross entropy here and if you have p and q as explicit values it's even simpler to calculate this cool i'm going to use your dot product trick and i'm going to do p and then it's it's the mp.log of the q exactly all right and the grader agrees which is good news that's our cross entropy it's calculated in a very similar way to calculating the entropy if you've got these distributions so the cross entropy is what shows up in your actual neural network losses so we're getting much much closer to actually having a neural network loss here you won't see it calculated with exactly this formula you know if you're working with say data with labels the cross entropy people use tricks to calculate it we'll also see that if it's something like a regression problem people replace the cross entropy formula it's you know if you actually calculate this out you'll find it turns into the squared error it turns into the absolute error it turns into all these other loss functions but this is the sort of like loss function underneath all these other ones that gives us the specific things that we compute for one machine learning model or another cool so one last bit here of additional entropy calculating things is the kl divergence the cross entropy gives us one way to measure how different two probability mass functions are like if q is really far away from p i'll get a larger cross entropy the more different q is from p but it's not quite a distance um one thing that we like about distances is that a point should have a distance zero from itself so the distance from one point to the same point should be zero whereas if i take the cross entropy of p with itself i'll get the entropy that would just be p times negative log p uh so instead of using the cross entropy it's often convenient to use a different quantity to measure the difference between two probability distributions which is the kl divergence so i gave the formula there which is the sort of like definition formula that people often use if you rearrange this really quickly just take that logarithm and split it apart and turn this into two sums you'll see that turns into divergence is equal to cross entropy minus entropy uh and this is the formula i prefer to think of if somebody asked me a question like oh does the kale diver just do this or what's the definition of the kl divergence i immediately think okay it's the difference between the cross entropy and the entropy it's the way we normalize the cross entropy down to zero and then maybe i like unpack that definition i unzip it into the full formula or into this more compact version that's up there so it gives you good intuition i think about what this thing does depending on the exact application the loss function you're using might be the cross entropy or might be the kl divergence they only differ by this entropy term here and depending on the derivatives you take maybe that term has derivative zero so it vanishes and so sometimes it's useful to think of it as a kl divergence sometimes useful think of your losses as coming from the cross entropy okay i guess a quick question is do you have any rules of thumb that is like when you would see one versus the other yeah i guess if you're doing things that are closer to generative modeling you're more likely to actually care about that entropy term so with things like variational autoencoders this okay still be in there um there's a different kind of divergence in gans and that term becomes really important but if you're doing something that's closer to supervised learning where all you're trying to do is match your particular outputs the particular things you've been given that will just show you that cross-entropy term okay cool um so it's kind of a divide between discriminative and generative modeling that's the more old-school term for those two things but yeah you'll always care about the cross entropy you will sometimes care also about that entropy term cool thanks great question yeah now let's uh let's do a quick function to compute it okay so i'm going to not use any functions like this representation i'm going to just calculate it directly and i'm going to use your dot product trig again and i'm going to pass in the p and then it's going to be mp.log or negative mp that log of q divided by p and i'm now wondering whether numpy is going to let me do that is that going to do a element-wise divide or i guess we could have a check and see if it's we can have a check for sure i think you're right that it is that element-wise divide that you want yeah awesome in general when you use your normal math expressions divide times plus numpy interprets that as i want to take all the elements of this array and like add them together or maybe i want to add the same number to every element in this array and it's only if you use the at the special at operator for matrix multiplication or you use a numpy function that's defined only for arrays like dot that you start getting things that you know change the shape of the array and don't just treat it as like a for loop over applying this to the elements of the array okay cool that's good now we're ready to put this together into the definition of a typical loss function in machine learning ml models often output probability distributions and the goal of training from a galactic view is to align those probability distributions that come out of our model when it's fed with inputs with what we believe to be the true probability distribution on the basis of some samples so we have samples of inputs and labels and that gives us like a kind of estimate of the probability distribution and we want our model's output distribution to match that another way of thinking of it is to say we want our model to be as as little surprised as possible uh when it encounters the data that it sees the way we do that is by the like cross entropy and kl divergence by minimizing those things because those calculate a form of surprise when presented with data and so we tend to do is compute the cross entropy based on our samples on the model's output for data that's labeled with specific numbers we'll actually calculate this fairly explicitly and so we'll calculate a cross entropy of the labels on the model's output so the one thing that stops us from directly just applying cross entropy to model output and label is that our models generally output not directly a probability mass function but they output just arbitrary numbers right if i go through a linear layer i've got i don't have numbers that necessarily sum to one so the usual is to sort of merge these two things together soft max and cross entropy into one step just so that we can connect as close as possible with pi torch and keras and other approaches for machine learning i wanted to explicitly write out this soft max cross entropy combo here and am i correct in saying that is usually referred to as the negative log likelihood loss in some of the or at least in pi torch yeah this combination is called the negative log likelihood loss the name has changed a couple of times i think and i think there's differences in convention between pi torch and keras but yeah this combo goes by that name in pi torch and negative log likelihood is another term another way of saying surprise so if i were writing pytorch i would have tried to call it the surprise loss that negative log likelihood terminology is a little older a little more standard than calling it the surprise okay i'm being asked to implement this you were saying there's some tricks i've come across one of those tricks but i'm not going to go for it now the trick i've heard is you do the log some x trick where you do one of the operations first and then the other and it works out more efficient am i correct in saying that yeah there's this trick involving when you do the logarithm and the main thing there isn't actually speed it's numerical stability oh yeah you'll calculate good gradients but let's rather than doing that what i want to emphasize here is that this thing here is just a combination of two things that we've already done um our soft max function which we wrote and our cross entropy function which we also wrote cool i'm going to do this first and that'll give me my probabilities i might actually just give them a name and then with that i'm going to do my negative log likelihood or sorry my cross entropy which i'm going to do with again your trick which will be probs np dot log of probs oh no so i have been passed in p so um applies that to the logic's increase okay so it'll be between this and this let's go ahead and run this yep and see what the grader says oh you wrote np dot softmax but we wrote the softmax function right oh okay yes true i i just assumed that this existed as a function but we wrote the software so this should do it or at least we should get a new error it'll run failed again and this says is it close so no so it didn't give me a nice error message oh check the argument order okay it did actually give me a nice error message then so in that case maybe i want to have done p and here and probs here yeah great we've placated our testing suite the explanation for why that should be the case is that the model probabilities are what give us the surprise so what we're trying to calculate is how surprised is our model so that should be inside the logarithm whereas in order to get the average like on average how how often would be we'd be surprised if we did this over and over again that's what we need the true distribution for this p that's why we have probabilities from the model inside the log probabilities from the labels not inside the log okay and then the one the one last thing i would say is we've now re-implemented the cross-entropy in a second place and my preference would be to just reuse the definition of the cross entropy here so if we later decide we want to make our cross entropy function better in some way we don't have to make that change in multiple places okay now that we're here this isn't too bad oh actually no i did it wrong though i should have done this yeah i do like the instinct to like pull out variables and name them it can make it more explicit what you're doing when somebody's reading your code later i think that's more important than saving online or saving on space computers are big they can hold a lot of code but it's fairly clear here you've just like translated the docs string into code which is also nice and readable so either way is fine cool all right so to close out here i want to put all of the ideas almost in this class together to sort of like glimpse how all of this comes together in a machine learning pipeline our linear algebra our calculus and our ideas from probability what we're going to do is first we're going to see how we go from cross entropy to a different loss function that's maybe more familiar the squared error and then we're going to see that taking derivatives and doing gradient descent with these two different approaches gives us the same result okay and then we'll have seen a derivative which calculates a linear approximation to a loss function given by a surprise used to train a simple ml model the connection here the way that we go from broad ideas about probability distributions and their cross entropies to a concrete specific loss function is through the gaussian distribution the normal distribution that we talked about in the lecture so when you take the gaussian probability distribution the probability density function that we have here if we take the negative logarithm of it it takes all that stuff that's up inside the exponent and pulls it down and turns it into this kind of simpler looking formula here so we just have the difference between x and the mean value of the gaussian distribution squared that gives us the surprise and this thing yes then we have there's like a normalization constant there that makes sure that the probability density function integrates to one that the probabilities are normalized so we've got that in there but then the core thing that changes as we change x or we change mu is that thing on the left and that's just a squared error the neat thing about gaussians is that they turn a hard problem of like how surprised am i by this input into a simple problem which is how far away is this value from another value which is nice simple function easy to compute cool we're going to implement this as a function the gaussian surprise that computes this value for a mean parameter mu and an array of sample values data so data is our x here and mu is that mean parameter so to write this one i think this one is one that does really benefit actually from naming each piece and then adding combining the like named pieces together okay so data here is my x and mu is this input okay and yeah i'm seeing z here but there's no normalization constant being passed in is this something that i'm going to have to figure out yeah and actually i would recommend writing there's some benefit to actually like writing code backwards rather than writing code the way the computer experiences it which is you know a series of instructions that build up to something start with the code that you want to write to close this thing out which is probably like a direct translation of that line of code up there and then look at that line of code and be like okay why doesn't this run to start off we'll write something that maybe doesn't run data minus mu and then i'm gonna do that yep and then i will just ignore that other thing and see if the grader will help me oh this is this is a fun one the gaussian surprise here says take in an array of data and calculate the total surprise for having observed all of that data and what you've done there is done the broadcasted version which gives you the surprise for each one separately okay so i want a sum of each element in these having been applied with with these am i correct there yep so you want the squared error done for all the data at once and then you just want to add those up okay on the outside of this whole thing yeah i should be able to i know intuitively i could do like four data and i'm resisting the urge to do a loop actually see it is it is nice to write the loop version and make sure you got the thing correct but i think the thing you're looking for is np.sum here okay well mp.some i was reaching for that but then if i do theta minus mu that'll just sum the differences and then i'm going to do the square after that okay so that sums up the differences and takes the square of adding up the differences so if i have two things one is plus one away from you and the other is minus one away from you if i add up the differences i get zero and then you'd square that and get a squared error of zero okay so what i really want is to square inside the sum okay i'm gonna do my loop and then i'm going to figure out without the loop afterwards good idea so i'll just do x minus y squared for x y in zip mu data okay i think maybe this is why this is maybe confusing mu is just gonna be fixed here u is just a single value being passed in and so you actually you don't have to zip them together or anything okay that is definitely where i was confused i think i can survive now with summing the differences so i want to do x oh well actually i'll do i'll do it here so i can do no zip um and then no mu and i'm just going to call this x and then i'm just going to do x minus mu here what have i done wrong now i have a wrong bracket somewhere oh here's the brackets in data yeah and then i also want to multiply that by 0.5 you got it let's see now okay squared errors for x equals mu is zero so surprise is n times a half log z i don't have log c exactly so we haven't gone to the z yet but this suggests that we've got at least some things we're getting the right output type is what the first thing says and then the greater equal thing is checking to make sure that the surprise is always non-negative so like we haven't made any too gross errors that helped us like figure out what the right data type was now we need to get the the right answer okay i would like to fix this as well before we go on yeah good idea this is probably a good learning moment as you say um so if i do data minus mu that'll do a element-wise subtraction oh yeah yeah i guess so mu is just a single number so this does is it broadcasts uh the term is broadcasting so it treats mu as though we're an array exactly the same shape as data and then does element y subtraction okay so this will subtract mu the scalar value from every element in data yes now i want to square all of those um values individually so i can do if i do this will that square all of my values independently so that now i'll have to square the differences and then all i need to do is sum them okay i'm struggling to remember i think oh okay so i just had this out here originally for my sump cool that i think that's a really great instance of how you should think your way through some maybe difficult tensor calculations where you've got arrays and you're doing mathematical operations on them just break it out into a for loop and think about the components and then once you're done and you're confident that is running correctly and giving you the right answers then you can try and turn it into something that maybe runs more efficiently is more like sort of pythonic and fits the way numpy likes you to write things while you still have that nice working example to compare to always to make sure that you're getting the right answer um but yeah i think whenever i come across anything super difficult like that i turn it into for loops unroll it like that so i can feel clearly about what i'm trying to do especially when there's things like broadcasting and like in that case i figured out that i had a misunderstood at least that this was not a um vector it does tend to find bugs that you you know you otherwise wouldn't have found exactly yeah if you tried to for loop over mu you would get a very clear error that's this is a scalar you can't for loop over that and your shapers will maybe be more informative it'll be like oh the length you know you said that you wanted to loop over with a range of a particular length and then that didn't work and you get a clear error that says here's the shape that is wrong in what you just wrote with your for loops it can be much more explicit and so it's a it's a nice way to debug like approach your code from a different angle maybe approach a problem from a different angle and some things will be more clear from that direction cool okay so we still have the issue that is this normalization constant with z yeah that i don't have access to so my suggestion would be right now when i said like you know write the code you wish you could write would just be like plus 0.5 times log z right yeah log of z yeah np dot log oh yes okay and this code won't run yet so the grader is just gonna be like wait what the hell is z so what we need to do is write down what z is by going back up to the formula ah i see okay z is just two pi yep so that's something that it uh makes sure that the thing integrates to one you could do the integral by hand if you're that type of intrepid individual or you can just look up these normalization constants or how to calculate them with a simpler formula which is what i usually do is this the same as in university probability classes where you'll have a big log book of probabilities um and you have to look up that specific one for a given function am i is this a different say yeah so they come about for the same reason actually so what i'm thinking of is like if you take a statistics class and you get like a value for a statistic the t statistic or the z statistic or whatever then you go look up a value that tells you something like the p value or things like that on the basis of that statistic yes in both cases there is a gnarly integral that is like really hard to evaluate and you either need an expert at evaluating integrals to tell you what those answers will be or you need to approximate them with a computer in some way and then in the time before you could just pass around code really easily and run it wherever you wanted like we can now the basic answer was oh we'll just write the answers down and pass those around instead so yeah it's uh i think in the end both of them come from really gnarly hard to evaluate integrals that's why we look these values up okay cool so two pi here is just this number that somebody wrote down so at some time you know calculated by hand that we're not getting the benefit of that and plugging it into our formula okay where are we at here the zero so the surprise is n times half log z have i done that correctly time to empty that log pi the one tricky bit here i actually didn't didn't see this is that formula up there is the surprise for one value so we get a one-half log z every single time we evaluate on a value you could do one of two things you could either put it inside of the sum there or you could multiply it by the number of terms in the sum multiply it by the length of the data interesting okay i could so i could do that now but i don't think i'd really understand it is that shown in this formula or is there some implicit i here that i'm not seeing yeah there's an implicit i there the surprise on observation x i is one half of x i minus mu squared plus one half log z or log z ah no log one of these for each of the x's and i'm calculating one x so if i did that this is correct for like my first eye okay sweet i don't have a preference of which is more elegant here i would say maybe the length one happy to be wrong about that i think that will run faster what you would have to do otherwise is like every time every single time you come across a data point you're adding one half log z but we know that if we do that n times what we'll get is n times one half log z and so you're sort of turning you're taking that and algebraically figuring out what the right answer is multiply by len data and calculating it in one fell swoop instead of writing the slower code let's check that that's right yeah great that is a little that is a tricky bit of that problem there if this jump to using like an entire array of data at once so that is that's definitely a tricky bit there yeah so this one this one here caught me a few times that you know between this being a scalar and then this term being also a constant that you know i should have should have noticed it up there but i didn't and then also that it's not just a constant this added in it for for the whole it's for every single value of x so yeah a couple of tricks but you help me through it yeah maybe another way to organize these exercises would be to do it for one data point then do it for all the data at once so split this up into two exercises yeah or or have you alongside help me through it that's the other way to do it so we're trying to connect this to machine learning models and you aren't going to find a gaussian surprise function like we just wrote in tensorflow or pytorch anywhere what you'll find in set is something that calculates something like the squared error this one's much more straightforward to write because there aren't those confusions about constants or anything like that i just no okay i need to do the difference that'll be applied to each value then squared then sum together yeah nice sick so this is what you'll more commonly see things like the means greater and the sum of squared errors when people are actually writing up their loss functions when they're writing out their code but where that came from is that came from that gaussian surprise they differ in their actual numerical like output values right because there's no log z in this second one but if you take a gradient z never changes as you change mu so that just vanishes when you take a gradient and so they all they differ by is by a factor of one half there's that one half at the beginning of the gaussian surprise if you scroll up just a little bit it's one half times the squared error or the squared difference and so that stays around in the gradient so the scales of the gradients are different and you know we use learning rates to change the scales of our gradients all the time so the scale is not that meaningful so these really are once it comes time to do optimization these are effectively the same function the sum of squared errors and the gaussian surprise and this just shows you that what i'm saying is correct here that if i take a gradient descent step using that gaussian surprise function you implemented i get the same answer if i do it with the sum of squared errors instead down there in that last assert block there is saying that these two gradient descent steps give you the same answer and it's goes that the half learning rate exactly nice and just like scaling the gradients does not change what the best value is so at the optimum the gradients are zero so multiply them by a half doesn't change anything it's still zero so the optima still stay the same the goal of our optimization still stays the same whether we're using the sum of squared errors or the gaussian surprise let's demonstrate that by writing a run gradient descent and then the way i'm going to check that you implemented gradient descent correctly is i'm going to check that it gives the same answer when optimizing you for both the gaussian surprise and the sum of squared error i didn't give you a definition of gradient descent here i guess i gave you the gd step there so you can reuse that function which does a lot of the work but it's it's what we learned i think in a different class where it's the step in the opposite direction of the gradient by a given learning rate yeah and that is one step of gradient descent so all you need to do to run gradient descent is not just take one step but take many steps yeah okay so this would be like your epoch number where in each step for any each epoch we evaluate the gradients for all of the data usually that's done in mini batches but in this case i gather we'll be doing it per element in the data i would say rather than doing mini batches or doing it element wise the gd step expects to get an array of data and so let's do the gd step for n steps for n steps yeah for in in steps or range of 10 steps i don't actually need the steps and then i'm going to call grading descent and it's going to return something which is going to be my my new value for or the parameter mu so mu here is the parameter of this gaussian distribution and it's like the parameters of our model because once you have that parameter you know the probability distribution over the outputs just like once you have the parameters of a neural network you also have a probability distribution over the outputs and so that mu there is our parameter and i guess i would focus in terms of like writing the next line here i would focus on just the api of gd step like you're gonna expect those four things there okay i'll just print them down here so i have them so mu zero is that data is the same f is the same the learning rate is also passed in i'm using n steps here and then i want to update which that's fine so i'm now renaming that and then i'm going to keep doing that after n steps i'll now have like a museo that was updated 10 times and then i'm going to return that value yep let's see yay got it the only thing i might change about the way you implemented that function is i might give an alias to mu uh yeah arrow like call it mu t equals mu0 at the start or or even this would be probably at least then it's not saying it's the zero like i i don't know why are you keep why would you keep around at mu0 i put it in the api of the function just because i think of the names of arguments in my functions as a way to communicate to the person calling my function what i'm expecting so calling it mu zero tells them hey i am expecting the starting value of mu and then if i need to rename that or change something about it in the body of the function i'm happy to do that i want the name in the docs string in the function signature to be as like communicative as possible to the person calling my function okay and do i need to copy it if i'm going to do that orders numpy copy it for me mu t will just be a view on the same array you'll be overwriting it so it might be polite actually to copy mu0 in case maybe someone's passing this in and using it later for some and then also using it later yeah so these are the kind of rough tough things about implementing a good version of gradient descent and a good version of like your machine learning models and their parameters that are why you don't want to write gradient descent yourself you know and you don't want to write your neural network library from scratch like you know what you actually use but if you do it at least once yeah that's true i feel like using autograd here is at least saving us a lot of pain that at least we don't have to implement the grading step ourself but yeah you're right that it's nice that training loops exist in in packages you don't have to think about but in order to be able to like make use of those effectively and to debug them it's really useful to go through an exercise like we've done in the course of this whole class actually and think through where are all these ideas coming from and what would be the basic bare bones implementation of a bunch of these things like calculating a gradient like running gradient descent or doing all these linear algebra operations writing these things out in maybe forms that are not very numerically stable or maybe not very efficient but are easy for us to understand and give the same answer and you can take that intuition and run with it to be able to understand the stuff that's actually useful that's out there in like pie torch and keras i completely agree great i'm glad and hopefully the folks enjoying this course from home also agree i gave some recommendations for additional exercises or or resources for learning math for machine learning in the lecture but i would close out just our exercise video series by saying if you want to really understand this stuff there's no substitute for like building a toy basic version of these things yourself and some of the resources i pointed to help you do that programmer's introduction to mathematics by jeremy kuhn is a good example build your own ex on github is another one these are great places to try that and really get comfortable with these things and become in the end more comfortable with the machine learning models and tools you're building all right thanks a lot scott for working through all these exercises with me uh you've been a really great sport as i've trapped you in my little learning traps and you've done a great job thanks i've learned a lot [Music] you