Maximum Likelihood for the Binomial Distribution, Clearly Explained!!!

Key Takeaways

maximum likelihood the binomial distribution that's what we'll talk about today stat quest hello I'm Josh stormer and welcome to stat quest today we're going to talk about maximum likelihood for the binomial distribution and it's gonna be clearly explained note this stat quest follows up on the stat quest maximum likelihood clearly explained as well as the stat quest probability versus likelihood and lastly this stat quest assumes you are already familiar with the binomial distribution if not check out the stat quest the binomial distribution and test clearly explained in the stat quest on the binomial distribution and test we use the binomial distribution aka this nasty-looking thing to determine if in general people like orange Fanta more than grape Fanta in the context of this problem X is the number of people who preferred orange Fanta in this case x equals 4 n is the total number of people we asked about whether they preferred orange Fanta or grape Fanta in this case in equals 7 and P is the probability somebody would randomly choose orange Fanta over grape Fanta in this case P equals 0.5 all together the left side of the equation reads the probability of X the number of people who say they prefer orange Fanta given in the number of people we asked and P the probability of picking orange Fanta then we just plug the numbers in and chugged away at the math and the probability that 4 out of 7 people would randomly prefer orange Fanta is zero point 2 7 3 now if we want to calculate the likelihood of P equals 0.5 then all we need to do is rearrange the left side of the equation that is to say we change this to this now the left side of the equation reads the likelihood of P the probability of picking orange Fanta given in the number of people we asked and X the number of people who say they prefer orange Fanta the right side of the equation however stays the same and this is now the likelihood of P equals 0.5 given that four out of seven people would randomly prefer orange Fanta just a reminder when we calculate likelihoods for P we can plug in different values for it while the observed data in equals 7 and x equals four remains fixed in other words we can calculate the likelihoods for different values of P given that four out of seven people said they preferred orange Fanta for example the likelihood of P equals 0.25 given that four out of seven people said they prefer two orange Fanta is plug and chug plug and chug plug and chug 0.058 the likelihood of P equals 0.25 given that four out of seven people would randomly prefer orange Fanta is less than 0.2 7/3 the likelihood when P equals 0.5 we can also calculate the likelihood of P equals 0.57 given that four out of seven people said they preferred orange Fanta plug and chug plug and chug plug and chug and we get zero point two nine four the likelihood of P equals zero point five seven given that four out of seven people would randomly prefer orange Fanta is greater than zero point two seven three the likelihood when P equals zero point five we can plot the likelihood with a bunch of different values for P between zero and one tada this peak is the maximum likelihood the slope of the curve at the peak is zero that means we can solve for the value for P that results in the maximum likelihood by finding where the derivative I II the slope is equal to 0 so let's do it here's the original likelihood function with n equals 7 and x equals 4 the first thing we do is take the log of the likelihood function we do this because the original likelihood function and its log will both reach the maximum using the same value for P and it's way easier to take the derivative of the log of the likelihood function compared to the original function to see this here is a plot of the likelihood function and here is the log of the likelihood function both have peaks at the same value for P the log function turns the multiplication into addition and it turns the exponents into multiplication if this log stuff is freaking you out well don't freak out just watch the stat quest on logs now we're ready to take the derivative but first because we are running out of room we'll move this to the top of the screen okay now we take the derivative with respect to P this first part doesn't contain P at all so it's derivative equals zero the derivative of the second part is just four times one over P the derivative of this last part is a little tricky since we need to apply the chain rule so we start with 7 minus 4 times the derivative of the log of 1 minus P and we multiply that by the derivative of 1 minus P then we simplify and plug it in BAM [Music] now we set the derivative to zero because we want to find the peak where the slope of the curve equals zero and that will tell us which value for P gives the maximum likelihood now multiply both sides by P times 1 minus P now multiply out 4 times 1 minus P now combine negative 4 P and negative 3p then just solve for P the maximum likelihood estimate for P is for the number of people who preferred orange Fanta divided by 7 the total number of people we asked double bam okay we just solved for the maximum likelihood estimate for P when we have data for X and n however we don't actually need data to determine a general formula for the maximum likelihood for P this formula will give us the maximum likelihood estimate for P when there are X successes in n trials that's how you say it using fancy statistics lingo we'll start with the original likelihood function however this time all of the variables P X and n are unknown just like before we take the log of the likelihood function because it will make solving for the derivative way easier and just like before the log function turns the multiplication into addition and the exponents into multiplication now we're ready to take the derivative and just like before because we are running out of room we will move this to the top of the screen okay now we take the derivative with respect to P the first part does not contain P so it's derivative is zero the derivative of the second part is just x times 1 over P and just like before we have to use the chain rule to figure out the derivative of this last part so we start with n minus x times the derivative of the log of 1 minus P and we multiply that by the derivative of 1 minus P then we simplify and plug it in then just like before we set the derivative to zero note different values for n and X will result in different curves but the slope is still zero at the maximum likelihood now multiply both sides by P times 1 minus P now multiply out x times 1 minus P and now negative XP and positive XP cancel each other out then just solve for P in this case the maximum likelihood estimate for P is X the number of successes divided by n the total number of trials BAM some of you may be saying to yourself duh what's the big deal the maximum likelihood estimate for P is just the average that's obvious well I agree once you know the solution it's pretty obvious but now we also have a mathematical proof that backs up our intuition and to me that's a very comforting thing triple bail hooray we've made it to the end of another exciting stat quest if you liked this tech quest and want to see more of them please subscribe and if you want to support stat quest well click the like button below and consider buying one or two of my original songs alright until next time quest on