The Exponential Distribution

Key Takeaways

the next distribution we'll consider is the exponential distribution so a random variable X is distributed as exponential with parameter Lambda and Lambda is also a parameter we see in the Plus on random variable but this will be a different Lambda uh it's distributed with this uh exponential Lambda if it has the following PDF so I'm just going to give you the PDF right away and we'll consider the we'll try we'll derive the expected value the variance the MGF and then we'll see some interesting properties about this guy okay so uh the PDF is given by this so again we'll do our standard form F big x uh Little X is equal to and again it's going to be two things so it's going to be uh Lambda e the Lambda x if x is bigger than or equal to 0 and Zer otherwise okay so this is only defined for non- negative values of X so and then zero otherwise so if we draw a quick graph it looks a little bit like this so Lambda e the Lambda X so so given that Lambda has to be positive which is another restriction here we see that it'll look like this so if we just say Lambda is one for Simplicity then it's just going to be e to the minus X and we know how that looks right if we have zero and this goes out to Infinity over this way then e Theus X looks like e to the X reflected across the y AIS so it simply just looks like we start here at uh one it'll just taper downward just like that okay so uh and if we increase Lambda then this just kind of changes the shape of this guy for us and to do a quick reality check we should definitely check that this PDF does integrate to one over 0 to Infinity so if we take the integral from 0 to infinity and inside we want to put Lambda eus Lambda X let me pull Lambda out eus Lambda x uh DX then we get Lambda out front this eus Lambda X integrates as -1 over Lambda eus Lambda X evaluated from 0 to Infinity so we're going to do have to do a lot of these uh e minus Lambda x 0 to Infinity calculations here so it's important that you realize how to do them so this this uh computes nicely this is minus one now we have e to the Lambda Infinity so we have e to the minus Lambda infinity and since Lambda is positive that's really e to the power of a negative huge number and that becomes zero as we see from the graph right it tapers down toward zero so really that's 0o minus and we plug zero into here we get 1 so we get 1 * 1 which is 1 which is good that is what we're supposed to be getting so we see the integral under this whole graph is indeed one uh okay so let's go ahead and calculate the three things we usually calculate expected value variance and MGF and then we'll talk about a few cool things about this distribution uh they're not too hard to calculate we want to find the expected value of our X that's going to be integral from 0 to Infinity remember I'm just taking shortcuts now uh really it's integral from negative Infinity to Infinity but we see that if we were to extend this over here by our definition it would have be zero everywhere else so this would be just identically zero so there's really nothing to integrate there okay so we just have that integral and it's going to be X Lambda eus Lambda X DX now this is going to involve integration by parts so first let's pull the Lambda out front and we're left with the integral 0 To infinity x eus Lambda X DX all right cool let's use integration by parts U equal x uh let's say DV = eus Lambda X DX okay so V must equal -1 over Lambda e to the Lambda X and du equal DX putting that together we get UV minus V du so UV is x * this quantity so it's minus X over Lambda eus Lambda x uh minus V du so we get V and du is here so minus with this - 1 over Lambda becomes A+ 1 Lambda uh we get e to - Lambda X DX okay nice and let's put our bounds in so this is from 0 to infinity and again this uh a cool thing about these General calculations we'll see right now so this has to also be evaluated from 0 to Infinity but let's notice something if I plug Infinity into this guy right here then I'm going to get Negative Infinity over Lambda e to minus Lambda Infinity so that seems unclear at first let's do a little side track over here so we have minus uh X over Lambda and I'll put this e to Theus Lambda X in the denominator so it's e to the Lambda X down here now what grows faster X or e to the Lambda x e grows faster and we can see that just by a simple graph here's X's growth rate the graph of x and e to the X grows like that it grows uh exponentially right so that's the graph and we see that e to the X grows very very very fast so this is going to dominate this term and this is going to go to Infinity much faster causing this hole to be zero so plugging Infinity into here causes zero if we plug zero into here we get zero also because because the den numerator become zero so this term almost always uh in this general form will go to zero and we won't have to worry about it at all all we got to worry about is this part right here now this we've actually seen what is this integral right here it's integral from 0 to Infinity e Theus Lambda X DX where do we see that before right here 0 to Infinity eus Lambda X DX and what did we uh find that that was that turned out to be if we follow it down here it turned out to be1 Lambda * -1 so let's write that so remember this Lambda out front we took out in the very beginning still exists so there's that Lambda there's this 1 over Lambda and there's the integral which we said was -1 over Lambda * -1 so really this minus minus becomes a plus so this becomes a plus we have a Lambda one over Lambda cancellation and all we're left with is really 1 over Lambda the expected value of exponential distribution is 1 the parameter Okay cool so that's pretty easy to remember and we'll see the variance is pretty easy to remember as well but it'll take a little work for us to calculate now the variance is remember using our trusty variance formula expected value of x^2 minus expected value of x squared okay cool uh now this we just calculated so this whole thing right here is 1 over Lambda squar and I'll put a minus here to signify that minus um so then we need to calculate this it's done in the same way we need to do integral from 0 to Infinity of Lambda eus Lambda X now we have to put X2 cuz we have X2 up here DX so again let me pull the Lambda out and we'll do integration by parts on this uh pretty painlessly so it's eus Lambda X DX now I say painlessly because a lot of the work is done for us we have to do the first step and we know the integral that will come out of that so this is equal to Lambda um and then what we're going to use as our integration by parts we're going to say uh we're going to say U where will I do this I'll do this uh right up here so we'll say U = x^2 we'll say DV equal eus Lambda X DX just like it was before so we get V = 1/ Lambda eus Lambda X and we get uh du = 2x DX okay sorry for that bad uh in the margin here but uh we see that what we get is this Lambda up front and then we get uh we get UV so it's going to be x^2 and then V is - over Lambda eus Lambda X this guy evaluated from 0 to Infinity minus uh V du okay so we have V is right here so again we have plus one over Lambda and then du is uh right here so we have so doing this uh integral VD term we have V right here and then we have integral of vdu is so we'll put a two up here in fact we'll put the X we'll put the E to Theus Lambda X we'll put the DX okay so uh that's all we have 0 to Infinity now we're pretty much done actually because let's look at this term evaluating this from 0 to Infinity we have the same deal we had before with that first term if you put Infinity the growth rate of this e is faster than x^ squ um so this will go to zero in that case if you put zero into here x becomes zero numerator is zero the whole thing is zero again this thing nicely goes to zero and doesn't cause us any trouble um so the next thing we have to worry about is so this Lambda and this 2 over Lambda cancels to become just a two so we have 2 integral 0 To infinity x eus x uh e Lambda X DX now you might be thinking oh no we have to use integration by parts again but really we don't we know what this is because we did that calculation for the expected value that is exactly what the expected value was right this is the expected value because it's just uh x * the PDF so the expected value we know is 2 over uh the expected value is 1 over Lambda sorry so then this will become 2 over Lambda cool so we found that the expected value of x^2 is 2 over Lambda so we need to do the calculation 2 over Lambda - 1 Lambda 2 what is 2 over Lambda - 1 over Lambda 2 that's going to be now the next thing we want to do is calculate the variance of this exponential random variable so we're going to use our typical trick variance of some exponent from some random variables equal to expected value of that the random variable x^2 - respect value of x^2 okay so this we know we found this was 1/ Lambda so we have to square it so this is min-1 over Lambda squ incorporating this minus sign into here uh now this is something new we got to find so it's not that hard we just use the formula again so 0 To infinity eus x DX and the Lambda I'll put out front Okay we need to do this now this seems a little bit daunting maybe we have to do here thing we'll have to do the uh integration by parts twice we actually won't because once we do it once we'll find something that looks a lot familiar okay so let's go ahead and get started so we're going to say U = x^2 DV is going to equal e Theus Lambda X DX again so this is familiar so this is just here we get this and then we get du = 2x DX so we get uh UV so we get x^2 Lambda eus Lambda x from 0 to infinity and we'll see this will cancel nicely minus integral of VD so V this is 1 over Lambda negative this will become A+ 1 over Lambda and then we'll put all the remaining terms inside here uh so we'll get X VD so we'll get X DX and actually there's a two here let me put that up here okay so we have 2 over Lambda times this integral right here now let's deal with this guy first now if we plug Infinity into here for X we see that uh this e to the ne Lambda X which is in the denominator will grow a lot faster than x^2 therefore that'll be zero if you put zero into here this will be zero because x^2 is Zer at zero therefore we have the numerator being zero so this again nicely doesn't give us any trouble and just goes to zero silently so then we have 2 over Lambda and actually I need to put in this original Lambda that was out here so putting that up here putting carrying that over here we have 2 over Lambda and then this looks very familiar what is it this kind of looks like the expected value doesn't it because it's the PDF or kind of part of the PDF time XD X so it's the expected value except we're missing this one over we're missing this Lambda term over here so it's actually the PDF it's the expected value uh divided by Lambda so what is the expected value it was one over Lambda and we take that and divide it by Lambda we get one over Lambda squar so it's 1 over Lambda squ so we have some nice cancellations um we have this Lambda with this Lambda and we get this is 2 over Lambda 2 so that is the expected value of x^2 term right here so all we have to do 2 over Lambda squ minus 1 over Lambda Square gives us 1 over Lambda squ so this is equal to the variance of this exponential random variable right here that's very nice now let's find the moment generating function um this won't give us too much trouble either so what we need to do we need to find the expected value of e to the s x where X is our exponential variable and S is some real number okay so uh let's just go ahead and Tackle this using our normal means so 0 to Infinity uh and we're going to put e to the SX and we're going to put Lambda e to the negative Lambda X this is really nice because we have these e terms coming together and they're going to make life very easy for us DX is equal to I'll pull the Lambda out front and then we have integral from 0 to Infinity of e to the x s minus Lambda DX now uh this is a very basic integral because this is a real number this is a real number so all I have to do is take Lambda over uh s minus Lambda e to the X s- Lambda evaluated from 0 to Infinity so I put Infinity into here now this is a little caveat of this thing uh let's let's first assume we're in the world where Lambda is bigger than S okay so that means that s minus Lambda is going to be smaller than zero so that means if we put e to the X so being smaller than zero means the graph looks like that okay tapering downward just like our PDF look like so putting Infinity into here we see that that goes to zero so we have Lambda / s- Lambda 0 minus and putting 0 into here we see that clearly is one so this is a Nega 1 so we can just flip the denominator so we get Lambda over Lambda - s and that's going to be our uh moment generating function under this assumption that Lambda is greater than S so this is actually our MGF now let's just explore what happens if uh if that's not true what if we had Lambda is less than S well if Lambda is less than S then this s- Lambda is positive and the graph doesn't look like that at all it actually looks like uh that right so considering only starting at zero now that's a big problem because if we plug in Infinity into here that's going to be Infinity it's going to go to infinity and this won't have a value therefore this MGF is only defined if the parameter exceeds whatever you're putting in there so you can't put values that are values of s that are bigger than Lambda so that's a restriction um that we have to keep in mind also a way to see that is if you did put values that are bigger than Lambda if s was big than Lambda then this denominator would be negative Lambda up here is positive so this whole thing would be negative so that you're telling that expected value of e to the something is negative impossible because uh this e to the something is a positive uh always has to be a positive variable it's the expected value cannot possibly be negative so you know something must be fishy so here's our MGF I'll label all these things MGF here's our variance and the expected value uh was 1 over Lambda Okay cool so we found all the major things now let's do a few cool Explorations in to this guy the first the the major exploration I guess the primary one we'll be doing is looking at this kind of example so let's say you have this is going to represent time let's say time starts someplace at zero you just any arbitrary time and let's say we'll put a Time s here and let's put a Time T here so this is in time interval right here of length T so we're going to consider two probabilities first is is the probability that uh X is less than or equal to T and the second probability will consider as a conditional probability so make sure you're familiar with this is the probability that x minus s is less than or equal to T given that X is bigger than S now this looks a little bit uh strange maybe intimidating at first but let's just go through what it means what is the first one X here is still an exponential random variable and we're going to be thinking of it as a kind of a waiting time how long you have to wait so maybe you can give physical meaning to this so this makes it more uh intuitive so let's say uh I've heard this with a bank problem so let's say you walk into a bank and uh you are the first person in line so let's say there's a window window at the bank right here and then someone is being served currently and you're the next guy in line and let's say the waiting time uh so X is going to be how long a person wait how long a person takes at the window is distributed as an exponential distribution so first we're saying what is the probability that the time they wait is less than or equal to this time T and the next thing we're asking uh let me put it in this way let's say you've been waiting for S minutes already okay so you started at zero you walked in you got in line and at Time s you're still standing there waiting for the person okay so you're sitting there thinking oh my how much longer do I have to wait and intuitively maybe you're thinking that uh because I waited so long I must not have that much longer to wait and if we're in this world where these times are distributed exponentially you would be wrong because uh we'll see that calculation in just a second but so this means that x - s so what does this mean this is the total waiting time the total amount of time this person spends at the window minus the time you've already waited so in a sense uh it is the remaining time you must wait okay so you've waited s time uh X is the total time so maybe what if x is here that means that uh you've already waited s time so this s this x - s is this extra time you have to wait when you make that thought in your head where you're like how much longer do I have to wait that much longer is x - s uh what is the probability that that time is less than or equal to T so uh let's say this is T exactly so what is the probability that we're in this range less than or equal to T given under the condition that we're already passed s okay so let's calculate both of these guys so the first one is probably the easiest probability that X is less than or equal to T remember um if you want to find the probability that a random variable is less than or equal to some value all you have to do is integrate the PDF so the PDF from zero till that value so in this case it's zero because exponential starts at zero the PDF is given by uh Lambda e to Theus Lambda X DX and we can easily solve for this this is just uh Lambda we take the Lambda out we have -1 over Lambda e Lambda x 0 to T So then this Lambda negative 1 over Lambda becomes a negative 1ga 1 e to the Lambda T and then we have minus E to that we put zero in here we get one so it's really 1 - eus Lambda T that's our first probability now what's our second one so this one's going to be a little bit more involved to calculate but really not that bad so we have uh let's rephrase it first it's going to be probability that X is less than or equal to t + S given that X is bigger than S so I did it like this this because I want the X alone so I can do calculations in terms of it now this is conditional I'm going to rewrite it uh using the definition of conditional probability so it's probability that X is less than or equal to t + S and X is bigger than S over probability that X is bigger than S this is the definition of conditional probability good um now this top part X is less than or equal to t plus S and S is bigger than S uh then that is this part we can just drop because if s is bigger than S uh if and on the second calculation let's do it on this small sheet of paper just to keep everything clean uh so I Rewritten it into this form because I want uh X to be alone so I can do calculations in terms of it okay so the second probability is what using the definition of conditional probability it's going to be the uh it's going to be X is less than or equal to t + S and X is bigger than S over probability that X is bigger than S this is just a definitional conditional probability of uh review it really fast if you need to but um this is exactly what this conditional bar means uh okay cool so then let's look at the top part the probability that X is the basically X is saying the time you're waiting is uh less than or equal to t plus s and it's bigger than s means that it must be in the range let me draw your picture again here is s uh here is let's say here is t plus s so the probability that your waiting time is between these two guys or sorry the probability your waiting time is less than here that's what the first part is saying right the x is less than or equal to t plus s and it's bigger than s means it must be captured in that range exactly so this is saying it's the probability that uh X is less than or equal to t plus s but bigger than S all over probability that X is bigger than S now this again we're just going to turn to our trusty PDF and we're just going to evaluate this guy so it's going to be first part let's do the top is integral from and our bounds here are s and t plus s t + S uh eus Lambda X DX bringing the Lambda out uh okay so then here we have Lambda and then we have over Lambda minus that just bring that out so just negative 1 there uh and then we have e to the Lambda t + S minus E the Lambda s okay uh and then that is just going to be e to the Lambda X S1 minus E the Lambda T So e Lambda T and uh what I did was I took this negative one to and I used it to flip these two guys flipped the um I did this minus this instead and then I took out a e to the Lambda s from both and then what I was left with was one here and E the Lambda T here and now the bottom is what so the bottom this is evaluated as um again bring the Lambda over here this is greater than S so it's s to Infinity e to the Lambda X DX this is just Lambda over Lambda again uh so e to the infinity is 0 minus E to the Lambda s so we have a negative here negative here we just get e to the Lambda s so what we want is this divided by this so This nicely cancels and we get 1us e to the Lambda T now that is very interesting because that was exactly the answer to the previous question which was what is the probability that X is less than or equal to T so we see uh this is often called the memory lessness property I can actually write that right here so it's kind of a long word memory lessness okay this is called the memorylessness property of the exponential uh random variable which is to say that it doesn't care uh how long you've been waiting the amount of time you have left to wait is as if you just walked in which is kind of sad but uh uh it's the our calculation show that it's correct so it's saying that if you walk into that bank you walk into that window and uh you start waiting and you think you think to yourself just as you start waiting what is the probability that I have to wait less than or equal to 5 minutes and you get some number and that number is here with five plugged in for T um then you you wait you wait an hour and you're waiting an hour and the window is still not clear for some reason you think wow it's been an hour I must be the probability must be so high that it's less than or equal to 5 minutes now uh actually it's not it's the same probability you just calculated uh under the assumption that these waiting times are distributed exponentially now that's a very shaky assumption not really true but uh this is very important if you do a Plus on process or uh if if you do the Plus on process which we may talk about if there's time but um you you could look into that and that the memory lessness property is very important in that so uh needless to say the exponential random variable is a very important random variable in our uh in our whole structure here and we'll see how it relates to the gamma random variable when we talk about that guy