Time Series Talk : ARMA Model

Key Takeaways

in this video we're gonna be talking about the ARMA model which is the autoregressive moving average model now if all these words seem kind of familiar from previous time series videos it's because we've already had a video talking about the auto regressive or AR model and we've had a couple of videos talking about the moving average or ma model it's just that in this video we're gonna be putting them together into this thing called the ARMA model so let's set up the situation and then we'll talk a little bit about what the components of the ARMA model the most simple type of ARMA model would be as well as if you see a time series in the wild how can you identify what kind of ARMA model it might be so the setup in this video is that you are a light bulb manufacturer and your challenge every month is to figure out how many light bulbs you should make to meet your demand and that value is gonna be L sub T L for light bulb and that subscript T is just the given month you're in how many light bulbs should you create so you decide to model your number of light bulbs using an ARMA 1 1 model so I'm gonna fill in the blanks here with one one and an ARMA 1 one model is the basic ARMA model it's pretty much the minimum you need to have an ARMA model and this one comma one corresponds to the AR and the M a so the first one corresponds to the order of the autoregressive part and the second one corresponds to the order of the moving average part and as you might expect from the autoregressive video in the moving average video the ARMA 1 1 process is pretty much just a combination of all the terms you would see in those individual models so that is to say if we want to read this equation it basically says that L sub t or the number of light bulbs that you're gonna create this month is going to be equal to some coefficient beta sub 0 this is just a constant okay that's the constant beta sub 1 is a different coefficient and then it gets interesting which is L sub t minus 1 which is the number of light bulbs you needed to create last month so this is the autoregressive bit which is if we were to just stop here then this would be a AR 1 model because you're basically saying how many light bulbs do I need to create this month is a function of how many light bulbs I needed to create last month but of course we also have this ma one bit which is this part here which says that not only is it a function of the number of light bulbs I had to create last year it's also a function of this coefficient this V sub one is a coefficient and it's a function of epsilon sub T minus one which is my error from the previous time period from last month and recall that this basically says that last month I made some prediction about how many light bulbs to create of course my prediction was off in the end by something whether it was positive or negative this error is epsilon sub t minus one is how much I was off by in the previous period so I'm incorporating that error that how much I was wronged by into my new prediction for this month and there's a little bit distinction to be made here this would be the process itself and I have epsilon sub T at the end so this is the error from this month now if I want to figure out what what would I make for my prediction this month it looks very similar but of course it's L sub T hat remember in statistics or time series L sub T hat or anything hat is your predicted value whose real process is given by the thing above but of course we don't know the real process because if we could predict the error from this month we would pretty much have all the information we need to make a perfect prediction which is obviously not true so my predicted value for light bulbs created this month is gonna be equal to this coefficient beta sub naught plus the coefficient beta sub 1 L sub t minus 1 I do have access to the exact number of light bulbs I needed last month because last month is in the past and I do have access to past knowledge plus V sub 1 epsilon sub T minus 1 I also have access to my error from last month because it's past knowledge but I basically stop here I don't have access to the error from this month because it hasn't happened yet so my prediction would basically given by this function right here and that's how you would use an ARMA 1 1 model to make a prediction about something now it's probably pretty easy for you to see if I change this one to like a 3 if I change this other one to like a 4 what would change in the model instead of having just a light bulbs last month I would have less light bulbs last month last two months ago three months ago four months ago whatever I want from my AR lag my AR or Goerke and similarly for my moving average a bit I would have not only epsilon sub T minus 1 which was the error last month I would incorporate the error from 2 months ago from 3 months ago from 12 months ago whatever I set up as my order of my hem a bit this is just the simplest one to get us started now to close out this video of course we want to talk about if you just see a time series in the wild you get some data from online or wherever you're getting it from how do you figure out what order you should set for the AR bit and the MA bit now this is going to be going back to some concepts from previous videos and the concepts of autocorrelation and partial autocorrelation so if you haven't checked out those videos make sure you give that a watch to understand what this means but as we did before ACF helps us identify the MA order and P ACF helps us identify the AR order and without getting too much into the weeds again the intuition behind that is PA CF measures direct effects which is what the autoregressive it would do and ACF measures the moving average bit okay so if I were to draw a couple spikes in here let's say they look like this for a CF and they look like let's say this for P a CF and these error bands these dotted pink lines basically say anything within those bands we don't think is statistically different from zero we have no evidence to say it's different from zero so basically in the ACF remember anything that's outside those bands would be telling us the order of the moving average bit so here that would be 1 + 2 + 4 P ACF anything outside the Bands tells us the order of the autoregressive which would here would just be 1 so a good candidate for give given this information of the ACF and PA CF of this model would be an ARMA the AR Umbra comes from PA CF being one and MA comes from a CF which here would be to an ARMA 1 2 model okay and I do want to say take this with a grain of salt there are certain exceptions this is a good rule of thumb if you see a CF and PA CF but there's other kind of more nuanced rules you can take into effect into account when you look at ACF and PCF about how fast these lags are shutting off and on but that is really the topic of a whole separate video this is just kind of to get you started a rule of thumb if you want to figure out what order should my ARMA model be in terms of what's the AR and ma order then look at a CFP ACF and see if there's a natural shut off after a certain number of lags in either one and that'll inform you of the total order okay so I hope this kind of helped bring the AR and ma bits together into the full ARMA model that we have in time series okay so until next time