Animating Statistical Data in Python

Key Takeaways

what is going on guys welcome back in this video we're going to learn how to create professional bar chart animations in python using matplotlib in order to visualize real-world statistics like for example the gdp per capita by country like the one you can see here on the left uh but also other things like browser marketer and whatever you want to visualize we're going to learn how to create such graphs today so let's get right into it [Music] all right so for this video we're going to get started with an artificial example first we're going to make up some data in order to keep things simple and to focus on the actual animation process once we have done that we're going to move on to an actual data set provided by the world bank which is about the gdp per capita by country and uh we're going to take what we learned here with the artificial example and we're going to apply to a more complex real world example so in order to do all this we need to make sure that we have two libraries installed the first one is numpy the second one is matplotlib and if you don't have those two libraries installed you need to make sure that you have them by opening up the command line and typing pip install numpy and pip install matplotlib so once you have those two libraries installed you can import them by saying import numpy snp and import matpotlib.pyplot splt those are the usual aliases so np for numpy plt for the pi plot of matplotlib and the example that we're going to model here the artificial example is going to be quite simple you can choose whatever example you like we're going to make up some data now here so it doesn't really matter but i'm going to choose the example of browser market share so i'm going to pretend now for a second that we only have through three browsers and they make up 100 of the market which means that chrome firefox and safari in this case are going to have 100 market share together and then they're going to have their own market share and we're going to visualize how this market share changes of course again just artificial data so that we can focus on the plotting and then we're going to look at actual data so for that we're going gonna start here by saying labels equals and we're gonna define the labels uh to be the browser name uh the browser names and those are chrome firefox and safari and now we're gonna make up some market shares so we're going to say shares is actually a list of lists and those lists inside of the shares list are essentially for each year month or whatever interval you choose the individual market shares so we can start for example with chrome having uh 50 firefox having uh 30 and safari having 20 and then we can copy this so that we have uh let's say eight values here eight lists inside of the list and we can change the values here so from 50 to 40 staying at 40 going down to 30 going to 40 again 30 again 40 and 50. then here we're gonna have 30 40 40 uh 20 30 30 20 40 and then we're going to fill up the last column so that we have one in each row on this case 2 in this case one in this case one as well in this case five actually um here we have now three here we have four here we have four as well and here we have one again so those are just some values that we make up and we're going to have all of this inside of a numpy array so we're going to say np array of that list of lists so that we have a numpy array and the reason we want to have a numpy array is because when we apply an arithmetic operation onto a numpy array the operation is applied to all the individual elements and we're going to need that here in a second so basically when you take a numpy array and you add one to it you add one to each of those elements here and this is what we need to do for our example here um what we want to do in general here is we want to make an animation of these values changing and for that animation what you need to do is you don't want to jump from one value to another one you don't want to jump from 0.5 to 0.4 without a smooth smooth transition you don't want to be here and then to jump here in the visualization then to jump lower and higher again and lower you want to have a smooth transition that you go up and down with so you don't want to have this jumping effect you want to have a smooth professional transition and for that of course we need to generate values that are not here so we need to say if we go from 0.5 down to 0.4 we don't go from 0.5 to 0.4 we go from 0.5 to 0.49 then to 0.48 0.47 and you can even make this more granular to um to take smaller steps down so that you have an even smoother smoother animation you of course need to balance this with the speed of the animation you don't want to wait too long to get from one year to the next year for example or from one time section to another time section whatever the interval is here but this is the idea we want to generate steps we want to generate data that is not part of the data set we want to generate in between data if you want to call it that and first of all we want to create a list of years that all these rows belong to and for that we're going to say here years equals 2010 plus i for i in range and those are uh eight rows so we're gonna say here range eight so we have a couple of years that those rows belong to and then what we're going to do is we're going to uh create this uh array that is going to have our in between values our uh plot steps if you want to call it that so we're going to say plot steps equals np array and we're going to initialize that array with the first element of the shares array so we're going to say here a list is going to be passed that contains shares 0 and the reason we pass a list here is because we want to have the proper dimensions already this array should have a list of rows so we pass a list with one row so that we have the proper dimension so that we can append to that list later on alternatively you could also use the empty function or something like that and just specify the shape but we're going to do it like that and also we want to have this variable here called last shares this is going to have the previous row so we're going to say here shares equals 0. and the reason we want to have this is because we want to calculate a difference we want to have this row here and this is the next row and we want to know okay what's the difference between this row here and this row we need to take this row and subtract this row from it so that we know okay what changes from this row to this row and then we know what the step size is if we divide that thing by a hundred a thousand or just ten whatever um you want to have whatever granularity you wanna have but this is the basic idea here and for that we're gonna say now for i in range one and length shares the reason we choose one here is because we already have zero so we want to compare one to zero we want to compare this row to this row and because of that now we say that the differences are going to be shares i minus the last shares so this is what has to change and we can now say differences divided equals um 100 for example so that we know okay those are the step sizes that we need to take for the individual uh for the individual positions in the individual rows so once we have that we just need to apply that we need to save for j in range and we're gonna choose the same number here so a hundred if we choose a hundred here we need to choose 100 here if we choose 1000 here or 50 here we need to choose the same number down here as well because here we have the step sizes and here we have how many steps we take and they should be related to each other so if we have this step if the step is one and you divide the step by a hundred so you have a hundred values you also wanna or a hundred a hundredth of a step you wanna take a hundred steps so that you have one full step right i think it makes sense um and then what we're going to do here is we're going to say plot steps is equal to np append and we're going to append to the plot steps array the last shares plus the differences times j so we take that point that we had before and we move one such tiny step size times whatever we are here so if we have a hundred here this is going to cancel out a hundred times 100 times something divided by 100 is that something and before we reach that point we're going to get closer and closer and closer to that point uh so in the end we're going to have the actual value and before that uh we're not going to have the actual value we're going to get closer to that value now i'm not sure if it wouldn't make sense to actually have this to be 101 because i think that otherwise you never get to the actual value so i'm going to change that even though in my prepare code i have 100 but i'm going to try 101 because i think it makes more sense since when we define that a range is 101 it doesn't include 101 but it includes 100 whereas if we say range 100 it's only going to include up to 99. so then the last thing we need to do here is we need to say last shares is going to be equal to shares i so that in the next iteration we have one for last chairs and 2 for shares shares i right so we can take a look at that we can print the plot steps and you can see that we have actually oh i know why the problem is um what the problem is we need to provide the axis here we need to set a zero here in the end because otherwise we're not adding rows we're just adding values so it's very important when you call the append function that you actually provide the zero in the end here so that we know that we're adding rows and if we do that you can now see that we have individual rows for the individual individual steps so we start with this step here then this is the next step and so on and so forth and we move towards the next row having all that we now can move on to the actual visualization this is quite simple we're going to say that years is going to be an iterator over the years list and we're going to say that the year is going to be the next value from that iterator and then we're going to say 4 i and step in enumerate plot steps so that we have an index and the actual step we're going to say if we have this 100 step here so if we have a step number and i need to say equals equals zero if we have a step number here that is divisible by a hundred we have made one step and we can change the year so we can say year equals next years like that and then we can do the plotting and for the plotting we know that the x limit is never going to be uh we're never going to need an x value larger than one because the maximum value that a company or a browser in this case can have is one which stands for 100 and because of that we're gonna say from zero to one is the x limit and then we're gonna say bar h so horizontal bar it's gonna be the labels and the step that we're currently looking at and the color is going to be um in this case we're going to have green orange and cyan like blue basically um and then plt text is going to be placed at the bottom right we're going to say 0.8 0.2 for the coordinates uh and we're going to plot here we're going to we're going to write the text here and we're going to say also that the font size is going to be 14. then last but not least we're going to say pause 0.01 seconds plt clear figure and then once all of this is done plt show so that is enough we can run this and you can see now how the market shares change and i think that was it already no it was not that already but you can see how this changes you can see how the bars move you can see how the years change um this is the animation of the artificial of of the artificial example of the artificial data set uh so let's recap the code real quick before we move on to the actual data set to the actual real data set provided by the world bank uh we have labels we have shares for each row and one row represents one year we have the year values and here we create now steps in between the individual rows individual years we do that by saying we initialize the uh plot steps array with the right shape by passing a list of the first row here and then we store the last chairs here in this variable so the last just we looked at we calculate the differences we determine how granular we want to have the step size in this case we take a hundred steps for one year and we basically say okay one step has a certain size we divide that size by a hundred and we take a hundred such steps in order to get to our goal then we reiterate that process all the time until we have all rows done and then we basically just visualize the individual steps so now we're going to move on to an actual real world example by looking at the data provided by the world bank on gdp per capita by country over the years and if you want to have that same data set you can go to the world bank website or you can just google world bank gdp per capita data set and here on the right you can see that we can download the csv file i have done so already and this here is the csv file displayed in pycharm now one thing that is important here is you need to remove the first couple of lines because the original csv file has i think at the top description date and maybe a title or something like that and some blank lines the first line you want to have in your csv file is this one here with country name country code indicator name and so on um and then you want to have the actual data everything above that is useless for our purposes and also pandas is not going to be able to recognize the csv file as a data frame because of those additional lines at the top so speaking of pandas we also need to install pandas so we're going to open up a command line and we're going to type pip install pandas and once you have that you can say import pandas spd and then we can load the data frame by saying pd read csv gdp dot csv and we need to of course print this to see something and you can see now here we have the country name this is a column we have the country code and then we have the individual years um now we can also see that the years are starting from 1960 so we can actually generate here a list of years we can say years equals um we're going to use strings so we have the column names then 100 uh or not 119 60 plus i for i in range and in this case we want to go to 2020 so we need to provide uh 61 because the 60 is not included if we just provide if we just provide 60 that is that let's see if that worked this gives us from 1960 to 2020. that's perfect and we also want to specify which countries we are interested in because we're not going to look at all countries this would be huge and also for a lot of countries data is missing so we want to focus on major countries that are definitely uh represented in the data set uh sufficiently and we also want to focus on just a few of them so i'm going to focus here on countries equals i'm going to say united states canada germany france italy i'm gonna also go with austria which is the country i'm from and spain for example so those are the countries that we want to use so we're going to filter for those countries we're going to say now that the data frame is dataframe.query and we're going to write the following query here in quotations we're going to say backtick and we're going to have two backticks here the country name this is the column name has to be equal to at countries and add countries refers to this list here and if we do that and we print the data frame we should be able to see that only these countries are now represented in the data frame anymore so all the other countries are not part of the data frame anymore this is what we want to be working with next we're going to say countries equals df country name so that we have the series of the column of the country name column here stored in the uh countries variable so now we have a series and we also have the correct order from the data frame and then what we're going to do is we're going to say that the data frame is going to be um the country name plus all the years so everything that we have here this is why we had this as a string so that we can select the individual years columns we're going to have that and we're going to drop everything that is a nan and we're going to provide the axis once we're going to drop all the columns that have nand values so we only want to have years where all the values are available if we have if some value is missing in some of the columns we don't want to have it so we drop the full column so that we have a smooth transition we don't have any missing values um and this is what this command does so now we have this data frame we're going to lose a couple of years especially the early years but then i think from 1997 or something we're going to have everything until 2020 and then what we're going to do is we're going to create a new numpy array by saying data equals mpmt and the shape that we provide here is one and then seven because we have seven countries here now if you have eight countries you provide eight so one eight uh if you have two one two whatever you wanna have here the amount of countries you can also specify len countries if you want to and then what we're going to do is we're going to say four year in data frame columns starting from the second column because of course we don't want to have the country name and all the other columns are years four year in the data frame columns so 40 years that are still here after we filtered out all the nand values we're going to say that the data is going to be np append to the data we're going to append df that particular year to numpy so we're going to turn all the values from that here to a numpy array and we're going to append it here axis 0 so we have the rows if we now print that you're going to see what this is we now have a numpy array with the gdp per capita for each year uh for the seven countries and now we do the same thing that we did before so this is we we now have this this is now our shares list from the example before this is now the shares list the shares numpy array and we now just need to create the plot steps for that array right so what we do here now is we say plot steps this is the exact same procedure plot steps equals np array initialize with data 0 same as share 0 before and then we say last gdp is equal to data 0 and then we say 4 i in range starting at one going up until length of data we're going to say that the differences are data at index i minus the last jet at gdp and then we're going to say differences divided equals 100 and then we're going to say 4j in range 100. we're going to say that the plot steps to the plot steps are equal to np append to the plot steps we're going to append now the last gdp plus the differences times j so that we have these individual steps and again don't forget access 0 to add individual lists per row and then we're going to say just last gdp is going to be data i this is the exact same loop just that now we have last gdp instead of last shares and data instead of shares this is the exact same code so what we do next is we go for the plotting and for that we're going to define some colors so that we just have some pleasant colors to look at for the individual countries we're going to say ff5555 we're gonna say um five five fff55 i just prepared some colors too that i think look good then five five five five ff then five five ccff we need seven colors because we have seven countries ffaa55 ff55ff and then finally ffdd33 all right and now last but not least we we do the plotting itself so we say the years are the iterator over df columns starting from 1 because we only want to iterate over the columns that are still there so not the years that we don't have anymore and then we want to say 4i step in enumerate plot steps we're going to say if i modulo 100 equals 0 we're going to say try year equals next year the tries because we can't get the stop iteration error so we're going to say here accept stop iteration in this case we're just gonna pass so this is years not here um and then what we do is we say uh this this is now a little bit more complicated because what i want to do here for this visualization we didn't do that for the previous one is when two bars are equally long or one one is uh below the other one and then the one that is below the other one gains value and uh basically has now a larger bar than the previous one we want that bar to move up in the graph so we want the bars to change positions based on their length we want to have a sorted bar chart if you want to put it that way and because of that we need to copy we need to first sort the actual steps and we need to then sort also respectively the colors because we want to have the same colors for the individual countries and we also want to sort the country names so we don't want the barges to move in the plot we also want the um the labels to move and we want the colors to change respectively so that we have a nice effect and for that what we do is we say list underscore sorted it's going to be equal to step dot copy list sorted is going to be sorted here and we're going to then say list underscore index is going to be an empty list so the idea is that we sort the steps themselves and then we also want to know the indices how did the indices of the colors and the names change so that we can also rearrange those probably because we don't want to sort the colors because that's not the same as sorting the values we don't want to create a new order in the colors array we want to have the same order uh as in the step array so this is the only thing that we sort and then based on those index changes we want to also change the order of the other lists so list index is going to be an empty list and then we're going to say 4x in list sorted we're going to say list index dot insert zero list step dot index x so that gets the job done here this is giving us the indices telling us okay at which position um is the original element or or how did the original position change what is the new index in the new sorted list now of this element and what we want to do with that is we then want to say that the country names are the data frame country name series and we want to say that we want to turn this into numpy and we also want to say the same thing for the colors but first of all we want to order this country names so we want to take that list and order it based on this list index list so we're going to say ordered ordered names is equal to country names i for i in list index and we want to have also the ordered colors and this is going to be colors i for i and list index and all we need to do now is we need to call the map of the commands plt title gdp per capita plt bar h and we're gonna pass here ordered names we're gonna reverse that otherwise we're gonna have the reverse order we're gonna reverse the ordered names uh we're gonna keep the list sorted in the ordinary sorted way so without reversing it and then we're going to say color equals ordered colors and reverse this one as well then plt text we're going to have the year displayed as well at the position fifty thousand because seventy thousand is going to be the maximum value uh that we're going to allow for the x axis so 50 000 is going to be the position 0.2 is going to be the y position and we're going to plot the year the font size is going to be 18 i'm not even sure if that keyword has an effect to be honest but let's forget about this plt xlim is going to be from 0 to 70 000 as already mentioned then we're gonna pause for 0.001 seconds uh and then we're going to say plt clear figure and in the end plt show that is it if we didn't make any mistakes this should now work and it actually does so we can see that the countries are uh increasing in the beginning quite quite fast and um then it takes some more time than we have just some minor changes here but we can see united state estates is at the top uh you can see that the order changed now between canada and austria so they flipped uh the places and austria and canada flipped and also the colors flipped now they flipped again so this is the effect that i was talking about we also want to rearrange the colors and the labels because otherwise nothing would happen if we just sort the values so then the bars would have the uh descending size here but we wouldn't have the correct labels at the correct bars so you can see how this works we have uh countries growing faster growing slower changing positions and all that but this is how you do such bar chart animations in python using matpotlib so that's it for today's video i hope you enjoyed it and hope you learned something if so let me know by hitting the like button and leaving a comment in the comment section down below and of course don't forget to subscribe to this channel and hit the notification bell to not miss a single future video for free other than that thank you much for watching see you next video and bye [Music] you