Live Coding Stream: ESports Earnings Data Analysis with Python

Key Takeaways

[Music] let's go hey how's it going everyone it is January 8th Sunday January 8th the year of 2023. we're rolling here we're getting going hope for the volumes okay let me know if the volume is clipping or anything uh how's everyone in chat I'm gonna lower down the music I know you guys aren't as big of a fan of it oh time for some data stuff says xcode it's related to OCR have a really new what are you guys talking about be Norma be Norma welcome to chat um would you be able to help me with a project I want to do for my internship I can maybe yeah for a little bit I want to do my own thing too but I'm here for you so how's everyone doing tonight it is a wonderful Sunday night I hope you enjoyed your weekend a lot of good football a lot of good fun with friends and uh you know we're here tonight on Sunday we're gonna have fun day now I I think I mentioned when I streamed on Tuesday that yeah be normal send me uh send me a link or something if if you want me to help but uh sun on Tuesday was the last time I streamed and I was feeling a little stuffed up and I mentioned it there and I decided afterwards just for for Giggles to give myself a test and it turned out um I was I was pretty I had the the thing so I'm almost back to normal now now it's Sunday and that was Tuesday yeah so it hasn't been that long I'm feeling pretty good but um we're gonna have a lot of fun tonight I uh want to show you guys this look at this it's my wife's better camera But the lens the lens is so zoomed zoomed in this is the best I could do so we're working on maybe getting a better lens for this one what do you think what do you guys think nobody cares this is just a webcam I always use just use a normal webcam so I was trying to switch it up hey we have AO we have the black zillion we have raphaelic we have xcode nice cam upgrade yeah I mean it's it's a cam we already had but I'm just trying to use it uh see if it would work if it would work like is my my voice in sync with the camera that's the main thing maybe I'll switch to this every once in a while so keep you on your toes look how blurry my hand is look at the depth of field in this one now that is legit that is legit this webcam just okay yeah I mean not not horrible but webcam we got some people on YouTube dude it looks it looks cool I feel like it it makes my face look more wrinkly it's like showing too much detail for dramatic moments like that okay let's get to something that you guys care about not me talk about webcams what do we want to work on tonight everyone let's let's not go to that let's go to this okay so we have uh some people in the chat we have they say I see no delay looks fine legit kaggle kaggle some people say kaggle um what do we want to do on kaggle [Music] I look younger on the drama cam okay then I'll take that I'll take that because despite me not wanting to it seems like every day I'm getting older what the what's the deal with that can I just be young kaggle or chess be normal wants chess I don't know okay what what on kaggle like kaggle is so General we want to do a data set data engineering pipeline someone says so let's go ahead on Twitch if you're not on Twitch right now twitch.tv let me get the link let's put that link in there make sure you go in there I'm gonna make a poll on there so let's go ahead and bring this out and figure out and we're going to get the infinite Loop going here of me how do I make a pull uh manage pull new poll what should should we work on tonight I don't know if you can read this all right um now last time we were we were working on IRS data what Jesse PK subscribed we got a subscription already I need to spin the wheel let's spin the wheel we have we barely even started what are you doing here look at this people are following thank you Vikram for following uh we got subscriptions like what like what did I do to deserve this and it landed on do nothing oh that's for you but I spun the wheel for you so you're okay so continue working on it was SEC data right not IRS all right uh we did no I don't know if I want to do pose uh create new data set like we could create a brand new data set from scratch put it on kaggle have I tried Game Dev no never uh 50 Quid SEC data um so last time we did we tried to run some polar stuff and I totally shut down my stream by overloading it with too much with too much uh uh running CPUs so I was in trouble uh okay so we can create a new data set we can work on the chess data set uh chess computer vision project that that it's been a while since I worked on we can try to get some ideas with that um what other ideas the chat have love your vids thanks will how are you doing tonight I'm just I'm just in a really good move I don't know why uh how much did I pay for my Hermann chair okay so the story about the Hermann chair I I think it was a definitely a few hundred but there's a place near where I live and they just sell like these these are like the chairs you're gonna sit in this chair your chair like most of your life people let's let's get to the dramatic camera so you can't see this chair in this so I'm going to switch to this one no I can switch to this one so you sit in this chair like a lot of your life so you want to have good ergonomics right so there's a place near my house if you live near me I can tell you the place but you probably don't but they they just it's just like a warehouse full of chairs they're all used because all these companies buy these chairs and then you just get a huge discount so that's where I bought this chair very good deal it was clean and I was thinking between like the C and the B and I got the C I think there's like different models of this chair and I kind of regret it my chair is free it's terrible yeah I'll I've also sat in terrible chairs for years but when I started working at home full-time I was like I need to invest in a legit chair uh Jose George says chess sounds good the chest oh oh chess computer vision stuff uh would love to see some data engineering too your videos help me a lot what what type of data engineering are we talking about here um one one thing I was thinking is like we could get some sports data um we could also what are some other data sets we could try to create hmm um the other thing is like like pose estimation stuff we could look into we did this on one stream but we never actually finished but we could look at all the state of the art pose estimation stuff um and I guess the only other thing I had was like Twitter bot stuff that I don't want to talk about that I don't want to work on that hey Catholic two welcome are you a lurker do I know you buddy buddy thank you so much I can't see your thing but I'm gonna spin I'm gonna can't see if you're a first time chatter make sure you chat if you do that loft is in the chat my chair is oh lopped is the one who said your chair is free do a project in pie spark I haven't used Pi supermark in a long time and later on do nothing let's spin it one more time oh wait yeah I wasn't even showing it okay so type Pizza 10 times okay let's go ahead and do this I've done this on stream enough times that I know the deal this is an ongoing typing of the word Pizza that I've done from the very beginning of streaming and I can do it 10 more times no biggie this is worth the sub thank you so much for the subscription my friend I like airflow Pi spark big query how do you sub how sub do you have on Twitch how many Subs do I have uh not that many I mean I don't do it for the subs I mean I'm streaming coding and data science it's not like the sexiest thing out there but you know we're gonna have fun Okay so here's the things we could work on it's now a poll on Twitch okay already gotta vote for SEC data some people want to get some sports data so I'm going to pitch each of these the SEC data we worked on before so we were trying to manipulate it and get it into like a workable tabular format so we could look at companies filings um across the years and see if there's interesting stuff creating a new data set would just be kind of like let's look at different sports leagues and what sports data is out there is there something cool we can find here's an idea here's something that I did for my college work was when I was in college it was one of our projects is I pulled in Reddit uh thread histories for each game in the NFL or might have been in college football and then you can associate each of those with the um people the team that they root for because people put Flair in the Reddit data set for which team they root for it that's kind of interesting data set and then um what could we do with that we could do like sentiment analysis of that because it's kind of texty um I I always I actually think that's a really good idea I mean that's winning and might as well try to work on that if you guys are a game I'm game um type in chat just like what your soccer data yeah yeah we could do football data too if people want to vote on that uh tabular data so that that data is I guess less tabular but we could we could like align it the interesting thing would be like getting the tabular data on the score of the game and trying to get the but then you have to get time stamps of when everything happened which is yes laptop soccer is football to the whole world except for the country in which I reside but that's also like that's how I feel I I grew up with football being like what I think football is that I can't take that away from myself UFC that would be cool UFC data set I think like whatever data set we pull from a sport it would be cool if we pull that data and then also try to get either something from Twitter or something from something from Twitter or Reddit like related to it because it's kind of cool to see the how the text uh associated with it making a video of you map or the tools for your dimensionality reduction visualizations technique oh umap is pretty nice for that like there's a lot of ways to it's almost like clustering showing it um the thing with those umap plots if if anyone doesn't know what that is okay so just so you guys know there's time remaining in this for you guys to vote um so it's like a dimensionality reduction to try to visualize there's umap and then there's tsne what these usually for me create are these blobs that kind of are helpful but I've never really been in a situation where I'm like a u map representation of the data changed anything like allowed us to make any business understanding or other than it being like oh that's cool that's not to hate on it I'm just saying these are the sort of things in data science you see a lot in projects but you don't see in practice because it is sort of like a fit predict you just run the fit you run the predict it kind of takes a while for umap um oh this is cool uh but then it's like what do I do with it what do I do with the fact that I see it in this representation like it's kind of it's kind of cool I I've used it many times before with sports data like uh I can show you guys past projects I've worked it what is rude what is rude what are you guys saying is rude your thoughts on Rust versus python in the next five years I think that rust is probably gonna be be big okay so data set Sports data set one yay I think that Russ at least seems like a good back end for stuff I think Python's probably still here to stay but on real big data pipelines I would love to see Russ take over something like Java Rob I'm super sorry am I welcome here PK Patricia you're always welcome here I don't know why you're asking that uh soccer will be more text on Twitter than Reddit I'm assume yeah potentially good evening good evening Mr Yan not understanding the lack of response I do subscribe what did I say this is actually rude oh PK are you really are you really actually upset with me I don't really get what's going on here with okay okay but please nothing that's too Sports heavy but that's what the vote that's what the vote was on I'm just doing what the people asked for um looks like k-means clustering in 3D I kind of kind of that's what is Russ seems way too heavy for data science we did try to use rust in one of the streams so um like if I go to my channel and I go to my lives we did we did rust for data science for an entire stream once and it was pretty cool uh but it was I didn't get very far I need to spend a lot more time learning it and I would need a reason to spend more time to learn it uh well enough to really be legit in Rust um all right so so we're talking about a data set uh Sports data set for sports and I'm sorry uh if anyone is not happy with that in chat but we're gonna go with what what the people ask for um now let's do another vote in sports so can you guys give me in chat like what type of sport we should look for data on and be specific about the league that we should we should look into soccer yeah but soccer like it would that be um new pool see what you're seeing in chat soccer la la linga in Spain okay why isn't the copy work on this there we go Premier League maybe cybersport DOTA okay we can do that I know there's some DOTA data sets already out there Premier League UEFA NBA people are saying let's put NBA on here um yeah cyber Dota 2 is a good idea but I don't know anything about DotA let's let's see if we find one Formula One okay I'm gonna do Formula One now Granite like some of this I don't know anything about NFL injuries that's my dating job uh Esports uh I'll put Esports on here so what's UEFA so this is just more more um soccer so which one of these should we uh the Premier League honestly is the only one I've I've heard of so I'm gonna be uh but I'm gonna put Esports on here as an option so let's get a view let's go ahead and view this poll WNBA okay uh UEFA UFC will be fun oh man I missed uh Skid Row I missed your comment sorry maybe next time all right we're voting on the type of sport that we're trying to get data for and then the first thing we're gonna do here's the first thing we're gonna do is we're gonna just look we're gonna do some worldwide web exploration are there already data sets pre-packaged out there I think that like three people that like said that what they want to vote on um are voting on those okay Premier League's taken over uh so if we look at the Premier League I think we'll just see what's out there on in in the metaverse in Facebook's metaverse out there on for early but we'll start with like seeing if there's anything or existing on kaggle and if it is there then we can uh can we do that can you share the link on YouTube I don't know if it would work to YouTube people what the link to Twitch I want to vote for all options you can't that's not the way it works bath thirst 1000 can be interesting I don't know what that is what is Premier League it's uh it's football it's uh soccer football Esports is starting to win we're gonna have to do another vote to figure out which esport to look into uh Esports in Premier League are neck and neck I'm I'm trying to think if it's Esports what sport would we focus on I guess DotA vote for Esports Silva wants it can you vote on Esports okay more people want Esports we do what the people want you can't vote on video okay so do do that you need to be on uh you need to be on Twitch it's just the rules of these polls so you have to join that League of Legends World of Warcraft Arena check which game is the most watched on Twitch I'm sure it's Dota or I don't know I don't know what year uh browse let's browse games these are just recommended so these are recommended for me because these are ones I've looked at um how do I view it in the order of what's most popular I forgot my twitch password oh man Call of Duty on Esports how's it going clipped we're trying to figure out we're trying to figure out us an ease it's looking like Esports is gonna win this unless someone votes like we get two votes really quickly for Premier League we're gonna get a data set so let's while this is loading let's look Esports on kaggle and just see Esports earnings Esports Ernie's dot this could be interesting how old is this data set two years maybe we could scrape a newer version of this from Esports earnings because this is just like all Esports and then we like figure out who are the top earners in Esports does that sound cool to you guys earnings looks cool that's nice it's like a generic uh data set for GTA 5 yeah that's not really a one that you can get much stats on looks like the the pole is pittering out and I haven't done any data science yet the pole is pittering out and it looks like Esports is gonna win so let's just assume Esports is gonna win and then we'll explore this data set um here's a question should we work on this locally on this data set locally or do you guys actually enjoy when I do it in a kaggle notebook because then I can share that with you guys at the end that's another question um I'm assuming this has ended yes Esports did win so we're going to look at Esports earnings this is it guys tonight guys and girls in whatever's we are going to look at Esports earnings data tonight that's just what we're gonna do because that's what you guys asked for and I give the people what they ask for and we're gonna start on kaggle and yeah people are saying kaggle yeah that's my second camera we were talking about earlier it's it's going to be my new camera it's my wife's camera basically I'm just borrowing it great videos thank you Silva kaggle notebook is better okay so hey let's not get ahead of ourselves let's just start a new notebook with this data set let's turn the Dark theme on so you guys aren't Blinded by the Light and we're gonna start working on a kaggle data set which is better this view this View this View I need to duplicate this we're just gonna have some fun tonight hello you're live and I don't have Legend I'm Brazilian thanks what does that mean Legend hello you're live and don't have Legend don't have a legend kaggle man it'd be nice to have it kaggle please okay so people are definitely saying they'd want me to do a nicago notebook uh that's my pizza tab which I'm gonna slow down and then let's get over here man I'm so unorganized so unorganized tonight okay so mad I'm actually not mad uh let's Zoom this out so you guys can see everything maybe make it a little wider for your viewing enjoyment look good like old man here just like all right let's zoom in e Sports earnings data analysis let's go let's go let's get it Esports Emoji let's get it Esports a video game emoji and let's copy this and let's put it in chat a bunch you can emote that I think and let's add it here to the beginning of our thing so it looks pretty let's delete all this so Esports earnings and analysis all right we're gonna add an image Esports earnings analysis let's see if there's like a good image for this no that's all boring uh Esports just not news copy this image address go to I need to make this my first tab or else I'll keep on losing it and then we can add this here as an image to make it super Epic um markdown image resize I always forget how to do this yeah so this is the way I can do it where I actually can resize this image so it's not super huge there we go make this about 500 with let's take take a look at the earnings from Esports now normally I would just get way ahead of myself and I think this would be going ahead of myself and and like trying to recreate this data set from scratch but we probably should just explore the one that already exists Granite granted it's a few years old let's open this in a new tab it's a few years old but it's gonna be better than us spending all of our time trying to scrape the internet um let's just also make sure there's not a newer Esports earnings data set on kaggle so I'm going to click specifically on data sets this is newer this one's newer so yeah let's click on this one oh yeah we gotta upvote these two upvote that and also up with this one but let's go ahead and add this Esports earnings to this notebook as well because it looks like it has more recent data it has two files only and little worse of a description let's get our chat over here chat chat chat where did you go gave up on data set Grand Master uh yeah I did kind of I just it was a goal of last year that I did not achieve that was definitely one of the 2021 goals it's so hard here's what I can vent a little bit about it but I feel like there are some there's some backdoor ways of getting data sets popular basically like joining Discord servers where you just post it and then everyone will vote it and I didn't want to I noticed that's going on and I don't want to be a part of that I want mine to be like organic maybe I'm just being too uh yeah maybe I'm just overthinking it uh this happens sometimes on the kaggle kernels where like it gets stuck to scrolled down and have to reload it um so this version let's look at the most recent version and then otherwise it's like really hard to get people interested in data sets that you you've created because no one goes on kaggle just to find a data set so like even the data sets that I've created by the way if you guys appreciate them you can go here and vote them but also these are all the data sets that I've created put this in chat speaking of marketing um but if you're not interested in this stuff you're not gonna like randomly go and start of voting this and then the ones that were popular like this flight status prediction one is very popular but here's the thing 107 people have upvoted it I think that just went up by the way 31 of them are non non-novevis so these are the only ones that count towards me getting to this medal which a gold medal I would need this to be 50. so it's actually this is actually further away from getting a gold medal in my data sets this one is further away than like say this exchange rate one because this was upvoted by more people who are on kaggle 47 so this is really close to becoming gold but uh thanks clipped for upvoting but you need an upvote and you need to also be more than a novice on kaggle because if you're just a novice it doesn't count um let's look at our input data um so this is the Esports earnings for players team bike game let's do some basic Imports here this is what we're going to use for data science pandas numpy import matplotlib import Seaborn oh geez I hate that sound let me move this um and then let's also this might not be a big deal but we'll make this like 500. yeah and that's going to be good enough oh from Global import glob this will let us grab all the files in this directory so by default on a kaggle kernel all the data sets are in this input directory which is One Directory up from where you're working this is Esports earnings players by game using glob on this will create a list of all the files oh no I need a star here that that's like a wild card character now we have the highest earning teams highest earning players in the country and continent codes list um so we'll go ahead and and make a teams data frame what are these files players and these are like the continent codes I guess all right so this is all just reading in the data you guys got it I'm trying to go like really make sure everyone's with me I'll need to upvote you while at work tomorrow thanks for doing that guys let me know in chat if you're there if you're watching if you have any input all right so teams so this is just gonna go do we have anything with dates you're watching okay just making sure I'm not talking to no one out there you need to post data on ML subreddits like our data sets and our machine learning yeah but they don't like that they don't like when you soft Market unless you're like helping out what I tried to do is make content related to that like that Airline prediction one I made like a multiple streams multiple streams I made YouTube shorts tick tocks talking about it but it's not going to drive people to kaggle to upvote it um all right let's look at the game and value counts on this all right Counter Strike oh so it's the top 100 of each that's what it looks like they've only given us the top 100 each and there's no dates so this looks like they just pulled what's the shape of this 1000 I don't know what we're looking at here oh teams 928 teams so it's like 100 top 100 teams for each game this must have just been taken at a at a like slice in time so this is data set that's data set number one I don't know if it's as promising it's it's not that exciting it's not that exciting and then we have like this codes which I think are just country codes yeah this has nothing to do with the data set so data set number one is a little bit of a dud now let's look up data set number two because that could be a little bit more interesting because it is from its earnings from 1998 to 2001 uh 21. whoa that's a lot of data you would think so let's list out all of this so it's just two files it's just two files General Esports data in historical Esports data um so we'll make a historical data set which will be this file zero and then let's also make a General which will be this one so let's look at this what's going on here can't decode position zero sometimes this requires encoding it's worth a try all right one thing other thing we can also do is look here what other people have done encoding ISO okay ISO eight eight five nine if someone in chats wants to let me know why it's this encoding I don't know I need this bracket all right historical General so 7 000 rows in the historical in general has 558 small data sets these are small data sets but we can at least explore them all right so this is I think at least has a little bit more information Okay so Warcraft 3 Smash Bros melee I know Smash Bros so let's look at that I know this game [Music] um there's 195 rows of it would be way better if you built it yourself yeah but we're seeing what's out there already Age of Empires 2 okay I'll look at that Let's uh let's also yeah this is not a so we'll do parse dates in here and we'll parse this date column now this is actual date time column and we'll sort values by the date so what is it here this like oh it's always the first day of the month and it's four six one three eight why is that such random intervals all right oh this is what other thing we have to do is something like this where so our plots will look nicer [Music] um let's do this earnings [Music] all right so this should be earnings over time so is this right competitions don't happen every day but do they always happen on the first day of the month I think this is just like a should be the month so is each row is each row one month let's read about this the inspiration came from this original data set which I'll uh vote and this data set just has similar stuff but it's older yeah the two years ago version uh version four split the data into two files historical Esports which contains their earnings per month for every game okay so it should be per month General Esports data which contains the generic data on each game also add another variable online earnings okay um why are you let's see how many games there are in this 518 games let's take the top 10 games that we have here and make this plot we'll make this exact same plot that way we can see what the difference in the games looks like and we'll put this here all right let's let's go ahead and see oh this is game so We're looping through each game and we're trying to see how the earnings change over time oh yeah yeah we need to set the index as the date or else it's not gonna look very pretty um would you consider yourself an average above average or expert David data scientist I don't know I'm just I'm me um because I'm coming from up on a year as an analyst and willing to move to data scientists I'm way below my skill level well I've been working on a while but I don't think anything I do is unachievable uh uh just asking to better understand where I need to strive to be at I think you need a you need to figure that out yourself man but you're asking the right questions um okay let's look at all these games all right granted the y-axis uh let's get the palette let's get this color palette um let's make this enumerate so we get a index and add a color here let's change this color didn't there we go it's gonna break once it gets to the okay so how to make a all right so we're gonna make this a cycler this palette and let's see if this works invalid color oh yeah so we're gonna have to do this all right so I'm gonna get I'm gonna get one of my old notebooks up here and get something to cycle over cycle some some image color to cycle over your work look at my own work sorry guys this is going to take one second we're about to be there we're about to be there what are we listening to on YouTube here getting all thrown out uh oh where is my stuff where is my stuff most votes um what I'm looking for is there's a way to have a color palette that you can infinitely iterate over this is an old way I used to do it and I know I'm not uh um finding it maybe this one probably not even worth it should I give up on this yes I found it I found it I found it I found it I found it this is what I want all right a color cycle there we go I think this is how I do it finally okay now it's gonna cycle over all these colors sorry for that big delay in time oh [Music] so you guys have better Solutions s chat gbt oh man you're right Seaborn palette and intertools.cycle yeah so now now we have a color cycle palette where we can just call next on it and it's always going to go through it I mean I don't need to do it this way like that I could have done it I think this way and it would work the same but let's just keep what I know let's keep what I know with how I know it and not question don't question my authority all right so the bad part about this is every time you run it the starting color might be a little different um but it's interesting here okay Starcraft early 2000s big time and then not so much Warcraft even earlier Super Smash Bros melee I didn't know that was so popular later 2016 and such but was that a did that exist back there when did that game even come out oh it came out in 2001 there must have been a reboot right a reboot of this game um so melee Peaks later and then what was this huge one we're gonna have to look that up like these outliers we have to understand why they're so huge Starcraft 2 earnings over time so Starcraft 2 is like now we start getting into the baby stuff let's just look at the top game month combined so history if we look at this and then we go date unless we like we need the year combined so if we look at the historic data and then we take the date and we pull off the year we make that its own column then if we take this history we Group by Group by our year and our game and then we take this earnings and we're gonna sum this so what does this get us this gets us the total earnings for a game in a given year and then we're going to sort values by yeah we're just gonna sort values ascending equals faults this will give us the top earnings if this data set is complete as a data expert what takes you more time during the day in your company cleaning data or you have data analysis data it really depends it's like modeling managing expectations uh meetings and uh uh data wrangling and entering like random questions and then also making slide decks with your results so you need to do that pop quiz question for use let me know when you're ready pizza Force go shoot hit me hit me Pizza Forest has a pop quiz for me can you define the central limit theorem without looking it up ready go no you win even if I could I don't want to do that right now I think you have you asked me these type of questions before sansar sanskar you got me you got me I'm not fresh out of college so I haven't recently researched that um okay let's do a head of 10 on this now let's do a head of 30. and let's make this a horizontal bar plot oh that's ugly um we can sort this again before we plot it the second time and what this is showing us is we probably want to flatten the index what this is showing us is the title will be top games top games uh game year earnings let's find an external data set and make sure that this lines up so 2019 fortnite made the most then 2021 DotA um does that make sense to you guys in chat is that like a reasonable thing that fortnite in 2019 would have the most earnings why is it so high for for that all right so top hundred highest overall earnings this is interesting uh this is the people who have earned the most player IDs this would be cool data to scrape history so now we're actually on the the website that has it top games Battle Grounds mobile has over three billion dollars oh wait this is in 2023 Apex Legends 20 to oh that's that's three that's three billion 2019 ninja was still big in fortnite too okay so the fortnite just really really peaked like in one year that's that's interesting um there must have been all right so let's make this a little bit cleaner let's take this reset the index and then um let's just make this a plotting data frame and we're gonna make it like the game Plus make sure I'm on a copyright three free audio copyright free audio that's what it says okay um so what we're doing here is we're going to take this year and we're gonna like make it a string and this needs to have a plus so now it's going to look a little bit prettier could we also make this that's Millions okay that makes a lot more sense I don't even know where I was Esports earnings that is millions is these yeah these zeros were throwing me off okay makes a lot more sense what is it like is Elon Musk sponsoring these games or something I thought it was just an error okay so League of Legends uh player unknowns Counter-Strike we could rename Counter-Strike this with CS go right is that what how people refer to it should be a comma we will make this a dictionary and it will make this all right so this will be game year I'm really missing my auto formatter here hey Corey Potter thank you so much do I know you have we met in chat before thank you for subscribing what made you want to subscribe what led you to finding my channel let's spin this for Corey I spin the wheel every time that someone does it and look it landed on push-ups I got to do five push-ups for you that's a good change of pace here that's a good change of pace let's let's check out the second camera that's still there oh that that's broken ah thank you for that uh not sure if rename worked let's look at here there's many ways I could do that rename I could also just map it uh yeah you're right it didn't because it has the year in it hey elk l-c-k-y-n thank you for the subscription too let me know in chat where you guys are uh Corey Potter said have been watching YouTube videos and want to say you were here you decided to come over watch on Twitch I realized I had a prime sub that's right there's a PL Prime is a free way for you to subscribe to the twitch Channel uh Channel if you would like to awesome thanks for subscribing we got we got type pizza oh man they're all coming in one two three four five six seven eight nine ten that's for you thanks so much for the subscription uh and then let's spin this one more time I'm gonna type it 10 more times it's like the universe wants this thanks guys you don't like I said you don't need to subscribe it if you're gonna take it from Bezos might as well do that so thanks empowered and uh oh whoa what do we got here all time record shattered thank you twitch fam you guys are you guys are the best seriously hype hype train in it I don't even know how that hype train works but I'm all for it okay let's get back to let's get back to this not that let's get back to this okay you can't see the wheel oh oh shoot you guys didn't see the wheel I spun it twice and it relying on Pizza twice but I do really appreciate you guys can't see the screen can you see it now um so thanks Corey Potter and also thanks lckyn and also think thinks Empower through design all right we're doing Counter-Strike Global Offensive and it's not renaming and someone says I do the string method on it key error game year yeah string Methods don't have rename I thought it was just like this and not as a dictionary it's basically we want to do this and take rename so this takes a dictionary with one-to-one values so maybe the thing is I'm not getting the correct game name so game value counts are there like spaces in it that I'm missing [Music] no looks good try rename columns but won't that actually rename the column and I want to rename the value within the column oh I know what I was looking to do yeah it's replay I mean I'm an idiot I'm an idiot okay so I was trying to rename and it's replace and I do it this way this is how I always do it yeah I'm totally new I totally knew about that yeah CS go now can we make any of these other names shorter let's also do like let's do it title so that like there's not all these uppercase lowercase mixes that'll make it look a little bit cleaner what's p u b g is that this okay so I'll rename this place don't want to double back slash here all right so it's looking looking okay I don't know why it didn't pick up the mobile version of this now is this is it a different game if it's a mobile version versus the real version so like maybe make this mobile I'm glad you guys are here to help me because I yeah this looks a lot better so now now they won't be so like huge when we plot it um and then we're gonna actually have to do instead of this group by we're gonna okay so now we're actually gonna take our plot DF plot set index as our date and then no set index as our game year and then earnings there we go that's a lot prettier now let's also do let's do something a little different here let's do uh let's do a with of one so that they're wider and let's do it Edge color of black that looks a little bit nicer and this should be top top game year earnings can we color these by their different the year isn't doing date format in the chart I'm not sure about that we can make the colors different based on the the game that might help us visually see this and that actually makes me think where's our plot DF uh DF plot okay we could actually use Seaborn for this foreign plot where the X Out axis is the game year and the y-axis is the earnings uh this is data there we go we're going to actually make this Y and make this X make it a little bit bigger our same dimensions is up here eight by ten and then let's oops um let's redo the ordering on this and then let's make the Hue or the color be the game well why did it make it super skinny maybe it needs to be the color okay so we need a color for each game uh the years in doing Dave these are the earnings of esport players by game year not the game earning right yes yes yes yes top Esports earnings by game year but what I would like to do is make the color of each of these different I could do it with matplotlib here we could we could actually take this plot axis and I think we can get like the the rectangles out of this and re recolor those individually or we can just try to use Seaborn to do it like we are here and it might be a little bit easier it might be harder um so this column basically the game how many unique ones we have we have eight so we can take this these unique values and we can map them I picture this data as a stacked horizontal graph with the game on the Y and the stack values of each year on The X that's a cool idea let's do that so then we don't need to filter down uh what we can do is take this historical data Group by game year earnings sum this again and then we kind of just want to unstack this plot kind equals stacked equals true uh no wait kind is going to be a bar and then stacked is equals to true this might be too crazy you know what I'm thinking because when we unstack this we are going to get we got we have to get a subset set of games recovering from coven and just want to drop in and show you some love thanks crazy fanatic command I mentioned at the beginning of my stream but on Tuesday I tested myself after the stream and realized that I also was suffering from covid so I'm five days in I'm feeling better A lot better right now but I hope you feel better too crazy fanatic man and don't be too crazy out there don't be too big and crazy of a fanatic but I appreciate you being here and hanging out uh let's make a temporary data set with this all right so now what we have is every year is a call uh column and every game is here we can sum this and then sort values and that will just give us this will get give us like we could take the top 10 of these index and these are the top games and we could take this temporary data frame and we can just only look at the top games probably still too much here we actually want it like this I think no we did want it the way it was before so this looks kind of crazy because they're all stacked on top of it so we want it to be um stacked equals true and then I do think we want uh we want a horizontal bar all right we're also gonna remove this Legend it's like a matplot lib Kind of Night I don't know if this is cool to you guys if you guys uh think this is interesting what we're doing but I do I do it's almost like we need each year in there that would have taken me a half hour yeah that's why you got to get good I mean I don't mean you got to get good but like being able to quickly manipulate data for visualizing it is really underrated skill because doing this also in Excel like maybe some people could load this data into Excel and do it that way but it's way less reproducible and if you learn how to do it this way you can make like notebooks that you send out and other people can reproduce What You've Done now the key here would be getting the year to show up in each of these bars because these I removed the legend because it's a little bit too much like we're repeating colors here um total Esports earnings by game I'm not super stoked about this visually I don't know about you guys you're stuck doing something similar using plotly yeah plotly can also do this this is the most I'm interested in now nice so I don't know I like this idea I think so what's cool what's cool about this we can see that playerunknown Battleground well also we didn't rename the [Music] we we could do our same game renaming here you can actually just do it to the base game right like in the entire data set before we even do any of this stuff what'd I do wrong what did I do I broke something oh it's oh I I loaded that but I didn't add this year column all right so let's just start at the top of our notebook and work our way down yeah that's where we added the year wait playerunknown Battleground mobile did not change the name what's what's going on here I thought I did that I don't know let's see where it contains let's locate where this contains that game so it's not in here it's not in the hist version [Music] when we plot it here is it ever show up no hmm I'm just I'm a little miffed about why it's not showing there but we could also just do this again here oh because it's in the index maybe because it's in the index again I don't know I'm getting a little confused here what I did anyone in chat wanna fix the star between unknown and S when you change your temp data it didn't have the rename I think what I tried to do here was just rename it when I first read in this historic data set and then this playerunknown Battlegrounds doesn't exist anymore right so then when I do a hist year over time and I create the subset like this stuff shouldn't matter anymore I should be able to remove this and get the same result well granted it's not going to work because like let's just temporary temporarily remove that yeah see there it didn't work and player and not unknowns anyone know why it worked for Counter-Strike but it didn't work in playerunknown's Battlegrounds mobile I know why I know why it's because I did this string title here and I did not do it in I did not do it when I'm was trying to run it here and you gotta be consistent about this stuff because computers don't joke around there we go and now we could actually remove this stuff we could just make this this like our games are now the right names and then here yeah Arena of Valor yeah so does everyone understand why that didn't work I was doing a rename of these lowercase one names assuming that it would propagate down but I forgot that I did a rename on the titled version of the text and when you these game names had all uppercase for playerunknowns Battleground so if I had this renamed with the uppercase version it would have worked foreign not totally excited about this necessarily let's look at individual earners I want to look at individual earners real Quicks did you transition from software Dev to data science no no I transitions from engineering to data science and I had done no software stuff before that um so this has the historical stuff but we also want to do individual people who earn money we looked at the top 10 games in the you know I think these plots would look better if it was Group by the year and summed am I crazy here I feel like this makes a little bit more sense why is Agent Empires make so much why why is Egypt Empire yeah and then I almost feel like this doesn't need to be a year this needs to be a bar plot for each year so we can like see each year and then and then what if we did this let's make subplots so the number of rows is going to be the length uh well it's just gonna be 10 and then the number of columns will be one and let's yeah let's go ahead and share X and then and then let's put this axis is this so let's iterate over each axis in this oh look at that beauty I know it's not beautiful really but it's gonna get there all right so let's do a fig size now it's got to be like a lot taller to fit everything in we have to do the names are kind of big should we share the y-axis because these bar plots are a little bit misleading how much did you have to drink before I started the stream two beers thanks for asking FIFA faux fum why do you ask was it when HD came out oh so they don't distinguish in this between the HD version let's look up oral Wikipedia HD version I don't I don't see it here I don't see any references to it it's got It's kind of cool to see like where each really peaked because some games Peak and then double Peak um it looks like we're almost on the same scale let's let's make the Y Lim equal to zero and what is this a million let's Group by game year earnings and sumness sort index sort values faults that all right so the top earning was that fortnite 2019 year and the earnings was we can do these underscores 72 million so let's make this 72 million and now we're gonna actually see everything on an equal scale jeez how is this possible thanks for sharing this is insightful thanks for hanging out fee-fi-fo-fum I just didn't get the beer comment if it was supposed to be encouraging or not I feel encouraged I don't know if it was intended to be that way I got water right here by the way hey let's try our second cam oh wait it failed my second camera failed look at this when I want when I want to make that second camera work this is what I need to actually run it's very convoluted invalid oh you know what probably happened it ran out of battery that's what I think happened um good energy oh thanks yeah I'm feeling good I'm feeling I like a good sleep last night so what I'm confused by is I think our top 10 list is bogus because our top 10 list is by the top yeah the top ones that exist so let's let's go by let's go by groupie game and then earnings I always do grope by I always spell it wrong um so actually our top 10 games will be this and it'll be like the first 10 of these these are the top 10 earning games and that's really what we care about not just the top 10 that exist in this data set so I'm going to re-plot this by the top earning games what happened to fortnite what happened to fortnite is this right e escharts.com just keep enjoying myself escharts.com what's ES no this is yeah yeah esportsearnings.com tell me what you're looking at what the heck is this thing hernia belt is this a targeted ad jeez keep it family friendly what are these ads someone know it's about fortnite in 2019 here's the bizarre thing is I don't see this peak for fortnite in 2019 here so what were we seeing why are we seeing it here be the highest something's off something's off why is it there the highest but then here it's clearly not it says 2014 here let's just get to the root of the problem a lot of that data dies off in 2017 is that because the no one plays games anymore so let's just look at fortnite earnings let's sort the values by the year yeah this doesn't make any sense why am i showing fortnite here peaking at the wrong time it peaked too soon it's because of this share X yeah so there's something that I assume I want peaking too soon uh there's something I assumed here which you know what happens when you assume and I made it out of you and me here by assuming that we had every year accounted for for each game one way I think we can fix this is if we do a year this might be a quick fix as type category that makes this year column not like a numeric value but a category yes but now we have yes now we can share x equals true but now we're gonna have every year we have every year and we don't necessarily care about every year look at all these games that don't make anything compared so what we're going to do here is can we query where year is greater than 2000 and 10 looks like no one was really making money prior to that yet still going to show all the years because we have it as a categorical here so scratch that yeah would have to create a bunch of fault zeros otherwise I mean one way we could do it here too is we can do this query of 2010 or greater foreign we can do this in steps like make this data that is queried with this date here and then make this year uh a category is not liking this oh I need to remove this assign idea all right it's kind of hacky but that way that way does work okay so now we see the spike for fortnite in 2019 which totally makes sense because that was our Peak year how did no one make money on fortnite I guess it only came out in 2018. do the query before the as-type cat I mean this way works I just made like a subset which is only 2010 dates and then ran it on those um the the thing is here this fortnight this Fortnight earnings is just blowing everyone else out of the water so maybe doing this y limit uh we're kind of comparing them to each other though when you look at it this way you can't really see how fortnite was really dominant over all the other ones so what do you guys think we we hard code the y-axis when I have high outliers sometimes it's best to go off the second highest for for the Y so you just like you curtail it you just ignore the outlier oh go a log yeah log skill is always an option all right so let's see what it would look like with log scale so basically what we would do then is we take so so here here's maybe we should look at it this way if we um so we have this like we have this history Group by game year earnings sum right we have this the year earnings basically we want to take this we want to plot a histogram with a bunch of bins to see what the distribution of this is yeah this is what I was expecting it's kind of even hard to see because yeah it's kind of hard to see but this is definitely like um uh uh highly skewed data set with some values way over here to the right so uh What's the title of this distribution of game earn of Esports earning for a year for a game year I don't know if that makes sense so most this makes sense like most games or Esports earners earned almost nothing in that year total Esports earning um but if we take the log transform of this so we take earnings or we actually take the value so let's let's reset this index so this is the log transform of it and this you do a lot of times in machine learning because you want to have a feature that's like a as close to a normal distribution normal distributions are nice so this is the log earnings um so we're gonna do it like this [Music] oh I keep on resetting the index which is no bueno [Music] Auto detected range okay so when you take the log of zero uh it's going to be an Infinity right that we know that from La math so we can use log 1p which will just add one to any uh to the values and then take the the version up okay so now let's look at the distribution what's the distribution look like now it looks a lot more normal so it's basically like um yeah then you have to think about this in terms of like orders of magnitude as you go up here like there there's a normal distribution around a value that people make and then there's like Tails either way that like aren't linear [Music] so now if we do this on a log scale I don't know the thing is about doing the log transform is it makes it a little bit less intuitive for the person looking at it I have an intro to ml's course this semester any recommendations for learning linear algebra uh [Music] I don't I don't have any good ones off the top of my head because I learned it also in school like I did a lot of linear algebra in undergrad and that was a long time ago for me yeah so this at least lets us let's just see like it's it's nice to see that this is a normal distribution when it's log transformed I think you can easily in matplotlib change something to a log scale uh matplotlib make axis log scale and then at least looks better because it has like the um foreign set y scale log let's remove this yeah so this actually looks a little bit better the way that we can so what's the max so you can kind of see like the the 10 to the six well I don't know why this is four times but it kind of lets us compare them side by side I don't know hey Rob any experience tips with Quant modeling and investment making not really specifically Larry I've never done Quant banking but I have done forecasting so if you look at my tutorials on YouTube for forecasting all right it's around 11 30. let's run this notebook all the way through we made some plots I'm realizing they're there is no individual earners on this because the data set does not have it um now let's look at Esports earnings and when I search for data sets of Esports earnings it just takes us back to kaggle so I wonder if the these people have an API I'm guessing they don't guessing also the data set isn't large enough that it wouldn't be that hard just to scrape support terms of use they do have an API obtaining an API key usage of an API key requires X does this sentence make any sense uh uh I don't know if you guys can see all of it but usage of the of the API okay usage of the API requires the use of an API key you can acquire it registering our develop Development Area so this is usually pretty easy and then if we wanted to take this further what we would want to do is actually gain access to this API and just scrape all the data in order to ensure the continued access fast the load times restriction API we will play you may only access the API at most once per second please store any information retrieved from the API locally and refresh the information every once in a while so that's reasonable one per second and what are these endpoints like so we could look up a single player every second we can look up player tournaments it's always funny when you have to kind of go through an API to gain access to a data set that's like they could have they could just give us it all it's not like it changes that often unless the historic data does change because we kind of want it like a select star from this entire data set forecasting is sexy okay fee fi fo fum so if you're gonna continue this project this is the route we would go I think we would we would get as much data as we could from um Esports earnings we would abide by their API and only request once per second but then we would just try to scrape as much as we could there's no reason to try to like go into the HTML code here or like go into this source code and try to pull this data when they're giving us an API because they're basically why they give us an apis because they don't want people to be accessing they know people are going to try to pull this data down and they're going to give us like a a way a sanctioned way that they're cool with for us to pull this data so let's say that you are you are trying to make this like super interests you let's say that like this is your jam Esports you're totally into it and you want to make something for your resume concerning this so you want to make make a project that you can at least like gain some experience from you might want to start by trying to use the API for this site picking maybe a specific game that you're interesting in this is only earnings diving into match data would be more interesting yeah you're right we kind of went down this rabbit hole but but that's I mean the same thing if there's a website for give me a game DotA you could use the API to do data streaming for live dashboard of their websites data yeah so you could like basically take this data and make it and make it prettier because they I don't think they've really invested it time in making many dashboards on this it looks like uh DOTA this is the wrong two DotA Dota 2 matches data sets so someone's already made this yeah this might have been this is three years ago I don't know enough about DOTA to to explore this data even I don't have the domain expertise okay Destiny two honestly when I when I suggested us looking into sports data I didn't I didn't think we were going to go down the Esports path I thought it was at least gonna be something I I'd know the rules to um this is kind of cool let's see what this person did they took Destiny data so they used the Bungie API like look at this this is a great data science project or project for you to work on if this is something that interests you they use the Bungie API give this a little uptick evenly between Xbox One and PS4 specifically ECA gathered their total time played total experience in total content so I don't know what yeah I'm not I'm not sure what this API is but I guess you would have to learn about this um you see I gather Turtle time played blah there's a clear pattern to how much time people play on each day of the week let's look at this okay these these plots could use a little bit work but I like it's not what matters uh so they're seeing when the spikes occur of I'm guessing this blue line is like a smooth moving average or something net change in glimmer I don't even know what this means and then he says he sees like there's some falling whoa yeah so you could see well this was back in 2015 but you could see the number of people using it false might be because like winter break I don't know it's getting nice outside and people don't play games as much efficiency of playing time this is really cool he's put the complete raw data set on an old Google Drive that's no longer existent but this is sort of this sort of stuff you could definitely yeah this is eight years ago I'm I'm realizing it but it was good enough that like it was the first thing that came up when I searched for Destiny data set gain inspiration from that though okay uh Dylan says really appreciate your videos and some of your things you've learned while watching you help me do land a research position with one of my professors after acing his pattern recognition class nice Dylan Dylan two thumbs up I love it like more than I've ever imagined the amount of people who who say that they've uh furthered their careers or gotten uh or gained some knowledge through what I've put out there and it's helped them in their careers that's that's what it's all about so Dylan you couldn't have made the stream end on a higher note let me just throw out some spam here so um we do have a Discord server exclamation point Discord to find that so I I paste it multiple times to make sure people in YouTube can get the link um you can check out my YouTube if you haven't already exclamation point YouTube will get you there but also um www.youtube.com [Music] should take you there um please subscribe if you haven't already and then I'm also on Twitter thank you in my senior year at UGA uh Georgia nice you guys beat Virginia Tech when I was in school though so kind of on my bad side no I'll I'll let you pass so I think I linked everything oh that's weird this is my this is my Twitter uh you can follow me there I've linked that I've linked my YouTube Discord what else is there twitch obviously if you're not go dogs baby yeah yeah oh man why do I keep on going that all right so that's it for me tonight let's find someone to raid let's read a rando late night reverse engineering is anyone doing python oh shoot I should show you guys this I keep on going that face one python happy new year let's do some [ __ ] is he doing python yeah let's read let's raid Matt see I'm not gonna stick around that long but if you're in twitch you want to join us in this raid please stick around if not thanks for hanging out um as always love each other be nice to each other and I will see you guys next time made it through covet it's been two and a half years or three years being worried about getting covered granted vaccinated and also it's like a different strain than when it came off but I can't believe I actually got it this week like I got it like that thing that we've been talking about for so long I got it all right hopefully I'll be around Tuesday uh check out in the Discord I'll let know if I'm not gonna be there but yeah thanks everyone for hanging out let's do the raid and then uh YouTube will sign off see you guys all right YouTube you guys have a good one too