Where do data scientists/analysts go next? Love and hate in data analytics (ft. Shashank Kalanithi)

Key Takeaways

people sometimes get uncomfortable when i bring up money so much right but in my opinion the conversation around money is very very important at the end of the day while we might enjoy our careers we do this for money hello everyone my name is sophia i am a data scientist working at a tech company many people are interested in becoming data scientists or data analysts however people really talk about where do they go after their data scientists or data analyst jobs many data professionals are actually thinking about transitioning to other fields doing product managers ux designers engineers or others today we're very fortunate to have a very special guest shashank who has just left his data analyst job shashank was a senior data analyst at nordstrom he also has a very successful data science youtube channel with almost 100 000 subscribers he's going to share with us his data science journey why he's leaving his job and where he's going next so keep watching if you're interested thank you so much for joining us today of course thanks for having me how is the weather in seattle it's amazing right now it's 78 degrees fahrenheit um which i think is in the it's like the 25 or something uh around 25 degrees celsius for uh anyone not in america and um it's sunny no clouds in the sky uh the wind is not that high so it it's you can wear whatever you want and you'll be comfortable oh i'm so jealous i think austin is over 100 today oh my god wow i was i was in dallas recently dallas is like that too so it yeah it's crazy over there yeah so hot okay so you are a youtube celebrity for those who don't know about you could you tell us a little bit about yourself what is your background and how did you get into data science yeah um so you know my name is sean calanothy i have a youtube channel um and i actually i found sophia through uh two vues uh youtube channel so sophie has some great content and i wanna have a feature on my channel at some point in time but my youtube channel is specifically about data science and analytics and how to break into the world of data science and analytics from a job perspective so um the reason i differentiate that is that for me the purpose of my channel what i want to accomplish with it my mission is to help people get good jobs in this industry um i specify good jobs because there are people who work in the industry um but are making about forty thousand dollars a year or something in like new york city uh and you know i wouldn't consider that a good job i want people to get like good well-paying jobs in this industry and i that mission kind of came about from my own uh past my own history so i graduated from emory university as you might be able to see from my shirt with a uh degree in chemistry and i had no interest in going into chemistry so you know i had this degree that i couldn't uh really use all that much and uh didn't know what to do with it so i happened to get an internship and uh at a company and when i got a job with them they they put me in the bi team and then i was like oh this seems like a really cool career uh from bi work myself up into a data analyst position eventually to a senior data analyst position and uh i will soon be taking a up a position as a senior data engineer nice so you're still in the data world just uh yes different focus yes yes exactly um yeah for me it was kind of like i so i want to create my own startup at one point in time and so there's two reasons that i wanted to move out of data and like data analysis into either data science or data engineering um so i was actually pursuing both paths simultaneously uh and i i got both positions uh and i chose data engineer um to clarify which what are the two positions you were pursuing so i was pursuing a senior data scientist position and a senior data and a senior data engineer position yeah yeah so i was pursuing those two positions because while data analyst is a great place to be in the industry at the end of the day the people who are paid the most in this industry are going to be the data scientists and the data engineers um unless you go into management as a data analyst obviously a data analyst manager uh can be paid quite a bit but um there is i would say there's a effective limit to the amount of money most companies are willing to offer their data analysts um even if they're doing data science work so as a data analyst as a senior data analyst i was doing some data science work i was doing some data engineering work um but until i had that title transition uh you know my pay could only be so high a good question so anyone that follows my youtube channel knows that i've been talking about getting into like date like becoming a data scientist uh for you know years at this point and uh a question people probably have is you know oh wait why'd you switch to data engineer so a large part of it was it was um a position that was offered to me uh after i did the interview and everything that you know i i think a lot of our careers kind of just also happened to be like what do people offer us to do but for me i want to create a startup at one point in time and i think that there are a lot of opportunities in the data engineering space for startups i think there's a lot of opportunities in data science as well but what what i kind of the way i've kind of like digested information as to what happened in the industry in the last like 10 years right so uh data science has been like a mainstream thing for about 10 years now like i think 2012 or something is when that harvard business review article came out uh the one that called data science the sexiest profession of the 21st century um a bit premature to call something the sexiest profession of a century in the first decade of that century but you know um when you're harvard i guess you can say whatever you want but um that that article came out and i think that a lot of people got really uh a lot of companies got really hyped up on machine learning on ai they're like we want to implement it everywhere we want to use it everywhere um because they'd seen what companies with really good clean data like netflix uh apple like these tech companies that like manage their data very well what they do um and other companies right like more legacy companies were like oh okay we want to do that same thing with machine learning and ai and everything and then i think around 2014 2015 is when everyone started throwing ml and ai everything and eventually they figured out that you know machine learning and ai more so than anything else follows the uh gi uh gigo principle garbage and garbage out um and so i think that there was a bit of a dip in interest in the field and now is where data engineering people realize that there's an entire data engineering uh base that has to be created before really good um at scale data science can be performed and so i kind of want to my timeline to start a startup is such that i think i'll be able to tap into that and i think some proof of that is snowflake snowflake is not it's not a data science tool it's a data engineering tool um with data science capabilities attached to it obviously but the core of its proposition is we make the data engineering easy you know what do you think so why do you want to choose data engineer over software engineer what do you think are the key differences um i think so for me i think a lot of it's also i just happen to be in the data space so that's gonna you know that colors my view on that a bit another thing would be data engineering gives me a good focus on precisely what i want to do um and at the end of the day like i'm coding regularly so i'm not gonna say i'm a software engineer but um i pick up a lot of those skills just as a part of my job and so what data engineering does is it gives me a very specific set of problems problems focus on which is kind of where i think if you want to make your own startup that's kind of like you need to have a specialization of something or sorry one method to go about it is to have a specialization of some kind and by having that specialization you'll encounter problems uh just in your daily you know business and whenever you encounter a problem you're like okay this could this could be a solution or i could create a solution for this that i can sell to people so i think um i picked data engineering because it is a it's a bit more focused in software engineering and so um that's kind of why i wanted to focus on that gotcha along the same line so we have engineer data analyst data scientist what are the differences like people ask about this question all the time since you have experience in kind of all of those what's your perspective yeah so i have i have what i would call a somewhat unpopular opinion on this um because i've talked to people that say there's no difference between the positions and i i fundamentally disagree with that um go to any company go to any hr department they pay their data scientists more than they pay their data analysts you know the in and people sometimes get uncomfortable when i bring up money so much right but in my opinion the conversation around money is very very important at the end of the day while we might enjoy our careers we do this for money you know uh along with maybe you know fulfillment and all the the other stuff exists too i'm not saying it doesn't and i hope you find fulfillment in your job to some extent but at the end of the day we're here to make money um so that's a little bit of a tangent as to why i keep bringing that up um i think if you go to any hr department and you see like yes data scientists and data engineers get paid more than data analysts that's enough proof that there is obviously a difference in the positions now obviously if you work in any of these jobs you can be doing work in the other jobs as well um so i i think that that's where people get confused they think just because as a data scientist you're doing some data engineering work or as a data analyst you're doing some data science work that all of a sudden there's no difference between the positions but you know the fundamental work is quite different i would say a data engineer their job is to create the systems pipelines and overall infrastructure uh set all of that up in such a way that a data scientist is not spending um all of their time just getting data ready for whatever ml project statistics whatever they want to do um they say you know like the data prep part is like 80 of someone's a data scientist job i think if you have an organization with really good data engineers um that percentage should come down a bit because the data should be cleaned ready to go uh and allow data analysts and data scientists to start analyzing performing their jobs um a data analyst is someone i would say is attached to the business in many ways so that doesn't so for example like every day or when i worked as a data analyst at nordstrom i was part of the it organization but at my first company i was actually attached to the supply chain organization um and i think a data analyst is someone who is um they're attached to the uh business in such that they have a stakeholder of some kind like a business stakeholder i have a video that's that should be out by the time this video is out where i kind of like go over a day in the life of a data analyst where i talk about the whole stakeholder relationship with um a data analyst and they will use sql and maybe some python and a bi tool like tableau or power bi in order to explore the data and then create and then maybe some basic statistics in order to explore the data put something together and then show it to the stakeholder i kind of see a data scientist more as someone who um focuses on a very very specific problem for which there isn't necessarily data for yet so for example a data scientist might be uh creating a model to model out um let's say you work at a car dealership right uh you don't necessarily know how many people are in the car dealership at any given time right because there's the date you just don't have the data for that but you can create a model that predicts how many people are inside the store at any given time or the dealership at any given time that model would be created by a data scientist instead of a data analyst so i kind of see it as a data analyst works with data that already exists a lot of the time and a data scientist works uh oftentimes creates models to create new data that you know uh didn't exist prior got it so it sounds to me like data analyst is given a problem in solving the problem with the data that's already existing whereas data scientists is maybe explore their own problems and questions and explore their own data questions that are well defined for them correct correct exactly um and that that difference is important right because that difference is partially what informs the difference in salary and it's what informs the difference in education requirements for these positions um oftentimes and this is a bit of an uncomfortable thing for people to understand oftentimes data scientists do require a higher level of education uh and a higher level like the position requires a higher level of concentrated mental rigor um than the position of a data analyst does uh not all the time but oftentimes but at the same time i think if you're particularly interested in working at working with business working more closely with the business itself then you actually will probably enjoy being a data analyst more so um the positions are different one's not necessarily better than another but i i people say that and i agree uh but i think we should also be very clear that like like data scientists get paid more money so like don't you know just just because one's not better than the other which is true don't think that you can uh ease as easily make as much money as a as a data analyst as you can as a data scientist um that's something i want to express to people because i don't think people are willing to just say that you know that's a good point and it's important you know if you care about money then you know become a data scientist or data engineer if you really want to have like or or management insi as a data analyst if you're management then obviously that's different but um yeah as an individual contributor um you know if you want to make more money these are positions you have to aspire to yeah well speaking of money there is also the career ladder in the career growth right what do you think how do people grow in different routes and trajectories i know like you basically change the swimming lanes to data engineer and engineering um but what do people normally do i think in my experience what people normally do is they actually just go up in their own swim lane for the most part um so anecdotally anecdotally i don't know many people who have been data analysts for like five years plus and then become a data scientist i know people have become data and so they finish college um they have like say a master's degree or something uh they become a data analyst and then within a year they're a data scientist uh and the whole thing is the only reason they couldn't become a data anal a data scientist right out of colleges the the company didn't want to give them that position with no prior experience um but funnily enough anecdotally i have not seen many people transition from data analysts if they've been in that position for a number of years into data science um and you know going through that transition myself i can say it's not an easy one to make because the jobs are different um the work you do as a data analyst is adjacent but not the same as that of a data scientist and so you have to put in extra work to get like to make that swim lane transition um yeah that's kind of like what my take on that would be i think most people probably just go up in their own swim lane uh so say you're a data analyst right then you'll start off as a junior data analyst then you know maybe an associate senior uh and then from senior you can choose to either go into management or staff um and then if your staff then you can go into principle uh these are kind of like the standard uh terminology used in the tech industry so it's like you know junior associates senior staff principal or you go into management um now that being said that you know a funny thing um someone really close to me mentioned that they don't work in tech they mention this really interesting observation where they said tech is one of those funny industries where individual contributors seem to get paid more than managers uh oftentimes um which is it's a funny observation they made yeah and in a lot of other industries managers always get paid more but uh in tech you can get paid more as an individual depending you can get paid more as an individual contributor than even a director sometimes um which depends on the company depends on the situation depends on your skill set um but it's a really interesting uh thing we have going on in our industry yeah that's why a lot of people don't want to become manager they just want to be ics forever exactly you you can make more money with less responsibility yeah um now obviously the advantage of going into management is if you're able to climb the corporate ladder really well then you'll far supersede any ic if you can continue to climb it but the people who get like stuck at those at certain levels those are the people that have more responsibility make less money uh and you know where a lot of people would rather just be an ic mm-hmm yeah um i mean from my experience i feel like a lot of my data analyst or data science friends are going into like either product managers row or uh ux designer role or software engineer raw data every engineer every scientist wants to leave and go into product management you're 100 right yeah yeah every engineer i know okay so why why do everyone do something else well so here's the thing right so i i was talking about this with my brother the other day and um there's a great book i read it's called the code um so it was written by a university of washington professor and it's about the history of um silicon valley and they talk about you know for example how boston used to be the original silicon valley how the department of defense is uh the one was one of the main contributors of um silicon valley existing uh it's really funny because silicon valley tries to dissociate itself from government when it's it's it's a spawn of government um and you'll see this in any country you know go to you know china india like all of the big tech hubs are they exist because of government um because who's gonna spend that much money you know it's going to be the government um but a funny thing you'll notice a bit of a side note is uh you know look at china india and the united states right all of their tech hubs are located very far away from the capital so shenzhen is you know exact opposite end of china from beijing um bangalore is india's tech capital it's really far away from delhi it's in the southern part of india versus delhi being the north and then obviously silicon valley is on the exact opposite side of the continent from um uh washington dc uh so sometimes people like to say that's an example of like how they want to stay away from government even though these are all all these uh places were drastically fueled by government spending in the case of like uh the united states department of defense funding um in california to fight japan during world war ii uh in india it's india's um uh i think it's called like the drdo it's their defense research organization basically um they fund bangalore because bangalore's specifically far away from pakistan and china um so it keeps the location secure and really far away from india's main rivals in the region uh and then obviously in china shenzhen was deng xiaoping created shenzhen as a free economic zone because um he was trying to move the country off of communism but he couldn't do that right next to beijing when where all the communist leaders were so he does it all in shenzhen where no one could you know back then you had to physically travel to like see stuff right so he puts it in shenzhen it's right next to hong kong um and it's really far away from the centers of power that way that transition is easier right like how do you it's the only way to convince the communists that this is a good idea um i'm reading a great book by the way right right right now on deng xiaoping highly highly recommend it it should just be the first biography if anyone's interested in it but um sorry the whole reason i went on that tangent was to say that the industry has transitioned a lot i think from where it used to be a couple of people who were extremely interested in technology because they were just nerds you know they just love technology um to where now people have realized that tech is a relatively easy way to make money and when i say easy i mean compared to the uh three other traditional careers to make a lot of uh guaranteed income right a doctor lawyer um finance uh a doctor or a lawyer the amount of schooling you have to go through is crazy and then once you work as a doctor or lawyer you work endless hours finance it requires less schooling but the like 12 hour work days that people in front office finance work and like investment banking and stuff like that uh are just insane um of those three professions technology you can make a 200 000 plus salary working eight hours a day you know uh and and even that like it's debatable if you even need to work that much honestly like obviously it takes effort to get in but um i would say tech is probably the easiest money out there today uh and people figure that out right so because people figured that out a lot of people who aren't actually interested in the technology started getting into the field and i'm just theorizing right they started getting into the field right what me my brother called the good student type um their you know their whole thing is like you know i get good grades i go to school i get good grades um give me a safe profession that'll make me a lot of money which is nothing wrong with that at all at the end of the day a lot of immigrant children so you know i mean i'm the child of an immigrant right their parents came halfway around the world not to see them risk their entire uh life um uh economic prospects on a very risky venture their parents say hey go do something safe you know um we didn't like fly halfway around the world peter just risk everything um and so i think you have a lot of engineers who actually don't care about engineering all that much um and so they want to create something and that's where product management comes in right because it's a much more creative space um and you can create the products that everyone talks about at the end of the day i think that's what a lot of people want it's a lot more creative than engineering can be so kind of the way i tied all this together right is that we we've come from the age of massive government investment and and the only people in silicon valley and all these other places are people who are very interested in technology just because they are it's a kind of an era where because there are so many great tech jobs that are relatively low effort compared to you know finance law and uh becoming a doctor um you have a lot of the good student types who come in and you know become engineers they don't really care about the engineering work and so they eventually move into product management so uh really really just do product manager the first round why do they have to like go to engineer first and then switch because they're the good student type uh the good student type wants a set of uh hard skills technical skills that they can fall back on in case their career fails um if you're a product manager uh obviously product managers are very skilled right but there isn't like a hard skill that the product manager has if you're an engineer right if you can code if you lose your job tomorrow you go on to upwork you can charge 60 to 100 an hour yeah um i mean obviously if you're not an h1b and something like that like you know h1b is very complicated um but the engineering provides a uh like especially coding provides a very strong safety net to where if you like there's unlimited opportunities for you basically uh provided you know i mean obviously when you start talking to people like like situations like you know kids having like they have to stay in a certain location that makes it a bit more difficult um but save all of that you know if if i were to lose my job tomorrow i'd just go on top work you know um and and i could make enough money on upwork to have zero change to my life the only thing that would change my savings would go down but you know at the end of the day it's a very safe path so i think that's why uh because engineering is still safer than product management right and i think companies hire a lot more engineers than yes managers as well and also i feel i feel like more and more product managers now know how to code and how to contribute to the production code oh and steve jobs i think steve jobs is also a really important part of this like if you think about it right in a way steve jobs is like the ultimate product manager and he's what a lot of people so he attracted i don't know if you spent enough time in these like um um they're called uh uh finance gurus or something right there basically there's these people that like like advertise these really scummy techniques to like make money and everything um so i'm a big personal finance guy so i'm like i i've seen enough of these people to just be like but i i've noticed that a lot of people worship steve jobs um without understanding how much of a genius a man actually was all they see is someone who didn't know how to code and change the world and they're like oh i don't have to build up any skills i'm not saying you have to know how to code to become a product manager um but what i'm saying is i think some people use um steve jobs as an excuse to not learn how to code um when knowing how to code is a great solution to becoming a product manager it's a great way to break into the field um reliably because at the end of the day we have a shortage of software engineers in the us it you know we don't have nearly enough and uh luckily in the united states we're willing to pay insane salaries to software engineers and data engineers um a great example of a great counter example is japan um extremely technologically advanced country huge electronics industry huge tech industry software engineers don't get paid that much over there um and you'll notice even if you've ever used japanese software it from a user experience perspective it's not the best thing in the world honestly um and if you if you've ever owned a japanese car you'll know that like the ux of those cars is like it's not amazing it's not as good as american cars uh which is why for example toyota move their entire um the people that make the central console uh for all the american cars like the tundra tacoma and everything that's all done in america now it's not the software's not designed in japan so obviously engine software and everything's still designed in japan but you know interesting i didn't know that yeah but they don't pay their software engineering as much in japan as we do here that was a whole point of me saying that yeah do you think product managers get paid as much as software engineers on average no i don't think so um i i think you can be but i think on average now again i i don't think anyone unless you're a lawyer or a doctor or finance i don't think any career with a large number of people inside it pays as much as software engineering um yeah like there's petroleum engineering but there's no security in that so right yeah and then here is data engineer like salary wise comfortable with software engineers i think software engineers get paid slightly more but yeah it's comparable i'd say um yeah you're looking at differences that aren't too much in my opinion and like if you're really the thing is if you're really like a go-getter i think you can make up that difference in just other stuff but i i don't think anything pays as much as a software engineer that's not like a lawyer a doctor a dentist um you know finance um yeah i i can't think of anything honestly like sales obviously but sales is 100 performance german so sales people make the most money they can make more money than anyone um they're lucky i guess um i'm sure luck has something to do with it but the the good thing is because their job is entirely based on commission um or mostly based on commission as long as they don't have a cap on it um i mean sales people can make seven hundred thousand dollars in their thirties because if they're just that good so um but it requires a very specific type of person right like i don't i don't think you can some personality types i think just have a hard time with sales um like introverts unlike me well and i actually think that there are sales techniques that introverts can use um it's just less obvious to be fair yeah yeah yeah totally so okay going back to why you chose to become a data engineer you say you wanted to open up your startup why is that relevant like what is your startup ideas i mean you might not want to share that with us but like what directions are you thinking so i don't have any ideas right now really like nothing that like is well flushed out enough to really matter um for me i think the most ideal thing would be to create something like snowflake that being said i think that there is a really big gap between my skill set uh and you know the people that created snowflake but i think that um like i mentioned earlier the opportunity for data engineering solutions currently in the market is bigger than the opportunity for data science solutions because i think what happened was again um data science only works well when the engineering is done appropriately when the data is ready to go and i think in most companies that's what's lacking currently um we need more people who are sorry we need more good clean well-organized big data available for most companies and that's where i think um there's more opportunity in the data engineering space today than the data science space but it's just a hunch right now these are just my observations um i don't have data to support that so um i i wanted to get into data engineering because it gave me the a larger field to play with in my opinion a great example of this is um there is there was this guy that joma tech interviewed um so if anyone not aware he's like the youtuber um he interviewed this guy and the video was not called something along the lines of like i got rejected by 22 year old ceo um that 22 year old ceo right so that guy he's the world's youngest self-made billionaire he's like 25 years old uh and he created a website or like a service right that basically i think it creates image data sets um like artificial image data sets for people uh or for like people to train their algorithms on to me that is a combination data engineering data science solution um that heavily biases the engineering side uh and it's a great example of data science needs more data to work with like like for example these artificially created data sets um then exist currently and i think it's called scale.ai or something i think that's what they're called um so yeah that's kind of why i wanted to go into data engineering i feel like there's more opportunity here than in data science uh and i i don't consider myself a particularly um i don't think i would be i might be shooting myself in the foot but i don't think i would be a good business owner um it like like just naturally right so i want to give myself the maximum amount of chance of success possible do you think data engineer also include like ml engineer ml ops those kind of skills are needed for data engineer jobs i think it's a separate job but because when i think ml engineer and ml ops right i think when i think about the not the pipeline but like the timeline of developing a machine learning algorithm right like you uh get the data ready um sorry you get the data ready you um build your model you deploy it right uh the ml ops and ml engineers work on the opposite end of the data engineers right uh and the data scientists in the middle so the way i think about it right is that the data is made ready by the data engineer the data scientists will build their model and the data and the machine learning engineer will go ahead and either convert that model into a c plus plus basically they'll make it like faster um and work better on production systems and uh the ml ops engineers well you know that's kind of their job as well it's similar but different and so i kind of see the two of working on like opposite ends of the spectrum so yeah there's just so many job families in the data science world yeah but like most people i think a lot of people still want to become data scientist or data analyst at their first job why do you think so like even when i interview uh i interview a lot of software engineer candidates and they tell me they want to become a data scientist it's just very confusing to me why do people still want to become data scientists i don't know i just feel like soft engineers may not need to be become a data scientist they can just do their job well and do data science around their jobs they don't need to become a data scientist to do data science right they could incorporate any modeling they want in the product they're building um yeah it's just confusing i have interviewed several candidates they all told me the same thing just like then why are you interviewing for a self-engineer job why don't you interview for the data scientist stuff right especially because i i mean i think software engineers get paid more than data scientists on average um obviously depending on the specifications yeah on average my understanding is it's software engineer data engineer data scientist data analyst data engineer data scientist like very close together i think it just depends on the day of the week um but whenever i look it up online usually that's kind of the order that it follows um yeah i don't know that that's a really good question i think okay so data analyst i think is fairly the answer is fairly simple it is a well-paying career that requires a relatively little amount of education to get into it when i say that let me be very clear i only have a bachelor's degree i don't have a master's degree i don't have any extra degree so i consider myself a very lowly educated person personally um my formal education is very low i only have a bachelor's degree um i don't think degree matters it's what you have learned uh outside of a degree that matters more i agree too um especially the again i know so like i'll put this way i am yet to encounter a significant number of people who have done their master's degrees and are like yes that was an amazing use of my time my money my effort no one is excited about their master's degrees um which is you know like obviously i'm i'm being encouraged by various uh important people in my life to go get a master's degree but anyone i talk to with one doesn't seem particularly enthused with it or they're an h1b so obviously i mean i'm a u.s citizen right so if someone's an h1b obviously i understand getting a master's degree is like just practicality it is incredibly helpful uh because it makes getting the h1bs a lot easier i mean for me that's not an issue because i'm a us citizen so you know the obviously the situations are different as far as a data analyst um yeah i would say it's a good paying career um that requires it doesn't require any special education to get into in my opinion except for you know you'd need a bachelor's degree but i mean you could make 170 to two hundred thousand dollars a year as a data analyst 200 on the high end um only with a bachelor's degree you know and i think you can go anywhere in the country and that's a good living for a single person um obviously if you go to new york you have a spouse and two kids um then even two hundred thousand dollars a year is like not you you're not ballin you're talking about the bay area salary i don't think austin yeah yeah yeah yeah so these are bay area um seattle new york salaries um yeah you can't go to dallas and expect money for sure uh the point is you'll be paid well enough to live a good life um wherever you are right so in dallas you might be paid 150k max um you know but well i mean that might be a little bit high and as far as data science i i think a lot of it has to do with the hype of um yeah again that part was over but it's still going on it's so weird yeah i think for people in the industry the hype is kind of over uh for people who are not who have not been in the industry the hype is still going on um and the thing is at the end of the day in the in the united states uh we're super lucky in that there's so much innovation going on over here but there's basically an unlimited pool of these jobs available still um because luckily for us as you know like for say you're born in the u.s right or you're a u.s citizen luckily for you um the u.s immigration system is so slow um that we we can't bring in the number of immigrants we need to fill in these gaps uh and educating the number of people and but but america has so much capital like we have so much money available here that we can invest in all these different companies and we need more and more jobs available um so it it's you like for example you would have the same problem in china except they have an unlimited number of people there so you know even without like even though china is not very immigration friendly um people don't get paid nearly as much as they do over here but but even then like i guess no actually no it depends so in china semiconductor semiconductor manufacturing so the beijing government said that that's like a very um uh core industry that they want to focus a lot of attention on and that talent doesn't exist in mainland china right now it's all in taiwan but what happens is because of that um companies in china are willing to offer quadruple salaries uh and a bunch of stocks so smic um which is like china's version of tsmc is uh they offer like quadruple salaries and higher salaries than than taiwan is for the same jobs because the talent doesn't exist in china yet it's going to take at least a decade to develop and really more than that um and so they you know basically try to hire immigrants and i think in any country you go to there are going to be uh jobs that just because the country hasn't focused on them uh the private sector needs a lot of but at the same time there aren't enough people for those jobs and i think data science is very much one of those positions um but i think i agree with you at the end of the day we don't i don't think we need as many data scientists as we need software engineers or we need like you know at this point data engineers i would argue we need more of them um but yeah that probably that harvard business review article was like very influential yeah i guess the titles sound good it has the scientist in the title but it doesn't mean yeah you just well and i mean i i debate that it's the sexiest profession of the 21st century i would actually say software engineering um like that is it is with only a bachelor's degree you can make in excess of 200 000 a year as a software engineer you know it's again these are these are bay area salaries um but i would also say in the post covered world um companies are offering bay area new york salaries for remote positions not all the time but some are like i have proof that er yeah i have proof that that is happening um and uh obviously this isn't like an all the time thing but you can find those jobs out there so you could take a remote job that they base the salary off of a bay area or a new york based person and just live wherever you want um that's a little amazing the jobs exist they 100 exist like i i know for a fact those jobs exist um so you know something to watch out for uh yeah i think i think the name of that article was wrong software engineering is the sexiest job of the 21st century relatively low effort relatively low education tons of money i agree okay go back to your data job what do you love and hate the most about your data science data analyst job data analyst working with tableau and power bi i hate working with bi tools um because well so i i haven't worked with power bi that much i'll be here but i use tableau and it the number of times i've run into like issues with just like basic like just basic issues with like getting data the way i wanted to i'll put it this way in the field today there is still not an end-to-end solution for data analysts maybe that's my startup idea over there so for example let's say you have tableau right um tableau only works well if you have clean data already this the the way it works with sql its data manipulation features are just suck they're not any good at all and tableau prep is like okay um and then because of that it makes the tool harder to use so you have to go into a sql database uh and i don't know anyone that's written a bunch of sql queries like even like like sql ides are just not nearly as good as like um the ones that they have for like coding and development maybe that's another business idea there too like a a better sql ide um and in my anecdotally i actually find writing sql queries more complicated than coding sometimes um because of like the order you have to do things in and it makes no sense and it's like an ancient language that hasn't been improved upon in like years um but yeah i would say probably working with the bi tools is the least favorite part of my job the most favorite part of my job is working in python um i just i love coding um i have a lot of fun doing it and it is always fun to see everything come together in a clean pipeline that's well documented with pie docs and everything um and uh yeah yeah that would probably be good oh environment setup that's one of my least favorite parts too i hate saying this you don't use conda i do use conda actually um so it's made my life a lot easier thank you yeah it's made my life way easier and it's what i recommend to people at the end of the day i say like this is like the standard data science stack um you know use conda for everything um but yeah it it's what gets really annoying is when you have like package conflicts and stuff like that which again it's it's good with dealing with but it's annoying that that exists as an issue you know but yeah i know even with condy can take a while yeah yeah and that's why i use mini conda i don't use the full anacond but i use mini conor for everything because it's a lot faster um yeah me too i think i think i recommend people to use meaning honda yeah instead of anaconda yeah um okay i want to talk more about your new job are you excited about your new job like what do you actually do as a data engineer so i'm not 100 sure what i'm going to be doing uh because i'll be working at a as a startup or at a startup right and working at a startup i think that uh you're gonna be wearing a lot more hats than you would if you work at like you know a larger more uh a larger more developed company for example but that's kind of the reason that you join a startup as well is it's partially for that so i imagine that i'll be helping set up the infrastructure for our data scientists to do you know what they need to do um so for anyone wondering i'm going to be working at bet fanatics which is a spinoff of fanatics which is a company that sells sports jerseys in the united states um actually we do in the united states europe india and uh china uh and japan so we work in quite a few markets and the um cool thing about that business right versus for example like they're not just clothes because there's only one uh official seller of a lot of jerseys and for most sports teams right so if you have the dallas cowboys only one person can sell their jersey actually in the united states usually only one person can sell a jersey for an entire league um so you basically have a monopoly on that entire business so it's a cool position to be in um and it also means that companies like the nfl and the nba are like partners uh with us and like we get their money to like do things and so we're expanding they're spinning off a section of the company called bet fanatics which is getting into the sports betting operations which in the united states it's a fairly new concept that's only just becoming popular are you a sports fan is that why you join i'm a bit of a sports fan honestly i'm more interested in the problem than anything uh because i'm more of an f1 guy right um and i personally don't gamble um i i or if i gamble it's like 100 200 in vegas like for my entire trip you know um i am way too fiscally conservative to get into gambling but um yeah i i i like sports a little bit and i like um and i like the psychology behind gambling i've always found it very very interesting even though i don't participate myself well that's good do you know what kind of text deck you're going to be working on not at all but in preparation i'm actually reading um data intensive applications uh which is what i've heard is kind of like a date like one of the data engineering bibles um one of the many books that like is written in data engineering that's like you know really great to read so i'm hoping to crush most of it by the end of next week and then on monday i find out precisely what i'm doing so the i could have asked ahead of time uh but i like to keep that separation before starting a new job of like don't ask for any work ahead of time even i want to get ahead right i'm like really like eager to get ahead but i'm like you're never like you don't get to experience that like that like i have no work no work to do right now uh it's a very rare feeling right that's amazing just yeah yeah so i i didn't ask my boss what i'm doing specifically because i'm like i will i i will go and try and do that and that's why i should not ask until day one let me enjoy my time off are you more excited or anxious about joining a startup i'm more excited um i think what really worked out for me a lot of people are anxious right because of the um a potential job insecurity part of it especially with the economy where it is right now uh people are expecting a recession to come up um i'm not an economic expert i have no idea what's going to happen um but i i will say that you know there's no reason a recession can't happen that's kind of like my logic towards these things like are there a recession could happen tomorrow why not um so i think a lot of people are more like they're more insecure about like their job prospects when they join a startup right uh which is totally fair and then i i should also say i'm very lucky or more than lucky i'm in a position in my life right now we're like i don't need that much money um i live by myself um and i'm not an expensive person quite honestly like i was actually just uh i mean i was just telling you how i cheaped out on my uh laptop that i bought for uh youtube and everything and i'm paying the price right now because i have like no storage uh for my videos um you know but to illustrate i'm just an expensive person right and i think what gave me a lot of the confidence to where i'm not anxious anymore is that i did up work for uh about a year and a half and you know i was able to basically just live off my upwork income um again i'm not a heavy spender right i don't spend a lot of money so um and luckily you can charge a lot of money for data science and data analysts and data engineering skills so i kind of did everything um as a freelancer is that the thing i didn't know i didn't even know you could do data science work like as a freelancer yeah yeah yeah if you find a good client on upwork then you know you're good to go um it's a great website you just have to be very very careful so upwork is a like it's a skill to make sure you don't take on bad clients because otherwise you'll hate your like life so much that you'll be like this isn't worth it um you need to find clients who don't question you all the time you know let you do your own work um there are easy clients and there are hard clients i'll put it that way and quite honestly as an extra job since i was doing it like outside of my normal work in my opinion hard clients aren't worth it um so like i had one client they were just very very difficult to deal with so i just said hey you know take your money back i don't need it like i'm gonna go do something else um that way i could focus on my other clients who are just way easier to deal with so i think the reason i bring up that story is that upward gave me the confidence in my ability to make money to where like if the economy crashes tomorrow i'll figure something out um there's going to be someone out there who needs data science and data engineering skills even if the economy crashes so and you're a youtube youtuber a youtube celebrity and i actually predicted youtube would make more money in a recession right because if you think about it right recession comes people lose their jobs which is you know obviously horrible they want to upskill this is what happened in the previous recession people want to upskill so they go onto youtube and try to learn all the skills more ad revenue for myself um in um what do you call it in uh the previous recession if you ever look at a graph of student debt in america over time you'll see it explodes starting in 2008 a combination of the federal government removing and state governments removing funding from a lot of schools and from americans wanting to get more educated in order to uplift themselves in the workplace um which is where you get to the current student debt crisis but the point i'm making is in a recession people will spend a lot of money to upgrade their skills yeah that leads to my last question i think how do you become a successful youtuber that's a tough question i think um it's a so a lot of people don't know this is actually kind of like the fourth iteration of this youtube channel um i i've done youtube like on three other occasions and failed at all three of them um and you just started over or did you just yeah i delete the old videos and i just start over um yeah and the same youtube channel no completely sorry the same channel yes but different content so like the first one i did was like a political science channel because i'm like you know i'm a bit of a policy wonk but i found out that i didn't have enough so this i did this back in college and honestly i just didn't know enough about politics to say interesting stuff about it uh and that was my first mistake with that channel uh my second channel was like a music channel where i just like did compilations of different like video game music and stuff that i didn't like heard uh and you know i got some views over there but you know you can never turn that into an income and that's where i kind of learned the idea behind like originality and stuff i forgot exactly what the third channel was there was a third channel but i forgot what it was um oh um sorry um i used to live stream uh to china a lot um so i would like teach english to people in china and so there was really yeah yeah there was an app called hua jiao um like the pepper um oh yeah i know that app yeah so i used to live stream there and then the chinese government said foreigners aren't allowed to live stream into china um so then i learned okay so like the chinese market is not accessible to me um because you know chinese government and all so so what happened is like you needed a chinese state id in order to like do it and so we had a friend we were using theirs but then we're like this isn't a viable business you know um so yeah sorry the four iterations of me doing um video based content that was it uh and then i started doing this and then this just seemed to stick right so i think um one technique is figure out if something is working if you're not gaining any traction try something else you know just keep trying stuff until it works um because i think it honestly is fairly random to be 100 honest like so much of it is just what it does it's just the algorithm i i would like to say hard work and dedication can get you there although like you can't really guarantee in this industry um which is why i would try different things you know until you find something that you're very successful with um and the cool thing is a lot of people strive to come become successful youtubers and then end up doing something else uh because their youtube gives them a platform to do something else so don't be afraid of changing your goals um if you grew up in that like in a household similar to what the one like my household right um there was a very heavy emphasis placed on hard work and on um not giving up hard work i think is always useful i think not giving up is not necessarily the best lesson in the world because i think you in in a world where if we had an infinite amount of time on earth then not giving up would be a tremendous technique but you all if you're lucky you only have 90 years to live um which is less time than you think it is you know um of which you know the first 18 you're you know just a kid uh and then the last 20 you're probably not going to be like you know out there just bawling every single day but uh yeah sorry so yeah giving up no one just give up it's it's another hard thing too to know when to give up because people attest their pride to it you know i i think it's okay to just say either i'm not good at this thing which is okay sorry i'm not good at this thing and i don't care to be good at this thing um because it's important again we have a limited amount of time on earth right you can't be good at everything um and i i think the most successful people are partially the people who like understand what they're not good at and they go do something else um or they they find someone to help cover up those like gaps for them right so uh yeah i think being successful on youtube there's a couple of lessons over there try different things know when to give up uh and don't be afraid to take different paths right so for example say you start a youtube channel about data science and teaching people data science uh and then you have a discord community and you actually find out your discord community is way more successful than your youtube channel don't be afraid to pivot into that or you find out that instagram is more successful you don't forget don't be afraid to pivot into that or like maybe you start a boot camp and your boot camp is way more successful don't be afraid to pivot into that um and then a fourth tip i would give is don't uh keep a very close eye on the amount of money you're making if that if you're interested in that almost anyone i know when they say make a successful youtube channel what they really mean is how do i make a lot of money on youtube you know i i'm not aware of anyone who's doing it for free like who don't monetize their videos for example um except if you're like gary tan or something like that like the uh angel i mean the uh venture capitalist who like does youtube on the side obviously that guy's a billionaire he's not trying to make any money on youtube um at least no substantial money and so i would say always keep an eye on how much money you're making versus how much time you're putting in money is not the end-all metric of life um but i think it's very important to have a very keen sense of where your time is going relative to the amount of money you're making because while money is not the center of everything money is very important to living a stress-free life or a life of minimal stress you know um having more of it is generally good as long as you're not you know kind of like just a jerk and trying to acquire it in my opinion um so i would say those are kind of the four things i i would focus on um try a lot of different things out know when to quit don't be afraid to pivot uh as in don't get married to any single concept and then uh keep an eye on how much money you're making relative to how much time you're putting in obviously there is an investment period um but i would say you know you can't invest forever like you have a limited amount of time make sure you use it wisely will you ever become a full-time youtuber probably not um i given the given the work that i do on my current youtube like i don't i wouldn't want to do this full-time personally um and more importantly my youtube channel is very focused on helping people get jobs in data science which i think is partially predicated on me having a job you know um i i'm not going to say that if you don't have a job like a like a formal job right um you can't teach people how to get jobs um but i think being in the industry is a important advantage you can have if you're teaching people about the industry so uh plus i like my work and i like my work and to be 100 honest it's i've reached an income level now to where it's difficult to replace that um so that that's kind of the problem like tech right once you make enough money it becomes very difficult to replace that income especially when you consider healthcare and like stocks and all that stuff then you're at a point where you have to make a lot of money on youtube to reliably replace that i think for me at the end of the day although i love my audience and i love my youtube channel i don't have an interest in doing this full time that's probably what it comes down to for me like i don't want to teach data science all day um unless it's tightly coupled with your startup business then it will be that would be different i i see this youtube channel potentially a marketing funnel for whatever startup i decided to create uh maybe yeah yeah i i could see it being like that so um no that that's a really fair point um i get this question a lot and i would say yeah my answer is probably not the one people expect uh because i think a lot of people start youtube with the intention of turning it into a full-time job which you know makes perfect sense um or people just hate their job and have to have an outlet yeah yeah and i i would say i'm lucky that i don't hate my job at least not yet um i could 100 see that happening at some point in time but for me probably the startup would be the outlook for it so yeah yeah looking forward to see your startup good luck thank you okay i think that's that's it from all my questions thank you so much of course and thank you so much for whoever is watching this video right now and i'll see you next time be sure to subscribe to sophie's channel i'll see you guys later yeah subscribe and like thank you