Machine Learning Engineer Full Course - 10 Hours | Machine Learning Roadmap | Edureka

[Music] hello everyone and welcome to this video on the machine learning full course by edua Machine learning is a subset of artificial intelligence that enable systems to learn from data identify patterns and make decisions with minimal human intervention its importance lies in its ability to process and analyze vast amount of data quickly and accurately leading to more informed decisions and automation of complex tasks machine learning is used in various Fields such as healthare for disease prediction and personalized treatments Finance for fraud detection and algorithmic trading retail for customer segmentation and recommendation systems autonomous vehicles natural language processing like chat Bots and many more transforming Industries and driving Innovations with that said now let's outline today's agenda for this machine learning full course we'll start by understanding what machine learning is follow by supervised unsupervised and reinforced machine learning techniques next we'll explore the path to becoming a machine learning engineer highlighting the necessary steps and skills required for this exciting career then we'll explore essential algorithms like linear regression logistic regression and Deion tree which provide a solid foundation in machine learning models moving on we'll get hands-on experience with random forest and Ker's neighbors algorithms enabling us to apply these techniques to real world problems next we'll cover classification techniques such as Nave based classifier and support Vector machine which are essential for various predi of tasks after that we'll discuss clustering algorithms like K means and agglomerators we also cover key Concepts such as linear algebra probability and statistics in machine learning we'll then learn about cloud-based machine learning platforms like Azure and AWS essential for deploying and scaling machine learning models lastly we'll highlight essential skills for machine learning engineer and the top tools and framework used in the field we'll conclude this full course with frequently Asked machine learning interview questions and answers but before we begin please consider subscribing to our YouTube channel and hit the Bell icon to stay updated on the latest tech content from edura also visit the edura website for the machine learning course master's program the link to which is given in the description box below now let's get started with our first topic what is machine learning as you know we are living in a world of humans and machines the humans have been evolving and learning from the past experience since millions of years on the other hand the era of machines and robots have just begun in today's world these machines or the robots are like they need to be programmed before they actually follow your instructions but what if the machines started to learn on their own and this is where machine learning comes into picture machine learning is the core of many futuristic technological advancement in our world today you can see various examples or implementation of machine learning around us such as Tesla's self-driving car Apple Siri Sofia AI robot and many more are there so what exactly is machine learning well Machine learning is a subfield of artificial intelligence that focuses on the design of system that can learn from and make decisions and predictions based on the experience which is data in the case of machines machine learning enables computer to act and make data driven decisions rather than being explicitly programmed to carry out a certain task these programs are designed to learn and improve over time when exposed to new data let's move on and discuss one of the biggest confusion of the people in the world they think that all the three of them the AI the machine learning and the Deep learning all are same you know what they are wrong let me clarify things for you artificial intelligence is a broader concept of machines being able to carry out tasks in a smarter way it covers anything which enables the computer to behave like humans think of a famous Turing test to determine whether a computer is capable of thinking like a human being or not if you're talking to City on your phone and you get an answer you're already very close to it so this was about the artificial intelligence now coming to the machine learning part so as I already said machine learning is a subset or a current application of AI it is based on the idea that we should be able to give machine the access to data and and let them learn from themselves it's a subset of artificial intelligence that deals with the extraction of pattern from data set this means that the machine can not only find the rules for optimal Behavior but also can adapt to the changes in the world many of the algorithms involved have been known for decades centuries even thanks to the advances in the computer science and parall Computing they can now scale up to massive data volumes so this was about the machine learning part now coming over to deep learning deep Lear learning is a subset of machine learning where similar machine learning algorithm are used to train deep neural network so as to achieve better accuracy in those cases where former was not performing up to the mark right I hope now you understood that machine learning Ai and deep learning all three are different okay moving on ahead let's see in general how a machine learning work one of the approaches is where the machine learning algorithm is trained using a labeled or unlabeled training data set to produce a model new input data is introduced to the machine learning algorithm and it make prediction based on the model the prediction is evaluated for accuracy and if the accuracy is acceptable the machine learning algorithm is deployed now if the accuracy is not acceptable the machine learning algorithm is trained again and again with an argumented training data set this was just an high level example as there are many more factor and other steps involved in it now let's move on and subcategorize the Machine learning into three different types the supervised learning unsupervised learning and reinforcement learning and let's see what each of them are how they work and how each of them is used in the field of banking Healthcare retail and other domains don't worry I'll make sure that I use enough examples and implementation of all three of them to give you a proper understanding of it so starting with supervised learning what is it so let's see a mathematical definition of supervised learning supervised learning is where you have input variables X and an output variable Y and you use an algorithm to learn the mapping function from the input to the output that is y equal FX the goal is to approximate the mapping function so well that whenever you have a new input data X you could predict the output variable that is y for that data right I think uh this was confusing for you let me simplify the definition of supervised learning so we can rephrase the understanding of the mathematical definition as a machine learning method where each instances of a training data set is composed of different input attribute and an expected output the input attributes of a training data set can be of any kind of data it can be a pixel of image it can be a value of a database row or it can even be a audio frequency histogram right for each input instance an expected output value is associated the value can be discrete representing a category or can be a real or continuous value in either case the algorithm learns the input pattern that generate the expected output now once the algorithm is strain it can be used to predict the correct output of a neverseen input you can see an image on your screen right in this image you can see that we are feeding raw inputs as image of Apple to the algorithm as a part of the algorithm we have a supervisor who keeps on correcting the machine or who keeps on training the machine it keeps on telling him that yes it is a apple and no it is not an apple things like that so this process keeps on repeating until we get a final train model once the model is ready it can easily predict the correct output of a neverseen input in this slide you can see that we are giving an image of a green apple to the machine and the Machine can easily identify it as yes it is an apple and it is giving the correct result right let me make things more clearer to you let's discuss another example of it so in this Slide the image shows an example of a supervised learning process used to produce a model which is capable of recognizing the ducts in the image the training data set is composed of labelled picture of ducts and non ducts the result of supervised learning process is a predictive model which is capable of associating a label duck or not duck to the new image presented to the model now once trained the resulting predictive model can be deployed to the production environment you can see a mobile app for example once deployed it is ready to recognize the new pictures right now you might be wondering why this category of machine learning is named as supervised learning well it is called as supervised learning because the process of an algorithm learning from the training data set can be thought of as a teacher supervising the learning process we know the correct answers the algorithm iteratively makes while predicting on the training data and is corrected by the teacher the learning stops when the algorithm achieves an acceptable level of performance now let's move on and see some of the popular supervised learning algorithm so we have linear regression random forest and support Vector machines these are just for your information we'll discuss about these algorithms in our next video now let's see some of the popular use cases of supervised learning so we have cotana cotana or any other speech Automation in your mobile phone trains using your voice and once trained it start working based on that training this is an application of supervised learning suppose you are telling okay Google call Sam or you say Hey Siri call Sam you get an answer to it and the action is performed and automatically a call goes to Sam so these are just an example of supervised learning next comes the weather app based on some of the prior knowledge like when it is sunny the temperature is higher when it is cloudy humidity is higher any kind of that they predict the parameters for a given time so this is also an example of supervised learning as you're are feeding the data to the machine and telling that whenever it is sunny the temperature should be higher whenever it is cloudy the humidity should be higher so it's an example of supervised learning another example is biometric attendance where you train the machine and after couple of inputs of your biometric identity be your thumb your iris or your earlobe or anything once trained the machine can validate your future input and can identify you next comes in the field of banking sector in banking sector supervised learning is used to predict the credit worthiness of a credit card holder by building a machine learning model to look for faulty attributes by providing it with a data on deliquent and non-eloquent customers next comes the healthcare sector in the healthcare sector it is used to predict the patients readmission rates by building a regression model by providing data on the patients treatment Administration and readmissions to show variables that best correlate with readmission next comes the retail sector in retail sector it is used to analyze a product that a customer buy together it does this by building a supervised model to identify frequent item sets and Association rule from the transactional data now let's learn about the next category of machine learning the unsupervised part mathematically unsupervised learning is where you only have input data X and no corresponding output variable the goal for unsupervised learning is to model the underlining structure or distribution in the data in order to learn more about the data so let me rephrase you this in simple terms in unsupervised learning approach the data instances of a training data set do not have an expected output Associated to them instead unsupervised learning algorithm detects pattern based on on init characteristics of the input data an example of machine learning task that applies unsupervised learning is clustering in this task similar data instances are grouped together in order to identify clusters of data in this slide you can see that initially we have different varieties of fruits as input now these set of fruits as input X are given to the model now where the model is train using unsupervised learning algorithm the model will create clusters on the basis of its training it will grow the similar fruits and make their cluster let me make things more clearer to you let's take another example of it so in this Slide the image below shows an example of unsupervised learning process this algorithm processes an unlabelled training data set and based on the charactertics it groups the picture into three different clusters of data despite the ability of grouping similar data into clusters the algorithm is not capable to add labels to the group The algorithm only knows which data instances are similar but it cannot identify the meaning of this group so now you might be wondering why this category of machine learning is named as unsupervised learning so these are called as unsupervised learning because unlike supervised learning ever there are no correct answer and there is no teacher algorithms are left on their own to discover and present the interesting structure in the data let's move on and see some of the popular unsupervised learning algorithm so we have here K means a prioria algorithm and hierarchal clustering again these are just for your information sake we'll discuss about these algorithms in our next video now let's move on and see some of the examples of unsupervised learning suppose a friend invites you to his party and where you meet totally strangers now you'll classify them using unsupervised learning as you don't have any prior knowledge about them and this classification can be done on the basis of gender age group dressing educational qualification or whatever way you might like now why this learning is different from supervised learning since you didn't use any past or prior knowledge about the people you kept on classifying them on the go as they kept on coming you kept on classifying them yeah this category of people belong to this group this category of people belong to that group and so on okay let's see one more example let's suppose you have never seen a football match before and by chance you watch a video on the internet now you can easily classify the players on the basis of different Criterion like player wearing the same kind of Jersey are in one class player wearing different kind of Jersey are in different class or you can classify them on the basis of their playing style like the guy is a attacker so he's in one class he's a Defender he's in another class or you can classify them whatever Way You observe the things so this was also an example of unsupervised learning let's move on and see how unsupervised learning is used in the sectors of banking Healthcare and Retail so starting with banking sector so in banking sector it is used to segment customers by behavioral characteristic by surveying prospects and customers to develop multiple segments using clustering in healthcare sector it is used to categorize the MRI data by normal or abnormal images it uses deep learning techniques to build a model that learns from different features of images to recognize a different pattern next is the retail sector in retail sector it is used to recommend the products to customer based on their past purchases it does this by building a collaborative filtering model based on the past purchases by them I assume you guys now have a proper idea of what unsupervised learning means if you have any slightest doubt don't hesitate and add your doubt to the comment section so let's discuss the third and the last type of machine learning that is reinforcement learning so what is reinforcement learning well reinforcement learning is a type of machine learning algorithm which allows software agents and machine to automatically determine the ideal Behavior within a specific context to maximize its performance the reinforcement learning is about interaction between two elements the environment and the learning agent the learning agent leverages two mechanism namely exploration and exploitation when learning agent acts on trial and error basis it is termed as exploration and when it acts based on the knowledge gained from the environment it is referred to as exploitation and this environment rewards the agent for correct actions which is reinforcement signal leveraging the rewards obtained the agent improves its environment knowledge to select the next action in this image you can see that the machine is confused whether it is an apple or it's not an apple then the machine is trained using reinforcement learning if it makes correct decision it get rewards point for it and in case of wrong it gets a penalty for that once the training is done now the machine can easily identify which one of them is an apple let's see an example here we can see that we have an agent who has to judge from the environment to find find out which of the two is a duck the first task he did is to observe the environment next he select some action using some policy it seems that the machine has made a wrong decision by choosing a bunny as a duck so the machine will get penalty for it for example minus 50 point for a wrong answer right now the machine will update its policy and this will continue till the machine gets an optimal policy from the next time machine will know that bunny is not a duck let's see some of the use cases of reinforcement learning but before that let's see how Pavo trained his dog using reinforcement learning or how he applied the reinforcement method to train his dog Pavo integrated learning in four stages initially Pavo gave meat to his dog and in response to the meat the dog started salivating next what he did he created a sound with a bell for this the dog did not respond anything in the third part he tried to condition the dog by using the bell and then giving him the food seeing the food the dog started salivating eventually a situation came when the dog started salivating just after hearing the Bell even if the food was not given to him as the dog was reinforced that whenever the master will ring the bell he will get the food now let's move on and see how reinforcement learning is applied in the field of banking Healthcare and Retail sector so starting with the banking sector in banking sector reinforcement learning is used to create a next best offer model for a call center by building a predictive model that learns over time as user accept or reject offer made by the sales staff fine now in healthcare sector it is used to allocate the scars medical resources to handle different type of er cases by building a mark of decision process that learns treatment strategies for each type of er Cas next and the last comes the retail sector so let's see how reinforcement learning is apply to retail sector in retail sector it can be used to reduce excess stock with Dynamic pricing by building a dynamic pricing model that adjust the price based on customer response to the offers here's a quick question for you guys which of the following best defines machine learning option A a method of programming computers to perform specific tasks b a type of artificial intelligence that allows computers to learn from data c a technique for building websites d a way to write software without human intervention if you know the correct answer please leave it in the comment section below building on our understanding of machine learning the next topic is how to become a machine learning engineer where Learners will learn about the necessary skills educational pathway and career prospects in this exciting field who is a machine learning engineer machine learning Engineers are sophisticated programmers who develop machines and systems that can learn and apply knowledge without any specific Direction so they're basically just enthusiastic computer programmers but their focus goes beyond specifically programming machines to perform tasks so what they really do is they create programs that will enable machines to take actions without being specifically directed to perform those tasks now that we know who a machine learning engineer is let us talk about what does an ml engineer do machine learning Engineers are creators of the algorithms that allow a machine to find patterns in its own programming data teaching it to understand commands and even think for itself the artificial intelligence seen in automatic vacuums and self-driving cars is the thought children of these Engineers researching new technologies and implementing them in machine learning programs is one of the many tasks that a machine learning engineer does finding the best design and Hardware to use when building the robot or computer developing tangible prototypes to show stakeholders and also putting the machines through where number of rigorous tests to ensure their function is planned so by now you must have an idea about what they do and who they are so now let us understand what are the skills needed to become an ml engineer the first most important skill is the programming skill computer science fundamentals are extremely important for machine learning Engineers they include data structures algorithms computability and complexity and even computer architecture you must be able to apply Implement adapt or address them when programming practice problems coding competitions and hackathons are a great way to ha your skills so some of the programming languages that you must be familiar with is our language python Java programming so our language here is basically used for developing statistical software and data analysis whereas python lets you create analyze and organize large chunks of data with ease also Java helps in data description moving on the next important skill is probability and statistics a formal characterization of probability and techniques derived from it are at the heart of many machine learning algorithms these are a means to deal with uncertainty in the real world closely related to this field is of Statistics which provides various measures like mean median variance Etc also distributions like uniform normal binomial Etc and Analysis methods like hypothesis testing that are necessary for building and validating models from observed data is extremely important many machine learning algorithms are essentially extensions of statistical modeling procedures moving on the next skill that is really important is understanding data modeling and evaluation now what is data modeling data modeling is simply the process of estimating the underlying structure of a given data set with the goal of finding useful patterns like correlations clusters Etc and predicting properties of previously unseen instances a key part of this estimation process is continually evaluating how good a given model is depending on the task at hand you will need to choose an appropriate accuracy or error measure example log laws for classification sum of squared errors for regression Etc and also an evaluation strategy like testing training split sequential versus r Riz cross validation Etc iterative learning algorithms often directly utilize resulting errors to tweak the model examples are back propagation for neural networks so understanding these measures is very important even for just applying standard algorithms so also applying machine learning algorithms and different libraries are very important standard implementations of machine learning algorithms are widely available through libraries or packages or apis some of the examples are tensorflow sky kit learn but applying them effectively involves choosing a suitable model like the decision tree nearest neighbor neural net Etc also a learning procedure to fit the data as well as understanding how hyperparameters affect learning is really important you also need to be aware of the relative advantages and disadvantages of different approaches and also understand how bies and variance works overfitting and underfitting work all of the basics of machine learning and how do you apply them data science and machine learning challenges such as those on kaggle are a great way to get exposed to different kind of problems also to become an ml engineer you need to understand how software engineering and system design works so at the end of the day a machine learning engineer's typical output or deliverable is software and often it is a very small component that fits into a larger ecosystem of products and services you need to understand how these different pieces work together communicate with them and build appropriate interfaces for your component that others will eventually depend on careful system design may be necessary to avoid bottlenecks and let your algorithms scale well with increasing volumes of data software engineering best practices including requirements analysis system design modularity Version Control testing documentation Etc are in valuable for productivity collaboration quality and maintainability now that we have discussed the skills needed to become an ml engineer let us look at some of the major roles and responsibilities now the first and most important role is to create artificial intelligent products for the team well this is achieved when we are able to create machine learning models of our own right what's more important is that we need to build efficient applications the efficiency plays a really big role here there are some responsibilities of a machine learning engineer such as we need to be able to study some prototypes and then transform them into applications we also have to be able to design and build our own machine Learning Systems we have to be in a position where we put in some research to find the appropriate algorithms and tools necessary and yes we will be developing machine learning application based on what's required also what's important is to select the right data to find the correct data representation methods we also need to run machine learning tests and experiments to keep improving on our implementation for the use case and lastly we need to train the systems for topnotch accuracy but sometimes you will have to retrain them again based on the changes in the requirement well the show does seem like a lot for now but it really isn't that complex once you start learning and begin cracking the basics it's extremely simple moving on let's see some some of the salary and trends of a machine learning engineer according to the 2019 indeed report the best jobs in the US and Indian market is the machine learning engineer what's more interesting is that the role of an MLA engineer recorded a whooping T 44% increase since 2015 the average salary for a machine learning engineer in India is rupees 6 lak 89,4 60 rupees whereas the average salary for a machine learning engine in the United States of America is $112,000 so that's really a huge amount right moving on let's talk about some of the companies hiring machine learning Engineers the number of opportunities is exponentially growing and this is amazing because you'll be trending when you're a machine learning engineer and obviously you'll be paid really well everyone from Apple to Uber Facebook to Salesforce all these big players are on a constant ml engineers hiring SP and they obviously pay really high salaries for this now this takes us to the last part of today's session that is the future of machine learning what is perhaps most compelling about machine learning is it seemingly Limitless applicability there are already so many fields being impacted by Machine learning including education Finance computer science and much more there are also virtually no fields to which machine learning does not apply in some cases machine learning techniques are in fact desperately needed Healthcare is an obvious example machine learning techniques are already being applied to critical Arenas within the healthc care sphere impacting everyone from care variation reduction efforts to Medical scan analysis the world is unquestionably changing in Rapid and dramatic ways and the demand for machine learning Engineers is going to keep increasing exponentially the world's challenges are complex and they will obviously require complex systems to solve them machine learning Engineers are building these systems if this is your future then there's no time like the present to start mastering the skills and developing the mindset you're going to need to succeed after understanding the role of machine learning engineer our next focus is machine learning algorithm where Learners will explore various types of algorithms and their importance in building predictive models let me connect you to the real life and tell you what all are the things which you can easily do using the concepts of machine learning so you can easily get answer to the questions like which types of house lies in this segment or what is the market value of this house or is this a mail a Spam or not a Spam is there any fraud well these are some of the question you could ask to the machine but for getting an answer to these you need some algorithm the machine need to train on the basis of some algorithm okay but how will you decide which algorithm to choose and when okay so the best option for us is to explore them one by one so the first is classification algorithm where the category is predicted using the data if you have some question like is this person a male or a female or is this a male a Spam or not a Spam then these category of question would fall under the classification algorithm classification is a supervised learning approach in which the computer program learns from the input given to it and then uses this learning to classify new observation some examples of classification problems are speech recognization handwriting recognization biometric identification document classification Etc shall we move ahead okay so next is the anomaly detection algorithm where you identify the unusual data point so what is anomaly detection well it's a technique that is used to identify unusual pattern that do not confirm to expected Behavior or you can say the outliers it has many application in business like intrusion detection like identifying strange patterns in the network traffic that could signal a hack or system Health monitoring that is spotting a deadly tumor in the MRI scan or you can even use it for fraud detection credit card transaction or to deal with fault detection in operating environment so next comes the clustering algorithm and you can use this clustering algorithm to group the data based on some similar condition now you can get answer to which type of houses lies in this segment or what type of customer buys this product the clustering is a task of dividing the population or data points into number of groups such that the data point in the same groups are more similar to other data points in the same group than those in the other groups in simple words the aim is to segregate groups with similar trait and assign them into cluster now this clustering is a task of dividing the population or data points into a number of groups such that the data points in the X group is more similar to the other data points in the same group rather than those in the other group in other words the aim is to segregate the groups with similar traits and assign them into different clusters let's understand this with an example suppose you're the head of a rental store and you wish to understand the preference of your customer to scale up your business so is it possible for you to look at the detail of each customer and design a unique business strategy for each of them definitely not right but what you can do is to Cluster all your customer say into 10 different groups based on their purchasing habit and you can use a separate strategy for customers in each of these 10 different groups and and this is what we call clustering next we have regression algorithm where the data itself is predicted question you may ask to this type of model is like what is the market value of this house or is it going to rain tomorrow or not so regression is one of the most important and broadly used machine learning and statistics tool it allows you to make prediction from data by learning the relationship between the features of your data and some observe continuous valued response regression is used in a massive number of applic ation you know what stock prices prediction can be done using regression now you know about different machine learning algorithm how will you decide which algorithm to choose and when so let's cover this part using a demo so in this demo part what we'll do we'll create six different machine learning model and pick the best model and build the confidence such that it has the most reliable accuracy so for our demo part we'll be using the IIs data set this data set is quite very famous and is considered one of the best small projectors start with you can consider this as a hello world data set for machine learning so this data set consists of 150 observation of Iris FL there are four Columns of measurement of flowers in centimeters the fifth column being the PES of the flower observed all the observed flowers belong to one of the three species of Iris stosa Iris venica and Iris Versa well this is a good project because it is so well to understand the attributes are numeric so you have to figure out how to load and handle the data it is a classification problem there by allowing you to practice with perhaps an easier type of supervised learning algorithm it has only four attributes and 150 rows meaning it is very small and can easily fit into the memory and even all of the numeric attributes are in the same unit and the same scale it means you do not require any special scaling or transformation to get started with so let's start coding and as I told earlier for the demo part I'll be using Anaconda with python 3.0 install on it so when you install Anaconda how your Navigator would look like so this is my homepage of of my anaconda navigator on this I'll be using the Jupiter notebook which is a web-based interactive Computing notebook environment which will help me to write and execute my python codes on it so let's hit the launch button and execute our Jupiter notebook so as you can see that my Jupiter notebook is starting on Local Host 1890 okay so this is my Jupiter notebook what I'll do here I'll select new notebook Python 3 there's my environment where I can write and execute all my python codes on it so let's start by checking the version of the libraries in order to make this video short and more interactive and more informative I have already written the set of code so let me just copy and paste it down I'll explain you then one by one so let's start by checking the version of the Python libraries okay so there's the code let's just copy it copied and let's paste it okay first let me summarize things for you what we are doing here we are just checking the version of the different libraries starting with python we'll first check what version of python we are working on then we'll check what the version of scipi we are using then numai M plot Leb then Panda then psychic learn okay so let's execute the Run button and see what are the various version of libraries which we are using hit the run so we are working on python 3.6.4 scipi 1.0 numpy 1.14 mlot lip 2.12 Panda 0.22 and psyched learn of version 0.19 okay so these are the version which I'm using ideally your version should be more recent or it should match but don't worry if you lag few versions behind as the apis do not change so quickly everything in this tutorial will very likely still work for you okay but in case you getting an error stop and try to fix that error in case you're unable to find the solution for the error feel free to reach out edureka even after this class let me tell you this if you're not able to run the script properly you will not be able to complete this tutorial okay so whenever you get a doubt reach out to edura and just resolve it now if everything is working smoothly then now it's the time to load the data set so as I said I'll be using the RS flow data set for this tutorial but before loading the data set let's import all the modules function and the object which we are going to use in this tutorial same I have already written the set of code so let's just copy and paste them let's load all the libraries so these are the various libraries which will be using in our tutorial so everything should work fine without an error if you get an error just stop you need to work on your CP envirment before you continue any further so I guess everything should work fine let's hit the Run button and see okay it worked so let's now move ahead and load the data we can load the data direct from the UCI machine learning repository first of all let me tell you we are using Panda to load the data okay so let's say my URL is this so This is My URL for the UIA machine learning repository from where I'll be downloading the data set okay now what I'll do I'll specify the name of each column when loading the data this will help me later to explore the data okay so I'll just copy and paste it down okay okay so I'm defining a variable names which consist of various parameters including SE lend SLE width petal length petal width and class so these are just the name of column from the data set okay now let's define the data set so data set equals panda. read CSV inside that we are defining URL and the names that is equal to name as I already said we'll be using Panda to load the data all right so we are using panda. read CSV so we are reading the CSV file and inside that from where that CSV is coming from the URL which URL So This is My URL okay and names equal names it's just specifying the names of the various columns in that particular CSV file okay so let's move forward and execute it so even our data set is loaded in case you have some network issues just go ahead and download the iris data file into your working directory and load it using the same method but yeah make sure that you change the url to the local name or else you might get an error okay yeah data set is loaded so let let's move ahead and check our data set let's see how many columns or rows we have in our data set okay so let's print the number of rows and columns in our data set so our data set is data set. shape what this will do it will just give you the numbers of total number of rows and total number of column or you can say the total number of instances or attributes in your data set fine so print data set. shape what you getting 150 and five so 150 is the total number of rows in your data set and five is the total number of columns fine so moving on ahead what if I want to see the sample data set okay so let me just print the first 30 instances of the data set okay so print data set. head what I want is the first 30 instances fine this will give me the first 30 result of my data set okay so when I hit the Run button what I'm getting is the first 30 result okay 0 to 29 so this is how my sample data set looks like seel length seel WID petal length petal width and the class okay so this is how data set looks like now let's move on and look at the summary of each attribute what if I want to find out the count mean the minimum and the maximum values and some other percentiles as well so what should I do then for that print data set dot describe what it will give let's see so you can see that all the numbers are the same scale of similar range between 0 to 8 cm right the mean value the standard deviation the minimum value the 25 percentile 50 percentile 75 percentile the maximum value all these values lies in the range between 0 to 8 cm okay so what we just did is we just took a summary of each attribute now let's look at the number of instances that belong to each class so for that what we'll do print data set first of all so let's print data set and I want to group it Group by using class and I want the size of it size of each class fine and let's hit the Run okay so what I want to do I want to print print what data set how I want to get it I Want It by class so Group by class okay now I want the size of each class find the size of each class so Group by class. size execute the run so you can see that I have 50 instances of Iris setosa 50 instances of Iris verical and 50 instances of Iris verinica okay all are of data type integer of Base 64 fine so now we have a basic idea for data now let's move ahead and create some visualization for it so for this we are going to create two different types of plot first would be the univariate plot and the next would be the multivariate plot so we'll be creating univariate plots to better understand about each attribute and the next we'll be creating the multivariate plot to better understand the relationship between different attributes okay so we start with some univariate plot that is plot of each individual variable so given that the input variables are numeric we can create box and visus plot for it okay so let's move ahead and create a box and viscus plot so data set. plot what kind I want it's a box okay and do I need a subplot yes yeah I need subplots for that so subplots equal true what type of layout do I want so my layout structure is 2 cross 2 next do I want to share my coordinates X and Y coordinates no I don't want to share it so share x equal false and even share y that two equals false okay so we have here data set. plot kind equal box my subplots is true layout 2 cross two and then what I want to do it I want to see it so plot. show whatever I created show it okay execute it now this gives us a much Clear idea what the distribution of the input attribute now what if I had given the layout to 2 cross2 instead of that I would have given it 4 cross 4 so what it will result just see fine everything would be printed in just one single row hold on guys ARA has a doubt he's asking that why we are using the share X and sharey values what are these why we have assigned false values to it okay Arya so in order to resolve this query I I need to show you what will happen if I give True Values to them okay so be with me so share x equal true and share y that equals true so let's see what result we'll get you're getting it the X and Y coordinates are just shared among all the four visualization right so you can see that the SLE length and SLE WID has y values ranging from 0.0 to 7.5 which are being shared among both the visualization so is with the pedal length it has a shared value between 0.0 to 7.5 okay so that is why I don't want to share the value of X and Y so it's just giving us a cluttered visualization so ARA why I'm doing this I'm just doing it because I don't want my X and Y coordinates to be shared among any visualization okay that is why my share X and share y value are false okay let's execute it so this is a pretty much Clear visualization which gives a clear idea about the distribution of the input attributes now if you want you can also create a histogram of each input variable to get a clear idea of the distribution so let's create a histogram for it so data set do his okay I need to see it so plot. show let's see so this is my histogram and it seems that we have two input variables that have a gan distribution so this is useful to note as we can use the algorithms that can exploit this assumption okay so next comes the multivariate plot now that we have created the univariate plot to understand about each attribute let's move on and look at the multivariate plot and see the interaction between the different variables so first let's look at the scatter plot of all the attribute this can be helpful to spot structured relationship between input variables okay so let's create a scatter Matrix so for creating a scatter plot we need scatter Matrix and we need to pass our data set into it okay and then what I want I want to see it so plot. show so this is how my scatter Matrix looks like it's like that the diagonal grouping of some pair right so this suggests a high correlation and a predictable relationship all right this was our multivar plot now let's move on and evaluate some algorithm now it's time to create some model of the data and estimate that accuracy on the basis of unseen data okay so now we know all about our data set right we know how many instances and attributes are there in our data set we know the summary of each attribute now I guess we have seen much about our data set now let's move on and create some algorithm and estimate their accuracy based on the Unseen data okay now what we'll do we'll create some model of the data and estimate the accuracy based on some unseen data okay so for that first of all let's create a validation data set what is a validation data set validation data set is your training data set that will be using it to train our model fine all right so how we'll create a validation data set for creating a validation data set what we are going to do is we are going to split our data set into two part okay so the very first thing we'll do is to create a validation data set so why do we even need a validation data set so we need a validation data set to know that the model we created is any good later what we'll do we'll use the statistical method to estimate the accuracy of the model that we create on the Unseen data we also want a more concrete estimate of the accuracy of the best model on unseen data by evaluating it on the actual unseen data okay confused let me simplify this for you what we'll do we'll split the loaded data into two parts the first 80% of the data will use it to train our model and the rest 20% will hold back as the validation data set that will use it to verify a trained model okay fine so let's define an array there's my array what it will consist of it will consist of all the values from the data set so data set do values okay next I'll Define a variable X which will consist of all the column from the array from 0 to 4 starting from 0 to 4 and the next variable Y which would consist of the array starting from this so first of all we'll Define a variable X that will consist of the values in the arrays starting from the beginning 0 till 4 okay so these are the column which will included in the X variable and for a y variable I'll Define it as a class or the output so what I need I just need the fourth column that is my class column so I'll start it from the beginning and I just want the fourth column okay now I'll Define my validation size validation underscore size I'll Define it as 0.20 and I'll use a seed I'll Define seed equals 6 so this method seed sets the integer starting value used in generating random number okay I'll Define the value of c equal 6 I'll tell you what is the importance of it later on okay so let me Define first few variables such as xcore train test Yore train and Yore test okay so what we want to do is Select some model okay so model underscore selection but before doing that what we have to do is split our training data set into two halfs okay so do Trin underscore testore split what we want to split is the value of X and Y okay and my test size is equals to validation size which is 0.20 correct and my random state is equal to seed so what the seed is doing here it's helping me to keep the same Randomness in the training and testing data set fine so let's execute it and see what is our result it's executed next we'll create a test harness for this we'll use 10 fold cross validation to estimate the accuracy so what it will do it will split our data set into 10 parts train on the nine part and test on the one part and this will repeat for all combination of train and test plats okay so for that let's define again my C that was six already defined and scoring equals accuracy fine so we are using the metric of accuracy to evaluate the model so what is this this is a ratio of number of correctly predicted instances divided by the total number of instances in the data set multiplied by 100 giving a percentage example it's 98% accurate or 99% accurate things like that okay so we'll be using this scoring variable when we run the build and evaluate each model in the next step so our next part is building model till now we don't know which algorithm would be good for this problem or what configuration to use so let's begin with six different algorithm I'll be using logistic regression linear discriminant analysis K nearest neighbor classification and regression trees neis and code Vector machine well these algorithms which I'm using is a good mixture of of simple linear or nonlinear algorithms in simple linear which included the logistic regression and the linear discriminant analysis or the nonlinear part which included the KNN algorithm the cart algorithm that the name buers and the support Vector machines okay so we reset the random number seat before each run to ensure that evaluation of each algorithm is performed using exactly the same data splits it ensures the result are directly comparable okay so let me just copy and paste it okay so what we are doing here we are building five different types of model we are building logistic regression linear discriminant analysis K nearest neighbor Deion tree gajan neys and the support Vector machine okay next what we'll do we'll evaluate model in each turn okay so what is this so we have six different model and accuracy estimation for each one of them now we need to compare the model to each other and select the most accurate of them all so running this script we saw the following result so we can see some of the result on the screen what is this it is just the accuracy score using different set of algorithms okay when we are using logistic regression what is the accuracy rate when we are using discriminant algorithm what is the accuracy and so and so okay so from the output it seems that LDA algorithm was the most accurate model that we tested now we want to get an idea of the accuracy of the model on our validation set or the testing data set so this will give us a independent final check on the accuracy of the best model it is always valuable to keep a testing data set for just in case case you made a overfitting to the testing data set or you made a data leak both will result in a overly optimistic result okay you can run the ldm model directly on the validation set and summarize the result as a final score a confusion Matrix and a classification report it's time for another question which type of machine learning algorithm is used when the output variable is a category such as spam or not spam and the options are a regression B classification C clustering D Association if you know the correct answer please leave it in the comment section below after learning various types of machine learning algorithms the next topic is linear regression algorithm where Learners will explore one of the most fundamental and widely used algorithms in machine learning let us understand what regression in machine learning is so what exactly is regression the main goal of regression is the construction of an efficient model to predict the dependent attri Utes from a bunch of attribute variables a regression problem is where the output variable is either real or a continuous value like salary weight area Etc we can also define regression as a statistical means that is used in applications like housing investing Etc to predict the relationship between a dependent variable and a bunch of independent variables for example let's say in the finance application or investing we can actually predict the values of certain stock prices or you know those values depending on the the independent variables like how many years it takes for a stock to you know actually mature or how many days will it take to grow or those variables that you have in investing and depending upon that we can make a possible outcome or a possible prediction of how our stock is going to be invested in a profit state or a loss state or all those things or we can take another example like housing we can take different parameters like number of years it's been there how many people have used it or what is the area of the house depending on all these factors or how many rooms does a house have we can predict the price of a house so this is basically what regression really is so let us take a look at a various types of regression techniques that we have we have SIMPLE linear regression then we have polinomial regression support Vector regression decision regression we have random Forest regression and we have logistic regression as well that is also a type of regression that we have but for now we'll be focusing on simple linear regression so let's talk about how or what exactly simple linear regression first so one of the most interesting and common regression technique is simple linear regression in this we predict the outcome of a dependent variable y based on the independent variables X so the relationship between the variables is linear hence the word linear regression then comes the polinomial regression so in this regression Technique we transform the original features into a polom feature of a given degree and then perform regression on it so this is basically polinomial regression after this we have support Vector machine regression or we can also call it svr we identify a hyper plane with maximum margin such that the maximum number of data points are within those margins it is also quite similar to the support Vector machine classification algorithm then we have decision tree regression a decision tree can be used for both regression and classification but in this case of regression we use the ID3 algorithm which is iterative dichotomize 3 to identify the splitting node by reducing the standard deviation after this we have a random Forest regression which is basically an ensemble of predictions of several decision tree regressions so this is all about the types of regressions for now we are going to focus on simple linear regression so let's take a look at what exactly is simple linear regression simple linear regression is a regression technique in which the independent variable has a linear relationship with the dependent variable the straight line in the diagram is the best fit line and the main goal of the simple linear regression is to consider the given data points and plot the best fit line to fit the model in the best way possible so if you talk about a real life analogy to explain linear regression we can take an example of a car resale value so we have different parameters you know when we are talking about resale value of a car like how many years the car has been there in the market and how many kilometers it has been ridden the kind of mileage the car gives and then we have different parameters we can focus upon and all these independent variables somehow are linearly connected or interconnected to the price of the car so that is one example to understand linear regression we'll be doing that in the use case I'll be telling you about how you can predict the price of car now talking about linear regression terminologies there are a few terminologies that you have to be thorough with to begin with linear regression so first of all we have to talk about cost function so the best fit line can be based on the linear equation that is given here so in this the dependent variable that is to be predicted is denoted by y a line that t the y axis is denoted by The Intercept B 0 the B1 is the slope of the line and X represents the independent variables that determine the prediction of Y the error in the resultant prediction is denoted by e now talking about cost function the cost function provides the best possible values for B 0 and B1 to make the best fit line for the data points we do this by converting this problem into a minimization problem to get the best values for B 0 and B1 so with this the error is minim IED in this problem between the actual value and the predicted value and we choose the function above to minimize now we Square the error difference and sum the error over all the data points the division between the total number of data points and the produced value provides the average square error for all the data points it is also known as mean squared error and we can change the values of B 0 and B1 so that the msse or the mean squ error value is settled at the minimum so this is one terminology that is cost function that we use in line lar regression then we have the gradient descent so the next important terminology to understand linear regression is gradient Descent of course and it is a method of updating B 0 and B1 value to reduce the MSC which is the mean squared error the idea behind this is to keep iterating the B 0 and B1 values until we reduce the msse to the minimum now to update B 0 and B1 we take the gradients from the cost function and to find these gradients we take partial derivatives with respect to B 0 and B1 and these partial derivatives are the gradients and are used to update the values of B 0 and B1 I'm sure guys this might be a little confusing for you guys if you are new to this like gradient descent and cost function but you don't have to worry about this because in Python when we using linear regression we're going to be using the skarn or the psychic learn Library so you don't have to worry about this you just have to integrate your model with the linear regression module that we have already over there and you'll be down with and when I'm implementing the linear regression model you'll see how easy it is to actually Implement linear regression in Python so after this let's talk about a few advantages and disadvantages of linear regression so talking about the advantages first linear regression performs exceptionally well for linearly separable data and it is actually very easy to implement interpret and very efficient to train as well and even though the linear regression is prone to overfitting it handles it pretty well using dimensionally reduction techniques regularization and cross validation and and one more Advantage is that the extrapolation Beyond a specific data set so these are all the advantages that we have with linear regression let's talk about a few disadvantages as well so one of the most common disadvantage with linear regression is that it takes the Assumption of linearity between dependent and independent variables the next disadvantage is it is often very prone to noise and overfitting as well which is not a very good sign for any model if you are doing regression or classification in machine learning the next disadvantage stages it is very quite sensitive to outliers as well and the last one is that it is very prone to multicolinearity so these are all the advantages and disadvantages of linear regression here's a simple question for you guys in linear regression the goal is two and the options are a classify data into different categories B Predator continuous output variable based on the input variables C group similar data point together D discover patterns in large data sets if you know the correct answer please leave it in the comment section below having covered the linear regression algorithm in our previous module we move on to logistic regression algorithm well Learners will understand how this algorithm is used in classification problems so let's understand the what and why of logistic regression now this algorithm is most widely used when the dependent variable or you can say the output is in the binary format so here you need to predict the outcome of a categorical dependent variable so so the outcome should be always discrete or categorical in nature Now by discrete I mean the value should be binary or you can say you just have two values it can either be zero or one it can either be yes or no either be true or false or high or low so only these can be the outcomes so the value which you need to predict should be discrete or you can say categorical in nature whereas in linear regression we have the value of y or you can say the value you need to predict is in a Range so that is how there's a difference between linear regression and logistic regression now you must be having a question why not linear regression now guys in linear regression the value of y or the value which you need to predict is in a range but in our case as in the logistic regression we just have two values it can be either zero or it can be one it should not entertain the values which is below zero or above one but in linear regression we have the value of y in the range so here in order to implement logistic regression we need to clip this part so we don't need the value that is below zero or we don't need the value which is above one so since the value of y will be between only 0 and one that is the main rule of logistic regression the linear line has to be clipped at 0 and one now once we clip this graph it would look somewhat like this so here you getting a curve which is nothing but three different straight lines so here we need to make a new way to solve this problem so this has to be formulated into equation and hence we come up with logistic regression so here the outcome is either zero or one which is the main rule of logistic regression so with this a resulting curve cannot be formed formulated so hence our main aim to bring the values to 0 and one is fulfilled so that is how we came up with logistic regression now here once it gets formulated into an equation it looks somewhat like this so guys this is nothing but a S curve or you can say the sigmoid curve or sigmoid function curve so this sigmoid function basically converts any value from minus infinity to Infinity to your discrete values which a logistic regression wants or you can say the values which are in binary format either zero or one so if you see here the Val vales as either zero or one and this is nothing but just a transition of it but guys there's a catch over here so let's say I have a data point that is 0.8 now how can you decide whether your value is zero or one now here you have the concept of threshold which basically divides your line so here threshold value basically indicates the probability of either winning or losing so here by winning I mean the value is equals to one and by losing I mean the value is equals to zero but how does it do that let's say I have data point which is over here let's say my cursor is at 0.8 so here I'll check whether this value is less than my threshold value or not let's say if it is more than my threshold value it should give me the result as one if it is less than that then it should give me the result as zero so here my threshold value is 0.5 now I need to Define that if my value let's say 0.8 it is more than 0.5 then the value shall be rounded off to one and let's say if it is less than 0.5 let's say I have a value 0.2 then should reduce it to zero so here you can use the concept of threshold value to find your output so here it should be discrete it should be either zero or it should be one so I hope you CAU this curve of logistic regression so guys this is the sigmoid S curve so to make this curve we need to make an equation so let me address that part as well so let's see how an equation is formed to imitate this functionality so over here we have an equation of a straight line which is yal MX plus C so in this case I just have only one indep depent variable but let's say if we have many independent variable then the equation becomes M1 X1 + M2 X2 + M3 X3 and so on till MN xn now let us put in B and X so here the equation becomes yal B1 X1 + B2 X2 plus B3 X3 and so on till BN xn plus C so guys your equation of the straight line has a range from minus infinity to Infinity but in our case or you can say in logistic equation the value which we need to predict or you can say the Y value it can have the range only from 0 to 1 so in that case we need to transform this equation so to do that what we had done we have just divide the equation by 1 - y so now if Y is equal to 0 so 0 1 - 0 equal to 1 so 0 over 1 is again 0er and if we take Y is equal to 1 then 1 / 1 - 1 which is 0 so 1 / 0 is infinity so here my range is now between 0o to Infinity but again we want the range from minus infinity to Infinity so for that what we'll do we'll have the log of this equation so let's go ahead and have the logarithmic of this equation so here we have this transform it further to get the range between minus infinity to Infinity so over here we have log of Y 1 minus one and this is your final logistic regression equation so guys don't worry you don't have to write this formula or memorize this formula in Python you just need to call this function which is logistic regression and everything will be automatically for you so I don't want to scare you with the maths and the formulas behind mind it but it's always good to know how this formula was generated moving ahead let us see the various use cases wherein logistic regression is implemented in real life so the very first is weather prediction now logistic regression helps you to predict your weather for example it is used to predict whether it is raining or not whether it is sunny is it cloudy or not so all these things can be predicted using logistic regression whereas you need to keep in mind that both linear regression and logistic regression can be used in predicting the weather so in that case linear regression helps you to predict what will be the temperature tomorrow whereas logistic regression will only tell you whether it's going to rain or not or whether it's cloudy or not whether it's going to snow or not so these values are discrete whereas if you apply linear regression you'll be predicting things like what is the temperature tomorrow or what is the temperature day after tomorrow and all those things so these are the slight differences between linear regression and logistic regression now moving ahead we have classification problem so python performs multiclass classification so here it can help you tell whether it's a bird or it's not a bird then you classify different kind of mammals let's say whether it's a dog or it's not a dog similarly you can check it for reptile whether it's a reptile or not a reptile so in logistic aggression it can perform multiclass classification so this point I have already discussed that it is used in classification problems next it also helps you to determine the illness as well so let me take an example let's say a patient goes for routine checkup in hospital so what doctor will do it will perform various test on the patient and will check whether the patient is actually ill or not so what will be the features so doctor can check the sugar level the blood pressure then what is the age of the patient is it very small or is it a old person then what is the previous medical history of that patient and all of these features will be recorded by the doctor and finally doctor checks the patient data and determines the outcome of it illness and the severity of illness so using all the data a doctor can identify whether a patient is ill or not so these are the various us cases in which you can use logistic regression now I guess enough of theory part so let's move ahead and see some of the Practical implementation of logistic regression so over here I'll be implementing two projects wherein I have the data set of a Titanic so over here we predict what factors made people more likely to survive the sinking of the Titanic ship and in my second project we'll see the data analysis on the SUV cars so over here we have the data of the SUV cars who can purchase it and what factors made people more interested in buying it SUV so these will be the major questions as to why you should Implement logistic regression and what output will you get by it so let's start by the very first project that is Titanic data analysis so some of you might know that there was a ship called as Titanic which basically hit an iceberg and it sank to the bottom of the ocean and it was a big disaster at that time because it was the first voyage of the ship and it was supposed to be really really strongly built and one of the best ships of that time so it was a big disaster of that time and of course there's a movie about this as well so many of you might have watched it so what we have we have data of the passengers those who survived and those who did not survive in this particular tragedy so what you have to do you have to look at this data and analyze which factors would have been contributed the most to the chances of a person's Survival on the ship or not so using the logistic regression we can predict whether the person survived or the person died now apart from this we'll also have a look with the various features along with that so first let us explore the data set so over here we have the index value then the First Column is passenger ID then my next column is survived so over here we have two values a zero and a one so zero stands for did not survive and one stands for survive so this column is categorical where the values are discrete next we have passenger class so over here we have three values 1 two and three so this basically tells you that whether a passenger is traveling in the first class second class or third class then we have the name of the passenger we have the sex or you can say the gender of the passenger whether passenger is a male or female then we have the age we have the csb so this basically means the number of siblings or the spouses above the Titanic so over here we have values such as 1 zero and so on then we have parch so parch is basically the number of parents or children aboard the Titanic over here we also have some values then we have the ticket number we have the fair we have the cabin number and we have the embarked column so in my embarked column we have three values we have S C and Q so s basically stands for Southampton C stands for cherborg and Q stands for cown so these are the features that we'll be applying our model on so here we'll perform various steps and then we'll be implementing logistic regression so now these are the various steps which are required to implement any algorithm so now in our case we are implementing logistic regression so very first step is to collect your data or to import the libraries that are used for collecting your data and then taking it forward then my second step is to analyze your data so over here I can go through the various fields and then I can analyze the data I can check did the females or children survive better than the males or did the rich passenger survive more than the poor passenger or did the money matter as in who paid more to get into the ship what they evacuated first and what about the workers does the worker survived or what is the survival rate if you were the worker in the ship and not just a traveling passenger so all of these are very very interesting questions and you would be going through all of them one by one so in this stage you need to analyze your data and explore your data as much as you can then my third step is to Wrangle your data now data rangling basically means cleaning your data so over here you can simply remove the unnecessary items or if you have a null values in the data set you can just clear that data and then you can take it forward so in this step you can build your model using the train data set and then you can test it using the test so over here you will be performing a split which basically split your data set into training and testing data set and finally you will check the accuracy so as to ensure how much accurate your values are so I hope you guys got these five steps that you're going to implement in logistic regression so now let's go into all these steps in detail so number one we have to collect your data or you can say import the libraries so let me just show you the implementation part as well so I'll just open my jupyter notebook and I'll just Implement all of these steps side by side so guys this is my jupyter notebook so first let me just rename jupyter notebook to let's say like dining data analysis now our first step was to import all the libraries and collect the data so let me just import all the libraries first so first of all I'll import pandas so pandas is used for data analysis so I'll say import pandas as PD then I'll be importing numpy so I'll say import numpy as NP so numai is a library in Python which basically stands for numerical Python and it is widely used to perform any scientific computation next we'll be importing cbon so cbon is a library for statistical plotting so I'll say import cbon as SNS I'll also import mat plot lip so matplot lib library is again for plotting so I'll say import matplot li. pyplot as plld now to run this library in juper Notebook all I have to write in is percentage mat plot Li in line next I'll be importing one module as well so as to calculate the basic mathematical functions so I'll say import math so these are the libraries that I'll be needing in this Titanic data analysis so now let me just import my data set so I'll take a variable let's say Titanic data and using the pandas I will just read my CSV or you can say the data set I'll write the name of my data set that is titanic. CSV now I have already showed you the data set so over here let me just print the top 10 rows so for that I'll just say I'll take the variable Titanic data do head and I'll say the top 10 rows so now I'll just run this so to run this so I just have to press shift plus enter or else you can just directly click on the cell so over here I have the index we have the passenger ID which is nothing but again the index which is starting from one then we have the survived column which has the categorical values or you can say the discrete values which is in the form of zero or one then we have the passenger class we have the name of the passenger sex age and so on so this is the data set that I'll be going forward with next let us print the number of passengers which are there in this original data set so for that I'll just simply type in print I'll say number of passengers and using the length function I can calculate the total length so I'll say length and inside this I'll be passing this variable which is Titanic data so I'll just copy it from here I'll just paste it do index and next so let me just print this one so here the number of passengers which are there in the original data set we have is 891 so around this number we're traveling in the typ anic ship so over here my first step is done where you have just collected data imported all the libraries and find out the total number of passengers which are traveling in Titanic so now let me just go back to presentation and let's see what is my next step so we're done with the collecting data next step is to analyze your data so over here we'll be creating different plots to check the relationship between variables as in how one variable is affecting the other so you can simply explore your data set by making use of various columns and then you can plot a graph between them so you can either plot a correlation graph you can plot a distribution graph it's up to you guys so let me just go back to my jupyter notebook and let me analyze some of the data over here my second part is to analyze data so I just put this in header two now to put this in header two I just have to go on code click on markdown and I just run this so first let us plot a account plot where compare between the passengers who survived and who did not survive so for that I'll be using the cbon library so over here I have imported cbon as SNS so I don't have to write the whole name I'll simply say SNS do count plot I'll say x is who survived and the data that I'll be using is the Titanic data or you can say the name of variable in which you have stored your data set so now let me just run this so over here as you can see I have survived column on my x axis and on the y axis I have the count so zero basically stands for did not survive and one stands for the passengers who did survive so over here you can see that around 550 of the passengers who did not survive and there were around 350 passengers who only survived so here you can basically conclude that there are very less survivors than non-s survivors so this was the very first plot now let us plot another plot to compare the six as to whether out of all the passengers who survived and who did not survive how many were men and how many were female so to do that I'll simply say SNS do count plot I add the Hue as sex so I want to know how many females and how many males survive then I'll be specifying the data so I'm using Titanic data set and let me just run this okay I've done a mistake over here so over here you can see I have survived column on the x axis and I have the count on the Y now so here your blue color stands for your male passengers and orange stands for your female so as you can see here the passengers who did not survive that has a value zero so we can see that majority of males did not survive and if we see the people who survived here we can see the majority qu of female survive so this basically concludes the gender of the survival rate so it appears on average women were more than three times more likely to survive than men next let us plot another plot where we have the Hue as the passenger class so over here we can see which class at the passenger was traveling in whether it was traveling in class one 2 or three so for that I'll just write the same command I'll say SNS do count plot I'll keep my xaxis as a only I'll change my Q to passenger class so my variable named as P class and the data set that I'll be using is Titanic data so this is my result so over here you can see I have blue for first class orange for second class and green for the third class so here the passengers who did not survive were majorly of the third class or you can say the lowest class or the cheapest class to get into the Titanic and the people who did survive majorly belong to the higher classes so here one and two has more rise than the passenger who were traveling in the third class so here we have concluded that the passengers who did not survive were majorly of third class or you can say the lowest class and the passengers who were traveling in first and second class would tend to survive more next let us plot a graph for the age distribution over here I can simply use my data so we'll be using Panda's library for this I'll declare an array and I'll pass in the column that is age so I plot and I want a histogram so I'll say plot doist so you can notice over here that we have more of young passengers or you can say the children between the ages 0 to 10 and then we have the average age people and if you go ahead lesser would be the population so this is the analysis on the age column so we saw that we have more young passengers and more mediocre age passengers which are traveling in the Titanic so next let me plot a graph of air as well so I'll say Titanic data I'll say fair and again I'll plot a histogram so I'll say hist so here you can see the fair size is between 0 to 100 now let me add the bin size so as to make it more clear so over here I'll say bin is equals to let's say 20 and I'll increase the figure size as well so I'll say fix size let's say I'll give the dimensions as 10x 5 so it is bins so this is more clear now next let us analyze the other columns as well so I'll just type in Titanic data and I want the information as to what all columns are left so here we have passenger ID which I guess it's of no use then we have see how many passengers survived and how many did not we also see the analysis on the gender basis we saw whether the female tend to survive more or the men tend to survive more then we saw the passenger class where the passenger is traveling in the first class second class or third class then we have the name so in name we cannot do any analysis we saw the sex we saw the age as well then we have SSP so this stands for the number of siblings or the SP houses which are aboard the Titanic so let us do this as well so I'll say SNS do count plot I'll mention X as s SP and I'll be using the Titanic data so you can see the plot over here so over here you can conclude that it has the maximum value on zero so you can conclude that neither a children nor a spouse was on board the Titanic now second most highest value is one and then we have very less values for 2 3 4 and so on next if I go above we saw this column as well similarly you can do for parge so next we have parge or you can say the number of parents or children which were the Titanic so similarly you can do this as well then we have the ticket number so I don't think so any analysis required for Ticket then we have fair so fair we have already discussed as in the people who tend to travel in the first class usually pay the highest fair then we have the cabin number and we have embarked so these are the columns that we'll be doing data wrangling on so we have analyzed the data and we have seen quite a few graphs in which we can conclude which variable is better than the another or or what is the relationship they hold so third step is my data wrangling so data wrangling basically means cleaning your data so if you have a large data set you might be having some null values or you can say n values so it's very important that you remove all the unnecessary items that are present in your data set so removing this directly affects your accuracy so I'll just go ahead and clean my data by removing all the N values and unnecessary columns which which has a null value in the data set so next I'll be performing data wrangling so first of all I'll check whether my data set is null or not so I'll say Titanic data which is the name of my data set and I'll say is null so this will basically tell me what all values are null and it will return me a Boolean result so this basically checks the missing data and your result will be Boolean format as in the result will be true or false so false mean if it is not null and true means if it is null so let me just run this over here you can see the values as false or true so false is where the value is not null and true is where the value is null so over here you can see in the cabin column we have the very first value which is null so we have to do something on this so you can see that we have a large data set so the counting does not stop and we can actually see the sum of it we can actually print the number of passengers who have the N value in each column so I'll say Titanic uncore data is null and I want the sum of it so I'll say do sum so this will basically print the number of passengers who have the NN values in each column so we can see that we have missing values in each column that is 177 then we have the maximum value in the cabin column and we have very Less in the impact column that is two so here if you don't want to see this numbers you can also plot a heat map and then you can visually analyze it so let me just do that as well so I'll say SNS do heat eat map and say y tick labels false so I'll just run this so as we have already seen that there were three columns in which missing data value was present so this might be age so over here almost 20% of age column has a missing value then we have the caping columns so this is quite a large value and then we have two values for imar column as well add a cmap for color coding so I'll say cmap so if I do this so the graph becomes more attractive so over here your yellow stands for True or you can say the values are null so here we have concluded that we have the missing value of age we have a lot of missing values in the cabin column and we have very less value which is not even visible in the Embark column as well so to remove these missing values you you can either replace the values and you can put in some dummy values to it or you can simply drop the column so here let us first pick the Agee column so first let me just plot a box plot and they will analyze with having a column as age so I'll say SNS do boxplot I'll say x is equals to passenger class so it's p class I'll say Y is equals to age and the data set that I'll be using is Titanic set so I'll say data is equals to Titanic data you can see the age in first class and second class tends to be more older rather than we have it in the third class well that depends on the experience how much you earn or might be the N number of reasons so here we concluded that passengers who were traveling in class one and class two are tend to be older than what we have in the class three so we have found that we have some missing values in M now one way is to either just drop the column or you can just simply fill in some values to there so this method is called as imputation now to perform data wrangling or cleaning let us first print the head of the data set so I'll say titanic. head sorry it's Titanic SC data let's say I just want the five rows so here we have survive which is again categorical so in this particular column I can apply logistic regression so this can be my y value or the value that I need to predict then we have the passenger class we have the name then we have ticket number Fair cabin so over here we have seen that in Cabin we have a lot of null values or you can say the na values which is quite visible as well so first of all we'll just drop this column so for dropping it I'll just say Titanic _ data and I'll simply type in drop and the column which I need to drop so I have to drop the cabin column I'll mention the axis equals to one and I'll say in place also to true so now again I'll just print the head and let us see whether this column has been removed from the data set or not so I'll say Titanic do head so as you can see here we don't have C in column anymore now you can also drop the na values so I'll say Titanic data do drop all the na values or you can say NN which is not a number and I'll say in place is equals to True Titanic so over here let me again plot the heat map and let's say or the values which were before showing a lot of null values has it been removed or not so I'll say SNS do heat map I'll pass in the data set I'll check it is null I'll say y te labels is equals to false and I don't want color coding so again I'll say false so this is basically really help me to check whether my values has been removed from the data set or not so as you can see here I don't have any null values so it's entirely black now you can actually know the sum as well so I'll just go above so I'll just copy this part and I just use the sum function to calculate the sum so here that tells me that data set is clean as in the data set does not contain any null value or any n value so now we have wrangled our data you can say clean a data so here we have done just one step in data wrangling that is just removing one column out of it now you can do a lot of things you can actually fill in the values with some other values or you can just calculate the mean and then you can just fit in the null values but now if I see my data set so I'll say Titanic data do head but now if I see over here I have a lot of string values so this has to be converted to categorical variables in order to implement logistic regression so what we will do we will convert this to categorical variable into some dummy variabl and this can be done using pandas because logistic regression just take two values so whenever you apply machine learning you need to make sure that there are no string values present because it won't be taking these as your input variables so using string you don't have to predict anything but in my case I have the survived column so I need to prct how many people tend to survive and how many did not so zero stands for did not survive and one stands for survive so now let me just convert these variables into dami variables so I'll just use pandas and I'll say pd. get dummies you can simply press tab to autocomplete I'll say Titanic data and I'll pass the sex so you can just simply click on shift plus tab to get more information on this so here we have the type data frame and we have the passenger ID survived and passenger class so if you run this you'll see that zero basically stands for not a female and one stand for it is a female similarly for male zero stands for it's not male and one stand for male now we don't require both the these column because one column itself is enough to tell us whether it's male or you can say female or not so let's say if I want to keep only male I'll say if the value of male is one so it is definitely a male and it is not a female so that is how you don't need both of these values so for that I just remove the First Column let's say female so I'll say drop first and true so over here it has given me just one column which is male and has the value zero and one now let me just set this as a variable let's say sex so over here I can say sex. head I just want to see the first five rows sorry it's dot so this is how my data looks like now here we have done it for six then we have the numerical values in age we have the numerical values in browes then we have the ticket number we have the fair and we have embarked as well so in embar the values are in SC and Q so here also we can apply this get dummy function so let's say I'll take a variable let's say embar I'll use the Panda's Library I'll enter the column name that is embarked so let me just print the head of it so I'll say imar do head so over here we have c q and s now here also we can drop the First Column because these two values are enough whether the passenger is either traveling for Q that is cown s for southam and if both the values are zero then definitely the passenger is from chair boo that is the third value so you can again drop the first value so I'll say drop and true let me just run this so this is how my output looks like now similarly you can do it for passenger class as well so here also we have three classes 1 two and three so I'll just copy the whole statement so let's say I want the variable name let's say PCL I'll pass in the column name that is p class and I'll just drop the first column so here also the values would be 1 two or three and I'll just remove the First Column so here we just left with two and three so if both the values are zero then definitely the passengers traveling in the first class now we have made the values as categorical now my next step would be to concatenate all these new rows into a data set or you can say Titanic data using the pandas we'll just concatenate all these columns so I'll say p. concat and it say we have to concatenate sex we have concatenate embal and PCL and then I'll mention the access to one I'll just run this okay I need to print the head so over here you can see that these columns have been added over here so we have the male column which basically tells whether a person is male or it's a female then we have the embar which is basically q and s so if it's traveling from cstown the value would be one else it would be zero and if both of these values are zero it is definitely traveling from chair Bo then we have the passenger class as two and three so if the value of both these is zero then the passenger is traveling in class one so I hope you got this till now now these are the irrelevant columns that we have it over here so we can just drop these columns we're dropping P class the imar column and the sex column so I'll just type in Titanic data do drop and I'll mention the columns that I want to drop so I'll say I'll even delete the passenger ID because it's nothing but just the index value which is starting from one so I'll drop this as well then I don't want name as well so I'll delete name as well then what else we can drop we can drop the ticket as well and then I'll just mention the axis and I'll say in place is equals to True okay so my column name starts from uppercase so these has been dropped now let me just print my data set again so this is my final data set guys we have the survived column which has the Valu zero and one then we have the passenger class oh we forgot to drop this as well so no worries I'll drop this again so now let me just run this so over here we have the survive we have the age we have the S SP we have the par we have fair maale and these we have just converted so here we have just performed data rangling or you can say clean the data and then we have just converted the values of gender to male then Embark to Q ands and the passenger class to two and three so this was all about my data ranging or just cleaning the data then my next step is training and testing your data so here we will split the data set into train subset and test sub set and then what we'll do we'll build a model on the train data and then predict the output on your test data set so let me just go back to Jupiter and let us implement this as well over here I need to train my data set so I'll just put this in dat heading three so over here you need to Define your dependent variable and independent variable so here my Y is the output or you can say the value that I need to predict so over here I'll write Titanic data I'll take the column which is survived so basically I have to predict this column whether the passenger survived or not and as you can see we have the discrete outcome which is in the form of zero and one and rest all the things you can take it as a features or you can say independent variable so I'll say Titanic data do drop so we'll just simply drop this survive and all the other columns will be my independent variable so everything else are the features which leads to the survival rate so once we have defined the independent variable and the dependent variable next step is to split your data into training and testing subset so for that we'll be using SK loarn I'll just type in from SK learn. cross validation import train test plate now here if you just click on shift and tab you can go to the documentation and you can just see the examples over here I'll click on plus to open it and then I just go to examples and see how you can split your data so over here you have X train X test y train y test and then using this train test plate you can just pass in your independent variable and dependent variable and just Define a size and a random state to it so let me just copy this and I'll just paste it over here over here will train test then we have the dependent variable train and test and using the split function we'll pass in the independent and dependent variable and then we'll set a split size so let's say I'll put it at 0.3 so this basically means that your data set is divided in 0.3 that is in 7030 ratio and then I can add any random state to it so let's say I'm applying one this is not necessary if you want the same result as that of mine you can add the random state so this basically take exactly the same sample every time next I have to train and predict by creating a model so here logistic regression will graph from the linear regression so next I'll just type in from SK learn. linear model import logistic regression next I'll just create the instance of this logistic regression model so I'll say log model is equals to logistic regression now I just need to fit my model so I'll say log model do fit it and I'll just pass in my X train and Y train all right so here it gives me all the details of logistic regression so here it gives me the class weight dual fit intercept and all those things then what I need to do I need to make prediction so I'll take a variable let's it predictions and I'll pass on the model to it so I'll say log model do predict and I'll pass in the value that is X test so here we have just created a model fit that model and then we had made predictions so now to evaluate how my model has been performing so you can simply calculate the accuracy or you can also calculate a classification report so don't worry guys I'll be showing both of these methods so I'll say from SK learn. metric UT classification report so over here I use classification report and inside this I'll be passing in y test and the predictions so guys this is my classification report so over here I have the preion I have the recall we have the F1 score and then we have support so here we have the value of precision as 75 72 and 73 which is not that bad now in order to calculate the accuracy as well you can also use the concept of confusion Matrix so if you want to print the confusion Matrix I'll simply say from sklearn do Matrix import confusion Matrix first of all and then we'll just print this so here my function has been imported successfully so I'll say confusion Matrix and I'll again pass in the same variables which is y test and predictions so I hope you guys already know the concept of confusion Matrix so can you guys give me a quick confirmation as to whether you guys remember this confusion Matrix concept or not so if not I can just quickly summarize this as well okay Jag says a yes okay s is not clear with this so I'll just tell you in a brief what confusion Matrix is all about so confusion Matrix is nothing but a 2X two Matrix which has a four outcomes this basically tells us that how accurate your values are so here we have the column as predicted no predicted why and we have actual no and an actual yes so this is the concept of confusion Matrix so here let me just feed in these values which we have just calculated so here we have 105 105 21 25 and 63 so as you can see here we have got four outcomes now 105 is the value where a model has predicted no and in reality it was also a no so here we have predicted no and an actual no similarly we have 63 as a predicted yes so here the model predicted yes and actually also it was a yes so in order to calculate the accuracy you just need to add the sum of these two values and divide the whole by the sum so here these two values tells me where the model has actually predicted the correct output so this value is also called as true negative this is called as false positive this is called as true positive and this is called as false negative now in order to calculate the accuracy you don't have to do it manually so in Python you can just import accuracy score function and you can get the results from that so I'll just do that as well so I'll say from SK learn. metric import accuracy score and I'll simply print the accuracy and I'll pass in the same variables that is y test and predictions so over here it tells me the accuracy as 78 which is quite good so over here if you want to do it manually you have to plus these two numbers which is 105 + 63 so this comes out to almost 168 and then you have to divided by the sum of all the four numbers so 105 + 63 + 21 + 25 so this gives me a result of 214 so now if you divide these two number you'll get the same accuracy that is 78% or you can say 78 so that is how you can calculate the accuracy so now let me just go back to my presentation and let's see what all we have covered till now so here we have first split our data into train and test subet then we have built our model on the train data and then predicted the output on the test data set and then my fifth step is to check the accuracy so here we have calculated accuracy to almost 78% which is quite good you cannot say that accuracy is bad so here it tells me how accurate your results are so here my accuracy score defined that and hence we got a good accuracy so now moving ahead let us see the second project that is SUV data analysis so in this a car company has released new SUV in the market and using the previous data about the sales of their SUV they want to predict the category of people who might be interested in buying this so using the logistic regression you need to find what factors made people more interested in buying this SUV so for this let us see a data set where I have user ID I have gender as male and female then we have the age we have the estimated salary and then we have the purchased column so this is my discrete column or you can see the categorical column so here we just have the value that is zero and one and this column we need to predict whether a person can actually purchase a SUV or Not So based on these factors we will be deciding whether a person can actually purchase a SUV or not so we know the salary of a person we know the age and using these we can predict whether person can actually purchase SUV or not so let me just go to my jupter notebook and let us Implement logistic regression so guys I'll not be going through all the details of data cleaning and analyzing the part so that part I'll just leave it on you so just go ahead and practice as much as you can all right so my second project is SUV predictions all right so first of all I have to import all the librar so I say import numi as NP and similarly I'll do the rest of it all right so now let me just print the head of this data set so this we've already seen that we have columns as user ID we have gender we have the age we have the salary and then we have to calculate whether person can actually purchase a SUV or not so now let us just simply go onto the algorithm part so we'll directly start off with the logistic regression or how you can train a model so for doing all those things we first need to Define your independent variable and dependent variable so in this case I want my X stus independent variable I say data set. iock so here I'll be specifying all the rows so colon basically stands for that and in the columns I want only two and three dot values so here should fetch me all the rows and only the second and third colum which is age and estimated salary so these are the factors which will used to predict the dependent variable that is purchase so here my dependent variable is purchase and independent variable is of age and salary so I'll say data set do iog I'll have all the rows and I just want fourth column that is my purchased column do values all right so I've just forgot one one square bracket over here all right so over here I have defined my independent variable and dependent variable so here my independent variable is age and salary and dependent variable is the column purchase now you must be wondering what is this iog function so iog function is basically an indexer for Panda's data frame and it is used for integer based indexing or you can also say selection by index now let me just print these independent variables and dependent variable so if I print the independent variable I have the age as well as a salary next let me print the dependent variable as well so over here you can see I just have the values in zero and one so 0 stands for did not purchase next let me just divide my data set into training and test subset so I'll simply write in from SK learn. cross spit do cross validation import train test next I'll just press shift and tab and over here I'll go to the examples and just copy the same line so I'll just copy this I remove the points now I want the text size to be let's say 25 so I have divided the train and test split in 75 25 ratio now let's say I'll take the random State as zero so Random State basically ensures the same result or you can say the same sample is taken whenever you run the code so let me just run this now you can also scale your input values for better performing and this can be done using standard scaler so let me do that as well so I'll say from SK learn. preprocessing import standard scaler now why do we scale it now if you see a data set we are dealing with large numbers well although we are using a very small data set so whenever you're working in a PR environment you'll be working with large data set where you'll be using thousands and 100 thousands of duples so their scaling down will definitely affect the performance by large extent so here let me just show you how you can scale down these input values and then the pre-processing contains all your methods and functionality which is required to transform your data so now let is scale down for test as well as a training data set so I'll First Make an instance of it so I'll say standard Scala then I'll have xtrain I'll say sc. fit fitore transform and I'll pass in my xtrain variable and similarly I can do it for test wherein I'll pass the X test all right now my next step is to import logistic regression so I'll simply apply logistic regression by first importing it so I'll say from SK learn from SK learn. linear model import logistic regression over here I'll be using classifier so I'll say classifier dot is equals to logistic regression so over here I'll just make an instance of it so I'll say logistic regression and over here I just pass in the random state which is zero and now I'll simply fit the model and I simply pass in X train and Y train so here it tells me all the details of logistic regression then I have to predict the value so I'll say y PR is equals to classifier then predict function and then I just pass in X test so now now we have created the model we have scaled down our input values then we have applied logistic regression we have predicted the values and now we want to know the accuracy so to know the accuracy first we need to import accuracy score so I'll say from SK learn. metrics import accuracy score and using this function we can calculate the accuracy or you can manually do that by creating a confusion Matrix so I'll just pass in my y test and my y predicted all right so over here I get the accuracy is 89% so if you want to know the accuracy in percentage so I just have to multiply it by 100 and if I run this so it gives me 89% so I hope you guys are clear with whatever I have taught you today so here I have taken my independent variables as age and salary and then we have calculated that how many people can purchase the SUV and then we have calculated our model by checking the accuracy so over here we get the accuracy is 89 which is great here is another question logistic regression is mainly used for options are a predicting a continuous variable b solving clustering problems C binary classification problems D performing Association rule learning if you know the correct answer please leave it in the comment section below now that we have covered both linear and logistic regression the next topic is linear regression versus logistic regression where Learners will compare and contrast these algorithms to understand their specific use cases and differences let's compare the two models so first of all let's look at the definition of linear regression and logistic regression so the main aim of linear regression is to predict a continuous dependent variable based on the values of the independent variables but when it comes to logistic regression the aim is to predict a categorical dependent variable based on the values of independent variables these are the main aim of each of these models now let's look at the variable type now in linear regression the dependent variable is always continuous all right this is very important to remember because this is the main objective of linear regression okay it makes use of continuous dependent variables to predict continuous values similarly when it comes to logistic regression you're going to use categorical dependent variable to predict a categorical value all right now let's look at the estimation method so guys linear regression is based on the least Square estimation which basically says that the regression coefficients should be chosen in such a way that it minimizes the sum of the square distance of each observed response okay now this is in depth about linear regression so that's why I'm going to leave a link in the description now logistic regression on the other hand is based on maximum likelihood estimation okay this basically says that the coefficients should be chosen in such a way that it maximizes the probability of Y given some value of x next is the equation earlier we discussed this equation where for linear regression we have y is equal b + B1 into x + e okay similarly this is the equation for logistic regression so the next difference is the best fit line so guys linear regression aims at finding the best fitting straight line which is also called the regression line all right but when it comes to logistic regression if you try and map the relationship between the dependent and independent variable you're going to get a curve which is also known as as a sigmoid curve all right so in linear regression the relationship between the dependent and independent variable is represented using a straight line but when it comes to logistic regression the relationship between the dependent and independent variable is represented using a sigmoid curve now let's look at the relationship between dependent and independent variable now when it comes to linear regression there has to be a linear relationship between the two so when I say linear I mean that the variables have to vary linearly okay that's how the straight line is formed in the first place all right when it comes to logistic regression it's not necessary to have a linear relationship now the output of linear regression is always going to be a predicted integer value or basically a continuous value so when it comes to logistic regression the output has to be a binary value okay so it should either be Class A or Class B or it should be zero or 1 something like that finally we have applications now linear regression is mainly used to predict out comes like the expected number of sales and you know it's always used to predict some continuous value all right but when it comes to logistic regression it's mainly use in classification so when you want to classify a data set into two different classes then you use logistic regression you can find a lot of applications of linear regression in the business domain and logistic regression is mainly used in the cyber security image processing and classification domain with a solid understanding of the regression techniques our next topic is decision Tre algorithm where Learners will learn about this versatile algorithm used for both classification and regression tasks what is classification and if I have to tell you about classification uh like for example what happens is like we have two type of when we talk about machine learning machine learning is nothing but you know like a series of m instructions you give it to the computer so that it can learn the patterns from your data set right to give you an example imagine that there is a trending topic for example you found it you want to find it out whether uh Prime Minister Modi will be uh the second prime minister once again the Prime Minister for the country or not okay so now what you will do is you will collect the data set from multiple different sources and uh you will you will actually build it like a algorithm uh where you you will get a label as yes or no yes he will continue as a next prime minister or no he will not continue as a next prime minister so you will collect the data set and you will feed this data set to the computer and this process is called as a machine learning right so now in this case what happens is um uh so this this is about the classification right so now machine learning is basically of two type one is called as a supervised machine learning another is called as a unsupervised machine learning and the third one is a reinforcement machine learning so when we speak about supervised machine learning as the name suggests it provides some supervision right for example the teacher teaching the kid is a supervised machine learning right so we will give the trained examples we will give the trained data set with a pure label on top of that uh this is called as a supervised machine learning so if I draw in front of you this type of machine learning look like this supervised machine learning where what happens is like you would have the data set which is a structur data set and you would have one column which is called as a label what you want to predict okay and you would have a uh various predictors by which you want to predict to give you an example imagine that you want to predict a pricing of a community okay you want to predict what would be the pricing of apartment in a particular Community right now this can be the variable like you can see that uh what would be the number of how many floors it has you can have a variable like what is a pollution level how how many educational institution are nearby right so based upon that the pricing will change but this type of supervised machine learning why it is called supervisor machine learning because we provide the independent variable or we provide the predictors also we provide the label data set okay now this supervised machine learning is basically of two type this supervised machine learning one type one first is called as the you know regression based supervised machine learning okay and the second one is called as a classification based supervised machine learning now what is the difference between regression based and a classification based super Vis machine learning regression based supervised machine learning is that machine learning where what you want to predict is continous in nature okay imagine the you want to predict the Comm Community prices right which is a continuous value if it is a continuous value then we will go ahead with regression based supervised machine learning okay whereas if you want to predict something which is the discrete outcome to give you an example you want to predict that whether I will win the match or not okay I want to predict whether the particular employee will churn out from the company or not you want to predict you know like whether uh the person will have a cancer or not you're getting my point right so if you have the output which you want to predict is in the form of yes or no or true or false right this is called as the uh supervised machine learning but a classification based supervised machine learning Okay so classif ifications based supervised machine learning is the process of dividing the data set into different categories or group by adding a label okay so always remember that whenever you guys want to predict the classes in the data set whenever you want to predict you know like whether this will happen or not whether the whether a person will do a credit card fraud or not you're getting my point right whether the employee will turn out from the company or not right whether the particular person will have a diabetes as a disease or not all these questions wherever you want to find it out yes or no or true or false or you want to predict classes in the data set this is called as a classification based supervised machine learning okay so this is what is called as a classification based supervis machine learning and today I will teach you you know I will tell you about various form of classification based supervis machine learning although we will do a deep dive into decision tree right now you will be able to understand that decision tree how decision tree is connected to the network what we are learning today with classification based supervised machine learning okay so now uh we have various algorithms algorithm is nothing but set of mathematical equations for classification based uh supervis machine learning we first of all we have something called as decision tree then we would have something called as random Forest then knife base and then KNN which is called as a K nearest neighbors okay so let me give you few statements about this algorithm and we have many others but today we will focus on one of them which is decision tree so now first let's start with decision tree now what is a decision tree what I was telling you is believe me or not decision tree is something you use every day in your um in your daily life for example you take decisions and for ex today also you took a decision to attend this webinar right but how do you decide a decision based on various further decisions right for example for today joining the webinar you have seen that okay uh when this webinar is about okay so you said it is weekday or weekend then you might have said check it out right what is the time of the webinar then you might have checked it out what is the topic of the webinar right so and who is conducting this webinar So based upon this you took a decision shall I go ahead or not go ahead you're getting my point right so this is what is called as a decision tree we call it as decision tree because it is a graphical representation of all the possible solution to a decision it's like a tree like decision tree is like a tree why because a tree also start with a root and then it emerge into various branches similarly you have a decision tree which I'm showing to you here as a simplest algorithm which is being used for machine learning purposes now it is something like this imagine that you want to find it out that you know like do you want to go to a restaurant or or do you want to buy an hamburger okay so you have two choices either you can go for a restaurant or either you can buy a hamburger now how would you decide which one you would follow so you will start with what is called as a root note you will start with a root note that whether I am hungry or not right if I am hungry right then only I will go for all these activity if I am not hungry at all then simply go and sleep you got my point right so this is how you will start start with the first note which is to find it out whether I'm feeling hungry or not if I'm feeling hungry then I will decide that well I do I have money which is around $25 worth if I have money then I will go for a restaurant if I don't have a money then I will buy an hamburger you understood this this is simplest representation of decision tree basically you decide something on the basis of the previous outcomes and you can imagine any sort of example here imagine that you want to find out whether the person will do a credit card fraud or not again it will depend upon the previous circumstances that you know for example it will depend upon how much is the salary of the person you it will depend upon what is a job profile of the person it will depend upon the fact that you know like for example how many fraud or how many card this person have right so on basis of you will decide that whether this will do a credit card fraud or not so this is the simplest form of classification based algorithm then we we have the next algorithm which is called as a random Forest now what is a random Forest as a name suggest you built one decision tree in the last example right but one decision tree can sometime be overfitting as a case right so people said that you know like why should I only trust one decision tree for example whenever you take a bold decision in your life you don't trust only a single voice right you want to hear it from multiple different people to make your decision more more stronger right go to a doctor if the doctor says to you that you have this type of a disease you don't uh believe in that what you do is you also talk to second doctor third doctor Fourth Doctor to confirm that this is true or not okay so this is what is called as a random Forest as the name suggests why it is called random Forest it is called random Forest because now you're building the various number of decision trees here it's like a forest of all the trees right it is no longer a single tree it is a forest of complete trees so for example you can imagine that if it is my training data set I will I will split my training data set into multiple examples multiple decision trees will be built and then based upon the majority I will decide whether should I do this or not okay so it is also called as a bagging sort of methodology where we bring the outcome of various models or we bring the outcome of various trees all together to make the uh to make our powerful decision okay so this is another examp this is another algorithm which is called as a random Forest then after random Forest we have something called as knife base okay now what is a knife base algorithm knif base is a also a simplest algorithm but knif base is basically based on the base theorem okay so this algorithm is basically based on the base theorem and base theorem is based on the conditional probability ility right so it is based on the conditional probability which is your n base now in b n base what happens is like we decide that whether something will happen or not on the basis of probability so let me illustrate you with this example imagine that you want to find it out whether I would have a disease or not okay so first of all the probability of having a disease is 0.10 and the probability of not having a disease is 90 okay so if there is a probability of having a disease is 0.10 you will further find it out that what is a probability that my test will be positive given I am diseased what is the probability that my test to diagnose the disease is negative given I have a disease right similarly if you go into this direction if there is a probability of not having a disease is 090 you will find it out that what is the probability of having a disease test being positive with having with no disease and what is the probability of no disease given I don't have the disease which is 090 so basically here you check the outcomes here you check the outcome of all the part all the possible combinations this is what is called as the KN base algorithm or base theorem or uh you know conditional probability based theorem okay then we also have something which is called as a key nearest neighbor which is the one of the finest algorithm uh which also helps you in deciding the uh classification right now what is K nearest neighbor K nearest neighbor as the name suggest what happens in this case is we try to build you know uh we try to build uh basically it's like a neighbors so it's something like this like if I give you the data set say I give you the customer data set okay and I it is a transaction data set so customer number one has bought a product number one right from a particular vendor at a particular rate and this is a profit this customer has given to us and this is the revenues okay this is my CL customer number one similarly you would have customer number two customer number three customer number four right now if I ask you that which customer profiling is same is nearby same right so what you can do is you can group your customers or you will get to know that which customer behave in the similar way so you can say that customer 1 customer three and customer four they behave in the similar way because they are giving us the high profit and high Revenue margins you understood so this is what happens in the case of K nearest neighbor so K nearest neighbor what you generally do is you find it out that what what uh I would be able to you know uh find it out who is my nearest neighbor like what are the similarities in the patterns we have right this is what we use for K nearest neighbors and this you can see that for example the algorithm based on the distance based mechanism find it out that how many people or what type of audience is similar to the outcomes right so then moving further here the with the second topic let's go into detail about what is decision tree all about right so so far I have touched based on classification based algorithm right and I was teaching you different type of classification based algorithm but since the focus of today's class is decision tree let's take a deep dive into the decision tree okay so let's get started with decision tree a decision tree is a graphical representation of all possible solution to a decision based on a certain condition what does it mean it means that it is as simple as that imagine that this is a tree right so it is like a tree where you have a problem statement that should I accept a job offer or not imagine that you want to find it out that should I accept a job offer or not this is a problem statement which is there in your mind now how would you solve this problem statement with the help of a decision tree first of all you will start with what we call as a root node we will start with a root node starting with you know to find it out what is a salary okay what I'm what what is the salary I'm getting so if my salary is equal to or greater than equal to 50,000 I'll go here if my salary is not greater than equal to 50,000 I will say I will not accept this offer okay now imagine that you say that your salary is greater than 50,000 then you will check another uh another variable here you will check it out whether I have to commute more than one hour if I have to commute more than 1 hour I will decline the offer you're getting my point right then you will if you don't have to commute more than 1 hour then you will still consider this option and then you will check it out for example in this case we are checking it out whether you are getting the free offers also like coffee or some snacks or other things like that if yes then you will accept finally the offer otherwise you will decline the offer you got my point this is how a decision tree Works basically decision tree will keep on splitting keep on splitting unless and until you are able to find it out your decision okay so here the decision was shall I accept this or not right so it will keep on splitting keep on splitting unless and until you get your decision whether you do this or not like this example which I have to I have explained you okay now with this what happens is like let me if I go further and explain you further on this decision tree let's understand this more importantly right right let's understand this uh one by one so imagine that this is my data set okay now in my data set you can see that I have various colors given to you it's like a green color yellow color red color red color and a yellow color I am saying if this is a if there is a green color fruit with a diameter of three right and I can say it is a mango whereas I'm saying if it is a yellow color and a diameter three still will be called as a mango if it is a red color with one diameter then it is a grap and it can be a red color with a diameter of one still can be a gray but even a yellow with a diameter of three can be lemon now what is happening in this case is if you have this type of a data set where you want to predict the label of the fruit right you want to predict whether a fruit will be mango whether a fruit will be lemon right or whether a fruit in this case is mango and lemon or grape I have to build a classifier which is a decision Tre classifier on the basis of this data set imagine this is a problem statement given to you now first of all what you will do here is you will take this data set right you will take this data set and you will start with a root node root note imagine here is like is my diameter of the fruit greater than equal to three or not okay is my diameter of the fruit greater than equal to three or not if it is greater than equal to three right now if it is greater than equal to three you can see that you have three fruits here three rows of the data set here one is green with three mango yellow with three lemon yellow with three mango and wherever it fails in the condition you are left with where the diameter is not greater than equal to three it is less than equal to three then which are the data set we have red one grape and red one grape okay so I hope you understanding this right how we have starting with the decision tree with the based on one condition here right which is diameter now based upon this what I will do is I have to split it further here in this case I don't have to split it because I already got the result so if my diameter is greater than equal to three in my data set if the diameter is not equal to greater than equal to three I know it is a gripe right whereas if the diameter is greater than equal to three then it can be a mango or it can be a lemon I'm not sure on my decision so what I will do here is I will split it further so I have to split this further here the splitting is not required but if I split this further I may check it out now is the color equal to yellow or not now if the color is equal to Yellow then I have two fruit which is your this row which is you know um your mango or lemon and if the color is not equal to Yellow then you will have the another row of the data set which is you know which you are left with right so this is how you have done this work in the case of your decision tree okay and how you will find it out that which type of the criteria or which type of the algorithm I should or which type of criteria or variable I should choose here to split the tree is on the basis of gen index and the Information Gain which I will illustrate and show you in the few slides from now okay so I hope with this you understand the how your decision tree Works basically on the basis of the condition and if you get a pure subset then no need to split it further if there is no pure subset you will keep on splitting keep on splitting unless and until you get a pure subset okay so in this case what has happened is you got 100% mango here you got 50% mango and 50% lemon okay so now this is what I have told you in the previous slide like how does it work right now this all depend upon the genie index basis right now in the next subsequent slide let me tell you and explain you about Genie index and Information Gain how does it work right how does that something works on the basis of uh your gen index so imagine that you know like imagine that I have a feature here is the color green or not right basis of whether the feat whether you have a color green or not what would happen here is if the color is green you will get this row here if it is not you will get this these two rows here right in the next it may decide on the other bases right it can be is the diameter which is greater than equal to three or not and many other things okay now let's go ahead the example and the questions which is coming to your mind which is related to you know decision tree terminologies I'm pretty sure you would have these questions in your mind and you would be thinking that how should I decide that which feature I should use and which feature shouldn't I be using this is a question which you had and this is a excellent question for understanding the decision tree but before I go on to that I have to tell you about some of the terminologies which we commonly use while we build the decision tree so first of all decision tree looks like this type of a tree based structure Okay so so every decision tree will have it root node so root node is where the decision tree will start so it represent the entire population or sample and it is further get divided or into two or more homogeneous sets so as you know that this will be the first feature on the basis of your tree will start like tree start with a root your in this case your decision tree will also start with a root node okay now once you have the root after that in a tree what happens is you will start getting the branches right I hope you understand what is branches in this case we will keep on splitting keep on splitting unless and until we get a decision whether this will happen or not so it's like a branches okay then we would also have a parent or a child node so child node is nothing but when you have a branches then the branches can also have the outcomes so in the previous example like for example we have whether the diameter is greater than equal to three or not you remember whether the diameter is greater than equal to three or not if it is true what is happening if it is a false what is happening so this is a child or child node basically this is a intermediate node this is not the final decision which is being made so this is called as a parent or the child node then we also have other terminology here which is splitting which you know that we will keep on splitting unless and until you get a desired node and finally the tree end with a leaf node so always remember one thing you will start your tree with a root node root node is a node with which you will start the decision tree and you will end your decision as decision tree at a leaf note where you will get a decision that you should do this or not okay and now uh pruning is a activity where uh you you will cut down the decision tree if it is pruned lot amount of times I will even explain you this it is a case of overfitting you shouldn't build thousand you can shouldn't build thousand uh branches of the tree when it is not even required so I explain you this point now let's move further here and let's see this with the help of an example which was your uh geni index and your information gain right so this is where you were asking me that which question to ask and when right how would you decide that which feature has to be taken first right let's take up an example here and I would en everyone of you to hear me really fine here because this is on the basis how would you decide or how the algorithm decide to break this further and and I'm explaining you with the help of one simplest example and we will do some maths over it so imagine that there is a data set where I want to find it out whether I will play the match or not okay there is a cricket match or there is a football match or whatever it is I want to find out whether I will play the match or not okay now how do the decision tree look like decision Tre look like this if the Outlook if the Outlook is humid if the Outlook is humid and if the Outlook is humid and humidity is very high then I will not play the match on the other hand if the Outlook is humid and humidity is normal I may play the match okay if the Outlook is absolutely clear then I will always play on the other hand if Outlook is windy and the winds are very strong I may not play the match on the other hand if the Outlook is windy and the wind are weak I may play the match so now what is happening again is that you are not sure how you decided with Outlook how you decided with these notes Here how you decided with these features here let me try to show you this entire data set so this entire data set look like this okay so this is is basically your 14 days data set and this exactly happens in the case of your classification based algorithm you will get a data set like this so you can imagine that you want to find it out I would like whether I will play or not right this is what you want to essentially find it want to find it out whether I would play or not on basis of what on basis of four different variables you have four different variables or features in the data set is Outlook temperature humidity and wind right so now uh it can be like if the Outlook is sunny temperature is hot humidity is high wind is not there I will not play the match this is how you will read this one data point right similarly I have multiple data point now I have to decide how can I build a decision tree and out of these four features which feature should I use first as my root node right so this is what I have to decide and let's do it uh accordingly right so now what I'll do from here is uh we I will illustrate you what is Gen index what is Information Gain and how does this happens right and this is basically used to build your decision tree so before I make you understand about Genie index or Information Gain one thing which you should always remember is to understand the concept of impurity what is impurity impurity is nothing but you can see there is a basket where you have apples right now if you have a basket which is of apple and in another uh tray it is written the label as Apple now in this case you will never make a mistake you will never make a mistake because here you have an apple here you have only one label so everything will be perfect it will be 100% basically there is no impurity there is no problem in your data set right on the other hand let me flip the story in this case imagine that I have different fruits in the basket which is like apple you have a banana you have a grapes you have a you know cherries and many other things right and you have many apples many labels here in this case try imagining you have to match each fruit with its label right now in this case the impurity cannot be equal to zero the impurity will not be equal to zero in this case is because what would happen here is that you have a chances of misclassification right this is a very important concept right when you have perfect thing the misclassification will not happen but whereas if you have a multiple labels with multiple fruit misclassification or impurities will not be equal to zero right so this is associated with a term called as entropy maybe in your childhood days you have learned about entropy in your chemistry class in a simple sense what is entropy entropy is a randomness of the space sample space whenever you are not sure on your decision then entropy will be more imagine that I'm giving you the data set where it is like 51% of doing this thing 49% of not doing this thing 51% chance that the employee may leave the organization 49% chance that employee may not leave the organization so basically you not sure on your decision if you're not sure on your DEC decision then the entropy will be very high on the other hand if you are very sure on your decision then the entropy will be very low so basically we need that feature which can provide us lowest entropy rather than the highest entropy uh to select ASAT as a good feature so we generally find it out entropy by the help of this formula what we simply do is don't get scared with this formula because everything happen automatically in r or python right imagine just look at this formula what is this formula this formula says that what is the probability that something will happen multiply by log base 2 probability that something will happen subtract this with probability that something will not happen into log base 2 probability that something will not happen Okay let's take up an example don't worry about it uh let's take up an example that probability that I will win the match you will apply here and probability that I will not play the match or win the match you will apply here and then you will calculate the entropy let's take uh you know an example how you can do this in our case so in our case let me show you yeah how we will build the decision tree in our case in our case you just see here what is happening is that we have uh 14 instances or 14 rows of the data set where nine times if you see it I will play the match and five times I will not play the match okay so if you see carefully there are nine labels where I'm playing the match and there are five levels where I'm not playing the match right so first of all I have to find it out the total entropy how I will find the total entropy this is being determined by probability that I will play the match match sub multiply this with log base power two probability that I will play the match what are the chances that I will play the match 9 out of 14 all of you will be with me right this is 9 out of 14 multiply with loog base 2 9 out of 14 subtract this with what is the probability that I will not play the match five out of 14 multiply this with log base 2 5 out of 14 so once you calculate this you will find it out entropy of this entire system entropy of this entire system is .94 okay so this is the entropy of your entire sample space this is the first thing now how this entropy will help you in selecting which features you will take or not so let's go further so now we will take each feature one by one whether I should take Outlook whether I should take temperature whether I should pick up humidity or whether should I pick up windy right let's go one by one now first I'm plotting for Outlook imagine for Outlook how many times so what are the distinct value of Outlook Outlook can be sunny Outlook can be overcast or Outlook can be rainy right these are the three different combinations you can have for Outlook now if the Outlook is sunny two times I'm playing three times I'm not playing the match if the Outlook is overcast I'm always playing the match if the Outlook is rainy three times I'm playing two times I'm not playing the match right this is how I have bifurcated it what I will do in the next iteration is let me find it out the entropy of Outlook okay so if we start with Outlook uh remember this formula which is you know probability that I will play multiply with the probability that I will play right and subtract with probability which I will not play and log base two of not playing so 2x5 is the chances that I will not I will play into log base 2 2x5 this is a subtraction here right with I will play and log 3 3x5 I will play right so you will calculate this entropy when Outlook is sunny yeah so you got this entropy when the Outlook is sunning is 971 accordingly you will proceed with calculating the entropy when the Outlook is overcast Outlook is overcast always you are playing right if the out out look is overcast every time you're playing so it means that you will get a probability of zero if you apply in that formula you will get zero and third what what what would happen if the if the Outlook is sunny in that case you will again multiply and you will put the formula and you will get it out 971 right so you will find it out entropy for each and every distinct combination of your feature you got my point right Outlook Being Sunny Outlook being overcast Outlook Being Sunny uh you know overcast sunny and rainy this should be replaced here right and then what you will do is you will finally calculate the Information Gain Information Gain is nothing but what you will do is you will pick it up the chances you will pick it up the entire chances when you are playing right which is five out of 14 you remember five out of 14 were the total chances that I will play into if it is sunny plus 4 out of 14 if it is overcast 5 out of 14 if it is rainy right and you will calculate the information from this Outlook and once you calculate the information you will subtract this information from your total entropy which I found it out in the last slide which was 0.94 you will subtract this and you will get the information gain or this is also called as a Information Gain from a particular feature so basically these type of a calculation first of all don't get scared away that you have to do this calculation but what I'm trying to explain you is this is how your algorithm will work for each and every feature it will calculate the information gain from your feature right if you have the more information gain it means that this variable is of very very important in predicting that something will happen or not okay so for Outlook I got the The Information Gain as247 with all the calculation remember then we will proceed with wind if the wind is there or not right and then I will proceed with wind and I will calculate the information gain and say Information Gain I found it out is 048 right similarly we will calculate for all the four of them let me put it together for all of you so this is what happens here now if I put in front of all of you these were the four four different variable I have Outlook temperature humidity and wind right I'm calculating the information gain for each one of them and the Information Gain I got for Outlook is 247 so if the Information Gain Is highest in a particular feature that feature will become your root node you got my point so therefore we will pick it up Outlook as our root node similarly for when the tree get started lat on as a branch node also your Information Gain will be calculated and wherever whichever feature is giving you more information gain that will be picked up later in the dis uh your tree also so this is how you will build your decision tree and you will finally get a decision tree like this okay so now what I will do is um you know like I will quickly show you how do we do the decision tree uh in your python okay so I'll show you how do you do this in Python and then I will uh summarize for all of you that why decision trees or tree based algorithms are better than your um other algorithm okay so how do you choose basically that which algorithm you will select and when okay so I'll I'll describe you this but before let's jump on to Python and go there so what I have done here is like I had built uh the decision tree in front uh like uh you know already so quickly I will walk you through the commands Okay so what happens here is that um in Python uh as you would be well versed with this that we generally import packages in Python so I'm importing numpy I'm importing met plotti for plotting the chart I'm uh importing various packages from your sklearn which is for machine learning purposes right so I am importing your label and order your decision tree classifier classification report and I also I'm importing your uh tree right so we have all this which why I'm importing right after I import what I will do is I'm reading my my data set so I'm showing you this with the help of a iris data set which is one of the very popular data set for building the decision tree right for any particular source so imagine that this is my data set and I'm just showing you six rows of the data set where you know like I want to find it out whether a particular flower species will be Sosa verol or virginica I have three different flowers which I want to predict and on the basis of SE length petal length seal width and petal width so basically I have different dimensions of flowers length and width and based on that I want to find it out whether the particular species will be Sosa versal or virginica you got my point right so this is a data set so what I will do is I as you know with every machine learning data set we play with the data set so I'm checking the information here uh like what type of the data type it is so C length it is a float Peter length it is a float and species is a object why object because this is a categorical column so after this I'm also checking whether there is any null value present in the data set or not because if there is any null value then we have to get rid of that null value or we have to replace that null value with some imputed value right this is what I'm checking here once I do that I'm also plotting this because we usually do visualization right to understand the data set better so what I'm doing here in the with the help of SNS which is your SNS dopay plot I'm plotting all the possible plots so basically SE length with seel width what type of the combination look like so Cossa versical and virginica there are three different species you can see in the data set and this is what I'm getting a trend between your SE length and seal width similarly this is basically from seal width to Peter length right so this is how I'm understanding the patterns or I'm understanding the relationship between the variables in the data set so this is what I am understanding here I am also checking whether there's a correlation or not higher the shade it means there will be a strong correlation so all of these things is being done as a part of exploratory data analysis before even you start your machine learning model right once you do this after that what I have to do is after that what I'm trying to do is I am taking your specious column what I want to predict as my target variable which is your dependent variable and what I so this is my the dependent variable and with the help of what I want to predict will be my independent variable so I'm calling X all my independent variable and I'm calling Target as my dependent variable once I do this then you will also think about it that your um the variable which I want to predict is the flower species right but I want to convert this into zero and one class 012 class because your computer cannot understand text right computer can only understand the numbers so what I'm doing here is I am uh changing it to the uh I'm changing it to the class right so what I'm trying to do here is I'm saying wherever it is Cossa it will become zero wherever it is virginica it will become one and it is vers color as a third category it will become two so now imagine my data set my label to predict becomes 0 one or two in instead of three flower which was Cossa versal and virginica this is what I'm doing with the encoder here once I convert this and this become my target what I will do is as I told you that I will split this data set into training and test data set so basically 80% of the data is going into the uh training data set and remaining 20% I'm taking as a test data set okay now I will call decision tree classifier you know like I want to make a decision tree and I want to fit this on my training data set and the test data set and I will start building the decision tree and with the help of you know from uh so here is when I have created the decision tree and here is when I'm checking the prediction how accurate my decision tree is so I got my Precision which is good I got my recall I got my F1 score and I also get the support right so all these matrices are being used to calculate your uh how accurate your predictions are so higher the Precision better the results would be right and finally I'm showing to you that how does the tree look like so I had tried to uh plot this decision tree in front of you so basically it start with Peter length and you can see the genie index coming up here or the Information Gain which is Information Gain and Genie index are reciprocal to each other if you want to use Information Gain you will get that score if you don't want to use Information Gain you will get Genie index so they are both reciprocal of each other it is one and the same thing you use Genie index or you use Information Gain they are the two different Matrix to build your decision tree so you can see that based on a Peter length the tree gets splitted like this then based on the Peter length uh of different dimensions your tree further split then it further split then it further split and finally you get to know that whether the particular species will be versic or virginica okay so this is how your entire decision tree is built in in Python okay so this is how it's so simple I know uh it takes time to build this thing but once you are a good data scientist and you understand all these things it is very simple to build all these things very easily in python or R right so now uh finally going back how would you decide how would you decide that which algorithm you know which algorithm will be taken when so uh what happens is this is on the basis of pyit learn so it it starts something like this okay so it is on the basis of like this that first of all you see here that um whether how many samples you have how many data points you have in a data set right if you have more than 50 samples then you will go here if you have less than 50 data point then you know you will go here so you will get more data set right so if you have more data if you have greater than 50 data point then you will go further you split of your data set if it is not then you will you just the kind advice it get more data set okay so then you will decide that what you want to predict if you have a label data set then you will go here right and do clustering if you don't have the label data set then you will see whether you want to predict a quantity if yes you will go in regression if you want to predict uh if you just want to do exploratory analysis you can do dimensional reduction if you have a label data set uh you know for classification you can do all this classification so basically this is a cheat sheet which we generally use to decide that what we have to do with the data set and when here's a quick question for you guys in a decision tree what do leaves represent and the options are a input features B decision nodes C final outputs or classes D branching conditions if you know the correct answer please leave it in the comment section below building on a knowledge of d trees the next module is random Forest where Learners will explore this powerful emble method that combines multiple Deion trees to improve predictive accuracy let's understand what is a random Forest So Random Forest is constructed by using multiple decision trees and the final decision is obtained by majority votes of these decision tree so let me make things very simple for you by taking an example now suppose we have got three independent decision trees yeah we are just taking three decision trees and I have got an unknown fruit and I want that these trees would give me a result of what exactly this fruit is so I pass this fruit to the first decision tree the second decision tree and the third decision tree now a random Forest is nothing but a combination of these decision trees so the results are been fed into the random Forest algorithm so what it sees is that okay the first decision Tre classifies it as Peach the second decision tree says that it is an apple and the third one says that it is a peach So Random Forest classifier says that okay I've got the result as two Peach and one for an Apple so I would say that the unknown fruit is an Peach all right so this is based on the majority voting of the decision trees and that is how a random Forest classifier comes to a decision of predicting the unknown value okay so this was a classification problem so it took the majority vote now suppose if it was an regression problem it would have taken mean of it okay so now let's move on further to understanding what is a decision tree but before that we should understand that random Forest the building blocks are decision trees and that's why studying decision tree becomes important because if we understand one decision tree we can apply the same concept to random Forest okay so now let's move on forward and understand the important terms in random forest and this will also help us consolidate whatever we have learned so far so we have taken the same small decision Tre of the previous example and let's understand these are also the important terms which will be relevant to random Forest also so the first is the root node now here what happens is that the entire training data is been fed to the root node and then we've got here that each node will ask either true or false question with respect to one of the feature and then in response to that question it will partition the data set into different subsets that's what it is it is doing here based on the condition that it if the mass body mass is greater than equal to 3500 it asks a question either yes or no and based on that again for the partition is done and if not then it just classifies the species and then again what happens is that the splitting now this is very important here the splitting takes place either with the help of a Genie or entropy methods and these helps to decide the optimal split and we will be discussing about splitting methods very soon right okay and then we've got the decision nodes which provide the link to the leaf nodes and these are really important because then only the leaf nodes would tell us what actually the real predictions or to which class does this species belong so now coming to the leaf nde and these are the end points where no further division will take place and we will obtain our predictions okay so now coming up to another important thing here is working of random Forest so now for working of random Forest we will have to understand a few important Concepts like random sampling with replacement feature selection and also the emble technique which is used in random forest and that is bootstrap agregation which is also known as bagging so we will understand this with the help of an example which will be very simple and then we will go on understanding how feature selection is done in both the classification and the regression problem actually how random Forest select features for the construction of decision trees well in random Forest the best split is chosen based on jny impurity or Information Gain methods so this also we will understand now let us first understand random sampling with replacement now what happens here is that we've got a small subset of the same penguin data set wherein we've got some six rows and four features that means four columns and the arrows that you can see is that now we will be creating three subsets from this small subset right and these three subsets will become our decision trees and then we'll be constructing decision trees from these subsets so let us create our first subset and you can see here that the subset is randomly being created and for convenience sake let me just also show you the different subsets here okay so now for better understanding let us understand this that in the first subset if we focus we've got certain random rows here and we've got certain feature but we do not know how this feature has been selected we got Island and we got body mass but in the second subset we got Island and flipper length and in the third subset we got body mass and flipper length right now let's look at the rows now when I am talking about these features I will say this is feature selection and remember this term now coming to the second concept that is random sampling now random sampling is nothing but selecting randomly from your subset so I'm selecting randomly certain rows from my subset and creating further subset okay so what is replacement here replacement is can be seen here and can be understood with the second subset we see here that the gento species the throw is being repeated again and this is replacement that means that when we are working with repeated rows and this row can be repeated again in the second or the third subset then this is random sampling with replacement that means my random Forest can use a row multiple times in multiple decision trees right so this is the basic concept of random sampling with replacement and feature selection in random Forest another important term which I would like to bring into the notice is that when we are working with these type of small subsets these are also known as of bootstrap data sets and when we aggregate the results of all these data set it becomes bootstrap aggregation so just filling in the gaps so that later on the concepts become more clear so now let's move on to drawing decision trees of these subsets okay so let's draw the decision tree of the first subset again we taking body mass as the first root note and then based on a decision like if the mass is greater than equal to 3500 then take a decision either yes or no if it is no then the species is gin STP and if it is yes then again you partition based on island and if it is togin then it is Edy and if it is bisco then it is gento species okay so this is how we will construct two more decision trees of the remaining subsets so on the second subset let us just again create decision tree and here now we are taking flipper length and then based on a condition that if the flipper length is greater than equal to 190 then make a split if it is yes then the specie become gentu and if it is no that means again make a decision based on island and if it is Sten it is Ed and if the island is Dream Island then it is a chinst STP spey so this is how the decision tree of the second subset has been created and this is how it will take decisions right based on the tree length depth and also the features it is selecting okay so now now let's create the third decision tree of the third subset and we get a decision tree something like this where in body mass if it is greater than 4,000 and if it is yes then clearly it is a gento species and if it is no then again make a partition with the with respect to flip or length another feature here and then if it is again greater than equal to 190 then the species would be edly else it would be chin STP so this is how decision tree three will make a decision now let's just keep these decision trees with us okay and we will make sense of these trees just in a while okay but before that let us understand how feature selection is done in a random Forest how am I selecting The Columns so for classification by default the feature selection is taken as the square root of total number of all the features now suppose I've got here four features so it is a classification problem I will take the square root of these four features which becomes two so decision tree would be constructed based on two features each if suppose I had 16 features then it would be square root of 16 that would be four so four features would be taken in each decision tree all right and suppose if this would have been a regression problem then by default what would happen the features would be selected by taking the total number of features and dividing them by three okay so this is how by default the feature selection is being done by a random Forest okay now let us move on forward to consolidating our learning so now we are coming to emble techniques that is also known as bootstrap aggregation random Forest uses emble techniques and what is embling it just means that you are aggregating the result of the decision trees and taking the majority vot in case of classification and the mean in case of regression problems and giving the output okay so now we have again plotted all our decision trees here and Below we can see that there's an unknown data and I want to predict the species of this data so what will happen is that again let us just feed this problem to each of the decision tree and let's see what each decision tree makes the prediction so I just feed this unknown data to decision tree one and it says that okay the fishy seems to be chin strape okay and then decision tre2 says that based on the data it has been found that the speci is edly and then decision Tre three says that no I I with my decision tree this specie is chin strip okay now all these data has been fed to random forest classifier and it says that okay for chin strip I've got two votes for Ed it's got one vote so the new species would be chin strap right so this is how the bootstrap aggregation is done based on the majority voting and the decisions taken by different decision trees they have been combined together aggregated and we get an ensembled result in the random Forest okay so this was very simple concept of Ensemble techniques which has been used in random Forest okay so now let's move on forward to splitting methods so what are the splitting methods that we use in random Forest so splitting methods are many like J impurity Information Gain or Kai Square so let's discuss about J impurity so J impurity is nothing but it is used to predict the likelihood that a randomly selected example would be incorrectly classified by a specific node and it is called impurity metric because it shows how the model differs from a pure division right and another interesting fact about gen impurity is that the impurity ranges from 0 to one with zero indicating that all of the elements belong to a single class and one indicates that only one class exist now value which is like 0.5 this indicates that the elements they are uniformally distributed across some classes right now moving on forward to Information Gain now this is another splitting method which random Forest can use and Information Gain utilizes entropy so entropy is nothing but it is a measure of uncertainty so Information Gain let's talk about that first so the features they are selected that provide most of the information about a class right and this utilizes the entropy concept so let's see what is entropy this is a measure of randomness or uncertainty in the data right so we will understand this entropy with the help of a small example so don't worry about it so let's understand this entropy now suppose there's a fruit tray with four different fruits right and uh what do you feel about the entropy here that means a randomness of the data is it really easy to classify these fruits into the respective class so this becomes really uncertain and the data looks messy here but what if we just split here the into two trays where in the first tray would have peaches and oranges and in the second tray we'll have apples and lemons so now this becomes a little more certain we get low Randomness here and this is called as low entropy so when we move down from tree that means from root node to the leaf nodes the entropy reduces and we can also calculate Information Gain from this entropy that is the difference in entropy before and after the split that is known as Information Gain okay so once we move down the tree and start reducing the randomness from the data the entropy becomes lower and that is what we want in our data if there's low entropy that means we are likely that the predictions would be more accurate and we can make predictions very easily as compared to very messy data which has high entropy okay so that was about entropy and now let us just move on to the Practical demonstration or a handson on random Forest okay so now it's time for an Hands-On on random Forest so let us just import a few basic libraries of python in our Jupiter notebook and we will run this we will import pandas SPD nay NP and cbon as SNS now cabon is need needed here because we want to load a data set that is a penguin's data set with the help of cabon and this is already been preloaded in cabor this is already loaded data set and cbone has got mult multiple data sets you know for practice for beginners so it is a good way to practice for data sets now you can see this asri sign that means it is telling us to wait so let us just let it get loaded so we get got our data in an object called DF and we can see the first five entries here and this uh data frame is shown in the form of a table rows and columns and we see here some species Island Bill length Bill depth flipper length body mass and the sex of the penguin so our task is to specify or to classify these species of penguins into the respective correct species right so we see the shape of our data and we see that it is like 344 rows and seven columns and we will see the info so we see DF doino and this gives us along with the non-null count we also get the data type of the values so we have got species Island as the object data type whereas the bill length Bill depth triple length and body mass are in floating point or you can say floating data type and the sex is in object data type right so now moving on forward to calculating how many null values are there with the help of DF do is null do sum so we get certain like some around two null values in all these columns as you can say the features like Bill length Bill depth flipper length and body mass whereas there are 11 null values in sex feature right so what we do is since they are very small null values we can just drop it or you can also ignore them so here here in this data frame what I'm doing is I'm just dropping these null values and let us just check whether the they have been dropped or not with the help of again the same function do is null do sum and then we see that yes they are been dropped from the data frame now let us do some feature engineering with our data now we have seen that we have got some object data type in our data frame and before feeding it into algorithm That Is Random Forest we have to transform the categorical data or the object data type into the numeric so we are using here one hot encoding to convert the categorical data into numeric now there are various ways in Python which we can do that like one hot encoding or you can also use mapping function in Python but here we are using one hot encoding so let us just do that and we find here first of all let us apply it on the sex column and here we see that we have got two unique values in sex that is male and female and we use pandas here to get dummies that is how we will apply this one hot encoding because is this is how get dummies work so what happens is here is that the new unique values are converted into the respective columns in the data frame so we see here we have got two unique values males and female and they are being converted into the columns okay so one thing to note here is that we also get a problem of dummy trap because here we see only two unique values now suppose if I had six or seven unique values and I do this one hot encoding I would have lots of features in my data frame and that would lead to several complexities so what I do is uh to keep things simple I can use one hot encoding when my data frame or my unique counts are low when my unique values are less so since I had just two or three I can use it so I'm using here so what I do is again now one row one column as we can see here that it is redundant giving me extra information so I will just drop it so I drop this First Column and what I get in this data frame is only me so let us just infer whether I can also infer females from this or not so if the value is one that means the penguin is a male and if the value is zero that means the penguin is a female okay so only one column is needed for this data frame so I just kept one and dropped the other one okay now apply again one hot encoding to the island feature so in Island if we check the unique values we've got three unique values here togis bisco and dream island and the object is the data type right so again we will use pandas pd. get dummies and we will use apply it on the feature Island and let's get the head of it so we get here again the unique values were converted into columns and we get here respected three columns and then again we will just drop the First Column to get the remaining two columns so here also we can infer that if the island is toen if it is one then it is not dream neither bisco right so this is how you can read it from the data frame and understand that now remember this thing that these two Island and here sex these are two independent data frames these are not yet included in the main data frame so what we will do now is we will concatenate the above two data frames into the original data frame so what we do we again create a new data frame that is new data and let us just concat with the help of pd. concat function and we will concat what DF Island and sex and Xs is one that means in the column Okay so so when we will run this let's see the head of it so everything gets concatenated in a single data frame which is good for the feeding this data into or splitting the data into test and train data so now we have this new data frame and we've got some repeated columns here which needs to be deleted so what we do is we will delete sex and Island here which are just repeating because we've got here mail and we have also got here dream and togon so we do not require this island column neither this sex so we just drop it with the help of new data. drop and the column names X is one in place equals to True right and let's see the head of this data frame head of the data frame gives me five unique values right and now it is time to create a separate Target variable and what we'll do is we will store in a variable called y only species so what we do is from this new data. species we will just store the species in this Y and we see this y do head that is the first five species and we got the values here that means another Target variable is being created now so and you can also see the Y do unique values as Edy chinstrap and gentu so now we see here three unique values of the penguin that is chinstrap Ed and gentu and the data type is object here so again we need to convert this object into the numeric data type so now what we are doing is we are using the map function in Python and what we do is we map Ed to zero gin strap to one and gen 2 to two so this is how we see then all the values have been mapped to numeric this is another way to convert a categorical value into a numeric value in Python now what we do is let us just drop the target value species from our main data frame so we just drop it and let's see our new data frame so we see that we don't have any Target species here right okay so in X let's store this new data and perform the splitting of the data so what we do is from SK learn. model selection we will import our train test split and we will split our training data into 70% and 30% so test data becomes 30% and training data is some 70% and this random state is zero which means that I'm not fixing any random State and this is also used for the code reproducibility now suppose if I again run this code I will get the same result it will not change you can set this random state to any of the random number as per your choice and the result would differ okay so now let us print the shape of X Trin y train X test and Y test so we see here that it has been splitted into 70 and 30% and we get X train has 233 values here and seven features and X test has 100 values and seven features similarly y train you can see 233 values and Y test has 100 values that means the species okay so that has been perfectly splitted into 70 and 30% now what we do is we will train the random Forest classifier on the training set how do we do it we will import the random forest classifier from Escalon emble so we've already dealt with what is emble and then in classifier we will store this random forest and this n estimator is nothing but decision trees so we are creating some five decision trees here and the criteria is entropy and again random state is set to zero so let's see and then we will fit this xra and white train so this has been fitted and the criteria is entropy here all right so now let's make some some predictions and let's create a variable called y predict and we will just predict it on X test and we've also printed this y prediction and now let's print the confusion Matrix to check the accuracy of random Forest algorithm and what we do is from metries escal and matrices we will import classification report and confusion Matrix and also the accuracy score so we will just import them and then in cm variable we will print the confusion Matrix of Y test and Y prediction so we will print it and we see here the accuracy score also which is 98% so our random Forest classifier is giving us a very good accuracy of 98% and you can see your confusion Matrix that only two cases have been misclassified rest all the cases have been correctly classified by random Forest classifier okay so now let's move on to printing the classification report of v test and by prediction let's see and we get the precision as 96% that means the two predictions by the algorithm is 96% the recall or the true prediction rate is 100% which is very nice and Evan score is also good which is 98% so this is giving us a good result but what if if we change the criteria from entropy to Genie so let's just experiment with that too so let's try this with the different number of trees and change the criteria to geni coefficient so now again from Escalon do emble we will import random forest classifier and fit it okay and here here what we are doing is just we are using Seven Trees previously we used five and now in the criteria we will use jinny coefficient and random state is zero so let's run this and see whether there's a change in accuracy or not and let's predict this and let's check the accuracy score what is the accuracy score for this random forest classifier with Seven Trees so we get 99% accuracy with changing the criteria and changing the number of trees so you can just experiment with different number of trees and different number of decision trees let's just experiment with you know 12 decision trees and see what happens so you can see the accuracy reduced to 98% okay with seven we were getting 99 so let's just keep seven because it is giving us really good accuracy so this is about random forest classifier and how it works with several trees and different criteria to give us very good accuracy on our training and test data here is another question what is the primary advantage of using a random Forest over single decision tree and the options are a it is faster to train B it reduces overfitting by averaging multiple trees C it requires less data preprocessing D it is easier to interpret if you know the correct answer please leave it in the comment section Below in our previous module we covered random Forest we now shift Focus to the KNN algorithm where Learners will learn about the K nearest neighbors algorithm its Simplicity and its application in class classification and regression tasks the case that we are discussing is basically the KNN algorithm which is an algorithm which we used for mostly supervised machine learning which is where you have labeled data okay so what is KNN algorithm so as we said KNN algorithm is K nearest neighbors algorithm and it is an example of supervised learning algorithm where basically you try to classify a new data point based on the neighbors of that data point which is basically which data points are closer to it for example here as you can see you have on one side couple of gats and on the other side you have couple of on one side you have dogs and on the other side you have cats right now if a new data uh point is given to us there is a picture of a new animal and if it is lying somewhere here right then we know that it is nearer to the cats right and therefore we will classify it as cat whereas if it is sort of uh nearer to the dogs then we classify it as dog right so that's the like you know neighborhood for the dog and therefore we sort of uh classify it that new animal or the new picture as as being that of a dog and this is quite uh uh you know this is something which is we even see in our in our regular day-to-day life right you know we have had examples where our parents keep telling us okay don't play with those kinds of you know children or something because they are not good in their studies or probably they are not so good in their behavior because you would become like them right so it's again an example example from real life of classifying a particular person based on the company that they keep right or from the uh with the kind of people that they are so that's that's sort of uh the example and now let me actually go back okay what are the features of of K nearest neighbors algorithm okay so let's talk about uh the features of Canan so as as I said Canan is a super vised learning algorithm typically it is uh you know uh used for supervised learning kind of problems it's very simple as we mentioned intuitively you can uh you know it's about you know what kind of neighbors do you have so your class is predicted based on the your nearest neighbors as the name suggests and then it's a non-parametric technique so I would like to spend a couple of minutes here to discuss about what we mean by by non-parametric so typically you know the supervised machine learning algorithms are of two kinds right one is the parametric types and the second one is the non-parametric type when we say parametric what we mean is basically that the machine or the algorithm assumes that there is an underlying function or a distribution that is known of the particular data set like for example the linear regression would assume that the relationship between two two things X and Y is is linear in nature right um and and and similarly for a distribution a gajian distribution it would assume a normal distribution of the data points and so on so a lot of the supervised machine learning algorithms they assume some kind of function association between the predictor which is basically the root causes which help you in predicting and the variable that you are trying to predict whereas the there are certain algorithms like the KNN or the paren window or the linear discriminant analysis which is of the kind which is called non-parametric because it does not assume any particular kind of distribution or any particular kind of functional relationship of the data that you are trying to you know predict or you're trying to learn the the pattern of so KNN is one of the kind as I said pen window is another one uh which is is basically where you uh in in Parson window essentially you know the the volume of the data set uh or the area that the data set covers is known whereas you're trying to find the K there where which is the number of data points within that area or the volume right whereas in case of non-parametric technique like KNN it's the opposite where the K is known which is basically you would like to associate the class of of the data point that you are trying to predict based on the K number of neighbors around it now K can be three 4 five whatever right 10 20 and so on essentially the difference between pars and window and KNN is that in KNN you already know the K and then from the K you try to find out the volume and therefore then you try to find the the probability or the density underlying distribution and then there is the the disc discriminant analysis which is of again two kinds linear discriminant and multiple discriminant analysis where basically what you do is you you transform the underlying data the features into a higher Dimension and in such a way that in the new feature space after you have transformed the data you now try to apply a parametric approach like for example you will try to project the features onto a line or if it is on a sub Subspace which is higher Dimension than a line then essentially it becomes multiple discriminant analysis so basically um those are the three kinds of non-parametric techniques so even if you were not able to sort of get the full hang of what these three types are what you need to keep in mind is that non-parametric technique does not assume any kind of distribution or any kind of functional relationship of the underlying data and therefore it gives us a lot of flexibility whereas the parametric techniques they basically assume some kind of functional relationship between the data points uh or they assume some kind of distribution so where would you use a non-parametric versus a parametric technique right so basically you would use non-parametric technique you know where you do not know about the functional relationship that is one second is that you know there is uh maybe let's say large amount of data and so on and thirdly uh non-parametric technique like Cann does not work in very high dimensional data so you would also use parametric techniques in that case whereas if you have you know smaller data sets you would use uh you know typically non-parametric techniques and then also if you know understand that the relationship between the data points might be for example linear or something like that you would use a parametric technique so you're assuming that there is some kind of relationship like a linear relationship or something like that between the data point so that's where you will use the parametric technique so again just to summarize basically non-parametric is an approach where you do not assume any kind of distribution or functional relationship whereas parametric assumes a functional relationship or basically a distribution between the data points the other feature of KNN is that it is a lazy algorithm so what do we mean by lazy algorithm actually in most of of the supervised uh learning algorithms uh you basically train your model on the training data set then you have your model and then you apply this particular model to the test data set to then classify or predict uh you know for example whether a new image is that of a cat or a dog right so this can be you know some kind of algorithm like support Vector machine or regression or logistic regression or whatever right so you run this logistic regression or you run this regression or support Vector machine on the training data set and it learns the features or it learns the parameters of the model from that data set and then applies this learned model on the test data set whereas in case of KNN actually there's no training Step at all that's why it's called a lazy algorithm because what it does is at the time that you actually now want to predict at that point in time actually it will go and it will do all the calculations of the distance of the new data point like for example the new image of the cat or dog from all the other data points that you have so it will calculate all the distances and then we'll check for the those data points which are or the K data points which are nearest to this so that's why it's called lazy algorithm because nothing happens till the point or no calculations happen till the point you are actually trying to predict something so there is no training step involved okay and then it's used for both classification and regression as we just mentioned so it can be used to predict the values as well as be able to classify something like you know okay whether it is a cat or dog or if you're trying to let's say predict some value some forecast or something that for that also you can use it and then it is based on feature similarity which is basically what do we mean by feature similarity so feature can be things like like if for example you know you are looking at classifying cats versus dog right so is the eyes like a dog that can be one of the features is the how do the ears look that can be one of the features what about the tongue the face and so on so there can be multiple such features and how similar it these features are between two data points which is used basically uh by the KNN algorithm and then as I said there is no training step involved so these are the features of KNN algorithm and therefore now let's look at actually just some simple examples of how it works so as you can see here in this slide we have two classes of data so one is the all this blue data points and there is another one which is the orange data points now if you have a new data point which is this pink one here which class should it belong to should it belong to class A or should it belong to class B so what you would do is you would actually start calculating the distance of this pink data point from every square or blue triangle data point and then you will decide uh you will have to assume a particular K let's say k is three right which is I'm looking at the nearest three data points and in that case basically as you can see if we draw this Circle right then we see that two of the nearest data points within that circle is of the square orange kind so basically we will predict that this particular new data point belongs to class A whereas K value was seven right as in this particular example now you would see that four out of the seven is actually of the blue triangle kind and therefore we will now classify it as belonging to class B so essentially this prediction changes as you can see here depending on what is the K value so therefore the question is what should be the value of K right and typically what happens is you run a trial and error and basically you come up with okay what is the best K value but essentially what one needs to understand is that as the value of K increases basically the the partition line starts moving towards becoming more and more linear so it starts becoming less flexible and it starts assuming some kind of a linear dividing line or something like that so what happens is that in that as you increase the k your bias basically increases but your variation reduces so we know that in classification problems or in machine learning problems bias and variance are two things that we are trying to manage right bias is basically how close you are to the actual class or or to the actual uh value whereas variation is how much variability is there in your prediction so uh as the K increases the bias sort of increases but the variance reduces and then it is vice versa so if your K decreases let's say at K is equal to one where you are only looking at just one nearest neighbor and then predicting based on that actually the bias is the least which which means it is the most flexible K is equal to 1 is the will give you the most flexible uh sort of demarketing line or function whereas the variability will be the maximum so that that's the sort of the tradeoff and that's how we actually determine K so we have to get a k value in such a way based on trial and error that sort of maximizes our uh sort of or reduces the bias as well as the variance and and that's the kind of optimiz we are trying to do okay so how do we calculate the distance itself right and distance typically can be of many kinds so you know the example here is of the ukian distance but you can have other kinds of distances like Manhattan distance or mahalanobis distance and you can look up references for other kinds of distances now ukian distance is calculated for the point P1 and P2 as given here essentially ukian distance is nothing but the you know square root of sum of the x coordinates of these two points P1 and P2 and then y coordinate square of uh of of these two points P1 and P2 and uh this is just an example of one kind of distance and other kinds of distances like man Manhattan or mahalanobis are also there and then this calculating this distance becomes quite challenging especially in cases where you know you are trying to for example calculate let's say how close to LinkedIn profiles are right or trying to classify the category of electrocardiogram and so on so forth so there we have to bring in more creativity to just decide what kind of distance to use okay now let's move ahead we will now talk of some use cases where KNN can be used and this is an example of how KNN can be used for book recommendation so if you have purchased books on Amazon or wherever right some of these recommendations are based on on KNN algorith and then you know as we said you know KNN is like based on features right so maybe let's say what will be the nearest neighbors of a particular book it can be based on who is the author what is the topic and so on so forth and then there are other use cases like I mentioned so for classifying satellite images for classifying handwritten digits uh on on image analytics or or for classifying electrocardiograms um Etc uh you know typic KNN can be used okay so now actually we will get into some Hands-On okay so to start the Hands-On session I'll go to this Jupiter notebook that I already have installed on my system and I have a certain code written which uh we will take two examples both the examples are based on data sets which are available in the open source so you can easily get get access to that data so what we do is we start by importing the necessary libraries so we import pandas cbor numpy and matplot lip basically pandas and numpy are there for doing the data manipulation and also for storing data as metri or as arrays and and be able to perform some mathematical procedures on them and then cbor is basically used for plotting and mat plot lib for plotting as well and this line here get I python just helps us to run the images that we'll be creating in line with jupyter notebook instead of opening up a new window so let's run this and what it will do is it will import all these packages for us which we are going to use and then we will first import the breast cancer data that is available in your psychic learn data sets so we import that and then let's just initialize that data into a variable here called cancer so cancer here represents all the load the breast cancer data and now we will let's actually look at what this data is it's a bunch of attributes in this uh dictionary here so you have data the target which is basically nothing but whether it is a cancer or not so whether it is malignant or benign malignant means it's a bad cancer and benign means well it's just a tumor it's not cancerous targets name description feature names which is basically uh the features that will tell us whether a particular um case belongs to cancerous or or non-cancer or malignant or blind and then actually let's just print the description of this particular data here so as you can see uh we can use this uh command to print the description here and then we see that there are five 169 data points with about 30 attributes and these attributes are radius texture parameter Etc and uh the the Max and Min values of those are given here and then now let's look at some of the feature names so these are the feature names radius texture and so on and now let's actually set up a data frame of this particular data here using pandas this function here so there are 569 data points and all these are basically your features as we talked about and let's look at the Target variable which is nothing but whether it is telling us whether a particular data of Point belongs to malignant or benign so zero is cancerous and one is non-cancerous and then we convert the target into a data frame as well and then let's look at the couple of examples of how the data points look like this is you know one row of the data points which with all the several feature values that we have so basically we use this um package called standard scalar from pyit learn for pre-processing and for standardizing the variables and we initialize this standard scalar into a variable call scaler so standardizing is nothing but you know basically bringing all the samples to essentially the same range right because uh what might happen is some of the data point like for example temperature might be from 0 to 100 and some price might be from let's say 1,000 to 100,000 or whatever right so the absolute values can can lead to some issues with respect to the prediction therefore we have to standardize it or bring it between let's say minus one and one so and then a mean of zero right so we have to bring everything to the same scale to be able to compare the samples so we first of all we fit the this standardization U or normalization on the data set we have and uh that is we calculate the the variance and and the means and then we actually apply it on the data set to transform it to the actual values and then if we look at the scaled values now so let's look at the scaled values and this will give an example of the top five rows here so we can see now the values are between minus one and one or rather uh it is standardized essentially with a normal distribution and then we divide this data into test and train so basically we will train the model and then we will test it on a separate data set if you use the same random State we you should be able to get the same result otherwise you may get a different result here and essentially we are keeping the testing size to 30 which means that we are dividing the entire data set into two parts the train part which is having 70% of the data and the test part which is having 30% of the data and again we are using this package called train test split from the psychic learn package so we get the X and the Y's which are basically nothing but your train and uh the X's are your predictors and Y is your predicted variable whether it is cancerous or not and then now let's import the K nearest neighbors classifier this is the the actual algorithm which we are importing from Psychic learn package and now we initialize this particular algorithm and then we fit it on the data and some of the parameters as you can see uh is is basically what is the leaf size and so on so forth the nearest uh n neighbors we are taking so we are taking K is equal to 1 here basically as of now we will see the results based on that and then we will change it and see how the results vary and we now run it on the we now try to predict it and then we will now try to evaluate what the results look like so we have imported the classification report and confusion Matrix which is basically trying to see whether we were able to correctly classify the cancerous as cancerous and non-cancerous as non-cancerous or not so we can see that this is the actual and this is the predicted so basically some data points here five and four are classified wrongly otherwise all the others are classified well so if we look at the accuracy calculated accuracy actually so we see that the Precision which is true alarm right which is basically from the cancerous samples how many were you able to actually predict as cancerous if you see the accuracy is quite High almost 94 95% and then the recall which is from all of the cancerous samples how many were you able to actually predict accurately is about again 94 95% and F1 score is nothing but a combination of both precision as well as recall and that's quite good as well so with K is equal to 1 we able to get some already some good results now let's try to see how to choose the K value right so this is basically nothing but a a bunch of code that actually runs the K value from 1 to 40 and then tries to check the accuracy and this is just like doing a trial and error to see where we get the best result so that we can then use the best K value so if you can see this particular plot here after we plot the result from the running the trial error from 1 to 40 we see that the error actually starts decreasing and somewhere around this K is equal to 21 we get the minimum value of error so for us the best K value is 21 so now if we compare the results between K is equal to 1 and K isal to 21 we we should be able to see the prediction result so as you can see this was the result with k equal to 1 which is we get about 94 95% accuracy and then with K is equal to 21 we will see whether the accuracy improves right so we see that yes the accuracy has now gone up to almost 99% which is we earlier had nine misclassified data points uh out of uh all of the points and then here we have just two data points which are misclassified from the test d