Understanding Logistic Regression and Decision Tree Analysis

Key Takeaways

now I am sanchita and um and I am a data science uh an analytics professional I'm working at a private bank and today I'll be touching upon the topics of logistic regression and decision tree analysis uh these topics are quite close to me because I am an economist by degree and by training and an analytics professional by experience so just adding on to that further I would like to uh touch upon these topics and take a session on these so should I start okay so starting with the logistic regression logistic regression is a model which has been used in traditional statistics as well as in machine learning so it's a very hot topic these days whenever we talk about classification problems and uh in the case of classification algorithms logistic regression and decision tree the other two very famous models which I use so I thought that I'll cover these models today so coming to logistic regression it's actually derived from the family of generalized linear models so um a linear regression model for example uh is a very simplistic regression model in statistics second coming on to the logistic regression it's derived from a linear model but it's basically a classification algorithm where the response variable is binary or it can be multi-class but how is it different from a linear regression model is that here the predicted the the predicted variable the variable which we are predicting the target variable is in a class format or it is a categorical variable and not a continuous variable that is a major difference between a linear regression model and a logistic regression model so linear regression basically will be used to um to predict a continuous variable for example sales or Price there are real estate property prices what topic these days that you know like what is the price of a property predicting that from where you can earn greater profits and stuff like that so for example size and weight of a person predicting um the price so all these variables these are continuous variables so linear regression is used to predict these variables whereas logistic regression is used to predict a categorical variable which can be depicted in the form of like true or false yes or no one or zero so for example if the weight of a person is suppose 70 kgs or 80 kgs will be tag him or her under the obese category or not so here are the categorical variables are it's a class variable so classification algorithm derived from the word class so here the target variable is into a class format and the logistic function basically is derived from inserting a sigmoid function unlike a straight line in case of a linear regression so now we understood that what is a linear regression problem but still there would be questions asked that uh but still why can't we use a linear regression for classification that is that the only reason about the predicted variable so first definitely that's a main reason that the predicted variable needs to be continuous in case of a linear regression unlike a logistic regression but what can be the other reason for it so the other reason is basically as you can see a straight line over here like this is an arrow it's a straight line so basically these the green dotted points are the points which we use in the logistic regression these are random points sorry so these are just random points and in case of a linear regression suppose if we need to predict a class variable by using a linear regression then we try to fit a line of best fit on this model so while trying to uh fit in the best fit line we get this red straight line as you can see in the picture this red straight line is the line of best fit as per a linear regression one so you can argue that okay fine we are using this line and the scan predictor model okay fine but consider a case where there is a value which is given on the extreme right of the axis for example at the end of the axis for example if I take X as the weight of an individual and the dependent variable would be I'm categorizing if the person is obese or not then X for example is uh 150 kgs so a person weighing 150 kgs if I try to use that um here in this line so I will get a point in the outlier segment which will be like way ahead Y is equal to 1 right so one point why we are not considering linear regression for these kind of problems is the impact of outliers so when a outlier is uh signified in a data set or categorized in a data set or found basically in the sample set then linear regression cannot deal with it when the predicted variable is a class variable or is a categorical variable in that case it cannot handle a linear regression cannot handle such cases that is why we are using logistic regression here second point is that um this line goes till the negative axis also and this line goes on to positive Infinity as well so here we are saying that we can get values uh varying from negative Infinity to positive Infinity it means that we can also get for a person who is weighing 10 kgs we can get a y value of or maybe minus 10 which is not possible right we cannot classify a person who's being 10 kgs as the Obesity ratio to be minus 10 we either need it either 0 or 1. so again a linear regression fails in such a case where we are not able to actually categorize it into 0 and 1. so therefore here comes the logic of a logistic regression wherein we say that okay fine we are getting a set of values and we are imposing such a function in for those values which will help us to segregate the class of that particular variable like whether that person will fall into zero class or whether that person will fall into one category so here like I've also done a working um I'll just show you the working so I am sorry I am not a very good artist so I mean I would urge you to forgive me for my utterly bad and writing in bad drawing but I think that this is actually the best way to you know explain so for example I'm coming on to the logistic regression graph and I'm trying to explain this so here the logistic regression the logic behind is is that we operate on the threshold values and the probability values so suppose in the prediction the model Run Hotel predicted we get a set of probabilities and that probability is we have to find a threshold value okay certain classes so that is how we are dividing it into two classes two categories so here for example this is 0.5 correct and for example I find that this is 75 kgs or 70 degrees whatever today these days people are lot into yoga and I think so probably 70 kgs right so um I can say that if a person is 70 kgs then the probability or the threshold or the probability is 0.5 and now if I set it as a threshold value then I can say that a person who is above 70. these cases for these cases the threshold value or the probability sorry the probability value will be greater than 0.5 and it will be less than one so here it will lie in this area so all the people who are lying in this area they will be categorized as one it means that they are obese and all the people who are less than maybe 70 kgs they will be categorized they will lie into this area and the probability will be less than 0.5 so they will be categorized into class 0 which is not obese so basically the graph is derived this way and it's different from a linear regression because here linear regression there are various points is a continuous line so here there are various points and a line of best fit is inserted over here now when a line of best fit is inserted we say key this is the line of respect and we say okay how did this line appear we use a different logic of the residual error in this case and we minimize the residual error and for whichever line the residual error is minimized we select that line so again this is a very continuous uh derivation of the target variable continuous unlike this we are not dividing the target variable into categories over here unlike a logistic regression Where We Are so again a mathematical uh explanation of the graph simple graph explanation yeah uh and how are we divided into two categories so again in the PPT as you can see that I have given a segregation of linear regression and logistic regression so here these are all the points which I have told you so again it's a straight line it's a S curve sigmoid function logistic regression and here basically we take the possibility of a particular event taking place in case of logistic regression and here we are just talking about the uh the change in the dependent variable upon a unit change in the independent variable here again it's a straight line it's a S curve and here the OLS method is used ordinarily Square which I already mentioned that you minimize the residual error here the maximum likelihood a method is used for the estimation so maximum likelihood in case of Maximum likelihood we see that which is the curve so again you will find many curves like this like this so for whichever curve the the multiplication of all these probabilities will be maximum we will select that as the sigmoid function for example over here the residual error to be minimum is the category is the criteria here the maximum likelihood function it means that the multiplication of all the probabilities needs to be maximum for that best fit sigmoid function so that is something which we get over here so again we use a logic function and there is a concept known as the odds ratio it is basically the probability of success to the probability of failure where your p is the probability of success and one minus p is the probability of failure so here the logs uh the log of odds ratio is actually depicted as a straight line and upon further derivation we get p as the as this equation which is e of a plus BX upon 1 plus e of a of a plus b x so this is actually the y axis when you see a function like this so coming up to the performance of logistic uh regression I will also um before turning on to this I will also turn on to the the Practical implementation of the logistic regression model I will it's a short demo though I have taken a very basic data set but uh I will show you that so before moving on to the decision tree I will I will have a short or demo of that as well so here um like here so also here we are focusing on the performance of the logistic regression model so now we know that how the logistic regression is derived what is the function the logit function and basically what what is the use case over here why are we not using the linear regression and what is the max behind the logistic regression and the interpretation of this graph basically now coming on to the performance of the logistic regression so now we have run the logistic regression and now we get a certain output and what are the metrics what are the evaluation metrics which we use to evaluate a logistic regression model that whether like what is the fit of a logistic impression what is accuracy level and how did our logistic regression model perform so here for example the first is AIC so in linear regression I don't know how many of you are from a statistics background but uh in statistics there is a concept known as adjusted R square where R square is basically the like we minimize it it's basically the minimum of the residual error which we get analogous to that in logistic regression is AIC which is the information criteria and we prefer a model with a minimum AIC value second is a confusion Matrix here a confusion Matrix is given so here uh I would like to take some time and explain this so here a confusion Matrix where we get four kind of values a two positive a two negative a false negative and a false positive so suppose in the data in our data the actual value of the actual class is divided into two good and bad so when the actual class is good and the predicted class is also good then it's a case of a true positive short form is TP and when the actual is good but the predicted is bad then it's a false negative and why it's a false negative because it's actually a positive value it's actually a good case but we are categorizing it into a bad case so it's a false negative when the actual value or the actual predict or the actual class is bad but the predicted is good then it's a false positive it means that a certain um it means that a certain attribute or a certain set of attributes were were actually categorized as bad but they were predicted as good now that is a case of a false positive and when the actual is bad but the predicted is also bad then it's a case of true negative it means it's correct there is no error over here actual was bad predicted is also bad so for example a person who is maybe 110 kgs and that person is Beats right so in actual data it will fall into this category of that but the logistic regression model gave it as a good category it means that the the model categorized it as a non-obese person then that is a case of the false positive so false positive and false negative are also type 1 and type 2 errors in the linear regression model but maybe when I'll take a session on the linear regression then I can explain that so here I'm not going very deep into the type 1 and the type 2 error but the false positive and false negative they are actually a signification with that naturally signify that yeah so here what is a TPI which is a true positive rate so that is actually given by D upon D plus c which is true positive upon Pro positive plus false negative and fpr which is a false positive rate is given by false positive upon false positive plus true negative these two and the derivation of this is basically used in The ROC curve so Roc curve is it is the receiver operating characteristic it is one more performance metric of a logistic regression model which we actually see to to depict that how the logistic regression model is performing so here the ROC case is given it's a function like this and here it it is actually depicting the false positive rate the points of false positive rate and the true positive rate so you can get these points basically I X X Y uh coordinates and out of that you are building a curve oh Roc curve is a metric it's a very important metric and basically a model which is performing good will have a Roc curve which is tilted Above This 45 degree line it means that the greater The ROC or the greater the area under the curve which is also known as EUC the better the model is and what will be a perfect model a perfect model will be at this point where there is no false positive and there are only two positive cases so it's here it signifies that it's a perfect model so the more curved the ROC is the greater degree to which it is tilted Above This 45 degree nine the more better the model is we also have concepts of precision and respond and the F1 score so the F1 score is nothing but it's the harmonic mean of the Precision and the report so they are also important metrics which we use so now coming to one depiction of the logistic regression module so in the logistic regression model we are using a very uh famous data set and you will find this data you will easily find this data set on GitHub or kaggle or many other websites so it's a very small data set and it's just a sample data set with no with not much of feature engineering I have used this um basically to amplify on the logistic regression part and not focus much on the Eda part which is the exploratory data analysis I'm not covering that in this session and I'm also not covering two complex feature engineering that is because uh a lot of time is actually taken up in future in complex feature engineering and the complex ETL therefore I've taken a short sweep sweet and simple law uh data set and on that I'll be performing the logistic regression I can also send you the code of the logistic regression after the session or maybe analytics Libya can help me in that help me in distributing the code sure we will be sharing the professional yeah thanks so here I'll just share my screen so here I've already uh written the code I have run it and I've given the output also in the PPT for the decision tree and for the logistic regression I'm running it in front of you for the decision tree also I'll show the implementation button on you know demo form so here first step in Jupiter uh we are using python for running the logistic regression model here firstly we are importing numpy as MP then as important as SPD and then import importing C1 as SNS so basically C1 is a data visualization or Library pandas again uh it signifies a data frame so basically the usage of data frame in that we have to use pandas and number for all for performing all the mathematical functions we are using Nonpoint so here in the second step we are basically reading the text file over here it's a it's a CSV file so PD dot read CSV the data set and this is the data set it looks like this third step we are checking if there are any null values or not so again I have not delved too much into the Eda part that would have taken like half an hour or 40 minutes all together so I have taken uh again reiterating that I'm not doing much of Eda over here so here I have taken uh yeah so here I am checking that if there are any null values in this so sorry I'm not running it yeah so first we load the libraries then we read the data set then we check if there are any null values or not so here zero zero zero it means that we do not get any null values then we are seeing the length of our data frame it means that basically how many rows we have in the data set then we are seeing a shape what is the size of the rows what is the size of the volume it's 14 and 6. then DF dot hand basically uh depicts that we get the top five rows of the data set and all the columns Al again is the last five rows then this is the basic description of the data what is the pound what are the unique values what are the frequency so here basically we use SQL on library for uh uh importing pre-processing so it's it's basically a data processing tool which is the label encoder over here and why are we using a label encoder because for example temperature is mild and hot humidity is normal and high wind is weak and strong so we are just giving it labels we are encoding it into labels here using SK loan so basically these were the string functions these were string normal high these are words weak strong these are string functions now we are converting it into integers we are applying string to integer and then we are transforming these people so again done this and the new data frame is something like this after that we divide um data into the attribute set and label so what is the label over here label is basically the the target the the variable which we are predicting and feature calls are basically the feature columns so in statistics or in the data set language basically the input variables are also known as the at they also call the attributes or features so we run the model using all the attributes and the target variable is basically um the dependent variable over here so in our case in our so in our data set the target variable is playing because yeah and the input variables are outlook temperature humidity and wind so here it's not only about playing tennis but for playing any sport we basically need to see that how is the weather forecast what is the temperature what is the humidity what is the wind like for example you cannot play badminton if if it's too windy right and if the temperature is soaring if it's too too too hot and you know if you're facing scorching heat then obviously you won't go out to play so these are a set of uh variables again humidity is also one factor so Outlook temperature humidity wind these are the factors which we uh which we use to decide whether we are going to play or not so these are the feature columns so again this is X x depicts the feature columns which are outlook temperature humidity and wind and Y depicts the target variable which is that depending on these a combination of these whether I will go for playing outside or not then in case of logistic regression what we do is that next step is that we are dividing it into training and assets splitting the data basically so again or a brief on that why are we splitting the data is that um because we want to train the model now for training a model we cannot train the whole model right so in that case what will you predict then so we are training 75 of our model and 25 basically is the test category which the data on which the model is not performed and we will just test it to uh basically signify that how well our model has performed or how correct our models so here are the train and test split apples and here we are performing uh the importing of the logistic regression again from Escalon Library so here we are naming the variable as classifier is equal to Logistics regression and here we are fitting the trained model and these are basically the parameters so I have not tuned any parameters again tuning of parameters would have involved greater time and we already shot on time and way ahead my my portion as of now so uh we are using this model to train and we are not tuning any uh parameters and we are just creating these parameters as default parameters so we are fitting the model and after that we are predicting so the model is done and now the next step is predicting what's the X test we are predicting the test data now and coming on to uh the metrics which I just covered in the PPT here we are seeing that how is the model performing so basically like what is the accuracy portion so here are the accuracy is like 75 and here again it's uh so four there are four values in the test so again my data set was very small so only 25 percent of the values could be used in the test and here for example um just a brief the actual was one the actual class was one and the predicted was zero so here we are getting this false actual was one predicted was one this is correct actual was one predicted was one and actually was Zero predicted was zero so out of these four values one is where we are getting a false result the other three are positive or we are getting a two result so three out of 4 is 75 therefore the accuracy rate of 75 we are getting after that we come on to the confusion Matrix here I talked about true positive two negative I explained it to you in the PPT so basically I talked about the Precision the recall the F1 so here this is basically the conclusion Matrix so uh logistic regression we are done with the the implementation part now coming to the decision tree okay so again what is your decision so the set now we are digesting to the second classification algorithm which is a decision tree so decision tree and logistic regression both are categorized into supervised learning technique and they both are used in classification decision tree is also used in the regression problems as well as the classification problem so why it is called a decision tree simple right because you are taking a decision on the basis of a tree that is why your decision so although it's not that simple because it there are a lot of uh technical jargon terms involved in case of a decision tree but the idea basically is that we are splitting the data sets on the basis of various features and we start with the root node and the decision Tree starts by asking a question so this this was something interesting when I was actually learning it and I was learning it myself reading through all the Articles seeing the videos and everything so what is a decision trees and basically it starts with a question right and after so for example just treat it as a person asking that question for example if I am asking a question and if I'm satisfied with that answer then I will not further uh you know delve deep into it similarly a decision tree when it it asks a question and when it is satisfied it will it will not split further but if it says that okay fine I won't delve deeper into the question and it will split further and further and further so again the terms are basically that we start with a root node which is also known as a decision more and decision model basically the first feature on which the splitting starts and then we come on to the subtrees and the leaf nodes so basically uh after the decision node a splitting happens and uh the end of a sub tree is a leaf node why a leaf node act because at the leaf node no further splitting happens that is why so it's called a leaf node and basically this is the yeah so yeah so this is a part of the subtree next is parent or child node parent or child node is also known as a root node which is also known as the decision mode parent node is from where the tree starts so uh basically a decision tree is a supervised learning technique all the points which I've said earlier without a screen visible the other points over here again I was talking about a decision node and a leaf node so decision node is also known as a root node and here these are the leaf nodes Leaf node is a part of a subtree after the leaf node no further splitting happens so you can see that or this is the end this is also the end this is the end and why this is the end because we got a satisfactory output over here it means satisfactory output means that we were able to segregate all the uh all the uh at all the samples into one class I will depict that later on also in in my output but basically no further splitting happens when we are clear with the class if we are not clear then we will split it further then coming module attribute selection measures now how will we suppose we have a set of all the variables in the late NS data set also we had Outlook temperature humidity wind so on what basis will we decide that which attribute we should select and which attribute will come where so for that there are various techniques used one is the information game basically this is used to decide on the root node and one famous or technique to decide that from where the data will actually start second is Genie index or entropy so again in my code also in my python code you will see that we can employ both these methods and I will also show you how we are employing both the methods so we can employ either the genie index or the entropy as a attribute in the as a parameter in the of the decision tree so here again the information gain as I had already mentioned that it's a node or the attribute which is having the highest information again it is split first and there is a certain formula which we use to find out the information gain which is given over here if I would have had more time I would have actually shown you the calculation of um the training the training data set over here which is the plate LS data set and I had actually shown you the the rough work but it's fine basically we are using this formula over here to find out the information again and entropy is basically a metric to measure the impurities similar to Jenny index that is also a measure of impurity so it is said that we should take that feature which has a minimum Genie index that feature should actually be selected in a tree so entropy formula is minus P of s s is again a class yes or no so basically these are the positives then log of probability of yes then probability of No and then log of probability of no and then coming on to the GD index the index is given by a formula of one minus P Square is p is again the probability so here basically the the it is a decision tree working so here in our play tennis data set we actually finding as to which node will we select first so we have four variables with us right like output temperature and humidity and wind so out of these four which one on which factor will the tree finally split or what will be the root node so here basically we are using the the genie impurity Genie index also it is said but in the according to the SK loan Library it is a genie impurity so mean you can call it either ways so Janine priority or Genie index is calculated Again by this formula so again although we do not have much time but I will just touch upon this to give you a brief introduction and who can actually do a mathematical calculation of this and the reason behind choosing a small data set was also that we could easily do a mathematical calculation so here G1 is basically 0.48 which is 1 minus 2 by 5 Square minus three by five square so as I've already mentioned that Genie index is basically 1 minus P Square so here p is the probability so here if we take Outlook yeah so for example G2 is 1 minus uh 0.4 whole Square minus four by four whole Square so here if we are taking overcast suppose as a Outlook category so here I am just signifying overcast overcast and overcast over here so here this is yes this is also yes this is also yes and this is also yes so there are four yes and there are zero nodes so G2 is given by 1 minus 0 by 4 Square minus 4 by 4 Square which is 0. again G3 is when we are considering other categories of sunny and urine here again G1 is other category sunny or rain and now we are calculating the weighted average of G1 G2 and G3 so the weighted average here is point zero point three four two of Outlook basically and how did we if you are wondering how did we get 5 by 14 and 4 by 14 and then 5 by 14 so basically uh Outlook is divided into three categories and overcast there are four cases and the total number of cases are 14 and out of overcast there are four cases so that is how we are getting 4 out of 14 over here and for sunny and Rain there are five five cases each therefore we are getting 5 by 14 and 5 by 14 over here and then we these are the weights and then we are multiplying it by the G1 G2 and G3 values to obtain the 0.342 now as you would have clearly remember in the previous slide I mentioned that wherever the genie impurity or the genie indexes is the lowest we will select that feature so here after all the calculations we find that Outlook The genome Purity for the Outlook is the lowest so this basically becomes a root node and the first step the first step becomes like this where out where the Outlook is divided into Sunny overcast and rainy now after this again we are doing this calculation to find out that which feature will be the next feature so after this we talk about the advantages of the decision tree and the disadvantages of the decision tree so basically again as I mentioned earlier it is the process which a human follow while making any decisions in life right so you remember that I Elders or our teachers they've already said that you know in your mind make a flowchart that you know what you are aspiring to do how will you do it so basically things are achieved by making a flowchart decisions the decision making process so basically it's a form of that so here you we are just using python for that and you may flowchart every day in your brain so you are actually running a decision tree model every day in your brain by you know making a flowchart and then abiding byte so disadvantages May uh first the most important is actually overfitting issue overfitting issue in that case uh there are a lot of layers the depth is like too much so what happens is that we just keep splitting the decision tree into layers and layers and layers uh sometimes for example uh if at the end of the decision tree I get 21 and 2 so out of 23 um out of 23 if I know that 21 RS and only two are no then I can simply categorize it into uh yes right that the output of this is yes class so I will disregard the two cases because they are very minuscule but sometimes overfitting happens and we do not stop the decision three till we get 0 in one class either we get 0 and no or either we get 0 in yes so till that time we just keep splitting the the decision tree so that is actually a problem of overfitting and that's a that's uh a very major disadvantage for that we basically what we basically do is that we fix a maximum depth so again tuning of parameters includes one of that includes fixing a maximum depth of the decision Gene so again we have used a play tennis data and what I'll do is that I'll just quickly go through the the code so here again we start with loading of libraries uh import numpy as uh NP and import pandas SPD then we read the data set the data set appears like this and then again we are doing uh and we are running it then we get the play tennis the data set including the the output then here what we are doing is that we are dropping the the target variable over here again get the feature columns barring play which is the target variable so the feature columns become output temperature humidity and we're in the same case as the logistic regression here I am importing tree again from SK loan and I'm giving a variable of clf to the decision tree classifier and here as you can see I have mentioned the Criterion as entropy entropy was given in my PPT where I've given the formula also of entropy although the calculation is depicted of of Gene but entropy calculation is also not very tough especially for a for a small data set your rant random state are selected as 0 and the maximum depth I have selected as four so not more than four levels of segregation I need in my data in my decision tree so here clf is again fit of X of Y in the decision tree model where X are all my feature volumes and my is my target variable which is playing here again I'm plotting the tree and field is equal to True means that I get colored volumes so uh that is the significance of field is equal to true and plot tree I am using as a function oh my goodness yeah so here's my tree and I have depicted a very neat output in my PPT just let me know if my PPT is available is visible yes it is visible okay so yeah here's my decision tree output where x0 is Outlook if you remember in my feature columns that was actually the first variable which I had mentioned and X1 is temperature X2 is humidity X3 is wind so basically uh the most important feature as depicted in the calculation also was Outlook and my decision tree uh via python the output which I get uh on python is is also depicting the same so here again I am getting Outlook X of 0 as the road node it is basically the decision mode where the first splitting happens after that so here entropy is depicted as 0.94 and uh in my model just one minute it will take I can change the Criterion to Ginny over your and then run the model again so here again uh instead of entropy Genie is written and here again we are starting from the decision mode of Outlook yeah so here entropy is given um while the cash so if you will calculate the entropy and the genie index for this particular data set you will find that the genie index is uh line between 0 to 0.5 while entropy lies between 0 and 1 so I've also made a simple graph for you to understand so this is basically the genie impurity and this is basically the entropy there it lies between 0 to 1 and Genie is from 0 to 0.5 so in the decision tree uh entropy is mentioned as 0.94 the samples are 14 because I've not done any splitting uh I have taken the full uh data set and here in the first instance the entropy we get was Zero so here there is no further splitting why because the value which we get is 0 and 4. 0 and 4 it means that in one class we get all 0 and in one class we get all four it means that we know that what is the class of this particular section so here we are not splitting it further here we are splitting it because 5 fall in yes and five fall in no so that is how we are splitting it and again we are splitting it further into okay so X2 is humidity so after uh Outlook after when you are doing the calculations again of Genie you will find that after Outlook the lowest Ginny or the lowest entropy is for humidity parameters so that is how the second split is at humidity so at this point again it is split into Outlook and it is split into wind so when X2 is humidity when it is less than or equal to 0.5 now humidity less than or equal to 0.5 is normal because high is 1 in the label encoding we have mentioned high as 1 and normal as 0. so when it is less than 0.5 it means that the classes that the humidity is normal so when the humidity is normal after that again on just on the basis of that we cannot take a decision right so then we will come on to the Outlook and we will also come on to how windy it is so again the wind is weak here because uh wind parameter is less than or equal to 0.05 and Outlook less than or equal to 1.5 means that it is raining so again this further splitting keeps on happening and at the end with the maximum depth of four we get this output where as you can see the class is 1 comma 0 it means that in one category we get all zeros and in one category we get all the other values here also zero comma one here also one comma zero here also zero comma one so these are called The Leaf nodes because here no further splitting of the decision tree happens so this basically ends my brief introduction or basically a brief depiction of the logistic regression and the decision tree I have used these reference links you can find these in my PPT you can go over three over these articles as well in the end I would just like to show you one video if it works then it's good so here basically what we do is that um we basically uh fix we we fit the function we fit the Lord we fit the odds ratio into a larger function and if you are asking that how this sigmoid function is derived then it's basically derived from the mle which is the maximum likelihood estimator and the coefficients are basically A and B so on that basis we derive the sigmoid function okay so I could get clearly the question we can move to the next question so Microsoft X I'm sorry this is the maximum likelihood estimated okay so we'll move ahead with the next question Microsoft Excel makes a use of polynomial regression when fitting a trend line to data points on an X Y scatter plot Excel multiple regression may you help explain curvilinear regression and relations to Euler number E power loss uh logistic regression and then from linear regression to non-linear so it's a very complex question now yeah exactly even I was not able to understand that if you want me to paste this in chat otherwise you can see it in the Q a part so should I randomly pick questions uh no it's up to you or like you can go in the sequence how it is basically uh how do we pick the maximum depth for a decision tree so again uh it's basically in the uh it is a part of tuning of parameters and um I had taken a small data set that's why I have taken a maximum depth of three or four if it's a very large data set then maybe you can have a a larger number maybe eight or nine also will do depending on the size of your data set then maths behind the logistic regression and the decision tree so here basically uh so here basically the maths is that when we are applying yeah so here when we applying log of T upon 1 minus p so log P upon 1 minus takes various values so we get a access from minus one to one and on that axis we get a straight line straight linear fit and we get minus 1 to 1. and this is a plus BX I'm not writing it but it's a plus BX so the coefficients Being A and B on the basis of these for example if we take a value we derive we insert that value here in this function and when we are inserting this we get a certain P value over here so that P value is basically the dependent variable value here on the y-axis not this when we are inserting the values of minus 1 to 1 in this function you will notice you can apply a formula and also notice that we will get the values between 0 to 1 only although these values are minus one to one but we are when we are using these values here in this function of p is equal to this we get the values between 0 to 1 and hence the logistic regression formula and the math behind it well if analytics video want I can have a different session all together on just the maths or I mean it will help me learn also because we can absolutely have another session as well you can get in touch with our team and we can schedule one for sure and similarly for tuning of uh parameters as well the maximum depth and everything so again the Criterion the maximum depth the random State all these are parameters uh similarly for Eda also I have not man I did not get the time also to actually take a complex data set and do all the Eva so Ada can be a little extensive so let us understandable yes yeah definitely and this complex question [Music] logistic regression then from logistic regression to non-linear I can actually delve deep into it and then explain probably uh as of now I uh it won't be correct if I give you an answer so let me learn also and then probably answer this okay and also uh like guys if you get any questions later on you can surely get in touch with her on LinkedIn we have shared the LinkedIn link as well so you know it will help both of you uh just two three questions I'll take just I think two three minutes of your time Priyanka sure sure absolutely yeah so um why recall is one so recall is basically the ratio of true positive upon true positive plus false negative so when we are getting only the two positives and when the false negative is zero in that case The Recoil is one so in that case the uh accuracy of the model so we are getting all the two positives so in that case the recall is one who decides the cutoff for the performance indicators see um I have employed few decision tree models in my work or till now so basically um 0.8.85 is considered to be a good model Point uh above that is considered to be too good to be true but on an average 0.75.8.85.89 they are considered to be uh metrics which are which indicate a good performance of the logistic regression model which is a vectometric Genie index okay so which is a better metric Journey index or entropy uh so entropy since entropy involves log in the formula it is usually not recommended for huge data sets because the computation time is much more in case of an entropy in case of a log formula so only for a small data set generally entropy is recommended but there is no hard and fast rule Genie index is again recommended for large data sets the computation is less cumbersome in case of Jenny index how to arrive at optimal number of tree size again it depends on the data set usually when the data set is very large you take more number of Maximum depth depth no gradient descent algorithm is a different model all together okay I think this question which case DT is useful if logistic regression is efficient then DT when to use a logistic regression and all I hope that these this this question was answered by my video because my video showed this case this thing only so uh I think I can share the video also with the analytics with their team and they can further distribute it sure yeah absolutely I think this question was answered in that video because in that video I showed that uh when to use logistic regression and windows entering basically so yeah I think foreign