Building and Evaluating Models: Linear Regression, Decision Trees, and Validation

Key Takeaways

And here instead of machine learning it it should be written as supervised learning. In supervised learning we have classification algorithms and we have regression algorithms. Just give me a second. Yes. No matter. So in supervised learning we have classification based algorithms and regression based algorithm. Now the question is what is classification exactly and how it is different from regression. So classification means whenever I want to classify anything for example if you want to know the stock price will go up or down. So there are two possible cases stock price can either go up or can go down. So in general it should be three cases. Stock price can either go up or down or will remain the same. Right? So I know there are three fixed outputs that you can get. You cannot get anything out of it. Right? So whenever you know the the outputs whenever you know like these are the fixed set of output you cannot get any output apart from it. This is called as a classification based approach. Okay. What is regression based approach? A regression based approach is wherever the output is not a single value. It is a continuous value. Okay. Now what is continuous value? In continuous value imagine it as if you want to predict the stock price of tomorrow. Now the stock price of tomorrow can be 100 100 it can be 101 it can be 101.5 it can be 101.23 two three or it can be any continuous value right so it's it's like temperature yes so it has a wide range of values right so in classification where I know there is a fixed value okay in regression where the value can vary it's a continuous value in a specific range okay so in classification we can have discrete value that's correct so and in regression we have uh we have things like that right So in classification we have couple of algorithms and regression we have couple of algorithms as well and how I made this mind map I I was using charge let's be frank with it right and here you can see this is the exact mind map so if I'm copying and pasting it down that will make more sense uh just to show you that yes you you can use these tools to to learn things and to teach things effectively as So here you can see this is what I have right. So in classification we have couple of algorithms in regression we have couple of algorithms as well. Okay. Now if I'm talking about classification, so there are couple of algorithms which are very specific to classification, right? And there are a couple of algorithms which are very strict to just regression. But there are a couple of algorithms that can do both classification and regression. And here you can see wherever we have written both these are the algorithms that can do both classification as well as regression. We also have written neural networks. Here I need to like fine-tune uh this mind map so that it will not take the deep learning ones. It will only take the machine learning algorithms. But this is what we have. Okay. Now our task is to start with one of the machine learning algorithm. Okay. And the machine learning algorithm we are going to take is linear regression. Okay. This is the algorithm we are going to start with. Okay. And then I'll modify the same algorithm and give you the task to do it to modify the same algorithm and make logistic regression on top of it. And you might be thinking sir here I have written machine learning algorithm logistic regression which is regression based but how can we modify the same algorithm just a little bit so that we can get a classification algorithm that is logistic regression. So yes this is maths. So you can modify little bit of little bit and uh you can get a totally different value. Okay. So this is the mind map that we have. These are the machine learning models that we have. Now do we have any other machine learning algorithm apart from them as well? Yes, we have a lot of them. Right. These are the we call call them as sort which is state-of-the-art algorithms. Okay. Very famous one. Okay. Some of the algorithms are way older like you can ask LGBT like when was this algorithm first used? You might be shocked like these are 20 30 year old algorithms as well. we still be using, right? And that's that's correct because these algorithms are still relevant, are still working. Okay, but we also have things like deep learning where we can have algorithms that are very straightforward like in very latest algorithm that are just released. For example, transformer is released in 2017. Okay. So, this is what we have machine learning models. Okay. And this is another one image that I have attached like which one can be used for risk classification, regression and both. But I think this particular thing will serve the purpose. Okay. Now the thing is what are we going to do today? Okay. In today's class our main agenda is to discuss about linear regression algorithm. Okay. We are going to write our own algorithm from scratch. Okay. so that you can have a clear understanding what exactly happens in an algorithm. Whenever I say machine learning model, what exactly do we have in a model? What exactly do we have in an algorithm? That is something we'll try to we'll try to see. We'll also see the maths behind it. Okay? And at the end I'll show you how you can do exactly the same thing by just writing minimal code. Okay? Because we are because we are going to do the same thing in two ways. First way is when I'm I'm going to write the algorithm write everything from scratch in in the another way I'm going to use inbuilt libraries that can do the same thing for you exactly same thing exactly same output for with just two or three lines of code and here I'm going to write like a couple of hundred lines of code okay so that is the agenda of today's class we are going to write an algorithm by ourself okay but before writing the algorithm let's do one thing okay I'm going to create a a table for you. Okay. So, let me ask chat GPT uh and uh let me create a new one. So, return uh a table. Okay. Return the data of users data of 10 users. Just want to give a sample uh with number of years of uh experience and their salary just these two things. Okay, return data in Tableau format. Now it would be great if the spellings are correct but this algorithms are so intelligent that they can somehow take these things as well. So here you can see we have user ID, okay, years of experience and the salary. This is what we have and for how many users I have the data of 10 users as mentioned. So I'm going to take this screenshot and attach it here. Okay. Imagine this is the data that you have here and you want to make a very simple machine learning model. A very simple machine learning model that can take any user. For example, if the user has five year experience, I can predict what would be the salary of that person. Okay, this is a very simple problem statement that we have. Now, you might be thinking sir, it would be great if rather than just having years of asking for years of experience, it would be great if can ask for designation as well, right? Or if you can ask for the location, right? If you can ask for the educational background because for same one year experience candidate we who is earning probably 36 LPA might be earning 36 LP as well with the same years of experience for the same designation right so these kind of thing can happen but I want to make the things very simplified that imagine that I want to make an algorithm where based on the years of experience I want to predict the salary of the person. This is the basic problem statement that we have. Does this make sense? I hope the problem statement is pretty clear and straightforward. Okay. How much math is required for now? Like uh I'll make it as simple as I can. Uh and but basically you need to have some good knowledge on statistics and on top of it there will be some derivatives and also the things that comes into play but I'll make I'll try to make it as simplific as simplified as I can. Okay. Now the thing is this is the input. Years of experience is the input and based on the input I want to predict the output and the output is nothing but the salary. Okay. If this is just the data definitely more data you have the better the algorithm will become and better answer you can get. But imagine this is what we have. Now can we do one thing right? I I'm going to write a very simple machine very simple algorithm very simple code not even a machine learning one but I'll try to make an algorithm that can based on the s of experience that can predict the salary of the person okay can I do one thing okay you can also write the code by yourself okay or you can also ask charg do so okay I'm going to make it very simplified and I'm asking charg to do so okay now I'm asking Okay. Uh, take the data of years in a list and salary in another list in Python format. Wait for my next command. Okay. So now what I'm expecting it to do, it will take two list. One has years of experience, second has salaries. Does this make sense? Right? This is a very simplified things. I've taken years of experience and salary. Okay? Now, what I'm going to do, I wanted to find a multiplication factor that can map both of them. Right? I want to find a factor that can act as this that can map this year's with this salary. this year with this salary. This year with this salary, this year with this salary that's a very basic maths, right? And how can we do it? If I want to make 1 as 3 lakh 60,000, 3 as 6 lakhs, 5 as 6 10,000 eventually you need to find you need to basically divide both of them. Divide 3 lakh 60,000 by 1, 6 lakhs by 3, 10 lakh 50,000 by 5 and find the average. Right? And now I'm going to do the same thing with Python. Okay, you can again coming to the same point. You can write the code by yourself as well. This is a very basic Python I'm talking about. Okay, but I'm asking Chibd to do so. Okay. Now divide every years every year with salary and give me the multiplication factor of each and return it in a list format. Okay. And I want to divide. So if I'm going to the same part. So to to find the multiplication factor you can divide salary by years or or basically what is the dependent feature here the dependent part is salary. Salary depends upon years of experience. So salary is something I'm going to calculate. So salary is something I'm going to calculate. If I'm writing a very simple equation y is equals to m * x where y is nothing but the output where with the salary salary will be calculated based on what based on years of experience. So now what do you want to find? I want to find this salary. But so far I have the salary. I want to find this m and to find m I can divide it by years. Right? Because if it's going from right to the left so it will be added in the denominator part. So I need to divide salary with this very basic thing. So let's ask it to divide every salary with year. Does this make sense? Yes, it does. Now your task is to do the exact same thing and execute it. Now what it will do it will write a basic for loop very basic for loop of dividing both of them and give the output in this particular format. So this is what we have. So why it is returning none for one of the case. Okay. So and here I can mention as well run the code to get the output because if I'm asking it to running run the code to get the output the answer will be most probably correct. Right? So you can see it is writing the code first of all writing the years of experience then writing the salary then it will write a for loop pro probably in list comprehension format or however it's showing it but is going for both of them and thus the final multiplication factor that it has. Now why we are getting none for one of the cases pro because of zero okay so it is zero pro this is why we are getting none but imagine if you don't have zero in the list so this is the factor that I have these are the factors that I have now can you find the average factor like now return the average once I have the average so this is the average factor that I have you can multiply You can take a a very basic format of it. But this is what we have. Now this is the number that can help you take from the input to the output. This is the number through which you can map this input with this output. Very basic maths. I'm taking all of them dividing it into with the years. If I'm not taking zero then it would be great because 0 is something that is making the whole thing uh a bit miserable for me because we are getting some null values. But this is the final value that I'm getting and how I'm getting it because I'm taking all of them and basically dividing them. That is it. Okay. And uh I hope you understood it. Anyone having any difficulty, any doubt, do let me know in the chats. Okay, this is this is a very basic maths that we are going with. Okay, and this is why I'm not even writing the code by yourself by myself because this is very basic Python. Clear. Once we have it, this is the number that I have. Now what I can do whenever imagine if there is a new person coming in I want and who want to predict this I and my task is to predict their salary. And to predict this what I want I need this m which is nothing but the parameter and I am going to multiply it with the years of experience. Now what I need to do imagine if there is a person coming in with five years of experience what should be the salary of that person? Now I have calculated one thing which is nothing but m which is a multiplication factor. Okay. So passing this m here which is 21 9841. Multiply both of them and you will get the final result. What should be the salary of the person? It should be 10 lakh 99,000 somewhere around 11 lakh should be the salary of the person. Okay. And that's pretty simple. If the person coming in with sorry 12 is years of experience the salary of the person should be somewhere around 26 lakhs. very basic maths through just mathematics I have calculated the salary of the person. Okay. And here you can see if you check the database do we have anyone with 12 years of experience? No. But do I have the result of the person who uh has 12 years of experience and what should be the salary of that person? Yes, I have it. Okay. And now because based on this particular data set I have calculated a factor and this particular factor is something that will map this input with output this feature with label. Does this make sense what what we are doing here? This is a very basic details that we have. Okay. So this is the factor that we have calculated. This m is something that we have calculated. Now tell me one thing once I have calculated this M once this M is calculated do I need to have this data set again I have the data set and based on the data set only I have calculated this M but once the M is calculated do I need to have the data set again do I need the data set once I have The answer should be pretty simple, right? Once the data once this M is calculated, I don't need data, right? Because I only need data to calculate this M. And once I have this M, I can save this M anywhere. I can save it in a text file. I can save it in I can save it where wherever that I want. Okay? And then whenever a new person is coming in my only task is to take this of years of experience multiply it with M and you will get the salary of the person. So this is a very basic algorithm that we have made and this is the datadriven algorithm again. This is something we have calculated based on data. Right now the fun part is this this is working this totally fine. But imagine rather than taking just years of experience and salary what if we have three things right rather than just taking years of experience and salary if I want one more thing which is let's add years of experience and their salary years of experience along with that let me add one more thing which is nothing but the uh what I would say the degree or let's suppose job group and their salary. Okay. Now what will happen now here rather than just having just two values where just years of experience and based on that I'm predicting the salary. What if I have two inputs? What will happen in that particular case? Any guesses? If I have two inputs now, the salary should be predicted based on years of experience as well as job rule. What need to be done? Okay, first of all, whenever applying machine learning algorithms uh specifically where we need to perform some statistical analysis, these things can be only done uh on maths on numbers. So whenever I have job roles which is categorical in nature, which is text in nature, I cannot have these things calculated, I cannot perform any operation on it as we can on years of experience. So best thing we can do at this particular time is I will convert these job rules into some sort of numeric representation. Using some sort of numeric representation is the best thing you can do. Okay? And that's something I'm mentioning now like uh convert this job role into uh label encoding. Okay. Now what is label encoding? Label encoding is pretty simple. This is a part comes under feature engineering by the way where every job role will be designated with uh a number right for example data analyst is nothing but zero. Data scientist will become one. Business analyst will become two. machine learning engineer will become one. If there is a one more data analyst, that person will become zero gap. Okay. Now, usually there is a better way of doing the same thing by writing rather than going for label encoding just go for uh one hot encoding. That's a better way of doing it. But now if I'm going for it now return just the label encoded column years of experience and salary. Okay. Now I will be having the same data as I have here but now this time I'll be having one extra column. So based on the job role and years of experience I want to predict the salary of the person. Now tell me what I need to do because this time things will be a bit difficult because now I don't have just two columns I have three columns and here salary depends upon years of experience as well as job role. What should I do now? How is this calculated? This is nothing but a label calculation. Right? A label calculation means by default it will take the first one second one and if there is a more data analyst label be zero you if there is a more data scientist that label should be again one okay this is a very basic details that I'm talking about okay nothing else I hope I hope that is clear to you okay now once this part is done now how will my model calculate Now if I go back this m that we have calculated is just one number summarizing the whole data. This is one number that is summarizing the whole data. Here I just have 10 rows. But imagine if I have thousand rows or if I have 1,000 rows, if I have 5,000 rows or 10,000 rows or even a million rows. Irres if I have just two columns one input and one output irrespective of how many rows of data I have I need just one particular parameter one particular number that can summarize this whole data and that's pretty that's pretty great right if I have 10 rows that's fine but if I have 100 rows thousand rows though the number though though though the calculation will be much higher but I will be having this particular number that is summarizing the whole data This number is nothing but what we call it as a parameter. This is a parameter. This 21 9841 is one parameter that is summarizing this whole data set. Okay. And whenever if I'm talking treating it as a machine learning model. So this is a machine learning model. But inside this model, what exactly do we have? We just have one number. That's the brain behind the model. We have one number that can summarize this whole data set. Okay, you can treat it like a knob that you might have used a radio, right? So in the radios, we have two knobs. One is for volume, another one is for frequency. Let's talk about just the frequency one. That's something going to take you the final value. You need to keep on tuning these knob until unless you have found a perfect value, right? And imagine rather than just having one knob if you have two knobs thing will be a lot more complicated because we need to go for all the combination of both the knobs then only you can get a perfect fit to get the value and the same same kind of thing also uh the companies of locker companies also using to go for the locks you might have seen couple of movies and not in movies you might have seen them in in your home as well. Okay. Now imagine just rather than just having two or three knobs, if you have 100 knobs, if you have thousand knobs, if you have a million knobs and if you have a billion knobs that you need to keep on tweening or keep on tweaking to get the best fit for your data and this is exactly how machine learns. If you have heard about charges, right? or GPT model has billions of parameters. Llama has released their own uh open-source model that also have billions of parameters. Okay. And now we have made a very simple model with just one parameter. So whenever talking about what are the six billion parameter means 6 billion parameters means the model has 6 billion numbers into it and usually they are not integer numbers they are floatingoint numbers and in floating point as well it is 32 bit long after decimal 32 digits after decimal this is how long these strings are okay this is how long these uh this floats are not not strings okay I hope you have a clear understanding now what exactly is a parameter. A parameter is something that we will calculate based on the data. And once it is calculated, I don't need data. I have the calculated field. I have the parameters that will help you do the exact same thing. Okay. Now here I need one parameter. But in this particular case, I need two parameters. The multiplication factor of this one and this one. Okay? And if you know a bit of maths and if you might have seen this particular equation this is y = m * x + c exactly don't predict salary right but these parameters exactly don't predict salary yes this multiplication factor has nothing to do with salary this m factor even this algorithm that we have written don't know that we are predicting salary for This algorithm the fund is pretty simple. Whatever number coming in multiply it with this particular number and you will get the answer. That is it. Irrespective if you are getting years of experience, if you are getting the salary of the person even if like you can give this the uh the age or to any number give any number to the age the main fun is that that value will be multiply with the parameter and give you the result. That's it. Okay. And this is very simplified calculation that we have done. Okay. And this is nothing but the equation of a line. Okay. So y is nothing but the output which is something you will get as a salary. X is nothing but the input. M is the factor that we have and C is the intercept. Okay. We don't have enough time to discuss about intercept and slope as well. Uh but I hope you know that like this this is basically showing the the equation of a line. This is the equation of a line that we have. Okay. And in this particular part, I'm not taking the C part the C part of it. If you are taking taking the C part of it as well, the output would be a lot higher. Okay. But in that particular case, you need to calculate two things. You need to find not only M but C as well. Okay, this is a simple part here. But to find it the same thing in this particular case the formula will be a bit different because here the formula would be m_sub_1 x1 because this is first input this is the second input and this is the output y wherever I'm saying x I mean to say input whenever I'm saying y I mean to say output so this is x1 just to show you so that you will remember so this thing will become x1 This is x1. This is x2 and this is your y. And once we have this the formula to find it means the final formula to calculate the salary is m1 * x1 plus m_sub_2 * x2 + c. So here how many multiplication factors do we need? You need to find this m1 and m2 exactly the same thing that has that I have mentioned here. You need one parameter that has to be calculated. In this particular case I have two parameter that has to be calculated. Okay. And to be more specific to get a better result you need to find three parameters. So now to summarize if I'm talking about the linear regression algorithm only now I'm coming to the point that this is how it works. Linear regression algorithm will is used for linear relation data. First of all, where there's a direct relation into uh the input and the output. But the fun will be pretty simple. Here you need to draw a line that can summarize the whole data. You need to draw a line that can summarize this whole data. Okay? And here in this particular case to find that summary of the line, you need to calculate M and C. And here in this particular case m1, m_sub_2 and c. If I have three inputs and one output then in that particular case x1, x2, x3 are going to be the input and y is going to be the output. So in that particular case, the formula would look something like this. Can we get this doc? Yes, you will be getting it. So in that particular case, you will be getting m3x3 plus c. So how many parameters you are going to have in this particular code? In this part you will be having four parameters that has to be calculated. First parameter is m1, m2, m3 and c. Okay. Now the final goal of my algorithm is to calculate these parameters. Okay. And in some advanced algorithms you also have like millions of billions of parameters as I mentioned as well. There will be a lot of other factors as well just rather than going for this part but this is how it goes. Okay. So whenever someone said uh that okay tell me what are the parameters we have in machine learning. So parameters are nothing but a trained model. What a trained model is storing and a trained model is nothing but storing numbers, floatingoint numbers. Okay, sometime they are just one array of numbers, sometime there will be multiple array of numbers and whenever some data coming in the operation will be performed on the incoming data based on the train model and then you will get the output and this is what it does. Okay. Now I hope you have a clear understanding what's happening now. And you can do the same thing here as well. But here the issue is if if you want to calculate the C part as well. So now this is going to be a quadratic equation. Okay. Because you need to calculate the C part as well. There are two variables that has to be found. Okay. Now to find these two variables or three variables or four variables or how many variables do we have? we have an algorithm that can help you do so and the algorithm is called as gradient descent. Okay. And now I'm coming to the main coding part. Okay. Now what do we have in gradient descent? Let me showcase the code to you that what exactly we are going to do. I'm using a data set which is of diabetes. This is of diabetic patients. Okay. And I'm using skarn library to load the data set. I'm also using mattplot lib and numpy to load the uh to load some of the add-ons things such that like whenever I want to do plot the graph or perform any numeric computations or array computations so it would be easier for me. I hope you guys are already aware about numpy mattplot and skarn. uh if not uh you guys can go to YouTube geeks for geeks and there will be plenty of sessions that you can get like a numpy series something I have made probably two years back it's available on geeks for geeks channel as well you guys can check that out okay now the fun is I've executed it and I've executed this part okay now what's happening in this particular part let me add a new code cell here okay and those who are not very much aware about where exactly I'm writing the code I'm writing wring the code on an environment which is called as Google collab. Okay, this is a pre-written notebook but you can also create your own notebook and write the code for by yourself. I'm going to share this particular file with you at the end so that you guys can also try it by yourself and check how much you know and what are the thing that you can you can find and things like that. Okay, we have to do with you now. No, that's totally fine. you you can do it after the end of the session because all the whole code is pretty much sorted if you are running it it will it will run okay so the fun is if I'm going for data of data now how you will get to know I need to go for data do data only because I've loaded data if I'm executing data here you can see it is starting with a key value pair so it's nothing but a dictionary right those who are already know Python they can see so if I'm writing data keys So these are all the keys that I have. So I can access data targets frame description. So let's if I'm finding DR in this particular way as well you can find it. So this is the description of the data set. So if I'm printing it because this is the raw format that we have this is what we have. So in this particular case you can see like this these are the details of the data set. These are the columns we have. These are the columns we have and first 10 columns are numeric predictive values. Number of instances we have the data 442 patient and the value that we need to predict with the diabetics or disease progression. Okay. Diabetes data set. This is of diabetes data set. So you are going to predict the final diabetes of the person. Diabetic value of the person. These are the inputs. How many inputs do we have? 1 2 3 4 5 6 7 8 9 10. So there are 10 inputs. Based on 10 inputs, you need to predict one output. If you go back here, this data set has two inputs, one output. So for two year input, one output, this is the this is the formula for three input, one output. This is the formula. Now imagine what would be the formula if you are predicting it for 10 inputs. So it would be m1 x1 + m2 x2 + m3 x3 up to m10 is to x10 and plus c there will be one single intercept that we have. Okay. So in that particular case how many parameters do we need to calculate? Do give me an answer in the chat. How many parameters you need to calculate in this particular case where I have 10 inputs and one output? In the case of 10 input, one output. So m1 x1 m2 x2 m3 x2 because we have 10 inputs. So there are 10 m that we need to calculate. You directly but there is one c as well right that you need to calculate. So total number of parameters are not 10 but 11. 10 RMS 10 m you need to find and one bias as well you need to find or intercept you also need to find so number of parameters are 11 in the case of this one the if the formula is pretty straightforward here you can see I need to find M and C to get a better answer you are getting it for M as well but whenever going for two or multiple of them adding C part into it will give you eventually better result even for single one you will get by adding C part into it it will give a better result Okay. And the issue you are facing with zero will be overcome if you are using the zero part into it as well. If you are adding the C part, this particular issue will also be resolved. Okay. So this is why we need 11 parameters here. So let's go back to the code and see how these parameters are being handled. Okay. So far I'm just loading the data and splitting and transforming it. But now let me tell you what exactly is happening here. So if I'm showing you X. So X is nothing but the whole data that we have. Okay. And let me rather than going for the X here, let me show you what do we have the data exactly here. So these are all the columns that we have. And if I'm going for data of data, this is the data that we have. You might be thinking what exactly do we have in this data. Okay. So let's do one thing. Let's check the shape of it. The shape of it is 442 is to 10. That means there are 442 rows and 10 columns. And that exactly what we are expecting, right? Because this data is of 442 patients. So 442 rows and 10 column is something that we already have. Okay? And if you want to show the data of the first patient, you can go for indexing. And these are the first, this is the data of first patient. This is the age, this is the sex, this is BMI, this is BP. might be thinking okay age should be in integer format right how can age be in the decimal format so that is something we are doing because we are feature engineered on this in tomorrow's lecture we are going to build a whole pipeline of project this is just to know how algorithm is being applied so that part is something I'm skipping for now that the data how the data is normalized exactly but this is exactly what we have why we are doing it Why age is in this particular format and sex how sex can be written in this particular format? Okay, because all these values are normalized in such a way that all these columns will have the same range of values because age can have 0 to 100, sex can have 0 to 1, BMI can be another number, BP can have another range. So normalizing will return all of them with exact same range between minus1 to 1. Okay. So this is the data of first patient and what would be the label of it and to print the label of it it would be data of target of zero. So this particular patient with these particular details has a blood uh has a diabetes of diabetic number of 151 and what if for the next patient 75 for the next patient. So this is nothing but if you can imagine it as a data frame. So I have 442 patients with 10 row as an input and one row as an output. 10 row as an input one as an output always range from minus1 to 1. Yes. Because machine learning algorithms okay machine learning algorithm can work even if the range is not between 0 minus 1 to 1 as well. If the range is between 0 to 1 as well then also it will work. If the even if the the thing is if the number is minus4 to4 then also it will work but usually it should be a normalized format. Whenever the data is normalized it comes in the standard deviation range and standard deviation range is minus4 to 4. Okay. So if I'm talking about like uh bul curve and okay so if I'm talking about standard deviation here you can see the data usually in this particular format the data is between 0 to three standard deviation and three standard deviation positive. So it should be in the range of this particular kind of a scenario. So between minus3 to 3 - 4 to 4 that's the maximum we can go with okay but usually it should be in the range minus1 to 1. Why exactly minus1 to 1? Because as I've told you in deep learning algorithms and or or in neural networks whenever uh each parameter is calculated in a decimal format and the decimal is very long. It's a 32 digits after decimal as well. And whenever this steep you want to go, the best thing you can do is go for floats, right? Going for float is the best thing you can do. So going for a float is the main reason why the value should be very close to zero. Okay, a bit more, a bit lesser than that, that's totally fine. And there there is a benefit of using machine learning algorithms as well because in machine learning algorithms for some of the algorithms even if are not normalizing the data into the exactly the same range but they should be uh not very close to zero that also fine because the algorithm will predict it. Okay. But for deep learning algorithms it will not work. Okay. So this is the data that we have. So all the data 442 patients is something that I have here. Okay. Now the question is I'm going to take this data and find M and C's. This is what we need to find right? How many M's you need to find? You need to find M comm X dot shape. Okay. But to print X dot shape first of all let's discuss about one more terminology that we have. And the terminology is pretty simple. Whenever you are training a model, you will not give all the data to the model to train itself. Okay? Because I want to evaluate the model as well. And to evaluate the model, the best thing is you can divide the data into two chunks. 80% for training our model and 20% of it on which you have not trained the model. We are going to use that 20% that remaining 20% for test the data. Yes, this is this correct subie. This is train and test split. Okay, this is exactly what we are we are planning to do. Okay, you can you can uh think of it as whenever you are going for a class test. Okay, how will you check if the person is intelligent or not? If I will ask couple of question which are not from the syllabus which is something that you have not trained on so far and if you can answer those question then only you can see. So best thing I can do is rather than giving you all the question to train yourself I'll I'll take couple of questions to myself that I' have not broadcasted to you and then out of box is something that I'm going to give you means out of box means the question on which you have not trained. So the same kind of thing I'm also doing here. So I'm taking this data set and doing a split of train and test. Okay. Now what is this train and test? Here you can see this is 02 mentioned here.2 02 is nothing but we are splitting the data of 20% for testing and 80% for training right so if I check the original data set so original data set has 442 rows so 442 rows are for training and after that 353 are for training now in total we have 442 but now I just have 353 for training now Where exactly do we have the remaining one? The remaining one is something I have it as test data. Okay. So a remaining one is something I have it test data. So if I check the shape of it with the correct spelling of shape of of course. So I have 353 rows on which I'm going to train my model and remaining 89 rows where I'm going to check if my model is predicting the output correctly or not. Okay. So these are the inputs that we have. So basically what I have done I have taken the same data set. Okay. Where where I have the same 442 rows and uh 10 columns. I have taken 353 rows for training and 89 for testing. And these are the inputs. So I must have the output as well. So the output for X is Y. Output of X test is Y test. Right? So output of So if I'm checking Y test, it will be labels. So just 89 values here. And if I'm checking Y dot shape, this is the output here as well. So this is the output of first row. This is the output of first row of test data. Now this makes sense. So we are splitting the data and once the data is splitted, my task is pretty simple. I need to take this X. What is this X? This is the X and check the shape of it. The shape of it is it has 353 rows and 10 columns on this particular data set. I'm going to train my model. So because it has 10 columns. So this 10 value is something I'm saving. So if I'm writing x= to m, n, this 353 will be saved here and 10 will be saved here. So n is storing the value which is nothing but 10. Now how many weights I need to have? I need to have 10 weights and one bias because m1 w1 m2 w2 m3 w3 up to m3 m10 w10. So I'll be having 10 weights and one bias. So based on this exact same thing if I'm writing np dot zeros of 10 so it will give an giant array of just 10 values is starting from random and one bias. So what I need to do exactly just to give you a brief I'll take x of0 this data multiply by this array. So this multiply by this this multiply by this this multiply by this just a basic dot product and add a bias to it. So this whole thing multiply by this particular array and if you are performing dotproduct of it my means by multiplying both of them and by adding just by and after that adding it and after that adding a bias the same thing will happen the same formula will happen y is equals to m1 x1 m2 x3 m3 x3 and so on. So once this part is calculated what we are going to do we'll check the existing weights and biases and by default you might be thinking why all the weights are something zero because by default I am initializing the weights as zero. Once the weights are initialized I will go through the data set and update these weights and make the final weights as as the as the as the final row. Right now the weights are zeros. But after doing all these calculation there are a couple of like this where the calculations are happening. These are going to be the final weights and this is going to be the final bias that eventually if there is a new data coming in I'll multiply that data with the weights add them with the bias and I will get the final prediction of the person. Right? So what are these weights? What what's happening in these weights? These weights will be calculated throughout the process and now we will find what exactly the process. Okay. Can you explain the weight part again? Yes. So let's summarize whatever we have discussed now in terms of weights. The data split part is pretty clear I think for you guys. Okay. Now what I'm going to do I'm going to find this particular scenario. In this particular scenario this is what we have. But here I need to find m1 * x1 plus m_sub_2 * x2 plus m_sub_3 * x3 and I need to do it for all of them right do it till m10 okay because I don't want to write for all of them let it write for So this is going to be the for final formula. This is how the final formula will look right and this is not only till m10. You need to also add a bias add a bias on top of it as well. So this is imagine that's the final formula that I have that I'm adding here. Okay. And okay, it would be great if it is here only because I need to go back and see. But this is going to be the formula and we need to add a final bias here. Now we need to calculate what these m1 m2 rather than by rather than defining them in a separate variable that this is m1 this is m2 this is m3 I've added all of them in one giant array. This is m1. This is m2. This is m3. Diff m4. This is x1. This is x2. This is x3. This is x4. You might be thinking this is x. This is another x. This is a variable x which is storing all the rows of the data. But if I'm talking about this first row for this particular this is the first value that has to be multiplied with this, this value with this, this value with this, this value with this and so on. Okay. And by default I'm starting initializing weights with zero. And throughout the process that I'm going to use it will this process will find the weights. And these are the final weights that we have. Okay. I hope this particular part is clear to you. Now we'll go ahead with how are we exactly going to find the weights. Okay. Shall we go ahead? And here comes a little bit math about it. Okay. Shall we go ahead? Do give me a yes. Yes, everyone. Okay. Okay. Great. So now the fun is pretty simple. Now what do we have in M? Now don't get confused. Okay. There are some naming convention that I have changed a little bit. Now W is something storing all the weights. Okay. M is number of rows. N is number of columns. So how many weights? 10. How many bias? One. So how many parameters in total? 11. Parameter will be number of weights plus number of biases. Okay. Bias will be one only. Weight will be multiple as per the number of variables that you want to count. Okay. So can I say the formula that I have mentioned here. Okay. add B here as well and replace M with W. Okay, so that I can add the for final formula. But if you look closely to the formula, okay, let me go to the formula only. Then then only it would be easier for you. I can use this way if not directly copy and pasting it. Okay. So if you look at this particular formula okay because this is the data that we have. So I have how many weights to calculate? I have 10 weights to calculate and one bias. Okay. And if you look closely if you I need to add 10 weights and one bias. Can I say this is the formula? Can I say this is the formula that is defining this exact situation or not situation? The correct part be should be equation. Okay, this equation is giving you the exact same thing. Right, we need to find it for all of them. And here here n is nothing but 1 to n. n is nothing but from 1 to 10. This is the exact same equation that I have. Right? So right now we have random weights, random W's. So if I have random W's, so based on random W's but actual X and and random B, I will get an output for sure. But the output will not be correct output. These are these are random values, right? So I will take first of all these random value. Okay? And there's the formula that will help you do exactly the same thing. I will take all the x's multiply them with w which is random weights zero and bias which is random as well. So the output would be the error would be huge because eventually I will get the output zero because all the x will become zero here by multiplying with zero and plus by 0. So all the output will be zero. So every for every patient the output you will get the diabetes value you will get is zero because eventually you have W and B0 but now what we are going to do this is the equation that I have written to define the cost okay just don't get confused with it this is nothing but a function that we have written okay this is the predicted value that we will calculate from random value of W and Okay, this is the actual value. Y because this is also something that I already have because this is a labeled data. I know the input. I know the output. So based on the output, this is the actual output that I have. This is the predicted output that I will get based on random M and B. Now I'm finding the square error in between. Subtract them and find how much error do they have. How closer these values are? If they are not very closer, I want to make them closer because what exactly will happen if a patient is having exact diabetic value 150 but if my model is predicting 0. So 0 - 150 the error is 150 right or minus 150 right. So I need to minimize this error and to minimize this error this is the function that I have written and using derivatives means this fun will have two variables that has to be calculated right or change in two variables that has to be calculated okay and this is where you need to have bit of knowledge in derivatives okay you need to find two values w and b so I need to take the same function and do the derivative ative of it with respect to w and with respect to b. Okay, if you look closely what will be the derivative? Now why I'm having one upon m so that what is m? M is the total number of rows. So whatever error you will get by writing one upon m you will get the average error per row. Why one upon two? because this two whenever doing the derivative two will become in front and two and two will cancel out that's it okay and now if I am doing the derivative of it with respect to w this is something that you will get okay and those who know a little bit of derivatives so they know about the chain rule as well okay and same goes here as well now you might be thinking s what why I'm having x1 here but not x1 here because here if you look at the equation this is y1 how is y1 calculated based on this. So the final equation would be 1 upon 2 m this whole thing x i w i + b i - yi means you will calculate this whole thing here. So in place of it you will be having xw + b minus yi to the square. Now if I'm doing the derivative of this with respect to w so x is a constant b is not so x will be there. If I'm doing it with with respect to B, this whole thing will be vanished. So this is why we have X part here. This is a very basic derivative that we have. Okay. Which one is predicted and actual? Wherever you have hat here, this is the predicted one. This is the actual one. Okay. So this is the function that will tell you how much W need to be changed. This function will tell you how much B need to be changed. Simple, right? How much w need to be changed? How much b need to be changed? And how to write the same thing in terms of functions? This is how to do it. This is how easy it is, right? 1 upon m dotproduct x with the error I have mentioned. And instead of error, I have writen y minus y. So you can also place it here. That's also totally fine. Right? And then the same formula that will give you DW. Now what is this DW? This whole thing. What is this DB? This whole thing. And that's it. It will tell you how much W need to be changed. How much B need to be changed. Does this make sense? And once you have done it eventually what you need to do you need to update the current W's and B's because this Yhat is something that is calculated based on existing W's. Existing W and B are zero. Right? Now I will change the existing W's and B's. This is what has to be changed in which direction we need to move. Right? How much W and B's has to be changed. And now I'm going to take the existing W's and B's and change the change them as per this particular formula. And to change them is this particular formula. Here you can see we have alpha here. Alpha is nothing but a learning rate. This is the mathemat uh this the coding way of writing the same thing. Pretty simple, pretty straightforward. But the thing is what is this alpha? This is learning rate. What is the use of learning rate? Yes, it is 0.01. I've mentioned here like someone uh somewhere must have mentioned. Okay, not right now but probably I'll mention after that. Okay, here I must have mentioned it. This is the learning rate that I mentioned. But what is the learning rate? At which rate the model will learn at each epoch? So learning rate, yes, we can see it this way. But basically in learning rate so imagine in the first step the error is 170 okay in the second step the error is three in the third step the error is 50 so if I'm writing dw is equals to w the change is directly w means whatever change that has to be happened if I'm replacing that change in the and updating an existing w say w will be very steadily changed very rapidly the W will change. I don't want to rapidly change W. I want to make a smaller step so that I'll have a smooth transition to get best weights. Okay. And to do that, the best thing you can do is take the exact same thing and by adding a very small learning rate factor, very small factor which is alpha and this alpha is nothing but learning rate. Okay? And I'm taking this very small learning rate multiply them with the existing factor that we have and update the weights so that the weights will be updated very steadily. Okay. And this is what we have. This is how they are being updated. And here I have mentioned learning rate as well. Now eventually the same step has to be happened multiple times. Okay. So that means if I go here, this is the second step. by adding it here. Okay. So, first of all, I'll find yi. Based on yi, I'm going to find these new weights and biases. Based on these new weights and biases, I'm going to check what exactly need to be changed. What are going to be new weights? Okay, this will tell you what has to be changed. This will tell you the new weights and biases. And by default these weights and biases that you existing that you have right now by default these weights and biases are initialized with zero. Right? First of all I'll initialize the current weights and biases which are zero. And once they are initialized I'll calculate the error based on this. I'll find the change and I'll do this. So this is the three-step process that I'll follow. And that's pretty straightforward. Right? This is step number one. This is step number two. And this is step number three. But not that simple. Okay. So this is step one. This is step three. And this is what we have. Now on top of it, you need to update it. Keep updating it. Okay. This step one. Based on that, you will find this. Based on that, you will find the new W. But you need to keep on updating it. And you need to keep on updating it until unless you are getting correct outputs. And to keep on updating it, you need to pass the exact same thing in a loop. So you need to pass this thing in a loop. And you need to do this iteration multiple times, couple of hundreds or couple of thousand times. And same thing is happening here. I passed this whole thing in a for loop. for how many times number of iterations means how many times you will go through the whole data set I'll find the predicted values by default weights are zero and bias is zero so what I'll do I have defined the learning rate I have defined the number of iteration for how many times you need to go through the whole data set and check the errors and update the weights so how many times weights will be updated the number of time this it this loop will work how many time loop will work 10,000 times. So 10,000 times weight will be updated but very steadily with very small steps at a time. And if you look closely first of all I'll find y predict then I'm printing cost. Why I'm printing cost? There is nothing to do of cost here. This is this is just to print them that the cost should be minimized. The actual value and the predicted value for both of them the cost should be minimized. then only we can see the model is learning. Okay, even if you are not running this then also your model will work. But this is just a way this is just uh to show you that if your model is learning or not. Okay. So this particular number that you can see will tell you the about the cost. This one is the second step as I mentioned here the error that we need to find. The third step is updating the weights. So here I am updating the parameters. For that purpose I have already have a function which is called as update parameters. So I'm getting new W andB. So I have initial W andB they are changed. Then I'll calculate again. Change them again. Change them again. Change them again. Every time I'm done with thousand iteration I'll print the history and print the current weights or basically save the weights in the waste weight format. That's it. Okay. So now if I'm executing it basically I need to run all of them otherwise it is going to throw in an error because in between cells I have not executed. So here you can see in iteration number zero I started with this error but then it start decreasing and here you can see it is decreasing at a rapid rate. So it will still decrease. So you can go for couple of more iterations. Okay. Shape. Okay. Let's run it again. I don't know why exactly it is going on error. So here you can see this is what we have. So model is being trained. And here you can see this is the minimum error that we have got. Now if I print these parameters. Now what are these parameters? I have initialized a dictionary. Inside this dictionary I'm saving key as weights. This W is nothing but the array that we have. the weights right I'm converting the weight to a list so if I open this dictionary I will be having these parameters and these are the parameters that I have these are the weights and this is the bice pretty simple right so now the fun is pretty simple you can save this parameters in a JSON format so if you go back here here you can see these are your parameters you can download this this is a model that you have very simple model. Now what I can do I'm here in this particular case what exactly I'm doing just want to show you I'm going for x test if I'm going for test data this is what we have right this is x test this is what we have exactly do we have it here I think I might have changed the code a little bit but if I am okay okay okay I need to run this whole thing again. So here you can check this is what we have this X test. This is unseen data. So I can take this X test here. I'm taking first 10 random rows. Take S test mean I'm taking first 10 rows of testing data. Doing a dotproduct of it with the existing weights and adding bias into it. y = to mx + c. But now doing the same thing for first 10 rows so that I'm printing the inputs the exact outputs as well as the output predicted by my model. And this is how my model is performing. Actual output of that patient should be 219 but mine is 115. But I've trained my model from scratch. This is not the great model that is giving you the perfect output. But this is working right. So this is exactly what a model is. What does append function do? Append function will basically append the value into the list. Okay. So if I go back what exactly it is doing here you can see there's a list that I have initialized named as cost history. So whatever data we can see here the same data you can see in the cost history function as well. So this is a list which is storing how the list is changing. So what you can do you can write for example this is just want to show you why exactly I'm doing it. Okay to plot this data in a line graph. And now my agenda is that the cost should decrease. And let's see if the cost is decreasing or not. And you can see cost is decreasing. This is exactly what I want. Right? So loop is using training data. Right? Yes, loop is using training data. But this one if I'm talking about this one using test data, this for loop using test data. So first of all, now if I want to show the whole pipeline and summarize whatever we have done in this lecture uh or specifically in training the model, I have the data set. Inside the data set, I have taken two things. Let me zoom a little bit in. I've divided the data set into two parts. Training and testing. On training, I'll have training data 352 rows I think. And in testing, I have 20% of the data, which is almost 80 80 90 rows. Now, once I have it, I have initialized random weights and biases as shown here. Okay. And before that there should be one more step that I should be adding which is initializing random weights and biases. And once that part is done once these random initialization of weights and biases are being done after that the repetitive task should be till here there are only these three steps keep on repeating not initializing the weights and biases because initially whenever I'm initializing the weights and biases weights are is zero bias is bias is equals to zero again rather than weight is equals to zero it is nothing but an array of zeros and usually they are 10 values and let's calculate 1 2 3 4 5 6 7 8 9 10. Yes. So first step is to initialize the weights and biases. Once initialize weights and biases, calculate the error and keep on repeating this process until and unless you found the best weights and biases and that's it. Increase the width of it. And this is what we usually do. Okay? And this is the process that will keep on repeating. These three steps will keep on repeating. Not this, not the rest of them. Okay. And we started from here. As you can see, we started from a very basic model that how we can make one to one relation with just one M. And then we have discussed how we can do the exact same thing. What will happen if I'm having two of them. Okay. Then we can see what will happen if you have the data that looks something like this. Okay. And here just to make the things a little bit easier for you to visualize uh I can also do one thing that whenever I'm taking X and whenever I'm taking Y I can ask to put to this play data of X and Y in data frame. Okay, now it will write the code for me. Now definitely I can write the exact same code by myself as well but okay my to do if if we can use to do so. So here exactly what I have done. This is the exact data that I have. These are the input. This is the output. These are the features. This is the label. And if you want to show just first 10 rows, this is the data that I have. So let's take a screenshot now and let's paste it here. So in this particular case specially to run this specific thing, this is the model that I'm going to use. This is the formula that I need to use. So here I need these many weights, these many biases. And to do exactly the same thing, how exactly I've done, I've used this specific approach of updating the weights and biases. So this is what linear regression is. This is what linear regression algorithm is. Okay. So hope you guys are clear with it. Okay. And so this is day one where we have discussed about introduction to machine learning. Let me add all these things in in this as well so that it would be easier for you to understand what exactly we have done day by day. So this is day one of the lecture with introduction to machine learning and now in day two of the lecture this is exactly what we have discussed. It's definitely a bit more discussion that we have done as compared to the last lecture because last lecture is more of the theoretical part. Here we have discussed uh the practical implementations of it, practical aspects of it as well. So this is day two of ML models. Okay. Great. Now let me share all the support files with you so that it would be easier for you to go ahead. So this is day one where we have discussed about the theoretical aspects of it and this is day two where we have discussed not only this particular part but the coding part of it as well. Okay. Now how can we access this particular file? Now to access this file uh there are many ways and uh I'll tell you a very simple way because this file is of no use. So let's not put it in trash. What you can do? You can search Ashi Jangra here. Go to the first link. Go to repositories. Okay, I I'll share the link at the end as well. And go for machine learning from scratch. There is a repository named as machine learning from scratch. go here and this is the exact code file which I have used. What you can do? You can download this file through here and upload it on collab and run it. Okay. And you can see this is exactly the same code that we have discussed and this code is exactly with markdown. So this is a full code for that. Okay, we have done the discussion till here. We have not done the discussion after this like how exactly we will get the predictions and what are the different ways to calculate the model, evaluate the model. That's something we are going to discuss in the next class. Okay. And here there is a bonus part as well how you can exact same thing that we have done for defining everything but using how you can do the same thing using inbuilt libraries. So this is stoastic gradient descent gradient descent only to perform linear regression and this is how you can do the exact same thing with just one snippet of code okay so this is what we have this the code file that you need to take so I'm copying this file and save it in excal draw so that it would be easier for you to access it okay so here I have added the excellent Eventually you will you you will be able to use it. Okay. And uh what else you will be needing? You will be needing this escal. Yes. So let me share this link with you. Copy the link. And here is the final link. Okay. So here's the final link shared with you which has all the nodes. So please check it. This the final link with all the nodes and inside this notes only uh we also have this. Okay. Can you please recap the cost? Yes, for sure. The cost is pretty simple. Basically cost is nothing but finding Help. Yeah. performance. So sorry guys there is some electricity happen here. So this is this something that we have discussed so far and I have shared the link with you as well but let me share it with it with you again. So that is it for today's class. Uh in the next class which is nothing but day three as well. Uh we are going to discuss about what else do we have how we can make a end toend machine learning project because here you can see we have made a project for sure we have made a model but our main agenda is to know what exactly a model is how how a model learns we have discussed specifically about linear regression there are lot of other algorithms as well for which we have not discussed yet because if you want to dig deeper into what a machine learning is what an algorithm is what a model is, what a parameter is. Stick to one algorithm and dig deeper into it. We can have detailed discussion, we can have discussion on five different algorithms, but we'll not catch exactly what one algorithm is doing. So it would be great if we can have discussion on one algorithm in detail and that's something we have discussed in this class as well. We started with linear regression. This is the process and this is the GitHub link that I have added in excal draw as well. In the next class we'll discuss about a project. We'll build a end toend project and we'll see how we can deploy it as well. That's the part of next class. Okay. So thank you so much everyone for joining and yes debug you can have it. Yes. Yes we are going to discuss about that as well. So come on Facebook. Thank you. Great. Okay. That's that's great. Okay, recap of cost. Cost is nothing but just to see if your model is performing or not because eventually we are printing this cost and that should be minimized. If you are not printing the cost, calculating the cost, that's also fine. It's basically a progress track if your model is learning or not. Attendance link. Sorry, I don't I don't have the attendance link. This this will be it will be automatically marked I think because you have joined from zoom, right? So it will be automatically marked. Okay. Why do we split the data? So that we can give an unseen data to my model and check it seem like why you are not getting question from the nodes that are shared with you by your schools because this is not exactly how they can check your intelligence in the final exam. In entrance exam there need to be couple of questions that are out of syllabus. then only you can check for better predictions as well to check if your model has not crammed the data and knows how to do it. Okay. So go for the recordings of this session. Go for the excelra and these are the lecture notes. Okay. And uh yes that's it. I'll see you in the next lecture. Then take care everyone and uh take care everyone. Bye.