Cross Validation using sklearn and python | Machine Learning

Key Takeaways

hello all today we will be discussing about Krantz validation in machine learning this is a very important topic when we are actually calculating accuracy for a machine learning use case this is supervised machine learning use it's let it be a classification or regression beam now let us just go ahead and have a steep discussion about cross-validation but before that let us consider that I have a dataset and in that data set I have some features like f1 f2 f3 f4 and suppose I have my target output which is basically indicated by my p-value okay now usually whenever we whenever we take this input data and suppose let us consider that this we have thousand records this we have actually thousand records now when I have to give this particular data with respect to this particular output to a machine learning algorithm first of all what we do is that we do something called a strain testlet now when we do the strain test grid that basically means that we will be taking somewhere like 70 30 percent a strain test you know and it will be randomly picking up all the records so that you know there is no specific manner in which we are actually giving this record so machine learning algorithm then what we do is that after doing the train to split of 70 or 30 or it may be 75 and 25 it depends on the number of data set that we have number of records that we now when when we do this train displayed then what we do is that we give the 70% of our data set to our model now model gets trained with respect to our input and output feature and then once the model is getting trained after that we test this particular model we test the accuracy of this model by giving the input of the remaining test 30% of the data and then we see the output and then we try to verify this predicted output build with the real test output right with the real test output now when we see this particular I am getting my model a Chris is given somewhere around 83% okay now when when when we get this particular 83% it is with respect to one type of train and one type of test data now it may so happen that the 70% of the Train data and the 30% of test data some of the reports may come over here also in the next hydration suppose I want to do a separate train to split by using a different random number variable okay there is a parameter inside train test which is called as random number and suppose if I give a random number of something different and that and what happens is that it again randomly picks up some other 70% of the data as my train set and then I then then the test data is the remaining 30% of the date I'll it randomly picks up picks up it shuffles the data set and randomly picks it up now at that time when I give this kind of data again to my model then the model may be giving my accuracy somewhere like 78% or it may also increase or decrease based on the type of data that is basically selected in my training set and the test set so this random number shuffling and the train displayed actually works for one type of data itself it is it is I it completely depends on what kind of data basically on the 70% of the site what kind of data or basically on the 30 questions of the day now imagine that we should try to know in order to prevent this like every time we are shuffling in in the ran in the Korean test plane and giving to our given the data to a model for the training purpose right all the time we'll be getting some different accuracy so we will not be able to you know come to accuracy parameter that what is basically the accuracy of the model by just seeing one type of train and estimate so because of that we basically use something called as cross-validation now the cross-validation works somewhere like this now let us consider that I have total number of Records as 500 so 500 is my data set now if I try to divide this okay now I'll be considering that let me consider for 50 days I said for 50 a record should be always in my training data set and then 50 records should be always in my agitated this is basically test they can see okay now what happens is that cross-validation works in such a way that suppose I will take this 500 data silicon and I know that I have to have 50 tests it tests 50 data set in my test to 50 records in my test so what I do is that I will take all this 500 dataset once is particular square box let me just take the first 50 data set as my as my test data okay now the remaining 450 what I can do is that okay remaining 450 now okay I not consider 450 because I've just made five groups instead what I'll do is that if I'm taking 500 as my total data set then what I'll do is that 400 I'll keep it in my train and remaining 100 I will be keeping in my test okay now I can divide this into five different experiments in the first experiment my first hundred records will be my test dataset and then the remaining data set that I have this will actually become my train data set okay I always remember see how how the cross-validation will happen in the first iteration my first record first 100 record will basically my test and the remaining all will be my training data set now what I'll do I'll take this data set I'll give it I'll train it in my model I'll take this training data set of 400 over here give it to my model and then test with this remaining test data and get one kind of accuracy so this will be my accuracy one similarly what I will do is that in my second experiment I'll just move towards one ahead step that is my next hundred data set which will be my test data set over here I will consider this as matrix test data set and remaining data set will be my training data set then I will give it to my model okay my model will get trained and it'll give me an another accuracy similarly what I will do is that I will complete the citation for all the experiments based on the size of the test data now why I am saying based on the size of the test data we have taken my test data sizes hundred so if the total number of records are basically 500 then what will happen is that the first iteration I will get 100 remaining 400 will be my training dataset similarly I'll complete all the I could iterations then I will be seeing that based on the number of iterations I will be having back many number of accuracy so here I have five iterations I'll let you have a curacy one accuracy to a curiosity for Krishna now since I am having different accuracy what my final cross-validation will do is that it will try to find out the me nawfal this accuracy mean of all the secrecy now when I find out the mean of all the security this will then be giving me accuracy which is actually which is actually representable for the complete data set wherein you're shuffling the data and you're trying to take it as a train and test it so it is actually using different train and test data and actually calculating your accuracy and this is how a cross validation work simple guys serious in the first experiment what I am doing I'm just considering that my top hundred records will be my test data set remaining all remaining all this will be my training data set right now I'll give it to my model train it and this will happen for all the experiments now how many number of experiments will be there based on how many number of Records I have right and then how many data set I'm taking it as my test data set Ohan five hundred divided by 100 is nothing but Phi so this will actually be five experiments you know and based on this what will happen every one every experiment will have different different accuracy then finally all this accuracy mean will be we will be calculating mean of all this accuracy will we will be calculating and finally we get the type of accuracy that is representable for the whole dataset and this is pretty much pretty much good I am pretty much valid when compared to the tree line test ok trained and disciplined and sometimes cross-validation also helps you to choose like what algorithms you should use how I'm saying that suppose you applying logic regression aware for all these particular experiments okay and then in the second time you apply decision tree now suppose your logistic regression average is coming somewhere around sixty eight percent accuracy the decision tree may give use and accuracy and if three percent right so how do you change this algorithm might sell it by seeing the cross-validation score that is your mean accuracy of seventy three percent this is much more better than the sixty-eight percent so you may choose decision tree for this particular problem statement and similarly you can apply it for other other types of algorithms and this was all about cross-validation guys I just also help you to see a coding functionality just give me a sec can so here I have actually made a Jupiter notebook file of model selection and cross-validation here I've considered a data set which is called as purchased data set C is B and here you can see that after applying I'm trying to see with Kenny or near K nearest neighbors classifiers and logistic regression you can see that after applying cross-validation I get a mean score of 78 percent and a means call of sixty four percent can go through this particular file I'll provide this in these already uploaded in the detailing I'll provide the link in my description of this particular video I hope you particular I hope you like this video guys please make sure that is subscribe to general thanks for supporting everyone please share with all your friends or I do come up with very interesting content and I'm pretty much sure that I'll come again with much more contents in my next videos thank you one and all have a great day thank you