Full Machine Learning Project — Fourier Transformation & Clustering (Part 5b)

Key Takeaways

hey everyone and welcome to the second half of part 5 of this series where we got a fitness tracker with Biden let's continue where we left off so first of all I've added some extra information to the document that we have been using in the previous uh episode so make sure to check it out again and the most important thing the that is updated over here is a new python file that we will be using so that is the frequency abstraction dot by file so make sure to add this file just like the previous files also to your project directory so your workspace in Visual Studio code so make sure it is in the features folder and then here we have the uh the temporal sorry the frequency abstraction dot by so this is the new one that we will be using in this episode so let's continue where we left off so uh the last thing we added to the script was the sum of square attribute to add some additional features let's now continue with temporal abstraction so let's first quickly cover what we mean by temporal abstraction what we are basically going to do we are going to use a rolling window walking over our data and for a certain window size we are going to compute the average so the mean and the standard deviation so here is a visual image of a rolling average in bandas and you can find out more about this Source over here by following this link but basically this gives a good um understanding of what we are going to accomplish so here we have the raw signal in this case so this is from another data set and then here in the orange line over here you can see a rolling average basically meaning that depending on the window size we're going to look back for example let's say five steps and then we're going to compute the mean the average of those five previous steps and then we're going to input that as a new feature and we walk over the whole data frame of the whole data set and for each of the records in there we compute this value and we can do this with all kind of statistical properties like min max median mode mean Etc and this can help us to basically summarize the data for a given window size and by that adding extra information providing extra information for our model so we will be using the mean and the standard deviation that those are the values that we will be Computing and luckily bundus has a very nice feature a very nice method in order to do this very quickly so let's see how that works so we'll start off as usual by defining a new data frame and creating a copy of the last data frame that we've used so we could continue from the DF squared so make sure that all the previous codes that we've created in the previous episode is active again in the interactive environment over here so make sure to run everything and then you can check that we have DF squared and it should look like this so 15 columns and 9009 rows and then let's continue by creating a copy next we will be using the numerical abstraction class that we have imported so let's define a new variable over here called num apps and set it to numerical abstraction and basically by doing this we initiate a new class instance like we've done previously already but now let's have a look at the numerical abstraction class over here and see what is in here so we have two functions we have an aggregate value function and we have an abstract numerical function and basically this is a convenient function to extract the different statistical properties for a given window size like I've just explained so as we can see as input we take a data frame and data table then we select the columns for which we want to extract the features then we specify a window size so that determines so how far do we want to look back into our data frame and then also the aggregation function which is uh one of the values that you see over here so if mean Max Min median standard deviation so that is how this temporal abstraction.pi file works with the class numerical abstraction in it so by initiating the class like this we can now use this variable over here and call the different functions in order to process the data set and add all the features and now just like the previous examples we're going to Loop over all the columns for which we want to add the temporal features and we have the predictor columns which contains the first six Columns of the data frame but now we're also going to add the columns that we've just created in this section over here so the sum of squares so that will be two additional columns so that is Arc R and gyroscope are so if we add that to the predictor columns we can see that we get a new list and now all we have to do is store that into the variable itself in order to store it in place so now we can see we have added two extra columns to the predictor columns now the next step is to determine the window size and as I've just explained the window size basically captures how many values we want to look back for each of the values so if we look at the data frame over here if for example we're at this value over here if we use a window size of 4 for example it will look at itself then this value this value and this value and then it will compute the mean or the standard deviation based on these four values and that is the value that will be added on this row and then in a new column all the way on the right so that is how that works and now finding the optimal window size is as usual a process of trial and error and also something that we should reflect on and evaluate later after we've created our initial models then we can also play around with the window size and see how it affects the performance of the overall model we also have to keep in mind by using a window size of for example five this means that we can't use the first four values of the data frame and later we will be splitting the data based on the individual sets just like we did in previous examples so we have to remember that by setting a very large window size we also throw away a lot of data that we can use because it will will result in missing values and to keep things simple for now we are going to start off with a window size of one second so in order to compute that remember that we have a step size of 200 milliseconds so in order to determine the window size for one second we take 1000 milliseconds and we divide it by 200 so our step size which will result in five meaning that in order to get a window size of one second we need five steps so we need five instances so to make this neat let's just make it an INT and then we have a window size of five now the next step is to Loop over all the predictor columns and then compute the mean and the standard deviation so as usual we're going to create a formula Loop where we say four call in predictor columns so this will create the loop we will say DF temporal because the function will add to the data frame so we're going to say numerical abstraction and then we're going to call abstract numerical and then remember we can check out what function what what parameters the function takes over here so we have data table column window size aggregation function so we'll start off with the data table which is the df10 Pro itself then we need the columns so that is going to be call but it has to be in a list format then the window size and then first we're going to compute the mean and then second we're going to command C and command V to copy paste this line and then just replace this with standard deviation this will Loop over all the predictor columns The Columns over here and it will add to the day of temporal data frame the numerical abstractions with a window size of five and it will compute the mean and the standard deviation so let's run this code and look at the results so this takes about the process because it is looping over all the columns and doing the computations so let's see how long it takes all right it's finished and now let's have a look at our day F temporal so we can see that we have additional a lot of the additional columns in our data frame now because for all of the particular columns we create two additional columns mean and standard deviation so let's look over here and see what we have and here we can see that for every variable over here we have a temporal mean with a Windows as a 5 temporal standard deviation with a window size of five and that for all of the columns and we can also see that the first four values are missing because because the window size is 5 we use the value itself and the four previous for a total of five records and it can't compute these because it doesn't have values earlier than that now this is starting to look good but remember just like in previous examples where we looked over all the data that we are basically introducing noise or arrows into the data set because by looking back five steps and looping over the whole data frame whenever you switch from one set so for example a squat to the next set in a data set which could be for example a bench press the first value so the first row of that new set so for example the bench press will look four steps back into the data frame and then use those values that are from a squat to compute the mean values and standard deviation so we are introducing data that is originally from a squat into a bench press set and that is of course something we we don't want because it will probably result in a worse performance of our model because we are introducing values that are just not true that are not relevant for the set so in order to fix that we are going to split this Loop up into a subset where we basically make a subset based on the set and then compute the values for each of the individual sets so let's do that right now and we will start off by creating a list that we will be appending subsets to so we'll start off with an empty list and then we're going to Loop over the unique sets in the data frame so for us in DF temporal and then we'll take the set column and then call Unique then let's select a subset the unique sets and set this to S so we'll Loop over the unique sets then create a subset based on the selection and in order to avoid some warnings we create a copy and now with the subset created we're basically going to copy and paste what we've done previously but now make sure to swap this around with the subset and then once that's done we have to add it to the JF temporal list and we say dot append and then we just append the subset and now once the loop is completed we call PD Dot concat and then we put in rdf temporal list to bring back everything together so the DF temporal list will contain a list of separate subset data frames and then by calling the pd.comcat method we bring everything back together into one data frame again let's make sure that the list is stored and now let's run this piece of code to Loop over the whole data frame all right and we're finished and now let's have a look at the concatenation see if that makes sense okay looking good so let's store this into the same sorry not in the list but in the original data frame so we're going to override it and then let's check it out okay awesome so what we've just done is we've looked over the unique sets and for each set we've added the mean and the standard deviation and by looking at a data frame right now we don't see much of a difference compared to the previous results it's still same amount of rows same amount of columns and the first contain missing values in this data frame over here there are actually a lot of there are a lot more missing values basically for each set the first four rows will be missing and we can check that out by calling info on the data frame and then have a look at all the values that we've just created over here and we can see that there are quite a few missing values in total but by doing this we have basically countered the data a spill so to say of the different sets into the next set so we lose some of the data but it's still the best option to go with in this case all right and now finally let's have a look at the result so we can just take the subset variable and that will contain the last set of the loop so this is just at random but this is set 71 and let's just take an example of the Y acceleration and then we're going to add copy and paste this and make sure it's the Y and then so this is the mean and then take the standard deviation as well so let's just check it out yes okay then create a plot all right so here we can have a look at what we've accomplished so we have the Y acceleration in blue and then we can see the mean is in red orange over here and we can see it's basically a very it smoothened out version of the original value and that is because it's taking the mean over a period of time so it Smooths everything out and then on top here we can see the standard deviation as well let's copy this line of code and do the same for the gyroscope so select this and hit command D two times and then type in gyroscope do that as well and here we can see the same graph but now for the gyroscope data so also a smooth and smoothened version of the original value and the standard deviation all right and with the temporal abstraction out of the way that brings us to the frequency features and these require some of an introduction we will be using a technique called Discrete Fourier transformation in this case and basically to give you an idea of what a Fourier trans transformation is I have some information provided here so first of all here is a description from the book machine learning for the quantify itself so the idea of a Fourier transformation is that any sequence of measurements that we perform can be represented by a combination of sinusoid functions with different frequencies this is an illustration that I found online so it's from this Source over here I think this really illustrates well how we can decompose a signal into its different frequencies so that is basically the goal of applying the Discrete Fourier transformation we're going to look at the individual measurements and then using a lot of complicated math that we won't get into in this video decompose the original signal into the different frequencies so we will be extracting different frequency components and here's another description of why this is helpful to machine learning so more efficient analysis of the data because it provides a way to better understand and model complex data sets as the frequency components produced by the DFT can provide insights into patterns and trends that would not otherwise be visible visible some of the features that we will be extracting are the amplitude for each of the relevant frequencies that are part of the time window the max frequency weighted frequency which is the afference average and the power spectral entropy so I have to say here and state that these frequencies are part of the machine learning for the Quantified Self case studies that are explained in the book and also on git and the code that we will be using to compute all these values is in the frequency abstraction.pi file that we've added to the workspace at the beginning of this video so here you can see how everything is calculated so here we have the max frequency the weighted the power spectral entropy and here the amplitudes for each of the frequencies available in the window but the math Behind these calculations is very complex and I have to admit that this is also outside of my area of expertise so we're not going to dive into this so here you have a Wikipedia page where you can find some information about the math Behind these calculations so I won't and also can't explain how everything works behind the scenes that we are about to do but that is also not my goal I just want to show you what a Fourier transformation looks like so input data output data and then later show you how the new features that we've generated affect our model so please keep that in mind this is also outside of my area of expertise I'm just following the principles from the book and the GitHub repository of the machine learning for the Quantified Self alright so let's get started so we're going to create a new frequency data frame and create a copy of the previous data frame and we're also going to reset the index because the functions that we are about to use expect and discrete as an input and now we have a Time series index so let's just reset the index and then check what it looks like so with the same data frame a copy but now we have a discrete index from zero all the way to the last value and now just like with the numerical abstraction we're going to import the class and initiate a new instance so we first have to add the import so we're going to do from and this is the frequency abstraction we're going to import the Fourier transformation class so let's run this and then come back to our code and we're create a new variable called freak apps and then set that equal to the Fourier transformation class and let's initiate it and let's have a look at what we are actually doing right now so we are creating a new instance of this class and in here we have two functions so we have find ffta transformation and then we have the abstract frequency function so these are the functions that we will be using and we also have some documentation over here about what kind of parameters we need as an input we are going to use the abstract frequency which takes as input the data table The Columns the window size and the sampling rate so it's kind of similar to what we saw with the temporal abstraction and then the numerical abstraction so we have a data table column window size and then here we have aggregation function and with the frequency see abstract frequency we need a sampling rate so let's see how that works so let's start off by defining the sampling rate and the window size and if we come back to the documentation over here we can see that the sampling rate expresses the number of samples per second and that is 5 in our case but let's define it properly this will result in five and then for the window size this time we go are going to set the window size to the average length of a repetition so that was about 2.8 seconds so we will set it to 2800 milliseconds then divide it by 200 which will give us a window size of 14 meaning that we are going to compute all the frequency features with a window size of 14 okay so let's see how that works and let's just first apply the Fourier transformation to one column them as usual to get an understanding of what this will result in so we call our frequency apps variable and then do a DOT and then call abstract frequency and then let's check what do we need we need data table column window size sampling rate all right so we just pass in the data frame start with the Y acceleration and then we're going to do let's check what's the first one first window size so we do window size and then sampling rate let's have a look so this takes some time to compute and we get an error and that is because just like with the numerical abstraction the column is expected to be in a list because we can also input multiple columns so let's run that one more time should work right now so this takes some time all right and once is finished we can have a look at our new data frame good and we can see that we have some additional columns so if we compared it to our previous data frame the temporal one we had 29 nine columns and now we have 41 columns so if we have a look at the columns we can see that starting from here these are all the columns that were added so max frequency weighted and the power spectral entropy and then all the different amplitudes so this is just for one variable so you can imagine how if we are going to Loop over the data frame over all of the predicted columns we are going to add quite a few values columns to the data frame so now let's have a quick look at what we've actually did so let's just first create a subset for set 15 just random and then plot this over here so here we can see the Y acceleration for set 15 and now let's have a look at some of the values that we've added and here you can see what that looks like so I've uploaded a couple Max weighted power respect to entropy and two random amplitudes so all the complicated math that is happening over here results into the values that you see over here so that was for one column now let's apply the same trick that we did over here by looping over the data frame and all the columns and also splitting by set and here I'm going a bit faster because it's basically the same as we've already did so we create a empty list that we are going to store the subsets in then we're going to Loop over the unique sets within the data frame then I'm also going to add a print statement here because this can take a while so you get an idea of where it's at and then we create the subset based on the S That We're looping over and then we're going to extract all the frequency features and it's nice that we can add a list of columns in here so we don't have to actually Loop over everything but we can just do it like this and I think yeah we could have also we we could have actually done that here as well I I noticed so the loop over here was probably not necessary but hey if it works let's just not touch it but let's just make this a bit more efficient by just adding the predictor columns in here instead of looping over everything and let's see what we're doing right now so here you can see that it's applying the Fourier transformation set by set and here you can see that it takes a while that's why it's nice to to have the print statements and depending on the machine that you're running this on this could even take longer or go faster so let's wait for this to complete all right and we're done so now let's do the same trick again where we do the PD concats and then put in the frequency lists and then let's see what that results in so we have a total of 107 columns and we can see that we have now more missing values in the frequency domain because we use a larger window size and we will deal with that in a bit but for now let's just store this in the data frame and overwrite it like this so then we have our original data frame again now with all the additional values and we're going to add one more statement over here and that is the set index we're basically going to take the index and set it back or the original index and set it as index again and drop the discrete index that we're seeing over here which should also be reset as you can see because of the loop that we were using so now let's just run this and then let's have a look and now we have our original index so that was just an intermediate step in order to get this function to work and now we're back to where we were and now before we continue I have to correct a small error so I will indicate this in video so you won't make that error but just to keep that in mind I'm going to put the gyroscope R here and not the Y so we were missing some values because we were adding the gyroscope y another time that was already in the predictor columns so in order to correct this in my situation I have to basically run everything over here again but now with the correct values and then do that also one more time for all the frequency features all right and after running everything again we should now have this final data frame with 119 columns so if you've been following along and paying attention to the mistake that I've made and corrected it earlier on this should be where you are at okay so the next step is dealing with overlapping windows and now you can imagine since we've added all these extra columns which are all based based on a rolling window that the values in all of the columns between the different rows are highly correlated and this is typically something we want to avoid when we are building models because this could cause overfitting in order to tackle that we're going to allow for a certain percentage of overlap and remove the rest of the data so let's see what that looks like but to start off we are going to drop all of the missing values that are currently present in the data frame and in order to do that we're going to take the latest data frame so they have freak and then call drop and let's see if we run the original data frame we can see that there are 909 rows in here but if we drop all the missing values we can see that we got rid of quite a few rows over here so about 1300 in total and that is the result of the window size of 14 steps that we've applied during the calculation of the frequency features we get rid of those and then store it into the data frame and now we can see we have a new data frame with less rows in total but no missing values now and in order to deal with the overlapping Windows we're going to get rid of some part of the data and when you look in the literature typically an allowance of 50 percent is recommended meaning that in our case we will get rid of 50 of the data by skipping every other row and this will result in a lot of data loss but it has been shown to pay off in the long run by making your models less prone to overfitting and in our case we still have enough data so we will use the 50 window or overlap in this case but when you have very small data sets you could also change this number to allow for 80 or 90 percent overlap meaning that you just will you will get rid of less rows overall but in order to do this we can use the I look method of the bundles library and typically how you use this is use first specifies the rows that you want so for example I want all rows all columns if I do it like this I get the original data frame but now what we can do we can also just do it like this and then specify just all the columns and then say I want everything up until the second row so that is how you normally use I look but there is another way you can use eye lock and that is if you add another semicolon over here or sorry another column then you can specify that you want every other row or every second row basically so if I run this we can see that we now have a result data frame that is half the size of the original data frame and if we look over here we start at zero then we go to 200 and then to 400. if we compare that to parse data frame we see that we start at zero and then we go to 400 so we skip the 200 one meaning that it will skip every other row and then starting from the first initial row of the data frame and in doing so so by removing every other row we basically reduce the correlation between the records the adjacent records that are in the data frame and yeah as I've explained this reduces a potential overfitting of our model and has proved to be a solid method for dealing with highly correlated values as the result of multiple aggregated features using a window so let's store that into our original data frame make sure that it is stored and saved check it out okay so we now have a much smaller data frame still 119 columns and now it's time for clustering and for this we will be using the K means clustering library from the SK learn package so first we have to import that so we're going to import from SK learn cluster import K means not K means hey we don't want k-means plus we want K means this is the one yes all right so import that make sure it loads correctly and then we can continue with adding clusters so first very briefly what is clustering and K means clustering in this case so it's an unsupervised machine learning algorithms that we can use to group data into clusters based on their similarity and how Decay means algorithm works is we start off by defining K that is the total amount of clusters that we want to use and then it basically calculates the distance between each of the data points and the centroids that it has defined So based on how close they are to that centroid and so this is a popular feature engineering technique and as I've said it's an unsupervised learning algorithm so you can basically provide the data and the amount of clusters that we want to use to the algorithm and then it will basically group all of our individual rows to a specific cluster so we can basically group based on a unsupervised learning algorithm and this is typically done for segmenting customers purchase Behavior anomaly detection image compressions so a very useful technique and we will be applying it to our sensor data set in order to see what kind of groups it can come up with and again just like with the principal component analysis we can use the elbow method here again but now using different values namely really the inertia which is the sum of squared distances of samples to the closest cluster Center and we will be Computing that for each K so for each number of clusters and we will basically Loop over various numbers of K and then create a plot like this which is taken from this resource this webpage over here so here you get a better understanding of how this works and also see some more code examples but this is basically what we will be doing and we will use this to determine the optimal amount of K for our data set okay so let's come back to our code and we'll start off by defining our cluster columns but because we definitely don't want to cluster on everything that is in there we are going to limit it only to the acceleration data so that will be the X excuse me the X Y and Z of the accelerometer data that is what we will be using to cluster on and then we're also going to set a range for the amount of K the amount of clusters that we want to test and for this we'll Define K values and we're going to state that in a range from 2 up until 10 and now we are also going to store the inertia in a list which will first Define as empty make sure to run everything and now let's Loop over the data frame and create the Clusters but before we start let's create a quick copy of the data frame so we create day F cluster will be a copy of day F and then let's just copy it so now we have a new data frame and then let's go ahead and say 4K in K values we're going to create a subset based on the cluster sorry not the cluster columns first the DF cluster and then creating a subset by inserting the cluster columns so this will be a subset based on only the columns that we want to cluster on and now we're going to Define our k-means model and we're going to set that equal to K means and then K will be equal to K and then the init will be equal to 20 in this case and the random state will set to zero oh sorry it's n in its in here not the init but the n in it and now let's continue by specifying the cluster labels and we're going to do a few steps in one here we're going to call the K means and this is a scikit learn model and typically you if you work with the scikit-learn library you first do a fit and then you do a predict but most scikit-learn models also have a fit predict method that you can use to instantly train the model and also make predictions and that is what we're going to use in this case so we're going to to use this model and then input our subset as the input so what this will do we have initiated a k means model over here with K set equal to Decay That We're looping over the next step what it will do it will take that model and call the fit predict then input the subset and create the cluster label and what it will do by calling the fit predict it will for each of the rows that is in the subset it will assign the row to a cluster and on the first iteration of this Loop K will be 2 a row can either be assigned to Cluster 1 or cluster 2 so that will literally be value 1 or value 2. then on the second iteration K will be 3 and then naturally each row will be assigned to one of three clusters one two or three and it will do that all the way until K is 10. we will store the cluster labels over here but the next thing that we have to do is we have to store the initias because we want to use the inertia's to plot them on a graph and then use the elbow technique to determine the optimal amount of K so this is just a preparation step in order to determine what we are going to use as K so the final step over here is to call the inertia's list called append and then we're going to call K means and then we can call Dot inertia underscore so if we take a look at the documentation from the scikit learn library and look at the k-means library here we can see all the parameters that we can adjust to tweak the performance of the model but we can also see the attributes and these are the attributes that we can call upon after fitting the model so after training and here you can see we can call inertia underscore which translates to the sum of squared distances of samples to their closest cluster Center so that is how this works and why we can call k-means inertia underscore so to summarize Loop over all the values create a subset train the model export the inertia let's run this see if everything works oh and we get an error over here because as you can see it's n clusters and not K so I have K is K here but we need n clusters equals K in this case and now if we run this it will succeed and then we can plot the energy as let's make sure I run this twice the values over here so 2 all the way to 10 and now we can use this list to create a plot and visually inspect whether we can determine an elbow and in order to plot the graph I'm just gonna copy and paste this code over here and we get this results and this is looking like a textbook example as we can clearly spot the elbow over here so you could discuss like shoot the optimal amount be four or five but I would argue that 5 would be the optimal amount of clusters in this case as you can clearly see that this is like the elbow joint and afterwards the decrease in sum of squared distance diminishes so what we will now do is we will basically this piece of code over here but now we're going to instead of looping over K we're just going to set n cluster so k equal to 5 and then Define our model as that so with those parameters then we're going to create the subset again which is as you can see the same as before so our day of cluster is still the data frame that was just a copy of the frequency data frame and now we are going to call the cluster label so we can even get rid of that what we can do is we can say day F cluster and then specify a new column called cluster and then call fit predict on the subset with make sure to run this line first that is set to 5 run all of this and then let's see what the result is so now if I have a look at this data frame we can see that we have one additional column 120 in total and if we go all the way to the back I can see that we have a cluster column over here and we can only see on our there's also a 4 over here the cluster starts at zero so previously I indicated that would that it started at one but I see now that it starts at zero so it's zero index we have a total of five clusters so zero all the way to four and each record is assigned to a specific cluster so now let's have a look at what that looks like officially to see if the Clusters make sense and in order to do this I'm going to copy and paste another block of code in here that you can follow along yourself but just to save some time I'm going to run this and this is a very cool Plus in my opinion so what we are doing right here is we are using the product projection 3D from the modplotlip library and then We're looping over the data frame and then splitting by cluster so we're calling the unique method again to Loop over all the individual unique clusters then create a subset and then this is the interesting part we're creating a scatter but instead of what you typically do is inserting two values so typically it's an X and A Y and you get a two-dimensional scatter now we can input three variables x y and c and by stating that this is a projection is 3D we get a 3D plot with the three different axes so this is very cool and here on the legend in the top right corner you can see which color corresponds to which cluster so this looks very cool and we can clearly we tell that the Clusters are groups together meaning that there are green clusters over here yellow blue Etc so you can also see some some noise over here so for example there are also some points over here that um are allocated to the blue cluster which is cluster zero but you could argue like do they really belong to this cluster um but of course it's this model is never perfect but from just this visual inspection it looks really good but now to really put it to the test we should compare this plot to the same plot but then splitting not by cluster but by label so by exercise so let's do that right now and this will be another block of code which is basically the same but now instead of looping over the unique clusters and then creating a subset based on cluster we're going over the unique labels and then subsetting by label so everything else is the same and now if we run this we get another plot and now the interesting part what we can do here is we can have a look at the individual exercises and see where they belong within this three-dimensional plot so we can see that the bench press and the overhead press very close to each other which is to be expected because the movements look a lot like each other then also in green and gray we have the deadlift and the row which also look very similar to each other and those are grouped over here then we can clearly see that the squat is very different from all the other exercises and the rest is all over the place which is also to be expected because rest is pretty random because the participants were not restricted to certain movements by looking at the Clusters and then at the images over here we can clearly see that for example cluster 3 definitely corresponds to the squad over here and similarly we can see that cluster 0 probably captures the brand bench press and the overhead press data and here you could also argue the decision to create five clusters because we have six labels in total and we are using five clusters so you see that it's grouping the overhead press and the bench press it's grouping them together in the same cluster and also you don't really see that much of a difference between the deadlift and the row over here which if I look at the Clusters also seem to be grouped together so this is very interesting but potentially a lot of valuable information that could benefit our predictive modeling and that brings us to the final step of this episode and that is as always exporting the day data so we will take our cluster data frame and we are going to pickle this so by calling two pickle and then we'll store this in the intermediate files again so data interim and then we're going to store this as O3 data features dot pickle let's run that have a look at the final data set so you should have a cluster column over here 120 columns in total this amount of rows and then we can export it and that brings us to the end of this episode now please like this video And subscribe to the channel and then I'll see you next week in part 6 where we dive into probably the most exciting part of this whole project and that is predictive modeling