GPT Engineer... Generate an entire codebase with one prompt

Key Takeaways

like what the what is going on here man I just literally provided it with an example of the data and the buff and I have a complete data science project over here this took me like five minutes man what is going on in this world of AI so we have another very exciting thing that I want to show you guys this is basically what I wish Auto GPT was a couple of weeks ago but I couldn't quite get it to work this way as you remember it got stuck in Loops when it tried to create files but now we have this new thing all of a sudden which is called GPT engineer and if you're an engineer then this is a must watch and you have to play around with this because this will change everything and what it basically allows you to do is you can basically start off with a prompt with a question a coding related question so an example they say like hey code a snake game and then what it will do it will start to work on that but the cool thing is it it can write to files so it will create code files on your within your project directory and it will split it up very dynamically very neat into the various files so these could be classes functions Etc depending on what kind of language you use this is really cool so in this video I'll show you how to set up how you can get started so let's get into it alright so we're starting off on the GPT engineer GitHub repository and we are going to first clone this repository now if you are an engineer you should kind of like already know how to work with GitHub but just I'm going to show you how to quickly like set this up then follow the instructions over here and then you should be all good to go so I'm coming to my folder over here I'm in my YouTube projects folder over here and I'm quickly going to open up a terminal over here and I'm just going to clone the repository over here which will basically download the folder and here you can see you have a GPT engineer folder now over here so I'm going to CD into that and then I'm going to open up vs code within that folder now this is not like a hard requirement you can also do it like with another IDE and then use the terminal but I'm using vs code over here so I'm just going to quickly save this as a workspace so now I can just like easily open up this project in vs code from the workspace folder over here alright so that's step one then we have to install the requirements and for that make sure that you are working in a python environment that you can use so let's just open up a random python file let's say the main file over here and I'm going to select my AI experiments Konda environment for this and then what you would normally do if you have for example a new environment you could open up a terminal and then have a look at the requirements over here so it basically only requires open Ai and typer which for me are already installed in AI experiments over here but you can just like come back to the get the repository over here and do like the PIP install R requirements that txt if you paste it in the terminal and run it alright so then the next step would be to create a new project basically so there's an example in here and you start off by just copying and pasting that one so I'm just gonna con command C command V and now I have a new project over here and now I have an idea over here that I want to try and that is one where we are going to do an experiment with a neural network and more specifically if we then open up the main prompt so this is basically where you start so create a new folder or create a copy and fill in the main prompt in your folder this is basically the starting point so here you can see in the example there is we are writing snake in Python and the components are split into separate files keyboard control alright so I've quickly changed the prompt so this is basically what I want to test so create code to train and test a neural network using a typical machine learning pipeline use a fake data set and regression meaning numerical data processes it with the pipeline splitting into train tests and evaluated the performance by creating line plots use skle learn and math mode clip and now my rationale behind this basically is this is something I tried previously with auto GPT to complete to create basically complete like data science machine learning projects and not just like using gpt4 for example to get it like step by step but really create like Cookie Cutter like boiler templates that you can use so I'm really curious to see what this will come up with so this is the main prompt alright so now all that's left to do before we can actually run this is to export the open AI API key so how that works is you basically copy or type this over here say so this is on Mac I will show you how to do it on Windows in a bit but you start with export open AI API key and then you just like fill in your key e and then hit enter so what this will do within this terminal session it will store this variable basically in the various scripts that will be run if you use GPT engineer now have access to this key so this is a little different from using the dot end file which I typically show in my videos you can also do that but that requires you to change some of the scripts this is the most straightforward one and so like I said this is on Mac and on Linux and if you're on Windows basically the only thing that you have to do is change up export and set it to set so instead of export you change it to set and everything else is the same then you do the key and then another neat little trick that you can do so on default basically this uses the GPT 4 model as you can see right here so that is in the main.pi but if you don't have access to gpt4 yet you can just like Swap this out with any other model from open AI so for example you could put the GPT 3 and hold turbo model in here alright so now with that out of the way we have everything we need in order to run the main.pi so that is done by recalling python main dot Pi make sure you're in the root of the project and then just calling your projects which is in my case neural uh not project in your own network all right so let's run it and let's see what it will come up with so at first says okay I need it's going to make a summary of the areas that need clarifications so this is the really cool thing about GPT engineer is that it starts off with like the main prompt but then it proceeds to ask clarifying questions so can provide more information about the fake data set such as number of features size of data targets variable okay let's see so for data let's use an example of industrial iot data bundles data frame 100 000 records use flow of a pump as a target's variable and come up with other variables on your own so then what type of neural network would you like to use let's just say like keep it sklearn pre-processing steps so let's say data cleaning and feature engineering and scaling so I'm just leaving any errors I'm just gonna like type it out like I'm a three-year-old and see what it can come up with because the thing is I want to replace the work that I do with this so I can be an idiot and play GPT engineer just do everything so it's generating I guess the steps or function it says so these are like the function functions that it's um that it has to create and now it starts off to write like the main.pi okay so this really exciting already so it's now actually creating the code let's wait for this to finish okay and it's finished so this is so exciting so let's actually see because I've been like watching what it was doing and it actually looks really good but now let's okay so now we have the workspace folder over here we have a bunch of python files which first of all like is really amazing to me like if you compare it to like doing it with GPT you still have like that manual process you can ask questions you can get the same thing because under the hood of course it's using the same model but doing it in such a like automated way where you can like you see the python files show up in your actual IDE to me it's like man where is this all going like damn okay we have a main.bi file it's the general flow of everything that we have to do like generate data process train evaluate okay so let's just fire up an interactive session and like okay because this is where the real test of course is like is this going to work and then here there's one error and that's I already noticed that because there's no import over here so we can pretty easily like fix that by saying like hey from SK learn that pre-processing import standard scaler quickly spin up another session quickly see okay this should be everything that we need plot results okay all right let's generate the data back to this train the model and now we can evaluate the model and get the R2 and the mean squared error so let's see and we can now print those and boom we have a very solid model and a really low mean squared error so the neural network was able to capture basically the underlying formula that is in here to capture the data basically all right so let's try one more test because I'm really curious to see how it will handle this what I did basically is I created a data folder within like the main project and I have now provided basically the main prompt in the data set test project with a little bit more information so first of all I'm saying like hey based on if we're running the code which is run from the workspace folder I've basically linked to where the CSV file is so I say like hey here's the data set that we're going to analyze so it's going two steps back into the data folder and then Bike Share CSV then I've provided it with the structure of the data which is just loading the data frame into loading the data frame and then doing running.info and then just copy and pasting the results I just like put it in here to see if it will work this way and now I've provided a little more like in-depth instruction on how to handle the data set because based on what the previous model so to say was asking I've provided it with some additional information here already and now I say we want to basically compare three machine learning models and compare them using R2 and mean squared error so let's see what happens if we run this okay and I see that it put this all into like a nice little function but I just gonna I'm just going to run it line by line to see what we got going on over here so first of all like the data path it like took up on that and we can like instantly load the data so that's really good so now we pre-process the data which is in this case just doing so what is it doing so it's okay so it's going to drop some columns x y and scale it okay so now it does all in one pretty neat pretty neat okay and now we're going to train the models let's see if it actually works okay it goes really fast and now evaluate the models this actually worked okay this is really really cool so we have the R2 like 70 60 like 80 so the random Forest is really the best one over here but what the I didn't put like any effort into this and we now okay so this is a pretty straightforward data set but we have a a random force with a score R2 score of like 80. it's a completely like properly structured piece of code this can open up like so many possibilities like again this is just like another step towards creating like fully automated pipelines of setting up projects we've now basically removed the human bottleneck of like typing into chat GPT copy and pasting it figuring out which file to put it in and then loading it um into the main file for example which which is done over here so this second experiment with the data set test the two to probably like 5 five minutes to set up this whole project alright so that is GPT engineer I'm very curious to see and to hear what you guys are going to try out with this so let me know in the comments what you will be doing with this you now know how it works I'm definitely going to continue experiment with this see what I can come up with and then in the meantime please if you got value from this video leave a like also subscribe to the channel it will really help me out also make sure to check out the links in the pinned comment right below this video there will be two links that could be very interesting to you so the first one is for the data freelancer Mastermind now this is for you if you want to launch and scale your freelancing career in data but don't know where to start and the second one is a sign up for my newsletter so this is really for people that are serious about learning data science and artificial intelligence and want to stay in the loop of all the things that we're doing behind the scenes so go check those out and then I want to thank you guys for watching and I'll see you in the next one [Music] foreign