My Real AI Coding Workflow (build anything)

Key Takeaways

Today I'm going to show you my real AI coding workflow. I'm going to build an app completely from scratch using AI and show you how I set up my environment, how I prompt the model, the various tools that I use, and how I overcome the issues that are bound to happen when you're coding with AI. I want this video to be really realistic and very pragmatic so that you can see how an experienced developer uses these tools to build something quickly and effectively. So with that said, let's dive in. So before we can build anything, we need to know what it is we're going to build. It sounds obvious, but this is the first step in my workflow and a lot of people skip this and it leads to really poor results later on. You want to spend a lot of time here researching alternatives, seeing what already exists, picking your tech stack, your tools, which I'm going to go through in 1 minute, and coming up with a really good high-level outline before you just dive in and start building a random product with AI. Now, for this particular video, what I want to build is an AI shorts tool. So, something similar to like Opus Clip, if you've heard of that before, where effectively I drop in like a long YouTube video, and then AI automatically generates a bunch of shorts for me where it has captions. It kind of reframes my face because I need to go from like a 16x9 to a 9x6 clip, so like horizontal to vertical. I want to have like obviously good moments. I want to have a title, caption, all of that kind of stuff. So, that's my idea. You can build anything that you want but we start from the idea and then what we need to do after that is start doing some research and understanding first of all what tech stack are we going to use what tools are we going to have in that particular stack and then how can we set up our coding environment so that we access all of those tools in an efficient way which you're going to see here in just one second now I've done a little bit of research before this video just to make this a little bit faster and I'll walk through how I did that but I created a quick slideshow I actually used claude code here to make this slideshow for me where I essentially just kind of dumped in my idea, gave it a prompt and say, "Hey, this is what I want to build, help me do some research, figure out the tools that I should use, and determine a tech stack that would make sense for this project." I went back and forth with it with multiple questions, and I just kind of started gathering, okay, what tools make sense, what flow makes sense, what should the app look like? And that is phase one. And I don't want to spend too much time there cuz I know a lot of you understand that. But let me go through the tools that I'm going to use for this particular video so you can see kind of how I've mapped that out. So while I was doing some research, I kind of realized that the easy part of this application is actually picking what the clip should be. So essentially looking at like the transcript of the video and going, okay, like this portion of text makes sense to be a clip. But the more difficult part is actually how we can get those clips to be turned into real clips, right? So, like I have the timestamp sections I want, but how do I take the video? How do I turn it into another format? How do I add captions? How do I stream it? How do I do this quickly? How do I download it, upload it? All of this kind of video processing related stuff. Yes, you can do it locally on your computer. If you want to build a web app or something though, it's a little bit more complex. So, for this particular video, the three tools that I kind of realized that I was going to need is Whisper. This comes from Grock and it's going to allow us to do essentially the audio transcription. So, we can pass in an audio file. We can get back all of the text that's in there. We're going to be using Claude. That's kind of our AI model, which is going to handle actually selecting the clips and then writing the different captions and titles for the video. And then we're going to use a platform called ImageKmit. They've actually been a long-term sponsor of this video and they reached out to sponsor this particular video because obviously this makes sense where what we're going to be able to do is turn whatever moment we have into a real video. I'll talk about how that works later, but don't worry, it's free to use in case you want to follow along. Okay, so at this point, I kind of know the main components that I'm going to need for this app. And I also decided, okay, I want this to be a web application, so I'm going to build this using Nex.js. And that's something that I talked with Claude about, and we kind of decided, okay, that text stack is going to make sense. Now, after that, how are we actually going to build this? And now, this comes into the AI tooling that we're going to use, which I'm going to show you how to set up live in just one second. Now I'm going to be using cursor and we're going to use the clawed model to build this out. There's a lot of other models that we can use but typically I find when I'm building something completely from scratch that whatever the frontier claw model is whenever you're watching this video typically performs a little bit better. And then when I want to do large refactors, revisions or if I have a lot of repeated mistakes, I'll switch over to something like GPT 5.5 or a faster model if I just want to do UI related changes like composer 2.5. So, what I'm actually going to do here is I'm going to start setting up cursor. So, it has some agent skills and tooling that's going to make it easier to work with the tools that we need for this particular project. So, with that in mind, let's start by if you don't already have it, downloading cursor, opening it up, and I'm going to show you how I set up my environment. Okay, so I've got cursor open here, and the first step is just to open a new folder, which I've already done. And you'll notice that I've just opened my kind of agents window over here on the side. And what I've done is I've just selected the model I want to use, which is Opus 48 because Fable 5 is not available right now. And I've gone into edit. I'm going to turn on the fast mode here. I'm going to leave the thinking mode. I'm going to give this the 1 million context window. And I'm going to put it in medium, which is kind of the sweet spot for a lot of the stuff that we want to build. You can go high or extra high, but it's going to take quite a bit longer. Okay. Now, this is going to cost me some money, by the way, in cursor. It's not going to be absolutely free to run, but this is personally what I use. I then have the agent modes and all this stuff, which we can change, but for now, I'm just going to leave it in kind of one agent window. And I prefer to use this view where I can actually see all of the files rather than going into this new kind of agents window thing that they have which is I think the default view that it brings you into where you're just chatting with the agent. I don't really like that because I find that it just I don't know for me it's easier when I can see the code that it's writing and I just work kind of in this one chat window right here. And the reason I use cursor is that I found it performs a lot better than things like claw code or like codec because while it's using the same models, it has a coding harness that's built by the cursor team that I just find works a lot better. That's my personal experience and that's why I use it. And then you'll notice that I still do keep clawed open here. I have claude code open and sometimes I use the chat windows. And if I just want to have kind of a side conversation or I'm doing my research or I want whatever I do to kind of be saved and easier for me to look up across multiple devices, then I'll do that directly inside of cursor here. So you can see like I built that presentation. I was doing some research on the video um whatever that I wanted to build and I do that in a separate window. Of course I don't do that in cursor. I only go to cursor when I want to write the code. So the first thing that I ultimately want to do here inside of cursor is start developing a plan for what it is that we're going to build out. So what I'll typically do is I'll start dictating into Cursor and describing my idea and then telling it all of the decisions that I've already made. So I've got this tool on my computer. It's called Whisper Flow. You'll see it working here in just 1 second, but allows me just to speak into this microphone and then directly have all of my words transcribed into text, which obviously makes this whole process a lot faster and I would recommend that you use. Uh, so I'm just going to show you how that works and then we can dive into that because that is kind of my dictation that I use for AI. It actually works natively in cursor and it can like tag files and stuff. So what I'm going to say is the following. I want to build an AI shorts tool. The basic idea is I want to upload a large file, some standard 16x9 video, and then I want to convert that into multiple shorter clips. The idea is that I want to take this long video, I want to generate those shorter clips, and I want to do that by analyzing the audio transcripts. I would like to build this using Nex.js and make this a web app and I want to use the following tools. First, for the transcription of the video, I want to use whisper from Grock. So, I'll provide a Grock API key. I then want to use Claude, specifically the sonnet model for actually picking out the moments in my video and then generating titles and captions for those particular shorts. Lastly, I want to use image kit for handling all of the image and video related tasks. For example, I wanted to reframe the face of the user. I wanted to add captions. I want it to give me the streamable video format. And I want it to handle actually uploading my video, allowing me to download it, etc. Can you create a high-level plan and kind of architecture document for this application so we have the context in our project? Ask me any questions that you need before we proceed. Okay. Okay, so I know that's a long prompt, but effectively what I'm trying to do here is I'm trying to give some initial context that we can start generating so we can have some markdown files in our application or yeah, I guess in our application that we can use kind of forever and in different chats. So if I want to open up multiple agent chats or I want to paralyze this, it can always come back to that initial context and understand what this app is and the decisions that we've already made. So what I'm going to do is I'm going to run this right now. And by the way, if you want to check out Whisper Flow, it's free to use. actually have a long-term partnership with them, too. So, I have a link in the description you guys can use. And you can see it gives you a bunch of stats and stuff and like you obviously type a lot faster or sorry, speak a lot faster than you're able to type. Okay, so we're going to wait for this to finish. That's what I'll do in this video. I'll skip through a lot of the long prompts. Once it's back, I'll be right back. I'll kind of show you what it's generated. And actually, we're already here where it's asking me a few questions. So, I'm going to go through and answer these different questions. I always recommend you do that in the beginning prompt. You tell the model, hey, ask me some questions. And now they actually have a built-in feature where you can answer it inside of cursor. So I'm going to answer all of those and then we'll continue setting up the environment. Okay, so I've answered the questions here and while it goes through that, I just want to show you kind of why I know the capabilities that these tools have. So what I actually did is obviously again before starting this video and what I would recommend to you is choose the tools you're going to use. So in my case, I found ImageKit and of course they've sponsored the channel so I know that they exist. And if we scroll down here, I've kind of figured out, okay, they have automatic image and video optimization. they have this like resize, crop, and transform feature, right? I know that they can do kind of alignment on the face. So, they can take a video and they can focus it on your face and kind of move it around. So, because I know of all of those features in this tool, I know what to ask the AI model to do because I've read through the documentation, I understand kind of the capabilities of the tool. Now, same thing here with Grock, right? Like I looked and I saw, okay, they have this whisper large V3 turbo. It's very fast speech recognition. And then of course I know with claw that I can use an API key and I can kind of get the information back by analyzing the text. So because I have that baseline context and I kind of know what's possible with these tools, I can better inform the AI model. Now what we're going to do next after this finishes kind of our planning prompt and we get that initial context is we're going to start adding various MCP servers to claw or to cursor sorry which is what I always do based on any of the tools that we're using. So now a lot of the tools that exist, think firecrawl, think you know, whatever. All of these like just new frameworks, they all probably have an agent scale or some kind of MCP server that help to explain to the model how to use them. So in the case of image kit, they have this link, I'll leave it in the description, build with AI, where they have agent skills, MCP servers, and plain text documentation. So rather than me having to know all of the docs and copy and paste stuff in and share like a URL to a website, I can install this skills, which we can actually go ahead and do right now so that the agent understands how to use this tool. So what I'm going to do is I'm going to go here to the cursor um setup. I'm just going to copy this and I'm just going to go run this command in my terminal. So let me open up the terminal and go ahead and run this. And it should just install these skills for me and automatically add them here inside of cursor. And you can see it does that and we now have agents skills and we get a bunch of skills built in directly from image kit that now cursor can start using. Now as well as that we also want to add the MCP server. So it says hey we can just add these MCP servers right to cursor MCP.json. So rather than me even doing that I'll just go tell cursor you know add these MCP servers. Okay. And it should be able to go and automatically add them for me. And then what we'll have to do is just authenticate with the MCP server so we can sign in and it can manage our image kit account. And obviously for this video, if you want to follow along, you will need an image kit account. Like I said, it is completely free to use. And of course, they have like a pro version you can upgrade to where you get better bandwidth, more memory, you know, higher upload limits, etc. And if you want to get the information from that account once you create one, which you can do from the link in the description, go to developer options and you'll see your endpoint, which you need, your public key, and your private key, which is effectively how you authenticate with image kit. Okay, so I just reloaded the window because it told me that it added the uh MCP server here. And in order for the MCP server to work, I just need to open the MCP settings. So what I'm doing in cursor here is I'm using the command prompt, command, shiftp, or control shiftp. I'm going to go MCP and then open MCP settings. And you'll see now that our MCP servers are added. Also notice I have the GitHub MCP server which you can add as well. And I'm going to go and press on connect here for my image kit API and paste in my image kit private key. So again, this is all me just setting up the environment. So it's going to be more efficient later. So let me just view and copy this key. Okay. So now you can see that I'm connected to image kit. It has all of these tools. So it can manage my image kit account for me right from the MCP server. And then I have this public MCP where it can search through the documentation and stuff uh if it needs to use that. So now I'm kind of set up cursor, right? I have my MCP server. I've added the different skills that I need. And if I was using other tools, I would go through and I would add all of the ones for those tools. And I want to just quickly kind of have a look at the documentation that we have here and just make sure that this makes sense and that it's kind of outlined, you know, the tools that we have. Okay, so it looks pretty good to me in terms of the architecture and the road map. Of course, we can change it later. For now, what I'm going to do is I'm just going to keep the changes. And sorry, I need to go to this one and go keep the changes of what it's created. And I'm going to just tell it to start setting up git here just so that we can save everything and we can roll back if we need to. So, I would highly recommend adding the git mcp server, the github mcp server, so it can create repositories for you. So what I'm going to tell the model is create a new repository for me and start setting up the project so that we're always going to be making commits anytime we make major changes so that we can roll back in the future. Add this as a specific rule for this project that we're always going to want to commit after any major changes and we should just do this automatically. So now what I'm doing is I'm adding another feature where I'm adding rules to cursor here. So cursor has the ability to generate rules which we'll probably see it do here in one second. Rules are something that will always be brought into the uh what do you call it here kind of uh prompt. So you can see it's created a folder for us called rules and it's kind of made this rule file so that cursor always knows or the models that we're using always know hey I want to do this thing whenever I'm writing code. Okay. So actually it's moving it out for some reason or it's doing something with this. But the point is it's going to make this rule and then as we keep working with this we won't need to tell it to make commits. It should just do that automatically. Okay. So now it's created the rule. It made the GitHub repository. I can see it. It showed up in my Git account and we're essentially ready to go and start actually building this out. So, what I'm going to do is I'm just going to gather the API keys that I need because I'm going to have my image kit public key, private key, URL, etc. And then the Grock and Enthropic API key. And I'm realizing here that it's using Grock, not Grock or however you spell this cuz it's the one with the Q that I want to use, not K. So, of course, we're going to have to adjust that. So, let me just tell the model. And this is common. It's going to make those mistakes. I don't want to use Grock. I want to use grock with a Q. So, we're going to have to just change this. Say Grock like this uh for the transcription. Okay. And let's tell it that and hopefully it will adjust that. Okay. So, it adjusted that for me. We're going to keep going. Now, I'm going to fill in the API keys and then we're just going to kick off and start building the app. Okay. So, I've just put the stuff in env.local here. Now, what I could do is just say, okay, let's begin the task here. Start building everything out based on the plan that we've set up. And then if I want in cursor now they have this multi- aent feature or multitask feature I'll select it. I don't normally use it because sometimes it just goes crazy and it makes it a little bit unmaintainable. But for this project because we've set it up pretty well and we've kind of spent the time to structure it. Probably this is going to work. So let's run it and see if it can kind of spin up multiple agents and build this relatively quick. Okay. So the first prompt is finished. It built everything out. Uh if I go back here you can see we kind of have like these projects. For some reason this one failed. Of course, we're going to have to start debugging, but we can upload a video. That part worked. And if I go now to image kit because I upload a video from here, we should be able to see it if we go to the media library. And we can see here that it's successfully loading the video. So, the upload is working. And this is just like a a cut of a larger video I did just so it's a bit shorter. You can see 335 megabytes. And we are good. Now, uh something to note is if you are using the free version of Image Kit, the maximum upload size is 100 megabytes. So, keep that in mind. on the pro version that I'm using right now. It's 2 gigabytes and there's a bunch of other things you can see if you go to the uh billing feature. Uh I don't have it because I already upgraded, but you'll be able to see like all the different things in case you want to upgrade. Regardless, what I'm going to do now is just tell it, okay, you know, this is failing. So upload a video and we got this result. So literally, this is all I do when it fails. If there's an error message, I just copy it and I say after uploading and moving to the transcription phase, I get this error message. the video was uploaded successfully and it's in image kit but for some reason the transcription is not working. Can we debug this? So I'm trying to give it a little bit more info than just hey this is the error message so it knows at least the first phase is working. And now hopefully we're going to go here and debug this. Okay. And as this is running I'm kind of realizing that what it's doing doesn't really seem to make sense. It seems like what it's attempting to do maybe is send the entire video over to Grock for the transcription which isn't what I want. So just to clarify, I'm going to step in here and say just to clarify, we should make sure that we convert the video to so it's just audio. So we don't have any video and we're just sending the audio file to Grock for the transcription so it's significantly smaller. Image kit can handle this. And I know that ImageKit is able to give us just the pure audio off of the video. There's a lot of transformations that it can do for us and then the model will be able to figure out all of those because it's connected to the MCP server. But again, I know that because I've read through a lot of the docs and I've looked at like what's actually possible here with image kit and I know that it can do the refframing. I know that it can do the resizing. I know that it can give us adaptive bit rate streaming. So, we can actually view it in real time at different resolutions. And then I know that it is able to give us just the audio. And there's like URL transformations. So, you can see here that the way that this kind of works is that you don't need to do anything special. you just add these query parameters to the end of the video URL that's generated and it will automatically apply all those transformations for you very quickly which is why you would use this. So anyways, let's wait for this to complete and hopefully we'll be good to go. Okay, so it looks like it worked. So I'm just going to upload another video and we'll see now if the transcription is proper and what the state of our app is. And again guys, I'm showing this to you because this is what happens when you build with AI. There are mistakes, there are issues, and I actually am reading what the model is saying here to understand what it's doing and looking at the various tool calls because a lot of times I can catch very quickly if it's going to go down the wrong direction and just immediately interject or in this case again I just told it, hey, like fix this and it went back to the sub agent because we're in the multitask mode. Adjusted it and now hopefully this will work. Okay, so I just retested this. Looks like it's working now. I mean this is pretty impressive. It has the transcript down here and then we have the clips. The clips are not loaded yet. I think they take a second to uh spin up here from image kit, but at least it's picked like what sections and then okay, you can see now it's kind of showing up. Uh looks like there's a bug now with the clips where it's giving me like a full 4-minute long clip and I'm not getting the actual transformed version. So, let me send it another prompt and kind of tell it what's going on. But at least we're progressing and making progress. And actually, what I'm going to do in this case is I'm going to take a photo of this. I find this is super helpful too for the models. They can actually see what's on the UI. They should be able to see it themselves, but even still, I like to just highlight something specific. So, I'm going to add the photo and I'm going to tell it, okay, so the transcription is working now. However, we didn't actually generate any of the transformed clips with image kit. Right now, we just have the original video showing up five times. We need to actually generate the five different videos. Let's make sure we're doing those in parallel at the same time and that we're adding the captions and that we're doing the face refframing and as well as converting to 9x6. Okay, so just giving again some detail so it kind of knows what we need to do. And let's run it and see what we get. Okay, so after a little bit of back and forth, there was a few small errors that I fixed. I just cut those out. You can see that we now have the generated clips. If I play one, you can see that it kind of follows my face around to tell you >> and the clip selection is not great. So, we of course could adjust the system prompt and we can, you know, keep going back and forth with it. But I can download, I can rerender it, and now it's appearing and you can see we actually even have a thumbnail appearing as well, which is im uh generated from image kit as well. Now, one thing that we're missing is the captions. So, image kit has the ability to actually burn captions right onto the video so that it's automatically playing them for us. And for short form video, of course, we want that. So, I'm just going to go back to the model and just tell it to do that. And hopefully, it adds that. Okay, so all of this is working. However, we just don't have the burned in captions. Can we please add the captions to each of the clips? Okay, so it's been a few minutes now and we are back and we can see now that it has the uh captions at the bottom. Now, they're not really appearing high enough. So, that's going to be another prompt that we need to pass. Let me just mute this. But, if we maybe let my mouse go away here for a second, we should see that there are captions down below on the actual clip. Again, hard to see when the uh kind of video like I guess what do you call player is covering it. So we could tell it just to move it up, but for now at least that's complete. So at this point the project is pretty much done. Now I know there was a lot of back and forth and you know there's a lot of kind of messiness going on in the end of this video but that's just the reality of coding with AI and I want to kind of summarize what we did. So we start at the beginning of the video by really doing some deep research and understanding what it is that we want to build making key decisions like the tech stack the tools different APIs whatever that we're going to be using. Then we created a bit of context inside of cursor. So we kind of generally told it what our goal was, what we were attempting to build and then made those decisions and had the model give us kind of a little bit of an interview. After that, we started setting up cursor to install any agent skills, MCP servers, and different rules or tooling so that it knows what to do for the majority of the project. So right now all of this code is actually just available on GitHub because of the fact that we had the GitHub MCP server because we added the image kit agent skills and the MCP server as well. We didn't have to directly go to the docs and copy and paste stuff or read a bunch of information. We could just have the model do that itself because we said we want to use these tools and then we provided it to the model. From there we just started iteratively prompting again. We start by getting something working. There's a few bugs. We go back and forth and then we fix it. And now we have this finished application. Of course, there's more to do and to fine-tune, but generally speaking, we got this working with probably five, six, maybe 10 prompts if you don't count the ones that I cut out of the video, uh, because the tooling was there in the first place. Now, had we not added the tools, had we not added the MCP server, had we not spent the time to plan this, of course, it would have taken significantly longer and we may not have even gotten a completed project. So, that said, guys, that's going to wrap up this video. I hope you found it useful. If you did, make sure to leave a like, subscribe to the channel, and I will see you in the next one.