Learn Drone Programming with Python – Tutorial

freeCodeCamp.org · Beginner ·👁️ Computer Vision ·4mo ago

Key Takeaways

Learn drone programming using Python and the Pyimverse simulator, covering 3D movement and computer vision across five practical missions.

Full Transcript

Learn drone programming using a Python-based simulator. This comprehensive course will teach you how to use Pyimverse, which is a highfidelity simulator where you can master AI drone programming using Python without the risk of expensive hardware crashes. You will progress from the fundamentals of 3D movement to deploying advanced computer vision across five practical missions, including gesture control and autonomous line following. Mortazza created this course. >> In many parts of the world, 2026 did not begin with fireworks. It began with thousands of synchronized drones lighting up the night sky. But drone shows are just the surface. Because these machines are already reshaping the real world. Agricultural drones spraying entire fields in minutes. Delivery drones flying across city skylines. Firefighting drones battling flames where humans can't reach. Police drones monitoring cities from above. Aerial advertisement screens floating in the sky. Even cars launching embedded drones directly from their roofs. This is not a demo. This is happening now. And this is only the beginning. YouTubers like Mark Robber and MKBHD have gathered tens of millions of views showcasing how these machines work. Universities are launching drone programs and companies are hiring drone developers. After the AI revolution in software, the next revolution is AI in the physical world. Drones and robotics. Machines that don't just calculate, they move, see, and act. And the people who would lead this revolution are not just pilots. They are programmers. But learning with real hardware is expensive. Developer kits cost between $500 to $2,000. Batteries last up to 10 to 15 minutes. One crash can mean costly repairs and most consumer drones don't even allow full custom control. Every hardware limitation pulls you away from what really matters. Writing intelligent code. So we take a smarter path. Just like real airplane pilots train in a flight simulator before flying a real aircraft. You will begin in simulation. We will start inside Pyim, a Python drone simulator and focus purely on programming. You will start by mastering the fundamentals. How drones move, how Python controls them, and how commands translate into real flight. Then you will move beyond the basics into real world applications using Pyim built-in mission environments designed to simulate practical scenarios from industry and research. In this course, you will complete five core missions. Garage navigation. Here you will master precision movement. Image capture using the drone's camera. hand gesture control to connect vision with motion, body following game where you will build intelligent tracking behavior and line following to program full autonomy. By the end of this course, you won't just know how drones fly, you will know how to program them. So, let's go ahead and get started. The term drone originally refers to a continuous low humming sound, but today it represents advanced unmanned aerial vehicles used across multiple industries. Over the past few years, drones have evolved significantly as technology has improved and components like motors, batteries, and GPS systems have become more affordable and efficient. Learning to program a drone allows you to automate tasks, customize flight patterns, and build mission-based operations. This skill is becoming increasingly valuable as industries move towards smarter and more automated systems. Drones come in different configurations. A quadcopter has four propellers. A hexacopter has six. An ocopter has eight. Each design offers different levels of stability and lifting capacity depending on its purpose. Have a look at the drone components. A frame that holds all components together. Motors that generate lift for flight. Propellers that create thrust by spinning. An ESC electronic speed controller that controls motor speed. Power distribution board that distributes battery power. Flight controller that is the brain of the drone. Battery that supplies electrical power. Receiver or Bluetooth that receives control signals. Camera that captures photos and videos. Video transmitter that sends video wirelessly. Sensors that measure movement and position. A barometer that measures altitude. a GPS that tracks drone location and an IMU that measures angle and acceleration. So the first thing we will need is PIMverse. So we will head to pyimburse.com. And what exactly is piverse? It is a simulator that has a lots uh that has lots of missions where you can program your drone to execute those missions. So for example here you can see in four lines of code you can basically run the simulator you can connect to it and fly the drone. So it says spice inverse import drone you import the drone class you initiate the object and then you connect to it and then you take off. So if we click on this so that's how simple it is with just four lines of code you can start running and uh that's the idea you can download connect to it control and you can optimize your code without going into the real hardware now uh these are some of the examples now some of them are free and some of them will be paid after the Kickstarter campaign so there is a Kickstarter campaign for this project. If we go down, uh yeah, here. So, back us on Kickstarter and you will get lifetime access for all the pro uh features and all the pro scenes for life, which means any new updates that come along, any new drone uh scenarios or uh missions come along, it will be free. You only have to pay once. So, that is exclusive for Kickstarter. You can check it here. And it is about to launch. So there are already 336 followers as of now who are following this project and waiting for the Kickstarter. Uh and here are some of the examples of realworld applications where uh drones are being used right now. So here for delivery, here for firefighting, for agriculture, here are for drone shows. You can even use uh uh you can even create drone shows in Pyimburse. I believe there is an example here somewhere. This is carjs. Yeah, there you go. Drone show. So here you can control up to 1,000 drones. So it depends on your PC as well how much it can handle. Uh and yeah, so then there are rescue missions. So all of these scenarios are being created as we go along. So new scenarios will be uh added. Anyways, so moving on, we will simply click on download. So here we have the Pi Simburse installer. We will double click it and it will tell you that u it's an unknown publisher. You can click on run anyway. Okay, we will create the desktop icon and we will hit install. And there you go. So now we can simply hit on finish. So now we will open this up. And first it will check if there are any updates and if there are it will say uh that you need to download. Once you have downloaded you can press start. Okay. So now we can basically log in. So first of all you'll go to sign up and you will sign up here. Once that is done we will go ahead and sign in. So there we have it. So this is the main interface, the main menu over here. And we have all these different missions. And here you can see we have some pro ones as well. And then there are these free ones. Now uh if you haven't used this before, then there will be some of these that you need to download if you want to use them. So by default, there will be, I believe, three or four scenarios or missions that are available. And then if you want to use more you can simply hit download and this will download quickly and then there you go and you can start. So that's the basic idea. Now what we will do is we will uh first of all go ahead and do all the installations before we continue over here. But you can click on start and this will give you the sample scene. Here we have garage. You have different levels. Uh you also have your profile. Uh if you click on this it will show you your profile and then uh you have all your progress here as well. If you go back and there you go. So let's go ahead and uh check out the other installations. So after installing Pymverse we have to install Python. So Python is the programming language we'll be using. So all you have to go uh all you have to do is you have to go to downloads and then you can see view full uh list of downloads. Now the thing is here you can see we are uh in 2026. So you can see some of them are already expired. So I do not recommend using those. But here uh a safe one would be uh either 13 or 14. So uh I would say we can go for 13. And here you can see it is still uh the end of 29 or the mid of 29. So that's pretty good. So we can go ahead and download our uh 13. You can select the specific versions here. So for example, we are going to select which one is 13. So 31312. Let's select this. There seems to be an issue with the CSS file probably. Uh anyways, but you can find here Windows installer 64-bit and you can simply download and install this. So that's pretty basic. And then we also need PyCharm. Now here's the thing. If you want to use an AI IDE uh like cursor, you can go ahead and do that completely. But what I recommend is that for beginners, creating a virtual environment, making sure all the files are installed properly. It's a little bit difficult. So that's why we are taking the easy approach. So we will download PyCharm for our environment and running the code. And for anything related to AI coding, we will be using cursor. Now, if you don't have the pro subscription of cursor or your limit is exceeded, don't worry. You can still use chat gpt because we don't have very complicated code. Uh what we have to do is we have to do minor changes in our code base. So, we don't have a lot of files. So, if you have one single file and you have to work around it, then it's not a big deal to use uh any uh model that is not agentic. So but if you have aentic AI then it's better. So what we will do is we will go to PyCharm. Here you will click on download. So here what they have done now is earlier we had uh the free version separate and the pro version separate. But now there's only one version. So free forever plus 1 month of pro included. So you can simply download this and install. I would recommend to install Python first and then install PyCharm so that PyCharm can detect your Python. And then the last thing we will do is we will download a cursor. And I will be using cursor for this project but again you can use uh any AI software like DeepS chat GPT or Gemini for all your reasoning or coding and or even claude if you want. So now we will go ahead and create our project in PyCharm. So we'll write here pi simverse course and over here we have to select the python version. So as you can see I have lots of different versions of python uh python 3.9 3.7 314 and 3.13 3.10 and you have other versions as well you can download but as I mentioned we are going to use uh 13. So that is what we are going to select and based on this we are going to create our virtual environment. So there you go our course project has been created. Python verse course and env is basically our uh virtual environment where we will install everything. So to install everything it's very simple. Uh what we have to do is we have to go to file and we will go to settings and then here in the project we are going to click on python interpreter and then we can add. So here we are going to search for pyth and we will click on this and install package. Now the other way to do this is very simple as well. You go to terminal and in the terminal make sure you select this and then go to command prompt. Don't just write here because this is not our local directory uh our virtual environment. So make sure you are in virtual environment by clicking on command prompt and then you can see here we are in our virtual environment and then here you can write pip install pi simur. So and that will install the same thing. So hopefully that has already been installed. So we are going to write here right click and create new python file and we will write first flight. So in our first flight uh we are going to test whether our pyburverse in is installed or not. So we are going to write from pi inverse import drone. So if that is showing you automatically it means you are good to go. Piimverse is installed and you are connected. So if I run this now and I don't get any errors. There you go. I didn't get any errors. It means everything is running well and we are good to go. Okay. So uh now what we can do is we can run our drone. We will simply go and we are going to uh create our drone object. So we are going to write drone is equals to drone and then we are going to connect to this drone. So drone dot connect and then drone dot uh or let's let's just keep it there. drone connect. So these are the three lines you need to actually connect. So what I like to do is I like to move this on one side. So I will push it till the end and I will drop it. And the other one I like to open up uh pi inverse. So here I will do the same thing and I will drop it here. There you go. And then I can simply close the project. And this will give me a very clean interface where on the left hand side I have all the uh what do you call missions and pyimburse and on the right hand side I am coding. So I can keep going back and forth. And this is designed by the way to accommodate this half feature halfsplit feature. So you can run any of these missions in half screen. So uh we'll go to free play and then we will hit start. So here what we will do is we will right click and we are going to run our first flight and fingers crossed and response connected. So this means our drone was connected successfully. So there was a communication in between. Now we didn't tell it to fly but it worked right. So that's the idea. Uh let me open the sound of the desktop as well so you can hear the propellers as well. So here we can now write drone dot takeoff and then we can write uh or let's let's just run that or let's write here we will import time import time and then we are going to write time dot sleep and we will give it like 3 seconds before the simulation shuts off. So we are going to run our first simulation. So fingers crossed and voila. There you go. So we have our first drone flight. Excellent. So after 3 seconds as you can see it turned off and it landed by itself. Now uh you can also write drone dot uh land and then you can give it like a few seconds to calm down right cool off. So this is your first flight how you can basically take off and how you can land. And the good thing is you don't have to connect manually anything. Everything is done internally. So all you have to do is you have to run it and then that's pretty much it. There you go. So that's the basic idea of how you can connect to your drone and how you can operate your first flight. So if you have done this and if it was connected and the drone actually flew, that is an achievement by itself. So give yourself a round of applause and comment below that first flight done. Okay. So now we'll move on to the next part. So now that we have understood how to take off and connect to our drone, let's go ahead and understand the basic movements. So you have two options of movements. The one way is to give a specific distance. So you will say move forward for example 10 cm. The other way is to give speed. So you are continuously giving speed rather than saying move this much. So both of them have their own applications and their own usage. So we will learn both of them and understand how they work. So let's look at the first example where we will do uh up down movement left right and rotation and all of that. So now we will say that all of this part is the initialization. the first initial part and all of this is the last part. We can remove the sleep because we were just simulating a time uh before landing. So we can remove that and this is where our code will lie. So whatever code we write we will write here. So the first thing we are going to understand is how to move forwards and backwards. So uh or so the first thing we are going to learn is how to move up and down. So once it takes off then we are going to tell it to go down and then we will tell it to go up. So we will say drone dot move back uh move down and then over here we are going to give in the distance in cm. So for example uh we will say move down let's say 20 cm. So by default when we take off it will go up 1 m which is 100 cm. So we are telling it to go down 20 cm and then we will tell it to move up. We are going to give a delay here. Time dot sleep. And we are going to give a delay of let's say 1 second or let's say 2 seconds. And then we are going to say drone dot move uh moved up. And then we are going to tell it to move up let's say 30 cm. And then we are going to sleep again for two seconds and then it will land. So let's go ahead and try that out. So it went down, now it went up and now it will land. So the exact behavior we were expecting. It went uh down first then up and then it landed. So now that uh this is done let's move on to left and right. So instead of move down we will say move uh move underscore left 20 and then we will say move underscore right let's say 30. So let's try that out. There you go. It goes left and then it goes right and then it should land. Right. Similarly, we can do forwards and backwards. So, we will say forward 20 and then we will say backwards uh let's say 30. So let's instead of 20 let's put 80 and here we will put 50. So let's run that. moving forward going backwards and then lens. Okay. So next we can uh understand how exactly can we rotate. So let's comment all of this and we will simply say so here we can say drone do.rotates let's say 5°. So it it depends on you. These angles are in degrees. So let's say we will tell it to rotate 5°. You see there was a slight movement there and now it's angled a little bit uh to the right side. Anyways, and if you put a minus it will go to the negative side. Now you can also set the speed of the drone. So you can write here drone dot set speed and we can let's say by default it's 20 m uh 20 cm/s. So let's say we'll put 50 cm/s and then we can tell it to move forward. Let's say 150. So that's 150 cm. And let's run that. There you go. So now you can see the speed has increased. So now we will look into the second method of how we can control the drone. And for that I would recommend we go ahead and create a new Python file. And we write here RC control. So this is basically the continuous control we were talking about. So what we will do is we'll copy all of this and we will go to RC control and we are going to paste it. Here we can pretty much remove the center part and here we will write while true because this is continuous. So it needs a loop that needs to keep updating otherwise it will send one value and that's it. So we need to keep giving the value again and again. So we will say drawn dot uh RC send RC control and then over here we have the left right value the forward backwards up down and y. So what we can do is we can write here left right equals let's say zero and then forward backward equals 0 and then we will write up down equals 0 and then yaw which is the rotation equals z. So and then we can give in these values over here. So I'm writing the whole thing so that it's easier for beginners to understand. Otherwise you can write for example LR or FB and UD up and down things like that. So to make it a little bit easier. So there you go. So now if you wanted to change we can simply change from here. So let's say back and forward uh we can give in the value of let's say 50. So this is our speed 50 and this is 50 cm/s. So now if we run this it will take off and then with the speed of 50 it will keep going forward and as you can see it keeps going forward. There you go. That's forward backwards is 50. And there you go. And now it got stuck. So let's let's do one thing and we will add a yaw as well. So that will allow the rotation as well. So what we will do is what I like to do is I like to keep these files separate. So these are my templates and when I ask AI to do something I ask it to use these templates to write the code, right? So it becomes very easy. So for example, if I was doing object detection, I would keep yolo very basic as one file and then I will give reference to AI. Okay, go ahead and use that. Right? So that's the basic idea. So now we are going to go to our exciting part which is the missions. So what exactly are missions? These are different scenarios, real world scenarios that are created so that you can test out and see how you would code your drone. So that's the basic idea. So here we have the free ones. You can see we have six of them right now. And then there are some pro versions as well. So uh we have jogging, obstacle avoiding, traffic surveillance and a lot more in the build that will be released soon. So again you can click on get pro and uh that will take us uh to the Kickstarter campaign where you can back this project and you can get lifetime access to all of the new uh what do you call scenes that will be coming later on as well. Now for the ones that we will get started with the first one is the karate scene. So if we go to that here we will see we have three levels. So what we will be doing is we will be trying out one level and then it will be up to you to complete level two and three. So whatever levels there are more. So uh and of course you can click on the scorecard here the leaderboard and you can see uh right now who is leading and where exactly you are standing for this exact uh what do you call mission right so you can keep improving and you can keep getting new stars okay so we will click on start and this will give us the mission briefing that what exactly you do you need to do so we can click on let's fly and The basic idea is very simple that you have to fly and you have to reach that target area. Right? So what I will do is I will turn off the volume so that it doesn't interfere with my audio. So and then I will go to project and here we are going to create uh should we create a directory um or we can create just a new Python file. Right? So we can say mission one garage. There you go. So here we are going to write in all of our code. So what we will do is uh we will take hints from our previous code. We will copy all of that and we will paste it here and we will uh minimize this. So over here we are going to remove the stuff that we don't need. So for example this is the initial initialization and this is the ending. So in between everything we can remove right. So this is our let's say template code that we will use uh over and over again. So over here what we need to do is we need to take off right and then we need to move to this target. So we have go up, go forward and let's say go right or you can go right first and then you can go forward. So let's try forward. So drone dot move forward not backwards forward. And right now we don't have the exact value of where it will uh how far the target is. Right? So, and this is on purpose so that you can experiment and try to land in the correct position rather than just saying okay go 2 m or 3 m or something like that. So that's why it's like this. So let's say we will start with 200. So 200 means 2 2 m 200 cm or 2 m. So we are going to right click and run this. And it says connected. And there you go. Our drone is in the sky. And it is moving forward. That's good. And you can see here in the mini map as well, uh the drone is moving, right? So it gives us the top view. And from that top view, we can tell how far did it land, right? And we can add a delay here. Right now it's 1 second. So now you can see we are a little bit off. So we can maybe say 200 uh let's say 30 or maybe 50 right we can write that and then we need to move towards the right so drone dot move right and how much rights again let's start with 200 and we will see so now we can hit run. And it's flying. It's going forward. And hopefully this time around it will be closer to our target. And there is a shadow that is telling us that okay, we are close. Not that far away hopefully. And oh, we might actually get Oh, okay. Okay, we got there uh in 32 seconds and we got a star. Uh but it could be better. We could write let's say 230. Let's see if that will work. Okay. So now it's entering and it's almost at the center. Maybe a little bit more and then you will get the perfect one. But again uh we are we have completed the task and then we have gained one star. So this was garage level one. It is the first uh mission that you should always do because it's very very simple. All you have to do is find the values of how far they are. And then uh similarly we have level two and level three. So uh they get a little bit harder as you go along. So here we have level two. So now we have three targets. So here you can see these are the targets. So you have to go to each one of them one by one. Right? So again this is manual that you are doing. Later on, you will also learn how to do this automatically, right? That's the best part where you actually find the targets automatically and it works, right? So, now that we have completed level one of mission one, I will give you the task of completing level two and level three and try to rank and get better results uh and uh get a better score in the leaderboard. Now, we will go on to the second mission. So, let's go ahead and do that. So our second mission is image capture. So if we go here, press start, we have two levels of image capture. So we are going to click on start and there you go. So how does it work? Uh you have to move around and go to that target and then you have to take a snapshot. Uh basically you're taking a picture. So how exactly do you have to do this? Uh now you could do this automatically by saying okay move forward this much move right and left but this is not designed to be like that it's designed so that you use your own keyboard to actually run this. So how exactly can we do that? Let's have a look of how we can use this with our keyboard to run our drone. So we are going to right click and create a new file and we will write here mission 2 image capture. Now image capture is more about keyboard control. So you can also say that it is keyboard control rather than image capture uh as you wish. Right? So we are going to learn how to use the camera of the drone and we are also going to use uh we are also going to learn how to use it with keyboard. Right? So the first thing that I like to do again because this is something that we are doing as a template. Right? So maybe in other programs we might need to use keyboard. So if we are doing keyboard control we should put it in a separate file right. So I'm going to create a new directory and I'm going to write here templates right so that we can later on use these. So first flight is basically your template as well how exactly the drone moves. The RC control again is a template and then again we will create a new template and we will write here keyboard control. Okay. So now to run this what we will do is and uh we will also need image capture right uh we can create here a python file here we are going to write drone video stream. So we need to understand how exactly can we get the drone footage. We are doing two things here and then once we have these templates we can combine them together to create our uh project or mission. Right? So the reason we are doing these templates so later on if you have other projects you can just reuse them. Okay. So that being said uh if we look at the stream first let's go ahead and do that. So we are going to write the same thing that we have done before. So we will go to first flight and we are going to copy this. So here we are going to write this and the center part as you know it's generic. So we are going to remove that. And now that we have this uh the drone is going to connect it's going to take off. But after that we need a continuous footage right? So we will write here while true or you can give a command that okay if I press this button it should land or if that is not true then do this. So you can do stuff like this but for now we are going to keep it straightforward. Now before we take off we are also going to write drone dot stream on. And to make sure the stream on works properly, we are also going to give it a delay time do sleep and we'll give it a 1second delay. So sometimes you have to do this, sometimes you don't have to do this. So it's better to just add that so that the stream on works properly. So now the question is how exactly do you get your frame? And it's very similar to how you get your frame in Open CV. So all you have to do is you have to write frame and then is success equals drone dot get frame. That's it. And that will give you your frame. Pretty easy, right? And then we can use OpenCV to actually show the frame. So in order to use OpenCV, we have to install it. So we will go to our terminal. Make sure it is env or uh you can go from here to settings as I have shown earlier. Here you can write pip install open cvv python. Okay. So now that this is installed we can import it here. import cv2 and then here we can simply write im show uh cv2 dot im show and then we are going to write our window name for example drone feed and then we will write here frame then we have to write cv2 dot weight key and we will give it a 1 millcond delay So now if we run this, let me turn the volume off and it goes up and there you go. So this is our drone feed that we are getting. Amazing, right? So that's how simple it is to get it up and running. So now that we have our stream done, again we are doing this manually. We are not using AI for this. so that we know the core principles of how this works. Once we understand the core principles, we can use these templates to give to our AI to make some changes or create some new files, right? So, uh the missions and all that. Okay. So now we can also use keyboard control. So we are going to ask our AI model to use keyboards and using the keyboard we should be able to move around uh with our drone. So we will tell it uh exactly that uh if I press this button the speed should increase. If I press this button the speed should decrease or move left or move right. But if you think about this we cannot say move the drone 10 cm to the left or 10 cm to the right because then whenever we press it will move it will stop then we press again it will move it will stop. It should be continuous. If you think about remotes they are continuous. So if I keep pressing it, it should keep going and then it should stop when I release. So how does that work? It works by sending in the speed rather than the value of distance. So how do we send speed? We send using RC control. Right? So here is our RC control. So we already have the template of RC control. So we can give our AI and we can tell it to control the drone using this RC control method and the keyboard. Right? So that's how uh simple our things become. So now we will open our AI model. Again you can go to chat GPT and give it the template file. So here we have the template file. You can simply copy this and give it to CH because these are not very long scripts. You can easily do this in charge as well. But for now we are using cursor. So I'm going to drag it in. And here I'm going to write using using this template code of how to control the drone speeds speeds. create a new script inside keyboard control to control the drone using keyboard keys. I think that's pretty good, right? Straightforward to the point. So, let's wait for it to complete. Okay, so now it has completed. And if we go to our keyboard control, keyboard control for the drone uses the same RC channels as RC control, left, right controls, Windows and short terminal window has focus and all of that. So it's using system and Windows 32. Okay. Um uh don't make it Windows specific because we might use it in Mac as well. First suggest the alternative before writing any code. So sometimes it gives a very specific answer. So and it starts working on stuff that we don't really like. So maybe it will give us several options and we can select from those. So one of the options is actually to use Open CV. We can use that because it's there. And then we have third party libraries like this one keyboard, right? Um we could use those. No extra packages. um by input read jar one API usage pip install okay so I think I have used this before keyboard and I think that's a good library so let's just tell it to pick option three So let's look at the code what it did before. Yeah, seems fine. Like you have a few variables that you keep changing like the speed and yaw and those you send at the end over here by capping them, right? So it it pretty much takes care of u different scenarios. So unless you have a very specific one, you need to give it the exact same thing, then you can refer to that. So it says install and run pip install keyboard Python uh templates and keyboard control. So but in keyboard control. Okay. Yeah, we have the code. I thought it was not there. Okay. So here we have imports, keyboard and all of that. So now let's go to PyCharm. And here we are going to go to our terminal and we are going to write pip install keyboard keyboard. And while that is installing let's go to keyboard control and have a look at it. Um try import keyboard. Ah we don't need try. you know we have installed it RC value range. Okay. So minimum RC maximum RC speed. Okay. Um seems fine to me. Let's let's run it and see if it will work. There you go. The drone is in the sky and we have to use what is it? So, a ws a d. So, it's not left right. Oh, w. Oh, there you go. It moves forward and it's typing W here as well. Okay, that's nice. We can remove that. We can click over here. And there you go. So, W. And I'm pressing S to go back. Whoa, whoa, whoa, whoa, whoa. That is way too much speed or the rotation. Your left and your right is way too much. We can fix that. And then left. Okay, up and down is also doing the same thing. Forward and backwards. Okay, because it says here up and down is the same. And then we do left and right with using the uh left and right keys. These. So again, it's up to you if you want to change or add stuff to this. But how can we go up? Q and E. So if we press Q, there you go. It goes up. If you press E, it goes down. Yeah, that's good. Now my concern is the rotation. So let's check the rotation speed. Do we have something else for rotation? The yaw equals minus speed. Oh, okay. So what we need to tell it that uh it needs to change keep the movement or lateral movement speed or keep the rotation speed in a separate variable. Right? so that we can adjust that according to our needs rather than using the same lateral speed as um the rotation speed. Okay, so there you go. It's done. And if you look at it here, rotation speed is 50. So now we can make it like five maybe, right? So let's run that. Connected. And there you go. Our simulator is up and running. And if we press Okay, there you go. Now, now it's much better, right? So, we can move around and we can go up. It's a little bit tough to go up and down. So we can say that uh the up and down should be from the arrow up and down keys instead. So rather than doing the same forward and backwards using the arrow keys, we can do up and down with that. Right? So that will be a little bit easier I believe. Okay, that's done. Let's go back and let's see what it did. Says up and down are Yeah, now it's up and down, right? So, let's run that. And our simulator is up and running. We can move forward, left, and right. And we can go up. There you go. Can go down. Let's try to complete the mission. And there you go. So now it says press zed to take a snapshot. So there you go. We press zed and now it's task accomplished. Right? But here's the thing. It did not actually take a snapshot, right? So we did not actually save the file. So in order to do that what we will do is we will create our image capture script and we will be using the keyboard control and the RC control uh not the RC control sorry the drone video stream. So now we will tell the AI to do that for us because it's a combination right. So now we will go and here it already knows all about this. So we can ask this AI agent to do this for us. Okay. So now using the uh where is it? Drone video screen uh video stream script and the keyboard control script. Now using the dashcript and dashcript create the following uh program in the file. Okay. Mission to image capture file. Okay. What exactly uh is the functionality? Now uh the user should be able to move the drone using the keyboard. Okay. When the meanwhile, the drone footage should also be shown. Right? When the user presses the Z button, it should capture a screenshot and save save it in the in a new directory. Okay. So, let's see how that works out. Okay. So now that this is completed, what I like to do, I always like to read the summary of what exactly did it do and look at the code a little bit so that I understand what exactly is happening. So it's saying here it combined the behavior video as drone video stream uh then each loop get frame. Okay. And then cv2 will show the image. That's good. Keyboard control same logic up down left right and send RC control each loop. Okay. when you press Z, the current frame is saved under a new directory, mission captures with timestamped file name. So this is actually something that is really good. So the AI actually understands that if I'm running this again and again, I will have to overwrite my images if I name them 1 2 3 4 or maybe capture 1 capture 2 something like that. So it's giving a timestamp so that the file names are unique, right? So and it's giving the example of that as well. The directory is created on the first run uh and no burst of key repeats. So there you go. So this is amazing right? So it is taking care of the different scenarios and one of the most common ones is that when you press a key it might be pressed twice or thrice. It might be triggered that way. Right? So it is taking care of that by itself. I remember once I was uh writing a program and I had to spend it was with Arduino or something and it was with a physical device with a robot and whenever I press the key it would just do it two three times right and that is an actual problem that you face. So thanks to the AI model it already knows that. Okay. So uh let's look at the code. So here we have the mission captured. Uh here we have speed rotation speed is separate captures in mission captures 2. This is the folder name. And then we have both of these. Uh this is the main uh function. And then uh we have a condition zed was pressed and that's false. And there you go. So that's pretty much the idea. So let's run. Okay, we are not running this. We need to rightclick and run mission image capture. So there you go. It's connected and you can see now it's running. And what we will do, there you go. We have our drone feed. We can move fast. Oh. Oh. Oh. Let's go, go, go, go, go, go, go. We'll move, move, move. Right. Right. And I think we need to move a little bit to the center. And we'll move forward a little bit. And there you go. And we'll press Z. And it lands. There you go. 25 seconds. We got three stars. Amazing. And there is our mission captures. And if we click on it. Ooh, nothing is there. If we stop it, do we get something there? Oh, we didn't get anything. That's a bummer. We actually pressed said and it did work. Uh, let me check if it's the other issue. It may be because we might have to keep our drone capture open, but that that should not be the case. If I press Z now, does it come here? No. Oh, yeah, it does. Yeah. See, that's the issue. So, you need to keep this open and you need to click on this. This is one of those things with OpenCV, right? So, it's not the simulator. So, if you move around and then uh you go up up up up up and then you move uh go down go down. I'm very bad at this. Okay. Uh let's rotate rotate go down go down go down forward. Forward. There you go. And then you press it. Okay. So once you have to click there. Once you have to click there. So that's not good. So, we will tell our AI to fix this. We don't want this. Okay. So, the Z the Z capture only works when the when the drone stream window is active. That should not be the case, right? Because doesn't make sense. one time I have to click one window then I have to click the other window and then it's a mess right so let's tell it to fix it okay so now it has completed and it is saying that single CV2 weight key its return value is stored in key and used for both Z and quit when drone feed window has focus zed from either source captures okay okay so it's using from opencv as well and it's using for from keyboard as well. So hopefully both of them running together should give us this. So if we reset this and if we run this now, oh my bad. So there is our drone. So we will not keep this in focus. We will click on this the simulator and then we will move it. So, let's move up and forward at the same time. And we will rotate. Rotate. Come on. Faster. Faster. Oh, come on. We need to move in. And capture. There you go. So, it captured. And did it give an image here? There you go. That's our new image. Perfect. AI to the rescue. So, that's the basic idea of uh this. Now what we can do is we can go back and here you have level two as well which has multiple targets. So you have to go one place and then you have to go inside the rooms and then you have to capture multiple images. So that will be a very fun one to do. So make sure you complete this task. Right. So now we are going to move to the next one. So now we are going to move on to mission number three which is hand gesture. So if we click on that there you go. So we have level one and level two. So what exactly is the idea? So the idea is that you move left and right uh with your hand and that moves the drone itself. So we have to touch this box and then we have to touch this box. So that's the basic idea. So how exactly can we do this? So first of all what we will do is we will create a template. The template to get our hand gestures right. So we need to detect our hand and we need to see how does it move. So we will be using media pipe for this task because it is open source. It's free to use uh and it's available by Google. So if you go down these are all the different models that we have. Image classification segmentation hand landmark detection. And this is the one that we are going for gesture recognition and face detection and all that. So if we go to hand landmark detection and this takes us to uh this notebook and here we have all these uh different cells that we can run. So in most cases the AI pretty much knows how to do this or it will scan and get some code. uh if you want to give directly you can also click on this and uh give it the code from this Google collab uh as a sample. This is I believe only for uh image and not video and uh not any live stream but you can still give it an example of how this will work. So we can go back and here in the templates we can create a new file and we can call it and gesture media pipe dotpy. So this is our template. So what we will do is we just want to run hand gesture. So here we are going to write uh inside this file create the hand uh detection code uh hand detection script. Uh use media pipe pipe to do this. Below is sample code. The and gesture should be detected in live uh open CV camera feed not images. Okay. So we need from a live webcam rather than images. So oh I forgot to give in the code. Uh there you go. And then we run it. And while this is running, we can go back and over here we will go to our terminal. And in the terminal, we are going to write pip install media pipe. So it can install that. Okay. So now it has completed and it's saying what I have done. imports media pipe task open CV and vision drawing helpers model path uh hand landmark created with video running mode and there's a camera loop with video capture zero and then it is drawing with utilities uh okay and then you can press Q to close and you need a model file that is required we need to download it so it's saying download it from here or from the script folder You can invoke this. So let's copy that. Click on the top here. It will open up our PowerShell and we can simply paste it and run. There you go. So now we have our handland marker. What we can do is we can create a new folder and we can write here not inside templates. It should be outside. We can write here models and we can move our landmark into models. And if we go back to PyCharm over here, we will see now we have models and then hand landmark. So wherever they have given the reference uh let's open that first. Where is our template for hand gesture? Now wherever they have given reference we will give in this. So we will write here models slash and landmark and there you go. So let's run this. So it is using video capture zero. This number should be at the top somewhere. So here we can write camera camera index equals zero and then we can copy this and we can paste it here. So yeah let's run this and see if it will work. I have a few cameras connected, so I'm not sure. So, it's saying model not found. It's saying download it from this save as hand landmark in the same folder as the script. Okay. So because we will be using it um in different what do you call projects probably so we don't want to keep it in the same script uh in the same folder so let's tell our AI model so it is going into scripts directory and then it's doing that okay that's why so if we remove this and path join we don't need that let's remove that. Let's run it. And again, it's saying the same thing. So, it is I think you need we need to go back and then we need to check in the models folder. And there you go. So, now it's going it's running. So, here is my image webcam. If I There you go. So, there is our webcam feed live and it is detecting both our hands. And there you go. So pretty amazing what AI can do rather quickly, right? So now that we have our template for hand gesture, now we need to add it, merge it with our um drone code, right? So what we will do is we will uh right click and we will create a new Python file and we will call it mission 3 uh and gesture. So hand gesture control is our mission. And now we'll go back to cursor. So here we will give this hand gesture create this new script so that we move the drone or should we talk about the drone? No, let's not talk about the drone. create this new script so that uh it uses uses the and it uses the let's give it in the template hand gesture template to find one hand and then when the hand is moved moved to the left. It prints prints out left and right accordingly. So you can split the screen into two parts. parts left and right. Also add a dead zone where no left or right is shown. Right? So I want to go step by step. Once we have this done, then we can move on to the next step which is controlling the drone with it. Right? Okay. Let's go ahead and check this. Uh it has completed templates hand gesture media pipe get model path detector hands number is two that should be one right uh yeah it's given it as one uh okay then mission three imports all of these one hand then there are different zones left dead zone and right and there you go so let's run that mission three. Let's see what it does. There you go. So, now it says left. Now it says right, dead zone, left and right. There you go. So, that's pretty cool. So, the text for dead zone is not really visible, but again, it's dead, right? Uh but anyhow, so that's that's pretty good, right? So now we can tell it to add more features. So using this um okay, now we need to move the drone left and right uh based on this hand gesture. So in order to tell it how to move left and right, we are going to give in our template for uh RC control. So we are going to use RC control to move left and right. This is the example code for drone movements. There you go. So the reason we have given this as separate variables is so that the AI can understand okay this is left and right that's why we didn't write LR or FB something like that because it's a template for the AI to understand how exactly PI symbols works at the back end. Okay. So now it has completed and it is saying that these are the summaries. We have imported and we have a speed. Uh then we have startup drone.connect zone to left zone to right. Every frame we are sending the left and right which is the first one. There you go. That's correct. And on exit it will land the drone. Okay. So that's pretty good. And it is using the speed of 50. So uh and that's your uh variable as well for the speed. Okay, let's go back and let's see if that will work. We will open that up and finally we have our mission ready. So let's see if that will work. So there you go. Now it's working and we can go to the right and the left and there you go. Our mission is complete. So we are basically controlling using our hand we are controlling our drone. So now that we have our hand gesture working what we can do is we can actually play a game with it. So that is our level two game and there is scoring on this as well. So you have to move around and you have to uh basically hit all of these different obstacles that are coming your way. So let's try that out. There you go. So now, there you go. Yeah, now it's easier to imagine. So, it depends on you. What do you prefer? I'm very bad at this game. So, there you go. Game over. And I did how many hits? Eight hits. I believe you can do better. So yeah, it's up to you which direction do you prefer? Left is left, right is right or whatever you prefer, you can basically add it there. So now we are going to move on to our fourth mission which is body follower. So what exactly is body follower? We are going to use the same principles of hand gestures but instead of controlling the drone with hands, we are going to control with our complete body. So if you remember that game uh called flappy bird uh where you had to tap to move up and down. Similarly, we have a game over here where you can move the drone up and down and then you will have obstacles that you have to avoid. So the first thing we will do is we will learn how to go up and down and then again we can go ahead and play our game and we can see how many we cross. So in level one we are going to start this and you can see you can see we have to go up and hit right. So that's the basic idea of uh body following. So what we will do is we will go back to our cursor and here uh we are going to create a new templates and here hand gesture media pipe we are going to create a new file and we are going to call it body follower media pipe dopy. Now this is very easy for us. Why? Because we can create a new agent and we can simply give it hand gesture media pipe and we can say similar to this create the body follower media pipe. That's it literally. So hopefully that will work around. So the task is complete and we can look at it. summary of what was added. Doc string to understand what exactly it does. Imports paths. Uh now see it uses the models path that we added earlier. Then the detector itself, the drawing and the main loop. So that's uh pretty good. So what I've done is I have a picture of a person standing. Hopefully you can see here. And what I will do is I will turn on the webcam and then I will show the person standing because I don't want to stand and jump and all that. I will try it but for testing at least we can do this right. So uh we will go back to our code and here we are going to go to our templates and we are going to go to body follower media pipe. Let's run that. Did it download? Uh it's saying no model found. So we can simply download and just drag it in. Okay. So when I clicked on it, it gave me this. So it's not able to find the link. So let's just give this here. Uh file not found. Let's just say that. Okay. So here we have summary of what changed. The old URL was this is no longer valid. Now it provides three post landmark markers light, full and heavy. So default model is this uh and it is recommended that we use light. Okay, we can use light. Then you have full and heavy. So again it it's up to you uh what you want to use. And if you don't get good results with one of them, you can of course shift to the other one. So I'm trying to copy this lights. Okay. So now it's downloaded and what I can do is I can simply drag it into the models folder and I'll just make sure that this is neat and clean. There you go. Post landmark light. So if we go back now and we see in the body follower the model path are we using light? Yes, we are using light. So let's run that and hopefully we are going to get something good. Okay, it's saying start id connection start no tupil object has no attribute start. So let's give this error to cursor. Uh fix this error. Okay. So it is saying draw uh draw landmarks expect each connection to be an object with start and end. So well the syntax probably was the issue. Sometimes these libraries they keep changing the syntax and it becomes difficult. In fact uh if you go below the version of 10 in media pipe uh the code will be completely different. So just make sure you are using above uh 10 or below 10 and you know which one you are using. So that will be helpful. Anyways, so let's open that up. And there you go. you have your body and there it is focused and all good. So that's pretty good. Now that we have this all we have to do is we have to do the same thing where we will ask uh our AI to actually create the code for us. Right? So the jumping code. Okay. So we will close the templates and in we are going to create a new file and we are going to we are going to write mission 4 body_follower.py Python right so modify follower uh we are going to write that in this new file create create the following script. Uh we need to detect the jump of the person. suggest some methods to detect jump given we have the media media pipe post landmarks. So, first it's it's a good idea because you can have multiple ways of detecting a jump, right? It's not just going up one pixel. It's it's a complete motion of frames. So, we need to find a good way to do this. So, this is complete and it says that actually it wrote the code as well. So it gave us five methods and then it picked what method hip center vertical position method one in the dock string. Okay. So here in the dock string they are given hip center vertical position implemented below track midpoint of left hip and right hip body goes up decreases then lands history when y drops. So, it's taking the positions and taking the average. Well, well, it should work. That's not bad. Ankle height vers relative to hips. H, I don't think that will be very good. Vertical velocity of hip center. Oh, this will be good. Smooth hip center. Compute frame takeoff position. Counter jump of takeoff or on landing with debounce. Oh, we can add both. So I think this will be better. Nose or shoulder height. Same height as hip center but nose midpoint more sensitive to head bobbling. No. Knee ankle rise together when both left knee and right knee. Um yeah that could work but I think the best one would be number three. So it already implemented number one. So let's see if that will work. So if it works, we can keep that or we can use a new one. Okay. So right now one jump was detected by default. So let's see. Oh, jump three is detected now. Okay, four is detected. So the sensitivity is too much. Seven. Or let me let me try it myself. If I stand here and I jump. If I just move. Yeah, it's okay. I guess it's okay. It's not that great. So what we can do uh instead is we can say use meth uh implement implement method two instead. Was it two? Oh no my bad. It was method three. Method three instead. Hip vertical velocity. There you go. It has implemented summary of changes. Method three behavior is smoothing hip center Y for five frames average to reduce. Then it's taking the velocity. Uh it's counting the takeoff and the landing and then at least the bounce of 12 frames between counter jumps. Okay. A lot of variables though. Uh let's see if that will work. Uh do we have any parameters here? No. So nothing to tune. You should have something to tune, right? Oh, this is body follower. My bad. Need to go to mission 4. Mission 4. We have Yeah. So here is the left hip, right hip, and here are the parameters to check out and vary. So let's run that. And so we have one jump already. Two. Okay. So now we have four, five, six. So a little bit shake. No. Seven. Oh, that seems to work. Let's try it out. Let me go back. Yep, this is better. But I guess we need to make it a little bit harder to um what do you call detect? So we can simply tell it make it a little bit harder to detect the jump. So here are the values changed. So the smoothness has been from 5 to 7. Takeoff velocity it needs to be a stronger motion. Landing velocity needs to be clearer and debounce frame is a little bit longer to detect. So let's run this and see if it will work. So I think this is much better now. Now if you wanted to, you could tune it further to get better results, but uh that's the idea. So let's go ahead and try out our hits. So what we need to do now is we need to connect it to our drone. And uh once it connects, whenever we jump, it should go and hit this ceiling. Okay. So, let's go and do that. Uh whenever the jump is detected, move the move the drone up. Okay. So for this um for this we can should we give should we give direct or should we say I think we can give just move up or does it have to be a cont no it has to be a continuous loop my bad it has to be a continuous loop so we do have to give RC control because we are using a webcam so you cannot say go up this much because it will use a time delay in that uh whenever it's doing a distance movement. So uh that's not very useful for us. So we'll use that as the RC control. Uh whenever the jump is detected, move the drone up. There you go. Okay. So it says it has completed. Start drone connect. Take off on jump blah blah blah. it will give the speed of 50 and up down is 50 during post jump and then it will give a zero value. Okay, so let's try to run that and we have our simulator on one side and let's see if that that will work. I think it went up. Okay. So, here we have. There you go. It jumps. It's weird uh trying to detect both at the same time. And there you go. It goes up. Let me let me do it myself. Let's run that again. There we have it. So now we are jumping to actually uh complete our task. Now what we can do is we can go to our game and we can actually play that game. Okay. And it will wait for the connection to start to actually start playing the game. too late. I need uh I think we should have this as full screen then it will be a little bit easier. Right. So if I jump now. There you go. I'm jumping. Oh, I need to jump more. Oh, come on. Let me let me try it again. I will Can we make it smaller? No. Okay. But I will put it here. So, I need to jump. Jump. Jump. There you go. And now I need to jump again. Jump. That's a lot of exercise. And uh it will become faster and faster. So, you have to be careful. jump. Okay. Okay. Uh there you go. I was able to cross only two. Okay. So that was how you can actually uh play this game. It's a lot of fun to actually do that and it is quite tiring as well. So you have to do a lots of jumps. So later on there will be lots of more games and examples like this. So right now it's uh just uh two of these games and the other ones are mostly educational but uh in future there will be a lot more uh scenes and missions where you will be able to do lots of these different uh games and uh other stuff. So now we are going to go ahead and start with line following. So let's go ahead and click that. Here we have three levels. So what we will do is we will write the code for level one and you can use the same code for level two and three but depends on your code how good it is. So if you create a good enough code for level three that should work with level one right but not vice versa. If level one is good it's not compulsory that it will be able to achieve level two or level three. So um and at the end of the day this is all about time right? So what you have to do is you have to do this in the fastest time possible. Anyways, so we will do only for level one and then it's up to you to optimize for level two and three and get the best results. And in fact, we are also going to have a competition for this and we will create a completely new um path that will be just for the competition and uh you can participate in that and you can win prizes as well. Anyways, so uh whenever that is released, it will be shown in the uh what do you call the simulator by itself. Anyways, so if we click on start, uh this is our simulator and here it's showing us that this is how you have to fly around and move along. Okay. So we are going to do the very very basic of uh something like this. So what we will do is we are going to detect color and from that color we are going to detect whether we should go left or right. So this is not the best method to do this but it is one of the easiest methods and uh it will be very beginner friendly. So as always we are going to go to templates and we are going to create a new template and we will call this color detection because color detection is something we might use later on as well. So it's better to have a template for that so the AI can understand. So in order to use color detection we will use OpenCV. Now having and debugging and finding the right color it's a little bit more time consuming if you go directly with OpenCV. So what we will do is we will install CV zone. So we will write here pip install CV zone and that has a built-in wrapper for color detection. So we are going to use that instead. So here we will write from CV zone dot uh color module import color finder. So we will press control and click on the color finder. And if we go down at the end it will give us an example code. So here this is our example code. We will copy all of this and we will come back to our color detection and we are going to paste it here. And of course we need to import CV2 because again it's a wrapper for it. So here it says my colorinder equals colorinder. So this is where we are calling this object uh this class to create an object. Now track bar equals true. If you are turning on the track bars, it means you are debugging and you are trying to find the color. Okay? So if you want to find the color, make it true. If you don't want to, then just uh make it false and that should work, right? So that's the basic idea. And here you can see we have all these different parameters. And then uh here we are not importing CV zone. Okay, we need to import import CV zone. There you go. And then it will just stack the images and show us the results. So let's run that on video capture number zero. It will run on my webcam and then I will show you how exactly it works. There you go. This is my webcam. Oh, that's very long. Um, we are basically stacking all these together. So let's make it smaller. If we go to our stack, this is image stack. And over here we have a scale of one. So we will put 0.5. 0.5. And we will run that again. There you go. So now we have this. And if we come here, these are our track bars. So what we can do is for example, if I wanted to detect this red color, right? So what I will do is I will bring this down or red color is actually a little bit harder to detect from other colors. Let's pick another color. Let's say this green, right? So if I wanted to detect this green, I can move this around until it disappears. Then I can move this around until this disappears. And there you go. And then I will keep moving those until I get uh the correct values. And there you go. So now you can see this is being detected. Again it's a rough idea but you get it right. And once that is done you can stop it and this is where you will get the exact value to detect this pen. Right? So what I can do is I can copy this and next time around instead of giving these HS values we will just paste our new HS values. And here instead of true we will make it false. Now if we run this it will directly detect our pen. We don't have to do any debugging or anything. No track bars nothing. So it will directly detect our pen. There you go. Yeah. So that's how simple it is. So what we will do now uh we are going to tell our AI to use. We will go to our AI and we will say where is it drones color detection. So use the drone camera for color detection instead. Instead and we can give it the what where is it? uh drone video stream as an example of how that could be done, right? So, let's see how that works out. We could have created a new agent. Actually, we should have created anyways, but it still works hopefully. Let's see. Okay, so the summary is that video source uses the drone camera setup uh video stream frame reading, display frames, shutdown on exit. So, that's all good. And if we go to our PyCharm um there we have Pi inverse import drone then we connect we turn on the stream and we take off okay now in takeoff normally we go 1 m high right but here we are going to give it a different height so takeoff height let's say we will make it um 30 right so that's 30 cm mters from the ground because we we don't want to go really high. We want to have a good view. Okay, let's turn that off. And there you go. So now it went a little bit up. And there you go. We are getting our line. Now once we have this line uh actually let's uh let's let's change the size of it to a square rather than this long um instead of three columns we will put two columns. So then it will make it a square anyway. And then we also will change our track bar to true so that we can detect the exact red color uh that we have for this line. So there you go. So this is our track bar and this is our feed. So now let's move them and we'll make sure uh the red is been detected. It's a little bit noisy though. Yeah. So again I'm doing it roughly but you can spend some time on this to make sure you get some good results. So yeah there you go. Now we are getting like perfect. So let's stop that. We are going to copy all of this and then we are going to paste it here and we will rerun it. Oh but we need to change that to false. There you go. And now we are detecting our red line. Amazing. Okay. So now that we understand this, we are going to tell our AI model to detect this line and based on that move left and right uh to solve the line following points. So now we will go ahead and create our mission file. We are going to name it mission uh mission 5 line follower followover. Okay. So now what we will do is we will bring in our color detection code. Actually I was thinking that um color detection should be a code that is standalone by itself. So we should not involve the drone uh in it. So what we can do is we can just copy this because we have already written that. This is part of this mission. So that's why we can copy that here. But for color module I think we should keep it empty from the drone part. Right. So we can go to our drone and here we said use the drone camera for uh detection and all that. So we can simply revert that and we can click on continue. And if I go back now our color detection should be just about color. No drone involved. And then our mission 5 already has the code with the drone. Right? So a little trick we played with the AI anyways. So uh now that we are getting the line and uh we what we can do is we can also show the AI that what exactly are we getting so that it understands um what exactly to code right so let's run this uh actually are we running mission 5 no we are just running the color part so that's why it's not connected and all that anyway so now it's connected and there is our image so we can take a screenshot. Uh on Windows you can press Windows shift and S and that will allow you to take a screenshot and I will take the screenshot of this and I will press stop and then I will go back to our AI and I will paste it here. So let's create a new agent. Why not? So and then we will go to line forward. Given this line follow code. Uh right now it can right now it can detect the line and create a mask for it. Example image shown in example image given. Okay. Now I want to detect Now I want to detect. In fact, you know what? Uh okay, let's just ask it. Now I want to follow this line uh and move the drone accordingly. Use your to rotate. give very small values of rotation. Let's run that and see what happens. Okay, so there we have it. Summary of what was added. Line follower get line center. Okay, it will get the mask and then the steering is right and left. then the yaw gain and then we have forward speed. We have a constant forward speed and then you can adjust these three things. Okay, so that sounds good. If we go to our line follower and let's run it. There you go. Oh. Oh, something happened. Something happened. That's good. Oh, it went off track and the game is over. Okay. Let's run it again. We need to see what exactly went wrong. Oh, okay. It actually did follow, but then it did not go all the way. So, we need to tune it a little bit. The idea is good, but we need to tune it. Um what we can do uh for the very first thing we can decrease the speed. So if it's a slower speed it will be easier to move around and then don't change too much of yaw. Let's say we make it four. And y again let's keep it the same. Okay. It's going okay. Oh, it's not really okay. What we can Oh, it went bad. It fell off the arena. So, as you saw earlier, uh we were not able to complete uh with the basic method. So, what I've done is I have created the P file as well, but I am going to keep it a secret so that you guys can try it out for yourself because there will be a competition for this as well. So we are going to run this now and let's see what we get. So let me bring this here as well. So there you go. We are getting results. Now again it's not perfect. It's very slow and you know it moves around here and there where it should not. But still you can see it is able to do it is able to complete the task. Now this is the best part. you can optimize it as much as you can and then get the better results. So check how much time do you get. So the lower the better. So you can even check your ranking world ranking uh and you can improve that as well. So this is basically our line follower. I have used P for that and you can see the results. If you tune it better, if you try other techniques, if you improve it, optimize it, then you will get some better results with this. So, finally, we have completed our course and I hope that this was very informative and useful for you. So the idea is that we are going to learn lots of more examples and missions and we will be creating all these different scenarios which will replicate realworld applications. So rescue missions, agriculture and drone shows and lots of other scenarios. So if you are interested in this do check out our Kickstarter. So this is our Kickstarter page. So, we will be launching very soon uh in a few weeks and uh you will get lifetime access if you just purchase one time and you will be able to get access to all the different uh levels and all the different missions forever that will be uploaded. So, you pay one time and you get lifetime access. And uh in terms of these files, I will try to upload them somewhere so you can easily download them and try it out for yourself. Now the good thing is AI is taking over the coding part. So make sure you code it using the new methodology of actually creating these templates and then uh creating actual scripts out of these templates. So this is something that I have been testing around and I have found useful. So that's why I have created this complete course around this. So I hope you have learned something new. I hope this was interesting for you. If it was, let me know in the comments below. And what would you like to see next? Let me know that as well. And I will see you in the next one.

Original Description

Learn drone programming using Python and the high-fidelity Pyimverse simulator. The course is designed to take you from the fundamentals of 3D movement to deploying advanced computer vision across five practical, real-world missions. By focusing on simulation first, you can develop intelligent autonomous code for tasks like gesture control and line following without the risk of expensive hardware crashes. ✏️ Course developed by @murtazasworkshop Download Pysimverse here: https://pysimverse.com/ Kickstarter Launching on 14th April Back The Project Here: https://www.kickstarter.com/projects/cvweb/pysimverse ❤️ Support for this channel comes from our friends at Scrimba – the coding platform that's reinvented interactive learning: https://scrimba.com/freecodecamp Contents — 00:00 Introduction to Python Drone Programming — 00:31 Real-World Applications: Drones in 2026 — 01:42 Why Use a Simulator? Avoiding Hardware Crashes — 02:42 Overview of the 5 Practical Missions — 03:16 Understanding Drone Configurations (Quads, Hexas, Octos) — 04:14 Anatomy of a Drone: Motors, ESCs, and Sensors — 05:06 Downloading and Installing the Pyimverse Simulator — 06:07 Kickstarter Campaign and Pro Features — 07:27 Installation: Python 3.13 and PyCharm Setup — 12:12 Creating Your First Drone Project — 14:24 Writing the "First Flight" Script (Takeoff & Landing) — 18:35 Basic 3D Movement: Up, Down, Left, and Right — 22:40 Adjusting Drone Speed and Rotation (Yaw) — 24:01 Continuous Control with Send RC Methods — 27:39 Mission 1: Garage Navigation & Precision Landing — 33:39 Mission 2: Video Streaming & Image Capture — 41:16 Developing AI-Assisted Keyboard Controls — 57:56 Mission 3: Hand Gesture Control with MediaPipe — 1:11:29 Mission 4: Body Follower & Vertical Jump Detection — 1:29:38 Mission 5: Autonomous Line Following — 1:42:43 Implementing PID Control for Smoother Flight — 1:46:06 Course Wrap-up & World Rankings Leaderboard 🎉 Thanks to our Champion and Sponsor supporters: 👾 @ome
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from freeCodeCamp.org · freeCodeCamp.org · 0 of 60

← Previous Next →
1 React: Production Server Setup Part 2 - Live Coding with Jesse
React: Production Server Setup Part 2 - Live Coding with Jesse
freeCodeCamp.org
2 cookies vs localStorage vs sessionStorage - Beau teaches JavaScript
cookies vs localStorage vs sessionStorage - Beau teaches JavaScript
freeCodeCamp.org
3 Browser history tutorial - Beau teaches JavaScript
Browser history tutorial - Beau teaches JavaScript
freeCodeCamp.org
4 Graph Data Structure Intro (inc. adjacency list, adjacency matrix, incidence matrix)
Graph Data Structure Intro (inc. adjacency list, adjacency matrix, incidence matrix)
freeCodeCamp.org
5 React: Parameterized Routing with Next.js - Live Coding with Jesse
React: Parameterized Routing with Next.js - Live Coding with Jesse
freeCodeCamp.org
6 React: Dealing with jQuery Issues - Live Coding with Jesse
React: Dealing with jQuery Issues - Live Coding with Jesse
freeCodeCamp.org
7 setInterval and setTimeout: timing events - Beau teaches JavaScript
setInterval and setTimeout: timing events - Beau teaches JavaScript
freeCodeCamp.org
8 Browser and Device Testing - Live Coding with Jesse
Browser and Device Testing - Live Coding with Jesse
freeCodeCamp.org
9 Last Minute Updates - Live Coding with Jesse
Last Minute Updates - Live Coding with Jesse
freeCodeCamp.org
10 Post Launch Updates - Live Coding with Jesse
Post Launch Updates - Live Coding with Jesse
freeCodeCamp.org
11 React: Setting Up Google Analytics - Live Coding with Jesse
React: Setting Up Google Analytics - Live Coding with Jesse
freeCodeCamp.org
12 React: Masonry Layout - Live Coding with Jesse
React: Masonry Layout - Live Coding with Jesse
freeCodeCamp.org
13 Load Balancing Digital Ocean Droplets - Live Coding with Jesse
Load Balancing Digital Ocean Droplets - Live Coding with Jesse
freeCodeCamp.org
14 try, catch, finally, throw - error handling in JavaScript
try, catch, finally, throw - error handling in JavaScript
freeCodeCamp.org
15 Load Balancing: SSL Passthrough Setup - Live Coding with Jesse
Load Balancing: SSL Passthrough Setup - Live Coding with Jesse
freeCodeCamp.org
16 Graphs: breadth-first search - Beau teaches JavaScript
Graphs: breadth-first search - Beau teaches JavaScript
freeCodeCamp.org
17 React: Masonry Layout Part 2 - Live Coding with Jesse
React: Masonry Layout Part 2 - Live Coding with Jesse
freeCodeCamp.org
18 React: WordPress API Live Search - Live Coding with Jesse
React: WordPress API Live Search - Live Coding with Jesse
freeCodeCamp.org
19 Creating WordPress Custom Post Types - Live Coding With Jesse
Creating WordPress Custom Post Types - Live Coding With Jesse
freeCodeCamp.org
20 Dates - Beau teaches JavaScript
Dates - Beau teaches JavaScript
freeCodeCamp.org
21 Miscellaneous Front End Updates - Live Coding with Jesse
Miscellaneous Front End Updates - Live Coding with Jesse
freeCodeCamp.org
22 Merging a Pull Request from GitHub - Live Coding with Jesse
Merging a Pull Request from GitHub - Live Coding with Jesse
freeCodeCamp.org
23 React + Prettier + Standard JS - Live Coding with Jesse
React + Prettier + Standard JS - Live Coding with Jesse
freeCodeCamp.org
24 React: Sortable Responsive Table - Live Coding with Jesse
React: Sortable Responsive Table - Live Coding with Jesse
freeCodeCamp.org
25 Geolocation Sorting by Distance - Live Coding with Jesse
Geolocation Sorting by Distance - Live Coding with Jesse
freeCodeCamp.org
26 Tradeoff Matrix - Agile Software Development
Tradeoff Matrix - Agile Software Development
freeCodeCamp.org
27 The Definition of Ready - Agile Software Development
The Definition of Ready - Agile Software Development
freeCodeCamp.org
28 Getting first React job without experience - Ask Preethi
Getting first React job without experience - Ask Preethi
freeCodeCamp.org
29 React: Google Analytics Click Tracking - Live Coding with Jesse
React: Google Analytics Click Tracking - Live Coding with Jesse
freeCodeCamp.org
30 Submitting a PR to an Open Source Project - Live Coding with Jesse
Submitting a PR to an Open Source Project - Live Coding with Jesse
freeCodeCamp.org
31 Should I go back to school to get CS degree? - Ask Preethi
Should I go back to school to get CS degree? - Ask Preethi
freeCodeCamp.org
32 Hero Section CSS Changes - Live Coding with Jesse
Hero Section CSS Changes - Live Coding with Jesse
freeCodeCamp.org
33 Working Agreement - Agile Software Development
Working Agreement - Agile Software Development
freeCodeCamp.org
34 A day at Pennybox with Co-Founder Reji Eapen
A day at Pennybox with Co-Founder Reji Eapen
freeCodeCamp.org
35 React: Sorting and Filtering Data - Live Coding with Jesse
React: Sorting and Filtering Data - Live Coding with Jesse
freeCodeCamp.org
36 React: Sorting and Filtering Data Part 2 - Live Coding with Jesse
React: Sorting and Filtering Data Part 2 - Live Coding with Jesse
freeCodeCamp.org
37 React: Building a New UI - Live Coding with Jesse
React: Building a New UI - Live Coding with Jesse
freeCodeCamp.org
38 Definition of Done - Agile Software Development
Definition of Done - Agile Software Development
freeCodeCamp.org
39 Getting started with jQuery (tutorial) - Beau teaches JavaScript
Getting started with jQuery (tutorial) - Beau teaches JavaScript
freeCodeCamp.org
40 Making a React Blog with WordPress Content - Live Coding with Jesse
Making a React Blog with WordPress Content - Live Coding with Jesse
freeCodeCamp.org
41 React, NextJS, CSS - Live Coding with Jesse
React, NextJS, CSS - Live Coding with Jesse
freeCodeCamp.org
42 jQuery events - Beau teaches JavaScript
jQuery events - Beau teaches JavaScript
freeCodeCamp.org
43 React/NextJS Routing and WordPress API Custom Types - Live Coding with Jesse
React/NextJS Routing and WordPress API Custom Types - Live Coding with Jesse
freeCodeCamp.org
44 React: Working with API Data - Live Coding with Jesse
React: Working with API Data - Live Coding with Jesse
freeCodeCamp.org
45 React: Refactoring Components - Live Streaming with Jesse
React: Refactoring Components - Live Streaming with Jesse
freeCodeCamp.org
46 jQuery effects - Beau teaches JavaScript
jQuery effects - Beau teaches JavaScript
freeCodeCamp.org
47 More React Refactoring - Live Coding with Jesse
More React Refactoring - Live Coding with Jesse
freeCodeCamp.org
48 animate in jQuery - Beau teaches JavaScript
animate in jQuery - Beau teaches JavaScript
freeCodeCamp.org
49 "Finishing" My React Site - Live Coding with Jesse
"Finishing" My React Site - Live Coding with Jesse
freeCodeCamp.org
50 Starting a New React Project (P2D1) - Live Coding with Jesse
Starting a New React Project (P2D1) - Live Coding with Jesse
freeCodeCamp.org
51 React Project 2 Day 2: Learning Material UI - Live Coding with Jesse
React Project 2 Day 2: Learning Material UI - Live Coding with Jesse
freeCodeCamp.org
52 The Agile Manifesto - Agile Software Development
The Agile Manifesto - Agile Software Development
freeCodeCamp.org
53 jQuery: get and set with http, text, val, and attr - Beau teaches JavaScript
jQuery: get and set with http, text, val, and attr - Beau teaches JavaScript
freeCodeCamp.org
54 React Project 2 Day 3 - Live Coding with Jesse
React Project 2 Day 3 - Live Coding with Jesse
freeCodeCamp.org
55 The INVEST approach to product backlog items
The INVEST approach to product backlog items
freeCodeCamp.org
56 React Project 2 Day 4 - Live Coding with Jesse
React Project 2 Day 4 - Live Coding with Jesse
freeCodeCamp.org
57 Chickens and Pigs - Agile Software Development
Chickens and Pigs - Agile Software Development
freeCodeCamp.org
58 React Project 2 Day 5 - Live Coding with Jesse
React Project 2 Day 5 - Live Coding with Jesse
freeCodeCamp.org
59 jQuery: add and remove DOM elements - Beau teaches JavaScript
jQuery: add and remove DOM elements - Beau teaches JavaScript
freeCodeCamp.org
60 React Project 2 Day 6 - Live Coding with Jesse
React Project 2 Day 6 - Live Coding with Jesse
freeCodeCamp.org

This tutorial teaches drone programming using Python and the Pyimverse simulator, covering the fundamentals of 3D movement and computer vision. By the end of the course, you'll be able to deploy advanced computer vision models across five practical, real-world missions.

Key Takeaways
  1. Install Python and the Pyimverse simulator
  2. Complete the introductory mission to understand 3D movement fundamentals
  3. Program a drone to navigate through a simulated environment
  4. Deploy a computer vision model to detect objects
  5. Complete the remaining four missions to apply computer vision to real-world scenarios
💡 Simulation is a crucial step in developing intelligent autonomous drones, allowing you to test and refine your code in a safe and controlled environment.

Related Reads

📰
Stop Wasting GPU Dollars on Blurry Data: Meet VigilCV, the 223µs Vision Gatekeeper
Learn how VigilCV helps prevent silent model failures in computer vision pipelines, saving GPU resources and improving model reliability
Medium · Deep Learning
📰
The GeoNodes Grind: Bridging Procedural Power and Real-time Performance
Learn to optimize GeoNodes for real-time performance and procedural power in graphics rendering
Dev.to · kirolos nadi
📰
Computer Science Internship Preparation: A Student Guide
Prepare for a computer science internship by improving your resume, GitHub projects, and technical interview skills
Medium · Programming
📰
Building User-Selected Object Tracking: ByteTrack, Cosine Similarity, and a Tiny Neural Network
Learn to build user-selected object tracking using ByteTrack, Cosine Similarity, and a Tiny Neural Network for accurate object detection
Medium · Machine Learning
Up next
This System Captures The Whole Stadium At Once | AI Chooses The Perfect Shot
Anik Singal
Watch →