Automate AWS EC2 Setup: Launch Jupyter Notebook with Python & Boto3

Key Takeaways

So now we are going to proceed with our next step. So for that I'm going to open my command prompt in the local setup. So I'll be pressing window r and I'm going to open the command prompt everybody. Now the thing is you should be having the Jupiter notebook setup ready in your local okay or you can say that in your command prompt you have already configured the AWS credentials and you are having your setup ready because what exactly we're going to do as I told you in this particular part I'm going to show you like how using the Jupyter notebook in the local we are going to set up an EC2 instance in the AWS account and do the required installation. Okay. So again I can quickly check Python space-en version. Okay. And I have already configured AWS credentials. I had shown you in my past videos that how to configure AWS credentials with the required permissions. So either for the practice purpose you can configure the credentials with the administrator privileges and once your demo is done deactivate the access keys or the other way is you can give the required access which is used to perform this particular demonstration. For example in this demo I want to launch an EC2 instance. Okay. So I have already provided the enough access in order to perform this demo. So how I'm going to start my Jupyter notebook in the local I will be typing here Jupiter notebook and let's see what I'm going to get after typing Jupyter notebook. It should be running Jupyter notebook and I will be getting a prompt in the browser that is going to load a notebook in my local machine. You can see the response from here. It is saying that extension manager is this. And here you can see it has opened a Jupyter notebook in my local machine. Everybody, this is my local browser. You can also relate it from local host. Now what you have to do, you can open a new notebook if you want. Okay. And you can also explore all these things like you have Python that is a kernel. This is a terminal. This is console and many more things you can explore from here. Okay. You can go to file. I'm going to open a new notebook everybody here. And then this is my script. Now I'm going to give you a highlevel idea what the script is doing. First of all, we are importing the required libraries. Let me do that. So you can take it step by step even. So import the required libraries. You can run this. Once this is executed successfully, the next step is I'm going to provide a set of parameters. Now, now the region name is the name where you are going to launch an EC2 instance. So, for example, I am providing the region name as AP South one because I will be launching the EC2 instance in Mumbai region. Okay, those who are new to AWS, they can always take the region code from the top means right now I'm talking about Asia Pacific Mumbai which is having a region code as AP- South. So I've taken this code from here. Okay. So this code has been taken. Now next thing is we have to provide the AMI. What AMI is? I told you AMI stands for Amazon machine image. Now AMI is a region specific means each region would be having the different AMIs with the AMI ids. When I'm talking about different AMIs here I'm specifically mentioning the AMI ids. Okay. So I will be needing an AMI ID here. How can I get it? For example, if I have to launch an Ubuntu instance in the AP South one which is a Mumbai region. Okay, I'll be choosing the AMI ID. How? Let's go to instances. I'm just clicking on launch instance this time to take the AMI ID. So from where you can get the AMI ID in this section once this Amazon machine image section would be loaded I will be copying the AMI ID for Ubuntu AMI okay let's select this and scrolling down from here you can take the AMI ID let me make a copy of it and I'll be pasting it here okay then instance type would be the one which you want to launch. So in my case I'm launching an instance of type T3.m medium but I told you use the one which you want to basically launch as a part of free tier and then key name is basically a key pair which will be used to connect to this EC2 instance externally using remote client. Obviously I'll not be doing that. If I would be required to troubleshoot something, I would be using EC2 instance connect. But here I have to provide the one which is existing in my account. If you are new to it, go back and expand these horizontal lines. If you scroll down here, you can see under network and security, we have key pairs. Go to key pairs and let's create a new key pair everybody. So let's say Jupiter keeper something like this keep rest all settings default just copy this name and we're going to use this name as a part of our code okay so no need to worry about if you don't have any key pair existing in your account just create a new in the way I have told you this is the connection which we are establishing using boto3 which is basically a SDK for python account. This particular instruction is helping me out to get my IP which I will be adding in the security group rules. Okay. And here I'm providing a security group name. So let's say if this security group would already be existing in my account when I am going to execute this piece of code, it will be saying security group already exist. So let me give it another name called Jupiter security group new. Okay. Restore settings. What we will be doing? We will be creating a security group. This is the first thing. We are allowing some rules on port 22 and this port 8888 means whatever you had done from the management console now we are doing it using code. And finally we are passing a user data. This user data is doing the same thing which you had done using commands like installing the python pip or virtual environment configuring the environment and providing certain properties and finally it is starting Jupyter notebook. Okay. So when I would be running this EC2 instance this particular user data script would be running at the time of launch. Okay. And finally after waiting for some time I will be connecting to the Jupyter notebook here. Okay let's do it. Let me quickly take this particular piece of code everybody. You can go through it if you want. I have given you a highle idea what exactly I'm doing from here. Okay. So we're starting from defining some parameters which will be required to launch an EC2 instance. We are creating an EC2 object using BTO3 client. We are fetching the IP address. This is the my IP address. In your case, it will be your IP address which is provided to you by your internet provider. Okay. This is a security group name. This is a security group description. The first thing we are doing, we are creating a security group. Later we are aligning some rules and the traffic we are aligning from my IP address. the one which we had fetched in the above part of our code. If there would be some error, we are going to throw some exceptions. Okay, this is a user data. This is to install and run the Jupiter. So, all the prerequisite which will be used, we are doing it from here. Finally, we are calling the run instance method and we are passing all the required things. We are printing the instance ID. We are waiting for the instance to come up and running. We are fetching the public IP address. And once the public IP address is fetched, we can access our Jupyter notebook using http/ublic IP888. Okay. And we have to wait for certain time in order to get our Jupyter notebook ready. This would be happen because we are passing a user data script which I told you will be running at the time of launch. Let's run this piece of code and let's see what happens. You can see the response here. Let's wait for the response to get. It is detecting my IP address. It's creating a security group. Launching the instance. Waiting for instance to run. Instance is running. And now I can access Jupyter notebook after 2 3 minutes. The first thing you all can see here how faster it was. So it's your choice. Do you want to launch an EC2 instance using management console or do you want to launch it programmatically? So you have both the ways. Whatever you prefer, whatever you want, you can follow the same set of instruction. Let me click here and let's see what I'm going to get after couple of minutes. In the meantime, I can quickly give you a brief summary what we have done here. We started with launching an EC2 instance using Ubuntu image and then we basically chosen the instance type and all the required properties for launching an EC2 instance. Once done, we connected to our EC2 instance. did the required configuration in terms of installing certain packages, setting up virtual environment, manage the configurations, allow the rules and finally we had access the Jupyter notebook in the browser after providing the I password at the login page. Okay, later I told you how can you do the same thing programmatically. How can you write a piece of code and using that piece of code you can achieve the same result. Okay. So let's quickly wait and we will be seeing a Jupyter notebook. So we'll be waiting till the time I'm going to get the access to Jupyter notebook and once it is done we are going to conclude this particular part of our discussion. So you can see here your Jupyter notebook is running in the browser and you can do the same thing if you have to install some additional packages you can open the terminal and again you can install the additional packages in a similar manner. Okay. So this is something I wanted to show you. So you can see this was a Jupyter notebook running. This was a Jupyter notebook running in my local machine. And again this was a one which was running from a past EC2 instance which we had launched using management console. Okay. So this is something I wanted to show you and I hope you can find it interesting and you can gaze an understanding how can you continue your journey in cloud on whatever role you are into.