Generate & Rotate Dynamic Credentials w/ Vault & Python App in 20 mins

Hello friends, welcome back to my channel. In this project, we are going to look at secret management system with zero trust using Hashiko vault. So we going to build a secure Python application that connects to a database which is a posgress SQL database and it uses Hashiko vault to dynamically generate secrets such as username and password in order to connect to the database. Now all the parts of this project are going to be running in a docker container. So we will use docker and docker compost to orchestrate the whole thing. Like comment and subscribe to my channel for more videos like this. Stick around for the entire video. And now let's get into it. By the way, when you go to the GitHub repository, which is icon C, you'll find the repository for this project right here. Go to repositories and you'll find secret management vault docker. Go in there and you see all the scripts or the commands that you need. Everything here and have all the explanation in the readmi file as well. By the way, I've also cloned this files and everything inside this repo into my local machine. So I have exact same thing here. So we are basically going to run it from our local machine. Before I started everything, just know that I'm doing this project on an Ubuntu machine. This is Ubuntu 24.04. That's just by the way. Now for some prerequisites, we need docker docker compose. We need hash vault. We need posgress SQL. That's going to be our database. And we need a Python application. That's what we are going to use to access the database which is the posgress SQL. Our vault is going to be our secret management system. And docker and docker compose. We will use that to containerize applications and also to orchestrate multiple docker application services and that is with the docker compose. So first just make sure that your system is updated. I'm running everything on ibuntu. So you can just run this here to update your system. So it's updated. And then you would need to install Docker, Docker Compose on Zip Pro and JQ. JQ is JSON query. We are going to use this tool here to query JSON formatted data. Now I already have all of these installed. So I'll just check the versions for Docker and Docker Compose. If you don't have it installed, just run that command on your terminal and that should give you the results. So you can clearly see the Docker and Docker Compose versions right there. Since I have that installed, what I'm going to do next is to create a Docker compos script. And this is going to have the services for vault and postgress SQL. And I have the code here and everything, but I've already created a file here as well. So just going to open the docker compos file right. So in docker compose when you want to run multiple docker containers in a single file use a docker compos file. So in our case we want to run vault and posgress. So each is a service. So this is the vault service and this is the posgress sql service. Start by specifying services and then you list all the services that you have. So for vault we are using the hashop vault colon latest as our image. We are calling the container vault and the port is 8200. These are the default ports for vault. And then some environment variable we using the token root and the address is 0000 at the port 8200. And then we have a command here that's going to start the old server for dev environment. And then we have a posgress database here which is using the posgress image tag 15 we calling it posgress and we have a user root password root pw and then we are creating a database my db and then we are porting 5432 to 6432. Now this is because I already have an application running on 5432 on my local machine. So I'm porting the 5432 of this Postgress container to 6432 instead instead of 5432 right and then because it's a database we want to persist data so we need a volume attached to it and usually Postgress stores data in var poss data so we need to mount that into the location of where we want to store our data which is pg data now in a docker compos file whenever you write script like this. You need to also reference the volume outside of the service itself. Right? So with that we can easily run this command here docker compos appd and this is going to run both vault and posgress containers. Press enter and you see it started here. First created a network a volume that's the volume you specified there and then the two containers. You can also verify this using docker ps and you should see both containers are running. This is vault container and this is the posgress container. Next we need to configure vault and posgress. Before that we need vault cli installed. I've already installed it. So let me first clear the screen. When I do vault v this is my vault cla that is installed version 1.20. So if you haven't installed it and you are working on Ubuntu as well, you can use this snap utility to install it. I'm going to check default status and you can see that there's a warning that the address is on set and so it's going to use the default the local host. Remember when I run the docker compos file the address for vault here is 000000. So it's really on set. It's open. That's why it's routing it to the local host but at the port 8200. That's okay. We will set the address soon. So we initialize these environment variables to set the correct address. I'm just going to run both of them together. This is going to set the address for volt and the token for vault. With that set, you can now enable database secret engine. And that is from the command vault secret enable database. Paste it here. And this should enable the database secret engine for vault. With that out of the way, we can now configure vault to connect to the Postgress SQL database. One thing you should notice since both posgress and vault are running as docker containers from our docker compos orchestration we did earlier we need to make sure that they are both running on the same network so that they can communicate with each other. Usually when multiple containers are on the same network the way they communicate with each other is by using a unique identifier and that unique identifier is their host name. Now when I do docker ps the host name of our containers is basically the same as the names that we specified or the container names we gave it. And so for the host name for our posgress database which is going to be this location right here. If you are working let's say with virtual machines you would put the host name in this very location. But since we are using containers you need to replace that host name with the name of the container which is posgress. So we run the command v write database config my posgress database we are importing some login name and then we will allow some row my role we will create this role later with more instructions and then the connection is to the posgress database we use some username password and then we are trying to connect to the posgress container now the username and the passwords is exactly what you see here root and root pw this is also the same thing we had in the docker compos file here root and root pw for connecting to the posgress sql. So going to copy this command here paste it here and run it. So you see that data has been written to this database. Now with that we can now create a vault role for managing dynamic credentials. So for that we will use the command vault write and then we are creating another database for rows and we are calling it my role inside the rows directory. For our database name remember over here the database that we wrote is my posgress database. So we have to reference that database name right here. And then we are going to write some SQL command to create the row. You see creation statements and it says create role. So you provide a name and then the login password is going to be the password that we referenced earlier which is root pw and this will be expiration and then we want to grant some permissions to or add some permissions to that role. We are granting select on all tables in the schema. Run this command here. Press enter and you see that the row has also been written to its database right there. Let's test it to see if it's able to generate any credentials at this point. And by credentials, I mean any dynamic credentials. It cannot be the exact same credentials when you generate it multiple times. It has to be constantly changing. And that's what we mean by the zero trust principle. So I'll run this vote read command here which is going to read some credentials in the myroll database. So paste this here and you can see here that it generates some credential. Now the list duration is an hour and this is renewable because it's dynamic. It has to be constantly changing. This is a password that is just generated and this is a username. Now you see that even though this looks like a table format but it's actually a key value store it has a key and then it corresponding value which can easily be transformed into a JSON format. That's why we needed the jq or the JSON query earlier. So we are getting credentials at this point. So that is good. So next we will look at how to use a Python application to fetch secrets from vault in order to connect to our Postgress database. So we are going to create a Python script or the Python application where we try to access our database Postgress SQL and the credentials that we are going to use to access it is going to come from vault and they will be generated dynamically and so of course I have the script here but I've created a file app.py inside our system here. So this is app.py with all the code that is over here. So you can just copy and paste it in there. Let's go through the app.py script briefly. So first it's importing the libraries or the packages we need. So we have requests and then cycle pg2. So first remember the environment variables we set for v the token and the address right here. We run this command here. We are doing same in our script here. And then we are going to fetch the dynamic credentials that volt supplies. V is going to pick up that results. Remember this result is going to look like this table form. And I did mention earlier that this is also like a JSON format. So it will query this and then it will grab the username and the password which is the last two lines that is generated right there. And you can see here it takes the rest which is the actual output and then it uses JSON here to write it properly in a JSON format and then it will use this to grab the usernames and the password. Next it's going to print the usernames and the passwords. Next to connect to the posgress SQL database we need the cycle PG2 and then of course we have to do connect. We need to specify the database name. Our database name is my DB. Remember in the docker compos file database is my DB right there. Right. The username and the password is is going to come from the username and password that vote supplies us over there. And then the host name for our Postgress. Remember when we run the command because we using containers the host name we used was posgress which is this section right here or the name of the posgress container and then the port is of course 5432. Then we will use cursor and execute. Execute will just help us to run some SQL commands and then we want to try to see if we can get some information from the database just to show that we've actually connected to it. So we are trying to fetch the date time and we are using the fetch one function to get that. After that we will print the date time or the database time that we got from this particular command right there. At the end we want to exit all connections. So once a dynamic credential is generated we connect to the database do whatever we need to do and then we exit until the next time we need to connect to the database again from our application. So that is pretty much our app.py. And now to run it, we want to containerize it as well since everything is running in docker containers. So we need to write a docker file for this Python app. And that is pretty straightforward. By the way, I have the Docker file here as well. And the content is exact same as what's over here. So we using a Python 3.10 slim. That's a very lightweight Python image. And we are setting a working directory / app. And then we are copying our Python app which is app.py into this same location which is / app. And then we want to run some shell commands. First a shell command to install the packages that we need. Remember in app.py we need this packages. So we need to install them. So that's why we have these two. And then the command to run the app.py Pi script itself is Python app.py. That's how you would run it on your terminal. But inside a container, you run it as a source script. So you need to provide it as a command. So cmd. So it's pretty straightforward. Docker file here. So now first let's build the docker image for Python app. I'm going to clear the screen for now. Paste this. And this is going to build the app and call it vault python- app. We using dot because the docker file is inside the current directory. Press enter. Right. So it's done. I'm going to clear the screen again. Docker images. And this is our Docker image that is just built. After building the image, next we need to run it. Now, in order to run it, you need to make sure that the Python app is also running on the exact same network that all the other containers are running on. So, the easiest way to do it is just give it a flag dash network and then specify one of the containers and its name and it's basically going to put it on the exact same network that the vault container is running on. That's one of the easiest way to do it. So we have docker run, we have the vault address that we set, the environment variable and the token is root and the name of our app. Now you see we have d-r here. That's because I don't want this container to be running forever. I just want it to run and then execute this command here in app.py. It needs to execute this command here and then these commands over there. then everything should exit. That's why I have this flag. If you don't want that, you can keep it running by removing this dar. Now also make sure that this is exact same name as the docker image that you built. You can provide the tag as latest but if you leave it like this that is also going to run. So I will copy this command here. Paste it here. Press enter. And you can see some output here. It says connecting. It says vault issued some dynamic username and password. And you see those new usernames and passwords. And then it's able to connect successfully using those two usernames and passwords to the posgress SQL database. And it's able to print the database time right there. Now, because it's dynamic, if you run it multiple times, it's always going to generate a different username and passwords. So, I'm going to run it multiple times here. So second time make this a little bigger. This is the second time and you can see that the username here and the password here is different from the username and password here. But it's still able to connect to the posgress SQL database. One more time again you see this username and the password is different from this username and password. It's also different from this username and password. But for all the cases connection to the database is still successful. So that's what we mean by dynamic credentialing or dynamic credential generation to ensure zero trust principle in applications and database communication. Before we leave, I'm going to clean up everything. So when I run docker ps, I have these two containers running. Remember there is also a docker volume in the docker network because everything was specified in the docker compos file here. So in order to get rid of everything just use the docker compos down v. The -v is just going to get rid of the volume as well. Run this command here and you see that it's removed the two containers. It removed the volume and then it got rid of the network as well. So everything is pretty much gone. Like, comment, and subscribe to my channel for more videos like this. And I will see you on the next one.