GenAI Deployment Guide: Dockerizing LLM Apps with AWS ECR & Parameter Store

Key Takeaways

All right. Now that we know how the application works and we have looked at the front end and the back end, it's now time to deploy the application. To deploy the app, we are going to first package it into a Docker container. Now there are two things that we have to look at. One is thevb file. As we can see the AWS credentials consist of the account ID, the access key ID and the secret access. This is the credentials for my particular user in AWS. This application actually runs in appunner where we would be creating a separate role and giving it access to all these services. So it's a bad practice if I'm going to supply these enviables inside my application which is actually created just for me. And one of the good practice is to actually always use IM roles and policies. And these are the application variables that I have to provide to my application. The default region, the bedrock ID, the OpenAI API key and the OpenAI model. All right. So what I do is I have to now create this as an end toend package. To create the package we are using a docker file. In the docker file we are working with one of the python 3.11 slim which is the smaller base image for python 3.11. We create a working directory at / app. We copy the requirements.txt txt all the packages and then we finally install all the package. Then in terms of code, we just copy the app.py and agent.py file and we then simply run our function using python app.py. So this docker file can then help us to create the docker container and then we'll push it to the docker container registry. So before we do let's quickly have a look at the architecture. So as we can see the docker file would be helping us to create the docker container. Then we'll finally push the docker container up here and the application would then be deployed in appunner. Apprunner would then pull the docker container and run the app. But before that before running it it will also go to parameter store to get all the environment variables that it needs. It will also inject these environment variables. So these would be available to the application. Now to be able to call all these services currently it was using the application in my local was using my AWS credentials but now we would be attaching an IM role to the appunner. What this role does is gives it permission to connect to knowledge basis and also to parameter store to fetch environment variables. It connects to knowledge basis to obviously retrieve all the documents from a given query and parameter store to get the open AI API key so that it can finally use that API key to call open AAI. All right. So to finally start with the deployment part which is the left side of the application um architecture diagram. Let's start with creating our ECR container. So we'll go to the dashboard. We'll mention ECR in search. This takes us to the ECR dashboard. Click on create and we can give it a nice name Chennai bot and that's it. Click on create. Finally, it's created. We can see the push commands. Now, first we have to log into this Docker registry to be able to push things. We'll go back to the code. Now we'll just make sure that source envir all the local my user credentials are loaded so that it doesn't interfere with any other AWS credentials if they are all on this machine. I'm logging into this registry. The login has succeeded. Finally I'll build the image for genib bot using the build command. One thing that I would mention is that I would always append it with platform Linux x64. This is because my machine a MacBook over here it has ARM based platform and just to not interfere with that the type of of image this has to be is going to be x86 and that's why it's a good practice to simply mention that the platform should be Linux x86 and so that the docker container is not created in as an ARM based image. All right. So I just write platform. Okay. It worked nicely. The build is done. I can finally tag it. I can then All right. Now the push is done. This container is now pushed to the AWS ECR. We can go to the dashboard. We can refresh it and we can see that the image is pushed. Now that we have ECR in place, let's also go and set up parameter store. So on the search we will mention parameter store. Here we'll click on create parameter. We'll name our parameter OpenAI API key. We'll mention it as secure string. So it would be encrypted and stored in AWS. And in the value, we would be putting the ENV value that we have just to make sure we don't have any spaces anywhere. All right. So with this this parameter is created. OpenAI API key. It's nice. Now that we had the parameter also stored in OpenAI API key, the value is stored in parameter store. We can finally use Approna to deploy the application.