Deploy Production-Ready Agents in 22 Minutes with AgentCore Runtime

Key Takeaways

Three lines of code. That is all you need to take an AI agent that runs on your laptop and host it on Amazon Bedrock agent core runtime. Let me show you. We have a simple logistics helper agent built using the Strands agents SDK as the agent framework and OpenAI as the model provider. We'll go over the agent code in a minute, but we're currently looking at the diff that shows the three lines that make it deployable on agent core runtime. We have one import, one instance and one decorator. Then the runtime environment, authentication, networking, scaling and session isolation are handled for you. With agent core runtime, you can deploy agents in just a few minutes and a lot of the heavy lifting is abstracted away. Runtime is both framework and model agnostic meaning you can host agents built with line graph crew AI ad strands agents or even your own custom framework. Any framework will do. You can bring your code and without any major rewrites or difficult integrations. It runs in a managed serverless runtime. Beyond agents you can also run your custom MCP servers or agentto agent servers with runtime. I have an end-to-end example so you can learn the ropes of agent core runtime. We will start by launching the logistics helper agent to runtime. And the first version of the agent uses stub tools with mock shipment data. We will wire it up and deploy it to agent core runtime using the agent core starter toolkit on the command line. So you can see just how fast it is to get something live. Then we will swap those mocked out tools for real SQL queries against an Amazon relational database service or RDS database running inside of a private VPC. And because production workloads should be using infrastructure as code, we'll deploy that version using the AWS cloud development kit or CDK. So the runtime, networking, and permissions are repeatable and can be version controlled. Let's take a look at our logistics helper agent. At the top of the file, we have our imports. Some important ones include import agent and tool from strands and the open AI model for the model provider also from strands. We then configure logging and set up some variables that we'll need later. There is also some mocked up data that the tools will be using as the fake shipment data for our logistics company. Next, we define several tools using the at@ tool decorator from the strands agents SDK. Get shipment status returns the current location and latest event for shipment and find delayed shipments returns any at risk cargo. After that, we then define the open AI model that we want to use for our agent. This function loads an open AI API key from AWS secrets manager and creates the open AI model that we'll pass in to our agent. Doing it this way keeps API keys out of configuration files or environment variables. Below that, we define a system prompt that instructs the agent to act like an internal logistics assistant. Then we create the agent using strands passing in our open AI model, our tools and our system prompt. Finally, here at the bottom, we have a function that kicks off the agent, taking in a query from the user, calling the agent, and then it returns the result. Now, let's turn this into an agent core runtime application with those three lines of code. Scrolling back up to the top, we then need to add the import. For this, we are importing from bedrock agent core. import Bedrock agent core app. Then we need to create the runtime app object. For this, we have app equals Bedrock agent core app. Then scrolling back down to the bottom, we need to define our entry point. To do this, we use the decorator at app.ent entry point. Agent Core will call this entry point whenever a runtime receives a request passing in a JSON payload. That is all the wiring we need for our first deployment. What this does behind the scenes is it automatically sets up endpoints that fulfill the service contract that agent core runtime needs for HTTP invocations. It will set up a post endpoint called invocations on port 8080 that will receive incoming requests and a get endpoint called ping. Everything else including the authentication, networking, scaling and session creation and handling is done by agent core runtime. Now let's use the agent core starter toolkit to configure and deploy the agent. The agent core starter toolkit makes it easy to configure and deploy agents to runtime. First, you need to install the starter toolkit. To do that, I'll run pip install bedrock a core starter toolkit. If you were starting completely from scratch, you can have the command line set up a new project for you by running a core create. Since I already have all of the files I need for this use case, I will start by simply running agent core configure. This is going to take me through a series of prompts for it to collect information about the configuration for our agent. First, it's asking us for our entry point. This is the name of the file where that app entry point decorator exists. The name of our file is agent. py. Then we need to give the agent a name. For this, I will call it logistics agent. After that, we can then point it to a dependency file that is requirements.txt. Now, it's asking us about deployment mode. You can select from either containerbased or direct code deployment. Direct code deployment is the easiest way to deploy your code package when it's under 250 megabytes in size. So, we're going to select that method. What this is going to do is it's going to take your Python files and your dependencies, bundle them up into a zip file, and upload that to S3. There's no need for Docker container repos or custom container builds. If your package size is under 250 megabytes and you are working in Python with common frameworks like Langraph or Strands, this is usually the fastest way to iterate and have a good starting point. Updates deploy in about 10 seconds. However, the container deployment method is the more flexible option and likely what you would use when graduating to a real production ready deployment. Instead of bundling up a zip file, the toolkit generates a Docker file and launches an AWS code build job that builds the full container image for you whenever you deploy the agent. That image gets pushed to a container repository in your account hosted on Amazon Elastic Container Registry. So you would use the container deployment method when your code package is bigger than 250 megabytes when you need full control of the underlying base image or if your runtime is not a currently supported runtime. So we will select direct code deployment here. Then it's asking us for the Python runtime version the execution role. In this case I'm going to provide an ARN for an IM role that already exists in my account. This is going to be associated with the runtime. So that way your agent has access to AWS credentials that it can use to securely invoke AWS APIs. Then we can provide the S3 bucket that we want the zip to be uploaded to. I'm going to let the toolkit create a new one for me. And then we have options for authorization configuration. For authorization configuration, runtime supports either OOTH or AM. We're going to use IM authorization. So we'll select no for this. And then for the request header allow list. This is asking us if we want any specific request headers to be passed through to our agent code. We don't need that for our use case. So we will select no. And then we will also skip memory configuration. Now that our agent is configured to deploy we can simply type agent core deploy. Let's give this a few seconds to complete. Now the agent is ready and we can invoke it from the command line by running agent core invoke passing in the query of what is the status of shipment shipref 10001 which is referencing some of the mocked data that our tools have access to. It's good to note here that agent core is a serverless service. So this starts a new runtime environment up from zero and creates a new session for the agent to run in. And when you run a command like agent core invoke, the toolkit signs the request using whatever local IM credentials you have set up. We can see the response has come back giving the result for the question we passed in. So this thing is working. It's now fully hosted, authenticated, session managed and remotely accessible. Next, let's talk about how runtime sessions work under the hood before we connect this agent to a real database. When you deploy an agent, each invocation gets its own instance of the agent called a session. Each session gets its own lightweight microVM and each agent session has its own compute, memory, and file system. That means if two users invoke the same agent at the same time, they're actually hitting separate instances of that agent that are isolated from each other. Sessions are ephemeral but can live for up to eight hours which is good for long running agent workflows like deep research. The default behavior is that it will live for 15 minutes. This is of course configurable so you can control how long a session sticks around. Once a session is done and has been idle for long enough, it will automatically shut down and the resources will be cleaned up. that gives you a secure, stateful workspace where your agent can maintain short-term context across invocations without risking data leakage between users. This behavior is controlled by session IDs. When you submit an invocation to your agent, you can optionally include a session ID in the payload. This makes it so that the runtime automatically routes that request to the matching agent session. If no session exists or session ID is provided, a new one is created. This is a foundational concept to understand how runtime handles both scalability and security. So now let's talk about updating and testing your agent code. You can easily redeploy your agent by making the changes and then typing in agent core deploy again. But you can also deploy locally to test your changes before you launch the runtime using agent core dev. Let's test that out. I will type in agent core dev and this will spin up a development server with hot reloading enabled for rapid iteration. Once it's up and running here, I can then open up a new terminal session. And then from here, I can run agent core invoke-dev passing in that same payload that we had before with query and then the question of what is the status of shipment ship refer1. closing out the JSON and then we can hit enter. This is going to invoke the agent locally. You can see in the other terminal session here the post has come in and the invocation completed successfully and the completion or the response shows up in the other terminal session. This is a great feature of the toolkit that you can use during development. Now let's make things a bit more real world. The mocked out tools we have been using are great for testing and development, but a real logistics operation team needs our agent to use live shipment data. In this next step, we are going to keep the same agent and swap out the mocked tools for real SQL queries against an Amazon RDS database that already exists inside of a private subnet in a VPC that I have in my AWS account. I'm not going to walk through the provisioning of the network and database here. We'll treat it as an existing system that our agent needs to talk to. Most of the code for this is the same as the mocked version. The important change are the tools and the database connection setup. There's a function that loads the database configuration details from AWS systems manager parameter store. It reads the RDS endpoint, the database name, and the ARN for a secret that holds the username and password. Then it calls out to AWS Secrets Manager to retrieve those credentials. Doing it this way keeps those connection details and passwords in managed services instead of in our source code or environment variables. Down beneath that, we have a function called get DB connection that uses a package PG8000 to talk to the database. This uses a connection object that is global, which means that it can be reused inside of a warm session and it doesn't need to reconnect on every single request. That is one example where the session model we just covered really matters. Next, we have the implemented tools that actually query the database. The functions are named exactly the same as before and each function is still decorated with the at tool decorator from strands and it still returns data that the model can reason over. The difference is that inside each tool we call get DB connection and then we have a parameterized SQL query set up that will run against the logistics database and then we take those results we convert them to JSON and return them. From the agents point of view nothing has changed and the next function has a different query that is specific to finding delayed shipments. After that the system prompt the model initialization for the open AI model and the agent initialization from strands is exactly the same. The same applies here to our entry point. Nothing has changed there. Now we need to deploy this version in a way that follows best practices for production workloads. There are two things to consider here. deploying this in a way where it can securely connect to our private database running in AWS and deploying this in a repeatable reliable way using infrastructure as code tools. For the mocked version, we use the agent core starter toolkit. That's perfect for development and I love using this workflow. You can type agent core configure and then agent core deploy and you are up and running. But once you're ready to really deploy your production agent, you want to be following an infrastructure as code approach. For infrastructure as code, we will use the AWS cloud development kit. And for the secure database connection, we will use the VPC connectivity feature of agent core runtime. Let's take a look at the CDK stack I have created to deploy the agent. At the top of the stack, I have to import the existing VPC where the RDS database is running and we will deploy our agent into a private subnet in the same VPC to ensure secure connectivity. Then we are referencing a security group for the runtime. This is a firewall that allows the private database connection between the agent and the database. Next, we will package up the agent code and its dependencies as a zip file. To do that, we need to package the agent code as a direct code artifact and we are calculating some local files that contain the agent code and some helper scripts. Then we set up an S3 asset that points at the agent directory for bundling. It spins up a lightweight Python container image with a bundling script that runs inside of the container. This bundler copies in the agent code, installs the dependencies into the same directory targeting Linux ARM 64 and Python 3.12, and then it runs a small helper script to zip everything up at the root level. As a core runtime requires ARM 64 compatibility. So, if you're working on a machine that isn't ARM 64 compatible, then this script is super important because it will download the correct packages. After we have the code bundling defined, I need to define the IM role and permissions that the agent will need. We have the permissions defined for Cloudatch logs, for X-ray, for Cloudatch metrics for monitoring, AWS systems manager parameter store for retrieving those credentials and finally AWS secrets manager for retrieving the database credentials and the OpenAI API key. Then for the actual runtime definition itself, we are using the low-level cloud for resource AWS bedrock agent core runtime as the type. In the properties section, we set up the agent runtime name which is logistics agent CDK. We have a description. We pass in the ARN for the IM role. Then we set up the network configuration for the runtime. The network mode is VPC and the network mode config accepts a list of private subnet ids and the runtime security group. That information will be passed in at runtime from environment variables. After that, we then reference the code artifact we created earlier pointing to the S3 bucket and prefix where the zip file is with our code. The entry point is set to agent. py and the runtime is set to python 312 which matches the environment we targeted when we installed the dependencies in the bundling step. Then we have another file here called app.p py and this is the code that gathers all of the information from the environment variables and then it actually passes those into our stack at runtime. When we deploy this and invoke it, agent core runtime will create elastic network interfaces in those private subnets, attach that security group and give the runtime sessions private IP addresses inside the VPC. It also connects agent core to the VPC using a VPC endpoint. That's the plumbing that lets the agent talk to the RDS instance over a private connection instead of using the public internet. It's also important to note here that there are VPC endpoints for other services as well like AWS systems manager and AWS secrets manager and there is a NAT gateway in a public subnet in my VPC so that my agent can make calls to open AI from a private subnet. To deploy this we will run CDK deploy and the CDK does three main things. It runs the bundling command in a docker container to build the zip for the right architecture. It uploads that zip to S3 as an asset and it creates or updates Bedrock agent core runtime with the VPC configuration, the AM ro and the pointer to that S3 asset. Let's give this a second to complete. The agent is done deploying and we can invoke it. But let's first take a look at what this deployed in the AWS console first. Here we are in the AWS console and you can see the running agent. We can select this to view details about this agent, including the versions, the endpoints, a link to logs, and a link to the observability dashboard. It also provides you with a sample script that you can use to invoke the agent using the agent core SDK. I'll go ahead and copy this Python example. Then we can head back to the IDE and we can create a new file and call that file invoke agent. py and we can paste in that code. Let's go over what this code is doing. It's using boto3 to create a client for bedrock agent core and then it sets up a payload which contains the prompt we want to send in. Let's go ahead and change this prompt to make it something that's more realistic for our use case. Let's say what shipments are at risk of being delayed. Then after that it calls the invoke agent runtime API passing in the ARN of the agent, the session ID, the payload and the qualifier. A qualifier is used when you want to target a specific version or endpoint for your agent. Versioning is automatically supported with runtime and each deployment creates a new version. Then there are endpoints and these act essentially as aliases to specific versions. The default endpoint automatically points to the newest version, but you can create additional endpoints for various use cases like feature testing or pointing to specific environments. Okay, let's go ahead and give this a run. So, heading back to our terminal session here. I'm going to type in Python 3 invoke agent. py and that will give it a run. All right, great. We can see that our agent has responded. That means that our agent is running with a secure connection to the database running on RDS. Now let's head back to the AWS console to explore one more feature, logs. Observability and logs are integrated automatically. Next to the latest version, there's a link to view logs. Selecting logs opens up the available log streams in Cloudatch logs. From there, we can then select the latest log streams. Here, all of your logs are automatically collected and sent to Cloudatch without you needing to set up anything beyond the permissions to do so. If we had any errors thrown during the invocation, they would show up here and we could come here to investigate. All right, that wraps up our agent core runtime walkthrough. We started with a simple strands agent using mock data, deployed it quickly using the starter toolkit, upgraded it to use tools that connect to a real RDS database inside of a VPC, and then move the deployment into a CDK stack so it can run as a production service. Check the description and a link for the full code if you're interested. Now that you know the basics, check out Agent Core runtime, get those agents into production, and drop a comment below to let me know what you're building. See you next time.