Build Intelligent AI Apps: Build an Agent

Key Takeaways

Sometimes I get so absorbed in my work that my coffee just sits there. But imagine if it could order a refill on its own or warn me when it's cold. That's exactly what an agent does. And today we have Julian joining us. He'll walk us through building agents that don't just sit there, but actually act on our behalf. I think that's super cool because I could use an agent or two. Julian, please teach us how. Over to you. >> In this second video of our Java and AI basics series, we'll focus on creating AI agents using Java and longchain 4J. So what is an AI agent? It's a program that can perform tasks on the behalf of a user by understanding natural language commands and taking appropriate actions. The initial power of such an agent lies in its ability to interact with various tools and APIs in order to accomplish something complex. But their true power comes when different agents work together and combine their unique strengths. In this video, we're going to use three agents and make them work together to achieve something in common. So the first agent will be an evolution of what we created in the first video. It's an author who is able to write a poem for you using an LLM. The second agent will be an actor. The actor will be able to transform a text, so this poem into an audio file. For this that agent will use another LLM and will use a tool which is able to transform a text to an audio file. The author and the actor will need to work together and for that we need a third agent which will call the supervisor. The supervisor will be able to coordinate them and orchestrate them so they work together correctly. Let's have a look at how this is using a whiteboard. So here we are the user. So you would be here on the left. Let me draw a little person here. And you're going to ask the supervisor, please write a poem for me on this specific topic. Then the supervisor can orchestrate the auto and the actor to achieve this goal. There are two ways to orchestrate those agents using 4J. Either we use what we call pure AI. So the supervisor will use an LLM like AGPT5 and using that LLM it will decide by itself which aent to call first, which a to call second and if they call them at all. That's probably the most powerful way to use AI agents. Uh and that's what we call pure AI. The second way to use uh the supervisor is to use an API which is pretty rich in launch 4J and that that API describes the workflow. So the workflow that we would use here is a sequence. We call first the author and then the actor. That API is of course rich. There's more complex workflows than just sequential calling. You can do loops, you can do parallel callings, etc. Uh but you get it. It's either you use AI to orchestrate your agents or you use a workflow. In this example, which is going to be very simple, we're going to use a sequence. So we'll ask the author to write a poem and we'll ask the actor to tell that poem. Now how do those agents work the author and the actor which are let's say sub aents for the supervisor. So the author will use GPT5 mini here to create the poem. It will get it back and it will send it back to the supervisor. Once the supervisor got the poem it can call the actor and the actor will use another LLM here mineral 3B from Mistral. We're using here another LM just to show that it's possible. And why would you use another one? Maybe because it's more accurate for what you're doing. Maybe because it's faster, maybe because it's cheaper. You got different reasons to change your LM. And the LM is linked to an agent. So Mral will come back to the actor and will use a tool called MAR TTS. We'll detail what it is just afterwards. But that tool is able to do text to speech. That's what TTS means. and it will transform the poem into an audio file will come back to the actor which will come back to the supervisor and which will come back to you with a finalized file. Now this all work together also because thanks to launch 4J you got a shared context for all those actors. So they will share the text of the poem the audio file etc. So they can answer you and work together. Now let's cut this and understand better how this all works. So let's go back to where we stopped with the first video. And on the first video, we stopped here with a command line runner, which was calling Open AI and sending back the poem. So we're going to do something a little bit more complex, of course. Now we're going to transform this into our first agent. Now, how do we do this? Well, first thing is that we need to add a new dependency to have a support in longchain 4J. So there's a new module in launch 4J which is called you guess it adjantic. Here it is. And I'm just going to refresh Maven to be sure that my class pass is up to date. Now we've got aic support in launchion 4J. Let's code our first aent. So I'm going to say new interface and that first interface will be the auto. So we'll call it auto agent. Uh so the auto agent what does it does? It creates a string and it was a poem for you. Uh instead of putting a name, let's put topic. So we'll have a topic and it will write a poem on that topic. It will send back this string. Let's just come back here. Welcome to demo application. What is your topic and it will come back with the topic here. So let's now configure that agent. Here you need basically two uh annotations from 4J. The first one is called user message. So that's the message that our user will send to the LLM. And so the message will be something like this. Write a poem about this topic. Uh so here autocomp completion does not understand yet the specifics of the uh template language that we're using here. So you need two curly braces and not one. We'll see that it's going to learn and so next time it will work correctly. So write a about this topic. That's our first user message. And we can also add a second uh annotation here just called agent just to tell the agent who is and so you are a poet. Yeah. Write what to about with this given topic. We could be more specific. You could be like a 19th century poet, a romantic poet, that kind of thing. And the user message is what we ask it to do to write a poem about this specific topic. So the first agent is done. Let's configure it. So auto agent. So for this we're going to use the agentic services from launch 4G. Uh we'll call it we tell it to create an agent using the the interface that we just created the auto agent. It's going to use a chat model. So the one which is above uh using GPT5 mini and uh well we're going to build it. And that's also that's not all. We also need one more thing an output name. So this is going to output our poem. So let's call the output poem. So that's what I called just before the shared context. So the context first will have a topic. That topic will be sent here and it will give back a string and that string in our shed context will be called poem. That's the poem that we want. Let's write the segment agent now. So the uh the actor let's actor agent which is also an interface. So that actor what does it does? Well it's going to give back a string which could be like the file name. It's not very important. uh and it will u transform this poem to an audio file and yeah it will take po as a string. Let's add two annotations like before. So the user message so let's tell it uh transform the poem into a node file. That's pretty good. And again it understood, oh this was wrong and it's missing here a quote. So come on. Yeah, this is good. Uh yeah, Copilot messed up a little bit, but then as you can see, it understood my new templating uh system that we just used before. So it's pretty smart. So transform this P into an audio file. And this agent is a voice actor. So you are a voice actor. Yeah, we don't need that because that's basically what we going to ask it to do. So the agent is what it is and the user message is what we tell it to do. Let's configure it like we just did before with with so actor agent. So agentic service agent build. So it's not going to use the same model as sorry actor agent. It's not going to use the same model as before. We said we wanted to use ministral uh uh 3B. So let me just copy paste this. Create a new model here. Let's call it min and meral. No 3D. Uh, so we're going to use Mistral as our LLM and we're going to need a tool to transform our poem into an audio file. So let's add a tool here, new text to speech tool. And we're going to need to code this one, of course. Let's create the class and have a look at how we configure a tool using launch 4j. So for this we're going to use as we explained in the introduction we're going to use marts. So it's a Java program that transform a text into a voice. It's written in pure Java. So I'm using it because I'm a Java person. But here we're just going to use it uh running on the side. So it could be anything. Uh I've already coded the integration. Let's have a look at how this works. Let me just take the code here. Here it is. And here it is. So first of all, we're going to use Docker to run Marts inside a container. So let's open up a terminal. Let's copy paste this. And Marts should be running now inside the container. If we use Docker Air, go to dashboard. Here it is. It just started. Wonderful. Now what are we doing here? So we've got a notation at tool. This is used by launchion 4j. So we tell 4j this tool converts the provided text to speech and saves it as output.wave. So this this is what the tool can do. And then this is of course specifically coded to use Merry TTS. So basically we send the text file uh through an HTTP uh request and we get back some some stream and we write that stream to output.wave. So we'll have here an output output.wave file which will be created. So let's let's go back to the uh to the configuration here. So the the actor agent now uses ministry as an LLM and uses our text to speech tool to generate the file. Now we need to link all of this together. So for that we're going to use our supervisor. So here is how it works. Uh so this time I'm not creating a specific class for this. Uh it's something very generic. I'm going to use an untyped agent. I will call it supervisor and it's going to be using second sequence builder. So the sequence builder will be a sequence of the two actors that we just the two aents that we just configured before. Those will be called here sub sub aents. Subent one is the auto. Subent two is the actor. And we don't need a tool. We just need to build that. And now in order to run this, we're going to create a context that will be shared between all those agents. So let's create a map for this. It's a map of string and objects. We'll call it context. Yeah. And let's import this. So what's happening here is that we create a context. In the top context we put a first item which is topic. So topic is what we have here. So that's the topic of the poem. Once this topic is transformed into a poem here there will be of course a new item in our map which will be the poem. It will be sent to the actor and the actor then will create the wave file out of that poem. Now let's run this supervisor invoke the context and let's run this to see how it all works together. So it's asking my topic. So my topic is the Java virtual machine. And let's wait a little bit. This should create an output. a file here with our poem told by the voice actor and that should be pretty fast. So yeah, don't expect that poem to be wonderfully said to have something a bit fast running inside Docker. We we're not having a very good texttospech system. So the output is air. Let's call it VC. So, I'm using VC, which is a cool French open source uh uh software to run to listen to music and files. And here it is. Let's listen to it. >> Beneath the glass and icons, humming in the dark, a quiet engine reads a language of steps and sparks by code minus neat soldiers folded. Now that we've seen how we can create a function for let me summarize what we just saw. So let's go back to the dashboard here. So again what we saw is that we can make a work together and be orchestrated by a supervisor agent. Those agents can use one LLM and can use one or many tools. They should be pretty specialized. So they're going to use an LLM which is specific for what they want to do and they're going to use the minimum set of tools. If they have too many tools, they will be confused like anyone and those tools are there to help them act and create something on your behalf. Uh and then also just to to to summarize what's very important here for the supervisor, there are two ways to super to orchestrate your agents. Either you use pure AI, so you've got another LLM, you send it the text and it decides which agent to call at which moment or you do like I just did here. You we have a workflow API which is more direct maybe a bit more simple but still pretty rich for normal usage and that gives you more control on what you want to do. So we are coming to an end of this for this video. Thank you so much for following it and see you in another video. Thank you. Goodbye. >> Hey Julian, thank you so much for showing us how we can easily integrate AI into our own applications. If you also want to learn and take your first steps to integrate AI, you can go to aka.ms/java and AI for beginners to find resources. It's also linked in the description of this video. We will see you in the next episode.