GenAI for Java Developers 3: Apps

Key Takeaways

In our last session, Rory explained the core techniques of generative AI. And if you're anything like me, theory is helpful. I'll nod my head, say yes, I understand. But it's when I try it myself that the real questions and the real learnings begin. Hi, I'm a cloud advocate here at Microsoft. And today is all about practice. We're building not one, not two, but three working applications in this episode. Rory is back with us and he's going to be leading us on this journey. We will be brewing up a pet story generator, an offline AI app, and even a calculator service. Three very different applications in just one session. That what makes AI so exciting. Rory, over to you. Well, now that you've finished setting up your environment and we've learned the basics in techniques for generative AI, let's look at creating some fun apps that you can go in and see the underlying principles. So, we're going to start as we did before on the generative AI beginners Java repo. And as we did before, you're going to go in and create a GitHub code space. So, we're going to open up that code space and everything's set up for you via the dev container. And I've already set up a GitHub token to make sure that I can use the free tier of the GitHub models. So, once this is started, we're going to go into the chapter that we're going to define our apps. So, we've got our practical samples there. And the first app that I want to show you, there's only three apps, is the calculator app. Now, the calculator app is very interesting because it uses something called MCP. So, let's go start it up. So, let's go into the MCP server application and we can just start it here or you can start it via command line and everything's set up already with the Java language server and Maven. And once we start it, it's going to go and register a tool. And as we saw before with functions, tools are great because they give you the ability to do things with your LLM in a very defined manner. So we're going to just close that there. And if we go into the service here, the calculator service, you'll see there that we've annotated it there with at service and then at tool, which is a MCP mechanism to say, hey, listen, here is a tool that I want you to call. And this is a calculator service similar to what we saw with the functions. And it just goes add. And we've got all of the other services here. Now, calling this is pretty simple. You can either call it via command line. We've got some nice test clients here and I'm going to use the lang chain forj client. And what this lang chain forj client does is it actually lets you talk to the calculator service. You can see there there's the calculator service. I'm using secure sockets events. And then it says wow get me the tools and calculate the sum of 24 uh.5 and 13 using the calculator tool. Now, this is different to normally calling the MCP service cuz what we're doing is we're injecting a GitHub model. So, if you go here, you can see there uh let's go find there's the GitHub model. We're injecting GPT40 mini which knows how to talk to tools into the MCP service. So, let's go in there and let's run that lang chain for Jane and it should actually come back to us and say, well, you're talking to an MCP server. And after this, I'm going to show you a very easy way to generate the exact code that I just did. And there we go. There. What's the square root of 144? Ah, the square root of 144 is 12. So now to test this, I can go into one of our extensions. You need to install it. It's called the AI toolkit. You can go into the extensions and install the AI toolkit. [snorts] And in the AI toolkit, you'll see there that I've got a little MCP server uh tab there. I want to go and start that up. I want to start that service there. And you you need to have the the Java service running. And then once you have that, you can go into the calculator service. You can right click on this. I know. Look how cool this is. And you can go connect to agent which will allow you to build a agent that calls your tool. What do you mean tool Rory? And like where where is this tool instructed there? So we're going to use the nano to it and um the nano uh LLM to make sure we don't get rate limited. And then when we go into the tools here, look at that. I've got my tool list there. Calculator. I can actually go in there and edit the tool list and it will show me all the tools. There's add, add add, add, divide, all the tools that I can get there. I can also go in and what we saw in the previous episode is you can go in and create uh functions. You can go there's the get weather one that we demoed. You can go there and uh do that. But we want to see with MCP. So I've got my calculator tool there and I now I can generate the code like I saw there. I can go into the OpenAI SDK into Java and I can generate all the code that I need to actually go in and call my tools and have an end to-end agent app running. So, let's not save that. And now I want to just uh go into the playground there. I got one there. Um calculate uh let's add that there. 500 + 5,000 using the calculator tool. I hit hit enter. And now it's going to, and remember this is in a a code space here. It's going to go say, do I have a calculator tool? Oh, there I do. I've got the calculator tool add. And you can see there the inputs A and B and the output the sum of and it's calling it exactly like I would with the lang chain for J. So this is MCP. You can go into that example there and you can play around with it. The next app that I want to show you, it's pretty exciting. And these apps allow you to go in and create LLMs on the front end. So, let's go into source here. Let's close that MCP calculator. We'll go into main Java and I want to go into the pet story application here. And I want you to to see what what is going to run because I'm going to run an application in a in in in this uh code space. What this application does, it creates pet stories, stories of your pet. But the the novelty really is that it uses and I'll show you the app now. It uses a builtin LLM in your JavaScript in your browser. So if I go choose file here and let's choose the multis poodle here and then I go analyze image and it's going to analyze the image but it's doing it in your browser. Cool. We've got the classification multisterior. Now we can generate the story and what this is going to do it's going to take that generated story and it's going to go push it to GitHub actions. Now, GitHub actions is great except the problem with GitHub actions and we saw it throughout this uh this series is it can be rate limited. So, you want to actually go in and also use Azure if you are uh in production. Alternatively, check there's the the pet stories um there you can use something called Foundry local. So, let's go into uh local and I'm going to call my command prompt here. Let's go in and stop that all there. Now, I'm going to go out of my code space cuz I need to run this locally. There isn't currently a Linux install for this. So, we're going to go into Foundry local. And if you haven't got it installed, you're going to install uh let's go into there. And you're going to install it with simple windgit install Microsoft Foundry local. And what this installs is a backend LLM. We saw with that other example in the JavaScript that you can actually get a front-end LLM. And the front-end LLM is actually embedded. If you if you see it here, we don't want to do that. Open recent. We want to go into here. The front end LLM is actually embedded in the pet story here in the the HTML. So you see there there's the index.html. Let's close that up there. And it pulls in. Let's go in there. It pulls in an LLM model from hugging space Zenova transformer, but I want one that runs in the back back end. So that's great. It runs in the front end. But with LLM local, so let's go into practical samples here. And you can see that there's sorry, Foundry local. Foundry local. All I have to do is start Foundry and it will pull in the exact model I need and also start running it. So if you want to change the port, you can change the port. There says service started and it gives you the port. Loading the model. knows what model to run and sees that I've got a GPU already there and I can go uh tell me a joke why don't scientists trust Adam because they make up everything. So this is running locally and I can download an a a lot of models there. Now to communicate with that it's pretty simple. All I do is I use the OpenAI um SDK and I can go in there now and I can just run this. So, not only can I run a LLM in the front end, but instead of GitHub models, I can actually run Foundry. And there it is. It's saying um what is the model? Hi, I'm Fire an AI language model created by Microsoft. So, we've seen three different ways today to actually create apps. We saw how to add tools with MCP and then you can generate the code from that and you can even go in and create an agent from that. We also looked at how if you wanted to augment it with a front-end model via the um the the pet story application and then finally we looked at how to use Foundry Local if you want to augment the backend model. And these are common practices that I've seen in the interweb of how people can create their apps, add tools with MCP and then augment it with models and through the pipeline create end toend applications using GitHub models using Azure and also using Java. >> Thank you so much Rory. I appreciate so much the level of detail you went into into your session, but not just that, how fun and entertaining you keep it the entire time. For everybody who joined us for this episode, if you would want to visit resources related to this episode, you can find them at aka.ms/java and aai for beginners. Link is in the description of this video. We'll see you in the next episode.