Intelligent Apps with Azure Kubernetes Service (AKS)

Key Takeaways

hi my name is Paul you Cloud native developer Advocate at Microsoft and in this short video I'm going to show you how you can run your intelligent apps on AKs we'll start by talking about what makes an app intelligent and then we'll talk about why you would want to run these apps on a platform like AKs and then I'll show you a demonstration of a brand new quick start tutorial that we've recently published on the Microsoft learn site so what makes an app intelligent well to keep it very simple when we say intelligent apps all that means is that your app takes advantage of artificial intelligence to enhance or augment the user experience generative AI has exploded onto the scene and organizations Microsoft included are taking advantage of these large language models and building capabilities on top of that via copilots chat Bots and whatnot we're starting to see a paradigm shift where natural language can be effectively used to enhance the user experience but just because the Paradigm is starting to shift that doesn't necessarily mean that you have to start from scratch by using Cloud native architectures such as microservices you can easily bolt on these capabilities to your existing apps as well and there's no other better platform to run your Cloud native apps than AKs AKs offers you a platform that allows you to deploy scale secure and manage all of your Cloud native deployments with with ease it offers you a complete end-to-end Cloud native development experience AKs also integrates seamlessly with other Azure services including Azure key Vault Azure storage Microsoft Defender Microsoft enter ID just to name a few and AKs is cncf certified to be kubernetes conformant what that means is that all the open source tooling that's available in the kubernetes ecosystem well you can bring that onto your AKs cluster as well things like istio cada Dapper Etc they can be installed using the open source installer packages or you can take advantage of some of these AKs add-ons where we will install and manage that for you bottom line is that the AKs platform will give you a pretty comprehensive Cloud native development experience to host these intelligent apps now let's switch gears just a little bit and talk about our demo scenario and the business problem that it's aiming to solve with the new quick start we actually introduced a brand new sample app called AKs store demo this app is taking a little bit of inspiration from another sample app called red dog that was actually developed by our gbb friends at Microsoft and in this particular scenario it's just a pet supply store where customers can log in view products check out and whatnot but we also have back office capabilities in the business scenario here is that we want to be able to reduce the cognitive load on some of our employees and be able to lean on open AI models to help us generate product descriptions and these product descriptions can be generated using the name and keywords and so I'll show you how we can literally bolt on AI services and enhance the employee experience in this case here's a simple view of the application architecture it all starts with the customer they visit the storefront where they can view products and add them to the shopping cart the product catalogs is retrieved from a product microservice and there's a simple checkout process in the store where orders are submitted to the order service the order service in turn sends the order to a message queue for later processing here in this case we're using Revenue queue from there we have employees who can perform an administrative tasks from the store admin site the employee manages the product catalog and so this site is also connected to the product service the employee also needs to be able to process orders and so here we have a service called the make line service where it actually retrieves orders from the message queue and saves the order to a persistent database in this case mongodb now the whole point of this quick start is to show you how you can add intelligent capabilities to your existing app and run them on nks and so we've done that by literally bolting on an AI service that will use open Ai and help us generate product descriptions here we're using the AI service written in Python and using the semantic kernel SDK and the SDK allows us to orchestrate interactions with open AI models and these open EI models they could be either on Azure open AI service or they could be on openai directly now that we have a good idea of how the application is constructed let's deploy this to AKs so we're going to follow this quick start guide and deploy our sample app to AKs to do that just open up a browser Tab and just do a quick search for open Ai and AKs and it should be the first item that you see on the list here click on the link and it should take it to the document that's titled deploy an application that uses open EI on Azure kubernetes service one of the important things that you need to know is number one obviously you do have to have an Azure subscription but number two you do have to request access to Azure open AI if you're going to be using the models on Azure and so what you can do if you haven't already done this is just go ahead and click that link and fill out this form it's basically a responsible AI Checker just to make sure that your intent to use the model is all is all good and whatnot from there we have step-by-step instructions that actually show you how to deploy this application and it all starts by creating a resource Group creating the AKs cluster and deploying the sample manifest to deploy this application that you see here now we're going to deploy everything that you see in the Box except for the two components down below the open Ai and the AI service let's go ahead and start our process here so once you have access to Azure open AI what you need to do is go into the Azure portal and just do a search for open AI and you should see the open AI logo and then we can actually create a brand new open AI service so what I'll do here is uh I will create a new Resource Group call it my Resource Group and I will put it into a region and I'll give it a name select the pricing tier click next just click through and after validation we'll go ahead and create the deployment all right so it's completed the next thing that we'll need to do is we'll actually need to go into the Azure open AI account and we need to deploy our model what you want to do is go here to model deployments and then click the manage deployment button this will open up a another portal where you can manage your openai models and also use the chat playground and whatnot so we'll create a brand new deployment select the model and what we want to use here is the gpt35 turbo model and the model does require a or the deployment does require a name on Azure open Ai and so I'll just use the same exact uh name as the model click create and we should be good to go there now one thing to note in a in The Next Step we're going to need the the keys and the endpoint so just make sure that you have this stuff handy what we'll do next is we'll actually create an AKs cluster in the resource Group now rather than creating this in your own terminal you can literally click click this green open Cloud shell button what this will do is it'll open up a Azure Cloud shell right here in the dock so you don't have to bounce back and forth between web pages so that's pretty handy so let's go ahead and do that so what I'll do is I've literally just copied this command azaks create command to create the AKs cluster all right our cluster is deployed we can actually connect to the cluster now so let's grab this command to do a z AKs get credentials okay so now I should be able to run my Cube CTL get notes command and I should be able to see my nodes you can see they're all ready now I can actually deploy the application and as stated we're just going to deploy the top half for now and then we're going to deploy the AI service and have it connected to our open AI model in just a little bit so scroll down here and let's grab the yaml Manifest to deploy all those resources and you can see it deployed quite a bit of resources for us and let's just make sure that we can test that so I can go Cube CTL get deploy we can see that they're starting to come online so we'll just get that a little bit now there are instructions for deploying open AI we've already done that but it does say that we need to grab the key and the endpoint and we need that because we're going to need to fill that in into our yaml manifest so the AI service deployment manual manifest there's a few pieces of information that is missing here and so we need the openai deployment name and then we also need the openai endpoint and the open AI API key if you're doing this from openai directly you can just click this tab for open Ai and you can see that all you need there is just the API key and the org ID well we're doing this on Azure so let me go ahead and copy this yaml manifest and we'll make some edits to it so we'll create a new manifest called AI service .yaml and let's open that up in our lightweight code editor here and let's paste in the Manifest okay so what we need is the deployment name I know what that is that is GPT three five Turbo because we give it the same name and I need the endpoint and the API key so let's go back to the Azure portal and I can grab my key put the key in here and then I can grab my endpoint copy that to clipboard and then paste that in right here okay now that I saved and closed the file I can apply that yellow manifest so let's go ahead and do that okay so now we have the two Services created now we should be in a pretty good state so let's just do a qctl get pods we can see that everything is running except for the AI service so we'll just give that a few seconds to come online what we can do in the meantime is we can go grab the IP for the store admin site and we see our admin portal is here and we have a list of products we can also grab the public IP of the storefront just to see what it looks like for our end users and this is or the storefront looks like for customers so they can actually come down here browse products and add them to the shopping cart and submit the order just like that now from a admin perspective we can actually hit the add product button over here and it would give the product a name because remember it requires the name and keywords to generate the description and so let's just do this um rough and tough chew toy and let's give it a price and let's give us some product keywords so it's a dog it's a chew toy notice here we have this button called ask open AI what this will do is it will call out to the AI service and call out to the open EI chat completion endpoint so we'll go ahead and ask it there we go so it returned back with this nice well-crafted product description and we can hit save the product we can see it's here if we go back to the list we see we have a brand new product here and if I reload my product page over here you can see that we have a brand new product available now that you've seen a simple yet common use case of adding intelligent capabilities to your app it's your turn to explore a bit more here are some resources that I think you should check out next number one go try the quick start for yourself go through the dock and let us know if you have any questions or comments about it if you're not really familiar with kubernetes no worries check out the kubernetes learning path that we have it's a curated list of resources that you should check out that will well put you on the right path I've also like to note that we will be hosting a webinar in the month of August at the Microsoft reactor where we'll walk you through how you can deploy your apps to Azure container apps and Azure kubernetes service if you are familiar with deploying your applications to containerized Services already and you just want to figure out what more can you do with let's say a semantic kernel SDK check out this blog post the symmetic kernel what it is and why it matters because there's a lot more than you could do with this sort of technology in the SDK there's things that you can do like embedded search using Vector databases in and whatnot and also we have a couple of other blog posts that's floating out there around how to deploy Azure open Ai and chat gbt on AKs using both terraform and biceps so be sure to check those two things out hopefully this is enough to get you started and if you have any questions just feel free reach out either in GitHub issues or on Twitter and let us know I can't wait to see what you build next thanks