Writing LLM Apps with Azure AI and PromptFlow

Key Takeaways

foreign [Music] [Music] thank you [Music] [Music] foreign [Music] thank you [Music] questions [Music] [Applause] hello and welcome to this episode of the AI show how is everybody doing today my name is Seth Juarez and uh today we're going to talk about large language model apps we've been um playing with prompt flow for the last couple of weeks today I'm going to show you something new hoping you'll like it to get you some docs and get your feedback here so my name is Seth Juarez I'm excited to be here where's everybody coming from tell me where you're coming from but before we get started I'm going to turn the music down here a little bit maybe even off I do recognize that today is September 11th I just wanted to make sure that everyone recognized uh you know that was uh I still remember what was going on that day I was driving into work I worked for a company called Holman's of Nevada I was a senior software Dev I led a team of Engineers of Consulting software Engineers I remember driving into work I live in Las Vegas at the time and I remember hearing stuff on the radio that sounded weird so I called in and I asked what's going on and they said they told me what was going on and uh proud New Yorker here I lived in New York from when I was uh you know I was like eight until I was 14. I went to PS 94 Public School 94 and is intermediate school 97 ending as prawl and so New York has a special part of my heart and so I just want to recognize that today is uh 22 years since it happened so I just want to bring that out and so if you if you're wondering yeah it's clear to me but life goes on and we want to make sure that we we make those of us that sacrifice that day proud of what we do okay so I just want to bring that up because I I just don't want to be like I said I I lived in New York not at the time I was in in Vegas after but uh it was it was difficult for me so I sure as it was difficult for folks but I just want to I want to recognize that right off the bat and so let me put the music back on I remember those uh that passed away with fondness and those that heroically uh helped that day all right all right let's see where everybody's coming from my friends um oh yeah um yeah let's reminisce a little bit um uh here's Sam I was in New York City as well and saw the Smoke on the sky it was my 60th 60th day in the U.S this affected everyone yeah uh thank you Sam um I was in seventh grade that day yeah New Jersey raised so yeah all right let's see where everyone's coming from uh let's see we have somebody from Raleigh North Carolina never forget indeed Austin Texas hello my friend Atlanta New Jersey UAE welcome my friend uh Detroit uh Detroit Detroit area the Netherlands Austria Seattle I'm so grateful for everyone spending their time with me this morning or I'll make it worth your while today uh I'll make it worth your while today we have some really cool stuff to show you today and I say we I mean me uh uh London homeroom that's awesome turn it up uh Hungary brasileiro here welcome my friend Belgium uh thank you for coming uh I thought we'd start today um someone also gave me some feedback uh I'd love to hear your thoughts on accessibility for this show they said it was difficult to um to pay attention when there's extra music and you know what I I understand that 100 I have a hard time paying attention because there's always seems to be music in my head so having extra extra music is probably hard uh so I totally get that uh so I'll let this uh this music play out and then we won't have that much music afterwards uh Surly Dev is here my friends hello hello my friend welcome oh Greg legon 11 Hagen my friend I have not been I have not I have not traveled anywhere for work in a long time it's been I think it was 2020. the last assignment I had is I went to California to visit uh produce production lot to help them with something that changes tomorrow I have to go I have to go to New York for one day it's gonna be great I'm excited uh and so having said that uh I used to visit with Greg a lot when we I used to go back to the Midwest to a wonderful conference called that conference yes that conference it's great all right so let's let me tell you what we're doing today uh just so we can get a feel for it I've got my handy dandy pen here uh today uh what we're going to do is we're going to look at a we're going to look at a new a new Tool uh for uh building llms llm apps not llms themselves but apps and this is important right because it's important to recognize what's the difference between building an llm and building an llm app it's not the same thing not the same thing uh number two uh we'll uh make something with it make something uh and then number three I'll take your cues uh questions and answer them live on air and then I'll show you a couple of community things that we can do uh when uh when we're offline and trying to trying to work with something called prompt flow I may put music on and I'm going to put it really really low because it feels like hello any hello kind of feels like that a little bit so I'll turn something on but I'll put it really low super low there you go at least I can hear it if you can't hear that's okay yeah you heard the echo Sam that was on purpose I was I was do it I have I have a thing that lets me change my voices okay uh so uh without further Ado let me go to uh vs code I have here um uh can we have a recording for those that work absolutely yes yes uh this will be recorded um um that will be recorded so feel free I'll make sure to tell you where so I want to show you that I'm actually starting from nothing uh and I'm starting from Visual Studio code Visual Studio code okay so I'm a huge fan huge fan of visual circuit so there's a new extension uh that I want you to check out uh right here called prompt flow for vs code they like secretly released it uh I want to see this number go up to like I would love it if like at the end of the month this thing was like at 5 000. and then in the next month it was like a ten thousand so I mean we all have homework now 465 installs look two people said it was great was one of them to me was it me huh these dudes I work with them I want to see your reviews in there as well uh prompt flow for vs code uh which is um pretty cool I feel like I need to like prom flow for vs code I feel like I need to be like [Music] [Applause] yeah let's install it let's install it let me let me turn this down again and put the put some music on here okay uh so I'm gonna install it um prompt flow problem flow no extensions just really look at it search extensions prom yes there it is there it is so I'm going to install this now there you go uh so now that it's installed uh I have prompt flow and so you should see uh you should see this uh little thing over here um this thing this is such a cool icon look there's a p in there yeah okay let me turn it down a little more prompt flow okay so now that we have that we have this thing here uh the first thing it's going to tell you is that um it's not installed so let's go to okay so this is a python thing so the first thing we're going to do is we're going to like I said this is all from scratch python minus M uh I like making virtual environments for all my stuff uh yes uh you did notice that I made a new thing yes I want it to be the actual thing let me move this over so that Control Plus this is much bigger nice there you go now my face fits just nicely in there okay so I like making virtual environments for stuff and I don't know why this happens but you have to reopen it for the inverse and so for those wondering this is prom flow is a python python thing and so for it to work um it needs to have a python environment and I don't like polluting my my base environment and so I you can see here I uh I made a new one and the other thing I do is uh obviously uh git ignore hit ignore uh another question does it work with conda absolutely it works with con any python python thing so the first thing I'm going to do is I'm going to dot venvy this thing cannot uh be in there and then I'm gonna say get uh get in it because that's what we do that's what we do when we're making projects by the way if it's too small let me know this is just my visual studio code I keep this on the right uh because I don't like this thing moving around uh so uh all right so that's the first thing we do it looks like to make this thing work uh we're gonna install some dependencies so I'm clicking this uh for for to this is the instructions or conda to manage environments okay so notice that you can see conda here absolutely uh so this is to make condo work for Activation or you can activate your virtual environment notice that icolumn.bam you can call them whatever you want but I do that too and it looks like we need to install a couple of things so I like to do this requirements.text okay uh and what I'm gonna do is I am just going to take these two goodies and I'm going to put them in here uh because I like to have everything documented as I go so that this can be reproduced um so there you go um there's that so I'm gonna say uh pip install Mine by the way every once when I look over because I'm seeing all the all the things you're saying in the chat I want to make sure that I am attentive to that um pip install pip install oops I'm in the wrong spot pip install minus r requirements.txt and so now I've got this thing going uh I put them in here by the way prompt flow and prompt flow tools these are these are all you're probably wondering where can I look at the code for that um I'm glad you asked uh let's see uh GitHub I'm gonna google it with Bing there it is uh here it is it's public you can all go look at this now um come complain at us here there's tons of places to complain at us oh uh yeah go in there show us what you want uh you can see it's all here there's also some documentation here for those that are wondering uh I'm just I'm just gonna go off the cuff if that's okay all right uh another question that is a good one what app do you use to zoom and draw arrows in your presentation and you zoom it Zoom it it's a great great tool okay um okay so uh let's go back to visual studio code here uh there you go oh I got it in Python dot XC um oops uh python dot XC by the way this is all happening I'm not this is not like okay this is not like a recording so you're gonna see me mess up and that's okay okay so now in theory uh prom flow should be good to go so let's refresh nice nice um nice let's see this nice so now we're good uh so when I go back uh hopefully everything will work okay so there you go all right so nice so now what I'm gonna do is I am going to create a prompt flow from scratch artisanally um artisanal from scratch so let me do this and the way you do it is you right click and say new flow in this directory so let's do that so new flow in this directory I'm going to do a chat flow because that's what I'm used to and um files with the same name are oh nice nice what is is it doing stuff oh it deleted that I don't like that so prompt flow and prompt flow tools that's what it was before cool beans so now notice it's dropped this dot prompt flow thing in there uh that's interesting and it looks like there's some tools in there so okay so there's a lot of files that it's dropping and you're like what is it doing so when I click on this notice that the prompt flow is a um it's just a it's just uh what can I say it's just a yaml description of a of the thing so it looks like I got the right environment looks like I might need to restart because it's still thinking about like everything so I'll I will restart this here so let me open it again uh put it over here nice okay so let's see if this changed yeah so now notice that that things look different uh you're also going to see that I've already done stuff in there and that's okay uh I just wanted to show you from the beginning what's going on okay so uh notice that this is basically now a graph that represents your prompt flow uh when we were looking at this uh we thought uh editing this is gonna be a pain in the behind so we have this these little helper things here that lets you like select the environment so for example those that asked about conda certainly works with conduct but I'm using the virtual environment uh but you have this thing called the visual editor and all the visual editor does is it manages the prompt flow directed acyclic graph a dag because notice that information from a uh for a prom flow has to flow top down into a call and then out all right so let me click on visual editor and this here is the prompt flow that you're going to see right uh this is uh this is it I mean this is pretty cool because again all the files live locally and you can do everything that you've seen before but locally it's important to recognize that this visualization is to help you make this yaml file that describes the graph okay so the first thing you need to do is we need to connect it to Azure open AI so to manage Connections in prom flow you go over here and you go to connections and if you want to add a new connection you hit plus and I'm going to connect it to Azure open AI and this is cool because like um here let me zoom out it's a little big what you do is you name these things you give it the key in the API base and then you hit create connection but the important bit um the important bit and this is really important is this part right here the connection information will will be stored in a local database with API encrypted for safety prom flow will only use the connection information on the keys when instructed by you EG manage connections use connections to run the flow so even though it looks like you're making a yaml file and you're saving it with your stuff you're not that's not what this is doing and that's important because that folks might be like well wait a minute you can't do that well yeah you don't check this stuff in I've already created a couple of connections myself and so I'm just going to use them I'm just going to use them but this is the way I created them and the way you edit them is you right click and you say update connection and what comes out is this exact same thing with the things filled out but the API key is is start out so I don't need that so I'm going to close that uh one thing that that we've found that we're probably going to fix is to create the connection you have to click this thing right here if you didn't see it right away that's okay don't worry just hit create connection okay so I've already created the connection so I'm just going to click this and select the the um select my thing and notice that you have the exact same stuff from before notice that you have the function calling as well uh built in here you have the advanced parameters like top P frequent penalty logit bias is it Logan or Lodge I say logit okay and then you can see that if you've used prompt flow before you take the inputs from like above so inputs.question you can see it's in the drop down box the other thing that's important is this notion of um this chat thing um what is the actual prompt for this well notice that we are using Ginger files and so you can see exactly like this is the actual prompt that's in there so if you go here and then you go to the prompt it's it's this thing and it's Jinja and if you don't know what Ginger is it's like it's cool stuff um uh it's it's like a cool way of like doing like I think it's used in Django to do HTML um Generation Um so yeah and yeah so there's a good comment so conveniently you could use git secrets to keep it safe even when having this code in control no keys or yes the reality is that this this uh these connections are not stored in the project you don't see any they're stored like in the prompt flow you know vs code extension area and they're locked up and we don't use them for anything other than we you only use them for what's in here okay uh all right so we've got that uh in theory we should be able to test it now so to test this uh uh let's do this and I'm going to run it in interactive mode so uh let me let me do this real quick so there's a bunch of stuff up here like for example play test debug Etc I don't know even what is this one build uh and uh sort Etc so um I'm just gonna run it and show you what it does I'm going to run it in interactive mode so let's do that so run interactive mode and again I really haven't done very much honestly uh okay so now I'm just in the chat flow who won won the first first Euro Vision Song Contest uh and what this will do is it will go through and do it uh you can see it's it's pretty good uh and it's all running locally modulo the calls to um Azure open AI of course so module of the calls to Azure open AI this will just run um uh who was the best singer ever at the Eurovision song contest um there you go uh so Ctrl C you can also you can also put like I'm Gonna Leave This uh empty on the chat and say who won the year the Eurovision song contest uh and then you can run it in standard mode which I think will just take these inputs and do this thing let me move that out of the way there you go most recent winner as of May 2021 you can start to see the limitation of this thing uh the contest we can't oh okay cool uh so you can see this running the other thing that's interesting is you can see this prom flow API calls and you can see that when I select a node you're seeing the API calls here as well as the the type the inputs the outputs and then any errors right so there's the input uh the output I don't know why the output is saying that so let me go here boom uh very good here's the inputs all right the messages Etc and then the output I don't the output shouldn't be raw uh so I'll have to look at that uh looks like it's a generator is the output uh oh yeah it's a generator because it's it's doing uh it's it's streaming it so that's why you don't see it um right away okay cool beans so let's make this thing more uh I don't know more interesting we're going to go back to our contoso example from before do you remember we did this we were doing this a little bit where we're trying to look stuff up so the first thing I'm going to do is I'm going to go to this thing and I'm going to remember let's see if I can remember the other thing we're going to add is a customer ID so that we can bring in like customers for the customer number six control s um so the question is which extension am I using it's called prompt flow prompt flow so if you go here and then type in prompt flow from flow this one either way um uh I showed it here again our goal homework for today we want to get this thing to like five thousand do it all right um so I put a customer ID in there now what I'm going to do is I'm going to add uh I'm going to add a node here and I'm going to use an uh um an llm tool oh no let's use it embedding let's use an embedding because I want to um I want to embed the question question embedding boom uh and the input is going to come from the questions so we'll use this and the deployment name is the text embedding it's a sigma the best we don't want to do that though we want we want actual people actual people not Bots man okay so there's a question embedding and then I am going to I'm going to do a search this is the same one we were making last time uh then what I'm gonna do is I'm gonna do I'm gonna call uh let me just I'm going to call call your search so uh Richie dokumentos okay so we'll do this in a new file um cool so notice a couple of things notice a couple of things number one when I when I put this thing down you're probably wondering where do I edit the code well the the code is actually in here this is the actual code it's actual real code uh uh and so I don't want you to think like oh it's hiding it no this is all the stuff you check in uh I'm just going to copy some code because I don't want to write it and mess it up in front of you because I've written this code so many times uh this is a uh let me this is kind of one two like I wanted I want it to be big but not so big uh so notice that there is this this uh question that comes in the index the embedding and we're gonna do a search on that okay and return that looks like we have a problem here with our requirements so one of the things I need to add is um why do I need do I have an Azure key credential okay so what I'm going to do is I'm going to go to the requirements.txt and add um Azure search this thing here is a beta 4. this is with the one with vectors and so now I can say something like it's on minus r or sorry pip pip install minus r requirements.txt beep beep beep beep I hope that's okay um is that I did I break it I don't know let's see now uh let me close it and let me open it up okay it says it doesn't see it but let me scroll up and see what the warnings are dude uh Marco renderer work Zug I don't know what that means does anyone know what that means I think it should work let's try it though so what I'll do is I'll set a break point right here set a break point right here uh oh the other thing I need to do of course it's not going to work is I need I need to regenerate no no no it's generating the tool here we go for some reason it doesn't like um it doesn't like um the the things I put in there so this let me let me close it and open it again for some reason oh here we go yeah it's for some reason I do not have the right things in my um there I I'm looking at I have another project open I did see this maybe this can as a problem before so um pip install minus r requirements.tags oh okay looks like that fixed it I don't even know what this is they told me to add that one time um okay so that works let me go back here um question there you go now it works so the question name that we want is inputs not question the index name that we want and if you're looking at this you're like what in the heck is he doing let me let me open up a my Azure cognitive search over here so you can look at it because it's going to ask me to log in I just don't I don't wanna you know okay where are you uh here we go Hubert data we have some indexes and contoso track is the index we want and you can see that the contoso Trek has a Content file name title and it also has like an embedding so contoso Trek is the name of the index so we'll go down here down here contoso track the output is this question embedding and we're going to use this search connection and just a reminder connections are these things that connect you to other things um like cognitive search open AI Etc you can you can even make a custom one I'm gonna make a custom one soon um okay so this is going swimmingly now what we need to do is we need to actually do the llm call oh sorry we need to build a uh prompt so let's go over my bot broke it yeah thank you so now I'm going to go like this and I'm going to add a prompt and this prompt is going to be the customer prompt hurry up we're going to run out of time here customer is all my Yap in here uh nice so notice that this is a customer prompt.jinga and you can change anything in there that you like right I am going to do this um I'm gonna take this out because we can't do this quite yet there you go and so this is the this is the um this is the prompt you're an AI agent for contoso Trek Outdoor Products retailer as the agent you answer question hold on you are an agent for the contoso Trek Outdoor Products retailer as the agent you answer questions briefly succinctly and in a personal manner using markdown and even add some versatile flare with appropriate emojis you should always reference factual statements the search results based on relevant documents search results based on relative is maybe incomplete or irrelevant you do not make assumptions on the search result to be on Strictly once returned wow there's like two minutes of that I don't know if I can keep going uh so this is the prompt and so now when I go back to this flow you can see that it's asking about the the um the documentation and then the history and then what I'm going to do is I'm going to go up here and I'm going to say that the output comes from dupe the oh well no no we got to pass this into an llm what am I even doing so here's an llm tool we're going to call this one um the llm oops I pushed the wrong thing llm response new file Fantastico I'm going to choose this one uh chat API por favor gpt35 turbo a course uh and then this thing also has its own Ginger file uh the reason why because you could put it all in the same place the reason why I didn't is because I wanted to this is this is how you test like various um uh various prompts um and so this prompt text is going to come from above and the question so now when I do this The Prompt text comes from customer prompt dot output and the question comes from question uh okay one of the things I want you to notice is that this element response when the user asks the question I'm forcing the user to also say please be brief use my name in the response reference uh okay but it doesn't know the name so that's gonna break it's gonna make up names just FYI I'm telling you that right now uh and now finally that we've done this let's go up here and the output is going to be the output of the llm response thank you my Brazilian Portuguese is working well oh man my face was covered the whole time with that maybe it made the show better Obrigado there there you go now you can see me okay uh he's like dude you're covering the screen with your goofball face I'm sorry that was I like that I'm like that old guy that's like sitting there on the freeway with their blinker on the whole time you know what I'm saying that's me that's blink I'm blinker guy what is going on Seth uh let's see uh let's go up and change the question one more time here so we're gonna say uh but tell me tell me about your hiking jackets no I'm gonna debug it because I want to make sure I want to make sure I just said let me set a break point right here I'm so dumb I'm like the guy with the blinker on you know the guy right and you're just like is are you're like you're like driving along you know like uh just driving along with your with your blinker on [Music] oh man I should have looked over at the thing tell me if I'm doing something wrong and I'll fix it it's like I'm sitting here driving with the blinker on click click click click click click click just living life living my life in Florida you know with a fishing hat I never go fishing click click click click click all right now let's go to this thing right here and let's debug it I want to show you something cool so when I hit the bug notice that it's actually gonna debug the code um this is the best part um and I think it's you're gonna love this because uh online it was a little harder to do this you could do it but now notice that you're actually doing the thing so here's the the question coming in index name contoso track tell me about your hiking jackets here's the embedding and the embedding so I'm going to F10 this F10 uh I'm getting the vector embedding here Boop and then I'm gonna do a search search oops Ctrl Z uh so now when you look at docs you should see there's a couple of docs there's two of them the rain guard hiking jacket and the trailblaze hiking pants I think you made a mistake there on the search uh but that's okay uh we'll let the llm figure it out so we'll continue this and hit run and the flow will come out the nodes will come out and um let's see what the outputs are let's go to prompt flow here all of these things are now completed wow I need to I need to control minus this it's too too small um and so the inputs come in and goes to the question embedding retrieves the documentation puts the question of the prompt the LM response comes and the outputs the question is how do we call this index uh right here so I'm doing the embedding right here you can see that I'm taking the question and doing it an ADA embedding to retrieve the documentations I'm I did this artisanally so I actually called Azure cognitive search using a search client and you can even see here the inputs is this embedding and then the outputs is this this gloriousness here all right these things so inputs outputs there's no errors there's the node name so we'll go back down uh very helpful stuff here very helpful uh and uh then when I look at the llm response you can see that the output is a generator uh uh which is kind of a bummer um so someone's asking about the index we can talk about this in next episode because we're gonna run out of time but a vector index is essentially just like a search index I stuffed a bunch of documents into it if you want to see like the actual code um I did it really fast using this code so this is the index definition uh you can see it says contoso Trek this is I added some semantic configuration some headers and I sucked in some documents did the embedding and then just push the record into it and then I pushed them all into Azure cognitive search there's a way to do this uh automatically too but this was like 100 lines of code and I was like I'll just write it so I can do it every time so that's how I index them myself okay so there's the output uh wonderful uh now the other thing now that we need to add is we need to add a way for the customer to be represented so what I'm going to do is I'm going to add another python node for a python for a customer customer lookup new file nice uh nice and as I've been doing before I'm just gonna take some code that I've written only because I'm probably gonna mess it up but by the way this is this is everything you need uh to do this stuff and I hope as you've noticed notice that now I need to have a cosmos DB uh package in there I don't have it in there so I'll just add it to the requirements.txt and then I will say a pip install minus r requirements be happy be happy I don't there you go now it's happy and so I'll go over here and this thing should now when I refresh it let's refresh it uh all this does is it looks at like the inputs and outputs so I'm going to say that the thing coming in is the customer ID the connection I'm going to use is Hubert Cosmos because it's a custom connection and actually let me show you this custom connection uh so I'm going to update it notice that all I'm doing is I'm passing like an end point a database ID and a container ID and then this is not not changing but again these connections um uh this is not a file that lives in your in your in your in your app it doesn't live there it's it's it's part of uh uh prom flow so again the connection information will be stored in a local database with API encrypted for safety prompt flow will only use the connection information we'll use when instructed by you EG manage connections use connections to run flow Etc okay all right so let's go back to our flow dot dag Dot geml and now what we want to do is we want to end the prompt inject more stuff about the customer right so uh I am going to take this text right here I'm going to add this in here which is customer context so let me go to let me go to this thing here because I want to show you what's in the Hubert Cosmos database so you can see what data is that I'm returning so let me go to the data Explorer here's contoso outdoor here's the customers notice that the customer ID number six is Emily Rodriguez and she's purchased some things uh you can see that in there uh so that's what's going on there okay now let's go back to my code here I think I did the writing because she's got a lot of purchase yes she's got a lot of purchases so I don't think I need this anymore so I added a cosmos client I'm using I'm using the connection the custom connection to pull some configs out some secrets that's what's coming out of this thing I'm reading the customer ID from the container and then I'm I'm shortening the number of orders because you only have so much space in your prompt for text so I'm restricting their uh their previous orders to just the last two and then the response goes there so let's uh let's do a little debuggeration here let me close this and let me go over here because I think this thing needs a customer ID right that and now this thing this thing I need to I think I just let me did I change the control s so this flow dot there you go uh generating tool metadata it needs a customer now this is wrong so this thing this thing need a customer this thing needs a customer and this customer is the output of the customer lookup.output okay let's do this uh uh uh hi their name thanks for got it right that's cool all right so let's uh let's do a little quick debug no that's not what I want let's do a little quick debug to make sure it's retrieving the right things uh so we'll do F10 here F10 oh F10 looks like it's calling both of these so I can take this breakpoint out F5 oh that I must have hit F5 too many times it finished so what's the output oh there's the output hi there Emily we have a great selection of hiking hold on I gotta make this like a commercial hi Emily we have a great selection of hiking jackets you might be interested in our rain guard hiking jacket it's made with waterproof and breathable fabric has adjustable cuffs and hem multiple pockets for storage Plus as a base member you can enjoy a discount on your purchase [Music] yeah thank you that was delightful so now we have a full prompt flow um going on here let me run it let me just do it in uh standard mode here and just show you that it's running it um all of it there you go nice a very nice okay wow that's pretty cool I mean look at this uh this wow this outputs this output's even even better um hold on hold on even better like hi Emily our ringgard hiking jacket would be a great addition to your outdoor gear collection it's lightweight waterproof and breathable perfect for your hiking needs you previously purchased Trek Star hiking sandals and the Alpine Explorer tent which would complement this jacket nicely we also have a user guide reviews and warranty information available if you'd like more details let me know if you have any other questions that was perfect um that's perfect that's a great answer here's a question so uh so Cosmos DB is being leveraged for Vector search as well as storage of index Docs so no uh uh yes yes and no so I am using a combination of unstructured documentation that's been indexed and structured documentation so for example when you release this thing let's just say this thing you want you want to build it or something the idea is that you're building an API that takes the chat history the question and a customer ID and Builds an llm app and it takes it takes those things and it goes to the prom flow and generates a response that's what this is that's what this thing is all about so um you see it here um pretty cool stuff right um so that's it uh that's the whole thing um in a nutshell oh okay so we we built this like I was yapping more than I was building to be honest so notice that this is the cosmos DB and let's change the user to like Jason let's go to I don't know let's I don't know pick a numbers I'll do seven we'll do number seven here just so you can see that it's actually calling this thing so let me run it and let me just run it in standard mode here uh there it goes it's running it that's hold on let me let me run it again let me run it again because I want to make sure that I get it like really good so it's working yeah it's coming back [Applause] there you go that's it hello Jason Brown thank you for your interest in our hanging jackets we have the rain Garden it has a reflective blah blah blah it would go great with your Sky View crew person 10. uh oh it's like five Sharp it's like you and me were born on the same day brothers or from another mother Well turns out there is this thing called build that you can do I haven't pushed that button yet um the other way you can do is you can push it to uh azure we have a way to do deployments on this stuff ml.azure.com uh you can like here's the same flow uh that I did before this is the same one online actually that's why I could copy code you can actually do a deploy right here as well uh and you're gonna start to see the integration of this with uh this as well great question all right so let me let me look at my notes make sure I got everything going uh make a thing um I did the thing now let's go to some of the links that you may want if you want to learn about prom flow you can totally do that easy peasy lemon squeezy uh this is the docs to prompt flow you can you can get those there I love how my face is covered up it I think it makes this show better you know what I'm saying maybe it makes this show better I don't know uh if you want the prompt flow extension you can totally go here uh this thing let's see if you've done let's see if if we've done some homework here only 460. well it went up a couple thank you thank you uh it went up a couple okay so we got that there uh then we also have the uh prompt flow local uh quick start guide uh you can totally go here click Start this guide will walk you through the first steps using this first step using a prompt flow um I'm gonna go through and fix some of this we write this stuff too fast sometimes uh notice that this in this case I use a virtual environment but this uses a conda environment which is just I know you can see the flow directory you can look at the flow.yaml here let's look at this and and no I guess that's not what I wanted let's dig this a little bit you can see the same thing I did um each of these things relate to these things right here inputs and outputs at the top and each one of these things are in this code here and the reality is that this all this thing does it this view is just a visual representation of this because um this is all this is really the asset and the other thing that's cool is I can get status get add dot get commit minus m initial prompt flow uh oh control C no no no no no no no no no no no I needed there's so much git ignoring I need to do like my cash off with you uh we don't need the dot prompt flow uh or maybe we do I don't know I should ask what this is oh I see what this yeah this is you don't need this uh so I'm going to take that out how do you undo something with Git un uncheck in let's go here how do I go to the folder and say undo uh on stage changes unstage changes on stage on stage on stage nice uh we'll keep that okay cool so now uh this get ignore was added so let me do get add Dot status nice so this is exactly all the files we want and I'm gonna say a good commit minus M initial prompt flow assign all my commits and now you can see I I gotta find a way to this is ignored so let me let me re open it up again I don't know I have to I'll check the other one in so you can actually see it um but there you go I built a custom prom float from escarash from scratch as we say uh visual editor here so download it let's get to 5000 by the end of the month uh it's only it's only uh at the beginning of the month so please go download it try it out uh install it um couple of things I also want to mention uh is we have a Discord Channel too if you want to go to that Discord Channel they were like hey Seth we made a Discord Channel and I was like what is that [Music] further evidence that I'm the guy from Florida with his blinker on still [Music] that's me so yep okay so uh go to our Discord channel uh I'll be there uh if you if you at me is that how you is that what the kids do in Discord they add people uh there's stuff in there we want your feedback this is obviously still early but I think it's great because now you can build uh llm applications from escarash all right let's see what people are saying here uh get your comments and questions uh uh love the progress on the documentation thank you all thank y'all um I see the final result as an interesting pipeline back by the Seattle what about delivery yep you can you can make that there is this thing I haven't played with this yet um but yeah it's all there we're trying to do the right thing um uh ugh uh what else we got uh a great sales pitch they're talking about like the rain guard hiking jacket totally made up uh we talked about the index Etc uh well I'll just send you a buzzing is my code available on GitHub not yet not yet uh but the reality is that like the only code I wrote and this is important is uh cognitive search and uh customer lookup but yeah I'll put this online uh soon uh if not if I don't have it now uh by the next show I'll show you where it is and if you join the Discord Channel I can tell you there too okay so the last thing I wanted to do today as we finish up is look at the GitHub communities GitHub communities page uh let's go to discussions this is ours machine learning and MLM Ops uh nice um symante kernel is okay um I I like this a little bit better because it's very transparent what's happening all the time I don't like anything that hides llm calls nut hides it but uh like pushes it down the stack it's just I don't I don't like it my opinion but still okay so let me go to discussions here uh let me go to machine learning and l l mobs new discussion um flow extension in vs code to flow vs code okay okay a new uh vs code extension um of your impression impress Impressions thoughts uh oh what happened my my screen just blinked uh and then we'll do this [Music] preview here are the docks uh uh it's on the documentation yes uh uh let me do this here documentation a prom oh God [Music] oh what is this oh this is new I've never seen that uh uh prom flow microsoft.github.io oh shoot I gotta put the closing music closing music on here because we need to walk it off here in a second so uh let me put this in here thank you [Music] go in there and tell me what you think [Music] labels [Music] okay all right there we go all right my friends it's been another wonderful what I wonder if I can put this up at the top can I move this up in this in all righty my friends it's been a fantastic day being with you thank you so much for spending some time with me learning all about how to do prom flow local uh just a couple of reminders if you want to learn more about the prom flow tool here is the website uh that has the getting started let's talk about some of the docs you can find right here here's the vs code extension that you can get and then our Discord Channel if you'd like to join happy to answer any questions thank you so much for being with us we've been learning all about the new vs code extension for prom flow here on the AI show live thank you so much for being with us and hopefully we'll see you next time [Music] thank you