Workshop: Build Multi-Agent Workflows with Low-Code Interfaces - AI PM Community Session #61

Key Takeaways

[Music] so little bit about uh background about myself so I'm working at a company called cdk Global I'm an AI product manager I own the the platform AI capabilities because we want to build uh platform services so that we can Infuse AI across our product abilities uh and cdk Global yeah few of you may not know but some of you may know but so cdk Global is the largest car dealership software company in United States so if you go anytime to a car dealership there's a 60% 70% chance that you are being serviced by a car dealership which is being powered by U pdk products we are spread across the whole life cycle starting from the the CRM to purchasing a card uh eventually to also in the fixed operation which is where you get your cars service so yeah so really excited to be here to talk to a little bit about the agents uh so first of all I want to talk first like what agents are how M multi-agent AI works and show you low code no code agent workflow like where exactly the agents are moving towards uh I want to reinfor for some topics about agents which we discussed in the last session I was there because some people could not join but like like the first section will be a lot similar and there's a reason for that because I want everybody to start thinking more about like how the world is progressing towards agents and every day I'm seeing new new capabilities coming out in agents and agents are really becoming the the next level of how people are leveraging uh Genera AI so uh we'll talk about agent what M agents are why should pmk what are the different kinds of multi AI agent and some low code no code interface way last time we dug into the collab notebooks to really uh write uh detailed code but as you can imagine like sometimes there are different personas for product managers like some product managers are very technical uh they can write code very easily they can get into python but then there are lot many product managers who want to start getting a flavor of like how we can use low code no code capabilities to to build AI products so this is the session where we will start exploring how you cannot just only build some low code no code capabilities for the normal LM based power tools but even agentic Solutions and few days back I saw another player popping up in this space so right now this content is really tip of the sphere Ive really checked today that whether the my python code will run in background when I show up this low code no code solution so glad it worked but like we are making rapid development so but I want you folks to be at the tip of the spare so that you can use this agent workflow as product managers to build poc's yourself the days are gone when you will have a big team of engineering counterparts who you can work with these days the engineering team size are shrinking that's one major reason why we're seeing so many layoffs happening is like you don't need that many people but what is happening these days in the industry is in fact my VP said the exact same thing to me like anit I want you to start building poc's yourself if you can showcase it to the the VPS the CTO the cpos probably show it in hackathon to really get funding to get it out there to get it in the production grade environment so that's why this session will give you a flavor of how to start thinking about it if you go to go back to my previous session that was for technical product managers who can write uh code but that was not that technical but I think it was it had some python lines that you write but the one that I'm going to show you today does not have any uh python codes to write except six lines that you have to do because right now we not there yet that you can get a web based low code no code interface for right now because as the market progresses you'll be able to do that but I'll show you how easy it is to write those six to seven lines of code so that you can build your uh agentic workflow so first of all uh let's talk about like what an agent actually is so agents versus llm the one simple difference is that agents are able to perform a specific task without human intervention of course you have to give them uh instruction for the first time but they can start doing the work again and again with llm what happened you have keep on asking a question like for example if you want to get some kind of a travel plan built up for going to Paris you have to go to an llm could be anthropic could be your uh chat GPT you have to ask a question if you had an agent and we'll show you how agents could work you don't have to have to ask like so you can get some kind of an email with all these steps taken care of so that's the difference between an llm and an agent so how AI is getting infused across the company so and you'll see where agents are becoming more and more important so right now how the companies are integrating AI is like they are infusing AI across their multiple products they're using it for sentiment analysis they using it for prediction they're using it for classification uh I can give you some example like what could be happening if you want to do like car dealership employees right like they want to uh like upsell cell capabilities uh to like when they are trying to service a car that's some kind of predictive maintenance capabilities what will that do is like and it's good for customers because customers sometime do not want to go back uh when they forgot that they have had some other repairs to be done so again you have to put your car in the dealership so that's how the companies are using it right now but the the companies are tomorrow but the I believe the future is already here but it is not distributed evenly think about companies like anthropic think about companies like chat GPT they have reimagined the UI starting with AI itself like anthropic and Char GPT have one like one major product which is an AI power product so you see their UI is so different compared to other companies so this is the world that we are moving towards what squa is telling is that in futures companies will be like a neural network the way the neural network will be is like you will have uh customer service neural network you'll have Finance neural network you'll have manufacturing neural network and they all can operate together in an agentic workflow so you've heard of this concept of uh like what Sam Alon talks about is a one person billion dollar company how do you get to that for that you need to have agent working in tandem with each other so that's why if you think this is a a cute animation of how this really works like so let's say you are a company which wants to develop some kind of a game so here multiple Department design coding testing and documenting are working together and you have a chat Dev manager which is working to and eventually the output will be uh some kind of a software which will be given back and you have seen this uh discussion about uh a Devon which is like valued at $2 billion doar like it's just like six year six month old like and what it was trying to do was it was trying to uh automate some task of a junior engineer uh we know that like there were some uh criticism that it was mocked up but now after seeing this open AI has come up with some kind of a benchmark for developers called Dev is Benchmark like how you can test your capabilities so as Sam Alman and everybody says this is the worst it will be it's going to become better and better so like what you're seeing right now the chat GPT if you don't like it well wait for chat GPT 5 and other when they come across they are going to have a step change because the way the neural network architectures are in Transformer like you just need more compute more data uh and and and you need to just increase the number of parameters that's what you need to really keep on improving the the the capability of of the model you can see here like llama came out with the new model it was very close to what chat GPT 3.5 or 4 was doing then some new model will keep on coming you can see that why this is happening because they are literally training it more and more data and they are uh like adding a lot more parameters and then they using do of compute so that's why this is the worst it's going to be so future is going to be really bright uh like we just waiting for when it happen but I want to people to be at the Forefront of the future so uh so this is an example of like how like currently it works like so right now what happens in the regular environment you give a prompt uh llm you ask a question to llm llm will give you an answer and then you can reiterate and keep on improving the pr that is the first way that you use chat GPD with then in the chat GPD you can also do one more thing like you you can upload some document like when you do not want to just rely on the chat GPT information because sometimes there's a cut off uh on the chat GPT because chat GPT is still not having the current information or if you want to uh build some products which are for your company you don't definitely will expose that to open AI you will have your own models and where you'll have your own data so this is a rag based concept retrieval augmented generation where you give uh uh information to the llm with the prompt and you give additional context which is connected to this Vector DB uh and then llm will give you an answer which is very specialized uh specifically for your use case for your business for example if your company has very specific uh customer service uh guidelines which you will not get on Char gbt so that's where you will use it but if you still see here what's happening right now it's like it's very uh uh like it's a very iterative process you have to keep on asking a question there's an input you have to be a human input needs to be there prompt prompt what does an agent will let you do and that's where the world is moving towards like it's not there yet but it's it's coming very soon because we can already see some benchmarks are doing well agent has uh like the the same prompt it has memory because it has had some previous information already available it has also built in some reasoning capabilities the Q&A like you heard of like Chain of Thought like but it's mostly Chain of Thought on steroids you can keep on iteratively asking it question again and again without you being needed to be involved it could be multiple llms talking to each other and asking questions these llms can also call some tools it can call some tools such as and and we'll show you how we going to have this example in the low code no code solution like tools such as like you can call zapier you can call some apis and so it can do lot many things So eventually you'll just give an prompt it's going to be working in intive fashion and eventually you'll get an uh output out there so this is what the world is moving towards and I just don't want you to take my word for it I have data to back how this is operating toward uh this is a coding Benchmark I got it from uh Andrew deep learning AI website they have a benchmark called human evaluation so if you'll see here uh like let me explain you what this axis are the y axis is leveraging gp4 or GPT 3.5 as we know four is a lot more powerful than that and the the x-axis is the accuracy if you're at 100% like you have a human level accuracy uh as you know that we are still not there so what what's happening right now if you use GPT 3.5 a zero short learning what exactly is a zero short learning uh zero short learning is when you do not give it any context you just ask a simple question like hey I have this movie I'm going for a movie today called drus drus if I see a review if I want to ask a question like hey hey CH GP tell me uh like uh the like what the reviews are saying like I'll give a some kind of a prompt review that hey like I like the movie I did not like the movie so you don't give any more information and from that it will tell you whether uh it was uh the review is a positive review or a negative review that's what a zero shot is few shots are when you give some example like hey look at this five six examples from past five six examples from past could be like for example uh like Barbie hey I I love Barbie and the the output is positive hey uh I did not like this x movie and the the centimeters negative so you're giving some example that is a few short learning so as you can see Zero short 48% versus zero short 67% doing better however if you start leveraging multi-agents when you're adding agentic workflow in the process even at 3.5 which are these different options like reflection tool use planning it starts crushing gp4 uh in the process so definitely even with agentic workflows uh and and you have not used uh the the GPT 4 you can still start performing better and better but like as you start going towards multi agent which is where we are like you can even are getting very close to the human level accurac so there are different examples of multi-agent systems uh last time we focused specifically on cre AI so highly recommend going back to the uh the recording uh like we give a very detailed overview of like how cre Works uh but today we're going to talk about other major player because they have come out with a low code no code solution which is which is by Microsoft we also have Lang graph and Mos langra is by Lang chain like I'll show you a screen in short like within few days back they also came up with a low code no code solution and Mos Ki is by data breakes I would have loved to show it but and we'll see if we can show some kind of a video around it but literally uh MOS Ki did show in the data break Summit about how they're using a low code no code solution uh to to make this happen uh so let's look at some examples of what is low code no code Solutions are existing for multi-agent systems uh today's topic is going to be around autogen studio uh which is Microsoft and I'll show you how easy it is to set it up the the second option is the langra studio like I literally looked yesterday night and I saw that even they have come with the the ID so I recommend checking it out and let me know how this is and this is uh the multii agent tooling by data brakes let's see if we able to see the recording I believe yeah and because I give uh next 10 to 15 minutes on that training uh like to at least show how the low code no code solution works but let's see if we are able to make recording and uh jacine let me know that you're able to hear me when I yes no everything's perfect stop Shar and I'll show the is like slack or uh email or even file a ticket if you need to is it all this we're gonna use and with that we're go ahead and jump right in so here I am inside uh so there's three capabilities that we're going to use inside of Mosaic AI to actually build this data intelligence so the first is we're going to use our Tools catalog actually build the data intelligence the next we're going to build uh and understand our quality with agent evaluation and then we're going to be able to debug and improve our quality with mlflow tracing capability so with that let's dive in dats so here we are inside Unity catalog you can see that I have some functions that I'm going to use as tools and these are governed alongside my AI my unstructured data and my structured data so to help demystify what a tool is we're going to go ahead and click into our franchise sales so you can see in here it's just a simple SE query that's accessing my sensitive transaction data and this is where it's really important that your tools are governed alongside your data because only the people who have access to this underlying transactions table are able to successfully call this tool and that's why we need this centralized governance across data Ai and some of the other tools that we've created that are leveraging our Enterprise data are in here as well so for by City And so basically what they are trying to do here is like she's going to talk to you about like how to help uh a cookie marketing team uh to come up with like different different uh like images that they can use for uh for sending out to their customers so like so and they're going to use their own data and they're going to leverage llm models really do that in a very seamless manner my country are just like helper functions to help me get the sales data and then this franchise reviews tool is actually grabbing customer reviews from our social media site so all of these tools are leveraging my Enterprise data so we're gonna go ahead now and extend a base model with these so I'm G to come over here into the AI playground we're going to jump in and then from the AI playground I'm going to select a tools enabled base model and so I'm going to you can tell it's tools enabled it has this little icon on the right so I'm going to go ahead and select Mama 3 from here I'm now going to add hosted tools these are my Unity catalog tools that are hosted inside the secure and scalable data R environment so in here we're going to access the tools that we just showed you in the AI schema so I can so what they're doing if you recall is the function calling so yeah we are using this hosted function calling to call other products so that's how agents are very different from how the the normal syntactic sugar to grab all of those tools and then my marketing team has created a tool for me so I'm going to copy paste real quick because it's kind of long so we're going to put this in here and this tool is going to generate an Instagram image using the shutter stock image AI model that Patrick just announced as well as a caption so now it's time to actually test this before I forget we're going to crank our temperature down to zero because this so you know like uh these are the hyper parameters that we are tuning I think if you attend the last session I went very detailed into the different hyperparameters with Kwang where we talked about what is temperature what is top P what is top K so highly recommend checking it out like but temperature is like think of it as a concept from entropy like you don't want entropy to be very high you want it entropy to be very low if you have very low entropy so like you will get a very specific uh like uh text that you can leverage uh so definitely that's why she's trying to do like because it's a demo that they're doing a data and AI Summit and you don't want to be crazy in front of thousands of people that they're going to that's why she is ranking down the temperature to zero live demo a prompt so this prompt is going to say hey send marketing and Instagram post with an image and a teline for the best selling cookie in San Francisco store so we can increase our sales and show that we listen to customer feedback so we're going to go ahead and here uh and click uh oh she made some she made some mistakes yeah I'll just move on like yeah she fixed it yeah retail. thank goodness for error messages so we're g to come in here and now we're going to add this in and now we're going to go ahead and do thank you thank you so you see what's happening here we using data breakes playground yeah like I don't have access to it because they have not made it public yet like I was nudging them but like it's still under the Ws but I thought like let me show you what they did and I'll show you what we have an autogen so if you see here what they're doing they're calling the meta Lama model uh like there's a system promt which is there and it is calling this franchise franchise prod function all right so what is happening here is gonna be a little bit magical so we're gonna come inside oh hold oh my goodness Sor right we're gonna have to bear with me we're gonna have to do it all again all right coming in here uh coming in here sorry this is why you don't do it live all right retail fr. a all of the functions in there we're going to add in our marketing tool that's going to generate our Instagram and all right we're in here we're going to send the prompt and we're going to make sure it's temperature zero which is what we kind of forgot there temperature zero live demo all right here we go now we're going to send this and what's happening is going be kind of magical so as we come in here you're going to see that llama 3 is going to do Chain of Thought reasoning so it's going to figure out which tools it needs to call in order to execute this so you can come in here and say oh the first thing I had to do was grab the franchise ID for my San Francisco store the second thing is it needed that to be able to access our franchise sales because it's trying to identify what that bestselling cookie then it's going to grab all of that sales data in here so I've grabbed all my sales data and from here identify the bestselling cookie is this almond busati cookie on said hey we need to show that we listen to customer feedback so we need to look into our customer reviews tool if you see there they're using that second thing there this agent is calling franchise review so you can see all the multi-agents are working together like so the first one was generating things and now you're looking at the data around the franchise reviews there it's gonna ask hey what do customers like about this mascotti cookie say oh they like the crunchy texture and unique flavor it's G to send all of this to my slack tool that's going to generate this Instagram image and caption send it to my marketing team on Flack they can review it before we post it on our social media and so now the moment weol waiting for to see what it actually returned so I'm G to jump over into my slack and this is the image that was generated by the shter stock image AI model that shows our image so the way what they have done is like they are multiple agents they're talking to each other they're calling the the chut stro image model based on the prompt that is done so if you see here like you're really helping uh the marketing really uh up their game and and and that's why these are things you can build in house like if you have data brakes and or there some other tools they keep coming out again and again but I want you to get the the front row seat as how this world is moving and then you can also see that it creates this customized uh caption it says our customers Rave about our biscotti for its crunchy texture unique flavor and perfect coffee dipping quality and so this is what has generated and this is how you can use data intelligence to extend your general intelligence to improve a base model now what happens if I remove the intelligence so I can come in here kind of like we did earlier and we're going to remove all of these uh Enterprise data en and we're going to run it all again so now we've taken away all of our Enterprise data access from this and now it's still going to generate an image and it's still going to generate a caption according to that prompt but it's going to be much more generic so as we jump back into slack and it's G to show me my new image soon down here okay successfully sent here we go so this is the image that I now created so all it had to go off of was that you can see that it is hallucinating and it is by Design that's why you need retrieval augmented generation like your own company's context this is exactly the live image that is Cookie and this I think this is San Francisco yeah yeah that's what it's trying to show because it's a San Francisco cookie based company so that's why like you need to ground your prompts uh with the the right kind of data cook that's in San Francisco and so try to create some kind of cool Instagram ad yeah and and if you take a deeper look at the caption it's very generic so it just says our bestselling cookie is backed by popular demand share your favorite cooking moments with us and so not really tailored to our specific business or the franchise or using our Enterprise data at all and this is why data intelligence is so important so we just showed how you can use the Tools catalog to feedback on the response they can come down here and they can say yeah and if you take a deeper look so I wna yeah I think I want to skip through this like quickly like what are the other things that you can do feedback so this is what's happening is the model is able to train like fine tune based on the data that you're giving it like like this is where the human is coming in and getting the feedback on the response they can come down here and they can say yes and they can explain why or why not uh that this answer was good and then they can go ahead and click done and submit this feedback this feedback that this submitted is then logged in Delta table in unit catalog in your account that you can then build an evaluation data set off of get confidence that you can go into production or as I have done I enabled late housee monitoring on top so that I can observe how my pilot program is running so you can see over here the different franchises that I set up this on and here is me tracking their negative scores with the agent over time and you can see that something is going terribly wrong with the Los Angeles franchise they're getting a lot of negative feedback on the agent so if I scroll a bit further to actually investigate their ratings and what questions they're getting they're rating forly we can see that their actual feedback that they're giving me on here is that it's returning a relevant reviews soing reviews from San Francisco stores or nonla stores to them or it's even hallucinated that there's a liberty chip cooking which we don't sell one of those at the cooking conom so we're going to need to dive in deeper figure out what's going wrong with our quality here so yeah just want to give you a flavor of like how they are using agentic workflows if you saw there like she also had the the monitoring setup here like where you can even track like how your different uh flows are working I will share this complete link of this data and AI Summit keynote day one so that you can go through that it'll be super helpful uh I think it starts from one minute yeah like what you know what I'll share the exact uh time stamp when it starts so that yeah I'll stop sharing and let's go back to doing some practical things again I will share my screen again here so we're going to talk now about autogen studio so autogen studio is a low code no code interface for the autogen so autogen it was built by Microsoft uh and very I think September 2023 I think autogen was created in autogen studio is like June 2024 like it it's very recent like so and as you know like they're still working more and more on this like to making better better so we're going to talk about like why uh this makes uh perfect sense uh like and you don't need to do much coding that's why these are not called No code solution they called Low code no code solution where you have to do some bit of uh uh coding like what I would recommend is that you should be decently good in the prompt engineering and should understand how the things work like I think that's the the thing that I would recommend getting better at like building better prompt I think and and we'll go through the the demo of like how so what was the goal of creating the autogen studio so it's going to help lower the bar in building multi-agent applications what I mentioned like because you can build these PCS like and maybe even for some internal company capabilities you can build out the capabilities you can do rapid prototyping that's what I also mentioned that because you want to do rapid prototyping probably show it in hackathon probably show it your leadership probably show it in your PM staff meeting to really uh get some kind of a buy that okay is there a PC that like are there some kind of a feasibility that we can not feasibility sorry the the the customer validation whether there's a need for this in the market so those are things that you can R and eventually with this autogen studio you can keep on improving the expertise and community of the whole group because there's a big Community behind now autogen studio so this is the reason why they built it out so now what we're going to do is like we're going to start uh getting into the the the meat and potatoes for this thing like how we do that like so what first we'll do it is like the first thing that you need is like for I think I showed in the last example also is like you need to have an either an open AI account or some API keys that you can so because you need to use uh these Keys uh in your low code no code solution so let's see how uh this yeah so this is yeah where you'll go to your platform. open. API keys and you'll create a new secret key uh like I have created a new secret key and I have copy pasted it and and put it inside this uh so that's where you will add all that information so like after we that then what we do is that we go back to the so yeah so next is you right now it is not available in the cloud environment so you have to use Python ID but these are the six these are the five steps you need to get to make the autogen studio work for you so uh and I'll show you like how easy it is to do that like so I have a pyam already enabled and I didn't want to do it in like live uh because like some yeah it it did work but at least I want to show you like how easy it is to do that so basically what I first of all did was like you need to first uh go here and then you need to say cond create in autogen studio let me paste this thing I think there's a for this will be helpful just copy paste these uh codes also so that people can use that if you want to do it live posted this so yeah you create your cond environment and you do autogen Studio python is equal to 3.11 like it should be not 3.8 it's 3.11 so this is one line you need to do it's going to create an cond environment for you so after you do all the the stuff you need to uh like it's going to do all the things it's going to do all the work for you and then after it is done you will be activating your cond environment cond activate autogen studio so you are enabling that autogen studio environment uh and then I know I'm going to change this key anyway but like this is the the key like the open a API key you will be pasting your key here uh and then after you need this key because if you don't use this key you will not be able to call the llms as we discussed previously right because eventually agents are still calling the agents calling the LM behind the scen so after you do that what you do it is you do a p install autogen studio so it's going to start installing autogen Studio like so all the stuff it's happening all on its own you don't have to do anything the last command that you need to just add is uh called autogen UI that's it you don't have to do anything else so it's doing all those work so you can see here that like it's a very easy setup you just need a py charm or some ID which will have this information after you go here it's doing all that fun stuff and see where it comes in yeah so yeah yeah I just made a mistake calling it autogen Studio U I should have been UI so after I do that autogen Studio UI that's it it's get it's ready so you you have a URL also ready which gets created you just have to click on it and boom you're in business here and now I'll show you like how autogen uh UI really works but you can see here it's not rocket science these are the five six steps that you need to add and then you will be able to move fast very quickly with it so let me uh show my uh no code no code solution here if you notice we had the same URL right 127 so this is running locally on my machine so I want to show you like at least how the the structure really looks so if you see here that you just have to type those five six commands and that's it like so you get access to this low code no code C solution capability first of all are the skills like what are the key skills that you can uh leverage here like like and and you can always keep adding more and more skills uh like in yeah you can have a lot more skills that's why it's good to have a pretty good handle on the The Prompt engineering but rest all will work on its own like a charm like provided yeah like you provided some information but like uh this is the what I want us to be become better and better at uh let's look at an example of one skill which already exists generate and save images so what is doing is that it's taking and and you don't have to write a code was always there what it's going to do is that function to paint draw illustrate images based on that what will the parameter what will be the image size so all that stuff is already available as part of you and eventually when you don't have information it's going to respond back like so this is the the kind of skills skills that you can have you can also use it to generate and save some p if you want like so that is the skills that it can give then the second is the models then it can it it can call because as you can see if you have multiple model like you have a open AI gp4 which is a highly performing model but costly but you can also have some uh local uh model like a like a open source models like llama 3 or Etc that's all you can also add to it like yeah there are this that you can add to reduce your uh compute cost and like as they keep coming out in more more model this will work then this took care of the models then is the agents what are the agents are going to do so if you recall in the last session what we did was like we were giving every agent a Persona uh there's a languages a Travel Group chart agent there's a planner assistant so why don't we look at like how a planner assistant would look like and of course like let's see what else we can also create so so the if you see here we're doing agent configuration as you can see here no code no code solution we are not getting into a collab notebook to write any code you're just typing all this information like this is the name of the agent what will this do what is the system message so you are a helpful as you're a helpful assistant that can suggest a travel plan for a user and your any context provided uh you are a primary coordinator who will receive suggestion or advice from other agents so you can see here it's a multi-level uh discussion that's happening uh like your final response must be the complete plan so you have to be very explicit telling that like hey uh like do not uh like run Haywire this is what eventually I want it to end with like so after you give a CL eventually it will get terminated uh so this is how work and as you can if you recall uh Casey also talked about temperature like so you don't want to be too creative that's why you try to keep it a little bit lower but if your use case requires specifically in marketing uh that like it should be a lot more creative you can definitely do that so so and then how many tokens that you need so token is like like the the input plus output tokens what because sometimes if you give it many tokens first of all like it's going to rack up my uh com uh like insurance cost bill because I'm going to use so many tokens so so but sometime you need a very detailed iten then you can do that but in interest of time uh and making sure that my credits also do not get exhausted I will still keep it at the like 24 2500 uh token limit uh and do you need any code configuration so so this takes care of the agent like these are all the personal if you recall in the crew. here what exactly what we're doing we creating a skill what they can do what are the models being used and what is the agent so that was more uh like you have to write little bit of quote there but here you don't have to write any code you just have to create agents and then like these are some good more examples user proxy agent like so you want to typically uh represent a user and execute code assistant agent plan generate code to solve task or group group chat like sometime you need to have multiple chats these are the few things that you can do with finally uh this is how the whole workflow will uh look like uh the workflow will look like is like how do you want your agents to talk to each other like if you have multiple agents like so like that's what a workflow will look like let's see what we have for travel workflow so travel workflow uh like the name description what's the the details uh like how it will summarize so with the last conversation and then if you can see here there's a user who's going to uh like ask answer or ask a question and then there's going to be deceiver who's going to take this information answer all your questions so these are the kind of things uh that you can do if you want to create a new workfl let's see the kind of an example you want something like an autonomous chat like where you can have an initi Rie like basically like a conversation or a sequence of items that uh you can use I'm seeing a question like how do I get access to autogen studio uh it's pretty straightforward use the like I'll be posting the the link of the presentation it's open source anybody can use it like the the the key steps that I mentioned that's what will let you access the autogen studio uh without any issue or you can even Google search and check like autogen Studio UI you will be able to you'll be in business okay so uh now this is where like how you are using the the the capabilities of this low code no code solution to make it work here so like the build was like we were doing all the tooling behind the scenes to get uh all of these agents in in check get our model in check get the workflow in check here is exactly what you are using to to test out things and all so uh let me you let me show you some very good examples uh like they already have some examples and and in t of time I think we going to focus on just showing you the examples and then you can try on your own and and this is so new that many people are coming with new and example so yeah like we can either you can build your own or you can see like how the the market is I'm going to click on the stock price agent and see what Happ so what's happening is the user has ask a question like plot a chart for NVIDIA and Tesla stock price for 2023 save the result to a file name Nvidia Tesla PNG so literally the agents are working behind the scene uh to to to and and there are the messages that are received so these are the kind of detail information which is being provide to the user proxy uh plot a chart save the result and assistant so so so this is what the agent has understood to PL a chart I am going to uh take this information install this P this stock plot the data using M plot and sa plot so so it's going it's already started the work it did a p install uh then it created this code itself and then the the code is getting run in the second side of thing so if you can see here that it already created the the Tesla uh and Nvidia stock price for 2024 until 01 so it's all running behind the scenes it can also tell you exactly like what are the the the code it Ed it also because as you know that these systems are very smart in generating code I have leverage and systems really generate a lot of code for myself and visualization so also give you a code like so you just had to ask one question and did all the work uh very seamlessly for you this was the code for the stock this was the code for like the scale like you see here if you recall this is what we talking about the skills right like so you had to just set it one time function to paint draw Illustrated images so it has called that specific uh part of the the the skill to really make this happen like so open Ai and the open AI they call Del to really create those images and create PNG and then eventually it saved it in the repos so you can see it like how easily it's able to do that uh we'll just wrap it up by talking to you a little bit about like the the travel specific one so you can see like it ran like very quickly because I've run it before for the the question was like user had asked like plan a two-day trip to Hawaii limited three activities per day as per as possible so what it has done is like it has quickly taken that information called llm and was able to give you like hey morning visit here and day like often go here in day two go for a hike to Diamond uh head Center D so yeah you can see here very simple lowo noode solutions that available and you can of course like you can add a lot more sessions you can ask questions so I just want to show a flavor of like what's possible so yeah that's all from my side I would love to open the floor for any questions you may have wow you did such a good job that was really interesting so thank you so much for such a great presentation we do have around 10 minutes left so anyone please feel free um to comment questions down below um or if you want you can also unmute yourself and ask a question okay let's see what questions we have here in the chat so we have one question that just came in oh someone raised their hand here um Gio I think you can unmute yourself if you want to ask your question what do you mean me yes sorry yeah I'm wondering is there a is there a path to deploy these agents that we would build onto like a production environment on a live application so autogen studio right now is not there yet to put in the live production but I did see Hara talking about Google vertex agent so so definitely uh this is uh uh like you can use Google vertex a agent to really build the agents and you saw the live demo from data breaks because data breaks like it's in uh private preview like even I'm waiting to get get my access to it because what they specialize in like you are if they if you're already a their subscriber like your company users like you can use their Data Insights to really build those agent so like so yeah or you can try vertic AI I think that should okay thanks and I did see uh Ain talking about like you can use verx agent builder direct production because they all are proposing it amazing I'm reading another question here in the chat um so we got asked um how many credits does a new user need to play with when they sign up for autogen okay uh why don't we check how many did we use right just did some short demo here not it should be a lot more clear you can see my screen yes so yeah we all are product managers we are dat driven let's see like somebody in aoni has the usage taken care of it is the you I'm not sure how real time will this be today's date is yeah 15 16 years it's not showing up yet but if you can see here this is how much it cost 24 cents uh this for yesterday I was trying to play Oh 16 AUST which is today yeah it cost me 24 cents today to to to to to exploit these things so yeah it all depends on how much you leverate so that's why you have lot of ways to even put your uh like limits in place so that you don't have to use it like so right now I put a $120 limit but you can choose put $5 limit $20 limit yeah it's a very itative process and one more thing to know about it is like like while you're talking about it like I want everybody to be aware about as a product manager about this specific website please bookmark it it's going to save you a lot of headache later in your life you are going to be working as an AI product manager you take you think about three things quality speed and price as a product manager these are the three tradeoffs that you have to work around with uh so uh if you can see here GPT 40 is the one of the most expensive one and that's what we use and that's why I incurred 25 cents for the session that I just administered but if you use other models it will be so it's so it will be 17th of that if you're using a mixer model or using a llama model so definitely like always uh like you have to do a tradeoff like hey do you care about price you care about speed do you care about quality so like these are the three triple constraints that you need to be aware about uh and and there is something like a quality versus price like this is a sweet spot like that people are using but everybody's use case is different sometimes you have a blank check from your company to explore these things sometimes you are uh like a oneman person trying to explore so so pick your battles based on this so highly recommend checking this out I'm I think question on it like let me base the L of this specific website also perfect awesome uh we have another question um it looks like Hara is raising their hand so if you want to unmute yourself and ask your question um yes uh hi hit um good to see you again I have a question on agentic AI in general and I also want to combine my questions with atin's question so give given a solution requirement how how do we think about the multi-agent scenario like how many agents how will they talk to each other how will they kind of like coordinate with each other to perform the tasks so curious to know how you would go about fitting agents into a solution like for example in your stock example yeah you get the stock price so should we put a trade should we sell buy you know options this thing that thing so very curious to get your thoughts on thinking about the multi-agent thing and they will they learn from each other exactly so there are uh some good example could be like we can look at things like Ai and that's what I gave the example in the last session also so if you look at what cre AI was doing was that cre AI was I'm trying to see if they have some examples yeah these are the kind of things so highly recommend checking this out like you're able to automate customer support inquiries and the one good example for that is like so and if you watch the recording you'll see that like you have uh one Junior customer support rep and you have a senior customer support rep both of them are aging so junior is asking a question to the the senior customer support rep and and he or not he or she it's it yeah it it is able to look at it and give it anwers so like you can literally work across it by having multiple of them like uh so there you have to find different personas but right now what's happening what I'm hearing the market is like so it's still very hard to to make this production great but like companies are trying this but one thing what is that you need to be very careful what use case where you chose your agents is like so as an AI product ma as you know that I'll give you one good example people are saying hey agents are not working for me for kyc related use cases first of all you cannot just rely on machines for doing no kyc is no your customer like those are very highrisk items you don't want your agents because agents are not very they are not deterministic they are very probalistic they can make mistakes so you you define your use cases where your agents make sense so for stock trading your to answer your question the way it's going to work is that uh you always as an AI product manager have a human in the Loof like you do not want to automate that last step but you need to probably do that analysis where like hey I have gone to this website Bloomberg I have gone to Wall Street Journal I've gone to Twitter and this is my rating buy or sell rating and then eventually a human needs to execute upon it like as you know that sometimes some Edge funds try to automated and they get in trouble so yeah like so it's like how I don't think we are there yet so it's still uh moving in the right direction but like some companies try to do it not successful but like people are still using it to like help them make better decisions but not like do that last step of like executing because they can be they are probabilistic in it hopefully that answer the question amazing yeah no thank you so much for everything this was a really great conversation um we are out of time but again I want to just thank everyone again for taking a time out of your day to join us we all really appreciate it and I hope that you all learn something new um so before we wrap up just a friendly reminder in case you're considering becoming an AI product manager I'm going to be putting the link to our upcoming program in the chat box below so feel free to check that out um again thank you so much we really appreciate you and we look forward to seeing you at our conversation next week thank you yeah and I see some more question yeah reach out to me and I can okay thank you all okay amazing yeah thanks everyone you