Knowledge in Copilot Studio | OD812

Microsoft Developer · Intermediate ·💻 AI-Assisted Coding ·1y ago

Key Takeaways

Microsoft Copilot Studio provides a knowledge platform that integrates with various data sources to enhance AI agents with more relevant and contextual answers. The platform uses retrieval augmented generation (RAG) to augment LLM knowledge with enterprise data.

Full Transcript

Welcome everyone. I'm so excited to present the content we got today for Microsoft build. I am Shri Kumar Nayer, a principal GPM on the Microsoft data versse team and today I'm joined with my colleague Rakkesh. Hi, I'm Rakkesh. I'm a principal PM lead on the data team working on Copilot Studio knowledge. All right. So, what we're going to show you today is how can you bring all your data and make it work the magic for your agents. So the session is going to focus on transforming data as knowledge for your agents. As we look at the enterprise scenario today, we have more than nine zetabytes of data spread across different enterprise systems and 44% of that data goes uncaptured. And the problem there is people don't know where the data is. people don't know how to access these systems and then 68% of that data going unused is a sheer waste of the data and the time that you spend collecting that data. So in this session we're going to see quickly how you can bring all the data come into play and transform your digital transformation using the AI revolution that's happening today. So what are some of the top challenges that we see today? As you have seen over the last couple of years, right, everybody is using LLM based technologies and LLMs are good at general knowledge, right? But what really uh the enterprises care for is their agents that are built needs to use the data that's residing within their enterprise sources. So that is a huge problem that's faced by thousands and thousands of enterprises out there today. Yeah. And they come in all shapes and all sizes as we know right some are structured some are unstructured some are semistructured and then above it all moving data from one place to another is a big nightmare takes months and lot of effort only to realize that I have to set up security and governance and everything again so given these challenges what do we have today to show we have the knowledge platform with Microsoft copilot studio which has come up with a unique solution of a knowledge platform that provides you real time integration as well as semantic uh structured rag that it can do on your data. So over the period of this uh session you will be seeing different patterns that we have implemented to suit your needs. In some places we will index the data, ingest the data and create indexes for you so that you can get the best quality of knowledge when it comes to getting answers through your agents. At the same time, there are places where your line of business systems have tight security and you don't want to uh tamper with that security. You want the system and data to stay in place while making the agents talk to them and do the rag process. By the way, what is this rag process? Do you know truckish? Isn't it retrieval augmented generation? Thought so. What is that? But it sounds fancy. It is indeed. Yes. So retrieval augmented generation is basically what we were talking about earlier in the session. Rakkesh was talking about LLMs have the world knowledge. But is that world knowledge what will help you in your for your agents to do the job in your enterprise or organization? Definitely not. So this is where rag process comes in. What we do is we augment the LLM knowledge with your data with your enterprise data making it more relevant and more reliable to be used in your enterprise. So with that as you can see here different data sources like data versse files and Salesforce and sharepoint all that knowledge coming together behind the knowledge platform and we provide tools to make that knowledge even more effective with indexes and metadata and glossery that you can build on your organizational knowledge that goes into this glossery. Of course, the power platform connections that you already have to these systems will play as an accelerator role in getting that knowledge retrieval going for your different agents that you're building. All right. So, what are these rag patterns that we're talking about? The very first pattern is ingest data where we ingest your data mostly unstructured data because we know LLMs do well when you have the semantic index, have the vector embeddings. So we'll ingest that data create the indexes on it and give you the best quality that is possible when it comes to the quality from the agents. The next pattern that we will be talking about is indexing your metadata. These are scenarios where ingestion of the data is not possible or not recommended for whatever reasons or or for that matter it's just your choice that you don't want to move the data. In such case we'll just ingest the metadata. And what type of metadata are we talking about? These are the table names, the column names. So that when we want to reference some of the user queries and map it to the references of a table where the data might be sitting, it makes the job much more easier to do the rag or the data grounding process. And these are scenarios where you would have like a dataware table or Salesforce table that are talking about some of your CRM process or for that matter even an ERP process because we do support some of the FNO finance and operations tables as well. The last but not the least is a readbased operation that we can still do with the power platform connector. This is assuming we do not have any data any metadata. You could still see how a maker can connect using power platform connector and connect to the source and then configure the knowledge source in the agent so that agents are able to give you a best user experience when it comes to getting knowledge out of your agents. With that, let's move to what we have here from the very first pattern. Ingesting data, especially unstructured data. As we already talked about uh this data gets ingested into data versse and you don't have to be worried about the security and governance because you already know data versse is the data platform that has secured security uh roles and the data governance principles all built in of course because it is the power platform and Dynamics 365's preferred data platform for storing all the data. Then comes creating semantic index where we leverage Azure AI search indexes under the hood and create the vector embeddings so that we can give you a much better semantic rag on all of this data. Let's come to the demo of it. So this is where I want to quickly show you on my screen laptop how I can do uh one drive knowledge. So if you have some files sitting in one drive, let's go look at how copilot studio can help you now connect to those files. And these are files that are evolving in the sense that if you make some changes to these files, your agents can immediately pick it up. And then if you you because you're choosing folders sometimes if you drop new files in these folders those are also going to be picked up by these agents thereby making sure you don't have to reconfigure all the agents again once you configured a one drive path as knowledge source for your agent. All right so Rakkesh and I are going to play a little HR game. We are going to pretend like HR professionals or at least we're going to make the agent prefer pretend like a hiring helper and let's see how an agent can help us hire good talent faster. So in this case as we said we're going to look at the first pattern of the knowledge where we ingest the knowledge. So here we got a hiring helper agent. You can see the standard topics and agents are there. There's no knowledge added. I'm going to quickly come and start adding knowledge here. So once you open this up, you will see all the different enterprise data sources that you can connect to. The very first one that you will see is files. And of course, up until now, you just had the option to drag drop a file. Now we have this new option that you can connect to one drive. It opens up uh one drive file picker where you can just browse go to your one drive and here I have some resumes of the candidates and I'm going to be choosing some candidates that I can use or for that matter you could also go back and select the folder itself. And now let's go ahead and select these files. You can give some more descriptions if you want to add a little more of those descriptions. And what's happening behind the scene is we are connecting to these files and adding them as references into what we should be syncing into data versse and then at the same time starting the file indexing process and some of the index embeddings creations. As part of that you can already see in the background we have added these files as knowledge and then all set once you go to your knowledge tab you will see the indexing and everything is ready everything is set for you to start asking questions for example you could ask who is the best candidate for the role of a product manager you never know they might all be product managers but you will see some of the deep research coming into play and uh how these agent helps helps you pick the right candidate for the job that you are trying to hire for. All right. So you already saw how we can use the files that are in one drive as reference knowledge to this hiring helper that we are configuring here. And as we do this, you will realize uh as with any HR process, you will now have to start collecting the list of the candidates, start looking at what is their availability and all that. So let's uh cut over to what Rakkesh has to show us where he's tracking these candidates on an Excel file and trying to use that as knowledge for the hiring agent. Over to you Rakkesh. Thank you Sri. So let's take a look at it. Right. So what Sri showed was a bunch of rums like PDF document PDF and word documents that are being added. But when you talk about unstructured uh knowledge, it's much more than that, right? Many customers just have their unstructured knowledge in like Excel files, CSV files and these contain very pertinent information that are help necessary for helping an agent make the right decisions. Right? So let's take a look what that is. So in this case I have a excel file that has the some a lot of details around the past hiring the roles when the candidate was applied when the phone screen was what was the outcome etc. So, I'm going to drop this thing uh this file as a knowledge source and I'm going to I should be able to ask complex questions around how long it took a candidate from move to move from the phone screen to the offer offer date or how what percentage of them were actually hired etc. And you can cut it across different role types different source types etc. So the way we do it is pretty simple. I come to copiler studio add knowledge and I can just drag and drop drop a file and just uh drop this file give a name and description I'm just going to add this thing um once it's added what we do is we index the content within the in in into the database uh service and depending on the file we might run additional tools like code interpreter as appropriately to really provide complex uh answers to complex questions the users might ask ask her on that. So that and once it's and it takes a few minutes for the indexing to complete and then you should be able to ask questions. So with that you saw how you could add different kinds of file whether it's word documents or a PDF documents or even entire folders into one drive or you can add an excel or a CSV uh into the agent and we would do the uh the right chunking approach and provide the response appropriately. So with that now take a look at what does the next pattern look like when it comes to knowledge. Yeah, this is so exciting, right? What you saw in the pattern one, you could just ingest uh so many files in one drive, you could look at Excel, you could actually do queries where you can do quantitative search on this Excel knowledge or rows of knowledge that is sitting in the Excel files. Uh let's now move on to the pattern two. Rakkesh, why do you think people need pattern two when pattern one is so good? Yeah, when we talk about uh knowledge scenarios, right, I classify into like two different patterns. The search and the query. Search is where you have to search for certain uh uh things within your enterprise knowledge sources and query is where you have to find a specific piece of information contained within your enterprise data corpus. Right? So depending on the scenario, you might do a combination of search or a query or one or the other. Yeah, that's a great point. So when it comes to structured and unstructured, the key difference is unstructured knowledge is all about just getting chunks of information. Whereas when you talk about structured, it's all about queries that require aggregation, joins, and questions that are like how many open opportunities are there on the table and those kind of questions. And not to forget there are requirements that some of your data uh does not have to move out of the system because you spend years configuring security on it. So in such cases we don't want to ingest those data. You tell us that we just want to connect to it. You just point to the data and then all the security settings out there are completely honored. And we already mentioned how we honor that is because we just connect to the data look at the metadata. just use this metadata as reference and then fire realtime queries to the source through the same protocols like O data or the connector protocols to get you the data that is needed and get the agent uh really what it needs to give you the right answer. So with that let's cut over to look at what some scenarios here look like. In this case, we are going to be talking about how you can make uh the data sitting in SharePoint list and snowflake in this hiring uh process and how do we how does all that come to life is basically what we're going to see in the next demo. Over to you Rakkesh. Thank you Sri. So uh let's take a look at the pattern too. Right. So we're really excited to uh showcase some of the new work that we have uh that we are launching at build. Right? Uh we have been listening to customer feedback a lot and sharepoint list constantly came across as one of the sources where uh you need to uh be able to ask questions like uh how many um what's the average are the computational questions on things that are used in SharePoint list. So let's start with uh one one of the canonical scenarios where SharePoint list is used in a hiring situation and let's take a look at it. You could see that I have a SharePoint uh list where I am tracking the interview status of different candidates right so you could see that uh the who the candidate was what where the resume is whether they have gone through phone screen the first first rounds and second rounds etc. So let's see how we can add this as a knowledge source in uh in an agent right and what you'll see is that it's the same similar experience that you are familiar with of adding a shareepoint source. So I click on add knowledge and click on the shareepoint tile and when I go to the browse items in addition to uh files and folders you'll see uh a list options also show up here. So in this case I can just do uh the interview tracker list here and uh confirm the selection and I'm going to just stick with the uh the default name and description for it. I'm just going to add this knowledge sources. So with that now um you will be able to ask very uh complex questions or mathematical questions on things contained within a SharePoint list directly you using on this agent. This pattern of uh not moving the data and using the query, it doesn't really end there. We can apply it to more complex systems, more u data warehouse systems and all. So we are really excited to announce uh the support for tabular sources like uh snowflake uh and data bricks and a number of other sources that are out there. So in this particular case, let's look at uh how we can add a snowflake knowledge source. So Snowflake as you might know is a one of the most popular data warehouse solutions and in this particular case in this HR scenario all my employee history current employee history is actually stored in this data warehouse. So I'm going to add this thing uh as a knowledge source and see how I can um leverage this data. So I click on this snowflake icon and I create a connection. The connection is already created in this agent. So I click on this employee history table. So before even add the adding the data I can click on this icon here to see to get a preview of what's the data that's contained inside this table. As you can see it contains a bunch of it's a snapshot of the data that talks about uh bonus percentage education skill etc of this uh uh different employees that I have. So I come I kind of come here and click next here and I can finish adding this knowledge. But not only that I can click on this advanced here and then provide really specific uh data clues to the system to help answer the questions through synonyms and glossery. So let's take a look at what that is. Right? So in synonyms I can uh click on this synonyms and say for each of these columns um what are the other names that may be associated with it? Right? So for example in my system there there's a column called education but people might refer to it as uh degree uh or something else and I can keep adding synonyms like that. Uh or another example could be uh uh PTO balance I can say it's like uh PTO balance is equal to vacation balance. So uh the system would know that when somebody asks about vacation it's actually referring to the PTO balance of these employees. Um so this helps map the specific column names that you have in the system to the natural way in which people might be asking questions against that agent and not only that through glossery you can actually add specific computer related terms um to help better transform this query. So for example, I could add something like FML which is uh it stands for family and u medical leave right which is a pretty common uh thing that we might ask or something like uh PIP might say uh performance uh improvement plan. So all the things that people might ask in using different acronyms, company lingo etc. uh this helps us the system convert those uh natural language queries that contain these uh these kind of phrases uh to really help uh form the right query and provide the right response to the user and I click on add here and um the knowledge source will be ready in a couple of minutes right the important thing to note here is that there is no data movement here the data resides in this case snowflake uh at runtime we are just converting the user query into a a API call that the system understands and simply executing the API call um enabling you to uh provide the response. So any uh source role based access control and any policies that you might have configured in your source system in this case snowflake will be completely honored um at runtime. Uh with that I'll hand it back to three. Well, that was mind-blowing. Starting with simple files going all the way to complex systems like Snowflake and then you saw how the glossery and synonyms comes into play. We talked about simple concepts that we all love, vacation balance and time offs. At the same time, some scary things like performance improvement plans and all that is your organizational language that you use that LLM may not be trained on. And this is where we provide those options for you to configure and train somewhat the agent to understand this enterprise language that you're using. And that's where the rag process comes in. You are augmenting the LLM knowledge for retrieval to do the best job that these agents do. With that uh it's time to move and look at the third pattern that we have here which is the uh read operation using power platform connectors. And what is actually happening here is uh we are using the power of the power platform connectors to bring together all the 1400 connectors that you have and use the read operation on it as a simple API call uh and make the connection. Sometimes these are not needing your semantic APIs or not needing the natural language to API conversions. In this case all you need is to read a ticket number or just look for an opportunity ID. In such cases, we use the read operation. At the same time, these connectors are also powerful and you will see that in the demo in a few while that they can connect to semantic indexes also that you might have created in Azure AI search. And that is one of the examples that we going to show you now how this hiring agent can connect to those semantic indexes and get your uh knowledge quality that you're looking for. With that, I'll hand it over to Rakkesh again to show you how the power of connectors and the Azure AI indexes that you may have in your subscription can be used as knowledge. Over to you Rakesh. Uh thank you Sri. Yeah. So we saw two patterns there initially, right? The first with respect to the injection of data. The second rel relates to the querying of the data without without actually data movement but really indexing the metadata of the systems. Um and now we're going to see a kind of like two scenarios. The first one is really uh one where we're using the power platform connector to see how you can u connect to a uh system and then leverage everything that is the goodness of that is created in the ashure side and then um add it as a knowledge right so let's take a look so the same experience that that we've been showing you here uh within the knowledge uh experience in co in copilot studio I click on add knowledge and I see the ashure AI uh s index here. So uh in this case I already created a connection using my power platform connection and I see different kinds of uh knowledge sources uh indexes that are already created in the Azure search. Right? So in this case I can click on here to get a preview of the size of the document. What kind of index is configured there and just pick uh the the the right index and with just one click I can add this as knowledge right this helps you to uh bring the best of what what you might have configured within your Azure site directly in a into copilot studio in just a couple of clicks. Right? Great. So now that we saw three different patterns right in the first pattern we were just indexing the data for different types of unstructured content and the second pattern uh we were quering the data uh without actually moving the data was indexing only the metadata to help translate a user question in natural language into a corresponding uh API call that system understands. And the third pattern we're just using the connector to kind of go to a particular source and then retrieve the right information. Now there are some scenarios where you will need to do a combination of these three together. So uh let's take a look at what such a scenario would be. Right? In this HR case, imagine u I'm an HR professional. I am using Salesforce as my uh system where all the tickets related to my candidate tracking is logged and I might also have some unstructured data like help articles that related to cander tracking uh monitoring etc all there. So how do we bring all of these together? Let's take a look at that. So similar to what you saw uh earlier. So you come to knowledge. I click on add knowledge. Then I'll see the Salesforce tab, Salesforce tile within my knowledge tab. And I I click on it. Um and as you saw earlier, you can use the same connection that is used for all Power Platform operations like the same connection that if you have a Salesforce action that you might want to add, it's the same connection that will be used. I click on that and I can add the different tables uh that are out that are in Salesforce there. So if I click on uh account for example right they come and take a look at the preview similar to how you saw the preview of the tabler sources. So as you could see the knowledgebased articles has information that an agent should have and not only that along with such an unstructured data you might have something called uh some a tabular data like cases right in this case the case would be case would be you have tabular data there you have the information related to different cases so similar to how you saw the tabular data as well so and now once I have selected both I come here and click on add and now with this both kinds of knowledge is being added to this agent. So what happens is depending on the user query if it related to the tabular data that's contained in Salesforce we will convert the user question into a a query and then execute against Salesforce. But if it's a question related to a knowledgebased article which is mainly unstructured data then that that that content is indexed and chunked on within the database service. So we'll use that to provide the response and if it needs both we'll use both of them to provide the response. So irrespective of what the user question is you will always get the uh get the right answer um depending on the scenario. With that uh let's take it back. All right, that was very exciting to see all the knowledge sources being accumulated for the hiring helper agent to now come into action. the one drive files, the excel files, the files that we had uh collected and given as resumes all coming into play and then of course the snowflake data that we provided besides the uh Salesforce data that we added at the end. And now let's take a look at the demo of how the hiring helper is going to help us hire candidates faster and find the best talent to get us do the job. Let's see this in action. So I'm going to ask a question about who is the eligible candidates for an open product manager role I have. So it's going to search through all the candidate rums that I added from one drive and the agent will provide me a response. As you can see it provided a pretty thoughtful response looking at the different résumés and even provided some additional qualitative aspects to it. Now let's take a look at the second scenario. Here I'm going to ask a complex mathematical question around the Excel file that I had uploaded. Uh it had a lot of information around different interview stages and timestamps and all. So here the system knows that beyond just doing a simple knowledge retrieval, it needs to invoke code interpreter behind the scenes to answer such a question. So it it does just that and provides you with a very specific answer that uh helps answer a complex question within an Excel file. So for the second pattern we discussed earlier, let's see how a query pattern can retrieve right results from the different knowledge source. So here my interview list is maintained in a sharepoint list and I'm going to ask questions around different interview stages and as you can see the system provides you with a specific answer as quantitive answer from the shareepoint list that's added. Now continuing on pattern two I'm going to ask a question about my existing employees salary which is stored in a snowflake data warehouse. As you saw earlier, since I added that knowledge source, the system is going to translate this user question around max salary into an SQL statement and then into an API call the system understands and provide me with the right response. As you can see, the max salary I recorded company is around $200,000. Now, for this scenario, we're going to see a combination of scenarios one and two. So I'm going to ask about what are the high priority cases that are open since the last 7 days and as you saw earlier the cases are contained in a Salesforce uh instance that I have added uh which is a classic tabular data source. So here it's going to do similar to what you saw with the snowflake example where it's going to convert the user question into a SQL statement and then execute against Salesforce. So looks like there's about this is one high priority case that's open right. So I'm going to ask a follow-up question based on that. So as you saw I'm just asking you using natural language what was this case about and the system knows the specific case I'm talking about at this instance. So it's going to come back with the details of that specific case. When I look at it I see that it's actually about an email delivery issue. So naturally my next question is how do I address such email delivery issues and you'll remember that I had added a Salforce knowledgebased articles as also which contains all this unstructured data around common troubleshooting uh things as knowledge. So the system that directly goes and fetches the details of the knowledgebased articles which includes a citation link to the article directly that's in Salesforce as well. So this way you get a full end toend experience. Wow, that is an awesome hiring agent that I would love to buy. Correct, Rakkesh? Absolutely. Yes. So don't just uh look at this hiring agent, but let's also look at so what our customers have been able to do with all these knowledge sources that we have lit up behind the scene. Hey, I'm Pravin Rajaram from uh CSX. I'm an software architect in enterprise architecture team. Uh CSX as a company we are uh one of the largest freight railroad transportation company in United States. Um we serve around 23 states across US and also part of Canada. We have around 21,000 track miles uh in the east coast region of the country. uh we move to uh different type of goods from agriculture, containers, coals, automobile a lot of uh through our network. Um so one of the uh uh use case we were looking into it is how we can help uh our customers and customer service support team uh using the AI uh capabilities which is evolving in the market. Uh so today our customers do business with CSX through ship csx.com which is our online portal. We have around 40,000 registered users and we have uh active 7,000 users per day who visit us to do different business functionality like planning, shipping uh trading and equipment or uh payment. They do a lot of stuff here. Uh so we wanted to give a um like first class experience for our customers. Uh so we started looking into tool Microsoft copilot studio and Azure AI foundry where we were able to build an AI agent called chessie which was embedded on our ship CSX customer portal uh where customers were able to get all the answers right uh without even having an involved. we were they were able to get all the answers at the first shot and at the same time when we embedded our business wanted to see how I can use um existing knowledge document like how to guides and frequently asked questions through this AI agent. So we looked into different solutions. One of the solution we ended up is the copilot studio where we were able to use all these documents upload into the AI agent within few hours. uh behind the scene he did all the magic for us and the outcome what we saw from these knowledge source was tremendous using the agent and then uh also our nature of the document how to guides and these frequently asked questions changes that uh pretty often uh so without involving technology or pro code here we were able to use copilot studios knowledge article capabilities where without having technology involved business is able to upload these documents and readily available for our AI agent. So overall the co-pilot studio gave a pretty good experience for our customers. We are seeing a lot of usage. At the same time we have the scalability to add more agents to the uh chassis and uh we are we hope we are going to give the better experience for the customers. Hi there everyone. My name is Martin from a cyber security company trusted by more than billion users. I represent the web development section where our solutions are used by tens of thousands of customers every day. To cope with all challenges, our solutions must be smart and secure. I want to share with you how we built a secure chatbot using Microsoft Copilot and Azure AI search. This reduced the number of support tickets handled by human agents, improved overall response time and enhance customer satisfaction. To cope with these challenges, we finally decided to use an orchestration layer based on the MCP framework which is fully under our control and uses an open source large language model with guard rails to protect the system. This layer acts as a doorman verifying the input and routing the user's questions to an appropriate AI solution that can provide the best answer. These AI solutions can be SAS platforms like Microsoft Copilot Studios which in our case are managed by different teams. Additionally, we can integrate CRM, human threads and integration to internal APIs which is always challenging to connect with SAS platforms for security reasons. There are many business scenarios where you want to provide more detailed information for a user's question than what is publicly available on your websites. To make this possible, we integrated Azure AI search as a knowledge source for Microsoft Copilot Studio. When we deployed the Azure AI search instance and imported our data, the integration with Microsoft Copilot Studio was seamless and easy. We just needed to add a new Azure AI search knowledge and set it up. When we properly configure the description, which helps Genai decide when to use this knowledge, followed by the connection settings and selecting the right vector index, we could enrich our public website information with internal data. [Music] [Music] Gino Energy is the largest oil producer in Ohio and one of the largest natural gas producers in the state. To do this, we leverage the Microsoft Power Platform to help us integrate our production data with all our applications. So that way we can make better, quicker decisions. Some of the problems that we faced as a business is how quickly we've scaled up. We've gone from trucking about 8,000 barrels of oil to trucking about 70,000 barrels of oil. Using the power platform, we can leverage the integration with thirdparty vendors. We can give apps to those are that are actually doing the trucking. So with the Microsoft Power Platform, you could build exactly what you need for your business users and ship it in days or weeks, not in a matter of months. Developed 30 different Power App applications that communicate to 30 or 40 different vendors, 150 different personnel in the field, and all that's live data coming from our field level. Big challenge we face being in Eastern Ohio is cell phone connectivity. We leverage data versse and its offline capabilities. They're in the field in the middle of nowhere trying to make decisions. We use offline capabilities via data versse to make that happen. We currently building agents with Snowflake as a knowledge source. So right now we have to for our field operators we have to write a lot of complex queries and reports. But a lot of times what happens is a field operator would look at the data and then want a small tweak to the data source. Previously, we had to have different business logic to try and get our data into the right spot. Now, with this connector, we're going to be able to consolidate all that into one place with makes it super easy to manage. So, that way we know there's one source of truth. This greatly simplify the workflow for us and help our business operators make much quicker decision. Now, they would be able to apply and get an answer immediately so that they can make a better decision. Wow, fantastic. It's so lovely to see our customers using all these new knowledge sources that we announcing at build and everything coming together. Um thank you Rakkesh and uh this was an awesome session. So but before we close let's take a look at some of the takeaways that we have here from this session. We definitely have knowledge sources that will help you get unstructured knowledge congested into data versse create the semantic index and be give you best of the knowledge sources that we can think of that included the files the one drive knowledge sources. We are also lighting up Salesforce and service now structured artic unstructured items there that can be used as pattern one and then of course confluence is also being added at the same time for pattern two where we are saying we will do the natural language to API conversion in uh at runtime where you don't have to move the data some of the structured knowledge sitting in service now azure SQL snowflake data bricks everything coming to light and you can get your hands on all of these knowledge resources soon after build. And then of course not to forget the last one where we are leveraging the power of the connectors where you also saw the demo of you how Azure AI search comes to a light there as knowledge sources. With that we would like to conclude this session. Thank you Rakkesh and hope you enjoy playing with all these knowledge sources and lighting up your agents. Thank you.

Original Description

Explore how to effectively leverage your enterprise knowledge in Microsoft Copilot Studio to enhance your agents with more relevant and contextual answers. You’ll discover the diverse types of knowledge available and learn how to optimize them to meet your specific business needs. Use knowledge from uploading files, using Dataverse tables and RAG search over connected data sources including SQL, Salesforce, Snowflake, and more. 𝗦𝗽𝗲𝗮𝗸𝗲𝗿𝘀: * Srikumar Nair * Rakesh Krishnan 𝗦𝗲𝘀𝘀𝗶𝗼𝗻 𝗜𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻: This is one of many sessions from the Microsoft Build 2025 event. View even more sessions on-demand and learn about Microsoft Build at https://build.microsoft.com OD812 | English (US) | AI, Copilot & Agents #MSBuild Chapters: 0:00 - Addressing Data Management Challenges in Enterprises 00:03:46 - Diverse Data Integration and Security Measures 00:18:59 - Setting up synonyms and glossaries for better data queries 00:19:44 - Finalization and deployment of knowledge sources without data movement 00:20:26 - Transition to Srikumar Nair Discussing System Transition and Configurations 00:27:01 - Combining Data Types to Provide Accurate Responses 00:29:51 - Combining Scenarios for Comprehensive Query Resolution 00:30:22 - Natural Language Query Demonstration 00:38:59 - Session Closure and Takeaways
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from Microsoft Developer · Microsoft Developer · 0 of 60

← Previous Next →
1 Prepare for the DP-300 exam & the Azure Database Administrator Associate cert | Data Exposed
Prepare for the DP-300 exam & the Azure Database Administrator Associate cert | Data Exposed
Microsoft Developer
2 What I Wish I Knew ... about landing a job in tech
What I Wish I Knew ... about landing a job in tech
Microsoft Developer
3 Igniting Developer Innovation with Vector Search
Igniting Developer Innovation with Vector Search
Microsoft Developer
4 Combining the power of vector search with Azure OpenAI then revolutionize image search with vectors!
Combining the power of vector search with Azure OpenAI then revolutionize image search with vectors!
Microsoft Developer
5 What I Wish I Knew ... about finding your place in tech
What I Wish I Knew ... about finding your place in tech
Microsoft Developer
6 Fluent UI React Insights: Accessible by default
Fluent UI React Insights: Accessible by default
Microsoft Developer
7 Signing Container Images with Notary Project
Signing Container Images with Notary Project
Microsoft Developer
8 What I Wish I Knew ... about finding your place in tech
What I Wish I Knew ... about finding your place in tech
Microsoft Developer
9 What programming languages does GitHub Copilot support?
What programming languages does GitHub Copilot support?
Microsoft Developer
10 What I Wish I Knew ... about how much your job can change
What I Wish I Knew ... about how much your job can change
Microsoft Developer
11 What I Wish I Knew ... about how much your job can change
What I Wish I Knew ... about how much your job can change
Microsoft Developer
12 How do I become more confident about AI?
How do I become more confident about AI?
Microsoft Developer
13 How do I become more confident about AI?
How do I become more confident about AI?
Microsoft Developer
14 Performance Demos of SQL’s Intelligent Query Processing Feedback capabilities | Data Exposed
Performance Demos of SQL’s Intelligent Query Processing Feedback capabilities | Data Exposed
Microsoft Developer
15 What I Wish I Knew ... about coming to Microsoft
What I Wish I Knew ... about coming to Microsoft
Microsoft Developer
16 What I Wish I Knew ... about coming to Microsoft
What I Wish I Knew ... about coming to Microsoft
Microsoft Developer
17 Revolutionizing Image Search with Vectors
Revolutionizing Image Search with Vectors
Microsoft Developer
18 Igniting developer innovation with Vector search and Azure OpenAI
Igniting developer innovation with Vector search and Azure OpenAI
Microsoft Developer
19 Getting Started with Azure AI Studio's Prompt Flow - Part 2
Getting Started with Azure AI Studio's Prompt Flow - Part 2
Microsoft Developer
20 What I Wish I Knew ... about finding my career path
What I Wish I Knew ... about finding my career path
Microsoft Developer
21 What I Wish I Knew ... about finding my career path
What I Wish I Knew ... about finding my career path
Microsoft Developer
22 Windows Terminal's journey to Open Source
Windows Terminal's journey to Open Source
Microsoft Developer
23 Can I trust the code that GitHub Copilot generates?
Can I trust the code that GitHub Copilot generates?
Microsoft Developer
24 What I Wish I Knew ... about interviewing
What I Wish I Knew ... about interviewing
Microsoft Developer
25 What I Wish I Knew ... about interviewing
What I Wish I Knew ... about interviewing
Microsoft Developer
26 What is the Microsoft TechSpark Program?
What is the Microsoft TechSpark Program?
Microsoft Developer
27 SQL Server 2022: Accelerate query performance while reducing query compile time - w/ no code changes
SQL Server 2022: Accelerate query performance while reducing query compile time - w/ no code changes
Microsoft Developer
28 What I Wish I Knew ... about discovering computer science
What I Wish I Knew ... about discovering computer science
Microsoft Developer
29 What I Wish I Knew ... about discovering computer science
What I Wish I Knew ... about discovering computer science
Microsoft Developer
30 Call center transcription and analysis using Azure AI
Call center transcription and analysis using Azure AI
Microsoft Developer
31 How to use Text Analytics for health in Azure AI Language
How to use Text Analytics for health in Azure AI Language
Microsoft Developer
32 Azure OpenAI-powered summarization in Azure AI Language
Azure OpenAI-powered summarization in Azure AI Language
Microsoft Developer
33 Accelerate data labeling using Azure OpenAI and Azure AI Language
Accelerate data labeling using Azure OpenAI and Azure AI Language
Microsoft Developer
34 Building a Private ChatGPT with Azure OpenAI
Building a Private ChatGPT with Azure OpenAI
Microsoft Developer
35 What I Wish I Knew ... about how to interview
What I Wish I Knew ... about how to interview
Microsoft Developer
36 What I Wish I Knew ... about how to interview
What I Wish I Knew ... about how to interview
Microsoft Developer
37 Getting Started with Azure AI Studio's Prompt Flow - Part 3
Getting Started with Azure AI Studio's Prompt Flow - Part 3
Microsoft Developer
38 Intelligent Apps with Azure Kubernetes Service (AKS)
Intelligent Apps with Azure Kubernetes Service (AKS)
Microsoft Developer
39 Getting Started with Azure Blob Storage | Data Exposed: MVP Edition
Getting Started with Azure Blob Storage | Data Exposed: MVP Edition
Microsoft Developer
40 Chat + Your Data + Plugins
Chat + Your Data + Plugins
Microsoft Developer
41 What I Wish I Knew ... about different career paths
What I Wish I Knew ... about different career paths
Microsoft Developer
42 What I Wish I Knew ... about different career paths
What I Wish I Knew ... about different career paths
Microsoft Developer
43 Advanced Dev Tunnels Features | OD122
Advanced Dev Tunnels Features | OD122
Microsoft Developer
44 Learn Live - Manage performance and availability in Azure Cosmos DB for PostgreSQL
Learn Live - Manage performance and availability in Azure Cosmos DB for PostgreSQL
Microsoft Developer
45 Plan your SQL Migration to Azure with confidence | Data Exposed
Plan your SQL Migration to Azure with confidence | Data Exposed
Microsoft Developer
46 What I Wish I Knew ... about social skills in a tech career
What I Wish I Knew ... about social skills in a tech career
Microsoft Developer
47 What I Wish I Knew ... about social skills in a tech career
What I Wish I Knew ... about social skills in a tech career
Microsoft Developer
48 All About Vectors, Search, and Function Calling in Azure OpenAI - Labor Day Special
All About Vectors, Search, and Function Calling in Azure OpenAI - Labor Day Special
Microsoft Developer
49 Introduction to project ORAS
Introduction to project ORAS
Microsoft Developer
50 What I Wish I Knew ... about finding the right major
What I Wish I Knew ... about finding the right major
Microsoft Developer
51 What I Wish I Knew ... about finding the right major
What I Wish I Knew ... about finding the right major
Microsoft Developer
52 What I Wish I Knew ... about how to approach programming
What I Wish I Knew ... about how to approach programming
Microsoft Developer
53 What I Wish I Knew ... about how to approach programming
What I Wish I Knew ... about how to approach programming
Microsoft Developer
54 Learn Live - Scale from a single node to multiple nodes with Azure Cosmos DB for PostgreSQL
Learn Live - Scale from a single node to multiple nodes with Azure Cosmos DB for PostgreSQL
Microsoft Developer
55 What I Wish I Knew ... about diversity in tech #1
What I Wish I Knew ... about diversity in tech #1
Microsoft Developer
56 What I Wish I Knew ... about diversity in tech #1
What I Wish I Knew ... about diversity in tech #1
Microsoft Developer
57 Get started with SQL Server AGs across Windows, Linux and Container Replicas | Data Exposed
Get started with SQL Server AGs across Windows, Linux and Container Replicas | Data Exposed
Microsoft Developer
58 Writing LLM Apps with Azure AI and PromptFlow
Writing LLM Apps with Azure AI and PromptFlow
Microsoft Developer
59 What I Wish I Knew ... about how cool working in tech could be
What I Wish I Knew ... about how cool working in tech could be
Microsoft Developer
60 Open Source foundation models in Azure Machine Learning & optimization techniques behind the scenes
Open Source foundation models in Azure Machine Learning & optimization techniques behind the scenes
Microsoft Developer

Microsoft Copilot Studio integrates with various data sources to enhance AI agents with more relevant and contextual answers. The platform uses RAG to augment LLM knowledge with enterprise data. Learn how to leverage this platform to build effective AI agents.

Key Takeaways
  1. Ingest data into Data Verse
  2. Create semantic index by leveraging Azure AI search indexes and vector embeddings
  3. Use Power Platform connections to accelerate knowledge retrieval
  4. Index metadata such as table names and column names
  5. Use Power Platform connector for read-based operations
  6. Add knowledge sources to an agent using Power Platform connection
  7. Build an AI agent using Microsoft Copilot Studio
  8. Embed the agent on a customer portal
  9. Use existing knowledge documents to train the agent
💡 The integration of Microsoft Copilot Studio with various data sources and the use of RAG to augment LLM knowledge enables the creation of more effective and contextual AI agents.

Related Reads

📰
Introducing Carrick
Learn about Carrick, a context layer for polyrepo TypeScript teams that enables AI coding agents to search and validate code across services
Dev.to AI
📰
Why I Switched from Ponytail to Guardsman for AI Coding
Learn how Guardsman improves AI coding by teaching agents to write accountable code, a crucial step beyond just writing less code
Dev.to · Antoinette C. Lennox
📰
How to Learn With AI Without Letting It Do the Thinking for You
Learn to leverage AI as a tool without losing critical thinking skills by setting boundaries and using it to augment your learning process
Medium · AI
📰
How to Learn With AI Without Letting It Do the Thinking for You
Learn to leverage AI as a tool without relying on it for thinking, using Rust as an example
Medium · Programming

Chapters (9)

Addressing Data Management Challenges in Enterprises
3:46 Diverse Data Integration and Security Measures
18:59 Setting up synonyms and glossaries for better data queries
19:44 Finalization and deployment of knowledge sources without data movement
20:26 Transition to Srikumar Nair Discussing System Transition and Configurations
27:01 Combining Data Types to Provide Accurate Responses
29:51 Combining Scenarios for Comprehensive Query Resolution
30:22 Natural Language Query Demonstration
38:59 Session Closure and Takeaways
Up next
Microsoft 365 Copilot ULTIMATE Guide: Excel, Word, Outlook & More!
Maksims Sics
Watch →