Igniting Developer Innovation with Vector Search

Key Takeaways

hey everyone you're not going to want to miss this episode of the AI show where we are going to be learning how you can ignite developer Innovation with Vector search and azure openai [Music] welcome we are joined by Liam who is a PM for Azure cognitive search with Vector search thank you so much for joining us why don't you tell us a little bit about yourself oh thanks for having me on the show Cassie it's uh really nice to be here we'll be back um so I am the group product manager for Azure cognitive search and I'm really excited to introduce a new capability and show you some of the cool things we've added to the product great so there's probably a lot of people that might be watching in being like what is even Vector search so why don't we start with what it is and um how and why people might want to use it absolutely absolutely so you know one of the places that I think would be great to start is this this blog post that we released which talks a little bit about what Azure cognitive search is and it walks through the details of it what we've done and some of the highlights of that but I think you know one of the things that I'd love to do in this session is just kind of help people understand what exactly is Vector search like why should I care what how does this compare to the the old traditional way of doing text search and just kind of the types of things that this opens up Cassie okay so we know that Vector embeddings are numbers and somehow we go from text to numbers can you kind of explain how that process works yeah absolutely this is this is a really interesting capability that it really opens up because in the traditional they will say the older way of searching it was all based on words like you could put in like a run run or running and it could handle different variations of that which was okay but what vectors have really unlocked is this ability to take objects and an object could be some text it could be an image could be a video I mean it could be a molecule or a gene and it describes that in a vector space and all a vector space is is a set of numbers that have a description of that and you can see on the left side how what we can do is we can use different types of models such as Azure open AI has an amazing one which I'll show using something called a to zero zero two which has this amazing ability to take text and make a vector representation of it and what you can do is once you have those descriptions what you can do is then when you ask a question like hey I'm looking for the capital of Canada and I can go through and that vectorized query goes through and it tries to find the the most closest match in that Vector space and that is the power of a vector database is it allows us to take massive numbers of vectors or content and quickly get those answers effectively which leads to amazing chat experiences and much more all right so it's it's really enabling a lot of these next level things that we've been seeing in the AI space so now I kind of understand a little bit about what vectors are how we're able to take many types of data sources turn them into a vector representation and then be able to perform different tasks in order to really enrich our applications and user experiences how does one use this how do we leverage this what are good scenarios to use it for so I have a couple of um or I have some code that I'd love to show to hope hopefully start showing you some of the things that we can do with this and luckily we have some friends coming to um to talk about some even more advanced scenarios in subsequent sessions with Derek and varsha but I thought it would be useful if we just kind of start with this Jupiter notebook and I can kind of walk through how that all works and it shows some of those scenarios does that work yeah that sounds awesome that's cool so well and by the way all this code can be referred to from this GitHub repo at the the top here or going to the the blog post that you referred to earlier let me just walk through this code so basically what I'm going to do here is I'm just in this first tab I'm just setting up all the Imports for python I'm going to be able to use Azure cognitive search which we're going to store all the vectors and do the querying against it we're going to be using Azure open AI because we're using Azure openai to take the text and actually vectorize that content so all I'm doing here is I'm just setting up my configuration to connect to Azure cognitive search I'm setting up my Azure open AI config and then I'm just creating some connections that I'll be using to interact with Azure content search okay so here it looks like you've already set up an Azure service is B Azure service I know that we're using the cognitive search service but we're using the vector search capability is there a different service that I need to set up or how do I do that yeah great great question Cassie and you're right I had previously created this Azure cognitive search service you simply just go to the Azure portal and just create the service and you do not need to do anything special to have Vector search It's just in a new field a new data type within your search index that stores these numbers and it allows us to do that similarity the search really really effectively awesome cool so let me keep on going here and I want to walk through a couple of other pieces of code here so I have two functions one is called generated embeddings any embeddings vectors you can think of them as the same thing all this does is it receives some text and it converts it to an embedding using the Azure open AI model called text embeddings a2002 and it returns that back and this other function that does something called cosine similarity and all that is is a way of us taking one vector and actually comparing it to another vector and seeing how close they are in similarity and so this will hopefully give us an idea of how vectors work and why this idea of being able to search is so useful so I see here in the engine you said you're using an openai model what models are supported can I bring my own or how do I choose which model to use when I'm doing my vector embeddings yeah it's really up to you you can use any model you like we do find that the vast majority of our customers have been using this 802 model because it's so effective it's so good over text however if you were you were thinking um back to what we were just showing in that architectural diagram you can do more than text like for example you can do image search and in a in a subsequent sex session and varsh is going to come in and show how they the Florence model from from Vision can be used to convert images to embeddings but you can also use open source models some people create their own custom models that are unique to their industry that they can use so we can support them all that's awesome and that's what I was wondering because sometimes you have really specific things to um the tasks that you're trying to solve and although these ones are generally the best right um for for all tasks sometimes you need something more specialized so all of that is supported and when you create um the the engine or when you're using the API here you can choose the best model that's exactly right that's exactly right cool so let's keep going and I want to show you an actual example so Cassie I love food so I I thought that you know a good good set of data to use would be you know recipes or just uh different food examples to to show this so I have two of my favorite foods I have enchilada uh and describes what that is and then pizza and I have so what I'm going to do is I'm going to generate embeddings for both of these and let's actually look at what that embedding looks like so if we look at the first vector and just look at the first 10 digits this is what ada002 describe that first sentence as that first set of text as now this means nothing to me but being able to take those numbers and actually compare them is actually very useful so let me now do something different and I'm going to say okay what is a good Mexican food and you can see here this is my query I'm going to convert it to a vector and then I'm going to say using that cosine similarity convert the first Vector to the question the second one and let's see which one is more similar and if we look at this we can see here that the first one refers to enchiladas is actually more relevant and this is why Ada the 002 model is so great like it doesn't like I there's nothing referring to Mexico or Mexican in that first sentence but it knew that the similarity between Mexican and enchilada was much more similar or if I change this to Italian and actually let's spell it here right we can actually see here that the the numbers change slightly so it handles these semantics so much more effectively than some of the ways that we were doing in in the textual search I think that was a really great way to break down how it's able to understand right we get words that are turned into numbers and then it's really easy to see what's similar right because in the embedded space they're closer together therefore they're similar exactly and so hopefully I've shown like you know I this is doing this all in Python this is just using out of the box code but realistically most people don't just have two sentences they have massive numbers of documents that they need to search over and so having to scan through every single one of these pieces of text and try to find most similar is not an efficient way of doing this and that's where Vector databases like Azure cutting search really allow us to scale that so you can put in massive amounts of vectors and search through them really quickly and efficiently and I'd love to show how that works within Congress search Cassie that'd be awesome I can like my brain's going with all the different ways that that would be useful in so many different fields I used to work in the legal field so tons of documents there and I yeah this is really cool yeah I don't think we need to think it's not hard to think about what scenarios people might want to use this for I think that people are probably going crazy with ideas right now it's so true and legal is such an interesting one as well because all the different ways that a single thing is referred to like all the different variations of those words is really hard and finding the right content can be hard if you don't know those semantics or those semitically similar words and this really helps in that space so let's go into a little bit more of an extensive example and so here I just have four different examples of content that I want to load into cognitive search that I want to do Vector search over so we have the first ones which talked about the enchilada and the pizza um there's another one here that has a little bit more about a secret ingredients for uh pizza sauce and this is just another random sentence you'll notice here that there's also we've added categories to this as well because sometimes filtering is important as well so let me just take that that okay song and all I'm going to do is for every single item in that array I'm just going to create an embedding for it a vector for it so it's going to run through it's just going to go through each each one and it's going to add that and if I look at this first one here you'll see that it looks you know pretty much similar right like it's got the same text sorry there's a lot of numbers here to scroll up but we can actually see here that uh we have the the first item the enchilada and here's that Vector representation of it so that's there now what I'm going to do is I'm going to load this into Azure cognitive search and I want to show you how this works in Azure cognitive search we have something called a search index index is just like almost like a table in a database where you have different fields or columns that you can store the data and those fields have different capabilities like for example I have a unique identifier for my recipe remember I was talking about the category which I can then filter or group if I want to I have the actual text of the recipe and here is that new data type we're referring to which is a vector field that allows me to store those numbers and use it for semantic search so I'm going to create that index we have some configuration for the vector work that we can do so like this helps us from the performance and accuracy uh trade-offs that you can apply when you're doing that search and then I'm gonna I'm not going to talk about this right now but I'm going to show you in a few minutes this other capability we have called semantic so I'm going to leave that for now and then we're going to just create that index now that I've created the index I'm going to take my documents load them up which includes all that Vector information it's now existing in Azure cognitive search and I can actually start doing my searches against it um so the first one that I'm going to do is I'm going to search for Italian food and of course I misspelled it here just to make sure it's working it's going to go through execute that search you can see how it's converting that query to an embedding and it's going to use my recipe Vector field because you can have multiple Vector fields in Azure cognitive search and let's print out the results and we see here that it did pretty well like we put in Italian food and we have all kinds of different results that came back here we have this one about pizza we have this one about the pizza sauce and some other items that are quite as relevant but it came up so as you can see here we got this result that came back but sometimes it's important to be able to adjust the categories based on what's important to you like maybe if I'm going to a jewelry store I might want to filter it based on whether it's a necklace or a bracelet or in this case I have categories of standard content and premium content and maybe I might want to say is okay I found a great match for here but I only want to bring back content that was standard so what I can do is because cognitive search has a lot more functionality than just the vectors I can make a change of this and just say let me filter where the category equals standard execute that exact same search and you can see here how it only brought back that specific information so being able to allow us to enable metadata and filtering from that is pretty critical as well yeah that's super helpful so I may have a category of documents that I want to already section out and then do my my uh Vector search on top of that to get really exact results exactly and so one of the things Cassie I love to do is talk a little bit more about some of the limitations of vector search and some of the the things that we've approached to help enable Vector search along with other different techniques and I think a good way to do that is if I take a different example here I'm going to change the query from Italian food to just food with cheese you'll notice in this result which was just using vectors it wasn't using text or anything like that you notice like I got an interesting result you know I said food with cheese and you could probably guess that my intent was like a recipe or food that includes cheese but you'll notice here why it came back like cheesy extremely semantically similar to the word cheese food and cook very similar and actually the amount of text that was there was pretty similar so you could understand from a recall perspective that vectors are really good at why it came back and why it was pretty effective but realistically that's not my intent I was probably wanting something more like an enchilada or pizza and that's why what we have is we've also added not only Vector search support but we've added additional capabilities and we have two things that really help there first is something called hybrid and I want to be very clear on what how we Define hybrid because there's a couple of different definitions but what we mean by hybrid is that it leverages scores from vectors as well as scores from text the traditional text and using both text is really good at recall so if we use both that combines and allows us to bump the relevancy generally and just what the industry generally says a lot better but on top of that we have another ranking layer called semantic search which is a set of models family models that come from Bing that add an extra layer of ranking so I I'd love to show you how what that looks like when I when I apply this yeah so these are just uh things you're going to add in so to your search query exactly add in those additional that's it okay cool yeah and so when you're asking about creating the vector search um that was that is one thing you do need to enable in your search service you just turn it on we have a free option and a paid option if you wish but you can see here I'm just telling it I'm going to use a semantic query I'm using it in English if you recall when it's creating the index I had a configuration for that so I'm going to use it and now I'm also going to pass in the query so not only am I passing in the the vector but I'm actually passing the text of food with cheese and if we search this now you'll notice that the results that come back are a bit more effective right like it did bring the cheesy one but it's much lower down in the list because all of these things of using text for hybrid as well as our updated semantic model which by by the way we just deployed a brand new updated model at the start of August which has been really really effective and this this has really made a big Improvement to the relevancy of results yeah that's awesome that's so cool to be able to see that adding those into your search queries can improve the results that much it's this is this is so important especially um earlier Seth and I were were talking about Enterprise chat DBT and a leveraging Azure cognitive search to recall information that can help the chat experience and this right here is so important because you can only send a certain amount of content into Azure open AI because of token limits and costs so it is critical to get the top results as as effectively as you can so this is really important not only to search but even to some of these generative AI applications yeah that's that's so true like when I've had playing with prompt flow and Azure ML and generating those prompts um using cognitive search with Vector embeddings has been extremely helpful in being able to get the correct documentation to give context to the large language model to get those answers so this adding your own data into large language models is a game changer in application development yeah absolutely and one of the things that I'd love to leave everyone with for some of the subsequent sessions that we're going to go through is that I showed a really simple example a very small set of data but we actually used Azure open AI to create a couple of thousand recipes as well as images for those recipes and so if I change to that index and I'm just going to do a quick search to give you an idea of what we're going to do I'm going to try to find an Italian recipe that includes meat and Parmesan which I completely misspelled so much in here and you can see here uh the result that comes back you know we have the image we're going to show some cool things around image search as well as the ability to like understand different variations of you know me misspelling all these things so I I'm sure hopeful that everyone's going to join for some of those subsequent sessions as well yeah that is super cool so the only thing that I changed here can you go back up to the the query that you sent in absolutely um the only thing you changed was the index you were pointing at the query and everything else you sent in is the same that is correct that was the only change they made uh it has the exact same um type of queries it's using semantic as you can see here I do have a couple additional fields in this one that I didn't have in the under index so that's basically the only change amazing and delicious I mean now I want Italian food yeah I know I that's the downside of doing this one it sure makes me very hungry every time but but yeah hopefully that gives everyone a good idea of what Vector search is and uh hopefully it inspires some ideas of things that you can do I mean the the possibilities are endless I think that probably a lot of people have had to like the problems that we're solving are problems that developers have had before but we haven't had these tools to do it this well yeah so I think that there's probably even different scenarios where people have tried to solve this and probably not get nearly the results that you can now do by adding in on Vector search and um these different embeddings into what you already have absolutely it really really has changed the game that's for sure okay so we went over a lot of really cool information on how to start using Vector search within Azure cognitive search where can people go to learn more and get started yeah so I think the best place is what we can refer to that back to this blog post that we're we're talking about but we also have the Azure cognitive search documentation which is always a great resource to to learn more and get started as well awesome so go check out these resources get started and thank you so much for hanging out with us today thank you foreign