Discover LlamaIndex: Bottoms-Up Development With LLMs (Part 1, LLMs and Prompts)

Key Takeaways

hey guys welcome back Logan here from love index today we're starting a new video series Bottoms Up development with llama index now what do I mean by Bottoms Up development well within Luma index there are a ton of components and any number of these you're probably going to need to customize when you're building you know your own query pipeline for your own application and so this video series is going to kind of cover these low-level components how you can customize them and how you can kind of stitch them all together into your own query pipeline so in order to kind of demonstrate this we have a project that we're going to kind of cover across multiple videos we're going to be building a sort of query engine or query pipeline across the long index documentation and through this process we're going to go through a ton of like really low level components lom's documents and nodes retrievers and show you how to customize these and how you can kind of stitch them all together and as well as how you can test and debug each component and so starting off first we're going to cover llms so right now in lab index we have a ton of moms that were recently introduced we have open AI hugging face any LM from Lang chain is supported uh Azure was just added it's actually not even on this list uh and you know custom llms and each of these have really low level basic methods uh complete for basic text completion uh chat for like chat response to like a list of Nest chat messages as well as streaming for each of those and then as well as there's async versions of all of these and so using these at this low level is super simple as we can see in this example down here you can instantiate your LOM super easily and really quickly you know ask it tell you a joke it'll give you a response you can look at the response text or you can actually look at the raw Json that came from that LOM response in addition to that there's also the kind of chat methods so in this case now the input instead of a string is going to be a list of messages and so this in this simple example here we have a system message asking it to talk like a pirate and then the same thing again the user is asking to tell me a joke and here uh chat ubt does its best to tell a pirate joke I guess not quite following instructions but you know close enough and that's basically it to these LOM components they're super simple to set up and use and it's a really good way to start testing these llms for your application you know we can prompt it with specific things that we're going to ask it later using you know documents that it's going to read later and really get a feel for you know is this llm going to work for my use case or do I need to do prompt engineering or find a different LOM and so in order to kind of walk through this a bit more I'm going to go through a notebook here that actually you know demonstrates this so like I mentioned before we're going to try and build a kind of query engine over the Llama index documentation it's going to be highly customized and basically touch like everything that can be customized but starting with llms um you can set your API keys right now I've just cut copied the documentation from llama index so I'm going to read a quick markdown file it's the starter example gonna instantiate our llm just like we saw before here I've copied the kind of internal prompt templates from llama index the two main ones that get used the most are the text QA template and the refine template the main difference between these two is that the refine template takes an existing answer and tries to refine it so either updated or repeat it after reading some new context and this basically only gets used when the text retrieved for a query doesn't fit into a single LOM call so we need to refine it and so these are the two prompts that I want to test right now I'm going to set those up and from there we could basically just start testing so using the starter example I'm going to ask you know how can I install llama index to install on the index you can follow the okay yeah that good answer uh next we're going to ask it how to create an index this one's gonna be a bit more complicated I think there's a few more steps to how to do that yep so it gave us a response here a pretty detailed it looks pretty good but actually I don't need all this information I just want it with code so let's see if we can get it to write just answering using code in this case I'm also going to use the stream complete method so this is going to return a generator and for every response in this generator we're going to print the Delta which is just you know the new change that has come in from the LOM or the new text so it's going to stream that really fast and that example looks perfect so obviously this is working pretty well with this template and with this content that we've loaded uh and we know this because we're testing this low-level Lon API that's the refine template here as well um here I give it one of the answers it gave before as an existing answer and I'm going to say no I want this to write your answer using only code so let's see if you know the LM can handle this refined template now I gave it encode not entirely all in code but this does actually look really correct and this is a it's a good answer nice in addition to this there are also the chat endpoints which as I mentioned before need a list of chat messages this is useful if you maybe want to you know give it a system prompt in this case we're giving it a system prompt telling it that it's a QA bot answer questions about Lum index so if we ask it how to create an index again we should get something uh pretty helpful I would say and again a long detailed response uh which is fine by me so yeah that's basically it uh in this video we kind of covered how to use these logo element low level LOM apis uh and how to kind of quickly get started with testing inputs and outputs using your own data and this is super helpful because if you know if you have a super specific use case you want to make sure that the llm you're using can handle you know the use case that you have intended for it in this case it looks pretty good so in the next video we're going to move on to testing uh document loading and how we can construct documents and nodes from the Lum index documentation so see you in the next video thanks