LlamaIndex Webinar: Time-based retrieval for RAG (with Timescale)

Key Takeaways

hey everyone uh Jerry here from llama index um we have another llama index webinar Series today and today we're excited to feature avar from time scale vector and we'll be talking about time based retrieval for rag so this is very interesting because I think for most uh rag based systems uh you know you don't really think about the concept of time but I've actually heard this from a lot of users on how do you actually think about you know uh well one like modeling and and storing your data with with time information and then two actually getting the LM to query the database with with time information um and so lots of interesting Concepts in here from both like the LM side as well as the system side so I'm really excited to dive into this uh and and after do you want to take a white for sure yeah first of all thanks so much for having us today uh really excited to chat about this whole concept of time based retrieval for uh for Rag and also some of the work that we've been doing at time scale Vector in order to enable this so uh by way of introduction you know Jerry mentioned uh my name is afar uh I think my official title is like product lead for time scale Vector uh but I really do a lot of things from you know writing code and examples that you're going to see today to you know giving presentations like this so really excited to tat you all I'm I'm uh speaking today from from New York City um let's get into an overview of what we're going to cover today here's a a short agenda for you know what you can expect from today's presentation um we'll start off with some motivation for what is time based retrieve you know Jerry mentioned that a lot of the time uh when folks think about R they just think about you know similarity search and trying to structure your knowledge based in such a way that you get the best results uh so I'll do some motivation for what is time based retrieval and why it can be actually be helpful for your RG systems uh secondly we'll take a look at some use cases that you can um design in mind uh with when you're actually building uh RG systems with time uh and I'll actually cover a real world use case of a customer of time scale today uh a company called Market reader uh so you can look forward to that uh thirdly I'll get into some of the technical challenges with performing time-based retrieval efficiently uh that a lot of ecta databases uh run into and then I'll cover the solution that we implemented with time scale and time scale Vector uh and then I'll go into some of the code for how you can actually leverage this capability in llama index so times Vector has an integration in llama index and uh we make it really easy for you to take advantage of a lot of the uh complex like systems level optimizations that we add uh in really a few lines of code AS you'll see uh finally I'll get into a live demo uh we just finished up uh this sample application that allows you to chat with uh a git commit uh your git commit history so I'll show you uh that in action uh that's built with streamlet and and llama index and times CCTA and then finally we'll get into some Q&A uh at any point during the session you know feel free to put your questions in the chat I'll address them like uh live if uh if it's quick but otherwise you know put it there and then we can deal with it at the end of today's session so that's an overview of what you can expect today let's get started into the first part which is what is time based retrieval so let's start with uh you know motivating what is uh what is simple retrieval I think this is the best way to start simple retrieval is basically when you uh want to retrieve records from your vector database that are most semantically similar to a given query so this is kind of the basic r that many of us have learned about and then what you do is that once you retrieve those records you can use that as context for an llm completion query and this is the uh augment in the retrieval augmented generation uh acronym so that's simple retrieval now the issue is that vectors are not uh vectors don't exist in a vacuum this is the alliteration that I came up with to explain the importance of metadata so often you don't just want the vector representation but you want uh the things that are associated with the vector vectors often represent chunks in documents they represent user messages these things have their own metadata associate associated with them and it's often important for us to incorporate that into our queries and into our uh RG systems in order to get better results for users so this is where uh metadata comes in and in particular time as a metadata component now I just mentioned your vectors they representing these real world things their documents their images their web pages and often these things have a Time associated with them for example you'll have a creation date of your document you'll have a publishing date of like an article you'd have a last updated uh date for a web page on your website so there's time associated with a lot of these things and what you can actually do is Leverage this in order to give users better results so coming back to the definition you what is time based retrieval the goal here with rags to give users the most relevant results possible what we can do the Insight that we have is you can actually use time as a search filter to increase the relevancy of results so uh that gives rise to time based retrieval where you retrieve vectors that are semantically more similar but they're also pertinent to a specific time frame and we'll see some examples in a minute about how you can constrain that time frame so timebase retrieval is essentially a two-step process step number one is to perform a similarity search with a Time filter and then two to pass on that context uh that uh both contains relevant information but relevant within a specific time period to the llm uh so that's an overview of what is time based retrieval let's get into now that you understand a little bit about the concepts some uh applications and use cases for or using them in your uh rag systems to give users better answers so I'm going to go through an overview of some of the use cases how you can actually Implement time based retrieval today and like what kind of problems you can solve with it so the first one is just search within a Time range so for example sometimes uh you know we're often building these uh chat with your data chat with your document um chat with your document kinds of applications uh and so what you can do is filter documents by things like create date or last update date or give the users the ability to constrain the search so that they can find the most relevant uh relevant documents um so that's number one number two is things like chat history so when you're building these chat Bots often uh you'd want to store the timestamp that the user sent the message or that the llm uh gave a uh or the AI Assistant gave a response and so what you can actually do if you're building these chat Bots and you want to do analytics and you want to do kind of analysis and summaries of a conversation uh you can search and retrieve chat history from a window of time in the past and that's uh one other capability that you can add with uh with time based uh retrieval thirdly uh one interesting use case is just to find the most recent embedding so for example let's say uh you're building an application and you want to find the most recent news or social media post related to a certain topic um and this is uh you know one of the main applications that we see and I'll show you a real world user uh in a in a moment and then lastly uh you really can use it to give an llm a sense of time so you can ask time based questions about a knowledge base this is something that I'll demo for you uh later on in the presentation um using as J mentioned earlier you basically give the LM this tool to say hey you can make these time filters uh and and answer uh these time based questions um Alex says in the chat uh he's using slack chat history as one of the the sources of knowledge and he found that time based retrieval in this case is a must 100% so that's another good example slack messages is is another one where you know and often the time is a is a is a marker of relevance uh you know we come back to this idea that we want to give users relevant results often you have messages that are very old that can be um you know less relevant than messages that are newer and sometimes vice versa sometimes you want to look at you know all the documents to find out things and and shun the new document so that gives you this idea of like using time as a proxy for relevance uh and implementing that in your systems so outside of llms uh there's also some use cases I just wanted to mention one is image similarity where you'd look at uh the images that are most similar but like from video in a certain time period um and then lastly anomaly detection where you'd want to look at uh you know anomalies uh you you compare a certain Vector to anomaly vectors uh from uh specific time period back in order to see like ah is this actually an anomaly or do I need to take action on it so these are some outside of llm use cases that I thought might be interesting um okay I wanted to talk to you about a real world company doing this this is actually a customer of time scale so little small disclaimer but I thought the use case was very interesting there's a company called Market reader basically what they do is uh you know they website says no what's moving and why what they do is they have um a a feed of asset prices so they have this time series data of asset prices stock prices and what they do is they help users understand why is the market moving so they see a certain movement in the market the stock might go up and down and then what they do is they want to find the most relevant news related to a specific Market event and so what they actually do is this uh you time based retrieval and action where they want to find the most pertinent news stories or news headlines related to a market movement uh but it needs needs to be recent it needs to be in the in the in the last uh you know 5 minutes or last 30 minutes or whatever time period that's interesting and so this is a you know really um a use case that exemplifies why time based retrieval is important because you want to do a similarity search on the topic maybe it's a stock name or something like that but you want to constrain that in the time period That's relevant uh for uh for the user which is usually you know the past minutes or past hours or so okay so uh Alex also says uh you know relevance in this case is key otherwise llm gets outdated and ends up hallucinating 100% right so thank you Alex for your comments really appreciate it um cool so now you can see this is an example of an actual company doing this uh and you know some inspiration for how you can uh do this in your uh applications now time based retrieval sounds great you know why why why doesn't everyone use this why isn't this used more uh let's get into some of the technical challenges and and how we at a time scale um propose a solution and what we implemented in time scale Vector so the challenge is that most Vector databases don't handle time well um you know most vector databas vector databases have kind of risen to fame this past year uh there seems to be a new one popping up every day and they handle Vector data really well you know they have these a&n indexes they have hybrid search capabilities Etc but time seems to be something that they don't uh handle very well and uh you that's mostly because they're specialized for Vector search in itself now uh traditionally there's two ways to deal with this uh when you have a vector database and you want to do Somey search with time filters and unfortunately like both of these methods don't work in practice so let's let's look at them one by one the first one is this idea called post filtering which is basically you know we talked about time based uh retrieval as a two-step process one you want to do your similarity search uh and then you want to have that um represent the only the vectors in that specific period of time while post filtering what what you do is you apply the time filter after a similarity search now uh what this looks like is you'd find the similar embeddings you'd filter out the ones not in the time range and then you'd uh remove then you'd return the results uh I see Ahmed ask a question I'll will answer that question in the Q&A part so thank you Ahmed um we'll will uh will'll get you in the Q&A okay but back to post filtering so post filtering you find SAR bearings you filter out the ones not in the time range and then you return the results now most of you can probably see the problem with this which is that when you do a similarity search some of the results that you get might not fit in the time range that you're interested in and so what happens is that you could end up with uh you know in the best case fewer results than you expect or in some cases no results because all of the things that you've returned from finding the most similar vectors just fall out of the time range that you're interested in so post filtering is not uh you know the best solution to use here there's this idea of pre-filtering which is applying the time filter before similarity search and this kind of seems what we want this seems like something that we want where you constrain the search only to the vectors in the relevant time period and then you perform the similarity search and return the results now the problem here is that most Vector databases aren't optimized for this kind of search because what happens is uh when you have an index on a collection or on a table or something like that uh the index is built over the entire set of vectors and so in order to use the index you need to uh search the entire Vector set and obviously um you know if some folks might know a little bit about approximate nearest neighbor search I won't get too much into it but basically there's all these uh fancy methods to Traverse graphs and things like that in a in an efficient way such that you can uh search through millions and billions of vectors in a short period of time but then again those graphs and those things are built off the whole Vector Set uh and so what happens is that if you want to filter out the the data that's only IR relevant in your time period what happens is that you end up not using that index because you're trying to like uh look at only a subgraph and again your graph is organized by semantic uh similarity or in this case represented by like the the vector embeddings and not by time and so you end up with slow queries because what happens is that you may be able to do a Time filter but then you just resort to doing a a brute force uh exact nearest neighbor search in the second step of Performing thear search and that often you know can take a lot of time and lead to slow slow queries so now that we understand the problem and understand these two approaches it kind of looks like you know this pre-filtering approach here this looks pretty good we kind of want something like this but we want to do it in an optimized way and so that's what we set out to do at time scale um and you know we we created this product called ector it's actually recently released in the past month and um you know I'm going to take you through right now about how we solve this uh hybrid such uh this sorry this this time based retrieval suchar problem using some of the capabilities uh that we introduced in the time series uh database product uh and combining that with Vector data to enable this uh unique uh time based retrieval capabilities but in a very efficient man so time scale Vector you know one thing to take away from this presentation today is that time scale Vector is a database for Vector relational and time series data so it basically combines uh all the uh all the the features of a Time series database relational database and lets you handle Vector uh data as well and so what uh what I'm going to do now is I'm just going to zoom out a little bit and tell you about times C Vector in general and then we can zoom back in to the time based search capabilities and I'll show you how we kind of uh solve that in uh in times SC Vector so just for your general knowledge you know what is timec Vector uh it's one Cloud postest database for Vector relational and time series data you know over here uh we basically take postgress and extend it for time series data using the time scale DB extension and then we extend it further using a PG vector and our own uh extension for handling Vector data so let's see what that looks like uh in a in a diagram so the first step is uh a cloud data platform so times SC Vector is a cloud database uh it has all the good things like ha and replicas and uh SSO and all that stuff uh then we have um the postgress database so each time scale Vector database is a postgress database and this allows you to store relational data and then you also get a time scale DB uh extension added on which gives you time series capabilities one of the capabilities that we're particularly interested in is this idea of automatic partitioning by time now this is something that uh you know when you uh when you take the capabilities of a Time series database you're often quering data by time and so you want to organize your data in such a way that it's really efficient to queri it uh by time uh and this automatic paring by time is what uh what we uh one of the capabilities that we introduce so on top of that uh we have some of the vector capabilities you get PG Vector indexes and the vector type and then time scale Vector adds its own search index uh inspired by the dis Anan algorithm as well as some capabilities for hybrid search uh by metadata and time filters which is you know what we're talking about today and then we have these integration and libraries uh one of which is llama index uh to allow you to basically take advantage of all the optimizations uh in the Frameworks that you use to build your app okay so now that you understand what is time scale Vector basically it's this Cloud postest database that can handle Vector relational time series data let's take a look at how uh you can actually efficiently uh time filter vectors by using time scale DB's capabilities and uh this uh Insight here is that like you know if you actually associate uh time with the vector you can actually utilize some of the time series capabilities in order to really uh speed up this time filtering and one of the key abstractions that we introduce in time in time scale DB and one of the key time series features is this idea called a hypertable so uh those of you who might know uh MySQL or postgress uh might know the concept of tables a table is just something that you uh you store your data in a table uh and a hypertable basically what it is is that it's a collection of smaller tables so you can imagine you know each of these little uh uh chunks here are themselves tables and what happens is that time scale Vector when uh data is ingested into the database it gets automatically partitioned by time into uh these uh smaller tables uh we call them chunks so each of these tables uh will contain data that's for a specified period of time maybe it's 7even days maybe it's 30 days maybe it's one year the user can actually specify that and what happens is that when you actually uh query the the the database you query it as if it's the single table so that's why it's called a hypertable so you query the hypertable but under the hood what happens is that when you're doing these timebase queries the database will uh know and exclude certain chunks or certain subtes that are not relevant so for example it'll know like hey I only want to search you know this uh these subtes for the query and we don't have to worry about all the other ones CU that's going to be efficient so that gives you some intuition for hyper tables and this is kind of the key feature that we leverage and so what we do when we do efficient time based Vector search and times Vector is really leverage this hyper table capability so the first thing we do is the data isn't just stored in a single table but it's stored in this hyper table so when you ingest it the data gets automatically partitioned by time into these smaller chunks uh so these are once again this is when I say chunk I just mean the subtable here here then what happens is that we create uh an approximate nearest neighbor index on each chunk so rather than the index B for all of your data we create it on a per chunk basis so that each of these subtes here have their ownn search index and then what happens when you do a search we can then perform the pre-filtering really efficiently because now we only need to look at the uh chunks that match the time predicate and we can filter out the chunks that don't uh so pre-filtering that that critical step is really efficient and you can perform the similarity search using the Ann index because those indexes are on a per uh chunk level rather than uh for all of your data and then what happens is that soorry I went too quickly so you perform the similarity search and then you can combine and rerank your results from the search using a merge sort in order to get uh both efficient time based search but also take advantage of that approximate nearest neighbor index so that's a little bit of theory for you um some background for some of the uh you know key capabilities that we introduced in time scale vector and why this idea of time based retrieval is kind of tricky when you dig into the system internals uh and we managed to you know come up with the solution that we offered through time scale Vector so now that you understand some of the theory behind this let's take a look at how you can actually Implement time based retrieval in your rag system using uh our favorite data framework llama index and time scale Vector as your vector database um so I'm going to cover the following things uh that we uh we at timescale Vector offer in llama index uh in order to make time based retrieval really easy so this is an overview and then I'll go through each of them one by one uh let me just take a second team okay so the first one is we need to have this uh notion of time in our nodes so nodes are these you know first class citizens in llama index so we need to be able to create uh have some time component in them and what time scale Vector does is allows you to create nodes with time based uid so we'll cover how to do that second you want to load your node into the vector store and you want to enable auto partitioning the good news is this is like a oneline uh setting that you enable so we'll cover that thirdly you know adding vectors sorry adding the index to the vector store you want to create that those indexes on the the subtes this is again you know on line procedure uh and then we allow you to do similarity search using a Time filter and then some of the more interesting things performing uh r with the time scale Vector store as a vector store Retriever and or a query engine and then in my opinion the most interesting thing is performing uh rag using the vector index Auto retriever where rather than specifying the uh time filter parameters yourself you can basically let the llm infer that from a user query so now that's this is an overview let's take a look at the code for each one of them um and once again you know at the end of this presentation I will send you uh I will give you a link for where you can find all of this code uh to use in your applications uh for now I've just put uh uh you know these images of of the code blocks on the screen um so you don't have to take a screenshot or anything these I'll send you the GitHub repo uh at the end of this presentation okay so the first one is time based uu IDs so the first idea here is like you want to have um the notion of time in your node and so what we do is there's a function over here to create a node um we want to uh in the ID for that node have uh time uh baked in so we use this uh uid type called uid V1 and what we have is in the library uh in the time scale Vector python client Library this function uh called uu ID from time which takes a daytime object and returns a uid that you can then use Excuse me uh that you can then use as a ID for your node in uh llama index okay one good uh good piece of news is that you only need to worry about this if you want to create nodes with time stamps in the past so this is like past data if you're like dealing with data and you want to have the current data and time associated with it uh you can just create a node uh as you normally would uh using um uh using Lum index and when it gets inserted into time scale Vector uh we automatically create a uid with the current data and time so you only have to go through the step if you want to associate a time and date in the past uh and again you know it's really easy uh there's a function that we provide to create this uid from time uh and that's how you can implement it so once you've done that what you want to do is load your nodes into Times SC vector and uh you want to do this in such a way that that auto partitioning that I described excuse me in the previous section is automatically enabled and as you can see you know llama index provides a really easy way to create uh Vector Stores um what you want to do here is you know import the time scale Vector store and then uh initialize it here with the service URL which is you know as I mentioned it's a cloud postc database you get a URL to your service you set your table name and then the most important part is to set a time partition interval so this would be the uh you know amount of time that you want each of these subtes to be uh representing and in this example I use seven days but you should pick a value that uh makes sense for your use case maybe if you're quaring data over decades then you want that to be like you know one year or something like that or if you're carrying data you know in the in the time span of like days and months you want to have it be uh you know in 30 minutes or in 60 minutes or something like that and so once you've set that to the time data that you want uh you just add the nodes to the uh Vector store and time scale Vector under the hood takes care of automatically partitioning it into the different subtes uh you don't have to worry about any of that so that's how you load the data into the vector store then adding the vector store uh adding the index to time scale Vector is actually just a oneline um oneline operation uh you just run this create index um create index uh command and uh it will add the time scale Vector index uh in this case is a disn index to your uh Vector store and once again you know that index is going to be um created on a per chunk basis in order to take advantage of uh the time base filtering uh we also offer other index types so there's hnsw and there's IVF flat both from PG Vector uh but in this example I'm just using the default one which is the times SC Vector dis n index so that's the index part you know one line of code and then we get to the similarity search so I'll walk through this uh you know part by part if you want to perform a similarity search what you first got to do is Define your query string so in this case I'm asking you know what's new with time scale DB functions uh then you want to have some sort of filter variables for your query so you can either have uh timescale Vector provides three ways to filter um to perform a sar search with time filters so in this case I've got the start date as the 1st of August the end date as the 30th of August and then a Time Delta of seven days we'll see these different combinations then what you do is create whoops then what you do is create a vector store query in this step here and then there's three different ways that you can actually do the sari search so the first way is you define a start date and an end date uh and pass those as arguments so when you have uh this when you query your vector store you'd pass in the vector store query but then you you would also pass in the start date and end date uh which is a start date and end date that we defined above and that constrains the suchar to only uh vectors within that start date and end date the second method to do it is you can return the most similar vectors to a query from a particular start date and a Time Delta later so in this case uh you define instead of passing in the start end date you'd pass in a Time Delta in this case it would be like you know from the 1 of August and seven days later but it could be you know end days an arbitary amount of time later uh some queries uh you know do fit that that form quite well and then you know another common one is you know uh finding this this common form of query is good for finding like recent events or recent news and things like that where you define uh an end date and a Time Delta earlier so in this case it's like from the 30th of August and seven days earlier uh this is useful for finding you know recent news recent events things like that so that's how you do the similarity search with the time filter you can get a bit more complex by using the Retriever and the query engine and in this case you do everything that I did over here by creating your uh time scale Vector store and creating an index uh you create an index here and then all you do is when you create the retriever you need to specify this Vector store keyword ARS uh argument and you set your start date and your end date here uh and then very similarly when you initialize your query engine you must make sure to set this Vector store keyword a so that the query engine knows which start and end dates to use okay so this is how you would do it uh you know if you uh just want to take advantage of the time based sity search and using llama index I'm going to get into this idea that we actually just implemented this yesterday so this is very exciting uh this is the auto Retriever and Alex asked is there a way to let the LM decide if it should care about the time frame depending on the question ask and this is exactly that capability where you actually let the llm infer the time filter from the user query and there's like two steps in order to enable this so the first one is uh you need to set your vector store info and this is basically telling the llm uh hey tell me about your schema and your metadata so tell me about all the different fields tell me like you know what type they are how do how do I like basically give me some information so that I can know uh when to use this uh you know personify GPD for for a second there then uh what we do is we'd want to use this Vector index Auto Retriever and what we do is we pass a couple of things to it the first one is we pass the vector store info that we set on the slide uh before this uh once again I'll give you the code so you can uh dig into it a bit more and then then you what you do is you essentially want to build a query engine from that so then we build the query engine from from our uh vectors Vector index Auto Retriever and then what we do is we actually give it uh we convert the query engine to be a tool for an open AI agent to then leverage uh and interact with the vector store so this piece of code here basically converts the query engine to a tool and then what we do is we create this agent uh that's then going to uh answer our questions um and we give it access to the tool here so we say you know the tools is going to be the query engine tool and then when we want to answer the question we can chat with the chat engine and we make sure that it can have this function call available to it of query engine tool so what actually happens then is that uh the openi agent can understand both the vector store information that we passed to it and it has the ability to query the vector database and so when you are answering a user query it's a two-step process you make two open AI calls the first one is to um you know given a user query and some query parameters like what do we uh ask like what should our query be can you like tell me what filters and things that I should use and then secondly given that user query and some results from the vector store what is the answer can you perform a completion that's kind of the rag portion of it so that's the theory of the auto retriever uh does this tool also work with llama models that's probably a question for Jerry uh at the end of this uh I think so maybe in the future uh for now we just use the open Agent uh but we'll come to that at the end of this uh at the end of this session so that's the auto retriever at a high level once again you know I will get into the code and things uh in the resources section uh for the next five minutes or so I'm just going to give you a small demo of what we call the postgress time machine which is showing you time-based uh retrieval and rag in action where you can actually chat with the git commit history uh and this demo is built uh with a few things so just to give you an high level overview this uh as I said postgress time machine you can chat with the git commit history of the postgress uh database project and we built this using llama index as a data framework uh front end is in streamlit and the vector database is time scale Vector so let me uh exit out from this screen can everyone see the streamlet apps okay so this is a time machine there's a couple of um you know once again I'll I'll link to the code and and things like that for this so you can Fork this and play around with it there's a couple of things that we have so the first one is the ability to load data from a specific uh git repo so right now on this hosted version that I'm using that's disabled but if you're running it locally uh there's instructions for how to load data from a repo that you're interested in and then when you get to the actual chat interface you can select which repo you want to uh chat with in this case I only have one which is postgress so I'm going to select that then we give users various knobs and dials in order to uh Define how many months back they want to search so you can search uh you know up to 130 months back which I think is just over 10 years uh or if you set it at zero it has no limit and it'll just search the whole uh history that you have and then you can also choose how many commits to retrieve uh this is probably not something you'll expose to users but for our demo purposes we uh we had this here and so uh let's go to the fun part I'm first demo the auto query Retriever and show you you know how the llm uh generates different results for different queries so the first one is uh let me make this bigger hopefully everyone can see that uh what changes were made to this feature called jet in post in 20122 so we'll just run that and see what the llm says you know once again under the hood uh what's happening is that first the llm is inferring what query filters to run in this case we want to only have vectors from 2020 to and then once you have that quer filter then performs a sariy search to retrieve that and then does uh rag using those results so let's give it a few seconds um this is uh one of the one of the parts of this that we need to improve which is just making the completion faster but you know those two llm calls taking place so while this is working let's just uh wait a few minutes I've personally noticed dt4 has been pretty slow lately so it's not your fault yeah yeah yeah thanks for thanks for the save there uh so you can see here yeah this is using gbd4 under the hood uh we get a completion saying you know in 2022 these are the changes made it made and then what it does is that in the commit uh in the the data that was retrieved we have you know the author and then we have the um change description of what was made so in this case Andreas uh fre made changes to the function uh to the feature and this is kind of what he did and then the same with uh this guy Thomas Monroe these are the changes they made uh and you can kind of see that hey it references data from 2022 uh so you know it uh you know under the hood it only constraints the such to that let's try the same query but asking it about 2023 and let's see you know hopefully we get different results so that's going to run um and you know what uh what you can do is kind of uh you know expose some of the thought processes to the users we haven't done this in this case um but this is an example of uh you know once again the auto retrieval functionality in in action so let's give it some time to think while we're waiting if folks want to like note down their questions in the chat I see there's some so that's good let's keep those coming uh we'll answer them in a moment once this demo is done there we go okay so in this case uh you know we can see uh a little bit of different format this time um and uh these are the just to compare you know what was said in 2022 we had uh this guy Andre his friend making some making some changes well tracking back lvm changes and in this case we can see the uh both the people making the changes and the changes are different so uh we can kind of tell that the llm one is giving a correct answer and then two um using vectors only from that time period as context for their answer uh and you know the key thing here is this this 2023 and 2022 and you know that's the power of the auto retriever where the user can specify the filter in their question and this is really useful so for example like if you're building like a business analytics app and you want your CEO or whoever in the company to know like hey tell me about the performance in the past month uh this is kind of a cool way to implement that capability where people can specify what they're looking for in natural language and then your llm can then you know get them the results that they need inferring the filters uh there's one more thing that I'll fil that I'll demo which is um just about uh uh yeah just about like you know the L&M knowing like uh when and um inferring like when the first commit for something was um so I'm going to ask who made the first changes to J and when did this happen so let's see what that gives us uh you know once again you can play around with this for any repo you're interested in I think the next version we should probably make is for llama index itself just to see you know when different features were added and what was some of the things behind it um this is I'm not sure if that's right actually uh but this is kind of the um the answer that we get and you know once again you can play around with the how many months back you want to search and how many commits you want to retrieve uh for the sake of time and and Force questions I'm going to end the demo here but uh you can actually play around with this uh all the code for this demo is in this repo called the vector cookbook and uh it's in this folder called tsv time machine you can Fork this and there's instructions about how to basically set up your own version of this and you can play around with it and explore uh the coding in more detail uh so that's it for the short demo maybe if there's time later we can get into more examples but um you know just some next steps for you you obviously learned uh a lot from this uh this webinar hopefully and here are some next steps to guide your learning so the first one is a tutorial that we have in the Llama index docs which is about how to use the time scale Vector store and there's a section there about performing similarity search with time based filtering so it goes through all the code that I covered and uh gives you some commentary for how that works and and how you can use that in Lama index then you know as I just uh showed you on the other screen uh you can try out the sample app s just go to this uh time scale uh Vector cookbook repo on GitHub and you can find the the code to uh clone and and get started with from there and then finally uh if you want to learn a bit more about timescale vector and llama index uh there's a Guest block that we did that Jerry published a couple of weeks ago uh on the Llama index blog that you can uh learn more about and uh finally you know as a thank you for everyone attending this webinar today and everyone who signed up uh we're giving llama index users 90 days free of time scale Vector so once again it's a cloud postgress database so your database is in the cloud uh and you get 90 days free to play around with it you can actually use it to spin up the sample app that I showed uh and you can go to the URL on the screen in front of you which is ill.com to claim that uh and you can also find uh just navigating to the page really quick in this resources section uh the tutorials uh both for llama index and then also this um let me just refresh the page this uh Vector cookbook which uh you can find uh the the sample app uh living there so that's it from me today uh happy to get into your questions um and really uh thank you so much again for uh for having me on this L index webinar today awesome after thanks so much for the presentation this is uh one really great it was really polished and to it was a lot of stuff uh from both like the systems like uh time time like the the part partitioning aspect all the way to LM based like Auto retrieval I think you showed a lot of cool use cases here um cool yeah I I guess like in terms of the next steps we could go through some of the audience questions first and then if if um if needed I I have a bunch of questions I I can ask to awesome so for the questions if okay I I'll just read it out from the chat is that how you guys usually do it yeah I'll do it okay cool so um the first one that I don't think we answered uh live is from Ahmed who asks can I explain more about Market reader how are they keeping their Vector database up to dat news are flowing in every millisecond are they using llama index to index real-time news and data so a market reader this is a great again you know great question uh they're a customer of time scale and uh basically I think they have these apis and these uh providers of news where they actually ingest realtime streams of news and they store that time series data in hyper tables in time time scale DB itself so uh that's one use case and then what they do is they vectorize that that news uh into a embedding using I think the open AI model and um then they can perform similarity search on that uh in terms of if they're using llama index or not I actually am not sure I need to double check with them uh I can't remember since the last time I spoke but if they're not I'll definitely recommended to them uh to use but I think definitely you can use llama index you know using the uh approaches that I talked about in this session today uh so yeah hopefully that uh that provides more color on that question um then Alex asks can time scale be used with any postest instance for example AWS Aurora so time scale DB uh the way it works is that it's a postgress extension there's actually two um you know to to answer your question the best way to use time scale is uh on our Cloud hosted product uh plug for my employers there uh you know obviously using the product that the folks who built the the extension built is kind of the best way to get the latest and greatest of of time scale but uh you know you can self-host time scill DB as well there is a community version which is free uh but for example on AWS Aurora you only get a limited feature set so uh you know we do see a lot of folks actually switching over for the full feature Set uh from AWS Aurora and you still get uh you know the same postgress and the same postgress ecosystem out a need so hopefully that answer your question you once again you get 90 days free if you just want to check it out I encourage you to sign up and you know evaluate it for yourself oh um oh because because I think that the next like unanswer question I think some of these have been have been answered like in terms of recording the recording is on YouTube uh and so basically all all these webinars get put on our YouTube channel and so um they it'll live on the internet forever but it it's basically good to to kind of like um for especially if you want to look up a tutorial of how to use La index with like time base retrieval it'll be a good resource there um and then I think Alex asked about like the the like Auto retrieval basically which I think which I think you answered um and then the next piece that um I figured I would just take is um Flor does this tool use uh like also work with llama based models or basically any sort of open models um one comment I'll make is that a lot of the concepts that after described is more on the retrieval side so time based ret Ral uh that doesn't actually um require the L alignment in the loop uh so if you basically just want to tune the retrieval portion with with time based filtering you you totally can um and then the LM is just responsible for the synthesis portion in that case I think llama models should work pretty well um the one exception is like tool use through Auto retrieval if you actually want the LM to infer the time base filters the LM is responsible for actually producing like structured outputs that it can use to query the time uh the vector database in that case in that sense like we have found open source models tend to not do as well as like GT4 or stronger models in general um so that's just something I keep in mind especially if you're interested in like the auto roval stuff awesome uh there were some questions I think Adriano and April asked about the types of questions that you can ask I'll say yes to both of them uh it is possible to handle and uh undefined amount of time you know it just such as the whole of the data set that you have in your database so however much data back that's the amount of time that you'll uh search over and then uh April ask you know can you answer like when questions uh yes it can uh in that case I actually can't remember on the top of my head when J was introduced so I'll double check that but you know yeah in theory that definitely does work um yeah and then there's some questions about you know Jeff asked about the time scale to be open source version uh you can try it we actually uh all our demos are using the databases uh in the cloud hosted version uh you know once again I'd recommend using that just because it's easy to get started with but you can try with open source version uh I will say that the time scale Vector capabilities so the disin in index and some of the hybrid search capabilities that we offer are only available on the hosted product uh just because we're trying to move quickly and that's the fastest way to get the software out there um so you know you can uh you know try whichever version makes sense for you there well maybe one question I have actually is um especially on the time based partitioning how do you deal with um document updates like for a given document if time represents like the last update time how do you actually and and then let's say that that uh update time changes because the contents got updated how do you like reindex that document and rebalance like the partitions yeah so what actually happens is we we generally recommend and insert rather than an update pattern uh so what actually happens is just creating a new uh record in your database just like insert it again and then what we actually provide that some of the things that I didn't get into we provide a bunch of data life cycle features where you can actually automatically uh tier data compress data in some cases delete data older than a certain time period And so this allows you to kind of maintain uh you know make sure that your database doesn't blow too much by having these like millions and billions of Records um in terms of how you like want to deal with uh specific like document updates I think like the cleanest way is just to insert a new record and then um there's probably some you know uh edge cases where you know you do actually want to perform the update uh in terms of the table uh it will get uh placed in the partition that makes most sense for it that happens under the hood uh but in terms of the and also in terms of the vector index uh you don't need to actually rebuild the index uh that gets taken care of with the disan algorithm as a new uh new um uh sorry new eddings are are added to the database got it uh so that hopefully that you know gives some insight yeah I think that makes sense I think the fall of the Fall question here is I think Alex actually asked us as well is like is there a way to avoid like duplicates I think I've actually heard this from a variety of users where um oftentimes like maybe the same database will have like outdated information as well as like kind of new and more recent information and I'm curious if there's ways to handle that at the system level for for time scope yeah yeah I think um you know again this is where it really depends on like your application Level needs like most of the time do you actually want to keep around the old record that's a question that you know some applications do for like auditing purposes other applications uh maybe not uh so I think like basically what you want is is uh you know answer those questions for yourself and then I think um you know one way that you can do it is uh to have uh in the um in the time in the time period like a way to basically flag to say hey use the most updated version of this data and one way to do that is to actually tier data from uh one table to another another way is to actually use uh we have a data tiering functionality that actually tiers data that is infrequently accessed to Amazon S3 so uh you can still keep it accessible from the database but there's an option to turn that uh retrieval from S3 off so you can only actually query data that's inside the database there's multiple ways that you can implement it um but uh you know I I hesitate to give an absolute answer just because like I've seen so many different edge cases and uh you know it really just depends on what your application requirements are so yeah there is a way to avoid duplicates you're going to have to do a little bit of um you know work working around for that you know one easy way to do it is just to use an upset that also works uh if you if you don't care about the old value but if you want to keep the old value then you know there's a few decisions you have to make there I see cool s the info um yeah then I I have another question actually oh but sorry you gonna say I was just gonna say uh Florian asked a question I'm just reading it are LMS good at scheduling replanning based on time event chunks I'm not sure what that means Florian can you can you just explain that in the chat um yeah I actually I'm not sure what your what you're getting at there unfortunately you had a question though in the meantime Jerry oh yeah I well I fig for this question it's probably more related to just like um something around like uh the auto retrieval piece uh I'm actually not sure but but we can we can get back to it as well um the uh question I was going to ask is like is there like a preferred um time stamp representation um like is is all all data like kind of index is it like um like Epoch time is it like that that date like date um the year month day format that doesn't matter actually so it actually doesn't matter let me just uh whoops sorry let me go back to one of the slides that I had actually uh which is this slide here accidentally close the chat so let me open that up uh so basically whoops it's not this one it's actually this one here yeah so basically what happens and this is specifically for llama index for if you're just using time scale without llama index uh you can use uh the preferable one is timestamp z uh or time stamp TZ that has a time stamp and a time zone associated with it it's a postgress uh time stamp data type for llama index what we recommend folks to do and the way that we've implement this is actually uh by using a uu ID uh V1 I think yeah uid V1 uh that actually has a daytime portion in it and so what we do is uh you know most of the time uh users don't really uh you know you you have to have an ID and so what we thought is hey if you have to have an ID anyway and we want to have unique times for uh an object and the ID kind of uniquely identifies a node in this case uh why don't we kind of Leverage that and so what we do is uh we use this uh uid uh V Force to say hey if you have a time in the past you can create a u ID from a daytime object and then when you're actually creating your node uh you can specify hey use that uu ID that I just created as the node ID uh and as I mentioned if you don't care about uh having a time in the past associated with if you like you know want to have the current data and time associated with it when you uh create your node or when you just uh there's a function to like add nodes to the uh Vector store uh in llama index uh it'll automatically create this U ID D V1 with the current date and time under the hood and that's only if you enable this time partitions option here uh if that's off it'll just use the standard uid that llama index uses but if you enable time based partitioning that's kind of how we take care of it uh so you know to answer your question the key thing here is to use that uu ID V1 as your node ID is uh in llama index uh that that that that's my recommendation there great yeah makes a lot of sense and then I think for the rest of the um uh the rest of webinar we probably just go through the last two questions and and um and and go from there um so I think Matias asked how does filtering in time scale DB compared to quadrant filtering spicy question currently filtering quadrant based on time yeah to be honest I haven't looked at how quadrant does filtering I you know spent uh uh you know a couple of sections in the webinar talking about how time scale does it again the automatic partitioning by time um that's something we can definitely look into I think you know um again I'm not super familiar with how quadrant does it but I will say you know something in general is that time scale started off as uh a company that was focused just on time series use cases and so what we're leveraging here is a uh functionality that was built specifically for time series data and uh that is uh kind of what we're leveraging so we do have the advantage of you know having a lot of experience dealing with time based and efficiently retrieving uh and efficiently searching over it um that I'm not sure if if quadrant does have those same kinds of capabilities so that's my you know face value answer again I'll have to dig into the details and happy to you know chat offline about that and then I think the last question is just like um could you share the like I guess could you share the hit rate of this technique and I guess the implication there is just like when you do this like time based retrieval uh and and do like a&n with like the the time based partitioning like is there kind of like an accuracy component like is there like some parts where like by tuning the the um time partition you actually might get like slightly different results and ways to think about that so I think that um the Ann index part is I I I would say that in some in this case if so you think about like accuracy of results which is like you know one there's two components one is you know does this result actually fall into the time period that I'm interested in and then two you know is it actually semantically relevant to the query that I'm asking so if you take the first component uh this technique has a very high hit rate because once again you're only examining these subtes these chunks that are uh within the time period of your interest because the data has been automatically partitioned by time and we use this thing called chunk exclusion to exclude chunks outside the quy time range so that's the first component the second component uh it depends on the index type that you're using so in time scale Vector the standard index is a disn index and you can set those parameters there's actually a blog post uh if you go to the time scale uh ill.com website there's a Blog that describes uh the actual index type that we use and you can set certain parameters to basically make this tradeoff between uh accuracy and query speed uh The Benchmark that we have in the blog uses accuracy threshold of 99% uh and shows the query speed at that uh at that threshold uh but if you want to have you know 99.999% there's uh there's certain parameters that you can change I don't want to get into it too much now because it varies uh for the disn one for the hnsw one but in the blog we actually discuss hey if you want higher accuracy this is what you should set it to and if you want higher query speed at the expensive accuracy you should set it to something else and then we give you like a default that has a happy middle ground at around 99% accuracy um yeah that makes a lot of sense that was actually a question I was gonna ask you which is like the the trade-off between the the query speed and the accuracy um okay awesome uh I think we'll end it there because we're about at time at an hour and thanks so much avar for the great presentation and and answering all the questions um we'll have the recording up on YouTube as well uh to to share with the rest of the world but this is great and and um we'll see you guys next week thanks so much again thank you thanks everyone