Build a RAG System from Scratch with Python

Key Takeaways

Hey everyone, welcome back. In this video, we are going to build rack system from scratch and show exactly where ones break down. To make this concrete, we are going to use real data transcripts from three legs Fredman's broadcast. Jensen Yuang of Nvidia, Sam Altman of OpenAI and Elon Musk. And we'll build a simple rack system first show how it works and then hit with question it cannot generally answer. So let's get into the code. So before we start with the code you have to configure your env file with open AI API key and here we have set rack chunk size as 500 rag overlap as 80 rack top k as 20 and rack top end as five. So let's get into the code. So first we have imported the necessary libraries we need. Then we have loaded our env file and after that from the env file we have extracted the parameters like rack chunk size rag overlap rack top k and rack top n and after that we define two data classes. SRT blocks holds one subtitle block. Its index start and end time stamp and text junk data classes where we actually store and search. It carries the text, timestamps inherited from the SRT, character positions in the full transcript and the source file name and a token count. And after that we start with the phase one where we read the SRT files, split it on the blank lines. And that's how SRT blocks are separated. For each block, it extracts the index, pass the time stamp arrow notation and joins the remaining lines as text. Invalid blocks are silently skipped. After phase one, we start with the phase two where we have two steps. First, we define a function which strips noise. Transcript credits, bracketed annotation, musical symbols, HTML tags, anything that isn't actual speech. Then, we define another function reconstruct full text, which merges consecutive blocks into sentence level segments by accumulating blocks until it hits the period, question mark, or exclamation mark. This restores the natural sentence flow that SRT format breaks apart. And now, phase 3 does sliding window chunking over those segments. It uses stick token to count the tokens accurately, build chunks up to 500 tokens and then steps forward while keeping an 80 token overlap between chunks. It also build a character to segment lookup. So every chunk inherits correct time stamps from the original SRTs. Now phase four is dense retrieval. The vector store class to apps chroma DB with BG small embedding model. Documents get a special prefix represent this sentence for searching and queries get a different prefix represent this question. This asymmetric prompting is a BG best practice that improves retrieval accuracy. And then finally, the results are scored by cosign similarity. Now, phase fire adds a keyword-based layer using BM25, the same algorithm behind classic search engine. It tokenizes chunks by lowerassing and stripping punctuations, then scores them against the query by term frequency. This catches the exact keyword match that semantic embedding sometimes miss. And now phase six where we combine both retriever using reciprocal rank fusion. Instead of merging raw scores which are on incompatible scales, rrf uses rank position. Dense results get 60% weightage while sparse gets 40%. Each junk's final score is the sum of these weightage from each list. The result is a single rank list that benefits from both semantic understanding and keyword procession. And now phase seven does a final quality pass. The top 20 fuse candidates get scored by a cross encoder MS macro mini LM which reads the query and each chunk together rather than as a separate embedding. This is slower but more accurate. We keep only the top five. Now everything is tied up together in the podcast rack class. The ingest method runs the full pipeline. Parse clean chunk embed BM25 pipeline for one file bit dis caching. So reuploads are instant. Multiple transcripts are accumulated into single unified BM25 index and Chromma DB collection. So every query searches across all loaded files simultaneously. And then we have the retrieval which uses hybrid retrieval plus reanking across all ingested transcripts. We format our context. And then the load state method restores all of this after a page refresh. And now that we have seen the entire pipeline, let's see it in action. The entire back end is exposed through a streamlit interface app. py. I'll run it with streamlit run app.py command and upload the sam altman podcast srt and start asking question. So I'll open the terminal. So let's run our streamllet app. So first of all I'll activate my condor environment and that cond activate. Then I'll run the command streamllet run app. py. Let's run the command. Okay. So here I'll drop my sam altman podcast transcript. I'll click process. After processing the transcript, we can just ask our questions. So our transcript is processed. So now let's ask a simple question like what does Sam Alman say about the OpenAI boot saga and how did it affect him? It should answer the question correctly because this was discussed at a length in a concatenated part of the transcript. So let's see if it answers correctly or not. So it answers like in the podcast Sam Alman described the open air saga as definitely the most painful professional experience of my life and the chaotic and ashameful and upsetting. Okay, that's correct and that's what he said in the podcast and here are the sources it used. Okay, that's correct. So now let's upload other files too like Elon Musk and Jensen podcast transcript and let's drop them and ask it to process it. And now I have uploaded three different transcript files and I'll cross question it now. So all the transcripts are activated now. And that's correct. And now let's ask a different question from a different transcript and that is how does Jensen UI explain why Nvidia moved from chip scale to rack scale design. And similarly it should answer correctly because the answer is in the transcript. And Jensen Yuang explained that Nvidia moved from chip scale to rack scale because the problem they are trying to solve is no longer within a single computer that can be accelerated by one GPU. Okay, this is also correct. And now I am going to ask a question which will definitely break this rack system. And the question is why did Elon Musk who co-founded OpenAI and originally tried to recruit Ilia away from the Google for his own AI initiative end up end up suing OpenAI instead and what company did he build as his alternative so I'll hit enter so our rack system answered this but look at what it missed the entire Ilia angle is gone the recruitment story the tough battle against deep mind to get Ilia who was the lich chin of the open AI exist only in Ilon transcript and rag didn't retrieve believe it. It also got the lawsuit motivation slightly wrong, framed as he wanted control rather than open-source betrayal principle. And now I'm going to ask a question which will definitely break this rack system. So I will ask why did Elon Musk who co-ounded OpenAI and originally tried to recruit AI away from the Google for his own AI initiative end up suing OpenAI instead and what company did he build as his alternative? So let's wait for the answer. So, we got a partial answer, but look at what it drops entirely. The Ilia angle, the story of Elon recruiting Ilia in a tough battle against Deep Mind. Alain recruiting Ilia in a tough battle of Deep Mind. It also gets the lawsuit motivation wrong, framing it as Elon wanting control rather than open-source betrayal principle that Sam Alman describes. So, Rag gave us the half story and got the motivation wrong. Now, that's it for this video. In next video we will understand the limitation of this rack system and see how the graph rack performs on the So that's it for this video. For that's it for this video. In the next video we will understand the limitation of this rack system.