Three reasons why your business shouldn't build a custom LLM

Key Takeaways

thinking about building an llm for customer service based on your business's data my advice don't and here are three reasons why cost talent and [Music] prompting hey it's Pete and today on context I'm going to let you know why you shouldn't build an llm that's specific to your business all right let's start with reason one cost so a model the size of gpt3 that's around 175 billion parameters cost open AI around $5 million and about 9 days to train and unfortunately gpt3 just isn't up for the prime time for most customer facing use cases then there's gbt 4 which costs around 100 million and took way longer to train so if you're going to build an llm then you're going to need to be okay with spending a lot of coin now let's go to reason number two Talent I've seen a few Fortune 500 businesses dip their toe in and try build or fine-tune their own llm and so far none of them have got it right generally the build is led by a team of data scientists and while they might be the best in the business they're not AI researchers and Engineers of course they get the theory they understand how an llm is built but most likely they've never actually done one before and to be honest not that many people have there is a reason why open AI has salary packages at around 900k per year I bet you're currently questioning your career choice anyway this stuff is just really hard all right now for my third reason and that is prompting is all you really need so what do I mean by this well a recent study called can generalist Foundation models outcompete special purpose tuning case study in medicine Jesus can we just take a minute and seriously who the hell names these papers anyway this paper showed that effective prompting can unlock lock deep specialist capabilities in generalist models like gp4 without the need for extensive fine-tuning so how did it do this well it's all about a strategy and in this case a smart one called Med prompt so how does Med prompt work well you can kind of think of Med prompt like a highly skilled assistant who knows exactly where to find the best resources in a vast library and instead of reading every single book kind of like what you would do if you were to train an llm the assistant quickly picks out a few of the key resources that provide the answer in technical terms this is what Med prompt did it used a method called K nearest neighbor or KNN to find the most semantically similar training examples from a data set based on Vector embeddings of questions it then surfaced these to gbt 4 which then used a prompting technique called Chain of Thought to work out the answer the result gp4 with Med prompt not only matched but outper per form specialized models that were fine-tuned to medicine and by a significant margin so why why did this happen well you can kind of think about it like this are you more likely to answer a question by looking at a textbook with the answer to the question or studying hard and using your memory it's the book right and that's essentially why specialist llms don't perform as well they have essentially read all the books and they're trying to generate an answer from memory remember last episode where I went through the pros and cons of llms and nlus well there I talked about how llms are just trying to guess the most likely token in a series of tokens and while they do that really really well they don't actually have an understanding like you and I of what they're actually saying therefore the most important thing a large language model can do is coherently put together text and if you give a language model a question and the answer it's probably just going to nail it so what what does that mean for you well Med prompt what it's doing is totally replicable firstly use your company data to power a vector database like voice flow's knowledge base to deliver the right information to a large language model at the right time boom secondly try out different prompting techniques like Chain of Thought to make sure your agent can think through its answer before it provides it to a customer and finally remember this key Insight it's not about building the most advanced AI from scratch it's about smartly leveraging what's already there use your data scientists to create algorithms that your agent can leverage like detecting churn anticipating complaints or opportunities for your agent to upsell a product now that's my take on whether your business should build a large language model from scratch let me know yours and remember stay [Music] curious