How to evaluate upgrading your app to GPT-4o | LangSmith Evaluations - Part 18

OpenAI recently released GPT-4o, which reports significant improvements in latency and cost. Many users may wonder how to evaluate the effects of upgrading their app to GPT-4o? For example, what latency benefit will users expect to gain and are there any material differences in app performance when I switch to the new GPT-4o model. Decisions like this are often limited by quality evaluations! Here, we show the process of evaluating GPT-4o on an example RAG app with a 20 question eval set related to LangChain documentation. We show how regression testing in the LangSmith UI allows you to quickly pinpoint examples where GPT-4o shows improvements or regressions over your current app. GPT-4o docs: https://openai.com/index/hello-gpt-4o/ LangSmith regression testing UI docs: https://docs.smith.langchain.com/old/evaluation/faq/regression-testing RAG evaluation docs: https://docs.smith.langchain.com/old/cookbook/testing-examples/rag_eval Public dataset referenced in the video: https://smith.langchain.com/public/ea1f6ca5-de52-4d36-bd7b-fde3faa74a70/d Cookbook referenced in the video: https://github.com/langchain-ai/langsmith-cookbook/blob/515f4140cb2576ea93051ea5bb4adec652e31893/introduction/langsmith_introduction.ipynb