Quick Introduction to Fast Tool Calling with Groq and Llama3

In this video we'll look at tool calling with Groq's fine tuned Llama 3 model specialized in for function calling. We'll conduct some simple experiments profiling Llama 3 models, comparing Llama3 70B and 8B models for latency given a simple prompt. At the end we'll also take a quick look at the super cool demo by Groq that combines fast audio transcription with Whisper Large v3 and Groq's optimized llama3.1 model (released recently by Meta) for a quick look into the future of low latency LLMs! 📚 Chapters: 00:00 - Introduction and Overview 00:29 - Setting Up the Environment and Project 01:05 - Defining the Function and Setting Up the Model 02:29 - Handling Messages and Tool Calls 03:52 - Running and Compiling Responses 04:47 - Testing Our Setup with a Simple Calculation 05:26 - Experimenting with Multiple Models 07:28 - Profiling Latency for Model Comparison 09:00 - Reviewing the Profiler Function 10:04 - Running and Comparing Multiple Tests 12:55 - Single Test Profiling and Adjustment 15:59 - Exploring Additional Tools and Fast Inference Demo 16:51 - Final Thoughts and Speed Evaluation 🔗 Links - Subscribe!: https://www.youtube.com/channel/UCu8WF59Scx9f3H1N_FgZUwQ - Tiktok: https://www.tiktok.com/@enkrateialucca?lang=en - Twitter: https://twitter.com/LucasEnkrateia - LinkedIn: https://www.linkedin.com/in/lucas-soares-969044167/ - AI project tracker from Ben's Bites: https://bensbites.beehiiv.com/subscribe?ref=CoXhc4I0c0 Support the Channel! - Buy me a cup of coffee: https://tr.ee/7tYsD-tUu2 - Paypal: https://paypal.me/lucasenkrateia?country.x=PT&locale.x=pt_PT