The Winds of AI Winter (Q2 Four Wars of the AI Stack Recap)
Thank you for 1m downloads of the podcast and 2m readers of the Substack! ๐
This is the audio discussion following The Winds of AI Winter essay that also serves as a recap of Q2 2024 in AI viewed through the lens of our Four Wars framework. Enjoy!
00:00:00 Intro Song by Suno.ai
00:02:01 Swyx and Alessio in Singapore
00:05:49 GPU Rich vs Poors: Frontier Labs
00:06:35 GPU Rich Frontier Models: Claude 3.5
00:10:37 GPU Rich helping Poors: Llama 3.1: The Synthetic Data Model
00:15:41 GPU Rich helping Poors: Frontier Labs Vibe Shift - Phi 3, Gemma 2
00:18:26 GPU Rich: Mistral Large
00:21:56 GPU Rโฆ
Watch on YouTube โ
(saves to browser)
Chapters (25)
Intro Song by Suno.ai
2:01
Swyx and Alessio in Singapore
5:49
GPU Rich vs Poors: Frontier Labs
6:35
GPU Rich Frontier Models: Claude 3.5
10:37
GPU Rich helping Poors: Llama 3.1: The Synthetic Data Model
15:41
GPU Rich helping Poors: Frontier Labs Vibe Shift - Phi 3, Gemma 2
18:26
GPU Rich: Mistral Large
21:56
GPU Rich: Nvidia + FlashAttention 3
23:45
GPU Rich helping Poors: Noam Shazeer & Character.AI
28:14
GPU Poors: On Device LLMs: Mozilla Llamafile, Chrome (Gemini Nano), Apple Intell
35:33
Quality Data Wars: NYT vs The Atlantic lawyer up vs partner up
37:41
Quality Data Wars: Reddit, ScarJo, RIAA vs Udio & Suno
41:03
Quality Data Wars: Synthetic Data, Jagged Intelligence, AlphaProof
45:33
Multimodality War: ChatGPT Voice Mode, OpenAI demo at AIEWF
47:34
Multimodality War: Meta Llama 3 multimodality + Chameleon
50:54
Multimodality War: PaliGemma + CoPaliGemma
52:55
Renaming Rag/Ops War to LLM OS War
55:31
LLM OS War: Ops War: Prompt Management vs Gateway vs Observability
1:02:57
LLM OS War: BM42 Vector DB Wars, Memory Databases, GraphRAG
1:06:15
LLM OS War: Agent Tooling
1:08:26
LLM OS War: Agent Protocols
1:10:43
Trend: Commoditization of Intelligence
1:16:45
Trend: Vertical Service as Software, AI Employees, Brightwave, Dropzone
1:20:44
Trend: Benchmark Frontiers after MMLU
1:23:31
Crowdstrike will save us from Skynet
Playlist
Uploads from Latent Space ยท Latent Space ยท 39 of 60
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
![[LLM Paper Club] Llama 3.1 Paper: The Llama Family of Models](https://i.ytimg.com/vi/TgLSYIBoX5U/mqdefault.jpg)
37
38
โถ
40
41
42
43
44
45
46
47
48
49
50
51
![[Paper Club] ๐ On Reasoning: Q-STaR and Friends!](https://i.ytimg.com/vi/Y5-FeaFOEFM/mqdefault.jpg)
52
![[Paper Club] Writing in the Margins: Chunked Prefill KV Caching for Long Context Retrieval](https://i.ytimg.com/vi/VHwrhL_MSV4/mqdefault.jpg)
53
54
55
56
57
58
59
![[Paper Club] Who Validates the Validators? Aligning LLM-Judges with Humans (w/ Eugene Yan)](https://i.ytimg.com/vi/4o_ic83U1Kw/mqdefault.jpg)
60
Ep 18: Petaflops to the People โ with George Hotz of tinycorp
Latent Space
FlashAttention-2: Making Transformers 800% faster AND exact
Latent Space
RWKV: Reinventing RNNs for the Transformer Era
Latent Space
Generating your AI Media Empire - with Youssef Rizk of Wondercraft.ai
Latent Space
RAG is a hack - with Jerry Liu of LlamaIndex
Latent Space
The End of Finetuning โ with Jeremy Howard of Fast.ai
Latent Space
Why AI Agents Don't Work (yet) - with Kanjun Qiu of Imbue
Latent Space
Powering your Copilot for Data - with Artem Keydunov from Cube.dev
Latent Space
Beating GPT-4 with Open Source Models - with Michael Royzen of Phind
Latent Space
The State of Silicon and the GPU Poors - with Dylan Patel of SemiAnalysis
Latent Space
The "Normsky" architecture for AI coding agents โ with Beyang Liu + Steve Yegge of SourceGraph
Latent Space
The AI-First Graphics Editor - with Suhail Doshi of Playground AI
Latent Space
The Accidental AI Canvas - with Steve Ruiz of tldraw
Latent Space
The Origin and Future of RLHF: the secret ingredient for ChatGPT - with Nathan Lambert
Latent Space
The Four Wars of the AI Stack - Dec 2023 Recap
Latent Space
The State of AI in production โ with David Hsu of Retool
Latent Space
Building an open AI company - with Ce and Vipul of Together AI
Latent Space
Truly Serverless Infra for AI Engineers - with Erik Bernhardsson of Modal
Latent Space
A Brief History of the Open Source AI Hacker - with Ben Firshman of Replicate
Latent Space
Open Source AI is AI we can Trust โ with Soumith Chintala of Meta AI
Latent Space
Making Transformers Sing - with Mikey Shulman of Suno
Latent Space
A Comprehensive Overview of Large Language Models - Latent Space Paper Club
Latent Space
Why Google failed to make GPT-3 -- with David Luan of Adept
Latent Space
Personal AI Meetup - Bee, BasedHardware, LangChain LangFriend, Deepgram EmilyAI
Latent Space
Supervise the Process of AI Research โ with Jungwon Byun and Andreas Stuhlmรผller of Elicit
Latent Space
Breaking down the OG GPT Paper by Alec Radford
Latent Space
High Agency Pydantic over VC Backed Frameworks โ with Jason Liu of Instructor
Latent Space
This World Does Not Exist โ Joscha Bach, Karan Malhotra, Rob Haisfield (WorldSim, WebSim, Liquid AI)
Latent Space
LLM Asia Paper Club Survey Round
Latent Space
How to train a Million Context LLM โ with Mark Huang of Gradient.ai
Latent Space
How AI is Eating Finance - with Mike Conover of Brightwave
Latent Space
How To Hire AI Engineers (ft. James Brady and Adam Wiggins of Elicit)
Latent Space
State of the Art: Training 70B LLMs on 10,000 H100 clusters
Latent Space
The 10,000x Yolo Researcher Metagame โ with Yi Tay of Reka
Latent Space
Training Llama 2, 3 & 4: The Path to Open Source AGI โ with Thomas Scialom of Meta AI
Latent Space
[LLM Paper Club] Llama 3.1 Paper: The Llama Family of Models
Latent Space
Synthetic data + tool use for LLM improvements ๐ฆ
Latent Space
RLHF vs SFT to break out of local maxima ๐
Latent Space
The Winds of AI Winter (Q2 Four Wars of the AI Stack Recap)
Latent Space
Segment Anything 2: Memory + Vision = Object Permanence โ with Nikhila Ravi and Joseph Nelson
Latent Space
Answer.ai & AI Magic with Jeremy Howard
Latent Space
Is finetuning GPT4o worth it?
Latent Space
Personal benchmarks vs HumanEval - with Nicholas Carlini of DeepMind
Latent Space
Building AGI with OpenAI's Structured Outputs API
Latent Space
Q* for model distillation ๐
Latent Space
Finetuning LoRAs on BILLIONS of tokens ๐ค
Latent Space
Cursor UX team is CRACKED ๐ป
Latent Space
Choosing the BEST OpenAI model ๐
Latent Space
How will OpenAI voice mode change API design?
Latent Space
STEALING OpenAI models data ๐ฅท
Latent Space
[Paper Club] ๐ On Reasoning: Q-STaR and Friends!
Latent Space
[Paper Club] Writing in the Margins: Chunked Prefill KV Caching for Long Context Retrieval
Latent Space
The Ultimate Guide to Prompting - with Sander Schulhoff from LearnPrompting.org
Latent Space
llm.c's Origin and the Future of LLM Compilers - Andrej Karpathy at CUDA MODE
Latent Space
Prompt Engineer is NOT a job ๐
Latent Space
Prompt Mining LLMs for better prompts โ๏ธ
Latent Space
The six pillars of few-shot prompting ๐ง
Latent Space
Language Agents: From Reasoning to Acting โ with Shunyu Yao of OpenAI, Harrison Chase of LangGraph
Latent Space
[Paper Club] Who Validates the Validators? Aligning LLM-Judges with Humans (w/ Eugene Yan)
Latent Space
Can you separate intelligence and knowledge?
Latent Space
DeepCamp AI