Exponential Progress of AI: Moore's Law, Bitter Lesson, and the Future of Computation

Lex Fridman · Beginner ·📐 ML Fundamentals ·5y ago
Discussion of exponential progress in AI and computation, including Moore's Law and the Bitter Lesson by Rich Sutton. This was part of the AI paper club on our Discord. Join here: https://discord.gg/8RwBPRs Bitter Lesson: http://www.incompleteideas.net/IncIdeas/BitterLesson.html Slides for this video: https://bit.ly/2T4SeHt References sheet: https://bit.ly/bitter-lesson Lex + AI Podcast Discord: https://discord.gg/8RwBPRs OUTLINE: 0:00 - Overview 0:37 - Bitter Lesson by Rich Sutton 6:55 - Contentions and opposing views 9:10 - Is evolution a part of search, learning, or something else? 10:51 - Bitter Lesson argument summary 11:42 - Moore's Law 13:37 - Global compute capacity 15:43 - Massively parallel computation 16:41 - GPUs and ASICs 17:17 - Quantum computing and neuromorphic computing 19:25 - Neuralink and brain-computer interfaces 21:28 - Deep learning efficiency 22:57 - Open questions for exponential improvement of AI 28:22 - Conclusion CONNECT: - Subscribe to this YouTube channel - Twitter: https://twitter.com/lexfridman - LinkedIn: https://www.linkedin.com/in/lexfridman - Facebook: https://www.facebook.com/lexfridman - Instagram: https://www.instagram.com/lexfridman - Medium: https://medium.com/@lexfridman
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from Lex Fridman · Lex Fridman · 0 of 60

← Previous Next →
1 Ido Portal: Movement
Ido Portal: Movement
Lex Fridman
 2 Ryan Hall: Moral Victory
Ryan Hall: Moral Victory
Lex Fridman
 3 Jimmy Pedro: Judo | Take It Uneasy Podcast
Jimmy Pedro: Judo | Take It Uneasy Podcast
Lex Fridman
 4 Foundations of Deep Learning (Hugo Larochelle, Twitter)
Foundations of Deep Learning (Hugo Larochelle, Twitter)
Lex Fridman
 5 TensorFlow Tutorial (Sherry Moore, Google Brain)
TensorFlow Tutorial (Sherry Moore, Google Brain)
Lex Fridman
 6 Nuts and Bolts of Applying Deep Learning (Andrew Ng)
Nuts and Bolts of Applying Deep Learning (Andrew Ng)
Lex Fridman
 7 Sequence to Sequence Deep Learning (Quoc Le, Google)
Sequence to Sequence Deep Learning (Quoc Le, Google)
Lex Fridman
 8 Torch Tutorial (Alex Wiltschko, Twitter)
Torch Tutorial (Alex Wiltschko, Twitter)
Lex Fridman
 9 Theano Tutorial (Pascal Lamblin, MILA)
Theano Tutorial (Pascal Lamblin, MILA)
Lex Fridman
 10 Deep Reinforcement Learning (John Schulman, OpenAI)
Deep Reinforcement Learning (John Schulman, OpenAI)
Lex Fridman
 11 Deep Learning for Speech Recognition (Adam Coates, Baidu)
Deep Learning for Speech Recognition (Adam Coates, Baidu)
Lex Fridman
 12 Deep Learning for Natural Language Processing (Richard Socher, Salesforce)
Deep Learning for Natural Language Processing (Richard Socher, Salesforce)
Lex Fridman
 13 Foundations of Unsupervised Deep Learning (Ruslan Salakhutdinov, CMU)
Foundations of Unsupervised Deep Learning (Ruslan Salakhutdinov, CMU)
Lex Fridman
 14 Deep Learning for Computer Vision (Andrej Karpathy, OpenAI)
Deep Learning for Computer Vision (Andrej Karpathy, OpenAI)
Lex Fridman
 15 Foundations and Challenges of Deep Learning (Yoshua Bengio)
Foundations and Challenges of Deep Learning (Yoshua Bengio)
Lex Fridman
 16 MIT 6.S094: Introduction to Deep Learning and Self-Driving Cars
MIT 6.S094: Introduction to Deep Learning and Self-Driving Cars
Lex Fridman
 17 MIT 6.S094: Deep Reinforcement Learning for Motion Planning
MIT 6.S094: Deep Reinforcement Learning for Motion Planning
Lex Fridman
 18 MIT 6.S094: Convolutional Neural Networks for End-to-End Learning of the Driving Task
MIT 6.S094: Convolutional Neural Networks for End-to-End Learning of the Driving Task
Lex Fridman
 19 MIT 6.S094: Recurrent Neural Networks for Steering Through Time
MIT 6.S094: Recurrent Neural Networks for Steering Through Time
Lex Fridman
 20 MIT 6.S094: Deep Learning for Human-Centered Semi-Autonomous Vehicles
MIT 6.S094: Deep Learning for Human-Centered Semi-Autonomous Vehicles
Lex Fridman
 21 Chris Gerdes (Stanford) on Technology, Policy and Vehicle Safety - MIT Self-Driving Cars
Chris Gerdes (Stanford) on Technology, Policy and Vehicle Safety - MIT Self-Driving Cars
Lex Fridman
 22 Sertac Karaman (MIT) on Motion Planning in a Complex World - MIT Self-Driving Cars
Sertac Karaman (MIT) on Motion Planning in a Complex World - MIT Self-Driving Cars
Lex Fridman
 23 MIT Sloan: Intro to Machine Learning (in 360/VR)
MIT Sloan: Intro to Machine Learning (in 360/VR)
Lex Fridman
 24 MIT 6.S094: Deep Learning
MIT 6.S094: Deep Learning
Lex Fridman
 25 MIT Self-Driving Cars (2018)
MIT Self-Driving Cars (2018)
Lex Fridman
 26 MIT 6.S094: Deep Reinforcement Learning
MIT 6.S094: Deep Reinforcement Learning
Lex Fridman
 27 MIT 6.S094: Computer Vision
MIT 6.S094: Computer Vision
Lex Fridman
 28 MIT 6.S094: Deep Learning for Human Sensing
MIT 6.S094: Deep Learning for Human Sensing
Lex Fridman
 29 MIT AGI: Artificial General Intelligence
MIT AGI: Artificial General Intelligence
Lex Fridman
 30 MIT AGI: Building machines that see, learn, and think like people (Josh Tenenbaum)
MIT AGI: Building machines that see, learn, and think like people (Josh Tenenbaum)
Lex Fridman
 31 Ray Kurzweil: Future of Intelligence | MIT 6.S099: Artificial General Intelligence (AGI)
Ray Kurzweil: Future of Intelligence | MIT 6.S099: Artificial General Intelligence (AGI)
Lex Fridman
 32 Sacha Arnoud, Director of Engineering, Waymo - MIT Self-Driving Cars
Sacha Arnoud, Director of Engineering, Waymo - MIT Self-Driving Cars
Lex Fridman
 33 Lisa Feldman Barrett: How the Brain Creates Emotions | MIT Artificial General Intelligence (AGI)
Lisa Feldman Barrett: How the Brain Creates Emotions | MIT Artificial General Intelligence (AGI)
Lex Fridman
 34 Stephen Wolfram: Computational Universe | MIT 6.S099: Artificial General Intelligence (AGI)
Stephen Wolfram: Computational Universe | MIT 6.S099: Artificial General Intelligence (AGI)
Lex Fridman
 35 Emilio Frazzoli, CTO, nuTonomy - MIT Self-Driving Cars
Emilio Frazzoli, CTO, nuTonomy - MIT Self-Driving Cars
Lex Fridman
 36 Sterling Anderson, Co-Founder, Aurora - MIT Self-Driving Cars
Sterling Anderson, Co-Founder, Aurora - MIT Self-Driving Cars
Lex Fridman
 37 MIT AGI: Cognitive Architecture (Nate Derbinsky)
MIT AGI: Cognitive Architecture (Nate Derbinsky)
Lex Fridman
 38 MIT Advanced Vehicle Technology Study (MIT-AVT)
MIT Advanced Vehicle Technology Study (MIT-AVT)
Lex Fridman
 39 MIT-AVT: Data Collection Device (for Large-Scale Semi-Autonomous Driving)
MIT-AVT: Data Collection Device (for Large-Scale Semi-Autonomous Driving)
Lex Fridman
 40 Geoffrey Hinton: What are you excited about in deep learning?
Geoffrey Hinton: What are you excited about in deep learning?
Lex Fridman
 41 Ilya Sutskever: OpenAI Meta-Learning and Self-Play | MIT Artificial General Intelligence (AGI)
Ilya Sutskever: OpenAI Meta-Learning and Self-Play | MIT Artificial General Intelligence (AGI)
Lex Fridman
 42 Comfortably Numb Solo | Pink Floyd Cover by Lex Fridman
Comfortably Numb Solo | Pink Floyd Cover by Lex Fridman
Lex Fridman
 43 Yoshua Bengio: Deep Learning | Lex Fridman Podcast #4
Yoshua Bengio: Deep Learning | Lex Fridman Podcast #4
Lex Fridman
 44 Jeff Atwood: Stack Overflow and Coding Horror | Lex Fridman Podcast #7
Jeff Atwood: Stack Overflow and Coding Horror | Lex Fridman Podcast #7
Lex Fridman
 45 Eric Schmidt: Google | Lex Fridman Podcast #8
Eric Schmidt: Google | Lex Fridman Podcast #8
Lex Fridman
 46 Pieter Abbeel: Deep Reinforcement Learning | Lex Fridman Podcast #10
Pieter Abbeel: Deep Reinforcement Learning | Lex Fridman Podcast #10
Lex Fridman
 47 Deep Learning Basics: Introduction and Overview
Deep Learning Basics: Introduction and Overview
Lex Fridman
 48 Deep Learning State of the Art (2019)
Deep Learning State of the Art (2019)
Lex Fridman
 49 MIT 6.S091: Introduction to Deep Reinforcement Learning (Deep RL)
MIT 6.S091: Introduction to Deep Reinforcement Learning (Deep RL)
Lex Fridman
 50 Self-Driving Cars: State of the Art (2019)
Self-Driving Cars: State of the Art (2019)
Lex Fridman
 51 Drago Anguelov (Waymo) - MIT Self-Driving Cars
Drago Anguelov (Waymo) - MIT Self-Driving Cars
Lex Fridman
 52 Oliver Cameron (CEO, Voyage) - MIT Self-Driving Cars
Oliver Cameron (CEO, Voyage) - MIT Self-Driving Cars
Lex Fridman
 53 Karl Iagnemma & Oscar Beijbom (Aptiv Autonomous Mobility) - MIT Self-Driving Cars
Karl Iagnemma & Oscar Beijbom (Aptiv Autonomous Mobility) - MIT Self-Driving Cars
Lex Fridman
 54 Leslie Kaelbling: Reinforcement Learning, Planning, and Robotics | Lex Fridman Podcast #15
Leslie Kaelbling: Reinforcement Learning, Planning, and Robotics | Lex Fridman Podcast #15
Lex Fridman
 55 Greg Brockman: OpenAI and AGI | Lex Fridman Podcast #17
Greg Brockman: OpenAI and AGI | Lex Fridman Podcast #17
Lex Fridman
 56 Ian Goodfellow: Generative Adversarial Networks (GANs) | Lex Fridman Podcast #19
Ian Goodfellow: Generative Adversarial Networks (GANs) | Lex Fridman Podcast #19
Lex Fridman
 57 MIT 6.S093: Introduction to Human-Centered Artificial Intelligence (AI)
MIT 6.S093: Introduction to Human-Centered Artificial Intelligence (AI)
Lex Fridman
 58 Chris Lattner: Compilers, LLVM, Swift, TPU, and ML Accelerators | Lex Fridman Podcast #21
Chris Lattner: Compilers, LLVM, Swift, TPU, and ML Accelerators | Lex Fridman Podcast #21
Lex Fridman
 59 Rajat Monga: TensorFlow | Lex Fridman Podcast #22
Rajat Monga: TensorFlow | Lex Fridman Podcast #22
Lex Fridman
 60 Gavin Miller: Adobe Research | Lex Fridman Podcast #23
Gavin Miller: Adobe Research | Lex Fridman Podcast #23
Lex Fridman

Related AI Lessons

⚡
What Building a Flight Delay Prediction System Taught Me About Real-World Data Science
Learn how building a flight delay prediction system taught valuable lessons about real-world data science, including handling large datasets and model deployment
Medium · Machine Learning
 ⚡
THE EVALUATION PROBLEM
Learn to measure AI system performance to build trust in machine learning models
Medium · Machine Learning
 ⚡
Probability & Statistics — Deep Dive + Problem: Connected Components Labeling
Learn Probability & Statistics fundamentals and apply them to solve the Connected Components Labeling problem
Dev.to AI
 ⚡
The Hardware Behind AI: The Hidden Circuit Boards Powering Machine Learning and the Future of…
Discover the crucial hardware behind AI, from GPUs to advanced PCB design, and how it enables machine learning and next-generation computing.
Medium · Machine Learning

Chapters (14)

Overview
0:37 Bitter Lesson by Rich Sutton
6:55 Contentions and opposing views
9:10 Is evolution a part of search, learning, or something else?
10:51 Bitter Lesson argument summary
11:42 Moore's Law
13:37 Global compute capacity
15:43 Massively parallel computation
16:41 GPUs and ASICs
17:17 Quantum computing and neuromorphic computing
19:25 Neuralink and brain-computer interfaces
21:28 Deep learning efficiency
22:57 Open questions for exponential improvement of AI
28:22 Conclusion
Up next
AI Engineer Roadmap 2026 🤖🔥 How to Become an AI Engineer Step by Step #AIEngineer #AIRoadmap #AI
CydexCode
Watch →