Deep Learning State of the Art (2020)

Lecture on most recent research and developments in deep learning, and hopes for 2020. This is not intended to be a list of SOTA benchmark results, but rather a set of highlights of machine learning and AI innovations and progress in academia, industry, and society in general. This lecture is part of the MIT Deep Learning Lecture Series. Website: https://deeplearning.mit.edu Slides: http://bit.ly/2QEfbAm References: http://bit.ly/deeplearn-sota-2020 Playlist: http://bit.ly/deep-learning-playlist OUTLINE: 0:00 - Introduction 0:33 - AI in the context of human history 5:47 - Deep learning celebrations, growth, and limitations 6:35 - Deep learning early key figures 9:29 - Limitations of deep learning 11:01 - Hopes for 2020: deep learning community and research 12:50 - Deep learning frameworks: TensorFlow and PyTorch 15:11 - Deep RL frameworks 16:13 - Hopes for 2020: deep learning and deep RL frameworks 17:53 - Natural language processing 19:42 - Megatron, XLNet, ALBERT 21:21 - Write with transformer examples 24:28 - GPT-2 release strategies report 26:25 - Multi-domain dialogue 27:13 - Commonsense reasoning 28:26 - Alexa prize and open-domain conversation 33:44 - Hopes for 2020: natural language processing 35:11 - Deep RL and self-play 35:30 - OpenAI Five and Dota 2 37:04 - DeepMind Quake III Arena 39:07 - DeepMind AlphaStar 41:09 - Pluribus: six-player no-limit Texas hold'em poker 43:13 - OpenAI Rubik's Cube 44:49 - Hopes for 2020: Deep RL and self-play 45:52 - Science of deep learning 46:01 - Lottery ticket hypothesis 47:29 - Disentangled representations 48:34 - Deep double descent 49:30 - Hopes for 2020: science of deep learning 50:56 - Autonomous vehicles and AI-assisted driving 51:50 - Waymo 52:42 - Tesla Autopilot 57:03 - Open question for Level 2 and Level 4 approaches 59:55 - Hopes for 2020: autonomous vehicles and AI-assisted driving 1:01:43 - Government, politics, policy 1:03:03 - Recommendation systems and policy 1:05:36 - Hopes for 2020: Politics, policy an