Make-A-Video: Text-To-Video Generation Without Text-Video Data | Paper Explained

Aleksa Gordiฤ‡ - The AI Epiphany ยท Beginner ยท๐Ÿ“ฐ AI News & Updates ยท3y ago
๐Ÿš€ Find out how to get started using Weights & Biases ๐Ÿš€ http://wandb.me/ai-epiphany ๐Ÿ‘จโ€๐Ÿ‘ฉโ€๐Ÿ‘งโ€๐Ÿ‘ฆ Join our Discord community ๐Ÿ‘จโ€๐Ÿ‘ฉโ€๐Ÿ‘งโ€๐Ÿ‘ฆ https://discord.gg/peBrCpheKE In this video I cover the latest text-to-video paper from Meta: "Make-A-Video: Text-To-Video Generation Without Text-Video Data". I walk you through the 3-stage approach that consists of: * Training a DALL-E 2 type of a model * Integrating temporal information and tuning on unlabeled videos * Fine-tuning the frame interpolation module. โ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌ โœ… Paper: https://arxiv.org/abs/2209.14792 โœ… Website: https://makeavideo.studio/ โ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌ โŒš๏ธ Timetable: 00:00 Intro 00:25 (sponsored) Weights & Biases 01:37 Going through the generations 06:15 High-level paper overview 10:50 Results 15:40 Limitations 16:30 Diving deep: DALL-E 2 backbone 23:35 Expanding to 3D - temporal info integration 32:39 Frame interpolation 37:24 3-stage training 41:28 Outro โ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌ ๐Ÿ’ฐ BECOME A PATREON OF THE AI EPIPHANY โค๏ธ If these videos, GitHub projects, and blogs help you, consider helping me out by supporting me on Patreon! The AI Epiphany - https://www.patreon.com/theaiepiphany One-time donation - https://www.paypal.com/paypalme/theaiepiphany Huge thank you to these AI Epiphany patreons: Eli Mahler Petar Veliฤkoviฤ‡ โ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌ ๐Ÿ’ผ LinkedIn - https://www.linkedin.com/in/aleksagordic/ ๐Ÿฆ Twitter - https://twitter.com/gordic_aleksa ๐Ÿ‘จโ€๐Ÿ‘ฉโ€๐Ÿ‘งโ€๐Ÿ‘ฆ Discord - https://discord.gg/peBrCpheKE ๐Ÿ“บ YouTube - https://www.youtube.com/c/TheAIEpiphany/ ๐Ÿ“š Medium - https://gordicaleksa.medium.com/ ๐Ÿ’ป GitHub - https://github.com/gordicaleksa ๐Ÿ“ข AI Newsletter - https://aiepiphany.substack.com/ โ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌ #makeavideo #meta #texttovideo
Watch on YouTube โ†— (saves to browser)
Sign in to unlock AI tutor explanation ยท โšก30

Playlist

Uploads from Aleksa Gordiฤ‡ - The AI Epiphany ยท Aleksa Gordiฤ‡ - The AI Epiphany ยท 0 of 60

โ† Previous Next โ†’
1 Intro | Neural Style Transfer #1
Intro | Neural Style Transfer #1
Aleksa Gordiฤ‡ - The AI Epiphany
 2 Basic Theory | Neural Style Transfer #2
Basic Theory | Neural Style Transfer #2
Aleksa Gordiฤ‡ - The AI Epiphany
 3 Optimization method | Neural Style Transfer #3
Optimization method | Neural Style Transfer #3
Aleksa Gordiฤ‡ - The AI Epiphany
 4 Advanced Theory | Neural Style Transfer #4
Advanced Theory | Neural Style Transfer #4
Aleksa Gordiฤ‡ - The AI Epiphany
 5 Anyone can make deepfakes now!
Anyone can make deepfakes now!
Aleksa Gordiฤ‡ - The AI Epiphany
 6 What is Computer Vision? | The Art of Creating Seeing Machines
What is Computer Vision? | The Art of Creating Seeing Machines
Aleksa Gordiฤ‡ - The AI Epiphany
 7 Feed-forward method | Neural Style Transfer #5
Feed-forward method | Neural Style Transfer #5
Aleksa Gordiฤ‡ - The AI Epiphany
 8 Alan Turing | Computing Machinery and Intelligence
Alan Turing | Computing Machinery and Intelligence
Aleksa Gordiฤ‡ - The AI Epiphany
 9 Feed-forward method (training) | Neural Style Transfer #6
Feed-forward method (training) | Neural Style Transfer #6
Aleksa Gordiฤ‡ - The AI Epiphany
 10 What is Google Deep Dream? (Basic Theory) | Deep Dream Series #1
What is Google Deep Dream? (Basic Theory) | Deep Dream Series #1
Aleksa Gordiฤ‡ - The AI Epiphany
 11 Semantic Segmentation in PyTorch | Neural Style Transfer #7
Semantic Segmentation in PyTorch | Neural Style Transfer #7
Aleksa Gordiฤ‡ - The AI Epiphany
 12 How to get started with Machine Learning
How to get started with Machine Learning
Aleksa Gordiฤ‡ - The AI Epiphany
 13 How to learn PyTorch? (3 easy steps) | 2021
How to learn PyTorch? (3 easy steps) | 2021
Aleksa Gordiฤ‡ - The AI Epiphany
 14 PyTorch or TensorFlow?
PyTorch or TensorFlow?
Aleksa Gordiฤ‡ - The AI Epiphany
 15 3 Machine Learning Projects For Beginners (Highly visual) | 2021
3 Machine Learning Projects For Beginners (Highly visual) | 2021
Aleksa Gordiฤ‡ - The AI Epiphany
 16 Machine Learning Projects (Intermediate level) | 2021
Machine Learning Projects (Intermediate level) | 2021
Aleksa Gordiฤ‡ - The AI Epiphany
 17 Cheapest (0$) Deep Learning Hardware Options | 2021
Cheapest (0$) Deep Learning Hardware Options | 2021
Aleksa Gordiฤ‡ - The AI Epiphany
 18 How to learn deep learning? (Transformers Example)
How to learn deep learning? (Transformers Example)
Aleksa Gordiฤ‡ - The AI Epiphany
 19 How do transformers work? (Attention is all you need)
How do transformers work? (Attention is all you need)
Aleksa Gordiฤ‡ - The AI Epiphany
 20 Developing a deep learning project (case study on transformer)
Developing a deep learning project (case study on transformer)
Aleksa Gordiฤ‡ - The AI Epiphany
 21 Vision Transformer (ViT) - An image is worth 16x16 words | Paper Explained
Vision Transformer (ViT) - An image is worth 16x16 words | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 22 GPT-3 - Language Models are Few-Shot Learners | Paper Explained
GPT-3 - Language Models are Few-Shot Learners | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 23 Google DeepMind's AlphaFold 2 explained! (Protein folding, AlphaFold 1, a glimpse into AlphaFold 2)
Google DeepMind's AlphaFold 2 explained! (Protein folding, AlphaFold 1, a glimpse into AlphaFold 2)
Aleksa Gordiฤ‡ - The AI Epiphany
 24 Attention Is All You Need (Transformer) | Paper Explained
Attention Is All You Need (Transformer) | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 25 Graph Attention Networks (GAT) | GNN Paper Explained
Graph Attention Networks (GAT) | GNN Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 26 Graph Convolutional Networks (GCN) | GNN Paper Explained
Graph Convolutional Networks (GCN) | GNN Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 27 Graph SAGE - Inductive Representation Learning on Large Graphs | GNN Paper Explained
Graph SAGE - Inductive Representation Learning on Large Graphs | GNN Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 28 PinSage - Graph Convolutional Neural Networks for Web-Scale Recommender Systems | Paper Explained
PinSage - Graph Convolutional Neural Networks for Web-Scale Recommender Systems | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 29 OpenAI CLIP - Connecting Text and Images | Paper Explained
OpenAI CLIP - Connecting Text and Images | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 30 Temporal Graph Networks (TGN) | GNN Paper Explained
Temporal Graph Networks (TGN) | GNN Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 31 Graph Neural Network Project Update! (I'm coding GAT from scratch)
Graph Neural Network Project Update! (I'm coding GAT from scratch)
Aleksa Gordiฤ‡ - The AI Epiphany
 32 Graph Attention Network Project Walkthrough
Graph Attention Network Project Walkthrough
Aleksa Gordiฤ‡ - The AI Epiphany
 33 How to get started with Graph ML? (Blog walkthrough)
How to get started with Graph ML? (Blog walkthrough)
Aleksa Gordiฤ‡ - The AI Epiphany
 34 DQN - Playing Atari with Deep Reinforcement Learning | RL Paper Explained
DQN - Playing Atari with Deep Reinforcement Learning | RL Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 35 AlphaGo - Mastering the game of Go with deep neural networks and tree search | RL Paper Explained
AlphaGo - Mastering the game of Go with deep neural networks and tree search | RL Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 36 DeepMind's AlphaGo Zero and AlphaZero | RL paper explained
DeepMind's AlphaGo Zero and AlphaZero | RL paper explained
Aleksa Gordiฤ‡ - The AI Epiphany
 37 OpenAI - Solving Rubik's Cube with a Robot Hand | RL paper explained
OpenAI - Solving Rubik's Cube with a Robot Hand | RL paper explained
Aleksa Gordiฤ‡ - The AI Epiphany
 38 MuZero - Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model | RL Paper explained
MuZero - Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model | RL Paper explained
Aleksa Gordiฤ‡ - The AI Epiphany
 39 EfficientNetV2 - Smaller Models and Faster Training | Paper explained
EfficientNetV2 - Smaller Models and Faster Training | Paper explained
Aleksa Gordiฤ‡ - The AI Epiphany
 40 Implementing DeepMind's DQN from scratch! | Project Update
Implementing DeepMind's DQN from scratch! | Project Update
Aleksa Gordiฤ‡ - The AI Epiphany
 41 MLP-Mixer: An all-MLP Architecture for Vision | Paper explained
MLP-Mixer: An all-MLP Architecture for Vision | Paper explained
Aleksa Gordiฤ‡ - The AI Epiphany
 42 DeepMind's Android RL Environment - AndroidEnv
DeepMind's Android RL Environment - AndroidEnv
Aleksa Gordiฤ‡ - The AI Epiphany
 43 When Vision Transformers Outperform ResNets without Pretraining | Paper Explained
When Vision Transformers Outperform ResNets without Pretraining | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 44 Non-Parametric Transformers | Paper explained
Non-Parametric Transformers | Paper explained
Aleksa Gordiฤ‡ - The AI Epiphany
 45 Chip Placement with Deep Reinforcement Learning | Paper Explained
Chip Placement with Deep Reinforcement Learning | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 46 Text Style Brush - Transfer of text aesthetics from a single example | Paper Explained
Text Style Brush - Transfer of text aesthetics from a single example | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 47 Graphormer - Do Transformers Really Perform Bad for Graph Representation? | Paper Explained
Graphormer - Do Transformers Really Perform Bad for Graph Representation? | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 48 GANs N' Roses: Stable, Controllable, Diverse Image to Image Translation | Paper Explained
GANs N' Roses: Stable, Controllable, Diverse Image to Image Translation | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 49 VQ-VAEs: Neural Discrete Representation Learning | Paper + PyTorch Code Explained
VQ-VAEs: Neural Discrete Representation Learning | Paper + PyTorch Code Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 50 VQ-GAN: Taming Transformers for High-Resolution Image Synthesis | Paper Explained
VQ-GAN: Taming Transformers for High-Resolution Image Synthesis | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 51 Multimodal Few-Shot Learning with Frozen Language Models | Paper Explained
Multimodal Few-Shot Learning with Frozen Language Models | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 52 Focal Transformer: Focal Self-attention for Local-Global Interactions in Vision Transformers
Focal Transformer: Focal Self-attention for Local-Global Interactions in Vision Transformers
Aleksa Gordiฤ‡ - The AI Epiphany
 53 AudioCLIP: Extending CLIP to Image, Text and Audio | Paper Explained
AudioCLIP: Extending CLIP to Image, Text and Audio | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 54 RMA: Rapid Motor Adaptation for Legged Robots | Paper Explained
RMA: Rapid Motor Adaptation for Legged Robots | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 55 DALL-E: Zero-Shot Text-to-Image Generation | Paper Explained
DALL-E: Zero-Shot Text-to-Image Generation | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 56 DETR: End-to-End Object Detection with Transformers | Paper Explained
DETR: End-to-End Object Detection with Transformers | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 57 DINO: Emerging Properties in Self-Supervised Vision Transformers | Paper Explained!
DINO: Emerging Properties in Self-Supervised Vision Transformers | Paper Explained!
Aleksa Gordiฤ‡ - The AI Epiphany
 58 DeepMind DetCon: Efficient Visual Pretraining with Contrastive Detection | Paper Explained
DeepMind DetCon: Efficient Visual Pretraining with Contrastive Detection | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 59 Do Vision Transformers See Like Convolutional Neural Networks? | Paper Explained
Do Vision Transformers See Like Convolutional Neural Networks? | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 60 Fastformer: Additive Attention Can Be All You Need | Paper Explained
Fastformer: Additive Attention Can Be All You Need | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany

Related AI Lessons

โšก
Big Tech firms are accelerating AI investments and integration, while regulators and companies focus on safety and responsible adoption.
Big Tech firms are investing billions in AI, driving growth and transformation, while prioritizing safety and responsible adoption
Dev.to AI
 โšก
From Smart Nation to A Blackbox Nation : Would Singapore ever turn against AI?
Explore the potential backlash against AI in Singapore and its implications on the nation's smart city initiatives
Medium ยท AI
 โšก
Big Tech firms are accelerating AI investments and integration, while regulators and companies focus on safety and responsible adoption.
Big Tech firms are investing billions in AI, driving growth and transformation, while prioritizing safety and responsible adoption
Dev.to AI
 โšก
Day 11: Did AI Write My Book About AI? (The Honest Truth)
Explore the role of AI in writing a book about AI and the honest truth behind it
Medium ยท AI

Chapters (11)

Intro
0:25 (sponsored) Weights & Biases
1:37 Going through the generations
6:15 High-level paper overview
10:50 Results
15:40 Limitations
16:30 Diving deep: DALL-E 2 backbone
23:35 Expanding to 3D - temporal info integration
32:39 Frame interpolation
37:24 3-stage training
41:28 Outro
Up next
Make a dropdown menu from a range of cells in Google Sheets
Spreadsheet Class
Watch โ†’