Generating Wikipedia by Summarizing Long Sequences

Connor Shorten · Beginner ·🧠 Large Language Models ·6y ago
This video explores the paper "Generating Wikipedia by Summarizing Long Sequences". Natural Language Processing models that can generate summaries of source documents on a single topic such as "Generative Adversarial Networks" or "Reinforcement Learning" are one of the NLP applications that I find the most interesting! This paper is frequently cited for introducing the Transformer decoder architecture, but there is a lot more interesting details about this paper. I also think the approximations proposed to full attention in this paper are really interesting! Paper Link: https://arxiv.org/pdf/1801.10198.pdf Thanks for watching! Please Subscribe!
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from Connor Shorten · Connor Shorten · 0 of 60

← Previous Next →
1 DenseNets
DenseNets
Connor Shorten
 2 DeepWalk Explained
DeepWalk Explained
Connor Shorten
 3 Inception Network Explained
Inception Network Explained
Connor Shorten
 4 StackGAN
StackGAN
Connor Shorten
 5 StyleGAN
StyleGAN
Connor Shorten
 6 Progressive Growing of GANs Explained
Progressive Growing of GANs Explained
Connor Shorten
 7 Improved Techniques for Training GANs
Improved Techniques for Training GANs
Connor Shorten
 8 Word2Vec Explained
Word2Vec Explained
Connor Shorten
 9 Must Read Papers on GANs
Must Read Papers on GANs
Connor Shorten
 10 Unsupervised Feature Learning
Unsupervised Feature Learning
Connor Shorten
 11 Self-Supervised GANs
Self-Supervised GANs
Connor Shorten
 12 Embedding Graphs with Deep Learning
Embedding Graphs with Deep Learning
Connor Shorten
 13 Transfer Learning in GANs
Transfer Learning in GANs
Connor Shorten
 14 ReLU Activation Function
ReLU Activation Function
Connor Shorten
 15 AC-GAN Explained
AC-GAN Explained
Connor Shorten
 16 SimGAN Explained
SimGAN Explained
Connor Shorten
 17 DC-GAN Explained!
DC-GAN Explained!
Connor Shorten
 18 ResNet Explained!
ResNet Explained!
Connor Shorten
 19 Graph Convolutional Networks
Graph Convolutional Networks
Connor Shorten
 20 Neural Architecture Search
Neural Architecture Search
Connor Shorten
 21 Henry AI Labs
Henry AI Labs
Connor Shorten
 22 Video Classification with Deep Learning
Video Classification with Deep Learning
Connor Shorten
 23 BigGANs in Data Augmentation
BigGANs in Data Augmentation
Connor Shorten
 24 Introduction to Deep Learning
Introduction to Deep Learning
Connor Shorten
 25 EfficientNet Explained!
EfficientNet Explained!
Connor Shorten
 26 Self-Attention GAN
Self-Attention GAN
Connor Shorten
 27 Curriculum Learning in Deep Neural Networks
Curriculum Learning in Deep Neural Networks
Connor Shorten
 28 Deep Learning Podcast #1 | Edward Dixon | Stochastic Weight Averaging
Deep Learning Podcast #1 | Edward Dixon | Stochastic Weight Averaging
Connor Shorten
 29 Deep Compression
Deep Compression
Connor Shorten
 30 Skin Cancer Classification with Deep Learning
Skin Cancer Classification with Deep Learning
Connor Shorten
 31 Deep Learning Podcast #2 | Edward Peake | Deep Learning in Medical Imaging
Deep Learning Podcast #2 | Edward Peake | Deep Learning in Medical Imaging
Connor Shorten
 32 The Lottery Ticket Hypothesis Explained!
The Lottery Ticket Hypothesis Explained!
Connor Shorten
 33 SqueezeNet
SqueezeNet
Connor Shorten
 34 GauGAN Explained!
GauGAN Explained!
Connor Shorten
 35 AutoML with Hyperband
AutoML with Hyperband
Connor Shorten
 36 DL Podcast #3 | Yannic Kilcher | Population-Based Search
DL Podcast #3 | Yannic Kilcher | Population-Based Search
Connor Shorten
 37 Weakly Supervised Pretraining
Weakly Supervised Pretraining
Connor Shorten
 38 Image Data Augmentation for Deep Learning
Image Data Augmentation for Deep Learning
Connor Shorten
 39 Unsupervised Data Augmentation
Unsupervised Data Augmentation
Connor Shorten
 40 Wide ResNet Explained!
Wide ResNet Explained!
Connor Shorten
 41 RevNet: Backpropagation without Storing Activations
RevNet: Backpropagation without Storing Activations
Connor Shorten
 42 GANs with Fewer Labels
GANs with Fewer Labels
Connor Shorten
 43 BigBiGAN Unsupervised Learning!
BigBiGAN Unsupervised Learning!
Connor Shorten
 44 Self-Supervised Learning
Self-Supervised Learning
Connor Shorten
 45 Multi-Task Self-Supervised Learning
Multi-Task Self-Supervised Learning
Connor Shorten
 46 Self-Supervised GANs
Self-Supervised GANs
Connor Shorten
 47 Population Based Training
Population Based Training
Connor Shorten
 48 Show, Attend and Tell
Show, Attend and Tell
Connor Shorten
 49 Siamese Neural Networks
Siamese Neural Networks
Connor Shorten
 50 WaveGAN Explained!
WaveGAN Explained!
Connor Shorten
 51 VAE-GAN Explained!
VAE-GAN Explained!
Connor Shorten
 52 Evolution in Neural Architecture Search!
Evolution in Neural Architecture Search!
Connor Shorten
 53 AI Research Weekly Update August 18th, 2019
AI Research Weekly Update August 18th, 2019
Connor Shorten
 54 Weight Agnostic Neural Networks Explained!
Weight Agnostic Neural Networks Explained!
Connor Shorten
 55 AI Research Weekly Update August 25th, 2019
AI Research Weekly Update August 25th, 2019
Connor Shorten
 56 Neuroevolution of Augmenting Topologies (NEAT)
Neuroevolution of Augmenting Topologies (NEAT)
Connor Shorten
 57 CoDeepNEAT
CoDeepNEAT
Connor Shorten
 58 AI Research Weekly Update September 1st, 2019
AI Research Weekly Update September 1st, 2019
Connor Shorten
 59 Randomly Wired Neural Networks
Randomly Wired Neural Networks
Connor Shorten
 60 Genetic CNN
Genetic CNN
Connor Shorten

Related AI Lessons

⚡
The Invisible Scaffolding — How Normalization Keeps Deep Models from Falling Apart
Learn how normalization keeps deep models stable and why it's a crucial design decision in large language models
Medium · LLM
 ⚡
Stop Telling the AI What to Build
Learn how to collaborate with AI using the Greek sculptor's method, focusing on discovery rather than instruction
Medium · AI
 ⚡
L’IA ne simule pas les émotions. Elle les gouverne.
Discover how AI governs emotions in various applications, and why this matters for human-AI interaction
Medium · AI
 ⚡
The Future of Artificial Intelligence and Machine Learning: Key Trends Every Developer Should Know
Stay ahead of the AI and ML curve by understanding key trends in intelligent systems, and apply them to your development work for a competitive edge
Medium · Machine Learning
Up next
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)
Watch →