Part 1: Welcome to the Distributed Data Parallel (DDP) Tutorial Series

In the first video of this series, Suraj Subramanian breaks down why Distributed Training is an important part of your ML arsenal. The series starts with a simple non-distributed training job, and ends with launching a training job for a GPT-esque model across several machines in a cluster. At each step, Suraj walks you through the code used, and best practices for your DDP training. Chapters 00:00 Intro 00:25 Why Distributed Training 00:50 Structure of this tutorial series Tutorial page for this video → https://bit.ly/3LrnACh Clone this repo to get started → https://bit.ly/3Rk5eo4 These tutorials assume familiarity with model training in PyTorch: Get started with PyTorch → https://bit.ly/3qUdjVw PyTorch 101 Video Series → https://bit.ly/3DF9Xxq Like this video and subscribe to the PyTorch channel for more videos like this!