Let's code on cloud GPUs with VSCode and Jupyter notebooks

In this video I show how to connect VSCode to cloud GPUs for remote development. This is an extremely simple, and free way to set up a remote development environment that is persistent and scalable. I also show how to run forks of the environment (as jobs) on their own machines to trivially parallelize workloads. The Studio offers at least 3 ways of coding, 1) connect your local VSCode, 2) Use the native web-based VSCode on the Studio or 3) Run Jupyter notebooks on the browser. Studio is a much more powerful alternative to Colab that is production grade and highly scalable. Chapters: 00:00 Introduction 00:25 Start a Studio 00:38 VSCode on a cloud CPU machine 00:55 Jupyter notebooks on a cloud CPU machine 01:40 Connect your local VSCode to the cloud machine 02:58 SSH and terminal access 03:05 Install python and system packages 03:40 Explain the persistent cloud environment 04:10 Example 1: Running a Python script for training a model 04:48 The optimal development workflow for GPUs 05:02 Run on a GPU (without code changes) 05:30 Start another Studio for a clean environment (studio.lightning.ai) 06:50 Python script automatically uses the GPU 07:50 Run a hyperparameter sweep from the Studio 11:20 Making code changes from local VSCode to remote server 14:00 Launch async Jobs from the Studio 15:20 Add 4 GPUs to the Studio 16:13 Monitor and interface with the Jobs 17:08 How to speed up model training by scaling to more GPUs 18:23 Profile GPU utilization 19:12 Start Tensorboard to compare models training 20:15 Switch back to CPU to debug 21:15 Summary