Image Classification and CNN
Building a CNN Classifier with PyTorch: Part 1
Building a CNN Classifier with PyTorch: Part 2
Introduction to DDP with PyTorch
Single Node Multi-GPU Training with torchrun
Multi-node Distributed Training with PyTorch
Lab Setup

Summary

In this notebook we covered the basics of how Distributed Data Parallel (DDP) works, highlighted major code modifications needed to convert a serial training script into a distributed training script, and made these modifications for a simple example. In the next script we will discuss fault tolerance and apply the content covered in this tutorial to the training of the DesignSafe Image Classifier example.

Back
 
Next
© Chishiki-AI  |   Cornell University    |   Center for Advanced Computing    |   Copyright Statement    |   Access Statement
CVW material development is supported by NSF OAC awards 1854828, 2321040, 2323116 (UT Austin) and 2005506 (Indiana University)