Image Classification and CNN
Building a CNN Classifier with PyTorch: Part 1
Building a CNN Classifier with PyTorch: Part 2
Introduction to DDP with PyTorch
Single Node Multi-GPU Training with torchrun
Multi-node Distributed Training with PyTorch
Lab Setup

Create Data Distributed Sampler (2)

Next we modify the dataloader such that we are using the DistributedSampler:

  • Load data across gpus
  • The sampler returns a iterator over indices, which are fed into dataloader

Note that we have set up a the DistributedSampler and Dataloader for our training and validation data as well.

Back
 
Next
© Chishiki-AI  |   Cornell University    |   Center for Advanced Computing    |   Copyright Statement    |   Access Statement
CVW material development is supported by NSF OAC awards 1854828, 2321040, 2323116 (UT Austin) and 2005506 (Indiana University)