Finally, we need to define standard PyTorch train and evaluation functions. The function train iterates over multiple epochs and all batches of data in each epoch. Model parameters are updated for each batch with the given loss function and optimizer. The model accuracy and loss are computed with testing data at every epoch.

There are two major modifications made in this training loop compared to part 1:

1. We call the function step associated with torch.optim.lr_scheduler.ReduceLROnPlateau object instantiated earlier after each epoch. This will check for validation loss plateaus. If the plateaus are identified, the learning rate will be lowered.
2. After each epoch we check to see if the validation accuracy of our model has improved. If so we save the best performing model so far. To save the model we use torch.save which saves an object to disk.