Pytorch Lightning Learning rate finder

pytorch

Using PyTorch Lightning’s built-in learning rate finder to plot loss vs learning rate and pick an optimal value

Published

January 1, 2023

The following plots the loss against learning rate in order to discover an “optimal” learning rate.

lr_finder = trainer.tuner.lr_find(lightning_module, train_dataloaders=train_dl)
fig = lr_finder.plot(suggest=True)
fig.show()

When all else fails, 1e-3.