Deep Learning with PyTorch Quick Start Guide

Parameter groups

When an optimizer is instantiated, it is the as well as a variety of hyperparameters such as the learning rate. Optimizers are also passed other hyperparameters specific to each optimization algorithm. It can be extremely useful to set up groups of these hyperparameters, which can be applied to different parts of the model. This can be achieved by creating a parameter group, essentially a list of dictionaries that can be passed to the optimizer. 

The param variable must either be an iterator over a torch.tensor or a Python dictionary specifying a default value of optimization options. Note that the parameters themselves need to be specified as an ordered collection, such as a list, so that parameters are a consistent sequence ...