Optimizer module
- class nnodely.basic.optimizer.SGD(optimizer_defaults={}, optimizer_params=[])[source]
Stochastic Gradient Descent (SGD) optimizer.
See also
Official PyTorch SGD documentation: torch.optim.SGD
- Parameters:
name (str) – The name of the optimizer.
optimizer_defaults (dict) – A dictionary of default optimizer settings.
optimizer_params (list) – A list of parameter groups for the optimizer.
- name
The name of the optimizer.
- Type:
str
- lr
Learning rate. Default is 0.01.
- Type:
float, optional
- momentum
Momentum factor. Default is 0.0.
- Type:
float, optional
- dampening
Dampening for momentum. Default is 0.0.
- Type:
float, optional
- weight_decay
Weight decay (L2 penalty). Default is 0.0.
- Type:
float, optional
- nesterov
Enables Nesterov momentum. Default is False.
- Type:
bool, optional
- add_defaults(option_name, params, overwrite=True)
Adds default settings to the optimizer.
- Parameters:
option_name (str) – The name of the option to add.
params (any) – The parameters for the option.
overwrite (bool, optional) – Whether to overwrite existing settings. Default is True.
- set_defaults(optimizer_defaults)
Sets the default optimizer settings.
- Parameters:
optimizer_defaults (dict) – A dictionary of default optimizer settings.
- set_params(optimizer_params)
Sets the parameter groups for the optimizer.
- Parameters:
optimizer_params (list) – A list of parameter groups for the optimizer.
- set_params_to_train(all_params, params_to_train)
Sets the parameters to be trained by the optimizer.
- Parameters:
all_params (dict) – A dictionary of all parameters in the model.
params_to_train (list) – A list of parameters to be trained.
- unfold(params)
Unfolds the parameter groups into a flat list.
- Parameters:
params (list) – A list of parameter groups.
- Returns:
A flat list of parameter groups.
- Return type:
list
- Raises:
KeyError – If the params argument is not a list.
- class nnodely.basic.optimizer.Adam(optimizer_defaults={}, optimizer_params=[])[source]
Stochastic Gradient Descent (SGD) optimizer.
See also
Official PyTorch Adam documentation: torch.optim.Adam
- Parameters:
name (str) – The name of the optimizer.
optimizer_defaults (dict) – A dictionary of default optimizer settings.
optimizer_params (list) – A list of parameter groups for the optimizer.
- name
The name of the optimizer.
- Type:
str
- lr
Learning rate. Default is 0.001.
- Type:
float, optional
- betas
Coefficients used for computing running averages of gradient and its square. Default is (0.9, 0.999).
- Type:
tuple of (float, float), optional
- eps
Term added to the denominator to improve numerical stability. Default is 1e-8.
- Type:
float, optional
- weight_decay
Weight decay (L2 penalty). Default is 0.0.
- Type:
float, optional
- amsgrad
Whether to use the AMSGrad variant of this algorithm. Default is False.
- Type:
bool, optional
- add_defaults(option_name, params, overwrite=True)
Adds default settings to the optimizer.
- Parameters:
option_name (str) – The name of the option to add.
params (any) – The parameters for the option.
overwrite (bool, optional) – Whether to overwrite existing settings. Default is True.
- set_defaults(optimizer_defaults)
Sets the default optimizer settings.
- Parameters:
optimizer_defaults (dict) – A dictionary of default optimizer settings.
- set_params(optimizer_params)
Sets the parameter groups for the optimizer.
- Parameters:
optimizer_params (list) – A list of parameter groups for the optimizer.
- set_params_to_train(all_params, params_to_train)
Sets the parameters to be trained by the optimizer.
- Parameters:
all_params (dict) – A dictionary of all parameters in the model.
params_to_train (list) – A list of parameters to be trained.
- unfold(params)
Unfolds the parameter groups into a flat list.
- Parameters:
params (list) – A list of parameter groups.
- Returns:
A flat list of parameter groups.
- Return type:
list
- Raises:
KeyError – If the params argument is not a list.