openfl.utilities.optimizers.numpy.yogi_optimizer.NumPyYogi

openfl.utilities.optimizers.numpy.yogi_optimizer.NumPyYogi#

class openfl.utilities.optimizers.numpy.yogi_optimizer.NumPyYogi(*, params=None, model_interface=None, learning_rate=0.01, betas=(0.9, 0.999), initial_accumulator_value=0.0, epsilon=1e-08)[source]#

Bases: NumPyAdam

Yogi optimizer implementation.

Implements the Yogi optimization algorithm using NumPy. Yogi is an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. It is a variant of Adam and it is more robust to large learning rates.

Original paper: https://papers.nips.cc/paper/8186-adaptive-methods-for-nonconvex-optimization

Parameters:

  • params (Dict[str, ndarray] | None)

  • learning_rate (float)

  • betas (Tuple[float, float])

  • initial_accumulator_value (float)

  • epsilon (float)

params#

Parameters to be stored for optimization.

Type:

dict, optional

model_interface#

Model interface instance to provide parameters.

learning_rate#

Tuning parameter that determines the step size at each iteration.

Type:

float

betas#

Coefficients used for computing running averages of gradient and its square.

Type:

tuple

initial_accumulator_value#

Initial value for gradients and squared gradients.

Type:

float

epsilon#

Value for computational stability.

Type:

float

__init__(*, params=None, model_interface=None, learning_rate=0.01, betas=(0.9, 0.999), initial_accumulator_value=0.0, epsilon=1e-08)[source]#

Initialize the Yogi optimizer.

Parameters:

  • params (dict, optional) – Parameters to be stored for optimization. Defaults to None.

  • model_interface – Model interface instance to provide parameters. Defaults to None.

  • learning_rate (float, optional) – Tuning parameter that determines the step size at each iteration. Defaults to 0.01.

  • betas (tuple, optional) – Coefficients used for computing running averages of gradient and its square. Defaults to (0.9, 0.999).

  • initial_accumulator_value (float, optional) – Initial value for gradients and squared gradients. Defaults to 0.0.

  • epsilon (float, optional) – Value for computational stability. Defaults to 1e-8.

Return type:

None

Methods

__init__(*[, params, model_interface, ...])

Initialize the Yogi optimizer.

step(gradients)

Perform a single step for parameter update.
step(gradients)[source]#

Perform a single step for parameter update.

Implement Yogi optimizer weights update rule.

Parameters:

gradients (dict) – Partial derivatives with respect to optimized parameters.

Return type:

None