openfl.utilities.optimizers.numpy

class openfl.utilities.optimizers.numpy.NumPyAdagrad(*, params: Dict[str, ndarray] | None = None, model_interface=None, learning_rate: float = 0.01, initial_accumulator_value: float = 0.1, epsilon: float = 1e-10)

Adagrad optimizer implementation.

Implements the Adagrad optimization algorithm using NumPy. Adagrad is an algorithm for gradient-based optimization that adapts the learning rate to the parameters, performing smaller updates for parameters associated with frequently occurring features, and larger updates for parameters associated with infrequent features.

Original paper: http://jmlr.org/papers/v12/duchi11a.html

Class Attributes:

• params (dict, optional) – Parameters to be stored for optimization.

• model_interface – Model interface instance to provide parameters.

• learning_rate (float) – Tuning parameter that determines the step size at each iteration.

• initial_accumulator_value (float) – Initial value for squared gradients.

• epsilon (float) – Value for computational stability.

step(gradients: Dict[str, ndarray]) → None

Perform a single step for parameter update.

Implement Adagrad optimizer weights update rule.

Parameters:

gradients (dict) – Partial derivatives with respect to optimized parameters.

Raises:

KeyError – If a key in gradients does not exist in optimized parameters.

class openfl.utilities.optimizers.numpy.NumPyAdam(*, params: Dict[str, ndarray] | None = None, model_interface=None, learning_rate: float = 0.01, betas: Tuple[float, float] = (0.9, 0.999), initial_accumulator_value: float = 0.0, epsilon: float = 1e-08)

Adam optimizer implementation.

Implements the Adam optimization algorithm using NumPy. Adam is an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments.

Original paper: https://openreview.net/forum?id=ryQu7f-RZ

Class Attributes:

• params (dict, optional) – Parameters to be stored for optimization.

• model_interface – Model interface instance to provide parameters.

• learning_rate (float) – Tuning parameter that determines the step size at each iteration.

• betas (tuple) – Coefficients used for computing running averages of gradient and its square.

• initial_accumulator_value (float) – Initial value for gradients and squared gradients.

• epsilon (float) – Value for computational stability.

step(gradients: Dict[str, ndarray]) → None

Perform a single step for parameter update.

Implement Adam optimizer weights update rule.

Parameters:

gradients (dict) – Partial derivatives with respect to optimized parameters.

Raises:

KeyError – If a key in gradients does not exist in optimized parameters.

class openfl.utilities.optimizers.numpy.NumPyYogi(*, params: Dict[str, ndarray] | None = None, model_interface=None, learning_rate: float = 0.01, betas: Tuple[float, float] = (0.9, 0.999), initial_accumulator_value: float = 0.0, epsilon: float = 1e-08)

Yogi optimizer implementation.

Implements the Yogi optimization algorithm using NumPy. Yogi is an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. It is a variant of Adam and it is more robust to large learning rates.

Original paper: https://papers.nips.cc/paper/8186-adaptive-methods-for-nonconvex-optimization

Class Attributes:

• params (dict, optional) – Parameters to be stored for optimization.

• model_interface – Model interface instance to provide parameters.

• learning_rate (float) – Tuning parameter that determines the step size at each iteration.

• betas (tuple) – Coefficients used for computing running averages of gradient and its square.

• initial_accumulator_value (float) – Initial value for gradients and squared gradients.

• epsilon (float) – Value for computational stability.

step(gradients: Dict[str, ndarray]) → None

Perform a single step for parameter update.

Implement Yogi optimizer weights update rule.

Parameters:

gradients (dict) – Partial derivatives with respect to optimized parameters.

adagrad_optimizer	Adagrad optimizer module.
adam_optimizer	Adam optimizer module.
base_optimizer	Base abstract optimizer class module.
yogi_optimizer	Adam optimizer module.