openfl.utilities.optimizers.numpy.adam_optimizer.NumPyAdam

class openfl.utilities.optimizers.numpy.adam_optimizer.NumPyAdam(*, params: Dict[str, ndarray] | None = None, model_interface=None, learning_rate: float = 0.01, betas: Tuple[float, float] = (0.9, 0.999), initial_accumulator_value: float = 0.0, epsilon: float = 1e-08)

Bases: Optimizer

Adam optimizer implementation.

Implements the Adam optimization algorithm using NumPy. Adam is an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments.

Original paper: https://openreview.net/forum?id=ryQu7f-RZ

Class Attributes:

  • params (dict, optional) – Parameters to be stored for optimization.

  • model_interface – Model interface instance to provide parameters.

  • learning_rate (float) – Tuning parameter that determines the step size at each iteration.

  • betas (tuple) – Coefficients used for computing running averages of gradient and its square.

  • initial_accumulator_value (float) – Initial value for gradients and squared gradients.

  • epsilon (float) – Value for computational stability.

Methods

step

Perform a single step for parameter update.
step(gradients: Dict[str, ndarray]) → None

Perform a single step for parameter update.

Implement Adam optimizer weights update rule.

Parameters:

gradients (dict) – Partial derivatives with respect to optimized parameters.

Raises:

KeyError – If a key in gradients does not exist in optimized parameters.