openfl.utilities.optimizers.numpy.adam_optimizer.NumPyAdam

openfl.utilities.optimizers.numpy.adam_optimizer.NumPyAdam#

class openfl.utilities.optimizers.numpy.adam_optimizer.NumPyAdam(*, params=None, model_interface=None, learning_rate=0.01, betas=(0.9, 0.999), initial_accumulator_value=0.0, epsilon=1e-08)[source]#

Bases: Optimizer

Adam optimizer implementation.

Implements the Adam optimization algorithm using NumPy. Adam is an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments.

Original paper: https://openreview.net/forum?id=ryQu7f-RZ

Parameters:

  • params (Dict[str, ndarray] | None)

  • learning_rate (float)

  • betas (Tuple[float, float])

  • initial_accumulator_value (float)

  • epsilon (float)

params#

Parameters to be stored for optimization.

Type:

dict, optional

model_interface#

Model interface instance to provide parameters.

learning_rate#

Tuning parameter that determines the step size at each iteration.

Type:

float

betas#

Coefficients used for computing running averages of gradient and its square.

Type:

tuple

initial_accumulator_value#

Initial value for gradients and squared gradients.

Type:

float

epsilon#

Value for computational stability.

Type:

float

__init__(*, params=None, model_interface=None, learning_rate=0.01, betas=(0.9, 0.999), initial_accumulator_value=0.0, epsilon=1e-08)[source]#

Initialize the Adam optimizer.

Parameters:

  • params (dict, optional) – Parameters to be stored for optimization. Defaults to None.

  • model_interface – Model interface instance to provide parameters. Defaults to None.

  • learning_rate (float, optional) – Tuning parameter that determines the step size at each iteration. Defaults to 0.01.

  • betas (tuple, optional) – Coefficients used for computing running averages of gradient and its square. Defaults to (0.9, 0.999).

  • initial_accumulator_value (float, optional) – Initial value for gradients and squared gradients. Defaults to 0.0.

  • epsilon (float, optional) – Value for computational stability. Defaults to 1e-8.

Raises:

  • ValueError – If both params and model_interface are None.

  • ValueError – If learning_rate is less than 0.

  • ValueError – If betas[0] is not in [0, 1).

  • ValueError – If betas[1] is not in [0, 1).

  • ValueError – If initial_accumulator_value is less than 0.

  • ValueError – If epsilon is less than or equal to 0.

Return type:

None

Methods

__init__(*[, params, model_interface, ...])

Initialize the Adam optimizer.

step(gradients)

Perform a single step for parameter update.
step(gradients)[source]#

Perform a single step for parameter update.

Implement Adam optimizer weights update rule.

Parameters:

gradients (dict) – Partial derivatives with respect to optimized parameters.

Raises:

KeyError – If a key in gradients does not exist in optimized parameters.

Return type:

None

Contents