openfl.utilities.optimizers.numpy.adagrad_optimizer.NumPyAdagrad

class openfl.utilities.optimizers.numpy.adagrad_optimizer.NumPyAdagrad(*, params=None, model_interface=None, learning_rate=0.01, initial_accumulator_value=0.1, epsilon=1e-10)

Bases: Optimizer

Adagrad optimizer implementation.

Implements the Adagrad optimization algorithm using NumPy. Adagrad is an algorithm for gradient-based optimization that adapts the learning rate to the parameters, performing smaller updates for parameters associated with frequently occurring features, and larger updates for parameters associated with infrequent features.

Original paper: http://jmlr.org/papers/v12/duchi11a.html

Parameters:

• params (Dict[str, ndarray] | None)

• learning_rate (float)

• initial_accumulator_value (float)

• epsilon (float)

params

Parameters to be stored for optimization.

Type:

dict, optional

model_interface

Model interface instance to provide parameters.

learning_rate

Tuning parameter that determines the step size at each iteration.

Type:

float

initial_accumulator_value

Initial value for squared gradients.

Type:

float

epsilon

Value for computational stability.

Type:

float

__init__(*, params=None, model_interface=None, learning_rate=0.01, initial_accumulator_value=0.1, epsilon=1e-10)

Initialize the Adagrad optimizer.

Parameters:

• params (dict, optional) – Parameters to be stored for optimization. Defaults to None.

• model_interface – Model interface instance to provide parameters. Defaults to None.

• learning_rate (float, optional) – Tuning parameter that determines the step size at each iteration. Defaults to 0.01.

• initial_accumulator_value (float, optional) – Initial value for squared gradients. Defaults to 0.1.

• epsilon (float, optional) – Value for computational stability. Defaults to 1e-10.

Raises:

• ValueError – If both params and model_interface are None.

• ValueError – If learning_rate is less than 0.

• ValueError – If initial_accumulator_value is less than 0.

• ValueError – If epsilon is less than or equal to 0.

Return type:

None

Methods

__init__(*[, params, model_interface, ...])	Initialize the Adagrad optimizer.
step(gradients)	Perform a single step for parameter update.

step(gradients)

Perform a single step for parameter update.

Implement Adagrad optimizer weights update rule.

Parameters:

gradients (dict) – Partial derivatives with respect to optimized parameters.

Raises:

KeyError – If a key in gradients does not exist in optimized parameters.

Return type:

None