openfl.utilities.data_splitters.numpy.LogNormalNumPyDataSplitter

class openfl.utilities.data_splitters.numpy.LogNormalNumPyDataSplitter(mu, sigma, num_classes, classes_per_col, min_samples_per_class, seed=0)

Bases: NumPyDataSplitter

Class for splitting numpy arrays of data according to a LogNormal distribution.

Unbalanced (LogNormal) dataset split. This split assumes only several classes are assigned to each collaborator. Firstly, it assigns classes_per_col * min_samples_per_class items of dataset to each collaborator so all of collaborators will have some data after the split. Then, it generates positive integer numbers by log-normal (power) law. These numbers correspond to numbers of dataset items picked each time from dataset and assigned to a collaborator. Generation is repeated for each class assigned to a collaborator. This is a parametrized version of non-i.i.d. data split in FedProx algorithm. Origin source: https://github.com/litian96/FedProx/blob/master/data/mnist/generate_niid.py#L30

Parameters:

  • mu (float) – Distribution hyperparameter.

  • sigma (float) – Distribution hyperparameter.

  • num_classes (int) – Number of classes.

  • classes_per_col (int) – Number of classes assigned to each collaborator.

  • min_samples_per_class (int) – Minimum number of collaborator samples of each class.

  • seed (int, optional) – Random numbers generator seed. Defaults to 0.

Note

This split always drops out some part of the dataset! Non-deterministic behavior selects only random subpart of class items.

Methods

split

Split the data.
split(data, num_collaborators)

Split the data.

Parameters:

  • data (np.ndarray) – numpy-like label array.

  • num_collaborators (int) – number of collaborators to split data across. Should be divisible by number of classes in data.