Class - LocalDataSource

Class - LocalDataSource#

class openfl.federated.data.sources.local_data_source.LocalDataSource(name, source_path, base_path, hash_func=<built-in function openssl_sha384>, max_dataset_size=0)[source]#

Bases: DataSource

This class represents a local data source.

Parameters:

  • name (str)

  • source_path (Path)

  • hash_func (Callable[[...], hashlib._Hash])

__init__(name, source_path, base_path, hash_func=<built-in function openssl_sha384>, max_dataset_size=0)[source]#

Initialize a LocalDataSource object.

Parameters:

  • name (str) – The name of the data source.

  • source_path (Path) – The path to the source data, relative to base_path.

  • base_path (Path) – The base path to the data source.

  • hash_func (Callable[..., hashlib._Hash]) – The hash function from hashlib

  • data. (to use to hash the)

  • max_dataset_size (int) – The maximum size of the dataset in GB.

Methods

__init__(name, source_path, base_path[, ...])

Initialize a LocalDataSource object.

compute_file_hash(path)

Compute the hash of the file.

enumerate_files()

Enumerate all files in the data source.

from_dict(ds_dict, base_path)

Create a DataSource from a dictionary.

get_source_full_path()

Return the full path to the source data.

is_valid_hash_function(func)

read_blob(path)

Read a blob from the data source.

to_dict()

Convert the object to a dictionary using the serialization rules.
compute_file_hash(path)[source]#

Compute the hash of the file. Return hash on hexstring format.

Parameters:

path (str)

Return type:

str

enumerate_files()[source]#

Enumerate all files in the data source.

Return type:

Generator[str, None, None]

classmethod from_dict(ds_dict, base_path)[source]#

Create a DataSource from a dictionary.

Parameters:

ds_dict (dict) – The dictionary to convert.

Returns:

The created DataSource.

Return type:

DataSource

get_source_full_path()[source]#

Return the full path to the source data.

read_blob(path)[source]#

Read a blob from the data source.

Parameters:

path (str)

Return type:

bytes

to_dict()[source]#

Convert the object to a dictionary using the serialization rules.

Return type:

Dict[str, Any]

Contents