Class - S3DataSource

class openfl.federated.data.sources.s3_data_source.S3DataSource(name, uri, endpoint=None, access_key_env_name=None, secret_key_env_name=None, secret_name=None, hash_func=None)

Bases: DataSource

Class for S3 data

Parameters:

uri (str)

__init__(name, uri, endpoint=None, access_key_env_name=None, secret_key_env_name=None, secret_name=None, hash_func=None)

Initialize a DataSource.

Parameters:

• type (DataSourceType) – The storage type of the data source.

• name (str) – The name of the data source.

• uri (str)

Methods

__init__(name, uri[, endpoint, ...])	Initialize a DataSource.
compute_file_hash(path)	Compute the hash of the object or file.
enumerate_files()	Enumerate all files in the data source
from_dict(ds_dict)	Create a DataSource from a dictionary.
is_valid_hash_function(func)
read_blob(path)	Read a blob from the data source.
to_dict()	Convert the object to a dictionary using the serialization rules.

compute_file_hash(path)

Compute the hash of the object or file.

Parameters:

path (str) – Path to the file.

Returns:

The file’s hash.

Return type:

str

enumerate_files()

Enumerate all files in the data source

Return type:

Generator[str, None, None]

classmethod from_dict(ds_dict)

Create a DataSource from a dictionary.

Parameters:

ds_dict (dict) – The dictionary to convert.

Returns:

The created DataSource.

Return type:

DataSource

read_blob(path)

Read a blob from the data source. :param path: Path to the file. :type path: str

Returns:

The file’s content.

Return type:

bytes

to_dict()

Convert the object to a dictionary using the serialization rules.

Return type:

Dict[str, Any]