Quickstart#
In this guide, we will train a simple Convolutional Neural Network (CNN) on MNIST handwritten digits dataset. We will simulate a Federated Learning experiment between two collaborators, orchestrated by an aggregator, via few CLI commands.
OpenFL must be installed for this tutorial. Refer to the installation guide.
(Optional) For a simplified setup without TLS, follow these steps as applicable throughout the process:
Set
use_tls: falsein the FL plan before initialization.Skip all certificate-related commands (incl. CA setup).
Manually populate the collaborator names in
cols.yaml. For example:
collaborators:
- bob
- charlie
Create a workspace#
We will reuse an existing workspace. OpenFL provides many reference model templates. Check them out under workspaces.
!fx workspace create --prefix ./mnist_example --template keras/mnist
%cd ./mnist_example
!pip install -r requirements.txt > /dev/null
Show code cell output
Hide code cell output
Creating Workspace Directories
Creating Workspace Templates from /home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl-workspace/keras/mnist in /home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/docs/tutorials/mnist_example
[10/14/25 21:57:19] INFO Loading DEFAULTS for section aggregator from file ]8;id=903067;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=198290;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#186\186]8;;\
/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl-workspace/workspace/plan/defaults/aggre
gator.yaml.
INFO Loading DEFAULTS for section assigner from file ]8;id=67260;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=280420;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#186\186]8;;\
/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl-workspace/workspace/plan/defaults/assig
ner.yaml.
INFO Loading DEFAULTS for section collaborator from file ]8;id=908936;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=78353;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#186\186]8;;\
/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl-workspace/workspace/plan/defaults/colla
borator.yaml.
INFO Loading DEFAULTS for section compression_pipeline from file ]8;id=436269;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=472782;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#186\186]8;;\
/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl-workspace/workspace/plan/defaults/compr
ession_pipeline.yaml.
INFO Loading DEFAULTS for section data_loader from file ]8;id=935663;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=478433;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#186\186]8;;\
/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl-workspace/workspace/plan/defaults/data_
loader.yaml.
INFO Loading DEFAULTS for section network from file ]8;id=552825;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=323519;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#186\186]8;;\
/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl-workspace/workspace/plan/defaults/netwo
rk.yaml.
INFO Loading DEFAULTS for section task_runner from file ]8;id=25694;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=924443;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#186\186]8;;\
/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl-workspace/workspace/plan/defaults/task_
runner.yaml.
INFO Loading DEFAULTS for section tasks from file ]8;id=631154;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=507695;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#186\186]8;;\
/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl-workspace/workspace/plan/defaults/tasks
_keras.yaml.
INFO FL-Plan hash is 71f93e3c5870ca465935df0180c419181cd57832297069eb737d448dfa00cb6f8474dfab1489dd6621521dbf084e2565 ]8;id=502489;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=664589;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#305\305]8;;\
INFO FL-Plan hash is d806a6e6440eab83603c7fbc0948bd51ea7c9ed61431e05db9b49d5cf46d1e9d37bccc4a8e3aa841379dcfed3ed028e9 ]8;id=765654;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=666251;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#305\305]8;;\
INFO Parsing Federated Learning Plan : SUCCESS : ]8;id=204791;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=69851;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#152\152]8;;\
/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl-workspace/keras/mnist/plan/plan.yaml.
INFO aggregator: ]8;id=476496;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=3007;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#157\157]8;;\
settings:
best_state_path: save/best.pbuf
db_store_rounds: 2
init_state_path: save/init.pbuf
last_state_path: save/last.pbuf
persist_checkpoint: true
persistent_db_path: local_state/tensor.db
rounds_to_train: 10
template: openfl.component.Aggregator
assigner:
settings:
task_groups:
- name: learning
percentage: 1.0
tasks:
- aggregated_model_validation
- train
- locally_tuned_model_validation
- name: evaluation
percentage: 0
tasks:
- aggregated_model_validation
template: openfl.component.RandomGroupedAssigner
collaborator:
settings:
db_store_rounds: 1
opt_treatment: RESET
use_delta_updates: false
template: openfl.component.Collaborator
compression_pipeline:
settings: {}
template: openfl.pipelines.NoCompressionPipeline
data_loader:
settings:
batch_size: 256
collaborator_count: 2
data_group_name: mnist
template: src.dataloader.KerasMNISTInMemory
network:
settings:
agg_addr: build-29946529-project-638300-openfl
agg_port: 58428
cert_folder: cert
client_reconnect_interval: 5
enable_atomic_connections: false
hash_salt: auto
require_client_auth: true
transport_protocol: grpc
use_tls: true
template: openfl.federation.Network
task_runner:
settings: {}
template: src.taskrunner.KerasCNN
tasks:
aggregated_model_validation:
function: validate_task
kwargs:
apply: global
batch_size: 32
metrics:
- accuracy
locally_tuned_model_validation:
function: validate_task
kwargs:
apply: local
batch_size: 32
metrics:
- accuracy
settings: {}
train:
function: train_task
kwargs:
batch_size: 32
epochs: 1
metrics:
- loss
New workspace directory structure:
mnist_example
├── src
│ ├── taskrunner.py
│ ├── __init__.py
│ ├── mnist_utils.py
│ └── dataloader.py
├── cert
├── plan
│ ├── cols.yaml
│ ├── data.yaml
│ ├── plan.yaml
│ └── defaults
├── .workspace
├── data
├── local_state
├── logs
├── requirements.txt
└── save
7 directories, 10 files
✔️ OK
/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/docs/tutorials/mnist_example
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/IPython/core/magics/osm.py:417: UserWarning: This is now an optional IPython functionality, setting dhist requires you to install the `pickleshare` library.
self.shell.db['dhist'] = compress_dhist(dhist)[-100:]
Initialize a Plan#
This step builds an entire FL experiment plan, along with the initial set of parameters that will be used in the experiment.
We supply localhost as the aggregator address, for simulation purposes. The command below supports models initializations (e.g. when using pre-trained models).
For a complete list of supported options, refer to fx plan initialize.
!fx plan initialize -a localhost
Show code cell output
Hide code cell output
2025-10-14 21:57:53.791057: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-10-14 21:57:53.798437: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:57:53.801630: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:57:53.810029: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1760479073.823094 652 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1760479073.826978 652 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-10-14 21:57:53.842253: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
[10/14/25 21:57:55] INFO FL-Plan hash is e4e8c761622a69752bf38c954836e189e5e13b2cb73975d28a428814984758ba75930316897fa1ba02230a6125399e72 ]8;id=442621;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=170198;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#305\305]8;;\
INFO Parsing Federated Learning Plan : SUCCESS : ]8;id=758558;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=570981;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#152\152]8;;\
/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/docs/tutorials/mnist_example/plan/plan.yaml.
INFO aggregator: ]8;id=346778;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=337506;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#157\157]8;;\
settings:
best_state_path: save/best.pbuf
db_store_rounds: 2
init_state_path: save/init.pbuf
last_state_path: save/last.pbuf
persist_checkpoint: true
persistent_db_path: local_state/tensor.db
rounds_to_train: 10
template: openfl.component.Aggregator
assigner:
settings:
task_groups:
- name: learning
percentage: 1.0
tasks:
- aggregated_model_validation
- train
- locally_tuned_model_validation
- name: evaluation
percentage: 0
tasks:
- aggregated_model_validation
template: openfl.component.RandomGroupedAssigner
collaborator:
settings:
db_store_rounds: 1
opt_treatment: RESET
use_delta_updates: false
template: openfl.component.Collaborator
compression_pipeline:
settings: {}
template: openfl.pipelines.NoCompressionPipeline
data_loader:
settings:
batch_size: 256
collaborator_count: 2
data_group_name: mnist
template: src.dataloader.KerasMNISTInMemory
network:
settings:
agg_addr: build-29946529-project-638300-openfl
agg_port: 58428
cert_folder: cert
client_reconnect_interval: 5
enable_atomic_connections: false
hash_salt: auto
require_client_auth: true
transport_protocol: grpc
use_tls: true
template: openfl.federation.Network
task_runner:
settings: {}
template: src.taskrunner.KerasCNN
tasks:
aggregated_model_validation:
function: validate_task
kwargs:
apply: global
batch_size: 32
metrics:
- accuracy
locally_tuned_model_validation:
function: validate_task
kwargs:
apply: local
batch_size: 32
metrics:
- accuracy
settings: {}
train:
function: train_task
kwargs:
batch_size: 32
epochs: 1
metrics:
- loss
INFO Initialized minimal dataloader for model creation ]8;id=758243;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/utilities/dataloading.py\dataloading.py]8;;\:]8;id=728876;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/utilities/dataloading.py#54\54]8;;\
INFO Building `src.taskrunner.KerasCNN` Module. ]8;id=516607;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=134799;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#243\243]8;;\
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/keras/src/layers/convolutional/base_conv.py:113: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead.
super().__init__(activity_regularizer=activity_regularizer, **kwargs)
2025-10-14 21:57:55.751156: E external/local_xla/xla/stream_executor/cuda/cuda_driver.cc:152] failed call to cuInit: INTERNAL: CUDA error: Failed call to cuInit: UNKNOWN ERROR (303)
INFO Model: "sequential" ]8;id=710148;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/keras/src/utils/summary_utils.py\summary_utils.py]8;;\:]8;id=426228;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/keras/src/utils/summary_utils.py#389\389]8;;\
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type) ┃ Output Shape ┃ Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ conv2d (Conv2D) │ (None, 13, 13, 16) │ 272 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_1 (Conv2D) │ (None, 5, 5, 32) │ 8,224 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten (Flatten) │ (None, 800) │ 0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense (Dense) │ (None, 100) │ 80,100 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense) │ (None, 10) │ 1,010 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 89,606 (350.02 KB)
Trainable params: 89,606 (350.02 KB)
Non-trainable params: 0 (0.00 B)
INFO Building `openfl.pipelines.NoCompressionPipeline` Module. ]8;id=494229;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=675069;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#243\243]8;;\
WARNING Following parameters omitted from global initial model, local initialization will determine values: [] ]8;id=890425;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/plan.py\plan.py]8;;\:]8;id=910597;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/plan.py#132\132]8;;\
INFO Saving model state to save/init.pbuf ]8;id=476223;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/plan.py\plan.py]8;;\:]8;id=905429;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/plan.py#140\140]8;;\
WARNING Patching Aggregator Addr in Plan 🠆 localhost ]8;id=56933;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/plan.py\plan.py]8;;\:]8;id=662426;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/plan.py#157\157]8;;\
INFO FL-Plan hash is f98a8114522480d1529ff5d62a5d4fa00af1e0cb0e88ba9551907ef69f60afdfda76378c86eaf8ee454a5b16d6e4792a ]8;id=722102;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=853475;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#305\305]8;;\
INFO ['plan_f98a8114'] ]8;id=365492;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/plan.py\plan.py]8;;\:]8;id=841054;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/plan.py#171\171]8;;\
✔️ OK
Create a certificate authority (CA)#
OpenFL supports mTLS, which ensures secure communication between the collaborators and the aggregator. This step generates a certificate authority (CA) that will be used to sign the certificates of the collaborators. The CA is generated only once and can be reused for multiple experiments.
No additional arguments are required for this command.
!fx workspace certify
Show code cell output
Hide code cell output
2025-10-14 21:57:57.514774: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-10-14 21:57:57.515514: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:57:57.518733: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:57:57.527147: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1760479077.540263 663 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1760479077.544077 663 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-10-14 21:57:57.559508: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
Setting Up Certificate Authority...
1. Create Root CA
1.1 Create Directories
1.2 Create Database
1.3 Create CA Request and Certificate
2. Create Signing Certificate
2.1 Create Directories
2.2 Create Database
2.3 Create Signing Certificate CSR
2.4 Sign Signing Certificate CSR
3 Create Certificate Chain
Done.
✔️ OK
Aggregator Setup#
The fx aggregator subcommand options to manage the aggregator-side certificate signing, generation and verification.
Here, we generate the aggregator’s certificate and private key. The aggregator’s certificate is then signed by the CA generated in the previous step.
For all commands that follow, --silent is used to skip manual confirmation. In practice, users are recommended to verify the details before proceeding.
!fx aggregator generate-cert-request --fqdn localhost
!fx aggregator certify --fqdn localhost --silent
Show code cell output
Hide code cell output
2025-10-14 21:58:01.292175: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-10-14 21:58:01.292895: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:01.296058: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:01.304452: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1760479081.317450 669 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1760479081.321252 669 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-10-14 21:58:01.336740: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
Creating AGGREGATOR certificate key pair with following settings: CN=localhost, SAN=DNS:localhost
Writing AGGREGATOR certificate key pair to: /home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/docs/tutorials/mnist_example/cert/server
The CSR Hash 5087f6e1a6516b0a9eb2c81d552b8ec1606fd74c8e0e9fe3fb83ff9e9885b8b172235c5e695d3009d9e2a511b437c095
✔️ OK
2025-10-14 21:58:04.813832: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-10-14 21:58:04.814564: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:04.817808: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:04.826323: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1760479084.839524 678 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1760479084.843329 678 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-10-14 21:58:04.858713: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
The CSR Hash for file server/agg_localhost.csr = 5087f6e1a6516b0a9eb2c81d552b8ec1606fd74c8e0e9fe3fb83ff9e9885b8b172235c5e695d3009d9e2a511b437c095
Warning: manual check of certificate hashes is bypassed in silent mode.
Signing AGGREGATOR certificate
✔️ OK
Collaborator Setup#
Following aggregator, fx collaborator subcommand manages the collaborator-side certificate signing, generation and verification. In addition, because collaborators bring their own local datasets, we populate dataset information under plan/data.yaml.
Setup first collaborator bob#
This step registers --data_path against collaborator’s name -n under plan/data.yaml.
!fx collaborator create --data_path 0 -n bob --silent
Show code cell output
Hide code cell output
2025-10-14 21:58:08.315291: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-10-14 21:58:08.316022: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:08.319226: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:08.327802: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1760479088.341130 684 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1760479088.344931 684 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-10-14 21:58:08.360448: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
✔️ OK
The next step generates bobs certificate and key pair.
!fx collaborator generate-cert-request -n bob --silent
Show code cell output
Hide code cell output
2025-10-14 21:58:11.654607: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-10-14 21:58:11.655356: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:11.658652: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:11.667477: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1760479091.681473 690 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1760479091.685316 690 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-10-14 21:58:11.701182: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
Creating COLLABORATOR certificate key pair with following settings: CN=bob, SAN=DNS:bob
Moving COLLABORATOR certificate to: /home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/docs/tutorials/mnist_example/cert/col_bob
The CSR Hash cdb8a397573cc662b4d06f36f6d9af4723236a0f8c63981175ad1a15d5471ac9a757556e866ed76a10e3fc2edbc94527
Archive col_bob_to_agg_cert_request.zip with certificate signing request created
This file should be sent to the certificate authority (typically hosted by the aggregator) for signing
✔️ OK
Certify bob with the CA.
Note: Here, we assume CA to be the aggregator. Hence the command below runs on the
aggregatorside. In reality,bob’s certificate signing request (CSR) package generated in the last step, is sent to theaggregator, which then signs the certificate with the CA. The signed certificate then has to be sent back tobob.
!fx collaborator certify --request-pkg col_bob_to_agg_cert_request.zip --silent
Show code cell output
Hide code cell output
2025-10-14 21:58:15.171115: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-10-14 21:58:15.171846: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:15.175058: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:15.183534: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1760479095.196790 696 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1760479095.200635 696 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-10-14 21:58:15.215912: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
The CSR Hash for file col_bob.csr is cdb8a397573cc662b4d06f36f6d9af4723236a0f8c63981175ad1a15d5471ac9a757556e866ed76a10e3fc2edbc94527
Signing COLLABORATOR certificate, Warning: manual check of certificate hashes is bypassed in silent mode.
Registering bob in /home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/docs/tutorials/mnist_example/plan/cols.yaml
✔️ OK
Setup second collaborator charlie#
As before, we create a collaborator by registering --data_path against collaborator’s name -n under plan/data.yaml.
!fx collaborator create --data_path 1 -n charlie --silent
Show code cell output
Hide code cell output
2025-10-14 21:58:18.777672: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-10-14 21:58:18.778405: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:18.781568: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:18.789998: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1760479098.802967 702 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1760479098.806784 702 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-10-14 21:58:18.822280: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
✔️ OK
Now, generate charlies certificate and key pair.
!fx collaborator generate-cert-request -n charlie --silent
Show code cell output
Hide code cell output
2025-10-14 21:58:22.189653: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-10-14 21:58:22.190365: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:22.193573: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:22.202085: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1760479102.215224 711 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1760479102.219060 711 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-10-14 21:58:22.234502: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
Creating COLLABORATOR certificate key pair with following settings: CN=charlie, SAN=DNS:charlie
Moving COLLABORATOR certificate to: /home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/docs/tutorials/mnist_example/cert/col_charlie
The CSR Hash ee91bed1c5f091647ad90a64593fa932e1019681ae7fa358783cae7a343b67744102572af3dc458576f707428eaec13d
Archive col_charlie_to_agg_cert_request.zip with certificate signing request created
This file should be sent to the certificate authority (typically hosted by the aggregator) for signing
✔️ OK
Certify charlie with the CA.
Note: Again, we assume CA to be the aggregator. Hence the command below is expected to be run on the
aggregatorside. In practice,charlie’s certificate signing request (CSR) package generated in the last step, is sent to theaggregator, which then signs the certificate with the CA. The signed certificate then has to be sent back tocharlie.
!fx collaborator certify --request-pkg col_charlie_to_agg_cert_request.zip --silent
Show code cell output
Hide code cell output
2025-10-14 21:58:25.760979: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-10-14 21:58:25.761719: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:25.764967: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:25.773525: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1760479105.786918 717 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1760479105.790759 717 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-10-14 21:58:25.806122: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
The CSR Hash for file col_charlie.csr is ee91bed1c5f091647ad90a64593fa932e1019681ae7fa358783cae7a343b67744102572af3dc458576f707428eaec13d
Signing COLLABORATOR certificate, Warning: manual check of certificate hashes is bypassed in silent mode.
Registering charlie in /home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/docs/tutorials/mnist_example/plan/cols.yaml
✔️ OK
Import CA signed certificates#
This step imports the CA-signed certificates of the collaborator and the aggregator into the workspace. We are back on the individual collaborator nodes for this step.
!fx collaborator certify --import agg_to_col_bob_signed_cert.zip
!fx collaborator certify --import agg_to_col_charlie_signed_cert.zip
Show code cell output
Hide code cell output
2025-10-14 21:58:29.289859: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-10-14 21:58:29.290595: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:29.293799: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:29.302274: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1760479109.315417 723 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1760479109.319224 723 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-10-14 21:58:29.334579: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
Certificate updated in the PKI directory
✔️ OK
2025-10-14 21:58:32.770449: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-10-14 21:58:32.771183: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:32.774439: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:32.782961: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1760479112.796303 729 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1760479112.800152 729 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-10-14 21:58:32.815881: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
Certificate updated in the PKI directory
✔️ OK
Start the simulation#
This step starts the simulation of the FL experiment. The aggregator will orchestrate the training process between the collaborators.
In practice, this command is executed on respective aggregator/collaborator nodes.
!fx aggregator start &\
fx collaborator start -n bob &\
fx collaborator start -n charlie
Show code cell output
Hide code cell output
2025-10-14 21:58:36.478264: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-10-14 21:58:36.480073: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:36.486677: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:36.504028: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1760479116.534401 740 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1760479116.540251 740 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-10-14 21:58:36.572074: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-10-14 21:58:36.629706: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-10-14 21:58:36.630351: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:36.630795: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-10-14 21:58:36.631326: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:36.636507: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:36.638062: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2025-10-14 21:58:36.654821: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2025-10-14 21:58:36.656091: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1760479116.683139 738 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1760479116.683846 739 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1760479116.691094 738 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
E0000 00:00:1760479116.691600 739 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-10-14 21:58:36.719536: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-10-14 21:58:36.723175: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
[10/14/25 21:58:39] INFO FL-Plan hash is f98a8114522480d1529ff5d62a5d4fa00af1e0cb0e88ba9551907ef69f60afdfda76378c86eaf8ee454a5b16d6e4792a ]8;id=187885;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=206463;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#305\305]8;;\
INFO Parsing Federated Learning Plan : SUCCESS : ]8;id=257104;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=97328;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#152\152]8;;\
/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/docs/tutorials/mnist_example/plan/plan.yaml.
INFO aggregator: ]8;id=254698;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=988058;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#157\157]8;;\
settings:
best_state_path: save/best.pbuf
db_store_rounds: 2
init_state_path: save/init.pbuf
last_state_path: save/last.pbuf
persist_checkpoint: true
persistent_db_path: local_state/tensor.db
rounds_to_train: 10
template: openfl.component.Aggregator
assigner:
settings:
task_groups:
- name: learning
percentage: 1.0
tasks:
- aggregated_model_validation
- train
- locally_tuned_model_validation
- name: evaluation
percentage: 0
tasks:
- aggregated_model_validation
template: openfl.component.RandomGroupedAssigner
collaborator:
settings:
db_store_rounds: 1
opt_treatment: RESET
use_delta_updates: false
template: openfl.component.Collaborator
compression_pipeline:
settings: {}
template: openfl.pipelines.NoCompressionPipeline
data_loader:
settings:
batch_size: 256
collaborator_count: 2
data_group_name: mnist
template: src.dataloader.KerasMNISTInMemory
network:
settings:
agg_addr: localhost
agg_port: 58428
cert_folder: cert
client_reconnect_interval: 5
enable_atomic_connections: false
hash_salt: auto
require_client_auth: true
transport_protocol: grpc
use_tls: true
template: openfl.federation.Network
task_runner:
settings: {}
template: src.taskrunner.KerasCNN
tasks:
aggregated_model_validation:
function: validate_task
kwargs:
apply: global
batch_size: 32
metrics:
- accuracy
locally_tuned_model_validation:
function: validate_task
kwargs:
apply: local
batch_size: 32
metrics:
- accuracy
settings: {}
train:
function: train_task
kwargs:
batch_size: 32
epochs: 1
metrics:
- loss
INFO Data paths: {'one': '1', 'bob': '0', 'charlie': '1'} ]8;id=555899;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/collaborator.py\collaborator.py]8;;\:]8;id=292763;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/collaborator.py#80\80]8;;\
Data = {'one': '1', 'bob': '0', 'charlie': '1'}
INFO 🧿 Starting a Collaborator Service. ]8;id=696623;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/collaborator.py\collaborator.py]8;;\:]8;id=496158;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/collaborator.py#82\82]8;;\
INFO Building `src.dataloader.KerasMNISTInMemory` Module. ]8;id=850196;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=11394;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#243\243]8;;\
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
warnings.warn(
8192/11490434 [..............................] - ETA: 0s
[10/14/25 21:58:39] INFO FL-Plan hash is f98a8114522480d1529ff5d62a5d4fa00af1e0cb0e88ba9551907ef69f60afdfda76378c86eaf8ee454a5b16d6e4792a ]8;id=477166;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=503331;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#305\305]8;;\
INFO Parsing Federated Learning Plan : SUCCESS : ]8;id=211145;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=718336;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#152\152]8;;\
/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/docs/tutorials/mnist_example/plan/plan.yaml.
565248/11490434 [>.............................] - ETA: 1s INFO aggregator: ]8;id=620266;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=177266;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#157\157]8;;\
settings:
best_state_path: save/best.pbuf
db_store_rounds: 2
init_state_path: save/init.pbuf
last_state_path: save/last.pbuf
persist_checkpoint: true
persistent_db_path: local_state/tensor.db
rounds_to_train: 10
template: openfl.component.Aggregator
assigner:
settings:
task_groups:
- name: learning
percentage: 1.0
tasks:
- aggregated_model_validation
- train
- locally_tuned_model_validation
- name: evaluation
percentage: 0
tasks:
- aggregated_model_validation
template: openfl.component.RandomGroupedAssigner
collaborator:
settings:
db_store_rounds: 1
opt_treatment: RESET
use_delta_updates: false
template: openfl.component.Collaborator
compression_pipeline:
settings: {}
template: openfl.pipelines.NoCompressionPipeline
data_loader:
settings:
batch_size: 256
collaborator_count: 2
data_group_name: mnist
template: src.dataloader.KerasMNISTInMemory
network:
settings:
agg_addr: localhost
agg_port: 58428
cert_folder: cert
client_reconnect_interval: 5
enable_atomic_connections: false
hash_salt: auto
require_client_auth: true
transport_protocol: grpc
use_tls: true
template: openfl.federation.Network
task_runner:
settings: {}
template: src.taskrunner.KerasCNN
tasks:
aggregated_model_validation:
function: validate_task
kwargs:
apply: global
batch_size: 32
metrics:
- accuracy
locally_tuned_model_validation:
function: validate_task
kwargs:
apply: local
batch_size: 32
metrics:
- accuracy
settings: {}
train:
function: train_task
kwargs:
batch_size: 32
epochs: 1
metrics:
- loss
INFO 🧿 Starting the Aggregator Service. ]8;id=959690;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/aggregator.py\aggregator.py]8;;\:]8;id=205161;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/aggregator.py#94\94]8;;\
INFO Building `openfl.component.RandomGroupedAssigner` Module. ]8;id=659002;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=85990;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#243\243]8;;\
[10/14/25 21:58:39] INFO FL-Plan hash is f98a8114522480d1529ff5d62a5d4fa00af1e0cb0e88ba9551907ef69f60afdfda76378c86eaf8ee454a5b16d6e4792a ]8;id=851074;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=260837;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#305\305]8;;\
INFO Parsing Federated Learning Plan : SUCCESS : ]8;id=35027;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=679735;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#152\152]8;;\
/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/docs/tutorials/mnist_example/plan/plan.yaml.
3129344/11490434 [=======>......................] - ETA: 0s INFO aggregator: ]8;id=155344;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=594477;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#157\157]8;;\
settings:
best_state_path: save/best.pbuf
db_store_rounds: 2
init_state_path: save/init.pbuf
last_state_path: save/last.pbuf
persist_checkpoint: true
persistent_db_path: local_state/tensor.db
rounds_to_train: 10
template: openfl.component.Aggregator
assigner:
settings:
task_groups:
- name: learning
percentage: 1.0
tasks:
- aggregated_model_validation
- train
- locally_tuned_model_validation
- name: evaluation
percentage: 0
tasks:
- aggregated_model_validation
template: openfl.component.RandomGroupedAssigner
collaborator:
settings:
db_store_rounds: 1
opt_treatment: RESET
use_delta_updates: false
template: openfl.component.Collaborator
compression_pipeline:
settings: {}
template: openfl.pipelines.NoCompressionPipeline
data_loader:
settings:
batch_size: 256
collaborator_count: 2
data_group_name: mnist
template: src.dataloader.KerasMNISTInMemory
network:
settings:
agg_addr: localhost
agg_port: 58428
cert_folder: cert
client_reconnect_interval: 5
enable_atomic_connections: false
hash_salt: auto
require_client_auth: true
transport_protocol: grpc
use_tls: true
template: openfl.federation.Network
task_runner:
settings: {}
template: src.taskrunner.KerasCNN
tasks:
aggregated_model_validation:
function: validate_task
kwargs:
apply: global
batch_size: 32
metrics:
- accuracy
locally_tuned_model_validation:
function: validate_task
kwargs:
apply: local
batch_size: 32
metrics:
- accuracy
settings: {}
train:
function: train_task
kwargs:
batch_size: 32
epochs: 1
metrics:
- loss
INFO Data paths: {'one': '1', 'bob': '0', 'charlie': '1'} ]8;id=545345;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/collaborator.py\collaborator.py]8;;\:]8;id=289412;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/collaborator.py#80\80]8;;\
Data = {'one': '1', 'bob': '0', 'charlie': '1'}
INFO 🧿 Starting a Collaborator Service. ]8;id=542915;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/collaborator.py\collaborator.py]8;;\:]8;id=300908;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/collaborator.py#82\82]8;;\
INFO Building `src.dataloader.KerasMNISTInMemory` Module. ]8;id=978501;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=448178;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#243\243]8;;\
A local file was found, but it seems to be incomplete or outdated because the auto file hash does not match the original value of 731c5ac602752760c8e48fbffcf8c3b850d9dc2a2aedcf2cc48468fc17b673d1 so we will re-download the data.
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
5595136/11490434 [=============>................] - ETA: 0s INFO Building `openfl.pipelines.NoCompressionPipeline` Module. ]8;id=619747;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=338597;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#243\243]8;;\
INFO Importing 🡆 Object WaitForAllPolicy from openfl.component.aggregator.straggler_handling Module. ]8;id=263259;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=773653;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#267\267]8;;\
INFO Connector defaults: None ]8;id=727396;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=788814;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#355\355]8;;\
INFO Building `openfl.component.Aggregator` Module. ]8;id=511692;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py\plan.py]8;;\:]8;id=91621;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py#243\243]8;;\
INFO Persistent checkpoint is enabled, setting persistent db at path local_state/tensor.db ]8;id=62026;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/component/aggregator/aggregator.py\aggregator.py]8;;\:]8;id=987627;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/component/aggregator/aggregator.py#157\157]8;;\
INFO Initializing persistent db at local_state/tensor.db ]8;id=524451;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/databases/persistent_db.py\persistent_db.py]8;;\:]8;id=142683;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/databases/persistent_db.py#36\36]8;;\
7757824/11490434 [===================>..........] - ETA: 0s
INFO Starting Aggregator gRPC Server ]8;id=554280;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/transport/grpc/aggregator_server.py\aggregator_server.py]8;;\:]8;id=340555;file:///home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/transport/grpc/aggregator_server.py#354\354]8;;\
10166272/11490434 [=========================>....] - ETA: 0s
11493376/11490434 [==============================] - 0s 0us/step
EXCEPTION : Bad magic number for file header
557056/11490434 [>.............................] - ETA: 1sTraceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/bin/fx", line 7, in <module>
sys.exit(entry())
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/cli.py", line 310, in entry
error_handler(e)
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/cli.py", line 229, in error_handler
raise error
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/cli.py", line 308, in entry
cli(max_content_width=120)
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/click/core.py", line 1462, in __call__
return self.main(*args, **kwargs)
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/click/core.py", line 1383, in main
rv = self.invoke(ctx)
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/cli.py", line 131, in invoke
super().invoke(ctx)
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/click/core.py", line 1850, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/click/core.py", line 1850, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/click/core.py", line 1246, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/click/core.py", line 814, in invoke
return callback(*args, **kwargs)
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/interface/collaborator.py", line 84, in start_
collaborator = plan_obj.get_collaborator(collaborator_name)
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py", line 548, in get_collaborator
data_loader = self.get_data_loader(collaborator_name)
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py", line 473, in get_data_loader
self.loader_ = Plan.build(**defaults)
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/openfl/federated/plan/plan.py", line 250, in build
instance = getattr(module, class_name)(**settings)
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/docs/tutorials/mnist_example/src/dataloader.py", line 42, in __init__
X_train, y_train, X_valid, y_valid = load_mnist_shard(
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/docs/tutorials/mnist_example/src/mnist_utils.py", line 89, in load_mnist_shard
(X_train, y_train), (X_valid, y_valid) = _load_raw_datashards(
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/checkouts/latest/docs/tutorials/mnist_example/src/mnist_utils.py", line 48, in _load_raw_datashards
X_train_tot = f['x_train']
File "/home/docs/checkouts/readthedocs.org/user_builds/openfl/envs/latest/lib/python3.10/site-packages/numpy/lib/_npyio_impl.py", line 251, in __getitem__
bytes = self.zip.open(key)
File "/home/docs/.asdf/installs/python/3.10.17/lib/python3.10/zipfile.py", line 1546, in open
raise BadZipFile("Bad magic number for file header")
zipfile.BadZipFile: Bad magic number for file header
1212416/11490434 [==>...........................] - ETA: 0s
1687552/11490434 [===>..........................] - ETA: 0s
2228224/11490434 [====>.........................] - ETA: 0s
2867200/11490434 [======>.......................] - ETA: 0s
3506176/11490434 [========>.....................] - ETA: 0s
4202496/11490434 [=========>....................] - ETA: 0s
4751360/11490434 [===========>..................] - ETA: 0s
5210112/11490434 [============>.................] - ETA: 0s
5816320/11490434 [==============>...............] - ETA: 0s
6471680/11490434 [===============>..............] - ETA: 0s
7143424/11490434 [=================>............] - ETA: 0s
7766016/11490434 [===================>..........] - ETA: 0s
Next steps#
The simulation will run for a few epochs. Once the simulation is complete, the model will be saved under the save directory. The model can be converted to framework native formats using the fx model save command.
For detailed information on options supported via CLI, refer to the CLI reference.
For an in-depth understanding of the TaskRunner architecture, refer to the TaskRunner API.