7-1-3. Aggregator

The Aggregator plays a crucial role in integrating the learned models of distributed encrypted states. Below are the main roles and flow of the Aggregator.

1. Integration of SubJob results and generation of intermediate data The Aggregator receives the results of the processed SubJobs and applies a unique integration processing logic corresponding to the ProgramId to integrate them. In this integration process, the learning results obtained from each SubJob are combined to update the overall learning model. The integrated learning model is saved as intermediate data (checkpoint file) to be used in the next epoch. This ensures the consistency of the distributed learning process and enables efficient learning.

2. Generation of final results When all epochs are completed and the final learning model is generated, a final integration process by the Aggregator is carried out. In this final integration process, the intermediate data generated in each epoch is integrated to produce a complete learning model as the processing result of the parent job. This final learning model is used for inference and prediction in the AI system.

The Aggregator plays an indispensable role in the distributed learning process. By properly integrating the learning results obtained from each SubJob, it ensures the quality and consistency of the overall learning model. Also, the Aggregator handles learning results in an encrypted state, protecting data privacy and security.

The function of the Aggregator occupies an important position in the architecture of EMETH. It manages the distributed learning process and integrates learning results, enabling efficient processing of large data sets. Also, the Aggregator collaborates with the Orchestrator, contributing to the control of the overall processing flow.

The Aggregator incorporates the concept of federated learning, which is widely used in fields such as machine learning and deep learning. In federated learning, each node learns using its own data, and by sharing only the learning results, it improves the overall learning model while protecting data privacy. The Aggregator extends this idea and achieves even higher privacy protection using encryption.