EMETH White Paper
  • EMETH White Paper
  • 1. Introduction
    • 1-1. The Need for a Global Distributed Computing Platform
  • 2. Project
    • 2-1. EMETH's Vision
    • 2-2. Overview of EMETH Project
      • 2-2-1. Data Privacy Protection
      • 2-2-2. Hybrid Parallel Processing
      • 2-2-3. EMETH L2 Roll Up
  • 3.Token Economy
    • 3-1. EMETH ($EMETH) Token Overview
    • 3-2. Token Allocation
    • 3-3. EMETH Token Utility
    • 3-4. GPU Mining Program
      • 3-4-1. Staking Program.
    • 3-5. Calculation Method for JOB Execution Fees
    • 3-6. Overview of fee state transition
  • 4. Node
    • 4-1. Benefits that EMETH node can enjoy
    • 4-2. How to Become a EMETH Node
      • 4-2-1. How to set up EMETH Node for Windows users
      • 4-2-2. How to set up EMETH Node for Ubuntu users
      • 4-2-3. How to set up EMETH Portable for mobile device users
  • 5. Service
    • 5-1. AI Inference
      • 5-1-1 Pricing
    • 5-2. Rent GPUs
  • 6. DAO
    • 6-1. EMETH DAO
  • 7. EMETH Architecture
    • 7-1. Overview
      • 7-1-1. Splitter
      • 7-1-2. Aggregator
      • 7-1-3. Verifier
      • 7-1-4. Signer
    • 7-2. Layer 1 Entire Process
    • 7-3. Layer 2 Entire Process
  • 8. ROADMAP
Powered by GitBook
On this page
  1. 7. EMETH Architecture
  2. 7-1. Overview

7-1-1. Splitter

The Splitter plays a crucial role in encrypting executable jobs where possible and distributing training data to nodes. The following are the main roles and process flow of the Splitter:

  1. Calling the Splitter corresponding to the ProgramId and dividing the dataset The ProgramId is a unique identifier for the program, and the Splitter selects the appropriate subprogram based on this identifier. The Splitter converts the given dataset into an optimal format for machine learning algorithms and divides it according to the NumParallel. NumParallel indicates the number of subprograms to be processed in parallel, and the dataset is evenly divided based on this number.

  2. Dividing Jobs and registering SubJobs to EMETH Core (L1) / EMETH Execution Layer (L2) The Splitter divides the Job corresponding to the split dataset into multiple SubJobs. Each SubJob is assigned a unique ID, maintaining its relationship with the parent Job. The divided SubJobs are registered with EMETH Core (L1) / EMETH Execution Layer (L2), preparing for distributed processing. The ProgramId differs between parent Jobs and SubJobs, expressing the hierarchical structure of the program. (Example: ProgramId:1 = Parent Job processing for overall GPT3 training, ProgramId:11 = SubJob processing for ProgramId:1)

The Splitter skillfully divides programs and jobs, applying confidential encryption to protect data confidentiality while enabling efficient distributed processing. This functionality allows for fast and secure processing even when dealing with large-scale datasets.

The role of the Splitter is extremely important in EMETH's architecture. The division and distribution of data form the foundation for parallel processing, significantly improving overall performance. Furthermore, the protection of data through confidential encryption is essential from privacy and security perspectives.

Previous7-1. OverviewNext7-1-2. Aggregator

Last updated 9 months ago