7-2. Entire Process

The EMETH architecture is designed to enable the decentralization of processes beyond language model training. The processing is performed through the following steps:

1. Job registration by the requester

   • The requester registers the parent Job information on the Blockchain and pays EMT tokens equal to feeLimit (= fuelLimit * fuelPrice) + VERIFIER_FEE.

   • The dataset associated with the Job is stored in S3 (Stack).

   • If the processing has not started, the requester can cancel the request by sending a cancel statement to the Blockchain. In this case, the paid feeLimit + VERIFIER_FEE is fully refunded.

2. Continuous synchronization cache by Cache servers

   • Cache servers continuously synchronize and cache Task data on the Blockchain.

3. Execution of processing by the Orchestrator

   • The Orchestrator monitors the information from Cache servers and performs the following:

     • Calls the Splitter corresponding to the ProgramId to divide the dataset into NumParallel parts.

     • Divides the Job into multiple SubJobs, associates them with the parent Job, and registers them on the Blockchain. The ProgramId differs between the parent Job and SubJobs.

     • Schedules multiple epoch processing (Jobs) according to NumEpoch.

4. Continuous synchronization cache by Cache servers

   • Cache servers continuously synchronize and cache Job data on the Blockchain.

5. Execution of processing by Emeth Nodes

   • Emeth Nodes monitor the information from Cache servers and perform the following:

     • Look for Jobs with attractive feeLimit (= FuelLimit * FuelPrice) and Deadline.

     • Send a process statement to the Blockchain to express the intention to start processing and deposit EMT tokens equal to deposit (= feeLimit * DEPOSIT_RATE + VERIFIER_FEE).

     • Download the dataset for the relevant Job from S3 (Stack) and start processing corresponding to the ProgramId.

     • Store the processing results in S3 (Stack) and send a completion report (submit) to the Blockchain.

   • Emeth Nodes can also withdraw after expressing the intention to start processing. In this case, feeLimit * DECLINE_PENALTY_RATE is confiscated, and the confiscated EMT tokens are burned.

6. Processing of timed-out Jobs by Verifiers

   • Verifiers monitor the information from Cache servers and search for Jobs that have timed out (now > deadline).

   • Send a timeout processing (timeout) to the Blockchain. In this case, the Emeth Node that was processing the Job has feeLimit * TIMEOUT_PENALTY_RATE confiscated.

7. Verification of processing results by Verifiers

   • Verifiers verify the submitted Jobs.

   • Start the Verifier program corresponding to the ProgramId for verification. If the processing result passes the verification, a verify command is sent to the Blockchain; if it fails, a rejectResult command is sent.

8. Reprocessing and completion confirmation of SubJobs by the Orchestrator

   • The Orchestrator re-registers (reprocesses) Jobs that have been timeout, declined, or rejectResult.

   • Confirms that all SubJobs are completed and calls the Aggregator program corresponding to the ProgramId to submit the results as the parent Job's results.

In this way, the EMETH architecture is designed to efficiently manage decentralized processing and obtain highly reliable results.