Learn · Supercomputing
HPC on VirtEngine: SLURM Jobs On-Chain
HPC is the marketplace's most distinctive capability: batch supercomputing capacity, offered and settled on-chain, executing through the schedulers clusters already run. No re-platforming, no container shim around a batch queue — a native job model.
Why HPC needs its own module
Supercomputing does not sell like cloud: work arrives as jobs with resource requirements, walltime limits, and partition targets, scheduled by systems like SLURM against allocation policies. Forcing that model into long-running container leases loses everything that makes HPC HPC.
The x/hpc module models batch work natively: on-chain HPC jobs are first-class marketplace objects with their own lifecycle, offered and priced through the same exchange economics as everything else.
Scheduler adapters, not replacements
The provider daemon's HPC integration connects to existing cluster controllers through native adapters — SLURM with munge or JWT authentication and per-partition configuration, plus MOAB and Open OnDemand. The cluster's scheduler remains in charge of its own resources; the marketplace becomes another source of authorized work.
Job lifecycle, engineered for failure
A job lifecycle service polls on-chain jobs, dispatches them through the adapter, and tracks them to completion with configurable concurrency limits and timeouts. State recovery is crash-safe: a daemon restart does not orphan running jobs or double-dispatch queued ones. Job events, security events, and usage reports flow to a dedicated audit log.
Settlement identical to cloud workloads
HPC usage batches enter the same pipeline as container leases: signed records, the 24-hour dispute window, escrow drawdown, the governed take. For facilities, that means spare-cycle monetization lands as settled chain payments — no new billing infrastructure, no invoicing external users.
Who this serves
Facilities monetize idle cycles at prices they set per partition, without disturbing primary-user allocations. Tenants — ML teams above all — get access to scheduler-class capacity with real interconnects, through an order they can post today instead of an allocation process measured in months. The operational detail lives in docs/hpc-provider-operations.md, docs/hpc-node-agent.md, and docs/hpc-workload-publishing.md in the protocol repo.
Asked about hpc on virtengine
Do we replace our scheduler?
No. Adapters connect the provider daemon to your existing controller; the scheduler keeps its resources and the marketplace becomes another source of authorized work.
What survives a daemon crash?
Running jobs are not orphaned and queued jobs are not double-dispatched — state recovery is crash-safe by design, with concurrency limits and timeouts.
How are partitions exposed to the market?
Through per-partition configuration: facilities choose which partitions and job classes the market may schedule into, without disturbing primary-user allocations.