Opti-om

Demo

About

Our project is a Web3 marketplace for GPU kernel optimizations that help locally hosted AI models generate more tokens per minute on existing hardware. It connects developers who create optimizations, community verifiers who test them, and users who want faster local AI without upgrading their GPUs. Running AI locally gives users more control over their models, data, and operating costs. However, configuring and optimizing those models requires technical expertise. Our goal is to make useful performance improvements easier to discover, evaluate, and install, while rewarding the developers who produce them. GPU kernels are the programs that execute operations such as matrix multiplication and attention on a GPU. Tuners submit optimized implementations, together with the model, GPU configuration, software requirements, and integration needed to use them. Our success metric is an improvement in actual model generation throughput—not simply a faster isolated kernel. The proposed publishing flow offers two routes. Tuners who verify through World ID can submit without a publishing deposit, subject to per-person limits. Other tuners deposit SUI into escrow before submitting. Both routes create a pending submission that must undergo performance verification. Human verification does not automatically make an optimization trustworthy. The platform assigns submissions to eligible community verifiers with compatible GPUs. Verifiers install a local benchmark harness that checks compatibility and runs the baseline and optimized implementations on their own hardware. It checks numerical correctness, measures generated tokens per minute, and repeats tests using controlled settings. Each report records the tested configuration and package version, so results can be tied to a specific optimization. The platform compares these reports to determine whether an improvement is repeatable across participating machines. Accepted packages display their measured performance, compatibility requirements, and verification evidence. Verifier reputation is tracked separately from package performance. This is community-based verification: multiple independent tests strengthen confidence, but World ID and blockchain records alone do not prove that a GPU measurement is truthful. World provides the human identity layer for publishing allowances and verifier participation. Sui provides the payment and coordination layer for submission deposits, purchase records, and programmable distribution of revenue to contributors. GPU computation runs locally and off-chain, while package identifiers, result references, and payment records can be recorded on-chain. Users browse verified packages for their model and hardware, pay once through Sui, and download an installable optimization. The purchase can include official updates, compatibility tools, and support. Local benchmarking lets users confirm the benefit on their own machine and return to the baseline if necessary. Purchase-gated distribution can control official access, although downloaded software cannot be made completely immune to copying. Our long-term vision is a community-driven ecosystem where improvements to local AI become reusable products: tuners earn from useful work, verifiers contribute reproducible evidence, and users get more performance from the hardware they already own.

How it's made

Our architecture connects a Web3 marketplace to a local GPU benchmarking harness. The web application coordinates submissions, verification jobs, results, and purchases, while the computationally expensive work runs off-chain on participants’ hardware. The optimization layer targets GPU kernels used during PyTorch model inference. Each submission pairs an optimization with its supported model, GPU requirements, software configuration, and integration instructions. The local Python harness compares the baseline and optimized implementations on the same machine. It checks numerical correctness and measures actual text-generation throughput in tokens per minute using repeated, warmed-up runs with consistent prompts and generation settings. The key engineering decision is to compare relative improvements on each verifier’s machine rather than compare raw speeds between different laptops. Two laptops with the same GPU model can behave differently because of power limits, cooling, and background activity. Running both implementations on each machine makes the comparison more useful. Reports include configuration details and package hashes so that results refer to a specific version of the optimization. World ID provides the human verification layer. It supports publishing allowances for verified participants and helps limit duplicate identities in the verifier pool. Identity verification remains separate from performance verification: the local harness measures the optimization, while World establishes the participant’s eligibility. Sui provides the settlement layer for publishing deposits, escrow, purchase records, and distribution of payments to tuners. Keeping payment logic on-chain makes transactions auditable, while keeping model execution off-chain allows users to benchmark and run optimizations on their own GPUs. The backend connects these components by assigning compatible verification jobs, collecting benchmark reports, updating submission status, and checking purchase eligibility before granting official package access. Buyers pay once for an optimization package they can install locally. The most distinctive part is using participants’ own machines as a distributed benchmark network. Instead of treating a tuner’s advertised speedup as sufficient evidence, the architecture requires other participants to reproduce the improvement. This produces community-backed performance evidence while acknowledging that reports from user-controlled hardware are not cryptographic proofs of honest execution.