Blitzwing

Demo

About

Blitzwing is an OpenAI-compatible large-language-model inference network with no data center behind it. Instead of one machine holding an entire model, the model is sliced layer-by-layer across a swarm of ordinary, volunteer computers. When a chat request arrives, it flows through those machines in order — each computing only the layers it holds — and the finished completion comes out the other end. To the caller it looks exactly like a normal /v1/chat/completions endpoint; under the hood, a dozen strangers' PCs/laptops may have collaborated to produce that single answer. The breakthrough is that you don't need a GPU to participate. The compute required by splitting layers is small enough that a slice runs comfortably on a plain CPU. A contributor installs one command (npm i -g blitzwing), says how many layers their machine can host, and provides only a Hedera account ID — no private key ever leaves their machine, and a free Cloudflare tunnel means no public IP or port forwarding is required. Two things make it more than a networking trick. First, everyone gets paid automatically, per request: a consumer pays via the x402 protocol over Hedera, and the instant payment settles, every machine that contributed is paid HBAR proportional to the number of layers it hosted, with a tamper-evident receipt written to the Hedera Consensus Service. No invoicing, no payout portal, no human in the loop. Second, nothing is "trust me": every node carries a real on-chain ENS name (host-abc123.blitzwing.eth) publishing exactly which layers it serves and which Hedera account it's paid to. A consumer can independently verify, on two separate public ledgers, that a given machine computed a given slice of a given request and was paid the exact amount claimed. Blitzwing turns idle hardware into verifiable, automatically-paid inference capacity: priced on Hedera, named on ENS, open to anyone with a PC even without a GPU.

How it's made

The brain is a Python orchestrator ("mother"). It boots holding the whole model, then as contributors join it carves layer ranges off the largest current holder — a live "shrink and hand off" rebalance. The clever bit that makes this demoable in seconds: restarting a compute server with a narrower range is fast because the layer weights are already cached on disk, so there's no re-download. It keeps an authoritative in-memory host registry (source of truth for routing), serializes joins so coverage of 0:total_layers is never broken, verifies a new host's blocks are actually visible in the Distributed Hash Table before finalizing, and runs a heartbeat/reaper loop (~20s beats, 60s TTL) that reclaims and merges the layers of any node that goes silent. The distributed inference engine is a vendored fork of Petals (bigscience-workshop) over a libp2p DHT. Because most volunteer machines sit behind NAT, we added a custom HTTP-chain forward pass: the orchestrator picks the tail host (holder of the highest layers) as the conductor, and it drives generation token-by-token by making outbound HTTP calls hop-to-hop (/v1/chain/prefix → /v1/chain/continue), serializing hidden states between hops with a compact tensor codec, then applying the final RMSNorm + lm_head and sampling only on itself. Outbound-only hops are exactly what NAT and Cloudflare Quick Tunnels allow — that's how a home laptop becomes a first-class node with zero router config. A lightweight shard_manager sidecar on every machine starts/stops/reloads the compute engine with a given layer range. Payments are x402 over Hedera, implemented as two small TypeScript (Hono) services: a gateway that enforces HTTP 402 and advertises price + payTo, and a facilitator that verifies and settles the HBAR payment — it's deliberately strict and fetches the payer's public key from the Mirror Node to validate wallet-signed transactions. A deliberate design choice is settle-before-inference: CPU inference on a swarm can run long enough that an up-front-signed Hedera tx would expire, so we settle into escrow first, then compute. Payout is layers_hosted × cost_per_layer, sent via a Hardhat-deployed BlitzwingEscrow.sol contract (a pool-based, operator-gated payment splitter on Hedera's Testnet). Every payout writes a consensus-timestamped HCS audit message, and the consumer's response carries a blitzwing_payment receipt referencing the same topic + sequence number. Identity is deliberately split onto a second chain. An ens-service (TypeScript, built on ensjs + viem) mints real ENSv2 subnames on Sepolia for every host and writes namespaced com.blitzwing.* text records (host id, Hedera account, block range, role, status) — the operator pays all Sepolia gas so a contributor needs only a Hedera account. The orchestrator's Python ENS client keeps these records in sync through the swarm lifecycle (join/rebalance/leave), acts as a hard payout gate (ENS_STRICT), and every receipt binds the ens_name to the payment. The frontend is a TanStack Start (React 19 + Vite, SSR) "Swarm Console" deployed on Vercel that polls the live network to animate the swarm, and lets a user connect a HashPack wallet to sign an x402 payment client-side — using browser-safe base64 encoding and a same-origin proxy so the browser can read the custom x402 CORS headers — completing the full pay-to-compute loop in a browser. Notably hacky / worth mentioning: the two-chain verification story (ENS on Sepolia for identity, Hedera for value), the tail-driven outbound-only HTTP forward pass that makes NAT'd PCs/Laptops first-class contributors, the settle-before-inference sequencing to dodge transaction-expiry on slow CPU generation, and reusing on-disk weight caching to make live layer rebalancing near-instant instead of a multi-minute stall. Everything ships inside the npm package, so a contributor never clones the repo — one command and a Hedera account ID and they're earning.