WorldMod

Demo

About

Go in as much detail as you can about what this project is. Please be as clear as possible! (min 280 characters) World Mod is a permissionless network for physical-world data. Anyone with a phone opens a web page, does a short physical task while the camera watches, and gets paid in USDC once the recording is accepted. A buyer posts what they need and escrows what it pays. Nothing to install, no hardware to buy, no gas to hold. **Why this exists.** Robots cannot learn from the internet the way language models did. They need first-person video of humans handling objects, and the datasets that exist today were collected by expensive, centralised research programs with recruited participants and custom hardware. The scarce resource is not storage or compute. It is someone doing a physical thing while a sensor is pointed at it. World Mod makes that contribution nearly frictionless and pays the person who made it. **How an episode earns**  1. **A buyer posts a bounty** ("15 seconds of keyboard typing, hands in frame") and escrows the full budget in real USDC. A bounty is only listed once it is funded. 2. **A contributor opens the app inside World App.** Their World wallet is their identity, and a one-time Selfie Check keeps one person from farming payouts across many devices. 3. **They record a 15-second task** on their phone. A head strap gives the best first-person footage but is optional; holding the phone works too. The app shows a live hand skeleton so they can see they are in frame. 4. **The recording is sealed on the device** before a single byte leaves it: hashed, signed with the contributor's key, and queued so going offline mid-task loses nothing. 5. **The server scores every frame.** Are the hands in frame? Does the camera motion agree with the gyroscope (a screen replay fails this)? Is it a near-duplicate of something already submitted? We measured the replay check against real recordings: genuine takes correlate at 59% on average, replays at 24%. 6. **The buyer's acceptance threshold is checked inside a TEE** by a Chainlink CRE Confidential Workflow, so the bar a buyer sets is never visible on-chain or to the operator. A pass is written on-chain as a DON-signed report. A rejection makes no on-chain call at all. 7. **The contributor is paid from escrow** in real USDC, into a balance only they can withdraw. They never need gas. **What accepted episodes become.** Episodes are bundled into a licensed dataset with a real IPFS content address and an on-chain registry entry. A buyer can tokenise that dataset as a KYC-gated Bond on Hedera, mint licence seats to the dataset's creator, and pay licence-fee coupons in USDC. Meanwhile a world model trains over the episodes and runs live during the next take, showing the wearer what it thinks happens next. A second Confidential Workflow settles each contributor's share of model improvement on-chain. **What is real today.** All six registry and escrow contracts are deployed on both Ethereum Sepolia and Hedera testnet, using Circle's testnet USDC on both, not a mock token. Real episodes from three contributors have been captured, scored, anchored, paid and tokenised. A real Selfie Check has completed end to end against World's sandbox. **What we say plainly.** A browser has no secure enclave, so every episode carries a "heuristic" trust level in the UI: the checks measure whether a capture is plausible, not that pixels came from a real camera. With five episodes the world model does not yet beat a "predict no change" baseline, and the buyer dashboard shows that curve rather than hiding it. We would rather ship a real pipeline that improves with every episode than a mock that looks finished.

How it's made

 **The phone.** A Next.js PWA, installable on iOS and Android. Capture is getUserMedia plus MediaRecorder with per-browser codec negotiation recorded into the manifest, DeviceMotionEvent for 60Hz IMU, and TensorFlow.js hand-pose detection with self-hosted 4MB lite models for the live skeleton. The manifest is canonicalised with RFC 8785 and the IMU is hashed as a binary stream, because float formatting across two runtimes is where byte-exactness dies. Scoring used to run on the phone too. We measured it dropping capture to 26.9fps with 818ms gaps during movement, so it moved to the server; a client-computed score was never authoritative anyway. **The server.** ffmpeg decodes every frame. Optical flow is correlated against the gyroscope with lag estimation, because the frame and IMU clocks are offset and a zero-lag correlation scored a genuine capture below a spoof. Fixing that took one episode from 5.2% to 59.6%. Perceptual hashing catches duplicates. Episode bytes and dataset metadata are pinned to IPFS through a local kubo node. **Contracts.** Foundry. EntityRegistry, AssetRegistry, EpisodeRegistry, BountyEscrow, DatasetRegistry and FederatedRound, plus a native WMOD token whose one shipped function is a ValidatorBond that stakes and slashes validators. Contributors sign EIP-712 payloads that a relayer submits, so nobody needs gas. Escrow credits a pull-based balance so nobody else can redirect a payout. The same contracts are deployed on Ethereum Sepolia and Hedera testnet, switched by one environment variable. **Chainlink CRE.** Two Confidential Workflows, both registered with cre.handlerInTee. The episode validator fetches the buyer's minimum framing and plausibility thresholds with runtime.getSecrets so they are only ever decrypted inside the enclave, pulls scored episodes over HTTPClient, compares in-enclave, and on a pass calls donRuntime.report then evmClient.writeReport into a consumer contract registered as an EpisodeRegistry validator. The utility oracle holds the utility weighting exponent as a secret, computes normalised basis-point shares in-enclave, and the DON co-signs them into a second consumer that is now BountyEscrow's registered oracle. Both consumers follow Chainlink's ReceiverTemplate and accept the production Sepolia KeystoneForwarder. This replaced the product's biggest trust assumption: one EOA acting as validator and oracle. **Hedera.** The core contracts call the HTS precompile in their constructors to self-associate with Circle's HTS USDC. Forge's local EVM has no HTS precompile, so those three had to be deployed via raw viem over Hashio instead of forge script. On top, a licensed dataset becomes an Asset Tokenization Studio Bond: the app reads the DatasetRegistry live, maps it through a tested dataset-to-bond mapping with a valid ISIN, issues via Bond.create, grants KYC with a real Terminal3 Verifiable Credential, mints licence seats to the dataset creator, sets a 5% coupon, and distributes it in real USDC. Four Bonds are source-verified on Sourcify for chain 296. **World.** The app is a MiniKit mini app. MiniKit.walletAuth gives contributors a recoverable identity, and MiniKit.signTypedData signs the same EIP-712 payloads the previous Privy signer did, a transport swap rather than a redesign. Selfie Check uses IDKitRequestWidget with the selfieCheckLegacy preset and a server-signed RP context. The verify route calls World's production v4 verify API and only records a verification when World says success. **World model.** A PyTorch encoder plus GRU dynamics head, exported to ONNX and checked against the PyTorch graph over an eight-step recurrent sequence (worst-case difference 2.5e-5). During capture the phone streams a cropped frame and motion sample twice a second, the server keeps the GRU hidden state per session, and returns the nearest-neighbour frame from the same take. **Hacky things worth mentioning.** The Hedera ATS SDK has no raw-key signing path; its public wallet list is MetaMask, WalletConnect and enterprise custody only. We wired an ethers.Wallet directly onto its transaction adapter and stubbed the hard global.window gate. That stub then broke Next.js server rendering across the whole app, because React checks typeof window in its own internals, so ATS issuance now runs in a short-lived child process that starts, issues one Bond, prints one line of JSON, and exits. Granting KYC and setting coupons both revert on a fresh Bond because the diamond owner does not receive _KYC_ROLE or _CORPORATEACTIONS_ROLE automatically; we found that by decoding reverts on the mirror node, and in the process discovered our own account-id and private-key pair had been mismatched from day one. Nobody had verified the ATS ResolverProxy on Hedera testnet before, so we recovered the compiler version from the on-chain metadata CBOR and rebuilt 153 source files to a byte-exact match.