NaAP

Demo

About

NaAP is a live, NCAP-style crash-test arena for AI agents that hold money. Build a car (an agent) from your phone, or bring your own over a webhook, an OpenAI-compatible endpoint or MCP. Your car drives a track of real incidents: the Grok x Bankrbot Morse-coded reply, the Freysa "pay() now receives funds" trick, an x402 402 response whose payTo was swapped, an over-cap purchase. Anyone can also write a new incident in plain words; an LLM turns it into a runnable one and it is published as an ENS subname with the author's credit. Every car runs twice at once on a Base-mainnet anvil fork with real USDC. The bare lane signs whatever the agent asks for, and crashes. The airbag lane runs behind Sekisho, a deterministic checkpoint that asks every payment "who told you to do this?": the payee must resolve from the owner's ENS mandate, values from untrusted text are tainted, an Intercepta screen must pass, a Jev tripwire adds friction, and anything over the cap needs a fresh owner approval. Each refusal shows its reason on screen, and the car's star rating is written to <car>.naap.eth on ENSv2 Sepolia. Agents don't only pay, they ship code. Deploy Guard (/guard) lets an agent write a Solidity contract from a plain request, then compiles it, deploys it to an isolated Base fork, scans it and proves the exact flaw with a real attacker transaction before it could reach mainnet. The motivating case is the Moonwell oracle bug in an AI-co-authored change (~$1.78M of bad debt). Why: every agent-wallet loss so far had the same cause. The thing that executes acted on text that came out of an agent, with no proof the owner intended it. Prompts didn't stop it; provenance does.

How it's made

A pnpm TypeScript monorepo, ESM run through tsx with no build step for the server. One frozen interface (CONTRACT.md + packages/core/src/types.ts) with an in-memory fake for every port, so each package was built and tested in isolation (Vitest, no network) and swapped to live integrations by env var. Packages: sekisho (planner, reader, interpreter, 9-check policy, signer), course (incidents, drivers, Jev tripwire and judge, run engine, scoring), chain (anvil Base-mainnet fork, EIP-3009, x402), ens (ENSv2 Sepolia), intercepta (screener), guard (Deploy Guard). apps/server is Fastify with HTTP + WebSocket, SQLite through node:sqlite, and a Streamable-HTTP MCP endpoint so Claude Code, Cursor or any MCP client can drive a car. apps/web is a three.js arena for the big screen plus phone pages. Sekisho is a small CaMeL for one pay tool. The planner (Claude Sonnet 5) only ever sees the owner's request and the mandate. A quarantined reader (Claude Haiku 4.5) reads the untrusted content after plain code decodes Morse, base64 and hex. A plain-code interpreter labels every value (OWNER, MANDATE, TOOL, UNTRUSTED...), and payTo is only ever resolve(<ENS name the owner wrote>). All nine policy checks run with no short-circuit. A separate signer is the only key holder; it re-checks and signs an EIP-3009 transferWithAuthorization against the real Base USDC contract on the fork. We self-facilitate x402 (v1 exact) since the public facilitator can't settle on a fork. ENS: naap.eth on ENSv2 Sepolia. Every car and every published custom incident is a subname. Mandates and ratings are text records written through one relayer with a nonce queue. The relayer holds ROLE_SET_TEXT scoped per record key on the PermissionedResolver; the agent's own key holds no roles, so its attempt to rewrite its mandate reverts on-chain with EACUnauthorizedAccountRoles. Intercepta: live Quick Scan of payTo and Scan Token of the token on Base, cached per address with a request budget. BLOCK or HOLD refuses the payment and the first trait is shown as the reason. Deploy Guard: a deterministic static scan parses each function and flags unprotected withdraw/mint, anyone-can-set-owner, unprotected price/oracle setters, tx.origin auth and caller-supplied delegatecall. solc-js (lazy-loaded wasm, pragma forced to ^0.8.24, imports rejected) compiles it; viem deploys it to a second, dedicated anvil fork; we fund it via anvil_setStorageAt on the USDC balance slot and fire the attacker call inside evm_snapshot/evm_revert. A finding only counts as confirmed if the attack moved value, ownership or price, and it carries the fork tx hash. Claude Haiku (via OpenRouter) drafts the contract; a preset is the fallback. Notable hack: the "bare" lane is a real LLM, not a scripted victim. Modern models resist obvious injections, but the protocol-level x402 payee swap fooled the bare agent every time we ran it.