EigenTensor Denver

Demo

About

The problem it solves building on-chain applications that use the GPU is complex. often, you're reliant on tools written by other people that limit what you can achieve using the GPU, since allowing developers to directly access the GPU on-chain is dangerous from a security perspective. we solved this by developing a platform-agnostic, memory safe way to execute arbitrary work on GPUs, which runs on an EigenLayer AVS. best of all, it integrates tightly with the popular ML library "tinygrad", which thus means we support the ability to cut-up GPU work into chunks automatically and split it between nodes using the Exo library. User Interaction and Data Flow DEMO (Local): 1. Run MNIST Demo From Subfolder In Repo, 2. Run NextJS Client. 3. Your Demo Is Running On Port 3000. DEPLOYMENT (AVS): 1. Run Othentic-CLI within the Main AVS Repo. 2. Install Package From EigenTensor Library. 3. `anytensor-server` The project architecture and development process 1. Othentic CLI. 2. We Hacked Internal Parts of Tinygrad We had to read through and test the entirety of the Tinygrad library to discover these undocumented functionalities. # Architecture & Development EigenTensor enables verifiable GPU computation in a decentralized context through a layered architecture: ```` ┌───────────────────┐ ┌────────────────────┐ ┌───────────────────┐ │ Client Library │──────▶ EigenTensor AVS │──────▶ Execution Node │ │ (TensorContext) │ │ (Othentic SDK) │ │ (GPU Runner) │ └───────────────────┘ └────────────────────┘ └───────────────────┘ │ ▼ ┌───────────────────┐ │ Consensus Layer │ │ (Result Verify) │ └───────────────────┘ ```` ## Core Components 1. **TensorContext API**: User interface for defining computational graphs 2. **Graph Compiler**: Converts tinygrad ops into verifiable format using BUFFER UOp 3. **AVS Layer**: Connects to EigenLayer via Othentic SDK for task distribution 4. **Execution Engine**: Runs computations on GPU nodes 5. **Consensus Mechanism**: Verifies outputs using Manhattan distance 6. **Demo**: MNIST classifier with Next.js frontend ## Development Process 1. **Discovery**: Reverse-engineered tinygrad's internals to enable placeholder tensors 2. **Core Library**: Built TensorContext API and graph serialization 3. **Testing**: Validated with MNIST model 4. **Integration**: Connected to EigenLayer via Othentic SDK 5. **Frontend**: Created demo interface Product Integrations Othentic/EigenLayer Key differentiators and uniqueness of the project EigenTensor offers significant advantages over alternatives: | Feature | EigenTensor | Kite | Autonome | Hyperbolic | |---------|-------------|------|----------|------------| | Full GPU Capability | ✅ | ❌ | ❌ | ❌ | | Memory Safety | ✅ | ✅ | ✅ | ✅ | | Multi-GPU Support | ✅ | ❌ | ❌ | ❌ | | Familiar ML API | ✅ | ❌ | ❌ | ❌ | | Consensus Verification | ✅ | ✅ | ✅ | ✅ | | No Custom Hardware | ✅ | ✅ | ❌ | ✅ | Trade-offs and shortcuts while building 1. Using Tinygrad Over Onnx (tinygrad supports inbuilt splitting, has better optimizations, etc, but was hard to hack on). 2. Chosing Othentic over Tangle (better documented). 3. Using simple MNIST demo rather than demoing an LLM, which should work equally well. We had to read through and test the entirety of the Tinygrad library to discover these undocumented functionalities: The core innovation of EigenTensor came from discovering and exploiting a subtle implementation detail in TinyGrad's tensor representation system. TinyGrad operates by building computational graphs that represent operations on tensors, rather than executing operations immediately. When you add two tensors, TinyGrad creates a node in this graph connecting them with an ADD operation. The actual GPU computation only happens when you call .realize() on a tensor. Our challenge was creating a system where computational graphs could be defined without specifying all input data upfront. We needed "placeholder" tensors that could be substituted with real data at execution time. This would allow us to share executable graphs that nodes could run with their own inputs. The solution involved modifying how TinyGrad's BUFFER operation works. Normally, a BUFFER operation takes a two-element tuple that encodes tensor information like shape and size. We discovered we could extend this tuple with a third element containing a special placeholder label string, without breaking TinyGrad's internal operations. This allowed us to substitute input values into the computational graph by specifying the placeholder label. This hack was necessary because TinyGrad's official metadata systems (like VOID patterns) were incompatible with graph composition operations - using them would have made it impossible to combine graphs involving placeholder tensors. Our approach was the only viable method we found after extensive testing. With this technique, we could create computational graphs that referenced input tensors. Additional Features Supports Exo Inference library