AI3D Transcendence

Demo

About

The problem it solves Human lives are fleeting, and our digital footprints often disappear or become scattered over time. AI3D Transcendence solves the ephemeral nature of online identity by allowing individuals to preserve a living, interactive version of themselves—stored immutably on the Internet Computer (ICP) blockchain. By storing key background data about the subject and providing access to real-time search and news, we create a persona that can react to current events based upon prior knowledge and experiences. User Interaction and Data Flow Users begin by specifying whose identity or “footprint” they want to replicate. Our AI agent crawls publicly available data related to that individual (e.g., posts, articles, social media). Next, a Unity WebGL frontend generates a 3D avatar and clones the user’s voice. Users can then chat directly with this AI-driven persona in real time, simulating a conversation with that individual’s “digital essence.” All essential AI parameters, persona context, and references are anchored on-chain (ICP) to ensure secure, lasting storage. 1. Input: Username or identity. 2. Web Scraping: AI agent searches the internet (via DuckDuckGo) for relevant information. 3. AI Reconstruction: Large Language Models process this data to create a coherent persona. 4. 3D + Voice: Unity WebGL + voice cloning library generate an interactive, lifelike avatar. 5. On-Chain Storage: Key data, metadata, or references saved on ICP for permanence. 6. User Chat: Frontend streams real-time AI responses, voice, and animations. The project architecture and development process Backend: A FastAPI server hosting our custom AI agent. It uses OpenAI models, (GPT-4o) to synthesize personality, style, and knowledge from public data. AI Agent: Scrapes the web for new information (duckduckgo_search), then merges it into a persona context. It also provides short, direct responses in the “voice” of the impersonated user using Pydantic AI tools for LLM abstraction. Unity WebGL Frontend: Renders a 3D avatar (via faced.io + custom rigging) and streams AI responses. Voice cloning is handled through an external service or library, synchronizing mouth movement in Unity with the generated audio. On-Chain Storage: ICP canisters store critical persona data and hashed references to ensure the digital persona remains verifiable and tamper-proof. Development Flow: Initially built out the AI agent architecture in Python, integrated Unity for the interactive layer, and finally connected to ICP for permanent storage. Product Integrations OpenAI (GPT-4o): Generates text responses and persona synthesis. DDGS (DuckDuckGo Search): Retrieves public information and news to build or update the persona. Pydantic AI: Provides AI agent framework for tool calling, state, and LLM abstractions Unity WebGL: Frontend for 3D avatars and live conversation. ICP: Decentralized storage for user data, ensuring immutability. faced.io + sdxl-lightning: Assists in generating realistic 3D avatars and imagery from textual or photographic prompts. Voice Cloning: Synthesizes the person’s speech patterns. Key differentiators and uniqueness of the project Digital Immortality: Goes beyond typical LLM chatbots by fully recreating someone’s “presence” with a 3D avatar and cloned voice. On-Chain Persistence: Uses ICP to store data so this AI persona can live on indefinitely without fear of central server failures or censorship. Non-Invasive: No implants or direct brain-computer interfaces; it’s purely AI-based, gleaning traces from publicly available data. Personalized Experience: The user’s “footprint” is intimately reconstructed, offering a highly tailored conversational style and memory. Trade-offs and shortcuts while building Privacy vs. Public Data: We only scrape publicly available info to respect privacy, but this limits completeness if the user has minimal public activity. On-Chain Storage Costs: Storing large files (avatars, detailed voice prints) fully on-chain is expensive. We store references/hashes on ICP instead of raw data to reduce costs. Accuracy vs. Authenticity: The persona’s authenticity depends on the quality and quantity of online traces. We prioritized quick scraping and generation over thorough, manual curation. Future Improvements: Potentially refine how user data is filtered to avoid outdated or inaccurate info, and add advanced privacy controls or user-supplied data ingestion. Additional Features Voice Cloning: Implemented during the BUIDLathon to create more immersive conversations. 3D Avatar Upgrades: We integrated a new pipeline for dynamic facial expressions in Unity specifically for ETHDenver. ICP Integration: Enhanced on-chain metadata references to ensure verifiability and reduce data storage overhead.