GameCraft AI

Every control in the HUD has a ? tip with a short explanation and a link here. This page is the long form: what each piece is, how it works on the fleet, and the steps to use it.

Studio HUD

#workspaceCo-Design Room

Chat with Pixel in free form. Ideas you type (or speak) update your living GDD and earn XP.

The Co-Design Room is the primary hearth. Pixel is a warm game-design co-pilot: you brainstorm in plain language; the Invisible Editor (Scribe) quietly extracts structured fields into your Game Design Document.

How to use

Type an idea or use the mic button
Watch Pixel reply and system notes appear
Genre, setting, loop, and mechanics fill in over time
Contradictions may summon a Plot Hole boss

#dualDual-Pane Hearth

Split view: Socratic chat on the left, live GDD canvas on the right (WebSocket-synced).

Dual-Pane runs the multi-agent matrix: Pixel streams conversational tokens while Scribe patches the GDD in the background. The right pane listens on a WebSocket for live GDD updates.

How to use

Open Dual-Pane Hearth from the sidebar
Chat or use the mic in the left companion
Watch the Living GDD Canvas flash when Scribe patches fields
On mobile, switch Chat / GDD Tome tabs

#voiceVoice Console

Hands-free design: Lemonade Whisper turns speech into text; Kokoro speaks Pixel’s replies.

Voice Console records from your mic, sends audio to the FastAPI gateway → Lemonade Whisper STT, runs a Socratic turn with Pixel, then plays Kokoro TTS (or browser fallback).

How to use

Allow microphone access when prompted
Tap the mic, speak, tap again to stop
Review transcript and Pixel’s spoken reply

#gddGDD Architect Canvas

Your living Game Design Document — genre, setting, core loop, mechanics, characters.

The GDD is the structured canon of your game. Fields start undefined; talking with Pixel and completing quests fills them. Server-side Scribe extraction and local NLP both contribute.

How to use

Design in Co-Design or Dual-Pane
Open this tab to review structured fields
Return to Pixel to fill gaps

#cozy-micCozy Mic Button

Hold-style record control with live volume ring; uploads WebM to Whisper.

Uses MediaRecorder + Web Audio analyser for amplitude. On stop, parent handles STT + chat pipeline.

AI agents

#pixelPixel (Conversationalist)

Your warm co-designer. Celebrates ideas, asks gentle questions, never blank-pages you.

Pixel is the conversational layer of the dual-agent engine. Replies come from the fleet LLM (LiteLLM / Lemonade). Offline, a friendly local fallback still keeps the room feeling alive.

How to use

Talk naturally about worlds, feelings, and player verbs
Answer Pixel’s questions one step at a time

#scribeScribe / Invisible Editor

Background agent that extracts GDD fields from chat without interrupting the conversation.

After each user message, Scribe analyzes text for genre, setting, coreLoop, victoryCond, mechanics, and characters, then patches project canon and can broadcast via WebSocket.

#matrixMulti-Agent Matrix

Runs Pixel (streamed) and Scribe (background) concurrently — chat stays warm while the GDD updates.

POST /gateway/matrix/prompt-concurrent returns an SSE token stream from Pixel while a background task extracts GDD deltas and notifies Live GDD clients.

#socraticSocratic Companion

Guided design dialogue with optional speech synthesis — used in Dual-Pane and voice flows.

Socratic turns hit /gateway/socratic/turn: Pixel reply + optional Kokoro audio. Dual-Pane prefers the matrix stream when available.

#live-gddLiving GDD Canvas

Real-time GDD view. Connects via WebSocket; flashes when Scribe patches fields.

Subscribes to /gateway/ws/project/{id}. Events like gdd_updated refresh genre, setting, loop, lore, and mechanics without a full page reload.

Asset pipelines

#assetsAsset Synthesizer

Generate concept art (ComfyUI/Flux on st.auta), 3D blockouts (Blender), and keep a vault of results.

Three layers: (1) free Quick Concept controls, (2) ComfyUI Sprite Synthesizer locked until Level 5, (3) 3D Blender Sandbox locked until Level 10. Outputs land in the vault and Adventurer Backpack.

How to use

Name an asset, pick type, write a visual prompt → Synthesize
At Level 5+, use ComfyUI chamber for sprites
At Level 10+, generate sword/shield/tower meshes
Check Backpack to equip showcase items

#comfyComfyUI Sprite Synthesizer

GPU image generation on st.auta (Flux). Unlocks at Level 5.

Prompts are sent through the FastAPI gateway to ComfyUI. Results save under storage, register as inventory items, and award XP. Below Level 5 the chamber shows a lock overlay (RpgFeatureGate).

How to use

Reach Level 5 by designing with Pixel and completing quests
Enter a cozy pixel-art style prompt
Generate Sprite — wait for st.auta render

#blender3D Blender Sandbox

Procedural low-poly blockouts (sword / shield / tower) via headless Blender on st.auta. Level 10.

Gateway calls the Blender worker; GLB/mesh URLs open in a Three.js viewer. Also queues island-style jobs when using the blender queue API.

How to use

Reach Level 10
Pick mesh type and complexity
Generate and orbit the preview

#quick-conceptQuick Concept Controls

Anytime concept generation — no level gate. Uses Comfy when online, then falls back.

Name + type + prompt → /gateway/assets/concept or Next.js asset API. Good for placeholders before you unlock the full Comfy chamber.

RPG meta-game

#inventoryAdventurer Backpack

Nine inventory slots for generated assets. Drag or click to equip into the Active Showcase.

Generated 2D/3D assets are registered server-side (rarity scales with level). Equip items for the showcase chamber; equipping awards a small XP tick.

How to use

Generate assets first (Assets tab)
Open Backpack → drag item or click to equip
See prompt/rarity in Active Showcase

#questsDesign Quests Journal

Micro-tasks that teach the design loop and award XP (genre, setting, loop, mechanics, assets).

Local quests complete when GDD fields fill. Gateway Quest Board can regenerate LLM-driven quests from your GDD and complete them for progression XP.

How to use

Design with Pixel to auto-complete field quests
Use Regenerate on the gateway board for fresh goals
Claim +XP on board quests when ready

#equipActive Showcase / Equip

Drop zone for backpack items — pin a hero asset while you keep designing.

Equipping POSTs to /gateway/inventory/equip and awards a small XP tick. Showcase shows thumbnail, prompt, and rarity.

Related: Adventurer Backpack

#progressionXP & Levels

Level N needs N×300 XP. Level-ups unlock Comfy (5) and Blender (10) chambers.

XP comes from chat, quests, assets, boss victories, and equipping. Profile is stored by the gateway progression service; the character sheet mirrors level and achievements.

How to use

Chat and complete design micro-tasks
Watch the sidebar XP bar
Level 5 → Comfy; Level 10 → Blender

#level-gatesFeature Level Gates

Creative chambers lock until you reach the required architect level.

UI uses RpgFeatureGate overlays; API hard-gates sprite (5) and 3D model (10) routes with 403 feature_locked. Keep designing to unlock.

#achievementsAchievements

Badges for milestones (e.g. Grand Architect at Level 5).

Evaluated when XP is awarded. Shown on the RpgCharacterSheet under Achievements.

Related: XP & Levels

#bossPlot Hole Defense (Boss)

When your design contradicts itself, a Lore Guardian boss appears — write a lore fix to win +300 XP.

Boss-check runs after chat (heuristics + optional LLM scan). The modal asks for a logical resolution; /gateway/boss/resolve-battle judges the solution and patches lore on victory.

How to use

Design freely — paradoxes get detected
Read the paradox description
Write a multi-sentence mechanical/lore fix
Unleash Lore Correction — or Flee

#character-sheetRPG Character Sheet

Sidebar card: level, XP bar, and achievements from the progression API.

Fetches /gateway/progression/profile/{userId}. Plays a fanfare when level increases between refreshes.

Related: XP & Levels · Achievements

#xp-barXP Progress Bar

Progress toward the next architect level (threshold = level × 300).

Updates when you chat, finish quests, generate assets, defeat bosses, or equip items. Level-up banner + chime fire on threshold cross.

Related: XP & Levels

Fleet services

#fleetFleet AI Services

Lemonade (STT/TTS) on stak; ComfyUI + Blender on st.auta; LLM via LiteLLM/Lemonade.

The Next.js app proxies /gateway/* to FastAPI on ardra. Health at /gateway/health reports comfyui, blender, whisper, tts, pixel_llm. Placeholders keep the HUD usable if a worker is down.

#sttSpeech-to-Text (Whisper)

Microphone audio → Lemonade Whisper-Tiny transcript via /gateway/voice/transcribe.

Used by Voice Console, CozyMic, and useVoiceEngine. Browser SpeechRecognition is a fallback on the classic Co-Design mic.

Related: Voice Console · Cozy Mic Button

#ttsText-to-Speech (Kokoro)

Pixel’s replies can be spoken via Lemonade Kokoro (/gateway/voice/tts).

Also exposed as /voice-stream/speak for direct audio responses. Soft failure if TTS is offline — text still appears.

#co-pilot-coreCo-Pilot Core Status

Shows whether Pixel+Scribe are bound to a cloud project or running local-only.

After login, bootstrap creates/loads a studio project and GDD. Offline mode still allows local chat and mock assets.

#gatewayAPI Gateway

Browser calls /gateway/* which rewrites to the FastAPI orchestration engine on port 8000.

See live OpenAPI docs at /gateway/docs when the backend is up. Modules include chat, assets, voice, quests, boss, progression, inventory, matrix, blender, hardware.

Related: Fleet AI Services

Auth & studio

#authSign-in (Zitadel)

Social login via Zitadel (Google/GitHub). Studio data binds to your account after auth.

Auth.js sessions protect /dashboard. Bootstrap POST creates a multi-tenant project row in Postgres (Prisma) while the FastAPI gateway keeps file-backed RPG/inventory state.

Related: Co-Pilot Core Status

Gateway health: /gateway/health · OpenAPI: /gateway/docs