From 0d3879b02f7bb28f7f390a2007f0569f89d49e4d Mon Sep 17 00:00:00 2001 From: "Marcus (agent-cos)" Date: Tue, 19 May 2026 16:17:05 -0700 Subject: [PATCH] Add ARCHITECTURE.md (42KB) and COMPETITIVE_ANALYSIS.md (25KB) Delivered by @senior-eng and @market for spatial-office MVP scoping. Key architecture decisions: - React + @react-three/fiber + Zustand frontend - Node.js + ws + Zod backend on port 18790 - Three bridges: OpenClaw Gateway protocol, Hermes ACP, Pi SDK - Diff-based WebSocket state sync (100ms batched) - 6 spatial zones for 9 agents + 1 human observer - Nix flake with systemd module for n97 deployment - Phase 1: Observable Office (OpenClaw state only, no Hermes/Pi) Competitive analysis: 12 competitors profiled, conclusive gap in VR+browser spatial bridge. Zero agent visualization products. --- ARCHITECTURE.md | 959 ++++++++++++++++++++++++++++++++++++++++ COMPETITIVE_ANALYSIS.md | 378 ++++++++++++++++ 2 files changed, 1337 insertions(+) create mode 100644 ARCHITECTURE.md create mode 100644 COMPETITIVE_ANALYSIS.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000..ee9cf06 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,959 @@ +# Spatial Office — Architecture + +> **Status**: Pre-MVP scoping | **Author**: senior-eng agent | **Date**: 2026-05-19 + +--- + +## Table of Contents + +1. [System Architecture Diagram](#1-system-architecture-diagram) +2. [Component Breakdown](#2-component-breakdown) +3. [Data Model](#3-data-model) +4. [Rendering Approach](#4-rendering-approach) +5. [WebSocket Protocol](#5-websocket-protocol) +6. [Nix Flake Structure](#6-nix-flake-structure) +7. [Integration Points](#7-integration-points) +8. [MVP Scope vs. Future Phases](#8-mvp-scope-vs-future-phases) + +--- + +## 1. System Architecture Diagram + +``` +┌──────────────────────────────────────────────────────────────────────────────┐ +│ SCOTT'S BROWSER │ +│ ┌──────────────────────────────────────────────────────────────────────┐ │ +│ │ SPATIAL OFFICE FRONTEND │ │ +│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────┐ │ │ +│ │ │ Isometric Scene │ │ UI Overlay │ │ State Manager │ │ │ +│ │ │ (Three.js) │ │ (React) │ │ (Zustand) │ │ │ +│ │ │ - Tile map │ │ - Agent cards │ │ - WS client │ │ │ +│ │ │ - Agent sprites │ │ - Zone panels │ │ - Entity cache │ │ │ +│ │ │ - Path animation │ │ - Event log │ │ - Diff engine │ │ │ +│ │ └────────┬─────────┘ └────────┬─────────┘ └──────────┬──────────┘ │ │ +│ │ └─────────────────────┼────────────────────────┘ │ │ +│ └─────────────────────────────────┼───────────────────────────────────────┘ │ +│ │ WebSocket (wss://n97:18790) │ +└────────────────────────────────────┼────────────────────────────────────────────┘ + │ +┌────────────────────────────────────┼────────────────────────────────────────────┐ +│ │HOST: n97 (NixOS, GMKtec NucBoxG5) │ +│ ┌────────────────────────────────┼─────────────────────────────────────────┐ │ +│ │ SPATIAL OFFICE BACKEND │ │ +│ │ │ │ +│ │ ┌──────────────────────┐ ┌──────────────────────┐ │ │ +│ │ │ WS Server │ │ State Engine │ │ │ +│ │ │ (Node.js / ws) │ │ - Agent registry │ │ │ +│ │ │ Port 18790 │ │ - Zone assignment │ │ │ +│ │ │ - Channel rooms │ │ - Activity tracker │ │ │ +│ │ │ - Broadcast diffs │ │ - Event log │ │ │ +│ │ └──────────┬───────────┘ └──────────┬───────────┘ │ │ +│ │ │ │ │ │ +│ │ ┌──────────┴─────────────────────────┴─────────────────────────────┐ │ │ +│ │ │ BRIDGE LAYER │ │ │ +│ │ │ │ │ │ +│ │ │ ┌─────────────────┐ ┌─────────────────┐ ┌──────────────────┐ │ │ │ +│ │ │ │ OpenClaw Bridge │ │ Hermes Bridge │ │ Pi Bridge │ │ │ │ +│ │ │ │ │ │ │ │ │ │ │ │ +│ │ │ │ Connects as WS │ │ Connects as ACP │ │ HTTP API bridge │ │ │ │ +│ │ │ │ backend client │ │ client to │ │ to pi coding │ │ │ │ +│ │ │ │ to OpenClaw GW │ │ Hermes ACP │ │ agent for │ │ │ │ +│ │ │ │ (port 18789) │ │ server │ │ project tasks │ │ │ │ +│ │ │ │ │ │ │ │ │ │ │ │ +│ │ │ │ Reads: │ │ Reads: │ │ Receives: │ │ │ │ +│ │ │ │ - system- │ │ - ACP tool call │ │ - Build status │ │ │ │ +│ │ │ │ presence │ │ events │ │ - Task progress │ │ │ │ +│ │ │ │ - event:agent │ │ - Task outputs │ │ - PR status │ │ │ │ +│ │ │ │ - event:tick │ │ - Skill updates │ │ │ │ │ │ +│ │ │ │ - sessions │ │ │ │ Sends: │ │ │ │ +│ │ │ │ state via │ │ │ │ - Feature │ │ │ │ +│ │ │ │ chat.send │ │ │ │ requests │ │ │ │ +│ │ │ │ │ │ │ │ - Bug reports │ │ │ │ +│ │ │ └────────┬────────┘ └────────┬────────┘ └────────┬─────────┘ │ │ │ +│ │ └───────────┼────────────────────┼────────────────────┼─────────────┘ │ │ +│ └──────────────┼────────────────────┼────────────────────┼──────────────────┘ │ +│ │ │ │ │ +└─────────────────┼────────────────────┼────────────────────┼──────────────────────┘ + │ │ │ + ▼ ▼ ▼ + ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ + │ OpenClaw GW │ │ Hermes Agent │ │ Pi Coding Agent │ + │ (ws://127.0.0.1 │ │ (ACP server) │ │ (pi-mono) │ + │ :18789) │ │ │ │ │ + │ │ │ Self-reflective │ │ CLI / SDK mode │ + │ Agent registry: │ │ learning loop │ │ │ + │ - cos │ │ │ │ Builds/deploys │ + │ - engineering │ │ Inter-agent │ │ spatial-office │ + │ - senior-eng │ │ communication │ │ project │ + │ - trust │ │ protocol │ │ │ + │ - market │ │ │ └──────────────────┘ + │ - health │ └──────────────────┘ + │ - content │ + │ - presentations │ + │ - vault │ + └──────────────────┘ +``` + +### Key Design Decisions + +| Decision | Rationale | +|----------|-----------| +| **Separate WS server** (port 18790) vs. extending OpenClaw GW | OpenClaw GW is a control plane, not an app server. A dedicated WS server keeps the spatial-office protocol clean and lets us iterate without touching the GW. The backend connects to the GW as a `backend` client (trusted loopback, shared gateway token). | +| **Bridge layer** as in-process modules | All three bridges run in the same Node.js process as the WS server. No need for separate microservices at this scale. Each bridge is a module that polls/streams from its source and pushes state diffs into the state engine. | +| **ACP for Hermes integration** | Hermes runs as an ACP server. The Hermes bridge uses `@openclaw/acpx` or a direct ACP client to connect, send tasks, and receive tool-call events. This is the standard path: OpenClaw → ACP → Hermes. | + +--- + +## 2. Component Breakdown + +### 2.1 Frontend (`packages/frontend`) + +**Stack**: TypeScript, Vite, React 18, Zustand, Three.js (via `@react-three/fiber`) + +``` +packages/frontend/ +├── index.html +├── vite.config.ts +├── src/ +│ ├── main.tsx # React entry +│ ├── App.tsx # Root layout (scene + UI overlay) +│ ├── scene/ +│ │ ├── OfficeScene.tsx # Three.js canvas + camera setup +│ │ ├── TileMap.tsx # Isometric tile grid renderer +│ │ ├── AgentSprite.tsx # Per-agent animated sprite +│ │ ├── ZoneMarker.tsx # Zone boundary overlays +│ │ └── PathAnimator.ts # Smooth interpolation between grid cells +│ ├── ui/ +│ │ ├── HUD.tsx # Top bar: time, agent count, events +│ │ ├── AgentCard.tsx # Click-to-inspect detail panel +│ │ ├── ZonePanel.tsx # Zone occupancy summary +│ │ ├── EventFeed.tsx # Real-time event log +│ │ └── MiniMap.tsx # Small overhead reference map +│ ├── state/ +│ │ ├── store.ts # Zustand store (agent states, zones, events) +│ │ ├── wsClient.ts # WebSocket client with reconnect +│ │ └── types.ts # Shared frontend types +│ └── assets/ +│ ├── tiles/ # Tile sprites (floor, wall, desk, plant) +│ ├── agents/ # Agent avatar sprites (9 departments) +│ └── zones/ # Zone type indicators +``` + +**Why `@react-three/fiber`**: It gives us React-idiomatic Three.js with declarative scene composition, and we already get the React ecosystem (state, hooks, devtools) for free. The isometric scene is modest — 50×50 tiles max, ~15 sprites — so the fiber overhead is negligible. + +### 2.2 Backend (`packages/backend`) + +**Stack**: TypeScript, Node.js 22, `ws` library, Zod for validation + +``` +packages/backend/ +├── tsconfig.json +├── src/ +│ ├── index.ts # Entry: starts WS server + bridges +│ ├── server.ts # WebSocket server (port 18790) +│ ├── state/ +│ │ ├── engine.ts # In-memory state + diff engine +│ │ ├── agentRegistry.ts # Known agents (9 depts) +│ │ ├── zoneManager.ts # Zone definitions + occupancy +│ │ └── eventLog.ts # Ring buffer of recent events +│ ├── bridges/ +│ │ ├── openclaw.ts # OpenClaw GW bridge +│ │ ├── hermes.ts # Hermes ACP bridge +│ │ └── pi.ts # Pi coding agent bridge +│ ├── protocol/ +│ │ ├── messages.ts # WS message types (Zod schemas) +│ │ └── diff.ts # State diff computation +│ └── validation/ +│ └── schemas.ts # Zod schemas for all entities +``` + +**Why in-memory state**: 9 agents with simple state (position, activity, status) is trivially small. No database needed for MVP. The event log uses a ring buffer (last 500 events). If persistence is desired later, add SQLite. + +**WS library**: `ws` (npm, ~20M weekly downloads) is the de facto Node.js WebSocket library. Fast, well-maintained, no dependencies. We wrap it with a thin protocol layer. + +### 2.3 Bridge: OpenClaw (`bridges/openclaw.ts`) + +The OpenClaw bridge connects to the Gateway WS on `ws://127.0.0.1:18789` as a `backend` client, authenticating with the shared gateway token. + +**Connection handshake** (per [Gateway Protocol](https://docs.openclaw.ai/gateway/protocol)): + +```typescript +const connect = { + type: "req", + id: "spatial-office-1", + method: "connect", + params: { + minProtocol: 3, + maxProtocol: 4, + client: { id: "spatial-office", version: "0.1.0", platform: "linux", mode: "backend" }, + role: "operator", + scopes: ["operator.read"], // read-only: presence, agent events, sessions + auth: { token: GATEWAY_SHARED_TOKEN }, + }, +}; +``` + +**What it reads**: + +| GW Method/Event | What We Get | Mapped To | +|----------------|-------------|-----------| +| `system-presence` | Connected devices (mac app, webchat, nodes) → gateway uptime, client count | Top-level system health indicator | +| `event:agent` | Streaming agent turns — when an agent starts processing a message, tool calls, completion | Agent activity state (`working`, `thinking`, `idle`) | +| `event:presence` | Periodic presence updates from connected clients | Scott's presence (is he watching?) | +| `event:tick` | Gateway heartbeat every ~15s (configurable) | Liveness signal for the bridge | +| `event:health` | Gateway health snapshot | Gateway status indicator | +| Chat sessions | Polled via `chat.send` with agent-specific queries (or use the sessions_history API) | Current task description per agent | + +**Agent state mapping** (how we infer activity from event:agent): + +```typescript +// event:agent payload includes: +// - agentId: which agent is working +// - status: "accepted" | "streaming" | "ok" | "error" +// - tool calls: embedded in streaming content + +function mapAgentState(event: AgentEvent): AgentActivity { + // When event:agent arrives with status:"accepted" → agent is "thinking" + // When we see tool calls → agent is "working" (extract tool name as context) + // When event:agent completes (status:"ok") → agent is "idle" + // When no agent events for >5min → agent is "idle" +} +``` + +**Fallback for agents without active sessions**: Poll the QMD session transcript store. Each agent's session directory at `~/.openclaw/agents//sessions` can be read for recent activity. This requires filesystem access (same host, so fine). + +### 2.4 Bridge: Hermes (`bridges/hermes.ts`) + +The Hermes bridge connects to a Hermes agent running as an ACP server. The bridge: + +1. Opens an ACP session with Hermes +2. Periodically queries: "What are you working on? What agents have you communicated with recently?" +3. Parses Hermes responses for inter-agent communication events +4. Maps these to spatial events (agents moving to "huddle" when Hermes facilitates communication between them) + +**ACP connection** (using `@openclaw/acpx` as reference): + +```typescript +import { ACPClient } from "@agentclientprotocol/sdk"; // or @openclaw/acpx + +const client = new ACPClient({ + transport: "stdio", // or "ws" if Hermes exposes WS + command: "hermes", // Hermes CLI + args: ["acp", "serve"], +}); +``` + +**What it extracts**: + +| Hermes Output | Spatial Office Mapping | +|--------------|----------------------| +| "Agent X asked me to help with Y" | Move Agent X to collaboration zone, show "pairing with Hermes on Y" | +| Skill extraction events | Show Hermes in self-improvement zone, "refining skill X" | +| Cron task execution | Show relevant agents moving to their task zones | +| Learning loop iteration | Self-improvement zone activity | + +**Why ACP matters here**: ACP is the standard protocol for driving external harnesses from OpenClaw. Hermes adopted it early. The bridge acts as an ACP *client* that drives Hermes and extracts structured output, rather than scraping logs. This is the same pattern described in the "Hermes drives, OpenClaw contains" architecture, just flipped: OpenClaw contains the spatial office, and Hermes is one of the observed agents within it. + +### 2.5 Bridge: Pi (`bridges/pi.ts`) + +The Pi bridge tracks the pi coding agent's activity on the spatial-office repo itself (dogfooding). + +**Connection**: Pi's `@earendil-works/pi-coding-agent` package exposes an SDK with `createAgentSession()` and `SessionManager`. The bridge can: + +1. **Programmatic mode**: Use `createAgentSession()` in SDK mode to spawn a pi session that works on spatial-office tasks +2. **Status polling**: Pi writes session logs; the bridge tails those logs for activity indicators + +For MVP, the simplest approach is HTTP polling of pi's task status: + +```typescript +// Pi exposes a simple HTTP health/status endpoint (we add this to pi-mono fork) +const status = await fetch("http://localhost:18991/status"); +// Response: { currentTask: "implementing tile renderer", progress: 0.4, repo: "spatial-office" } +``` + +**What it reports**: + +- Pi's current task on the spatial-office repo +- Build/deploy status +- PR status (via Forgejo API at `os.scoble.me/forgejo`) + +--- + +## 3. Data Model + +### 3.1 Entities + +```typescript +// ── Agent ──────────────────────────────────────────── +interface Agent { + id: string; // "cos", "engineering", "senior-eng", etc. + name: string; // Display name: "COS Agent" + department: Department; // enum of 9 values + position: GridPosition; // { row: number, col: number } + facing: Direction; // "N" | "NE" | "E" | "SE" | "S" | "SW" | "W" | "NW" + activity: AgentActivity; // Current state + currentTask: string | null; // "Reviewing PR #42", "Memory maintenance", etc. + lastHeartbeat: number; // Unix ms timestamp + sessionKey: string | null; // OpenClaw session key if active +} + +interface AgentActivity { + status: AgentStatus; // "idle" | "thinking" | "working" | "blocked" | "offline" + zone: ZoneId; // Which zone the agent is in + detail: string | null; // "Running: web_search", "Calling: sessions_spawn" + startedAt: number; // When this activity began +} + +type AgentStatus = "idle" | "thinking" | "working" | "blocked" | "offline"; +type Department = "cos" | "engineering" | "senior-eng" | "trust" | "market" + | "health" | "content" | "presentations" | "vault"; + +// ── Zone ───────────────────────────────────────────── +interface Zone { + id: ZoneId; + name: string; // "Engineering Pod", "Quiet Room", "Coach Corner" + type: ZoneType; // "collaboration" | "solo" | "self-improvement" | "social" + bounds: GridRect; // { rowStart, colStart, rowEnd, colEnd } + capacity: number; // Max agents in zone + occupants: string[]; // Agent IDs currently in zone + features: ZoneFeature[]; // "whiteboard", "terminal", "coffee", "coach-bot" +} + +type ZoneId = string; +type ZoneType = "collaboration" | "solo" | "self-improvement" | "social"; +type ZoneFeature = "whiteboard" | "terminal" | "coffee" | "coach-bot" | "library"; + +interface GridRect { + rowStart: number; + colStart: number; + rowEnd: number; + colEnd: number; +} + +// ── GridPosition ───────────────────────────────────── +interface GridPosition { + row: number; // 0-indexed, origin at top-left of isometric view + col: number; +} + +// ── Event ──────────────────────────────────────────── +interface OfficeEvent { + id: string; + timestamp: number; // Unix ms + type: EventType; + agentId: string | null; // Which agent triggered it + zoneId: string | null; // Where it happened + payload: Record; +} + +type EventType = + | "agent.moved" // Agent repositioned + | "agent.activity_changed" // Status or zone changed + | "agent.offline" // Agent heartbeat lost + | "agent.online" // Agent returned + | "hermes.message" // Inter-agent communication observed + | "pi.task_update" // Pi task progress changed + | "zone.full" // Zone hit capacity + | "system.heartbeat"; // Periodic system health + +// ── World State (the full snapshot) ────────────────── +interface OfficeState { + version: number; // Monotonic counter for diff-based sync + agents: Record; + zones: Record; + events: OfficeEvent[]; // Last N events (ring buffer) + meta: { + gatewayUptime: number; + connectedClients: number; + scottPresent: boolean; + lastUpdate: number; + }; +} +``` + +### 3.2 Zone Map (MVP Office Layout) + +``` + ┌──────────────────────────────────────┐ + │ COFFEE / SOCIAL │ (top) + │ [☕ social zone 4x2] │ + ├──────────┬───────────────┬────────────┤ + │ │ COLLABORATION│ │ + │ SOLO │ POD A │ SOLO │ + │ WORK │ [4 desks] │ WORK │ + │ [6 │ Whiteboard │ [6 │ + │ desks] │ Terminal │ desks] │ + │ ├───────────────┤ │ + │ │ COLLABORATION│ │ + │ │ POD B │ │ + │ │ [4 desks] │ │ + │ │ Whiteboard │ │ + ├──────────┴───────────────┴────────────┤ + │ SELF-IMPROVEMENT AREA │ (bottom) + │ [Coach Bot] [Library] [Quiet Pods] │ + └──────────────────────────────────────┘ +``` + +**Grid**: 24 columns × 15 rows. Each tile is 64×32px isometric diamond. + +**Zone assignments for 9 agents**: + +| Zone | Type | Capacity | Default Assignment | +|------|------|----------|-------------------| +| Pod A (left) | collaboration | 4 | engineering, senior-eng, pi (when active) | +| Pod B (right) | collaboration | 4 | market, content, presentations | +| Solo West | solo | 6 | cos, trust | +| Solo East | solo | 6 | health, vault | +| Coach Corner | self-improvement | 3 | Any agent doing memory maintenance / coach review | +| Coffee Bar | social | 4 | Any agents in "idle" or "observing" state | + +--- + +## 4. Rendering Approach + +### 4.1 Why Three.js (not Phaser, not Canvas 2D) + +| Criterion | Three.js | Phaser | Canvas 2D | +|-----------|----------|--------|-----------| +| Isometric projection | Trivial (orthographic camera at 45° rotation, 35.264° elevation) | Built-in isometric plugin but 2D-only | Manual tile math, no depth sorting | +| 3D depth effects | Native (agents can have height, z-fighting resolved) | 2D render order only | Manual z-index | +| Animation (tweening) | Built-in `gsap` + Three.js animation loop | Built-in tween system | Manual requestAnimationFrame | +| React integration | `@react-three/fiber` (25K+ stars) | `@yonatanmn/react-phaser` (unmaintained) | Manual DOM | +| Bundle size | ~150KB gzipped (with tree-shaking) | ~1MB | 0KB | +| Community/ecosystem | Massive | Large (game dev) | Native | + +**Decision**: Three.js with `@react-three/fiber`. The isometric view is fundamentally a 3D scene viewed from a fixed orthographic angle. Three.js gives us proper depth sorting, smooth path animation with `useFrame`, and the React integration means we can use Zustand state directly in the render loop. + +### 4.2 Camera Setup + +```typescript +// Orthographic camera positioned for true isometric projection +// True isometric: rotate Y by 45°, then X by 35.264° (atan(1/√2)) +const camera = new THREE.OrthographicCamera( + -width / 2, width / 2, // left, right + height / 2, -height / 2, // top, bottom + 0.1, 1000 // near, far +); + +// Position camera above and center +camera.position.set(0, -20, 20); +camera.lookAt(0, 0, 0); + +// For a 2:1 isometric pixel ratio (standard in games): +// X angle = 30° from horizontal (this gives the 2:1 diamond shape) +// Actually, use rotY=45°, rotX=-35.264° for true isometric +// But for a cleaner 2:1 pixel ratio (easier art), use rotX=-30° +``` + +**Alternative — fake isometric with 2D sprites on a tilted plane**: +We can place flat sprites on a plane tilted at 30° rather than building a full 3D office. This is simpler for art assets (flat PNG sprites for agents and furniture). The Three.js `Sprite` object always faces the camera, so for isometric we'd use quads (`PlaneGeometry`) tilted to match the floor plane. + +### 4.3 Tile Map + +The office is a 2D grid (`Array>`) where each cell has: + +```typescript +interface Tile { + type: TileType; // "floor" | "wall" | "desk" | "empty" | "door" + zoneId: ZoneId | null; // Which zone this tile belongs to + occupant: string | null; // Agent ID if someone is on this tile +} + +type TileType = "floor" | "wall" | "desk" | "empty" | "door" | "plant" | "whiteboard"; +``` + +**Rendering loop** (each frame): + +1. Sort tiles by `row + col` for correct isometric depth (farther tiles render first) +2. For each tile: render floor sprite, then desk/wall sprite +3. For each agent: render agent sprite at grid position, apply z-offset for "standing on desk" +4. Apply `gsap` tweens for smooth movement between tiles + +**Performance**: 24×15 = 360 tiles max, ~200 visible at a time. Three.js handles this effortlessly without instancing. + +### 4.4 Agent Visual Design + +Each of the 9 agents gets a simple isometric sprite (64×64 or 96×48px) with: +- **Department color** (distinct hue per department) +- **Department icon** (simple glyph: gear for engineering, shield for trust, book for content, etc.) +- **Status indicator** (small colored dot: green = idle, yellow = thinking, blue = working, red = blocked) +- **Speech bubble** (optional, appears when agent is "talking" via Hermes) + +Sprites are PNGs drawn in a 2:1 isometric pixel art style. We can generate these programmatically as a first pass (colored boxes with text labels) and replace with proper art later. + +### 4.5 UI Overlay (React) + +The UI overlay sits on top of the Three.js canvas (absolute positioned div) and provides: + +- **Agent card**: Click an agent → card slides in showing: name, department, current activity, task description, session key, last heartbeat +- **Zone panel**: Sidebar showing zone occupancy and type +- **Event feed**: Scrollable log of recent events (last 50) +- **Minimap**: Small 2D overhead view showing the full office layout +- **Camera controls**: Zoom in/out, pan (though the isometric view is mostly fixed) + +--- + +## 5. WebSocket Protocol + +### 5.1 Connection + +The frontend connects to `ws://n97:18790` (or `wss://` with TLS proxy later). + +```typescript +// No auth needed for MVP (local network only). +// Client sends a hello message: +{ + type: "hello", + version: 1, + client: "browser", +} +``` + +### 5.2 Server → Client Messages + +| Message | Payload | Frequency | +|---------|---------|-----------| +| `state.full` | `OfficeState` (complete snapshot) | On connect, after reconnect | +| `state.diff` | `{ version, changes: StateDiff[] }` | On every state change (throttled to 100ms batches) | +| `event` | `OfficeEvent` | Real-time as events occur | +| `heartbeat` | `{ ts: number }` | Every 15 seconds | + +**State diff format** (avoids sending the full state on every tick): + +```typescript +interface StateDiff { + op: "agent.update" | "agent.move" | "zone.update" | "event.append"; + path: string; // e.g., "agents.engineering.activity.status" + value: unknown; // New value +} + +// Example diff: agent moved +{ + op: "agent.move", + path: "agents.engineering", + value: { + position: { row: 5, col: 12 }, + activity: { status: "working", zone: "pod-a", detail: "pairing with senior-eng", startedAt: 1716152400000 } + } +} + +// Batch of diffs sent every 100ms: +{ + type: "state.diff", + version: 47, + changes: [ + { op: "agent.move", path: "agents.engineering", value: { ... } }, + { op: "agent.update", path: "agents.cos.activity.status", value: "thinking" }, + { op: "event.append", path: "events", value: { type: "agent.moved", agentId: "engineering", ... } }, + ] +} +``` + +### 5.3 Client → Server Messages + +| Message | Payload | Description | +|---------|---------|-------------| +| `agent.click` | `{ agentId: string }` | Scott clicked an agent (triggers detail expansion in UI, handled locally) | +| `zone.click` | `{ zoneId: string }` | Scott clicked a zone | +| `camera.move` | `{ x, y, zoom }` | Scott panned/zoomed (local state, not broadcast) | + +For MVP, client→server messages are minimal. The view is primarily read-only observation. + +### 5.4 State Throttling + +The backend batches diffs every 100ms. If no changes occur, no message is sent. The heartbeat ensures the client knows the connection is alive. + +```typescript +// Backend diff batching +class DiffBatcher { + private diffs: StateDiff[] = []; + private timer: NodeJS.Timeout | null = null; + + push(diff: StateDiff) { + this.diffs.push(diff); + if (!this.timer) { + this.timer = setTimeout(() => this.flush(), 100); + } + } + + flush() { + if (this.diffs.length > 0) { + broadcast({ type: "state.diff", version: this.version++, changes: this.diffs }); + this.diffs = []; + } + this.timer = null; + } +} +``` + +--- + +## 6. Nix Flake Structure + +### 6.1 `flake.nix` + +```nix +{ + description = "Spatial Office — isometric collaborative workspace for agents and humans"; + + inputs = { + nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable"; + flake-utils.url = "github:numtide/flake-utils"; + }; + + outputs = { self, nixpkgs, flake-utils }: + flake-utils.lib.eachDefaultSystem (system: + let + pkgs = import nixpkgs { inherit system; }; + + # Node.js toolchain + nodeToolchain = with pkgs; [ + nodejs_22 + nodePackages.pnpm + nodePackages.typescript + ]; + + # Build the frontend + buildFrontend = pkgs.stdenv.mkDerivation { + name = "spatial-office-frontend"; + src = ./packages/frontend; + nativeBuildInputs = nodeToolchain; + buildPhase = '' + pnpm install --frozen-lockfile + pnpm run build + ''; + installPhase = '' + mkdir -p $out + cp -r dist/* $out/ + ''; + }; + + # Build the backend + buildBackend = pkgs.stdenv.mkDerivation { + name = "spatial-office-backend"; + src = ./packages/backend; + nativeBuildInputs = nodeToolchain; + buildPhase = '' + pnpm install --frozen-lockfile + pnpm run build + ''; + installPhase = '' + mkdir -p $out + cp -r dist/* $out/ + cp -r node_modules $out/ + mkdir -p $out/bin + echo '#!/usr/bin/env bash' > $out/bin/spatial-office-backend + echo 'exec ${pkgs.nodejs_22}/bin/node $out/index.js' >> $out/bin/spatial-office-backend + chmod +x $out/bin/spatial-office-backend + ''; + }; + in + { + packages = { + frontend = buildFrontend; + backend = buildBackend; + default = pkgs.buildEnv { + name = "spatial-office"; + paths = [ buildFrontend buildBackend ]; + }; + }; + + devShells.default = pkgs.mkShell { + buildInputs = nodeToolchain ++ (with pkgs; [ + # Development tools + nodePackages.vite + nodePackages.eslint + ]); + + shellHook = '' + echo "🏢 Spatial Office dev shell" + echo " pnpm dev — start frontend + backend in dev mode" + echo " pnpm build — production build" + ''; + }; + + # NixOS module for running as a systemd service on n97 + nixosModules.spatial-office = { config, lib, pkgs, ... }: + let + cfg = config.services.spatial-office; + in + { + options.services.spatial-office = { + enable = lib.mkEnableOption "Spatial Office service"; + port = lib.mkOption { type = lib.types.int; default = 18790; }; + openclawTokenFile = lib.mkOption { type = lib.types.path; }; + }; + + config = lib.mkIf cfg.enable { + systemd.services.spatial-office = { + description = "Spatial Office Backend"; + wantedBy = [ "multi-user.target" ]; + after = [ "network.target" ]; + environment = { + PORT = toString cfg.port; + OPENCLAW_GATEWAY_URL = "ws://127.0.0.1:18789"; + OPENCLAW_TOKEN_FILE = cfg.openclawTokenFile; + NODE_ENV = "production"; + }; + serviceConfig = { + ExecStart = "${self.packages.${system}.backend}/bin/spatial-office-backend"; + Restart = "always"; + RestartSec = 5; + User = "spatial-office"; + Group = "spatial-office"; + }; + }; + + users.users.spatial-office = { + isSystemUser = true; + group = "spatial-office"; + }; + users.groups.spatial-office = {}; + }; + }; + } + ); +} +``` + +### 6.2 Why Nix + +- **Reproducibility**: `nix build` produces identical outputs on any machine. The n97 is NixOS, so this is the native deployment path. +- **Node.js pinning**: No "works on my machine" — Node 22, pnpm, and TypeScript versions are locked. +- **Systemd service**: The NixOS module handles service lifecycle, user creation, and restart policy. +- **Dev shell**: `nix develop` gives every developer the exact same toolchain. + +### 6.3 Directory Layout + +``` +spatial-office/ +├── flake.nix +├── flake.lock +├── package.json # Workspace root (pnpm workspaces) +├── pnpm-workspace.yaml +├── packages/ +│ ├── frontend/ +│ │ ├── package.json +│ │ ├── vite.config.ts +│ │ ├── tsconfig.json +│ │ └── src/ # (as described in §2.1) +│ └── backend/ +│ ├── package.json +│ ├── tsconfig.json +│ └── src/ # (as described in §2.2) +├── assets/ # Design files, tile templates +├── README.md +└── ARCHITECTURE.md # This file +``` + +--- + +## 7. Integration Points + +### 7.1 OpenClaw Integration + +**What we need from OpenClaw**: + +| Data | Source | How | +|------|--------|-----| +| Which agents exist | `agents.list` in Gateway config (`~/.openclaw/openclaw.json`) | Backend reads config file on startup, or GW exposes this via `config.get` RPC | +| Agent online/offline | `system-presence` WS method | Bridge calls this on startup + polls every 30s | +| Agent activity (working/thinking/idle) | `event:agent` WS events | Bridge subscribes to agent events for all 9 agent IDs | +| Current task description | Session transcripts or `chat.send` polling | Bridge reads `~/.openclaw/agents//sessions` most recent file | +| Scott's presence | `event:presence` with Scott's device ID | Updates `meta.scottPresent` | + +**Gateway config read**: +The backend needs to know about the 9 agents. It can either: +1. Read `~/.openclaw/openclaw.json` directly (same host, permissions allowing) +2. Call the Gateway's `config.get` RPC method with `operator.admin` scope +3. Accept a config file listing agent IDs explicitly + +Option 3 is simplest for MVP. A `spatial-office.config.json`: + +```json +{ + "agents": [ + { "id": "cos", "name": "COS Agent", "department": "cos", "color": "#FF6B6B" }, + { "id": "engineering", "name": "Engineering Agent", "department": "engineering", "color": "#4ECDC4" }, + ... + ], + "zones": [ ... ] +} +``` + +**Security**: The backend needs `operator.read` scope only. It connects via loopback with the shared gateway token. No admin access required. + +### 7.2 Hermes Integration + +Hermes runs as an ACP server. The spatial-office backend is an ACP *client*. + +**Architecture choice**: Hermes is NOT one of the 9 Scobleclaw agents. It's an external reasoning engine that the agents *use* through ACP. In the spatial office: + +- Hermes appears as a "utility" — like a whiteboard or a coach bot +- When Agent A asks Hermes something, Agent A moves to a collaboration pod with a Hermes indicator +- When Hermes processes a learning loop, the self-improvement zone shows activity + +**ACP protocol flow**: + +``` +Spatial Office Backend Hermes ACP Server + │ │ + ├──── initialize ─────────────>│ "What agents have communicated + │ │ through you recently?" + │<─── agent_messages ──────────┤ [{from: "engineering", to: "senior-eng", + │ │ topic: "spatial-office PR review"}] + │ │ + ├──── tools/list ─────────────>│ Discover available Hermes tools + │<─── tools ───────────────────┤ + │ │ + ├──── tools/call ─────────────>│ "Describe your current task" + │<─── result ──────────────────┤ "Refining the code-review skill + │ │ after 12 evaluations" +``` + +**For MVP**: The Hermes bridge starts simple — a periodic poll (every 30s) that asks Hermes for its current state. We don't need real-time streaming from Hermes in phase 1. The ACP session stays open, the bridge sends a query, receives a response, and maps it to spatial events. + +### 7.3 Pi Integration + +Pi (`@earendil-works/pi-coding-agent`) is the coding agent that builds the spatial-office project itself. In the office, Pi appears as a 10th "agent" (or a special entity) that: + +- Sits in a designated development workstation +- Shows current task: "Implementing TileMap renderer [▓▓▓▓░░░░] 40%" +- Moves to collaboration pod when doing code review with engineering +- Shows build status from the Forgejo CI + +**Integration approach**: + +1. **Pi session manager**: The backend spawns a Pi session using the SDK: + ```typescript + import { createAgentSession } from "@earendil-works/pi-coding-agent"; + const { session } = await createAgentSession({ + model: myModel, + tools: ["read", "bash", "write"], + sessionManager: SessionManager.file({ dir: "./pi-sessions" }), + }); + ``` + +2. **Status endpoint**: We add a lightweight HTTP server to the Pi session that exposes: + ``` + GET /status → { task: "...", progress: 0.4, status: "working" } + ``` + +3. **Forgejo webhooks**: The backend listens for Forgejo push/PR events for the spatial-office repo to track build status + +**For MVP**: The Pi bridge is a polling loop that checks pi's session log file. The spatial-office repo's CI status is checked via the Forgejo API (`GET /api/v1/repos/sscoble/spatial-office/commits`). + +--- + +## 8. MVP Scope vs. Future Phases + +### MVP (Phase 1) — "The Observable Office" + +**Goal**: Scott can open a browser tab and see the 9 agents in an isometric office, with their current activity inferred from OpenClaw. + +| Feature | Included? | Rationale | +|---------|-----------|-----------| +| Isometric tile map (fixed layout) | ✅ | Core visual | +| 9 agent sprites with department colors | ✅ | Core visual | +| Agent activity state (idle/thinking/working/blocked) from OpenClaw | ✅ | Core data source | +| Agent movement between zones based on activity | ✅ | Core mechanic | +| Click agent → detail card | ✅ | Scott's view | +| Zone occupancy indicators | ✅ | Visual clarity | +| Event feed (last 50 events) | ✅ | Debugging + insight | +| WebSocket real-time sync | ✅ | Required for live updates | +| Nix flake build | ✅ | Deployment requirement | +| OpenClaw GW bridge (operator.read) | ✅ | Primary data | +| **Hermes bridge** | ❌ (deferred) | Adds complexity without visual payoff in MVP | +| **Pi bridge** | ❌ (deferred) | Pi tasks can be inferred from OpenClaw events (if Pi is an agent) | +| Agent path animation (smooth tweening) | ✅ | Quality-of-life, cheap to add | +| Scott presence indicator | ✅ | Trivial (1 presence check) | +| Mobile/responsive layout | ❌ | Desktop-first, n97 serves local network | +| Multiple observers (multi-client) | ❌ | Scott's personal view only | + +**MVP deliverables**: +1. Working frontend (Vite + React + Three.js) rendering the office +2. Working backend (Node.js + ws) bridging to OpenClaw +3. Nix flake that builds both and deploys as a systemd service on n97 +4. Accessible at `http://n97:18790` on the local network + +### Phase 2 — "Hermes + Pi Visibility" + +| Feature | Notes | +|---------|-------| +| Hermes ACP bridge | Real-time Hermes state, inter-agent comm visualization | +| Pi bridge | Pi's task status on spatial-office, Forgejo CI status | +| Agent "huddle" visualization | Two agents visibly move together when collaborating | +| Hermes message visualization | Speech bubbles or "Hermes link" lines between agents | +| Coach bot visualization | Self-improvement zone shows active Hermes learning loops | + +### Phase 3 — "Interaction" + +| Feature | Notes | +|---------|-------| +| Scott can send messages to agents from the UI | `chat.send` through OpenClaw bridge | +| Scott can drag agents between zones | Sends `system-event` to re-route agent tasks | +| Agent "whisper" (private message to one agent) | One-click from agent card | +| Sound cues | Ambient office sounds, notification chimes | +| Agent idle animations | Breathing, looking around, typing | + +### Phase 4 — "VR + Multi-User" + +| Feature | Notes | +|---------|-------| +| WebXR headset view | First-person perspective in the same office | +| Multiple human observers | Different views, shared state | +| Voice chat integration | Spatial audio between agents and humans | +| Custom office layouts | Drag-and-drop office editor | +| Real collaboration tools | Shared whiteboard, code viewer, screenshare | + +### Phase 5 — "Full Product" + +| Feature | Notes | +|---------|-------| +| External teams | Multiple organizations, each with their own office | +| Agent marketplace | Drop-in agent "employees" from a catalog | +| Analytics dashboard | Productivity, collaboration metrics, heat maps | +| API for third-party agent systems | Standard spatial-office protocol | + +--- + +## Appendix A: Technology Dependency Graph + +``` +spatial-office +├── @react-three/fiber (^8.x) → three (^0.170.x) → WebGL +├── @react-three/drei (^9.x) → helpers for fiber (OrbitControls, etc.) +├── react (^18.x) +├── react-dom (^18.x) +├── zustand (^5.x) → state management +├── gsap (^3.x) → smooth path animation +├── vite (^6.x) → build tool +├── typescript (^5.x) +├── ws (^8.x) → WebSocket server +├── zod (^3.x) → validation +├── @agentclientprotocol/sdk → ACP client (Hermes bridge, Phase 2) +└── @earendil-works/pi-coding-agent → Pi SDK (Pi bridge, Phase 2) +``` + +## Appendix B: OpenClaw Gateway Protocol — Methods We Use + +Per the [Gateway Protocol](https://docs.openclaw.ai/gateway/protocol) and [Presence](https://docs.openclaw.ai/concepts/presence) docs: + +| Method | Scope Required | Use | +|--------|---------------|-----| +| `connect` | (handshake) | Establish backend client connection | +| `system-presence` | (unrestricted) | Get connected devices/clients | +| `event:agent` (subscription) | `operator.read` | Stream agent turn events | +| `event:presence` (subscription) | (unrestricted) | Stream presence updates | +| `event:health` (subscription) | (unrestricted) | Stream gateway health | +| `event:tick` (subscription) | (unrestricted) | Gateway heartbeat | +| `chat.send` | `operator.write` | (Phase 3) Send messages to agents | +| `config.get` | `operator.admin` | (May not be needed; use config file instead) | + +## Appendix C: Open Questions + +1. **Agent session polling frequency**: How often should we poll? 5s is responsive but may be noisy. 30s is calmer. Start with 10s and tune. +2. **OpenClaw shared token**: Where is it stored, and how does the NixOS module securely access it? Option: `OPENCLAW_TOKEN_FILE` environment variable pointing to a file readable by the `spatial-office` user. +3. **Hermes ACP endpoint**: Is Hermes currently running on n97? If so, at what port/socket? If not, that's a prerequisite for Phase 2. +4. **Agent sprite art**: Programmatic (CSS-generated or Canvas-drawn) vs. hand-drawn pixel art. Start programmatic, iterate. +5. **n97 firewall**: Does n97 expose port 18790 to the local network? For Scott's browser to connect, this port must be open or tunneled. diff --git a/COMPETITIVE_ANALYSIS.md b/COMPETITIVE_ANALYSIS.md new file mode 100644 index 0000000..363d7a0 --- /dev/null +++ b/COMPETITIVE_ANALYSIS.md @@ -0,0 +1,378 @@ +# Spatial Office — Competitive Landscape Analysis + +**Date:** 2026-05-19 +**Author:** @market (Market & Product Lead) +**Project:** spatial-office (Scobleclaw dogfood → product) + +--- + +## Executive Summary + +The spatial collaborative office market is fragmented into three incompatible tiers: **VR-only** (Immersed, Horizon Workrooms, Arthur, Glue, MeetinVR), **2D-only** (Gather, Teamflow, Roam, Sococo), and **events-focused** (Spatial, Frame, Microsoft Mesh/Teams Immersive). + +**No product bridges VR and non-VR users in a single shared spatial environment that feels natural for both.** This is the gap spatial-office targets. + +The additional differentiator — visualizing AI agents as first-class "residents" in the spatial office — has zero direct competitors. This is a genuinely novel angle. + +--- + +## 1. Direct Competitors — Who's Doing What, What's Broken + +### VR-Heavy Players + +#### Immersed +- **What it is:** Multi-screen virtual workspace for VR headsets (Meta Quest, Apple Vision Pro, HTC Vive). Streams your physical computer into VR with up to 5 virtual monitors. +- **Funding:** Raised ~$12M; launched their own "Visor" headset (2025, mass production announced summer 2025) +- **What works:** Best-in-class virtual monitor experience. Keyboard passthrough. Solo focus mode is genuinely useful. +- **What's broken:** + - VR-only — no browser fallback for non-headset collaborators + - Collaboration is secondary (multi-screen sharing + whiteboard, not ambient presence) + - Their own hardware play (Visor) fragments focus — are they a software company or hardware? + - Subscription required for collaboration features +- **Key risk signal:** Meta sunsetting Horizon Workrooms in Feb 2026 pushed Immersed to welcome those refugees — but that's a niche within a niche + +#### Meta Horizon Workrooms +- **What it is:** Meta's first-party VR meeting/office app. Virtual meeting rooms, whiteboards, desktop streaming. +- **What works:** Tight Quest integration. Zoom integration for non-VR participants (call-in only, not spatial). +- **What's broken:** + - **Meta effectively killed it (Feb 2026).** Removed collaboration features, shifted to "personal office" only. + - WIRED (Jan 2023) documented company trials: "cumbersome sensor-tracking setups, user dizziness, inability to use a computer naturally inside the headset" + - Reddit (Jun 2024): "Horizon Workrooms removes collaboration. Any alternatives?" — users literally fleeing + - Teams/Immersive Spaces is Meta's enterprise play now — Workrooms was the casualty +- **Key lesson:** Even Meta with $70B metaverse investment couldn't make VR-first office work stick. The friction of "everyone needs a headset" is fatal. + +#### Arthur +- **What it is:** Enterprise VR collaboration platform. Virtual office with persistent rooms, AI-driven workflows, async collaboration tools. +- **Funding:** $2.5M seed (2020) +- **What works:** Enterprise-grade security, persistent spaces, supports both VR and PC (desktop app) +- **What's broken:** + - Requires a desktop app install — no browser-first approach + - Positioned for enterprise (sales cycles, procurement) — not product-led growth + - 3D environments are generic corporate spaces — no character, no delight + - Pivoting hard toward "AI co-workers" messaging — feels reactive to AI hype rather than native + +#### Glue +- **What it is:** Finnish VR collaboration platform. Immersive virtual spaces, expressive avatars, spatial audio. +- **What works:** Strong spatial audio. Supports VR (Quest, HTC, etc.) and desktop mode. Persistent sticky notes and whiteboards. Office 365 / Google Workspace integration. +- **What's broken:** + - Desktop mode is a separate experience, not true shared spatial presence + - Generic corporate aesthetic — all glass walls and boardroom tables + - Small company (Finland-based), limited market traction + - No browser-only option — requires app install + +#### MeetinVR +- **What it is:** VR-native meeting platform. 3D sketching, prototyping review, brainstorming in VR. +- **What works:** Strong 3D interaction tools. Good for design review / prototyping. +- **What's broken:** + - VR-only — no non-VR participation at all + - Narrow use case (design reviews), not persistent office + - Very small player — minimal market presence since 2016 + +### 2D Spatial Players (Browser-Based) + +#### Gather +- **What it is:** 2D pixel-art virtual office with proximity-based video chat. Walk your avatar around a map, video pops when near people. +- **Funding:** $77M total (Series B led by Sequoia + Index, $50M round). Revenue ~$9-10.8M (2024). YC W20. +- **What works:** + - **Proven product-market fit.** 74-person team, real revenue. The closest thing to a winner in this space. + - Browser-first, zero install. Walk up to someone = video chat starts. No scheduling needed. + - Spatial audio + proximity video is the killer feature. Feels like a real office. +- **What's broken:** + - 2D pixel art is deliberately retro — charming but zero immersion. You never feel "present." + - No VR support whatsoever — this is a 2D-only experience + - No spatial memory (everything looks the same from top-down view) + - No body language, no peripheral awareness, no "ambient presence" beyond a dot on a map + - Customization is shallow — you move furniture, not build experiences +- **Key insight:** Gather proved that proximity-based interaction IS the right model. They just execute it in 2D with no path to immersion. + +#### Teamflow +- **What it is:** 2D virtual office platform. "Recreates in-person team presence and spontaneous collaboration." +- **Funding:** Smaller player, est. 2020, 11-50 employees +- **What works:** App integrations (Asana, Salesforce, Figma, Google Docs embedded in rooms). Spatial audio. Screen sharing, whiteboards. +- **What's broken:** + - 2D-only — same limitations as Gather but with less charm + - Much smaller market share than Gather + - No VR. No AI agent concept. + - Pivoting/shrinking — minimal public presence in 2025-2026 + +#### Roam +- **What it is:** AI-powered virtual HQ. "The Office of the Future: People & AIs collaborating." +- **What works:** + - **Only competitor with AI agents as permanent office residents** — closest to spatial-office concept + - AI assistant: observes office activity, schedules meetings, follows up via email/chat, searches meeting transcripts ("Magic Minutes") + - GitHub integration shows PR status on office map. Claims 42% reduction in PR review time internally. + - Saves teams ~$31,500/year by replacing multiple tools (per customer case study) + - Browser-based, no install. 8-minute average meeting duration. +- **What's broken:** + - 2D-only — top-down floor plan. No immersion, no spatial presence signals + - AI agents are assistants, not "teammates" — they schedule and remind, don't inhabit the space + - No VR path — the 2D architecture doesn't translate + - Positioning as "tool consolidator" rather than "office experience" — valuable but not visionary + +#### Sococo (Legacy) +- **What it is:** Original virtual office (founded 2007). Floor plan with rooms, avatars, knock-to-enter. +- **What went wrong:** + - Acquired by Scaleworks, then team absorbed by Virbela — effectively abandoned + - Hostile cancellation practices (60-day written notice, no online cancel) + - "Running costs are high, features simplistic, integrations lacking" — Software Advice reviews + - No VR, no AI, no mobile-first — left behind by Gather and Roam +- **Key lesson:** First-mover advantage means nothing in spatial office. Incumbents can stagnate fast. + +### Event/Experience-Focused (Adjacent) + +#### Spatial.io +- **What it is:** 3D immersive spaces for events, galleries, brand experiences. Web, Mobile, VR. 2M+ creators. +- **What works:** No-code + Unity tools for building 3D spaces. True cross-platform (Web no-download, Mobile, VR). Integration with Office, Google Workspace, Slack. +- **What's broken:** + - Positioned for **events and brand experiences**, not persistent daily office use + - Spaces feel like event venues, not workplaces — you visit, you don't "live" there + - No spatial audio proximity model for casual presence + - Avatars are cartoonish — fun for events, unserious for work + - No AI agent support + +#### Frame (by Virbela) +- **What it is:** Immersive 3D collaboration spaces. Digital twins of physical offices. Web, desktop, mobile, VR. +- **What works:** True cross-platform support including browser. Can build digital twins of real offices. Trello built their NYC office in Frame for events/meetings. Live webcam streaming, screen sharing, whiteboards, synced media. +- **What's broken:** + - Meeting/event focused, not persistent daily workspace + - 3D environments are heavy — not optimized for quick casual drop-ins + - No spatialized ambient presence (you're either in a meeting or you're not) + - No AI agent concept + - Pricing is enterprise sales — not product-led growth + +#### Microsoft Mesh → Teams Immersive Spaces +- **What it is:** Microsoft's 3D immersive meeting spaces, now folded into Teams as "immersive events." +- **What works:** Spatial audio creates natural side conversations. Avatar-based interaction. No-code space building. Native Teams integration — zero friction for existing Teams users. +- **What's broken:** + - **Event-only — not a persistent office.** You enter immersive space from a Teams meeting, you leave when it ends + - Known issues: audio quality inconsistency, artifacts in immersive space (per Microsoft Learn docs) + - Only accessible through Teams View menu during a meeting — not a standalone persistent workspace + - Microsoft killed standalone Mesh in Dec 2025 — folded it into Teams as a feature, not a product + - No browser-only spatial presence — requires Teams client + +--- + +## 2. The Gap: VR/Non-VR Bridge + +### The Core Problem + +Every existing product forces a binary choice: + +| Approach | Products | Limitation | +|-----------|----------|------------| +| VR-only | Immersed, Workrooms, MeetinVR | Excludes >95% of potential users who don't own/wear headsets | +| 2D-only | Gather, Teamflow, Roam, Sococo | Zero immersion. No body language, spatial memory, or peripheral awareness | +| Desktop app + VR | Arthur, Glue | Requires install — no browser path. VR and non-VR modes are different experiences | +| Cross-platform events | Spatial, Frame, Teams Immersive | Meeting/event-focused. No persistent ambient presence | + +### The Specific Gap Nobody Fills + +**A single shared spatial environment where VR users (full immersion, body language via head/hand tracking) and browser users (isometric 3D, keyboard + mouse) share the SAME space with the SAME presence signals.** + +Nobody does this because: +1. VR rendering and browser rendering are fundamentally different pipelines — most companies pick one +2. Isometric perspective for browser users hasn't been explored as a work medium (it's used in games, not productivity) +3. The conventional wisdom is that avatar-based 2D (Gather) is "good enough for browsers" and 3D is "for VR only" + +### Why It Matters Now (2026) + +- **VR market is shrinking:** Global VR headset market declined 12% YoY in 2024 (IDC). Apple Vision Pro production ceased early 2025. Even Meta's $70B bet is pivoting. +- **But VR isn't going away for power users:** Meta Quest 3 holds 74.6% market share. ~1M+ users engage WebXR monthly on Quest. The people who DO use VR for work LOVE it. +- **Browser 3D is finally viable:** WebGPU hit baseline in every major browser (2026). WebXR is an Interop 2026 focus area (browser vendors closing gaps). Three.js is shipping WebGPU support. +- **AI agents change the equation:** An office with AI residents isn't just a meeting tool — it's a persistent team environment. Browser users need to see what AI agents are doing. VR users need to inhabit the same space. + +--- + +## 3. What In-Office Work Provides That Remote Loses + +### The Evidence + +**Microsoft Nature Study (2021)** — Analyzed 61,182 Microsoft employees over the first 6 months of remote work: +- Cross-group collaboration dropped **25%** from pre-pandemic levels +- Collaboration networks became "more static and siloed" +- Communication shifted from synchronous (real-time) to asynchronous (email/IM) +- Employees communicated more with strong ties, fewer weak ties formed + +**MIT Research (2022)** — Analyzed email networks at MIT during pandemic: +- **Weak ties declined by ~38%** — 5,100 missed weak-tie connections in 18 months +- Weak ties are the primary source of novel information and serendipitous innovation +- "A complete return to the workplace would result in a complete recovery of weak ties" (Ratti et al.) + +**Oxford Academic JCMC (2024)** — Socio-technical analysis of remote informal communication: +- "Familiar communication behaviors, norms, and rituals related to face-to-face informal communication change, leading to severe reduction of remote informal communication" +- Remote workers lose the natural rhythm of office interactions — impromptu check-ins, overheard conversations, lunch conversations + +**Nature Human Behaviour (2021)** — Microsoft study published in Nature: +- "Firm-wide remote work caused collaboration networks to become more static and siloed" +- Asynchronous communication increased at the cost of synchronous, rich interaction + +**Frontiers in Computer Science (2021)** — Ambient workspaces research: +- "Remote collaboration involving spatial interactions and enhancing co-presence of colleagues — bringing the rhythmicity and natural flow of the workday to remote settings" +- Recommendation: extend displays to reduce cognitive load, enable spatial interactions + +**arXiv (2025)** — Effects of Remote Working on Scientific Collaboration: +- "Remote work expanded collaboration networks but reduced research impact" +- "Remote and hybrid working hampers research quality by reducing spontaneous, serendipitous in-person interactions" +- "Pandemic accelerated the erosion of weak ties" + +### The Five Signals Remote Work Destroys + +1. **Ambient Presence** — Knowing who's around, who's busy, who's available without asking. The "glance across the room" that tells you whether to interrupt. +2. **Peripheral Awareness** — Overhearing conversations, seeing someone struggle with something you can help with, noticing body language shifts. +3. **Serendipitous Collisions** — Running into someone from another team in the hallway/kitchen. These are the weak-tie interactions that drive innovation. +4. **Spatial Memory** — "I remember where I was standing when we figured that out." Physical space anchors memories and ideas. Zoom grids are memory-killing. +5. **Body Language** — Posture, gesture, eye contact, lean-in/lean-out. Zoom shows faces but loses everything below the shoulders. VR captures more (head/hand tracking) but current avatars are cartoonish. + +### What This Means for spatial-office + +The product must deliver, not describe, these five signals. Competitors claim to "recreate the office" but deliver at most 1-2 of these. Gather does proximity + collision (#3). Teams Immersive does spatial audio (#2 partial). Nobody does all five — and nobody does them across both VR and browser simultaneously. + +--- + +## 4. Browser-First Spatial Rendering — State of the Art + +### WebXR +- **Status:** Production-ready. Meta's Immersive Web SDK (IWSDK) — open-source WebXR toolkit with hand tracking, spatial interactions running in-browser. +- **Scale:** 1M+ monthly WebXR users on Meta Quest (as of Connect 2024) +- **Key development:** WebGPU baseline in all major browsers (2026). Interop 2026 includes WebXR as a focus area — browser vendors actively closing gaps between implementations. +- **Relevance:** WebXR enables a SINGLE codebase for VR (in-headset) and browser (monitor) rendering. The same Three.js scene renders in both contexts. + +### Three.js +- **Status:** Industry standard for browser 3D. Now ships WebGPU support. Powers everything from data viz to games. +- **Isometric perspective:** Three.js supports orthographic cameras natively. An isometric view of a 3D scene is a one-line camera setting. This means the SAME 3D scene that a VR user experiences immersively can be rendered isometrically for a browser user. +- **Key libraries:** enva-xr (WebXR + Three.js with occlusion, lighting, physics), Mattercraft (ZapWorks, "unrivalled choice for WebXR development") + +### Why Isometric for Browser Users + +Isometric perspective (e.g., classic SimCity, Diablo, Monument Valley) offers several advantages over top-down 2D (Gather) or first-person 3D (VR): +- **Spatial memory:** Objects have height, depth, and position. You remember where things are in 3D space. +- **Peripheral awareness:** You can see activity happening across the room without turning. VR users look around; browser users pan/zoom. +- **Lower cognitive load:** Isometric is a "god view" — you see context without being IN it. Good for monitoring multiple things (agent activity, teammate presence). +- **Game-heritage UX:** Millions of people intuitively understand isometric navigation from games. No learning curve. + +### Who's Doing Interesting Work Here + +Nobody is doing isometric 3D for productivity/collaboration. The concept exists in: +- **Games:** Diablo, Hades, Factorio, RimWorld — all isometric, all deeply spatial, all have "base building" metaphors that map to office layout +- **City builders:** SimCity, Cities: Skylines — managing spatial resources with isometric view +- **No productivity tools use this perspective.** The entire field is green. + +--- + +## 5. Agent Visualization — The Unique Angle + +### Current State of Agent Visualization + +**Academic/Research:** +- **Stanford Generative Agents (Smallville):** 25 AI agents in a 2D sandbox world. Agents have daily schedules, interact, form relationships. The visualization is a 2D top-down map with agent sprites. This is the most influential agent visualization project but is purely 2D and research-only. +- **MA3DSG (2026):** Multi-agent 3D scene graph generation — multiple agents exploring and building unified 3D representations. Research-grade, not user-facing. +- **MASON:** Java-based multi-agent simulation library with 2D/3D visualization tools. Academic, not product-grade. + +**Commercial/Tools:** +- **SmythOS:** Agent-based modeling platform. Heat maps, density plots, animated trajectories for agent behavior. Enterprise-focused, not spatial office. +- **DeepNLP Agent Visualization Tool:** Timeline-based visualization of async multi-agent events. 2D, not spatial. +- **AutoGen (Microsoft):** Multi-agent conversation framework. No spatial visualization — purely text/API-based. +- **LangGraph / CrewAI:** Agent orchestration frameworks. Text logs and node graphs. No spatial visualization. + +**The Closest Analog: Roam** +- Roam is the only product putting AI agents in a virtual office context +- But their agents are "assistants" — scheduling, note-taking, reminders +- Agents don't have spatial presence — they're functions, not residents +- Visualized as icons/buttons in a 2D floor plan + +### What spatial-office Can Do Differently + +**AI agents as spatial residents:** +- Each agent has a "desk" or workspace in the isometric office +- Agents have activity states visible through spatial cues: working (pulsing glow), idle, in-conversation, awaiting input +- Browser users can see at a glance what all 9 agents are doing — spatial memory for agent state +- VR users can "walk over" to an agent's workspace and interact +- The 9-agent Scobleclaw system is the PERFECT dogfood — the product is literally the team + +**This is genuinely new.** No product visualizes a multi-agent AI system as a spatial team. The combination of: +1. Isometric 3D for browser users +2. Full VR for headset users +3. AI agents as spatial residents with visual state +4. Human team members also present in the same space + +...is a product category that doesn't exist yet. + +--- + +## 6. Key Risks — What Killed Previous Spatial Office Products + +### Risk 1: Hardware Fragmentation (FATAL) +- **What happened:** Every VR-first product (Workrooms, MeetinVR, Immersed) hit the wall of "not everyone has a headset." VR market is small and shrinking. Apple Vision Pro was a commercial failure (production ceased). Quest 3S is the only bright spot. +- **Mitigation for spatial-office:** Browser-first isometric is the primary experience. VR is an enhancement, not a requirement. This inverts the adoption funnel: everyone can join, power users get more. + +### Risk 2: Overpromising Immersion (Meta's $70B Lesson) +- **What happened:** Meta sold "the metaverse is your new office" when the technology wasn't ready. Uncanny valley avatars, motion sickness, clunky interfaces. The hype-to-reality gap was catastrophic. +- **Mitigation:** Isometric is deliberately NOT immersive — it's informative. It doesn't promise presence; it delivers awareness. VR presence is opt-in, not required. + +### Risk 3: "Good Enough" Incumbents (Zoom/Teams) +- **What happened:** Zoom and Teams already worked. Asking people to adopt a new platform for marginal benefit failed. "Teams and Zoom had already established themselves as good enough for remote collaboration" (Reworked, 2026). +- **Mitigation:** spatial-office isn't replacing Zoom — it's a different category. It's for the BETWEEN meetings, not the meetings themselves. Persistent ambient presence, not scheduled calls. If anything, it integrates with Zoom for formal meetings. + +### Risk 4: Enterprise Sales Death Spiral +- **What happened:** Arthur, Glue, Frame all sell to enterprises. Long sales cycles, procurement hell, low adoption per deal. You need a champion at every company. Product-led growth is the only sustainable model. +- **Mitigation:** Browser-first = no install = self-serve signup. Dogfood with Scobleclaw first. Freemium or team-tier pricing, not enterprise contracts. + +### Risk 5: Novelty Without Retention +- **What happened:** Sococo, early virtual offices — interesting concept, fun for a week, then everyone goes back to Slack. The "gamification" approach (avatars, rooms, knocking) proved shallow. +- **Mitigation:** Agent visualization provides daily utility — you check the office to see what agents are doing, not just to "hang out." The product earns its place in the daily workflow. + +### Risk 6: AI Hype Without Substance +- **What happened:** Arthur pivoted from "VR office" to "AI co-workers" messaging. Roam added AI assistants. These feel bolted-on rather than native. +- **Mitigation:** The Scobleclaw agents ARE the team. This isn't "AI features added to an office app" — it's an office built FOR an AI-human team. The agents have real work output, not just chatbot functionality. + +### What's Different Now (2026) +- **WebGPU + WebXR are production-ready:** Building a cross-platform 3D app in the browser is actually feasible now — it wasn't in 2020-2023 +- **AI agents are real:** Multi-agent systems (AutoGen, CrewAI, LangGraph) are mature. Visualizing them is a new problem people actually have +- **Remote work is permanent:** The "return to office" wars are largely settled. Hybrid/remote is the default for tech. The problem isn't going away +- **VR fatigue is real but VR utility is real too:** People don't want to LIVE in VR, but power users who DO use it report genuine productivity gains. A browser-first product with VR enhancement captures both realities + +--- + +## Competitive Positioning Map + +``` +HIGH IMMERSION (VR) +│ +│ Immersed ● MeetinVR ● +│ Workrooms ☠ (dead) +│ Arthur ● ← spatial-office target zone: +│ Glue ● VR + Browser, Isometric 3D, +│ AI Agents as residents +│ 🎯 +│ Frame ● +│ Spatial.io ● ─────────────── +│ │ +│ Teams Immersive ● │ ← THE GAP +│ (events only) │ +│ │ +│ Gather ● Roam ● │ +│ Teamflow ● Sococo ☠ │ +│ +└────────────────────────────────────── +LOW IMMERSION (2D) BROWSER-FIRST +``` + +--- + +## Recommendations + +1. **Position against Gather, not against VR products.** Gather proved the market ($10M+ revenue). They're the incumbent to displace. "Gather but isometric, with AI teammates" is the elevator pitch. + +2. **Lead with agent visualization.** The unique angle isn't "spatial office" (many have tried) — it's "see your AI team working alongside you." This is a new category. + +3. **Browser-first, VR-enhanced.** Ship the isometric browser experience first. Add WebXR VR mode second. Never require a headset. + +4. **Dogfood publicly.** The Scobleclaw team is the first customer. Show, don't tell. Live-stream the office. Make the product's own development visible. + +5. **Self-serve onboarding.** No demos, no sales calls, no enterprise procurement. Sign up → invite team (or agents) → see the office. < 60 seconds to value. + +6. **Integrate, don't replace.** Zoom/Meet for formal meetings. Slack/Discord for async. spatial-office for the space between — the ambient layer. + +--- + +*Analysis compiled from: WIRED, Nature Human Behaviour, Microsoft Research, MIT/UC Berkeley studies, Frontiers in Computer Science, arXiv, Crunchbase, Tracxn, XR Today, product websites (Immersed, Arthur, Glue, Frame, Roam, Gather, Teamflow, Spatial, Meta), G2 reviews, Reddit r/OculusQuest, TechCrunch, Forrester research.*