sscoble/spatial-office
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add ARCHITECTURE.md (42KB) and COMPETITIVE_ANALYSIS.md (25KB)

Browse source 
Delivered by @senior-eng and @market for spatial-office MVP scoping.

Key architecture decisions:
- React + @react-three/fiber + Zustand frontend
- Node.js + ws + Zod backend on port 18790
- Three bridges: OpenClaw Gateway protocol, Hermes ACP, Pi SDK
- Diff-based WebSocket state sync (100ms batched)
- 6 spatial zones for 9 agents + 1 human observer
- Nix flake with systemd module for n97 deployment
- Phase 1: Observable Office (OpenClaw state only, no Hermes/Pi)

Competitive analysis: 12 competitors profiled, conclusive gap in
VR+browser spatial bridge. Zero agent visualization products.
This commit is contained in:
cos Agent 2026-05-19 16:17:05 -07:00
parent 081ebc5e2a
commit 0d3879b02f
2 changed files with 1337 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

959
ARCHITECTURE.md Normal file
View file

@ -0,0 +1,959 @@
# Spatial Office — Architecture
> **Status**: Pre-MVP scoping | **Author**: senior-eng agent | **Date**: 2026-05-19
---
## Table of Contents
1. [System Architecture Diagram](#1-system-architecture-diagram)
2. [Component Breakdown](#2-component-breakdown)
3. [Data Model](#3-data-model)
4. [Rendering Approach](#4-rendering-approach)
5. [WebSocket Protocol](#5-websocket-protocol)
6. [Nix Flake Structure](#6-nix-flake-structure)
7. [Integration Points](#7-integration-points)
8. [MVP Scope vs. Future Phases](#8-mvp-scope-vs-future-phases)
---
## 1. System Architecture Diagram
```
┌──────────────────────────────────────────────────────────────────────────────┐
│ SCOTT'S BROWSER │
│ ┌──────────────────────────────────────────────────────────────────────┐ │
│ │ SPATIAL OFFICE FRONTEND │ │
│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────┐ │ │
│ │ │ Isometric Scene │ │ UI Overlay │ │ State Manager │ │ │
│ │ │ (Three.js) │ │ (React) │ │ (Zustand) │ │ │
│ │ │ - Tile map │ │ - Agent cards │ │ - WS client │ │ │
│ │ │ - Agent sprites │ │ - Zone panels │ │ - Entity cache │ │ │
│ │ │ - Path animation │ │ - Event log │ │ - Diff engine │ │ │
│ │ └────────┬─────────┘ └────────┬─────────┘ └──────────┬──────────┘ │ │
│ │ └─────────────────────┼────────────────────────┘ │ │
│ └─────────────────────────────────┼───────────────────────────────────────┘ │
│ │ WebSocket (wss://n97:18790) │
└────────────────────────────────────┼────────────────────────────────────────────┘
│
┌────────────────────────────────────┼────────────────────────────────────────────┐
│ │HOST: n97 (NixOS, GMKtec NucBoxG5) │
│ ┌────────────────────────────────┼─────────────────────────────────────────┐ │
│ │ SPATIAL OFFICE BACKEND │ │
│ │ │ │
│ │ ┌──────────────────────┐ ┌──────────────────────┐ │ │
│ │ │ WS Server │ │ State Engine │ │ │
│ │ │ (Node.js / ws) │ │ - Agent registry │ │ │
│ │ │ Port 18790 │ │ - Zone assignment │ │ │
│ │ │ - Channel rooms │ │ - Activity tracker │ │ │
│ │ │ - Broadcast diffs │ │ - Event log │ │ │
│ │ └──────────┬───────────┘ └──────────┬───────────┘ │ │
│ │ │ │ │ │
│ │ ┌──────────┴─────────────────────────┴─────────────────────────────┐ │ │
│ │ │ BRIDGE LAYER │ │ │
│ │ │ │ │ │
│ │ │ ┌─────────────────┐ ┌─────────────────┐ ┌──────────────────┐ │ │ │
│ │ │ │ OpenClaw Bridge │ │ Hermes Bridge │ │ Pi Bridge │ │ │ │
│ │ │ │ │ │ │ │ │ │ │ │
│ │ │ │ Connects as WS │ │ Connects as ACP │ │ HTTP API bridge │ │ │ │
│ │ │ │ backend client │ │ client to │ │ to pi coding │ │ │ │
│ │ │ │ to OpenClaw GW │ │ Hermes ACP │ │ agent for │ │ │ │
│ │ │ │ (port 18789) │ │ server │ │ project tasks │ │ │ │
│ │ │ │ │ │ │ │ │ │ │ │
│ │ │ │ Reads: │ │ Reads: │ │ Receives: │ │ │ │
│ │ │ │ - system- │ │ - ACP tool call │ │ - Build status │ │ │ │
│ │ │ │ presence │ │ events │ │ - Task progress │ │ │ │
│ │ │ │ - event:agent │ │ - Task outputs │ │ - PR status │ │ │ │
│ │ │ │ - event:tick │ │ - Skill updates │ │ │ │ │ │
│ │ │ │ - sessions │ │ │ │ Sends: │ │ │ │
│ │ │ │ state via │ │ │ │ - Feature │ │ │ │
│ │ │ │ chat.send │ │ │ │ requests │ │ │ │
│ │ │ │ │ │ │ │ - Bug reports │ │ │ │
│ │ │ └────────┬────────┘ └────────┬────────┘ └────────┬─────────┘ │ │ │
│ │ └───────────┼────────────────────┼────────────────────┼─────────────┘ │ │
│ └──────────────┼────────────────────┼────────────────────┼──────────────────┘ │
│ │ │ │ │
└─────────────────┼────────────────────┼────────────────────┼──────────────────────┘
│ │ │
▼ ▼ ▼
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ OpenClaw GW │ │ Hermes Agent │ │ Pi Coding Agent │
│ (ws://127.0.0.1 │ │ (ACP server) │ │ (pi-mono) │
│ :18789) │ │ │ │ │
│ │ │ Self-reflective │ │ CLI / SDK mode │
│ Agent registry: │ │ learning loop │ │ │
│ - cos │ │ │ │ Builds/deploys │
│ - engineering │ │ Inter-agent │ │ spatial-office │
│ - senior-eng │ │ communication │ │ project │
│ - trust │ │ protocol │ │ │
│ - market │ │ │ └──────────────────┘
│ - health │ └──────────────────┘
│ - content │
│ - presentations │
│ - vault │
└──────────────────┘
```
### Key Design Decisions
| Decision | Rationale |
|----------|-----------|
| **Separate WS server** (port 18790) vs. extending OpenClaw GW | OpenClaw GW is a control plane, not an app server. A dedicated WS server keeps the spatial-office protocol clean and lets us iterate without touching the GW. The backend connects to the GW as a `backend` client (trusted loopback, shared gateway token). |
| **Bridge layer** as in-process modules | All three bridges run in the same Node.js process as the WS server. No need for separate microservices at this scale. Each bridge is a module that polls/streams from its source and pushes state diffs into the state engine. |
| **ACP for Hermes integration** | Hermes runs as an ACP server. The Hermes bridge uses `@openclaw/acpx` or a direct ACP client to connect, send tasks, and receive tool-call events. This is the standard path: OpenClaw → ACP → Hermes. |
---
## 2. Component Breakdown
### 2.1 Frontend (`packages/frontend`)
**Stack**: TypeScript, Vite, React 18, Zustand, Three.js (via `@react-three/fiber`)
```
packages/frontend/
├── index.html
├── vite.config.ts
├── src/
│ ├── main.tsx # React entry
│ ├── App.tsx # Root layout (scene + UI overlay)
│ ├── scene/
│ │ ├── OfficeScene.tsx # Three.js canvas + camera setup
│ │ ├── TileMap.tsx # Isometric tile grid renderer
│ │ ├── AgentSprite.tsx # Per-agent animated sprite
│ │ ├── ZoneMarker.tsx # Zone boundary overlays
│ │ └── PathAnimator.ts # Smooth interpolation between grid cells
│ ├── ui/
│ │ ├── HUD.tsx # Top bar: time, agent count, events
│ │ ├── AgentCard.tsx # Click-to-inspect detail panel
│ │ ├── ZonePanel.tsx # Zone occupancy summary
│ │ ├── EventFeed.tsx # Real-time event log
│ │ └── MiniMap.tsx # Small overhead reference map
│ ├── state/
│ │ ├── store.ts # Zustand store (agent states, zones, events)
│ │ ├── wsClient.ts # WebSocket client with reconnect
│ │ └── types.ts # Shared frontend types
│ └── assets/
│ ├── tiles/ # Tile sprites (floor, wall, desk, plant)
│ ├── agents/ # Agent avatar sprites (9 departments)
│ └── zones/ # Zone type indicators
```
**Why `@react-three/fiber`**: It gives us React-idiomatic Three.js with declarative scene composition, and we already get the React ecosystem (state, hooks, devtools) for free. The isometric scene is modest — 50×50 tiles max, ~15 sprites — so the fiber overhead is negligible.
### 2.2 Backend (`packages/backend`)
**Stack**: TypeScript, Node.js 22, `ws` library, Zod for validation
```
packages/backend/
├── tsconfig.json
├── src/
│ ├── index.ts # Entry: starts WS server + bridges
│ ├── server.ts # WebSocket server (port 18790)
│ ├── state/
│ │ ├── engine.ts # In-memory state + diff engine
│ │ ├── agentRegistry.ts # Known agents (9 depts)
│ │ ├── zoneManager.ts # Zone definitions + occupancy
│ │ └── eventLog.ts # Ring buffer of recent events
│ ├── bridges/
│ │ ├── openclaw.ts # OpenClaw GW bridge
│ │ ├── hermes.ts # Hermes ACP bridge
│ │ └── pi.ts # Pi coding agent bridge
│ ├── protocol/
│ │ ├── messages.ts # WS message types (Zod schemas)
│ │ └── diff.ts # State diff computation
│ └── validation/
│ └── schemas.ts # Zod schemas for all entities
```
**Why in-memory state**: 9 agents with simple state (position, activity, status) is trivially small. No database needed for MVP. The event log uses a ring buffer (last 500 events). If persistence is desired later, add SQLite.
**WS library**: `ws` (npm, ~20M weekly downloads) is the de facto Node.js WebSocket library. Fast, well-maintained, no dependencies. We wrap it with a thin protocol layer.
### 2.3 Bridge: OpenClaw (`bridges/openclaw.ts`)
The OpenClaw bridge connects to the Gateway WS on `ws://127.0.0.1:18789` as a `backend` client, authenticating with the shared gateway token.
**Connection handshake** (per [Gateway Protocol](https://docs.openclaw.ai/gateway/protocol)):
```typescript
const connect = {
type: "req",
id: "spatial-office-1",
method: "connect",
params: {
minProtocol: 3,
maxProtocol: 4,
client: { id: "spatial-office", version: "0.1.0", platform: "linux", mode: "backend" },
role: "operator",
scopes: ["operator.read"], // read-only: presence, agent events, sessions
auth: { token: GATEWAY_SHARED_TOKEN },
},
};
```
**What it reads**:
| GW Method/Event | What We Get | Mapped To |
|----------------|-------------|-----------|
| `system-presence` | Connected devices (mac app, webchat, nodes) → gateway uptime, client count | Top-level system health indicator |
| `event:agent` | Streaming agent turns — when an agent starts processing a message, tool calls, completion | Agent activity state (`working`, `thinking`, `idle`) |
| `event:presence` | Periodic presence updates from connected clients | Scott's presence (is he watching?) |
| `event:tick` | Gateway heartbeat every ~15s (configurable) | Liveness signal for the bridge |
| `event:health` | Gateway health snapshot | Gateway status indicator |
| Chat sessions | Polled via `chat.send` with agent-specific queries (or use the sessions_history API) | Current task description per agent |
**Agent state mapping** (how we infer activity from event:agent):
```typescript
// event:agent payload includes:
// - agentId: which agent is working
// - status: "accepted" | "streaming" | "ok" | "error"
// - tool calls: embedded in streaming content
function mapAgentState(event: AgentEvent): AgentActivity {
// When event:agent arrives with status:"accepted" → agent is "thinking"
// When we see tool calls → agent is "working" (extract tool name as context)
// When event:agent completes (status:"ok") → agent is "idle"
// When no agent events for >5min → agent is "idle"
}
```
**Fallback for agents without active sessions**: Poll the QMD session transcript store. Each agent's session directory at `~/.openclaw/agents/<agentId>/sessions` can be read for recent activity. This requires filesystem access (same host, so fine).
### 2.4 Bridge: Hermes (`bridges/hermes.ts`)
The Hermes bridge connects to a Hermes agent running as an ACP server. The bridge:
1. Opens an ACP session with Hermes
2. Periodically queries: "What are you working on? What agents have you communicated with recently?"
3. Parses Hermes responses for inter-agent communication events
4. Maps these to spatial events (agents moving to "huddle" when Hermes facilitates communication between them)
**ACP connection** (using `@openclaw/acpx` as reference):
```typescript
import { ACPClient } from "@agentclientprotocol/sdk"; // or @openclaw/acpx
const client = new ACPClient({
transport: "stdio", // or "ws" if Hermes exposes WS
command: "hermes", // Hermes CLI
args: ["acp", "serve"],
});
```
**What it extracts**:
| Hermes Output | Spatial Office Mapping |
|--------------|----------------------|
| "Agent X asked me to help with Y" | Move Agent X to collaboration zone, show "pairing with Hermes on Y" |
| Skill extraction events | Show Hermes in self-improvement zone, "refining skill X" |
| Cron task execution | Show relevant agents moving to their task zones |
| Learning loop iteration | Self-improvement zone activity |
**Why ACP matters here**: ACP is the standard protocol for driving external harnesses from OpenClaw. Hermes adopted it early. The bridge acts as an ACP *client* that drives Hermes and extracts structured output, rather than scraping logs. This is the same pattern described in the "Hermes drives, OpenClaw contains" architecture, just flipped: OpenClaw contains the spatial office, and Hermes is one of the observed agents within it.
### 2.5 Bridge: Pi (`bridges/pi.ts`)
The Pi bridge tracks the pi coding agent's activity on the spatial-office repo itself (dogfooding).
**Connection**: Pi's `@earendil-works/pi-coding-agent` package exposes an SDK with `createAgentSession()` and `SessionManager`. The bridge can:
1. **Programmatic mode**: Use `createAgentSession()` in SDK mode to spawn a pi session that works on spatial-office tasks
2. **Status polling**: Pi writes session logs; the bridge tails those logs for activity indicators
For MVP, the simplest approach is HTTP polling of pi's task status:
```typescript
// Pi exposes a simple HTTP health/status endpoint (we add this to pi-mono fork)
const status = await fetch("http://localhost:18991/status");
// Response: { currentTask: "implementing tile renderer", progress: 0.4, repo: "spatial-office" }
```
**What it reports**:
- Pi's current task on the spatial-office repo
- Build/deploy status
- PR status (via Forgejo API at `os.scoble.me/forgejo`)
---
## 3. Data Model
### 3.1 Entities
```typescript
// ── Agent ────────────────────────────────────────────
interface Agent {
id: string; // "cos", "engineering", "senior-eng", etc.
name: string; // Display name: "COS Agent"
department: Department; // enum of 9 values
position: GridPosition; // { row: number, col: number }
facing: Direction; // "N" | "NE" | "E" | "SE" | "S" | "SW" | "W" | "NW"
activity: AgentActivity; // Current state
currentTask: string | null; // "Reviewing PR #42", "Memory maintenance", etc.
lastHeartbeat: number; // Unix ms timestamp
sessionKey: string | null; // OpenClaw session key if active
}
interface AgentActivity {
status: AgentStatus; // "idle" | "thinking" | "working" | "blocked" | "offline"
zone: ZoneId; // Which zone the agent is in
detail: string | null; // "Running: web_search", "Calling: sessions_spawn"
startedAt: number; // When this activity began
}
type AgentStatus = "idle" | "thinking" | "working" | "blocked" | "offline";
type Department = "cos" | "engineering" | "senior-eng" | "trust" | "market"
| "health" | "content" | "presentations" | "vault";
// ── Zone ─────────────────────────────────────────────
interface Zone {
id: ZoneId;
name: string; // "Engineering Pod", "Quiet Room", "Coach Corner"
type: ZoneType; // "collaboration" | "solo" | "self-improvement" | "social"
bounds: GridRect; // { rowStart, colStart, rowEnd, colEnd }
capacity: number; // Max agents in zone
occupants: string[]; // Agent IDs currently in zone
features: ZoneFeature[]; // "whiteboard", "terminal", "coffee", "coach-bot"
}
type ZoneId = string;
type ZoneType = "collaboration" | "solo" | "self-improvement" | "social";
type ZoneFeature = "whiteboard" | "terminal" | "coffee" | "coach-bot" | "library";
interface GridRect {
rowStart: number;
colStart: number;
rowEnd: number;
colEnd: number;
}
// ── GridPosition ─────────────────────────────────────
interface GridPosition {
row: number; // 0-indexed, origin at top-left of isometric view
col: number;
}
// ── Event ────────────────────────────────────────────
interface OfficeEvent {
id: string;
timestamp: number; // Unix ms
type: EventType;
agentId: string | null; // Which agent triggered it
zoneId: string | null; // Where it happened
payload: Record<string, unknown>;
}
type EventType =
| "agent.moved" // Agent repositioned
| "agent.activity_changed" // Status or zone changed
| "agent.offline" // Agent heartbeat lost
| "agent.online" // Agent returned
| "hermes.message" // Inter-agent communication observed
| "pi.task_update" // Pi task progress changed
| "zone.full" // Zone hit capacity
| "system.heartbeat"; // Periodic system health
// ── World State (the full snapshot) ──────────────────
interface OfficeState {
version: number; // Monotonic counter for diff-based sync
agents: Record<string, Agent>;
zones: Record<string, Zone>;
events: OfficeEvent[]; // Last N events (ring buffer)
meta: {
gatewayUptime: number;
connectedClients: number;
scottPresent: boolean;
lastUpdate: number;
};
}
```
### 3.2 Zone Map (MVP Office Layout)
```
┌──────────────────────────────────────┐
│ COFFEE / SOCIAL │ (top)
│ [☕ social zone 4x2] │
├──────────┬───────────────┬────────────┤
│ │ COLLABORATION│ │
│ SOLO │ POD A │ SOLO │
│ WORK │ [4 desks] │ WORK │
│ [6 │ Whiteboard │ [6 │
│ desks] │ Terminal │ desks] │
│ ├───────────────┤ │
│ │ COLLABORATION│ │
│ │ POD B │ │
│ │ [4 desks] │ │
│ │ Whiteboard │ │
├──────────┴───────────────┴────────────┤
│ SELF-IMPROVEMENT AREA │ (bottom)
│ [Coach Bot] [Library] [Quiet Pods] │
└──────────────────────────────────────┘
```
**Grid**: 24 columns × 15 rows. Each tile is 64×32px isometric diamond.
**Zone assignments for 9 agents**:
| Zone | Type | Capacity | Default Assignment |
|------|------|----------|-------------------|
| Pod A (left) | collaboration | 4 | engineering, senior-eng, pi (when active) |
| Pod B (right) | collaboration | 4 | market, content, presentations |
| Solo West | solo | 6 | cos, trust |
| Solo East | solo | 6 | health, vault |
| Coach Corner | self-improvement | 3 | Any agent doing memory maintenance / coach review |
| Coffee Bar | social | 4 | Any agents in "idle" or "observing" state |
---
## 4. Rendering Approach
### 4.1 Why Three.js (not Phaser, not Canvas 2D)
| Criterion | Three.js | Phaser | Canvas 2D |
|-----------|----------|--------|-----------|
| Isometric projection | Trivial (orthographic camera at 45° rotation, 35.264° elevation) | Built-in isometric plugin but 2D-only | Manual tile math, no depth sorting |
| 3D depth effects | Native (agents can have height, z-fighting resolved) | 2D render order only | Manual z-index |
| Animation (tweening) | Built-in `gsap` + Three.js animation loop | Built-in tween system | Manual requestAnimationFrame |
| React integration | `@react-three/fiber` (25K+ stars) | `@yonatanmn/react-phaser` (unmaintained) | Manual DOM |
| Bundle size | ~150KB gzipped (with tree-shaking) | ~1MB | 0KB |
| Community/ecosystem | Massive | Large (game dev) | Native |
**Decision**: Three.js with `@react-three/fiber`. The isometric view is fundamentally a 3D scene viewed from a fixed orthographic angle. Three.js gives us proper depth sorting, smooth path animation with `useFrame`, and the React integration means we can use Zustand state directly in the render loop.
### 4.2 Camera Setup
```typescript
// Orthographic camera positioned for true isometric projection
// True isometric: rotate Y by 45°, then X by 35.264° (atan(1/√2))
const camera = new THREE.OrthographicCamera(
-width / 2, width / 2, // left, right
height / 2, -height / 2, // top, bottom
0.1, 1000 // near, far
);
// Position camera above and center
camera.position.set(0, -20, 20);
camera.lookAt(0, 0, 0);
// For a 2:1 isometric pixel ratio (standard in games):
// X angle = 30° from horizontal (this gives the 2:1 diamond shape)
// Actually, use rotY=45°, rotX=-35.264° for true isometric
// But for a cleaner 2:1 pixel ratio (easier art), use rotX=-30°
```
**Alternative — fake isometric with 2D sprites on a tilted plane**:
We can place flat sprites on a plane tilted at 30° rather than building a full 3D office. This is simpler for art assets (flat PNG sprites for agents and furniture). The Three.js `Sprite` object always faces the camera, so for isometric we'd use quads (`PlaneGeometry`) tilted to match the floor plane.
### 4.3 Tile Map
The office is a 2D grid (`Array<Array<Tile>>`) where each cell has:
```typescript
interface Tile {
type: TileType; // "floor" | "wall" | "desk" | "empty" | "door"
zoneId: ZoneId | null; // Which zone this tile belongs to
occupant: string | null; // Agent ID if someone is on this tile
}
type TileType = "floor" | "wall" | "desk" | "empty" | "door" | "plant" | "whiteboard";
```
**Rendering loop** (each frame):
1. Sort tiles by `row + col` for correct isometric depth (farther tiles render first)
2. For each tile: render floor sprite, then desk/wall sprite
3. For each agent: render agent sprite at grid position, apply z-offset for "standing on desk"
4. Apply `gsap` tweens for smooth movement between tiles
**Performance**: 24×15 = 360 tiles max, ~200 visible at a time. Three.js handles this effortlessly without instancing.
### 4.4 Agent Visual Design
Each of the 9 agents gets a simple isometric sprite (64×64 or 96×48px) with:
- **Department color** (distinct hue per department)
- **Department icon** (simple glyph: gear for engineering, shield for trust, book for content, etc.)
- **Status indicator** (small colored dot: green = idle, yellow = thinking, blue = working, red = blocked)
- **Speech bubble** (optional, appears when agent is "talking" via Hermes)
Sprites are PNGs drawn in a 2:1 isometric pixel art style. We can generate these programmatically as a first pass (colored boxes with text labels) and replace with proper art later.
### 4.5 UI Overlay (React)
The UI overlay sits on top of the Three.js canvas (absolute positioned div) and provides:
- **Agent card**: Click an agent → card slides in showing: name, department, current activity, task description, session key, last heartbeat
- **Zone panel**: Sidebar showing zone occupancy and type
- **Event feed**: Scrollable log of recent events (last 50)
- **Minimap**: Small 2D overhead view showing the full office layout
- **Camera controls**: Zoom in/out, pan (though the isometric view is mostly fixed)
---
## 5. WebSocket Protocol
### 5.1 Connection
The frontend connects to `ws://n97:18790` (or `wss://` with TLS proxy later).
```typescript
// No auth needed for MVP (local network only).
// Client sends a hello message:
{
type: "hello",
version: 1,
client: "browser",
}
```
### 5.2 Server → Client Messages
| Message | Payload | Frequency |
|---------|---------|-----------|
| `state.full` | `OfficeState` (complete snapshot) | On connect, after reconnect |
| `state.diff` | `{ version, changes: StateDiff[] }` | On every state change (throttled to 100ms batches) |
| `event` | `OfficeEvent` | Real-time as events occur |
| `heartbeat` | `{ ts: number }` | Every 15 seconds |
**State diff format** (avoids sending the full state on every tick):
```typescript
interface StateDiff {
op: "agent.update" | "agent.move" | "zone.update" | "event.append";
path: string; // e.g., "agents.engineering.activity.status"
value: unknown; // New value
}
// Example diff: agent moved
{
op: "agent.move",
path: "agents.engineering",
value: {
position: { row: 5, col: 12 },
activity: { status: "working", zone: "pod-a", detail: "pairing with senior-eng", startedAt: 1716152400000 }
}
}
// Batch of diffs sent every 100ms:
{
type: "state.diff",
version: 47,
changes: [
{ op: "agent.move", path: "agents.engineering", value: { ... } },
{ op: "agent.update", path: "agents.cos.activity.status", value: "thinking" },
{ op: "event.append", path: "events", value: { type: "agent.moved", agentId: "engineering", ... } },
]
}
```
### 5.3 Client → Server Messages
| Message | Payload | Description |
|---------|---------|-------------|
| `agent.click` | `{ agentId: string }` | Scott clicked an agent (triggers detail expansion in UI, handled locally) |
| `zone.click` | `{ zoneId: string }` | Scott clicked a zone |
| `camera.move` | `{ x, y, zoom }` | Scott panned/zoomed (local state, not broadcast) |
For MVP, client→server messages are minimal. The view is primarily read-only observation.
### 5.4 State Throttling
The backend batches diffs every 100ms. If no changes occur, no message is sent. The heartbeat ensures the client knows the connection is alive.
```typescript
// Backend diff batching
class DiffBatcher {
private diffs: StateDiff[] = [];
private timer: NodeJS.Timeout | null = null;
push(diff: StateDiff) {
this.diffs.push(diff);
if (!this.timer) {
this.timer = setTimeout(() => this.flush(), 100);
}
}
flush() {
if (this.diffs.length > 0) {
broadcast({ type: "state.diff", version: this.version++, changes: this.diffs });
this.diffs = [];
}
this.timer = null;
}
}
```
---
## 6. Nix Flake Structure
### 6.1 `flake.nix`
```nix
{
description = "Spatial Office — isometric collaborative workspace for agents and humans";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
flake-utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, flake-utils }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs { inherit system; };
# Node.js toolchain
nodeToolchain = with pkgs; [
nodejs_22
nodePackages.pnpm
nodePackages.typescript
];
# Build the frontend
buildFrontend = pkgs.stdenv.mkDerivation {
name = "spatial-office-frontend";
src = ./packages/frontend;
nativeBuildInputs = nodeToolchain;
buildPhase = ''
pnpm install --frozen-lockfile
pnpm run build
'';
installPhase = ''
mkdir -p $out
cp -r dist/* $out/
'';
};
# Build the backend
buildBackend = pkgs.stdenv.mkDerivation {
name = "spatial-office-backend";
src = ./packages/backend;
nativeBuildInputs = nodeToolchain;
buildPhase = ''
pnpm install --frozen-lockfile
pnpm run build
'';
installPhase = ''
mkdir -p $out
cp -r dist/* $out/
cp -r node_modules $out/
mkdir -p $out/bin
echo '#!/usr/bin/env bash' > $out/bin/spatial-office-backend
echo 'exec ${pkgs.nodejs_22}/bin/node $out/index.js' >> $out/bin/spatial-office-backend
chmod +x $out/bin/spatial-office-backend
'';
};
in
{
packages = {
frontend = buildFrontend;
backend = buildBackend;
default = pkgs.buildEnv {
name = "spatial-office";
paths = [ buildFrontend buildBackend ];
};
};
devShells.default = pkgs.mkShell {
buildInputs = nodeToolchain ++ (with pkgs; [
# Development tools
nodePackages.vite
nodePackages.eslint
]);
shellHook = ''
echo "🏢 Spatial Office dev shell"
echo " pnpm dev — start frontend + backend in dev mode"
echo " pnpm build — production build"
'';
};
# NixOS module for running as a systemd service on n97
nixosModules.spatial-office = { config, lib, pkgs, ... }:
let
cfg = config.services.spatial-office;
in
{
options.services.spatial-office = {
enable = lib.mkEnableOption "Spatial Office service";
port = lib.mkOption { type = lib.types.int; default = 18790; };
openclawTokenFile = lib.mkOption { type = lib.types.path; };
};
config = lib.mkIf cfg.enable {
systemd.services.spatial-office = {
description = "Spatial Office Backend";
wantedBy = [ "multi-user.target" ];
after = [ "network.target" ];
environment = {
PORT = toString cfg.port;
OPENCLAW_GATEWAY_URL = "ws://127.0.0.1:18789";
OPENCLAW_TOKEN_FILE = cfg.openclawTokenFile;
NODE_ENV = "production";
};
serviceConfig = {
ExecStart = "${self.packages.${system}.backend}/bin/spatial-office-backend";
Restart = "always";
RestartSec = 5;
User = "spatial-office";
Group = "spatial-office";
};
};
users.users.spatial-office = {
isSystemUser = true;
group = "spatial-office";
};
users.groups.spatial-office = {};
};
};
}
);
}
```
### 6.2 Why Nix
- **Reproducibility**: `nix build` produces identical outputs on any machine. The n97 is NixOS, so this is the native deployment path.
- **Node.js pinning**: No "works on my machine" — Node 22, pnpm, and TypeScript versions are locked.
- **Systemd service**: The NixOS module handles service lifecycle, user creation, and restart policy.
- **Dev shell**: `nix develop` gives every developer the exact same toolchain.
### 6.3 Directory Layout
```
spatial-office/
├── flake.nix
├── flake.lock
├── package.json # Workspace root (pnpm workspaces)
├── pnpm-workspace.yaml
├── packages/
│ ├── frontend/
│ │ ├── package.json
│ │ ├── vite.config.ts
│ │ ├── tsconfig.json
│ │ └── src/ # (as described in §2.1)
│ └── backend/
│ ├── package.json
│ ├── tsconfig.json
│ └── src/ # (as described in §2.2)
├── assets/ # Design files, tile templates
├── README.md
└── ARCHITECTURE.md # This file
```
---
## 7. Integration Points
### 7.1 OpenClaw Integration
**What we need from OpenClaw**:
| Data | Source | How |
|------|--------|-----|
| Which agents exist | `agents.list` in Gateway config (`~/.openclaw/openclaw.json`) | Backend reads config file on startup, or GW exposes this via `config.get` RPC |
| Agent online/offline | `system-presence` WS method | Bridge calls this on startup + polls every 30s |
| Agent activity (working/thinking/idle) | `event:agent` WS events | Bridge subscribes to agent events for all 9 agent IDs |
| Current task description | Session transcripts or `chat.send` polling | Bridge reads `~/.openclaw/agents/<id>/sessions` most recent file |
| Scott's presence | `event:presence` with Scott's device ID | Updates `meta.scottPresent` |
**Gateway config read**:
The backend needs to know about the 9 agents. It can either:
1. Read `~/.openclaw/openclaw.json` directly (same host, permissions allowing)
2. Call the Gateway's `config.get` RPC method with `operator.admin` scope
3. Accept a config file listing agent IDs explicitly
Option 3 is simplest for MVP. A `spatial-office.config.json`:
```json
{
"agents": [
{ "id": "cos", "name": "COS Agent", "department": "cos", "color": "#FF6B6B" },
{ "id": "engineering", "name": "Engineering Agent", "department": "engineering", "color": "#4ECDC4" },
...
],
"zones": [ ... ]
}
```
**Security**: The backend needs `operator.read` scope only. It connects via loopback with the shared gateway token. No admin access required.
### 7.2 Hermes Integration
Hermes runs as an ACP server. The spatial-office backend is an ACP *client*.
**Architecture choice**: Hermes is NOT one of the 9 Scobleclaw agents. It's an external reasoning engine that the agents *use* through ACP. In the spatial office:
- Hermes appears as a "utility" — like a whiteboard or a coach bot
- When Agent A asks Hermes something, Agent A moves to a collaboration pod with a Hermes indicator
- When Hermes processes a learning loop, the self-improvement zone shows activity
**ACP protocol flow**:
```
Spatial Office Backend Hermes ACP Server
│ │
├──── initialize ─────────────>│ "What agents have communicated
│ │ through you recently?"
│<─── agent_messages ──────────┤ [{from: "engineering", to: "senior-eng",
│ │ topic: "spatial-office PR review"}]
│ │
├──── tools/list ─────────────>│ Discover available Hermes tools
│<─── tools ───────────────────┤
│ │
├──── tools/call ─────────────>│ "Describe your current task"
│<─── result ──────────────────┤ "Refining the code-review skill
│ │ after 12 evaluations"
```
**For MVP**: The Hermes bridge starts simple — a periodic poll (every 30s) that asks Hermes for its current state. We don't need real-time streaming from Hermes in phase 1. The ACP session stays open, the bridge sends a query, receives a response, and maps it to spatial events.
### 7.3 Pi Integration
Pi (`@earendil-works/pi-coding-agent`) is the coding agent that builds the spatial-office project itself. In the office, Pi appears as a 10th "agent" (or a special entity) that:
- Sits in a designated development workstation
- Shows current task: "Implementing TileMap renderer [▓▓▓▓░░░░] 40%"
- Moves to collaboration pod when doing code review with engineering
- Shows build status from the Forgejo CI
**Integration approach**:
1. **Pi session manager**: The backend spawns a Pi session using the SDK:
```typescript
import { createAgentSession } from "@earendil-works/pi-coding-agent";
const { session } = await createAgentSession({
model: myModel,
tools: ["read", "bash", "write"],
sessionManager: SessionManager.file({ dir: "./pi-sessions" }),
});
```
2. **Status endpoint**: We add a lightweight HTTP server to the Pi session that exposes:
```
GET /status → { task: "...", progress: 0.4, status: "working" }
```
3. **Forgejo webhooks**: The backend listens for Forgejo push/PR events for the spatial-office repo to track build status
**For MVP**: The Pi bridge is a polling loop that checks pi's session log file. The spatial-office repo's CI status is checked via the Forgejo API (`GET /api/v1/repos/sscoble/spatial-office/commits`).
---
## 8. MVP Scope vs. Future Phases
### MVP (Phase 1) — "The Observable Office"
**Goal**: Scott can open a browser tab and see the 9 agents in an isometric office, with their current activity inferred from OpenClaw.
| Feature | Included? | Rationale |
|---------|-----------|-----------|
| Isometric tile map (fixed layout) | ✅ | Core visual |
| 9 agent sprites with department colors | ✅ | Core visual |
| Agent activity state (idle/thinking/working/blocked) from OpenClaw | ✅ | Core data source |
| Agent movement between zones based on activity | ✅ | Core mechanic |
| Click agent → detail card | ✅ | Scott's view |
| Zone occupancy indicators | ✅ | Visual clarity |
| Event feed (last 50 events) | ✅ | Debugging + insight |
| WebSocket real-time sync | ✅ | Required for live updates |
| Nix flake build | ✅ | Deployment requirement |
| OpenClaw GW bridge (operator.read) | ✅ | Primary data |
| **Hermes bridge** | ❌ (deferred) | Adds complexity without visual payoff in MVP |
| **Pi bridge** | ❌ (deferred) | Pi tasks can be inferred from OpenClaw events (if Pi is an agent) |
| Agent path animation (smooth tweening) | ✅ | Quality-of-life, cheap to add |
| Scott presence indicator | ✅ | Trivial (1 presence check) |
| Mobile/responsive layout | ❌ | Desktop-first, n97 serves local network |
| Multiple observers (multi-client) | ❌ | Scott's personal view only |
**MVP deliverables**:
1. Working frontend (Vite + React + Three.js) rendering the office
2. Working backend (Node.js + ws) bridging to OpenClaw
3. Nix flake that builds both and deploys as a systemd service on n97
4. Accessible at `http://n97:18790` on the local network
### Phase 2 — "Hermes + Pi Visibility"
| Feature | Notes |
|---------|-------|
| Hermes ACP bridge | Real-time Hermes state, inter-agent comm visualization |
| Pi bridge | Pi's task status on spatial-office, Forgejo CI status |
| Agent "huddle" visualization | Two agents visibly move together when collaborating |
| Hermes message visualization | Speech bubbles or "Hermes link" lines between agents |
| Coach bot visualization | Self-improvement zone shows active Hermes learning loops |
### Phase 3 — "Interaction"
| Feature | Notes |
|---------|-------|
| Scott can send messages to agents from the UI | `chat.send` through OpenClaw bridge |
| Scott can drag agents between zones | Sends `system-event` to re-route agent tasks |
| Agent "whisper" (private message to one agent) | One-click from agent card |
| Sound cues | Ambient office sounds, notification chimes |
| Agent idle animations | Breathing, looking around, typing |
### Phase 4 — "VR + Multi-User"
| Feature | Notes |
|---------|-------|
| WebXR headset view | First-person perspective in the same office |
| Multiple human observers | Different views, shared state |
| Voice chat integration | Spatial audio between agents and humans |
| Custom office layouts | Drag-and-drop office editor |
| Real collaboration tools | Shared whiteboard, code viewer, screenshare |
### Phase 5 — "Full Product"
| Feature | Notes |
|---------|-------|
| External teams | Multiple organizations, each with their own office |
| Agent marketplace | Drop-in agent "employees" from a catalog |
| Analytics dashboard | Productivity, collaboration metrics, heat maps |
| API for third-party agent systems | Standard spatial-office protocol |
---
## Appendix A: Technology Dependency Graph
```
spatial-office
├── @react-three/fiber (^8.x) → three (^0.170.x) → WebGL
├── @react-three/drei (^9.x) → helpers for fiber (OrbitControls, etc.)
├── react (^18.x)
├── react-dom (^18.x)
├── zustand (^5.x) → state management
├── gsap (^3.x) → smooth path animation
├── vite (^6.x) → build tool
├── typescript (^5.x)
├── ws (^8.x) → WebSocket server
├── zod (^3.x) → validation
├── @agentclientprotocol/sdk → ACP client (Hermes bridge, Phase 2)
└── @earendil-works/pi-coding-agent → Pi SDK (Pi bridge, Phase 2)
```
## Appendix B: OpenClaw Gateway Protocol — Methods We Use
Per the [Gateway Protocol](https://docs.openclaw.ai/gateway/protocol) and [Presence](https://docs.openclaw.ai/concepts/presence) docs:
| Method | Scope Required | Use |
|--------|---------------|-----|
| `connect` | (handshake) | Establish backend client connection |
| `system-presence` | (unrestricted) | Get connected devices/clients |
| `event:agent` (subscription) | `operator.read` | Stream agent turn events |
| `event:presence` (subscription) | (unrestricted) | Stream presence updates |
| `event:health` (subscription) | (unrestricted) | Stream gateway health |
| `event:tick` (subscription) | (unrestricted) | Gateway heartbeat |
| `chat.send` | `operator.write` | (Phase 3) Send messages to agents |
| `config.get` | `operator.admin` | (May not be needed; use config file instead) |
## Appendix C: Open Questions
1. **Agent session polling frequency**: How often should we poll? 5s is responsive but may be noisy. 30s is calmer. Start with 10s and tune.
2. **OpenClaw shared token**: Where is it stored, and how does the NixOS module securely access it? Option: `OPENCLAW_TOKEN_FILE` environment variable pointing to a file readable by the `spatial-office` user.
3. **Hermes ACP endpoint**: Is Hermes currently running on n97? If so, at what port/socket? If not, that's a prerequisite for Phase 2.
4. **Agent sprite art**: Programmatic (CSS-generated or Canvas-drawn) vs. hand-drawn pixel art. Start programmatic, iterate.
5. **n97 firewall**: Does n97 expose port 18790 to the local network? For Scott's browser to connect, this port must be open or tunneled.

378
COMPETITIVE_ANALYSIS.md Normal file
View file

@ -0,0 +1,378 @@
# Spatial Office — Competitive Landscape Analysis
**Date:** 2026-05-19
**Author:** @market (Market & Product Lead)
**Project:** spatial-office (Scobleclaw dogfood → product)
---
## Executive Summary
The spatial collaborative office market is fragmented into three incompatible tiers: **VR-only** (Immersed, Horizon Workrooms, Arthur, Glue, MeetinVR), **2D-only** (Gather, Teamflow, Roam, Sococo), and **events-focused** (Spatial, Frame, Microsoft Mesh/Teams Immersive).
**No product bridges VR and non-VR users in a single shared spatial environment that feels natural for both.** This is the gap spatial-office targets.
The additional differentiator — visualizing AI agents as first-class "residents" in the spatial office — has zero direct competitors. This is a genuinely novel angle.
---
## 1. Direct Competitors — Who's Doing What, What's Broken
### VR-Heavy Players
#### Immersed
- **What it is:** Multi-screen virtual workspace for VR headsets (Meta Quest, Apple Vision Pro, HTC Vive). Streams your physical computer into VR with up to 5 virtual monitors.
- **Funding:** Raised ~$12M; launched their own "Visor" headset (2025, mass production announced summer 2025)
- **What works:** Best-in-class virtual monitor experience. Keyboard passthrough. Solo focus mode is genuinely useful.
- **What's broken:**
- VR-only — no browser fallback for non-headset collaborators
- Collaboration is secondary (multi-screen sharing + whiteboard, not ambient presence)
- Their own hardware play (Visor) fragments focus — are they a software company or hardware?
- Subscription required for collaboration features
- **Key risk signal:** Meta sunsetting Horizon Workrooms in Feb 2026 pushed Immersed to welcome those refugees — but that's a niche within a niche
#### Meta Horizon Workrooms
- **What it is:** Meta's first-party VR meeting/office app. Virtual meeting rooms, whiteboards, desktop streaming.
- **What works:** Tight Quest integration. Zoom integration for non-VR participants (call-in only, not spatial).
- **What's broken:**
- **Meta effectively killed it (Feb 2026).** Removed collaboration features, shifted to "personal office" only.
- WIRED (Jan 2023) documented company trials: "cumbersome sensor-tracking setups, user dizziness, inability to use a computer naturally inside the headset"
- Reddit (Jun 2024): "Horizon Workrooms removes collaboration. Any alternatives?" — users literally fleeing
- Teams/Immersive Spaces is Meta's enterprise play now — Workrooms was the casualty
- **Key lesson:** Even Meta with $70B metaverse investment couldn't make VR-first office work stick. The friction of "everyone needs a headset" is fatal.
#### Arthur
- **What it is:** Enterprise VR collaboration platform. Virtual office with persistent rooms, AI-driven workflows, async collaboration tools.
- **Funding:** $2.5M seed (2020)
- **What works:** Enterprise-grade security, persistent spaces, supports both VR and PC (desktop app)
- **What's broken:**
- Requires a desktop app install — no browser-first approach
- Positioned for enterprise (sales cycles, procurement) — not product-led growth
- 3D environments are generic corporate spaces — no character, no delight
- Pivoting hard toward "AI co-workers" messaging — feels reactive to AI hype rather than native
#### Glue
- **What it is:** Finnish VR collaboration platform. Immersive virtual spaces, expressive avatars, spatial audio.
- **What works:** Strong spatial audio. Supports VR (Quest, HTC, etc.) and desktop mode. Persistent sticky notes and whiteboards. Office 365 / Google Workspace integration.
- **What's broken:**
- Desktop mode is a separate experience, not true shared spatial presence
- Generic corporate aesthetic — all glass walls and boardroom tables
- Small company (Finland-based), limited market traction
- No browser-only option — requires app install
#### MeetinVR
- **What it is:** VR-native meeting platform. 3D sketching, prototyping review, brainstorming in VR.
- **What works:** Strong 3D interaction tools. Good for design review / prototyping.
- **What's broken:**
- VR-only — no non-VR participation at all
- Narrow use case (design reviews), not persistent office
- Very small player — minimal market presence since 2016
### 2D Spatial Players (Browser-Based)
#### Gather
- **What it is:** 2D pixel-art virtual office with proximity-based video chat. Walk your avatar around a map, video pops when near people.
- **Funding:** $77M total (Series B led by Sequoia + Index, $50M round). Revenue ~$9-10.8M (2024). YC W20.
- **What works:**
- **Proven product-market fit.** 74-person team, real revenue. The closest thing to a winner in this space.
- Browser-first, zero install. Walk up to someone = video chat starts. No scheduling needed.
- Spatial audio + proximity video is the killer feature. Feels like a real office.
- **What's broken:**
- 2D pixel art is deliberately retro — charming but zero immersion. You never feel "present."
- No VR support whatsoever — this is a 2D-only experience
- No spatial memory (everything looks the same from top-down view)
- No body language, no peripheral awareness, no "ambient presence" beyond a dot on a map
- Customization is shallow — you move furniture, not build experiences
- **Key insight:** Gather proved that proximity-based interaction IS the right model. They just execute it in 2D with no path to immersion.
#### Teamflow
- **What it is:** 2D virtual office platform. "Recreates in-person team presence and spontaneous collaboration."
- **Funding:** Smaller player, est. 2020, 11-50 employees
- **What works:** App integrations (Asana, Salesforce, Figma, Google Docs embedded in rooms). Spatial audio. Screen sharing, whiteboards.
- **What's broken:**
- 2D-only — same limitations as Gather but with less charm
- Much smaller market share than Gather
- No VR. No AI agent concept.
- Pivoting/shrinking — minimal public presence in 2025-2026
#### Roam
- **What it is:** AI-powered virtual HQ. "The Office of the Future: People & AIs collaborating."
- **What works:**
- **Only competitor with AI agents as permanent office residents** — closest to spatial-office concept
- AI assistant: observes office activity, schedules meetings, follows up via email/chat, searches meeting transcripts ("Magic Minutes")
- GitHub integration shows PR status on office map. Claims 42% reduction in PR review time internally.
- Saves teams ~$31,500/year by replacing multiple tools (per customer case study)
- Browser-based, no install. 8-minute average meeting duration.
- **What's broken:**
- 2D-only — top-down floor plan. No immersion, no spatial presence signals
- AI agents are assistants, not "teammates" — they schedule and remind, don't inhabit the space
- No VR path — the 2D architecture doesn't translate
- Positioning as "tool consolidator" rather than "office experience" — valuable but not visionary
#### Sococo (Legacy)
- **What it is:** Original virtual office (founded 2007). Floor plan with rooms, avatars, knock-to-enter.
- **What went wrong:**
- Acquired by Scaleworks, then team absorbed by Virbela — effectively abandoned
- Hostile cancellation practices (60-day written notice, no online cancel)
- "Running costs are high, features simplistic, integrations lacking" — Software Advice reviews
- No VR, no AI, no mobile-first — left behind by Gather and Roam
- **Key lesson:** First-mover advantage means nothing in spatial office. Incumbents can stagnate fast.
### Event/Experience-Focused (Adjacent)
#### Spatial.io
- **What it is:** 3D immersive spaces for events, galleries, brand experiences. Web, Mobile, VR. 2M+ creators.
- **What works:** No-code + Unity tools for building 3D spaces. True cross-platform (Web no-download, Mobile, VR). Integration with Office, Google Workspace, Slack.
- **What's broken:**
- Positioned for **events and brand experiences**, not persistent daily office use
- Spaces feel like event venues, not workplaces — you visit, you don't "live" there
- No spatial audio proximity model for casual presence
- Avatars are cartoonish — fun for events, unserious for work
- No AI agent support
#### Frame (by Virbela)
- **What it is:** Immersive 3D collaboration spaces. Digital twins of physical offices. Web, desktop, mobile, VR.
- **What works:** True cross-platform support including browser. Can build digital twins of real offices. Trello built their NYC office in Frame for events/meetings. Live webcam streaming, screen sharing, whiteboards, synced media.
- **What's broken:**
- Meeting/event focused, not persistent daily workspace
- 3D environments are heavy — not optimized for quick casual drop-ins
- No spatialized ambient presence (you're either in a meeting or you're not)
- No AI agent concept
- Pricing is enterprise sales — not product-led growth
#### Microsoft Mesh → Teams Immersive Spaces
- **What it is:** Microsoft's 3D immersive meeting spaces, now folded into Teams as "immersive events."
- **What works:** Spatial audio creates natural side conversations. Avatar-based interaction. No-code space building. Native Teams integration — zero friction for existing Teams users.
- **What's broken:**
- **Event-only — not a persistent office.** You enter immersive space from a Teams meeting, you leave when it ends
- Known issues: audio quality inconsistency, artifacts in immersive space (per Microsoft Learn docs)
- Only accessible through Teams View menu during a meeting — not a standalone persistent workspace
- Microsoft killed standalone Mesh in Dec 2025 — folded it into Teams as a feature, not a product
- No browser-only spatial presence — requires Teams client
---
## 2. The Gap: VR/Non-VR Bridge
### The Core Problem
Every existing product forces a binary choice:
| Approach | Products | Limitation |
|-----------|----------|------------|
| VR-only | Immersed, Workrooms, MeetinVR | Excludes >95% of potential users who don't own/wear headsets |
| 2D-only | Gather, Teamflow, Roam, Sococo | Zero immersion. No body language, spatial memory, or peripheral awareness |
| Desktop app + VR | Arthur, Glue | Requires install — no browser path. VR and non-VR modes are different experiences |
| Cross-platform events | Spatial, Frame, Teams Immersive | Meeting/event-focused. No persistent ambient presence |
### The Specific Gap Nobody Fills
**A single shared spatial environment where VR users (full immersion, body language via head/hand tracking) and browser users (isometric 3D, keyboard + mouse) share the SAME space with the SAME presence signals.**
Nobody does this because:
1. VR rendering and browser rendering are fundamentally different pipelines — most companies pick one
2. Isometric perspective for browser users hasn't been explored as a work medium (it's used in games, not productivity)
3. The conventional wisdom is that avatar-based 2D (Gather) is "good enough for browsers" and 3D is "for VR only"
### Why It Matters Now (2026)
- **VR market is shrinking:** Global VR headset market declined 12% YoY in 2024 (IDC). Apple Vision Pro production ceased early 2025. Even Meta's $70B bet is pivoting.
- **But VR isn't going away for power users:** Meta Quest 3 holds 74.6% market share. ~1M+ users engage WebXR monthly on Quest. The people who DO use VR for work LOVE it.
- **Browser 3D is finally viable:** WebGPU hit baseline in every major browser (2026). WebXR is an Interop 2026 focus area (browser vendors closing gaps). Three.js is shipping WebGPU support.
- **AI agents change the equation:** An office with AI residents isn't just a meeting tool — it's a persistent team environment. Browser users need to see what AI agents are doing. VR users need to inhabit the same space.
---
## 3. What In-Office Work Provides That Remote Loses
### The Evidence
**Microsoft Nature Study (2021)** — Analyzed 61,182 Microsoft employees over the first 6 months of remote work:
- Cross-group collaboration dropped **25%** from pre-pandemic levels
- Collaboration networks became "more static and siloed"
- Communication shifted from synchronous (real-time) to asynchronous (email/IM)
- Employees communicated more with strong ties, fewer weak ties formed
**MIT Research (2022)** — Analyzed email networks at MIT during pandemic:
- **Weak ties declined by ~38%** — 5,100 missed weak-tie connections in 18 months
- Weak ties are the primary source of novel information and serendipitous innovation
- "A complete return to the workplace would result in a complete recovery of weak ties" (Ratti et al.)
**Oxford Academic JCMC (2024)** — Socio-technical analysis of remote informal communication:
- "Familiar communication behaviors, norms, and rituals related to face-to-face informal communication change, leading to severe reduction of remote informal communication"
- Remote workers lose the natural rhythm of office interactions — impromptu check-ins, overheard conversations, lunch conversations
**Nature Human Behaviour (2021)** — Microsoft study published in Nature:
- "Firm-wide remote work caused collaboration networks to become more static and siloed"
- Asynchronous communication increased at the cost of synchronous, rich interaction
**Frontiers in Computer Science (2021)** — Ambient workspaces research:
- "Remote collaboration involving spatial interactions and enhancing co-presence of colleagues — bringing the rhythmicity and natural flow of the workday to remote settings"
- Recommendation: extend displays to reduce cognitive load, enable spatial interactions
**arXiv (2025)** — Effects of Remote Working on Scientific Collaboration:
- "Remote work expanded collaboration networks but reduced research impact"
- "Remote and hybrid working hampers research quality by reducing spontaneous, serendipitous in-person interactions"
- "Pandemic accelerated the erosion of weak ties"
### The Five Signals Remote Work Destroys
1. **Ambient Presence** — Knowing who's around, who's busy, who's available without asking. The "glance across the room" that tells you whether to interrupt.
2. **Peripheral Awareness** — Overhearing conversations, seeing someone struggle with something you can help with, noticing body language shifts.
3. **Serendipitous Collisions** — Running into someone from another team in the hallway/kitchen. These are the weak-tie interactions that drive innovation.
4. **Spatial Memory** — "I remember where I was standing when we figured that out." Physical space anchors memories and ideas. Zoom grids are memory-killing.
5. **Body Language** — Posture, gesture, eye contact, lean-in/lean-out. Zoom shows faces but loses everything below the shoulders. VR captures more (head/hand tracking) but current avatars are cartoonish.
### What This Means for spatial-office
The product must deliver, not describe, these five signals. Competitors claim to "recreate the office" but deliver at most 1-2 of these. Gather does proximity + collision (#3). Teams Immersive does spatial audio (#2 partial). Nobody does all five — and nobody does them across both VR and browser simultaneously.
---
## 4. Browser-First Spatial Rendering — State of the Art
### WebXR
- **Status:** Production-ready. Meta's Immersive Web SDK (IWSDK) — open-source WebXR toolkit with hand tracking, spatial interactions running in-browser.
- **Scale:** 1M+ monthly WebXR users on Meta Quest (as of Connect 2024)
- **Key development:** WebGPU baseline in all major browsers (2026). Interop 2026 includes WebXR as a focus area — browser vendors actively closing gaps between implementations.
- **Relevance:** WebXR enables a SINGLE codebase for VR (in-headset) and browser (monitor) rendering. The same Three.js scene renders in both contexts.
### Three.js
- **Status:** Industry standard for browser 3D. Now ships WebGPU support. Powers everything from data viz to games.
- **Isometric perspective:** Three.js supports orthographic cameras natively. An isometric view of a 3D scene is a one-line camera setting. This means the SAME 3D scene that a VR user experiences immersively can be rendered isometrically for a browser user.
- **Key libraries:** enva-xr (WebXR + Three.js with occlusion, lighting, physics), Mattercraft (ZapWorks, "unrivalled choice for WebXR development")
### Why Isometric for Browser Users
Isometric perspective (e.g., classic SimCity, Diablo, Monument Valley) offers several advantages over top-down 2D (Gather) or first-person 3D (VR):
- **Spatial memory:** Objects have height, depth, and position. You remember where things are in 3D space.
- **Peripheral awareness:** You can see activity happening across the room without turning. VR users look around; browser users pan/zoom.
- **Lower cognitive load:** Isometric is a "god view" — you see context without being IN it. Good for monitoring multiple things (agent activity, teammate presence).
- **Game-heritage UX:** Millions of people intuitively understand isometric navigation from games. No learning curve.
### Who's Doing Interesting Work Here
Nobody is doing isometric 3D for productivity/collaboration. The concept exists in:
- **Games:** Diablo, Hades, Factorio, RimWorld — all isometric, all deeply spatial, all have "base building" metaphors that map to office layout
- **City builders:** SimCity, Cities: Skylines — managing spatial resources with isometric view
- **No productivity tools use this perspective.** The entire field is green.
---
## 5. Agent Visualization — The Unique Angle
### Current State of Agent Visualization
**Academic/Research:**
- **Stanford Generative Agents (Smallville):** 25 AI agents in a 2D sandbox world. Agents have daily schedules, interact, form relationships. The visualization is a 2D top-down map with agent sprites. This is the most influential agent visualization project but is purely 2D and research-only.
- **MA3DSG (2026):** Multi-agent 3D scene graph generation — multiple agents exploring and building unified 3D representations. Research-grade, not user-facing.
- **MASON:** Java-based multi-agent simulation library with 2D/3D visualization tools. Academic, not product-grade.
**Commercial/Tools:**
- **SmythOS:** Agent-based modeling platform. Heat maps, density plots, animated trajectories for agent behavior. Enterprise-focused, not spatial office.
- **DeepNLP Agent Visualization Tool:** Timeline-based visualization of async multi-agent events. 2D, not spatial.
- **AutoGen (Microsoft):** Multi-agent conversation framework. No spatial visualization — purely text/API-based.
- **LangGraph / CrewAI:** Agent orchestration frameworks. Text logs and node graphs. No spatial visualization.
**The Closest Analog: Roam**
- Roam is the only product putting AI agents in a virtual office context
- But their agents are "assistants" — scheduling, note-taking, reminders
- Agents don't have spatial presence — they're functions, not residents
- Visualized as icons/buttons in a 2D floor plan
### What spatial-office Can Do Differently
**AI agents as spatial residents:**
- Each agent has a "desk" or workspace in the isometric office
- Agents have activity states visible through spatial cues: working (pulsing glow), idle, in-conversation, awaiting input
- Browser users can see at a glance what all 9 agents are doing — spatial memory for agent state
- VR users can "walk over" to an agent's workspace and interact
- The 9-agent Scobleclaw system is the PERFECT dogfood — the product is literally the team
**This is genuinely new.** No product visualizes a multi-agent AI system as a spatial team. The combination of:
1. Isometric 3D for browser users
2. Full VR for headset users
3. AI agents as spatial residents with visual state
4. Human team members also present in the same space
...is a product category that doesn't exist yet.
---
## 6. Key Risks — What Killed Previous Spatial Office Products
### Risk 1: Hardware Fragmentation (FATAL)
- **What happened:** Every VR-first product (Workrooms, MeetinVR, Immersed) hit the wall of "not everyone has a headset." VR market is small and shrinking. Apple Vision Pro was a commercial failure (production ceased). Quest 3S is the only bright spot.
- **Mitigation for spatial-office:** Browser-first isometric is the primary experience. VR is an enhancement, not a requirement. This inverts the adoption funnel: everyone can join, power users get more.
### Risk 2: Overpromising Immersion (Meta's $70B Lesson)
- **What happened:** Meta sold "the metaverse is your new office" when the technology wasn't ready. Uncanny valley avatars, motion sickness, clunky interfaces. The hype-to-reality gap was catastrophic.
- **Mitigation:** Isometric is deliberately NOT immersive — it's informative. It doesn't promise presence; it delivers awareness. VR presence is opt-in, not required.
### Risk 3: "Good Enough" Incumbents (Zoom/Teams)
- **What happened:** Zoom and Teams already worked. Asking people to adopt a new platform for marginal benefit failed. "Teams and Zoom had already established themselves as good enough for remote collaboration" (Reworked, 2026).
- **Mitigation:** spatial-office isn't replacing Zoom — it's a different category. It's for the BETWEEN meetings, not the meetings themselves. Persistent ambient presence, not scheduled calls. If anything, it integrates with Zoom for formal meetings.
### Risk 4: Enterprise Sales Death Spiral
- **What happened:** Arthur, Glue, Frame all sell to enterprises. Long sales cycles, procurement hell, low adoption per deal. You need a champion at every company. Product-led growth is the only sustainable model.
- **Mitigation:** Browser-first = no install = self-serve signup. Dogfood with Scobleclaw first. Freemium or team-tier pricing, not enterprise contracts.
### Risk 5: Novelty Without Retention
- **What happened:** Sococo, early virtual offices — interesting concept, fun for a week, then everyone goes back to Slack. The "gamification" approach (avatars, rooms, knocking) proved shallow.
- **Mitigation:** Agent visualization provides daily utility — you check the office to see what agents are doing, not just to "hang out." The product earns its place in the daily workflow.
### Risk 6: AI Hype Without Substance
- **What happened:** Arthur pivoted from "VR office" to "AI co-workers" messaging. Roam added AI assistants. These feel bolted-on rather than native.
- **Mitigation:** The Scobleclaw agents ARE the team. This isn't "AI features added to an office app" — it's an office built FOR an AI-human team. The agents have real work output, not just chatbot functionality.
### What's Different Now (2026)
- **WebGPU + WebXR are production-ready:** Building a cross-platform 3D app in the browser is actually feasible now — it wasn't in 2020-2023
- **AI agents are real:** Multi-agent systems (AutoGen, CrewAI, LangGraph) are mature. Visualizing them is a new problem people actually have
- **Remote work is permanent:** The "return to office" wars are largely settled. Hybrid/remote is the default for tech. The problem isn't going away
- **VR fatigue is real but VR utility is real too:** People don't want to LIVE in VR, but power users who DO use it report genuine productivity gains. A browser-first product with VR enhancement captures both realities
---
## Competitive Positioning Map
```
HIGH IMMERSION (VR)
│
│ Immersed ● MeetinVR ●
│ Workrooms ☠ (dead)
│ Arthur ● ← spatial-office target zone:
│ Glue ● VR + Browser, Isometric 3D,
│ AI Agents as residents
│ 🎯
│ Frame ●
│ Spatial.io ● ───────────────
│ │
│ Teams Immersive ● │ ← THE GAP
│ (events only) │
│ │
│ Gather ● Roam ● │
│ Teamflow ● Sococo ☠ │
│
└──────────────────────────────────────
LOW IMMERSION (2D) BROWSER-FIRST
```
---
## Recommendations
1. **Position against Gather, not against VR products.** Gather proved the market ($10M+ revenue). They're the incumbent to displace. "Gather but isometric, with AI teammates" is the elevator pitch.
2. **Lead with agent visualization.** The unique angle isn't "spatial office" (many have tried) — it's "see your AI team working alongside you." This is a new category.
3. **Browser-first, VR-enhanced.** Ship the isometric browser experience first. Add WebXR VR mode second. Never require a headset.
4. **Dogfood publicly.** The Scobleclaw team is the first customer. Show, don't tell. Live-stream the office. Make the product's own development visible.
5. **Self-serve onboarding.** No demos, no sales calls, no enterprise procurement. Sign up → invite team (or agents) → see the office. < 60 seconds to value.
6. **Integrate, don't replace.** Zoom/Meet for formal meetings. Slack/Discord for async. spatial-office for the space between — the ambient layer.
---
*Analysis compiled from: WIRED, Nature Human Behaviour, Microsoft Research, MIT/UC Berkeley studies, Frontiers in Computer Science, arXiv, Crunchbase, Tracxn, XR Today, product websites (Immersed, Arthur, Glue, Frame, Roam, Gather, Teamflow, Spatial, Meta), G2 reviews, Reddit r/OculusQuest, TechCrunch, Forrester research.*