spatial-office/ARCHITECTURE.md

Spatial Office — Architecture

Status: Pre-MVP scoping | Author: senior-eng agent | Date: 2026-05-19

---

Table of Contents

1. System Architecture Diagram
2. Component Breakdown
3. Data Model
4. Rendering Approach
5. WebSocket Protocol
6. Nix Flake Structure
7. Integration Points
8. MVP Scope vs. Future Phases

---

1. System Architecture Diagram

┌──────────────────────────────────────────────────────────────────────────────┐
│                              SCOTT'S BROWSER                                  │
│  ┌──────────────────────────────────────────────────────────────────────┐    │
│  │                    SPATIAL OFFICE FRONTEND                            │    │
│  │  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────────┐   │    │
│  │  │ Isometric Scene  │  │   UI Overlay    │  │   State Manager     │   │    │
│  │  │ (Three.js)       │  │   (React)       │  │   (Zustand)         │   │    │
│  │  │ - Tile map       │  │   - Agent cards │  │   - WS client       │   │    │
│  │  │ - Agent sprites  │  │   - Zone panels │  │   - Entity cache    │   │    │
│  │  │ - Path animation │  │   - Event log   │  │   - Diff engine     │   │    │
│  │  └────────┬─────────┘  └────────┬─────────┘  └──────────┬──────────┘   │    │
│  │           └─────────────────────┼────────────────────────┘              │    │
│  └─────────────────────────────────┼───────────────────────────────────────┘    │
│                                    │ WebSocket (wss://n97:18790)                │
└────────────────────────────────────┼────────────────────────────────────────────┘
                                     │
┌────────────────────────────────────┼────────────────────────────────────────────┐
│                                   │HOST: n97 (NixOS, GMKtec NucBoxG5)           │
│  ┌────────────────────────────────┼─────────────────────────────────────────┐  │
│  │                      SPATIAL OFFICE BACKEND                               │  │
│  │                                                                           │  │
│  │  ┌──────────────────────┐  ┌──────────────────────┐                      │  │
│  │  │   WS Server          │  │   State Engine       │                      │  │
│  │  │   (Node.js / ws)    │  │   - Agent registry    │                      │  │
│  │  │   Port 18790         │  │   - Zone assignment  │                      │  │
│  │  │   - Channel rooms    │  │   - Activity tracker │                      │  │
│  │  │   - Broadcast diffs  │  │   - Event log        │                      │  │
│  │  └──────────┬───────────┘  └──────────┬───────────┘                      │  │
│  │             │                         │                                   │  │
│  │  ┌──────────┴─────────────────────────┴─────────────────────────────┐    │  │
│  │  │                        BRIDGE LAYER                               │    │  │
│  │  │                                                                   │    │  │
│  │  │  ┌─────────────────┐  ┌─────────────────┐  ┌──────────────────┐   │    │  │
│  │  │  │ OpenClaw Bridge │  │  Hermes Bridge  │  │   Pi Bridge      │   │    │  │
│  │  │  │                 │  │                 │  │                  │   │    │  │
│  │  │  │ Connects as WS  │  │ Connects as ACP │  │ HTTP API bridge  │   │    │  │
│  │  │  │ backend client  │  │ client to       │  │ to pi coding     │   │    │  │
│  │  │  │ to OpenClaw GW  │  │ Hermes ACP      │  │ agent for        │   │    │  │
│  │  │  │ (port 18789)    │  │ server          │  │ project tasks    │   │    │  │
│  │  │  │                 │  │                 │  │                  │   │    │  │
│  │  │  │ Reads:          │  │ Reads:          │  │ Receives:        │   │    │  │
│  │  │  │ - system-       │  │ - ACP tool call │  │ - Build status   │   │    │  │
│  │  │  │   presence      │  │   events        │  │ - Task progress  │   │    │  │
│  │  │  │ - event:agent   │  │ - Task outputs  │  │ - PR status      │   │    │  │
│  │  │  │ - event:tick    │  │ - Skill updates │  │                  │   │    │  │
│  │  │  │ - sessions      │  │                 │  │ Sends:           │   │    │  │
│  │  │  │   state via     │  │                 │  │ - Feature        │   │    │  │
│  │  │  │   chat.send     │  │                 │  │   requests       │   │    │  │
│  │  │  │                 │  │                 │  │ - Bug reports    │   │    │  │
│  │  │  └────────┬────────┘  └────────┬────────┘  └────────┬─────────┘   │    │  │
│  │  └───────────┼────────────────────┼────────────────────┼─────────────┘    │  │
│  └──────────────┼────────────────────┼────────────────────┼──────────────────┘  │
│                 │                    │                    │                      │
└─────────────────┼────────────────────┼────────────────────┼──────────────────────┘
                  │                    │                    │
                  ▼                    ▼                    ▼
    ┌──────────────────┐  ┌──────────────────┐  ┌──────────────────┐
    │  OpenClaw GW     │  │  Hermes Agent    │  │  Pi Coding Agent │
    │  (ws://127.0.0.1 │  │  (ACP server)    │  │  (pi-mono)       │
    │   :18789)        │  │                  │  │                  │
    │                  │  │  Self-reflective │  │  CLI / SDK mode  │
    │  Agent registry: │  │  learning loop   │  │                  │
    │  - cos           │  │                  │  │  Builds/deploys  │
    │  - engineering   │  │  Inter-agent     │  │  spatial-office  │
    │  - senior-eng    │  │  communication   │  │  project         │
    │  - trust         │  │  protocol        │  │                  │
    │  - market        │  │                  │  └──────────────────┘
    │  - health        │  └──────────────────┘
    │  - content       │
    │  - presentations │
    │  - vault         │
    └──────────────────┘

Key Design Decisions

Decision	Rationale
Separate WS server (port 18790) vs. extending OpenClaw GW	OpenClaw GW is a control plane, not an app server. A dedicated WS server keeps the spatial-office protocol clean and lets us iterate without touching the GW. The backend connects to the GW as a backend client (trusted loopback, shared gateway token).
Bridge layer as in-process modules	All three bridges run in the same Node.js process as the WS server. No need for separate microservices at this scale. Each bridge is a module that polls/streams from its source and pushes state diffs into the state engine.
ACP for Hermes integration	Hermes runs as an ACP server. The Hermes bridge uses @openclaw/acpx or a direct ACP client to connect, send tasks, and receive tool-call events. This is the standard path: OpenClaw → ACP → Hermes.

---

2. Component Breakdown

2.1 Frontend (packages/frontend)

Stack: TypeScript, Vite, React 18, Zustand, Three.js (via @react-three/fiber)

packages/frontend/
├── index.html
├── vite.config.ts
├── src/
│   ├── main.tsx                    # React entry
│   ├── App.tsx                     # Root layout (scene + UI overlay)
│   ├── scene/
│   │   ├── OfficeScene.tsx         # Three.js canvas + camera setup
│   │   ├── TileMap.tsx             # Isometric tile grid renderer
│   │   ├── AgentSprite.tsx         # Per-agent animated sprite
│   │   ├── ZoneMarker.tsx          # Zone boundary overlays
│   │   └── PathAnimator.ts         # Smooth interpolation between grid cells
│   ├── ui/
│   │   ├── HUD.tsx                 # Top bar: time, agent count, events
│   │   ├── AgentCard.tsx           # Click-to-inspect detail panel
│   │   ├── ZonePanel.tsx           # Zone occupancy summary
│   │   ├── EventFeed.tsx           # Real-time event log
│   │   └── MiniMap.tsx             # Small overhead reference map
│   ├── state/
│   │   ├── store.ts                # Zustand store (agent states, zones, events)
│   │   ├── wsClient.ts             # WebSocket client with reconnect
│   │   └── types.ts                # Shared frontend types
│   └── assets/
│       ├── tiles/                  # Tile sprites (floor, wall, desk, plant)
│       ├── agents/                 # Agent avatar sprites (9 departments)
│       └── zones/                  # Zone type indicators

Why @react-three/fiber: It gives us React-idiomatic Three.js with declarative scene composition, and we already get the React ecosystem (state, hooks, devtools) for free. The isometric scene is modest — 50×50 tiles max, ~15 sprites — so the fiber overhead is negligible.

2.2 Backend (packages/backend)

Stack: TypeScript, Node.js 22, ws library, Zod for validation

packages/backend/
├── tsconfig.json
├── src/
│   ├── index.ts                    # Entry: starts WS server + bridges
│   ├── server.ts                   # WebSocket server (port 18790)
│   ├── state/
│   │   ├── engine.ts               # In-memory state + diff engine
│   │   ├── agentRegistry.ts        # Known agents (9 depts)
│   │   ├── zoneManager.ts          # Zone definitions + occupancy
│   │   └── eventLog.ts             # Ring buffer of recent events
│   ├── bridges/
│   │   ├── openclaw.ts             # OpenClaw GW bridge
│   │   ├── hermes.ts               # Hermes ACP bridge
│   │   └── pi.ts                   # Pi coding agent bridge
│   ├── protocol/
│   │   ├── messages.ts             # WS message types (Zod schemas)
│   │   └── diff.ts                 # State diff computation
│   └── validation/
│       └── schemas.ts              # Zod schemas for all entities

Why in-memory state: 9 agents with simple state (position, activity, status) is trivially small. No database needed for MVP. The event log uses a ring buffer (last 500 events). If persistence is desired later, add SQLite.

WS library: ws (npm, ~20M weekly downloads) is the de facto Node.js WebSocket library. Fast, well-maintained, no dependencies. We wrap it with a thin protocol layer.

2.3 Bridge: OpenClaw (bridges/openclaw.ts)

The OpenClaw bridge connects to the Gateway WS on ws://127.0.0.1:18789 as a backend client, authenticating with the shared gateway token.

Connection handshake (per Gateway Protocol):

const connect = {
  type: "req",
  id: "spatial-office-1",
  method: "connect",
  params: {
    minProtocol: 3,
    maxProtocol: 4,
    client: { id: "spatial-office", version: "0.1.0", platform: "linux", mode: "backend" },
    role: "operator",
    scopes: ["operator.read"],  // read-only: presence, agent events, sessions
    auth: { token: GATEWAY_SHARED_TOKEN },
  },
};

What it reads:

GW Method/Event	What We Get	Mapped To
system-presence	Connected devices (mac app, webchat, nodes) → gateway uptime, client count	Top-level system health indicator
event:agent	Streaming agent turns — when an agent starts processing a message, tool calls, completion	Agent activity state (working, thinking, idle)
event:presence	Periodic presence updates from connected clients	Scott's presence (is he watching?)
event:tick	Gateway heartbeat every ~15s (configurable)	Liveness signal for the bridge
event:health	Gateway health snapshot	Gateway status indicator
Chat sessions	Polled via chat.send with agent-specific queries (or use the sessions_history API)	Current task description per agent

Agent state mapping (how we infer activity from event:agent):

// event:agent payload includes:
// - agentId: which agent is working
// - status: "accepted" | "streaming" | "ok" | "error"
// - tool calls: embedded in streaming content

function mapAgentState(event: AgentEvent): AgentActivity {
  // When event:agent arrives with status:"accepted" → agent is "thinking"
  // When we see tool calls → agent is "working" (extract tool name as context)
  // When event:agent completes (status:"ok") → agent is "idle"
  // When no agent events for >5min → agent is "idle"
}

Fallback for agents without active sessions: Poll the QMD session transcript store. Each agent's session directory at ~/.openclaw/agents/<agentId>/sessions can be read for recent activity. This requires filesystem access (same host, so fine).

2.4 Bridge: Hermes (bridges/hermes.ts)

The Hermes bridge connects to a Hermes agent running as an ACP server. The bridge:

1. Opens an ACP session with Hermes
2. Periodically queries: "What are you working on? What agents have you communicated with recently?"
3. Parses Hermes responses for inter-agent communication events
4. Maps these to spatial events (agents moving to "huddle" when Hermes facilitates communication between them)

ACP connection (using @openclaw/acpx as reference):

import { ACPClient } from "@agentclientprotocol/sdk"; // or @openclaw/acpx

const client = new ACPClient({
  transport: "stdio",  // or "ws" if Hermes exposes WS
  command: "hermes",   // Hermes CLI
  args: ["acp", "serve"],
});

What it extracts:

Hermes Output	Spatial Office Mapping
"Agent X asked me to help with Y"	Move Agent X to collaboration zone, show "pairing with Hermes on Y"
Skill extraction events	Show Hermes in self-improvement zone, "refining skill X"
Cron task execution	Show relevant agents moving to their task zones
Learning loop iteration	Self-improvement zone activity

Why ACP matters here: ACP is the standard protocol for driving external harnesses from OpenClaw. Hermes adopted it early. The bridge acts as an ACP client that drives Hermes and extracts structured output, rather than scraping logs. This is the same pattern described in the "Hermes drives, OpenClaw contains" architecture, just flipped: OpenClaw contains the spatial office, and Hermes is one of the observed agents within it.

2.5 Bridge: Pi (bridges/pi.ts)

The Pi bridge tracks the pi coding agent's activity on the spatial-office repo itself (dogfooding).

Connection: Pi's @earendil-works/pi-coding-agent package exposes an SDK with createAgentSession() and SessionManager. The bridge can:

1. Programmatic mode: Use createAgentSession() in SDK mode to spawn a pi session that works on spatial-office tasks
2. Status polling: Pi writes session logs; the bridge tails those logs for activity indicators

For MVP, the simplest approach is HTTP polling of pi's task status:

// Pi exposes a simple HTTP health/status endpoint (we add this to pi-mono fork)
const status = await fetch("http://localhost:18991/status");
// Response: { currentTask: "implementing tile renderer", progress: 0.4, repo: "spatial-office" }

What it reports:

• Pi's current task on the spatial-office repo
• Build/deploy status
• PR status (via Forgejo API at os.scoble.me/forgejo)

---

3. Data Model

3.1 Entities

// ── Agent ────────────────────────────────────────────
interface Agent {
  id: string;                // "cos", "engineering", "senior-eng", etc.
  name: string;              // Display name: "COS Agent"
  department: Department;    // enum of 9 values
  position: GridPosition;    // { row: number, col: number }
  facing: Direction;         // "N" | "NE" | "E" | "SE" | "S" | "SW" | "W" | "NW"
  activity: AgentActivity;   // Current state
  currentTask: string | null; // "Reviewing PR #42", "Memory maintenance", etc.
  lastHeartbeat: number;     // Unix ms timestamp
  sessionKey: string | null; // OpenClaw session key if active
}

interface AgentActivity {
  status: AgentStatus;       // "idle" | "thinking" | "working" | "blocked" | "offline"
  zone: ZoneId;              // Which zone the agent is in
  detail: string | null;     // "Running: web_search", "Calling: sessions_spawn"
  startedAt: number;         // When this activity began
}

type AgentStatus = "idle" | "thinking" | "working" | "blocked" | "offline";
type Department = "cos" | "engineering" | "senior-eng" | "trust" | "market"
                | "health" | "content" | "presentations" | "vault";

// ── Zone ─────────────────────────────────────────────
interface Zone {
  id: ZoneId;
  name: string;              // "Engineering Pod", "Quiet Room", "Coach Corner"
  type: ZoneType;            // "collaboration" | "solo" | "self-improvement" | "social"
  bounds: GridRect;          // { rowStart, colStart, rowEnd, colEnd }
  capacity: number;          // Max agents in zone
  occupants: string[];       // Agent IDs currently in zone
  features: ZoneFeature[];   // "whiteboard", "terminal", "coffee", "coach-bot"
}

type ZoneId = string;
type ZoneType = "collaboration" | "solo" | "self-improvement" | "social";
type ZoneFeature = "whiteboard" | "terminal" | "coffee" | "coach-bot" | "library";

interface GridRect {
  rowStart: number;
  colStart: number;
  rowEnd: number;
  colEnd: number;
}

// ── GridPosition ─────────────────────────────────────
interface GridPosition {
  row: number;   // 0-indexed, origin at top-left of isometric view
  col: number;
}

// ── Event ────────────────────────────────────────────
interface OfficeEvent {
  id: string;
  timestamp: number;         // Unix ms
  type: EventType;
  agentId: string | null;    // Which agent triggered it
  zoneId: string | null;     // Where it happened
  payload: Record<string, unknown>;
}

type EventType =
  | "agent.moved"            // Agent repositioned
  | "agent.activity_changed" // Status or zone changed
  | "agent.offline"          // Agent heartbeat lost
  | "agent.online"           // Agent returned
  | "hermes.message"         // Inter-agent communication observed
  | "pi.task_update"         // Pi task progress changed
  | "zone.full"              // Zone hit capacity
  | "system.heartbeat";      // Periodic system health

// ── World State (the full snapshot) ──────────────────
interface OfficeState {
  version: number;           // Monotonic counter for diff-based sync
  agents: Record<string, Agent>;
  zones: Record<string, Zone>;
  events: OfficeEvent[];     // Last N events (ring buffer)
  meta: {
    gatewayUptime: number;
    connectedClients: number;
    scottPresent: boolean;
    lastUpdate: number;
  };
}

3.2 Zone Map (MVP Office Layout)

                    ┌──────────────────────────────────────┐
                    │            COFFEE / SOCIAL            │  (top)
                    │         [☕ social zone 4x2]          │
                    ├──────────┬───────────────┬────────────┤
                    │          │  COLLABORATION│            │
                    │  SOLO    │  POD A        │  SOLO      │
                    │  WORK    │  [4 desks]    │  WORK      │
                    │  [6      │  Whiteboard   │  [6        │
                    │  desks]  │  Terminal     │  desks]    │
                    │          ├───────────────┤            │
                    │          │  COLLABORATION│            │
                    │          │  POD B        │            │
                    │          │  [4 desks]    │            │
                    │          │  Whiteboard   │            │
                    ├──────────┴───────────────┴────────────┤
                    │       SELF-IMPROVEMENT AREA           │  (bottom)
                    │   [Coach Bot] [Library] [Quiet Pods]  │
                    └──────────────────────────────────────┘

Grid: 24 columns × 15 rows. Each tile is 64×32px isometric diamond.

Zone assignments for 9 agents:

Zone	Type	Capacity	Default Assignment
Pod A (left)	collaboration	4	engineering, senior-eng, pi (when active)
Pod B (right)	collaboration	4	market, content, presentations
Solo West	solo	6	cos, trust
Solo East	solo	6	health, vault
Coach Corner	self-improvement	3	Any agent doing memory maintenance / coach review
Coffee Bar	social	4	Any agents in "idle" or "observing" state

---

4. Rendering Approach

4.1 Why Three.js (not Phaser, not Canvas 2D)

Criterion	Three.js	Phaser	Canvas 2D
Isometric projection	Trivial (orthographic camera at 45° rotation, 35.264° elevation)	Built-in isometric plugin but 2D-only	Manual tile math, no depth sorting
3D depth effects	Native (agents can have height, z-fighting resolved)	2D render order only	Manual z-index
Animation (tweening)	Built-in gsap + Three.js animation loop	Built-in tween system	Manual requestAnimationFrame
React integration	@react-three/fiber (25K+ stars)	@yonatanmn/react-phaser (unmaintained)	Manual DOM
Bundle size	~150KB gzipped (with tree-shaking)	~1MB	0KB
Community/ecosystem	Massive	Large (game dev)	Native

Decision: Three.js with @react-three/fiber. The isometric view is fundamentally a 3D scene viewed from a fixed orthographic angle. Three.js gives us proper depth sorting, smooth path animation with useFrame, and the React integration means we can use Zustand state directly in the render loop.

4.2 Camera Setup

// Orthographic camera positioned for true isometric projection
// True isometric: rotate Y by 45°, then X by 35.264° (atan(1/√2))
const camera = new THREE.OrthographicCamera(
  -width / 2, width / 2,   // left, right
  height / 2, -height / 2,  // top, bottom
  0.1, 1000                  // near, far
);

// Position camera above and center
camera.position.set(0, -20, 20);
camera.lookAt(0, 0, 0);

// For a 2:1 isometric pixel ratio (standard in games):
// X angle = 30° from horizontal (this gives the 2:1 diamond shape)
// Actually, use rotY=45°, rotX=-35.264° for true isometric
// But for a cleaner 2:1 pixel ratio (easier art), use rotX=-30°

Alternative — fake isometric with 2D sprites on a tilted plane: We can place flat sprites on a plane tilted at 30° rather than building a full 3D office. This is simpler for art assets (flat PNG sprites for agents and furniture). The Three.js Sprite object always faces the camera, so for isometric we'd use quads (PlaneGeometry) tilted to match the floor plane.

4.3 Tile Map

The office is a 2D grid (Array<Array<Tile>>) where each cell has:

interface Tile {
  type: TileType;          // "floor" | "wall" | "desk" | "empty" | "door"
  zoneId: ZoneId | null;   // Which zone this tile belongs to
  occupant: string | null; // Agent ID if someone is on this tile
}

type TileType = "floor" | "wall" | "desk" | "empty" | "door" | "plant" | "whiteboard";

Rendering loop (each frame):

1. Sort tiles by row + col for correct isometric depth (farther tiles render first)
2. For each tile: render floor sprite, then desk/wall sprite
3. For each agent: render agent sprite at grid position, apply z-offset for "standing on desk"
4. Apply gsap tweens for smooth movement between tiles

Performance: 24×15 = 360 tiles max, ~200 visible at a time. Three.js handles this effortlessly without instancing.

4.4 Agent Visual Design

Each of the 9 agents gets a simple isometric sprite (64×64 or 96×48px) with:

• Department color (distinct hue per department)
• Department icon (simple glyph: gear for engineering, shield for trust, book for content, etc.)
• Status indicator (small colored dot: green = idle, yellow = thinking, blue = working, red = blocked)
• Speech bubble (optional, appears when agent is "talking" via Hermes)

Sprites are PNGs drawn in a 2:1 isometric pixel art style. We can generate these programmatically as a first pass (colored boxes with text labels) and replace with proper art later.

4.5 UI Overlay (React)

The UI overlay sits on top of the Three.js canvas (absolute positioned div) and provides:

• Agent card: Click an agent → card slides in showing: name, department, current activity, task description, session key, last heartbeat
• Zone panel: Sidebar showing zone occupancy and type
• Event feed: Scrollable log of recent events (last 50)
• Minimap: Small 2D overhead view showing the full office layout
• Camera controls: Zoom in/out, pan (though the isometric view is mostly fixed)

---

5. WebSocket Protocol

5.1 Connection

The frontend connects to ws://n97:18790 (or wss:// with TLS proxy later).

// No auth needed for MVP (local network only).
// Client sends a hello message:
{
  type: "hello",
  version: 1,
  client: "browser",
}

5.2 Server → Client Messages

Message	Payload	Frequency
state.full	OfficeState (complete snapshot)	On connect, after reconnect
state.diff	{ version, changes: StateDiff[] }	On every state change (throttled to 100ms batches)
event	OfficeEvent	Real-time as events occur
heartbeat	{ ts: number }	Every 15 seconds

State diff format (avoids sending the full state on every tick):

interface StateDiff {
  op: "agent.update" | "agent.move" | "zone.update" | "event.append";
  path: string;    // e.g., "agents.engineering.activity.status"
  value: unknown;  // New value
}

// Example diff: agent moved
{
  op: "agent.move",
  path: "agents.engineering",
  value: {
    position: { row: 5, col: 12 },
    activity: { status: "working", zone: "pod-a", detail: "pairing with senior-eng", startedAt: 1716152400000 }
  }
}

// Batch of diffs sent every 100ms:
{
  type: "state.diff",
  version: 47,
  changes: [
    { op: "agent.move", path: "agents.engineering", value: { ... } },
    { op: "agent.update", path: "agents.cos.activity.status", value: "thinking" },
    { op: "event.append", path: "events", value: { type: "agent.moved", agentId: "engineering", ... } },
  ]
}

5.3 Client → Server Messages

Message	Payload	Description
agent.click	{ agentId: string }	Scott clicked an agent (triggers detail expansion in UI, handled locally)
zone.click	{ zoneId: string }	Scott clicked a zone
camera.move	{ x, y, zoom }	Scott panned/zoomed (local state, not broadcast)

For MVP, client→server messages are minimal. The view is primarily read-only observation.

5.4 State Throttling

The backend batches diffs every 100ms. If no changes occur, no message is sent. The heartbeat ensures the client knows the connection is alive.

// Backend diff batching
class DiffBatcher {
  private diffs: StateDiff[] = [];
  private timer: NodeJS.Timeout | null = null;

  push(diff: StateDiff) {
    this.diffs.push(diff);
    if (!this.timer) {
      this.timer = setTimeout(() => this.flush(), 100);
    }
  }

  flush() {
    if (this.diffs.length > 0) {
      broadcast({ type: "state.diff", version: this.version++, changes: this.diffs });
      this.diffs = [];
    }
    this.timer = null;
  }
}

---

6. Nix Flake Structure

6.1 flake.nix

{
  description = "Spatial Office — isometric collaborative workspace for agents and humans";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs { inherit system; };

        # Node.js toolchain
        nodeToolchain = with pkgs; [
          nodejs_22
          nodePackages.pnpm
          nodePackages.typescript
        ];

        # Build the frontend
        buildFrontend = pkgs.stdenv.mkDerivation {
          name = "spatial-office-frontend";
          src = ./packages/frontend;
          nativeBuildInputs = nodeToolchain;
          buildPhase = ''
            pnpm install --frozen-lockfile
            pnpm run build
          '';
          installPhase = ''
            mkdir -p $out
            cp -r dist/* $out/
          '';
        };

        # Build the backend
        buildBackend = pkgs.stdenv.mkDerivation {
          name = "spatial-office-backend";
          src = ./packages/backend;
          nativeBuildInputs = nodeToolchain;
          buildPhase = ''
            pnpm install --frozen-lockfile
            pnpm run build
          '';
          installPhase = ''
            mkdir -p $out
            cp -r dist/* $out/
            cp -r node_modules $out/
            mkdir -p $out/bin
            echo '#!/usr/bin/env bash' > $out/bin/spatial-office-backend
            echo 'exec ${pkgs.nodejs_22}/bin/node $out/index.js' >> $out/bin/spatial-office-backend
            chmod +x $out/bin/spatial-office-backend
          '';
        };
      in
      {
        packages = {
          frontend = buildFrontend;
          backend = buildBackend;
          default = pkgs.buildEnv {
            name = "spatial-office";
            paths = [ buildFrontend buildBackend ];
          };
        };

        devShells.default = pkgs.mkShell {
          buildInputs = nodeToolchain ++ (with pkgs; [
            # Development tools
            nodePackages.vite
            nodePackages.eslint
          ]);

          shellHook = ''
            echo "🏢 Spatial Office dev shell"
            echo "  pnpm dev          — start frontend + backend in dev mode"
            echo "  pnpm build        — production build"
          '';
        };

        # NixOS module for running as a systemd service on n97
        nixosModules.spatial-office = { config, lib, pkgs, ... }:
          let
            cfg = config.services.spatial-office;
          in
          {
            options.services.spatial-office = {
              enable = lib.mkEnableOption "Spatial Office service";
              port = lib.mkOption { type = lib.types.int; default = 18790; };
              openclawTokenFile = lib.mkOption { type = lib.types.path; };
            };

            config = lib.mkIf cfg.enable {
              systemd.services.spatial-office = {
                description = "Spatial Office Backend";
                wantedBy = [ "multi-user.target" ];
                after = [ "network.target" ];
                environment = {
                  PORT = toString cfg.port;
                  OPENCLAW_GATEWAY_URL = "ws://127.0.0.1:18789";
                  OPENCLAW_TOKEN_FILE = cfg.openclawTokenFile;
                  NODE_ENV = "production";
                };
                serviceConfig = {
                  ExecStart = "${self.packages.${system}.backend}/bin/spatial-office-backend";
                  Restart = "always";
                  RestartSec = 5;
                  User = "spatial-office";
                  Group = "spatial-office";
                };
              };

              users.users.spatial-office = {
                isSystemUser = true;
                group = "spatial-office";
              };
              users.groups.spatial-office = {};
            };
          };
      }
    );
}

6.2 Why Nix

• Reproducibility: nix build produces identical outputs on any machine. The n97 is NixOS, so this is the native deployment path.
• Node.js pinning: No "works on my machine" — Node 22, pnpm, and TypeScript versions are locked.
• Systemd service: The NixOS module handles service lifecycle, user creation, and restart policy.
• Dev shell: nix develop gives every developer the exact same toolchain.

6.3 Directory Layout

spatial-office/
├── flake.nix
├── flake.lock
├── package.json            # Workspace root (pnpm workspaces)
├── pnpm-workspace.yaml
├── packages/
│   ├── frontend/
│   │   ├── package.json
│   │   ├── vite.config.ts
│   │   ├── tsconfig.json
│   │   └── src/           # (as described in §2.1)
│   └── backend/
│       ├── package.json
│       ├── tsconfig.json
│       └── src/           # (as described in §2.2)
├── assets/                 # Design files, tile templates
├── README.md
└── ARCHITECTURE.md         # This file

---

7. Integration Points

7.1 OpenClaw Integration

What we need from OpenClaw:

Data	Source	How
Which agents exist	agents.list in Gateway config (~/.openclaw/openclaw.json)	Backend reads config file on startup, or GW exposes this via config.get RPC
Agent online/offline	system-presence WS method	Bridge calls this on startup + polls every 30s
Agent activity (working/thinking/idle)	event:agent WS events	Bridge subscribes to agent events for all 9 agent IDs
Current task description	Session transcripts or chat.send polling	Bridge reads ~/.openclaw/agents/<id>/sessions most recent file
Scott's presence	event:presence with Scott's device ID	Updates meta.scottPresent

Gateway config read: The backend needs to know about the 9 agents. It can either:

1. Read ~/.openclaw/openclaw.json directly (same host, permissions allowing)
2. Call the Gateway's config.get RPC method with operator.admin scope
3. Accept a config file listing agent IDs explicitly

Option 3 is simplest for MVP. A spatial-office.config.json:

{
  "agents": [
    { "id": "cos", "name": "COS Agent", "department": "cos", "color": "#FF6B6B" },
    { "id": "engineering", "name": "Engineering Agent", "department": "engineering", "color": "#4ECDC4" },
    ...
  ],
  "zones": [ ... ]
}

Security: The backend needs operator.read scope only. It connects via loopback with the shared gateway token. No admin access required.

7.2 Hermes Integration

Hermes runs as an ACP server. The spatial-office backend is an ACP client.

Architecture choice: Hermes is NOT one of the 9 Scobleclaw agents. It's an external reasoning engine that the agents use through ACP. In the spatial office:

• Hermes appears as a "utility" — like a whiteboard or a coach bot
• When Agent A asks Hermes something, Agent A moves to a collaboration pod with a Hermes indicator
• When Hermes processes a learning loop, the self-improvement zone shows activity

ACP protocol flow:

Spatial Office Backend          Hermes ACP Server
        │                              │
        ├──── initialize ─────────────>│  "What agents have communicated
        │                              │   through you recently?"
        │<─── agent_messages ──────────┤  [{from: "engineering", to: "senior-eng",
        │                              │    topic: "spatial-office PR review"}]
        │                              │
        ├──── tools/list ─────────────>│  Discover available Hermes tools
        │<─── tools ───────────────────┤
        │                              │
        ├──── tools/call ─────────────>│  "Describe your current task"
        │<─── result ──────────────────┤  "Refining the code-review skill
        │                              │   after 12 evaluations"

For MVP: The Hermes bridge starts simple — a periodic poll (every 30s) that asks Hermes for its current state. We don't need real-time streaming from Hermes in phase 1. The ACP session stays open, the bridge sends a query, receives a response, and maps it to spatial events.

7.3 Pi Integration

Pi (@earendil-works/pi-coding-agent) is the coding agent that builds the spatial-office project itself. In the office, Pi appears as a 10th "agent" (or a special entity) that:

• Sits in a designated development workstation
• Shows current task: "Implementing TileMap renderer [▓▓▓▓░░░░] 40%"
• Moves to collaboration pod when doing code review with engineering
• Shows build status from the Forgejo CI

Integration approach:

1. Pi session manager: The backend spawns a Pi session using the SDK:

   import { createAgentSession } from "@earendil-works/pi-coding-agent";
   const { session } = await createAgentSession({
     model: myModel,
     tools: ["read", "bash", "write"],
     sessionManager: SessionManager.file({ dir: "./pi-sessions" }),
   });
   

2. Status endpoint: We add a lightweight HTTP server to the Pi session that exposes:

   GET /status → { task: "...", progress: 0.4, status: "working" }
   

3. Forgejo webhooks: The backend listens for Forgejo push/PR events for the spatial-office repo to track build status

For MVP: The Pi bridge is a polling loop that checks pi's session log file. The spatial-office repo's CI status is checked via the Forgejo API (GET /api/v1/repos/sscoble/spatial-office/commits).

---

8. MVP Scope vs. Future Phases

MVP (Phase 1) — "The Observable Office"

Goal: Scott can open a browser tab and see the 9 agents in an isometric office, with their current activity inferred from OpenClaw.

Feature	Included?	Rationale
Isometric tile map (fixed layout)	✅	Core visual
9 agent sprites with department colors	✅	Core visual
Agent activity state (idle/thinking/working/blocked) from OpenClaw	✅	Core data source
Agent movement between zones based on activity	✅	Core mechanic
Click agent → detail card	✅	Scott's view
Zone occupancy indicators	✅	Visual clarity
Event feed (last 50 events)	✅	Debugging + insight
WebSocket real-time sync	✅	Required for live updates
Nix flake build	✅	Deployment requirement
OpenClaw GW bridge (operator.read)	✅	Primary data
Hermes bridge	❌ (deferred)	Adds complexity without visual payoff in MVP
Pi bridge	❌ (deferred)	Pi tasks can be inferred from OpenClaw events (if Pi is an agent)
Agent path animation (smooth tweening)	✅	Quality-of-life, cheap to add
Scott presence indicator	✅	Trivial (1 presence check)
Mobile/responsive layout	❌	Desktop-first, n97 serves local network
Multiple observers (multi-client)	❌	Scott's personal view only

MVP deliverables:

1. Working frontend (Vite + React + Three.js) rendering the office
2. Working backend (Node.js + ws) bridging to OpenClaw
3. Nix flake that builds both and deploys as a systemd service on n97
4. Accessible at http://n97:18790 on the local network

Phase 2 — "Hermes + Pi Visibility"

Feature	Notes
Hermes ACP bridge	Real-time Hermes state, inter-agent comm visualization
Pi bridge	Pi's task status on spatial-office, Forgejo CI status
Agent "huddle" visualization	Two agents visibly move together when collaborating
Hermes message visualization	Speech bubbles or "Hermes link" lines between agents
Coach bot visualization	Self-improvement zone shows active Hermes learning loops

Phase 3 — "Interaction"

Feature	Notes
Scott can send messages to agents from the UI	chat.send through OpenClaw bridge
Scott can drag agents between zones	Sends system-event to re-route agent tasks
Agent "whisper" (private message to one agent)	One-click from agent card
Sound cues	Ambient office sounds, notification chimes
Agent idle animations	Breathing, looking around, typing

Phase 4 — "VR + Multi-User"

Feature	Notes
WebXR headset view	First-person perspective in the same office
Multiple human observers	Different views, shared state
Voice chat integration	Spatial audio between agents and humans
Custom office layouts	Drag-and-drop office editor
Real collaboration tools	Shared whiteboard, code viewer, screenshare

Phase 5 — "Full Product"

Feature	Notes
External teams	Multiple organizations, each with their own office
Agent marketplace	Drop-in agent "employees" from a catalog
Analytics dashboard	Productivity, collaboration metrics, heat maps
API for third-party agent systems	Standard spatial-office protocol

---

Appendix A: Technology Dependency Graph

spatial-office
├── @react-three/fiber (^8.x) → three (^0.170.x) → WebGL
├── @react-three/drei (^9.x)  → helpers for fiber (OrbitControls, etc.)
├── react (^18.x)
├── react-dom (^18.x)
├── zustand (^5.x)            → state management
├── gsap (^3.x)               → smooth path animation
├── vite (^6.x)               → build tool
├── typescript (^5.x)
├── ws (^8.x)                 → WebSocket server
├── zod (^3.x)                → validation
├── @agentclientprotocol/sdk   → ACP client (Hermes bridge, Phase 2)
└── @earendil-works/pi-coding-agent → Pi SDK (Pi bridge, Phase 2)

Appendix B: OpenClaw Gateway Protocol — Methods We Use

Per the Gateway Protocol and Presence docs:

Method	Scope Required	Use
connect	(handshake)	Establish backend client connection
system-presence	(unrestricted)	Get connected devices/clients
event:agent (subscription)	operator.read	Stream agent turn events
event:presence (subscription)	(unrestricted)	Stream presence updates
event:health (subscription)	(unrestricted)	Stream gateway health
event:tick (subscription)	(unrestricted)	Gateway heartbeat
chat.send	operator.write	(Phase 3) Send messages to agents
config.get	operator.admin	(May not be needed; use config file instead)

Appendix C: Open Questions

1. Agent session polling frequency: How often should we poll? 5s is responsive but may be noisy. 30s is calmer. Start with 10s and tune.
2. OpenClaw shared token: Where is it stored, and how does the NixOS module securely access it? Option: OPENCLAW_TOKEN_FILE environment variable pointing to a file readable by the spatial-office user.
3. Hermes ACP endpoint: Is Hermes currently running on n97? If so, at what port/socket? If not, that's a prerequisite for Phase 2.
4. Agent sprite art: Programmatic (CSS-generated or Canvas-drawn) vs. hand-drawn pixel art. Start programmatic, iterate.
5. n97 firewall: Does n97 expose port 18790 to the local network? For Scott's browser to connect, this port must be open or tunneled.