pi-mono/packages/chord/docs/tutorial/book.md

2058 lines
97 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
title: Together: build a shared todo app with Chord
cover: Build It / Together
kicker: A tutorial · Chord v2 and its app kit · 2026-11-03
subtitle: One small app, built a step at a time, from an empty server to two people, a page, a terminal, a theme, three languages, an edit applied while they are connected, and one click traced across its processes
stats:
- 11 | steps, one runnable example
- 4 | host kinds one plugin runs on
- 1 | worker process per list
- 0 | restarts for a new theme, a language or an edit
note: |
The app is `packages/chord/test/examples/61-tutorial/`, runnable with the line in its `main.ts`. Every code block
below is quoted from its files, and every console output from what its steps print:
`test/examples/61-tutorial/tutorial.test.ts` fails when a quote or an output drifts. The design is
`packages/chord/docs/design.md`; where this tutorial says "the rule", design.md has the full statement. The last
chapter is about p2, whose code lives outside this repository.
colophon: Together · first edition · 2026-11-03 · Chord v2 slices 1-13, the app kit, example 61
---
# Before we begin {.preface}
Mara keeps the household's shopping list on her laptop. She wants it in a browser tab and in a terminal, live in both.
Ada, her flatmate, should be able to add coffee from her phone without an account: pick a name and go. And Mara wants
to change the app while it runs: add a dark theme in the evening, fix a small annoyance in how todos are added, without
anyone being thrown out.
That is four kinds of program (a server, a process per list, a page, a terminal client) and three kinds of change (new
data, a new person, new code) in one small app. Wiring that by hand means a web server, a socket protocol, a way to
start and stop processes, sessions, a bundler for the page and a reload story for each. Chord does the wiring, and its
**app kit** (`@earendil-works/chord/app`) makes the decisions an app would otherwise make itself: where data lives,
how people sign in, how plugins are found and rolled out.
This tutorial builds that app. Each chapter adds one thing, shows the code, shows what happens when it runs, and
states the rule behind it. The app is one directory, `test/examples/61-tutorial/`; run every step at once from
`packages/chord`:
```sh
node --conditions=source --experimental-strip-types test/examples/61-tutorial/main.ts
```
With Node 26.5.1 or later the terminal client of chapter V runs too (its sandbox needs that Node); with an older one,
that step says so and the rest runs.
## Ten things to know
:::findings
1. **An app is its API, its plugins and a start script.** `startServer` makes the data directory, the store, the
server host, HTTP with sign-in, the workers and the plugin roots; `startWorker` does a worker's half.
2. **A host is one participant.** The server, each list's worker, each browser tab and each terminal are hosts,
connected by links. Tabs and terminals are clients: they connect and ask, nobody asks them.
3. **An object is reached by its contract.** A contract names an object type, its params and its members; a `remote`
contract also says which host serves it. Any host asks for `Todos { list: "groceries" }`, and Chord routes the
request to that list's worker.
4. **Who calls is stamped, never sent.** Every call carries a principal the link stamped from the sign-in. A factory
picks a variant per principal; a todo records who added it from the principal, not from its arguments.
5. **A plugin has one file per host kind.** `server.ts`, `worker.ts`, `web.ts`, `tui.ts`: each declares what that kind
of host serves and contributes. Plugins import only the app's API.
6. **Plugins meet in registries.** The page shows `Panels`, the terminal `Screens`, the kit's `Themes` restyle both;
plugins contribute, one plugin (or the entry) composes.
7. **Changes apply when you say so.** Saving a file marks its plugin dirty; `applyChanges` builds and rolls out to the
server, the workers, the pages and the terminals, in place.
8. **Text is rendered per viewer.** A plugin's messages are typed code, a translation is a plugin, each user picks a
language, and servers store keys, never sentences.
9. **One click is one trace.** Every process records spans to the telemetry store it is given (here one SQLite file
in the data directory); carriers in the calls' headers join the page's call, the server's relay and the worker's
method into one trace, which `TelemetryStore` reads back. `ctx.trace` adds spans of your own.
10. **Six traps:** load before use; a reference acts as whoever made it; await what you start; a watch starts at the
current value; the plain `t` has no language of its own; a trace starts where a span starts.
:::
## Words used throughout
A **host** is one participant of the app, with an **address** if others can reach it (the server, `List({ id })`) or
without one (a **client**: a page, a terminal). A **link** connects two hosts. A **principal** is who calls, typed by
the app and stamped by links. An **object** is state and methods at one identity; its **contract** (`local` for its own
host only, `remote` for any host) names its **params** (which instance) and members. A remote contract's **variants**
are the member sets different principals get. A **factory** declares an object's parts: **documents** and **tables**
(stored data, changed in **transactions**), **live queries** (a table's rows Chord keeps current), **derives**
(values computed from others), **natives** (things opened and closed, such as a file or a database cursor). A **reference** is what asking for an object returns. A **facet** is a
plugin's code for one host kind; a **registry** is an extension point plugins **contribute** to. A **message** is a
text with a key, a **catalog** a plugin's messages in one language, a **locale** a language tag (`de`, `pt-BR`). A **rollout** applies
new plugin code to the hosts in order. A **span** is one timed piece of work a host recorded, a **trace** the spans of
one request across hosts, joined by the **carriers** calls carry.
# One Server {#server lede="The API, the principals and a start script: the smallest app the kit runs, and who may come in."}
An app built with the kit has a fixed shape: its API in `src/api/v<n>/`, its built-in plugins in `plugins/`, a start
script and a worker entry (design §23, "The app kit"). Everything else is a plugin. This chapter writes the three
pieces of base code and starts the server.
## The principals
A **principal** says who is calling. The app defines the type; the kit needs a variant for the server, one for its
workers, and adds the owner, `{ kind: "owner" }`, which the app's type must include:
<!-- quote: src/principals.ts -->
```ts
// The todo app's principals: its server and list workers, the kit's owner, and the users who signed in with a name.
/** Who someone is in the todo app. */
export type TodoPrincipal =
| { readonly kind: "server" }
| { readonly kind: "worker"; readonly list: string }
| { readonly kind: "owner" }
| { readonly kind: "user"; readonly id: string; readonly name: string };
```
The user variant is for chapter VII. A principal travels with every call and is part of objects' identities, so it
stays small: a kind and an id. A user's profile, if the app had one, would be data in an object keyed by the id.
## The API
Plugins never import Chord. They import the app's **API**, a module the app names with a version in its path
(`@todo/app/v1`). It re-exports the part of Chord the app supports and adds what is the app's own:
<!-- quote: src/api/v1/index.ts -->
```ts
export { BACKGROUND_CONTEXT } from "../../../../../../src/context/index.ts";
export {
arr,
bool,
type Ctx,
defineMessages,
defineState,
defineTable,
eq,
forbidden,
int,
invalid,
local,
MessageSchema,
type MessagesOf,
method,
nul,
num,
obj,
PluginError,
query,
remote,
type Static,
str,
type Theme,
Themes,
type Translation,
Translations,
union,
} from "../../../../../../src/index.ts";
export type { TodoPrincipal } from "../../principals.ts";
```
`defineMessages`, `Translations` and the message schema are chapter VIII's. `invalid` and `forbidden` make Chord's own errors: a plugin refusing arguments throws `invalid("args", message)`,
one refusing a caller `forbidden(message)`, and the caller gets the same `{ code, reason }` Chord's built-in checks
give (chapter IX). `PluginError` is for codes of the plugin's own.
and a `defineFacet` bound to the principal type, so every factory sees a `TodoPrincipal`:
<!-- quote: src/api/v1/index.ts -->
```ts
/** Defines a facet of the todo app: its factories get the app's principals. */
export function defineFacet(setup: (host: FacetHost<TodoPrincipal>, self: FacetSelf) => void) {
return chordFacet<TodoPrincipal>(setup);
}
```
The API is a **host package**: a plugin's build leaves it out, and each process hands the plugin its own instance when
it evaluates it. The plugin sees the very `Themes` registry and `List` host kind the process has, not a copy. A
version in the path lets the app ship `v2` beside `v1` later; one facet uses one version (design §21, "The plugin
model").
The API is also a real package: a **workspace package** of the app's repository, named in its `package.json` with an
`exports` entry per module:
<!-- quote: package.json -->
```json
{
"name": "@todo/app",
"version": "0.0.1",
"private": true,
"description": "The tutorial's todo app: its API (v1 and its per-kind modules), base code and built-in plugins",
"type": "module",
"exports": {
"./v1": "./src/api/v1/index.ts",
"./v1/server": "./src/api/v1/server.ts",
"./v1/web": "./src/api/v1/web.ts",
"./v1/tui": "./src/api/v1/tui.ts"
}
}
```
The repository's root `package.json` lists it in `workspaces` (here Chord's repository lists
`packages/chord/test/examples/61-tutorial`), so `npm install` links `node_modules/@todo/app` to it. Then Node,
TypeScript, the tests and esbuild resolve `@todo/app/v1` alike, from base code, from a plugin's files and from a test
importing a plugin's `contract.ts`; plugin builds still leave it out and pass the process's instance.
## The start script
`startServer` takes what the app decides and does the rest. `start.ts` names the API with this process's modules, the
facet kinds (which files a plugin may have), and the principals:
<!-- quote: start.ts -->
```ts
export const API = {
name: "@todo/app",
versions: ["v1"],
modules: { "@todo/app/v1": v1, "@todo/app/v1/server": v1server },
};
/** The app's facet kinds: a plugin has one file per kind it runs on (`server.ts`, `worker.ts`, `web.ts`, `tui.ts`). */
export const KINDS = {
server: { platform: "node" },
worker: { platform: "node" },
web: { platform: "browser", packages: { react: {}, "react/jsx-runtime": {} } },
tui: { platform: "node" },
} as const;
```
<!-- quote: start.ts -->
```ts
export function startTodo(
dataDir: string,
root: string,
telemetry: TodoTelemetry,
context: Context,
): Promise<App<TodoPrincipal>> {
return startServer<TodoPrincipal>(
{
name: "todo",
dataDir,
root,
listen: { host: "127.0.0.1", port: 0 },
api: API,
kinds: KINDS,
principal: { server: { kind: "server" }, worker: (address) => ({ kind: "worker", list: address.key }) },
// The server's spans and metrics; each list's worker opens the same file (`worker.ts`).
telemetry: telemetry.store,
workers: [
{
kind: v1.List,
entry: fileURLToPath(new URL("./worker.ts", import.meta.url)),
execArgv: ["--conditions=source", "--experimental-strip-types", "--no-warnings"],
env: { TODO_TELEMETRY: telemetryFile(dataDir) },
start: () => ({}),
// Who may reach a list's worker: everyone signed in, and the server itself.
authorize: (principal) => principal.kind !== "worker",
},
],
// The page, and the owner's trace viewer: the same page, which shows the viewer at `/traces`.
web: { entry: fileURLToPath(new URL("./src/web/main.ts", import.meta.url)), page, paths: ["/", "/traces"] },
tui: { entry: fileURLToPath(new URL("./src/tui/main.ts", import.meta.url)) },
onError: (error) => console.error("todo:", error.kind, error.error),
},
context,
);
}
```
`workers` is chapter II, `web` chapter IV (its second path, `/traces`, chapter XI), `tui` chapter V, `telemetry`
chapter XI. `listen` asks for a free port on loopback, as the example does; a real app picks a port. The worker entry
is a few lines (its telemetry store is chapter XI's too):
<!-- quote: worker.ts -->
```ts
// The todo app's worker entry: one process per list. `startWorker` opens the list's store in its directory, connects
// to the server and loads the plugins' worker facets. The worker records to the app's telemetry store, the file its
// server named in the environment: its methods called through the server continue the caller's trace.
import { startWorker } from "../../../src/node/app/index.ts";
import { openTelemetry } from "./src/telemetry.ts";
import { API, KINDS } from "./start.ts";
const telemetry = await openTelemetry(process.env.TODO_TELEMETRY!);
startWorker({ api: API, kinds: KINDS, telemetry: telemetry.store, onError: (error) => console.error("list:", error) });
```
## Running it
Step 1 of `main.ts` starts the app, tries the page without signing in, then does what a browser does with the owner
link:
<!-- quote: main.ts -->
```ts
const telemetry = await openTelemetry(telemetryFile(dataDir));
const app = await startTodo(dataDir, root, telemetry, context);
const link = new URL(app.ownerLink());
const url = link.origin;
console.log("1. owner link:", `http://127.0.0.1:<port>${link.pathname}#<token>`);
console.log(" data directory:", (await readdir(dataDir)).sort().join(", "));
console.log(" GET / signed out:", (await fetch(url)).status);
const signIn = await fetch(`${url}/_chord/owner`, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ token: link.hash.slice(1) }),
});
const cookie = signIn.headers.get("set-cookie")!.split(";")[0]!;
console.log(" the owner signs in:", signIn.status, cookie.replace(/=.*/, "=…"));
console.log(" GET / signed in:", (await fetch(url, { headers: { cookie } })).status);
```
<!-- output: main -->
```text
1. owner link: http://127.0.0.1:<port>/_chord/owner#<token>
data directory: logs, owner-token, plugins, server, telemetry.sqlite, telemetry.sqlite-shm, telemetry.sqlite-wal, workers
GET / signed out: 401
the owner signs in: 200 chord-session=…
GET / signed in: 200
```
`startServer` made the data directory: `server/` (its store, the cookie secret, its files), `workers/` (one directory
per worker), `plugins/` (the user's plugin root, beside the app's built-in one), `logs/` and `owner-token`;
`telemetry.sqlite` (with SQLite's `-shm` and `-wal` files) is the app's own, opened before the server starts
(chapter XI). The page
answered 401: HTTP **denies by default**, before any plugin code runs, unless a route says `public: true`.
The **owner link** (`/_chord/owner#<token>`) is how the owner gets in: a single-use token, valid ten minutes, made at
each start. It sits in the URL's fragment, which browsers never send, so it never reaches a log; the link's page posts
it and receives a session cookie. A command-line client on the same machine reads `owner-token` instead (mode 0600,
chapter III). Nothing is the owner because it comes from loopback: any local process could.
# A List on Its Own Worker {#worker lede="A host kind, a remote contract, a worker facet with a table and a live query, and the principal that adds a todo."}
Each list gets its own process, a **worker**, with its own store: a list that misbehaves cannot take the others down,
and an idle list costs nothing. This chapter declares where lists live, what a list offers, and the worker code that
keeps its todos.
## Where a list lives
A **host kind** declares addressable hosts with params; `List({ id: "groceries" })` is an **address**. The app's
host kinds are base code, in the API:
<!-- quote: src/api/v1/index.ts -->
```ts
/** A todo list: a worker process of its own, with its own store. */
export const List = hostKind("list", { id: str() });
```
A `remote` contract says, with `hostAddress`, at which address its objects live. The contract for a list's todos is
the lists plugin's, in its `contract.ts`, which its facets of every kind import, and so do the steps and its test:
<!-- quote: plugins/lists/contract.ts -->
```ts
import {
arr,
bool,
int,
List,
MessageSchema,
method,
nul,
num,
obj,
remote,
type Static,
str,
union,
} from "@todo/app/v1";
/** A todo: its number in the list, its text, whether it is done, who added it and when. */
export const Item = obj({ id: int(), text: str(), done: bool(), by: str(), at: num() });
/** A todo (see `Item`). */
export type Item = Static<typeof Item>;
/** The list's latest notice, as data each viewer renders in its own language; `null` before the first. */
export const Notice = union([MessageSchema, nul()]);
/**
* A list's todos as one principal sees them: the newest 50 (live) and the list's notice, adding one and ticking one off
* (or on again); the owner also clears the done ones.
*/
export const Todos = remote("todo.todos", {
version: 1,
params: { list: str() },
hostAddress: ({ list }) => List({ id: list }),
variants: {
member: {
state: { items: arr(Item), notice: Notice },
methods: { add: method(str()), tick: method(obj({ id: int(), done: bool() })) },
},
owner: {
state: { items: arr(Item), notice: Notice },
methods: { add: method(str()), tick: method(obj({ id: int(), done: bool() })), clear: method() },
},
},
});
```
Its `notice` is a message as data, which chapter VIII renders in each viewer's language. `params` select the
instance: `Todos { list: "groceries" }` is one object per principal. Nobody who asks names a
host: the caller computes the address from the params and sends the request there; a host that cannot serve it passes
it on to its parent or to a route (design §3, "Host kinds and addresses"). The kit's `workers` option is that route:
<!-- quote: start.ts -->
```ts
workers: [
{
kind: v1.List,
entry: fileURLToPath(new URL("./worker.ts", import.meta.url)),
execArgv: ["--conditions=source", "--experimental-strip-types", "--no-warnings"],
env: { TODO_TELEMETRY: telemetryFile(dataDir) },
start: () => ({}),
// Who may reach a list's worker: everyone signed in, and the server itself.
authorize: (principal) => principal.kind !== "worker",
},
],
```
`env` names the app's telemetry store for the worker (chapter XI). `authorize` runs for every request routed to a
list before anything starts; the first request for an address starts
its worker, and a worker idle for five minutes (the default `idleMs`) stops. **Variants** are the member sets a
contract offers: the owner's `Todos` has `clear`, a member's does not. The factory picks one per principal.
## The worker facet
The lists plugin is a directory with a `package.json` (its name is the plugin's name) and one file per host kind it
runs on. `worker.ts` is its **worker facet**: what every list's worker serves. It keeps the todos in a **table** of the
worker's store:
<!-- quote: plugins/lists/worker.ts -->
```ts
/** Every todo of the list, by its number. */
const Items = defineTable("todo.items", {
version: 1,
columns: { id: int(), text: str(), done: bool(), by: str(), at: num() },
key: ["id"],
});
/** The newest todos first. */
const NEWEST = query(Items).orderBy((item) => item.id, "desc");
/** How many todos the live query holds. */
const WINDOW = 50;
```
A table holds many records with queries and pages; it lives in an object, never in a facet. Persisted data lives only
in `local` objects with empty params (one per host and type), so its address never depends on who asked (design §9,
"Lifetime, persistence"). That object is the plugin's store, `Store`, a local contract of its own file:
<!-- quote: plugins/lists/worker.ts -->
```ts
/** The list's todos, once per worker: empty params, so it lives as long as the worker and may keep data in its store. */
const Store = local("todo.store", {
version: 1,
params: {},
members: {
items: arr(Item),
notice: Notice,
add: method(obj({ text: str(), by: str() })),
tick: method(obj({ id: int(), done: bool() })),
clear: method(obj({ by: str() })),
},
});
```
A `local` object with empty params lives as long as its host: it starts with the worker and is never ended by a count
of references. Its factory declares its parts and returns its members:
<!-- quote: plugins/lists/worker.ts -->
```ts
host.serve(Store, (_params, self) => {
const items = self.table(Items);
// The newest todos, live: read before the store goes live, read again after every commit writing the table.
const newest = self.query(items, NEWEST, WINDOW);
const notices = self.state(Notices);
return {
// Oldest first, as the list shows them.
items: self.derive([newest], ([rows]) => [...rows].reverse()),
notice: self.derive([notices], ([value]) => value.notice),
```
A **factory** is synchronous and only declares; Chord activates the object afterwards. `self.table` declares the
table. `self.query(items, NEWEST, WINDOW)` declares a **live query** over it: the rows of `NEWEST`, at most 50, as a
value the object can expose or derive from. Chord runs it before the store goes live, and again after every commit
of the store that writes `items`, and sends watchers only what changed. The store's derive turns the rows around,
oldest first, for the list.
A live query reads only its own object's tables: only the store writes them, so its own commits are exactly the
moments the rows can change, and a commit that writes no table with a query costs nothing. The limit is required:
each such commit runs the query once more, at most `WINDOW` rows. That is cheap for a list a person edits and wrong
for a table written hundreds of times a second (design §11, "there may be dragons").
Writes go through **transactions**: `self.tx(fn, ctx)` runs `fn` with `tx`, and what it stages is committed at once
when `fn` returns, or not at all if it throws. The live query follows in the same commit; the store keeps nothing
else in memory, not even the next todo's number, which it reads in the transaction:
<!-- quote: plugins/lists/worker.ts -->
```ts
add: async ({ text, by }, ctx) => {
// A span of its own under the method's: a trace shows the check, and why it refused.
await ctx.trace("todos.validate", { chars: text.length }, async () => {
// Refused as Chord refuses arguments that fail their schema: the caller gets `invalid/args`.
if (text === "") throw invalid("args", "a todo needs text");
});
await self.tx(async (tx) => {
// The number after the newest todo's: the store's transactions run one after another.
const { items: last } = await tx.page(items, NEWEST, 1);
tx.put(items, { id: (last[0]?.id ?? 0) + 1, text, done: false, by, at: Date.now() });
}, ctx);
return null;
```
A todo without text is refused with `invalid("args", …)`, from the API: the error crosses every link as it is, so a
caller on a page or in a terminal gets `{ code: "invalid", reason: "args" }`, as for arguments that fail the
contract's schema. The check runs in a span of its own (`ctx.trace`), which chapter XI shows in a trace.
## The view per principal
`Todos` is the remote face of the store, one object per principal. Its factory gets the principal the link stamped:
<!-- quote: plugins/lists/worker.ts -->
```ts
host.serve(Todos, (_params, principal, self) => {
const store = self.get(Store, {});
// Who adds a todo comes from the principal the link stamped, never from the arguments.
const by = principal.kind === "user" ? principal.name : principal.kind;
const members = {
items: self.derive([store.items], ([items]) => items),
notice: self.derive([store.notice], ([notice]) => notice),
add: (text: string, ctx: Ctx) => store.add({ text, by }, ctx),
tick: (tick: { id: number; done: boolean }, ctx: Ctx) => store.tick(tick, ctx),
};
if (principal.kind !== "owner") return { variant: "member", members };
return { variant: "owner", members: { ...members, clear: (ctx: Ctx) => store.clear({ by }, ctx) } };
});
```
`self.get(Store, {})` declares a dependency: the view goes live once the store is, and holds a **reference** to it
for its whole life. `by` comes from the principal. A client could put anything in the arguments of `add`; it cannot
change what its link stamped. That is the rule for every plugin: identity from the principal, never from arguments.
## Running it
Step 2 asks for the groceries list twice, from a root acting as the owner and from a root of the server itself:
<!-- quote: main.ts -->
```ts
const OWNER = { kind: "owner" } as const;
await app.host.root(
async (ctx) => {
const todos = await ctx.get(Todos, { list: "groceries" });
await todos.add("milk", ctx);
await todos.add("eggs", ctx);
await todos.tick({ id: 1, done: true }, ctx);
await until(() => todos.items.value[0]?.done === true);
console.log("2. as the owner:", todos.variant, "|", show(todos.items.value));
},
context,
{ principal: OWNER },
);
await app.host.root(async (ctx) => {
const todos = await ctx.get(Todos, { list: "groceries" });
await todos.add("bread", ctx);
await until(() => todos.items.value.length === 3);
console.log(" as the server:", todos.variant, "|", show(todos.items.value.slice(-1)));
}, context);
```
<!-- output: main -->
```text
2. as the owner: owner | [x] milk (owner), [ ] eggs (owner)
as the server: member | [ ] bread (server)
workers: list|groceries
workers/list/list%7Cgroceries: store.sqlite, store.sqlite-shm, store.sqlite-wal
```
A **root** (`host.root(fn, ctx, { principal })`) is a call scope for code outside objects; its `ctx` carries the
principal and cancellation, and the references `ctx.get` takes end when `fn` settles. The first request started the
list's worker, `list|groceries`, with its own store in `workers/list/list%7Cgroceries/`.
## Trap: who a reference acts as
The two roots made the same call and got different todos: `milk (owner)` and `bread (server)`. **A reference acts as
whoever made it.** `ctx.get` in a root or a method acts as that context's principal; `host.get`, and `self.get` in a
`local` object, act as the host; `self.get` in a remote object acts as that object's principal (design §4). So a route
whose request came from the owner reaches the worker as the owner (chapter III), and a guest's call cannot borrow the
server's authority. The surprise is the reverse case: a server method that wants to act as the server towards a worker
must not use its caller's `ctx.get`; it takes the reference with `self.get` in a local server object, or in a root of
the server's own.
## Trap: await what you start
Every write above is awaited, and that is not style. What is started with a context ends with it: a root's scope seals
when `fn` settles, a method's context ends with its call. A `void todos.add(text, ctx)` inside a root that returns at
once is cancelled before it commits; Chord rejects it and reports `transaction_cancelled` through `onError`, so an
empty `catch` cannot hide a write that never happened (design §8). Fire and forget only with a context that does not
end, as the page does with `BACKGROUND_CONTEXT` in chapter IV.
# Routes {#routes lede="A server facet: a JSON API for command-line clients, running as whoever sent the request."}
The page will talk Chord over a WebSocket. A script on Mara's machine is happier with HTTP. A **server facet**
contributes routes to the kit's `Http` registry, which the API's server module re-exports:
<!-- quote: src/api/v1/server.ts -->
```ts
export { Http } from "../../../../../../src/node/http/index.ts";
```
<!-- quote: plugins/lists/server.ts -->
```ts
// The lists plugin on the server: a JSON API for command-line clients. A route runs as whoever sent the request (the
// kit answered 401 before this code runs when nobody signed in), so its `ctx.get` reaches the list's worker as them.
import { defineFacet } from "@todo/app/v1";
import { Http } from "@todo/app/v1/server";
import { Todos } from "./contract.ts";
export default defineFacet((_host, self) => {
self.contribute(Http, {
routes: {
"GET /api/lists/:list": async (request, ctx) => {
const todos = await ctx.get(Todos, { list: request.params.list! });
return { kind: "json", value: todos.items.value };
},
"POST /api/lists/:list": async (request, ctx) => {
const todos = await ctx.get(Todos, { list: request.params.list! });
await todos.add(String(await request.json()), ctx);
return { kind: "json", value: "added" };
},
},
});
});
```
A **registry** is an extension point: one owner declares how contributions combine (here the kit's HTTP component
builds a router), plugins contribute one value each. `self.contribute` in a facet adds the plugin's routes while the
plugin is loaded. A route runs in a root with the request's principal, so `ctx.get(Todos, …)` reaches the list's
worker as whoever signed in, and gets their variant.
Step 3 is a command-line client with the owner token file:
<!-- quote: main.ts -->
```ts
const token = (await readFile(app.ownerTokenFile, "utf8")).trim();
const cli = { authorization: `Bearer ${token}` };
const posted = await fetch(`${url}/api/lists/groceries`, {
method: "POST",
headers: { ...cli, "content-type": "application/json" },
body: JSON.stringify("butter"),
});
console.log("3. POST /api/lists/groceries:", await posted.json());
const listed = (await (await fetch(`${url}/api/lists/groceries`, { headers: cli })).json()) as Item[];
console.log(" GET /api/lists/groceries:", show(listed));
console.log(" GET without a token:", (await fetch(`${url}/api/lists/groceries`)).status);
```
<!-- output: main -->
```text
3. POST /api/lists/groceries: added
GET /api/lists/groceries: [x] milk (owner), [ ] eggs (owner), [ ] bread (server), [ ] butter (owner)
GET without a token: 401
```
`butter` is the owner's: the route acted as the request's principal. Without a token the kit answered 401 before the
route ran. Another machine has no token file; it **pairs** instead: it asks `POST /_chord/pair` for a code, the owner
approves the code in a signed-in browser, and the client's next poll returns its token (example 54).
# A Page {#page lede="A web entry, a web facet with React, the panels registry, and why a watch starts at the current value."}
A page is a client host in the browser. The kit builds its **base bundle** from the app's web entry at start (Chord,
React, the app's web modules), serves the plugins' web facets as small scripts beside it, and serves the page through
the app's template (design §23, "Web and TUI").
## The page's registry
Plugins put panels on the page through a registry the API's web module declares, with Chord's React hooks re-exported
for them:
<!-- quote: src/api/v1/web.ts -->
```ts
import type { ComponentType } from "react";
import { defineRegistry } from "../../../../../../src/index.ts";
export { useHold, useMessages, usePageState, useWatch } from "../../../../../../src/react/index.ts";
/** A panel of the page: its name (a key, never shown) and its React component, which shows its own title. */
export type Panel = { readonly name: string; readonly Component: ComponentType };
/** The page's panels, one per contributing plugin, in plugin-name order. */
export const Panels = defineRegistry<Panel, readonly Panel[]>("todo.panels", { version: 1 });
```
A panel shows its own title, in the page's language; its `name` is only a key. The web entry serves `Panels` (its
`compose` keeps the contributions in plugin-name order), renders every panel, and starts the page's connection:
<!-- quote: src/web/main.ts -->
```ts
/** The page's own facet: it serves `Panels`, composing the contributions in plugin-name order. */
const core = defineFacet((host) => host.serve(v1web.Panels, { compose: (table) => table.map(({ value }) => value) }));
function Page() {
const panels = useRegistry(v1web.Panels) ?? [];
// The page's one other path (`start.ts` serves the page at `/` and `/traces`).
if (location.pathname === "/traces") return h("main", { className: "wide" }, h("h1", null, "Todo"), h(TraceViewer));
return h(
"main",
null,
h("h1", null, "Todo"),
...panels.map(({ name, Component }) => h("section", { key: name }, h(Component))),
);
}
void startPage({
connect: () => new WebSocket(`${location.origin.replace(/^http/, "ws")}/chord`),
modules: { "@todo/app/v1": v1, "@todo/app/v1/web": v1web, react: React, "react/jsx-runtime": jsx },
core: [{ name: "@todo/web", facet: core }],
render: (host) => createRoot(document.getElementById("root")!).render(h(HostProvider, { host }, h(Page))),
});
```
The path `/traces` is the owner's trace viewer (chapter XI). `startPage` makes the page's host, connects it to `/chord`
(reconnecting after drops), loads the current plugin scripts with the entry's core facets, and keeps them current.
`HostProvider` gives the React tree that host.
## The web facet
`plugins/lists/web.ts` contributes a panel. Its component holds the list's `Todos` while it is mounted and re-renders
when its items change:
<!-- quote: plugins/lists/web.ts -->
```ts
function TodoList() {
const { t } = useMessages(messages);
const list = location.hash.slice(1) || "groceries";
const todos = useHold(Todos, { list });
const items = useWatch(todos?.items) ?? [];
const notice = useWatch(todos?.notice);
// Kept by the page, not the component: the new code's component finds the half-typed todo.
const [text, setText] = usePageState(`lists.draft.${list}`, "");
const add = (event: { preventDefault(): void }) => {
event.preventDefault();
if (todos === undefined || text.trim() === "") return;
void todos.add(text, context);
setText("");
};
return h(
"div",
{ className: "lists-panel" },
h("h2", null, t("title")),
h(
"ul",
null,
...items.map((item) =>
h(
"li",
{
key: item.id,
className: item.done ? "lists-done" : undefined,
onClick: () => void todos?.tick({ id: item.id, done: !item.done }, context),
},
item.text,
h("span", { className: "lists-by" }, item.by),
),
),
),
h(
"form",
{ onSubmit: add },
h("input", {
value: text,
placeholder: t("placeholder"),
onChange: (event: { target: { value: string } }) => setText(event.target.value),
}),
),
todos?.variant === "owner"
? h("button", { type: "button", onClick: () => void todos.clear(context) }, t("clearDone"))
: null,
notice === undefined || notice === null ? null : h("p", { className: "lists-notice" }, t.render(notice)),
);
}
export default defineFacet((_host, self) => {
self.contribute(Translations, messages.translations);
self.contribute(Panels, { name: "groceries", Component: TodoList });
});
```
`useMessages(messages)` gives the plugin's texts in the page's language (chapter VIII; `messages` is the plugin's
`messages.ts`, and the facet contributes its catalogs to the page's `Translations`). `useHold` holds a reference while
the component is mounted and releases it on unmount; it is `undefined` on the first render. `useWatch` follows a member. `usePageState` is `useState` whose value the page keeps under a key instead of the
component: chapter IX shows why the half-typed todo needs it. Clicks call methods with `BACKGROUND_CONTEXT`, which never ends, so a click's call
is not cancelled when the handler returns. `todos?.variant === "owner"` narrows the reference: only the owner's has
`clear`, and TypeScript knows it. The panel's look reads the active theme's CSS variables, with class names prefixed
by the plugin's name (a `*.module.css` file would scope them instead):
<!-- quote: plugins/lists/lists.css -->
```css
.lists-panel input { width: 100%; box-sizing: border-box; padding: 0.4rem; background: var(--bg); color: var(--fg);
border: 1px solid var(--accent); border-radius: var(--radius, 0); }
```
The template in `start.ts` puts the assets the kit made into the app's own HTML:
<!-- quote: start.ts -->
```ts
function page(request: { readonly nonce: string }, assets: PageAssets): string {
return `<!doctype html><html lang="en"><head><meta charset="utf-8"><title>Todo</title>${pageScripts(assets, request.nonce)}</head><body><div id="root"></div></body></html>`;
}
```
## Running it
Step 4 reads the page as a browser gets it and watches the list from the server:
<!-- quote: main.ts -->
```ts
const html = await (await fetch(url, { headers: cli })).text();
console.log("4. the page loads:");
for (const [, asset] of html.matchAll(/(?:src|href)="([^"]+)"/g)) console.log(` ${hashless(asset!)}`);
const groceries = await app.host.get(Todos, { list: "groceries" }, context);
const watch = await groceries.items.watch(context);
console.log(" watch.value:", done(watch.value), "done");
const changed = Promise.withResolvers<readonly Item[]>();
watch.start(async (call) => {
if (call.kind === "value") changed.resolve(call.value);
});
await groceries.tick({ id: 2, done: true }, context);
console.log(" the listener's first call:", done(await changed.promise), "done");
await watch.stop();
```
<!-- output: main -->
```text
4. the page loads:
/_chord/web/base.<hash>.css
/_chord/web/plugin/@todo~lists.<hash>.css
/_chord/web/plugins.<hash>.js
/_chord/web/base.<hash>.js
watch.value: 1 done
the listener's first call: 2 done
```
The page is the template, the base bundle (`base.<hash>.js` and `.css`), one combined file with every plugin's web
script and one stylesheet per plugin with CSS: two scripts on a cold load, cacheable forever by their hashes. After the
owner opened the owner link in Chromium, the page shows the list:
:::figure | The page after the owner signed in: the groceries list, live, each todo with who added it; the owner's variant shows "Clear done".
![The todo page](screens/page.png)
:::
## Trap: a watch starts at the current value
`watch.value` was the value when the watch was taken (one todo done); the listener's first call was the change after
it (two done). **A watch's listener gets only changes.** Code that renders from a listener alone draws nothing until
the first change. Read `handle.value` first, then `start` (design §11, "Watchables"). `useWatch` and the terminal's
`TuiScope.watch` do this for you; `self.on`, an object's routine, is the one consumer whose first call is the current
value.
# A Terminal {#terminal lede="The same plugin in a terminal: a terminal facet with TuiScope, run in the launcher's sandbox."}
The terminal client is a client host too, run by a **launcher**: a trusted Node program that signs in, downloads the
terminal's base bundle and the plugins' terminal scripts, and runs them in a **sandbox**, a child process with no file
system, network, processes or terminal of its own. The launcher relays its link and draws its frames. Code from the
server never touches Mara's machine directly (decision "The terminal's sandbox is a child process").
The API's terminal module declares the parts of the screen:
<!-- quote: src/api/v1/tui.ts -->
```ts
export type Screen = {
readonly name: string;
readonly open: (scope: TuiScope, theme: TerminalTheme) => () => readonly string[];
};
/** The terminal's parts, in plugin-name order. */
export const Screens = defineRegistry<Screen, readonly Screen[]>("todo.screens", { version: 1 });
```
The terminal entry serves `Screens` and redraws on every change. A **`TuiScope`** holds references, follows values and
registries, calls one callback on every change, and releases everything when disposed; it is the terminal's
counterpart of the React hooks, for any terminal library:
<!-- quote: src/tui/main.ts -->
```ts
/** The terminal's own facet: it serves `Screens`, composing the contributions in plugin-name order. */
const core = defineFacet((host) => host.serve(v1tui.Screens, { compose: (table) => table.map(({ value }) => value) }));
void startTerminalRuntime({
modules: { "@todo/app/v1": v1, "@todo/app/v1/tui": v1tui },
core: [{ name: "@todo/tui", facet: core }],
render: (runtime) => {
const { terminal, theme } = runtime;
const screens = new TuiScope(runtime.host, () => draw()).registry(v1tui.Screens);
let opened: { screens: readonly v1tui.Screen[]; scope: TuiScope; lines: (() => readonly string[])[] } | undefined;
const draw = (): void => {
const current = screens() ?? [];
if (opened?.screens !== current) {
opened?.scope.dispose();
const scope = new TuiScope(runtime.host, () => draw());
opened = { screens: current, scope, lines: current.map((screen) => screen.open(scope, theme)) };
}
const lines = opened.lines.flatMap((read) => read());
terminal.write(`\x1b[2J\x1b[H${lines.join("\r\n")}\r\n`);
};
theme.onChange(draw);
terminal.onInput((data) => {
if (data === "q") runtime.exit(0);
});
draw();
},
});
```
When a reload changes `Screens`, the entry disposes the old scope and opens the screens again. The lists plugin's
terminal facet draws its title, the todos and the list's notice; `scope.messages(messages)` is the terminal's
`useMessages` (chapter VIII):
<!-- quote: plugins/lists/tui.ts -->
```ts
export default defineFacet((_host, self) => {
self.contribute(Translations, messages.translations);
self.contribute(Screens, {
name: "groceries",
open: (scope, theme) => {
const t = scope.messages(messages);
const todos = scope.hold(Todos, { list: "groceries" });
const items = scope.watch(todos.items);
const notice = scope.watch(todos.notice);
return () => {
const shown = notice();
return [
theme.fg("accent", t("title")),
...(items() ?? []).map((item) =>
item.done ? theme.fg("muted", `[x] ${item.text}`) : `[ ] ${item.text} ${theme.fg("muted", item.by)}`,
),
...(shown === undefined || shown === null ? [] : [theme.fg("warning", t.render(shown))]),
];
};
},
});
});
```
`theme.fg("muted", …)` colours by role, never by a fixed colour. Step 5 runs the launcher on a scripted terminal, its
languages fixed (by default they are the machine's system locale):
<!-- quote: main.ts -->
```ts
if (sandboxAvailable()) {
const output = Object.assign(new PassThrough(), { columns: 60, rows: 12 });
output.on("data", (chunk: Buffer) => frames.push(chunk.toString()));
terminal = await startTerminal({
url,
credentials: { kind: "file", path: app.ownerTokenFile },
input: new PassThrough(),
output,
colors: "truecolor",
// The machine's languages (its system locale by default), here fixed for the book's output.
locales: ["en-US"],
});
await until(() => screen(frames).includes("butter"));
console.log("5. the terminal shows:");
for (const line of screen(frames).trim().split("\r\n")) console.log(` ${line}`);
} else console.log("5. (the terminal client needs Node 26.5.1 or later for its sandbox)");
```
<!-- output: tui -->
```text
5. the terminal shows:
Groceries
[x] milk
[x] eggs
[ ] bread server
[ ] butter owner
```
# A Theme {#theme lede="A theme is data in a registry; a plugin added while the app runs restyles the page and the terminal in place."}
Mara wants the evening look. A theme is data: colours by role, a few web extras, terminal overrides. The kit's `Themes`
registry collects them; each user picks one dark and one light theme, the system preference chooses (decision "Looks
and themes"). The Harbor theme is a plugin with its data in one file:
<!-- quote: later/harbor/theme.ts -->
```ts
export const harbor: Theme = {
id: "harbor",
name: "Harbor",
mode: "dark",
colors: {
accent: "#f2a65a",
bg: "#12232e",
fg: "#d8e2e8",
muted: "#6b8796",
error: "#e5676b",
warning: "#e9c46a",
success: "#7fb685",
},
web: { radius: "6px" },
tui: { muted: "66" },
};
```
and two facets contributing it, one for pages and one for terminals (registries are per host, so each kind gets its
own contribution):
<!-- quote: later/harbor/web.ts -->
```ts
import { defineFacet, Themes } from "@todo/app/v1";
import { harbor } from "./theme.ts";
export default defineFacet((_host, self) => self.contribute(Themes, harbor));
```
The plugin is not in the app's root at start. Step 6 copies it in while everything runs and applies the change:
<!-- quote: main.ts -->
```ts
await cp(fileURLToPath(new URL("./later/harbor/", import.meta.url)), join(root, "harbor"), { recursive: true });
await until(() => app.plugins.state.value.set);
const added = await app.applyChanges(context);
console.log("6. harbor added, rolled out to:", added.steps.map(({ name }) => name).join(", "));
const scripts = await app.host.get(WebScripts, {}, context);
console.log(" the page's plugins:", scripts.set.value.plugins.map(({ name }) => name).join(", "));
scripts.release();
```
<!-- output: main -->
```text
6. harbor added, rolled out to: server, list, web, tui
the page's plugins: @todo/french, @todo/harbor, @todo/lists
```
<!-- output: tui -->
```text
the terminal redraws its title in Harbor's accent: "\u001b[38;2;242;166;90mGroceries"
```
A directory appearing in a plugin root marks the plugin set changed (`app.plugins.state.value.set`); nothing happens
until `applyChanges`, which builds the new plugin and rolls it out: the server, the list workers, then the page and
terminal surfaces. The page's host learned of the new script set through Chord, fetched Harbor's web script and loaded
it in place; the terminal did the same with its terminal script and redrew in Harbor's accent. With the browser
preferring dark:
:::figure | The same page after the Harbor plugin was added and applied: no reload, the CSS variables on `:root` changed.
![The page in the Harbor theme](screens/harbor.png)
:::
# A Second User {#user lede="A one-line login plugin, a client host of Ada's, and why a host must load before it asks."}
Ada joins with a name. The kit ships an ephemeral login as a facet (`POST /_chord/login { name }` signs in as `{ kind:
"user", id, name }`, no account stored); the API's server module exports it for the app's principal type:
<!-- quote: src/api/v1/server.ts -->
```ts
/** The kit's ephemeral login: `POST /_chord/login { name }` signs in as `{ kind: "user", id, name }`. */
export const login = ephemeralLoginFacet<TodoPrincipal>();
```
and the app adds it as a **single-file plugin**, a file named `<name>.<kind>.ts` at the root's top:
<!-- quote: plugins/login.server.ts -->
```ts
// The kit's pick-a-name login, added to the app as a single-file plugin: a server facet, one line.
import { login } from "@todo/app/v1/server";
export default login;
```
Step 7 does what Ada's page does: logs in, then connects a client host of her own over the links endpoint with her
cookie:
<!-- quote: main.ts -->
```ts
const login = await fetch(`${url}/_chord/login`, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ name: "ada" }),
});
const adaCookie = login.headers.get("set-cookie")!.split(";")[0]!;
const ada = createHost<TodoPrincipal>({ onError: (error) => console.error("ada:", error.kind) });
try {
ada.hold(Todos, { list: "groceries" }, context);
} catch (error) {
console.log("7. ada's hold before load:", (error as Error).message);
}
await ada.load([], context);
const socket = new WebSocket(`${url.replace(/^http/, "ws")}/chord`, { headers: { cookie: adaCookie } });
ada.connect(webSocketLink(socket as unknown as WebSocketLike), { principal: { kind: "server" } }, context);
const adas = await ada.get(Todos, { list: "groceries" }, context);
await adas.add("coffee", context);
console.log(" ada's variant:", adas.variant);
if (adas.variant === "member") {
// @ts-expect-error `clear` is the owner's: TypeScript refuses it, and so does the worker.
const clearing: Promise<null> = adas.clear(context);
await clearing.catch((error: ChordError) => console.log(" ada's clear:", `${error.code}/${error.reason}`));
}
await until(() => groceries.items.value.some(({ by }) => by === "ada"));
console.log(" the server's reference sees:", show(groceries.items.value.slice(-1)));
```
<!-- output: main -->
```text
7. ada's hold before load: todo.todos: the host has not loaded; call host.load first (a host with nothing to load calls load([]))
ada's variant: member
ada's clear: invalid/member
the server's reference sees: [ ] coffee (ada)
```
## Trap: load before use
Ada's first `hold` threw at once: **a host serves and binds nothing before `load`.** Requests made before it would wait
for a publication nobody started, so Chord refuses them with `not_ready` ("call `host.load` first"); a client with no
plugins of its own calls `load([])` (design §3). `startPage`, the launcher and the kit do it for you; code that makes
a host itself must not forget.
Ada got the `member` variant; her `clear` is refused twice, by TypeScript (the `@ts-expect-error`) and by the worker
(`invalid/member`), because what a reference offers is decided by the provider's variant, not by the caller's code.
Her todo is hers: `coffee (ada)`.
# Languages {#languages lede="Messages in code, a French plugin, a language chosen live, and a notice each viewer reads in its own language."}
Mara reads German; Ada would rather read French. Both look at the same list, served by the same worker, so its text
cannot be formatted once for everyone: **text people read is rendered per viewer, never by a server** (decision
"Messages and languages in the app kit"). A plugin writes its texts once, in code; languages are contributions; the
language is a preference of each user.
## Messages in code
The lists plugin keeps its texts in one file, `messages.ts`, which every facet that renders imports (and the worker,
below):
<!-- quote: plugins/lists/messages.ts -->
```ts
export const messages = defineMessages("todo.lists", {
en: {
title: "Groceries",
placeholder: "Add a todo",
clearDone: "Clear done",
open: (n: number) => (n === 1 ? "1 open" : `${n} open`),
cleared: (by: string) => `${by} cleared the done todos`,
},
de: {
title: "Einkäufe",
placeholder: "Neues Todo",
clearDone: "Erledigte entfernen",
open: (n) => `${n} offen`,
cleared: (by) => `${by} hat die erledigten Todos entfernt`,
},
});
```
The first catalog defines the keys and their argument types, and the English catalog is the source of truth; the
German one gets its functions' parameter types from it, and a missing key, an unknown one or a wrong argument
(`t("open", "3")`) is a type error, not a blank in production. The namespace (`todo.lists`) keeps two plugins' `title`
apart. Functions cover plurals and word order; dates and numbers inside them go through `formatTime` and
`formatNumber`, which use the locale being rendered.
The panel renders with `useMessages(messages)` and the terminal screen with `scope.messages(messages)` (chapters IV
and V): both give a `t` bound to their host's language, and re-render or redraw when it changes. Each facet that
renders also contributes the plugin's own catalogs to its host's `Translations`:
<!-- quote: plugins/lists/web.ts -->
```ts
export default defineFacet((_host, self) => {
self.contribute(Translations, messages.translations);
self.contribute(Panels, { name: "groceries", Component: TodoList });
});
```
Registries are per host, and a host learns from them which languages its plugins offer: the page matches the
browser's languages against them before it picks one.
## A translation plugin
French comes from a plugin of its own, written by someone who never touches the lists plugin. Its catalog is typed by
the lists plugin's messages, a type-only import erased at build:
<!-- quote: plugins/french/fr.ts -->
```ts
import type { MessagesOf, Translation } from "@todo/app/v1";
import type { messages } from "../lists/messages.ts";
const lists: MessagesOf<typeof messages> = {
title: "Courses",
placeholder: "Nouvelle tâche",
clearDone: "Retirer les tâches faites",
open: (n) => `${n} à faire`,
cleared: (by) => `${by} a retiré les tâches faites`,
};
export const french: readonly Translation[] = [{ namespace: "todo.lists", locale: "fr", messages: lists }];
```
and its web and terminal facets contribute that list, the same line in each:
<!-- quote: plugins/french/web.ts -->
```ts
export default defineFacet((_host, self) => self.contribute(Translations, french));
```
A contribution is a list, so one plugin can translate the app and every plugin at once. A catalog may be partial: a
missing key falls back to the locale's base (`pt-BR` to `pt`), then to the namespace's first catalog, key by key, and
each fallback is reported once (the page's `onError`, kind `messages`). Two catalogs for one namespace and locale: the
first by plugin name wins and the other is reported; a translation contributed with `override: true` replaces the
plugin's own (a better German).
## Choosing a language
The language is a preference each user keeps on the server, `LocaleChoice`, one per principal like the theme choice of
chapter VI. Until a user chooses, the device decides: the browser's `navigator.languages` on a page, the system locale
in a terminal (the launcher's `locales`), matched against the languages the catalogs offer, else English. Step 8 has
the owner choose German while the terminal runs:
<!-- quote: main.ts -->
```ts
const ownerChooses = (locale: string) =>
app.host.root(async (ctx) => (await ctx.get(LocaleChoice, {})).choose(locale, ctx), context, { principal: OWNER });
await ownerChooses("de");
```
<!-- output: main -->
```text
8. the owner chooses German
ada's pick: fr
```
<!-- output: tui -->
```text
the terminal shows:
Einkäufe
[x] milk
[x] eggs
[ ] bread server
[ ] butter owner
[ ] coffee ada
```
Every client of the owner holds the owner's `LocaleChoice`, so the terminal redrew in German at once; nothing
restarted. Ada's pick (French) is hers and changes nothing for the owner. A page does the same and also sets `lang`
and `dir` on its root element (right to left for Arabic or Hebrew, through the app's logical CSS properties).
## Text from the server
Clearing the done todos should tell everyone who cleared them. If the worker wrote "owner hat die erledigten Todos
entfernt", Ada would read German too. So the list's state carries a **message as data**, a key and its parameters,
declared with the kit's generic schema:
<!-- quote: plugins/lists/contract.ts -->
```ts
/** The list's latest notice, as data each viewer renders in its own language; `null` before the first. */
export const Notice = union([MessageSchema, nul()]);
```
The store writes it with `messages.message`, typed against the English catalog like `t` (`messages.message("cleared",
42)` does not compile), and each viewer renders it with `t.render`:
<!-- quote: plugins/lists/worker.ts -->
```ts
clear: async ({ by }, ctx) => {
await self.tx(async (tx) => tx.deleteWhere(items, (item) => eq(item.done, true)), ctx);
// The key and its parameters, typed against the English catalog: every viewer renders its own text.
await notices.change((draft) => {
draft.notice = messages.message("cleared", by);
}, ctx);
return null;
},
```
<!-- output: main -->
```text
the notice: { namespace: 'todo.lists', key: 'cleared', params: [ 'owner' ] }
the server renders it: owner cleared the done todos
```
<!-- output: tui -->
```text
the terminal renders it: owner hat die erledigten Todos entfernt
```
Parameters are plain data (strings, numbers, ISO dates), so a message crosses links like any state. Across a link
the member is the generic schema, so `t.render` checks the namespace and the key at runtime: a message of another
namespace or a key the catalogs no longer have shows the key, reported. Never translated: the model's prompts and
answers, log lines, error codes.
## Trap: the plain `t`
The server's `messages.render(…)` printed English. **The plain `t` has no language of its own:** inside a render it
uses the language being rendered (a message function calling another plugin's `t`, `formatTime`), outside one the
first catalog. That is right for a server, a test or a log line, and wrong in a component: a page or a terminal
renders with `useMessages(t)` or `scope.messages(t)`, bound to its host's language. There is no module-wide current
language, because one process runs many hosts and a server renders for many people.
On the page, `screenshots.ts` has the owner choose French and clear the done todos while a todo is half typed:
<!-- quote: screenshots.ts -->
```ts
await app.host.root(
async (ctx) => {
await (await ctx.get(LocaleChoice, {})).choose("fr", ctx);
const todos = await ctx.get(Todos, { list: "groceries" });
if (todos.variant === "owner") await todos.clear(ctx);
},
context,
{ principal: { kind: "owner" } },
);
```
<!-- output: screenshots -->
```text
wrote docs/tutorial/screens/french.png
after choosing French: lang="fr", "oat mi" still typed
the page did not reload: true
```
:::figure | The page after the owner chose French: the lists plugin's texts from the French plugin, the notice rendered from its key, the half-typed "oat mi" kept; no reload.
![The page in French](screens/french.png)
:::
# Changing Code {#reload lede="An edit while Ada is connected: the watcher marks, applyChanges rolls out, every object takes its fate."}
Ada typed " tea " with spaces, and a blank todo should be refused like an empty one. The fix is one line of the lists
plugin's worker facet: the view trims the text before the store sees it. Step 9 makes the edit in the running app's
copy of the plugins, as an editor or an agent would, and applies it:
<!-- quote: main.ts -->
```ts
const BEFORE = "add: (text: string, ctx: Ctx) => store.add({ text, by }, ctx),";
const AFTER = "add: (text: string, ctx: Ctx) => store.add({ text: text.trim(), by }, ctx),";
const workerFile = join(root, "lists", "worker.ts");
await writeFile(workerFile, (await readFile(workerFile, "utf8")).replace(BEFORE, AFTER));
await until(() => app.plugins.state.value.dirty.includes("@todo/lists"));
const applied = await app.applyChanges(context);
console.log("9. lists changed, rolled out to:", applied.steps.map(({ name }) => name).join(", "));
for (const host of applied.steps.find(({ name }) => name === "list")!.hosts) {
const fates = "report" in host ? host.report.objects.map(({ object, kind }) => `${object} ${kind}`) : [];
console.log(` on ${host.host}:`, [...new Set(fates)].join(", "));
}
await adas.add(" tea ", context);
await until(() => groceries.items.value.at(-1)?.text === "tea");
console.log(" ada adds ' tea ':", show(groceries.items.value.slice(-1)));
const refused = await adas.add(" ", context).then(
() => "added",
(error: ChordError) => ({ code: error.code, reason: error.reason }),
);
console.log(" ada adds ' ':", refused);
```
<!-- output: main -->
```text
9. lists changed, rolled out to: server, list, web, tui
on list|groceries: todo.store swapped, todo.todos(groceries) swapped
ada adds ' tea ': [ ] tea (ada)
ada adds ' ': { code: 'invalid', reason: 'args' }
```
The watcher marked `@todo/lists` dirty when the file changed; `applyChanges` rebuilt it (one esbuild call per facet)
and rolled it out host by host. On each host Chord stages the new code (it runs the new factories against a recording
`self` to see what they declare), commits it in one step, and gives each object of the changed plugin its **fate**
(design §21):
| The object | Fate |
| --- | --- |
| opens no natives, and neither does its new factory | **swapped** in place: new methods, the same documents; running calls finish on the old code |
| opens natives | **restarted**: calls wait, running ones finish, natives close and open again with the new code |
| its new code waits for a dependency | **parked** until the dependency is live |
| its type was removed | **ended**; its holders wait until something serves it again |
The store opens no natives, and neither does its new factory: its live query is not a native, Chord runs it. So the
store was **swapped**, not restarted. The sequence on the list's worker:
1. The store's new attempt starts beside the old one: the same object, instance, table and line. Calls arriving now
wait at a gate; calls already running finish on the old code.
2. The new attempt runs its live query (the same read as at the first activation) and goes live; the gate opens,
and references move to its members by name, each with one frame from its last value.
3. The old attempt is sealed once its calls settled; its query leaves with it, so later commits run only the new one.
The views were swapped too, and before the store: Chord starts the fates dependents first, so each view's new
attempt found the store still live, read its old values until step 2 and moved to its new members with one frame;
its calls to the store waited at the gate. Had the store gone first, the views would have found it between steps 1
and 2 and **parked** for that moment. Ada's reference never showed `loading`, and her next add was trimmed.
Had the store kept its window as a one-time read in `self.open`, it would count as holding a native and **restart**
at every reload of its plugin: running calls drained, natives closed and opened again, a new instance.
Ada's blank todo is now empty after trimming, so the store refused it with
`invalid("args", …)` (chapter II): her client got `{ code: "invalid", reason: "args" }`, Chord's own error, which
crossed the worker's link and the server's as it was. Had the edit broken the plugin, that plugin alone would have kept its old code
and the report would name the error.
A contract that changes shape needs a new `version`; old consumers keep working when the provider maps their version
(`accepts`), otherwise they wait with `contract_mismatch` until they reload. A stored document or table that changes
shape bumps its own version with a migration. Contract versions protect callers, storage versions protect data.
## On the page
An edit of a web facet applies in place too: the page fetches the one changed script and reloads its host, with no
page reload. `screenshots.ts` adds a count of open todos to the panel (`t("open", n)`, in the page's language) while half a
todo is typed:
<!-- quote: screenshots.ts -->
```ts
const BEFORE = `h(
"ul",`;
```
<!-- output: screenshots -->
```text
wrote docs/tutorial/screens/page.png
wrote docs/tutorial/screens/harbor.png
wrote docs/tutorial/screens/edited.png
typed "oat mi"; after the edit: "oat mi"
the page did not reload: true
```
:::figure | After the edit of the web facet, applied in place: the count appears, and the half-typed "oat mi" is still in the box: the page kept it under its key while React mounted the new component.
![The page after the edit](screens/edited.png)
:::
The page did not reload, and everything outside the changed plugin's components kept its state. The changed panel
itself is new code: its component is a new function, which React mounts anew, so its own `useState` would start over
(design §21). What must survive a plugin update (a half-typed todo, a scroll position) is **page state**:
`usePageState(key, initial)` keeps the value in the page's host under the key, not in the component, so the new
component finds it; a terminal's `TuiScope` has the same store as `scope.pageState(key, initial)`. Page state is any
value, never sent anywhere, and gone with the page.
# Moving a List {#export lede="Export a worker's data while it runs, import it under another address."}
A list's data is what its worker created: its store and its files. Step 10 exports the groceries list while its worker
runs and imports it as a new list:
<!-- quote: main.ts -->
```ts
const archive = join(base, "groceries.tar");
await app.exportData(List({ id: "groceries" }), archive, context);
console.log("10. exported groceries:", (await readFile(archive)).length > 0 ? "an archive" : "nothing");
await app.importData(archive, List({ id: "picnic" }), context);
const picnic = await app.host.get(Todos, { list: "picnic" }, context);
console.log(" imported as picnic:", show(picnic.items.value));
picnic.release();
await app.importData(archive, List({ id: "groceries" }), context).catch((error: Error) => {
console.log(" importing over groceries:", error.message.replace(base, "…"));
});
```
<!-- output: main -->
```text
10. exported groceries: an archive
imported as picnic: [ ] bread (server), [ ] butter (owner), [ ] coffee (ada), [ ] tea (ada)
importing over groceries: importData: list|groceries has data (its worker runs)
```
`exportData` stops nothing: the running worker writes its own store at one consistent point (the store's export holds
its commit queue while it reads) and its files, as one archive with a manifest naming the app. With no worker running,
the server takes the worker's process lock and reads the store itself. `importData` refuses an address that has data;
replacing a list is a delete, then an import (decision "App kit naming and export"). Registering an imported list in
an index of the app's, if it had one, is the app's.
# Seeing What Happened {#telemetry lede="One telemetry store for every process, one trace from a click to a commit, a span of your own, and a small viewer for the owner."}
Ada says that adding a todo from her phone feels slow some evenings. Her todo crosses three processes: her page, the
server's relay and the list's worker. A log line per process would say what each one did, but not which lines belong
to her click, or where the time went. Chord records **telemetry** through pi-telemetry: **spans** (a named, timed
piece of work with attributes), **events** on spans, and **metrics** (counters and histograms). Each host has one
**adapter**, given to `createHost({ telemetry })` and NOOP by default, which decides where they go. A **carrier** in
each call's header names the caller's span, so the spans of one request, recorded by different processes, form one
**trace** (design §29). Mara wants to see them without reading SQL: a small trace viewer would be nice. This chapter
records, follows one click, adds a span, reads the store from a script, and ends with the viewer.
## Recording
The app keeps its telemetry in one SQLite file in the data directory, through Chord's store
(`@earendil-works/chord/telemetry-sqlite`). The app's base code opens it, for any process that asks:
<!-- quote: src/telemetry.ts -->
```ts
/** The telemetry store's file in the data directory `dataDir`. */
export function telemetryFile(dataDir: string): string {
return join(dataDir, "telemetry.sqlite");
}
/** Opens the telemetry store at `file` for this process (its directory made, private, when missing). */
export async function openTelemetry(file: string): Promise<TodoTelemetry> {
// Spans carry who called what: the data directory is private before the store's first file appears in it.
await mkdir(dirname(file), { recursive: true, mode: 0o700 });
const db = await openNodeSqliteDatabase(file);
const store = await createSqliteTelemetry(db);
return {
store,
close: async () => {
await store.close();
await db.close();
},
};
}
```
`createSqliteTelemetry` returns a **recorder**: an adapter that keeps each settled span in memory and writes them in
one batch a second, one row per span with its events and links inline; metrics are summed per series and 10 seconds;
rows older than seven days are deleted in the same batch, and when 10,000 spans wait, new ones are dropped and counted.
Recording never runs SQL. The same object is the store's **read interface**, `TelemetryStore` (below).
Step 1 opened the store before the server (chapter I), and `startTodo` gives it to `startServer`, which passes it to the
server's host and its HTTP component:
<!-- quote: start.ts -->
```ts
// The server's spans and metrics; each list's worker opens the same file (`worker.ts`).
telemetry: telemetry.store,
```
The kit carries no spans between processes: each process records to the adapter it was given (slice 9, §5). So each
list's worker opens the same file with a recorder of its own; the server names the file in the worker's environment:
<!-- quote: start.ts -->
```ts
env: { TODO_TELEMETRY: telemetryFile(dataDir) },
```
<!-- quote: worker.ts -->
```ts
const telemetry = await openTelemetry(process.env.TODO_TELEMETRY!);
startWorker({ api: API, kinds: KINDS, telemetry: telemetry.store, onError: (error) => console.error("list:", error) });
```
One file for every process is the app's choice. SQLite in WAL mode lets several processes write it, each waiting for
the others' locks (up to 5 seconds), and every span id is global, so rows of different processes never collide. An app
with a collector would give each process an adapter that exports there instead; nothing else changes.
## One trace across three hosts
A page in a browser cannot record yet: `startPage` takes no telemetry adapter. Step 11 makes the page itself, as Ada's
client was made in chapter VII: a client host of this process with a recorder of its own on the same file, signed in
with the owner's cookie from step 1. A click is one span of the page, and the call runs under it:
<!-- quote: main.ts -->
```ts
const pageTelemetry = await openTelemetry(telemetryFile(dataDir));
const page = createHost<TodoPrincipal>({
telemetry: pageTelemetry.store,
onError: (error) => console.error("page:", error.kind),
});
await page.load([], context);
const pageSocket = new WebSocket(`${url.replace(/^http/, "ws")}/chord`, { headers: { cookie } });
page.connect(webSocketLink(pageSocket as unknown as WebSocketLike), { principal: { kind: "server" } }, context);
const pageTodos = await page.get(Todos, { list: "groceries" }, context);
const click = (text: string) =>
pageTelemetry.store.startSpan({ name: "page.add" }, (span) => pageTodos.add(text, withTelemetry(span, context)));
await click("honey");
await click(" ").catch(() => "refused");
```
`startSpan` runs the callback in a new root span; `withTelemetry(span, context)` makes it the context's span, so the
call of `add` starts its `chord.call` under it and puts that span's carrier in the call's header. Each recorder writes
once a second, so the step reads the store until the worker's spans are there, takes the older of the two traces whose
first span is `page.add` and prints it with pi-telemetry's `renderTrace` (main.ts masks the lengths of time, which
differ at every run):
<!-- output: main -->
```text
11. adding honey from the page:
page.add
chord.call type="todo.todos" member="add" object="todo.todos(groceries)" address.kind="list" outcome="ok"
chord.relay type="todo.todos" member="#0" upstream.kind="list" outcome="ok"
chord.method type="todo.todos" member="add" object="todo.todos(groceries)" principal.kind="owner" outcome="ok"
chord.method type="todo.store" member="add" object="todo.store" principal.kind="worker" outcome="ok"
· chord.commit documents=0 ops=0 duration=<ms>
todos.validate chars=5
```
Three processes recorded these six spans. The page recorded `page.add` and `chord.call`, the consumer's call until its
reply. The server recorded `chord.relay`: it forwarded the call to the list's worker without knowing member names
(`#0` is the method's index). The worker recorded the rest: `chord.method` for the owner's `Todos` view, then the
view's call of the store, which is a method span too, by `worker`, because the store acts as its host (chapter II).
The commit is an event on the span its transaction ran under, and the renderer prints a span's events before its
children. Its counts are documents and their ops; a commit of table rows counts neither, so this one says 0 and 0.
:::sequence animate | One click, one trace: each process starts its spans with its own recorder, and the carrier in each call's header names the parent span on the other side.
actor page "page" "recorder 1"
actor server "server" "recorder 2"
actor worker "list|groceries" "recorder 3"
page -> page: page.add, chord.call
page => server: call add, carrier of chord.call
server -> server: chord.relay
server => worker: call #0, carrier of chord.relay
worker -> worker: chord.method ×2, todos.validate, commit
worker --> server: reply
server --> page: reply
:::
The carrier is a field of the `call` and `hold` headers, at most 8 keys of at most 256 characters each. The receiver
reads it with its own adapter; an absent or malformed one gives a new root, never an error. With the NOOP adapter
nothing is injected and no span starts: a span site costs one check (§30).
The trace holds the click and nothing else. Work that serves many requests is its own trace, **linked** to the span that
first needed it: a worker's start (`chord.resolve`), an object's activation (`chord.activate`). A link is a reference to
a span of another trace; the renderer prints it and never follows it.
## Trap: a trace starts where a span starts
The page took its reference to `Todos` before the click, under `BACKGROUND_CONTEXT`, and that hold is in no trace of
the page: **a context without a recording span records nothing.** A host gives such work no span of its own, so its
calls start no `chord.call` and send no carrier, and the provider's `chord.method` becomes a root of its own trace
(the server's relay of the page's hold is one: `chord.relay` with `member="hold"`). So were the server's calls in step 2:
on the worker each is a root `chord.method`. Start a span where the work begins, a click handler or a job, and pass it
down with `withTelemetry`. The kit's HTTP component does this for every request: a `chord.http` span (method, route
pattern, status), with the route's `ctx` under it.
## A span of your own
The store's `add` checks the text in a span of its own:
<!-- quote: plugins/lists/worker.ts -->
```ts
await ctx.trace("todos.validate", { chars: text.length }, async () => {
// Refused as Chord refuses arguments that fail their schema: the caller gets `invalid/args`.
if (text === "") throw invalid("args", "a todo needs text");
});
```
`ctx.trace(name, attributes, fn)` starts a child of the method's span and gives `fn` a child `ctx` with the same
principal and `get`, so calls made inside are parented under it. It is the last line of the trace above:
`todos.validate chars=5`. Its attributes are small, flat values; the todo's text is data and stays out of telemetry,
its length is enough. When `fn` throws, the span fails and the error passes through as it was: the caller still gets
`invalid/args`. A check this small does not need a span of its own; a lookup, a model request or a file read does.
## Reading
The store is read through `TelemetryStore`, pi-telemetry's read interface modeled on Jaeger's: `findTraces` (complete
traces, newest first, filtered by their first span's name, attributes, errors or a time window, paged by a cursor),
`getTrace(id)` and `findMetrics` (a metric's rows, one per series and 10 seconds, with count, sum, min and max).
`trace.ts` is a script for a data directory, the app running or not:
<!-- quote: trace.ts -->
```ts
if (id !== undefined) {
const trace = await telemetry.store.getTrace(id);
console.log(trace === undefined ? `no trace ${id}` : renderTrace(trace.spans));
} else {
// Complete traces, newest first; a trace's first span (its root, by start time) names it.
const { traces } = await telemetry.store.findTraces({ name: values.name, limit: 10 });
for (const { traceId, spans } of traces) {
const start = Math.min(...spans.map((span) => span.startedMs));
const length = Math.max(...spans.map((span) => span.endedMs)) - start;
const failed = spans.some((span) => span.status.status === "error") ? " error" : "";
console.log(`${traceId} ${spans[0]!.name} ${spans.length} spans ${length.toFixed(2)} ms${failed}`);
}
// Metrics are stored per series and 10 s: a series is one set of attributes, here the direction.
for (const direction of ["sent", "received"]) {
const rows = await telemetry.store.findMetrics({ name: "chord.link.bytes", attributes: { direction } });
const bytes = rows.reduce((sum, row) => sum + row.sum, 0);
const messages = rows.reduce((sum, row) => sum + row.count, 0);
console.log(`chord.link.bytes ${direction}: ${bytes} bytes in ${messages} messages`);
}
}
```
Step 11 runs it twice from a process of its own: the newest `page.add` traces, then the refused one by its id (ids,
lengths and counts masked):
<!-- output: main -->
```text
trace.ts <data directory> --name page.add:
<id> page.add 6 spans <ms> error
<id> page.add 6 spans <ms>
chord.link.bytes sent: <n> bytes in <n> messages
chord.link.bytes received: <n> bytes in <n> messages
trace.ts <data directory> <the refused one's id>:
page.add error ChordError: a todo needs text
chord.call type="todo.todos" member="add" object="todo.todos(groceries)" address.kind="list" outcome="error" error call_failed: a todo needs text
chord.relay type="todo.todos" member="#0" upstream.kind="list" outcome="error" error relay_failed: a todo needs text
chord.method type="todo.todos" member="add" object="todo.todos(groceries)" principal.kind="owner" outcome="error" error method_failed: a todo needs text
chord.method type="todo.store" member="add" object="todo.store" principal.kind="worker" outcome="error" error method_failed: a todo needs text
todos.validate chars=0 error trace_failed: a todo needs text
```
A trace's id is its root span's. The refused add failed on every span of its path, each with a fixed category
(`call_failed`, `relay_failed`, `method_failed`, `trace_failed`) and the message cut at 500 characters; a cancelled call
is not an error. It has no commit: nothing was written. `chord.link.bytes` counts the encoded messages of every host's
links, each message once where it was sent and once where it was received. A metric carries no ids, only attributes
with few values (`direction`), so its series stay few. A running recorder writes a metric's current 10 seconds only
once they are over, or when it closes: `trace.ts` sees the page's bytes, whose recorder closed, and the server's and
workers' of the intervals already over. The script is a recorder too, which records nothing; Chord ships only the
store, so reads go through a recorder of the file.
## A trace viewer
Two halves make the viewer: routes on the server that read the store, and a page that shows what they answer. Server
code of an app is either the start script or a plugin, and routes come from plugins (chapter III), so the routes are a
small plugin, `plugins/traces/`. Its server facet serves one host-lifetime object, which opens the store's file as a
native and contributes the routes while it is live:
<!-- quote: plugins/traces/server.ts -->
```ts
host.serve(Traces, (_params, self) => {
const info = self.get(AppInfo, {});
// The server records to the same file with its own recorder; this one only reads.
const telemetry = self.open(
async () => openTelemetry(telemetryFile(info.dataDir.value)),
(opened) => opened.close(),
);
self.contribute(Http, {
routes: {
"GET /api/traces": async (_request, ctx) => {
ownerOnly(ctx);
const { traces } = await telemetry.value.store.findTraces({ limit: RECENT });
return { kind: "json", value: traces.map(({ traceId, spans }) => summaryOf(traceId, spans)) };
},
"GET /api/traces/:id": async (request, ctx) => {
ownerOnly(ctx);
const trace = await telemetry.value.store.getTrace(request.params.id!);
if (trace === undefined) return { kind: "error", status: 404 };
return { kind: "json", value: { id: trace.traceId, spans: rowsOf(trace.spans) } };
},
},
});
return {};
});
```
The kit gives a plugin the server's process values as objects: `AppInfo` names the data directory, and the API's server
module re-exports it with the app's `openTelemetry`. `self.open` declares a native, opened once `AppInfo` is live and
closed when the object ends; a reload of the plugin restarts the object and opens the file anew. An object may
contribute when it lives as long as its host (`local`, empty params; decision R21). Spans name who called what, so
only the owner reads them:
<!-- quote: plugins/traces/server.ts -->
```ts
function ownerOnly(ctx: Ctx): void {
if (ctx.principal?.kind !== "owner") throw forbidden("the traces are the owner's");
}
```
A route's principal is `undefined` only on a public route, which these are not; a signed-in member gets 403. `rowsOf`
orders a trace's spans as a tree, parents before their children, and turns attributes, events and links into text. The
spans name no host; the SQLite store's span ids begin with the id of the recorder, one per process, and the viewer
calls each recorder a host:
<!-- quote: plugins/traces/server.ts -->
```ts
function processOf(span: StoredSpan): string {
return span.id.slice(0, span.id.lastIndexOf(":"));
}
```
That is a detail of this store, not of the read interface (see the epilogue). The page is the app's own page at a
second path: `start.ts` serves it at `/` and `/traces` (chapter I), and the web entry picks the viewer by the path:
<!-- quote: src/web/main.ts -->
```ts
// The page's one other path (`start.ts` serves the page at `/` and `/traces`).
if (location.pathname === "/traces") return h("main", { className: "wide" }, h("h1", null, "Todo"), h(TraceViewer));
```
`TraceViewer` (`src/web/traces.ts`) fetches the list and the trace chosen in the URL's fragment, and draws each span
with its host, its length as a bar on the trace's time line, its attributes and events, a failed span in the theme's
error colour, and each link as the ids it refers to, never fetched. The bars are positioned with inline styles, which
the page's CSP allows only because React sets them through the DOM:
<!-- quote: src/web/traces.ts -->
```ts
const bar = {
left: `${(row.offsetMs / total) * 100}%`,
width: `${Math.max((row.durationMs / total) * 100, 0.5)}%`,
};
```
`screenshots.ts` adds honey from a recording page and refuses a blank todo, as step 11 does, then opens the viewer on
honey's trace:
<!-- quote: screenshots.ts -->
```ts
await browser.open(`${origin}/traces#${encodeURIComponent(trace.traceId)}`);
await browser.wait("document.querySelectorAll('.traces-tree li').length === 6");
await shoot("traces.png");
```
<!-- output: screenshots -->
```text
wrote docs/tutorial/screens/traces.png
the viewer shows page.add: 6 spans on 3 hosts
```
:::figure wide | The owner's trace viewer at `/traces`, in the Harbor theme: the newest traces (the refused add marked in red, the rest the app's own work: relays, activations, HTTP requests), and honey's trace as a tree across the page, the server and the worker. Starts are stored in whole milliseconds, so the bars of spans this short shift by up to one.
![The trace viewer](screens/traces.png)
:::
The list is mostly the app's own work: each HTTP request, relayed hold and activation is a trace of its own, as the
trap above says. A real viewer would filter by name (`findTraces({ name })`) or by error (`errors: true`), both of
which the read interface has; this one is about 270 lines of TypeScript in two files and 20 of CSS, and no Chord API was added for it.
# Testing One Plugin {#testing lede="testPlugin builds a plugin as production does and loads it alone on a host of its own."}
The steps above test the whole app. Most rules are checked faster on one plugin. `testPlugin` builds the plugin as
production builds it (the same esbuild call, the API's imports checked), evaluates it with the app's real API module and
loads it on a new host with an in-memory store, here at a list's address so it may serve `Todos`:
<!-- quote: plugins/lists/worker.test.ts -->
```ts
let t: TestedPlugin<v1.TodoPrincipal>;
beforeEach(async () => {
t = await testPlugin<v1.TodoPrincipal>(
fileURLToPath(new URL("./", import.meta.url)),
{
kind: "worker",
api: { name: "@todo/app", versions: ["v1"] },
modules: { "@todo/app/v1": v1 },
address: v1.List({ id: "groceries" }),
principal: { kind: "worker", list: "list|groceries" },
},
context,
);
});
afterEach(() => t.close(context));
```
Each test runs code as a principal of its choice with `t.root`:
<!-- quote: plugins/lists/worker.test.ts -->
```ts
it("lets the owner clear the done todos", async () => {
await t.root(
async (ctx) => {
const todos = await ctx.get(Todos, { list: "groceries" });
if (todos.variant !== "owner") throw new Error(`the owner got ${todos.variant}`);
await todos.add("milk", ctx);
await todos.add("eggs", ctx);
await todos.tick({ id: 1, done: true }, ctx);
await todos.clear(ctx);
expect(todos.items.value.map(({ text }) => text)).toEqual(["eggs"]);
expect(todos.notice.value).toEqual({ namespace: "todo.lists", key: "cleared", params: ["owner"] });
},
context,
{ principal: OWNER },
);
expect(t.unserved()).toEqual([]);
```
What the plugin uses and no plugin of the test serves would wait unbound; `t.unserved()` lists those names, so a test
knows which stub facets to add. Registries it contributes to that nobody serves are collected (`t.contributions(R)`).
Run it like any test:
```sh
node ../../node_modules/vitest/dist/cli.js --run test/examples/61-tutorial/plugins/lists/worker.test.ts
```
The test imports the API by its package name and `Todos` from the plugin's own `contract.ts`, as the facets do:
<!-- quote: plugins/lists/worker.test.ts -->
```ts
import * as v1 from "@todo/app/v1";
```
<!-- quote: plugins/lists/worker.test.ts -->
```ts
import { Todos } from "./contract.ts";
```
Both resolve because the API is a workspace package npm linked (chapter I); the plugin's build bundles its own copy of
`contract.ts`, and the two bind, since contracts are data (the same name, version and schema). A third test checks
the refusal of chapter II:
<!-- quote: plugins/lists/worker.test.ts -->
```ts
const refused = await todos.add("", ctx).catch((error: ChordError) => error);
expect(refused).toMatchObject({ code: "invalid", reason: "args" });
```
# Writing a p2 Plugin {#p2 lede="What is different in p2, and one small plugin: a tool for the agent and a chip for the people."}
p2, the coding agent's server with its web page and terminal, runs on Chord v2 but not on the app kit. It is older than
the kit and keeps its own decisions: its own data directory and SQLite files, its own sign-in, devices and sharing,
its own page and terminal bundles. It uses Chord's components directly: `createHost` with p2's principals,
`createWorkers` for one worker per session, `createPlugins` to find, build, watch and roll out its plugins. What a
plugin author sees is the same model: contracts, factories, registries, facets per host kind.
| | The todo app (kit) | p2 |
| --- | --- | --- |
| host kinds | `Server`, `List({ id })` | `Server`, `Session({ id })` (`serverHost`, `sessionHost`) |
| principals | server, worker, owner, user | `owner` (with `access`), `guest` (with `sessions` and `revision`), `worker`, `server`, `host` |
| API | `@todo/app/v1`, `/server`, `/web`, `/tui` | `@p2/app/v1` (browser-safe), `/node` (server and worker), `/server`, `/worker`, `/web`, `/tui` |
| other host packages | React for the web | pi-ai, pi-durable, pi-schema, pi-env, pi-tui, `ws`, `yaml`: plugins import them directly |
| plugins | `plugins/<name>/`, `package.json` named `@todo/<name>` | `apps/p2/plugins/<name>/`, `package.json` named `<name>` |
| a Chord store | the kit's, per host: persisted documents and tables | none: durable data is Durable's (documents with a conversation) or p2's own files |
| page and terminal | kit surfaces, `Panels` and `Screens` | p2's bundles; the page's `Slots` registry, `useObject`, `useReadable` |
| applying changes | `app.applyChanges` | the owner's Reload button, `npm run reload` in `apps/p2`, the `reload_plugins` tool |
## The scratchpad plugin
The example gives the agent a `scratch_note` tool and shows the session's notes to everyone who may see the session.
Its contract places one object per session on the session's worker:
<!-- p2: apps/p2/plugins/scratchpad/contract.ts -->
```ts
export const Scratchpad = remote("p2.scratchpad", {
version: 1,
params: { session: param.str() },
hostAddress: sessionHost,
variants: { reader: { state: { notes: arr(str()) } } },
});
```
The worker facet holds the notes in a worker-wide store (a local object with empty params, as in chapter II). It gives
the agent its tool through p2's `Extensions` registry, a keyed registry of Durable extensions, and serves the view to
whoever `canView` the session:
<!-- p2: apps/p2/plugins/scratchpad/worker.ts -->
```ts
const Notes = defineState("p2.scratchpad.notes", {
schema: obj({ items: arr(str()) }),
version: 1,
initial: () => ({ items: [] }),
});
```
<!-- p2: apps/p2/plugins/scratchpad/worker.ts -->
```ts
export default defineFacet((host) => {
host.serve(ScratchpadStore, (_params, self) => {
const notes = self.state(Notes);
self.contribute(Extensions, {
scratchpad: defineExtension({
name: "scratchpad",
tools: [
defineTool({
name: "scratch_note",
description:
"Keep a short note for this session (a decision, an open question). Everyone viewing the session sees the notes.",
parameters: obj({ text: str({ minLength: 1, maxLength: 500 }) }),
replay: "unsafe",
execute: async ({ text }, _api, context) => {
await notes.change((draft) => {
draft.items.push(text);
}, context);
return {
output: [{ type: "text" as const, text: `Noted (${notes.value.items.length} in all).` }],
};
},
}),
],
}),
});
return { notes: self.derive([notes], ([value]) => value.items) };
});
host.serve(Scratchpad, ({ session }, principal, self) => {
if (!canView(principal, session)) throw forbidden("viewers only");
const store = self.get(ScratchpadStore, {});
return { variant: "reader", members: { notes: self.derive([store.notes], ([notes]) => notes) } };
});
});
```
The notes are an in-memory document: p2 opens no Chord store, so `persist: true` is not available, and the notes last
as long as the session's worker. Data that must outlive it goes into a Durable document (p2's chat-mode keeps its switch
that way). The tool runs inside Durable with a Durable `Context`, which is Chord's own context type, so the store's
handle takes it and the write is awaited. The page gets a chip in the composer's bar through `Slots`:
<!-- p2: apps/p2/plugins/scratchpad/web.tsx -->
```tsx
function NotesChip() {
const { session } = useRoute();
const pad = useObject(Scratchpad, session === undefined ? undefined : { session });
const notes = useReadable(pad?.notes) ?? [];
if (notes.length === 0) return null;
return (
<span className="chip" title={notes.join("\n")}>
Notes · {notes.length}
</span>
);
}
export default defineFacet((_host, self) => self.contribute(Slots, { "composer:scratchpad": NotesChip }));
```
## Testing it in p2
p2's tests load facets on a bare host with `testHost` (`test/p2/helpers.ts`), stub what they need, and reach the host
as any principal over a memory link with `fixture.client(principal)`:
<!-- p2: apps/p2/test/p2/scratchpad.test.ts -->
```ts
it("keeps the agent's notes and shows them to the session's viewers", async () => {
const extensions = await fixture.host.registry(Extensions, BACKGROUND_CONTEXT);
const tool = extensions.value.get("scratchpad")!.tools![0]!;
await tool.execute({ text: "ship on friday" }, undefined as never, BACKGROUND_CONTEXT);
const pad = await fixture.client(OWNER).get(Scratchpad, { session: "s1" }, BACKGROUND_CONTEXT);
assert.deepEqual(await until(pad.notes, (notes) => notes?.length === 1), ["ship on friday"]);
});
```
Run from `apps/p2` with `node --import ./src/source.ts --test test/p2/scratchpad.test.ts`. Load the plugin into a running
p2 with the owner's Reload button or `npm run reload -- scratchpad`; a new plugin directory is picked up by the same
reload. The trap of chapter II applies with p2's names: a server method that must reach a session's worker as the
server runs in a root of the server's own (`serverOf(self)().root`), not through its caller's `ctx.get`.
# Traps and Open Questions {.epilogue lede="The six traps once more, and what in Chord v2 was hard to explain while writing this."}
**The traps.** Load before use: a host binds nothing before `load`; a client with nothing to load calls `load([])`. A
reference acts as whoever made it: `ctx.get` as the caller, `host.get` and a local object's `self.get` as the host.
Await what you start: a write started with a context that ends is cancelled and reported `transaction_cancelled`. A
watch starts at the current value: read `handle.value`, then `start`; the listener gets changes only. The plain `t`
has no language of its own: components and screens render with `useMessages(t)` and `scope.messages(t)`. A trace
starts where a span starts: work under a context without a recording span records nothing, and its provider's span is
a root of its own.
**Found while writing the example, and fixed in Chord since.** Integer table keys were typed `string` (a leftover
of the switch from TypeBox), so `tx.update(items, { id }, ...)` with a number failed `tsc`; the store now uses it. A
reloaded web plugin lost its own React state (now `usePageState`,
chapter IX), a plugin's `contract.ts` could not be imported outside builds (the API is now a workspace package,
chapter I), plugins could not throw Chord's `invalid` (Chord exports `invalid` and `forbidden`, chapter II), and a
live window over a table needed a one-time read in `self.open`, which made the store restart at every reload of its
plugin (now `self.query`, a live query Chord runs, chapters II and IX).
**Found while writing the telemetry chapter, still open.** A page cannot record: `startPage` takes no telemetry
adapter, so chapter XI's page is a client host in Node, and a browser's click starts its trace at the server's relay.
`chord.commit` counts documents and their ops only, so a commit that wrote table rows reports `documents=0 ops=0`.
Spans name no host, and the viewer tells processes apart by the SQLite store's id format (`<recorder>:<n>`), which
the read interface does not promise. The store keeps a span's start in whole milliseconds (`Date.now()`) and its
length in fractions (`performance.now()`), so on a trace of a few milliseconds the viewer's bars shift by up to one. And
reading needs a recorder: `chord/telemetry-sqlite` exports no read-only
store, so a script or a viewer opens a recorder that records nothing.
# The App's Files {.appendix}
| File | Kind | What it is |
| --- | --- | --- |
| `src/principals.ts` | base | the principal type |
| `package.json` | base | the API as the workspace package `@todo/app`, its `exports` |
| `src/api/v1/index.ts` | base, every kind | `@todo/app/v1`: Chord's part (with `invalid`, `forbidden`, `defineMessages`, `Translations`), `defineFacet`, `List` |
| `src/api/v1/server.ts` | base, server | `Http`, the ephemeral `login`, `AppInfo`, `openTelemetry` |
| `src/api/v1/web.ts` | base, web | `Panels`, `useHold`, `useMessages`, `usePageState`, `useWatch` |
| `src/api/v1/tui.ts` | base, tui | `Screens` |
| `start.ts`, `worker.ts` | base | `startServer`, `startWorker`, each with the telemetry store |
| `src/telemetry.ts` | base | the telemetry store's file in the data directory, `openTelemetry` |
| `src/web/main.ts`, `src/web/style.css` | base, web entry | `startPage`, the panels, the viewer at `/traces` |
| `src/web/traces.ts` | base, web entry | the owner's trace viewer |
| `src/tui/main.ts` | base, terminal entry | `startTerminalRuntime`, the screens |
| `plugins/lists/` | plugin | contract (`Item`, `Notice`, `Todos`), messages (English, German), worker (table, store, views), server (routes), web (panel), tui (screen), its test |
| `plugins/french/` | plugin | French for the lists plugin, for pages and terminals |
| `plugins/login.server.ts` | single-file plugin | the kit's ephemeral login |
| `plugins/traces/` | plugin | the trace viewer's routes over the telemetry store (server), the owner's only |
| `later/harbor/` | plugin, added in step 6 | the Harbor theme for pages and terminals |
| `main.ts`, `screenshots.ts`, `tutorial.test.ts` | example | the steps, the screenshots, the checks |
| `trace.ts` | example | a data directory's newest traces, one trace, a metric |
# Reference {.appendix}
| Call | Chapter | What it does |
| --- | --- | --- |
| `startServer<P>({ name, dataDir, root, listen, api, kinds, principal, workers, web, tui, onError }, ctx)` | I | the app's server, with the kit |
| `startWorker({ api, kinds, onError })` | I | a worker's entry |
| `app.ownerLink()`, `app.ownerTokenFile` | I, III | the owner's sign-in, for a browser and for local clients |
| `hostKind(name, params)`, `remote(name, { version, params, hostAddress, variants })`, `local(name, { version, params, members })` | II | host kinds and contracts |
| `host.serve(Type, factory)`; factory `self`: `table`, `query`, `derive`, `tx`, `get`, `contribute` | II | an object's parts |
| `host.root(fn, ctx, { principal })`, `ctx.get(Type, params)` | II | code outside objects, as a principal |
| `invalid(reason, message, details?)`, `forbidden(message, details?)` | II, IX | Chord's errors, thrown by a plugin |
| `self.contribute(Http, { routes })` | III | routes, denied by default |
| `startPage`, `HostProvider`, `useHold`, `useWatch`, `useRegistry` | IV | the page |
| `startTerminalRuntime`, `TuiScope`, `startTerminal` | V | the terminal |
| `usePageState(key, initial)`, `scope.pageState(key, initial)` | IV, IX | state the page keeps across plugin reloads |
| `self.contribute(Themes, theme)` | VI | a theme |
| `ephemeralLoginFacet<P>()`, `host.load([], ctx)`, `host.connect(link, { principal }, ctx)` | VII | a second user's client |
| `defineMessages(namespace, catalogs)`, `t.translations`, `self.contribute(Translations, list)` | IV, V, VIII | a plugin's messages and its catalogs |
| `useMessages(t)`, `scope.messages(t)`, `t.message(key, ...args)`, `t.render(message)`, `MessageSchema` | IV, V, VIII | texts in the viewer's language, messages as data |
| `MessagesOf<typeof t>`, `LocaleChoice` | VIII | a translation plugin, the user's language |
| `app.plugins.state`, `app.applyChanges(ctx)` | VI, IX | marks and rollouts |
| `app.exportData(address, path, ctx)`, `app.importData(path, address, ctx)` | X | moving a worker's data |
| `createSqliteTelemetry(db)`, `startServer({ telemetry })`, `startWorker({ telemetry })`, `createHost({ telemetry })` | XI | recording to the telemetry store |
| `adapter.startSpan({ name }, fn)`, `withTelemetry(span, ctx)`, `ctx.trace(name, attributes, fn)` | XI | spans of your own |
| `store.findTraces({ name, attributes, errors, startMin, startMax, limit, cursor })`, `store.getTrace(id)`, `store.findMetrics({ name, attributes })`, `renderTrace(spans)` | XI | reading traces and metrics |
| `AppInfo`, `self.open(open, close)` | XI | the data directory, a native (the viewer's store) |
| `testPlugin(dir, { kind, api, modules, address, principal, stubs }, ctx)` | XII | one plugin alone |