diff --git a/apps/p2/docs/handoff.md b/apps/p2/docs/handoff.md index 97dfc1f52..9bdb83cb9 100644 --- a/apps/p2/docs/handoff.md +++ b/apps/p2/docs/handoff.md @@ -155,9 +155,9 @@ From `AGENTS.md` and this project's history: 3. **Cleanup leftovers:** two HTTP helpers were not merged because their semantics differ (byte ranges: files serves `bytes=-` whole, workspace answers 416; body readers: sharing answers 400 past the limit, devices 413); decide one behavior each. -4. **MCP:** `plugins/mcp` (WP6 and WP7 of `docs/mcp-durable.md`: owner `mcp.json`, host stdio and HTTP, code mode - tools (`tool_search` without code mode), servers on environments, project `.pi/mcp.json`); OAuth (WP8) and code mode namespaces (WP9) - are open. +4. **MCP:** `plugins/mcp` (WP6-8 of `docs/mcp-durable.md`: owner `mcp.json`, host stdio and HTTP, code mode + tools (`tool_search` without code mode), servers on environments, project `.pi/mcp.json`, OAuth sign-in); code mode + namespaces (WP9) are open, and whether Radius sends its cookie on a sign-in's redirect back is unverified. 5. **npm:** deprecate `@earendil-works/pi-client`, `pi-protocol` and `pi-server` (1.1.0), which no longer exist. 6. `docs/p2-notes.md` TODO: the TypeScript 5.9.3 pin (Termux), an isolating plugin loader, sandboxing third-party facets. diff --git a/apps/p2/docs/mcp-durable.md b/apps/p2/docs/mcp-durable.md index adbf23574..4792570eb 100644 --- a/apps/p2/docs/mcp-durable.md +++ b/apps/p2/docs/mcp-durable.md @@ -84,11 +84,11 @@ p2 gets MCP servers. The design, in short: | 5 | p2 | **gate:** p2's Chord v2 port has landed; WP1-3 present | **done**: p2 imported at `05dc8d239`, deps in "feat(p2): MCP servers on the host, on environments and from projects" | 1-3, the port | | 6 | `apps/p2/plugins/mcp` | manager, view, worker object; owner config; host stdio and HTTP | **done**: "feat(p2): MCP servers on the host, on environments and from projects" | 2, 5 | | 7 | `apps/p2/plugins/mcp` | environment placement; project `mcp.json` | **done**: "feat(p2): MCP servers on the host, on environments and from projects" | 1, 6 | -| 8 | `apps/p2/plugins/mcp` | OAuth: flows, routes, loopback, pasting; web and TUI | open | 3, 6 | +| 8 | `apps/p2/plugins/mcp` | OAuth: flows, routes, loopback, pasting; web and TUI | **done**: see section 4.7 | 3, 6 | | 9 | `apps/p2/plugins/mcp`, `plugins/codemode` | code mode namespaces, progress, first-turn admission | open | 6 | Section 4.5 lists where the built APIs of WP2-4 differ from sections 5-7, section 4.6 where WP6-7 differ from sections -9-10. +9-10, section 4.7 where WP8 differs from section 11. --- @@ -215,6 +215,38 @@ export class RemoteProcess { - **Not watched:** a changed project `mcp.json` is read again at the worker's next `setLocations` (a commit, a move, the view re-activating), not when the file changes. +### 4.7 As built (WP8): differences from section 11 + +- **Which servers sign in:** as in pi, an owner HTTP server without an `Authorization` header (`oauthServerOf` in + `placement.ts`); its instances list `auth: "oauth"`. Only those wait for a sign-in on a 401 (`McpAuthRequiredError` or + `McpOAuthAuthorizationRequiredError`), also when a call's tokens are refused for good; a server refusing its + configured headers fails and retries instead. +- **Credentials:** `credentials.ts`, `/mcp-auth.json` through p2's `LockedFile` (0600), keyed like pi's + (`mcpNamespace(name)|url`). The refresh lock is a promise chain per server in the process, not `proper-lockfile`: one + p2 server owns a data directory (process lock), and nothing else reads this file. +- **Flows:** `sign-in.ts` (`SignIns`, a native of the manager), in memory, one per server (a new one replaces it), + 10 minutes each; listed (`signIns`: id, server, `via: "loopback" | "route"`, `waiting | failed`, error) in the + manager's `Listed` document. Not a Chord flow object: the manager's methods are `beginSignIn`, `completeSignIn` + (callback route), `pasteSignIn`, `cancelSignIn`, `signOut`, `probe`. A done flow disappears; the instances of its + server restart (`Instances.reconnect`). A pre-registered client (`oauth.clientId`) always uses loopback. +- **Links by server, not flow:** `GET /mcp/sign-in/` begins (or replaces) the server's flow, so the link is + stable (the `mcp_servers` section and call errors name it). The owner view has no `signIn` returning a link: its + `signIn({ server })` begins a loopback flow and returns the authorization URL (the TUI opens it); the web opens the + route. +- **Where the browser runs (owner addition):** the device owner (`access: "local"` on loopback) always returns to + loopback. Any other owner (Radius relay, an allowed public host) first gets a probe page (`pages.ts`): it fetches + `http://127.0.0.1:/` from a listener on the p2 host (`probe.ts`), which answers 204 with CORS only for the + page's origin and a nonce only this p2 issued. Answered: the browser runs on the p2 host, so `?via=loopback`; refused + or blocked: `?via=callback`, the route on the origin p2 is reached through. While the answer takes more than 1.5 s + (a browser asking for local network access), the page offers both. A wrong guess only means pasting. +- **Callback route:** `GET /mcp/callback` completes the route flow whose `state` matches, for owners only; a browser + without p2's sign-in there (a cookie not sent) gets a page saying to paste its address. +- **Model:** the `mcp_servers` section marks servers waiting for a sign-in; a call of one says to sign in from the MCP + panel or at `/mcp/sign-in/`. Plain text: p2's Markdown renders relative links as text. +- **UI:** web `plugins/mcp/web.tsx` (sidebar footer "MCP": instances, restart, sign in, sign out, paste, cancel); TUI + `/mcp` (Ctrl+R restart, Ctrl+S sign in through the launcher's `openUrl`, Ctrl+O sign out) and `/mcp paste `. +- **Not verified by hand:** whether Radius sends its cookie on the redirect back to the cortex origin (section 14). + --- ## 5. WP2: pi-mcp transport over a byte pipe diff --git a/apps/p2/docs/p2-notes.md b/apps/p2/docs/p2-notes.md index 35441538d..a681d4c9a 100644 --- a/apps/p2/docs/p2-notes.md +++ b/apps/p2/docs/p2-notes.md @@ -27,7 +27,7 @@ P2_FAKE=1 npm start # http://127.0.0.1:8788, data in ~/.pi/p2 | Variable | Default | Meaning | | --- | --- | --- | -| `P2_DATA_DIR` | `~/.pi/p2` | data directory: `catalog.sqlite`, `sessions/.sqlite`, `settings.json`, `models.json`, `envs.json`, `mcp.json`, `auth.json`, `radius.json`, `forwards.json` | +| `P2_DATA_DIR` | `~/.pi/p2` | data directory: `catalog.sqlite`, `sessions/.sqlite`, `settings.json`, `models.json`, `envs.json`, `mcp.json`, `mcp-auth.json`, `auth.json`, `radius.json`, `forwards.json` | | `PORT` | `8788` | listen port (loopback only), so p2 never collides with pi-mobile on 8787 | | `P2_FAKE` | unset | `1`: the faux provider (`faux/faux-1`, echoes the prompt; `slow …` streams long; `rename-session ` calls the tool; `subagent`, `subagent-bad`, `subagent-answer`, `subagent-configure`, `subagent-send`, `subagent-archive`, `subagent-status` exercise subagents, `subagent <json>` calls the tool with those arguments, as do `team <json>`, `history_search|read|handoff <json>`, `location_envs|dirs|move <json>`, `tool_search <json>` and `mcp__<server>__<tool> <json>`; `codemode` runs a script calling bash and read, `codemode-restart` one whose slow middle call a restart interrupts, `codemode <script>` runs that script; `json <text>` answers exactly `<text>`; a one-shot agent whose task is not `json …`/`slow…` answers a small value matching its schema) | | `P2_IDLE_MS` | `300000` | idle time before a session worker stops | @@ -990,13 +990,13 @@ translated phone drawer and intercept chat taps. reload during a script, SIGKILL and graceful restarts mid-call (finished calls do not rerun; the interrupted unsafe `bash` call returns an error the script catches); the removal mid-script (prompt error, turn ends, the script's thread goes); `codemode-plugin.test.ts` the sandbox across reload/removal. -- **MCP servers** (`plugins/mcp`, plan `docs/mcp-durable.md`, WP6 and WP7): the server's `McpManager` (local, +- **MCP servers** (`plugins/mcp`, plan `docs/mcp-durable.md`, WP6-8): the server's `McpManager` (local, host lifetime) owns every MCP process and connection; workers never spawn one or hold a credential. `<P2_DATA_DIR>/mcp.json` is the owner's `mcpServers` (pi-mcp's `validateMcpServerConfig`, plus p2's `shared` and `runOn`; invalid entries are skipped and listed in `errors`, `${VAR}` in headers and env values from the p2 host's environment; watched: an edited server restarts, a removed one closes; an invalid save keeps the last valid servers, also across a reload). Instances (`placement.ts`): stdio per session and location (key: session, location, server, hash of command, args, env, cwd and timeout; roots the location's cwd), `"shared": true` once on - the host, HTTP shared unless `"shared": false` (one per session); a 401 shows `sign-in` (OAuth is WP8). Stdio + the host, HTTP shared unless `"shared": false` (one per session); a 401 of a server without an `Authorization` header shows `sign-in` (OAuth below). Stdio servers on an environment (`runOn: "location"` on a remote location, or `{ env }`) start through its pi-env daemon (`openProcess` behind pi-mcp's `PipeTransport`, `transports.ts`): one connection per environment and revision of its `envs.json` entry, shared by its instances and closed with the last (`environments.ts`), opened through the envs @@ -1034,6 +1034,15 @@ translated phone drawer and intercept chat taps. short backoff), `mcp-leases.test.ts`, `mcp-environments.test.ts`, `mcp-project.test.ts`, `mcp-windows-command.test.ts`, `mcp-tools.test.ts`, `mcp-worker.test.ts` (fake provider: a project file, `tool_search`, a call, a relocation). +- **MCP sign-in** (`plugins/mcp/sign-in.ts`, `routes.ts`, `probe.ts`, `pages.ts`, `credentials.ts`; as built: + `docs/mcp-durable.md` section 4.7): HTTP servers without an `Authorization` header use OAuth with pi-mcp's auth + provider (stored tokens, refreshed, never a browser) and credentials in `<P2_DATA_DIR>/mcp-auth.json` (0600, keyed like + pi's). Owners sign in at `GET /mcp/sign-in/<server>` (the web MCP sheet opens it in a new tab): the device owner's + browser returns to a loopback callback on the p2 host; for a relayed or public owner a probe page first fetches a + nonce URL from a 127.0.0.1 listener of p2 (CORS for that origin only), and returns to loopback when it answers, else to + `GET /mcp/callback` on the origin p2 is reached through. Pasting the final redirect URL (web sheet, TUI `/mcp paste`) + always completes a waiting sign-in. One flow per server, in memory, 10 minutes; a pending one keeps the manager held. + Tests: `mcp-sign-in.test.ts` (a fake authorization server and MCP server). - **One-shot agents** (`plugins/agent`, port of Pi Mobile a9335ebea and 0e238b29f): code mode scripts' `tools.agent({ prompt, schema, label?, model?, thinkingLevel?, tools? })` (`callers: ["tools"]`, replay-safe). One conversation per call, owned by the tool task (abort cascades) and recorded in the task-scoped `p2.agent-run` doc, so a rerun diff --git a/apps/p2/plugins/mcp/contract.ts b/apps/p2/plugins/mcp/contract.ts index ab83e5f42..e50689aa5 100644 --- a/apps/p2/plugins/mcp/contract.ts +++ b/apps/p2/plugins/mcp/contract.ts @@ -1,6 +1,6 @@ -// MCP servers' contracts: the server's manager (host-internal: every MCP connection and process, by instance and lease) -// and its view per principal (`Mcp`: a session worker sets its locations and calls tools; the owner sees and restarts -// instances). +// MCP servers' contracts: the server's manager (host-internal: every MCP connection and process, by instance and lease, +// and the OAuth sign-ins) and its view per principal (`Mcp`: a session worker sets its locations and calls tools; the +// owner sees and restarts instances and signs in and out of HTTP servers). import type { CallToolResult, ToolAnnotations } from "@earendil-works/pi-mcp"; import { arr, int, nul, obj, oneOf, opt, type Static, str, union, unknown } from "@earendil-works/pi-schema"; @@ -31,6 +31,8 @@ export const InstanceSchema = obj({ /** The location it serves; null: shared by every session, or an HTTP server. */ location: union([LocationSchema, nul()]), runsOn: RunsOnSchema, + /** `oauth`: an HTTP server that signs in with OAuth (no `Authorization` header configured). */ + auth: opt(literal("oauth")), /** `sign-in`: the HTTP server asked for authorization. */ status: oneOf(["starting", "connected", "sign-in", "failed"]), error: opt(str()), @@ -40,6 +42,27 @@ export const InstanceSchema = obj({ /** One instance as the manager lists it: its server, where it runs, its status, error and tools. */ export type Instance = Static<typeof InstanceSchema>; +/** + * A sign-in to an HTTP server under way (`waiting`: for the browser's redirect back or a pasted redirect URL) or failed, + * one per server: returning to a loopback callback on the p2 host, or to p2's `/mcp/callback` route. No URL, state or + * code. + */ +export const SignInSchema = obj({ + id: str(), + server: str(), + via: oneOf(["loopback", "route"]), + status: oneOf(["waiting", "failed"]), + error: opt(str()), +}); +/** A sign-in as the owner sees it. */ +export type SignIn = Static<typeof SignInSchema>; + +/** A begun sign-in: signed in at once (stored tokens refreshed), or the authorization URL to open. */ +const SignInStartSchema = union([obj({ kind: literal("authorized") }), obj({ kind: literal("redirect"), url: str() })]); + +/** Completes sign-in `id` with the final redirect URL the owner copied from the browser's address bar. */ +const PasteSchema = obj({ id: str(), redirectUrl: str({ maxLength: 8192 }) }); + /** The instances one session's lease at a location uses, and why entries of its project were refused. */ const LeaseViewSchema = obj({ session: str(), location: LocationSchema, instances: arr(str()), errors: arr(str()) }); @@ -81,10 +104,11 @@ const Caller = { session: str(), generation: int(), incarnation: str() }; * moves the session to any configured environment. */ export const McpManager = local("p2.mcp.manager", { - version: 1, + version: 2, params: {}, members: { instances: arr(InstanceSchema), + signIns: arr(SignInSchema), /** The leases of every session and location, with the instances each uses. */ leases: arr(LeaseViewSchema), /** Why entries of `mcp.json` were skipped (the leases' projects' errors are in `leases`). */ @@ -102,16 +126,41 @@ export const McpManager = local("p2.mcp.manager", { CallResultSchema, ), restart: method(obj({ instance: str() })), + /** + * Begins a sign-in to HTTP server `server`, replacing its current one: the browser returns to a loopback callback on + * the p2 host, or to p2's callback route at `url`. + */ + beginSignIn: method( + obj({ + server: str(), + redirect: union([obj({ kind: literal("loopback") }), obj({ kind: literal("route"), url: str() })]), + }), + SignInStartSchema, + ), + /** Completes the route sign-in whose `state` the authorization response on the callback route carries. */ + completeSignIn: method( + obj({ state: str(), code: opt(str()), iss: opt(str()), error: opt(str()) }), + obj({ server: str() }), + ), + pasteSignIn: method(PasteSchema), + cancelSignIn: method(obj({ id: str() })), + /** Forgets the server's OAuth credentials; its instances reconnect without them. */ + signOut: method(obj({ server: str() })), + /** + * A loopback probe URL for a sign-in page on `origin`: only a browser on the p2 host reaches it (see + * `LoopbackProbe`). + */ + probe: method(obj({ origin: str() }), obj({ url: str() })), }, }); /** * MCP servers as a principal sees them: a session worker sets its locations and calls tools (its session, generation - * and incarnation from the link), the owner sees every instance with its errors and restarts them. Nobody else gets - * one. + * and incarnation from the link), the owner sees every instance with its errors and sign-ins, restarts them and signs in + * and out of HTTP servers. Nobody else gets one. */ export const Mcp = remote("p2.mcp", { - version: 1, + version: 2, params: {}, hostAddress: serverHost, variants: { @@ -136,11 +185,26 @@ export const Mcp = remote("p2.mcp", { }, owner: { /** `errors`: the owner's `mcp.json`'s, then every lease's (by session). */ - state: { instances: arr(InstanceSchema), errors: arr(str()) }, - methods: { restart: method(obj({ instance: str() })) }, + state: { instances: arr(InstanceSchema), errors: arr(str()), signIns: arr(SignInSchema) }, + methods: { + restart: method(obj({ instance: str() })), + /** + * Begins a sign-in returning to a loopback callback on the p2 host (pasting its final URL works from anywhere). + * Browsers open `/mcp/sign-in/<server>` instead, which also finds out where the browser runs. + */ + signIn: method(obj({ server: str() }), SignInStartSchema), + paste: method(PasteSchema), + cancelSignIn: method(obj({ id: str() })), + signOut: method(obj({ server: str() })), + }, }, }, }); +/** The p2 page that signs the owner in to HTTP server `server` (relative: it opens on the origin the viewer uses). */ +export function signInLink(server: string): string { + return `/mcp/sign-in/${encodeURIComponent(server)}`; +} + /** The worker's Durable extension of MCP tools. */ export const MCP_EXTENSION = "mcp"; diff --git a/apps/p2/plugins/mcp/credentials.ts b/apps/p2/plugins/mcp/credentials.ts new file mode 100644 index 000000000..b7d1de6bf --- /dev/null +++ b/apps/p2/plugins/mcp/credentials.ts @@ -0,0 +1,89 @@ +// The MCP servers' OAuth credentials, `<P2_DATA_DIR>/mcp-auth.json` (private, 0600): per server, by name and URL as in +// pi, the registered client, the tokens and a pending sign-in's PKCE verifier. Only the server's manager reads and +// writes it; tokens never leave the server. + +import { join } from "node:path"; +import { mcpNamespace } from "@earendil-works/pi-mcp"; +import type { McpOAuthServerStore, McpOAuthState } from "@earendil-works/pi-mcp/oauth"; +import { LockedFile, stripBom } from "@p2/app/v1/node"; + +/** The stored states by server key. */ +type States = Record<string, McpOAuthState>; + +/** The credential file and the refresh locks of its servers. */ +export class McpCredentials { + readonly #file: LockedFile; + /** The refresh running per server key: one p2 server per data directory, so a lock in this process suffices. */ + readonly #refreshing = new Map<string, Promise<unknown>>(); + + constructor(dataDir: string) { + this.#file = new LockedFile(join(dataDir, "mcp-auth.json")); + } + + /** The state store of server `name` at `url`, for pi-mcp's sign-in and auth provider. */ + forServer(name: string, url: string): McpOAuthServerStore { + const key = keyOf(name, url); + return { + load: () => this.#file.withLockAsync(async (text) => ({ result: parseStates(text)[key] })), + save: (state) => + this.#file.withLockAsync(async (text) => { + const states = parseStates(text); + states[key] = state; + return { result: undefined, next: serialize(states) }; + }), + withRefreshLock: (fn) => this.#withRefreshLock(key, fn), + }; + } + + /** Whether tokens are stored for server `name` at `url`. */ + signedIn(name: string, url: string): Promise<boolean> { + return this.#file.withLockAsync(async (text) => ({ + result: parseStates(text)[keyOf(name, url)]?.tokens !== undefined, + })); + } + + /** Forgets server `name` at `url`: its client, tokens and pending sign-in. */ + remove(name: string, url: string): Promise<void> { + const key = keyOf(name, url); + return this.#file.withLockAsync(async (text) => { + const states = parseStates(text); + if (!(key in states)) return { result: undefined }; + delete states[key]; + return { result: undefined, next: serialize(states) }; + }); + } + + /** Waits for pending writes. */ + close(): Promise<void> { + return this.#file.close(); + } + + /** Runs `fn` after the server's previous refresh (not reentrant, as pi-mcp's lock). */ + async #withRefreshLock<T>(key: string, fn: () => Promise<T>): Promise<T> { + const previous = this.#refreshing.get(key) ?? Promise.resolve(); + const run = previous.catch(() => {}).then(fn); + this.#refreshing.set(key, run); + try { + return await run; + } finally { + if (this.#refreshing.get(key) === run) this.#refreshing.delete(key); + } + } +} + +/** A server's key, as pi's: servers sharing a URL keep separate accounts. */ +function keyOf(name: string, url: string): string { + return `${mcpNamespace(name)}|${String(new URL(url))}`; +} + +/** The file's states (a trust boundary: an edited file that is not an object counts as none). */ +function parseStates(text: string | undefined): States { + if (!text?.trim()) return {}; + const value: unknown = JSON.parse(stripBom(text)); + // The file is p2's own: its entries are states pi-mcp wrote. + return typeof value === "object" && value !== null && !Array.isArray(value) ? (value as States) : {}; +} + +function serialize(states: States): string { + return `${JSON.stringify(states, null, 2)}\n`; +} diff --git a/apps/p2/plugins/mcp/instances.ts b/apps/p2/plugins/mcp/instances.ts index 7cd9dc316..e90f87f90 100644 --- a/apps/p2/plugins/mcp/instances.ts +++ b/apps/p2/plugins/mcp/instances.ts @@ -14,10 +14,10 @@ import { } from "@earendil-works/pi-mcp"; import { McpOAuthAuthorizationRequiredError } from "@earendil-works/pi-mcp/oauth"; import { PluginError } from "@p2/app/v1"; -import type { Instance, McpTool } from "./contract.ts"; +import { type Instance, type McpTool, signInLink } from "./contract.ts"; import { type ConnectEnv, EnvConnections } from "./environments.ts"; import type { InstanceSpec, Placement } from "./placement.ts"; -import { DaemonTooOldError, type Opened, type Transports, transportsFor } from "./transports.ts"; +import { type AuthProviders, DaemonTooOldError, type Opened, type Transports, transportsFor } from "./transports.ts"; /** How the p2 server names itself to MCP servers (`initialize`'s `clientInfo`). */ export type ClientInfo = { readonly name: string; readonly version: string }; @@ -35,16 +35,27 @@ export class Instances { /** Tools last listed per instance ID (a reload's previous supervisor): listed until the server answers again. */ readonly #seed: ReadonlyMap<string, readonly McpTool[]>; readonly #envs: EnvConnections; + readonly #auth: AuthProviders; readonly #running = new Map<string, Connection>(); readonly #listeners = new Set<() => void>(); #closed = false; - /** `connect` opens the connection to an environment its instances share. */ - constructor(client: ClientInfo, backoff: Backoff, seed: readonly Instance[], connect: ConnectEnv) { + /** + * `connect` opens the connection to an environment its instances share; `auth` gives the auth provider of an HTTP + * server that signs in with OAuth. + */ + constructor( + client: ClientInfo, + backoff: Backoff, + seed: readonly Instance[], + connect: ConnectEnv, + auth: AuthProviders, + ) { this.#client = client; this.#backoff = backoff; this.#seed = new Map(seed.map((instance) => [instance.id, instance.tools])); this.#envs = new EnvConnections(connect); + this.#auth = auth; } /** `listener` is called after every change of `list`; the result stops it. */ @@ -69,7 +80,7 @@ export class Instances { if (!this.#running.has(id)) { // Every leased ID comes from the same placement's instances. const spec = placement.instances.get(id)!; - const transports = transportsFor(spec.connect, this.#envs); + const transports = transportsFor(spec.connect, this.#envs, this.#auth); const connection = new Connection( spec, transports, @@ -106,6 +117,11 @@ export class Instances { connection.restart(); } + /** Connects every HTTP instance of owner server `server` again now (its credentials changed). */ + reconnect(server: string): void { + for (const connection of this.#running.values()) if (connection.isHttp(server)) connection.restart(); + } + /** Closes every instance (stdio servers get MCP's shutdown: stdin closed, then killed) and so every connection. */ async close(): Promise<void> { this.#closed = true; @@ -137,6 +153,9 @@ type State = /** Why the last attempt failed: the message for everybody, the stderr tail for the owner only. */ type Failure = { readonly message: string; readonly stderr?: string }; +/** The status message of an instance waiting for a sign-in. */ +const SIGN_IN_NEEDED: Failure = { message: "This server needs a sign-in" }; + /** One instance: its connection state, and its status, failure and tools as listed. */ class Connection { readonly #spec: InstanceSpec; @@ -172,8 +191,13 @@ class Connection { else void this.#connect(); } + /** Whether this is an instance of HTTP server `server`. */ + isHttp(server: string): boolean { + return this.#spec.server === server && this.#spec.connect.kind === "http"; + } + view(): Instance { - const { id, server, source, location, runsOn } = this.#spec; + const { id, server, source, location, runsOn, connect } = this.#spec; const failure = this.#failure; return { id, @@ -181,6 +205,7 @@ class Connection { source, location, runsOn, + ...(connect.kind === "http" && connect.oauth !== undefined ? { auth: "oauth" as const } : {}), status: this.#status, ...(failure === undefined ? {} @@ -189,13 +214,31 @@ class Connection { }; } - /** The error says why it is not connected, never the server's stderr (the model reads it). */ + /** + * The error says why it is not connected, never the server's stderr (the model reads it); for a sign-in, where the + * user signs in. Tokens refused for good during a call (revoked, say) make it wait for a sign-in. + */ async call(tool: string, args: Record<string, unknown>, signal: AbortSignal): Promise<CallToolResult> { - if (this.#state.kind !== "connected") { + const { server } = this.#spec; + const signInNeeded = () => + new Error( + `MCP server "${server}" needs a sign-in: ask the user to sign in from the MCP panel in p2's sidebar (or at ${signInLink(server)} on p2)`, + ); + if (this.#status === "sign-in") throw signInNeeded(); + const state = this.#state; + if (state.kind !== "connected") { const why = this.#failure === undefined ? "" : `: ${this.#failure.message}`; - throw new Error(`MCP server "${this.#spec.server}" is not connected${why}`); + throw new Error(`MCP server "${server}" is not connected${why}`); + } + try { + return await state.client.callTool(tool, args, { signal, timeoutMs: this.#spec.timeoutMs }); + } catch (error) { + // A restart meanwhile (a sign-in that just finished) decides on its own. + if (!this.#needsSignIn(error) || this.#state !== state) throw error; + void this.#stop(); + this.#show("sign-in", SIGN_IN_NEEDED); + throw signInNeeded(); } - return await this.#state.client.callTool(tool, args, { signal, timeoutMs: this.#spec.timeoutMs }); } restart(): void { @@ -271,11 +314,14 @@ class Connection { } } - /** A connect failed: wait for a sign-in, stay failed (a daemon too old), or try again after the backoff. */ + /** + * A connect failed: wait for a sign-in (an OAuth server without usable tokens), stay failed (a daemon too old), or + * try again after the backoff (a server refusing its configured headers too). + */ #failed(error: unknown, transport: McpTransport | undefined): void { - if (error instanceof McpAuthRequiredError || error instanceof McpOAuthAuthorizationRequiredError) { + if (this.#needsSignIn(error)) { this.#state = { kind: "idle" }; - this.#show("sign-in", { message: "This server needs a sign-in" }); + this.#show("sign-in", SIGN_IN_NEEDED); return; } if (error instanceof DaemonTooOldError) { @@ -287,6 +333,16 @@ class Connection { this.#wait(); } + /** Whether `error` means the user has to sign in: an OAuth server refusing the request with the tokens it has. */ + #needsSignIn(error: unknown): boolean { + const { connect } = this.#spec; + return ( + connect.kind === "http" && + connect.oauth !== undefined && + (error instanceof McpAuthRequiredError || error instanceof McpOAuthAuthorizationRequiredError) + ); + } + /** The connection dropped (a stdio server exited, say): reconnect after the backoff, tools still listed. */ #lost(transport: McpTransport): void { if (this.#state.kind === "connected" && Date.now() - this.#state.since >= this.#backoff.stableMs) diff --git a/apps/p2/plugins/mcp/pages.ts b/apps/p2/plugins/mcp/pages.ts new file mode 100644 index 000000000..156d595c9 --- /dev/null +++ b/apps/p2/plugins/mcp/pages.ts @@ -0,0 +1,72 @@ +// The browser pages of MCP sign-ins: a result (signed in, or why not) and the page that finds out whether the browser +// runs on the p2 host before choosing where the sign-in returns. Plain HTML with their own strict CSP. + +import { randomBytes } from "node:crypto"; +import type { ServerResponse } from "node:http"; + +/** A page saying `title`, then `message`. */ +export function resultPage(title: string, message: string): string { + return document(title, `<h1>${escapeHtml(title)}</h1><p>${escapeHtml(message)}</p>`); +} + +/** Sends `html` with `status` and a CSP allowing nothing but inline styles (and `extra` directives). */ +export function sendPage(response: ServerResponse, status: number, html: string, extra: readonly string[] = []): void { + response.writeHead(status, { + "Content-Type": "text/html; charset=utf-8", + "Content-Security-Policy": [ + "default-src 'none'", + "style-src 'unsafe-inline'", + "base-uri 'none'", + "form-action 'none'", + "frame-ancestors 'none'", + ...extra, + ].join("; "), + }); + response.end(html); +} + +/** + * Sends the page that asks the p2 host's loopback probe at `probeUrl` whether this browser runs there, then continues + * the sign-in to `server` with `?via=loopback` (it does) or `?via=callback` (it does not, or the browser blocks the + * request). While the answer takes long (a browser asking the user for local network access), the page offers both. + */ +export function sendProbePage(response: ServerResponse, server: string, probeUrl: string): void { + const nonce = randomBytes(16).toString("base64"); + const body = `<h1>Sign in to ${escapeHtml(server)}</h1> +<p id="checking">Checking whether this browser runs on the p2 computer…</p> +<div id="choose" hidden> +<p>Where does this browser run?</p> +<p><a href="?via=loopback">On the p2 computer</a></p> +<p><a href="?via=callback">On another device</a></p> +</div> +<noscript><p><a href="?via=loopback">This browser runs on the p2 computer</a></p><p><a href="?via=callback">This browser runs on another device</a></p></noscript> +<script nonce="${nonce}"> +let chosen = false; +const go = (via) => { if (!chosen) { chosen = true; location.replace("?via=" + via); } }; +for (const link of document.querySelectorAll("#choose a")) link.addEventListener("click", () => { chosen = true; }); +const slow = setTimeout(() => { document.getElementById("checking").hidden = true; document.getElementById("choose").hidden = false; }, 1500); +fetch(${JSON.stringify(probeUrl)}, { cache: "no-store" }).then( + (answer) => { clearTimeout(slow); go(answer.ok ? "loopback" : "callback"); }, + () => { clearTimeout(slow); go("callback"); }, +); +</script>`; + sendPage(response, 200, document(`Sign in to ${server}`, body), [ + `script-src 'nonce-${nonce}'`, + `connect-src ${new URL(probeUrl).origin}`, + ]); +} + +function document(title: string, body: string): string { + return `<!doctype html> +<html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"> +<title>${escapeHtml(title)} + +${body}`; +} + +function escapeHtml(text: string): string { + return text.replace( + /[&<>"']/g, + (char) => ({ "&": "&", "<": "<", ">": ">", '"': """, "'": "'" })[char] ?? char, + ); +} diff --git a/apps/p2/plugins/mcp/placement.ts b/apps/p2/plugins/mcp/placement.ts index 73bfafb0f..f56c2d8a1 100644 --- a/apps/p2/plugins/mcp/placement.ts +++ b/apps/p2/plugins/mcp/placement.ts @@ -14,6 +14,7 @@ import type { OwnerConfig, OwnerServer } from "./config.ts"; import type { RunsOn } from "./contract.ts"; import type { Lease } from "./leases.ts"; import { type ProjectConfig, type ProjectServer, parseProjectConfig } from "./project.ts"; +import type { OAuthServer } from "./sign-in.ts"; /** A program on an environment, started through its pi-env daemon; paths resolve once the daemon tells its system. */ export type RemoteConnect = { @@ -46,7 +47,13 @@ export type Connect = readonly hostVariables: boolean; } | RemoteConnect - | { readonly kind: "http"; readonly url: string; readonly headers: Readonly> } + | { + readonly kind: "http"; + readonly url: string; + readonly headers: Readonly>; + /** Signs in with OAuth (no `Authorization` header configured), as in pi. */ + readonly oauth: OAuthServer | undefined; + } | { readonly kind: "unplaced"; readonly reason: string }; /** One instance to run: its identity and where and how it runs. Equal configuration and key give an equal ID. */ @@ -139,16 +146,17 @@ function ownerSpec( const timeoutMs = server.config.timeout === undefined ? undefined : server.config.timeout * 1000; const owned = server.shared ? undefined : lease; if (server.kind === "http") { - const { url, headers = {} } = server.config; + const { url, headers = {}, oauth } = server.config; // Unshared HTTP servers are one per session, wherever its conversations work. - const key = ["http", server.name, owned?.session ?? null, hashOf({ url, headers, timeoutMs: timeoutMs ?? null })]; + const hash = hashOf({ url, headers, ...(oauth === undefined ? {} : { oauth }), timeoutMs: timeoutMs ?? null }); + const key = ["http", server.name, owned?.session ?? null, hash]; return { id: idOf(key), server: server.name, source: "owner", location: null, runsOn: "host", - connect: { kind: "http", url, headers }, + connect: { kind: "http", url, headers, oauth: oauthServerOf(server) }, timeoutMs, }; } @@ -158,6 +166,14 @@ function ownerSpec( return stdioSpec(placed, runsOn, server.config, timeoutMs, envs); } +/** How owner server `server` signs in with OAuth: as in pi, an HTTP server without an `Authorization` header. */ +export function oauthServerOf(server: OwnerServer): OAuthServer | undefined { + if (server.kind !== "http") return undefined; + const { url, headers = {}, oauth } = server.config; + if (Object.keys(headers).some((header) => header.toLowerCase() === "authorization")) return undefined; + return { name: server.name, url, config: oauth }; +} + /** The instance of project server `server` at `lease`'s location: always there, for that session alone. */ function projectSpec(server: ProjectServer, lease: Lease, envs: Readonly>): InstanceSpec { const timeoutMs = server.config.timeout === undefined ? undefined : server.config.timeout * 1000; @@ -238,7 +254,10 @@ function expandHome(value: string): string { return value.startsWith("~/") ? join(homedir(), value.slice(2)) : value; } -/** The configuration hash: what makes another process or connection (command, args, env, cwd; URL and headers; the timeout). */ +/** + * The configuration hash: what makes another process or connection (command, args, env, cwd; URL, headers and OAuth + * client settings; the timeout). + */ function hashOf(value: unknown): string { return createHash("sha256").update(JSON.stringify(value)).digest("hex").slice(0, 16); } diff --git a/apps/p2/plugins/mcp/probe.ts b/apps/p2/plugins/mcp/probe.ts new file mode 100644 index 000000000..206c2a4a8 --- /dev/null +++ b/apps/p2/plugins/mcp/probe.ts @@ -0,0 +1,79 @@ +// The loopback probe, a native of the manager: a listener on the p2 host's 127.0.0.1 that tells a sign-in page whether +// its browser runs on the p2 host. The page (on p2's Radius or public origin) fetches `http://127.0.0.1:/`: +// only a browser on this machine reaches this listener, and only this p2 knows the nonce, so another program or p2 on +// the browser's machine cannot answer for it. A wrong answer costs nothing more than a sign-in to paste. + +import { randomBytes } from "node:crypto"; +import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http"; + +/** How long a nonce answers. */ +const NONCE_MS = 5 * 60_000; + +/** The probe listener, started on first use. */ +export class LoopbackProbe { + /** The origin of the page each nonce was handed to, and when it expires. */ + readonly #nonces = new Map(); + #server: Promise | undefined; + #closed = false; + + /** A probe URL for a page on `origin`: it answers that origin's requests, with CORS, for a few minutes. */ + async url(origin: string): Promise { + if (this.#closed) throw new Error("The loopback probe is closed"); + const now = Date.now(); + for (const [nonce, entry] of this.#nonces) if (entry.expires <= now) this.#nonces.delete(nonce); + const nonce = randomBytes(24).toString("hex"); + this.#nonces.set(nonce, { origin, expires: now + NONCE_MS }); + this.#server ??= listen((request, response) => this.#handle(request, response)); + const server = await this.#server; + const address = server.address(); + // A TCP listener's address is an object. + const port = typeof address === "object" && address !== null ? address.port : 0; + return `http://127.0.0.1:${port}/${nonce}`; + } + + async close(): Promise { + this.#closed = true; + const server = await this.#server?.catch(() => undefined); + if (server !== undefined) await new Promise((resolve) => server.close(() => resolve())); + } + + /** + * 204 with CORS for the nonce's origin, else 404 without: other pages learn nothing. Preflights (Chrome's private + * network access asked for one) get the same answer, allowing the private network request. + */ + #handle(request: IncomingMessage, response: ServerResponse): void { + request.resume(); + const nonce = new URL(request.url ?? "/", "http://127.0.0.1").pathname.slice(1); + const entry = this.#nonces.get(nonce); + const origin = request.headers.origin; + if (entry === undefined || entry.expires <= Date.now() || origin !== entry.origin) { + response.writeHead(404, { "Cache-Control": "no-store" }).end(); + return; + } + const cors = { "Access-Control-Allow-Origin": origin, Vary: "Origin", "Cache-Control": "no-store" }; + if (request.method === "OPTIONS") + response + .writeHead(204, { + ...cors, + "Access-Control-Allow-Methods": "GET", + "Access-Control-Allow-Private-Network": "true", + }) + .end(); + else if (request.method === "GET") response.writeHead(204, cors).end(); + else response.writeHead(405, cors).end(); + } +} + +/** A listener on a free port of 127.0.0.1. */ +async function listen(handler: (request: IncomingMessage, response: ServerResponse) => void): Promise { + const server = createServer(handler); + await new Promise((resolve, reject) => { + server.once("error", reject); + server.listen(0, "127.0.0.1", () => { + server.off("error", reject); + resolve(); + }); + }); + server.unref(); + return server; +} diff --git a/apps/p2/plugins/mcp/routes.ts b/apps/p2/plugins/mcp/routes.ts new file mode 100644 index 000000000..463c47dc6 --- /dev/null +++ b/apps/p2/plugins/mcp/routes.ts @@ -0,0 +1,137 @@ +// The browser routes of MCP sign-ins (owners only). `GET /mcp/sign-in/` begins a sign-in and redirects to the +// authorization server: the browser returns to a loopback callback on the p2 host when it runs there, else to `GET +// /mcp/callback` on the origin it reaches p2 through (Radius, a public host). The device owner's browser runs on the +// p2 host; for any other owner a probe page finds out first (`LoopbackProbe`). + +import type { IncomingMessage, ServerResponse } from "node:http"; +import { type Ctx, isOwner, type StoreSelf, until } from "@p2/app/v1"; +import { type RequestAccess, Routes, type ServerParts, serverOf } from "@p2/app/v1/server"; +import { McpManager } from "./contract.ts"; +import { resultPage, sendPage, sendProbePage } from "./pages.ts"; + +/** Contributes the sign-in routes. */ +export function signInRoutes(self: StoreSelf): void { + const server = serverOf(self); + self.contribute(Routes, { + "GET /mcp/sign-in/*": (request, response, access, ctx) => signIn(request, response, access, server, ctx), + "GET /mcp/callback": (request, response, access, ctx) => callback(request, response, access, ctx), + }); +} + +/** The path of the callback route on p2's own origin. */ +const CALLBACK_PATH = "/mcp/callback"; + +/** + * Begins the sign-in to the server the path names. `?via=loopback|callback` is the probe page's answer; without it, the + * device owner signs in through loopback and any other owner gets the probe page. + */ +async function signIn( + request: IncomingMessage, + response: ServerResponse, + { principal, origin }: RequestAccess, + server: () => ServerParts, + ctx: Ctx, +): Promise { + if (!isOwner(principal)) return refuse(response, principal === undefined); + const url = new URL(request.url ?? "/", "http://localhost"); + let name: string; + try { + name = decodeURIComponent(url.pathname.slice("/mcp/sign-in/".length)); + } catch { + return sendPage(response, 400, resultPage("Unknown MCP server", "The sign-in link is malformed.")); + } + const via = url.searchParams.get("via"); + const local = principal.access === "local" && !origin.remote; + const manager = await ctx.get(McpManager, {}); + if (!local && via !== "loopback" && via !== "callback") { + const { url: probeUrl } = await manager.probe({ origin: origin.origin }, ctx); + return sendProbePage(response, name, probeUrl); + } + let start: Awaited>; + try { + start = await manager.beginSignIn( + { + server: name, + redirect: + local || via === "loopback" + ? { kind: "loopback" } + : { kind: "route", url: origin.origin + CALLBACK_PATH }, + }, + ctx, + ); + } catch (error) { + return sendPage(response, 502, resultPage(`Signing in to ${name} failed`, messageOf(error))); + } + if (start.kind === "authorized") + return sendPage(response, 200, resultPage(`Signed in to ${name}`, "You may close this page.")); + keepManager(name, server); + response.writeHead(302, { Location: start.url }); + response.end(); +} + +/** + * Keeps the manager, which holds the flow, while the sign-in to `name` waits for its browser step (until it finishes, + * fails or expires), also when no view holds it meanwhile. + */ +function keepManager(name: string, server: () => ServerParts): void { + void server() + .root(async (ctx) => { + const manager = await ctx.get(McpManager, {}); + await until( + manager.signIns, + (signIns) => !signIns.some((entry) => entry.server === name && entry.status === "waiting"), + ctx, + ); + }) + .catch(() => {}); +} + +/** Completes a route sign-in with the authorization response the browser brings back. */ +async function callback( + request: IncomingMessage, + response: ServerResponse, + { principal }: RequestAccess, + ctx: Ctx, +): Promise { + const params = new URL(request.url ?? "/", "http://localhost").searchParams; + if (!isOwner(principal)) + return sendPage( + response, + principal === undefined ? 401 : 403, + resultPage( + "Sign-in not finished", + "This browser is not signed in to p2 here. Copy this page's full address from the address bar and paste it in p2's MCP panel.", + ), + ); + const state = params.get("state"); + if (state === null) + return sendPage(response, 400, resultPage("Sign-in not finished", "The authorization response has no state.")); + const [code, iss, error] = [params.get("code"), params.get("iss"), params.get("error")]; + try { + const { server } = await (await ctx.get(McpManager, {})).completeSignIn( + { + state, + ...(code === null ? {} : { code }), + ...(iss === null ? {} : { iss }), + ...(error === null ? {} : { error: params.get("error_description") ?? error }), + }, + ctx, + ); + sendPage(response, 200, resultPage(`Signed in to ${server}`, "You may close this page.")); + } catch (failure) { + sendPage(response, 400, resultPage("Signing in failed", messageOf(failure))); + } +} + +/** Guests and people without a sign-in get no sign-in. */ +function refuse(response: ServerResponse, anonymous: boolean): void { + sendPage( + response, + anonymous ? 401 : 403, + resultPage("Not signed in", "Only p2's owner signs in to MCP servers. Open p2 and sign in first."), + ); +} + +function messageOf(error: unknown): string { + return error instanceof Error ? error.message : String(error); +} diff --git a/apps/p2/plugins/mcp/server.ts b/apps/p2/plugins/mcp/server.ts index 1b74d23d6..deb156d36 100644 --- a/apps/p2/plugins/mcp/server.ts +++ b/apps/p2/plugins/mcp/server.ts @@ -1,13 +1,15 @@ // The MCP plugin's server facet: the manager (the owner's configuration, the leases of session workers with their -// projects' configurations, the environments and where sessions work, the supervisor of instances) and its view per -// principal: a session worker's (its locations and calls, fenced to the session's current worker; its leases released -// when the view ends) and the owner's (every instance with its errors, restart). +// projects' configurations, the environments and where sessions work, the supervisor of instances, the OAuth sign-ins) +// and its view per principal: a session worker's (its locations and calls, fenced to the session's current worker; its +// leases released when the view ends) and the owner's (every instance with its errors and sign-ins, restart, signing in +// and out); and the sign-in routes browsers open. import { readFile } from "node:fs/promises"; import { join } from "node:path"; import { arr, int, nul, obj, str, union, unknown } from "@earendil-works/pi-schema"; import { type Ctx, + contribute, defineFacet, defineTypedState, deriveTyped, @@ -23,6 +25,7 @@ import { locationText, type Native, nativeOf, + PluginError, type Ref, type Self, sameLocation, @@ -39,11 +42,15 @@ import { EnvManagers, EnvService } from "../envs/contract.ts"; import { Catalog, type Session } from "../sessions/contract.ts"; import { type Child, ChildStore } from "../subagents/contract.ts"; import { type OwnerConfig, OwnerConfigFile, type OwnerServer } from "./config.ts"; -import { InstanceSchema, Mcp, McpManager, type ProjectFile } from "./contract.ts"; +import { InstanceSchema, Mcp, McpManager, type ProjectFile, SignInSchema } from "./contract.ts"; +import { McpCredentials } from "./credentials.ts"; import type { ConnectEnv } from "./environments.ts"; import { type ClientInfo, Instances, RECONNECT } from "./instances.ts"; import { heldLease, type Lease, type ProjectSource, releaseGeneration, setLocations } from "./leases.ts"; -import { type Placement, place, type WorkEnvironments } from "./placement.ts"; +import { oauthServerOf, type Placement, place, type WorkEnvironments } from "./placement.ts"; +import { LoopbackProbe } from "./probe.ts"; +import { signInRoutes } from "./routes.ts"; +import { type OAuthServer, SIGN_IN_TIMEOUT_MS, type SignInRedirect, SignIns } from "./sign-in.ts"; /** A tool call as the worker's view takes it; the manager's adds the caller. */ type CallArgs = { location: Location; server: string; tool: string; args: Record }; @@ -77,6 +84,7 @@ export function mcpServerFacet(viewLingerMs: number, connector: EnvConnector): F return defineFacet((host) => { host.serve(McpManager, managerFactory(connector)); host.serve(Mcp, view, { linger: viewLingerMs }); + contribute(host, "p2.mcp-routes", signInRoutes); }); } @@ -111,11 +119,14 @@ const Leased = defineTypedState("p2.mcp.leases", { initial: (): { leases: Lease[] } => ({ leases: [] }), }); -/** The supervisor's instances as it last listed them; a reload's new supervisor starts with their tools. */ +/** + * The supervisor's instances as it last listed them (a reload's new supervisor starts with their tools), and the + * sign-ins under way. + */ const Listed = defineTypedState("p2.mcp.instances", { - schema: obj({ instances: arr(InstanceSchema) }), - version: 1, - initial: () => ({ instances: [] }), + schema: obj({ instances: arr(InstanceSchema), signIns: arr(SignInSchema) }), + version: 2, + initial: () => ({ instances: [], signIns: [] }), }); /** The manager: its documents, the file and the supervisor as natives, and the routines that keep them in step. */ @@ -143,11 +154,27 @@ function managerFactory(connector: EnvConnector): Factory { // After a reload the leases are still here: the routine below starts what they use, with the tools listed // before. Not here: natives open before derives compute. const info = await clientInfo(server.info.value.app); - return new Instances(info, RECONNECT, listed.value.instances, connect); + return new Instances(info, RECONNECT, listed.value.instances, connect, (oauth) => + signIns.value.authProvider(oauth), + ); }, (opened) => opened.close(), ); - self.onLive((ctx) => followNatives(file, instances, configured, listed, ctx)); + const signIns = self.open( + async () => + new SignIns( + new McpCredentials(server.info.value.dataDir), + "p2", + (name) => instances.value.reconnect(name), + SIGN_IN_TIMEOUT_MS, + ), + (opened) => opened.close(), + ); + const probe = self.open( + async () => new LoopbackProbe(), + (opened) => opened.close(), + ); + self.onLive((ctx) => followNatives(file, instances, signIns, configured, listed, ctx)); // The current configuration, leases, environments and places of work, then every change of them, start and close // instances; methods that change leases reconcile at once too, so a call right after `setLocations` finds its // instance. @@ -174,11 +201,18 @@ function managerFactory(connector: EnvConnector): Factory { ); return { instances: deriveTyped(self, [listed], ([value]) => value.instances), + signIns: deriveTyped(self, [listed], ([value]) => value.signIns), leases, errors: self.derive([configured], ([{ errors, failure }]) => failure === null ? [...errors] : [...errors, `${failure}; keeping the last valid servers`], ), ...managerMethods(leased, () => instances.value, placement), + ...signInMethods( + configured, + listed, + () => signIns.value, + () => probe.value, + ), }; }; } @@ -220,10 +254,11 @@ function placements(): ( }; } -/** Copies what the file reads and the supervisor lists into the manager's documents until `ctx` ends. */ +/** Copies what the file reads, the supervisor lists and the sign-ins are into the manager's documents until `ctx` ends. */ async function followNatives( file: Native, instances: Native, + signIns: Native, configured: HandleOf, listed: HandleOf, ctx: Ctx, @@ -234,6 +269,7 @@ async function followNatives( void listed .change((draft) => { draft.instances = instances.value.list(); + draft.signIns = signIns.value.list(); }, ctx) .catch(() => {}); }; @@ -254,6 +290,7 @@ async function followNatives( .catch(() => {}); }), instances.value.subscribe(show), + signIns.value.subscribe(show), ]; show(); try { @@ -303,6 +340,60 @@ function managerMethods(leased: HandleOf, instances: () => Instan }; } +/** The manager's sign-in methods: flows of the owner's OAuth servers, the callback route's completion, the probe. */ +function signInMethods( + configured: HandleOf, + listed: HandleOf, + signIns: () => SignIns, + probe: () => LoopbackProbe, +) { + const oauthServer = (name: string): OAuthServer => { + const server = configured.value.servers.find((candidate) => candidate.name === name); + const oauth = server === undefined ? undefined : oauthServerOf(server); + if (oauth === undefined) throw new PluginError("not_found", `No MCP server "${name}" signs in with OAuth`); + return oauth; + }; + return { + beginSignIn: async ({ server, redirect }: { server: string; redirect: SignInRedirect }, ctx: Ctx) => { + const start = await asPluginError(signIns().begin(oauthServer(server), redirect)); + // Listed before the answer: a caller following `signIns` for this flow's end sees it under way first. + await listed.change((draft) => { + draft.signIns = signIns().list(); + }, ctx); + return start; + }, + completeSignIn: ({ state, code, iss, error }: { state: string; code?: string; iss?: string; error?: string }) => + signIns().complete( + error !== undefined || code === undefined + ? { state, error: error ?? "The authorization response has no code" } + : { state, code, ...(iss === undefined ? {} : { iss }) }, + ), + pasteSignIn: async ({ id, redirectUrl }: { id: string; redirectUrl: string }) => { + await signIns().paste(id, redirectUrl); + return null; + }, + cancelSignIn: async ({ id }: { id: string }) => { + signIns().cancel(id); + return null; + }, + signOut: async ({ server }: { server: string }) => { + await asPluginError(signIns().signOut(oauthServer(server))); + return null; + }, + probe: async ({ origin }: { origin: string }) => ({ url: await probe().url(origin) }), + }; +} + +/** `promise`, its failures as the caller's errors with their messages (a discovery that failed, an unset variable). */ +async function asPluginError(promise: Promise): Promise { + try { + return await promise; + } catch (error) { + if (error instanceof PluginError) throw error; + throw new PluginError("sign_in", error instanceof Error ? error.message : String(error)); + } +} + /** The project configuration the worker found at `location`: none, the file's text, or why it could not read it. */ function projectAt(location: Location, projects: readonly ProjectFile[]): ProjectSource { const file = projects.find((candidate) => sameLocation(candidate.location, location)); @@ -324,7 +415,12 @@ const view: Factory = (_params, principal, self) => { ...errors, ...leases.flatMap((lease) => lease.errors.map((error) => `session ${lease.session}: ${error}`)), ]), + signIns: manager.signIns, restart: (args, ctx) => manager.restart(args, ctx), + signIn: ({ server }, ctx) => manager.beginSignIn({ server, redirect: { kind: "loopback" } }, ctx), + paste: (args, ctx) => manager.pasteSignIn(args, ctx), + cancelSignIn: (args, ctx) => manager.cancelSignIn(args, ctx), + signOut: (args, ctx) => manager.signOut(args, ctx), }, }; return { variant: "worker", members: workerView(principal, manager, self) }; diff --git a/apps/p2/plugins/mcp/sign-in.ts b/apps/p2/plugins/mcp/sign-in.ts new file mode 100644 index 000000000..529bcc318 --- /dev/null +++ b/apps/p2/plugins/mcp/sign-in.ts @@ -0,0 +1,343 @@ +// MCP OAuth sign-ins, a native of the manager: the auth provider every OAuth HTTP connection sends its tokens with, and +// the sign-in flows, one per server at a time. A flow redirects the browser either to a loopback callback server on the +// p2 host (the browser runs there) or to p2's own `/mcp/callback` route (the browser reaches p2 through Radius or a +// public host); the final redirect URL pasted by the owner always completes it too. Flows live in memory: a restart +// forgets them, the stored tokens stay. + +import { randomUUID } from "node:crypto"; +import type { McpOAuthConfig } from "@earendil-works/pi-mcp"; +import { + completeMcpSignIn, + createMcpAuthProvider, + listenForLoopbackSignIn, + type McpAuthorizationResponse, + type McpAuthProvider, + type McpClientMetadataDocuments, + type McpOAuthSettings, + type McpSignInOptions, + type McpSignInRedirect, + type OAuthCallbackPage, + type OAuthCallbackServer, + type OAuthChallenge, + responseFromRedirectUrl, + startMcpSignIn, +} from "@earendil-works/pi-mcp/oauth"; +import { PluginError } from "@p2/app/v1"; +import { expandHostVariables } from "./config.ts"; +import type { McpCredentials } from "./credentials.ts"; +import { resultPage } from "./pages.ts"; + +/** An HTTP server that signs in with OAuth: its name, URL and the client settings of its `oauth` entry. */ +export type OAuthServer = { readonly name: string; readonly url: string; readonly config: McpOAuthConfig | undefined }; + +/** Where a sign-in's browser step returns: a loopback callback on the p2 host, or p2's callback route at `url`. */ +export type SignInRedirect = { readonly kind: "loopback" } | { readonly kind: "route"; readonly url: string }; + +/** A begun sign-in: signed in at once (a stored refresh token worked), or the authorization URL to open. */ +export type SignInStart = { readonly kind: "authorized" } | { readonly kind: "redirect"; readonly url: string }; + +/** A sign-in as the owner sees it: no URL, state or code. */ +export type SignInView = { + readonly id: string; + readonly server: string; + readonly via: "loopback" | "route"; + readonly status: "waiting" | "failed"; + readonly error?: string; +}; + +/** What the sign-ins do once a server's credentials changed (signed in or out): reconnect its instances. */ +export type CredentialsChanged = (server: string) => void; + +/** How long a sign-in waits for its browser step. */ +export const SIGN_IN_TIMEOUT_MS = 10 * 60_000; + +/** + * pi's Client ID Metadata Documents on pi.dev: they list the loopback redirect URIs `http://127.0.0.1/callback` and + * `http://localhost/callback` (any port), so loopback sign-ins may use them; p2's https callback route registers + * dynamically. + */ +const PI_CLIENT_METADATA_DOCUMENTS: McpClientMetadataDocuments = { + baseUrl: "https://pi.dev/oauth", + listsRedirectUrl: (redirectUrl) => { + const url = new URL(redirectUrl); + return ( + url.protocol === "http:" && + (url.hostname === "127.0.0.1" || url.hostname === "localhost") && + url.pathname === "/callback" + ); + }, +}; + +/** One sign-in under way (or failed, kept for the owner to see until replaced or dismissed). */ +type Flow = { + readonly id: string; + readonly server: OAuthServer; + readonly options: McpSignInOptions; + readonly authorization: McpSignInRedirect; + readonly callback: OAuthCallbackServer | undefined; + readonly timer: ReturnType; + status: "waiting" | "completing" | "failed"; + error?: string; +}; + +/** The auth providers and sign-in flows; subscribers hear of every change of `list`. */ +export class SignIns { + readonly #credentials: McpCredentials; + readonly #clientName: string; + readonly #changed: CredentialsChanged; + readonly #timeoutMs: number; + /** The last challenge of each server's connections (its resource metadata URL and scope), by server key. */ + readonly #challenges = new Map(); + /** The current flow of each server, by name. */ + readonly #flows = new Map(); + readonly #listeners = new Set<() => void>(); + + constructor(credentials: McpCredentials, clientName: string, changed: CredentialsChanged, timeoutMs: number) { + this.#credentials = credentials; + this.#clientName = clientName; + this.#changed = changed; + this.#timeoutMs = timeoutMs; + } + + /** `listener` is called after every change of `list`; the result stops it. */ + subscribe(listener: () => void): () => void { + this.#listeners.add(listener); + return () => this.#listeners.delete(listener); + } + + /** The flows, by server. */ + list(): SignInView[] { + return [...this.#flows.values()] + .map((flow) => ({ + id: flow.id, + server: flow.server.name, + via: flow.callback === undefined ? ("route" as const) : ("loopback" as const), + // Completing is still waiting to the owner: a paste meanwhile is refused as for any finished flow. + status: flow.status === "failed" ? ("failed" as const) : ("waiting" as const), + ...(flow.error === undefined ? {} : { error: flow.error }), + })) + .sort((a, b) => (a.server < b.server ? -1 : a.server > b.server ? 1 : 0)); + } + + /** The auth provider of `server`'s connections: stored tokens, refreshed; never a browser step. */ + authProvider(server: OAuthServer): McpAuthProvider { + return createMcpAuthProvider({ + serverUrl: server.url, + store: this.#credentials.forServer(server.name, server.url), + settings: () => settingsOf(server.config), + onChallenge: (challenge) => this.#challenges.set(keyOf(server), challenge), + clientName: this.#clientName, + clientMetadataDocuments: PI_CLIENT_METADATA_DOCUMENTS, + }); + } + + /** + * Begins a sign-in to `server`, replacing its current one. A pre-registered client (`oauth.clientId`) always uses the + * loopback callback: its redirect URIs are registered as configured. + */ + async begin(server: OAuthServer, redirect: SignInRedirect): Promise { + const settings = settingsOf(server.config); + const store = this.#credentials.forServer(server.name, server.url); + const listening = + redirect.kind === "route" && settings.clientId === undefined + ? { redirectUrl: redirect.url, callback: undefined } + : await listenForLoopbackSignIn({ + serverUrl: server.url, + store, + settings, + clientMetadataDocuments: PI_CLIENT_METADATA_DOCUMENTS, + renderPage: (page) => callbackPage(server.name, page), + }); + const options: McpSignInOptions = { + serverUrl: server.url, + store, + settings, + redirectUrl: listening.redirectUrl, + challenge: this.#challenges.get(keyOf(server)), + clientName: this.#clientName, + clientMetadataDocuments: PI_CLIENT_METADATA_DOCUMENTS, + }; + let start: Awaited>; + try { + start = await startMcpSignIn(options); + } catch (error) { + await listening.callback?.close().catch(() => {}); + throw error; + } + const previous = this.#flows.get(server.name); + if (previous !== undefined) this.#drop(previous); + if (start.kind === "authorized") { + await listening.callback?.close().catch(() => {}); + this.#notify(); + this.#changed(server.name); + return { kind: "authorized" }; + } + const flow: Flow = { + id: randomUUID(), + server, + options, + authorization: start, + callback: listening.callback, + timer: setTimeout(() => this.#fail(flow, new Error("The sign-in expired; start it again")), this.#timeoutMs), + status: "waiting", + }; + flow.timer.unref(); + this.#flows.set(server.name, flow); + flow.callback + ?.waitForCallback(start.state, new URL(start.redirectUrl).pathname) + .then( + (response) => this.#finish(flow, response), + (error: unknown) => this.#fail(flow, error), + ) + .catch(() => {}); + this.#notify(); + return { kind: "redirect", url: start.authorizationUrl }; + } + + /** + * Completes the route sign-in whose `state` the authorization response carries (an error response fails it); the + * result names its server. + */ + async complete( + response: { state: string } & ({ code: string; iss?: string } | { error: string }), + ): Promise<{ server: string }> { + const flow = [...this.#flows.values()].find( + (candidate) => + candidate.callback === undefined && + candidate.status === "waiting" && + candidate.authorization.state === response.state, + ); + if (flow === undefined) throw new PluginError("not_found", "This sign-in is unknown, finished or expired"); + if ("error" in response) { + this.#fail(flow, new Error(response.error)); + throw new PluginError("sign_in", response.error); + } + await this.#finish(flow, { + code: response.code, + state: response.state, + ...(response.iss ? { iss: response.iss } : {}), + }); + return { server: flow.server.name }; + } + + /** Completes sign-in `id` with the final redirect URL the owner copied from the browser. */ + async paste(id: string, redirectUrl: string): Promise { + const flow = this.#byId(id); + if (flow?.status !== "waiting") + throw new PluginError("not_found", "This sign-in is unknown, finished or expired"); + // A URL of another sign-in or no code is refused without failing this one. + let response: McpAuthorizationResponse; + try { + response = responseFromRedirectUrl(redirectUrl, flow.authorization); + } catch (error) { + throw new PluginError("invalid_url", messageOf(error)); + } + await this.#finish(flow, response); + } + + /** Ends sign-in `id`, under way or failed. */ + cancel(id: string): void { + const flow = this.#byId(id); + if (flow === undefined) return; + this.#drop(flow); + this.#notify(); + } + + /** Forgets `server`'s credentials and ends its sign-in; its instances reconnect, without them. */ + async signOut(server: OAuthServer): Promise { + const flow = this.#flows.get(server.name); + if (flow !== undefined) { + this.#drop(flow); + this.#notify(); + } + await this.#credentials.remove(server.name, server.url); + this.#changed(server.name); + } + + /** Ends every flow (their loopback servers close). */ + async close(): Promise { + const flows = [...this.#flows.values()]; + this.#flows.clear(); + await Promise.allSettled( + flows.map((flow) => { + clearTimeout(flow.timer); + return flow.callback?.close(); + }), + ); + await this.#credentials.close(); + } + + #byId(id: string): Flow | undefined { + return [...this.#flows.values()].find((flow) => flow.id === id); + } + + /** Exchanges the code of `response`; a failure fails the flow and is thrown. */ + async #finish(flow: Flow, response: McpAuthorizationResponse): Promise { + if (flow.status !== "waiting" || this.#flows.get(flow.server.name) !== flow) + throw new PluginError("not_found", "This sign-in is unknown, finished or expired"); + flow.status = "completing"; + try { + await completeMcpSignIn({ ...flow.options, authorization: flow.authorization, response }); + } catch (error) { + this.#fail(flow, error); + throw new PluginError("sign_in", messageOf(error)); + } + if (this.#flows.get(flow.server.name) === flow) { + this.#drop(flow); + this.#notify(); + } + this.#changed(flow.server.name); + } + + /** Marks `flow` failed (if it is still the server's current one and has not finished). */ + #fail(flow: Flow, error: unknown): void { + if (this.#flows.get(flow.server.name) !== flow || flow.status === "failed") return; + flow.status = "failed"; + flow.error = messageOf(error); + clearTimeout(flow.timer); + void flow.callback?.close().catch(() => {}); + this.#notify(); + } + + /** Removes `flow`: its timer stops and its loopback server closes. */ + #drop(flow: Flow): void { + if (this.#flows.get(flow.server.name) === flow) this.#flows.delete(flow.server.name); + clearTimeout(flow.timer); + void flow.callback?.close().catch(() => {}); + } + + #notify(): void { + for (const listener of [...this.#listeners]) listener(); + } +} + +/** The settings of an `oauth` entry: `${NAME}` in the client secret from the p2 host's environment. */ +function settingsOf(config: McpOAuthConfig | undefined): McpOAuthSettings { + if (config === undefined) return {}; + return { + clientId: config.clientId, + clientSecret: config.clientSecret === undefined ? undefined : expandHostVariables(config.clientSecret), + callbackPort: config.callbackPort, + callbackUrl: config.callbackUrl, + scope: config.scope, + clientName: config.clientName, + clientRegistration: config.clientRegistration, + authServerMetadataUrl: config.authServerMetadataUrl ? new URL(config.authServerMetadataUrl) : undefined, + }; +} + +/** A server's key for its challenges: its name and URL, as its credentials'. */ +function keyOf(server: OAuthServer): string { + return JSON.stringify([server.name, server.url]); +} + +/** The loopback callback's page after the redirect. */ +function callbackPage(server: string, page: OAuthCallbackPage): string { + return page.ok + ? resultPage(`Signed in to ${server}`, "You may close this page.") + : resultPage(`Signing in to ${server} failed`, page.details ? `${page.message} ${page.details}` : page.message); +} + +function messageOf(error: unknown): string { + return error instanceof Error ? error.message : String(error); +} diff --git a/apps/p2/plugins/mcp/tools.ts b/apps/p2/plugins/mcp/tools.ts index 762cfcbb3..a1607a11a 100644 --- a/apps/p2/plugins/mcp/tools.ts +++ b/apps/p2/plugins/mcp/tools.ts @@ -21,7 +21,7 @@ import { toLlmContent, } from "@earendil-works/pi-mcp"; import type { Context } from "@p2/app/v1"; -import { type Instance, MCP_EXTENSION, type McpTool } from "./contract.ts"; +import { type Instance, MCP_EXTENSION, type McpTool, signInLink } from "./contract.ts"; /** One tool of one server. */ export type ServerTool = { readonly server: string; readonly tool: McpTool }; @@ -33,9 +33,13 @@ export type ServerTool = { readonly server: string; readonly tool: McpTool }; */ export type McpReach = "codemode" | "tool_search"; -/** What the MCP extension offers: the session's servers by name, their tools, and how the model reaches them. */ +/** + * What the MCP extension offers: the session's servers by name, those waiting for the owner to sign in, their tools, + * and how the model reaches them. + */ export type McpOffer = { readonly servers: readonly string[]; + readonly signIn: readonly string[]; readonly tools: readonly ServerTool[]; readonly reach: McpReach; }; @@ -69,19 +73,24 @@ export function serverNames(instances: readonly Instance[]): string[] { return [...new Set(instances.map((instance) => instance.server))].sort(); } +/** The servers of `instances` waiting for a sign-in, by name. */ +export function signInNames(instances: readonly Instance[]): string[] { + return serverNames(instances.filter((instance) => instance.status === "sign-in")); +} + /** * The Durable extension offering `offer`'s tools, each calling through `call`, and the `mcp_servers` section. As in pi * (and Codex), every tool whose name sanitizes to one another tool's also gets pi's hash suffix, so no tool's name * depends on the order of the list and a new tool renames only those it collides with. */ -export function createMcpExtension({ servers, tools, reach }: McpOffer, call: McpCall): Extension { +export function createMcpExtension({ servers, signIn, tools, reach }: McpOffer, call: McpCall): Extension { const plain = new Map(); for (const { server, tool } of tools) { const name = createMcpToolName(server, tool.name); plain.set(name, (plain.get(name) ?? 0) + 1); } const assigned = new Set(); - const listing = serversSection(servers, reach); + const listing = serversSection(servers, signIn, reach); return defineExtension({ name: MCP_EXTENSION, tools: tools.map((entry) => { @@ -108,15 +117,19 @@ const MAX_SERVERS_SECTION_CHARS = 4096; /** * The `mcp_servers` section, as pi renders it for servers without a description: each server's namespace and how its - * tools are reached. Undefined without servers. + * tools are reached; for a server waiting for a sign-in, the link the user signs in with. Undefined without servers. */ -function serversSection(servers: readonly string[], reach: McpReach): string | undefined { +function serversSection(servers: readonly string[], signIn: readonly string[], reach: McpReach): string | undefined { if (servers.length === 0) return undefined; const intro = reach === "codemode" ? "MCP servers whose tools are not declared to you. Call the tools of `codemode` servers from codemode scripts." : "MCP servers whose tools are not declared to you. Load the tools of `tool_search` servers with `tool_search`."; - const lines = servers.map((server) => `- ${mcpNamespace(server)} (${reach})`); + const lines = servers.map((server) => + signIn.includes(server) + ? `- ${mcpNamespace(server)} (${reach}; needs a sign-in, no tools yet: the user signs in from the MCP panel in p2's sidebar or at ${signInLink(server)} on p2)` + : `- ${mcpNamespace(server)} (${reach})`, + ); const find = reach === "codemode" ? "searchTools()" : "tool_search"; const omitted = (count: number) => count > 0 ? [`- … ${count} more server${count === 1 ? "" : "s"}; find their tools with ${find}`] : []; diff --git a/apps/p2/plugins/mcp/transports.ts b/apps/p2/plugins/mcp/transports.ts index 6d6a2603f..5cadc72cc 100644 --- a/apps/p2/plugins/mcp/transports.ts +++ b/apps/p2/plugins/mcp/transports.ts @@ -1,14 +1,21 @@ // How an instance opens its MCP transport, by where it runs: a program on the p2 host (`StdioTransport`), a program // on an environment through its pi-env daemon (`PipeTransport` over `openProcess`, through `cmd.exe` on Windows), or -// a Streamable HTTP server. +// a Streamable HTTP server (with the stored OAuth tokens when it signs in). import { posix, win32 } from "node:path"; import { pathToFileURL } from "node:url"; import { type Connection, nodePlatform, openProcess, RemoteError } from "@earendil-works/pi-env"; -import { type McpTransport, PipeTransport, StdioTransport, StreamableHttpTransport } from "@earendil-works/pi-mcp"; +import { + type AuthProvider, + type McpTransport, + PipeTransport, + StdioTransport, + StreamableHttpTransport, +} from "@earendil-works/pi-mcp"; import { expandHostVariables } from "./config.ts"; import type { EnvConnections } from "./environments.ts"; import type { Connect, RemoteConnect } from "./placement.ts"; +import type { OAuthServer } from "./sign-in.ts"; import { windowsCommand } from "./windows-command.ts"; /** A transport, not started yet, and the `file://` roots its client offers. */ @@ -20,8 +27,14 @@ export type Transports = { open(): Promise; release(): void }; /** The environment's daemon predates the `process` operation: retrying does not help, updating it does. */ export class DaemonTooOldError extends Error {} -/** The transports of an instance connecting as `connect`; an environment's connection is taken from `envs`. */ -export function transportsFor(connect: Connect, envs: EnvConnections): Transports { +/** The auth provider of an HTTP server that signs in with OAuth. */ +export type AuthProviders = (server: OAuthServer) => AuthProvider; + +/** + * The transports of an instance connecting as `connect`; an environment's connection is taken from `envs`, an OAuth + * server's tokens from `auth`. + */ +export function transportsFor(connect: Connect, envs: EnvConnections, auth: AuthProviders): Transports { switch (connect.kind) { case "remote": { const lease = envs.acquire(connect.environment, connect.config); @@ -41,14 +54,20 @@ export function transportsFor(connect: Connect, envs: EnvConnections): Transport }), release: () => {}, }; - case "http": + case "http": { + const authProvider = connect.oauth === undefined ? undefined : auth(connect.oauth); return { open: async () => ({ - transport: new StreamableHttpTransport({ url: connect.url, headers: expandValues(connect.headers) }), + transport: new StreamableHttpTransport({ + url: connect.url, + headers: expandValues(connect.headers), + ...(authProvider === undefined ? {} : { authProvider }), + }), roots: [], }), release: () => {}, }; + } case "unplaced": return { open: async () => { diff --git a/apps/p2/plugins/mcp/tui.ts b/apps/p2/plugins/mcp/tui.ts new file mode 100644 index 000000000..95d665a03 --- /dev/null +++ b/apps/p2/plugins/mcp/tui.ts @@ -0,0 +1,207 @@ +// `/mcp` in the TUI (owners), after the web's MCP sheet: the running servers with status, place, tools and errors; +// Ctrl+R restarts one, Ctrl+S signs in to an HTTP server using OAuth (the authorization page opens through the +// launcher's `openUrl`, confirmed in its own screen), Ctrl+O signs out. The sign-in returns to a loopback callback on +// the p2 host; a browser elsewhere ends on a page that does not load, whose address `/mcp paste ` completes. + +import { + BACKGROUND_CONTEXT, + contribute, + defineFacet, + follow, + type Held, + type HeldIn, + liveIn, + locationText, + until, + whenLive, +} from "@p2/app/v1"; +import type { TuiContext } from "@p2/app/v1/tui"; +import { ListOverlay, type ListRow, lineText, type TuiCommand, TuiCommands } from "@p2/app/v1/tui"; +import { type Instance, Mcp, type SignIn } from "./contract.ts"; + +type OwnerView = HeldIn; + +const STATUS: Record = { + starting: "starting", + connected: "connected", + "sign-in": "sign-in needed", + failed: "failed", +}; + +const COLOR: Record = { + starting: "muted", + connected: "success", + "sign-in": "warning", + failed: "error", +}; + +const messageOf = (error: unknown): string => (error instanceof Error ? error.message : String(error)); + +/** `/mcp [paste ]`: the overlay, or a pasted redirect URL for the sign-in waiting for it. */ +function openMcp(ctx: TuiContext, args: string | undefined): void { + const ref = ctx.host.hold(Mcp, {}, BACKGROUND_CONTEXT); + const words = args?.trim().split(/\s+/).filter(Boolean) ?? []; + if (words[0] === "paste") { + void paste(ctx, ref, words[1]).finally(() => ref.release()); + return; + } + let instances: Instance[] = []; + let signIns: SignIn[] = []; + let status = ""; + let closed = false; + const stops: (() => void)[] = []; + void whenLive(ref) + .then(() => { + const view = liveIn(ref, "owner"); + if (closed || view === undefined) return; + stops.push( + follow(view.instances, (value) => { + instances = value; + ctx.requestRender(); + }), + follow(view.signIns, (value) => { + signIns = value; + ctx.requestRender(); + }), + ); + }) + .catch(() => {}); + const run = (label: string, action: (view: OwnerView) => Promise) => { + const view = liveIn(ref, "owner"); + if (view === undefined) return; + status = `${label}…`; + ctx.requestRender(); + void action(view) + .then(() => { + status = `${label}: done`; + }) + .catch((error: unknown) => { + status = `${label}: ${messageOf(error)}`; + }) + .finally(() => ctx.requestRender()); + }; + const byRow = (row: ListRow | undefined) => instances.find((instance) => instance.id === row?.key); + const signIn = (view: OwnerView, server: string) => + view.signIn({ server }, BACKGROUND_CONTEXT).then(async (start) => { + if (start.kind === "authorized") return; + const opened = await ctx.grant("openUrl", { kind: "external", url: start.url }); + status = opened.ok + ? `Signing in to ${server}: finish in the browser; if it ends on a page that does not load, /mcp paste ` + : `Open the sign-in page in a browser (${opened.error}); /mcp paste if it does not return`; + }); + const overlay = ctx.overlay( + "mcp", + new ListOverlay(ctx, { + title: () => `MCP servers (${instances.length})`, + rows: () => + instances.map((instance) => { + const flow = signIns.find((entry) => entry.server === instance.server); + const place = [ + instance.location ? locationText(instance.location) : "", + instance.runsOn === "host" ? "" : `on ${instance.runsOn.env}`, + ] + .filter(Boolean) + .join(" · "); + return { + key: instance.id, + label: lineText(`${instance.server} ${STATUS[instance.status]}`), + detail: lineText( + instance.status === "connected" + ? `${instance.tools.length} tool${instance.tools.length === 1 ? "" : "s"}` + : place, + ), + description: lineText( + flow?.status === "waiting" + ? "sign-in waiting for the browser (/mcp paste completes it)" + : flow?.status === "failed" + ? `sign-in failed: ${flow.error ?? ""}` + : (instance.error ?? place), + ), + color: COLOR[instance.status], + }; + }), + actions: [ + { + key: "ctrl+r", + label: "restart", + run: (row) => { + const instance = byRow(row); + if (instance) + run(`Restarting ${instance.server}`, (view) => + view.restart({ instance: instance.id }, BACKGROUND_CONTEXT), + ); + }, + }, + { + key: "ctrl+s", + label: "sign in", + run: (row) => { + const instance = byRow(row); + if (instance?.auth === "oauth") + run(`Signing in to ${instance.server}`, (view) => signIn(view, instance.server)); + }, + }, + { + key: "ctrl+o", + label: "sign out", + run: (row) => { + const instance = byRow(row); + if (instance?.auth === "oauth") + run(`Signing out of ${instance.server}`, (view) => + view.signOut({ server: instance.server }, BACKGROUND_CONTEXT), + ); + }, + }, + ], + footer: () => (status ? [ctx.theme.current.fg("muted", ` ${lineText(status)}`)] : []), + onClose: () => { + closed = true; + for (const stop of stops) stop(); + ref.release(); + overlay.close(); + }, + }), + { width: "100%", anchor: "bottom-center" }, + ); +} + +/** Completes the waiting sign-in whose redirect URL `url` is (each waiting one is tried: the URL's state picks it). */ +async function paste(ctx: TuiContext, ref: Held, url: string | undefined): Promise { + if (!url) return ctx.notify("Usage: /mcp paste "); + try { + await whenLive(ref); + const view = liveIn(ref, "owner"); + if (view === undefined) return ctx.notify("MCP servers are for the owner", "error"); + const signIns = await until(view.signIns, () => true, BACKGROUND_CONTEXT); + const waiting = signIns.filter((entry) => entry.status === "waiting"); + if (waiting.length === 0) return ctx.notify("No MCP sign-in is waiting", "error"); + let failure: unknown; + for (const entry of waiting) { + try { + await view.paste({ id: entry.id, redirectUrl: url }, BACKGROUND_CONTEXT); + return ctx.notify(`Signed in to ${entry.server}`); + } catch (error) { + failure = error; + } + } + ctx.notify(messageOf(failure), "error"); + } catch (error) { + ctx.notify(messageOf(error), "error"); + } +} + +const COMMANDS: Record = { + "p2.mcp": { + title: "MCP servers", + category: "Owner", + when: "owner && !overlay", + slash: { name: "mcp", argumentHint: "[paste ]" }, + run: (ctx, args) => openMcp(ctx, args), + }, +}; + +export default defineFacet((host) => + contribute(host, "p2.tui-mcp", (self) => { + self.contribute(TuiCommands, COMMANDS); + }), +); diff --git a/apps/p2/plugins/mcp/web.tsx b/apps/p2/plugins/mcp/web.tsx new file mode 100644 index 000000000..49cc59d51 --- /dev/null +++ b/apps/p2/plugins/mcp/web.tsx @@ -0,0 +1,223 @@ +// The MCP sheet (owners): the running servers with their status, errors and tools, restart, and signing in and out of +// HTTP servers that use OAuth. Sign in opens `/mcp/sign-in/` in a new tab (it finds out where the browser runs); +// while a sign-in waits, the final redirect URL can be pasted here, for a browser that cannot reach where it returns. + +import { BACKGROUND_CONTEXT, contribute, defineFacet, type HeldIn, locationText } from "@p2/app/v1"; +import { + ChevronIcon, + closeLayers, + openLayer, + Sheet, + Slots, + useComposerState, + useObject, + useOwner, + useReadable, + useSheet, +} from "@p2/app/v1/web"; +import { useState } from "react"; +import { type Instance, Mcp, type SignIn, signInLink } from "./contract.ts"; + +type OwnerView = HeldIn; + +const errorText = (reason: unknown): string => (reason instanceof Error ? reason.message : String(reason)); + +const STATUS: Record = { + starting: "Starting", + connected: "Connected", + "sign-in": "Sign-in needed", + failed: "Failed", +}; + +/** The sheet: configuration errors, then each server with its instances and sign-in. */ +function McpSheet({ onClose }: { onClose: () => void }) { + const view = useObject(Mcp, {}, { variant: "owner" }); + const instances = useReadable(view?.instances) ?? []; + const errors = useReadable(view?.errors) ?? []; + const signIns = useReadable(view?.signIns) ?? []; + const servers = [...new Set(instances.map((instance) => instance.server))]; + return ( + +

+ Servers of mcp.json run while sessions use them. Sign-in credentials are + stored on the server. +

+ {errors.map((error) => ( +

+ {error} +

+ ))} + {view && servers.length === 0 &&

No MCP server is running.

} + {view && ( +
+ {servers.map((server) => ( + instance.server === server)} + signIn={signIns.find((entry) => entry.server === server)} + /> + ))} +
+ )} + + ); +} + +/** One server: its instances (status, place, tools, error, restart), and signing in or out if it uses OAuth. */ +function ServerEntry({ + view, + server, + instances, + signIn, +}: { + view: OwnerView; + server: string; + instances: readonly Instance[]; + signIn: SignIn | undefined; +}) { + const { setError } = useComposerState(); + const perform = (action: Promise) => void action.catch((reason: unknown) => setError(errorText(reason))); + const oauth = instances.some((instance) => instance.auth === "oauth"); + const needsSignIn = instances.some((instance) => instance.status === "sign-in"); + const connected = instances.some((instance) => instance.status === "connected"); + return ( +
+ {server} + {instances.map((instance) => ( +
+

+ {STATUS[instance.status]} + {instance.location && · {locationText(instance.location)}} + {instance.runsOn !== "host" && · on {instance.runsOn.env}} + {instance.status === "connected" && ( + + {" "} + · {instance.tools.length} tool{instance.tools.length === 1 ? "" : "s"} + + )} +

+ {instance.error && instance.status !== "sign-in" &&

{instance.error}

} +
+ +
+
+ ))} + {oauth && signIn?.status === "waiting" && } + {oauth && signIn?.status === "failed" && ( +
+

{signIn.error ?? "The sign-in failed."}

+ +
+ )} + {oauth && ( +
+ {needsSignIn || signIn !== undefined ? ( + + {signIn?.status === "waiting" ? "Sign in again" : "Sign in"} + + + ) : null} + {connected && ( + + )} +
+ )} +
+ ); +} + +/** A sign-in waiting for the browser: paste the final redirect URL, or cancel. */ +function PendingSignIn({ view, signIn }: { view: OwnerView; signIn: SignIn }) { + const [value, setValue] = useState(""); + const [sending, setSending] = useState(false); + const [error, setError] = useState(); + return ( +
{ + event.preventDefault(); + setSending(true); + setError(undefined); + view + .paste({ id: signIn.id, redirectUrl: value.trim() }, BACKGROUND_CONTEXT) + .then(() => setValue("")) + .catch((reason: unknown) => setError(errorText(reason))) + .finally(() => setSending(false)); + }} + > +

+ Waiting for the sign-in in the browser. If it ends on a page that does not load ( + {signIn.via === "loopback" ? "a 127.0.0.1 address" : "p2's address"}), copy that page's full address from + the address bar and paste it here. +

+ setValue(event.target.value)} + /> + {error &&

{error}

} +
+ + +
+
+ ); +} + +/** The sidebar footer's MCP button: any authenticated owner. */ +function McpButton() { + const owner = useOwner(); + const open = useSheet("mcp") !== undefined; + if (!owner) return null; + return ( + <> + + {open && } + + ); +} + +export default defineFacet((host) => { + contribute(host, "p2.web-mcp", (self) => { + self.contribute(Slots, { "sidebar-foot:mcp": McpButton }); + }); +}); diff --git a/apps/p2/plugins/mcp/worker.ts b/apps/p2/plugins/mcp/worker.ts index e966488a7..d9e841bec 100644 --- a/apps/p2/plugins/mcp/worker.ts +++ b/apps/p2/plugins/mcp/worker.ts @@ -38,7 +38,15 @@ import { } from "@p2/app/v1/worker"; import { Subagents } from "../subagents/contract.worker.ts"; import { MCP_EXTENSION, Mcp, PROJECT_FILE_MAX_CHARS, type ProjectFile } from "./contract.ts"; -import { createMcpExtension, type McpCall, type McpOffer, type McpReach, serverNames, serverTools } from "./tools.ts"; +import { + createMcpExtension, + type McpCall, + type McpOffer, + type McpReach, + serverNames, + serverTools, + signInNames, +} from "./tools.ts"; /** How long to wait before setting the locations again after a failure. */ const RETRY_MS = 2000; @@ -67,7 +75,7 @@ function mcpWorker(self: StoreSelf): void { [self.registry(ExtensionSwitches)], ([switches]): McpReach => (switches.get(CODEMODE_EXTENSION) === true ? "codemode" : "tool_search"), ); - const extension = self.writable(extensionsFor({ servers: [], tools: [], reach: "codemode" }, call)); + const extension = self.writable(extensionsFor({ servers: [], signIn: [], tools: [], reach: "codemode" }, call)); self.contribute(Extensions, extension); self.onLive((ctx) => offerTools(view, reach, extension, call, ctx)); leaseLocations(self, view, harness, worker); @@ -84,8 +92,8 @@ function extensionsFor(offer: McpOffer, call: McpCall): Readonly, @@ -102,7 +110,12 @@ async function offerTools( let current = reaches.value; let offered = ""; const offer = (): void => { - const next: McpOffer = { servers: serverNames(servers), tools: serverTools(servers), reach: current }; + const next: McpOffer = { + servers: serverNames(servers), + signIn: signInNames(servers), + tools: serverTools(servers), + reach: current, + }; const key = JSON.stringify(next); if (key === offered) return; offered = key; diff --git a/apps/p2/plugins/sessions/web.tsx b/apps/p2/plugins/sessions/web.tsx index 78dec2101..c11cd2821 100644 --- a/apps/p2/plugins/sessions/web.tsx +++ b/apps/p2/plugins/sessions/web.tsx @@ -2,6 +2,7 @@ import { Access, BACKGROUND_CONTEXT, contribute, defineFacet, type HeldIn, liveIn, whenLive } from "@p2/app/v1"; import { + ChevronIcon, Drawer, navigate, openLayer, @@ -17,7 +18,7 @@ import { useRoute, useStatus, } from "@p2/app/v1/web"; -import { useCallback, useEffect, useMemo, useState, useSyncExternalStore } from "react"; +import { useCallback, useEffect, useMemo, useRef, useState, useSyncExternalStore } from "react"; import { Search, SessionReads } from "../search/contract.ts"; import { SessionStatus } from "../status/contract.ts"; import { Subagents } from "../subagents/contract.ts"; @@ -33,17 +34,59 @@ const subscribeVisibility = (listener: () => void) => { /** Whether the page is shown (not a background tab or a hidden app). */ const usePageVisible = () => useSyncExternalStore(subscribeVisibility, () => document.visibilityState === "visible"); -// Explicit mobile ordering. -const footer = ( - <> - - - - - - - -); +/** + * The footer's owner entries, in an explicit order, behind one "More" button whose menu opens upwards: one row of + * buttons no longer fits them. The entries stay mounted while the menu is closed, because each renders the sheet it + * opens (through a portal, so the hidden menu does not hide it). + */ +function FooterMenu() { + const [open, setOpen] = useState(false); + const menu = useRef(null); + useEffect(() => { + if (!open) return; + const outside = (event: PointerEvent) => { + if (!menu.current?.contains(event.target as Node)) setOpen(false); + }; + const onKey = (event: KeyboardEvent) => { + if (event.key === "Escape") setOpen(false); + }; + document.addEventListener("pointerdown", outside); + document.addEventListener("keydown", onKey); + return () => { + document.removeEventListener("pointerdown", outside); + document.removeEventListener("keydown", onKey); + }; + }, [open]); + return ( +
+ + {/* An entry's click opens its sheet, then closes the menu. */} + {/* biome-ignore lint/a11y/useKeyWithClickEvents lint/a11y/noStaticElementInteractions: the entries are buttons; this only notices their clicks. */} + +
+ ); +} + +const footer = ; function Sidebar() { const connected = useConnected(); const route = useRoute(); diff --git a/apps/p2/src/web/style.css b/apps/p2/src/web/style.css index 1e63559e8..69e5889b5 100644 --- a/apps/p2/src/web/style.css +++ b/apps/p2/src/web/style.css @@ -2980,6 +2980,43 @@ button.diff-number:hover { .foot-button.on { font-weight: var(--fw-strong); } +/* The footer's "More" menu: owner entries in a list opening upwards from the button. */ +.foot-menu { + position: relative; +} +.foot-menu-chevron { + display: flex; + transform: rotate(-90deg); + transition: transform 0.15s ease-out; +} +.foot-button[aria-expanded="true"] .foot-menu-chevron { + transform: rotate(90deg); +} +.foot-menu-list { + position: absolute; + bottom: calc(100% + var(--s1)); + left: 0; + z-index: 5; + display: flex; + flex-direction: column; + min-width: 180px; + padding: var(--s1) 0; + border: 1px solid var(--line); + border-radius: var(--radius); + background: var(--bg); + box-shadow: 0 6px 24px rgb(0 0 0 / 0.18); +} +.foot-menu-list[hidden] { + display: none; +} +.foot-menu-list .foot-button { + width: 100%; + padding: 0 var(--x); + text-align: left; +} +.foot-menu-list .foot-button:hover { + background: var(--hover); +} /* Sheets: bottom sheet on phones, centred dialog on wide screens. Above the workspace panel (35), below the viewer (40): only the top navigation layer is ever shown. */ diff --git a/apps/p2/test/p2/mcp-instances.test.ts b/apps/p2/test/p2/mcp-instances.test.ts index 200613976..6cd315678 100644 --- a/apps/p2/test/p2/mcp-instances.test.ts +++ b/apps/p2/test/p2/mcp-instances.test.ts @@ -21,6 +21,10 @@ const SIGNAL = new AbortController().signal; const NO_ENVIRONMENTS = (): never => { throw new Error("no environments in this test"); }; +/** No HTTP servers either. */ +const NO_OAUTH = (): never => { + throw new Error("no OAuth servers in this test"); +}; /** A supervisor running the fixture once for session s1 at a temporary directory; `env` added to its environment. */ function supervisor(t: { after(fn: () => unknown): void }, env: Record = {}, seed: Instance[] = []) { @@ -45,7 +49,7 @@ function supervisor(t: { after(fn: () => unknown): void }, env: Record { await instances.close(); rmSync(root, { recursive: true, force: true }); @@ -129,7 +133,7 @@ test("a new supervisor lists the tools its predecessor listed until the server a status: "connected", tools: [{ name: "echo", inputSchema: { type: "object" } }], }; - const instances = new Instances(CLIENT, BACKOFF, [seed], NO_ENVIRONMENTS); + const instances = new Instances(CLIENT, BACKOFF, [seed], NO_ENVIRONMENTS, NO_OAUTH); t.after(() => instances.close()); instances.reconcile(first.placement); assert.deepEqual( diff --git a/apps/p2/test/p2/mcp-sign-in.test.ts b/apps/p2/test/p2/mcp-sign-in.test.ts new file mode 100644 index 000000000..53291d40f --- /dev/null +++ b/apps/p2/test/p2/mcp-sign-in.test.ts @@ -0,0 +1,376 @@ +// MCP OAuth sign-ins (plugins/mcp/sign-in.ts, routes.ts, probe.ts) on a bare server host, against a fake authorization +// server whose MCP server takes only the tokens it issued: a 401 waits for a sign-in, the loopback flow (the owner +// view's `signIn` and the device owner's sign-in route), the callback route of relayed and public owners with the probe +// page and the probe, pasting, refusals (wrong state, guests, nobody), refreshing without a browser, signing out. +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import { mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, statSync, writeFileSync } from "node:fs"; +import { createServer, type IncomingMessage, type ServerResponse } from "node:http"; +import type { AddressInfo } from "node:net"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { test } from "node:test"; +import { constant, type P2Principal } from "@p2/app/v1"; +import { findRoute, Hosts, type RequestAccess, Routes, Server } from "@p2/app/v1/server"; +import { EnvService } from "../../plugins/envs/contract.ts"; +import { type Instance, Mcp, McpManager } from "../../plugins/mcp/contract.ts"; +import { mcpServerFacet } from "../../plugins/mcp/server.ts"; +import { Catalog, type Session } from "../../plugins/sessions/contract.ts"; +import { ChildStore } from "../../plugins/subagents/contract.ts"; +import { APP, holdIn, TEST, testHost, until } from "./helpers.ts"; + +const OWNER: P2Principal = { kind: "owner", access: "local" }; +const RELAY_OWNER: P2Principal = { kind: "owner", access: "relay" }; +const GUEST: P2Principal = { + kind: "guest", + id: "g1", + name: "Guest", + home: "s1", + sessions: { s1: "full" }, + revision: 1, +}; +/** The session whose worker leases the servers. */ +const SESSION: Session = { + id: "s1", + env: null, + cwd: "/", + title: "s1", + createdAt: 0, + updatedAt: 0, + archivedAt: null, + pinned: false, + model: { provider: "faux", modelId: "faux-1" }, + thinkingLevel: "off", + main: 1, + branches: [], +}; +/** The origin a relayed owner reaches p2 through (never contacted: the test brings the browser's redirect back). */ +const BRAIN = "https://brain.example"; + +const UNUSED = (): never => { + throw new Error("unused in these tests"); +}; + +/** + * An authorization server with an MCP server at `/mcp` that answers only requests with a token it issued and has not + * revoked: discovery, dynamic client registration, PKCE, rotating refresh tokens. + */ +async function authorizationServer(t: { after(fn: () => unknown): void }) { + const registrations: Record[] = []; + const authorizations: URLSearchParams[] = []; + const grants: string[] = []; + const challenges = new Map(); + const accessTokens = new Set(); + const refreshTokens = new Set(); + let issued = 0; + const issue = () => { + issued++; + accessTokens.add(`access-${issued}`); + refreshTokens.add(`refresh-${issued}`); + return { access_token: `access-${issued}`, refresh_token: `refresh-${issued}`, token_type: "Bearer" }; + }; + const json = (response: ServerResponse, status: number, body: unknown) => + response.writeHead(status, { "content-type": "application/json" }).end(JSON.stringify(body)); + const server = createServer(async (request, response) => { + const url = new URL(request.url ?? "/", origin); + const body = await read(request); + switch (url.pathname) { + case "/.well-known/oauth-protected-resource/mcp": + return json(response, 200, { resource: `${origin}/mcp`, authorization_servers: [origin] }); + case "/.well-known/oauth-authorization-server": + return json(response, 200, { + issuer: origin, + authorization_endpoint: `${origin}/authorize`, + token_endpoint: `${origin}/token`, + registration_endpoint: `${origin}/register`, + response_types_supported: ["code"], + code_challenge_methods_supported: ["S256"], + token_endpoint_auth_methods_supported: ["none"], + }); + case "/register": { + const metadata = JSON.parse(body) as Record; + registrations.push(metadata); + return json(response, 201, { ...metadata, client_id: `client-${registrations.length}` }); + } + case "/authorize": { + authorizations.push(url.searchParams); + const code = `code-${authorizations.length}`; + challenges.set(code, url.searchParams.get("code_challenge") ?? ""); + const redirect = new URL(url.searchParams.get("redirect_uri") ?? ""); + redirect.searchParams.set("code", code); + redirect.searchParams.set("state", url.searchParams.get("state") ?? ""); + return response.writeHead(302, { location: redirect.href }).end(); + } + case "/token": { + const params = new URLSearchParams(body); + grants.push(params.get("grant_type") ?? ""); + if (params.get("grant_type") === "authorization_code") { + const code = params.get("code") ?? ""; + const verifier = createHash("sha256") + .update(params.get("code_verifier") ?? "") + .digest("base64url"); + if (challenges.get(code) !== verifier) return json(response, 400, { error: "invalid_grant" }); + challenges.delete(code); + return json(response, 200, issue()); + } + if (!refreshTokens.delete(params.get("refresh_token") ?? "")) + return json(response, 400, { error: "invalid_grant" }); + return json(response, 200, issue()); + } + case "/mcp": { + const token = request.headers.authorization?.replace(/^Bearer /, "") ?? ""; + if (!accessTokens.has(token)) + return response + .writeHead(401, { + "www-authenticate": `Bearer resource_metadata="${origin}/.well-known/oauth-protected-resource/mcp"`, + }) + .end(); + if (request.method !== "POST") return response.writeHead(405).end(); + const message = JSON.parse(body) as { id?: number; method: string }; + if (message.id === undefined) return response.writeHead(202).end(); + const result = + message.method === "initialize" + ? { + protocolVersion: "2025-06-18", + capabilities: { tools: {} }, + serverInfo: { name: "oauth-fixture", version: "1.0.0" }, + } + : message.method === "tools/list" + ? { tools: [{ name: "whoami", inputSchema: { type: "object" } }] } + : { content: [{ type: "text", text: token }] }; + return json(response, 200, { jsonrpc: "2.0", id: message.id, result }); + } + default: + return response.writeHead(404).end(); + } + }); + await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); + t.after(() => new Promise((resolve) => server.close(resolve))); + const origin = `http://127.0.0.1:${(server.address() as AddressInfo).port}`; + return { + url: `${origin}/mcp`, + registrations, + authorizations, + /** Grant types of token requests. */ + grants, + /** Revokes every access token issued so far (refresh tokens stay valid). */ + expireAccessTokens: () => accessTokens.clear(), + }; +} + +async function read(request: IncomingMessage): Promise { + let body = ""; + for await (const chunk of request) body += String(chunk); + return body; +} + +/** Follows `url` like a browser up to the redirect it answers with, and returns where it redirects. */ +async function redirectOf(url: string): Promise { + const response = await fetch(url, { redirect: "manual" }); + const location = response.headers.get("location"); + assert.ok(location, `${url} answered ${response.status} without a redirect`); + return new URL(location, url); +} + +/** + * A server host with the MCP plugin and `mcp.json` holding the fake server as `docs`, plus `http`: requests to its + * routes as `access` (any principal and origin) through a local HTTP server. + */ +async function signInHost( + t: { after(fn: () => unknown): void }, + oauth: Awaited>, +) { + const root = realpathSync(mkdtempSync(join(tmpdir(), "p2-mcp-sign-in-"))); + const dataDir = join(root, "data"); + mkdirSync(dataDir); + writeFileSync(join(dataDir, "mcp.json"), JSON.stringify({ mcpServers: { docs: { url: oauth.url } } })); + const fixture = await testHost({ + facets: [mcpServerFacet(150, () => UNUSED)], + stubs: [ + [ + EnvService, + { configs: constant({}), tests: constant({}), failure: constant(null), test: UNUSED, reload: UNUSED }, + ], + [Catalog, { sessions: constant([SESSION]), get: UNUSED, create: UNUSED, activity: UNUSED, update: UNUSED }], + [ChildStore, { children: constant({}), report: UNUSED }], + [ + Server, + { + info: constant({ app: APP, dataDir, fake: true, allowedHosts: [], port: 0 }), + revokeOwners: () => {}, + revokeGuests: () => {}, + }, + ], + [Hosts, { workers: constant({ s1: { running: true, generation: 1 } }), stop: () => {} }], + ], + }); + const manager = await fixture.host.get(McpManager, {}, TEST); + // Session s1's worker leases the servers (shared ones start with their first lease). + const worker = await holdIn(fixture.client({ kind: "worker", session: "s1", generation: 1 }), Mcp, {}, "worker"); + await worker.setLocations({ locations: [{ env: null, cwd: root }] }, TEST); + const routes = (await fixture.host.registry(Routes, TEST)).value; + let access: RequestAccess = { + principal: OWNER, + origin: { origin: "http://127.0.0.1:8788", remote: false, port: 8788 }, + }; + const server = createServer((request, response) => { + const route = findRoute(routes, request.method ?? "GET", new URL(request.url ?? "/", "http://x").pathname); + if (route === undefined) return void response.writeHead(404).end(); + void fixture.root((ctx) => route(request, response, access, ctx)); + }); + await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); + const base = `http://127.0.0.1:${(server.address() as AddressInfo).port}`; + t.after(async () => { + worker.release(); + await until(manager.leases, (leases) => leases.length === 0); + await new Promise((resolve) => server.close(resolve)); + await fixture.close(); + rmSync(root, { recursive: true, force: true }); + }); + return { + dataDir, + worker, + owner: () => holdIn(fixture.client(OWNER), Mcp, {}, "owner"), + /** A request to the routes as `principal` reaching p2 through `origin` (loopback by default). */ + http: (path: string, principal: P2Principal | undefined, origin = "http://127.0.0.1:8788", headers = {}) => { + access = { + principal, + origin: { origin, remote: !origin.startsWith("http://127.0.0.1"), port: 8788 }, + }; + return fetch(base + path, { redirect: "manual", headers }); + }, + }; +} + +const docs = (instances: readonly Instance[]) => instances.find((instance) => instance.server === "docs"); + +test("a 401 waits for a sign-in; the owner view's loopback sign-in connects with the tokens", async (t) => { + const oauth = await authorizationServer(t); + const host = await signInHost(t, oauth); + const owner = await host.owner(); + const waiting = docs(await until(owner.instances, (list) => docs(list)?.status === "sign-in")); + assert.equal(waiting?.auth, "oauth"); + const start = await owner.signIn({ server: "docs" }, TEST); + assert.equal(start.kind, "redirect"); + const [flow] = await until(owner.signIns, (list) => list.length === 1); + assert.equal(flow?.via, "loopback"); + assert.equal(flow?.status, "waiting"); + // The browser goes to the authorization server, which redirects to the loopback callback on this machine. + const back = await redirectOf(start.kind === "redirect" ? start.url : ""); + assert.equal(back.hostname, "127.0.0.1"); + assert.equal(back.pathname, "/callback"); + const page = await fetch(back); + assert.equal(page.status, 200); + assert.match(await page.text(), /Signed in to docs/); + const connected = docs(await until(owner.instances, (list) => docs(list)?.status === "connected")); + assert.deepEqual( + connected?.tools.map((tool) => tool.name), + ["whoami"], + ); + assert.deepEqual(await until(owner.signIns, (list) => list.length === 0), []); + // The credentials are private to the owner of the data directory. + assert.equal(statSync(join(host.dataDir, "mcp-auth.json")).mode & 0o777, 0o600); + assert.match(readFileSync(join(host.dataDir, "mcp-auth.json"), "utf8"), /access-1/); +}); + +test("the sign-in route: loopback for the device owner; the probe page, the probe and the callback route for a relayed owner", async (t) => { + const oauth = await authorizationServer(t); + const host = await signInHost(t, oauth); + const owner = await host.owner(); + await until(owner.instances, (list) => docs(list)?.status === "sign-in"); + // The device owner goes straight to the authorization server, returning to loopback. + const local = await host.http("/mcp/sign-in/docs", OWNER); + assert.equal(local.status, 302); + assert.match( + new URL(local.headers.get("location") ?? "").searchParams.get("redirect_uri") ?? "", + /^http:\/\/127\.0\.0\.1:\d+\/callback$/, + ); + // A relayed owner first gets the probe page, with a probe only its origin may read. + const probePage = await host.http("/mcp/sign-in/docs", RELAY_OWNER, BRAIN); + assert.equal(probePage.status, 200); + const html = await probePage.text(); + const probe = /fetch\("(http:\/\/127\.0\.0\.1:\d+\/[0-9a-f]+)"/.exec(html)?.[1]; + assert.ok(probe, "the page probes the loopback listener"); + assert.match( + probePage.headers.get("content-security-policy") ?? "", + new RegExp(`connect-src ${new URL(probe).origin}`), + ); + const answer = await fetch(probe, { headers: { origin: BRAIN } }); + assert.equal(answer.status, 204); + assert.equal(answer.headers.get("access-control-allow-origin"), BRAIN); + assert.equal((await fetch(probe, { headers: { origin: "https://elsewhere.example" } })).status, 404); + assert.equal((await fetch(`${new URL(probe).origin}/unknown`, { headers: { origin: BRAIN } })).status, 404); + // On another device the page continues with the callback route on the origin p2 is reached through. + const routed = await host.http("/mcp/sign-in/docs?via=callback", RELAY_OWNER, BRAIN); + assert.equal(routed.status, 302); + const back = await redirectOf(routed.headers.get("location") ?? ""); + assert.equal(back.origin + back.pathname, `${BRAIN}/mcp/callback`); + const [flow] = await until(owner.signIns, (list) => list[0]?.via === "route"); + assert.equal(flow?.status, "waiting"); + // The registration names p2 and the https callback. + assert.deepEqual(oauth.registrations.at(-1)?.redirect_uris, [`${BRAIN}/mcp/callback`]); + // Nobody but the owner completes it: a browser without p2's sign-in is told to paste instead. + const anonymous = await host.http(`/mcp/callback${back.search}`, undefined, BRAIN); + assert.equal(anonymous.status, 401); + assert.match(await anonymous.text(), /paste it in p2/); + const guest = await host.http(`/mcp/callback${back.search}`, GUEST, BRAIN); + assert.equal(guest.status, 403); + // A response of another sign-in is refused and leaves this one waiting. + const wrong = await host.http(`/mcp/callback?code=code-9&state=other`, RELAY_OWNER, BRAIN); + assert.equal(wrong.status, 400); + assert.equal((await until(owner.signIns, (list) => list.length === 1))[0]?.status, "waiting"); + const done = await host.http(`/mcp/callback${back.search}`, RELAY_OWNER, BRAIN); + assert.equal(done.status, 200); + assert.match(await done.text(), /Signed in to docs/); + await until(owner.instances, (list) => docs(list)?.status === "connected"); +}); + +test("pasting the final redirect URL completes a sign-in; a URL of another sign-in is refused", async (t) => { + const oauth = await authorizationServer(t); + const host = await signInHost(t, oauth); + const owner = await host.owner(); + await until(owner.instances, (list) => docs(list)?.status === "sign-in"); + const start = await owner.signIn({ server: "docs" }, TEST); + const back = await redirectOf(start.kind === "redirect" ? start.url : ""); + const [flow] = await until(owner.signIns, (list) => list.length === 1); + const other = new URL(back); + other.searchParams.set("state", "someone-else"); + await assert.rejects(owner.paste({ id: flow!.id, redirectUrl: other.href }, TEST), /different sign-in/); + assert.equal((await until(owner.signIns, (list) => list.length === 1))[0]?.status, "waiting"); + // The loopback page never loaded (the browser runs elsewhere): its address, pasted, completes the sign-in. + await owner.paste({ id: flow!.id, redirectUrl: back.href }, TEST); + await until(owner.instances, (list) => docs(list)?.status === "connected"); + await until(owner.signIns, (list) => list.length === 0); +}); + +test("refused tokens refresh without a browser; signing out forgets them and waits for a sign-in", async (t) => { + const oauth = await authorizationServer(t); + const host = await signInHost(t, oauth); + const owner = await host.owner(); + const waiting = docs(await until(owner.instances, (list) => docs(list)?.status === "sign-in")); + const start = await owner.signIn({ server: "docs" }, TEST); + await fetch(await redirectOf(start.kind === "redirect" ? start.url : "")); + await until(owner.instances, (list) => docs(list)?.status === "connected"); + oauth.expireAccessTokens(); + await owner.restart({ instance: waiting!.id }, TEST); + await until(owner.instances, (list) => docs(list)?.status === "connected"); + assert.deepEqual(oauth.grants, ["authorization_code", "refresh_token"]); + assert.equal(oauth.authorizations.length, 1); + // A refresh token a stored sign-in still has signs in at once, without the browser. + assert.deepEqual(await owner.signIn({ server: "docs" }, TEST), { kind: "authorized" }); + await owner.signOut({ server: "docs" }, TEST); + await until(owner.instances, (list) => docs(list)?.status === "sign-in"); + assert.doesNotMatch(readFileSync(join(host.dataDir, "mcp-auth.json"), "utf8"), /access-/); +}); + +test("guests and anonymous visitors get no sign-in route; unknown servers fail", async (t) => { + const oauth = await authorizationServer(t); + const host = await signInHost(t, oauth); + assert.equal((await host.http("/mcp/sign-in/docs", GUEST, BRAIN)).status, 403); + assert.equal((await host.http("/mcp/sign-in/docs", undefined, BRAIN)).status, 401); + const unknown = await host.http("/mcp/sign-in/nope", OWNER); + assert.equal(unknown.status, 502); + assert.match(await unknown.text(), /No MCP server "nope" signs in with OAuth/); + const owner = await host.owner(); + await assert.rejects(owner.signIn({ server: "nope" }, TEST), /No MCP server "nope" signs in with OAuth/); +}); diff --git a/apps/p2/test/p2/mcp-tools.test.ts b/apps/p2/test/p2/mcp-tools.test.ts index db582fbb6..e19383a1b 100644 --- a/apps/p2/test/p2/mcp-tools.test.ts +++ b/apps/p2/test/p2/mcp-tools.test.ts @@ -27,7 +27,7 @@ const names = (tools: readonly ServerTool[]) => /** What the worker offers for `tools`: their servers, reached from code mode unless `reach` says otherwise. */ function offer(tools: readonly ServerTool[], reach: McpReach = "codemode") { - return { servers: [...new Set(tools.map((entry) => entry.server))].sort(), tools, reach }; + return { servers: [...new Set(tools.map((entry) => entry.server))].sort(), signIn: [], tools, reach }; } test("tool names: pi's, and every tool of a colliding name gets the hash suffix whatever the order", () => { @@ -103,10 +103,19 @@ test("the mcp_servers section names each server's namespace and how its tools ar "MCP servers whose tools are not declared to you. Load the tools of `tool_search` servers with `tool_search`.\n- mcp__app (tool_search)\n- mcp__my_docs (tool_search)", ); // A server still connecting has no tools yet, and is listed; without servers there is no section. - const starting = createMcpExtension({ servers: ["slow"], tools: [], reach: "codemode" }, async () => ({ + const starting = createMcpExtension({ servers: ["slow"], signIn: [], tools: [], reach: "codemode" }, async () => ({ content: [], })); assert.match(String(await render(starting)), /- mcp__slow \(codemode\)$/); + // A server waiting for a sign-in is listed with the link the user signs in with. + const locked = createMcpExtension( + { servers: ["my-docs"], signIn: ["my-docs"], tools: [], reach: "codemode" }, + async () => ({ content: [] }), + ); + assert.match( + String(await render(locked)), + /- mcp__my_docs \(codemode; needs a sign-in, no tools yet: the user signs in from the MCP panel in p2's sidebar or at \/mcp\/sign-in\/my-docs on p2\)$/, + ); assert.deepEqual(createMcpExtension(offer([]), async () => ({ content: [] })).sections, []); }); diff --git a/apps/p2/test/p2/mcp.test.ts b/apps/p2/test/p2/mcp.test.ts index 9291a446f..544da1cb3 100644 --- a/apps/p2/test/p2/mcp.test.ts +++ b/apps/p2/test/p2/mcp.test.ts @@ -584,9 +584,16 @@ test("placement: environments not in envs.json fail; HTTP servers connect with h // One shared docs connection for both sessions, one unshared one per session. assert.equal(status(all, "docs").length, 1); assert.equal([...sessions].filter((session) => session.startsWith("/mine:")).length, 2); - assert.equal(host.starts().length, 0); + // Only s2's own stdio server runs on this machine; s1's are on environments. + await until(s2.servers, isUp("here")); + assert.equal(host.starts().length, 1); assert.ok(authorizations.includes("Bearer secret-token")); assert.equal(textOf(await s1.call({ location: remote, server: "docs", tool: "lookup", args: {} }, TEST)), "found"); + // A call of a server waiting for a sign-in tells the model where the user signs in. + await assert.rejects( + s1.call({ location: remote, server: "locked", tool: "lookup", args: {} }, TEST), + /needs a sign-in: ask the user to sign in from the MCP panel in p2's sidebar \(or at \/mcp\/sign-in\/locked on p2\)/, + ); }); test("a project's mcp.json: its servers run at its location for its session, it turns the owner's on and off there, refusals are reported", async (t) => {