Commit graph

7423 commits

Author SHA1 Message Date
Armin Ronacher
7dc38ff0e9 feat(p2): MCP servers on the host, on environments and from projects
A manager on the p2 server owns every MCP connection and process. Session
workers hold one view each and lease the locations their conversations
work at; the manager runs the owner's mcp.json servers there (stdio per
session and location, or shared; HTTP shared by default) and offers their
tools as deferred Durable tools found through tool_search.

Stdio servers on an environment start through its pi-env daemon, one
connection per environment shared by its instances, through cmd.exe on
Windows. A project's .pi/mcp.json, read by the worker at each location,
may add stdio servers that run there and switch the owner's servers on or
off; the manager refuses everything else. A lease gets servers only on
environments its session works on.
2026-11-03 16:48:38 +01:00
github-actions[bot]
a57438e918 chore: update Nix model catalog pin 2026-11-03 13:56:54 +00:00
Mario Zechner
2501150892 docs(p2): p2 is an experimental prototype that will be gone soon 2026-11-03 13:56:27 +00:00
Cristina Poncela Cubeiro
d71c66e05b fix(nix): pack pi-schema for the install lock
pi-schema became a coding-agent runtime dependency, but the Nix package
neither packed it nor overrode its install-lock entry, which has no
integrity hash, so evaluation failed with "attribute 'integrity' missing".
2026-11-03 14:43:45 +01:00
Mario Zechner
05dc8d239d feat: add apps/p2 (imported from pi-private at 7dadff697)
p2 (Pi Mobile on Chord v2) moves from pi-private into pi as one commit. Root scripts that loop all workspaces
use --if-present; npm test, ./test.sh and the release script cover packages/* only (p2: cd apps/p2 && npm test);
sync-versions also updates apps' workspace pins.
2026-11-03 13:39:06 +00:00
robopipi[bot]
f409847473
fix(mcp): cancel SSE response streams that fail the message-size limit (#11735)
When an SSE event went over `maxMessageBytes`, `consumeSseStream()`
threw the size-limit error, but its `finally` only released the reader
lock. Releasing the lock does not cancel the response body. If the
server kept the response open, for example with heartbeats, the failed
request held its HTTP connection until the server ended it or the whole
client was closed. The same applies to the GET notification stream,
which uses the same function.

On error, `consumeSseStream()` now cancels the reader before it rethrows
and releases the lock. A failure in `cancel()` is ignored, so the caller
still gets the original error.

Testing:
- New test in `packages/mcp/test/streamable-http.test.ts`: it feeds an
oversized event through a stream that stays open and checks that the
stream's `cancel` callback runs. It fails without the change and passes
with it.
- Full mcp package tests: 48/48 pass. `npm run check` is clean.
- Reproduction over loopback: a local `node:http` server sends a
512-byte event, then a heartbeat every 20 ms, and the client reads it
through `fetch` with `maxEventBytes: 128`, three times.
- Before: none of the three server responses closed, and the server made
28 more heartbeat writes.
  - After: all three closed, with no further writes.

Fixes #11729

Requested by @badlogic

Co-authored-by: triagent <triagent@localhost>
2026-11-03 14:22:16 +01:00
Armin Ronacher
797e122cb1 fix(mcp): refresh with the app's redirect URI, cancel sign-in during the lock wait
Tokens from a Client ID Metadata Document sign-in through an https
callback route were refreshed with the loopback redirect URI, which the
documents do not list, so every expiry needed a browser sign-in.
createMcpAuthProvider takes the app's redirectUrl and startMcpSignIn
passes its own.

startMcpSignIn's wait for the refresh lock now ends when the signal
aborts. Only McpRefreshLockBusyError (coding-agent maps proper-lockfile's
ELOCKED to it) falls back to the browser step; other lock errors fail
the sign-in.
2026-11-03 14:17:21 +01:00
Armin Ronacher
76c040a2ad feat(mcp): shared tool naming, schema, output limit and config helpers
createMcpToolName, mcpNamespace, normalizeMcpInputSchema,
limitMcpContent (with a caller-provided saver) and validation of the
standard mcpServers shape move from coding-agent into pi-mcp, so other
applications name and limit MCP tools exactly like pi. Applications
validate their own fields on the tagged result; coding-agent keeps
exposure, auth.provider and its Client ID Metadata Document rule.
2026-11-03 14:17:21 +01:00
Armin Ronacher
bb266bf359 feat(mcp): MCP OAuth sign-in in two halves, moved from coding-agent
pi-mcp now has startMcpSignIn and completeMcpSignIn, so a server app can
start a sign-in in one request and complete it in another (a callback
route, a pasted redirect URL, or a loopback server via
listenForLoopbackSignIn/waitForLoopbackSignIn), plus createMcpAuthProvider.
Application specifics (client name, Client ID Metadata Document base URL
and listed redirect URIs) are options. coding-agent's signInMcpServer is a
thin wrapper over them.

authorizeMcp gains refreshOnly: a failed refresh no longer registers a
client or replaces a pending sign-in's PKCE verifier and state. Sign-in
refreshes with the stored client before dropping it for another redirect
URI.
2026-11-03 14:17:21 +01:00
Armin Ronacher
032656b6c6 fix(env): kill what a process program left running when it exits
When a program started by the process operation exits on its own, the
daemon now ends its process group (Unix) or its own job object (Windows)
before reaping it, so a wrapper like npx does not leave its server
running. The pid leaves the tracked groups before it is reaped, so a
racing close cannot signal a reused pid.
2026-11-03 14:17:21 +01:00
Armin Ronacher
c158fa2ee4 feat(mcp): PipeTransport, the stdio transport over a program started by the caller
StdioTransport and PipeTransport share the newline-delimited framing and
the stderr tail (transports/lines.ts). PipeTransport accepts pi-env's
RemoteProcess as its McpPipe, so p2 can run stdio MCP servers on
execution environments through the daemon's process operation.
2026-11-03 14:17:21 +01:00
David Brailovsky
a758f3ae4c
Anthropic message threads (#11545)
Anthropic can keep a conversation on the server (message threads beta).
A request can then continue from the previous reply and send only the
new messages, instead of the whole history every turn.

pi uses this when a session id is set and caching is on. It still builds
the full request, compares it with the previous one, and only sends the
new part if nothing earlier changed.

- Default: on only for the anthropic provider on api.anthropic.com.
  Gateways that drop unknown fields would lose the conversation.
  Override with compat.supportsMessageThreads.
- If the server rejects a continuation, pi resends the full request
  as a new thread. Other errors turn threads off for the session.
- New option StreamOptions.disableContinuation. The cache warmer sets
  it so warm requests don't replace the session's thread.
- cleanupSessionResources(sessionId) frees a session's thread state.
2026-11-03 14:07:57 +01:00
Mario Zechner
3551ec07eb docs(chord): p2 on Chord v2 and the v1 deletion done 2026-11-03 12:15:48 +00:00
Mario Zechner
ed4f488401 feat(chord): remove Chord v1 (/old, /old/node), its tests and docs
p2 runs on Chord v2; apps still on v1 pin @earendil-works/chord@1.5.0.
2026-11-03 12:11:22 +00:00
robopipi[bot]
90b42bca6e
fix(ai): remove Codex SSE retry abort listener after backoff (#11725)
The Codex Responses provider's local `sleep()` helper attached an
anonymous `abort` listener to the caller's signal and never removed it
when the timer fired. The SSE retry loop calls this helper before each
retry, so every completed backoff left one listener on the signal. Three
429 responses followed by a success left three listeners behind.

The helper now uses a named `onAbort` handler, removes it when the timer
fires, and registers it with `{ once: true }`. It still rejects with
`Error("Request was aborted")`, which the retry loop's catch block
depends on.

### Testing
- New regression test in `packages/ai/test/openai-codex-stream.test.ts`.
A stub fetch returns 429 with `retry-after: 0` three times and then a
completed SSE stream, using `maxRetries: 3` and the caller's own
`AbortController`. The test asserts that `getEventListeners(signal,
"abort")` is empty afterwards. It fails without the change ("got 3") and
passes with it.
- `node ../../node_modules/vitest/dist/cli.js --run
test/openai-codex-stream.test.ts`: 41/41 pass.
- A standalone script against the built package printed `listeners 3`
before the change and `listeners 0` after.
- `npm run check`: biome and the repo scripts pass. `tsc` fails, but
only in files this change does not touch: generated model ids in
`packages/ai` tests, `github-copilot.ts`, and missing bench dependencies
in `packages/schema`. I did not verify whether those failures also
happen on a clean checkout.

Fixes #11708

Requested by @badlogic

Co-authored-by: triagent <triagent@localhost>
2026-11-03 09:32:52 +01:00
Mario Zechner
d7704516e0 fix(chord): int() table keys type as numbers again (a TypeBox leftover) 2026-11-03 08:32:07 +00:00
robopipi[bot]
42849bb840
fix(mcp): clean up server children on stdio close after the server exited (#11724)
On POSIX, stdio MCP servers run in their own process group so that
closing the transport can kill the whole tree. A server's main process
can exit while a child it started is still running and still holds the
inherited stdout/stderr. In that case `StdioTransport.close()` returned
early, because the direct child already had an exit code. It skipped the
process-group SIGTERM/SIGKILL cleanup, so the child kept running. The
transport's close event also never fired.

A second problem: the group was removed from the set the host exit hook
kills as soon as the server process exited. So the leftover child also
survived pi exiting.

This change:
- On POSIX, skips the early return when the server already exited, so
the existing stdin-close → SIGTERM → SIGKILL group cleanup runs and
`close()` resolves once the pipes close. Windows keeps the early return.
- Keeps the process group tracked until the child's `close` event (all
pipes closed) instead of `exit`.

Testing:
- Added a test in `packages/mcp/test/stdio.test.ts` with a new fixture,
`exiting-parent-server.mjs`. The fixture starts a helper that inherits
stdout/stderr, then exits. The test checks that after `close()` the
helper is gone and the close event fired exactly once. Without the fix
it fails: the helper is still alive.
- `packages/mcp` tests pass (47 tests). `npm run check` passes.
- Manual reproduction with a shell-script server that backgrounds a
long-running process and exits. Before: `close()` returned in 0 ms, the
process stayed alive, no close event. After: it is killed and the close
event fires.

Fixes #11721

Requested by @badlogic

Co-authored-by: triagent <triagent@localhost>
2026-11-03 09:31:10 +01:00
Mario Zechner
24d3993b20 docs(chord): the Chord v2 tutorial, a todo app built with the app kit (example 61) 2026-11-03 08:30:05 +00:00
Mario Zechner
0c7c741988 fix(chord): channel test attaches its expectation before the deadline can pass 2026-11-03 07:29:13 +00:00
Mario Zechner
5bd014b1c7 feat(chord): mistakes that wait are reported at once
- get, hold and registry on a host that has not loaded throw not_ready at once.
- A registry with contributions and no provider is reported (registry_undeclared) at publication.
- A request waiting unbound is reported once after 5 s (unbound_long; one clock per host), and keeps waiting.
- An invalid plugin name makes load and reload throw before any change.
- A transaction cancelled by the end of its context is reported (transaction_cancelled) besides rejecting.
- Docs: who a reference acts as, no per-link call limit, await what you start in a root or a handler.
2026-11-03 07:19:52 +00:00
Mario Zechner
51bb8f5c06 fix(chord): createWorkers' route resolve type-checks in every program 2026-11-03 07:19:24 +00:00
Mario Zechner
8ff2cec028 docs(chord): p2 switch goes to the phone once green 2026-11-03 06:59:12 +00:00
Mario Zechner
5d74f40de9 docs(chord): changes cancelled by a seal are reported 2026-11-03 06:49:01 +00:00
Mario Zechner
fc7d91f4e8 docs(chord): mistakes that wait are reported at once; no per-link call limit 2026-11-03 06:39:27 +00:00
Armin Ronacher
9c9ce8547e feat(env): run programs with plain pipes through the daemon
The daemon's process operation runs a long-lived program with piped stdin, stdout and stderr, for programs that speak a protocol on stdin and stdout such as MCP servers. processWrite and processCloseStdin feed stdin in arrival order; output arrives as raw bytes per stream; cancelling or a lost connection kills the process group. hello reports it in features, and openProcess() is the client.

On Windows, readers and writers of child pipes now run their own overlapped operations and look at stop every 100 ms. The standard library opens these pipes for overlapped I/O, so CancelSynchronousIo never reached a blocked read: exec's readers stayed blocked while a descendant held a pipe open.

Spawning, reaping, argv parsing and target lookup are shared by exec, pty and process.
2026-11-03 00:53:07 +01:00
David Brailovsky
00967748dc feat: issue triage UI shows untriaged issues and agent conversations 2026-11-02 23:25:24 +00:00
Mario Zechner
db8307ec95 docs: no runtime checks of values TypeScript already guarantees 2026-11-02 23:25:24 +00:00
David Brailovsky
c856eb0b6e fix(chord): mark plugin-imports test fixture private so preview publishing skips it
Some checks are pending
CI / build-check (push) Waiting to run
CI / test (coding-agent) (push) Waiting to run
CI / test (durable) (push) Waiting to run
CI / test (other) (push) Waiting to run
CI / mcp-conformance (push) Waiting to run
Environments / daemons (push) Waiting to run
Environments / check (push) Blocked by required conditions
Environments / release-daemons (darwin-arm64, macos-latest) (push) Blocked by required conditions
Environments / release-daemons (darwin-x64, macos-latest) (push) Blocked by required conditions
Environments / release-daemons (linux-arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Environments / release-daemons (linux-x64, ubuntu-latest) (push) Blocked by required conditions
Environments / release-daemons (windows-arm64, windows-11-arm) (push) Blocked by required conditions
Environments / release-daemons (windows-x64, windows-latest) (push) Blocked by required conditions
Environments / tests (macos-latest) (push) Waiting to run
Environments / tests (ubuntu-24.04-arm) (push) Waiting to run
Environments / tests (ubuntu-latest) (push) Waiting to run
Environments / tests (windows-11-arm) (push) Waiting to run
Environments / tests (windows-latest) (push) Waiting to run
2026-11-03 00:04:15 +01:00
Mario Zechner
3c7a9ca0e7 feat(chord): example 59-chat keeps its messages in a table 2026-11-02 22:47:49 +00:00
Mario Zechner
d5c4b80ebf docs(chord): p2 moves to Chord v2, then v1 is deleted 2026-11-02 22:38:23 +00:00
Mario Zechner
e9c0ff4d84 feat(chord): Chord v2 is the package's root
src/next is src (the main export), src/next-node is src/node with short subpaths (/http, /workers, /plugins,
/plugins/runtime, /app, /surfaces, /web, /react, /tui, /tui/launcher); the foundations get /json and /watch;
Chord v1 moves to src/old (/old, /old/node) until p2 runs on v2; tests and docs move likewise (test/, docs/,
test/old, docs/old). Durable and env import the foundations from their subpaths.
2026-11-02 22:36:55 +00:00
David Brailovsky
0a4fad5489
fix(ai): leave room for OpenRouter's input overcount when clamping max tokens (#11712)
OpenRouter checks input + max_tokens against the context window before
routing, using its own tokenizer. For prose it counted up to 12% more
input tokens than the provider. pi left only 4096 tokens spare, so
models whose output limit is close to the window (e.g. DeepSeek flash)
were rejected with a 400, which pi then treated as a context overflow
and compacted.

The default output limit now leaves 15% of the estimated input unused
(at least 4096). When that would leave less than 32768 tokens of output,
only the fixed 4096 margin is kept and output is capped at 32768.

fixes #10497
2026-11-02 22:43:27 +01:00
Mario Zechner
1d81a69727 docs(chord): Chord v2 becomes the package's root 2026-11-02 21:40:16 +00:00
Mario Zechner
5bc4145165 feat(chord): Chord v2 web and terminal surfaces (slice 11)
- Web: base bundles built from the app's entry at start (content-hashed, served
  forever), plugin scripts that register themselves and one combined file,
  WebScripts/TuiScripts showing the current set, startPage with the connection
  loop, in-place plugin reloads and a reload on a new base bundle.
- React bindings: HostProvider, useHold, useWatch, useRegistry, useRefState.
- Terminal: startTerminal, a launcher that signs in with a token, a token file or
  device pairing and always runs the app's terminal code in a sandboxed child
  process (pipes, empty environment, --permission granting nothing, no SQLite,
  a builtin allowlist and a frozen process inside); output filter, confirmed
  copy/open/attach actions; startTerminalRuntime and TuiScope inside.
- Themes: the Themes registry, ThemeChoice per principal, CSS variables on the
  page, terminal colours by role; CSS modules in plugin builds.
- App kit: startServer gains web and tui; startApp renamed startServer,
  exportAddress/importAddress renamed exportData/importData and export no longer
  stops the worker; ephemeral sessions end with their client's last link.
- Examples 56-web, 57-tui, 58-themes, 59-chat.
2026-11-02 21:31:59 +00:00
Mario Zechner
9c248717da docs(chord): messages and languages in the app kit decided 2026-11-02 21:28:52 +00:00
Petr Houška
806cdd68e3
fix(coding-agent): only report packages as up to date when pi update checked them (#11711)
## Problem

#11707 prints `Packages are already up to date` whenever no package
changed. In two cases that claim is false.

**1. npm left the old version installed.** Extension updates install
`@latest` or the configured range. If npm resolves stale registry
metadata, it reinstalls the old version and exits 0. The existing check
warns, then the summary contradicts it:

```
Updating npm:foo...
npm:foo is still 1.2.0 (registry reports 1.3.0); npm registry metadata may be stale, try again in a few minutes
Packages are already up to date
```

**2. Offline mode.** With `--offline` or `PI_OFFLINE=1`,
`updateConfiguredSources()` returns before checking anything, and the
CLI still prints `Packages are already up to date`.

## Change

The CLI only prints "already up to date" when the update ran and printed
no warning:

| Situation | Output |
|---|---|
| Packages changed | `Updated N packages:` and the list (unchanged) |
| Nothing changed, no warnings | `Packages are already up to date`
(unchanged) |
| Nothing changed, a warning was printed | `No packages were updated,
see the warnings above` (or `<source> was not updated, ...`) |
| Offline mode | `Skipped package updates (offline mode)` |

The CLI counts `progress` update events, which are always warnings, and
does not try to map them to packages. That is deliberate: the git
warning about untracked files that could not be cleaned is printed even
though the commit did change, so a warning does not mean "this package
was not updated".

In offline mode, `update()` still runs, so `pi update <unknown source>`
still fails with "No matching package". `isOfflineModeEnabled()` is now
exported from `core/package-manager.ts` (not from the package index) so
the CLI uses the same check.

The changelog entry from #11707 is adjusted, since it is unreleased.

## Tests

`test/package-command-paths.test.ts`, both through `main()`:

- npm `view` reports 1.3.0 but `install` leaves 1.2.0: the
stale-metadata warning is printed, then `No packages were updated, see
the warnings above`, and no "up to date".
- `PI_OFFLINE=1`: `Skipped package updates (offline mode)`, no "up to
date".

Both fail without the fix. `cycles project package overrides in config
local mode`, `prints a pnpm metadata hint when self-update fails` and
`should expand ~ in exclusion and exact patterns` fail locally on
Windows, on `origin/main` as well.
2026-11-02 21:49:07 +01:00
Mario Zechner
65c85bda2c docs(chord): slice 11 spec, the sandbox is a child process 2026-11-02 20:47:13 +00:00
Mario Zechner
10cf3264fb docs(chord): design §23, the terminal's sandbox is a child process 2026-11-02 20:26:54 +00:00
Mario Zechner
128912d246 docs(chord): the terminal's sandbox is a child process 2026-11-02 20:26:39 +00:00
Petr Houška
342514bdf8
feat(coding-agent): list updated packages with version changes in pi update (#11707)
## Problem

`pi update`, `pi update --all` and `pi update --extensions` print only
`Updating <source>...` progress lines and a final `Updated packages`.
Users cannot see which extensions changed or to which version. Several
outdated npm packages in the same scope are installed as one batch, so
the progress output does not even name them.

## Change

`pi update` now prints a summary of what actually changed:

```
Updated 2 packages:
  npm:foo 1.2.0 -> 1.3.0
  git:github.com/user/bar (project) abc1234 -> def5678
```

When nothing changed it prints `Packages are already up to date` (or
`<source> is already up to date`).

- npm: compares the installed `package.json` version before and after
the batch install. The "before" version comes from the existing
`shouldUpdateNpmSource` check, so this adds no extra registry calls.
- git: compares `git rev-parse HEAD` before and after `updateGit`.
Commits are shortened to 7 characters.
- Packages whose version or commit did not change are omitted, including
npm installs that hit stale registry metadata (those already get the
existing warning).

## API

`PackageManager.update()` still returns `Promise<void>`.
`ProgressEvent.type` gains a `"result"` member. Each changed package is
reported through the existing progress callback as:

```ts
{ type: "result", action: "update", source: "npm:foo", message: "npm:foo 1.2.0 -> 1.3.0" }
```

The CLI collects these messages to print the summary. A separate type
keeps the existing contract that each `start` is followed by exactly one
`complete` or `error` for the same source. Reusing `complete` would have
produced a second `complete` for git packages and a `complete` without a
`start` for batched npm packages. The only compatibility cost is for
TypeScript consumers with an exhaustive `switch` on `event.type`.

## Tests

- `test/package-manager.test.ts`: npm update emits `start`, `complete`,
then a `result` with `npm:example@^1.0.0 (project) 1.0.0 -> 1.2.0`; an
up-to-date package emits no result.
- `test/git-update.test.ts`: git update emits a result with the old and
new commit; an up-to-date clone emits no result.

`should expand ~ in exclusion and exact patterns` fails locally on
Windows. It tests path resolution and is unrelated to this change.
2026-11-02 21:21:23 +01:00
Mario Zechner
15f00a786a docs(chord): example 60, session sync played with the store's outbox, applyBatch and archives 2026-11-02 20:13:38 +00:00
Mario Zechner
642e30b689 docs(chord): p3 meeting prep with pictures, sync example, branches 2026-11-02 17:18:31 +00:00
Mario Zechner
1746cfd515 docs(chord): i18n proposal, t.message and one file per host 2026-11-02 17:09:59 +00:00
Mario Zechner
48f9f1b423 docs(chord): i18n proposal, messages as object state 2026-11-02 17:03:46 +00:00
Mario Zechner
7dffeaf521 docs(chord): i18n proposal 2026-11-02 16:58:24 +00:00
Mario Zechner
1f8604f84f docs(chord): sync applies batches with the host stopped 2026-11-02 16:47:09 +00:00
Mario Zechner
95485d32f1 docs(chord): startServer, exportData without stopping; p3 notes on subagent context and tree 2026-11-02 16:15:54 +00:00
Mario Zechner
e64f313bd6 docs(chord): p3 feature set and work packages, meeting prep 2026-11-02 15:58:52 +00:00
Mario Zechner
cf74b67653 docs(chord): p3 meeting summary, session sync and index 2026-11-02 15:49:56 +00:00
Mario Zechner
d6af4d18e5 docs(chord): looks and themes decided; slice 11 spec gains themes 2026-11-02 15:23:22 +00:00