mirror of
https://github.com/badlogic/pi-mono.git
synced 2026-11-05 23:37:25 +00:00
Anthropic message threads (#11545)
Anthropic can keep a conversation on the server (message threads beta). A request can then continue from the previous reply and send only the new messages, instead of the whole history every turn. pi uses this when a session id is set and caching is on. It still builds the full request, compares it with the previous one, and only sends the new part if nothing earlier changed. - Default: on only for the anthropic provider on api.anthropic.com. Gateways that drop unknown fields would lose the conversation. Override with compat.supportsMessageThreads. - If the server rejects a continuation, pi resends the full request as a new thread. Other errors turn threads off for the session. - New option StreamOptions.disableContinuation. The cache warmer sets it so warm requests don't replace the session's thread. - cleanupSessionResources(sessionId) frees a session's thread state.
This commit is contained in:
parent
3551ec07eb
commit
a758f3ae4c
14 changed files with 885 additions and 22 deletions
|
|
@ -52,6 +52,7 @@ type ProxySerializableStreamOptions = Pick<
|
|||
| "reasoning"
|
||||
| "cacheRetention"
|
||||
| "sessionId"
|
||||
| "disableContinuation"
|
||||
| "headers"
|
||||
| "metadata"
|
||||
| "transport"
|
||||
|
|
@ -95,6 +96,7 @@ function buildProxyRequestOptions(options: ProxyStreamOptions): ProxySerializabl
|
|||
reasoning: options.reasoning,
|
||||
cacheRetention: options.cacheRetention,
|
||||
sessionId: options.sessionId,
|
||||
disableContinuation: options.disableContinuation,
|
||||
headers: options.headers,
|
||||
metadata: options.metadata,
|
||||
transport: options.transport,
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue