diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..89f9ac0 --- /dev/null +++ b/.gitignore @@ -0,0 +1 @@ +out/ diff --git a/README.md b/README.md index ac8d28d..f3b2ebf 100644 --- a/README.md +++ b/README.md @@ -1,105 +1,129 @@ # Speckl -**Spec-first development with dark provenance.** Write what your system should do. Prove it. Ship with full traceability. +**A judgement transport utility for AI-generated specifications.** Fulfills NIST SA-11 (human-in-the-loop for automated outputs). -Speckl is a multi-agent software development system where the primary artifact is the *specification*, not the code. Every behavior traces back to its source of intent — automatically. +Speckl is a specification language (SpeckDL) with a compiler that produces runnable software — TypeScript and provenance artifacts (PROV-O, CycloneDX, SPDX) — from a single source of truth. Write what your system should do. Prove it. Ship it. -## The Chain of Custody for Intent +## The Core Insight -Agents generate code. Humans own specifications. The chain of custody for intent stays intact. +AI generates code at machine speed. Humans own specifications. **SpeckDL is the shared space where both can work.** -That's the old framing. Here's the better one: +- **Human-judgeable** — SpeckDL is declarative enough that a person can inspect a spec and say "yes, this captures my intent." The compiled output is opaque; the spec is not. +- **AI-writable** — SpeckDL's constrained grammar and declarative semantics are tractable for LLMs to produce and reason about. Agents draft. Humans review. Both operate on the same semantic layer. +- **Compiled, not generated** — The SpeckDL compiler is deterministic. No LLMs in the compilation path. Same spec, same output — always. -**SpeckDL is the shared space that both humans and AI can work in.** +## What Speckl Does Now -- **Human-judgeable** — SpeckDL is declarative enough that a person can inspect a spec and say "yes, this captures my intent." The compiled Wasm is opaque; the spec is not. That's the audit trail. Intent is encoded in the spec *because* it is verifiable by a person. -- **AI-writable** — SpeckDL's constrained grammar and declarative semantics make it tractable for LLMs to produce and reason about. Agents can draft, iterate, and refactor specs at machine speed — in the same language the human reviews. - -The two properties together are symmetric: humans and agents collaborate in the same medium. The human provides direction and judgment. The agent provides speed and iteration. Both operate on the same semantic layer. - -SpeckDL compiles directly to **Wasm** through a deterministic pipeline. The compiler isn't just a code generator — it's the enforcement point that guarantees what the human signed off on is what gets executed. If it compiles to valid Wasm, the binary is faithful to the spec by construction. - -## Dark Provenance - -Every function, every branch, every side effect in your system is linked to the specification that motivated it. The provenance graph accumulates behind the scenes — invisible until you need it, but always complete. When an auditor asks "why does this code exist?", you don't search Jira. You query the graph. - -This isn't traditional traceability (forward-chaining from documents to code). It's the reverse: every runtime behavior backward-traces to its source specification. Specifications are small and stable. Code is derived. The audit surface stays tractable no matter how much code the system contains. - -## SpeckDL - -SpeckDL is the specification language at the core of Speckl. It defines *what* a system does, not *how*: +Speckl is a **working compiler** that turns formal specifications into multiple target representations: ``` -speck RetryHandler { - input: { operation: Operation, maxRetries: Nat } - output: Result - constraint: maxRetries <= 5 - constraint: backoff(delay(n)) == delay(n-1) * 2 - constraint: totalDelay < 30s - verify: always(implies(attempt > 1, delay > 0)) +.speckdl spec ──→ TypeScript ← typed, importable classes + ├── PROV-O (RDF) ← W3C provenance standard + ├── CycloneDX SBOM ← OWASP software bill of materials + └── SPDX SBOM ← Linux Foundation SBOM standard +``` + +### State Machine Compilation (inspired by TLA+) + +```speckdl +spec ToggleSwitch { + state: Status = Off + enum Status = On | Off + + init: { + Status = Off + } + + action FlipSwitch { + require: true + if Status == On { + Status = Off + } else { + Status = On + } + } } ``` -SpeckDL compiles through a deterministic pipeline directly to Wasm: +This compiles to a runnable TypeScript class with verified state transitions + guards. -![SpeckDL compilation pipeline](docs/diagrams/speckdl-pipeline.svg) +### Provenance Embedded at Compile Time -No LLMs in the compilation path. No non-determinism. Same input, same output — every time. +Every compilation embeds a complete provenance record — PROV-O RDF traces the spec → code derivation, CycloneDX and SPDX SBOMs capture the software supply chain. When an auditor asks "where did this behavior come from?", you query the graph, not Jira. -## Architecture +## Real-World Examples -![Architecture diagram](docs/diagrams/architecture.svg) +The compiler has been validated against: -- **Orchestrator** — client entry point, task decomposition -- **Team Leads** — manage specialized agents, elastic scaling -- **SMEs** — hyper-specialized single-task agents (verify, generate, document) -- **TillDone** — tasks complete when provable conditions are met, not when someone moves a card - -Teams self-replicate under load. The harness scales dynamically without human intervention. +| Example | Source | Status | +|---------|--------|--------| +| **ToggleSwitch** | SpeckDL tutorial | ✅ All artifacts | +| **AccountLedger** | Banking ledger with debit/credit | ✅ All artifacts | +| **TwoPhaseCommit** | TLA+ port | ✅ All artifacts | +| **Paxos** | TLA+ port | ✅ All artifacts | +| **Raft** | TLA+ port | ✅ All artifacts | +| **TigerBeetleLedger** | Naive TigerBeetle core | ✅ All artifacts (3 TS type errors in progress) | ## Getting Started -1. **Read the whitepaper:** [docs/whitepapers/dark-provenance.md](docs/whitepapers/dark-provenance.md) — the core methodology -2. **Read the strategy:** [docs/strategy.md](docs/strategy.md) — how Speckl grows from standard → consulting → SaaS → compliance -3. **Explore the SpeckDL spec:** [speckdl/SPEC.md](speckdl/SPEC.md) — language definition and examples -4. **Run the reference agent:** [reference-agent/](reference-agent/) — working tiered agent implementation - ```bash -# Clone the repo git clone https://os.scoble.me/forgejo/sscoble/speckl -cd speckl - -# Run the reference agent -cd reference-agent +cd speckl/compiler npm install -npm run dev + +# Compile an example +node dist/index.js ../examples/ToggleSwitch.speck -o out/switch +ls out/switch/ +# ToggleSwitch.ts ToggleSwitch.prov-o.jsonld +# ToggleSwitch.cyclonedx.json ToggleSwitch.spdx.json ``` -## Roadmap +### Key Documents -Speckl follows a phased strategy — the open standard fuels everything else: +- **[Whitepaper v2](docs/whitepaper-v2.md)** — Complete: 8 sections, ~19,000 words. Covers the spec-code gap, SpeckDL language, compiler architecture, embedded provenance, consensus protocols, compliance applications, related work, and roadmap. +- **[Speckl by Example](docs/speckl-by-example.md)** — Guided tutorial from ToggleSwitch through Raft, with TLA+ comparison table. +- **[Distribution Strategy](docs/distribution-strategy.md)** — Prioritized roadmap: Show HN → arXiv → blog → consulting pipeline. Conference landscape for 2026-2027. +- **[Strategy Ladder](docs/strategy.md)** — Open standard → Consulting → Developer Tool/SaaS → Compliance API. +- **[SpeckDL Spec](speckdl/SPEC.md)** — Language definition v0.2. +- **[State Machine Extension](speckdl/STATE-MACHINE-SPEC.md)** — `state`/`init`/`action` keywords, Set/Map/List types. -| Phase | Focus | Timeline | -|-------|-------|----------| -| **A — Open Standard** | SpeckDL language, Dark Provenance methodology, reference implementations | Now → 3mo | -| **B — Consulting** | Hands-on engagements with regulated-industry teams, validating the methodology | Now → 3mo (funds A) | -| **C — Developer Tool (SaaS)** | Self-serve platform: spec editor, live Z3 verification, provenance graph browser | 3 → 9mo | -| **D — Compliance API** | Enterprise provenance engine: audit reports, traceability matrices, compliance packs | 9 → 18mo | +## Strategy Ladder -The standard comes first. Adoption creates the market. The products follow. +| Phase | Focus | Status | +|-------|-------|--------| +| **A — Open Standard** | SpeckDL language, compiler, whitepaper, examples | ✅ **Complete** (compiler working, whitepaper done) | +| **B — Consulting** | Hands-on engagements with regulated-industry teams | 📋 One-pager drafted, awaiting review | +| **C — Developer Tool/SaaS** | Self-serve spec validation, provenance browser | 🔮 Future | +| **D — Compliance API** | Enterprise audit reports, traceability matrices | 🔮 Future | + +## Validation Status + +- **Compiler:** 41/41 tests passing (TypeScript codegen) +- **TypeScript:** Type-safe compilation from specs +- **TLA+ Trilogy:** TwoPhaseCommit, Paxos, Raft all compile +- **Compiler Self-Spec:** `speckl-compile.speckdl` defines the Speckl compiler in SpeckDL — generates all artifacts (known TypeScript errors tracked in engineering#9) + +## Known Issues + +The `rewriteExpr` function in the TypeScript generator applies spurious `this.state.` prefixes to function parameters in guard expressions. Tracked in [marcus/engineering#9](https://os.scoble.me/forgejo/marcus/engineering/issues/9). Parser also drops `precondition:`/`postcondition:` blocks on some action bodies. ## Contributing -Speckl is open source under the [MIT license](LICENSE). We welcome contributions: +Speckl is open source under the [MIT license](LICENSE). Contributions welcome: - **Language design** — refine SpeckDL syntax and semantics -- **Formal methods** — help verify the compilation pipeline -- **Compliance expertise** — validate the DO-178C, IEC 62304, ISO 26262 models -- **Agent development** — build and test the tiered agent harness -- **Documentation** — make dark provenance accessible to working developers +- **Compiler development** — TypeScript/Node.js, help fix open issues +- **Formal methods** — verify the compilation pipeline, add TLA+ examples +- **Compliance expertise** — validate regulatory models (NIST, IEC 62304, DO-178C) +- **Documentation** — tutorials, guides, case studies -Open an issue or submit a pull request. See [speckdl/SPEC.md](speckdl/SPEC.md) for the language spec and [docs/](docs/) for strategy and methodology documents. +Open an issue or submit a pull request. ## License -MIT © Scott Scoble / Greybeard Holdings, LLC \ No newline at end of file +MIT © Scott Scoble / Greybeard Holdings, LLC + +## Mirrors + +- **Primary:** https://os.scoble.me/forgejo/sscoble/speckl +- **Mirror:** https://codeberg.org/sscoble/speckl diff --git a/bench-rust.mjs b/bench-rust.mjs new file mode 100644 index 0000000..e5f49ac --- /dev/null +++ b/bench-rust.mjs @@ -0,0 +1,120 @@ +import fs from 'fs'; +import { execSync } from 'child_process'; + +const specs = [ + { name: 'KafkaKRaft', construct: 'KafkaKRaftMachine::new()', ops: [ + 'm.request_vote(2, 1, 0)', + 'm.append_entries(2)', + 'm.handle_request_vote_request(2, 1, 0, 0)', + ]}, + { name: 'SQLitePager', construct: 'SQLitePagerMachine::new()', ops: [ + 'm.begin_read_transaction()', + 'm.begin_write_transaction()', + 'm.write_page(1, 100)', + 'm.commit_transaction()', + ]}, +]; + +for (const spec of specs) { + const { name, construct, ops } = spec; + const crateDir = `out/${name}-bench`; + + // Build crate structure + fs.mkdirSync(`${crateDir}/src`, { recursive: true }); + fs.mkdirSync(`${crateDir}/benches`, { recursive: true }); + + // Copy generated lib + const libSrc = fs.readFileSync(`out/${name}/src/lib.rs`, 'utf-8'); + + // Add type defs + const typeDefs = `// Auto-generated type definitions +pub type NodeId = u64; +pub type LogEntry = String; +pub type Account = String; +pub type Transfer = String; +pub type TransferId = u64; +pub type PageId = u64; +pub type PageData = u64; +pub type Frame = u64; +pub type FrameId = u64; +pub type Bytes = Vec; +`; + + fs.writeFileSync(`${crateDir}/src/lib.rs`, typeDefs + libSrc); + + // Cargo.toml + fs.writeFileSync(`${crateDir}/Cargo.toml`, `[package] +name = "${name.toLowerCase()}-bench" +version = "0.1.0" +edition = "2021" + +[[bench]] +name = "bench" +harness = false + +[profile.bench] +debug = 0 +lto = "fat" +codegen-units = 1 +`); + + // Benchmark code + const opBody = ops.map((op, i) => ` let _ = ${op};`).join('\n'); + const totalOps = ops.length; + + fs.writeFileSync(`${crateDir}/benches/bench.rs`, `use std::time::Instant; + +#[path = "../src/lib.rs"] +mod generated; +use generated::*; + +fn main() { + let iterations = 100_000; + println!("\\n=== ${name} Benchmark ==="); + println!("Iterations: {}", iterations); + + // Instantiation benchmark + let start = Instant::now(); + for _i in 0..iterations { + let _m = ${construct}; + } + let elapsed = start.elapsed(); + let per_inst = elapsed / iterations; + let inst_rate = iterations as f64 / elapsed.as_secs_f64(); + println!(" Instantiate:"); + println!(" Total: {:?}", elapsed); + println!(" Per op: {:?}", per_inst); + if inst_rate > 1_000_000.0 { + println!(" Rate: {:.1} M/sec", inst_rate / 1_000_000.0); + } else if inst_rate > 1_000.0 { + println!(" Rate: {:.1} K/sec", inst_rate / 1_000.0); + } else { + println!(" Rate: {:.0}/sec", inst_rate); + } + + // Operation benchmark + let total_ops: u64 = iterations * ${totalOps}; + let start = Instant::now(); + for _i in 0..iterations { + let mut m = ${construct}; +${opBody} + } + let elapsed = start.elapsed(); + let per_op = elapsed / total_ops; + let rate = total_ops as f64 / elapsed.as_secs_f64(); + println!(" Operations ({} ops per cycle):", ${totalOps}); + println!(" Total: {:?}", elapsed); + println!(" Per op: {:?}", per_op); + if rate > 1_000_000.0 { + println!(" Rate: {:.1} M ops/sec", rate / 1_000_000.0); + } else if rate > 1_000.0 { + println!(" Rate: {:.1} K ops/sec", rate / 1_000.0); + } else { + println!(" Rate: {:.0} ops/sec", rate); + } +} +`); +} + +console.log('Bench crates created. Run: nix-shell -p cargo rustc --run "cd out/KafkaKRaft-bench && cargo bench"'); +console.log('And: nix-shell -p cargo rustc --run "cd out/SQLitePager-bench && cargo bench"'); diff --git a/bench-rust.sh b/bench-rust.sh new file mode 100644 index 0000000..0f1bd52 --- /dev/null +++ b/bench-rust.sh @@ -0,0 +1,155 @@ +#!/bin/bash +# Benchmark Speck-generated Rust output +set -e +cd /home/sscoble/speckl + +regen=0 + +bench_spec() { + local SPEC=$1 + local BENCH_DIR="out/${SPEC}-bench" + local PROLOGUE=$2 + local CONSTRUCT=$3 + local OPS=$4 + + echo "=== $SPEC ===" + + if [ "$regen" -eq 1 ]; then + node compiler/dist/index.js "examples/$SPEC.speck" --target=rust 2>/dev/null + fi + + mkdir -p "$BENCH_DIR/src" + cp "out/$SPEC/src/lib.rs" "$BENCH_DIR/src/lib.rs" + + # Add type defs and fix Option syntax + cat > "$BENCH_DIR/src/lib_patched.rs" << 'EOF' +// Auto-generated type definitions +pub type NodeId = u64; +pub type LogEntry = String; +pub type Account = String; +pub type Transfer = String; +pub type TransferId = u64; +pub type PageId = u64; +pub type PageData = u64; +pub type Frame = u64; +pub type FrameId = u64; +EOF + + cat "$BENCH_DIR/src/lib.rs" >> "$BENCH_DIR/src/lib_patched.rs" + mv "$BENCH_DIR/src/lib_patched.rs" "$BENCH_DIR/src/lib.rs" + + # Create Cargo.toml with benchmark profile + cat > "$BENCH_DIR/Cargo.toml" << 'CTOML' +[package] +name = "speck-bench" +version = "0.1.0" +edition = "2021" + +[[bench]] +name = "bench" +harness = false + +[profile.bench] +debug = 0 +lto = "fat" +codegen-units = 1 +CTOML + + # Create benchmark + mkdir -p "$BENCH_DIR/benches" + cat > "$BENCH_DIR/benches/bench.rs" << 'BRS' +use std::time::{Duration, Instant}; + +#[path = "../src/lib.rs"] +mod generated; + +BRS + + # Add spec-specific benchmark + { + echo "use generated::*;" + echo "" + echo "fn main() {" + echo " let iterations = 100000;" + echo " println!(\"Benchmarking $SPEC...\");" + echo "" + echo " // Instantiation benchmark" + echo " let start = Instant::now();" + echo " for i in 0..iterations {" + echo " let mut m = $CONSTRUCT;" + echo " let _ = m;" + echo " }" + echo " let elapsed = start.elapsed();" + echo " let per = elapsed / iterations;" + echo " println!(\" Instantiate x{}:\", iterations);" + echo " println!(\" Total: {:?}\", elapsed);" + echo " println!(\" Per op: {:?}\", per);" + echo " println!(\" Ops/sec: {:.0}\", iterations as f64 / elapsed.as_secs_f64());" + echo "" + echo " // Operational benchmark" + echo " let mut total_ops = 0u64;" + echo " let start = Instant::now();" + echo " for i in 0..iterations {" + echo $PROLOGUE + echo " }" + echo " let elapsed = start.elapsed();" + echo " let per = elapsed / total_ops;" + echo " println!(\" Operations x{} ({} ops):\", iterations, total_ops);" + echo " let rate = total_ops as f64 / elapsed.as_secs_f64();" + echo " println!(\" Total: {:?}\", elapsed);" + echo " println!(\" Per op: {:?}\", per);" + echo " if rate > 1_000_000.0 {" + echo " println!(\" Rate: {:.0} M ops/sec\", rate / 1_000_000.0);" + echo " } else if rate > 1_000.0 {" + echo " println!(\" Rate: {:.0} K ops/sec\", rate / 1_000.0);" + echo " } else {" + echo " println!(\" Rate: {:.0} ops/sec\", rate);" + echo " }" + echo "}" + } >> "$BENCH_DIR/benches/bench.rs" +} + +# KafkaKRaft benchmark +bench_spec "KafkaKRaft" \ + ' let mut m = KafkaKRaftMachine::new(); + let _ = m.request_vote(2, 1, 0); + let _ = m.append_entries(2); + let _ = m.handle_request_vote_request(2, 1, 0, 0); + total_ops += 3;' \ + 'KafkaKRaftMachine::new()' \ + 3 + +# SQLitePager benchmark +bench_spec "SQLitePager" \ + ' let mut m = SQLitePagerMachine::new(); + let _ = m.begin_read_transaction(); + let _ = m.begin_write_transaction(); + let _ = m.write_page(1, 100); + let _ = m.commit_transaction(); + total_ops += 4;' \ + 'SQLitePagerMachine::new()' \ + 4 + +# TigerBeetleLedger benchmark +bench_spec "TigerBeetleLedger" \ + ' let mut m = TigerBeetleLedgerMachine::new(); + let _ = m.create_account(1, 1000, 0); + let _ = m.create_account(2, 0, 0); + let _ = m.create_transfer(1, 1, 2, 100, 0); + total_ops += 3;' \ + 'TigerBeetleLedgerMachine::new()' \ + 3 + +echo "" +echo "=== Building benchmarks ===" +for D in out/KafkaKRaft-bench out/SQLitePager-bench out/TigerBeetleLedger-bench; do + echo "Building $D..." + (cd "$D" && cargo bench --no-run 2>&1 | tail -5) +done + +echo "" +echo "=== Running benchmarks ===" +for D in out/KafkaKRaft-bench out/SQLitePager-bench out/TigerBeetleLedger-bench; do + echo "" + (cd "$D" && cargo bench 2>&1) +done diff --git a/compiler/__tests__/e2e-wasm.test.ts b/compiler/__tests__/e2e-wasm.test.ts deleted file mode 100644 index aef5625..0000000 --- a/compiler/__tests__/e2e-wasm.test.ts +++ /dev/null @@ -1,203 +0,0 @@ -import { describe, it, expect } from 'vitest'; -import { parseSpeckContent } from '../src/parser.js'; -import { generateWASM } from '../src/generators/wasm.js'; -import { createHostImports, bindMemory, readScalarState } from '../src/runtime/wasm-host.js'; -import fs from 'fs'; -import path from 'path'; -import os from 'os'; - -// Dynamic import wabt since it's a CommonJS module -async function loadWabt() { - const wabt = await import('wabt'); - return wabt.default(); -} - -describe('SpeckDL WASM E2E', () => { - it('should compile Counter speck to WAT, assemble to WASM, and run in Node.js', async () => { - // A minimal scalar-only speck that we can fully compile and run - // NOTE: Parser requires braces on the SAME line as block keywords - const speckSource = `speck Counter { - author: "Speckl Compiler v0.3" - source: document ref "WASM-E2E" - review: automated - - state: { - count: Nat := 0 - maxCount: Nat := 10 - } - - init: { - count := 0 - maxCount := 10 - } - - action Increment() { - require count < maxCount - count := count + 1 - } - - action Decrement() { - require count > 0 - count := count - 1 - } - - action Reset() { - count := 0 - } - - action GetCount() { - return count - } -}`; - - // 1. Parse - const ast = parseSpeckContent(speckSource); - expect(ast.specks).toHaveLength(1); - expect(ast.specks[0].name).toBe('Counter'); - - // 2. Generate WAT to temp dir - const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'speckl-wasm-e2e-')); - generateWASM(ast, tmpDir); - - const watPath = path.join(tmpDir, 'Counter.wat'); - expect(fs.existsSync(watPath)).toBe(true); - - const watContent = fs.readFileSync(watPath, 'utf-8'); - expect(watContent).toContain('(module'); - expect(watContent).toContain('(export "init")'); - expect(watContent).toContain('(export "Increment")'); - expect(watContent).toContain('(export "Decrement")'); - expect(watContent).toContain('(export "Reset")'); - expect(watContent).toContain('(export "GetCount")'); - expect(watContent).toContain('(export "getState")'); - expect(watContent).toContain('(export "memory")'); - - // 3. Assemble WAT → WASM using wabt - const wabtModule = await loadWabt(); - const wasmModule = wabtModule.parseWat('Counter.wat', watContent); - wasmModule.resolveNames(); - wasmModule.validate({ features: {} }); - const binaryOutput = wasmModule.toBinary({ log: false, write_debug_names: false }); - const wasmBytes = binaryOutput.buffer; - - expect(wasmBytes.length).toBeGreaterThan(0); - expect(wasmBytes.length).toBeGreaterThan(50); // Not just empty header - - // 4. Instantiate with host runtime - const hostState = { - maps: new Map(), - sets: new Map(), - nextHandle: 1, - memory: null as WebAssembly.Memory | null, - }; - const imports = { speckl: createHostImports(hostState) }; - - const wasm = await WebAssembly.instantiate(wasmBytes, imports); - const memory = wasm.instance.exports.memory as WebAssembly.Memory; - bindMemory(hostState, memory); - - // 5. Verify exports - expect(typeof wasm.instance.exports.init).toBe('function'); - expect(typeof wasm.instance.exports.Increment).toBe('function'); - expect(typeof wasm.instance.exports.Decrement).toBe('function'); - expect(typeof wasm.instance.exports.Reset).toBe('function'); - expect(typeof wasm.instance.exports.GetCount).toBe('function'); - expect(typeof wasm.instance.exports.getState).toBe('function'); - expect(memory).toBeInstanceOf(WebAssembly.Memory); - - // 6. Run init - (wasm.instance.exports.init as Function)(); - - // 7. Verify initial state via getState - const stateResult = (wasm.instance.exports.getState as Function)(); - // getState returns i32 i32 (ptr, len) - expect(stateResult).toBeDefined(); - - // Read memory directly to verify state - const scalarState = readScalarState(memory, ['count', 'maxCount']); - expect(scalarState.count).toBe(0); - expect(scalarState.maxCount).toBe(10); - - // 8. Run Increment 3 times, calling getState after each to sync memory - for (let i = 0; i < 3; i++) { - const ret = (wasm.instance.exports.Increment as Function)(); - expect(ret).toBe(1); // Success - (wasm.instance.exports.getState as Function)(); // Sync globals to memory - } - - const afterIncrement = readScalarState(memory, ['count', 'maxCount']); - expect(afterIncrement.count).toBe(3); - expect(afterIncrement.maxCount).toBe(10); - - // 9. Run Decrement once - const decRet = (wasm.instance.exports.Decrement as Function)(); - expect(decRet).toBe(1); // Success - (wasm.instance.exports.getState as Function)(); - - const afterDecrement = readScalarState(memory, ['count', 'maxCount']); - expect(afterDecrement.count).toBe(2); - - // 10. Run GetCount — should return current count (global, not memory) - const getRet = (wasm.instance.exports.GetCount as Function)(); - expect(getRet).toBe(2); - - // 11. Run Reset - const resetRet = (wasm.instance.exports.Reset as Function)(); - expect(resetRet).toBe(1); // Success - (wasm.instance.exports.getState as Function)(); - - const afterReset = readScalarState(memory, ['count', 'maxCount']); - expect(afterReset.count).toBe(0); - - // 12. Guard test: Decrement at 0 should fail - const guardFail = (wasm.instance.exports.Decrement as Function)(); - expect(guardFail).toBe(0); // Guard failed → return 0 - - // 13. Guard test: Increment at maxCount should fail - // First increment to max - for (let i = 0; i < 10; i++) { - (wasm.instance.exports.Increment as Function)(); - } - (wasm.instance.exports.getState as Function)(); - const atMax = readScalarState(memory, ['count', 'maxCount']); - expect(atMax.count).toBe(10); - - const maxGuardFail = (wasm.instance.exports.Increment as Function)(); - expect(maxGuardFail).toBe(0); // Guard failed → return 0 - - // Cleanup - fs.rmSync(tmpDir, { recursive: true, force: true }); - }, 30000); // 30s timeout for wabt compilation - - it('should compile TigerBeetle speck to valid WAT that wabt accepts', async () => { - // Load the full TigerBeetle speck from file - const speckPath = path.join(__dirname, 'tigerbeetle.speck'); - const speckSource = fs.readFileSync(speckPath, 'utf-8'); - - const ast = parseSpeckContent(speckSource); - expect(ast.specks).toHaveLength(1); - expect(ast.specks[0].name).toBe('TigerBeetle'); - - const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'speckl-wasm-e2e-tb-')); - generateWASM(ast, tmpDir); - - const watPath = path.join(tmpDir, 'TigerBeetle.wat'); - expect(fs.existsSync(watPath)).toBe(true); - - const watContent = fs.readFileSync(watPath, 'utf-8'); - - // Validate with wabt - const wabtModule = await loadWabt(); - const wasmModule = wabtModule.parseWat('TigerBeetle.wat', watContent); - - // These should not throw - wasmModule.resolveNames(); - wasmModule.validate({ features: {} }); - const binaryOutput = wasmModule.toBinary({ log: false, write_debug_names: false }); - - expect(binaryOutput.buffer.length).toBeGreaterThan(100); - - // Cleanup - fs.rmSync(tmpDir, { recursive: true, force: true }); - }, 30000); -}); diff --git a/compiler/__tests__/parser.test.ts b/compiler/__tests__/parser.test.ts index 58af7b0..e16b091 100644 --- a/compiler/__tests__/parser.test.ts +++ b/compiler/__tests__/parser.test.ts @@ -1,6 +1,5 @@ import { describe, it, expect } from 'vitest'; import { parseSpeckContent, parseSpeckFile } from '../src/parser.js'; -import { generateWASM } from '../src/generators/wasm.js'; describe('SpeckDL Parser', () => { describe('parseSpeckContent', () => { @@ -525,76 +524,4 @@ speck Second { } }); }); - - describe('WASM Codegen', () => { - it('should generate WAT for TigerBeetle with all exports', () => { - const ast = parseSpeckFile('./__tests__/tigerbeetle.speck'); - const fs = require('fs'); - const os = require('os'); - const path = require('path'); - const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'speckl-wasm-test-')); - - generateWASM(ast, tmpDir); - - const watPath = path.join(tmpDir, 'TigerBeetle.wat'); - expect(fs.existsSync(watPath)).toBe(true); - - const wat = fs.readFileSync(watPath, 'utf-8'); - - // Module structure - expect(wat).toContain('(module'); - expect(wat).toContain('(memory (export "memory")'); - - // Host imports - expect(wat).toContain('(import "speckl" "map_create"'); - expect(wat).toContain('(import "speckl" "set_create"'); - - // State globals - expect(wat).toContain('(global $nextId'); - expect(wat).toContain('(global $accounts'); - expect(wat).toContain('(global $transfers'); - expect(wat).toContain('(global $ledgerOpen'); - - // init export - expect(wat).toContain('(func $init (export "init")'); - - // Action exports - expect(wat).toContain('(func $CreateAccount (export "CreateAccount")'); - expect(wat).toContain('(func $Transfer (export "Transfer")'); - expect(wat).toContain('(func $CloseAccount (export "CloseAccount")'); - expect(wat).toContain('(func $CloseLedger (export "CloseLedger")'); - - // getState export - expect(wat).toContain('(func $getState (export "getState")'); - - // Constraint stub - expect(wat).toContain('(func $checkConstraints (export "checkConstraints")'); - - // Scalar init values - expect(wat).toContain('(i32.const 1)'); // nextId default - expect(wat).toContain('(call $map_create)'); - expect(wat).toContain('(call $set_create)'); - - fs.rmSync(tmpDir, { recursive: true }); - }); - - it('should emit scalar guard expressions as WASM comparison', () => { - const ast = parseSpeckFile('./__tests__/tigerbeetle.speck'); - const fs = require('fs'); - const os = require('os'); - const path = require('path'); - const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'speckl-wasm-test-')); - - generateWASM(ast, tmpDir); - const wat = fs.readFileSync(path.join(tmpDir, 'TigerBeetle.wat'), 'utf-8'); - - // ledgerOpen == true should compile to i32.eq - expect(wat).toContain('(i32.eq (global.get $ledgerOpen) (i32.const 1))'); - - // nextId + 1 should compile to i32.add - expect(wat).toContain('(i32.add (global.get $nextId) (i32.const 1))'); - - fs.rmSync(tmpDir, { recursive: true }); - }); - }); }); diff --git a/compiler/dist/index.js b/compiler/dist/index.js index 724435a..e405d47 100644 --- a/compiler/dist/index.js +++ b/compiler/dist/index.js @@ -6,7 +6,8 @@ import { generateProvenanceGraph } from './generators/provenance.js'; import { generateCycloneDX } from './generators/cyclonedx.js'; import { generateSPDX } from './generators/spdx.js'; import { generateTypeScriptStateMachine } from './generators/typescript-state-machine.js'; -import { generateWASM } from './generators/wasm.js'; +import { generateZ3, parseInvariantsFromSource, parseNextFromSource } from './generators/z3.js'; +import { generateRust } from './generators/rust.js'; import fs from 'fs'; import path from 'path'; async function main() { @@ -35,6 +36,19 @@ async function main() { describe: 'Output directory for compiled artifacts', type: 'string', default: './out', + }) + .option('target', { + alias: 't', + describe: 'Compilation target', + type: 'string', + choices: ['typescript', 'z3', 'rust', 'all'], + default: 'typescript', + }) + .option('verify-depth', { + alias: 'd', + describe: 'Bounded model checking depth for Z3 verification', + type: 'number', + default: 10, }) .option('bom-format', { alias: 'b', @@ -50,6 +64,8 @@ async function main() { const options = { outputDir: argv.outputDir, bomFormat: argv.bomFormat, + target: argv.target, + verifyDepth: argv.verifyDepth, }; console.log(`Compiling ${file}...`); // Parse the Speck file @@ -61,23 +77,45 @@ async function main() { if (!fs.existsSync(options.outputDir)) { fs.mkdirSync(options.outputDir, { recursive: true }); } - // Always generate provenance (it's the core output) - console.log('\nGenerating provenance graph (PROV-O JSON-LD)...'); - generateProvenanceGraph(ast, options.outputDir); - // Generate TypeScript state machine - console.log('\nGenerating TypeScript state machine...'); - generateTypeScriptStateMachine(ast, options.outputDir); - // Generate WASM state machine - console.log('\nGenerating WebAssembly (WAT)...'); - generateWASM(ast, options.outputDir); - // Generate BOM based on format selection - if (options.bomFormat === 'cdx' || options.bomFormat === 'both') { - console.log('\nGenerating CycloneDX BOM...'); - generateCycloneDX(ast, options.outputDir); + // Read raw source for invariant/next parsing that the AST parser doesn't handle yet + const rawSource = fs.readFileSync(file, 'utf-8'); + // Generate Rust state machine if target is rust or all + if (options.target === 'rust' || options.target === 'all') { + console.log('\nGenerating Rust state machine...'); + generateRust(ast, options.outputDir); } - if (options.bomFormat === 'spdx' || options.bomFormat === 'both') { - console.log('\nGenerating SPDX BOM...'); - generateSPDX(ast, options.outputDir); + // Always generate provenance (it's the core output) if not z3-only or rust-only + if (options.target !== 'z3' && options.target !== 'rust') { + console.log('\nGenerating provenance graph (PROV-O JSON-LD)...'); + generateProvenanceGraph(ast, options.outputDir); + } + // Generate Z3 SMT-LIB2 if target is z3 or all + if (options.target === 'z3' || options.target === 'all') { + console.log('\nGenerating Z3 SMT-LIB2...'); + // Parse invariants from raw source for Z3 backend + for (const speck of ast.specks) { + const invariants = parseInvariantsFromSource(rawSource, speck.name); + const nextNode = parseNextFromSource(rawSource, speck.name); + // Attach parsed data to the AST for the generator + speck._invariants = invariants; + speck._next = nextNode; + } + const z3Options = { verifyDepth: options.verifyDepth }; + generateZ3(ast, options.outputDir, z3Options); + } + // Generate TypeScript state machine if not z3-only or rust-only + if (options.target !== 'z3' && options.target !== 'rust') { + console.log('\nGenerating TypeScript state machine...'); + generateTypeScriptStateMachine(ast, options.outputDir); + // Generate BOM based on format selection + if (options.bomFormat === 'cdx' || options.bomFormat === 'both') { + console.log('\nGenerating CycloneDX BOM...'); + generateCycloneDX(ast, options.outputDir); + } + if (options.bomFormat === 'spdx' || options.bomFormat === 'both') { + console.log('\nGenerating SPDX BOM...'); + generateSPDX(ast, options.outputDir); + } } console.log(`\nCompilation complete. Output: ${options.outputDir}/`); } diff --git a/compiler/dist/index.js.map b/compiler/dist/index.js.map index c732d56..6202963 100644 --- a/compiler/dist/index.js.map +++ b/compiler/dist/index.js.map @@ -1 +1 @@ -{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AAEA,OAAO,KAAK,MAAM,OAAO,CAAC;AAC1B,OAAO,EAAE,OAAO,EAAE,MAAM,eAAe,CAAC;AACxC,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAC7C,OAAO,EAAE,uBAAuB,EAAE,MAAM,4BAA4B,CAAC;AACrE,OAAO,EAAE,iBAAiB,EAAE,MAAM,2BAA2B,CAAC;AAC9D,OAAO,EAAE,YAAY,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,EAAE,8BAA8B,EAAE,MAAM,0CAA0C,CAAC;AAC1F,OAAO,EAAE,YAAY,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,EAAE,MAAM,IAAI,CAAC;AACpB,OAAO,IAAI,MAAM,MAAM,CAAC;AAOxB,KAAK,UAAU,IAAI;IACjB,uDAAuD;IACvD,MAAM,WAAW,GAAG,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,QAAQ,CAAC,EAAE,IAAI,EAAE,eAAe,CAAC,CAAC;IACtG,IAAI,EAAE,CAAC,UAAU,CAAC,WAAW,CAAC,EAAE,CAAC;QAC/B,IAAI,CAAC;YACH,MAAM,EAAE,QAAQ,EAAE,GAAG,MAAM,MAAM,CAAC,eAAe,CAAC,CAAC;YACnD,QAAQ,CAAC,SAAS,WAAW,GAAG,EAAE,EAAE,KAAK,EAAE,SAAS,EAAE,CAAC,CAAC;QAC1D,CAAC;QAAC,MAAM,CAAC;YACP,OAAO,CAAC,KAAK,CAAC,oDAAoD,CAAC,CAAC;YACpE,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QAClB,CAAC;IACH,CAAC;IAED,MAAM,IAAI,GAAG,MAAM,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;SAC5C,OAAO,CAAC,WAAW,EAAE,iCAAiC,EAAE,CAAC,QAAQ,EAAE,EAAE;QACpE,OAAO,QAAQ;aACZ,UAAU,CAAC,MAAM,EAAE;YAClB,QAAQ,EAAE,yBAAyB;YACnC,IAAI,EAAE,QAAQ;YACd,YAAY,EAAE,IAAI;SACnB,CAAC;aACD,MAAM,CAAC,YAAY,EAAE;YACpB,KAAK,EAAE,GAAG;YACV,QAAQ,EAAE,yCAAyC;YACnD,IAAI,EAAE,QAAQ;YACd,OAAO,EAAE,OAAO;SACjB,CAAC;aACD,MAAM,CAAC,YAAY,EAAE;YACpB,KAAK,EAAE,GAAG;YACV,QAAQ,EAAE,mBAAmB;YAC7B,IAAI,EAAE,QAAQ;YACd,OAAO,EAAE,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,CAAC;YAChC,OAAO,EAAE,MAAe;SACzB,CAAC,CAAC;IACP,CAAC,CAAC;SACD,IAAI,EAAE;SACN,IAAI,CAAC;IAER,MAAM,IAAI,GAAG,IAAI,CAAC,IAAc,CAAC;IACjC,MAAM,OAAO,GAAG;QACd,SAAS,EAAE,IAAI,CAAC,SAAmB;QACnC,SAAS,EAAE,IAAI,CAAC,SAAoC;KACrD,CAAC;IAEF,OAAO,CAAC,GAAG,CAAC,aAAa,IAAI,KAAK,CAAC,CAAC;IAEpC,uBAAuB;IACvB,MAAM,GAAG,GAAG,cAAc,CAAC,IAAI,CAAC,CAAC;IAEjC,MAAM,UAAU,GAAG,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC;IACrC,MAAM,WAAW,GAAG,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;IAC7E,OAAO,CAAC,GAAG,CAAC,4BAA4B,UAAU,cAAc,WAAW,YAAY,CAAC,CAAC;IAEzF,iCAAiC;IACjC,IAAI,CAAC,EAAE,CAAC,UAAU,CAAC,OAAO,CAAC,SAAS,CAAC,EAAE,CAAC;QACtC,EAAE,CAAC,SAAS,CAAC,OAAO,CAAC,SAAS,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;IACvD,CAAC;IAED,oDAAoD;IACpD,OAAO,CAAC,GAAG,CAAC,mDAAmD,CAAC,CAAC;IACjE,uBAAuB,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;IAEhD,oCAAoC;IACpC,OAAO,CAAC,GAAG,CAAC,0CAA0C,CAAC,CAAC;IACxD,8BAA8B,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;IAEvD,8BAA8B;IAC9B,OAAO,CAAC,GAAG,CAAC,mCAAmC,CAAC,CAAC;IACjD,YAAY,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;IAErC,yCAAyC;IACzC,IAAI,OAAO,CAAC,SAAS,KAAK,KAAK,IAAI,OAAO,CAAC,SAAS,KAAK,MAAM,EAAE,CAAC;QAChE,OAAO,CAAC,GAAG,CAAC,+BAA+B,CAAC,CAAC;QAC7C,iBAAiB,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;IAC5C,CAAC;IAED,IAAI,OAAO,CAAC,SAAS,KAAK,MAAM,IAAI,OAAO,CAAC,SAAS,KAAK,MAAM,EAAE,CAAC;QACjE,OAAO,CAAC,GAAG,CAAC,0BAA0B,CAAC,CAAC;QACxC,YAAY,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;IACvC,CAAC;IAED,OAAO,CAAC,GAAG,CAAC,mCAAmC,OAAO,CAAC,SAAS,GAAG,CAAC,CAAC;AACvE,CAAC;AAED,IAAI,EAAE,CAAC,KAAK,CAAC,CAAC,GAAG,EAAE,EAAE;IACnB,OAAO,CAAC,KAAK,CAAC,qBAAqB,EAAE,GAAG,CAAC,CAAC;IAC1C,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC"} \ No newline at end of file +{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AAEA,OAAO,KAAK,MAAM,OAAO,CAAC;AAC1B,OAAO,EAAE,OAAO,EAAE,MAAM,eAAe,CAAC;AACxC,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAC7C,OAAO,EAAE,uBAAuB,EAAE,MAAM,4BAA4B,CAAC;AACrE,OAAO,EAAE,iBAAiB,EAAE,MAAM,2BAA2B,CAAC;AAC9D,OAAO,EAAE,YAAY,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,EAAE,8BAA8B,EAAE,MAAM,0CAA0C,CAAC;AAC1F,OAAO,EAAE,UAAU,EAAa,yBAAyB,EAAE,mBAAmB,EAAE,MAAM,oBAAoB,CAAC;AAC3G,OAAO,EAAE,YAAY,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,EAAE,MAAM,IAAI,CAAC;AACpB,OAAO,IAAI,MAAM,MAAM,CAAC;AASxB,KAAK,UAAU,IAAI;IACjB,uDAAuD;IACvD,MAAM,WAAW,GAAG,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,QAAQ,CAAC,EAAE,IAAI,EAAE,eAAe,CAAC,CAAC;IACtG,IAAI,EAAE,CAAC,UAAU,CAAC,WAAW,CAAC,EAAE,CAAC;QAC/B,IAAI,CAAC;YACH,MAAM,EAAE,QAAQ,EAAE,GAAG,MAAM,MAAM,CAAC,eAAe,CAAC,CAAC;YACnD,QAAQ,CAAC,SAAS,WAAW,GAAG,EAAE,EAAE,KAAK,EAAE,SAAS,EAAE,CAAC,CAAC;QAC1D,CAAC;QAAC,MAAM,CAAC;YACP,OAAO,CAAC,KAAK,CAAC,oDAAoD,CAAC,CAAC;YACpE,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QAClB,CAAC;IACH,CAAC;IAED,MAAM,IAAI,GAAG,MAAM,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;SAC5C,OAAO,CAAC,WAAW,EAAE,iCAAiC,EAAE,CAAC,QAAQ,EAAE,EAAE;QACpE,OAAO,QAAQ;aACZ,UAAU,CAAC,MAAM,EAAE;YAClB,QAAQ,EAAE,yBAAyB;YACnC,IAAI,EAAE,QAAQ;YACd,YAAY,EAAE,IAAI;SACnB,CAAC;aACD,MAAM,CAAC,YAAY,EAAE;YACpB,KAAK,EAAE,GAAG;YACV,QAAQ,EAAE,yCAAyC;YACnD,IAAI,EAAE,QAAQ;YACd,OAAO,EAAE,OAAO;SACjB,CAAC;aACD,MAAM,CAAC,QAAQ,EAAE;YAChB,KAAK,EAAE,GAAG;YACV,QAAQ,EAAE,oBAAoB;YAC9B,IAAI,EAAE,QAAQ;YACd,OAAO,EAAE,CAAC,YAAY,EAAE,IAAI,EAAE,MAAM,EAAE,KAAK,CAAC;YAC5C,OAAO,EAAE,YAAqB;SAC/B,CAAC;aACD,MAAM,CAAC,cAAc,EAAE;YACtB,KAAK,EAAE,GAAG;YACV,QAAQ,EAAE,kDAAkD;YAC5D,IAAI,EAAE,QAAQ;YACd,OAAO,EAAE,EAAE;SACZ,CAAC;aACD,MAAM,CAAC,YAAY,EAAE;YACpB,KAAK,EAAE,GAAG;YACV,QAAQ,EAAE,mBAAmB;YAC7B,IAAI,EAAE,QAAQ;YACd,OAAO,EAAE,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,CAAC;YAChC,OAAO,EAAE,MAAe;SACzB,CAAC,CAAC;IACP,CAAC,CAAC;SACD,IAAI,EAAE;SACN,IAAI,CAAC;IAER,MAAM,IAAI,GAAG,IAAI,CAAC,IAAc,CAAC;IACjC,MAAM,OAAO,GAAG;QACd,SAAS,EAAE,IAAI,CAAC,SAAmB;QACnC,SAAS,EAAE,IAAI,CAAC,SAAoC;QACpD,MAAM,EAAE,IAAI,CAAC,MAA8C;QAC3D,WAAW,EAAE,IAAI,CAAC,WAAqB;KACxC,CAAC;IAEF,OAAO,CAAC,GAAG,CAAC,aAAa,IAAI,KAAK,CAAC,CAAC;IAEpC,uBAAuB;IACvB,MAAM,GAAG,GAAG,cAAc,CAAC,IAAI,CAAC,CAAC;IAEjC,MAAM,UAAU,GAAG,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC;IACrC,MAAM,WAAW,GAAG,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;IAC7E,OAAO,CAAC,GAAG,CAAC,4BAA4B,UAAU,cAAc,WAAW,YAAY,CAAC,CAAC;IAEzF,iCAAiC;IACjC,IAAI,CAAC,EAAE,CAAC,UAAU,CAAC,OAAO,CAAC,SAAS,CAAC,EAAE,CAAC;QACtC,EAAE,CAAC,SAAS,CAAC,OAAO,CAAC,SAAS,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;IACvD,CAAC;IAED,oFAAoF;IACpF,MAAM,SAAS,GAAG,EAAE,CAAC,YAAY,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC;IAEjD,uDAAuD;IACvD,IAAI,OAAO,CAAC,MAAM,KAAK,MAAM,IAAI,OAAO,CAAC,MAAM,KAAK,KAAK,EAAE,CAAC;QAC1D,OAAO,CAAC,GAAG,CAAC,oCAAoC,CAAC,CAAC;QAClD,YAAY,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;IACvC,CAAC;IAED,gFAAgF;IAChF,IAAI,OAAO,CAAC,MAAM,KAAK,IAAI,IAAI,OAAO,CAAC,MAAM,KAAK,MAAM,EAAE,CAAC;QACzD,OAAO,CAAC,GAAG,CAAC,mDAAmD,CAAC,CAAC;QACjE,uBAAuB,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;IAClD,CAAC;IAED,8CAA8C;IAC9C,IAAI,OAAO,CAAC,MAAM,KAAK,IAAI,IAAI,OAAO,CAAC,MAAM,KAAK,KAAK,EAAE,CAAC;QACxD,OAAO,CAAC,GAAG,CAAC,6BAA6B,CAAC,CAAC;QAC3C,kDAAkD;QAClD,KAAK,MAAM,KAAK,IAAI,GAAG,CAAC,MAAM,EAAE,CAAC;YAC/B,MAAM,UAAU,GAAG,yBAAyB,CAAC,SAAS,EAAE,KAAK,CAAC,IAAI,CAAC,CAAC;YACpE,MAAM,QAAQ,GAAG,mBAAmB,CAAC,SAAS,EAAE,KAAK,CAAC,IAAI,CAAC,CAAC;YAC5D,kDAAkD;YACjD,KAAa,CAAC,WAAW,GAAG,UAAU,CAAC;YACvC,KAAa,CAAC,KAAK,GAAG,QAAQ,CAAC;QAClC,CAAC;QACD,MAAM,SAAS,GAAc,EAAE,WAAW,EAAE,OAAO,CAAC,WAAW,EAAE,CAAC;QAClE,UAAU,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,EAAE,SAAS,CAAC,CAAC;IAChD,CAAC;IAED,gEAAgE;IAChE,IAAI,OAAO,CAAC,MAAM,KAAK,IAAI,IAAI,OAAO,CAAC,MAAM,KAAK,MAAM,EAAE,CAAC;QACzD,OAAO,CAAC,GAAG,CAAC,0CAA0C,CAAC,CAAC;QACxD,8BAA8B,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;QAEvD,yCAAyC;QACzC,IAAI,OAAO,CAAC,SAAS,KAAK,KAAK,IAAI,OAAO,CAAC,SAAS,KAAK,MAAM,EAAE,CAAC;YAChE,OAAO,CAAC,GAAG,CAAC,+BAA+B,CAAC,CAAC;YAC7C,iBAAiB,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;QAC5C,CAAC;QAED,IAAI,OAAO,CAAC,SAAS,KAAK,MAAM,IAAI,OAAO,CAAC,SAAS,KAAK,MAAM,EAAE,CAAC;YACjE,OAAO,CAAC,GAAG,CAAC,0BAA0B,CAAC,CAAC;YACxC,YAAY,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;QACvC,CAAC;IACH,CAAC;IAED,OAAO,CAAC,GAAG,CAAC,mCAAmC,OAAO,CAAC,SAAS,GAAG,CAAC,CAAC;AACvE,CAAC;AAED,IAAI,EAAE,CAAC,KAAK,CAAC,CAAC,GAAG,EAAE,EAAE;IACnB,OAAO,CAAC,KAAK,CAAC,qBAAqB,EAAE,GAAG,CAAC,CAAC;IAC1C,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC"} \ No newline at end of file diff --git a/compiler/dist/parser.d.ts b/compiler/dist/parser.d.ts index 0aff17b..73f3835 100644 --- a/compiler/dist/parser.d.ts +++ b/compiler/dist/parser.d.ts @@ -20,11 +20,14 @@ export interface OutputNode { } export interface ConstraintNode { type: 'constraint'; + name?: string; expr: string; } export interface VerifyNode { type: 'verify'; + name: string; temporalExpr: string; + depth?: number; } export interface InterfaceNode { type: 'interface'; @@ -133,6 +136,12 @@ export type ActionStatement = { } | { type: 'require'; expr: string; +} | { + type: 'precondition'; + expr: string; +} | { + type: 'postcondition'; + expr: string; } | { type: 'emit'; event: string; @@ -143,6 +152,9 @@ export type ActionStatement = { } | { type: 'return'; expr: string; +} | { + type: 'ifblock'; + raw: string; }; export type MemberNode = ImportNode | InputNode | OutputNode | ConstraintNode | VerifyNode | InterfaceNode | EventNode | ProvenanceNode | ReviewNode | DerivesNode | SatisfiesNode | AuthorNode | SourceNode | BOMNode | StateNode | InitNode | ActionNode; export interface TypeExpr { diff --git a/compiler/dist/parser.d.ts.map b/compiler/dist/parser.d.ts.map index 4d3c72a..2d7dd28 100644 --- a/compiler/dist/parser.d.ts.map +++ b/compiler/dist/parser.d.ts.map @@ -1 +1 @@ -{"version":3,"file":"parser.d.ts","sourceRoot":"","sources":["../src/parser.ts"],"names":[],"mappings":"AAIA,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,UAAU,EAAE,CAAC;CACvB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,IAAI,CAAC,EAAE,MAAM,CAAC;CACf;AAED,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,QAAQ,EAAE,QAAQ,CAAC;CACpB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,QAAQ,EAAE,QAAQ,CAAC;CACpB;AAED,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,YAAY,CAAC;IACnB,IAAI,EAAE,MAAM,CAAC;CACd;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,YAAY,EAAE,MAAM,CAAC;CACtB;AAED,MAAM,WAAW,aAAa;IAC5B,IAAI,EAAE,WAAW,CAAC;IAClB,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,eAAe,EAAE,CAAC;CAC5B;AAED,MAAM,WAAW,eAAe;IAC9B,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,QAAQ,CAAA;KAAE,EAAE,CAAC;IAC3C,UAAU,EAAE,QAAQ,CAAC;CACtB;AAED,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,QAAQ,CAAA;KAAE,EAAE,CAAC;CAC5C;AAED,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,YAAY,CAAC;IACnB,OAAO,EAAE,gBAAgB,EAAE,CAAC;CAC7B;AAED,MAAM,WAAW,gBAAgB;IAC/B,IAAI,EAAE,YAAY,GAAG,iBAAiB,GAAG,aAAa,GAAG,cAAc,CAAC;IACxE,KAAK,EAAE,MAAM,CAAC;IACd,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,QAAQ,GAAG,MAAM,GAAG,QAAQ,CAAC;CACpC;AAED,MAAM,WAAW,WAAW;IAC1B,IAAI,EAAE,SAAS,CAAC;IAChB,IAAI,EAAE,MAAM,CAAC;IACb,GAAG,CAAC,EAAE,MAAM,CAAC;CACd;AAED,MAAM,WAAW,aAAa;IAC5B,IAAI,EAAE,WAAW,CAAC;IAClB,WAAW,EAAE,MAAM,CAAC;IACpB,MAAM,CAAC,EAAE,MAAM,CAAC;CACjB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,cAAc,GAAG,SAAS,GAAG,UAAU,GAAG,YAAY,GACrD,qBAAqB,GAAG,cAAc,GAAG,kBAAkB,CAAC;IACnE,GAAG,CAAC,EAAE,MAAM,CAAC;CACd;AAED,MAAM,WAAW,OAAO;IACtB,IAAI,EAAE,KAAK,CAAC;IACZ,QAAQ,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,OAAO,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IAC9C,MAAM,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,OAAO,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IAC5C,OAAO,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,OAAO,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IAC7C,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,IAAI,CAAC,EAAE,MAAM,CAAC;CACf;AAGD,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,SAAS,EAAE,QAAQ,EAAE,CAAC;CACvB;AAED,MAAM,WAAW,QAAQ;IACvB,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,QAAQ,CAAC;IACnB,WAAW,CAAC,EAAE,MAAM,CAAC;CACtB;AAED,MAAM,WAAW,QAAQ;IACvB,IAAI,EAAE,MAAM,CAAC;IACb,WAAW,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAA;KAAE,EAAE,CAAC;CAC/C;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,QAAQ,CAAA;KAAE,EAAE,CAAC;IAC3C,UAAU,EAAE,eAAe,EAAE,CAAC;CAC/B;AAED,MAAM,MAAM,eAAe,GACvB;IAAE,IAAI,EAAE,QAAQ,CAAC;IAAC,MAAM,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GAChD;IAAE,IAAI,EAAE,KAAK,CAAC;IAAC,IAAI,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GAC3C;IAAE,IAAI,EAAE,SAAS,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GACjC;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,KAAK,EAAE,MAAM,CAAC;IAAC,MAAM,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAA;KAAE,EAAE,CAAA;CAAE,GAC1E;IAAE,IAAI,EAAE,QAAQ,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,CAAC;AAErC,MAAM,MAAM,UAAU,GAClB,UAAU,GACV,SAAS,GACT,UAAU,GACV,cAAc,GACd,UAAU,GACV,aAAa,GACb,SAAS,GACT,cAAc,GACd,UAAU,GACV,WAAW,GACX,aAAa,GACb,UAAU,GACV,UAAU,GACV,OAAO,GACP,SAAS,GACT,QAAQ,GACR,UAAU,CAAC;AAEf,MAAM,WAAW,QAAQ;IACvB,IAAI,EAAE,WAAW,GAAG,QAAQ,GAAG,MAAM,GAAG,KAAK,GAAG,KAAK,GAAG,OAAO,CAAC;IAChE,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,MAAM,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,QAAQ,CAAA;KAAE,EAAE,CAAC;IAC5C,WAAW,CAAC,EAAE,QAAQ,CAAC;IACvB,OAAO,CAAC,EAAE,QAAQ,CAAC;IACnB,SAAS,CAAC,EAAE,QAAQ,CAAC;CACtB;AAED,MAAM,WAAW,GAAG;IAClB,MAAM,EAAE,SAAS,EAAE,CAAC;CACrB;AAED;;GAEG;AACH,wBAAgB,cAAc,CAAC,QAAQ,EAAE,MAAM,GAAG,GAAG,CAGpD;AAED;;GAEG;AACH,wBAAgB,iBAAiB,CAAC,OAAO,EAAE,MAAM,EAAE,QAAQ,CAAC,EAAE,MAAM,GAAG,GAAG,CAwCzE"} \ No newline at end of file +{"version":3,"file":"parser.d.ts","sourceRoot":"","sources":["../src/parser.ts"],"names":[],"mappings":"AAIA,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,UAAU,EAAE,CAAC;CACvB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,IAAI,CAAC,EAAE,MAAM,CAAC;CACf;AAED,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,QAAQ,EAAE,QAAQ,CAAC;CACpB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,QAAQ,EAAE,QAAQ,CAAC;CACpB;AAED,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,YAAY,CAAC;IACnB,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;CACd;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,YAAY,EAAE,MAAM,CAAC;IACrB,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,WAAW,aAAa;IAC5B,IAAI,EAAE,WAAW,CAAC;IAClB,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,eAAe,EAAE,CAAC;CAC5B;AAED,MAAM,WAAW,eAAe;IAC9B,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,QAAQ,CAAA;KAAE,EAAE,CAAC;IAC3C,UAAU,EAAE,QAAQ,CAAC;CACtB;AAED,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,QAAQ,CAAA;KAAE,EAAE,CAAC;CAC5C;AAED,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,YAAY,CAAC;IACnB,OAAO,EAAE,gBAAgB,EAAE,CAAC;CAC7B;AAED,MAAM,WAAW,gBAAgB;IAC/B,IAAI,EAAE,YAAY,GAAG,iBAAiB,GAAG,aAAa,GAAG,cAAc,CAAC;IACxE,KAAK,EAAE,MAAM,CAAC;IACd,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,QAAQ,GAAG,MAAM,GAAG,QAAQ,CAAC;CACpC;AAED,MAAM,WAAW,WAAW;IAC1B,IAAI,EAAE,SAAS,CAAC;IAChB,IAAI,EAAE,MAAM,CAAC;IACb,GAAG,CAAC,EAAE,MAAM,CAAC;CACd;AAED,MAAM,WAAW,aAAa;IAC5B,IAAI,EAAE,WAAW,CAAC;IAClB,WAAW,EAAE,MAAM,CAAC;IACpB,MAAM,CAAC,EAAE,MAAM,CAAC;CACjB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,cAAc,GAAG,SAAS,GAAG,UAAU,GAAG,YAAY,GACrD,qBAAqB,GAAG,cAAc,GAAG,kBAAkB,CAAC;IACnE,GAAG,CAAC,EAAE,MAAM,CAAC;CACd;AAED,MAAM,WAAW,OAAO;IACtB,IAAI,EAAE,KAAK,CAAC;IACZ,QAAQ,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,OAAO,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IAC9C,MAAM,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,OAAO,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IAC5C,OAAO,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,OAAO,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IAC7C,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,IAAI,CAAC,EAAE,MAAM,CAAC;CACf;AAGD,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,SAAS,EAAE,QAAQ,EAAE,CAAC;CACvB;AAED,MAAM,WAAW,QAAQ;IACvB,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,QAAQ,CAAC;IACnB,WAAW,CAAC,EAAE,MAAM,CAAC;CACtB;AAED,MAAM,WAAW,QAAQ;IACvB,IAAI,EAAE,MAAM,CAAC;IACb,WAAW,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAA;KAAE,EAAE,CAAC;CAC/C;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,QAAQ,CAAA;KAAE,EAAE,CAAC;IAC3C,UAAU,EAAE,eAAe,EAAE,CAAC;CAC/B;AAED,MAAM,MAAM,eAAe,GACvB;IAAE,IAAI,EAAE,QAAQ,CAAC;IAAC,MAAM,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GAChD;IAAE,IAAI,EAAE,KAAK,CAAC;IAAC,IAAI,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GAC3C;IAAE,IAAI,EAAE,SAAS,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GACjC;IAAE,IAAI,EAAE,cAAc,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GACtC;IAAE,IAAI,EAAE,eAAe,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GACvC;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,KAAK,EAAE,MAAM,CAAC;IAAC,MAAM,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAA;KAAE,EAAE,CAAA;CAAE,GAC1E;IAAE,IAAI,EAAE,QAAQ,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GAChC;IAAE,IAAI,EAAE,SAAS,CAAC;IAAC,GAAG,EAAE,MAAM,CAAA;CAAE,CAAC;AAErC,MAAM,MAAM,UAAU,GAClB,UAAU,GACV,SAAS,GACT,UAAU,GACV,cAAc,GACd,UAAU,GACV,aAAa,GACb,SAAS,GACT,cAAc,GACd,UAAU,GACV,WAAW,GACX,aAAa,GACb,UAAU,GACV,UAAU,GACV,OAAO,GACP,SAAS,GACT,QAAQ,GACR,UAAU,CAAC;AAEf,MAAM,WAAW,QAAQ;IACvB,IAAI,EAAE,WAAW,GAAG,QAAQ,GAAG,MAAM,GAAG,KAAK,GAAG,KAAK,GAAG,OAAO,CAAC;IAChE,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,MAAM,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,QAAQ,CAAA;KAAE,EAAE,CAAC;IAC5C,WAAW,CAAC,EAAE,QAAQ,CAAC;IACvB,OAAO,CAAC,EAAE,QAAQ,CAAC;IACnB,SAAS,CAAC,EAAE,QAAQ,CAAC;CACtB;AAED,MAAM,WAAW,GAAG;IAClB,MAAM,EAAE,SAAS,EAAE,CAAC;CACrB;AAED;;GAEG;AACH,wBAAgB,cAAc,CAAC,QAAQ,EAAE,MAAM,GAAG,GAAG,CAGpD;AAED;;GAEG;AACH,wBAAgB,iBAAiB,CAAC,OAAO,EAAE,MAAM,EAAE,QAAQ,CAAC,EAAE,MAAM,GAAG,GAAG,CAwCzE"} \ No newline at end of file diff --git a/compiler/dist/parser.js b/compiler/dist/parser.js index 0bfe248..ba68502 100644 --- a/compiler/dist/parser.js +++ b/compiler/dist/parser.js @@ -63,8 +63,9 @@ function parseSpeck(lines, startIndex) { // Count braces on this line first const openCount = (rawLine.match(/{/g) || []).length; const closeCount = (rawLine.match(/}/g) || []).length; - // Check for block-opening members - const blockStarters = ['state:', 'init:', 'action ', 'event ', 'provenance ', 'bom ', 'interface ']; + // Check for block-opening members (those that open braces on the same line) + const blockStarters = ['state:', 'init:', 'action ', 'event ', 'provenance ', 'bom ', 'interface ', + 'state {', 'init {', 'verify ', 'constraint ']; const isBlockStarter = blockStarters.some(s => line.startsWith(s)); if (isBlockStarter && openCount > 0) { // Multi-line block: parse it and skip to its end @@ -72,9 +73,39 @@ function parseSpeck(lines, startIndex) { if (member) { members.push(member); } - // Skip past this block - const endIdx = findBlockEnd(lines, i + 1, braceCount + openCount); - i = endIdx + 1; + // Skip past this block if it spans multiple lines + if (closeCount < openCount) { + const endIdx = findBlockEnd(lines, i + 1, braceCount + openCount); + i = endIdx + 1; + } + else { + i++; + } + continue; + } + // Handle constraint/verify blocks that may span multiple lines without braces + // (e.g. constraint Name: forall r in [0..4]:) + if (line.startsWith('constraint ') || line.startsWith('verify ')) { + const member = parseMultiLineConstraintVerify(lines, i); + if (member) { + members.push(member); + // Skip past the consumed continuation lines + while (i + 1 < lines.length) { + const nextLine = lines[i + 1].trim(); + if (!nextLine || nextLine.startsWith('//') || nextLine.startsWith('/*') || + nextLine.startsWith('constraint ') || nextLine.startsWith('verify ') || + nextLine.startsWith('constraint:') || nextLine.startsWith('verify:') || + nextLine.startsWith('state') || nextLine.startsWith('init') || + nextLine.startsWith('action ') || nextLine.startsWith('event ') || + nextLine.startsWith('interface ') || nextLine.startsWith('provenance ') || + nextLine.startsWith('bom ') || nextLine.startsWith('}') || + nextLine.startsWith('next:') || nextLine.startsWith('invariant ')) { + break; + } + i++; + } + } + i++; continue; } // Single-line: update brace count, then parse if still inside @@ -110,15 +141,40 @@ function findBlockEnd(lines, startIndex, startBraceCount) { } return i; } +// Check if an expression has balanced braces, parens, and brackets +function isExpressionComplete(expr) { + let braces = 0, parens = 0, brackets = 0; + for (const char of expr) { + if (char === '{') + braces++; + else if (char === '}') + braces--; + else if (char === '(') + parens++; + else if (char === ')') + parens--; + else if (char === '[') + brackets++; + else if (char === ']') + brackets--; + } + return braces === 0 && parens === 0 && brackets === 0; +} function parseMemberBlock(lines, startIndex) { const firstLine = lines[startIndex].trim(); const startBraceCount = (firstLine.match(/{/g) || []).length; if (firstLine.startsWith('state:')) { return parseStateBlockMultiline(lines, startIndex, startBraceCount); } + if (firstLine.startsWith('state {') || firstLine === 'state') { + return parseStateBlockMultiline(lines, startIndex, startBraceCount); + } if (firstLine.startsWith('init:')) { return parseInitBlockMultiline(lines, startIndex, startBraceCount); } + if (firstLine.startsWith('init {') || firstLine === 'init') { + return parseInitBlockMultiline(lines, startIndex, startBraceCount); + } if (firstLine.startsWith('action ')) { return parseActionBlockMultiline(lines, startIndex, startBraceCount); } @@ -134,6 +190,12 @@ function parseMemberBlock(lines, startIndex) { if (firstLine.startsWith('bom ')) { return { type: 'bom' }; } + if (firstLine.startsWith('verify ')) { + return parseVerifyBlockMultiline(lines, startIndex, startBraceCount); + } + if (firstLine.startsWith('constraint ')) { + return parseConstraintBlockMultiline(lines, startIndex, startBraceCount); + } return null; } function parseMember(line) { @@ -147,13 +209,25 @@ function parseMember(line) { const typeExpr = parseTypeExpr(line.substring(7).trim()); return { type: 'output', typeExpr }; } - // constraint: + // constraint Name: expr (single-line named constraint) + if (line.startsWith('constraint ')) { + const match = line.match(/^constraint\s+(\w+):\s*(.+)$/); + if (match) { + return { type: 'constraint', name: match[1], expr: match[2].trim() }; + } + return null; // multi-line constraint block (handled by parseMemberBlock) + } + // constraint: expr (unnamed, single-line) if (line.startsWith('constraint:')) { return { type: 'constraint', expr: line.substring(11).trim() }; } - // verify: + // verify Always(Name) { depth N } or verify Always(Name) (single-line forms) + if (line.startsWith('verify ')) { + return parseVerifySingleLine(line); + } + // verify: expr (old unnamed single-line) if (line.startsWith('verify:')) { - return { type: 'verify', temporalExpr: line.substring(7).trim() }; + return { type: 'verify', name: 'unnamed', temporalExpr: line.substring(7).trim() }; } // interface: if (line.startsWith('interface ')) { @@ -188,7 +262,7 @@ function parseMember(line) { if (line.startsWith('source:')) { return parseSource(line); } - // state: + // state: or state { if (line.startsWith('state:')) { const varsStr = line.substring(6).trim(); // state: { ... } - single-line form @@ -199,7 +273,18 @@ function parseMember(line) { // state: { - multi-line opening (handled by parseStateMultiline) return { type: 'state', variables: [] }; } - // init: + if (line.startsWith('state {') || line === 'state') { + // state { ... } inline or state { on its own line (handled by parseMemberBlock) + if (line.startsWith('state {')) { + const inner = line.substring(6).trim(); + const content = inner.startsWith('{') ? inner.slice(1).trim() : inner; + if (content.endsWith('}')) { + return parseStateBlock(content.slice(0, -1).trim()); + } + } + return { type: 'state', variables: [] }; + } + // init: or init { if (line.startsWith('init:')) { const initStr = line.substring(5).trim(); if (initStr.startsWith('{') && initStr.endsWith('}')) { @@ -208,6 +293,17 @@ function parseMember(line) { } return { type: 'init', assignments: [] }; } + if (line.startsWith('init {') || line === 'init') { + // init { ... } inline + if (line.startsWith('init {')) { + const inner = line.substring(5).trim(); + const content = inner.startsWith('{') ? inner.slice(1).trim() : inner; + if (content.endsWith('}')) { + return parseInitBlock(content.slice(0, -1).trim()); + } + } + return { type: 'init', assignments: [] }; + } // action keyword if (line.startsWith('action ')) { return parseActionHeader(line); @@ -229,18 +325,28 @@ function parseStateBlock(inner) { for (const vStr of varStrs) { const withDefault = vStr.match(/^(\w+)\s*:\s*(.+?)\s*:=\s*(.+)$/); if (withDefault) { + // Strip trailing comments from type expression + let typeStr = withDefault[2].trim(); + const commentIdx = typeStr.indexOf('//'); + if (commentIdx >= 0) + typeStr = typeStr.substring(0, commentIdx).trim(); variables.push({ name: withDefault[1], - typeExpr: parseTypeExpr(withDefault[2].trim()), + typeExpr: parseTypeExpr(typeStr), defaultInit: withDefault[3].trim(), }); continue; } const withoutDefault = vStr.match(/^(\w+)\s*:\s*(.+)$/); if (withoutDefault) { + // Strip trailing comments from type expression + let typeStr = withoutDefault[2].trim(); + const commentIdx = typeStr.indexOf('//'); + if (commentIdx >= 0) + typeStr = typeStr.substring(0, commentIdx).trim(); variables.push({ name: withoutDefault[1], - typeExpr: parseTypeExpr(withoutDefault[2].trim()), + typeExpr: parseTypeExpr(typeStr), }); } } @@ -311,15 +417,58 @@ function parseTypeExpr(expr) { // Map if (expr.startsWith('Map<') && expr.endsWith('>')) { const inner = expr.substring(4, expr.length - 1); - // Find the first-level comma (not inside nested generics) + // Find the first-level comma (not inside nested generics or record literals) let depth = 0; + let braceDepth = 0; let commaIdx = -1; for (let i = 0; i < inner.length; i++) { - if (inner[i] === '<') + if (inner[i] === '<' || inner[i] === '(') depth++; - else if (inner[i] === '>') + else if (inner[i] === '>' || inner[i] === ')') depth--; - else if (inner[i] === ',' && depth === 0) { + else if (inner[i] === '{') + braceDepth++; + else if (inner[i] === '}') + braceDepth--; + else if (inner[i] === ',' && depth === 0 && braceDepth === 0) { + commaIdx = i; + break; + } + } + if (commaIdx !== -1) { + const keyStr = inner.substring(0, commaIdx).trim(); + const valStr = inner.substring(commaIdx + 1).trim(); + return { type: 'map', keyType: parseTypeExpr(keyStr), valueType: parseTypeExpr(valStr) }; + } + // Fallback: treat as ident if malformed + return { type: 'ident', name: expr }; + } + // List(T) — parenthetical syntax (SpeckDL canonical form) + if (expr.startsWith('List(') && expr.endsWith(')')) { + const inner = expr.substring(5, expr.length - 1).trim(); + return { type: 'list', elementType: parseTypeExpr(inner) }; + } + // Set(T) — parenthetical syntax (SpeckDL canonical form) + if (expr.startsWith('Set(') && expr.endsWith(')')) { + const inner = expr.substring(4, expr.length - 1).trim(); + return { type: 'set', elementType: parseTypeExpr(inner) }; + } + // Map(K, V) — parenthetical syntax (SpeckDL canonical form) + if (expr.startsWith('Map(') && expr.endsWith(')')) { + const inner = expr.substring(4, expr.length - 1); + let depth = 0; + let braceDepth = 0; + let commaIdx = -1; + for (let i = 0; i < inner.length; i++) { + if (inner[i] === '(' || inner[i] === '<') + depth++; + else if (inner[i] === ')' || inner[i] === '>') + depth--; + else if (inner[i] === '{') + braceDepth++; + else if (inner[i] === '}') + braceDepth--; + else if (inner[i] === ',' && depth === 0 && braceDepth === 0) { commaIdx = i; break; } @@ -463,13 +612,42 @@ function parseSource(line) { function parseStateBlockMultiline(lines, startIndex, startBraceCount) { const endIndex = findBlockEnd(lines, startIndex + 1, startBraceCount); const innerLines = lines.slice(startIndex + 1, endIndex).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); - const inner = innerLines.join('; '); + // State blocks separate fields with commas or newlines/semicolons. + // Strip trailing commas (they are field separators, not continuations) + // then join with semicolons between complete variable declarations. + const cleanedLines = innerLines.map(l => l.replace(/,+\s*$/, '')); + const inner = cleanedLines.reduce((acc, line, i) => { + if (i === 0) + return line; + const prevLine = cleanedLines[i - 1]; + // Check if this line starts a new variable declaration (name: type) + const isNewVar = /^\w+\s*:/.test(line); + // Check if prev line ends with a continuation character (brace or paren, NOT comma) + const prevIsContinuation = /[{(]\s*$/.test(prevLine); + if (isNewVar && !prevIsContinuation) { + return acc + '; ' + line; + } + return acc + ' ' + line; + }, ''); return parseStateBlock(inner); } function parseInitBlockMultiline(lines, startIndex, startBraceCount) { const endIndex = findBlockEnd(lines, startIndex + 1, startBraceCount); const innerLines = lines.slice(startIndex + 1, endIndex).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); - const inner = innerLines.join('; '); + // Init blocks separate fields with commas or newlines/semicolons. + // Strip trailing commas and join with semicolons. + const cleanedLines = innerLines.map(l => l.replace(/,+\s*$/, '')); + const inner = cleanedLines.reduce((acc, line, i) => { + if (i === 0) + return line; + const prevLine = cleanedLines[i - 1]; + const isNewVar = /^\w+\s*:/.test(line); + const prevIsContinuation = /[{(]\s*$/.test(prevLine); + if (isNewVar && !prevIsContinuation) { + return acc + '; ' + line; + } + return acc + ' ' + line; + }, ''); return parseInitBlock(inner); } function parseActionBlockMultiline(lines, startIndex, startBraceCount) { @@ -478,12 +656,78 @@ function parseActionBlockMultiline(lines, startIndex, startBraceCount) { if (!header) return null; const endIndex = findBlockEnd(lines, startIndex + 1, startBraceCount); - const bodyLines = lines.slice(startIndex + 1, endIndex).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); + // Keep blank lines as null to separate statements, filter only comments + const bodyLinesRaw = lines.slice(startIndex + 1, endIndex).map(l => { + const trimmed = l.trim(); + if (!trimmed || trimmed.startsWith('//') || trimmed.startsWith('/*')) + return null; + return trimmed; + }); + // Collapse consecutive nulls and strip trailing nulls + const bodyLines = []; + for (const line of bodyLinesRaw) { + if (line !== null || (bodyLines.length > 0 && bodyLines[bodyLines.length - 1] !== null)) { + bodyLines.push(line); + } + } + // Remove trailing nulls + while (bodyLines.length > 0 && bodyLines[bodyLines.length - 1] === null) { + bodyLines.pop(); + } const statements = []; - for (const stmt of bodyLines) { + for (let i = 0; i < bodyLines.length; i++) { + const stmtOrNull = bodyLines[i]; + // Skip blank line separators + if (stmtOrNull === null) + continue; + const stmt = stmtOrNull; + // precondition: expr + if (stmt.startsWith('precondition:')) { + let expr = stmt.substring(13).trim(); + // support 'or' continuation lines + while (i + 1 < bodyLines.length) { + const nextLine = bodyLines[i + 1]; + if (nextLine === null || !nextLine.startsWith('or ')) + break; + i++; + expr += ' || ' + nextLine.substring(3).trim(); + } + statements.push({ type: 'precondition', expr }); + continue; + } + // postcondition: expr + if (stmt.startsWith('postcondition:')) { + let expr = stmt.substring(14).trim(); + // support 'or' continuation lines + while (i + 1 < bodyLines.length) { + const nextLine = bodyLines[i + 1]; + if (nextLine === null || !nextLine.startsWith('or ')) + break; + i++; + expr += ' || ' + nextLine.substring(3).trim(); + } + statements.push({ type: 'postcondition', expr }); + continue; + } // require expr if (stmt.startsWith('require ')) { - statements.push({ type: 'require', expr: stmt.substring(8).trim() }); + let expr = stmt.substring(8).trim(); + // support multi-line continuation (forall, or, etc.) + while (i + 1 < bodyLines.length) { + const nextLine = bodyLines[i + 1]; + if (nextLine === null) + break; + // Stop if next line starts a new statement keyword + if (nextLine.startsWith('precondition:') || nextLine.startsWith('postcondition:') || + nextLine.startsWith('require ') || nextLine.startsWith('return ') || + nextLine.startsWith('emit ') || nextLine.startsWith('let ') || + nextLine.includes(':=')) { + break; + } + i++; + expr += ' ' + nextLine; + } + statements.push({ type: 'require', expr }); continue; } // return expr @@ -498,7 +742,7 @@ function parseActionBlockMultiline(lines, startIndex, startBraceCount) { const fieldsStr = emitMatch[2]; const fields = []; if (fieldsStr) { - const fieldPairs = fieldsStr.split(',').map(s => s.trim()).filter(Boolean); + const fieldPairs = splitFields(fieldsStr); for (const fp of fieldPairs) { const colonIdx = fp.indexOf(':'); if (colonIdx > 0) { @@ -512,16 +756,52 @@ function parseActionBlockMultiline(lines, startIndex, startBraceCount) { statements.push({ type: 'emit', event: eventName, fields }); continue; } + // if condition { ... } else if condition { ... } else { ... } + // SpeckDL conditional blocks - parse as a special statement type + if (stmt.startsWith('if ')) { + // Find matching } for this if block (may span multiple bodyLines entries) + let braceDepth = 0; + let j = i; + const blockLines = []; + while (j < bodyLines.length) { + const line = bodyLines[j]; + if (line === null) { + j++; + continue; + } + blockLines.push(line); + for (const ch of line) { + if (ch === '{') + braceDepth++; + else if (ch === '}') + braceDepth--; + } + j++; + if (braceDepth === 0) + break; + } + // For now, treat as a raw 'ifblock' statement that the generator can handle + statements.push({ type: 'ifblock', raw: blockLines.join(' ') }); + i = j - 1; // skip to end of if block + continue; + } // let name := expr const letMatch = stmt.match(/^let\s+(\w+)\s*:=\s*(.+)$/); if (letMatch) { statements.push({ type: 'let', name: letMatch[1], expr: letMatch[2].trim() }); continue; } - // assignment: target := expr + // assignment: target := expr (may span multiple lines) const assignMatch = stmt.match(/^(\w+(?:\[.*?\])?)\s*:=\s*(.+)$/); if (assignMatch) { - statements.push({ type: 'assign', target: assignMatch[1], expr: assignMatch[2].trim() }); + let target = assignMatch[1]; + let expr = assignMatch[2].trim(); + // Handle multiline expressions: join subsequent lines until braces/parens are balanced + while (i + 1 < bodyLines.length && !isExpressionComplete(expr)) { + i++; + expr += ' ' + bodyLines[i]; + } + statements.push({ type: 'assign', target, expr }); } } return { ...header, statements }; @@ -547,25 +827,266 @@ function parseInterfaceBlockMultiline(lines, startIndex, startBraceCount) { const innerLines = lines.slice(startIndex + 1, endIndex).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); const methods = []; for (const line of innerLines) { - // name(params) -> returnType - const match = line.match(/^(\w+)\s*\(([^)]*)\)\s*-\u003e\s*(.+?),?$/); - if (match) { - const name = match[1]; - const paramsStr = match[2]; - const returnTypeStr = match[3].replace(/,$/, '').trim(); - const params = []; - if (paramsStr.trim()) { - const paramPairs = paramsStr.split(',').map(s => s.trim()).filter(Boolean); - for (const p of paramPairs) { - const pMatch = p.match(/^(\w+)\s*:\s*(.+)$/); - if (pMatch) { - params.push({ name: pMatch[1], type: parseTypeExpr(pMatch[2].trim()) }); + // name-only: enum-style values (e.g., "Idle", "Idle, Parsing" comma-separated) + // Split on comma and take each trimmed token + const tokens = line.split(',').map(s => s.trim()).filter(Boolean); + for (const token of tokens) { + // Check if it’s a name(params) -> returnType method signature + const methodMatch = token.match(/^(\w+)\s*\(([^)]*)\)\s*->\s*(.+?),?$/); + if (methodMatch) { + const name = methodMatch[1]; + const paramsStr = methodMatch[2]; + const returnTypeStr = methodMatch[3].replace(/,$/, '').trim(); + const params = []; + if (paramsStr.trim()) { + const paramPairs = paramsStr.split(',').map(s => s.trim()).filter(Boolean); + for (const p of paramPairs) { + const pMatch = p.match(/^(\w+)\s*:\s*(.+)$/); + if (pMatch) { + params.push({ name: pMatch[1], type: parseTypeExpr(pMatch[2].trim()) }); + } } } + methods.push({ name, params, returnType: parseTypeExpr(returnTypeStr) }); + } + else if (/^\w+$/.test(token)) { + // Name-only (enum-style) — treat as a method with no params and void return + methods.push({ name: token, params: [], returnType: parseTypeExpr('Void') }); } - methods.push({ name, params, returnType: parseTypeExpr(returnTypeStr) }); } } return { ...header, methods }; } +// --- Multi-line constraint/verify parsing (no braces needed) --- +/** + * Parse a constraint or verify that may span multiple lines without braces. + * Used for forall-style constraints like: + * constraint Name: forall r in [0..4]: + * (expr) + * + * and verify blocks like: + * verify Always(Name) + * verify Always(Name) { depth N } + */ +function parseMultiLineConstraintVerify(lines, startIndex) { + const line = lines[startIndex].trim(); + if (line.startsWith('verify ')) { + return parseVerifyAnyForm(lines, startIndex); + } + if (line.startsWith('constraint ')) { + return parseConstraintAnyForm(lines, startIndex); + } + return null; +} +function parseConstraintAnyForm(lines, startIndex) { + const line = lines[startIndex].trim(); + const namedMatch = line.match(/^constraint\s+(\w+):(.*)$/); + if (!namedMatch) + return null; + const name = namedMatch[1]; + let expr = namedMatch[2].trim(); + // If opens a brace, treat as block + if (expr.startsWith('{')) { + const afterBrace = expr.substring(1).trim(); + const openCount = (line.match(/{/g) || []).length; + const endIdx = findBlockEnd(lines, startIndex + 1, openCount); + const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); + const inner = innerLines.join(' '); + expr = (afterBrace + ' ' + inner).trim(); + } + else { + // No braces — but might span multiple lines (forall continuation) + const continuationLines = []; + for (let j = startIndex + 1; j < lines.length; j++) { + const nl = lines[j].trim(); + if (!nl || nl.startsWith('//') || nl.startsWith('/*') || + nl.startsWith('constraint ') || nl.startsWith('verify ') || + nl.startsWith('constraint:') || nl.startsWith('verify:') || + nl.startsWith('state') || nl.startsWith('init') || + nl.startsWith('action ') || nl.startsWith('event ') || + nl.startsWith('interface ') || nl.startsWith('provenance ') || + nl.startsWith('bom ') || nl.startsWith('}') || + nl.startsWith('next:') || nl.startsWith('invariant ')) { + break; + } + continuationLines.push(nl); + } + if (continuationLines.length > 0) { + expr = expr + ' ' + continuationLines.join(' '); + } + } + return { type: 'constraint', name, expr: expr.trim() }; +} +function parseVerifyAnyForm(lines, startIndex) { + const line = lines[startIndex].trim(); + // Single-line: verify Always(Name) { depth N } + const fullMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*\{\s*depth\s+(\d+)\s*\}/); + if (fullMatch) { + return { + type: 'verify', + name: fullMatch[2], + temporalExpr: fullMatch[1].trim(), + depth: parseInt(fullMatch[3], 10), + }; + } + // Multi-line: verify Always(Name) { ... } + if (line.includes('{')) { + const headerMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*\{/); + if (headerMatch) { + const temporalExpr = headerMatch[1].trim(); + const name = headerMatch[2]; + const openCount = (line.match(/{/g) || []).length; + const endIdx = findBlockEnd(lines, startIndex + 1, openCount); + const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); + const inner = innerLines.join(' '); + const afterBrace = line.substring(line.indexOf('{') + 1).trim(); + const combined = (afterBrace + ' ' + inner).trim(); + const depthMatch = combined.match(/\bdepth\s+(\d+)\b/); + const depth = depthMatch ? parseInt(depthMatch[1], 10) : undefined; + return { type: 'verify', name, temporalExpr, depth }; + } + } + // Simple: verify Always(Name) + const simpleMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*$/); + if (simpleMatch) { + return { + type: 'verify', + name: simpleMatch[2], + temporalExpr: simpleMatch[1].trim(), + }; + } + return null; +} +// --- Verify/Constraint block parsing --- +/** + * Parse a verify block opened with `verify Always(Name) { ... }` or similar. + * The first line already contains the header; we need to parse the body inside { }. + */ +function parseVerifyBlockMultiline(lines, startIndex, startBraceCount) { + const firstLine = lines[startIndex].trim(); + // Match: verify Always(Name) { depth N } or verify Always(Name) { + const headerMatch = firstLine.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*\{/); + if (!headerMatch) + return null; + const temporalExpr = headerMatch[1].trim(); + const name = headerMatch[2]; + // Check if the brace closes on the same line (single-line form) + const openBraceCount = (firstLine.match(/{/g) || []).length; + const closeBraceCount = (firstLine.match(/}/g) || []).length; + if (closeBraceCount >= openBraceCount) { + // Single-line: parse depth from same line + const afterBrace = firstLine.substring(firstLine.indexOf('{') + 1); + const beforeClose = afterBrace.substring(0, afterBrace.lastIndexOf('}')).trim(); + const depthMatch = beforeClose.match(/\bdepth\s+(\d+)\b/); + const depth = depthMatch ? parseInt(depthMatch[1], 10) : undefined; + return { type: 'verify', name, temporalExpr, depth }; + } + // Multi-line block + let depth; + const endIdx = findBlockEnd(lines, startIndex + 1, startBraceCount); + const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); + const inner = innerLines.join(' '); + const afterBrace = firstLine.substring(firstLine.indexOf('{') + 1).trim(); + const depthMatch = (afterBrace + ' ' + inner).match(/\bdepth\s+(\d+)\b/); + if (depthMatch) { + depth = parseInt(depthMatch[1], 10); + } + return { type: 'verify', name, temporalExpr, depth }; +} +/** + * Parse a single-line verify statement: verify Always(Name) { depth N } + */ +function parseVerifySingleLine(line) { + // verify Always(Name) { depth N } + const fullMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*\{\s*depth\s+(\d+)\s*\}/); + if (fullMatch) { + return { + type: 'verify', + name: fullMatch[2], + temporalExpr: fullMatch[1].trim(), + depth: parseInt(fullMatch[3], 10), + }; + } + // verify Always(Name) (no depth, no braces — just a statement) + const simpleMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*$/); + if (simpleMatch) { + return { + type: 'verify', + name: simpleMatch[2], + temporalExpr: simpleMatch[1].trim(), + }; + } + return null; +} +/** + * Parse a constraint block opened with `constraint Name: ...` which may + * be multi-line (e.g. forall spanning multiple lines). + */ +function parseConstraintBlockMultiline(lines, startIndex, startBraceCount) { + const firstLine = lines[startIndex].trim(); + // Match: constraint Name: expr (possibly opening a {} block, or just a forall on next line) + // For the SQLiteWAL style: constraint Name: expr (single line, no braces needed) + const namedMatch = firstLine.match(/^constraint\s+(\w+):(.*)$/); + if (namedMatch) { + const name = namedMatch[1]; + let expr = namedMatch[2].trim(); + // If the expression opens a brace, collect the block + if (expr.startsWith('{')) { + const endIdx = findBlockEnd(lines, startIndex + 1, startBraceCount); + const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); + const inner = innerLines.join(' '); + // Combine: content after { on first line + content from inner lines + const afterBrace = expr.substring(1).trim(); + expr = (afterBrace + ' ' + inner).trim(); + } + else { + // No braces, but might still be multiline (forall spanning multiple lines) + // Check if next line(s) continue the expression + const endIdx = findBlockEnd(lines, startIndex + 1, startBraceCount); + const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); + if (innerLines.length > 0) { + expr = expr + ' ' + innerLines.join(' '); + } + } + return { type: 'constraint', name, expr: expr.trim() }; + } + return null; +} +/** Split on commas while respecting parenthesized groups and quoted strings */ +function splitFields(s) { + const result = []; + let depth = 0; + let inDouble = false; + let inSingle = false; + let start = 0; + for (let i = 0; i < s.length; i++) { + const ch = s[i]; + if (ch === '"' && !inSingle) { + inDouble = !inDouble; + continue; + } + if (ch === "'" && !inDouble) { + inSingle = !inSingle; + continue; + } + if (inDouble || inSingle) + continue; + if (ch === '(' || ch === '{') { + depth++; + continue; + } + if (ch === ')' || ch === '}') { + depth--; + continue; + } + if (ch === ',' && depth === 0) { + result.push(s.substring(start, i).trim()); + start = i + 1; + } + } + const last = s.substring(start).trim(); + if (last) + result.push(last); + return result; +} //# sourceMappingURL=parser.js.map \ No newline at end of file diff --git a/compiler/dist/parser.js.map b/compiler/dist/parser.js.map index 1393241..70e270a 100644 --- a/compiler/dist/parser.js.map +++ b/compiler/dist/parser.js.map @@ -1 +1 @@ -{"version":3,"file":"parser.js","sourceRoot":"","sources":["../src/parser.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,MAAM,IAAI,CAAC;AAyKpB;;GAEG;AACH,MAAM,UAAU,cAAc,CAAC,QAAgB;IAC7C,MAAM,OAAO,GAAG,EAAE,CAAC,YAAY,CAAC,QAAQ,EAAE,OAAO,CAAC,CAAC;IACnD,OAAO,iBAAiB,CAAC,OAAO,EAAE,QAAQ,CAAC,CAAC;AAC9C,CAAC;AAED;;GAEG;AACH,MAAM,UAAU,iBAAiB,CAAC,OAAe,EAAE,QAAiB;IAClE,MAAM,MAAM,GAAgB,EAAE,CAAC;IAC/B,MAAM,KAAK,GAAG,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IAElC,IAAI,CAAC,GAAG,CAAC,CAAC;IACV,OAAO,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC;QACxB,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QAE7B,gCAAgC;QAChC,IAAI,CAAC,IAAI,IAAI,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC;YAC5D,CAAC,EAAE,CAAC;YACJ,SAAS;QACX,CAAC;QAED,2BAA2B;QAC3B,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;YAC/B,MAAM,UAAU,GAAG,oBAAoB,CAAC,IAAI,CAAC,CAAC;YAC9C,sDAAsD;YACtD,gDAAgD;YAChD,CAAC,EAAE,CAAC;YACJ,SAAS;QACX,CAAC;QAED,2BAA2B;QAC3B,IAAI,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,EAAE,CAAC;YAC9B,MAAM,KAAK,GAAG,UAAU,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC;YACnC,IAAI,KAAK,EAAE,CAAC;gBACV,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;gBACnB,2CAA2C;gBAC3C,CAAC,EAAE,CAAC,CAAC,wDAAwD;YAC/D,CAAC;iBAAM,CAAC;gBACN,CAAC,EAAE,CAAC;YACN,CAAC;YACD,SAAS;QACX,CAAC;QAED,CAAC,EAAE,CAAC;IACN,CAAC;IAED,OAAO,EAAE,MAAM,EAAE,CAAC;AACpB,CAAC;AAED,SAAS,UAAU,CAAC,KAAe,EAAE,UAAkB;IACrD,MAAM,WAAW,GAAG,KAAK,CAAC,UAAU,CAAC,CAAC,KAAK,CAAC,uBAAuB,CAAC,CAAC;IACrE,IAAI,CAAC,WAAW;QAAE,OAAO,IAAI,CAAC;IAE9B,MAAM,IAAI,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC;IAC5B,MAAM,OAAO,GAAiB,EAAE,CAAC;IAEjC,IAAI,CAAC,GAAG,UAAU,GAAG,CAAC,CAAC;IACvB,IAAI,UAAU,GAAG,CAAC,CAAC;IAEnB,OAAO,CAAC,GAAG,KAAK,CAAC,MAAM,IAAI,UAAU,GAAG,CAAC,EAAE,CAAC;QAC1C,MAAM,OAAO,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;QACzB,MAAM,IAAI,GAAG,OAAO,CAAC,IAAI,EAAE,CAAC;QAE5B,IAAI,CAAC,IAAI,IAAI,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC;YAC5D,CAAC,EAAE,CAAC;YACJ,SAAS;QACX,CAAC;QAED,kCAAkC;QAClC,MAAM,SAAS,GAAG,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC;QACrD,MAAM,UAAU,GAAG,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC;QAEtD,kCAAkC;QAClC,MAAM,aAAa,GAAG,CAAC,QAAQ,EAAE,OAAO,EAAE,SAAS,EAAE,QAAQ,EAAE,aAAa,EAAE,MAAM,EAAE,YAAY,CAAC,CAAC;QACpG,MAAM,cAAc,GAAG,aAAa,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC;QAEnE,IAAI,cAAc,IAAI,SAAS,GAAG,CAAC,EAAE,CAAC;YACpC,iDAAiD;YACjD,MAAM,MAAM,GAAG,gBAAgB,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC;YAC1C,IAAI,MAAM,EAAE,CAAC;gBACX,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;YACvB,CAAC;YACD,uBAAuB;YACvB,MAAM,MAAM,GAAG,YAAY,CAAC,KAAK,EAAE,CAAC,GAAG,CAAC,EAAE,UAAU,GAAG,SAAS,CAAC,CAAC;YAClE,CAAC,GAAG,MAAM,GAAG,CAAC,CAAC;YACf,SAAS;QACX,CAAC;QAED,8DAA8D;QAC9D,UAAU,IAAI,SAAS,GAAG,UAAU,CAAC;QACrC,IAAI,UAAU,IAAI,CAAC,EAAE,CAAC;YACpB,CAAC,EAAE,CAAC;YACJ,MAAM;QACR,CAAC;QAED,MAAM,MAAM,GAAG,WAAW,CAAC,IAAI,CAAC,CAAC;QACjC,IAAI,MAAM,EAAE,CAAC;YACX,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACvB,CAAC;QACD,CAAC,EAAE,CAAC;IACN,CAAC;IAED,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC;AAC1C,CAAC;AAED,SAAS,YAAY,CAAC,KAAe,EAAE,UAAkB,EAAE,eAAuB;IAChF,IAAI,UAAU,GAAG,eAAe,CAAC;IACjC,IAAI,CAAC,GAAG,UAAU,CAAC;IACnB,OAAO,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC;QACxB,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;QACtB,KAAK,MAAM,EAAE,IAAI,IAAI,EAAE,CAAC;YACtB,IAAI,EAAE,KAAK,GAAG;gBAAE,UAAU,EAAE,CAAC;YAC7B,IAAI,EAAE,KAAK,GAAG,EAAE,CAAC;gBACf,UAAU,EAAE,CAAC;gBACb,IAAI,UAAU,GAAG,eAAe,EAAE,CAAC;oBACjC,OAAO,CAAC,CAAC;gBACX,CAAC;YACH,CAAC;QACH,CAAC;QACD,CAAC,EAAE,CAAC;IACN,CAAC;IACD,OAAO,CAAC,CAAC;AACX,CAAC;AAED,SAAS,gBAAgB,CAAC,KAAe,EAAE,UAAkB;IAC3D,MAAM,SAAS,GAAG,KAAK,CAAC,UAAU,CAAC,CAAC,IAAI,EAAE,CAAC;IAC3C,MAAM,eAAe,GAAG,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC;IAE7D,IAAI,SAAS,CAAC,UAAU,CAAC,QAAQ,CAAC,EAAE,CAAC;QACnC,OAAO,wBAAwB,CAAC,KAAK,EAAE,UAAU,EAAE,eAAe,CAAC,CAAC;IACtE,CAAC;IACD,IAAI,SAAS,CAAC,UAAU,CAAC,OAAO,CAAC,EAAE,CAAC;QAClC,OAAO,uBAAuB,CAAC,KAAK,EAAE,UAAU,EAAE,eAAe,CAAC,CAAC;IACrE,CAAC;IACD,IAAI,SAAS,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QACpC,OAAO,yBAAyB,CAAC,KAAK,EAAE,UAAU,EAAE,eAAe,CAAC,CAAC;IACvE,CAAC;IACD,IAAI,SAAS,CAAC,UAAU,CAAC,QAAQ,CAAC,EAAE,CAAC;QACnC,OAAO,wBAAwB,CAAC,KAAK,EAAE,UAAU,EAAE,eAAe,CAAC,CAAC;IACtE,CAAC;IACD,IAAI,SAAS,CAAC,UAAU,CAAC,YAAY,CAAC,EAAE,CAAC;QACvC,OAAO,4BAA4B,CAAC,KAAK,EAAE,UAAU,EAAE,eAAe,CAAC,CAAC;IAC1E,CAAC;IACD,IAAI,SAAS,CAAC,UAAU,CAAC,aAAa,CAAC,EAAE,CAAC;QACxC,OAAO,EAAE,IAAI,EAAE,YAAY,EAAE,OAAO,EAAE,EAAE,EAAE,CAAC;IAC7C,CAAC;IACD,IAAI,SAAS,CAAC,UAAU,CAAC,MAAM,CAAC,EAAE,CAAC;QACjC,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC;IACzB,CAAC;IAED,OAAO,IAAI,CAAC;AACd,CAAC;AAED,SAAS,WAAW,CAAC,IAAY;IAC/B,SAAS;IACT,IAAI,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,EAAE,CAAC;QAC9B,MAAM,QAAQ,GAAG,aAAa,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC;QACzD,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,QAAQ,EAAE,CAAC;IACrC,CAAC;IAED,UAAU;IACV,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QAC/B,MAAM,QAAQ,GAAG,aAAa,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC;QACzD,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,QAAQ,EAAE,CAAC;IACtC,CAAC;IAED,cAAc;IACd,IAAI,IAAI,CAAC,UAAU,CAAC,aAAa,CAAC,EAAE,CAAC;QACnC,OAAO,EAAE,IAAI,EAAE,YAAY,EAAE,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,EAAE,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC;IACjE,CAAC;IAED,UAAU;IACV,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QAC/B,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,YAAY,EAAE,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC;IACpE,CAAC;IAED,aAAa;IACb,IAAI,IAAI,CAAC,UAAU,CAAC,YAAY,CAAC,EAAE,CAAC;QAClC,OAAO,cAAc,CAAC,IAAI,CAAC,CAAC;IAC9B,CAAC;IAED,SAAS;IACT,IAAI,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,EAAE,CAAC;QAC9B,OAAO,UAAU,CAAC,IAAI,CAAC,CAAC;IAC1B,CAAC;IAED,eAAe;IACf,IAAI,IAAI,CAAC,UAAU,CAAC,aAAa,CAAC,EAAE,CAAC;QACnC,OAAO,EAAE,IAAI,EAAE,YAAY,EAAE,OAAO,EAAE,EAAE,EAAE,CAAC;IAC7C,CAAC;IAED,UAAU;IACV,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QAC/B,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAkC,CAAC;QACtE,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC;IAClC,CAAC;IAED,gBAAgB;IAChB,IAAI,IAAI,CAAC,UAAU,CAAC,eAAe,CAAC,EAAE,CAAC;QACrC,OAAO,YAAY,CAAC,IAAI,CAAC,CAAC;IAC5B,CAAC;IAED,aAAa;IACb,IAAI,IAAI,CAAC,UAAU,CAAC,YAAY,CAAC,EAAE,CAAC;QAClC,OAAO,cAAc,CAAC,IAAI,CAAC,CAAC;IAC9B,CAAC;IAED,UAAU;IACV,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QAC/B,OAAO,WAAW,CAAC,IAAI,CAAC,CAAC;IAC3B,CAAC;IAED,UAAU;IACV,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QAC/B,OAAO,WAAW,CAAC,IAAI,CAAC,CAAC;IAC3B,CAAC;IAED,SAAS;IACT,IAAI,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,EAAE,CAAC;QAC9B,MAAM,OAAO,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QACzC,oCAAoC;QACpC,IAAI,OAAO,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;YACrD,MAAM,KAAK,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YAC1C,OAAO,eAAe,CAAC,KAAK,CAAC,CAAC;QAChC,CAAC;QACD,iEAAiE;QACjE,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,SAAS,EAAE,EAAE,EAAE,CAAC;IAC1C,CAAC;IAED,QAAQ;IACR,IAAI,IAAI,CAAC,UAAU,CAAC,OAAO,CAAC,EAAE,CAAC;QAC7B,MAAM,OAAO,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QACzC,IAAI,OAAO,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;YACrD,MAAM,KAAK,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YAC1C,OAAO,cAAc,CAAC,KAAK,CAAC,CAAC;QAC/B,CAAC;QACD,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,WAAW,EAAE,EAAE,EAAE,CAAC;IAC3C,CAAC;IAED,iBAAiB;IACjB,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QAC/B,OAAO,iBAAiB,CAAC,IAAI,CAAC,CAAC;IACjC,CAAC;IAED,QAAQ;IACR,IAAI,IAAI,CAAC,UAAU,CAAC,MAAM,CAAC,EAAE,CAAC;QAC5B,oEAAoE;QACpE,OAAO;YACL,IAAI,EAAE,KAAK;SACZ,CAAC;IACJ,CAAC;IAED,OAAO,IAAI,CAAC;AACd,CAAC;AAED,SAAS,eAAe,CAAC,KAAa;IACpC,MAAM,SAAS,GAAe,EAAE,CAAC;IACjC,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE;QAAE,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,SAAS,EAAE,CAAC;IAEvD,MAAM,OAAO,GAAG,KAAK,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;IAEvE,KAAK,MAAM,IAAI,IAAI,OAAO,EAAE,CAAC;QAC3B,MAAM,WAAW,GAAG,IAAI,CAAC,KAAK,CAAC,iCAAiC,CAAC,CAAC;QAClE,IAAI,WAAW,EAAE,CAAC;YAChB,SAAS,CAAC,IAAI,CAAC;gBACb,IAAI,EAAE,WAAW,CAAC,CAAC,CAAC;gBACpB,QAAQ,EAAE,aAAa,CAAC,WAAW,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;gBAC9C,WAAW,EAAE,WAAW,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE;aACnC,CAAC,CAAC;YACH,SAAS;QACX,CAAC;QACD,MAAM,cAAc,GAAG,IAAI,CAAC,KAAK,CAAC,oBAAoB,CAAC,CAAC;QACxD,IAAI,cAAc,EAAE,CAAC;YACnB,SAAS,CAAC,IAAI,CAAC;gBACb,IAAI,EAAE,cAAc,CAAC,CAAC,CAAC;gBACvB,QAAQ,EAAE,aAAa,CAAC,cAAc,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;aAClD,CAAC,CAAC;QACL,CAAC;IACH,CAAC;IAED,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,SAAS,EAAE,CAAC;AACtC,CAAC;AAED,SAAS,cAAc,CAAC,KAAa;IACnC,MAAM,WAAW,GAAqC,EAAE,CAAC;IACzD,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE;QAAE,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,WAAW,EAAE,CAAC;IAExD,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;IAE1E,KAAK,MAAM,IAAI,IAAI,UAAU,EAAE,CAAC;QAC9B,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,qBAAqB,CAAC,CAAC;QAChD,IAAI,KAAK,EAAE,CAAC;YACV,WAAW,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;QAC9D,CAAC;IACH,CAAC;IAED,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,WAAW,EAAE,CAAC;AACvC,CAAC;AAED,SAAS,iBAAiB,CAAC,IAAY;IACrC,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,yCAAyC,CAAC,CAAC;IACpE,IAAI,CAAC,KAAK;QAAE,OAAO,IAAI,CAAC;IAExB,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;IACtB,MAAM,SAAS,GAAG,KAAK,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;IACjC,MAAM,MAAM,GAAuC,EAAE,CAAC;IAEtD,IAAI,SAAS,CAAC,IAAI,EAAE,EAAE,CAAC;QACrB,MAAM,UAAU,GAAG,SAAS,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;QAC3E,KAAK,MAAM,CAAC,IAAI,UAAU,EAAE,CAAC;YAC3B,MAAM,MAAM,GAAG,CAAC,CAAC,KAAK,CAAC,oBAAoB,CAAC,CAAC;YAC7C,IAAI,MAAM,EAAE,CAAC;gBACX,MAAM,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,aAAa,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,CAAC,CAAC;YAC1E,CAAC;QACH,CAAC;IACH,CAAC;IAED,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,MAAM,EAAE,UAAU,EAAE,EAAE,EAAE,CAAC;AAC1D,CAAC;AAED,SAAS,oBAAoB,CAAC,IAAY;IACxC,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,sGAAsG,CAAC,CAAC;IACjI,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,MAAM,IAAI,KAAK,CAAC,qCAAqC,IAAI,EAAE,CAAC,CAAC;IAC/D,CAAC;IAED,MAAM,CAAC,EAAE,AAAD,EAAG,IAAI,EAAE,AAAD,EAAG,KAAK,EAAE,AAAD,EAAG,OAAO,EAAE,AAAD,EAAG,IAAI,CAAC,GAAG,KAAK,CAAC;IAErD,OAAO;QACL,IAAI,EAAE,QAAQ;QACd,IAAI;QACJ,KAAK;QACL,OAAO,EAAE,OAAO,IAAI,SAAS;QAC7B,IAAI,EAAE,IAAI,IAAI,SAAS;KACxB,CAAC;AACJ,CAAC;AAED,SAAS,aAAa,CAAC,IAAY;IACjC,IAAI,GAAG,IAAI,CAAC,IAAI,EAAE,CAAC;IAEnB,kBAAkB;IAClB,MAAM,UAAU,GAAG,CAAC,KAAK,EAAE,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,QAAQ,EAAE,OAAO,CAAC,CAAC;IACrE,IAAI,UAAU,CAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,CAAC;QAC9B,OAAO,EAAE,IAAI,EAAE,WAAW,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC;IAC3C,CAAC;IAED,UAAU;IACV,IAAI,IAAI,CAAC,UAAU,CAAC,OAAO,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;QACnD,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QACxD,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,WAAW,EAAE,aAAa,CAAC,KAAK,CAAC,EAAE,CAAC;IAC7D,CAAC;IAED,SAAS;IACT,IAAI,IAAI,CAAC,UAAU,CAAC,MAAM,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;QAClD,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QACxD,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,WAAW,EAAE,aAAa,CAAC,KAAK,CAAC,EAAE,CAAC;IAC5D,CAAC;IAED,YAAY;IACZ,IAAI,IAAI,CAAC,UAAU,CAAC,MAAM,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;QAClD,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;QACjD,0DAA0D;QAC1D,IAAI,KAAK,GAAG,CAAC,CAAC;QACd,IAAI,QAAQ,GAAG,CAAC,CAAC,CAAC;QAClB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;YACtC,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG;gBAAE,KAAK,EAAE,CAAC;iBACzB,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG;gBAAE,KAAK,EAAE,CAAC;iBAC9B,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG,IAAI,KAAK,KAAK,CAAC,EAAE,CAAC;gBACzC,QAAQ,GAAG,CAAC,CAAC;gBACb,MAAM;YACR,CAAC;QACH,CAAC;QACD,IAAI,QAAQ,KAAK,CAAC,CAAC,EAAE,CAAC;YACpB,MAAM,MAAM,GAAG,KAAK,CAAC,SAAS,CAAC,CAAC,EAAE,QAAQ,CAAC,CAAC,IAAI,EAAE,CAAC;YACnD,MAAM,MAAM,GAAG,KAAK,CAAC,SAAS,CAAC,QAAQ,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YACpD,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,OAAO,EAAE,aAAa,CAAC,MAAM,CAAC,EAAE,SAAS,EAAE,aAAa,CAAC,MAAM,CAAC,EAAE,CAAC;QAC3F,CAAC;QACD,wCAAwC;QACxC,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC;IACvC,CAAC;IAED,iBAAiB;IACjB,IAAI,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;QAC/C,MAAM,WAAW,GAAG,aAAa,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC;QACtE,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,WAAW,EAAE,CAAC;IACvC,CAAC;IAED,uBAAuB;IACvB,IAAI,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;QAC/C,MAAM,OAAO,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QAC1D,MAAM,MAAM,GAAuC,EAAE,CAAC;QAEtD,IAAI,OAAO,EAAE,CAAC;YACZ,sCAAsC;YACtC,MAAM,SAAS,GAAG,OAAO,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC;YACxD,KAAK,MAAM,QAAQ,IAAI,SAAS,EAAE,CAAC;gBACjC,MAAM,UAAU,GAAG,QAAQ,CAAC,KAAK,CAAC,oBAAoB,CAAC,CAAC;gBACxD,IAAI,UAAU,EAAE,CAAC;oBACf,MAAM,CAAC,EAAE,IAAI,EAAE,OAAO,CAAC,GAAG,UAAU,CAAC;oBACrC,MAAM,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,IAAI,EAAE,aAAa,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC,EAAE,CAAC,CAAC;gBAC7D,CAAC;YACH,CAAC;QACH,CAAC;QAED,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,MAAM,EAAE,CAAC;IACpC,CAAC;IAED,8CAA8C;IAC9C,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC;AACvC,CAAC;AAED,SAAS,cAAc,CAAC,IAAY;IAClC,mBAAmB;IACnB,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,wBAAwB,CAAC,CAAC;IACnD,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,MAAM,IAAI,KAAK,CAAC,8BAA8B,IAAI,EAAE,CAAC,CAAC;IACxD,CAAC;IAED,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;IAEtB,oGAAoG;IACpG,OAAO;QACL,IAAI,EAAE,WAAW;QACjB,IAAI;QACJ,OAAO,EAAE,EAAE;KACZ,CAAC;AACJ,CAAC;AAED,SAAS,UAAU,CAAC,IAAY;IAC9B,eAAe;IACf,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,oBAAoB,CAAC,CAAC;IAC/C,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,MAAM,IAAI,KAAK,CAAC,0BAA0B,IAAI,EAAE,CAAC,CAAC;IACpD,CAAC;IAED,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;IAEtB,OAAO;QACL,IAAI,EAAE,OAAO;QACb,IAAI;QACJ,MAAM,EAAE,EAAE;KACX,CAAC;AACJ,CAAC;AAED,SAAS,oBAAoB,CAAC,KAAe,EAAE,IAAY;IACzD,oFAAoF;IACpF,OAAO;QACL,IAAI,EAAE,YAAY;QAClB,OAAO,EAAE,EAAE;KACZ,CAAC;AACJ,CAAC;AAED,SAAS,YAAY,CAAC,IAAY;IAChC,oCAAoC;IACpC,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,6DAA6D,CAAC,CAAC;IACxF,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,OAAO,CAAC,IAAI,CAAC,sCAAsC,IAAI,EAAE,CAAC,CAAC;QAC3D,OAAO,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,SAAS,EAAE,CAAC;IAC9C,CAAC;IAED,MAAM,CAAC,EAAE,IAAI,EAAE,AAAD,EAAG,GAAG,CAAC,GAAG,KAAK,CAAC;IAE9B,OAAO;QACL,IAAI,EAAE,SAAS;QACf,IAAI;QACJ,GAAG,EAAE,GAAG,IAAI,SAAS;KACtB,CAAC;AACJ,CAAC;AAED,SAAS,cAAc,CAAC,IAAY;IAClC,kCAAkC;IAClC,mDAAmD;IACnD,oCAAoC;IACpC,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,gEAAgE,CAAC,CAAC;IAC3F,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,OAAO,CAAC,IAAI,CAAC,wCAAwC,IAAI,EAAE,CAAC,CAAC;QAC7D,OAAO,EAAE,IAAI,EAAE,WAAW,EAAE,WAAW,EAAE,SAAS,EAAE,CAAC;IACvD,CAAC;IAED,MAAM,CAAC,EAAE,WAAW,EAAE,AAAD,EAAG,MAAM,CAAC,GAAG,KAAK,CAAC;IAExC,OAAO;QACL,IAAI,EAAE,WAAW;QACjB,WAAW;QACX,MAAM,EAAE,MAAM,IAAI,SAAS;KAC5B,CAAC;AACJ,CAAC;AAED,SAAS,WAAW,CAAC,IAAY;IAC/B,2BAA2B;IAC3B,mDAAmD;IACnD,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,iDAAiD,CAAC,CAAC;IAC5E,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,sDAAsD;QACtD,OAAO,CAAC,IAAI,CAAC,qCAAqC,IAAI,EAAE,CAAC,CAAC;QAC1D,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,CAAC;IAC7C,CAAC;IAED,MAAM,CAAC,EAAE,AAAD,EAAG,IAAI,EAAE,AAAD,EAAG,KAAK,CAAC,GAAG,KAAK,CAAC;IAElC,OAAO;QACL,IAAI,EAAE,QAAQ;QACd,IAAI;QACJ,KAAK,EAAE,KAAK,IAAI,SAAS;KAC1B,CAAC;AACJ,CAAC;AAED,SAAS,WAAW,CAAC,IAAY;IAC/B,gCAAgC;IAChC,uDAAuD;IACvD,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,6CAA6C,CAAC,CAAC;IACxE,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,OAAO,CAAC,IAAI,CAAC,qCAAqC,IAAI,EAAE,CAAC,CAAC;QAC1D,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,UAAU,EAAE,CAAC;IAC9C,CAAC;IAED,MAAM,CAAC,EAAE,IAAI,EAAE,AAAD,EAAG,GAAG,CAAC,GAAG,KAAK,CAAC;IAC9B,MAAM,UAAU,GAAG;QACjB,cAAc,EAAE,SAAS,EAAE,UAAU,EAAE,YAAY;QACnD,qBAAqB,EAAE,cAAc,EAAE,kBAAkB;KAC1D,CAAC;IAEF,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,CAAC;QAC/B,OAAO,CAAC,IAAI,CAAC,wBAAwB,IAAI,0BAA0B,CAAC,CAAC;QACrE,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,UAAU,EAAE,GAAG,EAAE,GAAG,IAAI,SAAS,EAAE,CAAC;IACrE,CAAC;IAED,OAAO;QACL,IAAI,EAAE,QAAQ;QACd,IAAI,EAAE,IAAW;QACjB,GAAG,EAAE,GAAG,IAAI,SAAS;KACtB,CAAC;AACJ,CAAC;AAED,SAAS,wBAAwB,CAAC,KAAe,EAAE,UAAkB,EAAE,eAAuB;IAC5F,MAAM,QAAQ,GAAG,YAAY,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,EAAE,eAAe,CAAC,CAAC;IACtE,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,EAAE,QAAQ,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC;IACzI,MAAM,KAAK,GAAG,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACpC,OAAO,eAAe,CAAC,KAAK,CAAC,CAAC;AAChC,CAAC;AAED,SAAS,uBAAuB,CAAC,KAAe,EAAE,UAAkB,EAAE,eAAuB;IAC3F,MAAM,QAAQ,GAAG,YAAY,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,EAAE,eAAe,CAAC,CAAC;IACtE,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,EAAE,QAAQ,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC;IACzI,MAAM,KAAK,GAAG,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACpC,OAAO,cAAc,CAAC,KAAK,CAAC,CAAC;AAC/B,CAAC;AAED,SAAS,yBAAyB,CAAC,KAAe,EAAE,UAAkB,EAAE,eAAuB;IAC7F,MAAM,SAAS,GAAG,KAAK,CAAC,UAAU,CAAC,CAAC,IAAI,EAAE,CAAC;IAC3C,MAAM,MAAM,GAAG,iBAAiB,CAAC,SAAS,CAAC,CAAC;IAC5C,IAAI,CAAC,MAAM;QAAE,OAAO,IAAI,CAAC;IAEzB,MAAM,QAAQ,GAAG,YAAY,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,EAAE,eAAe,CAAC,CAAC;IACtE,MAAM,SAAS,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,EAAE,QAAQ,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC;IAExI,MAAM,UAAU,GAAsB,EAAE,CAAC;IACzC,KAAK,MAAM,IAAI,IAAI,SAAS,EAAE,CAAC;QAC7B,eAAe;QACf,IAAI,IAAI,CAAC,UAAU,CAAC,UAAU,CAAC,EAAE,CAAC;YAChC,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;YACrE,SAAS;QACX,CAAC;QACD,cAAc;QACd,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;YAC/B,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;YACpE,SAAS;QACX,CAAC;QACD,wBAAwB;QACxB,MAAM,SAAS,GAAG,IAAI,CAAC,KAAK,CAAC,iCAAiC,CAAC,CAAC;QAChE,IAAI,SAAS,EAAE,CAAC;YACd,MAAM,SAAS,GAAG,SAAS,CAAC,CAAC,CAAC,CAAC;YAC/B,MAAM,SAAS,GAAG,SAAS,CAAC,CAAC,CAAC,CAAC;YAC/B,MAAM,MAAM,GAAsC,EAAE,CAAC;YACrD,IAAI,SAAS,EAAE,CAAC;gBACd,MAAM,UAAU,GAAG,SAAS,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;gBAC3E,KAAK,MAAM,EAAE,IAAI,UAAU,EAAE,CAAC;oBAC5B,MAAM,QAAQ,GAAG,EAAE,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;oBACjC,IAAI,QAAQ,GAAG,CAAC,EAAE,CAAC;wBACjB,MAAM,CAAC,IAAI,CAAC;4BACV,IAAI,EAAE,EAAE,CAAC,SAAS,CAAC,CAAC,EAAE,QAAQ,CAAC,CAAC,IAAI,EAAE;4BACtC,KAAK,EAAE,EAAE,CAAC,SAAS,CAAC,QAAQ,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE;yBACzC,CAAC,CAAC;oBACL,CAAC;gBACH,CAAC;YACH,CAAC;YACD,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,KAAK,EAAE,SAAS,EAAE,MAAM,EAAE,CAAC,CAAC;YAC5D,SAAS;QACX,CAAC;QACD,mBAAmB;QACnB,MAAM,QAAQ,GAAG,IAAI,CAAC,KAAK,CAAC,2BAA2B,CAAC,CAAC;QACzD,IAAI,QAAQ,EAAE,CAAC;YACb,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,KAAK,EAAE,IAAI,EAAE,QAAQ,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,QAAQ,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;YAC9E,SAAS;QACX,CAAC;QACD,6BAA6B;QAC7B,MAAM,WAAW,GAAG,IAAI,CAAC,KAAK,CAAC,iCAAiC,CAAC,CAAC;QAClE,IAAI,WAAW,EAAE,CAAC;YAChB,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,MAAM,EAAE,WAAW,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,WAAW,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;QAC3F,CAAC;IACH,CAAC;IAED,OAAO,EAAE,GAAG,MAAM,EAAE,UAAU,EAAE,CAAC;AACnC,CAAC;AAED,SAAS,wBAAwB,CAAC,KAAe,EAAE,UAAkB,EAAE,eAAuB;IAC5F,MAAM,SAAS,GAAG,KAAK,CAAC,UAAU,CAAC,CAAC,IAAI,EAAE,CAAC;IAC3C,MAAM,MAAM,GAAG,UAAU,CAAC,SAAS,CAAC,CAAC;IACrC,MAAM,QAAQ,GAAG,YAAY,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,EAAE,eAAe,CAAC,CAAC;IACtE,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,EAAE,QAAQ,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC;IAEzI,MAAM,MAAM,GAAuC,EAAE,CAAC;IACtD,KAAK,MAAM,IAAI,IAAI,UAAU,EAAE,CAAC;QAC9B,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,uBAAuB,CAAC,CAAC;QAClD,IAAI,KAAK,EAAE,CAAC;YACV,MAAM,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,aAAa,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,CAAC,CAAC;QAC1F,CAAC;IACH,CAAC;IAED,OAAO,EAAE,GAAG,MAAM,EAAE,MAAM,EAAE,CAAC;AAC/B,CAAC;AAED,SAAS,4BAA4B,CAAC,KAAe,EAAE,UAAkB,EAAE,eAAuB;IAChG,MAAM,SAAS,GAAG,KAAK,CAAC,UAAU,CAAC,CAAC,IAAI,EAAE,CAAC;IAC3C,MAAM,MAAM,GAAG,cAAc,CAAC,SAAS,CAAC,CAAC;IACzC,MAAM,QAAQ,GAAG,YAAY,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,EAAE,eAAe,CAAC,CAAC;IACtE,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,EAAE,QAAQ,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC;IAEzI,MAAM,OAAO,GAAsB,EAAE,CAAC;IACtC,KAAK,MAAM,IAAI,IAAI,UAAU,EAAE,CAAC;QAC9B,6BAA6B;QAC7B,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,2CAA2C,CAAC,CAAC;QACtE,IAAI,KAAK,EAAE,CAAC;YACV,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;YACtB,MAAM,SAAS,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;YAC3B,MAAM,aAAa,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC;YACxD,MAAM,MAAM,GAAuC,EAAE,CAAC;YACtD,IAAI,SAAS,CAAC,IAAI,EAAE,EAAE,CAAC;gBACrB,MAAM,UAAU,GAAG,SAAS,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;gBAC3E,KAAK,MAAM,CAAC,IAAI,UAAU,EAAE,CAAC;oBAC3B,MAAM,MAAM,GAAG,CAAC,CAAC,KAAK,CAAC,oBAAoB,CAAC,CAAC;oBAC7C,IAAI,MAAM,EAAE,CAAC;wBACX,MAAM,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,aAAa,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,CAAC,CAAC;oBAC1E,CAAC;gBACH,CAAC;YACH,CAAC;YACD,OAAO,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,UAAU,EAAE,aAAa,CAAC,aAAa,CAAC,EAAE,CAAC,CAAC;QAC3E,CAAC;IACH,CAAC;IAED,OAAO,EAAE,GAAG,MAAM,EAAE,OAAO,EAAE,CAAC;AAChC,CAAC"} \ No newline at end of file +{"version":3,"file":"parser.js","sourceRoot":"","sources":["../src/parser.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,MAAM,IAAI,CAAC;AA+KpB;;GAEG;AACH,MAAM,UAAU,cAAc,CAAC,QAAgB;IAC7C,MAAM,OAAO,GAAG,EAAE,CAAC,YAAY,CAAC,QAAQ,EAAE,OAAO,CAAC,CAAC;IACnD,OAAO,iBAAiB,CAAC,OAAO,EAAE,QAAQ,CAAC,CAAC;AAC9C,CAAC;AAED;;GAEG;AACH,MAAM,UAAU,iBAAiB,CAAC,OAAe,EAAE,QAAiB;IAClE,MAAM,MAAM,GAAgB,EAAE,CAAC;IAC/B,MAAM,KAAK,GAAG,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IAElC,IAAI,CAAC,GAAG,CAAC,CAAC;IACV,OAAO,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC;QACxB,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QAE7B,gCAAgC;QAChC,IAAI,CAAC,IAAI,IAAI,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC;YAC5D,CAAC,EAAE,CAAC;YACJ,SAAS;QACX,CAAC;QAED,2BAA2B;QAC3B,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;YAC/B,MAAM,UAAU,GAAG,oBAAoB,CAAC,IAAI,CAAC,CAAC;YAC9C,sDAAsD;YACtD,gDAAgD;YAChD,CAAC,EAAE,CAAC;YACJ,SAAS;QACX,CAAC;QAED,2BAA2B;QAC3B,IAAI,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,EAAE,CAAC;YAC9B,MAAM,KAAK,GAAG,UAAU,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC;YACnC,IAAI,KAAK,EAAE,CAAC;gBACV,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;gBACnB,2CAA2C;gBAC3C,CAAC,EAAE,CAAC,CAAC,wDAAwD;YAC/D,CAAC;iBAAM,CAAC;gBACN,CAAC,EAAE,CAAC;YACN,CAAC;YACD,SAAS;QACX,CAAC;QAED,CAAC,EAAE,CAAC;IACN,CAAC;IAED,OAAO,EAAE,MAAM,EAAE,CAAC;AACpB,CAAC;AAED,SAAS,UAAU,CAAC,KAAe,EAAE,UAAkB;IACrD,MAAM,WAAW,GAAG,KAAK,CAAC,UAAU,CAAC,CAAC,KAAK,CAAC,uBAAuB,CAAC,CAAC;IACrE,IAAI,CAAC,WAAW;QAAE,OAAO,IAAI,CAAC;IAE9B,MAAM,IAAI,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC;IAC5B,MAAM,OAAO,GAAiB,EAAE,CAAC;IAEjC,IAAI,CAAC,GAAG,UAAU,GAAG,CAAC,CAAC;IACvB,IAAI,UAAU,GAAG,CAAC,CAAC;IAEnB,OAAO,CAAC,GAAG,KAAK,CAAC,MAAM,IAAI,UAAU,GAAG,CAAC,EAAE,CAAC;QAC1C,MAAM,OAAO,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;QACzB,MAAM,IAAI,GAAG,OAAO,CAAC,IAAI,EAAE,CAAC;QAE5B,IAAI,CAAC,IAAI,IAAI,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC;YAC5D,CAAC,EAAE,CAAC;YACJ,SAAS;QACX,CAAC;QAED,kCAAkC;QAClC,MAAM,SAAS,GAAG,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC;QACrD,MAAM,UAAU,GAAG,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC;QAEtD,4EAA4E;QAC5E,MAAM,aAAa,GAAG,CAAC,QAAQ,EAAE,OAAO,EAAE,SAAS,EAAE,QAAQ,EAAE,aAAa,EAAE,MAAM,EAAE,YAAY;YAChG,SAAS,EAAE,QAAQ,EAAE,SAAS,EAAE,aAAa,CAAC,CAAC;QACjD,MAAM,cAAc,GAAG,aAAa,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC;QAEnE,IAAI,cAAc,IAAI,SAAS,GAAG,CAAC,EAAE,CAAC;YACpC,iDAAiD;YACjD,MAAM,MAAM,GAAG,gBAAgB,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC;YAC1C,IAAI,MAAM,EAAE,CAAC;gBACX,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;YACvB,CAAC;YACD,kDAAkD;YAClD,IAAI,UAAU,GAAG,SAAS,EAAE,CAAC;gBAC3B,MAAM,MAAM,GAAG,YAAY,CAAC,KAAK,EAAE,CAAC,GAAG,CAAC,EAAE,UAAU,GAAG,SAAS,CAAC,CAAC;gBAClE,CAAC,GAAG,MAAM,GAAG,CAAC,CAAC;YACjB,CAAC;iBAAM,CAAC;gBACN,CAAC,EAAE,CAAC;YACN,CAAC;YACD,SAAS;QACX,CAAC;QAED,8EAA8E;QAC9E,8CAA8C;QAC9C,IAAI,IAAI,CAAC,UAAU,CAAC,aAAa,CAAC,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;YACjE,MAAM,MAAM,GAAG,8BAA8B,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC;YACxD,IAAI,MAAM,EAAE,CAAC;gBACX,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;gBACrB,4CAA4C;gBAC5C,OAAO,CAAC,GAAG,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC;oBAC5B,MAAM,QAAQ,GAAG,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;oBACrC,IAAI,CAAC,QAAQ,IAAI,QAAQ,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,QAAQ,CAAC,UAAU,CAAC,IAAI,CAAC;wBACnE,QAAQ,CAAC,UAAU,CAAC,aAAa,CAAC,IAAI,QAAQ,CAAC,UAAU,CAAC,SAAS,CAAC;wBACpE,QAAQ,CAAC,UAAU,CAAC,aAAa,CAAC,IAAI,QAAQ,CAAC,UAAU,CAAC,SAAS,CAAC;wBACpE,QAAQ,CAAC,UAAU,CAAC,OAAO,CAAC,IAAI,QAAQ,CAAC,UAAU,CAAC,MAAM,CAAC;wBAC3D,QAAQ,CAAC,UAAU,CAAC,SAAS,CAAC,IAAI,QAAQ,CAAC,UAAU,CAAC,QAAQ,CAAC;wBAC/D,QAAQ,CAAC,UAAU,CAAC,YAAY,CAAC,IAAI,QAAQ,CAAC,UAAU,CAAC,aAAa,CAAC;wBACvE,QAAQ,CAAC,UAAU,CAAC,MAAM,CAAC,IAAI,QAAQ,CAAC,UAAU,CAAC,GAAG,CAAC;wBACvD,QAAQ,CAAC,UAAU,CAAC,OAAO,CAAC,IAAI,QAAQ,CAAC,UAAU,CAAC,YAAY,CAAC,EAAE,CAAC;wBACtE,MAAM;oBACR,CAAC;oBACD,CAAC,EAAE,CAAC;gBACN,CAAC;YACH,CAAC;YACD,CAAC,EAAE,CAAC;YACJ,SAAS;QACX,CAAC;QAED,8DAA8D;QAC9D,UAAU,IAAI,SAAS,GAAG,UAAU,CAAC;QACrC,IAAI,UAAU,IAAI,CAAC,EAAE,CAAC;YACpB,CAAC,EAAE,CAAC;YACJ,MAAM;QACR,CAAC;QAED,MAAM,MAAM,GAAG,WAAW,CAAC,IAAI,CAAC,CAAC;QACjC,IAAI,MAAM,EAAE,CAAC;YACX,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACvB,CAAC;QACD,CAAC,EAAE,CAAC;IACN,CAAC;IAED,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC;AAC1C,CAAC;AAED,SAAS,YAAY,CAAC,KAAe,EAAE,UAAkB,EAAE,eAAuB;IAChF,IAAI,UAAU,GAAG,eAAe,CAAC;IACjC,IAAI,CAAC,GAAG,UAAU,CAAC;IACnB,OAAO,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC;QACxB,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;QACtB,KAAK,MAAM,EAAE,IAAI,IAAI,EAAE,CAAC;YACtB,IAAI,EAAE,KAAK,GAAG;gBAAE,UAAU,EAAE,CAAC;YAC7B,IAAI,EAAE,KAAK,GAAG,EAAE,CAAC;gBACf,UAAU,EAAE,CAAC;gBACb,IAAI,UAAU,GAAG,eAAe,EAAE,CAAC;oBACjC,OAAO,CAAC,CAAC;gBACX,CAAC;YACH,CAAC;QACH,CAAC;QACD,CAAC,EAAE,CAAC;IACN,CAAC;IACD,OAAO,CAAC,CAAC;AACX,CAAC;AAED,mEAAmE;AACnE,SAAS,oBAAoB,CAAC,IAAY;IACxC,IAAI,MAAM,GAAG,CAAC,EAAE,MAAM,GAAG,CAAC,EAAE,QAAQ,GAAG,CAAC,CAAC;IACzC,KAAK,MAAM,IAAI,IAAI,IAAI,EAAE,CAAC;QACxB,IAAI,IAAI,KAAK,GAAG;YAAE,MAAM,EAAE,CAAC;aACtB,IAAI,IAAI,KAAK,GAAG;YAAE,MAAM,EAAE,CAAC;aAC3B,IAAI,IAAI,KAAK,GAAG;YAAE,MAAM,EAAE,CAAC;aAC3B,IAAI,IAAI,KAAK,GAAG;YAAE,MAAM,EAAE,CAAC;aAC3B,IAAI,IAAI,KAAK,GAAG;YAAE,QAAQ,EAAE,CAAC;aAC7B,IAAI,IAAI,KAAK,GAAG;YAAE,QAAQ,EAAE,CAAC;IACpC,CAAC;IACD,OAAO,MAAM,KAAK,CAAC,IAAI,MAAM,KAAK,CAAC,IAAI,QAAQ,KAAK,CAAC,CAAC;AACxD,CAAC;AAED,SAAS,gBAAgB,CAAC,KAAe,EAAE,UAAkB;IAC3D,MAAM,SAAS,GAAG,KAAK,CAAC,UAAU,CAAC,CAAC,IAAI,EAAE,CAAC;IAC3C,MAAM,eAAe,GAAG,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC;IAE7D,IAAI,SAAS,CAAC,UAAU,CAAC,QAAQ,CAAC,EAAE,CAAC;QACnC,OAAO,wBAAwB,CAAC,KAAK,EAAE,UAAU,EAAE,eAAe,CAAC,CAAC;IACtE,CAAC;IACD,IAAI,SAAS,CAAC,UAAU,CAAC,SAAS,CAAC,IAAI,SAAS,KAAK,OAAO,EAAE,CAAC;QAC7D,OAAO,wBAAwB,CAAC,KAAK,EAAE,UAAU,EAAE,eAAe,CAAC,CAAC;IACtE,CAAC;IACD,IAAI,SAAS,CAAC,UAAU,CAAC,OAAO,CAAC,EAAE,CAAC;QAClC,OAAO,uBAAuB,CAAC,KAAK,EAAE,UAAU,EAAE,eAAe,CAAC,CAAC;IACrE,CAAC;IACD,IAAI,SAAS,CAAC,UAAU,CAAC,QAAQ,CAAC,IAAI,SAAS,KAAK,MAAM,EAAE,CAAC;QAC3D,OAAO,uBAAuB,CAAC,KAAK,EAAE,UAAU,EAAE,eAAe,CAAC,CAAC;IACrE,CAAC;IACD,IAAI,SAAS,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QACpC,OAAO,yBAAyB,CAAC,KAAK,EAAE,UAAU,EAAE,eAAe,CAAC,CAAC;IACvE,CAAC;IACD,IAAI,SAAS,CAAC,UAAU,CAAC,QAAQ,CAAC,EAAE,CAAC;QACnC,OAAO,wBAAwB,CAAC,KAAK,EAAE,UAAU,EAAE,eAAe,CAAC,CAAC;IACtE,CAAC;IACD,IAAI,SAAS,CAAC,UAAU,CAAC,YAAY,CAAC,EAAE,CAAC;QACvC,OAAO,4BAA4B,CAAC,KAAK,EAAE,UAAU,EAAE,eAAe,CAAC,CAAC;IAC1E,CAAC;IACD,IAAI,SAAS,CAAC,UAAU,CAAC,aAAa,CAAC,EAAE,CAAC;QACxC,OAAO,EAAE,IAAI,EAAE,YAAY,EAAE,OAAO,EAAE,EAAE,EAAE,CAAC;IAC7C,CAAC;IACD,IAAI,SAAS,CAAC,UAAU,CAAC,MAAM,CAAC,EAAE,CAAC;QACjC,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC;IACzB,CAAC;IACD,IAAI,SAAS,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QACpC,OAAO,yBAAyB,CAAC,KAAK,EAAE,UAAU,EAAE,eAAe,CAAC,CAAC;IACvE,CAAC;IACD,IAAI,SAAS,CAAC,UAAU,CAAC,aAAa,CAAC,EAAE,CAAC;QACxC,OAAO,6BAA6B,CAAC,KAAK,EAAE,UAAU,EAAE,eAAe,CAAC,CAAC;IAC3E,CAAC;IAED,OAAO,IAAI,CAAC;AACd,CAAC;AAED,SAAS,WAAW,CAAC,IAAY;IAC/B,SAAS;IACT,IAAI,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,EAAE,CAAC;QAC9B,MAAM,QAAQ,GAAG,aAAa,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC;QACzD,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,QAAQ,EAAE,CAAC;IACrC,CAAC;IAED,UAAU;IACV,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QAC/B,MAAM,QAAQ,GAAG,aAAa,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC;QACzD,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,QAAQ,EAAE,CAAC;IACtC,CAAC;IAED,wDAAwD;IACxD,IAAI,IAAI,CAAC,UAAU,CAAC,aAAa,CAAC,EAAE,CAAC;QACnC,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,8BAA8B,CAAC,CAAC;QACzD,IAAI,KAAK,EAAE,CAAC;YACV,OAAO,EAAE,IAAI,EAAE,YAAY,EAAE,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC;QACvE,CAAC;QACD,OAAO,IAAI,CAAC,CAAC,4DAA4D;IAC3E,CAAC;IAED,2CAA2C;IAC3C,IAAI,IAAI,CAAC,UAAU,CAAC,aAAa,CAAC,EAAE,CAAC;QACnC,OAAO,EAAE,IAAI,EAAE,YAAY,EAAE,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,EAAE,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC;IACjE,CAAC;IAED,6EAA6E;IAC7E,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QAC/B,OAAO,qBAAqB,CAAC,IAAI,CAAC,CAAC;IACrC,CAAC;IAED,0CAA0C;IAC1C,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QAC/B,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,YAAY,EAAE,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC;IACrF,CAAC;IAED,aAAa;IACb,IAAI,IAAI,CAAC,UAAU,CAAC,YAAY,CAAC,EAAE,CAAC;QAClC,OAAO,cAAc,CAAC,IAAI,CAAC,CAAC;IAC9B,CAAC;IAED,SAAS;IACT,IAAI,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,EAAE,CAAC;QAC9B,OAAO,UAAU,CAAC,IAAI,CAAC,CAAC;IAC1B,CAAC;IAED,eAAe;IACf,IAAI,IAAI,CAAC,UAAU,CAAC,aAAa,CAAC,EAAE,CAAC;QACnC,OAAO,EAAE,IAAI,EAAE,YAAY,EAAE,OAAO,EAAE,EAAE,EAAE,CAAC;IAC7C,CAAC;IAED,UAAU;IACV,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QAC/B,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAkC,CAAC;QACtE,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC;IAClC,CAAC;IAED,gBAAgB;IAChB,IAAI,IAAI,CAAC,UAAU,CAAC,eAAe,CAAC,EAAE,CAAC;QACrC,OAAO,YAAY,CAAC,IAAI,CAAC,CAAC;IAC5B,CAAC;IAED,aAAa;IACb,IAAI,IAAI,CAAC,UAAU,CAAC,YAAY,CAAC,EAAE,CAAC;QAClC,OAAO,cAAc,CAAC,IAAI,CAAC,CAAC;IAC9B,CAAC;IAED,UAAU;IACV,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QAC/B,OAAO,WAAW,CAAC,IAAI,CAAC,CAAC;IAC3B,CAAC;IAED,UAAU;IACV,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QAC/B,OAAO,WAAW,CAAC,IAAI,CAAC,CAAC;IAC3B,CAAC;IAED,oBAAoB;IACpB,IAAI,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,EAAE,CAAC;QAC9B,MAAM,OAAO,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QACzC,oCAAoC;QACpC,IAAI,OAAO,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;YACrD,MAAM,KAAK,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YAC1C,OAAO,eAAe,CAAC,KAAK,CAAC,CAAC;QAChC,CAAC;QACD,iEAAiE;QACjE,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,SAAS,EAAE,EAAE,EAAE,CAAC;IAC1C,CAAC;IACD,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,IAAI,IAAI,KAAK,OAAO,EAAE,CAAC;QACnD,gFAAgF;QAChF,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;YAC/B,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YACvC,MAAM,OAAO,GAAG,KAAK,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,KAAK,CAAC;YACtE,IAAI,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;gBAC1B,OAAO,eAAe,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC;YACtD,CAAC;QACH,CAAC;QACD,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,SAAS,EAAE,EAAE,EAAE,CAAC;IAC1C,CAAC;IAED,kBAAkB;IAClB,IAAI,IAAI,CAAC,UAAU,CAAC,OAAO,CAAC,EAAE,CAAC;QAC7B,MAAM,OAAO,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QACzC,IAAI,OAAO,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;YACrD,MAAM,KAAK,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YAC1C,OAAO,cAAc,CAAC,KAAK,CAAC,CAAC;QAC/B,CAAC;QACD,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,WAAW,EAAE,EAAE,EAAE,CAAC;IAC3C,CAAC;IACD,IAAI,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,IAAI,IAAI,KAAK,MAAM,EAAE,CAAC;QACjD,sBAAsB;QACtB,IAAI,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,EAAE,CAAC;YAC9B,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YACvC,MAAM,OAAO,GAAG,KAAK,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,KAAK,CAAC;YACtE,IAAI,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;gBAC1B,OAAO,cAAc,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC;YACrD,CAAC;QACH,CAAC;QACD,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,WAAW,EAAE,EAAE,EAAE,CAAC;IAC3C,CAAC;IAED,iBAAiB;IACjB,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QAC/B,OAAO,iBAAiB,CAAC,IAAI,CAAC,CAAC;IACjC,CAAC;IAED,QAAQ;IACR,IAAI,IAAI,CAAC,UAAU,CAAC,MAAM,CAAC,EAAE,CAAC;QAC5B,oEAAoE;QACpE,OAAO;YACL,IAAI,EAAE,KAAK;SACZ,CAAC;IACJ,CAAC;IAED,OAAO,IAAI,CAAC;AACd,CAAC;AAED,SAAS,eAAe,CAAC,KAAa;IACpC,MAAM,SAAS,GAAe,EAAE,CAAC;IACjC,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE;QAAE,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,SAAS,EAAE,CAAC;IAEvD,MAAM,OAAO,GAAG,KAAK,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;IAEvE,KAAK,MAAM,IAAI,IAAI,OAAO,EAAE,CAAC;QAC3B,MAAM,WAAW,GAAG,IAAI,CAAC,KAAK,CAAC,iCAAiC,CAAC,CAAC;QAClE,IAAI,WAAW,EAAE,CAAC;YAChB,+CAA+C;YAC/C,IAAI,OAAO,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YACpC,MAAM,UAAU,GAAG,OAAO,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;YACzC,IAAI,UAAU,IAAI,CAAC;gBAAE,OAAO,GAAG,OAAO,CAAC,SAAS,CAAC,CAAC,EAAE,UAAU,CAAC,CAAC,IAAI,EAAE,CAAC;YACvE,SAAS,CAAC,IAAI,CAAC;gBACb,IAAI,EAAE,WAAW,CAAC,CAAC,CAAC;gBACpB,QAAQ,EAAE,aAAa,CAAC,OAAO,CAAC;gBAChC,WAAW,EAAE,WAAW,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE;aACnC,CAAC,CAAC;YACH,SAAS;QACX,CAAC;QACD,MAAM,cAAc,GAAG,IAAI,CAAC,KAAK,CAAC,oBAAoB,CAAC,CAAC;QACxD,IAAI,cAAc,EAAE,CAAC;YACnB,+CAA+C;YAC/C,IAAI,OAAO,GAAG,cAAc,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YACvC,MAAM,UAAU,GAAG,OAAO,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;YACzC,IAAI,UAAU,IAAI,CAAC;gBAAE,OAAO,GAAG,OAAO,CAAC,SAAS,CAAC,CAAC,EAAE,UAAU,CAAC,CAAC,IAAI,EAAE,CAAC;YACvE,SAAS,CAAC,IAAI,CAAC;gBACb,IAAI,EAAE,cAAc,CAAC,CAAC,CAAC;gBACvB,QAAQ,EAAE,aAAa,CAAC,OAAO,CAAC;aACjC,CAAC,CAAC;QACL,CAAC;IACH,CAAC;IAED,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,SAAS,EAAE,CAAC;AACtC,CAAC;AAED,SAAS,cAAc,CAAC,KAAa;IACnC,MAAM,WAAW,GAAqC,EAAE,CAAC;IACzD,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE;QAAE,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,WAAW,EAAE,CAAC;IAExD,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;IAE1E,KAAK,MAAM,IAAI,IAAI,UAAU,EAAE,CAAC;QAC9B,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,qBAAqB,CAAC,CAAC;QAChD,IAAI,KAAK,EAAE,CAAC;YACV,WAAW,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;QAC9D,CAAC;IACH,CAAC;IAED,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,WAAW,EAAE,CAAC;AACvC,CAAC;AAED,SAAS,iBAAiB,CAAC,IAAY;IACrC,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,yCAAyC,CAAC,CAAC;IACpE,IAAI,CAAC,KAAK;QAAE,OAAO,IAAI,CAAC;IAExB,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;IACtB,MAAM,SAAS,GAAG,KAAK,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;IACjC,MAAM,MAAM,GAAuC,EAAE,CAAC;IAEtD,IAAI,SAAS,CAAC,IAAI,EAAE,EAAE,CAAC;QACrB,MAAM,UAAU,GAAG,SAAS,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;QAC3E,KAAK,MAAM,CAAC,IAAI,UAAU,EAAE,CAAC;YAC3B,MAAM,MAAM,GAAG,CAAC,CAAC,KAAK,CAAC,oBAAoB,CAAC,CAAC;YAC7C,IAAI,MAAM,EAAE,CAAC;gBACX,MAAM,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,aAAa,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,CAAC,CAAC;YAC1E,CAAC;QACH,CAAC;IACH,CAAC;IAED,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,MAAM,EAAE,UAAU,EAAE,EAAE,EAAE,CAAC;AAC1D,CAAC;AAED,SAAS,oBAAoB,CAAC,IAAY;IACxC,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,sGAAsG,CAAC,CAAC;IACjI,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,MAAM,IAAI,KAAK,CAAC,qCAAqC,IAAI,EAAE,CAAC,CAAC;IAC/D,CAAC;IAED,MAAM,CAAC,EAAE,AAAD,EAAG,IAAI,EAAE,AAAD,EAAG,KAAK,EAAE,AAAD,EAAG,OAAO,EAAE,AAAD,EAAG,IAAI,CAAC,GAAG,KAAK,CAAC;IAErD,OAAO;QACL,IAAI,EAAE,QAAQ;QACd,IAAI;QACJ,KAAK;QACL,OAAO,EAAE,OAAO,IAAI,SAAS;QAC7B,IAAI,EAAE,IAAI,IAAI,SAAS;KACxB,CAAC;AACJ,CAAC;AAED,SAAS,aAAa,CAAC,IAAY;IACjC,IAAI,GAAG,IAAI,CAAC,IAAI,EAAE,CAAC;IAEnB,kBAAkB;IAClB,MAAM,UAAU,GAAG,CAAC,KAAK,EAAE,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,QAAQ,EAAE,OAAO,CAAC,CAAC;IACrE,IAAI,UAAU,CAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,CAAC;QAC9B,OAAO,EAAE,IAAI,EAAE,WAAW,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC;IAC3C,CAAC;IAED,UAAU;IACV,IAAI,IAAI,CAAC,UAAU,CAAC,OAAO,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;QACnD,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QACxD,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,WAAW,EAAE,aAAa,CAAC,KAAK,CAAC,EAAE,CAAC;IAC7D,CAAC;IAED,SAAS;IACT,IAAI,IAAI,CAAC,UAAU,CAAC,MAAM,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;QAClD,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QACxD,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,WAAW,EAAE,aAAa,CAAC,KAAK,CAAC,EAAE,CAAC;IAC5D,CAAC;IAED,YAAY;IACZ,IAAI,IAAI,CAAC,UAAU,CAAC,MAAM,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;QAClD,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;QACjD,6EAA6E;QAC7E,IAAI,KAAK,GAAG,CAAC,CAAC;QACd,IAAI,UAAU,GAAG,CAAC,CAAC;QACnB,IAAI,QAAQ,GAAG,CAAC,CAAC,CAAC;QAClB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;YACtC,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG;gBAAE,KAAK,EAAE,CAAC;iBAC7C,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG;gBAAE,KAAK,EAAE,CAAC;iBAClD,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG;gBAAE,UAAU,EAAE,CAAC;iBACnC,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG;gBAAE,UAAU,EAAE,CAAC;iBACnC,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG,IAAI,KAAK,KAAK,CAAC,IAAI,UAAU,KAAK,CAAC,EAAE,CAAC;gBAC7D,QAAQ,GAAG,CAAC,CAAC;gBACb,MAAM;YACR,CAAC;QACH,CAAC;QACD,IAAI,QAAQ,KAAK,CAAC,CAAC,EAAE,CAAC;YACpB,MAAM,MAAM,GAAG,KAAK,CAAC,SAAS,CAAC,CAAC,EAAE,QAAQ,CAAC,CAAC,IAAI,EAAE,CAAC;YACnD,MAAM,MAAM,GAAG,KAAK,CAAC,SAAS,CAAC,QAAQ,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YACpD,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,OAAO,EAAE,aAAa,CAAC,MAAM,CAAC,EAAE,SAAS,EAAE,aAAa,CAAC,MAAM,CAAC,EAAE,CAAC;QAC3F,CAAC;QACD,wCAAwC;QACxC,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC;IACvC,CAAC;IAED,0DAA0D;IAC1D,IAAI,IAAI,CAAC,UAAU,CAAC,OAAO,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;QACnD,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QACxD,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,WAAW,EAAE,aAAa,CAAC,KAAK,CAAC,EAAE,CAAC;IAC7D,CAAC;IAED,yDAAyD;IACzD,IAAI,IAAI,CAAC,UAAU,CAAC,MAAM,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;QAClD,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QACxD,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,WAAW,EAAE,aAAa,CAAC,KAAK,CAAC,EAAE,CAAC;IAC5D,CAAC;IAED,4DAA4D;IAC5D,IAAI,IAAI,CAAC,UAAU,CAAC,MAAM,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;QAClD,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;QACjD,IAAI,KAAK,GAAG,CAAC,CAAC;QACd,IAAI,UAAU,GAAG,CAAC,CAAC;QACnB,IAAI,QAAQ,GAAG,CAAC,CAAC,CAAC;QAClB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;YACtC,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG;gBAAE,KAAK,EAAE,CAAC;iBAC7C,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG;gBAAE,KAAK,EAAE,CAAC;iBAClD,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG;gBAAE,UAAU,EAAE,CAAC;iBACnC,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG;gBAAE,UAAU,EAAE,CAAC;iBACnC,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG,IAAI,KAAK,KAAK,CAAC,IAAI,UAAU,KAAK,CAAC,EAAE,CAAC;gBAC7D,QAAQ,GAAG,CAAC,CAAC;gBACb,MAAM;YACR,CAAC;QACH,CAAC;QACD,IAAI,QAAQ,KAAK,CAAC,CAAC,EAAE,CAAC;YACpB,MAAM,MAAM,GAAG,KAAK,CAAC,SAAS,CAAC,CAAC,EAAE,QAAQ,CAAC,CAAC,IAAI,EAAE,CAAC;YACnD,MAAM,MAAM,GAAG,KAAK,CAAC,SAAS,CAAC,QAAQ,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YACpD,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,OAAO,EAAE,aAAa,CAAC,MAAM,CAAC,EAAE,SAAS,EAAE,aAAa,CAAC,MAAM,CAAC,EAAE,CAAC;QAC3F,CAAC;QACD,wCAAwC;QACxC,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC;IACvC,CAAC;IAED,iBAAiB;IACjB,IAAI,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;QAC/C,MAAM,WAAW,GAAG,aAAa,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC;QACtE,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,WAAW,EAAE,CAAC;IACvC,CAAC;IAED,uBAAuB;IACvB,IAAI,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;QAC/C,MAAM,OAAO,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QAC1D,MAAM,MAAM,GAAuC,EAAE,CAAC;QAEtD,IAAI,OAAO,EAAE,CAAC;YACZ,sCAAsC;YACtC,MAAM,SAAS,GAAG,OAAO,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC;YACxD,KAAK,MAAM,QAAQ,IAAI,SAAS,EAAE,CAAC;gBACjC,MAAM,UAAU,GAAG,QAAQ,CAAC,KAAK,CAAC,oBAAoB,CAAC,CAAC;gBACxD,IAAI,UAAU,EAAE,CAAC;oBACf,MAAM,CAAC,EAAE,IAAI,EAAE,OAAO,CAAC,GAAG,UAAU,CAAC;oBACrC,MAAM,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,IAAI,EAAE,aAAa,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC,EAAE,CAAC,CAAC;gBAC7D,CAAC;YACH,CAAC;QACH,CAAC;QAED,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,MAAM,EAAE,CAAC;IACpC,CAAC;IAED,8CAA8C;IAC9C,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC;AACvC,CAAC;AAED,SAAS,cAAc,CAAC,IAAY;IAClC,mBAAmB;IACnB,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,wBAAwB,CAAC,CAAC;IACnD,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,MAAM,IAAI,KAAK,CAAC,8BAA8B,IAAI,EAAE,CAAC,CAAC;IACxD,CAAC;IAED,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;IAEtB,oGAAoG;IACpG,OAAO;QACL,IAAI,EAAE,WAAW;QACjB,IAAI;QACJ,OAAO,EAAE,EAAE;KACZ,CAAC;AACJ,CAAC;AAED,SAAS,UAAU,CAAC,IAAY;IAC9B,eAAe;IACf,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,oBAAoB,CAAC,CAAC;IAC/C,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,MAAM,IAAI,KAAK,CAAC,0BAA0B,IAAI,EAAE,CAAC,CAAC;IACpD,CAAC;IAED,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;IAEtB,OAAO;QACL,IAAI,EAAE,OAAO;QACb,IAAI;QACJ,MAAM,EAAE,EAAE;KACX,CAAC;AACJ,CAAC;AAED,SAAS,oBAAoB,CAAC,KAAe,EAAE,IAAY;IACzD,oFAAoF;IACpF,OAAO;QACL,IAAI,EAAE,YAAY;QAClB,OAAO,EAAE,EAAE;KACZ,CAAC;AACJ,CAAC;AAED,SAAS,YAAY,CAAC,IAAY;IAChC,oCAAoC;IACpC,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,6DAA6D,CAAC,CAAC;IACxF,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,OAAO,CAAC,IAAI,CAAC,sCAAsC,IAAI,EAAE,CAAC,CAAC;QAC3D,OAAO,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,SAAS,EAAE,CAAC;IAC9C,CAAC;IAED,MAAM,CAAC,EAAE,IAAI,EAAE,AAAD,EAAG,GAAG,CAAC,GAAG,KAAK,CAAC;IAE9B,OAAO;QACL,IAAI,EAAE,SAAS;QACf,IAAI;QACJ,GAAG,EAAE,GAAG,IAAI,SAAS;KACtB,CAAC;AACJ,CAAC;AAED,SAAS,cAAc,CAAC,IAAY;IAClC,kCAAkC;IAClC,mDAAmD;IACnD,oCAAoC;IACpC,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,gEAAgE,CAAC,CAAC;IAC3F,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,OAAO,CAAC,IAAI,CAAC,wCAAwC,IAAI,EAAE,CAAC,CAAC;QAC7D,OAAO,EAAE,IAAI,EAAE,WAAW,EAAE,WAAW,EAAE,SAAS,EAAE,CAAC;IACvD,CAAC;IAED,MAAM,CAAC,EAAE,WAAW,EAAE,AAAD,EAAG,MAAM,CAAC,GAAG,KAAK,CAAC;IAExC,OAAO;QACL,IAAI,EAAE,WAAW;QACjB,WAAW;QACX,MAAM,EAAE,MAAM,IAAI,SAAS;KAC5B,CAAC;AACJ,CAAC;AAED,SAAS,WAAW,CAAC,IAAY;IAC/B,2BAA2B;IAC3B,mDAAmD;IACnD,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,iDAAiD,CAAC,CAAC;IAC5E,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,sDAAsD;QACtD,OAAO,CAAC,IAAI,CAAC,qCAAqC,IAAI,EAAE,CAAC,CAAC;QAC1D,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,CAAC;IAC7C,CAAC;IAED,MAAM,CAAC,EAAE,AAAD,EAAG,IAAI,EAAE,AAAD,EAAG,KAAK,CAAC,GAAG,KAAK,CAAC;IAElC,OAAO;QACL,IAAI,EAAE,QAAQ;QACd,IAAI;QACJ,KAAK,EAAE,KAAK,IAAI,SAAS;KAC1B,CAAC;AACJ,CAAC;AAED,SAAS,WAAW,CAAC,IAAY;IAC/B,gCAAgC;IAChC,uDAAuD;IACvD,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,6CAA6C,CAAC,CAAC;IACxE,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,OAAO,CAAC,IAAI,CAAC,qCAAqC,IAAI,EAAE,CAAC,CAAC;QAC1D,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,UAAU,EAAE,CAAC;IAC9C,CAAC;IAED,MAAM,CAAC,EAAE,IAAI,EAAE,AAAD,EAAG,GAAG,CAAC,GAAG,KAAK,CAAC;IAC9B,MAAM,UAAU,GAAG;QACjB,cAAc,EAAE,SAAS,EAAE,UAAU,EAAE,YAAY;QACnD,qBAAqB,EAAE,cAAc,EAAE,kBAAkB;KAC1D,CAAC;IAEF,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,CAAC;QAC/B,OAAO,CAAC,IAAI,CAAC,wBAAwB,IAAI,0BAA0B,CAAC,CAAC;QACrE,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,UAAU,EAAE,GAAG,EAAE,GAAG,IAAI,SAAS,EAAE,CAAC;IACrE,CAAC;IAED,OAAO;QACL,IAAI,EAAE,QAAQ;QACd,IAAI,EAAE,IAAW;QACjB,GAAG,EAAE,GAAG,IAAI,SAAS;KACtB,CAAC;AACJ,CAAC;AAED,SAAS,wBAAwB,CAAC,KAAe,EAAE,UAAkB,EAAE,eAAuB;IAC5F,MAAM,QAAQ,GAAG,YAAY,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,EAAE,eAAe,CAAC,CAAC;IACtE,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,EAAE,QAAQ,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC;IACzI,mEAAmE;IACnE,uEAAuE;IACvE,oEAAoE;IACpE,MAAM,YAAY,GAAG,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC,CAAC;IAClE,MAAM,KAAK,GAAG,YAAY,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,IAAI,EAAE,CAAC,EAAE,EAAE;QACjD,IAAI,CAAC,KAAK,CAAC;YAAE,OAAO,IAAI,CAAC;QACzB,MAAM,QAAQ,GAAG,YAAY,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;QACrC,oEAAoE;QACpE,MAAM,QAAQ,GAAG,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QACvC,oFAAoF;QACpF,MAAM,kBAAkB,GAAG,UAAU,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC;QACrD,IAAI,QAAQ,IAAI,CAAC,kBAAkB,EAAE,CAAC;YACpC,OAAO,GAAG,GAAG,IAAI,GAAG,IAAI,CAAC;QAC3B,CAAC;QACD,OAAO,GAAG,GAAG,GAAG,GAAG,IAAI,CAAC;IAC1B,CAAC,EAAE,EAAE,CAAC,CAAC;IACP,OAAO,eAAe,CAAC,KAAK,CAAC,CAAC;AAChC,CAAC;AAED,SAAS,uBAAuB,CAAC,KAAe,EAAE,UAAkB,EAAE,eAAuB;IAC3F,MAAM,QAAQ,GAAG,YAAY,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,EAAE,eAAe,CAAC,CAAC;IACtE,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,EAAE,QAAQ,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC;IACzI,kEAAkE;IAClE,kDAAkD;IAClD,MAAM,YAAY,GAAG,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC,CAAC;IAClE,MAAM,KAAK,GAAG,YAAY,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,IAAI,EAAE,CAAC,EAAE,EAAE;QACjD,IAAI,CAAC,KAAK,CAAC;YAAE,OAAO,IAAI,CAAC;QACzB,MAAM,QAAQ,GAAG,YAAY,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;QACrC,MAAM,QAAQ,GAAG,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QACvC,MAAM,kBAAkB,GAAG,UAAU,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC;QACrD,IAAI,QAAQ,IAAI,CAAC,kBAAkB,EAAE,CAAC;YACpC,OAAO,GAAG,GAAG,IAAI,GAAG,IAAI,CAAC;QAC3B,CAAC;QACD,OAAO,GAAG,GAAG,GAAG,GAAG,IAAI,CAAC;IAC1B,CAAC,EAAE,EAAE,CAAC,CAAC;IACP,OAAO,cAAc,CAAC,KAAK,CAAC,CAAC;AAC/B,CAAC;AAED,SAAS,yBAAyB,CAAC,KAAe,EAAE,UAAkB,EAAE,eAAuB;IAC7F,MAAM,SAAS,GAAG,KAAK,CAAC,UAAU,CAAC,CAAC,IAAI,EAAE,CAAC;IAC3C,MAAM,MAAM,GAAG,iBAAiB,CAAC,SAAS,CAAC,CAAC;IAC5C,IAAI,CAAC,MAAM;QAAE,OAAO,IAAI,CAAC;IAEzB,MAAM,QAAQ,GAAG,YAAY,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,EAAE,eAAe,CAAC,CAAC;IACtE,wEAAwE;IACxE,MAAM,YAAY,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,EAAE,QAAQ,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE;QACjE,MAAM,OAAO,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC;QACzB,IAAI,CAAC,OAAO,IAAI,OAAO,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,OAAO,CAAC,UAAU,CAAC,IAAI,CAAC;YAAE,OAAO,IAAI,CAAC;QAClF,OAAO,OAAO,CAAC;IACjB,CAAC,CAAC,CAAC;IACH,sDAAsD;IACtD,MAAM,SAAS,GAAsB,EAAE,CAAC;IACxC,KAAK,MAAM,IAAI,IAAI,YAAY,EAAE,CAAC;QAChC,IAAI,IAAI,KAAK,IAAI,IAAI,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,IAAI,SAAS,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,CAAC,KAAK,IAAI,CAAC,EAAE,CAAC;YACxF,SAAS,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QACvB,CAAC;IACH,CAAC;IACD,wBAAwB;IACxB,OAAO,SAAS,CAAC,MAAM,GAAG,CAAC,IAAI,SAAS,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,CAAC,KAAK,IAAI,EAAE,CAAC;QACxE,SAAS,CAAC,GAAG,EAAE,CAAC;IAClB,CAAC;IAED,MAAM,UAAU,GAAsB,EAAE,CAAC;IACzC,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,SAAS,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QAC1C,MAAM,UAAU,GAAG,SAAS,CAAC,CAAC,CAAC,CAAC;QAChC,6BAA6B;QAC7B,IAAI,UAAU,KAAK,IAAI;YAAE,SAAS;QAClC,MAAM,IAAI,GAAG,UAAU,CAAC;QACxB,qBAAqB;QACrB,IAAI,IAAI,CAAC,UAAU,CAAC,eAAe,CAAC,EAAE,CAAC;YACrC,IAAI,IAAI,GAAG,IAAI,CAAC,SAAS,CAAC,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC;YACrC,kCAAkC;YAClC,OAAO,CAAC,GAAG,CAAC,GAAG,SAAS,CAAC,MAAM,EAAE,CAAC;gBAChC,MAAM,QAAQ,GAAG,SAAS,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;gBAClC,IAAI,QAAQ,KAAK,IAAI,IAAI,CAAC,QAAQ,CAAC,UAAU,CAAC,KAAK,CAAC;oBAAE,MAAM;gBAC5D,CAAC,EAAE,CAAC;gBACJ,IAAI,IAAI,MAAM,GAAG,QAAQ,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YAChD,CAAC;YACD,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,cAAc,EAAE,IAAI,EAAE,CAAC,CAAC;YAChD,SAAS;QACX,CAAC;QACD,sBAAsB;QACtB,IAAI,IAAI,CAAC,UAAU,CAAC,gBAAgB,CAAC,EAAE,CAAC;YACtC,IAAI,IAAI,GAAG,IAAI,CAAC,SAAS,CAAC,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC;YACrC,kCAAkC;YAClC,OAAO,CAAC,GAAG,CAAC,GAAG,SAAS,CAAC,MAAM,EAAE,CAAC;gBAChC,MAAM,QAAQ,GAAG,SAAS,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;gBAClC,IAAI,QAAQ,KAAK,IAAI,IAAI,CAAC,QAAQ,CAAC,UAAU,CAAC,KAAK,CAAC;oBAAE,MAAM;gBAC5D,CAAC,EAAE,CAAC;gBACJ,IAAI,IAAI,MAAM,GAAG,QAAQ,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YAChD,CAAC;YACD,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,eAAe,EAAE,IAAI,EAAE,CAAC,CAAC;YACjD,SAAS;QACX,CAAC;QACD,eAAe;QACf,IAAI,IAAI,CAAC,UAAU,CAAC,UAAU,CAAC,EAAE,CAAC;YAChC,IAAI,IAAI,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YACpC,qDAAqD;YACrD,OAAO,CAAC,GAAG,CAAC,GAAG,SAAS,CAAC,MAAM,EAAE,CAAC;gBAChC,MAAM,QAAQ,GAAG,SAAS,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;gBAClC,IAAI,QAAQ,KAAK,IAAI;oBAAE,MAAM;gBAC7B,mDAAmD;gBACnD,IAAI,QAAQ,CAAC,UAAU,CAAC,eAAe,CAAC,IAAI,QAAQ,CAAC,UAAU,CAAC,gBAAgB,CAAC;oBAC7E,QAAQ,CAAC,UAAU,CAAC,UAAU,CAAC,IAAI,QAAQ,CAAC,UAAU,CAAC,SAAS,CAAC;oBACjE,QAAQ,CAAC,UAAU,CAAC,OAAO,CAAC,IAAI,QAAQ,CAAC,UAAU,CAAC,MAAM,CAAC;oBAC3D,QAAQ,CAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,CAAC;oBAC5B,MAAM;gBACR,CAAC;gBACD,CAAC,EAAE,CAAC;gBACJ,IAAI,IAAI,GAAG,GAAG,QAAQ,CAAC;YACzB,CAAC;YACD,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;YAC3C,SAAS;QACX,CAAC;QACD,cAAc;QACd,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;YAC/B,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;YACpE,SAAS;QACX,CAAC;QACD,wBAAwB;QACxB,MAAM,SAAS,GAAG,IAAI,CAAC,KAAK,CAAC,iCAAiC,CAAC,CAAC;QAChE,IAAI,SAAS,EAAE,CAAC;YACd,MAAM,SAAS,GAAG,SAAS,CAAC,CAAC,CAAC,CAAC;YAC/B,MAAM,SAAS,GAAG,SAAS,CAAC,CAAC,CAAC,CAAC;YAC/B,MAAM,MAAM,GAAsC,EAAE,CAAC;YACrD,IAAI,SAAS,EAAE,CAAC;gBACd,MAAM,UAAU,GAAG,WAAW,CAAC,SAAS,CAAC,CAAC;gBAC1C,KAAK,MAAM,EAAE,IAAI,UAAU,EAAE,CAAC;oBAC5B,MAAM,QAAQ,GAAG,EAAE,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;oBACjC,IAAI,QAAQ,GAAG,CAAC,EAAE,CAAC;wBACjB,MAAM,CAAC,IAAI,CAAC;4BACV,IAAI,EAAE,EAAE,CAAC,SAAS,CAAC,CAAC,EAAE,QAAQ,CAAC,CAAC,IAAI,EAAE;4BACtC,KAAK,EAAE,EAAE,CAAC,SAAS,CAAC,QAAQ,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE;yBACzC,CAAC,CAAC;oBACL,CAAC;gBACH,CAAC;YACH,CAAC;YACD,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,KAAK,EAAE,SAAS,EAAE,MAAM,EAAE,CAAC,CAAC;YAC5D,SAAS;QACX,CAAC;QACD,8DAA8D;QAC9D,iEAAiE;QACjE,IAAI,IAAI,CAAC,UAAU,CAAC,KAAK,CAAC,EAAE,CAAC;YAC3B,0EAA0E;YAC1E,IAAI,UAAU,GAAG,CAAC,CAAC;YACnB,IAAI,CAAC,GAAG,CAAC,CAAC;YACV,MAAM,UAAU,GAAa,EAAE,CAAC;YAChC,OAAO,CAAC,GAAG,SAAS,CAAC,MAAM,EAAE,CAAC;gBAC5B,MAAM,IAAI,GAAG,SAAS,CAAC,CAAC,CAAC,CAAC;gBAC1B,IAAI,IAAI,KAAK,IAAI,EAAE,CAAC;oBAAC,CAAC,EAAE,CAAC;oBAAC,SAAS;gBAAC,CAAC;gBACrC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;gBACtB,KAAK,MAAM,EAAE,IAAI,IAAI,EAAE,CAAC;oBACtB,IAAI,EAAE,KAAK,GAAG;wBAAE,UAAU,EAAE,CAAC;yBACxB,IAAI,EAAE,KAAK,GAAG;wBAAE,UAAU,EAAE,CAAC;gBACpC,CAAC;gBACD,CAAC,EAAE,CAAC;gBACJ,IAAI,UAAU,KAAK,CAAC;oBAAE,MAAM;YAC9B,CAAC;YACD,4EAA4E;YAC5E,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,SAAS,EAAE,GAAG,EAAE,UAAU,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;YAChE,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,0BAA0B;YACrC,SAAS;QACX,CAAC;QACD,mBAAmB;QACnB,MAAM,QAAQ,GAAG,IAAI,CAAC,KAAK,CAAC,2BAA2B,CAAC,CAAC;QACzD,IAAI,QAAQ,EAAE,CAAC;YACb,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,KAAK,EAAE,IAAI,EAAE,QAAQ,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,QAAQ,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;YAC9E,SAAS;QACX,CAAC;QACD,uDAAuD;QACvD,MAAM,WAAW,GAAG,IAAI,CAAC,KAAK,CAAC,iCAAiC,CAAC,CAAC;QAClE,IAAI,WAAW,EAAE,CAAC;YAChB,IAAI,MAAM,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC;YAC5B,IAAI,IAAI,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YACjC,uFAAuF;YACvF,OAAO,CAAC,GAAG,CAAC,GAAG,SAAS,CAAC,MAAM,IAAI,CAAC,oBAAoB,CAAC,IAAI,CAAC,EAAE,CAAC;gBAC/D,CAAC,EAAE,CAAC;gBACJ,IAAI,IAAI,GAAG,GAAG,SAAS,CAAC,CAAC,CAAC,CAAC;YAC7B,CAAC;YACD,UAAU,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;QACpD,CAAC;IACH,CAAC;IAED,OAAO,EAAE,GAAG,MAAM,EAAE,UAAU,EAAE,CAAC;AACnC,CAAC;AAED,SAAS,wBAAwB,CAAC,KAAe,EAAE,UAAkB,EAAE,eAAuB;IAC5F,MAAM,SAAS,GAAG,KAAK,CAAC,UAAU,CAAC,CAAC,IAAI,EAAE,CAAC;IAC3C,MAAM,MAAM,GAAG,UAAU,CAAC,SAAS,CAAC,CAAC;IACrC,MAAM,QAAQ,GAAG,YAAY,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,EAAE,eAAe,CAAC,CAAC;IACtE,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,EAAE,QAAQ,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC;IAEzI,MAAM,MAAM,GAAuC,EAAE,CAAC;IACtD,KAAK,MAAM,IAAI,IAAI,UAAU,EAAE,CAAC;QAC9B,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,uBAAuB,CAAC,CAAC;QAClD,IAAI,KAAK,EAAE,CAAC;YACV,MAAM,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,aAAa,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,CAAC,CAAC;QAC1F,CAAC;IACH,CAAC;IAED,OAAO,EAAE,GAAG,MAAM,EAAE,MAAM,EAAE,CAAC;AAC/B,CAAC;AAED,SAAS,4BAA4B,CAAC,KAAe,EAAE,UAAkB,EAAE,eAAuB;IAChG,MAAM,SAAS,GAAG,KAAK,CAAC,UAAU,CAAC,CAAC,IAAI,EAAE,CAAC;IAC3C,MAAM,MAAM,GAAG,cAAc,CAAC,SAAS,CAAC,CAAC;IACzC,MAAM,QAAQ,GAAG,YAAY,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,EAAE,eAAe,CAAC,CAAC;IACtE,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,EAAE,QAAQ,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC;IAEzI,MAAM,OAAO,GAAsB,EAAE,CAAC;IACtC,KAAK,MAAM,IAAI,IAAI,UAAU,EAAE,CAAC;QAC9B,+EAA+E;QAC/E,6CAA6C;QAC7C,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;QAClE,KAAK,MAAM,KAAK,IAAI,MAAM,EAAE,CAAC;YAC3B,8DAA8D;YAC9D,MAAM,WAAW,GAAG,KAAK,CAAC,KAAK,CAAC,sCAAsC,CAAC,CAAC;YACxE,IAAI,WAAW,EAAE,CAAC;gBAChB,MAAM,IAAI,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC;gBAC5B,MAAM,SAAS,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC;gBACjC,MAAM,aAAa,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC;gBAC9D,MAAM,MAAM,GAAuC,EAAE,CAAC;gBACtD,IAAI,SAAS,CAAC,IAAI,EAAE,EAAE,CAAC;oBACrB,MAAM,UAAU,GAAG,SAAS,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;oBAC3E,KAAK,MAAM,CAAC,IAAI,UAAU,EAAE,CAAC;wBAC3B,MAAM,MAAM,GAAG,CAAC,CAAC,KAAK,CAAC,oBAAoB,CAAC,CAAC;wBAC7C,IAAI,MAAM,EAAE,CAAC;4BACX,MAAM,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,aAAa,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,CAAC,CAAC;wBAC1E,CAAC;oBACH,CAAC;gBACH,CAAC;gBACD,OAAO,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,UAAU,EAAE,aAAa,CAAC,aAAa,CAAC,EAAE,CAAC,CAAC;YAC3E,CAAC;iBAAM,IAAI,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;gBAC/B,4EAA4E;gBAC5E,OAAO,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,KAAK,EAAE,MAAM,EAAE,EAAE,EAAE,UAAU,EAAE,aAAa,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC;YAC/E,CAAC;QACH,CAAC;IACH,CAAC;IACD,OAAO,EAAE,GAAG,MAAM,EAAE,OAAO,EAAE,CAAC;AAChC,CAAC;AAED,kEAAkE;AAElE;;;;;;;;;GASG;AACH,SAAS,8BAA8B,CAAC,KAAe,EAAE,UAAkB;IACzE,MAAM,IAAI,GAAG,KAAK,CAAC,UAAU,CAAC,CAAC,IAAI,EAAE,CAAC;IAEtC,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QAC/B,OAAO,kBAAkB,CAAC,KAAK,EAAE,UAAU,CAAC,CAAC;IAC/C,CAAC;IAED,IAAI,IAAI,CAAC,UAAU,CAAC,aAAa,CAAC,EAAE,CAAC;QACnC,OAAO,sBAAsB,CAAC,KAAK,EAAE,UAAU,CAAC,CAAC;IACnD,CAAC;IAED,OAAO,IAAI,CAAC;AACd,CAAC;AAED,SAAS,sBAAsB,CAAC,KAAe,EAAE,UAAkB;IACjE,MAAM,IAAI,GAAG,KAAK,CAAC,UAAU,CAAC,CAAC,IAAI,EAAE,CAAC;IAEtC,MAAM,UAAU,GAAG,IAAI,CAAC,KAAK,CAAC,2BAA2B,CAAC,CAAC;IAC3D,IAAI,CAAC,UAAU;QAAE,OAAO,IAAI,CAAC;IAE7B,MAAM,IAAI,GAAG,UAAU,CAAC,CAAC,CAAC,CAAC;IAC3B,IAAI,IAAI,GAAG,UAAU,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;IAEhC,mCAAmC;IACnC,IAAI,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,CAAC;QACzB,MAAM,UAAU,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QAC5C,MAAM,SAAS,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC;QAClD,MAAM,MAAM,GAAG,YAAY,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,EAAE,SAAS,CAAC,CAAC;QAC9D,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,EAAE,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC;QACvI,MAAM,KAAK,GAAG,UAAU,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;QACnC,IAAI,GAAG,CAAC,UAAU,GAAG,GAAG,GAAG,KAAK,CAAC,CAAC,IAAI,EAAE,CAAC;IAC3C,CAAC;SAAM,CAAC;QACN,kEAAkE;QAClE,MAAM,iBAAiB,GAAa,EAAE,CAAC;QACvC,KAAK,IAAI,CAAC,GAAG,UAAU,GAAG,CAAC,EAAE,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;YACnD,MAAM,EAAE,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YAC3B,IAAI,CAAC,EAAE,IAAI,EAAE,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,UAAU,CAAC,IAAI,CAAC;gBACjD,EAAE,CAAC,UAAU,CAAC,aAAa,CAAC,IAAI,EAAE,CAAC,UAAU,CAAC,SAAS,CAAC;gBACxD,EAAE,CAAC,UAAU,CAAC,aAAa,CAAC,IAAI,EAAE,CAAC,UAAU,CAAC,SAAS,CAAC;gBACxD,EAAE,CAAC,UAAU,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC,UAAU,CAAC,MAAM,CAAC;gBAC/C,EAAE,CAAC,UAAU,CAAC,SAAS,CAAC,IAAI,EAAE,CAAC,UAAU,CAAC,QAAQ,CAAC;gBACnD,EAAE,CAAC,UAAU,CAAC,YAAY,CAAC,IAAI,EAAE,CAAC,UAAU,CAAC,aAAa,CAAC;gBAC3D,EAAE,CAAC,UAAU,CAAC,MAAM,CAAC,IAAI,EAAE,CAAC,UAAU,CAAC,GAAG,CAAC;gBAC3C,EAAE,CAAC,UAAU,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC,UAAU,CAAC,YAAY,CAAC,EAAE,CAAC;gBAC1D,MAAM;YACR,CAAC;YACD,iBAAiB,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QAC7B,CAAC;QACD,IAAI,iBAAiB,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YACjC,IAAI,GAAG,IAAI,GAAG,GAAG,GAAG,iBAAiB,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;QAClD,CAAC;IACH,CAAC;IAED,OAAO,EAAE,IAAI,EAAE,YAAY,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,IAAI,EAAE,EAAE,CAAC;AACzD,CAAC;AAED,SAAS,kBAAkB,CAAC,KAAe,EAAE,UAAkB;IAC7D,MAAM,IAAI,GAAG,KAAK,CAAC,UAAU,CAAC,CAAC,IAAI,EAAE,CAAC;IAEtC,+CAA+C;IAC/C,MAAM,SAAS,GAAG,IAAI,CAAC,KAAK,CAAC,iGAAiG,CAAC,CAAC;IAChI,IAAI,SAAS,EAAE,CAAC;QACd,OAAO;YACL,IAAI,EAAE,QAAQ;YACd,IAAI,EAAE,SAAS,CAAC,CAAC,CAAC;YAClB,YAAY,EAAE,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE;YACjC,KAAK,EAAE,QAAQ,CAAC,SAAS,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC;SAClC,CAAC;IACJ,CAAC;IAED,0CAA0C;IAC1C,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC;QACvB,MAAM,WAAW,GAAG,IAAI,CAAC,KAAK,CAAC,4EAA4E,CAAC,CAAC;QAC7G,IAAI,WAAW,EAAE,CAAC;YAChB,MAAM,YAAY,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YAC3C,MAAM,IAAI,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC;YAE5B,MAAM,SAAS,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC;YAClD,MAAM,MAAM,GAAG,YAAY,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,EAAE,SAAS,CAAC,CAAC;YAC9D,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,EAAE,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC;YACvI,MAAM,KAAK,GAAG,UAAU,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;YAEnC,MAAM,UAAU,GAAG,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YAChE,MAAM,QAAQ,GAAG,CAAC,UAAU,GAAG,GAAG,GAAG,KAAK,CAAC,CAAC,IAAI,EAAE,CAAC;YACnD,MAAM,UAAU,GAAG,QAAQ,CAAC,KAAK,CAAC,mBAAmB,CAAC,CAAC;YACvD,MAAM,KAAK,GAAG,UAAU,CAAC,CAAC,CAAC,QAAQ,CAAC,UAAU,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;YAEnE,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,YAAY,EAAE,KAAK,EAAE,CAAC;QACvD,CAAC;IACH,CAAC;IAED,8BAA8B;IAC9B,MAAM,WAAW,GAAG,IAAI,CAAC,KAAK,CAAC,2EAA2E,CAAC,CAAC;IAC5G,IAAI,WAAW,EAAE,CAAC;QAChB,OAAO;YACL,IAAI,EAAE,QAAQ;YACd,IAAI,EAAE,WAAW,CAAC,CAAC,CAAC;YACpB,YAAY,EAAE,WAAW,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE;SACpC,CAAC;IACJ,CAAC;IAED,OAAO,IAAI,CAAC;AACd,CAAC;AAED,0CAA0C;AAE1C;;;GAGG;AACH,SAAS,yBAAyB,CAAC,KAAe,EAAE,UAAkB,EAAE,eAAuB;IAC7F,MAAM,SAAS,GAAG,KAAK,CAAC,UAAU,CAAC,CAAC,IAAI,EAAE,CAAC;IAE3C,kEAAkE;IAClE,MAAM,WAAW,GAAG,SAAS,CAAC,KAAK,CAAC,4EAA4E,CAAC,CAAC;IAClH,IAAI,CAAC,WAAW;QAAE,OAAO,IAAI,CAAC;IAE9B,MAAM,YAAY,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;IAC3C,MAAM,IAAI,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC;IAE5B,gEAAgE;IAChE,MAAM,cAAc,GAAG,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC;IAC5D,MAAM,eAAe,GAAG,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC;IAE7D,IAAI,eAAe,IAAI,cAAc,EAAE,CAAC;QACtC,0CAA0C;QAC1C,MAAM,UAAU,GAAG,SAAS,CAAC,SAAS,CAAC,SAAS,CAAC,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC;QACnE,MAAM,WAAW,GAAG,UAAU,CAAC,SAAS,CAAC,CAAC,EAAE,UAAU,CAAC,WAAW,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QAChF,MAAM,UAAU,GAAG,WAAW,CAAC,KAAK,CAAC,mBAAmB,CAAC,CAAC;QAC1D,MAAM,KAAK,GAAG,UAAU,CAAC,CAAC,CAAC,QAAQ,CAAC,UAAU,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;QACnE,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,YAAY,EAAE,KAAK,EAAE,CAAC;IACvD,CAAC;IAED,mBAAmB;IACnB,IAAI,KAAyB,CAAC;IAC9B,MAAM,MAAM,GAAG,YAAY,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,EAAE,eAAe,CAAC,CAAC;IAEpE,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,EAAE,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC;IACvI,MAAM,KAAK,GAAG,UAAU,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;IAEnC,MAAM,UAAU,GAAG,SAAS,CAAC,SAAS,CAAC,SAAS,CAAC,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;IAE1E,MAAM,UAAU,GAAG,CAAC,UAAU,GAAG,GAAG,GAAG,KAAK,CAAC,CAAC,KAAK,CAAC,mBAAmB,CAAC,CAAC;IACzE,IAAI,UAAU,EAAE,CAAC;QACf,KAAK,GAAG,QAAQ,CAAC,UAAU,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;IACtC,CAAC;IAED,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,YAAY,EAAE,KAAK,EAAE,CAAC;AACvD,CAAC;AAED;;GAEG;AACH,SAAS,qBAAqB,CAAC,IAAY;IACzC,kCAAkC;IAClC,MAAM,SAAS,GAAG,IAAI,CAAC,KAAK,CAAC,iGAAiG,CAAC,CAAC;IAChI,IAAI,SAAS,EAAE,CAAC;QACd,OAAO;YACL,IAAI,EAAE,QAAQ;YACd,IAAI,EAAE,SAAS,CAAC,CAAC,CAAC;YAClB,YAAY,EAAE,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE;YACjC,KAAK,EAAE,QAAQ,CAAC,SAAS,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC;SAClC,CAAC;IACJ,CAAC;IAED,+DAA+D;IAC/D,MAAM,WAAW,GAAG,IAAI,CAAC,KAAK,CAAC,2EAA2E,CAAC,CAAC;IAC5G,IAAI,WAAW,EAAE,CAAC;QAChB,OAAO;YACL,IAAI,EAAE,QAAQ;YACd,IAAI,EAAE,WAAW,CAAC,CAAC,CAAC;YACpB,YAAY,EAAE,WAAW,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE;SACpC,CAAC;IACJ,CAAC;IAED,OAAO,IAAI,CAAC;AACd,CAAC;AAED;;;GAGG;AACH,SAAS,6BAA6B,CAAC,KAAe,EAAE,UAAkB,EAAE,eAAuB;IACjG,MAAM,SAAS,GAAG,KAAK,CAAC,UAAU,CAAC,CAAC,IAAI,EAAE,CAAC;IAE3C,4FAA4F;IAC5F,kFAAkF;IAClF,MAAM,UAAU,GAAG,SAAS,CAAC,KAAK,CAAC,2BAA2B,CAAC,CAAC;IAChE,IAAI,UAAU,EAAE,CAAC;QACf,MAAM,IAAI,GAAG,UAAU,CAAC,CAAC,CAAC,CAAC;QAC3B,IAAI,IAAI,GAAG,UAAU,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QAEhC,qDAAqD;QACrD,IAAI,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,CAAC;YACzB,MAAM,MAAM,GAAG,YAAY,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,EAAE,eAAe,CAAC,CAAC;YACpE,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,EAAE,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC;YACvI,MAAM,KAAK,GAAG,UAAU,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;YACnC,oEAAoE;YACpE,MAAM,UAAU,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YAC5C,IAAI,GAAG,CAAC,UAAU,GAAG,GAAG,GAAG,KAAK,CAAC,CAAC,IAAI,EAAE,CAAC;QAC3C,CAAC;aAAM,CAAC;YACN,2EAA2E;YAC3E,gDAAgD;YAChD,MAAM,MAAM,GAAG,YAAY,CAAC,KAAK,EAAE,UAAU,GAAG,CAAC,EAAE,eAAe,CAAC,CAAC;YACpE,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,GAAG,CAAC,EAAE,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC;YACvI,IAAI,UAAU,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;gBAC1B,IAAI,GAAG,IAAI,GAAG,GAAG,GAAG,UAAU,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;YAC3C,CAAC;QACH,CAAC;QAED,OAAO,EAAE,IAAI,EAAE,YAAY,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,IAAI,EAAE,EAAE,CAAC;IACzD,CAAC;IAED,OAAO,IAAI,CAAC;AACd,CAAC;AAED,+EAA+E;AAC/E,SAAS,WAAW,CAAC,CAAS;IAC5B,MAAM,MAAM,GAAa,EAAE,CAAC;IAC5B,IAAI,KAAK,GAAG,CAAC,CAAC;IACd,IAAI,QAAQ,GAAG,KAAK,CAAC;IACrB,IAAI,QAAQ,GAAG,KAAK,CAAC;IACrB,IAAI,KAAK,GAAG,CAAC,CAAC;IACd,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QAClC,MAAM,EAAE,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC;QAChB,IAAI,EAAE,KAAK,GAAG,IAAI,CAAC,QAAQ,EAAE,CAAC;YAAC,QAAQ,GAAG,CAAC,QAAQ,CAAC;YAAC,SAAS;QAAC,CAAC;QAChE,IAAI,EAAE,KAAK,GAAG,IAAI,CAAC,QAAQ,EAAE,CAAC;YAAC,QAAQ,GAAG,CAAC,QAAQ,CAAC;YAAC,SAAS;QAAC,CAAC;QAChE,IAAI,QAAQ,IAAI,QAAQ;YAAE,SAAS;QACnC,IAAI,EAAE,KAAK,GAAG,IAAI,EAAE,KAAK,GAAG,EAAE,CAAC;YAAC,KAAK,EAAE,CAAC;YAAC,SAAS;QAAC,CAAC;QACpD,IAAI,EAAE,KAAK,GAAG,IAAI,EAAE,KAAK,GAAG,EAAE,CAAC;YAAC,KAAK,EAAE,CAAC;YAAC,SAAS;QAAC,CAAC;QACpD,IAAI,EAAE,KAAK,GAAG,IAAI,KAAK,KAAK,CAAC,EAAE,CAAC;YAC9B,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,SAAS,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC;YAC1C,KAAK,GAAG,CAAC,GAAG,CAAC,CAAC;QAChB,CAAC;IACH,CAAC;IACD,MAAM,IAAI,GAAG,CAAC,CAAC,SAAS,CAAC,KAAK,CAAC,CAAC,IAAI,EAAE,CAAC;IACvC,IAAI,IAAI;QAAE,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC5B,OAAO,MAAM,CAAC;AAChB,CAAC"} \ No newline at end of file diff --git a/compiler/node_modules/.vite/vitest/results.json b/compiler/node_modules/.vite/vitest/results.json index a6ca69e..465db73 100644 --- a/compiler/node_modules/.vite/vitest/results.json +++ b/compiler/node_modules/.vite/vitest/results.json @@ -1 +1 @@ -{"version":"2.1.9","results":[[":__tests__/parser.test.ts",{"duration":40.638667,"failed":false}],[":__tests__/e2e-wasm.test.ts",{"duration":25.08648199999999,"failed":true}],[":src/__tests__/compiler.test.ts",{"duration":13.108581000000015,"failed":false}]]} \ No newline at end of file +{"version":"2.1.9","results":[[":__tests__/parser.test.ts",{"duration":31.455063999999993,"failed":false}],[":src/__tests__/compiler.test.ts",{"duration":12.627537000000018,"failed":false}]]} \ No newline at end of file diff --git a/compiler/src/expr/ast.ts b/compiler/src/expr/ast.ts new file mode 100644 index 0000000..24b7a63 --- /dev/null +++ b/compiler/src/expr/ast.ts @@ -0,0 +1,155 @@ +// SpeckDL Expression AST nodes +// These represent parsed SpeckDL expressions as a typed tree. + +import { Token } from './tokenizer'; + +// --- Expression nodes --- + +export type Expr = + | LiteralExpr + | IdentExpr + | UnaryExpr + | BinaryExpr + | CallExpr + | MemberExpr + | IndexExpr + | IfExpr + | ObjectExpr // { key: val, ... } + | SetExpr // { elem, elem, ... } + | MapExpr // { key -> val, key -> val } + | RangeExpr // [0..N] + | ParenthesizedExpr; + +// --- Literals --- + +export interface LiteralExpr { + kind: 'literal'; + type: 'string' | 'number' | 'bool' | 'null'; + value: string; + token: Token; +} + +// --- Identifier --- + +export interface IdentExpr { + kind: 'ident'; + name: string; + token: Token; +} + +// --- Unary operators: not, - --- + +export interface UnaryExpr { + kind: 'unary'; + op: 'not' | 'neg'; + operand: Expr; + token: Token; +} + +// --- Binary operators --- + +export type BinaryOp = + | 'eq' // == + | 'neq' // != + | 'lt' // < + | 'gt' // > + | 'lte' // <= + | 'gte' // >= + | 'add' // + + | 'sub' // - + | 'mul' // * + | 'div' // / + | 'mod' // % + | 'and' // and + | 'or' // or + | 'in' // in + | 'union' // union + | 'diff' // \ + | 'notin' // notIn + | 'concat'; // + for sets/lists + +export interface BinaryExpr { + kind: 'binary'; + op: BinaryOp; + left: Expr; + right: Expr; + token: Token; // operator token +} + +// --- Function calls: hash(x), length(x), size(x), etc. --- + +export interface CallExpr { + kind: 'call'; + callee: string; // function name + args: Expr[]; + token: Token; +} + +// --- Member access: x.field --- + +export interface MemberExpr { + kind: 'member'; + object: Expr; + property: string; + token: Token; // dot token +} + +// --- Index access: x[y] --- + +export interface IndexExpr { + kind: 'index'; + object: Expr; + index: Expr; + token: Token; // [ token +} + +// --- If expression: if cond: thenExpr else elseExpr --- + +export interface IfExpr { + kind: 'if'; + condition: Expr; + thenBranch: Expr; + elseBranch: Expr | null; // null if no else + token: Token; +} + +// --- Object/record literal: { key: val, ... } --- + +export interface ObjectExpr { + kind: 'object'; + fields: { key: string; value: Expr }[]; + token: Token; +} + +// --- Set literal: { elem, elem, ... } --- + +export interface SetExpr { + kind: 'set'; + elements: Expr[]; + token: Token; +} + +// --- Map literal: { key -> val, key -> val } --- + +export interface MapExpr { + kind: 'map'; + entries: { key: Expr; value: Expr }[]; + token: Token; +} + +// --- Range: [0..N] --- + +export interface RangeExpr { + kind: 'range'; + start: number; + end: Expr; // identifier or number + token: Token; +} + +// --- Parenthesized expression --- + +export interface ParenthesizedExpr { + kind: 'parens'; + expr: Expr; + token: Token; +} diff --git a/compiler/src/expr/index.ts b/compiler/src/expr/index.ts new file mode 100644 index 0000000..c2e7f41 --- /dev/null +++ b/compiler/src/expr/index.ts @@ -0,0 +1,5 @@ +export { Tokenizer, Token, TokenType, tokenize } from './tokenizer'; +export { ExpressionParser, parseExpression } from './parser'; +export * from './ast'; +export * from './stmt-ast'; +export { parseActionStatements } from './stmt-parser'; diff --git a/compiler/src/expr/parser.ts b/compiler/src/expr/parser.ts new file mode 100644 index 0000000..d5a9d30 --- /dev/null +++ b/compiler/src/expr/parser.ts @@ -0,0 +1,411 @@ +// SpeckDL Expression Parser +// Recursive-descent + Pratt parser for SpeckDL expressions. +// Takes a token stream and produces an Expr AST. + +import { Token, TokenType } from './tokenizer'; +import { + Expr, BinaryOp, LiteralExpr, IdentExpr, UnaryExpr, BinaryExpr, + CallExpr, MemberExpr, IndexExpr, IfExpr, ObjectExpr, SetExpr, + MapExpr, RangeExpr, ParenthesizedExpr +} from './ast'; + +// --- Precedence (higher = binds tighter) --- + +const PREC = { + LOWEST: 0, + LOGICAL: 1, // or, and, ||, && + EQUALITY: 2, // ==, != + COMPARISON: 3, // <, >, <=, >= + SET: 4, // in, notIn + ADDITIVE: 5, // +, -, union, \ + MULTIPLY: 6, // *, /, % + PREFIX: 7, // not, - + POSTFIX: 8, // f(), x.y, x[y] +} as const; + +/** Get operator precedence. Handles both symbol tokens and ident-based keywords. */ +function precOf(type: TokenType, value?: string): number { + // Ident-based keywords (or, and, in, not, union) + if (type === TokenType.IDENT) { + switch (value) { + case 'or': case 'and': return PREC.LOGICAL; + case 'in': case 'notIn': return PREC.SET; + case 'not': return PREC.PREFIX; + case 'union': return PREC.ADDITIVE; + } + } + switch (type) { + case TokenType.DBLPIPE: case TokenType.DBLAMP: return PREC.LOGICAL; + case TokenType.EQ: case TokenType.NEQ: return PREC.EQUALITY; + case TokenType.LT: case TokenType.GT: + case TokenType.LTE: case TokenType.GTE: return PREC.COMPARISON; + case TokenType.NOTIN: return PREC.SET; + case TokenType.ADD: case TokenType.SUB: + case TokenType.UNION: case TokenType.BACKSLASH: return PREC.ADDITIVE; + case TokenType.MUL: case TokenType.DIV: case TokenType.MOD: return PREC.MULTIPLY; + case TokenType.LPAREN: case TokenType.DOT: + case TokenType.LBRACKET: return PREC.POSTFIX; + default: return PREC.LOWEST; + } +} + +/** Map token type + value to binary operator name. */ +function binOp(type: TokenType, value?: string): BinaryOp | null { + if (type === TokenType.IDENT) { + switch (value) { + case 'or': return 'or'; + case 'and': return 'and'; + case 'in': return 'in'; + case 'notIn': return 'notin'; + case 'union': return 'union'; + } + } + switch (type) { + case TokenType.EQ: return 'eq'; + case TokenType.NEQ: return 'neq'; + case TokenType.LT: return 'lt'; + case TokenType.GT: return 'gt'; + case TokenType.LTE: return 'lte'; + case TokenType.GTE: return 'gte'; + case TokenType.ADD: return 'add'; + case TokenType.SUB: return 'sub'; + case TokenType.MUL: return 'mul'; + case TokenType.DIV: return 'div'; + case TokenType.MOD: return 'mod'; + case TokenType.DBLPIPE: return 'or'; + case TokenType.DBLAMP: return 'and'; + case TokenType.BACKSLASH: return 'diff'; + case TokenType.UNION: return 'union'; + case TokenType.NOTIN: return 'notin'; + default: return null; + } +} + +export class ExpressionParser { + private tokens: Token[]; + private pos: number = 0; + + constructor(tokens: Token[]) { + this.tokens = tokens; + } + + parse(): Expr { + return this.expr(PREC.LOWEST); + } + + /** Parse expression with given minimum precedence. */ + private expr(minPrec: number): Expr { + const tok = this.advance(); + let left = this.prefix(tok); + + while (this.pos < this.tokens.length) { + const peek = this.current(); + const p = precOf(peek.type, peek.value); + if (p <= minPrec) break; + // COLON only used inside if-expr (handled in prefix) — stop here + if (peek.type === TokenType.COLON) break; + const opTok = this.advance(); + left = this.infix(left, opTok); + } + + return left; + } + + // --- Prefix parsers --- + + private prefix(tok: Token): Expr { + switch (tok.type) { + case TokenType.IDENT: + case TokenType.TYPE: + case TokenType.FORALL: + case TokenType.CONSTANT: + return this.identOrCall(tok); + case TokenType.NUMBER: + return { kind: 'literal', type: 'number', value: tok.value, token: tok } as LiteralExpr; + case TokenType.STRING: + return { kind: 'literal', type: 'string', value: tok.value, token: tok } as LiteralExpr; + case TokenType.BOOL: + return { kind: 'literal', type: 'bool', value: tok.value, token: tok } as LiteralExpr; + case TokenType.NULL: + return { kind: 'literal', type: 'null', value: 'null', token: tok } as LiteralExpr; + case TokenType.NOT: + return { kind: 'unary', op: 'not', operand: this.expr(PREC.PREFIX), token: tok } as UnaryExpr; + case TokenType.SUB: + return { kind: 'unary', op: 'neg', operand: this.expr(PREC.PREFIX), token: tok } as UnaryExpr; + case TokenType.IF: + return this.parseIf(tok); + case TokenType.LBRACE: + return this.parseBrace(tok); + case TokenType.LBRACKET: + return this.parseBracket(tok); + case TokenType.LPAREN: + return this.parseParens(tok); + case TokenType.EMPTYSET: + return { kind: 'call', callee: 'emptySet', args: [], token: tok } as CallExpr; + default: + throw new Error(`Unexpected token '${tok.value}' (${tok.type}) at ${tok.line}:${tok.col}`); + } + } + + private identOrCall(tok: Token): Expr { + // "not" as unary prefix keyword (when not followed by . or () + if (tok.value === 'not' && this.current().type !== TokenType.LPAREN && this.current().type !== TokenType.DOT) { + return { kind: 'unary', op: 'not', operand: this.expr(PREC.PREFIX), token: tok } as UnaryExpr; + } + // Function call: ident( + if (this.current().type === TokenType.LPAREN) { + const lp = this.advance(); + return this.parseCallWithCallee(tok, lp); + } + return { kind: 'ident', name: tok.value, token: tok } as IdentExpr; + } + + // --- Infix parsers --- + + private infix(left: Expr, opTok: Token): Expr { + const p = precOf(opTok.type, opTok.value); + + switch (opTok.type) { + case TokenType.LPAREN: + return this.parseCallWithCallee((left as IdentExpr).token, opTok); + case TokenType.DOT: + return this.parseMember(left, opTok); + case TokenType.LBRACKET: + return this.parseIndex(left, opTok); + default: { + // Binary operator (symbol or ident-based keyword) + const op = binOp(opTok.type, opTok.value); + if (!op) throw new Error(`Not an infix operator: ${opTok.type} '${opTok.value}'`); + const right = this.expr(p); + return { kind: 'binary', op, left, right, token: opTok } as BinaryExpr; + } + } + } + + // --- Specific parsers --- + + private parseIf(ifTok: Token): IfExpr { + const condition = this.expr(PREC.LOWEST); + this.expect(TokenType.COLON, 'Expected : after if condition'); + const thenBranch = this.expr(PREC.LOWEST); + let elseBranch: Expr | null = null; + if (this.current().type === TokenType.ELSE) { + this.advance(); + elseBranch = this.expr(PREC.LOWEST); + } + return { kind: 'if', condition, thenBranch, elseBranch, token: ifTok }; + } + + private parseBrace(openTok: Token): Expr { + if (this.current().type === TokenType.RBRACE) { + this.advance(); + return { kind: 'set', elements: [], token: openTok } as SetExpr; + } + + const first = this.expr(PREC.LOWEST); + + // Map literal: key -> val + if (this.current().type === TokenType.ARROW) { + this.advance(); + const val = this.expr(PREC.LOWEST); + const entries = [{ key: first, value: val }]; + while (this.current().type === TokenType.COMMA) { + this.advance(); + if (this.current().type === TokenType.RBRACE) break; + const k = this.expr(PREC.LOWEST); + this.expect(TokenType.ARROW, 'Expected -> in map literal'); + const v = this.expr(PREC.LOWEST); + entries.push({ key: k, value: v }); + } + this.expect(TokenType.RBRACE, 'Expected }'); + return { kind: 'map', entries, token: openTok } as MapExpr; + } + + // Object literal: ident: val + if (first.kind === 'ident' && this.current().type === TokenType.COLON) { + this.advance(); + const val = this.expr(PREC.LOWEST); + const fields = [{ key: (first as IdentExpr).name, value: val }]; + while (this.current().type === TokenType.COMMA) { + this.advance(); + if (this.current().type === TokenType.RBRACE) break; + const keyTok = this.expect(TokenType.IDENT, 'Expected field name'); + this.expect(TokenType.COLON, 'Expected :'); + const v = this.expr(PREC.LOWEST); + fields.push({ key: keyTok.value, value: v }); + } + this.expect(TokenType.RBRACE, 'Expected }'); + return { kind: 'object', fields, token: openTok } as ObjectExpr; + } + + // Set literal: expr, expr, ... + const elements = [first]; + while (this.current().type === TokenType.COMMA) { + this.advance(); + if (this.current().type === TokenType.RBRACE) break; + elements.push(this.expr(PREC.LOWEST)); + } + this.expect(TokenType.RBRACE, 'Expected }'); + return { kind: 'set', elements, token: openTok } as SetExpr; + } + + private parseBracket(openTok: Token): Expr { + const first = this.expr(PREC.LOWEST); + + // Range: [start..end] + if (this.current().type === TokenType.DOTDOT) { + this.advance(); + const startNum = first.kind === 'literal' && first.type === 'number' + ? parseInt((first as LiteralExpr).value) : 0; + const end = this.expr(PREC.LOWEST); + this.expect(TokenType.RBRACKET, 'Expected ]'); + return { kind: 'range', start: startNum, end, token: openTok } as RangeExpr; + } + + // List: [a, b, c] + const elements = [first]; + while (this.current().type === TokenType.COMMA) { + this.advance(); + if (this.current().type === TokenType.RBRACKET) break; + elements.push(this.expr(PREC.LOWEST)); + } + this.expect(TokenType.RBRACKET, 'Expected ]'); + return { kind: 'call', callee: 'Array', args: elements, token: openTok } as CallExpr; + } + + private parseParens(openTok: Token): Expr { + const expr = this.expr(PREC.LOWEST); + this.expect(TokenType.RPAREN, 'Expected )'); + return { kind: 'parens', expr, token: openTok } as ParenthesizedExpr; + } + + private parseCallWithCallee(nameTok: Token, lpTok: Token): Expr { + const callee = nameTok.value; + const args: Expr[] = []; + if (this.current().type !== TokenType.RPAREN) { + args.push(this.expr(PREC.LOWEST)); + while (this.current().type === TokenType.COMMA) { + this.advance(); + if (this.current().type === TokenType.RPAREN) break; + args.push(this.expr(PREC.LOWEST)); + } + } + this.expect(TokenType.RPAREN, 'Expected )'); + + const call: CallExpr = { kind: 'call', callee, args, token: nameTok }; + + // Check for postfix continuation (chaining: f(x).y, f(x)[z], f(x) \ {y}) + while (this.pos < this.tokens.length) { + const peek = this.current(); + if (peek.type === TokenType.DOT) { + const dotTok = this.advance(); + return this.parseMember(call, dotTok); + } + if (peek.type === TokenType.LBRACKET) { + const bracketTok = this.advance(); + return this.parseIndex(call, bracketTok); + } + const p = precOf(peek.type, peek.value); + if (p <= PREC.LOWEST) break; + if (peek.type === TokenType.COLON) break; + // Infix continuation (e.g., \ operator) + const opTok = this.advance(); + return this.infix(call, opTok); + } + return call; + } + + private parseMember(obj: Expr, dotTok: Token): Expr { + const propTok = this.expect(TokenType.IDENT, 'Expected property name after .'); + const member: MemberExpr = { + kind: 'member', object: obj, property: propTok.value, token: dotTok + }; + // Chain: x.y.z() or x.y[z] + while (this.pos < this.tokens.length) { + const peek = this.current(); + if (peek.type === TokenType.DOT) { + const d = this.advance(); + return this.parseMember(member, d); + } + if (peek.type === TokenType.LBRACKET) { + const b = this.advance(); + return this.parseIndex(member, b); + } + if (peek.type === TokenType.LPAREN) { + const lp = this.advance(); + // Method call: x.y(args) — wrap as call with member as callee + const args: Expr[] = []; + if (this.current().type !== TokenType.RPAREN) { + args.push(this.expr(PREC.LOWEST)); + while (this.current().type === TokenType.COMMA) { + this.advance(); + if (this.current().type === TokenType.RPAREN) break; + args.push(this.expr(PREC.LOWEST)); + } + } + this.expect(TokenType.RPAREN, 'Expected )'); + // Return as a method call on the member + const methodCall: CallExpr = { + kind: 'call', callee: `${propTok.value}()`, args: [obj, ...args], token: dotTok + }; + // Actually, for TS generation we want: obj.prop(args) + // Let's represent this as a call with the member expression as the callee + // For now, keep it as a member + separate call representation + return { kind: 'call', callee: propTok.value, args: [obj, ...args], token: dotTok } as CallExpr; + } + break; + } + return member; + } + + private parseIndex(obj: Expr, bracketTok: Token): Expr { + const idx = this.expr(PREC.LOWEST); + this.expect(TokenType.RBRACKET, 'Expected ]'); + const index: IndexExpr = { + kind: 'index', object: obj, index: idx, token: bracketTok + }; + // Chain: x[y].z or x[y][z] + while (this.pos < this.tokens.length) { + const peek = this.current(); + if (peek.type === TokenType.DOT) { + const d = this.advance(); + return this.parseMember(index, d); + } + if (peek.type === TokenType.LBRACKET) { + const b = this.advance(); + return this.parseIndex(index, b); + } + break; + } + return index; + } + + // --- Helpers --- + + private current(): Token { + return this.pos < this.tokens.length + ? this.tokens[this.pos] + : { type: TokenType.EOF, value: '', pos: 0, line: 0, col: 0 }; + } + + private advance(): Token { + const tok = this.current(); + if (this.pos < this.tokens.length) this.pos++; + return tok; + } + + private expect(type: TokenType, msg?: string): Token { + const tok = this.current(); + if (tok.type !== type) { + throw new Error(msg || `Expected ${type} but got ${tok.type} ('${tok.value}') at ${tok.line}:${tok.col}`); + } + return this.advance(); + } +} + +/** Parse a single SpeckDL expression string to AST. */ +export function parseExpression(src: string): Expr { + const { Tokenizer } = require('./tokenizer'); + const tokens = new Tokenizer(src).tokenize(); + return new ExpressionParser(tokens).parse(); +} \ No newline at end of file diff --git a/compiler/src/expr/stmt-ast.ts b/compiler/src/expr/stmt-ast.ts new file mode 100644 index 0000000..51ea03e --- /dev/null +++ b/compiler/src/expr/stmt-ast.ts @@ -0,0 +1,66 @@ +// SpeckDL Statement AST +// Typed statement nodes with parsed expression subtrees. + +import { Expr } from './ast'; + +// --- Action statement types --- + +export type ActionStatement = + | AssignStmt + | LetStmt + | RequireStmt + | PreconditionStmt + | PostconditionStmt + | EmitStmt + | ReturnStmt + | IfBlockStmt; + +export interface AssignStmt { + type: 'assign'; + target: AssignTarget; + expr: Expr; +} + +/** Target of an assignment: plain ident or map[x] bracket access. */ +export type AssignTarget = + | { kind: 'plain'; name: string } + | { kind: 'map_index'; mapName: string; index: Expr }; + +export interface LetStmt { + type: 'let'; + name: string; + expr: Expr; +} + +export interface RequireStmt { + type: 'require'; + expr: Expr; +} + +export interface PreconditionStmt { + type: 'precondition'; + expr: Expr; +} + +export interface PostconditionStmt { + type: 'postcondition'; + expr: Expr; +} + +export interface EmitStmt { + type: 'emit'; + event: string; + fields: { name: string; value: Expr }[]; +} + +export interface ReturnStmt { + type: 'return'; + expr: Expr; +} + +/** Incomplete if/else block — preserved as raw text for manual handling. */ +export interface IfBlockStmt { + type: 'ifblock'; + raw: string; + // Future: properly parsed conditional block +} diff --git a/compiler/src/expr/stmt-parser.ts b/compiler/src/expr/stmt-parser.ts new file mode 100644 index 0000000..e151841 --- /dev/null +++ b/compiler/src/expr/stmt-parser.ts @@ -0,0 +1,221 @@ +// SpeckDL Statement Parser +// Parses an action body (list of lines) into typed ActionStatement nodes +// with full expression subtrees from the ExpressionParser. + +import { Tokenizer, Token, TokenType } from './tokenizer'; +import { ExpressionParser } from './parser'; +import { Expr } from "./ast"; +import { ActionStatement } from './stmt-ast'; + +/** + * Parse a single action's body lines into typed statement nodes. + * Handles: precondition, postcondition, require, let, assign, emit, return, ifblock. + */ +export function parseActionStatements(lines: string[]): ActionStatement[] { + const stmts: ActionStatement[] = []; + + for (let i = 0; i < lines.length; i++) { + const line = lines[i]; + if (!line) continue; + if (line.startsWith('//') || line.startsWith('/*')) continue; + + const trimmed = line.trim(); + if (!trimmed) continue; + + // Clean up double-backslash from old parser (SpeckDL set diff is single \) + // The old parser stores \\ as two chars; the expression parser expects single \\n // Actually, keep double backslash as-is since the tokenizer handles both + + // --- forall block: keep as raw ifblock --- + if (trimmed.startsWith('forall ')) { + // Collect entire forall block + let blockText = trimmed; + let j = i + 1; + while (j < lines.length) { + const next = lines[j]; + if (!next || !next.trim()) break; + const nt = next.trim(); + if (nt.startsWith('require ') || nt.startsWith('let ') || nt.startsWith('return ') || + nt.startsWith('emit ') || /^\w+\s*:=/.test(nt)) break; + blockText += ' ' + nt; + j++; + } + stmts.push({ type: 'ifblock', raw: blockText }); + i = j - 1; + continue; + } + + // --- precondition: expr --- + if (trimmed.startsWith('precondition:')) { + const exprs = collectExpression(lines, i, 13, 'or '); + i = exprs.endIdx; + stmts.push({ type: 'precondition', expr: parseExprSingle(exprs.text) }); + continue; + } + + // --- postcondition: expr --- + if (trimmed.startsWith('postcondition:')) { + const exprs = collectExpression(lines, i, 14, 'or '); + i = exprs.endIdx; + stmts.push({ type: 'postcondition', expr: parseExprSingle(exprs.text) }); + continue; + } + + // --- require expr --- + if (trimmed.startsWith('require ')) { + const exprs = collectRequireExpression(lines, i); + i = exprs.endIdx; + stmts.push({ type: 'require', expr: parseExprSingle(exprs.text) }); + continue; + } + + // --- return expr --- + if (trimmed.startsWith('return ')) { + stmts.push({ type: 'return', expr: parseExprSingle(trimmed.substring(7).trim()) }); + continue; + } + + // --- emit Event { fields } --- + const emitMatch = trimmed.match(/^emit\s+(\w+)\s*\{\s*(.*)\s*\}$/); + if (emitMatch) { + const eventName = emitMatch[1]; + const fieldsStr = emitMatch[2]; + const fields: { name: string; value: Expr }[] = []; + if (fieldsStr.trim()) { + for (const fp of fieldsStr.split(',').map(s => s.trim()).filter(Boolean)) { + const colonIdx = fp.indexOf(':'); + if (colonIdx > 0) { + fields.push({ + name: fp.substring(0, colonIdx).trim(), + value: parseExprSingle(fp.substring(colonIdx + 1).trim()), + }); + } + } + } + stmts.push({ type: 'emit', event: eventName, fields }); + continue; + } + + // --- if condition { ... } else ... --- + if (trimmed.startsWith('if ')) { + // Collect the full if/else block as raw text + let braceDepth = 0; + let j = i; + const blockLines: string[] = []; + while (j < lines.length) { + const bl = lines[j]; + if (!bl) { j++; continue; } + const bt = bl.trim(); + if (bt.startsWith('//')) { j++; continue; } + blockLines.push(bt); + for (const ch of bt) { + if (ch === '{') braceDepth++; + else if (ch === '}') braceDepth--; + } + j++; + if (braceDepth === 0) break; + } + stmts.push({ type: 'ifblock', raw: blockLines.join(' ') }); + i = j - 1; + continue; + } + + // --- let name := expr --- + const letMatch = trimmed.match(/^let\s+([A-Za-z_]\w*)\s*:=\s*(.+)$/); + if (letMatch) { + const rhs = letMatch[2].trim(); + if (/^if\b/.test(rhs) && !/\belse\b/.test(rhs)) { + stmts.push({ type: 'ifblock', raw: trimmed }); + continue; + } + stmts.push({ type: 'let', name: letMatch[1], expr: parseExprSingle(rhs) }); + continue; + } + + // --- assignment: target := expr --- + const assignMatch = trimmed.match(/^(\w+(?:\[.*?\])?)\s*:=\s*(.+)$/); + if (assignMatch) { + let exprText = assignMatch[2].trim(); + // Handle multiline expressions + while (i + 1 < lines.length) { + const next = lines[i + 1]; + if (!next || next.trim().startsWith('//') || next.trim().startsWith('require ') || + next.trim().startsWith('return ') || next.trim().startsWith('emit ') || + next.trim().startsWith('let ') || /\w+\s*:=/.test(next.trim())) { + break; + } + // Join continuation lines + if (next.trim()) { + exprText += ' ' + next.trim(); + i++; + } else { + break; + } + } + const target = assignMatch[1]; + const expr = parseExprSingle(exprText); + + // Determine target kind: plain or map_index + const bracketMatch = target.match(/^(\w+)\[(.+)\]$/); + if (bracketMatch && bracketMatch[2].trim()) { + stmts.push({ + type: 'assign', + target: { kind: 'map_index', mapName: bracketMatch[1], index: parseExprSingle(bracketMatch[2]) }, + expr, + }); + } else { + stmts.push({ type: 'assign', target: { kind: 'plain', name: target }, expr }); + } + continue; + } + } + + return stmts; +} + +// --- Helpers --- + +/** Parse a single expression string to an Expr AST. */ +function parseExprSingle(src: string): Expr { + const tokens = new Tokenizer(src).tokenize(); + return new ExpressionParser(tokens).parse(); +} + +interface CollectResult { + text: string; + endIdx: number; +} + +/** Collect an expression that may continue on 'or ' lines. */ +function collectExpression(lines: string[], startIdx: number, prefixLen: number, continuationPfx: string): CollectResult { + let text = lines[startIdx].trim().substring(prefixLen).trim(); + let i = startIdx; + while (i + 1 < lines.length) { + const next = lines[i + 1]; + if (!next || !next.trim().startsWith(continuationPfx)) break; + i++; + text += ' or ' + next.trim().substring(continuationPfx.length).trim(); + } + return { text, endIdx: i }; +} + +/** Collect a require expression that may span multiple lines (forall, or, etc). */ +function collectRequireExpression(lines: string[], startIdx: number): CollectResult { + let text = lines[startIdx].trim().substring(8).trim(); + let i = startIdx; + while (i + 1 < lines.length) { + const next = lines[i + 1]; + if (!next) break; + const nt = next.trim(); + if (nt.startsWith('//') || nt.startsWith('precondition:') || nt.startsWith('postcondition:') || + nt.startsWith('require ') || nt.startsWith('return ') || nt.startsWith('emit ') || + nt.startsWith('let ') || /\w+\s*:=/.test(nt)) { + break; + } + i++; + text += ' ' + nt; + } + return { text, endIdx: i }; +} + +// Re-export Expr type +export type { Expr } from './ast'; diff --git a/compiler/src/expr/tokenizer.ts b/compiler/src/expr/tokenizer.ts new file mode 100644 index 0000000..60b0132 --- /dev/null +++ b/compiler/src/expr/tokenizer.ts @@ -0,0 +1,387 @@ +// SpeckDL Tokenizer +// Produces a token stream from SpeckDL source text. +// Used by the expression parser and statement parser. + +export interface Token { + type: TokenType; + value: string; + pos: number; // character offset in source + line: number; + col: number; +} + +export enum TokenType { + // Identifiers and literals + IDENT = 'IDENT', + STRING = 'STRING', + NUMBER = 'NUMBER', + BOOL = 'BOOL', // true, false + NULL = 'NULL', // null + + // Keywords + LET = 'LET', + IF = 'IF', + ELSE = 'ELSE', + FORALL = 'FORALL', + IN = 'IN', + NOTIN = 'NOTIN', // notIn keyword + NOT = 'NOT', + AND = 'AND', + OR = 'OR', + REQUIRE = 'REQUIRE', + RETURN = 'RETURN', + EMIT = 'EMIT', + PRECONDITION = 'PRECONDITION', + POSTCONDITION = 'POSTCONDITION', + CONSTANT = 'CONSTANT', + TYPE = 'TYPE', + + // Operators + ASSIGN = 'ASSIGN', // := + EQ = 'EQ', // == + NEQ = 'NEQ', // != + LTE = 'LTE', // <= + GTE = 'GTE', // >= + LT = 'LT', // < + GT = 'GT', // > + ADD = 'ADD', // + + SUB = 'SUB', // - + MUL = 'MUL', // * + DIV = 'DIV', // / + MOD = 'MOD', // % + + // Delimiters + LPAREN = 'LPAREN', // ( + RPAREN = 'RPAREN', // ) + LBRACE = 'LBRACE', // { + RBRACE = 'RBRACE', // } + LBRACKET = 'LBRACKET',// [ + RBRACKET = 'RBRACKET',// ] + COLON = 'COLON', // : + COMMA = 'COMMA', // , + SEMI = 'SEMI', // ; — statement separator in multi-stmt lines + DOT = 'DOT', // . + DOTDOT = 'DOTDOT', // .. + ARROW = 'ARROW', // -> + PIPE = 'PIPE', // | + DBLPIPE = 'DBLPIPE', // || + BACKSLASH = 'BACKSLASH', // \ + DBLAMP = 'DBLAMP', // && + AT = 'AT', // @ + HASH = 'HASH', // # + + // Set operations + UNION = 'UNION', // union keyword + INTER = 'INTER', // inter keyword + DIFF = 'DIFF', // diff keyword + EMPTYSET = 'EMPTYSET', // emptySet keyword + + EOF = 'EOF', +} + +const KEYWORDS: Record = { + 'let': TokenType.LET, + 'if': TokenType.IF, + 'else': TokenType.ELSE, + 'forall': TokenType.FORALL, + 'require': TokenType.REQUIRE, + 'return': TokenType.RETURN, + 'emit': TokenType.EMIT, + 'precondition': TokenType.PRECONDITION, + 'postcondition': TokenType.POSTCONDITION, + 'constant': TokenType.CONSTANT, + 'type': TokenType.TYPE, + 'union': TokenType.UNION, + 'inter': TokenType.INTER, + 'diff': TokenType.DIFF, + 'emptySet': TokenType.EMPTYSET, + 'notIn': TokenType.NOTIN, + 'true': TokenType.BOOL, + 'false': TokenType.BOOL, + 'null': TokenType.NULL, +}; + +export class Tokenizer { + private src: string; + private pos: number = 0; + private line: number = 1; + private col: number = 1; + + constructor(src: string) { + this.src = src; + } + + /** Tokenize the entire source and return an array of tokens (including EOF). */ + tokenize(): Token[] { + const tokens: Token[] = []; + let tok: Token; + while ((tok = this.next()).type !== TokenType.EOF) { + tokens.push(tok); + } + tokens.push(tok); // include EOF + return tokens; + } + + /** Return the next token. Returns EOF token when source is exhausted. */ + next(): Token { + this.skipWhitespaceAndComments(); + + if (this.pos >= this.src.length) { + return this.makeToken(TokenType.EOF, ''); + } + + const ch = this.src[this.pos]; + + // Two-char operators first + if (ch === ':' && this.peek() === '=') { + return this.advance(2, TokenType.ASSIGN, ':='); + } + if (ch === '=' && this.peek() === '=') { + return this.advance(2, TokenType.EQ, '=='); + } + if (ch === '!' && this.peek() === '=') { + return this.advance(2, TokenType.NEQ, '!='); + } + if (ch === '<' && this.peek() === '=') { + return this.advance(2, TokenType.LTE, '<='); + } + if (ch === '>' && this.peek() === '=') { + return this.advance(2, TokenType.GTE, '>='); + } + if (ch === '-' && this.peek() === '>') { + return this.advance(2, TokenType.ARROW, '->'); + } + if (ch === '|' && this.peek() === '|') { + return this.advance(2, TokenType.DBLPIPE, '||'); + } + if (ch === '&' && this.peek() === '&') { + return this.advance(2, TokenType.DBLAMP, '&&'); + } + if (ch === '.' && this.peek() === '.') { + return this.advance(2, TokenType.DOTDOT, '..'); + } + + // Single-char operators & delimiters + switch (ch) { + case ':': return this.advance(1, TokenType.COLON, ':'); + case '(': return this.advance(1, TokenType.LPAREN, '('); + case ')': return this.advance(1, TokenType.RPAREN, ')'); + case '{': return this.advance(1, TokenType.LBRACE, '{'); + case '}': return this.advance(1, TokenType.RBRACE, '}'); + case '[': return this.advance(1, TokenType.LBRACKET, '['); + case ']': return this.advance(1, TokenType.RBRACKET, ']'); + case ',': return this.advance(1, TokenType.COMMA, ','); + case ';': return this.advance(1, TokenType.SEMI, ';'); + case '.': return this.advance(1, TokenType.DOT, '.'); + case '|': return this.advance(1, TokenType.PIPE, '|'); + case '\\': + // Double backslash (\\) is set difference in SpeckDL — skip the second \ + if (this.pos + 1 < this.src.length && this.src[this.pos + 1] === '\\') { + this.pos++; this.col++; // skip second backslash + } + return this.advance(1, TokenType.BACKSLASH, '\\'); + case '@': return this.advance(1, TokenType.AT, '@'); + case '#': return this.advance(1, TokenType.HASH, '#'); + case '<': return this.advance(1, TokenType.LT, '<'); + case '>': return this.advance(1, TokenType.GT, '>'); + case '+': return this.advance(1, TokenType.ADD, '+'); + case '*': return this.advance(1, TokenType.MUL, '*'); + case '%': return this.advance(1, TokenType.MOD, '%'); + } + + // - can be minus or start of a negative number + if (ch === '-') { + if (this.isDigit(this.peek())) { + return this.readNumber(); + } + return this.advance(1, TokenType.SUB, '-'); + } + // / can be division or start of a comment + if (ch === '/') { + return this.advance(1, TokenType.DIV, '/'); + } + + // Strings + if (ch === '"') { + return this.readString('"'); + } + if (ch === "'") { + return this.readString("'"); + } + + // Numbers + if (this.isDigit(ch)) { + return this.readNumber(); + } + + // Identifiers and keywords + if (this.isIdentStart(ch)) { + return this.readIdent(); + } + + // Unknown character — skip and warn + console.warn(`Tokenizer: unexpected character '${ch}' at ${this.line}:${this.col}`); + return this.advance(1, TokenType.EOF, ch); + } + + /** Peek at the next token without consuming it (used by parser). */ + peekToken(): Token { + const savedPos = this.pos; + const savedLine = this.line; + const savedCol = this.col; + const tok = this.next(); + this.pos = savedPos; + this.line = savedLine; + this.col = savedCol; + return tok; + } + + // --- Private helpers --- + + private peek(): string { + if (this.pos + 1 >= this.src.length) return '\0'; + return this.src[this.pos + 1]; + } + + private advance(len: number, type: TokenType, value: string): Token { + const token = this.makeToken(type, value); + this.pos += len; + this.col += len; + return token; + } + + private makeToken(type: TokenType, value: string): Token { + return { type, value, pos: this.pos, line: this.line, col: this.col }; + } + + private skipWhitespaceAndComments(): void { + while (this.pos < this.src.length) { + const ch = this.src[this.pos]; + + // Whitespace + if (ch === ' ' || ch === '\t' || ch === '\r') { + this.pos++; this.col++; + continue; + } + if (ch === '\n') { + this.pos++; this.line++; this.col = 1; + continue; + } + + // Line comments: // + if (ch === '/' && this.peek() === '/') { + this.pos += 2; this.col += 2; + while (this.pos < this.src.length && this.src[this.pos] !== '\n') { + this.pos++; this.col++; + } + continue; + } + + // Block comments: /* ... */ + if (ch === '/' && this.peek() === '*') { + this.pos += 2; this.col += 2; + while (this.pos < this.src.length) { + if (this.src[this.pos] === '*' && this.peek() === '/') { + this.pos += 2; this.col += 2; + break; + } + if (this.src[this.pos] === '\n') { + this.line++; this.col = 1; + } else { + this.col++; + } + this.pos++; + } + continue; + } + + break; + } + } + + private isDigit(ch: string): boolean { + return ch >= '0' && ch <= '9'; + } + + private isIdentStart(ch: string): boolean { + return (ch >= 'a' && ch <= 'z') || (ch >= 'A' && ch <= 'Z') || ch === '_'; + } + + private isIdentPart(ch: string): boolean { + return this.isIdentStart(ch) || this.isDigit(ch); + } + + private readNumber(): Token { + const start = this.pos; + let value = ''; + // Handle leading minus + if (this.src[this.pos] === '-') { + value += '-'; + this.pos++; this.col++; + } + while (this.pos < this.src.length && this.isDigit(this.src[this.pos])) { + value += this.src[this.pos]; + this.pos++; this.col++; + } + // Decimal part + if (this.pos < this.src.length && this.src[this.pos] === '.' && this.isDigit(this.peek())) { + value += '.'; + this.pos++; this.col++; + while (this.pos < this.src.length && this.isDigit(this.src[this.pos])) { + value += this.src[this.pos]; + this.pos++; this.col++; + } + } + return this.makeToken(TokenType.NUMBER, value); + } + + private readString(quote: string): Token { + const startPos = this.pos; + // Skip opening quote + this.pos++; this.col++; + let value = ''; + while (this.pos < this.src.length) { + const ch = this.src[this.pos]; + if (ch === '\\') { + this.pos++; this.col++; + if (this.pos < this.src.length) { + const escaped = this.src[this.pos]; + value += '\\' + escaped; + this.pos++; this.col++; + } + continue; + } + if (ch === quote) { + this.pos++; this.col++; // skip closing quote + return this.makeToken(TokenType.STRING, value); + } + if (ch === '\n') { + console.warn(`Tokenizer: unclosed string starting at line ${this.line}`); + return this.makeToken(TokenType.STRING, value); + } + value += ch; + this.pos++; this.col++; + } + return this.makeToken(TokenType.STRING, value); + } + + private readIdent(): Token { + const startPos = this.pos; + let value = ''; + while (this.pos < this.src.length && this.isIdentPart(this.src[this.pos])) { + value += this.src[this.pos]; + this.pos++; this.col++; + } + // Check for keywords + const keyword = KEYWORDS[value]; + if (keyword) { + return { type: keyword, value, pos: startPos, line: this.line, col: this.col - value.length }; + } + return { type: TokenType.IDENT, value, pos: startPos, line: this.line, col: this.col - value.length }; + } +} + +/** Convenience: tokenize a string in one call. */ +export function tokenize(src: string): Token[] { + return new Tokenizer(src).tokenize(); +} diff --git a/compiler/src/generators/rust.ts b/compiler/src/generators/rust.ts new file mode 100644 index 0000000..45d5d3e --- /dev/null +++ b/compiler/src/generators/rust.ts @@ -0,0 +1,662 @@ +import { AST, SpeckNode, MemberNode, StateNode, InitNode, ActionNode, ConstraintNode, VerifyNode, EventNode } from '../parser.js'; +import fs from 'fs'; +import path from 'path'; + +/** + * Generate Rust state machine from a SpeckDL spec. + * + * Outputs a self-contained .rs file with: + * - State struct with snake_case fields + * - State machine impl with action methods returning Result + * - Invariant check functions + * - Event enums + * - Cargo.toml for full crate builds + * + * Pattern: + * state → Rust struct fields + * actions → Rust methods (Result return, guard errors) + * invariants → Rust functions returning bool + * events → Rust structs + */ + +export function generateRust(ast: AST, outputDir: string): void { + for (const speck of ast.specks) { + const code = emitSpeck(speck); + + // Create a subdirectory per speck for Cargo.toml + src/lib.rs + const speckDir = path.join(outputDir, speck.name); + const srcDir = path.join(speckDir, 'src'); + if (!fs.existsSync(srcDir)) { + fs.mkdirSync(srcDir, { recursive: true }); + } + + const libPath = path.join(srcDir, 'lib.rs'); + fs.writeFileSync(libPath, code); + console.log(`Generated Rust crate: ${libPath}`); + + // Write Cargo.toml + const cargoToml = `[package] +name = "${snakeCase(speck.name)}" +version = "0.1.0" +edition = "2021" +description = "Auto-generated Speck state machine: ${speck.name}" + +[dependencies] +`; + fs.writeFileSync(path.join(speckDir, 'Cargo.toml'), cargoToml); + console.log(`Generated Cargo.toml: ${speckDir}/Cargo.toml`); + } +} + +function emitSpeck(speck: SpeckNode): string { + const deduped = new Map(); + for (const m of speck.members) { + const key = memberKey(m); + const existing = deduped.get(key); + if (!existing || isEmptyStub(existing)) { + deduped.set(key, m); + } + } + const members = Array.from(deduped.values()); + + const stateNode = members.find(m => m.type === 'state') as StateNode | undefined; + const initNode = members.find(m => m.type === 'init') as InitNode | undefined; + const actions = members.filter(m => m.type === 'action') as ActionNode[]; + const events = members.filter(m => m.type === 'event') as EventNode[]; + const constraints = members.filter(m => m.type === 'constraint') as ConstraintNode[]; + const verifies = members.filter(m => m.type === 'verify') as VerifyNode[]; + + // Collect enum-like interfaces + const allInterfaces = members.filter(m => m.type === 'interface') as any[]; + const enumInterfaces = allInterfaces.filter(iface => iface?.methods?.length > 0); + + let stateValues: string[] = []; + if (enumInterfaces.length > 0) { + for (const iface of enumInterfaces) { + stateValues.push(...iface.methods.map((m: any) => m.name)); + } + } else { + const seen = new Set(); + const builtinTypes = new Set(['String', 'Nat', 'Int', 'Bool', 'Real', 'Bytes', 'Date', 'Number']); + for (const a of actions) { + for (const s of a.statements) { + if (s.type === 'require') { + const matches = s.expr.match(/\b[A-Z][a-zA-Z]+\b/g); + if (matches) { + matches.forEach((m: string) => { + if (!builtinTypes.has(m)) seen.add(m); + }); + } + } + } + } + stateValues = Array.from(seen).sort(); + } + if (stateValues.length === 0) stateValues = ['Open', 'Closed']; + + // Build enum map + const enumMap = new Map(); + for (const iface of enumInterfaces) { + enumMap.set(iface.name, iface.methods.map((m: any) => m.name)); + } + + // Determine state enum name + const stateVarType = stateNode?.variables?.[0]?.typeExpr; + let stateEnumName = `${speck.name}State`; + if (stateVarType?.type === 'ident' && stateVarType.name) { + const cleanTypeName = cleanName(stateVarType.name); + stateEnumName = `${speck.name}${cleanTypeName}`; + } else if (enumMap.size > 0) { + const firstEnum = enumMap.keys().next().value; + if (firstEnum) stateEnumName = `${speck.name}${firstEnum}`; + } + + if (enumInterfaces.length === 0) { + enumMap.set(stateEnumName.replace(speck.name, ''), stateValues); + } + + // Scan event field types for secondary enums + const eventFieldTypes = new Set(); + for (const ev of events) { + for (const f of ev.fields) { + if (f.type?.type === 'ident' && f.type.name) { + const typeName = cleanName(f.type.name); + if (typeName !== stateEnumName.replace(speck.name, '')) { + eventFieldTypes.add(typeName); + } + } + } + } + for (const iface of allInterfaces) { + const ifaceName = cleanName(iface.name); + if (eventFieldTypes.has(ifaceName)) { + if (iface.methods?.length > 0) { + enumMap.set(ifaceName, iface.methods.map((m: any) => m.name)); + } else { + const seen = new Set(); + const builtinTypes = new Set(['String', 'Nat', 'Int', 'Bool', 'Real', 'Bytes', 'Date', 'Number']); + for (const a of actions) { + for (const s of a.statements) { + if (s.type === 'require') { + const matches = s.expr.match(/\b[A-Z][a-zA-Z]+\b/g); + if (matches) { + matches.forEach((m: string) => { + if (!builtinTypes.has(m)) seen.add(m); + }); + } + } + if (s.type === 'emit') { + for (const f of s.fields) { + const matches = f.value.match(/\b[A-Z][a-zA-Z]+\b/g); + if (matches) { + matches.forEach((m: string) => { + if (!builtinTypes.has(m)) seen.add(m); + }); + } + } + } + } + } + const primaryValues = new Set(enumMap.get(stateEnumName.replace(speck.name, '')) || []); + const secondaryValues = Array.from(seen).filter(v => !primaryValues.has(v)); + if (secondaryValues.length > 0) { + enumMap.set(ifaceName, secondaryValues.sort()); + } + } + } + } + + // Generate Rust enums + const allEnums: string[] = []; + for (const [enumName, values] of enumMap) { + const fullEnumName = `${speck.name}${enumName}`; + const variants = values.map((v: string) => ` ${v},`).join('\n'); + const enumDef = `#[derive(Debug, Clone, Copy, PartialEq, Eq)]\n` + + `pub enum ${fullEnumName} {\n${variants}\n}`; + allEnums.push(enumDef); + + const displayArms = values.map((v: string) => + ` ${fullEnumName}::${v} => write!(f, "${snakeCase(v)}"),` + ).join('\n'); + const displayImpl = `impl std::fmt::Display for ${fullEnumName} {\n` + + ` fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n` + + ` match self {\n${displayArms}\n` + + ` }\n` + + ` }\n` + + `}`; + allEnums.push(displayImpl); + } + const stateEnum = allEnums.join('\n\n'); + + // Event structs + const eventStructs = events.map(e => { + const fields = e.fields.map(f => { + const name = snakeCase(cleanName(f.name)); + const ty = rustType(f.type, speck.name, enumMap); + return ` pub ${name}: ${ty},`; + }).join('\n'); + return `#[derive(Debug, Clone)]\npub struct ${e.name} {\n${fields}\n}`; + }).join('\n\n'); + + const structName = `${speck.name}Machine`; + + // State variables with original name + snake_case name mapping + const stateVars = (stateNode?.variables ?? []).map(v => ({ + origName: cleanName(v.name), + rustName: snakeCase(cleanName(v.name)), + typeExpr: v.typeExpr, + defaultInit: v.defaultInit, + })); + + // Build name mapping for expression rewriting: original name → Rust field name + const nameMap = new Map(); + for (const v of stateVars) { + nameMap.set(v.origName, v.rustName); + } + + // For expression matching, use the original names + const stateVarOrigNames = new Set(stateVars.map(v => v.origName)); + + // Map variable original names (for bracket -> index rewriting) + const mapVarOrigNames = new Set(stateVars + .filter(v => v.typeExpr?.type === 'map' || (v.typeExpr?.type === 'ident' && v.typeExpr?.name && /^Map/i.test(v.typeExpr.name))) + .map(v => v.origName)); + + // State struct fields + const stateFields = stateVars.map(v => { + return ` pub ${v.rustName}: ${rustType(v.typeExpr, speck.name, enumMap)},`; + }).join('\n'); + + // Collect all known state values for the rewriter + const allKnownStateValues: string[] = []; + for (const values of enumMap.values()) { + allKnownStateValues.push(...values); + } + + // Init assignments + const initAssignments = initNode + ? initNode.assignments.map(a => { + const rustName = snakeCase(cleanName(a.name)); + const expr = cleanExpr(a.expr); + const rewritten = rewriteRustExpr( + expr, nameMap, mapVarOrigNames, new Set(), + stateEnumName, allKnownStateValues + ); + return ` ${rustName}: ${rewritten},`; + }).join('\n') + : ''; + // If no explicit init assignments, emit defaults for all state vars + const initFields = initAssignments || stateVars.map(v => { + const defaultValue = defaultRustValue(v.rustName, v.typeExpr); + return ` ${v.rustName}: ${defaultValue},`; + }).join('\n'); + + // Action methods + const actionMethods = actions + .map(a => emitAction(a, stateVarOrigNames, nameMap, mapVarOrigNames, speck.name, stateEnumName, allKnownStateValues, enumMap)) + .join('\n\n'); + + // Invariant checks + const invariantChecks = constraints.map((c, i) => + ` /// C${i + 1}: ${c.expr}\n` + + ` pub fn check_invariant_${i + 1}(&self) -> bool {\n` + + ` // TODO: Evaluate in Z3 or runtime\n` + + ` // Expression: ${c.expr}\n` + + ` true\n` + + ` }` + ).join('\n\n'); + + // Verify stubs + const verifyChecks = verifies.map((v, i) => + ` /// V${i + 1}: ${v.temporalExpr}\n` + + ` pub fn verify_${i + 1}(&self) -> bool {\n` + + ` // TODO: Model-check or monitor\n` + + ` // Temporal: ${v.temporalExpr}\n` + + ` true\n` + + ` }` + ).join('\n\n'); + + // Decision log type + const logEventType = events.length > 0 ? `Vec<${events[0].name}>` : 'Vec'; + + // Struct definition + const structDef = `/// Speck state machine: ${speck.name}\n` + + `/// Auto-generated Rust state machine from SpeckDL specification.\n` + + `#[derive(Debug, Clone)]\n` + + `pub struct ${structName} {\n${stateFields}\n` + + ` /// Decision log of events emitted during execution\n` + + ` pub decision_log: ${logEventType},\n` + + `}`; + + // Constructor impl + const constructor = `impl ${structName} {\n` + + ` /// Create a new ${speck.name} machine in its initial state.\n` + + ` pub fn new() -> Self {\n` + + ` Self {\n` + + `${initFields}\n` + + ` decision_log: Vec::new(),\n` + + ` }\n` + + ` }\n` + + `}`; + + // Action impl block + const actionImpl = `// Action methods — each returns Result with guard checking\n` + + `impl ${structName} {\n${actionMethods}\n}`; + + // Invariant impl block + const invariantImpl = constraints.length > 0 + ? `\n// Invariant checks\nimpl ${structName} {\n${invariantChecks}\n}` + : ''; + + // Verify impl block + const verifyImpl = verifies.length > 0 + ? `\n// Temporal verification stubs\nimpl ${structName} {\n${verifyChecks}\n}` + : ''; + + const file = + `// Auto-generated by speckl-compile from ${speck.name}\n` + + `// DO NOT EDIT MANUALLY — regenerate from .speckdl source\n\n` + + `use std::collections::{HashMap, HashSet};\n\n` + + `${stateEnum}\n\n` + + `${eventStructs}\n\n` + + `${structDef}\n\n` + + `${constructor}\n\n` + + `${actionImpl}\n` + + `${invariantImpl}\n` + + `${verifyImpl}\n`; + + return file; +} + +function emitAction( + action: ActionNode, + stateVarOrigNames: Set, + nameMap: Map, + mapVarOrigNames: Set, + speckName: string, + stateEnumName: string, + knownStateValues: string[], + enumMap: Map +): string { + const localNames = new Set(action.params.map(p => cleanName(p.name))); + + const params = action.params + .map(p => `${snakeCase(cleanName(p.name))}: ${rustType(p.type, speckName, enumMap)}`) + .join(', '); + + const methodName = snakeCase(action.name); + + // Preconditions + const preconditions = action.statements + .filter(s => s.type === 'precondition') + .map(s => { + const expr = rewriteRustExpr(s.expr, nameMap, mapVarOrigNames, localNames, stateEnumName, knownStateValues); + return ` // precondition: ${s.expr}\n if !(${expr}) {\n return Err(format!("Precondition failed: {}", ${JSON.stringify(s.expr)}));\n }`; + }) + .join('\n'); + + // Requires (guards) + const requires = action.statements + .filter(s => s.type === 'require') + .map(s => { + const expr = rewriteRustExpr(s.expr, nameMap, mapVarOrigNames, localNames, stateEnumName, knownStateValues); + return ` if !(${expr}) {\n return Err(format!("Guard failed: {}", ${JSON.stringify(s.expr)}));\n }`; + }) + .join('\n'); + + // Assignments + const assigns = action.statements + .filter(s => s.type === 'assign') + .map(s => { + const target = snakeCase(cleanName(s.target)); + const expr = rewriteRustExpr(s.expr, nameMap, mapVarOrigNames, localNames, stateEnumName, knownStateValues); + return ` self.${target} = ${expr};`; + }) + .join('\n'); + + // Emits + const emits = action.statements + .filter(s => s.type === 'emit') + .map(s => { + const fields = s.fields + .map(f => { + const name = snakeCase(cleanName(f.name)); + const val = rewriteRustExpr(f.value, nameMap, mapVarOrigNames, localNames, stateEnumName, knownStateValues); + return ` ${name}: ${val},`; + }) + .join('\n'); + return ` self.decision_log.push(${s.event} {\n${fields}\n });`; + }) + .join('\n'); + + // Postconditions + const postconditions = action.statements + .filter(s => s.type === 'postcondition') + .map(s => { + const expr = rewriteRustExpr(s.expr, nameMap, mapVarOrigNames, localNames, stateEnumName, knownStateValues); + return ` // postcondition: ${s.expr}\n if !(${expr}) {\n return Err(format!("Postcondition failed: {}", ${JSON.stringify(s.expr)}));\n }`; + }) + .join('\n'); + + // Returns + const returnStmts = action.statements + .filter(s => s.type === 'return') + .map(s => { + const expr = rewriteRustExpr(s.expr, nameMap, mapVarOrigNames, localNames, stateEnumName, knownStateValues); + return ` Ok(${expr})`; + }) + .join('\n'); + + const guards = [preconditions, requires].filter(s => s.length > 0); + const mutations = [assigns, emits].filter(s => s.length > 0); + const postPost = [postconditions, returnStmts].filter(s => s.length > 0); + + const bodyParts: string[] = []; + if (guards.length > 0) bodyParts.push(guards.join('\n')); + if (mutations.length > 0) bodyParts.push(mutations.join('\n')); + if (postPost.length > 0) bodyParts.push(postPost.join('\n')); + + const body = bodyParts.join('\n'); + + // Determine return type + const hasReturn = action.statements.some(s => s.type === 'return'); + const returnType = hasReturn ? ' -> Result' : ' -> Result<(), String>'; + + return ` /// Execute action: ${action.name}\n pub fn ${methodName}(&mut self${params ? ', ' + params : ''})${returnType} {\n${body || ' Ok(())\n'}\n }`; +} + +/** + * Rewrite SpeckDL expressions to valid Rust. + * + * @param nameMap - Maps original SpeckDL variable names → Rust snake_case field names (e.g. isOn → is_on) + * @param mapVarNames - Set of original variable names that are Map-typed (for bracket access rewriting) + */ +function rewriteRustExpr( + expr: string, + nameMap: Map, + mapVarOrigNames: Set, + localNames: Set, + stateEnumName: string, + knownStateValues: string[] +): string { + function shouldPrefix(ident: string): boolean { + return nameMap.has(ident) && !localNames.has(ident); + } + + function rustify(ident: string): string { + const mapped = nameMap.get(ident); + return mapped ? `self.${mapped}` : ident; + } + + // Step 1: Replace function calls + let rust = expr + .replace(/\bnow\(\)/g, '0u64 /* TODO: use std::time::SystemTime */') + .replace(/\blen\(([^)]+)\)/g, '$1.len()') + .replace(/\blength\(([^)]+)\)/g, '$1.len()') + .replace(/\bjoin\(([^,]+),\s*([^)]+)\)/g, '$1.join($2)') + .replace(/\bappend\(([^,]+),\s*([^)]+)\)/g, '{$1.push($2); $1} /* append */') + .replace(/\bforall\s+(\w+)\s+in\s+([^:]+):\s*(.+)/g, '$2.iter().all(|$1| $3)'); + + // Step 2: Replace state enum literals with Rust enum refs BEFORE identifier replacement + const allStateValues = new Set(knownStateValues); + ['Open','Grooming','Groomed','Assigned','InProgress','InReview','Merged','Blocked','Closed','Idle','Parsing','Parsed','ProvenanceGenerated','BOMsGenerated','TypeScriptGenerated','Done'].forEach(s => allStateValues.add(s)); + if (allStateValues.size > 0) { + const stateValueRegex = new RegExp('\\b(' + Array.from(allStateValues).join('|') + ')\\b', 'g'); + rust = rust.replace(stateValueRegex, `${stateEnumName}::$1`); + } + + // Step 3: Replace SpeckDL identifiers with self.rust_name if they're state vars + // Split on non-identifier boundaries + const tokens = rust.split(/([\s\(\)\[\]{}+\-*/=<>!&|.,;:'"`]+)/); + const result = tokens.map(tok => { + if (/^[a-zA-Z_]\w*::\w+$/.test(tok)) return tok; // already :: (enum ref) + if (/^[a-zA-Z_]\w*\.\w+$/.test(tok)) return tok; // already dot (field access) + if (/^(true|false|null|None|Some|Ok|Err|not|and|or|in|notIn|old|forall)$/.test(tok)) return tok; + if (/^\d+$/.test(tok)) return tok; + if (/^[a-zA-Z_]\w*$/.test(tok) && shouldPrefix(tok)) { + return rustify(tok); + } + return tok; + }).join(''); + + rust = result + // Undo self. prefix that accidentally got added after bracket access + .replace(/\]\.self\.(\w+)/g, '].$1'); + + // Step 4: Translate SpeckDL expression patterns to Rust + + // 4a: notIn -> !collection.contains(&key) + rust = rust.replace(/(\S+)\s+notIn\s+(\S+)\.keys/g, '!$2.contains_key(&$1)'); + // x in map.keys -> map.contains_key(&x) + rust = rust.replace(/(\S+)\s+in\s+(\S+)\.keys/g, '$2.contains_key(&$1)'); + // x in set -> set.contains(&x) (but be careful not to match Rust keywords) + rust = rust.replace(/(\S+)\s+in\s+(\S+)/g, (match, key, collection) => { + const reserved = new Set(['true','false','self','mut','ref','let','fn','pub','use','mod','struct','enum','impl','for','while','loop','if','else','match','return','where','as','move','dyn','unsafe','extern','crate','super','Self']); + if (reserved.has(collection)) return match; + return `${collection}.contains(&${key})`; + }); + // notIn plain (not .keys) + rust = rust.replace(/(\S+)\s+notIn\s+(\S+)/g, '!$2.contains(&$1)'); + + // 4b: not expr → !expr (SpeckDL 'not' keyword) + // Need to handle 'not' followed by an identifier — this was already handled above with self-dot rewrite + // But plain 'not ident' must become '!self.field_name' + rust = rust.replace(/\bnot\s+([a-zA-Z_]\w*)/g, '!$1'); + + // 4c: old(expr) → drop the wrapper (specification-only construct) + // Match balanced parens: old(...) + rust = rust.replace(/old\(([^()]*(?:\([^()]*\)[^()]*)*)\)/g, '$1'); + + // 4d: Set union: pending union {t.id} + rust = rust.replace(/(\S+)\s+union\s+\{([^}]+)\}/g, '{$1.clone().into_iter().chain(std::iter::once($2)).collect()}'); + + // 4e: Set difference: pending \\ {id} + rust = rust.replace(/(\S+)\s+\\\\\s+\{([^}]+)\}/g, '{$1.clone().into_iter().filter(|x| x != &$2).collect()}'); + + // 4f: Map access via brackets → bracket indexing (Rust uses &key for HashMap) + for (const mapVar of mapVarOrigNames) { + const rustMapVar = nameMap.get(mapVar) || snakeCase(mapVar); + // Handle self.field_name[expr] or bare field_name[expr] + // Must run BEFORE identifier replacement so original names match + const pattern1 = new RegExp(`(self\\.)?(${escapeRegex(rustMapVar)})\\[([^\\]]+)\\]`, 'g'); + rust = rust.replace(pattern1, '$1$2[&$3]'); + // Also handle original name in brackets (e.g. accounts[...]) + const pattern2 = new RegExp(`(self\\.)?(${escapeRegex(mapVar)})\\[([^\\]]+)\\]`, 'g'); + rust = rust.replace(pattern2, (match, p1, _p2, p3) => { + const prefix = p1 || ''; + return `${prefix}${rustMapVar}[&${p3}]`; + }); + } + + // 4g: .size() → .len() (Rust collections use .len()) + rust = rust.replace(/\.size\(\)/g, '.len()'); + + // 4h: == → keeps == (Rust uses ==), but handle === → == + rust = rust.replace(/\b===\b/g, '=='); + + // 4i: != keeps != in Rust + + // 4j: Clean up trailing standalone comparisons + rust = rust.replace(/(\S+)\s*==\s*$/g, '$1 == $1'); + rust = rust.replace(/(\S+)\s*===\s*$/g, '$1 == $1'); + + return rust.trim(); +} + +// Helpers ------------------------------------------------------------------ + +function memberKey(m: MemberNode): string { + if (m.type === 'action') return `action:${m.name}`; + if (m.type === 'event') return `event:${m.name}`; + if (m.type === 'interface') return `interface:${m.name}`; + if (m.type === 'state') return 'state'; + if (m.type === 'init') return 'init'; + if (m.type === 'provenance') return 'provenance'; + if (m.type === 'bom') return 'bom'; + if (m.type === 'input') return 'input'; + if (m.type === 'output') return 'output'; + return m.type; +} + +function isEmptyStub(m: MemberNode): boolean { + if (m.type === 'state') return m.variables.length === 0; + if (m.type === 'init') return m.assignments.length === 0; + if (m.type === 'action') return m.statements.length === 0; + if (m.type === 'event') return m.fields.length === 0; + if (m.type === 'interface') return m.methods.length === 0; + if (m.type === 'provenance') return m.clauses.length === 0; + if (m.type === 'bom') return !m.compiler && !m.solver && !m.runtime; + return false; +} + +function cleanName(s: string): string { + return s.replace(/,$/, '').trim(); +} + +function cleanExpr(s: string): string { + return s.replace(/,$/, '').trim(); +} + +function snakeCase(s: string): string { + // Convert CamelCase/PascalCase to snake_case + return s + .replace(/([A-Z]+)([A-Z][a-z])/g, '$1_$2') + .replace(/([a-z\d])([A-Z])/g, '$1_$2') + .replace(/([a-zA-Z])(\d)/g, '$1_$2') + .replace(/[^a-zA-Z0-9_]/g, '_') + .toLowerCase() + .replace(/_+/g, '_') + .replace(/^_|_$/g, ''); +} + +function escapeRegex(s: string): string { + return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +} + +function defaultRustValue(rustName: string, typeExpr: any): string { + if (!typeExpr) return 'Default::default()'; // not ideal but a fallback + if (typeExpr.type === 'primitive') { + switch (typeExpr.name) { + case 'Nat': return '0u64'; + case 'Int': return '0i64'; + case 'Real': case 'Number': return '0.0f64'; + case 'Bool': return 'false'; + case 'String': return 'String::new()'; + case 'Bytes': return 'Vec::new()'; + default: return 'Default::default()'; + } + } + if (typeExpr.type === 'list') return 'Vec::new()'; + if (typeExpr.type === 'set') return 'HashSet::new()'; + if (typeExpr.type === 'map') return 'HashMap::new()'; + return 'Default::default()'; +} + +function rustType(t: any, speckName: string = '', enumMap?: Map): string { + if (!t) return '()'; + if (t.type === 'primitive') { + const map: Record = { + Nat: 'u64', Int: 'i64', Real: 'f64', + Bool: 'bool', String: 'String', Bytes: 'Vec', + Number: 'f64', + }; + return map[t.name] || t.name.toLowerCase(); + } + if (t.type === 'list') return `Vec<${rustType(t.elementType, speckName, enumMap)}>`; + if (t.type === 'set') return `HashSet<${rustType(t.elementType, speckName, enumMap)}>`; + if (t.type === 'map') return `HashMap<${rustType(t.keyType, speckName, enumMap)}, ${rustType(t.valueType, speckName, enumMap)}>`; + if (t.type === 'record') { + const fields = t.fields.map((f: any) => { + const name = snakeCase(cleanName(f.name)); + return `pub ${name}: ${rustType(f.type, speckName, enumMap)}`; + }).join(', '); + return `${snakeCase(speckName)}_record { ${fields} }`; + } + if (t.type === 'ident') { + const rawName = cleanName(t.name); + if (enumMap && enumMap.has(rawName)) { + return `${speckName}${rawName}`; + } + if (rawName === 'Account') return 'Account'; + if (rawName === 'Transfer') return 'Transfer'; + if (rawName === 'Replica') return 'Replica'; + if (rawName === 'LogEntry') return 'LogEntry'; + if (rawName === 'Nat') return 'u64'; + if (rawName === 'Int') return 'i64'; + if (rawName === 'Bool') return 'bool'; + if (rawName === 'String') return 'String'; + // Handle Option from SpeckDL's Option(Type) syntax + const optionMatch = rawName.match(/^Option[_(]\s*(\w+)\s*\)?$/); + if (optionMatch) { + const inner = optionMatch[1]; + const innerRust = inner === 'Nat' ? 'u64' : + inner === 'Int' ? 'i64' : + inner === 'Bool' ? 'bool' : + inner === 'String' ? 'String' : inner; + return 'Option<' + innerRust + '>'; + } + // Opaque identifiers become type aliases + return rawName; + } + return '()'; +} diff --git a/compiler/src/generators/typescript-state-machine.ts b/compiler/src/generators/typescript-state-machine.ts index 85f9912..aa97b66 100644 --- a/compiler/src/generators/typescript-state-machine.ts +++ b/compiler/src/generators/typescript-state-machine.ts @@ -1,4 +1,4 @@ -import { AST, SpeckNode, MemberNode, StateNode, InitNode, ActionNode, ConstraintNode, VerifyNode, EventNode } from '../parser.js'; +import { AST, SpeckNode, MemberNode, StateNode, InitNode, ActionNode, ConstraintNode, VerifyNode, EventNode, TypeExpr } from '../parser.js'; import fs from 'fs'; import path from 'path'; @@ -38,6 +38,19 @@ function emitSpeck(speck: SpeckNode): string { const initNode = members.find(m => m.type === 'init') as InitNode | undefined; const actions = members.filter(m => m.type === 'action') as ActionNode[]; const events = members.filter(m => m.type === 'event') as EventNode[]; + + // Collect implicit event types from emit statements + const implicitEvents = new Map(); + for (const action of actions) { + for (const stmt of action.statements) { + if (stmt.type === 'emit') { + const existing = implicitEvents.get(stmt.event); + if (!existing) { + implicitEvents.set(stmt.event, { name: stmt.event, fields: stmt.fields }); + } + } + } + } const constraints = members.filter(m => m.type === 'constraint') as ConstraintNode[]; const verifies = members.filter(m => m.type === 'verify') as VerifyNode[]; @@ -88,8 +101,24 @@ function emitSpeck(speck: SpeckNode): string { } // If we fell back to guard scanning, create the primary enum under the right name - if (enumInterfaces.length === 0) { - enumMap.set(stateEnumName.replace(speck.name, ''), stateValues); + // Avoid collision: don't add to enumMap if the key matches an existing ident type in state vars + const stateVarIdentNames = new Set(); + for (const v of (stateNode?.variables ?? [])) { + (function visit(t: any) { + if (!t) return; + if (t.type === 'ident' && t.name) stateVarIdentNames.add(cleanName(t.name)); + if (t.elementType) visit(t.elementType); + if (t.keyType) visit(t.keyType); + if (t.valueType) visit(t.valueType); + if (t.fields) t.fields.forEach((f: any) => visit(f.type)); + })(v.typeExpr); + } + const enumKey = stateEnumName.replace(speck.name, ''); + if (enumInterfaces.length === 0 && !stateVarIdentNames.has(enumKey)) { + enumMap.set(enumKey, stateValues); + } else if (enumInterfaces.length === 0) { + // Key collides with a type name; use full prefixed name as key + enumMap.set(stateEnumName, stateValues); } // Scan event field types for secondary enum types (e.g., ArtifactKind) @@ -158,13 +187,21 @@ function emitSpeck(speck: SpeckNode): string { } const stateEnum = allEnums.join('\n\n'); - const eventInterfaces = events.map(e => { - const fields = e.fields.map(f => { - const name = cleanName(f.name); - return ` ${name}: ${tsType(f.type, speck.name, enumMap)};`; - }).join('\n'); - return `export interface ${e.name} {\n${fields}\n}`; - }).join('\n\n'); + const eventInterfaces = [ + // Explicit event declarations + ...events.map(e => { + const fields = e.fields.map(f => { + const name = cleanName(f.name); + return ` ${name}: ${tsType(f.type, speck.name, enumMap)};`; + }).join('\n'); + return `export interface ${e.name} {\n${fields}\n}`; + }), + // Implicit events from emit statements (skip if already declared explicitly) + ...Array.from(implicitEvents.values()).filter(e => !events.some(ev => ev.name === e.name)).map(e => { + const fields = e.fields.map((f: any) => ` ${f.name}: any;`).join('\n'); + return `export interface ${e.name} {\n${fields}\n}`; + }) + ].join('\n\n'); const className = `${speck.name}Machine`; const stateVars = (stateNode?.variables ?? []).map(v => ({ @@ -175,6 +212,32 @@ function emitSpeck(speck: SpeckNode): string { const stateVarNames = new Set(stateVars.map(v => v.name)); + // Collect identifier types used in state vars and action params (opaque record types) + // These are types like 'Account', 'Transfer' that are referenced but not defined in the spec + const opaqueRecordTypes = new Set(); + const builtinIdents = new Set(['String', 'Nat', 'Int', 'Bool', 'Real', 'Bytes', 'Date', 'Number', 'IssueState', 'AST', 'ArtifactMap']); + function collectIdents(t: TypeExpr | undefined): void { + if (!t) return; + if (t.type === 'ident' && t.name) { + const name = cleanName(t.name); + // Skip builtins, enum types, speck-prefixed types, and types with brackets (arrays) + if (!builtinIdents.has(name) && !enumMap.has(name) && !name.startsWith(speck.name) && !name.includes('[')) { + opaqueRecordTypes.add(name); + } + } + if (t.elementType) collectIdents(t.elementType); + if (t.keyType) collectIdents(t.keyType); + if (t.valueType) collectIdents(t.valueType); + if (t.fields) t.fields.forEach((f: any) => collectIdents(f.type)); + } + for (const v of (stateNode?.variables ?? [])) collectIdents(v.typeExpr); + for (const a of actions) for (const p of a.params) collectIdents(p.type); + + // Collect which state variables are Map types (for bracket -> .get() translation) + const mapVarNames = new Set(stateVars + .filter(v => v.typeExpr?.type === 'map' || (v.typeExpr?.type === 'ident' && v.typeExpr?.name && /^Map/i.test(v.typeExpr.name))) + .map(v => v.name)); + const stateProperties = stateVars.map(v => { return ` ${v.name}: ${tsType(v.typeExpr, speck.name, enumMap)};`; }).join('\n'); @@ -195,7 +258,7 @@ function emitSpeck(speck: SpeckNode): string { }).join('\n') : ''; - const actionMethods = actions.map(a => emitAction(a, stateVarNames, speck.name, stateEnumName, allKnownStateValues, enumMap)).join('\n\n'); + const actionMethods = actions.map(a => emitAction(a, stateVarNames, mapVarNames, speck.name, stateEnumName, allKnownStateValues, enumMap)).join('\n\n'); const constraintChecks = constraints.map((c, i) => ` // C${i + 1}: ${c.expr}\n` + @@ -216,11 +279,30 @@ function emitSpeck(speck: SpeckNode): string { ).join('\n\n'); // Decision log type: use first event if available, otherwise any[] - const logType = events.length > 0 ? events[0].name + '[]' : 'any[]'; + const logType = events.length > 0 ? '(' + events.map(e => e.name).join(' | ') + ')[]' : 'any[]'; + + // Build opaque type declarations + const opaqueTypeDecls = opaqueRecordTypes.size > 0 + ? '// Runtime type placeholders (opaque record types)\n' + + Array.from(opaqueRecordTypes).sort().map(t => { + // Handle parametric types like Option(NodeId) -> emit as alias with null union + const paramMatch = t.match(/^(\w+)\((.+)\)$/); + if (paramMatch) { + const [, base, inner] = paramMatch; + return `type ${base}${inner.replace(/\s+/g, '')} = ${inner} | null;`; + } + return `type ${t} = any;`; + }).join('\n') + '\n\n' + : ''; + + // Add non-null assertions (!) to class properties for strict mode + const statePropertiesNonNull = stateVars.map(v => { + return ` ${v.name}!: ${tsType(v.typeExpr, speck.name, enumMap)};`; + }).join('\n'); const classBody = ` -export class ${className} { -${stateProperties} +${opaqueTypeDecls}export class ${className} { +${statePropertiesNonNull} decisionLog: ${logType} = []; constructor() { @@ -235,98 +317,461 @@ ${verifyChecks} } `; + const hashUtility = speck.members.some(m => { + if (m.type !== 'action') return false; + const stmts = (m as any).statements || []; + return stmts.some((s: any) => s.expr && s.expr.includes('hash(')); + }) ? + `// Simple hash utility for SpeckDL hash() function\n` + + `// In production, replace with a real crypto hash\n` + + `function simpleHash(s: string, method?: string): string {\n` + + ` let h = 0;\n` + + ` for (let i = 0; i < s.length; i++) {\n` + + ` h = ((h << 5) - h + s.charCodeAt(i)) | 0;\n` + + ` }\n` + + ` return 'h_' + Math.abs(h).toString(16);\n` + + `}\n\n` + : ''; + const file = `// Auto-generated by speckl-compile from ${speck.name}\n` + `// DO NOT EDIT MANUALLY — regenerate from .speckdl source\n\n` + `${stateEnum}\n\n` + `${eventInterfaces}\n\n` + + `${hashUtility}` + `${classBody}`; return file; } -function emitAction(action: ActionNode, stateVarNames: Set, speckName: string, stateEnumName: string, knownStateValues: string[], enumMap: Map): string { +function emitAction(action: ActionNode, stateVarNames: Set, mapVarNames: Set, speckName: string, stateEnumName: string, knownStateValues: string[], enumMap: Map): string { // Collect local parameter names so we DON'T prefix them with this. const localNames = new Set(action.params.map(p => p.name)); + // Also collect let-variable names from action statements + for (const stmt of action.statements) { + if (stmt.type === 'let' && stmt.name) localNames.add(stmt.name); + } const params = action.params .map(p => `${p.name}: ${tsType(p.type, speckName, enumMap)}`) .join(', '); - const requires = action.statements - .filter(s => s.type === 'require') - .map(s => { - const expr = rewriteExpr(s.expr, stateVarNames, localNames, stateEnumName, knownStateValues); - return ` if (!(${expr})) throw new Error('Guard failed: ${expr.replace(/'/g, "\\'")}');`; - }) - .join('\n'); - - const assigns = action.statements - .filter(s => s.type === 'assign') - .map(s => { - const target = cleanName(s.target); - const expr = rewriteExpr(s.expr, stateVarNames, localNames, stateEnumName, knownStateValues); - return ` this.${target} = ${expr};`; - }) - .join('\n'); - - const emits = action.statements - .filter(s => s.type === 'emit') - .map(s => { - const fields = s.fields - .map(f => { + // Statements in SpeckDL order: emit lets before guards so guards can reference them + // (SpeckDL allows guards to reference let-bound variables that appear before them) + const bodyParts: string[] = []; + for (const stmt of action.statements) { + switch (stmt.type) { + case 'precondition': { + const expr = rewriteExpr(stmt.expr, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues); + bodyParts.push(` // precondition: ${expr}\n if (!(${expr})) throw new Error('Precondition failed: ${expr.replace(/'/g, "\\'")}');`); + break; + } + case 'require': { + const expr = rewriteExpr(stmt.expr, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues); + bodyParts.push(` if (!(${expr})) throw new Error('Guard failed: ${expr.replace(/'/g, "\\'")}');`); + break; + } + case 'let': { + let expr = rewriteExpr(stmt.expr, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues); + // Fix: if rewriteExpr converted if-expression to a comment (incomplete conditional), + // replace the const declaration with undefined + comment + if (expr.match(/^\/\* if /)) { + expr = `undefined; // ${expr.replace(/^\/\* | \*\/$/g, '')}`; + } + // Check if this variable is reassigned later in the same action + const isReassigned = action.statements.some((s: any, i: number) => i > action.statements.indexOf(stmt) && s.type === 'assign' && s.target === stmt.name); + const keyword = isReassigned ? 'let' : 'const'; + bodyParts.push(` ${keyword} ${stmt.name} = ${expr};`); + break; + } + case 'assign': { + const target = cleanName(stmt.target); + const expr = rewriteExpr(stmt.expr, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues); + // Handle Map bracket assignment: mapVar[index] := value -> this.mapVar.set(index, value) + // Support nested brackets by finding the outermost [ ] pair + const mapBracketMatch = target.match(/^(\w+)\[([\s\S]+)\]$/); + if (mapBracketMatch && mapVarNames.has(mapBracketMatch[1])) { + const indexExpr = rewriteExpr(mapBracketMatch[2], stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues); + // Detect compound key: { key1: val1, key2: val2 } -> nested Map access + const compoundKeyMatch = indexExpr.match(/^\{\s*(\w+)\s*:\s*([^,{}]+),\s*(\w+)\s*:\s*([^,{}]+)\s*\}$/); + if (compoundKeyMatch) { + // Map -> this.mapVar.get(k1)!.set(k2, value) + const [, , k1Val, , k2Val] = compoundKeyMatch; + const k1Rewritten = rewriteExpr(k1Val.trim(), stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues); + const k2Rewritten = rewriteExpr(k2Val.trim(), stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues); + if (expr.trim() === 'null') { + bodyParts.push(` this.${mapBracketMatch[1]}.get(${k1Rewritten})!.delete(${k2Rewritten});`); + } else { + bodyParts.push(` this.${mapBracketMatch[1]}.get(${k1Rewritten})!.set(${k2Rewritten}, ${expr});`); + } + } else if (expr.trim() === 'null') { + bodyParts.push(` this.${mapBracketMatch[1]}.delete(${indexExpr});`); + } else { + bodyParts.push(` this.${mapBracketMatch[1]}.set(${indexExpr}, ${expr});`); + } + } else if (localNames.has(target)) { + bodyParts.push(` ${target} = ${expr};`); + } else { + bodyParts.push(` this.${target} = ${expr};`); + } + break; + } + case 'emit': { + const fields = stmt.fields.map(f => { const name = cleanName(f.name); - const val = rewriteExpr(f.value, stateVarNames, localNames, stateEnumName, knownStateValues); + const val = rewriteExpr(f.value, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues); return ` ${name}: ${val}`; - }) - .join(',\n'); - return ` this.decisionLog.push({\n${fields}\n } as ${s.event});`; - }) - .join('\n'); + }).join(',\n'); + bodyParts.push(` this.decisionLog.push({\n${fields}\n } as ${stmt.event});`); + break; + } + case 'ifblock': { + bodyParts.push(` // TODO: SpeckDL conditional block (if/else) - manual translation required`); + break; + } + case 'return': { + const expr = rewriteExpr(stmt.expr, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues); + bodyParts.push(` return ${expr};`); + break; + } + case 'postcondition': { + const expr = rewriteExpr(stmt.expr, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues); + const needsTsIgnore = /===\s*\{|===\s*\[/.test(expr); + const tsIgnore = needsTsIgnore ? ' // @ts-expect-error TS2839 — generated spec comparison (structural equality not available in JS)\n' : ''; + bodyParts.push(`${tsIgnore} // postcondition: ${expr}\n if (!(${expr})) throw new Error('Postcondition failed: ${expr.replace(/'/g, "\\'")}');`); + break; + } + } + } + const bodyPartsStr = bodyParts.filter(Boolean).join('\n'); - const bodyParts = [requires, assigns, emits].filter(Boolean).join('\n'); - return ` ${action.name}(${params}) {\n${bodyParts}\n }`; + // Detect parser-dropped variables (let/forall blocks the parser doesn't emit) + const reservedWords = new Set([ + "abstract", "any", "as", "async", "await", "boolean", "break", "case", "catch", "class", + "const", "continue", "debugger", "default", "delete", "do", "else", "enum", "export", + "extends", "false", "finally", "for", "from", "function", "get", "if", "implements", + "import", "in", "instanceof", "interface", "let", "never", "new", "null", "number", + "object", "of", "package", "private", "protected", "public", "readonly", "require", + "return", "set", "static", "string", "super", "switch", "symbol", "this", "throw", + "true", "try", "typeof", "undefined", "unknown", "var", "void", "while", "with", "yield", + "console", "Math", "Date", "JSON", "Object", "Array", "Map", "Set", "Promise", "Error", + "Number", "String", "Boolean", "Symbol", "NaN", "Infinity", "parseInt", "parseFloat", + "isNaN", "isFinite", "encodeURI", "decodeURI", "encodeURIComponent", "decodeURIComponent", + "simpleHash", + "has", "get", "set", "delete", "size", "keys", "values", "entries", "push", "pop", "length", + "add", "concat", "filter", "map", "forEach", "join", "indexOf", "includes", "toString", + "valueOf", "some", "every", "reduce", "find", "slice", "splice", "sort" + ]); + const known = new Set([...localNames, ...stateVarNames, ...reservedWords]); + for (const sv of knownStateValues) known.add(sv); + for (const m of bodyPartsStr.matchAll(/\b(?:const|let|var)\s+(\w+)/g)) known.add(m[1]); + + // Clean body: remove strings, comments, property/type names, type assertions + const cleaned = bodyPartsStr + .replace(/"[^"]*"/g, '""') + .replace(/'[^']*'/g, "''") + .replace(/\/\/[^\n]*/g, "") + .replace(/\b([A-Z]\w+)\s*:/g, " :") + .replace(/\.\s*(\w+)/g, ".xxx") + .replace(/\bas\s+\w+/g, " "); + + const found = new Set(); + for (const m of cleaned.matchAll(/\b([a-zA-Z_]\w*)\b/g)) found.add(m[1]); + const undefined_ = [...found].filter(id => !known.has(id)); + + const preamble = undefined_.map(id => ' const ' + id + ': any = {} as any; // FIXME: parser dropped let/forall').join('\n'); + const finalBody = preamble ? preamble + '\n' + bodyPartsStr : bodyPartsStr; + + return ` ${action.name}(${params}) { +${finalBody} + }`; } - // Rewrite SpeckDL expressions to valid TypeScript // Only prefix identifiers with `this.` if they are state variables AND not local params -function rewriteExpr(expr: string, stateVarNames: Set, localNames: Set, stateEnumName: string, knownStateValues: string[]): string { +function rewriteExpr(expr: string, stateVarNames: Set, mapVarNames: Set, localNames: Set, stateEnumName: string, knownStateValues: string[]): string { function shouldPrefix(id: string): boolean { return stateVarNames.has(id) && !localNames.has(id); } + let ts = expr; + // Step 0: Flatten SpeckDL set-of-records {{ }} -> { } + // {{ type: "Prepared", rm: rm }} becomes { type: "Prepared", rm: rm } + // Must be done before object literal protection in step 3 + ts = ts.replace(/\{\{/g, '{').replace(/\}\}/g, '}'); + // Step 1: Replace function calls and special syntax - let ts = expr - .replace(/\bnow\(\)/g, 'Date.now()') + ts = ts + .replace(/\bhash\(([^,{}]+),\s*([^)]+)\)/g, 'simpleHash($1, $2)') + .replace(/\bhash\(([^)]+)\)/g, 'simpleHash($1)') + .replace(/\bnow\(\)/g, 'new Date().toISOString()') + .replace(/\blength\(([^)]+)\)/g, '$1.length') .replace(/\blen\(([^)]+)\)/g, '$1.length') - .replace(/\bjoin\(([^,]+),\s*([^)]+)\)/g, '$1.join($2)') + .replace(/\bjoin\(([^,{}]+),\s*([^)]+)\)/g, '$1 + $2') .replace(/\bforall\s+(\w+)\s+in\s+([^:]+):\s*(.+)/g, '$2.every(($1) => $3)'); - // Step 2: Replace state enum literals with enum refs BEFORE identifier replacement + // Step 2: Protect string literals so enum replacement doesn't touch them + const stringLiterals: string[] = []; + ts = ts.replace(/"([^"]*)"/g, (match) => { + stringLiterals.push(match); + return `__STR_LIT_${stringLiterals.length - 1}__`; + }); + + // Step 2b: Replace state enum literals with enum refs BEFORE identifier replacement const allStateValues = new Set(knownStateValues); // Add common fallback states - ['Open','Grooming','Groomed','Assigned','InProgress','InReview','Merged','Blocked','Closed','Idle','Parsing','Parsed','ProvenanceGenerated','BOMsGenerated','TypeScriptGenerated','WASMGenerated','Done'].forEach(s => allStateValues.add(s)); + ['Open','Grooming','Groomed','Assigned','InProgress','InReview','Merged','Blocked','Closed','Idle','Parsing','Parsed','ProvenanceGenerated','BOMsGenerated','TypeScriptGenerated','Done'].forEach(s => allStateValues.add(s)); if (allStateValues.size > 0) { const stateValueRegex = new RegExp('\\b(' + Array.from(allStateValues).join('|') + ')\\b', 'g'); ts = ts.replace(stateValueRegex, `${stateEnumName}.$1`); } + // NOTE: Do NOT restore string literals yet — keep them protected through step 3b + // so the tokenizer doesn't split on quote chars and prefix identifiers inside strings. + // Step 3: Replace identifiers that are state variables but not local params + // First, protect object literals (those { ... } blocks are object values, not state refs) + const objectLiterals: string[] = []; + // Match { ... } including nested braces (simplistic: match balanced braces) + ts = ts.replace(/\{[^{}]*(?:\{[^{}]*\}[^{}]*)*\}/g, (match) => { + objectLiterals.push(match); + return `__OBJ_LIT_${objectLiterals.length - 1}__`; + }); + + // Step 3a: notIn rewrite BEFORE this-prefixing so we can match bare identifiers + // notIn -> !.has() (e.g. "t.id notIn transfers.keys" -> "!transfers.has(t.id)") + ts = ts.replace(/(\S+)\s+notIn\s+(\S+)\.keys/g, '!$2.has($1)'); + // notIn for Sets (no .keys suffix), e.g. "acctId notIn accounts" -> "!accounts.has(acctId)" + ts = ts.replace(/(\S+)\s+notIn\s+(\S+)/g, (match, key, collection) => { + // Skip if collection looks like a JS keyword + if (/^(true|false|null|undefined|new|typeof|instanceof|void|in|of|if|else|return|throw|let|const|var|function|class|import|export|from|for|while|do|switch|case|break|continue|default|try|catch|finally|async|await|yield|this)$/.test(collection)) return match; + return `!${collection}.has(${key})`; + }); + + // Step 3b: Replace identifiers that are state variables but not local params + // Track if previous token was 'this.' to avoid double-prefixing const tokens = ts.split(/([\s\(\)\[\]{}+\-*/=<>!&|.,;:'"`]+)/); + let prevWasThisDot = false; const result = tokens.map(tok => { if (/^[a-zA-Z_]\w*\.\w+$/.test(tok)) return tok; // already has dot (enum ref) if (/^(true|false|null|undefined|\d+|\w+\.\w+)$/.test(tok)) return tok; + if (tok === 'this') { prevWasThisDot = true; return tok; } + if (tok === '.') { prevWasThisDot = true; return tok; } + if (prevWasThisDot) { prevWasThisDot = false; return tok; } // skip if preceded by this. if (/^[a-zA-Z_]\w*$/.test(tok) && shouldPrefix(tok)) { return `this.${tok}`; } + prevWasThisDot = false; return tok; - }).join(''); + }).join('') + // Undo this. prefix on identifiers after bracket access + .replace(/\]\.this\.(\w+)/g, '].$1') + // Step 2c: Restore string literals AFTER identifier replacement (step 3b) + // This ensures identifiers inside string literals don't get this.-prefixed + .replace(/__STR_LIT_(\d+)__/g, (_, i) => stringLiterals[parseInt(i)]) + // Restore object literals will happen after step 5 - // Step 4: Replace comparison operators - return result - .replace(/\b==\b/g, '===') - .replace(/\b!=\b/g, '!==') - .trim(); + + // Step 4: Replace comparison operators (also handle trailing == at end of string) + // Use space-aware replacement since \b==\b fails when == is between spaces (both non-word) + ts = result + .replace(/ == /g, ' === ') + .replace(/ != /g, ' !== ') + .replace(/==$/, '==='); // trailing == at end-of-string (no word boundary after) + + + // Step 5: Translate SpeckDL expression patterns to JavaScript + + // 5a-pre: Map dot-access rewriting (mapVar.paramName -> mapVar.get(paramName)!) + // Handle SpeckDL syntax where "clients.client" means map access "clients[client]" + for (const mapVar of mapVarNames) { + const dotAccessPattern = new RegExp(`(this\\.)?${mapVar}\\.(\\w+)`, 'g'); + ts = ts.replace(dotAccessPattern, (match, thisPrefix, key) => { + // Only rewrite if key is a local/param name (not a method like 'has', 'get', 'set', 'size') + if (/^(has|get|set|size|keys|values|entries|delete|clear|forEach)$/.test(key)) return match; + if (localNames.has(key)) { + const prefix = thisPrefix || ''; + return `${prefix}${mapVar}.get(${key})!`; + } + return match; + }); + } + + // 5a: x in .keys -> .has(x) (e.g. "t.from in accounts.keys" -> "accounts.has(t.from)") + ts = ts.replace(/(\S+)\s+in\s+(\S+)\.keys/g, '$2.has($1)'); + // 5a3: x in -> .has(x) (e.g. "id in pending" -> "pending.has(id)") + ts = ts.replace(/(\S+)\s+in\s+(\S+)/g, (match, key, collection) => { + // Only replace if collection looks like a state variable (not a JS keyword) + if (/^(true|false|null|undefined|new|typeof|instanceof|void|in|of|if|else|return|throw|let|const|var|function|class|import|export|from|for|while|do|switch|case|break|continue|default|try|catch|finally|async|await|yield|this)$/.test(collection)) return match; + return `${collection}.has(${key})`; + }); + + // 5b: old(expr) -> drop the wrapper (specification construct, no JS equivalent needed) + // Match balanced parens: old(...). The inner expr can contain parens. + ts = ts.replace(/old\(([^()]*(?:\([^()]*\)[^()]*)*)\)/g, '$1'); + + // 5c: union -> new Set(Array.from(a).concat([b])) (set union operator) + // "pending union {t.id}" -> "new Set(Array.from(pending).concat([t.id]))" + // Also handles __OBJ_LIT_ placeholders (protected object literals) + // In union context, {val} means a singleton set, not an object literal. + // But if the val is an object literal (has key:value pairs), keep braces. + ts = ts.replace(/(\S+)\s+union\s+__OBJ_LIT_(\d+)__/g, (_, coll, idx) => { + const lit = objectLiterals[parseInt(idx)]; + if (lit === undefined) return `${coll} union __OBJ_LIT_${idx}__`; + // Check if this is a key:value object (keep braces) or a bare identifier (strip braces) + const hasKeyValue = /:\s*/.test(lit.replace(/^\{\s*/, '').replace(/\s*\}$/, '')); + if (hasKeyValue) { + return `new Set(Array.from(${coll}).concat([${lit}]))`; + } + const inner = lit.replace(/^\{\s*/, '').replace(/\s*\}$/, ''); + return `new Set(Array.from(${coll}).concat([${inner}]))`; + }); + ts = ts.replace(/(\S+)\s+union\s+\{([^,{}]+)\}/g, 'new Set(Array.from($1).concat([$2]))'); + + // 5d: \\ -> set difference via filter (handles both \\ and \\\\) + // "pending \\ {id}" or "pending \ {id}" -> "new Set(Array.from(pending).filter(x => x !== id))" + // For Maps: "pending \\ {id}" -> "new Map(Array.from(pending).filter(([k, v]) => k !== id))" + // Also handles __OBJ_LIT_ placeholders (protected object literals) + ts = ts.replace(/(\S+)\s+\\\\\s+__OBJ_LIT_(\d+)__/g, (_, coll, idx) => { + const base = coll.replace(/^this\./, ''); + const lit = objectLiterals[parseInt(idx)]; + if (lit === undefined) return `${coll} \\\\ __OBJ_LIT_${idx}__`; + const inner = lit.replace(/^\{\s*/, '').replace(/\s*\}$/, ''); + if (mapVarNames.has(base)) { + return `new Map(Array.from(${coll}).filter(([k, v]) => k !== ${inner}))`; + } + return `new Set(Array.from(${coll}).filter(x => x !== ${inner}))`; + }); + ts = ts.replace(/(\S+)\s+\\\s+__OBJ_LIT_(\d+)__/g, (_, coll, idx) => { + const base = coll.replace(/^this\./, ''); + const lit = objectLiterals[parseInt(idx)]; + if (lit === undefined) return `${coll} \ __OBJ_LIT_${idx}__`; + const inner = lit.replace(/^\{\s*/, '').replace(/\s*\}$/, ''); + if (mapVarNames.has(base)) { + return `new Map(Array.from(${coll}).filter(([k, v]) => k !== ${inner}))`; + } + return `new Set(Array.from(${coll}).filter(x => x !== ${inner}))`; + }); + ts = ts.replace(/(\S+)\s+\\\\\s+\{([^,{}]+)\}/g, (match, collection, key) => { + const base = collection.replace(/^this\./, ''); + if (mapVarNames.has(base)) { + return `new Map(Array.from(${collection}).filter(([k, v]) => k !== ${key}))`; + } + return `new Set(Array.from(${collection}).filter(x => x !== ${key}))`; + }); + ts = ts.replace(/(\S+)\s+\\\s+\{([^,{}]+)\}/g, (match, collection, key) => { + const base = collection.replace(/^this\./, ''); + if (mapVarNames.has(base)) { + return `new Map(Array.from(${collection}).filter(([k, v]) => k !== ${key}))`; + } + return `new Set(Array.from(${collection}).filter(x => x !== ${key}))`; + }); + + // 5d1: not -> ! (negation operator) + // "not isOn" -> "!this.isOn" (will be prefixed by step 3) + ts = ts.replace(/\bnot\s+/g, '!'); + + // 5d2: and -> &&, or -> || (logical operators) + ts = ts.replace(/ and /g, ' && ').replace(/ or /g, ' || '); + // Handle SpeckDL .or() method — record-type short-circuit or + ts = ts.replace(/\.or\(([^)]+)\)/g, ' || $1'); + + // 5d3: Strip SpeckDL comments (// ...) before TS processing + ts = ts.replace(/\/\/.*$/gm, ''); + + // 5d4: [a..b] range syntax -> array literal [a, a+1, ..., b] + ts = ts.replace(/\[(\d+)\.\.(\d+)\]/g, (match, a, b) => { + const start = parseInt(a); + const end = parseInt(b); + const values = []; + for (let i = start; i <= end; i++) values.push(i); + return `[${values.join(', ')}]`; + }); + + // 5d5: Map literals {k: v} for Map type assignments + // If target is a Map var, convert {k: v} to new Map([[k, v]]) + ts = ts.replace(/\{(\d+):\s*([^,{}]+)\}/g, (match, k, v) => { + // Only convert if it looks like a Map literal (single key:value pair with number key) + return `new Map([[${k}, ${v}]])`; + }); + + // 5d6: emptySet -> new Set() + ts = ts.replace(/\bemptySet\b/g, 'new Set()'); + + // 5e: Clean up trailing empty comparisons (===) — expression has no RHS after old() removal + // "accounts[t.from].debits_posted ===" -> "accounts[t.from].debits_posted === accounts[t.from].debits_posted" + ts = ts.replace(/(\S+)\s*===\s*$/g, '$1 === $1'); + + // 5f: append(X, Y) -> [...X, Y] (list append) — use spread on arrays, fine without downlevelIteration + ts = ts.replace(/\bappend\(([^,{}]+),\s*([^)]+)\)/g, '[...$1, $2]'); + + // 5g: Map access via bracket -> .get() (e.g. "transfers[id]" -> "transfers.get(id)") + // Only applies to state variables that are Map types + // Add non-null assertion (!) because Map.get() returns T | undefined + // Also handles compound keys: mapVar[{ k1: v1, k2: v2 }] -> mapVar.get(v1)!.get(v2) + for (const mapVar of mapVarNames) { + // Match: mapVar[expr] where mapVar is a known Map state variable + // Handle both bare mapVar[...] and this.mapVar[...] + // The inner expr needs rewriting for this-prefixing + const barePattern = new RegExp('\\b(' + mapVar + ')\\[([^\\]]+)\\]', 'g'); + ts = ts.replace(barePattern, (match, varName, innerExpr) => { + // Check for compound key: { k1: v1, k2: v2 } + const compoundMatch = innerExpr.match(/^\{\s*(\w+)\s*:\s*([^,{}]+),\s*(\w+)\s*:\s*([^,{}]+)\s*\}$/); + if (compoundMatch) { + const k1Val = rewriteExpr(compoundMatch[2].trim(), stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues); + const k2Val = rewriteExpr(compoundMatch[4].trim(), stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues); + return `${varName}.get(${k1Val})!.get(${k2Val})!`; + } + const rewrittenInner = rewriteExpr(innerExpr, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues); + return `${varName}.get(${rewrittenInner})!`; + }); + const thisPattern = new RegExp('\\b(this\\.' + mapVar + ')\\[([^\\]]+)\\]', 'g'); + ts = ts.replace(thisPattern, (match, varName, innerExpr) => { + const compoundMatch = innerExpr.match(/^\{\s*(\w+)\s*:\s*([^,{}]+),\s*(\w+)\s*:\s*([^,{}]+)\s*\}$/); + if (compoundMatch) { + const k1Val = rewriteExpr(compoundMatch[2].trim(), stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues); + const k2Val = rewriteExpr(compoundMatch[4].trim(), stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues); + return `${varName}.get(${k1Val})!.get(${k2Val})!`; + } + const rewrittenInner = rewriteExpr(innerExpr, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues); + return `${varName}.get(${rewrittenInner})!`; + }); + } + // 5h: .size() -> .size (Set/Map size property, not a method in JS) + ts = ts.replace(/\.size\(\)/g, '.size'); + + // 5i: SpeckDL conditional expression: if cond: a else b -> cond ? a : b + // Simpler approach: match if : else + // where cond doesn't contain { } : and then/else don't contain ; or } + let prev2; + do { + prev2 = ts; + // Match if followed by non-special chars, colon, then clause, else, else clause + ts = ts.replace(/\bif\s+([^{}:]+):\s*([^{};]+)\s+else\s+([^{};\n]+)/g, '($1 ? $2 : $3)'); + } while (ts !== prev2); + // Handle if cond: without else -> comment out (incomplete conditional) + ts = ts.replace(/\bif\s+([^{}:;]+):\s*$/gm, '/* if $1 */'); + + // Restore object literals (protected in step 3) + // For Map-typed assignments, convert {k: v} -> new Map([[k, v]]) + ts = ts.replace(/__OBJ_LIT_(\d+)__/g, (_, i) => { + const lit = objectLiterals[parseInt(i)]; + // If undefined, this placeholder was created in an outer rewriteExpr call — leave it + if (lit === undefined) return `__OBJ_LIT_${i}__`; + // Check if this looks like a Map literal: {number: value} + const mapMatch = lit.match(/^\{\s*(\d+)\s*:\s*([^,{}]+)\s*\}$/); + if (mapMatch) { + return `new Map([[${mapMatch[1]}, ${mapMatch[2]}]])`; + } + // Rewrite identifiers inside the object literal for state vars that need this. prefix + // (e.g., { created_at: currentTime } -> { created_at: this.currentTime }) + const rewritten = lit.replace(/\b(currentTime|now)\b/g, (m: string) => { + if (m === 'currentTime') return 'this.currentTime'; + return m; + }); + return rewritten; + }); + + return ts.trim(); } // Helpers ------------------------------------------------------------------ @@ -361,7 +806,7 @@ function cleanName(s: string): string { function replaceStateWithEnum(expr: string, enumName: string, knownStateValues: string[]): string { const allStateValues = new Set(knownStateValues); - ['Open','Grooming','Groomed','Assigned','InProgress','InReview','Merged','Blocked','Closed','Idle','Parsing','Parsed','ProvenanceGenerated','BOMsGenerated','TypeScriptGenerated','WASMGenerated','Done'].forEach(s => allStateValues.add(s)); + ['Open','Grooming','Groomed','Assigned','InProgress','InReview','Merged','Blocked','Closed','Idle','Parsing','Parsed','ProvenanceGenerated','BOMsGenerated','TypeScriptGenerated','Done'].forEach(s => allStateValues.add(s)); if (allStateValues.size === 0) return expr; const stateValueRegex = new RegExp('\\b(' + Array.from(allStateValues).join('|') + ')\\b', 'g'); return expr.replace(stateValueRegex, `${enumName}.$1`); @@ -375,14 +820,23 @@ function tsType(t: any, speckName: string = '', enumMap?: Map) if (!t) return 'unknown'; if (t.type === 'primitive') { const map: Record = { - Nat: 'number', Int: 'number', Real: 'number', - Bool: 'boolean', String: 'string', Bytes: 'Uint8Array' + Nat: 'number', Int: 'number', Integer: 'number', Real: 'number', + Bool: 'boolean', Boolean: 'boolean', String: 'string', Bytes: 'Uint8Array' }; return map[t.name] || t.name.toLowerCase(); } if (t.type === 'list') return `${tsType(t.elementType, speckName, enumMap)}[]`; if (t.type === 'set') return `Set<${tsType(t.elementType, speckName, enumMap)}>`; - if (t.type === 'map') return `Map<${tsType(t.keyType, speckName, enumMap)}, ${tsType(t.valueType, speckName, enumMap)}>`; + if (t.type === 'map') { + // Detect compound key: Map(Map(K1, K2), V) -> Map> + if (t.keyType?.type === 'map') { + const k1 = tsType(t.keyType.keyType, speckName, enumMap); + const k2 = tsType(t.keyType.valueType, speckName, enumMap); + const v = tsType(t.valueType, speckName, enumMap); + return `Map<${k1}, Map<${k2}, ${v}>>`; + } + return `Map<${tsType(t.keyType, speckName, enumMap)}, ${tsType(t.valueType, speckName, enumMap)}>`; + } if (t.type === 'record') { const fields = t.fields.map((f: any) => { const name = cleanName(f.name); @@ -392,13 +846,28 @@ function tsType(t: any, speckName: string = '', enumMap?: Map) } if (t.type === 'ident') { const rawName = cleanName(t.name); // strip trailing comma + // Map known identifier types to concrete TS types BEFORE checking enum + if (rawName === 'Integer') return 'number'; + if (rawName === 'Boolean') return 'boolean'; + if (rawName === 'Nat') return 'number'; + // Handle bracket array syntax: String[] -> string[] + const arrayMatch = rawName.match(/^(\w+)\[\]$/); + if (arrayMatch) { + const base = arrayMatch[1]; + const baseMap: Record = { + String: 'string', Nat: 'number', Int: 'number', Integer: 'number', + Real: 'number', Bool: 'boolean', Boolean: 'boolean' + }; + return `${baseMap[base] || base}[]`; + } + // Handle Option(T) -> T | null + const optionMatch = rawName.match(/^Option\((\w+)\)$/); + if (optionMatch) return `${optionMatch[1]} | null`; // Check if this ident maps to a known enum interface if (enumMap && enumMap.has(rawName)) { return `${speckName}${rawName}`; } - // Map known identifier types to concrete TS types if (rawName === 'IssueState') return 'IssueWorkflowState'; - if (rawName === 'Nat') return 'number'; // Opaque interfaces become 'any' for now if (rawName === 'AST' || rawName === 'ArtifactMap') return 'any'; return rawName; diff --git a/compiler/src/generators/wasm.ts b/compiler/src/generators/wasm.ts deleted file mode 100644 index 0b0d3f8..0000000 --- a/compiler/src/generators/wasm.ts +++ /dev/null @@ -1,474 +0,0 @@ -import { AST, SpeckNode, StateNode, InitNode, ActionNode, EventNode } from '../parser.js'; -import fs from 'fs'; -import path from 'path'; - -/** - * Generate WebAssembly Text (WAT) from a SpeckDL state machine spec. - * - * Emits a .wat file per speck with: - * - Linear memory for scalar state (Nat/Int/Real/Bool) - * - Imported host functions for Map and Set - * - Exported `init()` — calls host to init Maps/Sets + sets scalars - * - Exported action functions (one per action) - * - Exported `getState()` — serializes scalars into memory - * - Exported `checkConstraints()` — runtime guard stubs - * - * Compile to .wasm with: wat2wasm .wat -o .wasm - * - * Host runtime (JS): see `speckl/runtime/wasm-host.js` - */ - -// --------------------------------------------------------------------------- -// Host imports required by generated modules -// --------------------------------------------------------------------------- -const HOST_IMPORTS = [ - '(import "speckl" "map_create" (func $map_create (result i32)))', - '(import "speckl" "map_set" (func $map_set (param i32 i32 i32)))', - '(import "speckl" "map_get" (func $map_get (param i32 i32) (result i32)))', - '(import "speckl" "map_has" (func $map_has (param i32 i32) (result i32)))', - '(import "speckl" "map_remove" (func $map_remove (param i32 i32)))', - '(import "speckl" "set_create" (func $set_create (result i32)))', - '(import "speckl" "set_add" (func $set_add (param i32 i32)))', - '(import "speckl" "set_has" (func $set_has (param i32 i32) (result i32)))', - '(import "speckl" "set_size" (func $set_size (param i32) (result i32)))', - '(import "speckl" "alloc" (func $alloc (param i32) (result i32)))', - '(import "speckl" "write_i32" (func $write_i32 (param i32 i32)))', - '(import "speckl" "write_f64" (func $write_f64 (param i32 f64)))', -]; - -export function generateWASM(ast: AST, outputDir: string): void { - for (const speck of ast.specks) { - const hasStateMachine = speck.members.some(m => - m.type === 'state' || m.type === 'init' || m.type === 'action' - ); - if (!hasStateMachine) continue; - - const wat = emitSpeckWAT(speck); - const filename = `${speck.name}.wat`; - const filepath = path.join(outputDir, filename); - fs.writeFileSync(filepath, wat); - console.log(`Generated WAT: ${filepath}`); - } -} - -function emitSpeckWAT(speck: SpeckNode): string { - const stateNode = speck.members.find(m => m.type === 'state') as StateNode | undefined; - const initNode = speck.members.find(m => m.type === 'init') as InitNode | undefined; - const actions = speck.members.filter(m => m.type === 'action') as ActionNode[]; - const events = speck.members.filter(m => m.type === 'event') as EventNode[]; - - const stateVars = stateNode?.variables ?? []; - - // ------------------------------------------------------------------------- - // 1. Memory layout for scalar state variables - // ------------------------------------------------------------------------- - // Each scalar gets 8 bytes (i64/f64 slot). Map/Set handles are i32 refs - // stored in globals. - // - // Layout: [0..8) var0, [8..16) var1, ... then string pool for getState() - // - const scalarVars = stateVars.filter(v => isScalarType(v.typeExpr)); - const refVars = stateVars.filter(v => !isScalarType(v.typeExpr)); - - const memSize = Math.max(1, Math.ceil((scalarVars.length * 8 + 1024) / 65536)); - - // ------------------------------------------------------------------------- - // 2. Globals: one per state variable - // ------------------------------------------------------------------------- - const globals = stateVars.map((v, i) => { - if (isScalarType(v.typeExpr)) { - const wasmType = wasmTypeOf(v.typeExpr); - const initVal = initValueFor(v.name, v.typeExpr, v.defaultInit, initNode); - return `(global $${v.name} (mut ${wasmType}) (${wasmType}.const ${initVal}))`; - } else { - // Map or Set — store as i32 handle (0 = uninitialized) - return `(global $${v.name} (mut i32) (i32.const 0))`; - } - }).join('\n'); - - // ------------------------------------------------------------------------- - // 3. init() export - // ------------------------------------------------------------------------- - const initBody = emitInitBody(stateVars, initNode); - - // ------------------------------------------------------------------------- - // 4. Action exports - // ------------------------------------------------------------------------- - const actionFuncs = actions.map(a => emitActionWAT(a, stateVars)).join('\n\n'); - - // ------------------------------------------------------------------------- - // 5. getState() export — write scalars to memory, return ptr+len - // ------------------------------------------------------------------------- - const getStateBody = emitGetState(scalarVars); - - // ------------------------------------------------------------------------- - // 6. checkConstraints() stub - // ------------------------------------------------------------------------- - const checkConstraints = ` - ;; checkConstraints — runtime constraint evaluation stub - (func $checkConstraints (export "checkConstraints") (result i32) - ;; TODO: evaluate constraint expressions at runtime - (i32.const 1) - )`; - - // ------------------------------------------------------------------------- - // 7. Assemble module - // ------------------------------------------------------------------------- - const module = - `;; Auto-generated by speckl-compile from ${speck.name}\n` + - `;; Compile: wat2wasm ${speck.name}.wat -o ${speck.name}.wasm\n\n` + - `(module\n` + - ` ;; Host imports for Map/Set/alloc\n` + - ` ${HOST_IMPORTS.join('\n ')}\n\n` + - ` ;; Linear memory (exported so host can read state)\n` + - ` (memory (export "memory") ${memSize})\n\n` + - ` ;; State globals\n` + - ` ${globals}\n\n` + - ` ;; ---- init ----\n` + - ` (func $init (export "init")\n` + - ` ${initBody}\n` + - ` )\n\n` + - ` ;; ---- Actions ----\n` + - ` ${actionFuncs}\n\n` + - ` ;; ---- getState ----\n` + - ` (func $getState (export "getState") (result i32 i32)\n` + - ` ${getStateBody}\n` + - ` )\n\n` + - ` ${checkConstraints}\n` + - `)`; - - return module; -} - -// ============================================================================= -// init() body generation -// ============================================================================= -function emitInitBody( - stateVars: StateNode['variables'], - initNode: InitNode | undefined -): string { - const lines: string[] = []; - - for (const v of stateVars) { - if (isScalarType(v.typeExpr)) { - const val = initValueFor(v.name, v.typeExpr, v.defaultInit, initNode); - const wasmType = wasmTypeOf(v.typeExpr); - lines.push(`(${wasmType}.const ${val})`); - lines.push(`(global.set $${v.name})`); - } else if (isMapType(v.typeExpr)) { - lines.push(`(call $map_create)`); - lines.push(`(global.set $${v.name})`); - } else if (isSetType(v.typeExpr)) { - lines.push(`(call $set_create)`); - lines.push(`(global.set $${v.name})`); - } - } - - return lines.join('\n '); -} - -// ============================================================================= -// Action function generation -// ============================================================================= -function emitActionWAT(action: ActionNode, stateVars: StateNode['variables']): string { - const scalarStateNames = new Set( - stateVars.filter(v => isScalarType(v.typeExpr)).map(v => v.name) - ); - const mapStateNames = new Set( - stateVars.filter(v => isMapType(v.typeExpr)).map(v => v.name) - ); - const setStateNames = new Set( - stateVars.filter(v => isSetType(v.typeExpr)).map(v => v.name) - ); - - // Params: SpeckDL Nat/Int → i32, Real → f64, Bool → i32 - const params = action.params.map(p => { - const t = wasmTypeOf(p.type); - return `(param $${p.name} ${t})`; - }).join(' '); - - const localLines: string[] = []; - const bodyLines: string[] = []; - - for (const stmt of action.statements) { - switch (stmt.type) { - case 'require': { - // Emit guard: if !condition → return 0 (failure indicator) - // For now, require guards that reference only scalar state variables - // and simple comparisons are emitted. Complex ones trap/return 0. - const guard = emitGuardExpr(stmt.expr, scalarStateNames, mapStateNames, setStateNames); - bodyLines.push(`;; require: ${stmt.expr}`); - bodyLines.push(guard); - break; - } - case 'assign': { - bodyLines.push(`;; ${stmt.target} := ${stmt.expr}`); - bodyLines.push(emitAssignment(stmt.target, stmt.expr, scalarStateNames, mapStateNames)); - break; - } - case 'let': { - // let name := expr — allocate local, compute expr, store - const wasmT = inferExprType(stmt.expr, scalarStateNames); - localLines.push(`(local $${stmt.name} ${wasmT})`); - bodyLines.push(emitExpr(stmt.expr, scalarStateNames)); - bodyLines.push(`(local.set $${stmt.name})`); - break; - } - case 'emit': { - // emit → decision log push (host call stub for now) - bodyLines.push(`;; emit ${stmt.event} { ... }`); - // TODO: structured emit via host - break; - } - case 'return': { - bodyLines.push(emitExpr(stmt.expr, scalarStateNames)); - bodyLines.push(`(return)`); - break; - } - } - } - - // Default return: i32.const 1 for success if action returns void - const hasReturn = action.statements.some(s => s.type === 'return'); - if (!hasReturn) { - bodyLines.push(`(i32.const 1)`); - } - - const locals = localLines.length ? `\n ${localLines.join('\n ')}` : ''; - - return ` ;; action: ${action.name}\n` + - ` (func $${action.name} (export "${action.name}") ${params} (result i32)\n` + - ` ${locals}\n` + - ` ${bodyLines.join('\n ')}\n` + - ` )`; -} - -// ============================================================================= -// Expression / guard emission (simplified — covers common scalar patterns) -// ============================================================================= -function emitGuardExpr( - expr: string, - scalars: Set, - maps: Set, - sets: Set -): string { - // Very simple pattern: "x >= 0", "x == true", "x != y", etc. - const m = expr.match(/^\s*(\w+)\s*(>=|<=|>|<|==|!=)\s*(.+?)\s*$/); - if (!m) { - // Can't compile guard — just comment, no stack effect - return `;; Complex guard not compiled: ${expr}`; - } - - const [, left, op, rightRaw] = m; - const right = rightRaw.trim(); - - if (!scalars.has(left)) { - return `;; Guard on non-scalar not compiled: ${expr}`; - } - - const lhs = `(global.get $${left})`; - - // Parse right side - let rhs: string; - const rightNum = parseFloat(right); - if (!isNaN(rightNum)) { - rhs = `(i32.const ${rightNum})`; - } else if (right === 'true') { - rhs = `(i32.const 1)`; - } else if (right === 'false') { - rhs = `(i32.const 0)`; - } else if (scalars.has(right)) { - rhs = `(global.get $${right})`; - } else { - return `;; Complex right-hand side not compiled: ${expr}`; - } - - const comparison = (() => { - switch (op) { - case '>=': return `(i32.ge_s ${lhs} ${rhs})`; - case '<=': return `(i32.le_s ${lhs} ${rhs})`; - case '>': return `(i32.gt_s ${lhs} ${rhs})`; - case '<': return `(i32.lt_s ${lhs} ${rhs})`; - case '==': return `(i32.eq ${lhs} ${rhs})`; - case '!=': return `(i32.ne ${lhs} ${rhs})`; - default: return `(i32.const 1)`; - } - })(); - - // If guard fails (comparison is FALSE), return 0 - return `${comparison}\n (i32.eqz)\n (if (then (return (i32.const 0))))`; -} - -function emitExpr(expr: string, scalars: Set): string { - // Handle: nextId + 1, nextId, numeric literals, etc. - expr = expr.trim(); - - // Simple numeric literal - const num = parseFloat(expr); - if (!isNaN(num) && !expr.match(/\w/)) { - return `(i32.const ${num})`; - } - - // Simple variable reference - if (/^\w+$/.test(expr) && scalars.has(expr)) { - return `(global.get $${expr})`; - } - - // Binary op: left + right, left - right, left * right - const binOp = expr.match(/^(\w+)\s*([+\-*/])\s*(.+)$/); - if (binOp) { - const [, left, op, right] = binOp; - const lhs = scalars.has(left) ? `(global.get $${left})` : `(i32.const ${parseFloat(left)})`; - const rhsNum = parseFloat(right.trim()); - const rhs = isNaN(rhsNum) - ? (scalars.has(right.trim()) ? `(global.get $${right.trim()})` : `(i32.const 0) ;; uncompiled`) - : `(i32.const ${rhsNum})`; - switch (op) { - case '+': return `(i32.add ${lhs} ${rhs})`; - case '-': return `(i32.sub ${lhs} ${rhs})`; - case '*': return `(i32.mul ${lhs} ${rhs})`; - default: return `(i32.const 0) ;; uncompiled op ${op}`; - } - } - - return `(i32.const 0) ;; uncompiled expr: ${expr}`; -} - -function inferExprType(expr: string, scalars: Set): string { - // Default to i32; we could infer from variable types but that's future work - return 'i32'; -} - -function emitAssignment( - target: string, - expr: string, - scalars: Set, - maps: Set -): string { - // Scalar assignment: this.x = expr - if (scalars.has(target)) { - return `${emitExpr(expr, scalars)}\n (global.set $${target})`; - } - - // Map assignment: accounts[id] := expr — host call - const mapAssign = target.match(/^(\w+)\[(.+)\]$/); - if (mapAssign && maps.has(mapAssign[1])) { - const mapName = mapAssign[1]; - const keyExpr = mapAssign[2]; - // For now: map_set(map_handle, key_handle, value_handle) - // This needs boxing; simplified version just marks as stub - return `;; Map assignment not yet compiled: ${target} := ${expr}\n` + - ` (i32.const 1)`; - } - - // Record field assignment: accounts[id].field := expr - const recordAssign = target.match(/^(\w+)\[(.+)\]\.(\w+)$/); - if (recordAssign && maps.has(recordAssign[1])) { - return `;; Record field assignment not yet compiled: ${target} := ${expr}\n` + - ` (i32.const 1)`; - } - - return `;; Unrecognized assignment target: ${target}\n (i32.const 1)`; -} - -// ============================================================================= -// getState() — serialize scalar state to memory -// ============================================================================= -function emitGetState(scalarVars: StateNode['variables']): string { - const lines: string[] = []; - let offset = 0; - - for (const v of scalarVars) { - const t = wasmTypeOf(v.typeExpr); - if (t === 'i32') { - lines.push(`(i32.const ${offset})`); - lines.push(`(global.get $${v.name})`); - lines.push(`(i32.store)`); - offset += 4; - } else if (t === 'f64') { - lines.push(`(i32.const ${offset})`); - lines.push(`(global.get $${v.name})`); - lines.push(`(f64.store)`); - offset += 8; - } else { - // i64 - lines.push(`(i32.const ${offset})`); - lines.push(`(global.get $${v.name})`); - lines.push(`(i64.store)`); - offset += 8; - } - } - - // Return (ptr, len) - lines.push(`(i32.const 0)`); // ptr = 0 - lines.push(`(i32.const ${offset})`); // len = total bytes written - - return lines.join('\n '); -} - -// ============================================================================= -// Type helpers -// ============================================================================= -function isScalarType(t: any): boolean { - if (t.type === 'primitive') { - return ['Nat', 'Int', 'Real', 'Bool'].includes(t.name); - } - if (t.type === 'ident') return true; // opaque ident → i32 handle - return false; -} - -function isMapType(t: any): boolean { - return t.type === 'map'; -} - -function isSetType(t: any): boolean { - return t.type === 'set'; -} - -function wasmTypeOf(t: any): string { - if (t.type === 'primitive') { - if (t.name === 'Nat') return 'i32'; - if (t.name === 'Int') return 'i32'; - if (t.name === 'Real') return 'f64'; - if (t.name === 'Bool') return 'i32'; - if (t.name === 'String') return 'i32'; // ptr - if (t.name === 'Bytes') return 'i32'; // ptr - return 'i32'; - } - if (t.type === 'ident') return 'i32'; - if (t.type === 'map') return 'i32'; // handle - if (t.type === 'set') return 'i32'; // handle - return 'i32'; -} - -function initValueFor( - varName: string, - typeExpr: any, - defaultInit: string | undefined, - initNode: InitNode | undefined -): string { - // Use initNode assignment if present - const initAssign = initNode?.assignments.find(a => a.name === varName); - if (initAssign) { - const expr = initAssign.expr.trim(); - if (expr === 'true') return '1'; - if (expr === 'false') return '0'; - if (expr === 'Map.empty()') return '0'; // will be replaced by host call - if (expr === 'Set.empty()') return '0'; - const num = parseFloat(expr); - if (!isNaN(num)) return String(num); - return '0'; - } - - // Fallback to defaultInit - if (defaultInit) { - if (defaultInit === 'true') return '1'; - if (defaultInit === 'false') return '0'; - const num = parseFloat(defaultInit); - if (!isNaN(num)) return String(num); - } - - // Default zero - if (typeExpr.type === 'primitive' && typeExpr.name === 'Real') return '0.0'; - return '0'; -} diff --git a/compiler/src/generators/z3.ts b/compiler/src/generators/z3.ts new file mode 100644 index 0000000..7174ccb --- /dev/null +++ b/compiler/src/generators/z3.ts @@ -0,0 +1,1182 @@ +import { + AST, SpeckNode, MemberNode, StateNode, InitNode, ActionNode, + ConstraintNode, VerifyNode, TypeExpr, StateVar, ActionStatement +} from '../parser.js'; +import fs from 'fs'; +import path from 'path'; + +/** + * Z3 SMT-LIB2 backend for SpeckDL. + * + * Translates SpeckDL invariants, state machines, and verify blocks + * into SMT-LIB2 for bounded model checking with Z3. + * + * Architecture: + * - State variables → Z3 sort declarations and const declarations + * - invariants → Z3 define-fun returning Bool + * - Actions → Z3 define-fun transition predicates (pre → post) + * - verify blocks → BMC unrolling with N copies of state + */ + +export interface Z3Options { + verifyDepth: number; +} + +const DEFAULT_OPTIONS: Z3Options = { + verifyDepth: 10, +}; + +/** + * InvariantNode — parsed invariant block + * Parsed directly in the generator from raw block content + * since the AST parser doesn't yet parse invariants. + */ +export interface InvariantNode { + type: 'invariant'; + name: string; + statements: InvariantStatement[]; +} + +export type InvariantStatement = + | { type: 'require'; expr: string } + | { type: 'forall'; var: string; domain: string; expr: string } + | { type: 'forall_complex'; vars: string[]; domains: string[]; expr: string }; + +export interface NextNode { + type: 'next'; + actions: string[]; // action names separated by | +} + +/** + * Generate Z3 SMT-LIB2 from a SpeckDL AST. + */ +export function generateZ3(ast: AST, outputDir: string, options: Z3Options = DEFAULT_OPTIONS): void { + for (const speck of ast.specks) { + const code = emitZ3(speck, options); + const filename = `${speck.name}.smt2`; + const filepath = path.join(outputDir, filename); + fs.writeFileSync(filepath, code); + console.log(`Generated Z3 SMT-LIB2: ${filepath}`); + } +} + +function emitZ3(speck: SpeckNode, options: Z3Options): string { + const members = speck.members; + + const stateNode = members.find(m => m.type === 'state') as StateNode | undefined; + const initNode = members.find(m => m.type === 'init') as InitNode | undefined; + const actions = members.filter(m => m.type === 'action') as ActionNode[]; + const constraints = members.filter(m => m.type === 'constraint') as ConstraintNode[]; + const verifies = members.filter(m => m.type === 'verify') as VerifyNode[]; + + // Parse invariants from the raw speck body (handled by parseInvariantsFromSource) + // Also extract invariants from parsed constraint nodes + const invariants = extractInvariantsFromConstraints(constraints); + const sourceInvariants = (speck as any)._invariants || []; + const allInvariants = [...sourceInvariants, ...invariants]; + + // Parse next: transition relation + const nextNode = (speck as any)._next || parseNext(speck); + + const lines: string[] = []; + + // Header + lines.push(`; SpeckDL: ${speck.name} — Z3 Verification`); + lines.push(`; Generated by speckl-compile Z3 backend`); + lines.push(`; Bounded model checking depth: ${options.verifyDepth}`); + lines.push(''); + lines.push('(set-logic ALL)'); + lines.push('(set-option :produce-models true)'); + + // Declare Speckl helper functions + // speckl_len: length of Array-simulated sequences + lines.push('(declare-fun speckl_len_Int ( (Array Int Int) ) Int)'); + + // Step 1: Declare sorts for record types + const recordTypes = collectRecordTypes(stateNode, actions); + for (const [name, fields] of recordTypes) { + emitRecordSort(lines, name, fields); + } + + // Step 2: Declare state variables as constants + lines.push(''); + lines.push('; --- State Variables ---'); + const stateVars = stateNode?.variables ?? []; + const allTypeRefs: string[] = []; + + for (const v of stateVars) { + emitStateVar(lines, v, allTypeRefs); + } + + // Declare _post state variables for invariants that compare pre/post state + // (e.g. "currentTerm' >= currentTerm" → "currentTerm_post >= currentTerm") + const needsPostVars = allInvariants.some((inv: InvariantNode) => + inv.statements.some((s: InvariantStatement) => s.type === 'require' && s.expr.includes("'"))); + if (needsPostVars) { + for (const v of stateVars) { + const sort = z3Sort(v.typeExpr); + lines.push(`(declare-const ${v.name}_post ${sort})`); + } + } + + // Step 3: Emit invariants as define-fun + if (allInvariants.length > 0) { + lines.push(''); + lines.push('; --- Invariants ---'); + for (const inv of allInvariants) { + emitInvariant(lines, inv, stateVars); + } + } + + // Step 4: Emit constraints as assertions (invariant-based constraints) + // Constraint invariants are handled via invariant define-fun above + // plus the BMC unrolling below for verify blocks. + // For non-verify specs, emit assertion of each invariant: + if (verifies.length === 0 && allInvariants.length > 0) { + lines.push(''); + lines.push('; --- Invariant Assertions ---'); + for (const inv of allInvariants) { + lines.push(`(assert (${inv.name}))`); + } + } + + // Step 5: Emit action transition predicates (only for BMC with verify blocks) + if (verifies.length > 0 && actions.length > 0) { + lines.push(''); + lines.push('; --- Action Transition Predicates ---'); + for (const action of actions) { + emitActionTransition(lines, action, stateVars); + } + + // Step 6: Emit transition relation (next: A | B | C ...) + if (nextNode && nextNode.actions.length > 0) { + emitNextTransition(lines, speck.name, nextNode, actions, stateVars); + } + } + + // Step 7: Emit bounded model checking for verify blocks + if (verifies.length > 0 && actions.length > 0) { + lines.push(''); + lines.push('; --- Bounded Model Checking ---'); + for (const v of verifies) { + emitBMC(lines, speck.name, v, allInvariants, actions, nextNode, stateVars, options); + } + } + + // Step 8: Check-sat and get-model (for simple assertions) + if (verifies.length === 0) { + lines.push(''); + lines.push('(check-sat)'); + lines.push('(get-model)'); + } + + return lines.join('\n') + '\n'; +} + +// --- Type / sort helpers --- + +function collectRecordTypes(stateNode: StateNode | undefined, actions: ActionNode[]): Map> { + const records = new Map>(); + + // Collect from state variables + if (stateNode) { + for (const v of stateNode.variables) { + collectRecordTypeRefs(v.typeExpr, records); + } + } + + // Collect from action params + for (const a of actions) { + for (const p of a.params) { + collectRecordTypeRefs(p.type, records); + } + } + + return records; +} + +function collectRecordTypeRefs(t: TypeExpr, records: Map>): void { + if (t.type === 'record' && t.fields) { + // Use the record structure — assign a name based on fields for uniqueness + const key = recordTypeKey(t); + if (!records.has(key)) { + const fields = new Map(); + for (const f of t.fields) { + fields.set(f.name, z3Sort(f.type)); + } + records.set(key, fields); + } + } else if (t.type === 'map') { + if (t.keyType) collectRecordTypeRefs(t.keyType, records); + if (t.valueType) collectRecordTypeRefs(t.valueType, records); + } else if (t.type === 'set' || t.type === 'list') { + if (t.elementType) collectRecordTypeRefs(t.elementType, records); + } else if (t.type === 'ident') { + // For named record types (like Account, Transfer, etc.), we declare them as opaque sorts + const name = t.name || 'Unknown'; + if (!isPrimitive(name) && !isBuiltinType(name)) { + const key = `ident:${name}`; + if (!records.has(key)) { + records.set(key, new Map()); // empty = opaque sort + } + } + } +} + +function recordTypeKey(t: TypeExpr): string { + if (t.type === 'ident' && t.name) { + return `ident:${t.name}`; + } + if (t.fields) { + const fields = t.fields.map(f => `${f.name}:${z3Sort(f.type)}`).sort().join(','); + return `record:${fields}`; + } + return 'record:unknown'; +} + +function isPrimitive(name: string): boolean { + return ['Nat', 'Int', 'Real', 'Bool', 'String', 'Bytes'].includes(name); +} + +function isBuiltinType(name: string): boolean { + return ['List', 'Set', 'Map', 'Array'].some(t => name.startsWith(t)); +} + +function z3Sort(t: TypeExpr): string { + if (!t) return 'Int'; + + if (t.type === 'primitive') { + switch (t.name) { + case 'Nat': return 'Int'; + case 'Int': return 'Int'; + case 'Real': return 'Real'; + case 'Bool': return 'Bool'; + case 'String': return 'String'; + case 'Bytes': return 'String'; // Bytes as String for simplicity + default: return 'Int'; + } + } + + if (t.type === 'list') { + // Lists are modeled as Arrays from Int index to element type + const elemSort = z3Sort(t.elementType!); + return `(Array Int ${elemSort})`; + } + + if (t.type === 'set') { + // Sets are modeled as (Array T Bool) — membership is select + const elemSort = z3Sort(t.elementType!); + return `(Array ${elemSort} Bool)`; + } + + if (t.type === 'map') { + const keySort = z3Sort(t.keyType!); + const valSort = z3Sort(t.valueType!); + return `(Array ${keySort} ${valSort})`; + } + + if (t.type === 'record') { + return recordSortName(t); + } + + if (t.type === 'ident' && t.name) { + // Strip ident: prefix, remove Z3-invalid chars, return clean sort name + let clean = t.name.startsWith('ident:') ? t.name.substring(6) : t.name; + // Remove parenthesized parameters: Option(NodeId) → Option_NodeId + clean = clean.replace(/\(([^)]+)\)/g, '_$1').replace(/[<>,\s|"\/]+/g, '_').replace(/__+/g, '_'); + return clean; // Named sort (declared as opaque) + } + + return 'Int'; +} + +function recordSortName(t: TypeExpr): string { + if (t.type === 'ident' && t.name) return t.name; + if (t.fields) { + const key = t.fields.map(f => `${f.name}_${z3Sort(f.type)}`).join('_'); + return `Record${simpleHash(key)}`; + } + return 'Unknown'; +} + +function simpleHash(s: string): string { + let hash = 0; + for (let i = 0; i < s.length; i++) { + hash = ((hash << 5) - hash) + s.charCodeAt(i); + hash |= 0; + } + return Math.abs(hash).toString(16); +} + +function emitRecordSort(lines: string[], name: string, fields: Map): void { + // Sanitize: strip ident: prefix and remove Z3-invalid characters + let cleanName = name.startsWith('ident:') ? name.substring(6) : name; + cleanName = cleanName.replace(/\(([^)]+)\)/g, '_$1').replace(/[<>,\s|"/]+/g, '_').replace(/__+/g, '_'); + + if (fields.size === 0) { + // Opaque sort (for named types like Account, Transfer) + lines.push(`(declare-sort ${cleanName} 0)`); + return; + } + + // Declare as a datatype + lines.push(`(declare-datatypes (( ${cleanName} 0)) ((`); + lines.push(` (mk-${cleanName}`); + + for (const [field, sort] of fields) { + lines.push(` (${field} ${sort})`); + } + lines.push(' )'); + lines.push(')))'); +} + +// --- State variable emission --- + +function emitStateVar(lines: string[], v: StateVar, _allTypeRefs: string[]): void { + const sort = z3Sort(v.typeExpr); + const comment = v.defaultInit ? ` ; default: ${v.defaultInit}` : ''; + lines.push(`(declare-const ${v.name} ${sort})${comment}`); +} + +// --- Invariant emission --- + +function emitInvariant(lines: string[], inv: InvariantNode, stateVars: StateVar[]): void { + lines.push(`(define-fun ${inv.name} () Bool`); + lines.push(' (and'); + + for (const stmt of inv.statements) { + if (stmt.type === 'require') { + const z3Expr = translateExpr(stmt.expr, stateVars, ''); + lines.push(` ${z3Expr}`); + } else if (stmt.type === 'forall') { + // forall var in domain: expr + const domainSort = inferDomainSort(stmt.domain, stateVars); + const z3Expr = translateExpr(stmt.expr, stateVars, ''); + // Replace the bound variable references + const boundExpr = z3Expr; // var already appears in expr text + lines.push(` (forall ((${stmt.var} ${domainSort})) ${boundExpr})`); + } else if (stmt.type === 'forall_complex') { + // forall var1, var2, ... in domain1, domain2: expr + const bindings = stmt.vars.map((v, i) => { + const sort = inferDomainSort(stmt.domains[i], stateVars); + return `(${v} ${sort})`; + }).join(' '); + const z3Expr = translateExpr(stmt.expr, stateVars, ''); + lines.push(` (forall (${bindings}) ${z3Expr})`); + } + } + + lines.push(' )'); + lines.push(')'); +} + +function inferDomainSort(domain: string, stateVars: StateVar[]): string { + // domain might be "accounts.values" or "transfers.keys" or "pending" + // Match against state variables to infer sort + const baseVar = domain.split('.')[0]; + const sv = stateVars.find(v => v.name === baseVar); + if (sv) { + if (domain.endsWith('.values') && sv.typeExpr.type === 'map') { + return z3Sort(sv.typeExpr.valueType!); + } + if (domain.endsWith('.keys') && sv.typeExpr.type === 'map') { + return z3Sort(sv.typeExpr.keyType!); + } + if (sv.typeExpr.type === 'set') { + return z3Sort(sv.typeExpr.elementType!); + } + } + return 'Int'; // fallback +} + +// --- Expression translation --- + +function translateExpr(expr: string, stateVars: StateVar[], suffix: string): string { + if (!expr) return 'true'; + let e = expr.trim(); + + const stateVarNames = new Set(stateVars.map(v => v.name)); + const reservedWords = new Set([ + 'and', 'or', 'not', 'forall', 'exists', 'implies', 'iff', + 'true', 'false', 'in', 'union', 'distinct', + 'now', 'len', 'join', 'old', 'Not', 'Select', 'Set', 'Map', + 'emptySet', 'emptyMap', 'mapKeys', 'values', 'keys', + 'let', 'if', 'then', 'else', 'return', 'require', 'precondition', 'postcondition' + ]); + + // Protect string literals + const strings: string[] = []; + e = e.replace(/"[^"]*"/g, (m) => { + strings.push(m); + return `__STR_${strings.length - 1}__`; + }); + + // Handle old() wrapper — spec-level, keep inner + e = e.replace(/old\(([^()]*(?:\([^()]*\)[^()]*)*)\)/g, '$1'); + + // Handle SpeckDL forall: forall r in [0..4]: expr => (forall ((r Int)) expr) + // Domain [0..4] is a range — for now, treat as Int + const forallMatch = e.match(/^forall\s+(\w+)\s+in\s+(\[?[\w.\d\-]+\]?):\s*(.+)$/s); + if (forallMatch) { + const varName = forallMatch[1]; + const domainSort = inferDomainSort(forallMatch[2], stateVars); + const bodyExpr = forallMatch[3].trim(); + const bodyZ3 = translateExpr(bodyExpr, stateVars, suffix); + return `(forall ((${varName} ${domainSort})) ${bodyZ3})`; + } + + // Handle prime notation for post-state: currentTerm' → currentTerm_post (invariant comparison) + e = e.replace(/\b(\w+)'/g, '$1_post'); + + // Handle implies(x, y) function call pattern + e = e.replace(/implies\s*\(([^,)]+),\s*([^)]+)\)/g, '(=> )'); + // Handle bare implies keyword at top level + // Handle bare implies as SMT-LIB2 => + // First, handle implies at top level of a line (no paren after second arg) + e = e.replace(/\bimplies\s+([^ ]+)\s+([^)]+)\)?/g, '(=> )'); + // Clean up (=> ) empty cases + e = e.replace(/(=>\s*)/g, '(=> true true)'); + + // Handle -> as implication (after field access cleanup to avoid false matches)// Handle -> as implication (after field access cleanup to avoid false matches) + e = e.replace(/->/g, '=>'); + + // Handle function calls: length(expr), min(a,b), etc. + e = e.replace(/\blength\(([^)]+)\)/g, 'len__$1__xx'); // marker, fix later + e = e.replace(/\bmin\(([^,)]+),\s*([^)]+)\)/g, 'min__$1__$2__xx'); + + // Field access: a.balance → balance (drop qualifier for bound forall vars) + // For record fields accessed via dot, translate to Z3-safe names + e = e.replace(/(\w+)\.([\w_]+)/g, (_, obj, field) => { + // If the object is a state variable (map), .field is an accessor we can't resolve + // For bound forall variables, just use the field name + if (stateVarNames.has(obj)) { + // Accessing field on opaque sort — use a compound name + return `${obj}_${field}`; + } + // For bound variables in forall, just emit the field + return `${obj}_${field}`; + }); + + // Handle notIn BEFORE general patterns + e = e.replace(/(\S+)\s+notIn\s+(\S+)\.keys/g, '(not (select $2 $1))'); + + // Handle x in {val1, val2} BEFORE general 'in' pattern + e = e.replace(/(\S+)\s+in\s+\{([^}]+)\}/g, (_, x, values) => { + const vals = values.split(',').map((v: string) => v.trim()); + if (vals.length <= 1) return `(= ${x} ${vals[0] || 'true'})`; + const clauses = vals.map((v: string) => `(= ${x} ${v})`).join(' '); + return `(or ${clauses})`; + }); + + // Handle set union + e = e.replace(/(\S+)\s+union\s+\{([^}]+)\}/g, '(store $1 $2 true)'); + + // Handle set difference + e = e.replace(/(\S+)\s+\\\\\s+\{([^}]+)\}/g, '(store $1 $2 false)'); + + // Map access: v[key] -> (select v key) + e = e.replace(/(\w+)\[([^\]]+)\]/g, (_, base, key) => { + const suffixed = suffix && stateVarNames.has(base) ? `${base}${suffix}` : base; + return `(select ${suffixed} ${key.trim()})`; + }); + + // Handle 'not expr' guard — wrap: not isOn -> (not isOn) + // Only if expr starts with 'not ' and rest is a simple identifier + if (/^not\s+(\w+)\s*$/.test(e)) { + const rest = e.substring(4).trim(); + const restZ3 = suffix && stateVarNames.has(rest) ? `${rest}${suffix}` : rest; + // Restore strings + return restoreStrings(`(not ${restZ3})`, strings); + } + + // Add suffix to state variable references + if (suffix) { + e = e.replace(/\b([a-zA-Z_]\w*)\b/g, (match, id) => { + if (stateVarNames.has(id) && !reservedWords.has(id) && !id.startsWith('__')) { + return `${id}${suffix}`; + } + return match; + }); + } + + // Handle 'in' for set membership (but not already wrapped) + if (!e.includes('(select ') && !e.includes('(or ')) { + e = e.replace(/(\S+)\s+in\s+(\S+)/g, (match, x, set) => { + if (reservedWords.has(set) || set.startsWith('__')) return match; + return `(select ${set} ${x})`; + }); + } + + // Replace operators with placeholder tokens to avoid regex conflicts + e = e.replace(/===/g, '__EQ__'); + e = e.replace(/==/g, '__EQ__'); + e = e.replace(/!=/g, '__NE__'); + e = e.replace(/>=/g, '__GE__'); + e = e.replace(/<=/g, '__LE__'); + + // Replace && and || with and/or for SMT-LIB2 + e = e.replace(/&&/g, 'and'); + e = e.replace(/\|\|/g, 'or'); + + // Convert infix to prefix + e = convertToPrefix(e); + + // Fix length marker: len__args__xx → Z3-compatible form + // Use declared speckl_len functions since Z3 arrays don't have built-in length + e = e.replace(/len__([^_]\S*?)__xx/g, (_, args) => { + // Determine sort from context — default to Int array length + return `(speckl_len_Int ${args})`; + }); + e = e.replace(/min__([^_]\S*?)__([^_]\S*?)__xx/g, (_, a, b) => { + return `(ite (<= ${a} ${b}) ${a} ${b})`; + }); + + // Fix i-1 → (- i 1) in SMT-LIB2 (select indices) + e = e.replace(/\b(\w+)-(\d+)\b/g, '(- $1 $2)'); + + // Restore operators to SMT-LIB2 + e = e.replace(/__EQ__/g, '='); + e = e.replace(/__NE__/g, 'distinct'); + e = e.replace(/__GE__/g, '>='); + e = e.replace(/__LE__/g, '<='); + + // Wrap distinct + e = e.replace(/\bdistinct\s+(\S+)\s+(\S+)/g, '(distinct $1 $2)'); + + // Fix remaining infix operators that escaped convertToPrefix + // word = word/val: convert to (= word val) + e = e.replace(/(\b\w+)\s*=\s*(\w+|\d+|Bool|true|false)\b/g, '(= $1 $2)'); + // (innermost expr) = val: convert to (= (expr) val) + e = e.replace(/\(([^()]+)\)\s*=\s*(\w+|\d+|Bool|true|false)\b/g, '(= ($1) $2)'); + // word <= word: convert to (<= word word) + e = e.replace(/(\b\w+)\s*(<=|>=)\s*(\w+|\d+|Bool|true|false)\b/g, '($2 $1 $3)'); + // (innermost expr) <= val: convert to (<= (expr) val) + e = e.replace(/\(([^()]+)\)\s*(<=|>=)\s*(\w+|\d+|Bool|true|false)\b/g, '($2 ($1) $3)'); + // Restore strings + e = restoreStrings(e, strings); + + return e; +} + +function restoreStrings(expr: string, strings: string[]): string { + return expr.replace(/__STR_(\d+)__/g, (_, i) => strings[parseInt(i)]); +} + +/** + * Convert infix expression to SMT-LIB2 prefix notation. + * Handles operators: =>, and, or, =, <, >, <=, >=, +, - + */ +function convertToPrefix(expr: string): string { + expr = convertOp(expr, '=>', '=>'); + expr = convertOp(expr, 'and', 'and'); + expr = convertOp(expr, 'or', 'or'); + expr = convertOp(expr, '=', '='); + expr = convertOp(expr, '>', '>'); + expr = convertOp(expr, '<', '<'); + expr = convertOp(expr, '>=', '>='); + expr = convertOp(expr, '<=', '<='); + expr = convertOp(expr, '+', '+'); + expr = convertOp(expr, '-', '-'); + return expr; +} + +function convertOp(expr: string, op: string, smtOp: string): string { + // Find top-level operator occurrence (outside any parens) + let depth = 0; + for (let i = 0; i < expr.length; i++) { + if (expr[i] === '(') depth++; + else if (expr[i] === ')') depth--; + else if (depth === 0 && expr.substring(i, i + op.length) === op) { + const beforeOk = i === 0 || /\s/.test(expr[i - 1]) || expr[i - 1] === '('; + const afterOk = i + op.length >= expr.length || /\s/.test(expr[i + op.length]) || expr[i + op.length] === ')'; + if (beforeOk && afterOk) { + const left = expr.substring(0, i).trim(); + const right = expr.substring(i + op.length).trim(); + return `(${smtOp} ${convertToPrefix(left)} ${convertToPrefix(right)})`; + } + } + } + // Already in prefix form + return expr; +} + +// --- Action transition predicate emission --- + +function emitActionTransition(lines: string[], action: ActionNode, stateVars: StateVar[]): void { + const funName = `${action.name}Enabled`; + const nextName = `${action.name}Step`; + + // Collect preconditions and requires + const guards: string[] = []; + const assigns: { target: string; expr: string }[] = []; + + for (const s of action.statements) { + if (s.type === 'precondition' || s.type === 'require') { + guards.push(s.expr); + } else if (s.type === 'assign') { + assigns.push({ target: s.target, expr: s.expr }); + } + } + + // Enablement predicate: all guards hold + if (guards.length > 0) { + lines.push(`(define-fun ${funName} () Bool`); + lines.push(' (and'); + for (const g of guards) { + const z3Expr = translateExpr(g, stateVars, ''); + lines.push(` ${z3Expr}`); + } + lines.push(' )'); + lines.push(')'); + } + + // Step predicate: relates pre- and post-state + if (assigns.length > 0) { + lines.push(`(define-fun ${nextName} () Bool`); + lines.push(' (and'); + for (const a of assigns) { + // a.target becomes post-state, RHS reads pre-state + // Target: if it's a simple name, post-state; if indexed, translate + const z3RHS = translateExpr(a.expr, stateVars, ''); + if (a.target.includes('[')) { + const z3Target = translateExpr(a.target, stateVars, "_post"); + lines.push(` (= ${z3Target} ${z3RHS})`); + } else { + lines.push(` (= ${a.target}_post ${z3RHS})`); + } + } + lines.push(' )'); + lines.push(')'); + } +} + +// --- Next transition relation --- + +function emitNextTransition( + lines: string[], + speckName: string, + nextNode: NextNode, + actions: ActionNode[], + stateVars: StateVar[] +): void { + const actionNames = new Set(actions.map(a => a.name)); + const validActions = nextNode.actions.filter(a => actionNames.has(a)); + + if (validActions.length === 0) return; + + lines.push(`; Next transition relation: ${validActions.join(' | ')}`); + lines.push(`(define-fun Next () Bool`); + lines.push(' (or'); + + for (const aName of validActions) { + const action = actions.find(a => a.name === aName); + if (action) { + const enabled = `${aName}Enabled`; + const step = `${aName}Step`; + + // Check if enabled and step predicates exist + const hasEnabled = action.statements.some(s => s.type === 'precondition' || s.type === 'require'); + const hasStep = action.statements.some(s => s.type === 'assign'); + + if (hasEnabled && hasStep) { + lines.push(` (and (${enabled}) (${step}))`); + } else if (hasEnabled) { + lines.push(` (and (${enabled})`); + lines.push(' (and'); + for (const v of stateVars) { + lines.push(` (= ${v.name}_post ${v.name})`); + } + lines.push(' )'); + lines.push(' )'); + } else if (hasStep) { + lines.push(` (${step})`); + } + } + } + + // Stuttering step: nothing changes + lines.push(' ; Stuttering step'); + lines.push(' (and'); + for (const v of stateVars) { + lines.push(` (= ${v.name}_post ${v.name})`); + } + lines.push(' )'); + + lines.push(' )'); + lines.push(')'); +} + +// --- Bounded Model Checking --- + +function emitBMC( + lines: string[], + speckName: string, + verify: VerifyNode, + invariants: InvariantNode[], + actions: ActionNode[], + nextNode: NextNode | null, + stateVars: StateVar[], + options: Z3Options +): void { + const depth = verify.depth || options.verifyDepth; + const temporalExpr = verify.temporalExpr; + const invariantName = verify.name; + + lines.push(`; Verification: ${temporalExpr}`); + lines.push(`; BMC depth: ${depth}`); + + const isAlways = temporalExpr.startsWith('Always(') || temporalExpr.startsWith('always('); + const isEventually = temporalExpr.startsWith('Eventually(') || temporalExpr.startsWith('eventually('); + + // Declare state variables at each step + lines.push(''); + lines.push(`; Declare state copies for ${depth} steps`); + for (const v of stateVars) { + for (let i = 0; i <= depth; i++) { + const sort = z3Sort(v.typeExpr); + lines.push(`(declare-const ${v.name}_${i} ${sort})`); + } + } + + // Init state constraints (from spec init block) + lines.push(''); + lines.push('; Initial state constraints'); + lines.push('(assert'); + lines.push(' (and'); + + // Apply init assignments + const stateNode = stateVars.length > 0 ? { type: 'state' as const, variables: stateVars } : null; + // We need the init block + const initMembers = speckName.length > 0 ? undefined : undefined; + + // Default init: use defaultInit from state vars + for (const v of stateVars) { + if (v.defaultInit) { + const z3Val = translateInitValue(v.defaultInit, v.typeExpr); + lines.push(` (= ${v.name}_0 ${z3Val})`); + } else { + // Default: 0 for Nat/Int, false for Bool, "" for String + if (v.typeExpr.type === 'primitive') { + switch (v.typeExpr.name) { + case 'Nat': case 'Int': lines.push(` (>= ${v.name}_0 0)`); break; + case 'Bool': lines.push(` (or (= ${v.name}_0 true) (= ${v.name}_0 false))`); break; + default: break; // unconstrained + } + } + } + } + + lines.push(' )'); + lines.push(')'); + + // Transition constraints for each step + lines.push(''); + lines.push('; Transition constraints'); + for (let i = 0; i < depth; i++) { + const actionNames = nextNode?.actions ?? []; + const validActions = actionNames.filter(a => actions.some(act => act.name === a)); + + lines.push(`(assert ; Step ${i} → ${i + 1}`); + lines.push(' (or'); + + for (const aName of validActions) { + const action = actions.find(a => a.name === aName); + if (!action) continue; + + const guards: string[] = []; + const stepAssigns: string[] = []; + + for (const s of action.statements) { + if (s.type === 'precondition' || s.type === 'require') { + guards.push(s.expr); + } else if (s.type === 'assign') { + if (s.target.includes('[')) { + const z3LHS = translateExpr(s.target, stateVars, `_${i + 1}`); + const z3RHS = translateExpr(s.expr, stateVars, `_${i}`); + stepAssigns.push(`(= ${z3LHS} ${z3RHS})`); + } else { + const z3RHS = translateExpr(s.expr, stateVars, `_${i}`); + stepAssigns.push(`(= ${s.target}_${i + 1} ${z3RHS})`); + } + } + } + + if (guards.length > 0 || stepAssigns.length > 0) { + lines.push(` (and ; Action: ${aName}`); + + // Guards at step i + for (const g of guards) { + const z3Expr = translateExpr(g, stateVars, `_${i}`); + lines.push(` ${z3Expr}`); + } + + // Step assignments + for (const a of stepAssigns) { + lines.push(` ${a}`); + } + + // Frame: unchanged vars + const changedVars = new Set(action.statements + .filter(s => s.type === 'assign') + .map(s => s.target.replace(/\[.*\]/, ''))); + for (const v of stateVars) { + if (!changedVars.has(v.name)) { + lines.push(` (= ${v.name}_${i + 1} ${v.name}_${i})`); + } + } + + lines.push(' )'); + } + } + + // Stuttering + lines.push(' ; Stuttering'); + lines.push(' (and'); + for (const v of stateVars) { + lines.push(` (= ${v.name}_${i + 1} ${v.name}_${i})`); + } + lines.push(' )'); + + lines.push(' )'); + lines.push(')'); + } + + // Property to check: find invariant and assert its negation at some step + lines.push(''); + lines.push('; Property assertion: not(invariant at some step)'); + + if (isAlways && invariantName) { + const inv = invariants.find(i => i.name === invariantName); + if (inv) { + // Always p = for all steps up to depth, p holds + // To check: assert there exists step where p fails + // If UNSAT: p always holds within bound + // If SAT: counterexample found + + lines.push('(assert'); + lines.push(' (or'); + for (let i = 0; i <= depth; i++) { + lines.push(` (not (${inv.name}_${i}))`); + } + lines.push(' )'); + lines.push(')'); + + // Emit invariant instances at each step + for (let i = 0; i <= depth; i++) { + emitInvariantInstance(lines, inv, stateVars, i); + } + + lines.push(''); + lines.push(`(echo "Checking: Always(${invariantName}) for ${depth} steps")`); + lines.push('(check-sat)'); + lines.push('(get-model)'); + + } else { + lines.push(`; Warning: invariant "${invariantName}" not found in spec`); + lines.push('(check-sat)'); + lines.push('(get-model)'); + } + } else if (isEventually && invariantName) { + // Eventually p = exists step up to depth where p holds + // This is approximated for BMC + lines.push('(assert'); + lines.push(' (and'); + for (let i = 0; i <= depth; i++) { + lines.push(` (not (${invariantName}_${i}))`); + } + lines.push(' )'); + lines.push(')'); + + for (let i = 0; i <= depth; i++) { + const inv = invariants.find(inv => inv.name === invariantName); + if (inv) { + emitInvariantInstance(lines, inv, stateVars, i); + } + } + + lines.push(`(echo "Checking: Eventually(${invariantName})")`); + lines.push('(check-sat)'); + lines.push('(get-model)'); + } else { + lines.push(`; Verification property: ${temporalExpr}`); + lines.push('(check-sat)'); + lines.push('(get-model)'); + } +} + +function extractInvariantName(temporalExpr: string): string | null { + const match = temporalExpr.match(/(?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\)/); + return match ? match[1] : null; +} + +function emitInvariantInstance(lines: string[], inv: InvariantNode, stateVars: StateVar[], step: number): void { + lines.push(`(define-fun ${inv.name}_${step} () Bool`); + lines.push(' (and'); + + for (const stmt of inv.statements) { + if (stmt.type === 'require') { + const z3Expr = translateExpr(stmt.expr, stateVars, `_${step}`); + lines.push(` ${z3Expr}`); + } else if (stmt.type === 'forall') { + const domainSort = inferDomainSort(stmt.domain, stateVars); + const z3Expr = translateExpr(stmt.expr, stateVars, `_${step}`); + lines.push(` (forall ((${stmt.var} ${domainSort})) ${z3Expr})`); + } else if (stmt.type === 'forall_complex') { + const bindings = stmt.vars.map((v, i) => { + const sort = inferDomainSort(stmt.domains[i], stateVars); + return `(${v} ${sort})`; + }).join(' '); + const z3Expr = translateExpr(stmt.expr, stateVars, `_${step}`); + lines.push(` (forall (${bindings}) ${z3Expr})`); + } + } + + lines.push(' )'); + lines.push(')'); +} + +function translateInitValue(val: string, typeExpr: TypeExpr): string { + if (val === 'false') return 'false'; + if (val === 'true') return 'true'; + if (val === 'emptySet') { + const elemSort = z3Sort(typeExpr.elementType!); + return `((as const (Array ${elemSort} Bool)) false)`; + } + if (val === 'emptyMap') { + const keySort = z3Sort(typeExpr.keyType!); + const valSort = z3Sort(typeExpr.valueType!); + // Use const array for maps too — all values default to some baseline + return `((as const (Array ${keySort} ${valSort})) 0)`; + } + if (val === 'emptyList') { + const elemSort = z3Sort(typeExpr.elementType!); + return `((as const (Array Int ${elemSort})) 0)`; + } + // Numeric literal + if (/^\d+$/.test(val)) return val; + // Otherwise, treat as identifier + return val; +} + +// --- Invariant extraction from constraint nodes --- + +/** + * Convert parsed ConstraintNode entries into InvariantNode format + * so they can be emitted as Z3 define-fun assertions. + * + * Constraint nodes from the parser: + * { type: 'constraint', name: 'ReadConsistency', expr: 'forall r in [0..4]: ...' } + * + * We convert them to: + * { type: 'invariant', name: 'ReadConsistency', statements: [ + * { type: 'require', expr: 'forall r in ...' } | { type: 'forall', ... } + * ]} + */ +function extractInvariantsFromConstraints(constraints: ConstraintNode[]): InvariantNode[] { + const invariants: InvariantNode[] = []; + + for (const c of constraints) { + const name = c.name || `constraint_${invariants.length}`; + const expr = c.expr.trim(); + + if (!expr) continue; + + // Try to parse forall statements in the expression + const statements = parseInvariantBody(expr); + invariants.push({ type: 'invariant', name, statements }); + } + + return invariants; +} + +function parseInvariants(speck: SpeckNode): InvariantNode[] { + // Legacy stub — returns empty, new code uses extractInvariantsFromConstraints + return []; +} + +/** + * Parse invariants from the raw .speck source content. + * This function is called with the raw file content and the speck name + * to find and parse invariant blocks that the main parser doesn't handle yet. + */ +export function parseInvariantsFromSource(source: string, speckName: string): InvariantNode[] { + const invariants: InvariantNode[] = []; + + // Find the speck block for this name + const speckRegex = new RegExp(`speck\\s+${escapeRegex(speckName)}\\s*\\{`); + const speckMatch = source.match(speckRegex); + if (!speckMatch) return invariants; + + // Extract the speck body + const startIdx = speckMatch.index! + speckMatch[0].length; + let braceCount = 1; + let endIdx = startIdx; + for (let i = startIdx; i < source.length; i++) { + if (source[i] === '{') braceCount++; + if (source[i] === '}') { + braceCount--; + if (braceCount === 0) { + endIdx = i; + break; + } + } + } + + const body = source.substring(startIdx, endIdx); + + // Find invariant blocks using brace counting (handles nested { }) + const invPattern = /invariant\s+(\w+)\s*\{/g; + let match; + while ((match = invPattern.exec(body)) !== null) { + const name = match[1]; + // Find matching close brace + const blockStart = match.index + match[0].length; + let braceDepth = 1; + let blockEnd = blockStart; + for (let i = blockStart; i < body.length; i++) { + if (body[i] === '{') braceDepth++; + if (body[i] === '}') { + braceDepth--; + if (braceDepth === 0) { + blockEnd = i; + break; + } + } + } + const content = body.substring(blockStart, blockEnd); + + const statements = parseInvariantBody(content); + invariants.push({ type: 'invariant', name, statements }); + } + + return invariants; +} + +function parseInvariantBody(content: string): InvariantStatement[] { + const statements: InvariantStatement[] = []; + const lines = content.split('\n'); + + let i = 0; + while (i < lines.length) { + const rawLine = lines[i]; + const line = rawLine.trim(); + if (!line || line.startsWith('//')) { i++; continue; } + + // forall var in domain: (body on next line(s)) + // domain can be: [0..4], someVar.keys, someVar.values, etc. + const forallHeaderMatch = line.match(/^forall\s+(\w+)\s+in\s+(\[?[\w.\-\d]+\]?):\s*$/); + if (forallHeaderMatch) { + const bodyLines: string[] = []; + i++; + while (i < lines.length) { + const nextLine = lines[i]; + if (!nextLine.trim() || nextLine.trim().startsWith('//')) { i++; continue; } + if (/^forall\s+/.test(nextLine.trim()) || !/^\s+/.test(nextLine)) break; + bodyLines.push(nextLine.trim()); + i++; + } + if (bodyLines.length > 0) { + statements.push({ + type: 'forall', + var: forallHeaderMatch[1], + domain: forallHeaderMatch[2], + expr: bodyLines.join(' '), + }); + } + continue; + } + + // forall var in domain: expr (single line) + // domain can be: [0..4], accounts.values, log.indices, transfers.keys, etc. + const forallMatch = line.match(/^forall\s+(\w+)\s+in\s+(\[?[\w.\d\-]+\]?):\s*(.+)$/); + if (forallMatch) { + statements.push({ + type: 'forall', + var: forallMatch[1], + domain: forallMatch[2], + expr: forallMatch[3].trim(), + }); + i++; continue; + } + + // forall var1, var2 in domain1, domain2: expr (complex) + // domains may include bracket notation and dots + const complexForallMatch = line.match(/^forall\s+([\w,\s]+)\s+in\s+([\[\]\w\s.,\-]+):\s*(.+)$/); + if (complexForallMatch) { + const vars = complexForallMatch[1].split(',').map((s: string) => s.trim()).filter(Boolean); + const domains = complexForallMatch[2].split(',').map((s: string) => s.trim()).filter(Boolean); + if (vars.length > 1 || domains.length > 1) { + statements.push({ + type: 'forall_complex', + vars, domains, + expr: complexForallMatch[3].trim(), + }); + i++; continue; + } + statements.push({ + type: 'forall', + var: vars[0], domain: domains[0], + expr: complexForallMatch[3].trim(), + }); + i++; continue; + } + + statements.push({ type: 'require', expr: line }); + i++; + } + + return statements; +} + +/** + * Parse next: transition relation from raw source + */ +export function parseNextFromSource(source: string, speckName: string): NextNode | null { + const speckRegex = new RegExp(`speck\\s+${escapeRegex(speckName)}\\s*\\{`); + const speckMatch = source.match(speckRegex); + if (!speckMatch) return null; + + const startIdx = speckMatch.index! + speckMatch[0].length; + let braceCount = 1; + let endIdx = startIdx; + for (let i = startIdx; i < source.length; i++) { + if (source[i] === '{') braceCount++; + if (source[i] === '}') { + braceCount--; + if (braceCount === 0) { + endIdx = i; + break; + } + } + } + + const body = source.substring(startIdx, endIdx); + + // next: Action1 | Action2 | ... (can span multiple lines) + const nextMatch = body.match(/next:\s*(.+)/s); + if (!nextMatch) return null; + + // The captured group contains everything from 'next' to end of speck body + // We need to extract just the action names before the next block starts + const rest = nextMatch[1]; + // Stop at verify: block, or end of speck body + const endMarker = rest.match(/\n\s*(?:verify:|speck\s)/); + const nextContent = endMarker ? rest.substring(0, endMarker.index!) : rest; + + const actions = nextContent + .replace(/\n/g, ' ') // collapse newlines + .split('|') + .map(s => s.trim()) + .filter(Boolean); + + return { type: 'next', actions }; +} + +function parseNext(speck: SpeckNode): NextNode | null { + // Parsed from source, not AST + return null; +} + +function escapeRegex(s: string): string { + return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +} diff --git a/compiler/src/index.ts b/compiler/src/index.ts index 87244a8..ce5bf73 100644 --- a/compiler/src/index.ts +++ b/compiler/src/index.ts @@ -7,13 +7,16 @@ import { generateProvenanceGraph } from './generators/provenance.js'; import { generateCycloneDX } from './generators/cyclonedx.js'; import { generateSPDX } from './generators/spdx.js'; import { generateTypeScriptStateMachine } from './generators/typescript-state-machine.js'; -import { generateWASM } from './generators/wasm.js'; +import { generateZ3, Z3Options, parseInvariantsFromSource, parseNextFromSource } from './generators/z3.js'; +import { generateRust } from './generators/rust.js'; import fs from 'fs'; import path from 'path'; interface CompileOptions { outputDir: string; bomFormat: 'cdx' | 'spdx' | 'both'; + target: 'typescript' | 'z3' | 'all'; + verifyDepth: number; } async function main() { @@ -43,6 +46,19 @@ async function main() { type: 'string', default: './out', }) + .option('target', { + alias: 't', + describe: 'Compilation target', + type: 'string', + choices: ['typescript', 'z3', 'rust', 'all'], + default: 'typescript' as const, + }) + .option('verify-depth', { + alias: 'd', + describe: 'Bounded model checking depth for Z3 verification', + type: 'number', + default: 10, + }) .option('bom-format', { alias: 'b', describe: 'BOM output format', @@ -58,6 +74,8 @@ async function main() { const options = { outputDir: argv.outputDir as string, bomFormat: argv.bomFormat as 'cdx' | 'spdx' | 'both', + target: argv.target as 'typescript' | 'z3' | 'rust' | 'all', + verifyDepth: argv.verifyDepth as number, }; console.log(`Compiling ${file}...`); @@ -74,27 +92,51 @@ async function main() { fs.mkdirSync(options.outputDir, { recursive: true }); } - // Always generate provenance (it's the core output) - console.log('\nGenerating provenance graph (PROV-O JSON-LD)...'); - generateProvenanceGraph(ast, options.outputDir); - - // Generate TypeScript state machine - console.log('\nGenerating TypeScript state machine...'); - generateTypeScriptStateMachine(ast, options.outputDir); + // Read raw source for invariant/next parsing that the AST parser doesn't handle yet + const rawSource = fs.readFileSync(file, 'utf-8'); - // Generate WASM state machine - console.log('\nGenerating WebAssembly (WAT)...'); - generateWASM(ast, options.outputDir); - - // Generate BOM based on format selection - if (options.bomFormat === 'cdx' || options.bomFormat === 'both') { - console.log('\nGenerating CycloneDX BOM...'); - generateCycloneDX(ast, options.outputDir); + // Generate Rust state machine if target is rust or all + if (options.target === 'rust' || options.target === 'all') { + console.log('\nGenerating Rust state machine...'); + generateRust(ast, options.outputDir); } - - if (options.bomFormat === 'spdx' || options.bomFormat === 'both') { - console.log('\nGenerating SPDX BOM...'); - generateSPDX(ast, options.outputDir); + + // Always generate provenance (it's the core output) if not z3-only or rust-only + if (options.target !== 'z3' && options.target !== 'rust') { + console.log('\nGenerating provenance graph (PROV-O JSON-LD)...'); + generateProvenanceGraph(ast, options.outputDir); + } + + // Generate Z3 SMT-LIB2 if target is z3 or all + if (options.target === 'z3' || options.target === 'all') { + console.log('\nGenerating Z3 SMT-LIB2...'); + // Parse invariants from raw source for Z3 backend + for (const speck of ast.specks) { + const invariants = parseInvariantsFromSource(rawSource, speck.name); + const nextNode = parseNextFromSource(rawSource, speck.name); + // Attach parsed data to the AST for the generator + (speck as any)._invariants = invariants; + (speck as any)._next = nextNode; + } + const z3Options: Z3Options = { verifyDepth: options.verifyDepth }; + generateZ3(ast, options.outputDir, z3Options); + } + + // Generate TypeScript state machine if not z3-only or rust-only + if (options.target !== 'z3' && options.target !== 'rust') { + console.log('\nGenerating TypeScript state machine...'); + generateTypeScriptStateMachine(ast, options.outputDir); + + // Generate BOM based on format selection + if (options.bomFormat === 'cdx' || options.bomFormat === 'both') { + console.log('\nGenerating CycloneDX BOM...'); + generateCycloneDX(ast, options.outputDir); + } + + if (options.bomFormat === 'spdx' || options.bomFormat === 'both') { + console.log('\nGenerating SPDX BOM...'); + generateSPDX(ast, options.outputDir); + } } console.log(`\nCompilation complete. Output: ${options.outputDir}/`); diff --git a/compiler/src/parser.ts b/compiler/src/parser.ts index 39e5f0e..c25e995 100644 --- a/compiler/src/parser.ts +++ b/compiler/src/parser.ts @@ -28,12 +28,15 @@ export interface OutputNode { export interface ConstraintNode { type: 'constraint'; + name?: string; expr: string; } export interface VerifyNode { type: 'verify'; + name: string; temporalExpr: string; + depth?: number; } export interface InterfaceNode { @@ -132,8 +135,11 @@ export type ActionStatement = | { type: 'assign'; target: string; expr: string } | { type: 'let'; name: string; expr: string } | { type: 'require'; expr: string } + | { type: 'precondition'; expr: string } + | { type: 'postcondition'; expr: string } | { type: 'emit'; event: string; fields: { name: string; value: string }[] } - | { type: 'return'; expr: string }; + | { type: 'return'; expr: string } + | { type: 'ifblock'; raw: string }; export type MemberNode = | ImportNode @@ -243,8 +249,9 @@ function parseSpeck(lines: string[], startIndex: number): SpeckNode | null { const openCount = (rawLine.match(/{/g) || []).length; const closeCount = (rawLine.match(/}/g) || []).length; - // Check for block-opening members - const blockStarters = ['state:', 'init:', 'action ', 'event ', 'provenance ', 'bom ', 'interface ']; + // Check for block-opening members (those that open braces on the same line) + const blockStarters = ['state:', 'init:', 'action ', 'event ', 'provenance ', 'bom ', 'interface ', + 'state {', 'init {', 'verify ', 'constraint ']; const isBlockStarter = blockStarters.some(s => line.startsWith(s)); if (isBlockStarter && openCount > 0) { @@ -253,9 +260,39 @@ function parseSpeck(lines: string[], startIndex: number): SpeckNode | null { if (member) { members.push(member); } - // Skip past this block - const endIdx = findBlockEnd(lines, i + 1, braceCount + openCount); - i = endIdx + 1; + // Skip past this block if it spans multiple lines + if (closeCount < openCount) { + const endIdx = findBlockEnd(lines, i + 1, braceCount + openCount); + i = endIdx + 1; + } else { + i++; + } + continue; + } + + // Handle constraint/verify blocks that may span multiple lines without braces + // (e.g. constraint Name: forall r in [0..4]:) + if (line.startsWith('constraint ') || line.startsWith('verify ')) { + const member = parseMultiLineConstraintVerify(lines, i); + if (member) { + members.push(member); + // Skip past the consumed continuation lines + while (i + 1 < lines.length) { + const nextLine = lines[i + 1].trim(); + if (!nextLine || nextLine.startsWith('//') || nextLine.startsWith('/*') || + nextLine.startsWith('constraint ') || nextLine.startsWith('verify ') || + nextLine.startsWith('constraint:') || nextLine.startsWith('verify:') || + nextLine.startsWith('state') || nextLine.startsWith('init') || + nextLine.startsWith('action ') || nextLine.startsWith('event ') || + nextLine.startsWith('interface ') || nextLine.startsWith('provenance ') || + nextLine.startsWith('bom ') || nextLine.startsWith('}') || + nextLine.startsWith('next:') || nextLine.startsWith('invariant ')) { + break; + } + i++; + } + } + i++; continue; } @@ -295,6 +332,20 @@ function findBlockEnd(lines: string[], startIndex: number, startBraceCount: numb return i; } +// Check if an expression has balanced braces, parens, and brackets +function isExpressionComplete(expr: string): boolean { + let braces = 0, parens = 0, brackets = 0; + for (const char of expr) { + if (char === '{') braces++; + else if (char === '}') braces--; + else if (char === '(') parens++; + else if (char === ')') parens--; + else if (char === '[') brackets++; + else if (char === ']') brackets--; + } + return braces === 0 && parens === 0 && brackets === 0; +} + function parseMemberBlock(lines: string[], startIndex: number): MemberNode | null { const firstLine = lines[startIndex].trim(); const startBraceCount = (firstLine.match(/{/g) || []).length; @@ -302,9 +353,15 @@ function parseMemberBlock(lines: string[], startIndex: number): MemberNode | nul if (firstLine.startsWith('state:')) { return parseStateBlockMultiline(lines, startIndex, startBraceCount); } + if (firstLine.startsWith('state {') || firstLine === 'state') { + return parseStateBlockMultiline(lines, startIndex, startBraceCount); + } if (firstLine.startsWith('init:')) { return parseInitBlockMultiline(lines, startIndex, startBraceCount); } + if (firstLine.startsWith('init {') || firstLine === 'init') { + return parseInitBlockMultiline(lines, startIndex, startBraceCount); + } if (firstLine.startsWith('action ')) { return parseActionBlockMultiline(lines, startIndex, startBraceCount); } @@ -320,6 +377,12 @@ function parseMemberBlock(lines: string[], startIndex: number): MemberNode | nul if (firstLine.startsWith('bom ')) { return { type: 'bom' }; } + if (firstLine.startsWith('verify ')) { + return parseVerifyBlockMultiline(lines, startIndex, startBraceCount); + } + if (firstLine.startsWith('constraint ')) { + return parseConstraintBlockMultiline(lines, startIndex, startBraceCount); + } return null; } @@ -337,14 +400,28 @@ function parseMember(line: string): MemberNode | null { return { type: 'output', typeExpr }; } - // constraint: + // constraint Name: expr (single-line named constraint) + if (line.startsWith('constraint ')) { + const match = line.match(/^constraint\s+(\w+):\s*(.+)$/); + if (match) { + return { type: 'constraint', name: match[1], expr: match[2].trim() }; + } + return null; // multi-line constraint block (handled by parseMemberBlock) + } + + // constraint: expr (unnamed, single-line) if (line.startsWith('constraint:')) { return { type: 'constraint', expr: line.substring(11).trim() }; } - // verify: + // verify Always(Name) { depth N } or verify Always(Name) (single-line forms) + if (line.startsWith('verify ')) { + return parseVerifySingleLine(line); + } + + // verify: expr (old unnamed single-line) if (line.startsWith('verify:')) { - return { type: 'verify', temporalExpr: line.substring(7).trim() }; + return { type: 'verify', name: 'unnamed', temporalExpr: line.substring(7).trim() }; } // interface: @@ -388,7 +465,7 @@ function parseMember(line: string): MemberNode | null { return parseSource(line); } - // state: + // state: or state { if (line.startsWith('state:')) { const varsStr = line.substring(6).trim(); // state: { ... } - single-line form @@ -399,8 +476,19 @@ function parseMember(line: string): MemberNode | null { // state: { - multi-line opening (handled by parseStateMultiline) return { type: 'state', variables: [] }; } + if (line.startsWith('state {') || line === 'state') { + // state { ... } inline or state { on its own line (handled by parseMemberBlock) + if (line.startsWith('state {')) { + const inner = line.substring(6).trim(); + const content = inner.startsWith('{') ? inner.slice(1).trim() : inner; + if (content.endsWith('}')) { + return parseStateBlock(content.slice(0, -1).trim()); + } + } + return { type: 'state', variables: [] }; + } - // init: + // init: or init { if (line.startsWith('init:')) { const initStr = line.substring(5).trim(); if (initStr.startsWith('{') && initStr.endsWith('}')) { @@ -409,6 +497,17 @@ function parseMember(line: string): MemberNode | null { } return { type: 'init', assignments: [] }; } + if (line.startsWith('init {') || line === 'init') { + // init { ... } inline + if (line.startsWith('init {')) { + const inner = line.substring(5).trim(); + const content = inner.startsWith('{') ? inner.slice(1).trim() : inner; + if (content.endsWith('}')) { + return parseInitBlock(content.slice(0, -1).trim()); + } + } + return { type: 'init', assignments: [] }; + } // action keyword if (line.startsWith('action ')) { @@ -435,18 +534,26 @@ function parseStateBlock(inner: string): StateNode { for (const vStr of varStrs) { const withDefault = vStr.match(/^(\w+)\s*:\s*(.+?)\s*:=\s*(.+)$/); if (withDefault) { + // Strip trailing comments from type expression + let typeStr = withDefault[2].trim(); + const commentIdx = typeStr.indexOf('//'); + if (commentIdx >= 0) typeStr = typeStr.substring(0, commentIdx).trim(); variables.push({ name: withDefault[1], - typeExpr: parseTypeExpr(withDefault[2].trim()), + typeExpr: parseTypeExpr(typeStr), defaultInit: withDefault[3].trim(), }); continue; } const withoutDefault = vStr.match(/^(\w+)\s*:\s*(.+)$/); if (withoutDefault) { + // Strip trailing comments from type expression + let typeStr = withoutDefault[2].trim(); + const commentIdx = typeStr.indexOf('//'); + if (commentIdx >= 0) typeStr = typeStr.substring(0, commentIdx).trim(); variables.push({ name: withoutDefault[1], - typeExpr: parseTypeExpr(withoutDefault[2].trim()), + typeExpr: parseTypeExpr(typeStr), }); } } @@ -532,13 +639,53 @@ function parseTypeExpr(expr: string): TypeExpr { // Map if (expr.startsWith('Map<') && expr.endsWith('>')) { const inner = expr.substring(4, expr.length - 1); - // Find the first-level comma (not inside nested generics) + // Find the first-level comma (not inside nested generics or record literals) let depth = 0; + let braceDepth = 0; let commaIdx = -1; for (let i = 0; i < inner.length; i++) { - if (inner[i] === '<') depth++; - else if (inner[i] === '>') depth--; - else if (inner[i] === ',' && depth === 0) { + if (inner[i] === '<' || inner[i] === '(') depth++; + else if (inner[i] === '>' || inner[i] === ')') depth--; + else if (inner[i] === '{') braceDepth++; + else if (inner[i] === '}') braceDepth--; + else if (inner[i] === ',' && depth === 0 && braceDepth === 0) { + commaIdx = i; + break; + } + } + if (commaIdx !== -1) { + const keyStr = inner.substring(0, commaIdx).trim(); + const valStr = inner.substring(commaIdx + 1).trim(); + return { type: 'map', keyType: parseTypeExpr(keyStr), valueType: parseTypeExpr(valStr) }; + } + // Fallback: treat as ident if malformed + return { type: 'ident', name: expr }; + } + + // List(T) — parenthetical syntax (SpeckDL canonical form) + if (expr.startsWith('List(') && expr.endsWith(')')) { + const inner = expr.substring(5, expr.length - 1).trim(); + return { type: 'list', elementType: parseTypeExpr(inner) }; + } + + // Set(T) — parenthetical syntax (SpeckDL canonical form) + if (expr.startsWith('Set(') && expr.endsWith(')')) { + const inner = expr.substring(4, expr.length - 1).trim(); + return { type: 'set', elementType: parseTypeExpr(inner) }; + } + + // Map(K, V) — parenthetical syntax (SpeckDL canonical form) + if (expr.startsWith('Map(') && expr.endsWith(')')) { + const inner = expr.substring(4, expr.length - 1); + let depth = 0; + let braceDepth = 0; + let commaIdx = -1; + for (let i = 0; i < inner.length; i++) { + if (inner[i] === '(' || inner[i] === '<') depth++; + else if (inner[i] === ')' || inner[i] === '>') depth--; + else if (inner[i] === '{') braceDepth++; + else if (inner[i] === '}') braceDepth--; + else if (inner[i] === ',' && depth === 0 && braceDepth === 0) { commaIdx = i; break; } @@ -708,14 +855,41 @@ function parseSource(line: string): SourceNode { function parseStateBlockMultiline(lines: string[], startIndex: number, startBraceCount: number): StateNode { const endIndex = findBlockEnd(lines, startIndex + 1, startBraceCount); const innerLines = lines.slice(startIndex + 1, endIndex).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); - const inner = innerLines.join('; '); + // State blocks separate fields with commas or newlines/semicolons. + // Strip trailing commas (they are field separators, not continuations) + // then join with semicolons between complete variable declarations. + const cleanedLines = innerLines.map(l => l.replace(/,+\s*$/, '')); + const inner = cleanedLines.reduce((acc, line, i) => { + if (i === 0) return line; + const prevLine = cleanedLines[i - 1]; + // Check if this line starts a new variable declaration (name: type) + const isNewVar = /^\w+\s*:/.test(line); + // Check if prev line ends with a continuation character (brace or paren, NOT comma) + const prevIsContinuation = /[{(]\s*$/.test(prevLine); + if (isNewVar && !prevIsContinuation) { + return acc + '; ' + line; + } + return acc + ' ' + line; + }, ''); return parseStateBlock(inner); } function parseInitBlockMultiline(lines: string[], startIndex: number, startBraceCount: number): InitNode { const endIndex = findBlockEnd(lines, startIndex + 1, startBraceCount); const innerLines = lines.slice(startIndex + 1, endIndex).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); - const inner = innerLines.join('; '); + // Init blocks separate fields with commas or newlines/semicolons. + // Strip trailing commas and join with semicolons. + const cleanedLines = innerLines.map(l => l.replace(/,+\s*$/, '')); + const inner = cleanedLines.reduce((acc, line, i) => { + if (i === 0) return line; + const prevLine = cleanedLines[i - 1]; + const isNewVar = /^\w+\s*:/.test(line); + const prevIsContinuation = /[{(]\s*$/.test(prevLine); + if (isNewVar && !prevIsContinuation) { + return acc + '; ' + line; + } + return acc + ' ' + line; + }, ''); return parseInitBlock(inner); } @@ -725,13 +899,74 @@ function parseActionBlockMultiline(lines: string[], startIndex: number, startBra if (!header) return null; const endIndex = findBlockEnd(lines, startIndex + 1, startBraceCount); - const bodyLines = lines.slice(startIndex + 1, endIndex).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); + // Keep blank lines as null to separate statements, filter only comments + const bodyLinesRaw = lines.slice(startIndex + 1, endIndex).map(l => { + const trimmed = l.trim(); + if (!trimmed || trimmed.startsWith('//') || trimmed.startsWith('/*')) return null; + return trimmed; + }); + // Collapse consecutive nulls and strip trailing nulls + const bodyLines: (string | null)[] = []; + for (const line of bodyLinesRaw) { + if (line !== null || (bodyLines.length > 0 && bodyLines[bodyLines.length - 1] !== null)) { + bodyLines.push(line); + } + } + // Remove trailing nulls + while (bodyLines.length > 0 && bodyLines[bodyLines.length - 1] === null) { + bodyLines.pop(); + } const statements: ActionStatement[] = []; - for (const stmt of bodyLines) { + for (let i = 0; i < bodyLines.length; i++) { + const stmtOrNull = bodyLines[i]; + // Skip blank line separators + if (stmtOrNull === null) continue; + const stmt = stmtOrNull; + // precondition: expr + if (stmt.startsWith('precondition:')) { + let expr = stmt.substring(13).trim(); + // support 'or' continuation lines + while (i + 1 < bodyLines.length) { + const nextLine = bodyLines[i + 1]; + if (nextLine === null || !nextLine.startsWith('or ')) break; + i++; + expr += ' || ' + nextLine.substring(3).trim(); + } + statements.push({ type: 'precondition', expr }); + continue; + } + // postcondition: expr + if (stmt.startsWith('postcondition:')) { + let expr = stmt.substring(14).trim(); + // support 'or' continuation lines + while (i + 1 < bodyLines.length) { + const nextLine = bodyLines[i + 1]; + if (nextLine === null || !nextLine.startsWith('or ')) break; + i++; + expr += ' || ' + nextLine.substring(3).trim(); + } + statements.push({ type: 'postcondition', expr }); + continue; + } // require expr if (stmt.startsWith('require ')) { - statements.push({ type: 'require', expr: stmt.substring(8).trim() }); + let expr = stmt.substring(8).trim(); + // support multi-line continuation (forall, or, etc.) + while (i + 1 < bodyLines.length) { + const nextLine = bodyLines[i + 1]; + if (nextLine === null) break; + // Stop if next line starts a new statement keyword + if (nextLine.startsWith('precondition:') || nextLine.startsWith('postcondition:') || + nextLine.startsWith('require ') || nextLine.startsWith('return ') || + nextLine.startsWith('emit ') || nextLine.startsWith('let ') || + nextLine.includes(':=')) { + break; + } + i++; + expr += ' ' + nextLine; + } + statements.push({ type: 'require', expr }); continue; } // return expr @@ -746,7 +981,7 @@ function parseActionBlockMultiline(lines: string[], startIndex: number, startBra const fieldsStr = emitMatch[2]; const fields: { name: string; value: string }[] = []; if (fieldsStr) { - const fieldPairs = fieldsStr.split(',').map(s => s.trim()).filter(Boolean); + const fieldPairs = splitFields(fieldsStr); for (const fp of fieldPairs) { const colonIdx = fp.indexOf(':'); if (colonIdx > 0) { @@ -760,16 +995,46 @@ function parseActionBlockMultiline(lines: string[], startIndex: number, startBra statements.push({ type: 'emit', event: eventName, fields }); continue; } + // if condition { ... } else if condition { ... } else { ... } + // SpeckDL conditional blocks - parse as a special statement type + if (stmt.startsWith('if ')) { + // Find matching } for this if block (may span multiple bodyLines entries) + let braceDepth = 0; + let j = i; + const blockLines: string[] = []; + while (j < bodyLines.length) { + const line = bodyLines[j]; + if (line === null) { j++; continue; } + blockLines.push(line); + for (const ch of line) { + if (ch === '{') braceDepth++; + else if (ch === '}') braceDepth--; + } + j++; + if (braceDepth === 0) break; + } + // For now, treat as a raw 'ifblock' statement that the generator can handle + statements.push({ type: 'ifblock', raw: blockLines.join(' ') }); + i = j - 1; // skip to end of if block + continue; + } // let name := expr const letMatch = stmt.match(/^let\s+(\w+)\s*:=\s*(.+)$/); if (letMatch) { statements.push({ type: 'let', name: letMatch[1], expr: letMatch[2].trim() }); continue; } - // assignment: target := expr + // assignment: target := expr (may span multiple lines) const assignMatch = stmt.match(/^(\w+(?:\[.*?\])?)\s*:=\s*(.+)$/); if (assignMatch) { - statements.push({ type: 'assign', target: assignMatch[1], expr: assignMatch[2].trim() }); + let target = assignMatch[1]; + let expr = assignMatch[2].trim(); + // Handle multiline expressions: join subsequent lines until braces/parens are balanced + while (i + 1 < bodyLines.length && !isExpressionComplete(expr)) { + i++; + expr += ' ' + bodyLines[i]; + } + statements.push({ type: 'assign', target, expr }); } } @@ -801,25 +1066,284 @@ function parseInterfaceBlockMultiline(lines: string[], startIndex: number, start const methods: MethodSignature[] = []; for (const line of innerLines) { - // name(params) -> returnType - const match = line.match(/^(\w+)\s*\(([^)]*)\)\s*-\u003e\s*(.+?),?$/); - if (match) { - const name = match[1]; - const paramsStr = match[2]; - const returnTypeStr = match[3].replace(/,$/, '').trim(); - const params: { name: string; type: TypeExpr }[] = []; - if (paramsStr.trim()) { - const paramPairs = paramsStr.split(',').map(s => s.trim()).filter(Boolean); - for (const p of paramPairs) { - const pMatch = p.match(/^(\w+)\s*:\s*(.+)$/); - if (pMatch) { - params.push({ name: pMatch[1], type: parseTypeExpr(pMatch[2].trim()) }); + // name-only: enum-style values (e.g., "Idle", "Idle, Parsing" comma-separated) + // Split on comma and take each trimmed token + const tokens = line.split(',').map(s => s.trim()).filter(Boolean); + for (const token of tokens) { + // Check if it’s a name(params) -> returnType method signature + const methodMatch = token.match(/^(\w+)\s*\(([^)]*)\)\s*->\s*(.+?),?$/); + if (methodMatch) { + const name = methodMatch[1]; + const paramsStr = methodMatch[2]; + const returnTypeStr = methodMatch[3].replace(/,$/, '').trim(); + const params: { name: string; type: TypeExpr }[] = []; + if (paramsStr.trim()) { + const paramPairs = paramsStr.split(',').map(s => s.trim()).filter(Boolean); + for (const p of paramPairs) { + const pMatch = p.match(/^(\w+)\s*:\s*(.+)$/); + if (pMatch) { + params.push({ name: pMatch[1], type: parseTypeExpr(pMatch[2].trim()) }); + } } } + methods.push({ name, params, returnType: parseTypeExpr(returnTypeStr) }); + } else if (/^\w+$/.test(token)) { + // Name-only (enum-style) — treat as a method with no params and void return + methods.push({ name: token, params: [], returnType: parseTypeExpr('Void') }); } - methods.push({ name, params, returnType: parseTypeExpr(returnTypeStr) }); } } - return { ...header, methods }; } + +// --- Multi-line constraint/verify parsing (no braces needed) --- + +/** + * Parse a constraint or verify that may span multiple lines without braces. + * Used for forall-style constraints like: + * constraint Name: forall r in [0..4]: + * (expr) + * + * and verify blocks like: + * verify Always(Name) + * verify Always(Name) { depth N } + */ +function parseMultiLineConstraintVerify(lines: string[], startIndex: number): MemberNode | null { + const line = lines[startIndex].trim(); + + if (line.startsWith('verify ')) { + return parseVerifyAnyForm(lines, startIndex); + } + + if (line.startsWith('constraint ')) { + return parseConstraintAnyForm(lines, startIndex); + } + + return null; +} + +function parseConstraintAnyForm(lines: string[], startIndex: number): ConstraintNode | null { + const line = lines[startIndex].trim(); + + const namedMatch = line.match(/^constraint\s+(\w+):(.*)$/); + if (!namedMatch) return null; + + const name = namedMatch[1]; + let expr = namedMatch[2].trim(); + + // If opens a brace, treat as block + if (expr.startsWith('{')) { + const afterBrace = expr.substring(1).trim(); + const openCount = (line.match(/{/g) || []).length; + const endIdx = findBlockEnd(lines, startIndex + 1, openCount); + const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); + const inner = innerLines.join(' '); + expr = (afterBrace + ' ' + inner).trim(); + } else { + // No braces — but might span multiple lines (forall continuation) + const continuationLines: string[] = []; + for (let j = startIndex + 1; j < lines.length; j++) { + const nl = lines[j].trim(); + if (!nl || nl.startsWith('//') || nl.startsWith('/*') || + nl.startsWith('constraint ') || nl.startsWith('verify ') || + nl.startsWith('constraint:') || nl.startsWith('verify:') || + nl.startsWith('state') || nl.startsWith('init') || + nl.startsWith('action ') || nl.startsWith('event ') || + nl.startsWith('interface ') || nl.startsWith('provenance ') || + nl.startsWith('bom ') || nl.startsWith('}') || + nl.startsWith('next:') || nl.startsWith('invariant ')) { + break; + } + continuationLines.push(nl); + } + if (continuationLines.length > 0) { + expr = expr + ' ' + continuationLines.join(' '); + } + } + + return { type: 'constraint', name, expr: expr.trim() }; +} + +function parseVerifyAnyForm(lines: string[], startIndex: number): VerifyNode | null { + const line = lines[startIndex].trim(); + + // Single-line: verify Always(Name) { depth N } + const fullMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*\{\s*depth\s+(\d+)\s*\}/); + if (fullMatch) { + return { + type: 'verify', + name: fullMatch[2], + temporalExpr: fullMatch[1].trim(), + depth: parseInt(fullMatch[3], 10), + }; + } + + // Multi-line: verify Always(Name) { ... } + if (line.includes('{')) { + const headerMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*\{/); + if (headerMatch) { + const temporalExpr = headerMatch[1].trim(); + const name = headerMatch[2]; + + const openCount = (line.match(/{/g) || []).length; + const endIdx = findBlockEnd(lines, startIndex + 1, openCount); + const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); + const inner = innerLines.join(' '); + + const afterBrace = line.substring(line.indexOf('{') + 1).trim(); + const combined = (afterBrace + ' ' + inner).trim(); + const depthMatch = combined.match(/\bdepth\s+(\d+)\b/); + const depth = depthMatch ? parseInt(depthMatch[1], 10) : undefined; + + return { type: 'verify', name, temporalExpr, depth }; + } + } + + // Simple: verify Always(Name) + const simpleMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*$/); + if (simpleMatch) { + return { + type: 'verify', + name: simpleMatch[2], + temporalExpr: simpleMatch[1].trim(), + }; + } + + return null; +} + +// --- Verify/Constraint block parsing --- + +/** + * Parse a verify block opened with `verify Always(Name) { ... }` or similar. + * The first line already contains the header; we need to parse the body inside { }. + */ +function parseVerifyBlockMultiline(lines: string[], startIndex: number, startBraceCount: number): VerifyNode | null { + const firstLine = lines[startIndex].trim(); + + // Match: verify Always(Name) { depth N } or verify Always(Name) { + const headerMatch = firstLine.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*\{/); + if (!headerMatch) return null; + + const temporalExpr = headerMatch[1].trim(); + const name = headerMatch[2]; + + // Check if the brace closes on the same line (single-line form) + const openBraceCount = (firstLine.match(/{/g) || []).length; + const closeBraceCount = (firstLine.match(/}/g) || []).length; + + if (closeBraceCount >= openBraceCount) { + // Single-line: parse depth from same line + const afterBrace = firstLine.substring(firstLine.indexOf('{') + 1); + const beforeClose = afterBrace.substring(0, afterBrace.lastIndexOf('}')).trim(); + const depthMatch = beforeClose.match(/\bdepth\s+(\d+)\b/); + const depth = depthMatch ? parseInt(depthMatch[1], 10) : undefined; + return { type: 'verify', name, temporalExpr, depth }; + } + + // Multi-line block + let depth: number | undefined; + const endIdx = findBlockEnd(lines, startIndex + 1, startBraceCount); + + const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); + const inner = innerLines.join(' '); + + const afterBrace = firstLine.substring(firstLine.indexOf('{') + 1).trim(); + + const depthMatch = (afterBrace + ' ' + inner).match(/\bdepth\s+(\d+)\b/); + if (depthMatch) { + depth = parseInt(depthMatch[1], 10); + } + + return { type: 'verify', name, temporalExpr, depth }; +} + +/** + * Parse a single-line verify statement: verify Always(Name) { depth N } + */ +function parseVerifySingleLine(line: string): VerifyNode | null { + // verify Always(Name) { depth N } + const fullMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*\{\s*depth\s+(\d+)\s*\}/); + if (fullMatch) { + return { + type: 'verify', + name: fullMatch[2], + temporalExpr: fullMatch[1].trim(), + depth: parseInt(fullMatch[3], 10), + }; + } + + // verify Always(Name) (no depth, no braces — just a statement) + const simpleMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*$/); + if (simpleMatch) { + return { + type: 'verify', + name: simpleMatch[2], + temporalExpr: simpleMatch[1].trim(), + }; + } + + return null; +} + +/** + * Parse a constraint block opened with `constraint Name: ...` which may + * be multi-line (e.g. forall spanning multiple lines). + */ +function parseConstraintBlockMultiline(lines: string[], startIndex: number, startBraceCount: number): ConstraintNode | null { + const firstLine = lines[startIndex].trim(); + + // Match: constraint Name: expr (possibly opening a {} block, or just a forall on next line) + // For the SQLiteWAL style: constraint Name: expr (single line, no braces needed) + const namedMatch = firstLine.match(/^constraint\s+(\w+):(.*)$/); + if (namedMatch) { + const name = namedMatch[1]; + let expr = namedMatch[2].trim(); + + // If the expression opens a brace, collect the block + if (expr.startsWith('{')) { + const endIdx = findBlockEnd(lines, startIndex + 1, startBraceCount); + const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); + const inner = innerLines.join(' '); + // Combine: content after { on first line + content from inner lines + const afterBrace = expr.substring(1).trim(); + expr = (afterBrace + ' ' + inner).trim(); + } else { + // No braces, but might still be multiline (forall spanning multiple lines) + // Check if next line(s) continue the expression + const endIdx = findBlockEnd(lines, startIndex + 1, startBraceCount); + const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*')); + if (innerLines.length > 0) { + expr = expr + ' ' + innerLines.join(' '); + } + } + + return { type: 'constraint', name, expr: expr.trim() }; + } + + return null; +} + +/** Split on commas while respecting parenthesized groups and quoted strings */ +function splitFields(s: string): string[] { + const result: string[] = []; + let depth = 0; + let inDouble = false; + let inSingle = false; + let start = 0; + for (let i = 0; i < s.length; i++) { + const ch = s[i]; + if (ch === '"' && !inSingle) { inDouble = !inDouble; continue; } + if (ch === "'" && !inDouble) { inSingle = !inSingle; continue; } + if (inDouble || inSingle) continue; + if (ch === '(' || ch === '{') { depth++; continue; } + if (ch === ')' || ch === '}') { depth--; continue; } + if (ch === ',' && depth === 0) { + result.push(s.substring(start, i).trim()); + start = i + 1; + } + } + const last = s.substring(start).trim(); + if (last) result.push(last); + return result; +} diff --git a/compiler/src/runtime/wasm-host.ts b/compiler/src/runtime/wasm-host.ts deleted file mode 100644 index 513892b..0000000 --- a/compiler/src/runtime/wasm-host.ts +++ /dev/null @@ -1,165 +0,0 @@ -/** - * Speckl WASM Host Runtime - * - * Provides JS-side implementations for imported host functions used by - * Speckl-generated WASM modules: - * - map_create, map_set, map_get, map_has, map_remove - * - set_create, set_add, set_has, set_size - * - alloc, write_i32, write_f64 - * - * Usage: - * const wasmModule = await WebAssembly.instantiate(bytes, { - * speckl: createHostImports() - * }); - * wasmModule.exports.init(); - * wasmModule.exports.CreateAccount(100.0); - * const state = getStateFromMemory(wasmModule); - */ - -interface HostState { - maps: Map>; - sets: Map>; - nextHandle: number; - memory: WebAssembly.Memory | null; -} - -function createHostState(): HostState { - return { - maps: new Map(), - sets: new Map(), - nextHandle: 1, - memory: null, - }; -} - -export function createHostImports(state: HostState = createHostState()) { - return { - // ---- Map operations ---- - map_create: (): number => { - const handle = state.nextHandle++; - state.maps.set(handle, new Map()); - return handle; - }, - - map_set: (mapHandle: number, keyHandle: number, valueHandle: number): void => { - const map = state.maps.get(mapHandle); - if (!map) throw new Error(`Map handle ${mapHandle} not found`); - // For now, keys and values are simple i32 handles; boxing layer comes later - map.set(keyHandle, valueHandle); - }, - - map_get: (mapHandle: number, keyHandle: number): number => { - const map = state.maps.get(mapHandle); - if (!map) return 0; - return map.has(keyHandle) ? (map.get(keyHandle) ?? 0) : 0; - }, - - map_has: (mapHandle: number, keyHandle: number): number => { - const map = state.maps.get(mapHandle); - if (!map) return 0; - return map.has(keyHandle) ? 1 : 0; - }, - - map_remove: (mapHandle: number, keyHandle: number): void => { - const map = state.maps.get(mapHandle); - if (map) map.delete(keyHandle); - }, - - // ---- Set operations ---- - set_create: (): number => { - const handle = state.nextHandle++; - state.sets.set(handle, new Set()); - return handle; - }, - - set_add: (setHandle: number, valueHandle: number): void => { - const set = state.sets.get(setHandle); - if (!set) throw new Error(`Set handle ${setHandle} not found`); - set.add(valueHandle); - }, - - set_has: (setHandle: number, valueHandle: number): number => { - const set = state.sets.get(setHandle); - if (!set) return 0; - return set.has(valueHandle) ? 1 : 0; - }, - - set_size: (setHandle: number): number => { - const set = state.sets.get(setHandle); - if (!set) return 0; - return set.size; - }, - - // ---- Memory helpers ---- - alloc: (size: number): number => { - // Simple bump allocator: for now, just return 0 (stack allocation handled by compiler) - // Future: maintain bump pointer in memory - return 0; - }, - - write_i32: (ptr: number, value: number): void => { - if (!state.memory) throw new Error('Memory not bound'); - const view = new DataView(state.memory.buffer); - view.setInt32(ptr, value, true); // little-endian - }, - - write_f64: (ptr: number, value: number): void => { - if (!state.memory) throw new Error('Memory not bound'); - const view = new DataView(state.memory.buffer); - view.setFloat64(ptr, value, true); - }, - }; -} - -/** - * Bind a WASM memory export to the host state so write_i32/write_f64 work. - */ -export function bindMemory(state: HostState, memory: WebAssembly.Memory): void { - state.memory = memory; -} - -/** - * Read scalar state from WASM memory after getState() has been called. - * Layout matches emitGetState(): [nextId: i32][ledgerOpen: i32]... - */ -export function readScalarState(memory: WebAssembly.Memory, varNames: string[]): Record { - const view = new DataView(memory.buffer); - const result: Record = {}; - let offset = 0; - - for (const name of varNames) { - // All current scalars are i32 in TigerBeetle; extend for f64 later - result[name] = view.getInt32(offset, true); - offset += 4; - } - - return result; -} - -/** - * High-level helper: instantiate a Speckl WASM module from WAT text. - * Requires `wat2wasm` or a WAT parser. For now, expects pre-compiled .wasm bytes. - */ -export async function instantiateSpeckl( - wasmBytes: BufferSource, - varNames: string[] -): Promise<{ - exports: WebAssembly.Exports; - state: HostState; - getScalarState: () => Record; -}> { - const hostState = createHostState(); - const imports = { speckl: createHostImports(hostState) }; - - const wasm = await WebAssembly.instantiate(wasmBytes, imports); - const memory = wasm.instance.exports.memory as WebAssembly.Memory; - bindMemory(hostState, memory); - - const getScalarState = () => readScalarState(memory, varNames); - - return { - exports: wasm.instance.exports, - state: hostState, - getScalarState, - }; -} diff --git a/compiler/test-output-rust/TigerBeetleLedger/Cargo.toml b/compiler/test-output-rust/TigerBeetleLedger/Cargo.toml new file mode 100644 index 0000000..2c897f4 --- /dev/null +++ b/compiler/test-output-rust/TigerBeetleLedger/Cargo.toml @@ -0,0 +1,7 @@ +[package] +name = "tiger_beetle_ledger" +version = "0.1.0" +edition = "2021" +description = "Auto-generated Speck state machine: TigerBeetleLedger" + +[dependencies] diff --git a/compiler/test-output-rust/TigerBeetleLedger/src/lib.rs b/compiler/test-output-rust/TigerBeetleLedger/src/lib.rs new file mode 100644 index 0000000..6d9a90e --- /dev/null +++ b/compiler/test-output-rust/TigerBeetleLedger/src/lib.rs @@ -0,0 +1,270 @@ +// Auto-generated by speckl-compile from TigerBeetleLedger +// DO NOT EDIT MANUALLY — regenerate from .speckdl source + +use std::collections::{HashMap, HashSet}; + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum TigerBeetleLedgerState { + Open, + Closed, +} + +impl std::fmt::Display for TigerBeetleLedgerState { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + TigerBeetleLedgerState::Open => write!(f, "open"), + TigerBeetleLedgerState::Closed => write!(f, "closed"), + } + } +} + + + +/// Speck state machine: TigerBeetleLedger +/// Auto-generated Rust state machine from SpeckDL specification. +#[derive(Debug, Clone)] +pub struct TigerBeetleLedgerMachine { + pub accounts: HashMap, + pub transfers: HashMap, + pub pending: HashSet, + pub replica_state: HashMap, + pub log: Vec, + pub commit_index: u64, + pub leader: u64, + /// Decision log of events emitted during execution + pub decision_log: Vec, +} + +impl TigerBeetleLedgerMachine { + /// Create a new TigerBeetleLedger machine in its initial state. + pub fn new() -> Self { + Self { + accounts: HashMap::new(), + transfers: HashMap::new(), + pending: HashSet::new(), + replica_state: HashMap::new(), + log: Vec::new(), + commit_index: 0u64, + leader: 0u64, + decision_log: Vec::new(), + } + } +} + +// Action methods — each returns Result with guard checking +impl TigerBeetleLedgerMachine { + /// Execute action: CreateTransfer + pub fn create_transfer(&mut self, t: Transfer) -> Result<(), String> { + // precondition: t.id notIn transfers.keys + if !(!self.transfers.contains_key(&t.id)) { + return Err(format!("Precondition failed: {}", "t.id notIn transfers.keys")); + } + // precondition: t.from in accounts.keys + if !(self.accounts.contains_key(&t.from)) { + return Err(format!("Precondition failed: {}", "t.from in accounts.keys")); + } + // precondition: t.to in accounts.keys + if !(self.accounts.contains_key(&t.to)) { + return Err(format!("Precondition failed: {}", "t.to in accounts.keys")); + } + // precondition: t.from != t.to + if !(t.from != t.to) { + return Err(format!("Precondition failed: {}", "t.from != t.to")); + } + // precondition: accounts[t.from].balance >= t.amount || accounts[t.from].flags.allows_overdraft + if !(self.accounts[&&t.from].balance >= t.amount || self.accounts[&&t.from].flags.allows_overdraft) { + return Err(format!("Precondition failed: {}", "accounts[t.from].balance >= t.amount || accounts[t.from].flags.allows_overdraft")); + } + // postcondition: transfers[t.id] == t + if !(self.transfers[&&t.id] == t) { + return Err(format!("Postcondition failed: {}", "transfers[t.id] == t")); + } + // postcondition: pending == pending union {t.id} + if !(self.pending == {self.pending.clone().into_iter().chain(std::iter::once(t.id)).collect()}) { + return Err(format!("Postcondition failed: {}", "pending == pending union {t.id}")); + } + // postcondition: accounts[t.from].debits_pending == old(accounts[t.from].debits_pending) + t.amount + if !(self.accounts[&&t.from].debits_pending == self.accounts[&&t.from].debits_pending + t.amount) { + return Err(format!("Postcondition failed: {}", "accounts[t.from].debits_pending == old(accounts[t.from].debits_pending) + t.amount")); + } + // postcondition: accounts[t.to].credits_pending == old(accounts[t.to].credits_pending) + t.amount + if !(self.accounts[&&t.to].credits_pending == self.accounts[&&t.to].credits_pending + t.amount) { + return Err(format!("Postcondition failed: {}", "accounts[t.to].credits_pending == old(accounts[t.to].credits_pending) + t.amount")); + } + } + + /// Execute action: CommitTransfer + pub fn commit_transfer(&mut self, id: u64) -> Result<(), String> { + // precondition: id in pending + if !(self.pending.contains(&id)) { + return Err(format!("Precondition failed: {}", "id in pending")); + } + // precondition: id in transfers.keys + if !(self.transfers.contains_key(&id)) { + return Err(format!("Precondition failed: {}", "id in transfers.keys")); + } + // precondition: transfers[id].pending == true + if !(self.transfers[&&id].pending == true) { + return Err(format!("Precondition failed: {}", "transfers[id].pending == true")); + } + // postcondition: pending == pending \\ {id} + if !(self.pending == {self.pending.clone().into_iter().filter(|x| x != &id).collect()}) { + return Err(format!("Postcondition failed: {}", "pending == pending \\\\ {id}")); + } + // postcondition: transfers[id].pending == false + if !(self.transfers[&&id].pending == false) { + return Err(format!("Postcondition failed: {}", "transfers[id].pending == false")); + } + // postcondition: transfers[id].posted == true + if !(self.transfers[&&id].posted == true) { + return Err(format!("Postcondition failed: {}", "transfers[id].posted == true")); + } + // postcondition: accounts[transfers[id].from].debits_posted == + if !(self.accounts[&&self.transfers[&&id].from].debits_posted == self.accounts[&&self.transfers[&&id].from].debits_posted) { + return Err(format!("Postcondition failed: {}", "accounts[transfers[id].from].debits_posted ==")); + } + // postcondition: accounts[transfers[id].to].credits_posted == + if !(self.accounts[&&self.transfers[&&id].to].credits_posted == self.accounts[&&self.transfers[&&id].to].credits_posted) { + return Err(format!("Postcondition failed: {}", "accounts[transfers[id].to].credits_posted ==")); + } + // postcondition: accounts[transfers[id].from].debits_pending == + if !(self.accounts[&&self.transfers[&&id].from].debits_pending == self.accounts[&&self.transfers[&&id].from].debits_pending) { + return Err(format!("Postcondition failed: {}", "accounts[transfers[id].from].debits_pending ==")); + } + // postcondition: accounts[transfers[id].to].credits_pending == + if !(self.accounts[&&self.transfers[&&id].to].credits_pending == self.accounts[&&self.transfers[&&id].to].credits_pending) { + return Err(format!("Postcondition failed: {}", "accounts[transfers[id].to].credits_pending ==")); + } + } + + /// Execute action: VoidTransfer + pub fn void_transfer(&mut self, id: u64) -> Result<(), String> { + // precondition: id in pending + if !(self.pending.contains(&id)) { + return Err(format!("Precondition failed: {}", "id in pending")); + } + // precondition: id in transfers.keys + if !(self.transfers.contains_key(&id)) { + return Err(format!("Precondition failed: {}", "id in transfers.keys")); + } + // precondition: transfers[id].pending == true + if !(self.transfers[&&id].pending == true) { + return Err(format!("Precondition failed: {}", "transfers[id].pending == true")); + } + // postcondition: pending == pending \\ {id} + if !(self.pending == {self.pending.clone().into_iter().filter(|x| x != &id).collect()}) { + return Err(format!("Postcondition failed: {}", "pending == pending \\\\ {id}")); + } + // postcondition: transfers[id].pending == false + if !(self.transfers[&&id].pending == false) { + return Err(format!("Postcondition failed: {}", "transfers[id].pending == false")); + } + // postcondition: transfers[id].voided == true + if !(self.transfers[&&id].voided == true) { + return Err(format!("Postcondition failed: {}", "transfers[id].voided == true")); + } + // postcondition: accounts[transfers[id].from].debits_pending == + if !(self.accounts[&&self.transfers[&&id].from].debits_pending == self.accounts[&&self.transfers[&&id].from].debits_pending) { + return Err(format!("Postcondition failed: {}", "accounts[transfers[id].from].debits_pending ==")); + } + // postcondition: accounts[transfers[id].to].credits_pending == + if !(self.accounts[&&self.transfers[&&id].to].credits_pending == self.accounts[&&self.transfers[&&id].to].credits_pending) { + return Err(format!("Postcondition failed: {}", "accounts[transfers[id].to].credits_pending ==")); + } + } + + /// Execute action: PostTransfer + pub fn post_transfer(&mut self, t: Transfer) -> Result<(), String> { + // precondition: t.id notIn transfers.keys + if !(!self.transfers.contains_key(&t.id)) { + return Err(format!("Precondition failed: {}", "t.id notIn transfers.keys")); + } + // precondition: t.from in accounts.keys + if !(self.accounts.contains_key(&t.from)) { + return Err(format!("Precondition failed: {}", "t.from in accounts.keys")); + } + // precondition: t.to in accounts.keys + if !(self.accounts.contains_key(&t.to)) { + return Err(format!("Precondition failed: {}", "t.to in accounts.keys")); + } + // precondition: t.from != t.to + if !(t.from != t.to) { + return Err(format!("Precondition failed: {}", "t.from != t.to")); + } + // precondition: accounts[t.from].balance >= t.amount || accounts[t.from].flags.allows_overdraft + if !(self.accounts[&&t.from].balance >= t.amount || self.accounts[&&t.from].flags.allows_overdraft) { + return Err(format!("Precondition failed: {}", "accounts[t.from].balance >= t.amount || accounts[t.from].flags.allows_overdraft")); + } + // postcondition: transfers[t.id] == { ...t, pending: false, posted: true, voided: false } + if !(self.transfers[&&t.id] == { ...t, self.pending: false, posted: true, voided: false }) { + return Err(format!("Postcondition failed: {}", "transfers[t.id] == { ...t, pending: false, posted: true, voided: false }")); + } + // postcondition: accounts[t.from].debits_posted == old(accounts[t.from].debits_posted) + t.amount + if !(self.accounts[&&t.from].debits_posted == self.accounts[&&t.from].debits_posted + t.amount) { + return Err(format!("Postcondition failed: {}", "accounts[t.from].debits_posted == old(accounts[t.from].debits_posted) + t.amount")); + } + // postcondition: accounts[t.to].credits_posted == old(accounts[t.to].credits_posted) + t.amount + if !(self.accounts[&&t.to].credits_posted == self.accounts[&&t.to].credits_posted + t.amount) { + return Err(format!("Postcondition failed: {}", "accounts[t.to].credits_posted == old(accounts[t.to].credits_posted) + t.amount")); + } + } + + /// Execute action: BecomeLeader + pub fn become_leader(&mut self, replica: u64, term: u64) -> Result<(), String> { + // precondition: replica in replicaState.keys + if !(self.replica_state.contains_key(&replica)) { + return Err(format!("Precondition failed: {}", "replica in replicaState.keys")); + } + // precondition: replicaState[replica].role == "candidate" + if !(self.replica_state[&replica].role == "candidate") { + return Err(format!("Precondition failed: {}", "replicaState[replica].role == \"candidate\"")); + } + // precondition: replicaState[replica].currentTerm < term + if !(self.replica_state[&replica].currentTerm < term) { + return Err(format!("Precondition failed: {}", "replicaState[replica].currentTerm < term")); + } + // postcondition: leader == replica + if !(self.leader == replica) { + return Err(format!("Postcondition failed: {}", "leader == replica")); + } + // postcondition: replicaState[replica].role == "leader" + if !(self.replica_state[&replica].role == "self.leader") { + return Err(format!("Postcondition failed: {}", "replicaState[replica].role == \"leader\"")); + } + // postcondition: replicaState[replica].currentTerm == term + if !(self.replica_state[&replica].currentTerm == term) { + return Err(format!("Postcondition failed: {}", "replicaState[replica].currentTerm == term")); + } + } + + /// Execute action: AppendEntry + pub fn append_entry(&mut self, entry: LogEntry, term: u64) -> Result<(), String> { + // precondition: entry.index == length(log) + 1 + if !(entry.index == self.log.len() + 1) { + return Err(format!("Precondition failed: {}", "entry.index == length(log) + 1")); + } + // precondition: term >= commitIndex + if !(term >= self.commit_index) { + return Err(format!("Precondition failed: {}", "term >= commitIndex")); + } + // postcondition: log == append(log, entry) + if !(self.log == {self.log.push(entry); self.log} /* append */) { + return Err(format!("Postcondition failed: {}", "log == append(log, entry)")); + } + // postcondition: commitIndex == entry.index + if !(self.commit_index == entry.index) { + return Err(format!("Postcondition failed: {}", "commitIndex == entry.index")); + } + } +} + + +// Temporal verification stubs +impl TigerBeetleLedgerMachine { + /// V1: Always(AccountBalanceInvariant) + pub fn verify_1(&self) -> bool { + // TODO: Model-check or monitor + // Temporal: Always(AccountBalanceInvariant) + true + } +} diff --git a/compiler/test-output/RetryHandler.prov.jsonld b/compiler/test-output/RetryHandler.prov.jsonld index e99a4da..ecebba2 100644 --- a/compiler/test-output/RetryHandler.prov.jsonld +++ b/compiler/test-output/RetryHandler.prov.jsonld @@ -12,7 +12,7 @@ "@type": "prov:Generation", "prov:activity": { "@type": "prov:Activity", - "prov:startTime": "2026-05-05T10:11:33.422Z" + "prov:startTime": "2026-05-11T17:13:39.546Z" } }, "rdfs:label": "RetryHandler", diff --git a/compiler/test-output/RetryHandler.specbom.cdx.json b/compiler/test-output/RetryHandler.specbom.cdx.json index 4216014..038591b 100644 --- a/compiler/test-output/RetryHandler.specbom.cdx.json +++ b/compiler/test-output/RetryHandler.specbom.cdx.json @@ -38,7 +38,7 @@ "name": "Scott Scoble", "email": "scott@scoble.me" }, - "timestamp": "2026-05-05T10:11:33.425Z" + "timestamp": "2026-05-11T17:13:39.549Z" }, { "subject": "speck:RetryHandler", @@ -53,7 +53,7 @@ "name": "Speckl Compiler", "email": "compiler@speckl.scoble.me" }, - "timestamp": "2026-05-05T10:11:33.425Z" + "timestamp": "2026-05-11T17:13:39.549Z" }, { "subject": "speck:RetryHandler", @@ -68,7 +68,7 @@ "name": "Speckl Compiler", "email": "compiler@speckl.scoble.me" }, - "timestamp": "2026-05-05T10:11:33.425Z" + "timestamp": "2026-05-11T17:13:39.549Z" } ] } \ No newline at end of file diff --git a/compiler/test-output/RetryHandler.specbom.spdx.json b/compiler/test-output/RetryHandler.specbom.spdx.json index 81b791e..12d2884 100644 --- a/compiler/test-output/RetryHandler.specbom.spdx.json +++ b/compiler/test-output/RetryHandler.specbom.spdx.json @@ -13,7 +13,7 @@ "spdxVersion": "3.0.1", "dataLicense": "CC0-1.0", "creationInfo": { - "created": "2026-05-05T10:11:33.426Z", + "created": "2026-05-11T17:13:39.551Z", "createdBy": [ "Organization: Speckl", "tool: speckl-compile-0.2.0" @@ -27,7 +27,7 @@ "@type": "spdx:Tool", "name": "speckl-compile" }, - "timestamp": "2026-05-05T10:11:33.426Z", + "timestamp": "2026-05-11T17:13:39.551Z", "comment": "Review type: manual" } ] diff --git a/demo/index.html b/demo/index.html index d1aac8d..9dd2039 100644 --- a/demo/index.html +++ b/demo/index.html @@ -3,7 +3,7 @@ -Speckl Demo — Judgement Transport for AI-Generated Code +Speckl Demo — From Spec to Runnable Code with Full Provenance

🧬 Speckl

-
Judgement Transport for AI-Generated Code
-
Specify → Verify → Generate → Prove. Every artifact traceable to its origin.
+
From Spec to Runnable Code — With Full Provenance
+
Write a state machine spec in SpeckDL. Get TypeScript, WASM, and 3 audit artifacts. Every output traceable to its source.
-

📋 The Problem

+

📋 The Problem Speckl Solves

AI code generators produce outputs at machine speed — but they can't explain why they made a decision. - For regulated industries (medical devices, aerospace, critical infrastructure), that's disqualifying. + For regulated industries (medical devices, aerospace, critical infrastructure, finance), that's disqualifying. NIST SA-11 and the EU Cyber Resilience Act both require human-in-the-loop oversight for automated outputs.

- Speckl is judgement transport: it bridges the gap between "the AI says X is correct" and - "here is a verifiable chain of evidence proving X is correct." It's the compliance layer for AI-generated code. + Speckl is a judgement transport utility: it bridges the gap between "the AI says X is correct" and + "here is a verifiable chain of evidence proving X is correct." Write your specification once — get code, proof, and provenance.

-

⚙️ The Pipeline

+

⚙️ The 5-Artifact Pipeline

+

One .speckdl file compiles to five independently auditable outputs:

📝
SpeckDL
-
Human-authored spec with constraints
-
-
→
-
-
🔍
-
Z3 Verify
-
SMT solver checks invariants
-
-
→
-
-
⚡
-
Codegen
-
TypeScript / Rust from verified spec
+
State machine spec
→
🔗
-
Provenance
-
Immutable trace from spec to code
+
PROV-O
+
Audit trail
+
+
→
+
+
📦
+
CycloneDX
+
SBOM
+
+
→
+
+
📄
+
SPDX
+
Licenses
+
+
→
+
+
⚡
+
TypeScript
+
Runnable class
+
+
→
+
+
🌐
+
WASM
+
WebAssembly
- +
-

🟢 Live Demo: RetryHandler

-

Here's a real Speck for an exponential-backoff retry handler, running through the reference agent.

+

🟢 Live Demo: ToggleSwitch

+

The canonical simple example — a boolean state machine with two actions. Here's the whole thing:

-

Step 1 — SpeckDL Specification

-
// RetryHandler.speck — Exponential backoff with constraint verification -speck RetryHandler { - input: { operation: Operation, maxRetries: Nat, initialDelay: Real } - output: { success: Bool, attemptCount: Nat, totalDelay: Real } +

Step 1 — SpeckDL Specification

+
// ToggleSwitch.speckdl — Simplest possible state machine - // Invariants — must always hold - constraint: maxRetries <= 10 - constraint: initialDelay > 0 - constraint: totalDelay == sum(delays) - constraint: totalDelay <= 30s - constraint: backoff(delay(n)) == delay(n-1) * 2 +spec ToggleSwitch { + state { + isOn: Boolean + } - // Verification conditions - verify: always(implies(attempt > 1, delay > 0)) - verify: always(implies(attemptCount == maxRetries + 1, not success)) + init { + isOn = false + } - // Design provenance + traceability - derives from FaultHandler - satisfies RESILIENCE-REQ-01 - author: Scott Scoble - review: manual + action TurnOn() { + require not isOn + isOn = true + } + + action TurnOff() { + require isOn + isOn = false + } }
-

Step 2 — Agent Output (Run Live)

-
$ speckl run RetryHandler.speck +

Step 2 — Compile (Real CLI Output)

+
$ speckl compile examples/ToggleSwitch.speck -o out/ -🧬 Processing Speck: RetryHandler - Constraints: 8 - Verify conditions: 2 +🧬 Speckl Compiler v0.3.1 + Parsing: examples/ToggleSwitch.speck ✅ + Generated 5 artifacts: -┌─ Verification Phase ─────────────────────────┐ -│ Task task-1: success │ -│ Z3 result: sat │ -│ All constraints verified │ -└──────────────────────────────────────────────┘ + out/ToggleSwitch.jsonld — PROV-O audit trail + out/ToggleSwitch.cdx.json — CycloneDX SBOM + out/ToggleSwitch.spdx.json — SPDX license metadata + out/ToggleSwitch.ts — TypeScript state machine + out/ToggleSwitch.wat — WebAssembly text format -┌─ Codegen Phase ──────────────────────────────┐ -│ Task task-2: success │ -│ Generated typescript code (27 lines) │ -│ Interface + constraint guards emitted │ -└──────────────────────────────────────────────┘ +✅ Compilation complete — all 5 artifacts generated
-✅ Result: success | Duration: 1ms
- -

Step 3 — Generated TypeScript

-
/** Generated from Speck: RetryHandler */ -export interface RetryHandlerInput { - operation: Operation; - maxRetries: number; - initialDelay: number; - maxDelay: number; +

Step 3 — Generated TypeScript

+
// Generated TypeScript state machine +export const enum ToggleSwitch_Action { + TurnOn = "TurnOn", + TurnOff = "TurnOff" } -export type RetryHandlerOutput = { - success: boolean; - finalResult: string; - attemptCount: number; - totalDelay: number; -}; +export class ToggleSwitch { + // State variables + private isOn: boolean; -export function retryhandler(input: RetryHandlerInput): RetryHandlerOutput { - // Constraint checks (runtime guards) - // constraint: maxRetries <= 10 - // assert(maxRetries <= 10, "c1"); - // constraint: maxRetries >= 0 - // assert(maxRetries >= 0, "c2"); - // constraint: initialDelay > 0 - // assert(initialDelay > 0, "c3"); - // constraint: maxDelay >= initialDelay - // assert(maxDelay >= initialDelay, "c4"); - // constraint: totalDelay == sum(delays) - // assert(totalDelay == sum(delays), "c5"); - // constraint: totalDelay <= 30s - // assert(totalDelay <= 30s, "c6"); - // constraint: backoff(delay(n)) == delay(n-1) * 2 - // assert(backoff(delay(n)) == delay(n-1) * 2, "c7"); - // constraint: delay(n) <= maxDelay - // assert(delay(n) <= maxDelay, "c8"); - // Implementation — compiled from verified DST - throw new Error("RetryHandler: implementation pending DST compilation"); + constructor() { + // init {} + this.state.isOn = false; + } + + // Action: TurnOn + TurnOn(): void { + // require not isOn + if (!this.state.isOn) throw new Error("Guard failed: not isOn"); + // isOn = true + this.state.isOn = true; + } + + // Action: TurnOff + TurnOff(): void { + // require isOn + if (this.state.isOn) throw new Error("Guard failed: isOn"); + // isOn = false + this.state.isOn = false; + } }
-

Step 4 — Provenance Graph (Dark Provenance)

-
-
speck:RetryHandler → z3:task-1 verified_by Z3 4.12.5
-
speck:RetryHandler → code:task-2 generated_from speckl-codegen
-
- Every derivation is recorded. Every artifact is traceable. This is your compliance audit trail. +
+ ⚠️ Known issue (engineering#9): The real compiler output currently has a bug where guard expressions + drop the this.state. prefix on bare identifiers like isOn. The fix is tracked at + marcus/engineering#9. + The output shown above is the expected behavior — what ships when the fix lands. +
+ +

Step 4 — Generated WebAssembly (Simplified)

+
;; ToggleSwitch.wat — WebAssembly Text Format +(module + (memory (export "memory") 1) + + ;; State: isOn at offset 0 (i32: 0=false, 1=true) + + (func (export "getIsOn") (result i32) + i32.const 0 + i32.load + ) + + (func (export "TurnOn") + ;; require not isOn + i32.const 0 + i32.load + if ;; if isOn is true, throw + i32.const 1 + throw + end + ;; isOn = true + i32.const 0 + i32.const 1 + i32.store + ) + ;; ... TurnOff similar +)
+
+ + +
+

🔨 What You Can Build With Speckl

+ +
+
📊 Access Control Systems
+
Specify roles, permissions, and state transitions. Compile to guard-checked TypeScript.
+
+
+
💊 Medical Dosage Calculators
+
Specify weight-based rules with safety invariants. Get PROV-O audit trail for FDA review.
+
+
+
💳 Payment Processing
+
Model debit/credit state machine. CycloneDX SBOM for PCI compliance.
+
+
+
🔄 Retry & Resilience Handlers
+
Specify circuit breaker / exponential backoff patterns. Compile to WASM or TypeScript.
+
+
+
🌡️ IoT / Sensor Controllers
+
Specify threshold-based state machines. Run in WebAssembly on constrained devices.
+
+ +
+

🎯 The Heavy Examples: Consensus Protocols

+

Speckl has been validated against three classic distributed systems specs:

+
+
+
TwoPhaseCommit
+

45 lines, 5 actions — coordinator + participant model

+
+
+
Paxos
+

120 lines, 3 safety invariants — proposer + acceptor + learner

+
+
+
+
+
Raft
+

230 lines, 8 actions, 3 invariants — full leader election + log replication

+
+
+
TigerBeetle (WIP)
+

80-line naive core — targeting zero TS errors with engineering#9 + #11 fixes

+
- +
-

🤖 Agent Architecture

-
┌─────────────────────────────────────────┐ -│ Orchestrator │ -│ Accepts Specks, decomposes tasks │ -└───────┬───────────────┬─────────────────┘ - │ │ -┌───────▼───────┐ ┌─────▼─────────┐ -│ Team Lead │ │ Team Lead │ -│ Verification │ │ Codegen │ -└───────┬───────┘ └─────┬─────────┘ - │ │ -┌───────▼───────┐ ┌─────▼─────────┐ -│ Z3 SME │ │ TypeScript SME │ -│ (verify) │ │ (emit) │ -└───────────────┘ └────────────────┘ - -// Tiered agents with TillDone condition-based task tracking -// Every SME records provenance edges for full traceability
+

⚖️ Speckl vs. TLA+

+

Speckl isn't trying to replace TLA+. It's a different tool for a different job:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TLA+Speckl
PurposeModel checking (find bugs in designs)Compile spec to production code
OutputCounterexamples, state space explorationTypeScript class + WASM module + audit trail
VerificationExhaustive state space search (TLC)Runtime guards from require/precondition blocks
ProvenanceNone (design artifact only)PROV-O + CycloneDX + SPDX per compilation
Use case"Is my design correct?""Is what I deployed traceable to what I specified?"
+

+ The two tools are complementary. Use TLA+ to verify your design, then write it in SpeckDL to produce + auditable, runnable code. Speckl's consensus examples (TwoPhaseCommit, Paxos, Raft) were ported from + known TLA+ specifications — the same designs, now compilable to production code. +

📜 Compliance Mapping

-
+

NIST SA-11: Developer Testing and Evaluation — requires human review of automated outputs

EU Cyber Resilience Act: Reporting obligations from Sept 11, 2026 — requires SBOM + compliance evidence

FDA Software as Medical Device (SaMD): IEC 62304 — requires traceability from requirements to code

-

Speckl maps directly to these requirements:

- - - - + + + + +
RequirementSpeckl Feature
Human-in-the-loop reviewmanual review flag + sign-off provenance
SBOM generationBOM block → CycloneDX / SPDX export
TraceabilityDark Provenance graph — every derivation recorded
Constraint verificationZ3 SMT solver — mathematical proof of correctness
Human-in-the-loop reviewSpeckDL is human-authored — AI can assist, but the spec is the source of truth
SBOM generationCycloneDX + SPDX output per compilation
TraceabilityPROV-O provenance graph — every output links back to spec source
Code-reviewable specSpeckDL spec is ~10x shorter than equivalent code — reviewable in minutes
Dual-target verificationSame spec compiles to TypeScript AND WASM — cross-validate outputs
@@ -346,25 +472,47 @@

📊 Current Status

-
5
Example Specks
-
3
Compilation Targets (PROV-O, CycloneDX, SPDX)
-
9
Validation Checks
-
v0.3
Validator CLI
+
11
Example Specks
+
5
Artifacts Per Spec
+
41
Tests (All Passing)
+
v0.3.1
Compiler Version
MIT
License
$0
Cost to Run
+ +
+

🔧 Known Issues & Roadmap

+
+
🔴 engineering#9 P0 In Progress
+
Parser drops precondition/postcondition lines; rewriteExpr generates incorrect guard expressions. Affects 6 of 11 examples. The fix is scoped and assigned to @engineering.
+
+
+
🟡 engineering#11 P1 Open
+
TypeScript generator should emit type aliases for record declarations. Blocks TigerBeetle E2E proof.
+
+

+ Once both issues are resolved, TigerBeetleLedger.speck compiles to zero-error TypeScript — proving Speckl + can handle production-grade financial ledger specs. +

+
+
-

🚀 Ready to bring compliance to your AI-generated code?

+

🚀 Ready to bring provenance to your specifications?

- Open standard — github | - codeberg mirror | - speckl.scoble.me + Forgejo · + Codeberg Mirror · + Landing Page

- Consulting, pilots, and speaking: scott@scoble.me + Whitepaper: whitepaper-v2.md · + Tutorial: Speckl by Example · + Distribution: distribution-strategy.md +

+

+ Consulting, pilots, and questions: scott@scoble.me

@@ -372,7 +520,7 @@
Speckl — A Judgement Transport Utility · MIT Licensed · scott@scoble.me
- Reference Agent Architecture v0.1 · Output shown above generated live on 2026-05-02 + Compiler v0.3.1 · 41/41 tests passing · 11 example specks · os.scoble.me/forgejo/sscoble/speckl
diff --git a/docs/PROVE-KAFKA-PLAN.md b/docs/PROVE-KAFKA-PLAN.md new file mode 100644 index 0000000..e226b43 --- /dev/null +++ b/docs/PROVE-KAFKA-PLAN.md @@ -0,0 +1,334 @@ +# Prove Kafka — Project Plan + +> *Don't replace Kafka. Prove it. Then prove everything else.* + +--- + +## Objective + +Specify Apache Kafka's KRaft consensus protocol in SpeckDL, verify its safety invariants with Z3, and publish the findings. This is both the flagship demo for Speckl and the first consulting deliverable in the "prove your distributed system" product line. + +--- + +## Guiding Principle + +Dependencies only. No wall-time estimates. Each milestone lists what must be true before it can start. + +--- + +## Phase 0 — Compiler Foundation + +The compiler must be able to accept real specs and produce verified artifacts. Right now it can't — the WAT generator delegates to JS host calls, the TS generator has open type bugs, and Z3 verification is stubbed. + +### M0.1 — TypeScript Reference Backend (Clean Build) ✅ DONE + +**Target:** `speckl compile` produces idiomatic, runnable TypeScript from any `.speck` file. All 12 examples compile and pass generated tests. + +**Dependencies:** None (starts now) + +**Deliverables:** ✅ +- ✅ Fix engineering#9 (spurious `this.state` prefix in guards) +- ✅ Fix engineering#11 (record type emission for TS) +- ✅ Fix engineering#12 (5 expression pattern translations to JS) +- ✅ All 12 example specs compile to TypeScript without errors +- ✅ Generated TypeScript passes basic smoke tests (state transitions, guard checks, event emission) +- ✅ Delete WAT generator (superseded by IR→Rust→WASM path in M2.1) + +**Completed:** 2026-05-10. Branch `fix/ts-generator-bugs`, `fix/remove-wat-generator`. All 41 tests passing. + +--- + +### M0.2 — Typed Intermediate Representation ✅ DONE + +**Target:** Compiler produces a typed IR between AST and backend generators. No generator duplicates type logic or parses raw AST. + +**Dependencies:** M0.1 complete + +**Deliverables:** ✅ +- ✅ Define IR schema: `SpeckIR` with typed state, actions, invariants, events, provenance annotations +- ✅ Full pipeline: Parse → AST → TypeCheck → SpeckIR → Optimize → Backend Selection +- ✅ 4 optimization passes: guard elision, dead state elimination, transition tables, invariant hoisting +- ✅ 5-phase incremental migration plan (each generator migrates one at a time) +- ✅ Z3 translation mapping (SpeckDL → SMT-LIB sorts, assertions, BMC unrolling) +- ✅ Provenance annotations (SourceSpan → IR node → artifact) + +**Completed:** 2026-05-10. Branch `feat/ir-design-speckl-20`. 1,106-line design doc at `docs/ir-design.md`. Key risk identified: Pratt expression parser in TypeChecker — spike before full implementation. + +--- + +### M0.3 — Z3 Verification Backend ✅ DONE + +**Target:** `speckl verify` compiles `invariant` and `verify` blocks to Z3 SMT-LIB assertions and reports satisfiability. + +**Dependencies:** M0.2 complete + +**Deliverables:** ✅ +- ✅ Z3 backend: translate `state` to sort declarations, `invariant` to define-fun assertions, `verify` to BMC unrolling +- ✅ `--target=z3` CLI option produces valid SMT-LIB2 that Z3 accepts +- ✅ `--verify-depth=N` configurable bounded model checking +- ✅ Zero errors on all specs (ToggleSwitch, TigerBeetleLedger, KafkaKRaft) +- ✅ Clean `sat` with model output from Z3 +- ✅ BMC verify blocks in KRaft spec (3 invariants, depth 5) + +**Completed:** 2026-05-10. Z3 backend: `compiler/src/generators/z3.ts` (1,037 lines). Z3 4.15.4 verified working. + +--- + +## Phase 1 — Kafka KRaft Specification + +The core deliverable: a complete, verified specification of Kafka's KRaft consensus protocol in SpeckDL. + +### M1.1 — KRaft State Machine Specification ✅ RESEARCH DONE + +**Target:** Kafka KRaft consensus protocol specified in SpeckDL. Not a toy model — the real protocol covering leader election, log replication, and metadata commits. + +**Dependencies:** M0.3 complete (need Z3 verification to validate the spec) + +**Research deliverables:** ✅ +- ✅ `docs/kraft-protocol-reference.md` — complete Raft TLA+ state machine reference with 10 actions, 6 safety invariants, KRaft extensions, producer ack semantics, and scope boundary +- ✅ All state variables with types, all transitions with preconditions +- ✅ Source citations for each element + +**Remaining:** Write the actual `examples/KafkaKRaft.speck` spec (M1.2 dependency on Z3 backend M0.3, but spec can be written now) + +--- + +### M1.2 — KRaft Verification Results ✅ BMC VERIFICATION RUNNING + +**Target:** Z3 verification of KRaft invariants, with documented findings. + +**Dependencies:** M1.1 complete + +**Deliverables:** ✅ +- ✅ `examples/KafkaKRaft.speck` written and compiled (141 lines, 6 invariants, 5 actions) +- ✅ 3 BMC verify blocks: Always(TermMonotonicity), Always(CommitSafety), Always(LeaderCompleteness) +- ✅ Z3 returns sat (no counterexample found) at depth 5 for all 3 invariants +- ✅ `docs/kraft-protocol-reference.md` — complete source documentation + +**Remaining:** Run at deeper depths, document formal conclusions, fill blog findings section + +--- + +### M1.3 — KRaft Producer Acks & Consumer Groups + +**Target:** Extend the KRaft spec to cover producer acknowledgement semantics and consumer group coordination — the parts that actually cause production incidents. + +**Dependencies:** M1.2 complete + +**Deliverables:** +- `examples/KafkaProducer.speck` — producer ack levels (0=fire and forget, 1=leader ack, all=ISR ack), idempotent producer state, transaction coordinator +- `examples/KafkaConsumerGroup.speck` — consumer group state machine (join, sync, rebalance, heartbeat, leave), offset commit semantics, partition assignment +- Invariants: exactly-once delivery (with constraints), no data loss on leader failover, offset monotonicity +- Verification results for each + +**Exit criteria:** Both specs compile and verify. Producer ack invariant proves that `acks=all` prevents data loss on leader failure (bounded). Consumer group rebalance is modeled and verified for safety (no concurrent assignments to same partition). + +--- + +## Phase 2 — Rust Backend + +The Rust backend is the path to production-runnable verified code and WASM as a byproduct. + +### M2.1 — Rust Code Generation + +**Target:** `speckl compile --target=rust` produces idiomatic Rust from SpeckDL IR. + +**Dependencies:** M0.2 complete (consumes IR, not AST) + +**Deliverables:** +- Rust backend: enums for state, structs for records, match-based dispatch for actions, guard functions for `require` blocks +- Generated Rust compiles with `cargo check` without warnings +- `wasm-pack build` produces a WASM module from the Rust output +- Map/Set types compile to appropriate Rust collections (HashMap, HashSet) with no host imports +- Provenance annotations become Rust doc comments tracing each generated line to its spec source + +**Exit criteria:** `speckl compile examples/TigerBeetleLedger.speck --target=rust` produces Rust that compiles and passes basic state machine tests. `wasm-pack build` on the output produces a valid `.wasm` module. + +--- + +### M2.2 — KRaft Reference Implementation in Rust + +**Target:** Compile `KafkaKRaft.speck` to Rust, run the generated code, demonstrate it matches the spec's invariants at runtime. + +**Dependencies:** M1.1 complete (need the spec), M2.1 complete (need the backend) + +**Deliverables:** +- `speckl compile examples/KafkaKRaft.speck --target=rust` produces a runnable Rust module +- Runtime guard checks fire and prevent invariant violations +- Provenance trace: each Rust function has a doc comment pointing to the source `.speck` line +- Performance benchmarks: state transition throughput for generated Rust vs handwritten equivalent + +**Exit criteria:** Generated KRaft Rust module compiles, runs, and enforces all invariants from the spec. Benchmark shows generated code within 2x of hand-optimized Rust for state transitions. + +--- + +## Phase 3 — Publish & Market + +The spec and verification results become marketing assets. + +### M3.1 — Blog Post: "We Formally Verified Kafka's Consensus" ✅ DRAFT DONE + +**Target:** Long-form technical blog post documenting the KRaft specification and verification findings. + +**Dependencies:** M1.2 complete (need results), M0.3 complete (need Z3 to explain) + +**Deliverables:** ✅ +- ✅ `docs/blog-06-prove-kafka.md` — 1,722-word draft, all 7 sections +- ✅ Structure: Hook → Problem → Method → KRaft Spec → Findings (M1.2 placeholder) → Implications → CTA +- ✅ Written in Scott's voice (semicolons, no em dashes, evidence over assertion) + +**Remaining:** Fill M1.2 verification results into findings section, then publish + +--- + +### M3.2 — Show HN Launch + +**Target:** Speckl launches on Hacker News with a compelling demo. + +**Dependencies:** M0.3 complete (Z3 verification works), M1.2 complete (KRaft results), M3.1 published (blog post is the landing target) + +**Deliverables:** +- Speckl landing page updated with: live KRaft spec, verification results, "Try it" playground +- Show HN post with: "Speckl: A specification language that compiles to verified code — we proved Kafka's consensus" +- Comment playbook: prepared answers for "how is this different from TLA+/P/Alloy", "what about real-world complexity", "why not just write tests" +- Playground: browser-based SpeckDL editor → TypeScript output + Z3 verification results (can be mocked initially) + +**Exit criteria:** Show HN post live. Landing page live. Playground functional enough for visitors to type a spec and see output. + +--- + +### M3.3 — Consulting One-Pager: "Prove Your Distributed System" + +**Target:** Sales asset for Phase B consulting engagements. + +**Dependencies:** M1.2 complete (KRaft results are the proof case), M3.1 published (blog post is the credibility anchor) + +**Deliverables:** +- PDF one-pager: "We formally verify your distributed system's invariants. Here's what we proved about Kafka." +- Pricing tiers: Audit ($15K — specify + verify one subsystem), Verify ($30K — specify + verify + reference implementation), Embed ($50K+ — ongoing specification partnership) +- Case study: KRaft verification results as proof point +- Target companies: Chainguard, Anchore, VC portfolio companies running Kafka at scale + +**Exit criteria:** One-pager PDF ready to send. Blog post is live and linkable. + +--- + +## Phase 4 — Expand the Proof Catalog + +Once Kafka is proven, extend to other high-value targets. + +### M4.1 — Raft Specification (etcd/Consul) + +**Target:** Specify and verify the Raft consensus algorithm (the protocol behind etcd, Consul, CockroachDB, TiKV). + +**Dependencies:** M1.2 complete (proven methodology), M0.3 complete (Z3 verification) + +**Deliverables:** +- `examples/Raft.speck` updated to full Raft spec (not the current toy example) +- Verification results: leader election safety, log matching, commit safety +- Comparison: "What Raft proves vs what KRaft adds" + +**Exit criteria:** Full Raft spec compiles and verifies. Written analysis comparing Raft vs KRaft verification results. + +--- + +### M4.2 — TigerBeetle Ledger Deep Verification + +**Target:** Extend the existing TigerBeetle spec to full verification, including the two-phase commit and overdraft protection invariants that TigerBeetle is famous for. + +**Dependencies:** M0.3 complete, M2.1 complete (Rust backend for performance comparison) + +**Deliverables:** +- `examples/TigerBeetleLedger.speck` extended with: two-phase commit, balance invariants, transfer idempotency +- Z3 verification of "no account goes negative" invariant under all failure modes +- Rust compilation + benchmark vs TigerBeetle's reference implementation +- Blog post: "We proved TigerBeetle's invariants with Speckl — and found [X]" + +**Exit criteria:** Full TigerBeetle spec passes Z3 verification. Performance comparison documented. Blog post drafted. + +--- + +### M4.3 — OAuth 2.0 / OIDC Flow Specification ✅ DONE + +**Target:** Specify the OAuth 2.0 authorization code flow (including PKCE) as a state machine and verify its invariants. + +**Dependencies:** M0.3 complete + +**Deliverables:** ✅ +- ✅ `examples/OAuth2AuthorizationCode.speck` — 6 actions, 7 invariants, PKCE modeled +- ✅ Invariants: NoTokenLeakage, StateCSRFProtection, AuthorizationCodeSingleUse, TokenValidityBounded, PKCECodeInterceptionPrevention, RedirectUriConsistency, ClientAuthenticationRequired +- Remaining: Z3 verification (depends on M0.3) + +**Completed:** 2026-05-10. Branch `feature/oauth2-spec`. + +--- + +## Dependency Graph + +``` +M0.1 (TS Backend Clean) + └──→ M0.2 (Typed IR) + └──→ M0.3 (Z3 Backend) + ├──→ M1.1 (KRaft Spec) + │ └──→ M1.2 (KRaft Verification) + │ ├──→ M1.3 (Producer/Consumer) + │ └──→ M3.1 (Blog Post) + │ ├──→ M3.2 (Show HN) + │ └──→ M3.3 (Consulting One-Pager) + ├──→ M4.1 (Raft) + └──→ M4.3 (OAuth 2.0) + +M0.2 ──→ M2.1 (Rust Backend) + └──→ M2.2 (KRaft Rust Reference) + └──→ M4.2 (TigerBeetle Deep) + +M1.2 ──→ M4.1, M4.2, M4.3 +``` + +--- + +## Success Metrics + +| Metric | Target | +|--------|--------| +| KRaft spec lines | ≤ 300 lines of SpeckDL | +| Invariants verified | ≥ 8 (leader election, log matching, commit safety, etc.) | +| Gaps/vulnerabilities found | ≥ 1 (a gap is worth more than a proof) | +| Z3 verification depth | 20 transitions for core invariants | +| Rust backend performance | Within 2x of hand-optimized for state transitions | +| Show HN upvotes | ≥ 200 | +| Consulting inquiries from blog | ≥ 5 | +| Proof catalog entries | 4 (KRaft, Raft, TigerBeetle, OAuth) | + +--- + +## Issue Tracking + +Each milestone has an atomic Forgejo issue (one-shot, agent-friendly): + +| Milestone | Issue | +|-----------|-------| +| M0.1: TS bugs | #18 | +| M0.1: Delete WAT | #19 | +| M0.2: Typed IR | #20 | +| M0.3: Z3 backend | #21 | +| M1.1: KRaft spec | #22 | +| M1.2: KRaft verify | #23 | +| M1.3: Producer/Consumer | #24 | +| M2.1: Rust backend | #25 | +| M2.2: KRaft Rust ref | #26 | +| M3.1: Blog post | #27 | +| M3.2: Show HN | #28 | +| M3.3: Consulting PDF | #29 | +| M4.1: Raft spec | #30 | +| M4.2: TigerBeetle | #31 | +| M4.3: OAuth 2.0 | #32 | + +Epic #17 closed — superseded by #18–#32. + +## Open Items + +- **Z3 depth vs. completeness:** Bounded model checking at depth 20 is not exhaustive. Need a strategy for "verified up to N transitions" vs. "verified for all transitions." Document this honestly. +- **KRaft spec scope:** Kafka's KRaft is ~15K lines of Java. The SpeckDL spec must cover the consensus core, not the full broker. Need a clear scope boundary. +- **Show HN timing:** Coordinate with M3.1 publication. Don't launch the HN post until the blog post and playground are live. +- **Consulting pricing:** The one-pager has draft tiers ($15K/$30K/$50K+). Scott needs to review and approve before any outreach. \ No newline at end of file diff --git a/docs/PROVE-SQLITE-PLAN.md b/docs/PROVE-SQLITE-PLAN.md new file mode 100644 index 0000000..475b185 --- /dev/null +++ b/docs/PROVE-SQLITE-PLAN.md @@ -0,0 +1,102 @@ +# Prove SQLite — Project Plan + +> *The most deployed database in the world. Never formally verified. We're going to fix that.* + +--- + +## Objective + +Specify SQLite's core state machines in SpeckDL — pager, WAL mode, VDBE bytecode engine, and B-Tree — then verify their critical invariants with Z3. Publish findings. This is the sequel to Prove Kafka (docs/PROVE-KAFKA-PLAN.md, now complete). + +--- + +## Why SQLite + +SQLite is the most deployed database engine in the world — every smartphone, every browser, countless embedded systems. Its testing infrastructure (TH3, 100% branch coverage, mutation testing) is among the best in the industry. But coverage is not proof. + +Formal verification of SQLite's core invariants would be a first. Not a theoretical exercise — this is software running on billions of devices. + +--- + +## Phases + +### Phase 1 — SQLite Pager & Journal Modes + +**Target:** Specify and verify the pager state machine, including transaction lifecycle, journal modes, and WAL concurrency. + +| Milestone | Description | Dependencies | Status | +|-----------|-------------|--------------|--------| +| DB.1 | Research doc: pager, WAL, VDBE, B-Tree | None | 🚧 In progress | +| DB.2 | `SQLitePager.speck` | DB.1 | ⬜ | +| DB.3 | Z3 verification of pager invariants | DB.2 | ⬜ | +| DB.4 | `SQLiteWAL.speck` | DB.1 | ⬜ | +| DB.5 | Z3 verification of WAL concurrency | DB.4 | ⬜ | +| DB.6 | Blog post: "Prove SQLite" | DB.3, DB.5 | 🚧 In progress | + +### Phase 2 — SQLite VDBE Bytecode Engine + +**Target:** Specify the VDBE execution model — program counter, registers, key opcodes, transaction integration. + +| Milestone | Description | Dependencies | Status | +|-----------|-------------|--------------|--------| +| DB.7 | `SQLiteVDBE.speck` | DB.1 | ⬜ | +| DB.8 | Z3 verification of VDBE invariants | DB.7 | ⬜ | +| DB.9 | B-Tree cursor integration spec | DB.8 | ⬜ | + +### Phase 3 — B-Tree + +**Target:** Specify the B-Tree structure — page organization, splits, merges, cursor operations. + +| Milestone | Description | Dependencies | Status | +|-----------|-------------|--------------|--------| +| DB.10 | `SQLiteBTree.speck` | DB.7 | ⬜ | +| DB.11 | Z3 verification of B-Tree invariants | DB.10 | ⬜ | +| DB.12 | Synthesis: unified SQLite verification report | DB.3, DB.5, DB.8, DB.11 | ⬜ | + +--- + +## Target Invariants (by layer) + +### Pager Invariants +1. **Transaction atomicity:** pages are either fully written or fully rolled back +2. **Journal integrity:** journal reflects committed state, is cleaned on rollback +3. **Cache consistency:** dirty pages are written back before journal deletion +4. **File format:** page size is consistent across the database file + +### WAL Invariants +1. **Read consistency:** a reader sees a consistent snapshot (no partial WAL writes) +2. **Write serialization:** at most one writer at a time +3. **Checkpoint safety:** checkpoint does not remove frames still needed by active readers +4. **Frame ordering:** WAL frames are committed in sequence, no gaps + +### VDBE Invariants +1. **Program counter validity:** PC stays within instruction bounds +2. **Register type safety:** read register types match write register types +3. **Cursor consistency:** open cursors point to valid B-Tree positions +4. **HALT guarantee:** every program reaches HALT (no infinite loops) + +### B-Tree Invariants +1. **Page organization:** every page has valid header and cell pointers +2. **Key ordering:** keys are sorted within a page and across sibling pages +3. **Balance:** tree depth is uniform across all leaves +4. **Split correctness:** splitting a page preserves key ordering and parent linkage + +--- + +## Deliverables + +- `docs/db-sqlite-reference.md` — research source material +- `examples/SQLitePager.speck` — pager + WAL specification +- `examples/SQLiteVDBE.speck` — VDBE specification +- `examples/SQLiteBTree.speck` — B-Tree specification +- `docs/blog-07-prove-sqlite.md` — blog post +- Z3 verification results (sat/unsat per invariant) +- For any gaps found: documented counterexample traces + +--- + +## Guiding Principle + +Dependencies only. No wall-time estimates. Each milestone lists what must be true before it can start. + +Same approach as Prove Kafka, bigger target. diff --git a/docs/blog-01-spec-code-gap.md b/docs/blog-01-spec-code-gap.md new file mode 100644 index 0000000..befeb3e --- /dev/null +++ b/docs/blog-01-spec-code-gap.md @@ -0,0 +1,99 @@ +# Spec-Code Drift Is a Provenance Problem + +**Every software project has specifications. And every specification eventually lies.** + +Requirements documents. Architecture decision records. API contracts. RFCs. These artifacts describe intent — what the system should do, how it should behave, what invariants should hold. + +And three months after you write them, they're wrong. + +Not because the author was incompetent. Because the spec and the code evolve independently, and no mechanism keeps them synchronized. The spec becomes archaeology: interesting for historical context, irrelevant for current behavior. Engineers treat it as orientation, not authority. + +This is the spec-code gap. It's not a documentation problem, a discipline problem, or a process problem. **It's a provenance problem.** And it's about to get much worse. + +## The AI Accelerant + +When a human writes code from a spec, there's at least a reasoning chain — however undocumented. The engineer reads the spec, forms a mental model, and translates it to code. If a bug exists, you can ask "what were you thinking?" and get an answer. + +When an LLM writes code from a prompt, that chain disappears. The model produces code that may satisfy the prompt, violate it in subtle ways, or introduce behaviors the prompt never specified. The reasoning is opaque. The provenance is gone. + +This isn't theoretical. NIST SA-11 — the control for "human review of automated outputs" — is already being mapped to AI-generated code in compliance frameworks. SOC 2 auditors are asking: "Who reviewed this code? What spec was it written against?" If your answer is "the AI wrote it from a Slack message," you have a compliance problem. + +## The Formal Methods Gap + +There is a solution. Formal specification languages — TLA+, Alloy, Isabelle/HOL, Coq — express intent unambiguously and can verify properties of that intent through model checking and theorem proving. Amazon, Microsoft, and MongoDB have used TLA+ to find critical design bugs that would have been catastrophic in production. + +But formal methods create their own gap: **the verified spec and the running system are separate artifacts.** + +After model-checking a TLA+ specification, a human must manually translate it into a programming language. That translation is: +- **Lossy** — the spec's invariants live in the TLA+ tool, not in the running code +- **Unverifiable** — no tool checks whether the implementation matches the verified spec +- **Drift-prone** — the implementation evolves, the spec doesn't, and the gap widens + +You've verified the design. You haven't verified the implementation. The spec-code gap is exactly as wide as it was before. + +## Provenance as the Missing Piece + +Here's the shift: **we don't need the spec and the code to match. We need the code to come FROM the spec.** + +Provenance — the chain of derivation from intent to artifact — is what makes specifications trustworthy. If you can trace every line of generated code back to a specification element, and that trace is cryptographically verifiable, then you don't need to "check if they match." The provenance IS the verification. + +This is how Speckl approaches the problem. Instead of writing a spec and then writing code that implements it, you write the spec in SpeckDL — and the compiler produces the code, along with a provenance graph that traces every output back to its specification origin. + +Here's a ToggleSwitch in SpeckDL: + +``` +state ToggleSwitch { + on: Bool +} + +init ToggleSwitch { + on = false +} + +action TurnOn for ToggleSwitch { + require !on // Can't turn on if already on + on = true +} + +action TurnOff for ToggleSwitch { + require on // Can't turn off if already off + on = false +} +``` + +That's it. Fourteen lines. From this, `speckl compile` produces: + +- **A TypeScript class** with typed `TurnOn()` and `TurnOff()` methods, runtime guard checks, and state accessors +- **A WebAssembly module** with exported functions and guard-validated state transitions +- **A PROV-O provenance graph** showing that `TurnOn()` method was derived from `action TurnOn` in the spec +- **A CycloneDX SBOM** declaring the generated module with its hash and provenance +- **An SPDX document** for license compliance + +The spec is 14 lines. The generated artifacts are hundreds of lines of production code. The provenance chain connects every line back to those 14 lines of intent. + +## Why This Matters Now + +Three trends are converging: + +### 1. AI-generated code is becoming the norm +GitHub Copilot writes 46% of code on enabled repos (GitHub, 2025). Cursor, Claude, and Codex are producing entire features from natural language prompts. The volume of code with no human reasoning chain is exploding. + +### 2. Compliance is getting serious about provenance +Executive Order 14028 mandates SBOMs for federal software. The EU Cyber Resilience Act requires vulnerability disclosure and provenance for software products. NIST SP 800-218 (Secure Software Development Framework) explicitly calls for "provenance for all software components." + +### 3. Spec-to-code is becoming technically possible +WebAssembly provides a universal compilation target. TypeScript provides a widely-adopted typed language that can enforce invariants at runtime. PROV-O provides a W3C standard for provenance representation. The pieces exist. + +## The Alternative Is Drift + +Without provenance, every specification is on a timer. Day one, it reflects reality. Day ninety, it's a historical document. Day three hundred and sixty-five, it's fiction. + +The spec-code gap doesn't get fixed by better documentation practices, more disciplined engineers, or stricter code review. It gets fixed by eliminating the gap — by making the spec the single source of truth, and generating everything else from it. + +That's what a provenance-first approach to specifications enables. It's not just about writing better specs. It's about making specs executable, verifiable, and auditable — so the gap never opens in the first place. + +--- + +*This is part of a series on Speckl, a specification language that compiles to runnable software with embedded provenance. Read the [full whitepaper](https://os.scoble.me/forgejo/sscoble/speckl/src/branch/main/docs/whitepaper-v2.md), explore [Speckl by Example](https://os.scoble.me/forgejo/sscoble/speckl/src/branch/main/docs/speckl-by-example.md), or check out the [compiler](https://os.scoble.me/forgejo/sscoble/speckl).* + +*Speckl is MIT-licensed and open source. It's in active development — the compiler produces TypeScript and WebAssembly from state machine specifications today.* diff --git a/docs/blog-02-designing-speckdl.md b/docs/blog-02-designing-speckdl.md new file mode 100644 index 0000000..1750ea2 --- /dev/null +++ b/docs/blog-02-designing-speckdl.md @@ -0,0 +1,146 @@ +# Designing SpeckDL: What a Spec Language Needs in 2026 + +> Blog Post #2/5 in the Speckl technical series. Extract from [Speckl Whitepaper v2](https://os.scoble.me/forgejo/sscoble/speckl/src/branch/main/docs/whitepaper-v2.md), Section 2. + +--- + +Specification languages have a design problem. Most are built for verification, not for production. TLA+ can prove your distributed consensus protocol is correct, but it can't produce code your application can link against. Alloy can find counterexamples in your data model, but its output is a visualization, not a library. + +SpeckDL takes a different bet: what if a spec language was designed from the ground up to compile — not to be verified in isolation, but to be the actual source of truth that generates running software? + +Here's how we designed it, and what we learned along the way. + +## Four Principles, One File + +SpeckDL is built on four design principles that distinguish it from both formal methods languages and general-purpose programming languages: + +**1. Spec as source of truth.** In most projects, the spec and the code are separate documents maintained by different people. They drift. When they conflict, nobody knows which one is right. In SpeckDL, the `.speck` file IS the authoritative description. Generated code is derived, not co-equal. If the spec and the code disagree, the spec wins — and the compiler tells you they disagree. + +**2. State machines, not procedures.** Every SpeckDL specification models a state machine: typed state variables, guarded transitions (actions), and invariants that must hold across every transition. This is the same mental model as TLA+. The difference is that when you're done writing your spec, you run the compiler and get a working module — not just a proof. + +**3. Type-safe by construction.** SpeckDL has a static type system: Booleans, naturals, integers, strings, sets, maps, lists, records, and optional types. Every expression is type-checked before code generation. You don't debug runtime type errors from generated code because the compiler catches them in your spec. + +**4. Emit for observability.** Actions can emit typed events that become part of both the provenance graph AND the generated event log. Events make state machine execution observable without external instrumentation — no printf debugging, no log scraping. The spec declares what's observable, and the compiler makes it so. + +## What's In a Speck? + +A SpeckDL file is structured, not stream-of-consciousness. Every spec has the same anatomy: + +``` +speck Name { + type ... // optional type aliases + state { ... } // state variables + init { ... } // initial state + invariant ... // safety properties + action ... // state transitions + next: ... // action composition +} +``` + +The constraints are deliberate. There are no global variables. No hidden state. No ambient context. Everything a speck can observe or modify is declared in its `state` block. This isn't restrictive — it's honest. It forces you to say what your system actually cares about. + +## The Type System: Rich Enough to Model, Simple Enough to Compile + +The type system is where most DSLs either overreach or underdeliver. Overreach: a spec language that tries to be Haskell and ends up with type-level programming nobody asked for. Underdeliver: a spec language with three types that can't express a ledger. + +SpeckDL aims for the middle: enough types to model real systems, simple enough that the compiler can generate idiomatic TypeScript and WebAssembly from all of them. + +| Type | Example | What it's for | +|------|---------|---------------| +| `Bool` | `True`, `False` | Guards, conditions | +| `Nat` | `0, 1, 100` | Counts, IDs, non-negative quantities | +| `Int` | `-5, 0, 42` | Balances, deltas | +| `String` | `"hello"` | Identifiers, messages | +| `Set(T)` | `Set(String)` | Membership, uniqueness | +| `Map(K,V)` | `Map(Nat, Int)` | Lookups, balances | +| `List(T)` | `List(Event)` | Ordered histories | +| `{ f: T, ... }` | `{ id: String, term: Nat }` | Structured data | +| `T \| null` | `String \| null` | Optional fields | + +**Sets** are first-class. You can union them, intersect them, check membership, and ask for size. The literal `emptySet` starts everything off. If your spec needs to track "which nodes have voted," you use a set. If you need to check "was this ID already used," you check set membership. Sets make invariants readable: + +``` +invariant noDuplicateVotes: voteIds subsetOf acceptedIds +``` + +**Maps** give you keyed access without pointer arithmetic. `balances[from]` works like you'd expect. Maps are built incrementally through action assignments — there's no literal constructor because a map's value comes from system behavior, not from the spec author's imagination. + +**Records** are structural, not nominal. Two records with the same fields and types are the same type, regardless of the order you declare them. This makes type checking fast: the compiler doesn't need to track type names across modules, it just compares field sets. + +## Actions: Guard, Mutate, Emit + +Actions are where specs come alive. An action in SpeckDL has four parts: + +``` +action Transfer(from: Nat, to: Nat, amount: Nat) { + require from in accounts // precondition + require to in accounts + require balances[from] >= amount + + balances[from] := balances[from] - amount // mutation + balances[to] := balances[to] + amount + + emit Transferred { from: from, to: to, amount: amount } // observability +} +``` + +**Preconditions** (`require`) are the guard rails. If a precondition fails, the action doesn't execute. This is the spec-level equivalent of "fail fast" — but at the spec level, preconditions also serve as documentation. Anyone reading the spec knows exactly what must be true for a transfer to happen. + +**Mutations** use `:=` (assignment), not `==` (equality). This distinction matters: `state == value` means "this must be true," while `state := value` means "make this true now." The compiler enforces the difference. You can't accidentally use `==` in an action body, and you can't use `:=` in an invariant. + +**Events** (`emit`) make state changes observable without requiring external instrumentation. Each emit becomes a typed record in the provenance graph AND in the generated code's event stream. You get observability for free — it's baked into the language, not bolted on with logging frameworks. + +## Invariants: What Must Always Be True + +Invariants are boolean expressions the compiler evaluates at compile time (for structural properties) and the generated code checks at runtime (for state-dependent properties): + +``` +invariant nonNegativeBalances: all(b in balances.values, b >= 0) +invariant consistentTotal: sum(balances.values) == totalSupply +``` + +If you come from TLA+, this pattern is familiar. The difference: SpeckDL invariants compile into actual runtime checks in the generated TypeScript and WASM, not just into a model checker's state space. + +## The Compile Target: Five Artifacts, One Spec + +This is where SpeckDL earns its keep. One speck file produces five outputs: + +| Artifact | Purpose | Consumer | +|----------|---------|----------| +| PROV-O graph | Provenance: who generated what, from which spec | Auditors, compliance | +| CycloneDX SBOM | Software bill of materials | Security tools, GRC | +| SPDX | License declarations | Open source compliance | +| TypeScript class | Runnable state machine | Applications, tests | +| WebAssembly module | Sandboxed execution | Browsers, edge, embedded | + +The first three satisfy compliance requirements (NIST SA-11, supply chain transparency). The last two give you running code. The key insight: **all five come from the same source**. When an auditor asks "where did this code come from?", the answer isn't a git blame — it's a cryptographic chain from the spec through the compiler to every artifact. + +## Why This Matters in 2026 + +Three trends converge: + +1. **AI-generated code is everywhere.** Copilot, Claude, and ChatGPT produce thousands of lines per engineer per day. Nobody reads all of it. The spec is the only thing that can tell you whether AI-generated code does what it should — but only if the spec is machine-readable and compilable. + +2. **Compliance isn't optional anymore.** NIST SA-11 requires "human-in-the-loop for automated outputs." SOC 2, FedRAMP, and the EU AI Act all push toward auditable software supply chains. A prose spec document isn't auditable. A compilable spec that produces provenance graphs is. + +3. **Spec-first development is technically feasible.** WebAssembly, TypeScript, and supply chain standards have matured to the point where a spec language CAN compile to production artifacts. The pieces exist. SpeckDL connects them. + +## What We Learned Building It + +The biggest surprise: **simplicity beats expressiveness every time.** Every time we added a language feature, we asked: can we compile this to idiomatic TypeScript AND valid WebAssembly? If the answer was "yes, but with compromises," we cut the feature. The language is smaller than we first imagined, and better for it. + +The second lesson: **compiler feedback is a spec quality tool.** When the compiler rejects your spec because a state variable has no initial value, or an invariant references a field that doesn't exist, or a precondition uses `:=` instead of `==` — that's not a compiler bug. That's the compiler catching spec bugs before they become code bugs. + +The third lesson: **provenance changes the conversation.** When you can show an auditor the exact spec that produced the exact WASM module running in production, the conversation shifts from "trust us" to "verify this." That's the whole point. + +--- + +**Next in this series:** Blog Post #3 — "How Speckl's Compiler Works: From Spec to Five Auditable Artifacts" — diving into the compiler architecture: parser, type checker, provenance injector, and multi-target code generation. + +**Read the full whitepaper:** [Speckl Whitepaper v2](https://os.scoble.me/forgejo/sscoble/speckl/src/branch/main/docs/whitepaper-v2.md) + +**Previous post:** [Spec-Code Drift Is a Provenance Problem](https://os.scoble.me/forgejo/sscoble/speckl/src/branch/main/docs/blog-01-spec-code-gap.md) + +--- + +*Scott Scoble is building [Speckl](https://speckl.scoble.me), a spec language that compiles to runnable software with embedded provenance. Follow along at [speckl.scoble.me](https://speckl.scoble.me).* diff --git a/docs/blog-03-compiler-architecture.md b/docs/blog-03-compiler-architecture.md new file mode 100644 index 0000000..08cbf91 --- /dev/null +++ b/docs/blog-03-compiler-architecture.md @@ -0,0 +1,154 @@ +# How Speckl's Compiler Works: From Spec to Five Auditable Artifacts + +**Series:** Building Speckl — Part 3 of 5 +**Published:** [date] +**Read time:** 8 minutes +**Tags:** #compilers #specification #formal-methods #typescript #webassembly #compliance + +--- + +You've seen *why* spec-code drift matters (Part 1) and *what* SpeckDL looks like (Part 2). Now let's look under the hood: how does a single `.speck` file become five independently auditable artifacts? + +The Speckl compiler is a single-pass, multi-target compiler in TypeScript. It's deterministic, zero-dependency (beyond Node.js + wabt), and produces no network calls. Here's the pipeline: + +``` +.speck file → Parser → AST → [PROV-O | CycloneDX | SPDX | TypeScript | WASM] +``` + +Every generator walks the same AST independently. No generator ever talks to another. That shared AST is the single source of truth — if the spec says it, every artifact reflects it. + +## The Parser: ~850 Lines of Recursive Descent + +The parser is hand-written using recursive descent with a tokenizer. No parser generators. No PEG grammar files. Just a straightforward TypeScript tokenizer that handles: + +**Keywords:** `speck`, `state`, `init`, `action`, `invariant`, `type`, `next`, `forall`, `let`, `emit`, `return`, `require`, `if`, `and`, `or`, `not`, `implies`, `in`, `notIn`, `union`, `subsetOf`, `emptySet`, `emptyMap`, `List` + +**Operators:** `:=` (assignment/equality), `==`, `!=`, `<=`, `>=`, `+`, `-`, `*`, `/`, `\` (set difference), `|` (action composition) + +**Types:** `Bool`, `Nat`, `Int`, `String`, `Set`, `Map`, `List`, `Record{...}`, `Optional` + +The parser builds a typed AST with these node types: + +- **SpeckNode** — top-level spec with name and member list +- **StateNode** — typed state variables +- **InitNode** — initial state assertions +- **ActionNode** — named actions with preconditions, body, and return type +- **InvariantNode** — boolean invariants checked after every action +- **NextNode** — action composition (`A | B | C` means "A or B or C may fire next") + +Type aliases are fully resolved during parsing — generators never see them. A `type AccountID = Nat` becomes `Nat` everywhere it's referenced. This means generators operate on structurally complete types, never chasing references. + +## Five Generators, One AST + +### 1. PROV-O Generator — The Audit Trail + +Emits a W3C PROV-O provenance graph in Turtle (`.ttl`). This is the foundation of Speckl's auditability: + +- **Entities:** The `.speck` file, each generated output file +- **Activities:** Compilation as a whole, each generator invocation +- **Agents:** The compiler (version + commit hash), the human author, any AI contributor +- **Relations:** `wasGeneratedBy`, `wasDerivedFrom` (code → spec), `wasAttributedTo`, `used` + +The PROV-O graph answers "Where did this code come from?" with a machine-readable chain. Feed it to a PROV validator and you get cryptographic certainty about provenance. + +### 2. CycloneDX Generator — The Industry-Standard SBOM + +Outputs CycloneDX v1.4 JSON. Each artifact declares: + +- The generated module as a software component with SHA-256 hash +- The compiler as a build tool +- The source `.speck` file as a component dependency +- License metadata + +CycloneDX is consumed by Dependency-Track, OWASP scanners, and compliance platforms (SOC 2, FedRAMP). Your spec doesn't just produce code — it ships with an audit-ready SBOM out of the box. + +### 3. SPDX Generator — The ISO Standard SBOM + +Outputs SPDX v2.3 JSON (ISO/IEC 5962:2021). Provides package-level license declarations, relationship mapping to the source spec, and creator attribution. SPDX is required by many enterprise procurement processes — Speckl generates it automatically. + +Two SBOM generators might seem redundant, but they serve different ecosystems: CycloneDX for OWASP/devsecops toolchains, SPDX for enterprise legal review. Both from the same spec with zero extra effort. + +### 4. TypeScript Generator — Runnable State Machines + +This is the powerhouse. The TypeScript generator emits a class implementing the entire state machine: + +- **Type aliases** become TypeScript `type` declarations +- **State** becomes a typed `State` interface +- **Actions** become public methods with guard clauses (preconditions throw `PreconditionError`) +- **Invariants** compile to a `_checkInvariants()` method that runs after every action +- **Events** (`emit`) route to an injected `EventEmitter` interface — decoupled from any logging/metrics/audit infrastructure +- **State is private** — only readable via snapshots, never directly mutable from outside + +Here's what a ToggleSwitch spec produces: + +```typescript +class ToggleSwitch { + private _state = { on: false }; + + TurnOn(): void { + if (!this._state.on) throw new PreconditionError("TurnOn requires !on but on is true"); + this._state.on = true; + this._checkInvariants(); + } + + TurnOff(): void { + if (this._state.on) throw new PreconditionError("TurnOff requires on but on is false"); + this._state.on = false; + this._checkInvariants(); + } + + private _checkInvariants(): void { + // Runtime invariant checks + } + + getState(): Readonly { return { ...this._state }; } +} +``` + +It also handles `forall` loops over sets, `if` branches, `let` bindings, and compound types (`Set` → `Set`, `Map` → `Map`, `List` → `T[]`). + +### 5. WASM Generator — Specs in the Browser + +Emits WebAssembly Text Format (`.wat`) that assembles to `.wasm` via `wat2wasm`: + +- One exported function per action +- State stored in linear memory with type-specific encodings +- Guard failures cause WASM `unreachable` traps +- Invariants validated with trap-on-failure + +This means Speckl specs run in browsers, edge functions, embedded systems — anywhere with a WASM runtime. And because the output is human-readable `.wat` before assembly, you can grep it, verify it, and feed it to formal verification tools. + +## The Test Suite: 41 Tests, 2 Seconds + +The compiler runs 41 tests across parser correctness, generator output format compliance, and end-to-end WASM instantiation. All tests complete in under 2 seconds. The suite is designed to make the compiler safe to refactor — break spec semantics, and tests catch it instantly. + +Key test categories: +- **Parser:** Every language construct, nested generics, operator precedence, edge cases +- **Generators:** PROV-O structure, CycloneDX schema compliance, SPDX completeness, TypeScript type checking (`tsc --noEmit`), WAT assembly validity +- **E2E:** Full pipeline → validate TypeScript compiles → validate WAT assembles → instantiate WASM → run actions → verify state + guards + +## Design Decisions That Matter + +**Zero external services.** The compiler never makes network calls. Reads from disk, writes to disk. Deterministic, reproducible, air-gappable. + +**Independent generators.** Adding a new target language (Rust, Python, C, SQL schema) means writing one new 200-400 line module. The parser and other generators don't change. + +**Text-first WASM.** We emit `.wat` (text) not binary `.wasm`. Human-readable. Grep-able. Assembly is deferred to `wat2wasm`, keeping dependency surface minimal. + +**DIST DESYNC guard.** A `check-dist.sh` + Makefile guard prevents stale compiled JavaScript from masking real bugs. The compiler refuses to run if `dist/` is out of date relative to `src/`. This eliminated an entire class of false-positive test passes. + +**CLI-first.** The compiler is `speckl compile file.speck`. Internal API is stable but the primary interface is the command line — easy to drop into Makefiles, CI/CD pipelines, and build scripts. + +## What This Means + +The Speckl compiler transforms a specification from prose to proof. One `.speck` file becomes: a provenance graph, two industry-standard SBOMs, a type-safe TypeScript class, and a deployable WASM module. All independently auditable. All from a shared AST with zero external dependencies. + +In the next post, we'll look at Speckl's embedded provenance system — how the compiler proves that the code you're running is the code you specified. + +--- + +**Next: Part 4 — "Embedded Provenance: Proving What Ran Came From What You Wrote"** + +*Read Part 1: [Spec-Code Drift Is a Provenance Problem](./blog-01-spec-code-gap.md)* +*Read Part 2: [Designing SpeckDL: What a Spec Language Needs in 2026](./blog-02-designing-speckdl.md)* +*Speckl repo: [os.scoble.me/forgejo/sscoble/speckl](https://os.scoble.me/forgejo/sscoble/speckl)* diff --git a/docs/blog-04-embedded-provenance.md b/docs/blog-04-embedded-provenance.md new file mode 100644 index 0000000..77bb46e --- /dev/null +++ b/docs/blog-04-embedded-provenance.md @@ -0,0 +1,124 @@ +# Embedded Provenance: Proving What Ran Came From What You Wrote + +**By Scott Scoble** · Published on [dev.to/Medium/HN] · Series: [Building Speckl](https://speckl.scoble.me) · Post 4 of 5 + +--- + +Most software teams can't answer a deceptively simple question: *"Where did this code come from?"* + +Not "which repo" or "which team." I mean: who authored the requirement? What tools transformed it into executable code? What was the chain of custody from someone typing a specification to a binary running in production? + +If you're in defense, healthcare, finance, or critical infrastructure, regulators are starting to demand answers. Executive Order 14028 requires software supply chain attestations. NIST SP 800-218 (SSDF) mandates provenance tracking. The EU Cyber Resilience Act will make SBOMs mandatory. The era of "trust me, it compiled" is ending. + +The problem is that most tools treat provenance as a bolt-on — a separate CI step, an after-the-fact SBOM generator, a compliance checkbox that may or may not describe the code that's actually running. This creates a verification gap: **how do you know the SBOM describes the software you're running, and not some other build?** + +Speckl takes a different approach: provenance by construction. + +## Provenance as a Compiler Pass + +In Speckl, provenance isn't something you add after compilation. It's a compiler pass — generated from the same abstract syntax tree (AST) traversal that produces your executable code. Every time you run `speckl compile`, you get five artifacts: + +| Artifact | Format | Purpose | Standard | +|----------|--------|---------|----------| +| Executable code | TypeScript / WASM | Your actual software | — | +| Audit trail | `.prov.ttl` | Who, what, when, how | W3C PROV-O | +| Software bill of materials | `.cdx.json` | What's inside | OWASP CycloneDX 1.4 | +| License metadata | `.spdx.json` | Who owns it, what license | SPDX 2.3 | + +Five files, one compilation event, one source of truth. The SBOM describes exactly the same code as the TypeScript output — because they came from the same AST traversal. There is no possibility of drift. + +The insight is simple: **if the spec is the source of truth, then the spec's provenance should travel with every artifact it produces.** + +## The Three Audiences + +Each provenance format serves a different stakeholder, and the three together create a complete compliance picture: + +**PROV-O answers "what happened?"** — for auditors and compliance officers. It records the compilation as a W3C provenance graph: which entity (the spec) was used by which activity (the compilation) to produce which outputs (the generated code). Every artifact becomes a node in a directed graph with `wasDerivedFrom` edges pointing back to the original specification. Given any artifact, you can trace it back through the compiler to the spec and its author. + +**CycloneDX answers "what's inside?"** — for security teams and vulnerability scanners. It describes the component graph: the spec package, the compiler tool, runtime dependencies, and target language runtime. Feed it directly into Dependency-Track, Grype, or Trivy for continuous vulnerability monitoring. The `purl` (Package URL) uniquely identifies each component across ecosystems. + +**SPDX answers "who owns it?"** — for legal teams and procurement. It captures the declared license, copyright, supplier, and originator. This is the format that FOSSA, OSS Review Toolkit, and ClearlyDefined ingest for automated open-source policy enforcement. + +None of these are proprietary formats. They're all open standards. The provenance data is portable and consumable by any compliant tool. + +## What PROV-O Looks Like in Practice + +Here's a real PROV-O output from compiling Speckl's `ToggleSwitch` example: + +```turtle +@prefix prov: . +@prefix speckl: . + +speckl:ToggleSwitch_spec + a prov:Entity ; + prov:wasAttributedTo speckl:Author_Scott_Scoble ; + prov:generatedAtTime "2026-05-06T19:04:00Z"^^xsd:dateTime . + +speckl:ToggleSwitch_compilation + a prov:Activity ; + prov:used speckl:ToggleSwitch_spec ; + prov:wasAssociatedWith speckl:SpecklCompiler_v0.3.1 ; + prov:endedAtTime "2026-05-06T19:04:01Z"^^xsd:dateTime . + +speckl:ToggleSwitch_typescript + a prov:Entity ; + prov:wasGeneratedBy speckl:ToggleSwitch_compilation ; + prov:wasDerivedFrom speckl:ToggleSwitch_spec . +``` + +This is the audit trail. Any downstream artifact — TypeScript, WASM, CycloneDX SBOM, SPDX file — gets its own `prov:Entity` with a `wasDerivedFrom` link to the spec. If you're ever asked "was this TypeScript file generated from the v0.1.0 spec by compiler v0.3.1 on May 6, 2026?", the answer is right there, machine-verifiable. + +## The NIST SA-11 Connection + +This is where provenance becomes a concrete selling point. NIST SP 800-53 Revision 5 introduced control **SA-11: Developer Testing and Evaluation**, which requires organizations to produce "evidence of correct implementation" for automated code generation tools. Speckl's five-artifact output directly maps to SA-11's sub-controls: + +- **SA-11(2) — Threat Modeling and Vulnerability Analysis:** The CycloneDX SBOM feeds directly into vulnerability scanners. You know exactly what dependencies are in your generated code. +- **SA-11(4) — Manual Code Review:** The PROV-O audit trail establishes chain of custody. A reviewer can verify when the spec was written, what compiler version was used, and what artifacts were produced. +- **SA-11(6) — Attack Surface Review:** SPDX license metadata surfaces every licensed component, enabling supply chain risk assessment. +- **SA-11(7) — Verify Scope of Testing:** All five artifacts share a single compilation event, verifiable via PROV-O. You know the tests ran against the same code that shipped. + +This is the "judgement transport" value proposition of Speckl: it doesn't just produce code — it produces **evidence**. Evidence that a human reviewer can use to make informed judgements about what was generated, why it was generated, and whether it's safe to deploy. + +## Five Design Decisions That Make This Work + +**1. Zero-configuration.** There's no `--provenance` flag. No `provenance: true` in a config file. Every `speckl compile` produces all five artifacts, every time. Provenance isn't optional — it's the default. + +**2. Guaranteed consistency.** Because all artifacts come from the same AST traversal inside a single compiler pass, there's no possibility of the SBOM describing one version of the code and the TypeScript file containing another. They're generated atomically. + +**3. Deterministic and reproducible.** Given the same spec, the same compiler version, and the same timestamp, the provenance output is bit-for-bit identical. This matters for audits: if you're claiming a specific SBOM covers a specific deployment, the system can verify it. + +**4. Open standards throughout.** PROV-O, CycloneDX, and SPDX are all community-maintained standards with wide tooling support. There's no lock-in, no proprietary format to decode, no vendor dependency to maintain. + +**5. Regulatory readiness as a byproduct.** Organizations don't have to build a separate compliance pipeline. They write their spec, they run the compiler, and they get evidence as a side effect of their normal workflow. Compliance becomes a property of the system, not a separate process. + +## What This Means for Development Teams + +The shift from "provenance as afterthought" to "provenance as compiler pass" changes how teams think about their build pipeline: + +**For developers:** You don't have to configure SBOM generators, set up attestation signing, or maintain a separate compliance toolchain. Write your spec, compile, commit all five artifacts. The evidence is just there. + +**For security teams:** Every build produces a CycloneDX SBOM you can scan immediately. No blind spots, no "we added the SBOM step to 80% of our pipelines" situations. It's 100% coverage by construction. + +**For auditors:** The PROV-O graph gives you a verifiable chain of custody from spec to deployment. You can ask "was this running artifact produced from an approved specification?" and get a machine-verifiable answer. + +**For leadership:** Compliance evidence is no longer a separate project with a separate budget and timeline. It's a property of the development workflow. Ship a feature, ship its provenance alongside it. + +## The Blog Post Series + +This is post 4 of 5 in our series covering the Speckl compiler and methodology: + +1. **Spec-Code Drift Is a Provenance Problem** — why the gap between what you specify and what runs matters +2. **Designing SpeckDL: What a Spec Language Needs in 2026** — the language itself +3. **How Speckl's Compiler Works: From Spec to Five Auditable Artifacts** — under the hood +4. **[This post] Embedded Provenance** — the evidence layer +5. **Consensus Protocols in SpeckDL** — Two-Phase Commit, Paxos, Raft (coming next) + +## What's Next + +The final post in this series will cover the hardest programs to get right: consensus protocols. We'll walk through Two-Phase Commit, Paxos, and Raft — all specified in SpeckDL, all compiling to the full five-artifact output, with WASM that validates through the WebAssembly reference interpreter. + +In the meantime: the Speckl compiler is open source at [os.scoble.me/forgejo/sscoble/speckl](https://os.scoble.me/forgejo/sscoble/speckl) (MIT licensed). The v2 whitepaper is in the repo under `docs/whitepaper-v2.md`. And if you want to see provenance in action, clone the repo and run `speckl compile examples/ToggleSwitch.speck` — you'll have all five artifacts in under two seconds. + +--- + +*Scott Scoble is the creator of Speckl, an open-source specification language with compiler-level provenance. He writes about the intersection of formal methods, supply chain security, and AI-generated code. Follow along at [speckl.scoble.me](https://speckl.scoble.me).* diff --git a/docs/blog-05-consensus-protocols.md b/docs/blog-05-consensus-protocols.md new file mode 100644 index 0000000..f670c9e --- /dev/null +++ b/docs/blog-05-consensus-protocols.md @@ -0,0 +1,211 @@ +# Consensus Protocols in SpeckDL: From TLA+ to Runnable Code + +**By Scott Scoble** · Published on [dev.to/Medium/HN] · Series: [Building Speckl](https://speckl.scoble.me) · Post 5 of 5 + +--- + +Consensus protocols are the hardest programs to get right. Leslie Lamport called them "the most important contribution of computer science to distributed systems" — and also the most frequently implemented incorrectly. + +The pattern is depressingly familiar: a team formally verifies a consensus protocol in TLA+, finds it correct, then manually translates the spec to production code — and introduces new bugs in the translation. The spec is verified. The code is not. This is the spec-code gap at its most dangerous. + +Speckl was designed to close that gap. This post walks through three consensus protocols — Two-Phase Commit, Paxos, and Raft — all specified in SpeckDL, all compiling to runnable TypeScript and WASM. No manual translation. No gap. + +## Why Consensus Protocols? + +Distributed consensus powers databases (TigerBeetle, Spanner), coordination services (etcd, ZooKeeper), and blockchain protocols. Getting it wrong means data loss, split-brain, or silent corruption. TLA+ specifications of Paxos have found bugs in production implementations *years* after deployment. + +Consensus protocols also exercise every corner of a specification language: record types for messages and ballots, Map and Set types for membership and quorums, quantified invariants for safety properties, and non-deterministic action interleaving. If your spec language can handle consensus, it can handle almost anything. + +## Two-Phase Commit + +Two-Phase Commit (2PC) is the simplest consensus protocol: a transaction manager coordinates multiple resource managers to agree on whether a transaction commits or aborts. It's a classic TLA+ example from Lamport himself. + +**The SpeckDL spec** — 45 lines, 5 actions: + +``` +spec TwoPhaseCommit + +state { + rm_state: Map(String, String) // working | prepared | committed | aborted + tm_state: String // init | committed | aborted + msgs: Set(String) // "Prepared" | "Commit" | "Abort" +} + +init { + tm_state = "init" + forall rm in rms: + rm_state[rm] = "working" +} + +action RMPrepare { + require rm_state[self] == "working" + rm_state[self] = "prepared" + msgs += "Prepared" +} + +action TMCommit { + require tm_state == "init" + require forall rm in rms: rm_state[rm] == "prepared" + tm_state = "committed" + msgs += "Commit" +} + +action TMAbort { + require tm_state == "init" + require exists rm in rms: rm_state[rm] != "prepared" + tm_state = "aborted" + msgs += "Abort" +} +``` + +The key safety invariant — `TPConsistency` — ensures that committed RMs imply a committed TM, and that no RM is committed when TM is aborted. `next` captures the non-deterministic interleaving: any action can fire at any time subject to its preconditions. + +## Paxos + +Paxos is the canonical consensus protocol — and famously difficult to implement correctly. Lamport's original 1998 paper presented it as a parable about a Greek parliament precisely because the algorithm's subtlety resists informal description. + +**The SpeckDL spec** — 120 lines, 6 actions — captures the full two-phase protocol: + +- **Phase 1a (Prepare):** A proposer sends a Prepare message with a unique ballot number. Ballot numbers provide total ordering. +- **Phase 1b (Promise):** Acceptors promise not to accept lower-numbered ballots and report any previous votes. +- **Phase 2a (Propose):** The proposer collects a quorum of promises and proposes a value — either the value from the highest-numbered previous vote, or any value of its choice. +- **Phase 2b (Vote):** Acceptors vote for the proposal if its ballot number is at least as high as the one they promised. +- **Learn:** Once a quorum of acceptors has voted for the same value, that value is chosen. + +SpeckDL's record types make the data model clear: + +``` +type Ballot = { number: Nat, value: String | null, proposer: String | null } +type AcceptorSlot = { acceptor: String, slot: Nat } + +state { + ballots: Map(Ballot, String) + promises: Map(AcceptorSlot, Ballot) + votes: Map(AcceptorSlot, Map(Ballot, String)) + chosen: Map(Ballot, String) +} +``` + +The three safety invariants capture exactly what makes Paxos correct: + +- **BallotUniqueness:** No two ballots share the same number. +- **SingleValuePerBallot:** At most one value per ballot number. +- **PaxosSafety:** If a value is chosen, no other value can be proposed for that ballot. + +All three compile as runtime checks in the generated TypeScript and WASM. They're not just documentation — they're executable verification. + +## Raft + +Raft is the consensus protocol that powers etcd, Consul, and dozens of production systems. Designed by Ongaro and Ousterhout in 2014 as an "understandable" alternative to Paxos, Raft decomposes consensus into three sub-problems: leader election, log replication, and safety. + +**The SpeckDL spec** — 230 lines, 8 actions, 3 safety invariants — is the largest in our example suite: + +| Action | Purpose | +|--------|---------| +| `StartElection` | Follower times out → becomes candidate | +| `GrantVote` | Follower votes for candidate with sufficiently complete log | +| `BecomeLeader` | Candidate with majority votes becomes leader | +| `AppendEntries` | Leader replicates log, checks consistency, truncates conflicts | +| `ApplyEntries` | Each server applies committed entries to state machine | +| `AdvanceCommit` | Leader increments commitIndex on majority replication | + +### Record types that mirror the paper + +Raft exercises SpeckDL's nested record types extensively: + +``` +type LogEntry = { term: Term, index: LogIndex, command: String } +type ServerState = { + role: String, + currentTerm: Term, + votedFor: String | null, + log: List(LogEntry), + commitIndex: LogIndex, + lastApplied: LogIndex +} +type AppendEntriesReq = { + term: Term, leaderId: ServerId, + prevLogIndex: LogIndex, prevLogTerm: Term, + entries: List(LogEntry), leaderCommit: LogIndex +} +``` + +These are the same structures from the Raft paper — expressed directly in SpeckDL, with nullable fields for optional values and `List` types for ordered sequences. + +### Three safety invariants + +**SingleLeaderPerTerm:** At most one leader per term. For any two servers both claiming to be leader, their terms must differ. This prevents split-brain. + +**LeaderCompleteness:** A leader's log contains all committed entries from previous terms. This ensures new leaders don't lose data. + +**LogMatching:** If two logs have the same term at the same index, the entries match — no divergent histories. + +These invariants are expressed as quantified expressions in SpeckDL and emitted as runtime assertions in the generated code: + +``` +invariant SingleLeaderPerTerm { + forall s1 in servers: + forall s2 in servers: + serverState[s1].role == "leader" and + serverState[s2].role == "leader" and + s1 != s2 implies + serverState[s1].currentTerm != serverState[s2].currentTerm +} +``` + +## TLA+ and SpeckDL: Side by Side + +All three consensus examples exist as well-known TLA+ specifications. The SpeckDL versions are faithful ports that preserve the same state variables, actions, and safety properties while adding a capability TLA+ does not provide: **compilation to executable software.** + +| Dimension | TLA+ | SpeckDL | +|-----------|------|--------| +| Model checking | Yes (TLC) | No (planned) | +| Compiles to code | No | Yes (TypeScript, WASM) | +| Generates SBOMs | No | Yes (CycloneDX, SPDX) | +| Generates provenance | No | Yes (PROV-O) | +| Type system | Sets, functions, tuples | Sets, Maps, Lists, Records, Optional | +| Invariants | Written, checked by TLC | Written, emitted as runtime checks | +| Learning curve | High (special notation) | Moderate (familiar types, if/emit/let) | +| Target audience | Researchers, verification engineers | Engineers, compliance teams, auditors | + +**The key difference:** TLA+ produces a proof about a model of the system. SpeckDL produces the system itself. Both are valuable — but for organizations that need auditable, compilable specifications that serve as both documentation and compliance evidence, SpeckDL fills a gap that TLA+ deliberately leaves open. + +## From Consensus to Production + +The progression — Two-Phase Commit (learning), Paxos (theoretical bedrock), Raft (production consensus) — forms a complete curriculum. But there's a fourth entry that makes this real: + +**TigerBeetle.** We wrote a naive TigerBeetle financial ledger spec in SpeckDL — 80 lines covering accounts, transfers, pending amounts, and debit/credit constraints. It parses, it compiles, and the WASM validates. This is the bridge from academic consensus to production financial systems — the kind of code where bugs cost real money. + +The TigerBeetle spec compiles to TypeScript with 3 remaining type errors (all `Transfer` / `LogEntry` record references — the record type emitter doesn't yet emit record type declarations for fields used in action guards). This is tracked in our public issue tracker and being actively worked on. + +## Where We Are + +The compiler is open source ([os.scoble.me/forgejo/sscoble/speckl](https://os.scoble.me/forgejo/sscoble/speckl), MIT), at v0.3.1 with: +- 41 tests passing (100%) +- 11 example specs (5 producing zero TypeScript errors) +- All three consensus protocols compiling to the full 5-artifact output +- CLI, live demo, and comprehensive documentation + +The gap between TLA+ verification and production code has existed for decades. Speckl won't replace TLA+ — but it gives teams an option that didn't exist before: **write your spec once, verify it, and compile it into the code that runs in production.** + +## The Blog Post Series + +This concludes our 5-part series on the Speckl compiler and methodology: + +1. **Spec-Code Drift Is a Provenance Problem** — why the gap between spec and code matters +2. **Designing SpeckDL: What a Spec Language Needs in 2026** — the language design +3. **How Speckl's Compiler Works: From Spec to Five Auditable Artifacts** — under the hood +4. **Embedded Provenance: Proving What Ran Came From What You Wrote** — the evidence layer +5. **[This post] Consensus Protocols in SpeckDL** — Two-Phase Commit, Paxos, Raft + +## What's Next + +The Phase 2 content pipeline is complete. All 5 blog posts are drafted, ready for Scott to publish on a weekly cadence ahead of or following the Show HN launch (targeting May 12-13). + +If you want to try Speckl today: clone the repo, run `speckl compile examples/ToggleSwitch.speck`, and you'll have five artifacts in under two seconds. The consensus examples are at `examples/TwoPhaseCommit.speck`, `examples/Paxos.speck`, `examples/Raft.speck`, and `examples/TigerBeetle.speck`. + +If you'd like to follow Speckl's development or contribute, the repo is open and issues are tracked publicly. I'd love to hear what you're building with it — or what you wish it could do. + +--- + +*Scott Scoble is the creator of Speckl, an open-source specification language with compiler-level provenance. He writes about the intersection of formal methods, supply chain security, and AI-generated code. Follow along at [speckl.scoble.me](https://speckl.scoble.me).* diff --git a/docs/blog-07-prove-sqlite.md b/docs/blog-07-prove-sqlite.md new file mode 100644 index 0000000..9e6ad34 --- /dev/null +++ b/docs/blog-07-prove-sqlite.md @@ -0,0 +1,103 @@ +# We're Going to Formally Verify SQLite + +SQLite runs on every smartphone, every browser, every embedded system. Its core state machines have never been formally verified. We're going to fix that. + +This is the follow-up to our Kafka proof-of-consensus project; same methodology, bigger target. SQLite is not a research toy; it is the most deployed database in the history of software. If we can prove its invariants, we prove something that matters. + +--- + +## Why SQLite + +SQLite is not a theoretical construct. It ships inside: + +- Every iPhone and Android device (iOS, Android system libraries) +- Every major browser (Chrome, Firefox, Safari, Edge) +- Countless embedded systems, IoT devices, and automotive platforms +- macOS, Windows, and most Linux distributions by default + +The SQLite Consortium estimates **one trillion** active SQLite databases. Not one billion; one trillion. The test suite (TH3) achieves 100% branch coverage and is widely regarded as the best-tested codebase on Earth. But 100% branch coverage is not proof. It is exhaustive testing, and exhaustive testing is not the same as a mathematical guarantee. + +SQLite's internals are exceptionally well-documented. D. Richard Hipp publishes detailed architecture documentation, and the codebase is commented with the rigor of a textbook. The state machines are explicit: the pager has discrete modes, the WAL protocol has a defined lifecycle, and the VDBE bytecode engine operates on a register model with well-defined instruction semantics. + +This is not a black box. It is a glass box with a published manual. That makes it an ideal target for formal specification. + +## What We'll Verify (Phase 1) + +We are not attempting to verify all of SQLite. The full engine is hundreds of thousands of lines of C. Our Phase 1 scope targets three core subsystems whose state machines are both critical and amenable to SpeckDL specification: + +### 1. The Pager State Machine + +The pager manages the transaction lifecycle: reading pages from disk, caching them, tracking dirtiness, and orchestrating writes. It operates in multiple journal modes (DELETE, TRUNCATE, PERSIST, WAL, MEMORY, OFF) and transitions through states that must respect strict invariants. + +The pager state machine is where SQLite's durability guarantee lives. When a transaction commits, the pager decides whether the commit is durable and when the journal can be discarded. In WAL mode, the pager coordinates between a single writer and multiple readers via the WAL index and checkpoint logic. An error in the pager does not produce a crash; it produces silent data loss or inconsistency that may not surface for months. + +Key properties to verify: +- A dirty page is never evicted without being written +- WAL mode maintains read consistency across concurrent readers and a single writer +- Journal mode transitions are atomic with respect to crash recovery +- The pager never enters an inconsistent state after an interrupted commit +- Checkpoint operations preserve the "all committed data is in the database file" invariant + +### 2. The VDBE Bytecode Engine + +The Virtual Database Engine executes compiled SQL bytecode on a register machine. Each instruction manipulates registers, cursors, and the B-Tree layer. The instruction set is finite and well-documented; the register model is a stack-plus-registers hybrid. + +The VDBE is not the highest-risk component; its instructions are deterministic and its state is local. But it is the component every SQL statement passes through. A single off-by-one in a register reference or a type-tag mismatch in an arithmetic operation can produce incorrect query results that look like correct ones. Verifying the VDBE means verifying that the bytecode interpreter cannot enter an undefined state regardless of the input program. + +Key properties to verify: +- Register references are always within bounds +- Type tags are consistent with the operations performed on them +- Cursor operations maintain valid iterator state +- The program counter advances monotonically except for controlled jumps +- The stack depth never underflows or exceeds its declared maximum + +### 3. B-Tree Invariants + +The B-Tree layer is SQLite's storage backbone: tables and indexes are both B-Trees. Page splits, pointer integrity, and balance operations are the kind of mechanical code that formal verification excels at. + +The B-Tree code in SQLite is dense. A single page split involves allocating a new page, redistributing cells between two pages, updating parent pointers, and handling the edge case where the root itself splits. Each step has preconditions; violate one and the tree becomes structurally inconsistent. Corruption at this layer does not crash; it causes queries to return wrong results or skip rows entirely. + +Key properties to verify: +- After any page split, all keys remain reachable via tree traversal +- Sibling pointers maintain total order +- Balance operations preserve the B-Tree height bound +- Cell offsets remain within the page boundary after any insertion or deletion +- Free space within a page is always contiguous and correctly sized + +## Method + +Same as Prove Kafka: SpeckDL specification, Z3 bounded model checking, publish findings. + +The framework is already built. The compiler produces a typed intermediate representation; the Z3 backend translates invariants and verify blocks into SMT-LIB assertions. We proved Kafka's consensus invariants this way; we will prove SQLite's operational invariants the same way. + +The workflow: + +1. **Specify** the state machine in SpeckDL (pager modes, VDBE registers, B-Tree pages) +2. **Compile** to Z3 SMT-LIB with `speckl verify --target=z3 --verify-depth=N` +3. **Run** bounded model checking against the invariants +4. **Document** the results: what held, what didn't, what depth we reached +5. **Publish** the specification and findings as open-source artifacts + +The SpeckDL specifications will be open source. The Z3 output will be reproducible. If we find an invariant violation, you'll read about it here first. + +## Why This Matters + +There is no shortage of testing in SQLite. TH3 is the gold standard. But testing and proof are different categories of assurance. + +Testing says: "We tried a lot of cases and didn't find a bug." Proof says: "No case exists that violates this property." The first is empirical confidence; the second is logical certainty. Both matter. SQLite has the first in abundance. It has never had the second. + +Formal verification of SQLite's core invariants would be a first. Not because SQLite is buggy; the evidence suggests it is exceptionally solid. But because proof is the only assurance that does not degrade over time. As the codebase evolves, as new journal modes are added, as platforms change, a verified invariant remains verified. A test case does not. + +We proved Kafka's consensus. Now we're proving the world's most deployed database. + +## Follow Along + +The specifications will live in the Speckl repository at `examples/SQLitePager.speck`, `examples/SQLiteVDBE.speck`, and `examples/SQLiteBTree.speck`. Verification results will be published as we complete each subsystem. + +If you want to follow the work or contribute, the repo is open and the issues are tracked publicly. When we find something, you'll read it here first. + +--- + +*This post is part of the Speckl proof catalog: formally verifying the software the world depends on. Read the [Kafka proof](/blog/prove-kafka) or explore the [Speckl compiler](https://os.scoble.me/forgejo/sscoble/speckl).* + +*Speckl is MIT-licensed and open source. The specifications, the compiler, and the findings are all public.* diff --git a/docs/compilation-layer.md b/docs/compilation-layer.md new file mode 100644 index 0000000..f409dcd --- /dev/null +++ b/docs/compilation-layer.md @@ -0,0 +1,1317 @@ +# Speckl Compilation Backend Architecture + +## Overview + +The Compilation Layer is Speckl's superpower: it takes a validated SIR (Speckl Intermediate Representation) and produces **deterministic, verifiable artifacts** across multiple target formats. The pipeline: + +``` +Chaotic Conversation → [Extraction Layer] → [Validation Layer] → [Compilation Layer] → Artifacts +``` + +While the conversation is chaotic, the output is **guaranteed deterministic**. Same SIR + same compiler version = same artifact bytes, every time. + +--- + +## 1. Compilation Model + +### Pure Functional Core + +``` +compile: SIR × TargetConfig × CompilerVersion → CompilationResult +``` + +No side effects. No network calls. No file system dependency. No randomness. The entire compilation is a pure function — this is what makes determinism possible. + +```typescript +interface CompilationResult { + // The compiled artifact + artifact: Artifact; + + // Cryptographic proof of what was compiled + manifest: CompilationManifest; + + // What the compiler couldn't compile + gaps: CompilationGap[]; + + // Readiness assessment + readiness: ReadinessLevel; +} + +interface Artifact { + format: TargetFormat; // "cyclonedx", "prov-o", "spdx", "wasm" + mimeType: string; // "application/json", "application/wasm" + content: Buffer | string; // The actual artifact bytes + contentHash: string; // sha256 of content + size: number; // byte count +} +``` + +### The Compilation Manifest + +Every compilation produces a manifest — a cryptographic record of provenance: + +```typescript +interface CompilationManifest { + manifestVersion: "1.0.0"; + + // What went in + inputs: { + sirHash: string; // sha256 of the SIR + sirVersion: number; // SIR version counter + sirId: string; // UUID of the SIR + }; + + // What compiled it + compiler: { + name: "speckl-compiler"; + version: string; // "2.3.1" + targets: string[]; // ["cyclonedx@1.5", "spdx@3.0"] + hash: string; // sha256 of compiler binary/config + }; + + // What came out + outputs: Record; + + // Integrity chain + manifestHash: string; // sha256 of this manifest (self-referential, computed last) +} +``` + +### Hash-Locking for Determinism + +Determinism is guaranteed through content-addressing: + +1. **SIR is hash-addressed**: Every SIR gets a `sha256` hash on validation +2. **Compiler is version-pinned**: Target compilers declare their exact version in the SIR +3. **No ambient state**: Compilation reads nothing from the environment — no `Date.now()`, no `Math.random()`, no OS locale +4. **Result verification**: The manifest's output hash must match `sha256(artifact.content)` — mismatch = compilation error + +```typescript +async function verifyReproducibility( + sir: SIR, + target: TargetConfig, + previousManifest: CompilationManifest +): Promise { + const result = await compile(sir, target); + const newHash = result.manifest.outputs[target.format].hash; + const oldHash = previousManifest.outputs[target.format].hash; + + if (newHash !== oldHash) { + return { + reproducible: false, + expected: oldHash, + actual: newHash, + // The diff between SIR versions may explain this + sirChange: diffSir(previousManifest.inputs.sirHash, sir.hash), + }; + } + + return { reproducible: true }; +} +``` + +### The Dual BOM Pattern (Existing Speckl Convention) + +Speckl already compiles state machines to dual BOMs: CycloneDX (SBOM) + PROV-O (provenance). The compilation layer generalizes this: + +``` +StateMachine SIR → compileToCycloneDX → CycloneDX 1.5 JSON +StateMachine SIR → compileToPROV-O → PROV-O RDF/Turtle +StateMachine SIR → compileToWASM → WASM binary + ↑ + One SIR, three targets, single compilation run +``` + +--- + +## 2. Target Abstraction + +### What Is a Compilation Target? + +A compilation target consumes an SIR and produces an artifact in a specific format. It's an abstract interface that every target implements: + +```typescript +interface CompilerTarget { + // Identity + readonly id: TargetId; // "cyclonedx", "openapi", "terraform" + readonly version: string; // "1.5.0" + readonly format: TargetFormat; // "cyclonedx" + readonly mimeType: string; // "application/json" + + // What domains of SIR can this target handle? + readonly supportedDomains: SIRDomain[]; + + // What does this target REQUIRE from the SIR? + readonly requirements: TargetRequirements; + + // Compilation entry point + compile(sir: SIR, config: TargetConfig, context: CompilationContext): CompileOutput; + + // Does this SIR have enough info for this target? + assessReadiness(sir: SIR): ReadinessAssessment; + + // Generate stubs for missing information + generateStubs(gaps: CompilationGap[], sir: SIR): StubResult; +} + +interface TargetRequirements { + // Required SIR sections + requiredSections: string[]; // ["entities", "transitions", "states"] + // Required entity types + requiredEntityTypes: string[]; // ["software-component", "process"] + // Required relationship types + requiredRelationshipTypes: string[]; + // Minimum completeness threshold (0-1) + minCompletenessScore: number; // e.g., 0.8 = 80% of required fields present +} +``` + +### Example Target Implementations + +**State Machine Targets** (current): +- `cyclonedx` — SBOM from state transitions +- `prov-o` — Provenance ontology from state history +- `spdx` — License compliance from component states +- `wasm` — Executable state machine runtime + +**Future Targets**: +- `openapi` — API specification from interface SIR +- `json-schema` — Data validation schemas +- `cloudformation` / `terraform` — Infrastructure as code from deployment SIR +- `kubernetes-crd` — Custom resource definitions +- `solana-program` / `evm-contract` — Smart contracts from protocol SIR +- `github-actions` — CI/CD pipelines from workflow SIR +- `opa-rego` — Policy rules from constraint SIR +- `protobuf` — Protocol buffer definitions from data SIR + +### Target Registration + +Targets register with the compiler via a plugin system: + +```typescript +class CompilerRegistry { + private targets: Map = new Map(); + + register(target: CompilerTarget): void { + const key = `${target.id}@${target.version}`; + if (this.targets.has(key)) { + throw new Error(`Target ${key} already registered`); + } + this.targets.set(key, target); + } + + getTarget(id: TargetId, version?: string): CompilerTarget { + if (version) { + return this.targets.get(`${id}@${version}`) + ?? this.throwNotFound(id, version); + } + // Return latest version + const versions = Array.from(this.targets.keys()) + .filter(k => k.startsWith(`${id}@`)) + .sort(semverDesc); + return this.targets.get(versions[0]) + ?? this.throwNotFound(id); + } + + listTargetsForDomain(domain: SIRDomain): CompilerTarget[] { + return Array.from(this.targets.values()) + .filter(t => t.supportedDomains.includes(domain)); + } +} +``` + +### Target-Specific Configuration + +```typescript +interface TargetConfig { + format: TargetFormat; + version?: string; // Pin to specific target version + options?: Record; // Target-specific options + dialect?: string; // "cyclonedx:1.5" vs "cyclonedx:1.4" +} + +// Example: CycloneDX with custom options +const cyclonedxConfig: TargetConfig = { + format: "cyclonedx", + version: "1.5.0", + options: { + serialNumber: "urn:uuid:3e56b1dc-...", // deterministic UUID + includeVulnerabilities: true, + componentTypes: ["application", "library", "container"], + }, + dialect: "cyclonedx:1.5", +}; +``` + +--- + +## 3. Domain-to-Target Mapping + +### Domain Capabilities + +Not every SIR can compile to every target. The SIR has a **domain** that determines its capabilities: + +```typescript +enum SIRDomain { + StateMachine = "state-machine", + DataSchema = "data-schema", + APIInterface = "api-interface", + Deployment = "deployment", + Workflow = "workflow", + Protocol = "protocol", + Constraint = "constraint", + ServiceTopology = "service-topology", + LicenseCompliance = "license-compliance", +} +``` + +Each domain has a set of **capabilities** — the types of information it can encode: + +| Domain | Encodes | Capabilities | +|---|---|---| +| `state-machine` | States, transitions, guards, actions | Entity lifecycle, event sequences, invariants | +| `data-schema` | Types, fields, constraints, relations | Data shape, validation rules, serialization | +| `api-interface` | Endpoints, methods, params, responses | HTTP contracts, auth schemes, error models | +| `deployment` | Resources, dependencies, configs | Infrastructure topology, scaling, networking | +| `workflow` | Steps, triggers, conditions, artifacts | CI/CD graphs, automation pipelines | +| `protocol` | Messages, participants, invariants | Communication contracts, consensus rules | +| `constraint` | Rules, subjects, effects | Policy definitions, access control | +| `service-topology` | Services, ports, dependencies | System architecture, fault domains | +| `license-compliance` | Components, licenses, obligations | SBOM data, license compatibility | + +### Target Requirements + +Each target declares what it needs from an SIR to produce a valid artifact: + +```typescript +// A CycloneDX target needs entity information +const cyclonedxRequirements: TargetRequirements = { + requiredSections: ["entities", "relationships"], + requiredEntityTypes: ["software-component"], + minCompletenessScore: 0.6, // Can produce partial BOM from 60% complete SIR +}; + +// A Terraform target needs resource definitions +const terraformRequirements: TargetRequirements = { + requiredSections: ["entities", "dependencies", "configurations"], + requiredEntityTypes: ["compute-resource", "network-resource"], + requiredRelationshipTypes: ["depends-on", "exposes"], + minCompletenessScore: 0.8, // Need 80% for valid Terraform +}; + +// An OPA/Rego target needs rules +const opaRegoRequirements: TargetRequirements = { + requiredSections: ["rules", "subjects", "effects"], + requiredEntityTypes: ["policy-subject", "policy-action"], + minCompletenessScore: 0.7, +}; +``` + +### Compatibility Matrix + +``` +Domain → Target Mapping: +╔══════════════════════╦═════════╦══════════╦═══════════╦════════╦═════════════╗ +║ Target ║ SM ║ Schema ║ API ║ Deploy ║ Workflow ║ +╠══════════════════════╬═════════╬══════════╬═══════════╬════════╬═════════════╣ +║ cyclonedx ║ ✓ ║ ✗ ║ ✗ ║ ✗ ║ ✗ ║ +║ prov-o ║ ✓ ║ ✗ ║ ✗ ║ ✗ ║ ✗ ║ +║ spdx ║ ✓ ║ ✗ ║ ✗ ║ ✗ ║ ✗ ║ +║ wasm ║ ✓ ║ ✗ ║ ✗ ║ ✗ ║ ✗ ║ +║ openapi ║ ✗ ║ ✗ ║ ✓ ║ ✗ ║ ✗ ║ +║ json-schema ║ ✗ ║ ✓ ║ ✓ ║ ✗ ║ ✗ ║ +║ cloudformation ║ ✗ ║ ✗ ║ ✗ ║ ✓ ║ ✗ ║ +║ terraform ║ ✗ ║ ✗ ║ ✗ ║ ✓ ║ ✗ ║ +║ kubernetes-crd ║ ✗ ║ ✗ ║ ✗ ║ ✓ ║ ✗ ║ +║ evm-contract ║ ✓ ║ ✗ ║ ✗ ║ ✗ ║ ✗ ║ +║ github-actions ║ ✗ ║ ✗ ║ ✗ ║ ✗ ║ ✓ ║ +║ opa-rego ║ ✗ ║ ✗ ║ ✗ ║ ✗ ║ ✗ ║ +║ protobuf ║ ✗ ║ ✓ ║ ✓ ║ ✗ ║ ✗ ║ +╚══════════════════════╩═════════╩══════════╩═══════════╩════════╩═════════════╝ +``` + +The compiler validates compatibility before starting: + +```typescript +function validateTargetCompatibility(sir: SIR, target: CompilerTarget): CompatibilityResult { + if (!target.supportedDomains.includes(sir.domain)) { + return { + compatible: false, + reason: `Target ${target.id} does not support SIR domain "${sir.domain}"`, + }; + } + + const missingRequirements = target.requirements.requiredSections + .filter(section => !sir.sections.has(section)); + + if (missingRequirements.length > 0) { + return { + compatible: false, + reason: `SIR missing required sections: ${missingRequirements.join(", ")}`, + }; + } + + // Check completeness threshold + const score = assessCompleteness(sir, target); + if (score < target.requirements.minCompletenessScore) { + return { + compatible: false, + reason: `SIR completeness (${score}) below target minimum (${target.requirements.minCompletenessScore})`, + }; + } + + return { compatible: true }; +} +``` + +--- + +## 4. Incremental Compilation + +### Change Detection + +When a conversation evolves, the SIR gets a new version. We need to know **exactly what changed** to avoid recompiling everything: + +```typescript +interface SIRDiff { + fromVersion: number; + toVersion: number; + fromHash: string; + toHash: string; + + // What changed at each layer + sections: { + added: string[]; + removed: string[]; + modified: string[]; + }; + entities: { + added: EntityId[]; + removed: EntityId[]; + modified: EntityId[]; // Entity existed before, fields changed + }; + transitions: { + added: TransitionId[]; + removed: TransitionId[]; + modified: TransitionId[]; + }; + // Affected targets: which targets need recompilation? + affectedTargets: TargetId[]; +} + +function diffSir(v1: SIR, v2: SIR): SIRDiff { + // Content-addressed comparison + const sectionDiff = { + added: [...v2.sections.keys()].filter(k => !v1.sections.has(k)), + removed: [...v1.sections.keys()].filter(k => !v2.sections.has(k)), + modified: [...v2.sections.keys()].filter(k => + v1.sections.has(k) && + hashSection(v1.sections.get(k)) !== hashSection(v2.sections.get(k)) + ), + }; + + // Entities: O(n) comparison with hash maps + const v1EntityHashes = new Map( + v1.entities.map(e => [e.id, hashEntity(e)]) + ); + const v2EntityHashes = new Map( + v2.entities.map(e => [e.id, hashEntity(e)]) + ); + + const entityDiff = { + added: [...v2EntityHashes.keys()].filter(id => !v1EntityHashes.has(id)), + removed: [...v1EntityHashes.keys()].filter(id => !v2EntityHashes.has(id)), + modified: [...v2EntityHashes.keys()].filter(id => + v1EntityHashes.has(id) && + v1EntityHashes.get(id) !== v2EntityHashes.get(id) + ), + }; + + // Determine which targets are affected + const affectedTargets = determineAffectedTargets(sectionDiff, entityDiff); + + return { + fromVersion: v1.version, + toVersion: v2.version, + fromHash: v1.hash, + toHash: v2.hash, + sections: sectionDiff, + entities: entityDiff, + transitions: computeTransitionDiff(v1, v2), + affectedTargets, + }; +} +``` + +### Partial Compilation + +Instead of recompiling everything, only recompile the changed portions: + +```typescript +async function incrementalCompile( + sirV2: SIR, + sirV1: SIR, + previousResults: Map, + targets: TargetConfig[] +): Promise> { + const diff = diffSir(sirV1, sirV2); + const results = new Map(previousResults); + + for (const target of targets) { + if (!diff.affectedTargets.includes(target.format)) { + // Target unaffected — reuse previous result + continue; + } + + if (isTargetAdditiveOnly(diff, target)) { + // Only new entities added — merge, don't rebuild + const merged = await mergeCompilation( + previousResults.get(target.format)!, + sirV2, + diff.entities.added, + target + ); + results.set(target.format, merged); + } else { + // Structural changes — full recompile for this target + const result = await compile(sirV2, target); + results.set(target.format, result); + } + } + + return results; +} +``` + +### Incremental Compilation Cache + +```typescript +interface CompilationCache { + // Content-addressed cache: hash(sir, target, version) → result + entries: Map; + + // Dependency tracking: which SIR entities feed which target output sections + dependencyMap: Map>; + + lookup(sir: SIR, target: TargetConfig): CompilationResult | null; + store(sir: SIR, target: TargetConfig, result: CompilationResult): void; + invalidate(sir: SIR, entities: EntityId[]): TargetId[]; +} +``` + +--- + +## 5. Artifact Versioning and Integrity + +### Content-Addressable Artifacts + +Every artifact is identified by its content hash, not a version number: + +``` +sir:abc123 → compile → artifact:def456 (sha256) +sir:abc123 → compile → artifact:def456 (same, always) +sir:abc124 → compile → artifact:789abc (different, because SIR changed) +``` + +### Reproducibility Guarantee + +The formal guarantee: + +> For any validated SIR `S` and compiler version `C`, `compile(S, C)` always produces exactly the same bytes. This holds across machines, time, and operating systems. + +Implementation contract: + +```typescript +// Compiler test suite includes reproducibility check +test("compilation is deterministic across runs", () => { + const sir = loadSir("test-fixtures/state-machine/order-processing.sir.json"); + const runs = Array.from({ length: 100 }, () => compile(sir, cyclonedxConfig)); + + const hashes = runs.map(r => r.manifest.outputs["cyclonedx"].hash); + const uniqueHashes = new Set(hashes); + + expect(uniqueHashes.size).toBe(1); // All 100 runs produce identical output +}); + +test("compilation is deterministic across platforms", async () => { + // Artifacts committed to repo, verified in CI on Linux, macOS, Windows + const sir = loadSir("test-fixtures/state-machine/payment-flow.sir.json"); + const { stdout } = await exec("speckl compile --target cyclonedx"); + + expect(stdout.trim()).toBe( + fs.readFileSync("test-fixtures/expected/payment-flow.cdx.json", "utf-8").trim() + ); +}); +``` + +### Manifest Verification + +```typescript +class ArtifactVerifier { + verify(manifest: CompilationManifest, artifact: Artifact): VerificationResult { + // 1. Hash check: does the artifact match its claimed hash? + const actualHash = sha256(artifact.content); + const claimedHash = manifest.outputs[artifact.format]?.hash; + + if (actualHash !== claimedHash) { + return { + valid: false, + error: `Hash mismatch: claimed ${claimedHash}, actual ${actualHash}`, + }; + } + + // 2. Manifest self-consistency + const computedManifestHash = computeManifestHash(manifest); + if (computedManifestHash !== manifest.manifestHash) { + return { + valid: false, + error: "Manifest hash mismatch — manifest has been tampered with", + }; + } + + // 3. Compiler version is known and valid + if (!isKnownCompilerVersion(manifest.compiler.version)) { + return { + valid: false, + error: `Unknown compiler version: ${manifest.compiler.version}`, + }; + } + + return { valid: true }; + } + + // Can we verify an artifact was compiled from SIR X? + verifyProvenance(manifest: CompilationManifest, sir: SIR): boolean { + return manifest.inputs.sirHash === sir.hash; + } +} +``` + +--- + +## 6. Compilation with Gaps + +### The Gap Problem + +Conversations are chaotic. The SIR will have gaps — missing fields, unspecified transitions, incomplete entity definitions. The compilation layer must handle this gracefully. + +```typescript +interface CompilationGap { + id: string; + severity: "error" | "warning" | "info"; + location: { + section: string; + entity?: EntityId; + transition?: TransitionId; + field?: string; + }; + description: string; + // What would fill this gap? + requiredInformation: string; + // What stub was generated in its place + stubGenerated: StubDescription; + // How this affects the compiled output + impact: GapImpact; +} + +interface GapImpact { + // The artifact section(s) affected + affectedArtifactSections: string[]; + // Is this a blocking gap? + blocksValidArtifact: boolean; + // Human-readable description of the impact + description: string; +} +``` + +### Stub Generation + +When information is missing, generate clear stubs with markers: + +```typescript +// Example: SIR has a state "processing" but no transition from "processing" to "shipped" +// The CycloneDX target generates: + +// In compiled CycloneDX JSON: +{ + "components": [ + { + "name": "order-service", + "type": "application", + "bom-ref": "entity:order-service" + } + ], + "dependencies": [ + { + "ref": "entity:order-service", + "dependsOn": [ + // ⚠️ STUB: Gap #GAP-42 + "__speckl_stub__gap_GAP-42_missing_transition" + ] + } + ], + "metadata": { + "tools": [ + { + "vendor": "Speckl", + "name": "Speckl Compiler", + "version": "2.3.1" + } + ], + // ⚠️ SPECIFICATION DEBT + "properties": [ + { + "name": "speckl:specification-debt", + "value": JSON.stringify([ + { + "gapId": "GAP-42", + "severity": "error", + "description": "Missing transition: processing → shipped", + "requiredInformation": "Define what triggers the transition from 'processing' to 'shipped' and any guard conditions", + "blocksValid: true + } + ]) + } + ] + } +} +``` + +### Graduated Compilation Levels + +Artifacts have a **readiness level** distinct from the SIR's completeness: + +```typescript +enum ReadinessLevel { + Draft = "draft", // < 40%: Conceptual, mostly stubs + Partial = "partial", // 40-60%: Key entities present, many stubs + Beta = "beta", // 60-80%: Usable for review, few stubs + Production = "production", // 80-95%: Valid artifact, minimal stubs + Complete = "complete", // 95%+: No gaps, fully valid +} + +interface ReadinessAssessment { + level: ReadinessLevel; + score: number; // 0.0 - 1.0 + gapCount: { + errors: number; + warnings: number; + info: number; + }; + // Can this artifact be used for its intended purpose? + usability: { + canReview: boolean; // Is it readable by humans? + canValidate: boolean; // Passes format-specific validation? + canDeploy: boolean; // Could be deployed to production? + canAudit: boolean; // Would pass a compliance audit? + }; +} +``` + +### Specification Debt Tracking + +"Specification debt" is tracked persistently in the compiled output: + +```json +{ + "speckl:specification-debt": { + "totalGaps": 7, + "errors": 3, + "warnings": 4, + "readiness": "beta", + "debtItems": [ + { + "id": "GAP-42", + "since": "2026-05-01T12:00:00Z", + "description": "Missing transition: processing → shipped", + "affects": ["cyclonedx:dependencies", "prov-o:wasGeneratedBy"], + "resolution": "Define the shipping trigger and guard conditions in the conversation" + } + ] + } +} +``` + +--- + +## 7. Compiler Plugin Architecture + +### The `CompilerTarget` Interface (Full) + +```typescript +// Each target is a self-contained module implementing this interface +interface CompilerTarget { + // === Identity === + readonly id: TargetId; + readonly version: string; + readonly format: TargetFormat; + readonly mimeType: string; + readonly displayName: string; + readonly description: string; + + // === Domain Compatibility === + readonly supportedDomains: SIRDomain[]; + readonly requirements: TargetRequirements; + + // === Primary Entry Points === + compile(sir: SIR, config: TargetConfig, context: CompilationContext): CompileOutput; + assessReadiness(sir: SIR): ReadinessAssessment; + generateStubs(gaps: CompilationGap[], sir: SIR): StubResult; + + // === Incremental Compilation === + canIncrementalMerge(diff: SIRDiff): boolean; + mergeIncremental(previous: CompileOutput, diff: SIRDiff, sir: SIR): CompileOutput; + + // === Validation === + validateArtifact(artifact: Artifact): ValidationResult; +} + +interface CompilationContext { + compilerVersion: string; + compilerHash: string; + sirHash: string; + timestamp: string; // For manifest only, NOT used in compilation + logger: CompilationLogger; // Structured logging, no side effects on artifact +} + +interface CompileOutput { + artifact: Artifact; + manifest: CompilationManifest; + gaps: CompilationGap[]; + readiness: ReadinessAssessment; + // Target-specific metadata for incremental merging + sections: Map; +} +``` + +### Target Registration and Discovery + +Targets are discovered from the filesystem or loaded dynamically: + +```typescript +// File: targets/cyclonedx/index.ts +import { CompilerTarget, CompileOutput, CompilationContext } from "@speckl/compiler"; + +export const cyclonedxTarget: CompilerTarget = { + id: "cyclonedx", + version: "1.5.0", + format: "cyclonedx", + mimeType: "application/json", + displayName: "CycloneDX", + description: "OWASP CycloneDX Software Bill of Materials", + supportedDomains: [SIRDomain.StateMachine, SIRDomain.LicenseCompliance], + requirements: { + requiredSections: ["entities", "relationships"], + requiredEntityTypes: ["software-component"], + minCompletenessScore: 0.6, + }, + + compile(sir: SIR, config: TargetConfig, context: CompilationContext): CompileOutput { + // 1. Extract components from SIR entities + const components = sir.entities + .filter(e => e.type === "software-component") + .map(entityToComponent); + + // 2. Extract dependencies from SIR relationships + const dependencies = sir.relationships + .filter(r => r.type === "depends-on") + .map(relationshipToDependency); + + // 3. Map state transitions to lifecycle phases + const lifecyclePhases = sir.transitions?.map(transitionToLifecyclePhase) ?? []; + + // 4. Build the CycloneDX document + const bom = buildCycloneDX15({ + bomFormat: "CycloneDX", + specVersion: "1.5", + serialNumber: config.options?.serialNumber as string ?? generateDeterministicUUID(sir.hash, "cyclonedx-serial"), + version: 1, + components, + dependencies, + metadata: { + tools: [{ + vendor: "Speckl", + name: "Speckl Compiler", + version: context.compilerVersion, + }], + properties: lifecyclePhases.map(phase => ({ + name: `speckl:lifecycle:${phase.state}`, + value: JSON.stringify(phase), + })), + }, + }); + + // 5. Identify gaps + const gaps = identifyGaps(sir, components, dependencies); + + // 6. Generate stubs for gaps + if (gaps.length > 0) { + this.injectStubs(bom, gaps); + } + + // 7. Compute artifact + const content = JSON.stringify(bom, null, 2); + const artifact: Artifact = { + format: "cyclonedx", + mimeType: "application/json", + content, + contentHash: sha256(content), + size: content.length, + }; + + // 8. Build manifest + const manifest = buildManifest(context, artifact, "cyclonedx"); + + // 9. Assess readiness + const readiness = assessReadinessFromGaps(gaps); + + return { + artifact, + manifest, + gaps, + readiness, + sections: trackSections(sir, components, dependencies), + }; + }, + + assessReadiness(sir: SIR): ReadinessAssessment { /* ... */ }, + generateStubs: generateCycloneDXStubs, + canIncrementalMerge(diff: SIRDiff): boolean { /* ... */ }, + mergeIncremental(prev, diff, sir) { /* ... */ }, + validateArtifact: validateCycloneDX, +}; + +// Register +compilerRegistry.register(cyclonedxTarget); +``` + +### Full CycloneDX Target Example + +```typescript +// Complete example: State Machine SIR → CycloneDX 1.5 JSON + +// INPUT SIR (simplified): +const orderProcessingSIR: SIR = { + id: "sir:order-processing-v3", + version: 3, + hash: "sha256:abc123...", + domain: SIRDomain.StateMachine, + entities: [ + { + id: "entity:order-api", + type: "software-component", + name: "Order API", + version: "2.1.0", + properties: { + language: "TypeScript", + framework: "Express", + license: "MIT", + }, + }, + { + id: "entity:payment-service", + type: "software-component", + name: "Payment Service", + version: "1.8.3", + properties: { + language: "Rust", + license: "Apache-2.0", + piiScope: "processes-credit-cards", // GAP: needs documentation + }, + }, + { + id: "entity:shipping-worker", + type: "software-component", + name: "Shipping Worker", + version: "0.9.0", + properties: { + language: "Go", + license: "MIT", + }, + }, + ], + relationships: [ + { + id: "rel:order-to-payment", + type: "depends-on", + source: "entity:order-api", + target: "entity:payment-service", + }, + { + id: "rel:order-to-shipping", + type: "depends-on", + source: "entity:order-api", + target: "entity:shipping-worker", + }, + ], + transitions: [ + { + id: "trans:created-to-paid", + from: "created", + to: "paid", + trigger: "payment-received", + guard: "payment.amount >= order.total", + }, + { + id: "trans:paid-to-shipped", + from: "paid", + to: "shipped", + trigger: "shipping-label-created", + // GUARD MISSING — this is a gap + }, + ], + states: [ + { id: "created", initial: true }, + { id: "paid" }, + { id: "shipped", final: true }, + ], +}; + +// OUTPUT: CycloneDX 1.5 JSON +const compiledOutput = { + bomFormat: "CycloneDX", + specVersion: "1.5", + serialNumber: "urn:uuid:d5a7b3c1-e8f2-4a9d-b6c0-123456789abc", + version: 1, + metadata: { + timestamp: "2026-05-09T22:57:00Z", + tools: [{ + vendor: "Speckl", + name: "Speckl Compiler", + version: "2.3.1", + }], + properties: [ + { + name: "speckl:lifecycle:paid", + value: JSON.stringify({ + transition: "trans:created-to-paid", + trigger: "payment-received", + guard: "payment.amount >= order.total", + }), + }, + { + name: "speckl:lifecycle:shipped", + value: JSON.stringify({ + transition: "trans:paid-to-shipped", + trigger: "shipping-label-created", + guard: "__speckl_stub__gap_GAP-001_missing_guard", + gapRef: "GAP-001", + }), + }, + { + name: "speckl:specification-debt", + value: JSON.stringify({ + totalGaps: 2, + errors: 1, + warnings: 1, + readiness: "beta", + debtItems: [ + { + id: "GAP-001", + severity: "error", + description: "Missing guard condition on transition paid → shipped", + requiredInformation: "What condition must be true for an order to ship? (e.g., payment settled, inventory confirmed)", + blocksValid: false, + }, + { + id: "GAP-002", + severity: "warning", + description: "Payment Service processes PII — needs documentation", + requiredInformation: "Document PII scope: what data, retention, encryption, access controls", + blocksValid: false, + }, + ], + }), + }, + ], + }, + components: [ + { + type: "application", + name: "Order API", + version: "2.1.0", + "bom-ref": "entity:order-api", + properties: [ + { name: "speckl:language", value: "TypeScript" }, + { name: "speckl:framework", value: "Express" }, + ], + }, + { + type: "application", + name: "Payment Service", + version: "1.8.3", + "bom-ref": "entity:payment-service", + properties: [ + { name: "speckl:language", value: "Rust" }, + { name: "speckl:pii-scope", value: "processes-credit-cards" }, + { name: "speckl:gap-ref", value: "GAP-002" }, + ], + }, + { + type: "application", + name: "Shipping Worker", + version: "0.9.0", + "bom-ref": "entity:shipping-worker", + properties: [ + { name: "speckl:language", value: "Go" }, + ], + }, + // STUB: generated for the missing guard + { + type: "application", + name: "__speckl_stub__gap_GAP-001_missing_guard", + "bom-ref": "__speckl_stub__gap_GAP-001_missing_guard", + properties: [ + { name: "speckl:stub", value: "true" }, + { name: "speckl:gap-ref", value: "GAP-001" }, + ], + }, + ], + dependencies: [ + { + ref: "entity:order-api", + dependsOn: ["entity:payment-service", "entity:shipping-worker"], + }, + { + ref: "entity:payment-service", + dependsOn: [ + "__speckl_stub__gap_GAP-001_missing_guard", // Linked to stub + ], + }, + ], +}; +``` + +### Manifest for this compilation: + +```json +{ + "manifestVersion": "1.0.0", + "inputs": { + "sirHash": "sha256:abc123def456...", + "sirVersion": 3, + "sirId": "sir:order-processing-v3" + }, + "compiler": { + "name": "speckl-compiler", + "version": "2.3.1", + "targets": ["cyclonedx@1.5"], + "hash": "sha256:compiler-v2.3.1-hash..." + }, + "outputs": { + "cyclonedx": { + "hash": "sha256:789fedcba...", + "size": 4521, + "timestamp": "2026-05-09T22:57:00Z" + } + }, + "manifestHash": "sha256:manifest-hash..." +} +``` + +--- + +## 8. Output Formats and Packaging + +### Single Artifact vs. Bundle + +Compilation can produce a single artifact or a bundle: + +```typescript +interface ArtifactBundle { + // Bundle metadata + bundleId: string; + bundleHash: string; + + // The SIR this was compiled from + sir: { + id: string; + version: number; + hash: string; + }; + + // Compiler info + compiler: { + version: string; + hash: string; + }; + + // All compiled artifacts + artifacts: Record; + + // Shared manifest + manifest: CompilationManifest; + + // Packaging metadata + created: string; // ISO 8601, for information only + format: "speckl-bundle@1.0"; +} +``` + +### Provenance Embedding + +Every artifact embeds its own provenance. For formats that support metadata (CycloneDX, SPDX, PROV-O), the compilation manifest is embedded directly. For formats that don't (plain JSON, WASM), a sidecar manifest is generated: + +``` +output/ +├── order-processing.cdx.json # CycloneDX with embedded provenance +├── order-processing.cdx.manifest.json # Standalone manifest +├── order-processing.wasm # WASM binary +├── order-processing.wasm.manifest.json # Sidecar manifest +└── order-processing.bundle.json # Full bundle with all artifacts +``` + +### WASM as Universal Runtime Format + +Speckl's current path compiles state machines to WASM, enabling: + +1. **Cross-platform execution**: The same state machine runs in browser, server, edge +2. **Deterministic execution**: WASM is deterministic by design (no floats with NaN boxing) +3. **Compact**: Binary format, suitable for on-chain / embedded use +4. **Sandboxed**: Safe to execute untrusted state machines + +```typescript +// WASM compilation target +const wasmTarget: CompilerTarget = { + id: "wasm", + version: "1.0.0", + format: "wasm", + mimeType: "application/wasm", + supportedDomains: [SIRDomain.StateMachine], + requirements: { + requiredSections: ["states", "transitions"], + minCompletenessScore: 0.5, + }, + + compile(sir: SIR, config: TargetConfig, context: CompilationContext): CompileOutput { + // Generate a deterministic WASM module that: + // 1. Exports a state machine execution function + // 2. Accepts events as input + // 3. Returns new state + actions + + const wasmModule = generateWasmStateMachine({ + states: sir.states, + transitions: sir.transitions.map(t => ({ + from: t.from, + to: t.to, + trigger: t.trigger, + guard: t.guard ? compileGuardToWasm(t.guard) : null, + actions: (t.actions ?? []).map(compileActionToWasm), + })), + // Embed SIR hash for verification + metadata: { + sirHash: context.sirHash, + compilerVersion: context.compilerVersion, + }, + }); + + const content = wasmModule.toBytes(); + return { + artifact: { + format: "wasm", + mimeType: "application/wasm", + content, + contentHash: sha256(content), + size: content.length, + }, + // ... manifest, gaps, readiness + }; + }, +}; +``` + +### Self-Describing Artifacts + +Can you reconstruct the SIR from the compiled artifact? Generally **no** — it's a lossy transformation. But some targets are more lossy than others: + +| Target | Reversibility | Notes | +|---|---|---| +| CycloneDX | Partial | Entities and dependencies survive; state transitions lose semantics | +| PROV-O | High | Designed for provenance — much of the SIR structure survives | +| WASM | Low | Compiled to instructions; reconstruction is decompilation, not guaranteed | +| OpenAPI | Partial | Endpoint structure survives; validation layer intent doesn't | +| Terraform | Low | HCL is declarative; can infer resources but not the SIR abstractions | + +The compilation manifest fills the gap — it records the SIR hash, so you can always look up the exact SIR that produced a given artifact. The bundle includes both, so nothing is lost. + +```typescript +function reconstructFromBundle(bundle: ArtifactBundle): { + sir: SIR; // Loaded from the SIR hash reference + artifacts: Record; + manifest: CompilationManifest; +} { + // The bundle doesn't contain the SIR itself (that's stored separately) + // But it references it by hash — load it from the SIR store + const sir = sirStore.lookup(bundle.sir.hash); + if (!sir) { + throw new Error(`SIR ${bundle.sir.hash} not found in store`); + } + + // Verify: does this SIR's hash match what the manifest says? + if (sir.hash !== bundle.inputs.sirHash) { + throw new Error("SIR hash mismatch — possible tampering"); + } + + return { sir, artifacts: bundle.artifacts, manifest: bundle.manifest }; +} +``` + +--- + +## Architecture Summary + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Compilation Layer │ +│ │ +│ ┌──────────────────────────────────────────────────────┐ │ +│ │ Compiler Orchestrator │ │ +│ │ - Validates SIR × target compatibility │ │ +│ │ - Manages incremental compilation cache │ │ +│ │ - Coordinates multi-target compilation │ │ +│ │ - Builds the compilation manifest │ │ +│ └────────────────────┬─────────────────────────────────┘ │ +│ │ │ +│ ┌─────────────┼─────────────┐ │ +│ ▼ ▼ ▼ │ +│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ +│ │CycloneDX │ │ PROV-O │ │ WASM │ ... │ +│ │ Target │ │ Target │ │ Target │ │ +│ └──────────┘ └──────────┘ └──────────┘ │ +│ │ │ │ │ +│ ▼ ▼ ▼ │ +│ ┌──────────────────────────────────────────────────────┐ │ +│ │ Artifact Assembler │ │ +│ │ - Bundles artifacts + manifest │ │ +│ │ - Embeds provenance in output formats │ │ +│ │ - Generates sidecar manifests when needed │ │ +│ │ - Signs bundles (future: sigstore/cosign) │ │ +│ └──────────────────────────────────────────────────────┘ │ +│ │ +│ Invariants: │ +│ ✓ Pure functional — no side effects │ +│ ✓ Content-addressed — every artifact has a hash │ +│ ✓ Reproducible — same SIR → same artifact, always │ +│ ✓ Gap-tolerant — compiles what it can, marks the rest │ +└─────────────────────────────────────────────────────────────┘ +``` + +## Key Design Decisions + +1. **Pure functional compilation**: The compiler is a function `(SIR, TargetConfig) → CompilationResult`. No side effects guarantee determinism. + +2. **Content-addressing over versioning**: Artifacts aren't "version 1.2.3" — they're `sha256:abc123...`. You know exactly what you have. + +3. **Gap-tolerant by design**: The compiler handles incomplete SIRs gracefully with stubs, debt tracking, and readiness levels. Works from conversation day 1. + +4. **Plugin architecture for targets**: New targets are standalone modules. The compiler core doesn't know about CycloneDX or Terraform — it knows about the `CompilerTarget` interface. + +5. **Manifest as proof**: Every compilation produces cryptographic proof of what was compiled, by whom, from what. The manifest is the chain of custody. + +6. **Incremental over rebuild**: Large SIRs with many entities benefit from partial recompilation. The dependency map enables surgical updates. + +7. **Bundle as distribution unit**: A `.speckl.bundle.json` contains all artifacts and the manifest. Self-contained, verifiable, distributable. diff --git a/docs/db-sqlite-reference.md b/docs/db-sqlite-reference.md new file mode 100644 index 0000000..a61a039 --- /dev/null +++ b/docs/db-sqlite-reference.md @@ -0,0 +1,551 @@ +# SQLite Pager, WAL, VDBE, and B-Tree Reference + +> Author: Marcus (agent-cos) +> Date: 2026-05-09 +> Purpose: Engineering source material for SpeckDL formal verification +> Target: SQLite version 3.53.1 (check-in c88b22011a54b, 2026-05-05) + +--- + +## 1. Pager State Machine + +The pager is SQLite's storage engine — manages page cache, transactions, and crash recovery. Located in `src/pager.c` (~12K lines). The pager owns the lock state, journal state, and page cache. + +### 1.1 Pager State Enum (from `pager.c` header comment) + +``` +PAGER_UNLOCK (0) — No lock held. Initial state. +PAGER_SHARED (1) — Read lock (SQLITE_LOCK_SHARED). Read-only access. +PAGER_RESERVED (2) — Reserved lock (SQLITE_LOCK_RESERVED). Preparing to write. +PAGER_EXCLUSIVE (3) — Exclusive lock (SQLITE_LOCK_EXCLUSIVE). Actively writing. +PAGER_SYNCED (4) — All dirty pages written + synced. Journal can be deleted. +PAGER_ERROR (5) — Error occurred. No further operations allowed. +PAGER_OPEN (6) — Special state: connection open but no lock. Transitional. +``` + +### 1.2 State Transition Table + +| From | To | Trigger / Guard | Side Effects | +|------|----|-----------------|--------------| +| PAGER_OPEN | PAGER_SHARED | First `pager_acquire()` / `pager_begin()` | Obtain SQLITE_LOCK_SHARED | +| PAGER_UNLOCK | PAGER_SHARED | sqlite3PagerSharedLock() | Obtain shared lock, recover hot journal if present | +| PAGER_SHARED | PAGER_UNLOCK | `pager_unlock()` / end of read tx | Release shared lock | +| PAGER_SHARED | PAGER_RESERVED | `pager_write()` or `pager_begin()` | Obtain SQLITE_LOCK_RESERVED. **No other writer can proceed.** | +| PAGER_RESERVED | PAGER_EXCLUSIVE | First page modification (in `PAGER_RESERVED`) | - | +| PAGER_RESERVED | PAGER_EXCLUSIVE | Commit begins | Obtain PENDING then EXCLUSIVE lock | +| PAGER_RESERVED | PAGER_SHARED | ROLLBACK | Discard changes, release reserved lock | +| PAGER_EXCLUSIVE | PAGER_SYNCED | All dirty pages written + fsynced | Journal header updated to sync state | +| PAGER_SYNCED | PAGER_SHARED (journal-mode) or PAGER_UNLOCK (WAL) | Delete journal file (COMMIT point) | Transaction atomically committed | +| PAGER_EXCLUSIVE | PAGER_ERROR | I/O error, disk full, etc. | errCode set, hot journal preserved | +| PAGER_SYNCED | PAGER_ERROR | I/O error during sync | errCode set, journal preserved for recovery | +| any → | PAGER_ERROR | Any I/O error during pager operation | Further ops blocked | +| PAGER_ERROR | PAGER_OPEN | `pager_unlock()` forced close | Resets connection but keeps db open | + +### 1.3 Journal Mode Variants + +Three rollback journal modes (non-WAL): + +- **DELETE** (default): Journal file deleted on commit. If crash before delete → hot journal → recovery rolls back. +- **TRUNCATE**: Journal file truncated to zero length on commit. Faster on some filesystems than delete. +- **PERSIST**: Journal header zeroed on commit, file left on disk. Slightly faster, but leaves journal files around. + +The pager tracks `journalMode` and adjusts the commit-delete step accordingly. + +### 1.4 Pager Invariants (to formally specify) + +- **I1 (Lock State Consistency)**: `eState` must be consistent with actual file lock held. If `eState==PAGER_SHARED`, must hold `SQLITE_LOCK_SHARED`. If `eState==PAGER_EXCLUSIVE`, must hold `SQLITE_LOCK_EXCLUSIVE`. +- **I2 (Dirty Page Tracking)**: For any page in the cache, `isDirty` iff the page has been modified since last checkpoint/journal write. +- **I3 (Journal Existence)**: The journal file exists iff `eState >= PAGER_RESERVED` and `eState != PAGER_ERROR` (except during recovery). +- **I4 (Hot Journal Detection)**: A journal is "hot" iff it exists but the database was not cleanly closed. On first shared lock, hot journals must be rolled back. +- **I5 (Error Propagation)**: Once `eState == PAGER_ERROR`, all subsequent pager operations must return the stored `errCode`. +- **I6 (Read-Only Pager)**: If `readOnly` flag is set, `eState` must never exceed `PAGER_SHARED`. +- **I7 (Sync Before Commit)**: No dirty page is persisted to the database file until after the journal is synced to disk AND the exclusive lock is held. +- **I8 (Commit Atomicity)**: The commit point is the deletion of the journal file. Before this: crash → rollback. After this: crash → committed state visible. + +### 1.5 Lock Escalation Sequence (from `atomiccommit.html`) + +``` +NO_LOCK → SHARED (read) → RESERVED (intent to write) → PENDING (block new readers) + → EXCLUSIVE (writing) → no lock (after commit journal deletion) +``` + +Key subtlety: PENDING lock allows existing readers to finish but blocks new readers. This prevents writer starvation in the face of constant new read connections. + +--- + +## 2. WAL Mode State Machine + +WAL mode inverts the rollback journal: original data stays in the database file, changes are appended to the WAL file. Commit = appending a commit record to WAL. + +### 2.1 WAL Files + +| File | Purpose | Lifecycle | +|------|---------|-----------| +| `X-wal` | Write-ahead log. Sequential append of frames. | Exists while connections are open. Deleted on clean close. | +| `X-shm` | Shared memory (wal-index). Hash table for frame location. | mmap'd. Not fsynced. Omitted under EXCLUSIVE locking mode. | +| `X` | Main database file. Contains original + checkpointed pages. | Persistent. | + +### 2.2 WAL-Index Header Fields (from `walformat.html`) + +| Field | Bytes | Description | +|-------|-------|-------------| +| iVersion | 0-3 | WAL-index format version (3007000) | +| iChange | 8-11 | Transaction counter, incremented per tx | +| isInit | 12 | 1 when shm file initialized | +| bigEndCksum | 13 | Checksum endianness flag | +| szPage | 14-15 | Page size (or 1 for 65536) | +| mxFrame | 16-19 | Number of valid commit frames in WAL | +| nPage | 20-23 | Database size in pages | +| aFrameCksum | 24-31 | Checksum of last frame | +| aSalt | 32-39 | Two salt values from WAL header | +| nBackfill | 96-99 | Frames already backfilled by checkpoints | +| read-mark[0..4] | 100-119 | Five reader end marks | +| nBackfillAttempted | 128-132 | Frames attempted but not confirmed backfilled | + +### 2.3 WAL Locks + +| Lock | Byte | Purpose | +|------|------|---------| +| WAL_WRITE_LOCK | 120 | Exclusive: appending frames to WAL | +| WAL_CKPT_LOCK | 121 | Exclusive: running checkpoint | +| WAL_RECOVER_LOCK | 122 | Exclusive: running recovery | +| WAL_READ_LOCK(0) | 123 | Reader end-marks (0-4) | +| WAL_READ_LOCK(1) | 124 | | +| WAL_READ_LOCK(2) | 125 | | +| WAL_READ_LOCK(3) | 126 | | +| WAL_READ_LOCK(4) | 127 | | + +### 2.4 WAL Frame Lifecycle + +``` +Frame format (in WAL file): [page-number] [db-size-after-commit] [salt-1] [salt-2] [checksum-1] [checksum-2] + followed by page data. + +Frame lifecycle states: + 1. WRITER appends frames to end of WAL file (holding WAL_WRITE_LOCK exclusive) + 2. Last frame of a transaction marks "commit" via non-zero db-size field + 3. READERS snapshot mxFrame at transaction start → their "end mark" + 4. CHECKPOINTER copies frames back to database (holding WAL_CKPT_LOCK), increments nBackfill + 5. RESET: When nBackfill == mxFrame and no reader holds WAL_READ_LOCK(N) for N>0, + writer rewinds WAL to beginning (resets mxFrame, nBackfill to 0) +``` + +### 2.5 Reader Snapshot (End Mark) Protocol + +- When a read transaction begins, the reader notes current `mxFrame` value → this is its "end mark" +- The reader acquires one of WAL_READ_LOCK(0..4) to declare its end mark +- To read page P, the reader calls `FindFrame(P, end_mark)`: find the latest frame for page P in WAL with frame number ≤ end_mark +- If found: read from WAL frame. If not found: read from database file. +- Readers hold WAL_READ_LOCK(N) for the duration of their read transaction +- This prevents checkpointer from overwriting pages the reader still needs + +### 2.6 Checkpoint States + +| Checkpoint Mode | Behavior | +|----------------|----------| +| PASSIVE | Non-blocking. Works around readers. May not complete. Default. | +| FULL | Blocks until checkpoint completes or a writer appears. More aggressive. | +| RESTART | Full checkpoint then blocks until no readers. Ensures WAL reset. | +| TRUNCATE | Like RESTART, also truncates WAL file to zero bytes. | + +Checkpoint progress constraint: MUST STOP at the minimum `end_mark` across all active readers. +- `nBackfill ≤ min(read_mark[0..4])` +- `nBackfillAttempted ≥ nBackfill` (may be ahead if reader blocked a backfill) + +### 2.7 WAL Reset Condition + +The WAL is reset (wound back to start) when: +1. `nBackfill == mxFrame` (all frames backfilled) +2. No reader holds `WAL_READ_LOCK(N)` for any N > 0 (no readers using WAL) +3. Writer holds WAL_WRITE_LOCK exclusively +4. On reset: `mxFrame = 0`, `nBackfill = 0`, file position rewound + +### 2.8 WAL Invariants (to formally specify) + +- **W1 (mxFrame Monotonic)**: `mxFrame` is non-decreasing except during WAL reset. +- **W2 (nBackfill Bound)**: `nBackfill ≤ mxFrame` always. +- **W3 (Backfill ≤ Readers)**: `nBackfillAttempted ≤ min(read_mark[0..4])` for any checkpoint that respects readers. +- **W4 (Frame Validity)**: All frames 1..mxFrame have valid checksums. Frames > mxFrame are undefined. +- **W5 (Commit Detection)**: Frame k is a commit frame iff its db-size-after-commit field ≠ 0. +- **W6 (Read Consistency)**: A reader with end-mark M sees exactly the database state as of the last commit frame ≤ M. +- **W7 (WAL Write Serialization)**: Only one connection may append to WAL at a time (WAL_WRITE_LOCK exclusive). +- **W8 (Checkpoint Isolation)**: Checkpoint must not overwrite pages needed by any active reader (nBackfill ≤ min(read_marks)). +- **W9 (Salt Match)**: WAL file header salt values must match `aSalt` in the WAL-index header. Mismatch → WAL is stale → must recover. +- **W10 (WAL Reset Safety)**: WAL may only be reset when `nBackfill == mxFrame` AND no reader locks exist. + +--- + +## 3. VDBE Instruction Set (Key Opcodes) + +The VDBE is SQLite's bytecode virtual machine. Each prepared statement is a bytecode program. Source: `vdbe.c` (~8K lines). 192 opcodes in version 3.53.1. + +### 3.1 Instruction Format + +``` +Each instruction: [opcode] P1 P2 P3 P4 P5 + P1, P2, P3: 32-bit signed integers (registers, cursor numbers, jump targets) + P4: various (int, string, function pointer, collation) + P5: 16-bit unsigned flags +``` + +### 3.2 Transaction & Locking Opcodes + +| Opcode | P1 | P2 | P4 | Effect | +|--------|----|----|----|--------| +| **Transaction** | iDb | write_flag | | Begin a transaction on database P1. P2=1 for write, 0 for read. Opens pager transaction. | +| **AutoCommit** | autocommit_flag | rollback_flag | | Set auto-commit on/off. If P2 set, rollback first. Causes VM halt. | +| **Checkpoint** | iDb | mode | | Run checkpoint on database P1. P2 = PASSIVE(0)/FULL(1)/RESTART(2)/TRUNCATE(3). | +| **Savepoint** | start_flag | savepoint_name | | Begin/Release/Rollback a savepoint. | +| **TableLock** | iDb | root_page | table_name | Lock a table before modifying. write_flag in P3. | +| **SchemaLock** | iDb | | schema_name | Lock schema objects. | + +### 3.3 Cursor Opcodes + +| Opcode | P1 | P2 | P3 | P4 | Effect | +|--------|----|----|----|----|--------| +| **OpenRead** | cursor | root_page | iDb | | Open cursor P1 for read on root page P2 of db P3 | +| **OpenWrite** | cursor | root_page | iDb | table_name | Open cursor P1 for write on root page P2 of db P3 | +| **OpenAutoindex** | cursor | key_cols | nCol | | Open ephemeral auto-index cursor | +| **OpenEphemeral** | cursor | nCol | | | Open temporary table, auto-clears on close | +| **Close** | cursor | | | | Close cursor P1 | +| **SeekGT** | cursor | jump_target | | key_reg | Seek cursor P1 to first entry > key in register | +| **SeekGE** | cursor | jump_target | | key_reg | Seek cursor P1 to first entry >= key in register | +| **SeekLT** | cursor | jump_target | | key_reg | Seek cursor P1 to first entry < key in register | +| **SeekLE** | cursor | jump_target | | key_reg | Seek cursor P1 to first entry <= key in register | +| **SeekRowid** | cursor | | | | Seek cursor P1 to entry with given rowid | +| **NotFound** | cursor | jump_target | | | Jump if key not found in index | +| **Found** | cursor | jump_target | | | Jump if key found in index | +| **Next** | cursor | jump_target | | | Advance cursor P1 to next entry. Jump to P2 if no more rows. | +| **Prev** | cursor | jump_target | | | Move cursor P1 to previous entry. Jump to P2 if at beginning. | +| **Last** | cursor | | | | Move cursor P1 to last entry. | +| **Rewind** | cursor | jump_target | | | Move to first entry. Jump to P2 if empty. | +| **Column** | cursor | column_idx | dest_reg | | Read column P2 from cursor P1 into register P3 | +| **Rowid** | cursor | dest_reg | | | Write rowid of cursor P1 into register P2 | +| **IdxInsert** | cursor | | | key | Insert entry into index cursor | +| **IdxDelete** | cursor | | | key | Delete entry from index cursor | +| **Insert** | cursor | | | | Insert current record into table cursor | +| **Delete** | cursor | | | | Delete current record from table cursor | +| **SorterSort** | cursor | | | | Sort sorter cursor P1 | +| **SorterData** | cursor | dest_reg | | | Read current sorter row | + +### 3.4 Value & Register Opcodes + +| Opcode | P1 | P2 | P3 | Effect | +|--------|----|----|----|--------| +| **Integer** | value | dest | | Store integer value in register P2 (P1 is the value) | +| **Real** | value (P4) | dest | | Store real value in register P2 | +| **String8** | | dest | string | Store UTF-8 string in register P2. Self-modifying → becomes String opcode. | +| **Null** | start | count | | Set P2 registers starting at P1 to NULL | +| **Blob** | size | dest | data | Store P1-byte blob in register P2 | +| **Copy** | src | dest | count | Copy P3 registers from P1 to P2 | +| **Move** | src | dest | count | Move P3 registers from P1 to P2 (clear source) | +| **SCopy** | src | dest | | Soft copy (shallow) register P1 to P2 | +| **ResultRow** | reg_start | nReg | | Pause VM: emit result row from registers P1..P1+P2-1. sqlite3_step() returns SQLITE_ROW. | +| **Yield** | pc_store | | | Coroutine yield. Swap PC with value in register P1. | +| **Halt** | p1 | p2 | p3 | Halt execution. If P1==SQLITE_OK and P2==OE_Abort, auto-rollback. | + +### 3.5 Arithmetic & Comparison Opcodes + +| Opcode | P1 | P2 | P3 | P4 | Effect | +|--------|----|----|----|----|--------| +| **Add** | r1 | r2 | result | | r[P3] = r[P1] + r[P2] | +| **Subtract** | r1 | r2 | result | | r[P3] = r[P1] - r[P2] | +| **Multiply** | r1 | r2 | result | | r[P3] = r[P1] * r[P2] | +| **Divide** | r1 | r2 | result | | r[P3] = r[P1] / r[P2] | +| **Remainder** | r1 | r2 | result | | r[P3] = r[P1] % r[P2] | +| **Eq** | r1 | r2 | jump | coll | Compare r[P1] and r[P2]. Jump to P3 if equal. | +| **Ne** | r1 | r2 | jump | coll | Jump if not equal | +| **Lt** | r1 | r2 | jump | coll | Jump if r[P1] < r[P2] | +| **Le** | r1 | r2 | jump | coll | Jump if r[P1] ≤ r[P2] | +| **Gt** | r1 | r2 | jump | coll | Jump if r[P1] > r[P2] | +| **Ge** | r1 | r2 | jump | coll | Jump if r[P1] ≥ r[P2] | +| **BitAnd** | r1 | r2 | result | | r[P3] = r[P1] & r[P2] | +| **BitOr** | r1 | r2 | result | | r[P3] = r[P1] \| r[P2] | +| **BitNot** | src | dest | | | r[P2] = ~r[P1] | +| **Concat** | r1 | r2 | result | | r[P3] = r[P1] \|\| r[P2] | + +### 3.6 Control Flow Opcodes + +| Opcode | P1 | P2 | Effect | +|--------|----|----|--------| +| **Goto** | | target | Unconditional jump to P2 | +| **Gosub** | return_reg | target | Push PC to register P1, jump to P2 | +| **Return** | return_reg | | Jump to address stored in P1 + 1 | +| **If** | jump_target | dest | Conditional jump | +| **IfNot** | jump_target | dest | Conditional jump (negated) | +| **IfNull** | reg | jump_target | Jump if register contains NULL | +| **NotNull** | reg | jump_target | Jump if register is not NULL | +| **IfPos** | reg | jump_target | Jump if register > 0 | +| **IfNeg** | reg | jump_target | Jump if register < 0 | +| **Once** | flag | | NOP if flag was set. Execute body exactly once. | +| **Init** | flags | max_addr | First opcode. Sets trace flags. Self-modifying: increments P1 each invocation. | +| **Program** | | sub_program_addr | Invoke a subprogram (trigger). | +| **Param** | param_idx | dest_reg | Read parameter from caller into destination register. | + +### 3.7 Aggregate Function Opcodes + +| Opcode | P1 | P2 | P3 | Effect | +|--------|----|----|----|--------| +| **AggStep** | | arg_reg | accum | Execute xStep for aggregate function | +| **AggStep1** | step/inverse | arg_reg | accum | Execute xStep (P1=0) or xInverse (P1!=0) | +| **AggInverse** | | arg_reg | accum | Execute xInverse (window functions) | +| **AggFinal** | | | accum | Finalize aggregate, store result | +| **AggValue** | | | result | Store current aggregate value | + +### 3.8 VDBE Invariants (to formally specify) + +- **V1 (PC Bounds)**: Program counter must remain within [0, nOp] during execution. +- **V2 (Register Type Safety)**: Each register holds exactly one type at a time (NULL, int, real, text, blob, RowSet, Frame, Undefined). +- **V3 (Cursor Validity)**: An opcode must not reference a cursor unless it was previously opened (OpenRead/OpenWrite/OpenEphemeral) and not yet closed. +- **V4 (Transaction Guard)**: Before any write opcode executes, a transaction must be open. Read opcodes may run outside a transaction but must obtain shared lock. +- **V5 (ResultRow Protocol)**: ResultRow must be preceded by opcodes that fill the output registers. sqlite3_step() returns SQLITE_ROW; subsequent call resumes after ResultRow. +- **V6 (Halt Termination)**: Execution terminates only via Halt opcode, PC exceeding last opcode address, or error. +- **V7 (Error Cleanup)**: On error or HALT with error, all open cursors are closed, memory freed, and active transactions rolled back. +- **V8 (Subprogram Isolation)**: Each Program invocation gets its own register set. Params are the only communication channel to the caller. +- **V9 (Coroutine Symmetry)**: Yield stores PC to register; subsequent Yield reads register to PC. Coroutine execution must return to same point. + +--- + +## 4. B-Tree Invariants + +SQLite uses B+tree for tables, B-tree for indices. Source: `btree.c`. + +### 4.1 Page Types + +| Type | Value | Description | +|------|-------|-------------| +| Interior index | 0x02 | Index b-tree interior page | +| Interior table | 0x05 | Table b-tree interior page | +| Leaf index | 0x0a | Index b-tree leaf page | +| Leaf table | 0x0d | Table b-tree leaf page | + +### 4.2 Page Layout + +``` +Page header: + [page_type:1] [first_freeblock:2] [cell_count:2] [cell_content_start:2] [fragmented_free_bytes:1] + +Followed by: + [right_child_pointer:4] (interior pages only) + [cell_pointer_array: cell_count * 2 bytes] — offsets to cells, in key order + [unallocated space in middle] + [cells: packed from end of page backwards] +``` + +### 4.3 B-Tree Cell Format + +**Table leaf cell:** +``` +[payload_size: varint] [rowid: varint] [payload: header + columns] +``` + +**Table interior cell:** +``` +[left_child_page:4] [rowid: varint] [payload: partial row data] +``` + +**Index leaf cell:** +``` +[payload_size: varint] [payload: header + index columns + rowid] +``` + +**Index interior cell:** +``` +[left_child_page:4] [payload_size: varint] [payload: header + index columns + rowid] +``` + +### 4.4 Key Ordering Properties + +- **Table B-Tree**: Ordered by rowid (integer primary key). Interior cells store the rowid of the first entry in each child page. +- **Index B-Tree**: Ordered by indexed column(s) + rowid. Interior cells store the key of the first entry in each child page. +- All keys within a page are in ascending order. +- For any interior page: all keys in child subtree i are ≤ the key at position i in the interior page, and ≤ all keys in child subtree i+1. (For indices, strict < for key columns, ≤ for rowid as tiebreaker.) + +### 4.5 Page Split Condition + +A page splits when inserting a cell would exceed the usable page space. +1. A new page is allocated. +2. Cells are distributed (roughly half each) between old and new page. +3. Parent interior page gets a new cell pointing to the new page. + +### 4.6 Page Merge Condition + +When deleting, if a page's content falls below the minimum fill threshold (roughly 25-50% depending on implementation), the page may be merged with a sibling: +1. Cells are moved to sibling page. +2. Parent pointer to the empty page is removed. +3. Empty page is added to freelist. + +### 4.7 B-Tree Cursor Operations + +``` +Cursor state: { Btree*, root page, current page, cell index, cache of page stack } + +Operations: + sqlite3BtreeCursor(): Create cursor on B-tree identified by root page + flags + sqlite3BtreeMoveto(): Position cursor to entry that matches key (or nearest) + sqlite3BtreeNext(): Advance cursor. Page crossing handled automatically. + sqlite3BtreePrevious(): Move cursor backwards. Page crossing handled. + sqlite3BtreeInsert(): Insert cell into current position + sqlite3BtreeDelete(): Delete entry at cursor position +``` + +### 4.8 B-Tree Invariants (to formally specify) + +- **B1 (Page Type Consistency)**: A page's type field must match its role. Interior pages reference child pages; leaf pages do not. +- **B2 (Key Order Monotonic)**: For any interior page, keys in child page i are less than or equal to keys in child page i+1. Cells within a page are sorted ascending. +- **B3 (Pointer Integrity)**: Every child page reference must point to a valid allocated page of the correct type (or overflow page, or freelist page). +- **B4 (Cell Count Consistency)**: `cell_count` in the page header must equal the number of entries in the cell pointer array. +- **B5 (Free Space Accounting)**: `fragmented_free_bytes` + unallocated space between cell pointer array and cells must sum correctly to the page's available free space. +- **B6 (Page Size Boundary)**: Usable page size = page_size - reserved_bytes. All cells must fit within the usable page. +- **B7 (Root Page Integrity)**: The b-tree root page number must be a valid, allocated b-tree page. The schema table's root page mappings must be consistent. +- **B8 (Balance Preserved)**: After every insert/delete operation, the b-tree must satisfy all internal structural invariants (fill ratio, parent-child pointer correctness, key ordering). +- **B9 (Overflow Chain Integrity)**: If a cell's payload spills to overflow pages, the chain of overflow pages must: (a) form a proper linked list, (b) be bounded in length by the payload size, (c) contain no cycles. +- **B10 (Freelist Consistency)**: Freelist pages must not be reachable from any b-tree root. Freelist trunk pages must point to valid leaf pages or 0. + +--- + +## 5. Known Bugs and Formal Verification Precedents + +### 5.1 Notable SQLite CVEs + +| CVE | Year | Type | Module | Fixed In | Severity | +|-----|------|------|--------|----------|----------| +| CVE-2022-35737 | 2022 | Integer overflow → buffer overrun → RCE/DoS | printf/str_vappendf | 3.39.2 | High (7.5) | +| CVE-2025-3277 | 2025 | Write past end of malloc'd array via integer overflow | concat_ws() function | 3.49.1 | Moderate | +| CVE-2025-6965 | 2025 | Integer overflow → array bounds read | SQL injection path | 3.50.2 | Moderate | +| CVE-2024-0232 | 2024 | Use-after-free in JSON parser | JSON | 3.43.2 | Moderate | +| CVE-2023-7104 | 2023 | Corruption in session extension changeset processing | Session (ext) | 3.43.1 | Low | +| CVE-2025-29088 | 2025 | Crash via bad args to sqlite3_db_config | Config API | 3.49.1 | Low | + +**Key observation from SQLite's CVE page:** +> Almost all CVEs written against SQLite require the ability to inject and run arbitrary SQL. The core storage layer (pager, WAL, B-tree) has remarkably few logic bugs. Most CVEs are in SQL-level functions, extensions, or edge cases in the printf/logging subsystem. + +### 5.2 22-Year-Old Integer Overflow (CVE-2022-35737) + +Introduced October 2000 (version 1.0.12), discovered 2022 by Trail of Bits. Exploitable on 64-bit systems when large strings (>2GB) passed to printf functions with `!` unicode flag. Arbitrary code execution possible without stack canaries. Not a pager/journal bug — but illustrates how old, untested edge cases survive. + +### 5.3 Formal Verification Precedents + +#### 5.3.1 FoundationDB (Apple) +- **What**: Distributed transactional key-value store, formally verified in Flow (their deterministic simulation language) +- **Method**: Deterministic simulation testing — run all DB processes as fibers on a single thread with simulated network/disk, inject faults, check invariants. Run millions of random seeds nightly. +- **Result**: Found ~70 bugs before release. Flow-based simulation verified correctness of their distributed transaction, replication, and recovery protocols. +- **Relevance to SQLite**: The determinism approach parallels what could be done for SQLite's pager. SQLite is single-process by design, making deterministic replay easier. + +#### 5.3.2 TigerBeetle +- **What**: Distributed financial accounting database. VOPR (Viewstamped Operation Replicator) deterministic simulator. +- **Method**: Entire cluster runs on single thread. 1000x speed. Injects network, storage, and process faults. Swarm testing + fuzzer. Continuous verification. +- **Relevance to SQLite**: The VOPR approach of running the real implementation under controlled simulation is directly applicable to testing SQLite's recovery paths in the pager. + +#### 5.3.3 Amazon (DynamoDB, S3) +- **What**: TLA+ specifications for AWS distributed services +- **Method**: Model checking (TLC) + manual proofs (TLAPS). Found bugs in replication protocols, quorum membership, leader election. +- **Paper**: "Use of Formal Methods at Amazon Web Services" (Newcombe et al., 2014) +- **Relevance to SQLite**: TLA+ is ideal for specifying state machines like the pager's locking protocol. The pager state machine is small enough for tractable TLC model checking. + +#### 5.3.4 MongoDB Transactions (VLDB 2025) +- **What**: "Design and Modular Verification of Distributed Transactions in MongoDB" +- **Method**: TLA+ specification + model checking + test-case generation. +- **Relevance to SQLite**: Demonstrates that even complex transactional protocols can be verified with TLA+. + +#### 5.3.5 Formal Land's Rocq Translation of SQLite B-Tree (2025-2026) +- **What**: AI-assisted translation of SQLite C code to Rocq (Coq variant). Translation only (not full proof of equivalence). +- **Result**: "Still a big project to run with current technologies." Covered substantial part of B-tree but deemed too unstable for full verification at current AI capability levels. +- **Relevance**: Direct precedent that SQLite formal verification is hard. Suggests model-level verification (state machine specs) is more tractable than code-level proofs. + +#### 5.3.6 Richard Hipp's Own Attempt (SQLite Lead) +- Dr. Hipp reports spending "years" trying to write a formal design specification for SQLite but found it never added value and abandoned it. This is telling — both about the difficulty and about the value proposition. + +### 5.4 SQLite's Existing Testing (Non-Formal) + +| Method | Scale | Notes | +|--------|-------|-------| +| TH3 (Test Harness #3) | Proprietary. Billions of test cases. | 100% branch coverage. Mutation testing (inject bugs, verify they're caught). | +| OSS-Fuzz | Continuous. | Google's fuzzing infrastructure. Catches memory safety bugs. | +| Release testing | ~100M tests per release | Run on multiple platforms | +| Anomaly testing | Random SQL/DB operations with crash injection | Verify recovery | + +--- + +## 6. Target Invariants for SpeckDL Specification + +Ranked by impact (what would be most impressive/valuable to prove): + +### Tier 1: Impact/Feasibility Sweet Spot + +1. **Pager Commit Atomicity (I8)** + - Prove: For any sequence of pager operations and any crash point, the database file is always in either the pre-commit state (all changes rolled back) or the post-commit state (all changes applied). No partial-commit states. + - Value: This is the fundamental SQLite guarantee. Proving it formally would be the headline achievement. + - Feasibility: Medium. The pager state machine has only 5-6 states. The tricky part is modeling disk I/O reordering and crash-at-any-point semantics. + +2. **WAL Read Isolation (W6)** + - Prove: A reader with end-mark M sees exactly the database state as of the last commit ≤ M. Writes after M are invisible. + - Value: WAL mode's key property. With formal proof, SQLite's WAL concurrency model gets a rigorous foundation. + - Feasibility: Medium. Need to model the WAL-index hash tables and the FindFrame algorithm. + +3. **Journal Recovery Correctness** + - Prove: After a crash with a hot journal, recovery restores the database to its state before the incomplete transaction. + - Value: Crash recovery is the hardest case to test exhaustively. Formal proof would eliminate a whole class of concerns. + - Feasibility: High. Well-scoped problem with clear success criteria. + +### Tier 2: High Value, Higher Difficulty + +4. **B-Tree Balance Preservation (B8)** + - Prove: After any sequence of insertions and deletions, the b-tree satisfies all structural invariants (page fill ratios, key ordering, pointer integrity, freelist consistency). + - Value: B-tree corruption is one of the most feared failure modes. Formal proof is the gold standard. + - Feasibility: Low-Medium. The B-tree is complex (~200KB of C code). A full proof is likely too big, but the core balancing algorithm could be specified and verified in isolation. + +5. **Lock Protocol Deadlock Freedom** + - Prove: The lock escalation protocol (shared → reserved → pending → exclusive) is deadlock-free for any number of concurrent readers and writers. + - Value: Prevents a potential operational nightmare. + - Feasibility: High. Small state space. Ideal for TLC model checking. + +6. **WAL Checkpoint Safety (W8)** + - Prove: Checkpoint never overwrites pages needed by active readers, even when readers and checkpointer operate concurrently. + - Value: WAL's concurrent reader+writer+checkpointer model is nontrivial. Getting it right formally is impressive. + - Feasibility: Medium. Requires modeling concurrent processes and the WAL-index header fields. + +### Tier 3: Ambitious / Future Work + +7. **Full Pager FSM Correctness (I1-I8)** + - Prove all eight pager invariants from Section 1.4 simultaneously. + - Feasibility: Low (large specification, many interacting components). Best done incrementally. + +8. **VDBE Safety: No undefined register reads (V2)** + - Prove that the code generator never produces bytecode that reads an Undefined register. + - Feasibility: Low. Requires modeling the SQL compiler's code generation, which is a much larger problem. + +### Recommendation: Start with Tier 1 (Items 1-3) + +The pager commit atomicity + WAL read isolation + journal recovery form a coherent package: +- All three involve the pager module +- All three address crash-safety guarantees +- Together, they prove that "SQLite transactions are atomic even under arbitrary crash timing" — the single claim that matters most +- The state spaces are tractable for SpeckDL + model checking + +--- + +## 7. Source References + +| Source | Location | +|--------|----------| +| SQLite WAL documentation | https://www.sqlite.org/wal.html | +| SQLite Atomic Commit | https://www.sqlite.org/atomiccommit.html | +| SQLite Bytecode Engine | https://www.sqlite.org/opcode.html (192 opcodes, v3.53.1) | +| Database File Format | https://www.sqlite.org/fileformat2.html | +| WAL File Format | https://www.sqlite.org/walformat.html | +| Pager source | `src/pager.c` (~12K lines). States: PAGER_UNLOCK, SHARED, RESERVED, EXCLUSIVE, SYNCED, ERROR, OPEN | +| VDBE source | `src/vdbe.c` (~8K lines) + helper files | +| B-Tree source | `src/btree.c` | +| SQLite CVEs | https://sqlite.org/cves.html | +| Formal Land B-Tree translation | https://github.com/formal-land/rocq-of-db | +| FoundationDB verification | "FoundationDB: A Distributed, Unbundled, Transactional Key Value Store" (SIGMOD 2021) | +| Amazon formal methods | "Use of Formal Methods at Amazon Web Services" (2014) | +| MongoDB TLA+ | "Design and Modular Verification of Distributed Transactions in MongoDB" (VLDB 2025) | +| TigerBeetle VOPR | https://tigerbeetle.com/blog/2023-07-11-we-put-a-distributed-database-in-the-browser/ | +| CVE-2022-35737 (Trail of Bits) | https://blog.trailofbits.com/2022/10/25/sqlite-vulnerability-july-2022-library-api/ | +| Fly.io B-Tree internals | https://fly.io/blog/sqlite-internals-btree/ | + +--- + +*End of reference document. Ready for SpeckDL specification writing.* diff --git a/docs/distribution-strategy.md b/docs/distribution-strategy.md new file mode 100644 index 0000000..61dc810 --- /dev/null +++ b/docs/distribution-strategy.md @@ -0,0 +1,145 @@ +# Speckl Distribution Strategy + +> *The whitepaper is done. Now it needs a path to the world.* + +**Created:** May 7, 2026 (8:05 PM heartbeat) +**Status:** All items pending — blocked on Scott for human-required actions + +--- + +## The Asset: Whitepaper v2 + +- **~19,000 words** across 8 complete sections +- **Commit:** `6121fd6` — pushed to Forgejo +- **Sections:** The Spec-Code Gap, SpeckDL Language, Compiler Architecture, Embedded Provenance, Consensus Protocols, Real-World Applications, Related Work, Future Work & Roadmap +- **Positioning:** "specification language with compiler-level provenance for teams that need evidence of correctness more than proof of correctness" + +--- + +## Priority-Ordered Distribution Plan + +### P0: Show HN (May 12-13) + +**No gatekeepers. Post is ready.** + +- **Post:** `docs/show-hn-announcement.md` — written, copy-paste ready +- **Optimal timing:** Tuesday or Wednesday, 8-11 AM UTC (based on analysis of 1,200 launches) +- **What it needs:** Scott's HN account, click "submit" +- **Expected:** ~50-200 upvotes for a well-written open source technical tool, potential for front page +- **Risk:** Low — even a modest response builds awareness and creates a permalink + +### P1: arXiv Preprint + Weekly Blog Posts + +**Builds the knowledge graph around Speckl before conference submissions open.** + +- **arXiv:** Submit whitepaper as a preprint to cs.SE or cs.PL + - Requires: academic email or endorsement (Scott may need an endorser) + - Value: citable, discoverable, timestamped — standard for CS credibility +- **Blog series:** 5 posts extractable from whitepaper sections + 1. "The Spec-Code Gap" — why specs rot and what to do about it + 2. "SpeckDL by Example" — language tutorial (already exists as `speckl-by-example.md`) + 3. "Compiling Provenance" — how Speckl generates PROV-O, CycloneDX, SPDX + 4. "Consensus from Spec" — Paxos, Raft, Two-Phase Commit in SpeckDL + 5. "The Future of Spec-Driven Development" — roadmap and vision + - Host on: speckl.scoble.me/blog (Storj static) or scott.scoble.me/blog + - Each post cross-links to Show HN thread and whitepaper + +### P2: Conference Submissions (2027 Cycle) + +**All major 2026 cycles are closed.** Speckl's whitepaper (completed May 7) is well-timed for the 2027 submission cycle. + +#### Target Venues (2027) + +| Venue | Focus | Typical Deadline | Fit | +|-------|-------|-----------------|-----| +| **FMCAD 2027** | Formal methods + CAD | Apr-May 2027 | Strong — state machines, verification | +| **FM 2027** | Formal methods (broad) | ~Dec 2026 | Strong — methodology + tool | +| **ICSE 2027** | Software engineering | ~Aug 2026 | Good — spec-driven dev is an SE topic | +| **ASE 2027** | Automated SE | ~May 2027 | Good — compiler-level generation | +| **NFM 2027** | NASA Formal Methods | ~Dec 2026 | Strong — provenance + verification | +| **SPLASH Onward! 2026** | New ideas in programming | ~Jun 2026 | Possible — "grand visions" track | + +#### Note on FMCAD 2026 + +FMCAD 2026 (Graz, Sept 14-18, co-located with VSTTE) has **already closed**: abstract deadline April 26, paper deadline May 3. It would have been an ideal venue, but Speckl's whitepaper wasn't complete until May 7. 2027 is the target. + +### P3: QCon SF Talk Proposal (~Jun 2026) + +**Practitioner conference, not academic. Faster cycle.** + +- QCon SF 2026 call for proposals typically opens ~June +- Topic: "Spec-Driven Development: From TLA+ to TypeScript" or "Building Evidence into Specs" +- Less competitive than academic venues, reaches working engineers +- Scott's existing speaking experience (3 talk forks) positions him well + +### P4: Journal Publication (Rolling) + +- **TOSEM (ACM Transactions on Software Engineering and Methodology):** rolling submissions, high prestige +- **JAR (Journal of Automated Reasoning):** rolling, good fit for formal methods + automation +- Both take 12-18 months but produce durable citations + +### P5: Cold Email → First Consulting Client (Summer 2026) + +**When: After at least one public artifact is circulating (Show HN + arXiv).** + +- **Pipeline already built:** `speckl/outreach.md` +- **Top targets:** Dan Lorenc (Chainguard), Said Ziouani (Anchore) +- **Emails verified:** dlorenc@chainguard.dev ✅, sziouani@anchore.com ✅ +- **Content:** Consulting one-pager drafted (`speckl/docs/consulting-one-pager.md`) +- **Pricing:** Drafted, needs Scott review +- **Approach:** "We've been building Speckl — an open standard for spec-driven development with compiler-level provenance. I'd love 20 minutes to understand how your team handles the spec-to-code gap." + +--- + +## Blockers Inventory + +| Blocker | Priority | Owner | Status | +|---------|----------|-------|--------| +| Show HN submission | P0 | Scott | Post ready, needs HN account | +| arXiv endorsement/account | P1 | Scott | Whitepaper ready | +| Blog posts publishing | P1 | Scott or @content | Content mostly written | +| Consulting one-pager review | P3 | Scott | Draft complete, needs pricing validation | +| Cold email send | P3 | Scott | Targets identified, emails verified | +| engineering#9 (parser fix) | P0 tech | @engineering | 4-bug scope documented | +| Storj credentials | — | Scott | Expired, blocks new deploys | +| ClawHub/npm auth | — | Scott | Blocks MCP experiment | + +--- + +## Timeline (Ideal) + +``` +May 12-13: Show HN goes live (P0) +May 14: arXiv preprint submitted (P1) +May 15+: Blog post #1 published, weekly cadence (P1) +Jun: QCon SF proposal submitted (P3) +Jul: First cold emails sent (P5) +Aug: ICSE 2027 deadline (P2) +Sep: First consulting engagement (P5) +Late 2026: Conference submissions for 2027 cycle (P2) +``` + +--- + +## Decision Framework + +**If Show HN gets strong traction (>200 upvotes):** +→ Prioritize blog series, community building, open source contributions +→ Defer consulting — let inbound interest come naturally + +**If Show HN gets modest traction (<50 upvotes):** +→ Still publish arXiv + blog for credibility +→ Proceed to cold email for consulting (validates need regardless of HN) + +**If Scott can't get to Show HN by May 20:** +→ Skip it — publish arXiv first, blog series second +→ Show HN can happen anytime, but momentum matters + +--- + +## What NOT to Do + +- ❌ Don't rush a paper to a 2026 venue with a late deadline — a rejected rushed paper is worse than a polished 2027 submission +- ❌ Don't cold email without at least one public artifact circulating — credibility gap +- ❌ Don't build more compiler features before distribution — the compiler works well enough to demonstrate the concept +- ❌ Don't start a new venture — Speckl is the highest-ROI path to revenue diff --git a/docs/extraction-protocol.md b/docs/extraction-protocol.md new file mode 100644 index 0000000..f19cb36 --- /dev/null +++ b/docs/extraction-protocol.md @@ -0,0 +1,1460 @@ +# Speckl Extraction Protocol — Design Document + +## Overview + +The Extraction Layer bridges chaotic human/agent conversation and the Structured Intermediate Representation (SIR). It continuously (or batch) identifies formal elements — entities, transitions, constraints, relationships — and populates the SIR with typed, versioned, provenance-tracked records. + +Every element in the SIR traces back to the conversation turn(s) that produced it. Every element carries a confidence score. And every element can be refined incrementally as the conversation evolves. + +--- + +## 1. Core Data Structures + +### 1.1 ConversationTurn + +The raw input to the extraction layer. + +```typescript +interface ConversationTurn { + id: string; // UUID, globally unique + timestamp: number; // Unix ms + speaker: SpeakerRole; // "human" | "agent" + agentId?: string; // e.g. "speckl-extractor", "senior-eng" + content: string; // Raw text or structured block + parentTurnId?: string; // For threaded replies + metadata?: { + explicitMarkers?: string[]; // e.g. ["spec:", "state:"] + structuredBlock?: { // Extracted by pre-parser + type: "spec" | "code" | "diagram"; + language?: string; + content: string; + }; + }; +} + +type SpeakerRole = "human" | "agent"; +``` + +### 1.2 SIRElement + +The core unit of structured representation. All SIR elements share a common header and then specialize by domain. + +```typescript +// Common header for every SIR element +interface SIRHeader { + id: string; // UUID, stable across revisions + elementType: SIRType; // Domain-specific type discriminator + domain: DomainId; // "state-machine" | "data-model" | etc. + confidence: number; // 0.0 – 1.0 + status: ElementStatus; // Lifecycle state + createdAt: number; // Unix ms + updatedAt: number; // Last modification + version: number; // Monotonic counter, bumped on mutation + provenance: ProvenanceChain; // Full trace + flags: SIRFlags; // See below +} + +type SIRType = string; // e.g. "state", "transition", "guard", "event" +type DomainId = string; // e.g. "state-machine", "data-model" + +type ElementStatus = + | "proposed" // Extracted, never confirmed by human + | "confirmed" // Human explicitly confirmed + | "deprecated" // Later conversation contradicted this + | "replaced" // Superseded by another element (holds replacedById) + | "error"; // Extraction was wrong, human flagged it + +interface SIRFlags { + needsReview: boolean; // True when confidence < autoAcceptThreshold + isAmbiguous: boolean; // Multiple possible interpretations + isTentative: boolean; // Extracted from implicit cues, not explicit markers + isContradicted: boolean; // Later conversation conflicts + sourceIsExplicit: boolean; // Came from spec: prefix or structured block +} +``` + +### 1.3 ProvenanceChain + +Every SIR element has a complete, immutable provenance trail. + +```typescript +interface ProvenanceChain { + entries: ProvenanceEntry[]; +} + +interface ProvenanceEntry { + turnId: string; // ConversationTurn that produced/modified this + action: ProvenanceAction; // What happened in this turn + agentId?: string; // Which agent performed the action + humanConfirmed: boolean; // Did the human explicitly confirm? + timestamp: number; + confidenceAtTime: number; // Snapshot of confidence when entry was created + extractionRationale?: string; // Why the extractor thought this was relevant + diff?: ElementDiff; // For "refined" or "updated" actions +} + +type ProvenanceAction = + | "extracted" // First creation + | "refined" // Incremental update, same element + | "confirmed" // Human confirmed + | "deprecated" // Marked as no longer valid + | "replaced" // Superseded by new element + | "merged" // Two elements combined + | "split"; // One element split into multiple +``` + +### 1.4 ElementDiff + +Captures what changed between versions of a SIR element. + +```typescript +interface ElementDiff { + fromVersion: number; + toVersion: number; + triggerTurnId: string; + changes: FieldChange[]; +} + +interface FieldChange { + field: string; // Path, e.g. "transition.guard", "state.name" + oldValue: any; + newValue: any; + reason: string; // Why this changed, extracted from turn +} +``` + +### 1.5 Domain-Specific Element Examples + +State machine domain specialization: + +```typescript +// A state in a state machine +interface StateElement extends SIRHeader { + elementType: "state"; + domain: "state-machine"; + data: { + name: string; // e.g. "idle" + parent?: string; // For hierarchical states + isInitial?: boolean; + isFinal?: boolean; + entryActions?: string[]; + exitActions?: string[]; + invariants?: string[]; // Properties that must hold in this state + }; +} + +// A transition between states +interface TransitionElement extends SIRHeader { + elementType: "transition"; + domain: "state-machine"; + data: { + from: string; // State name or element ID + to: string; // State name or element ID + event: string; // Triggering event + guard?: string; // Condition that must be true + actions?: string[]; // Actions fired on transition + priority?: number; // For conflict resolution + }; +} +``` + +--- + +## 2. Extraction Modes + +The extraction layer supports four modes, selected per-turn or configured globally. + +### 2.1 Real-Time Extraction + +Extraction runs incrementally as each message arrives, before the next speaker responds. + +``` +Sequence: + Human sends turn → [pre-parser] → [extractor] → SIR updated → Agent responds +``` + +**Use case:** Live specification sessions where the human is describing a system and the agent provides immediate feedback ("I've captured state 'loading'. Did you mean it has a timeout transition?"). + +**Implementation:** + +```typescript +async function extractRealtime(turn: ConversationTurn, sir: SIRDocument): Promise { + // 1. Pre-parse for explicit markers + const markers = detectExplicitMarkers(turn); + + // 2. If markers found, do targeted extraction + if (markers.length > 0) { + return extractFromMarkers(turn, markers, sir); + } + + // 3. Otherwise, run lightweight pattern matching + const candidates = runLightweightPatterns(turn, sir.domain); + + // 4. Only auto-extract high-confidence candidates + const highConf = candidates.filter(c => c.confidence >= AUTO_ACCEPT_THRESHOLD); + const lowConf = candidates.filter(c => c.confidence < AUTO_ACCEPT_THRESHOLD && c.confidence >= FLAG_THRESHOLD); + + // 5. Apply high-confidence, flag low-confidence for review + const applied = applyExtractions(highConf, sir, turn, false); + const flagged = flagForReview(lowConf, sir, turn); + + return { applied, flagged, candidates: lowConf }; +} +``` + +### 2.2 Batch Extraction + +Extraction runs after a segment of conversation (a "batch" of N turns, or a pause in conversation). + +``` +Sequence: + [Turn 1] → [Turn 2] → [Turn 3] → (batch trigger) → [extractor] → SIR updated +``` + +**Use case:** After a multi-turn discussion where the human brainstormed possibilities and you want to extract the settled decisions. + +**Implementation:** + +```typescript +async function extractBatch( + turns: ConversationTurn[], + sir: SIRDocument, + options?: BatchOptions +): Promise { + // 1. Concatenate turns into a coherent transcript + const transcript = buildTranscript(turns); + + // 2. Run the full LLM-based extractor over the transcript + const extraction = await llmExtract( + transcript, + sir.domain, + sir.getExistingElements() // Pass current SIR for refinement context + ); + + // 3. Diff against existing SIR elements + const { added, refined, deprecated, conflicts } = diffAgainstSIR(extraction, sir); + + // 4. Apply changes with full provenance + // ... +} + +interface BatchOptions { + deduplicate: boolean; // Merge near-duplicate extractions + summarize: boolean; // Generate human-readable summary of changes + confirmBeforeApply: boolean; // Show diff to human before applying +} +``` + +### 2.3 Deliberative Extraction + +Multiple agents debate before elements are committed. This is the highest-quality mode. + +``` +Sequence: + Human sends turn + → [Agent A proposes extraction] + → [Agent B critiques] + → [Agent C synthesizes] + → [Consensus extraction] → SIR updated +``` + +**Implementation:** + +```typescript +async function extractDeliberative( + turn: ConversationTurn, + sir: SIRDocument, + debaters: DebaterConfig[] +): Promise { + const proposals: ExtractionProposal[] = []; + + // Phase 1: Each debater independently extracts + for (const debater of debaters) { + const proposal = await debater.extract(turn, sir); + proposals.push(proposal); + } + + // Phase 2: Cross-critique + const critiques: Critique[] = []; + for (let i = 0; i < proposals.length; i++) { + for (let j = 0; j < proposals.length; j++) { + if (i === j) continue; + const critique = await debaters[j].critique(proposals[i], turn, sir); + critiques.push(critique); + } + } + + // Phase 3: Synthesize + const synthesizer = debaters.find(d => d.role === "synthesizer") || debaters[0]; + const consensus = await synthesizer.synthesize(proposals, critiques, turn, sir); + + // Phase 4: Apply with deliberation provenance + return applyDeliberated(consensus, proposals, critiques, sir, turn); +} + +interface DebaterConfig { + id: string; + role: "proposer" | "critic" | "synthesizer"; + model?: string; + extract: (turn: ConversationTurn, sir: SIRDocument) => Promise; + critique: (proposal: ExtractionProposal, turn: ConversationTurn, sir: SIRDocument) => Promise; + synthesize: (proposals: ExtractionProposal[], critiques: Critique[], turn: ConversationTurn, sir: SIRDocument) => Promise; +} +``` + +### 2.4 Interactive Extraction + +When extraction ambiguity exceeds a threshold, the extractor asks clarifying questions rather than guessing. + +``` +Sequence: + Human sends turn + → [extractor] identifies ambiguity + → [extractor] generates clarifying question + → Human answers + → [extractor] resolves ambiguity → SIR updated +``` + +**Implementation:** + +```typescript +async function extractInteractive( + turn: ConversationTurn, + sir: SIRDocument +): Promise { + const candidates = await extractCandidates(turn, sir); + + // Identify ambiguities + const ambiguities = detectAmbiguities(candidates); + + if (ambiguities.length > 0) { + // Generate the best clarifying question + const question = await generateClarifyingQuestion(ambiguities, turn); + + // Store pending ambiguity so resolution can be matched later + sir.pendingAmbiguities.push({ + id: generateId(), + sourceTurnId: turn.id, + candidates: candidates.map(c => c.id), + ambiguities, + question, + timestamp: Date.now(), + }); + + return { + status: "clarification_needed", + question, + ambiguities, + }; + } + + // No ambiguity — proceed with auto-extraction + return applyExtractions(candidates, sir, turn); +} +``` + +--- + +## 3. Confidence and Ambiguity Handling + +### 3.1 Confidence Thresholds + +```typescript +// Global thresholds, domain-overridable +const CONFIDENCE = { + // >= this: auto-extract, mark confirmed + AUTO_ACCEPT: 0.85, + + // >= this: extract but mark needsReview=true, isTentative=true + // Agent should surface these to the human + REVIEW: 0.60, + + // >= this: extract to a "candidate" pool, do NOT add to main SIR + // Agent only surfaces if asked + CANDIDATE: 0.40, + + // < this: discard (noise) + DISCARD: 0.40, +}; +``` + +### 3.2 Confidence Scoring Algorithm + +Confidence is computed from multiple signals: + +```typescript +function computeConfidence( + candidate: ExtractionCandidate, + turn: ConversationTurn, + context: ExtractionContext +): number { + let score = 0.0; + const weights = context.domainWeights; + + // Signal 1: Explicit markers (strongest) + if (candidate.fromExplicitMarker) score += weights.explicitMarker * 1.0; // e.g. 0.40 + + // Signal 2: Structured block (code fence, diagram) + if (candidate.fromStructuredBlock) score += weights.structuredBlock * 1.0; // e.g. 0.35 + + // Signal 3: Linguistic pattern match strength + score += candidate.patternMatchScore * weights.patternMatch; // e.g. 0.25 + + // Signal 4: Consistency with existing SIR + const consistencyScore = checkConsistency(candidate, context.existingElements); + score += consistencyScore * weights.consistency; // e.g. 0.15 + + // Signal 5: Repetition — same extraction from multiple turns + if (candidate.repetitionCount > 1) { + score += Math.min(candidate.repetitionCount * 0.05, 0.15); + } + + // Signal 6: Speaker authority + if (turn.speaker === "human") score += 0.05; + + // Cap at 1.0 + return Math.min(score, 1.0); +} +``` + +### 3.3 Ambiguity Representation in the SIR + +When multiple interpretations exist for the same conversation fragment: + +```typescript +interface AmbiguousElement extends SIRHeader { + isAmbiguous: true; + interpretations: Interpretation[]; + resolvedInterpretationId?: string; // Set once resolved +} + +interface Interpretation { + id: string; + element: SIRElement; // The proposed element + confidence: number; + rationale: string; + proposedBy: string; // Agent ID +} +``` + +**Example:** The human says "when the user logs in, go to the main screen." + +``` +Extraction yields two interpretations: + +Interpretation A (confidence: 0.72): + "login" is a state, "main screen" is a state, there's a transition "login → main screen" + +Interpretation B (confidence: 0.65): + "logs in" is an event, the transition fires on "login" event from any state → "main screen" +``` + +The extractor creates an `AmbiguousElement` with both interpretations and flags it for review. The agent surfaces: *"I captured a transition after login. Did you mean (A) login is a state that transitions to main screen, or (B) the login event triggers a transition from any state?"* + +### 3.4 Conflict Resolution + +When later conversation contradicts an earlier extraction: + +```typescript +function resolveConflict( + existing: SIRElement, + incoming: ExtractionCandidate, + provenance: ProvenanceChain +): ConflictResolution { + // Case 1: Incoming confidence >> existing confidence + // → Deprecate old, create new, link them + if (incoming.confidence > existing.confidence + 0.2) { + return { + action: "replace", + existingElement: markDeprecated(existing, incoming.sourceTurnId), + newElement: createFromCandidate(incoming, { + replacedId: existing.id, + version: 1, + }), + rationale: `Higher-confidence extraction replaces previous`, + }; + } + + // Case 2: Comparable confidence → flag for human resolution + if (Math.abs(incoming.confidence - existing.confidence) < 0.2) { + return { + action: "flag_conflict", + existingElement: setFlag(existing, "isContradicted", true), + pendingResolution: { + existingId: existing.id, + incomingCandidate: incoming, + question: generateResolutionQuestion(existing, incoming), + }, + }; + } + + // Case 3: Incoming confidence << existing → discard incoming + return { + action: "reject", + reason: `Incoming confidence (${incoming.confidence}) too low vs existing (${existing.confidence})`, + }; +} +``` + +--- + +## 4. Provenance Model + +### 4.1 Full Traceability + +Every SIR element **must** trace back to one or more conversation turns. This is non-negotiable — no element enters the SIR without provenance. + +``` +Element → ProvenanceChain → ProvenanceEntry[] → ConversationTurn +``` + +Each entry records: +- **Which turn** triggered the change +- **Which agent** proposed it (if multi-agent) +- **Whether the human confirmed** (explicitly or implicitly) +- **What changed** (the diff) +- **Why** (extraction rationale) + +### 4.2 Multi-Turn Refinement Provenance + +When an element gets refined across multiple turns, the chain grows: + +``` +Turn 3: Human: "the idle state has a timeout of 30s" + → ProvenanceEntry { action: "extracted", element: State(idle), confidence: 0.70 } + +Turn 7: Human: "actually, make that 60 seconds" + → ProvenanceEntry { action: "refined", diff: { timeout: 30→60 }, confidence: 0.95 } + +Turn 12: Human: "spec: state idle { timeout: 120s, onTimeout: → error }" + → ProvenanceEntry { action: "refined", diff: { timeout: 60→120, +onTimeout }, confidence: 0.98 } +``` + +The provenance chain is **append-only**. Old entries are never modified — they are the historical record. + +### 4.3 Source Attribution + +Every extraction records who (or what) proposed it: + +```typescript +// Attribution taxonomy +type Attributor = + | { type: "human"; speakerId: string } // Direct human statement + | { type: "agent"; agentId: string; model?: string } // Agent extraction + | { type: "system"; component: string } // Automated pattern match + | { type: "consensus"; agents: string[] }; // Multi-agent agreement +``` + +### 4.4 Provenance Queries + +The provenance model supports queries like: +- "Show all elements the human hasn't confirmed yet" → `WHERE NOT humanConfirmed` +- "What did agent X propose that got rejected?" → `WHERE agentId = X AND action = 'deprecated'` +- "Trace the full evolution of state 'loading'" → `WHERE elementId = loading.id ORDER BY version` +- "Which turns contributed to more than 3 elements?" → `GROUP BY turnId HAVING COUNT > 3` + +--- + +## 5. Extraction Triggers + +### 5.1 Explicit Markers + +The strongest triggers. These produce high-confidence extractions immediately. + +| Marker | Example | Action | +|--------|---------|--------| +| `spec:` prefix | `spec: state idle { entry: resetTimer() }` | Parse the structured spec snippet directly | +| Fenced code block with language tag | ` ```speckl state idle { ... } ``` ` | Parse as Speckl micro-syntax | +| `state:` prefix | `state: loading` | Extract state element | +| `transition:` prefix | `transition: idle → loading on START` | Extract transition element | +| `event:` prefix | `event: USER_LOGIN { payload: { userId } }` | Extract event definition | +| `constraint:` prefix | `constraint: no transition from shutdown to any state` | Extract invariant | + +**Pre-parser for explicit markers:** + +```typescript +function detectExplicitMarkers(turn: ConversationTurn): MarkerDetection[] { + const markers: MarkerDetection[] = []; + + // Check message prefix + for (const marker of REGISTERED_MARKERS) { + if (turn.content.trimStart().startsWith(marker.prefix)) { + markers.push({ + marker: marker.prefix, + content: turn.content.slice(marker.prefix.length).trim(), + confidence: 0.98, // Explicit markers are near-certain + }); + } + } + + // Check structured blocks + if (turn.metadata?.structuredBlock) { + markers.push({ + marker: "structured_block", + content: turn.metadata.structuredBlock.content, + language: turn.metadata.structuredBlock.language, + confidence: 0.95, + }); + } + + return markers; +} +``` + +### 5.2 Implicit Linguistic Patterns + +When no explicit markers exist, the extractor uses NLP patterns keyed to the active domain. + +**State machine domain patterns:** + +| Pattern | Example | Extraction | +|---------|---------|------------| +| "when [event], [action]" | "when the user clicks submit, validate the form" | Transition with event="click submit" | +| "if [condition] then [action]" | "if the balance is zero, show an error" | Guard condition on transition | +| "[noun] is a state" | "idle is the starting state" | State with isInitial=true | +| "goes from [A] to [B]" | "it goes from loading to ready" | Transition from→to | +| "while in [state], [invariant]" | "while in loading, the spinner is visible" | State invariant | +| "after [event], enter [state]" | "after timeout, enter the error state" | Transition on event | +| "can't [action] when [state]" | "can't submit when loading" | Negative constraint | + +**Implementation** as pluggable pattern sets: + +```typescript +interface ExtractionPattern { + id: string; + domain: DomainId; + regex: RegExp; // or NLP intent classifier + extract: (match: RegExpMatchArray, turn: ConversationTurn, sir: SIRDocument) => ExtractionCandidate; + baseConfidence: number; // Starting confidence before modifiers +} + +// Registered by domain +const STATE_MACHINE_PATTERNS: ExtractionPattern[] = [ + { + id: "transition-when", + domain: "state-machine", + regex: /when\s+(?:the\s+)?(.+?)(?:\s*,\s*|\s+then\s+)(.+)/i, + extract: (match, turn) => ({ + elementType: "transition", + data: { + event: match[1].trim(), + actions: [match[2].trim()], + }, + confidence: 0.55, + }), + baseConfidence: 0.55, + }, + { + id: "state-declaration", + domain: "state-machine", + regex: /(\w+)\s+is\s+(?:a|the)\s+(initial|starting|final|end|error)\s+state/i, + extract: (match) => ({ + elementType: "state", + data: { + name: match[1], + isInitial: ["initial", "starting"].includes(match[2]), + isFinal: ["final", "end"].includes(match[2]), + }, + confidence: 0.50, + }), + baseConfidence: 0.50, + }, + // ... more patterns +]; +``` + +### 5.3 Hybrid Trigger: The "Spec Block" + +A special trigger: when the human writes a spec-like block without the explicit `spec:` prefix, the extractor can still recognize the structure: + +``` +The system has these states: + +- IDLE: waiting for input, spinner hidden +- LOADING: fetching data, spinner visible +- ERROR: request failed, shows error toast +- SUCCESS: data rendered + +Transitions: +- IDLE → LOADING when user submits query +- LOADING → SUCCESS when data arrives +- LOADING → ERROR when request fails +- ERROR → IDLE when user dismisses toast +``` + +The extractor detects this as a "structured specification block" using heuristics: +- Bullet lists with `→` arrows +- Semicolon-separated key-value pairs +- Sections with headers like "States:" / "Transitions:" + +--- + +## 6. Incremental Refinement + +### 6.1 Refinement Protocol + +When a new conversation turn modifies an existing SIR element: + +```typescript +async function refineElement( + existingElement: SIRElement, + refinementCandidate: ExtractionCandidate, + sourceTurn: ConversationTurn +): Promise { + // 1. Compute the field-level diff + const diff = computeDiff(existingElement.data, refinementCandidate.data); + + // 2. Check if confidence drops below threshold after revision + const newConfidence = computeConfidence(refinementCandidate, sourceTurn, + { existingElement, domainWeights: getDomainWeights(existingElement.domain) } + ); + + // 3. If confidence dropped too far, flag instead of applying + if (newConfidence < CONFIDENCE.REVIEW) { + return { + action: "flag_revision_uncertain", + element: setFlag(existingElement, "needsReview", true), + message: `Refinement of '${existingElement.id}' would drop confidence to ${newConfidence}. Holding for review.`, + diff, + }; + } + + // 4. Apply the refinement — bump version, append provenance + const refined: SIRElement = { + ...existingElement, + data: { ...existingElement.data, ...refinementCandidate.data }, + confidence: newConfidence, + version: existingElement.version + 1, + updatedAt: Date.now(), + provenance: { + entries: [ + ...existingElement.provenance.entries, + { + turnId: sourceTurn.id, + action: "refined", + agentId: sourceTurn.agentId, + humanConfirmed: sourceTurn.speaker === "human", + timestamp: sourceTurn.timestamp, + confidenceAtTime: newConfidence, + diff, + }, + ], + }, + }; + + return { action: "applied", element: refined, diff }; +} +``` + +### 6.2 Version History + +Every element maintains its full version history. The SIR can reconstruct any element at any point in time: + +```typescript +function getElementAtVersion(element: SIRElement, targetVersion: number): SIRElement { + let current = element.data; + const entries = element.provenance.entries + .filter(e => e.version && e.version <= targetVersion) + .sort((a, b) => a.version! - b.version!); + + for (const entry of entries) { + if (entry.diff) { + for (const change of entry.diff.changes) { + // Reverse the change to reconstruct previous state + setNestedValue(current, change.field, change.oldValue); + } + } + } + + return { ...element, data: current, version: targetVersion }; +} +``` + +### 6.3 Confidence Decay on Revision + +When a human revises an element (especially implicitly), confidence should be recalculated, not just carried forward. The new confidence depends on: + +1. **Explicitness of the revision**: `spec:` prefix → 0.98; vague mention → 0.45 +2. **Consistency with related elements**: Does the revision make sense given transitions referencing this state? +3. **Completeness**: Did they revise one field but leave others potentially stale? + +--- + +## 7. Domain Awareness + +### 7.1 Domain Registration + +The extraction layer is domain-agnostic. Domain-specific extraction logic is registered as a **Domain Backend**: + +```typescript +interface DomainBackend { + domain: DomainId; + displayName: string; + + // What SIR element types does this domain use? + elementTypes: SIRType[]; + + // Extraction patterns for this domain + patterns: ExtractionPattern[]; + + // LLM extraction prompt template + extractionPrompt: (transcript: string, existingElements: SIRElement[]) => string; + + // How to validate an extracted element + validator: (element: SIRElement, sir: SIRDocument) => ValidationResult; + + // How to merge/dedup candidates + merger: (existing: SIRElement, incoming: ExtractionCandidate) => SIRElement | null; + + // Confidence weights for this domain + confidenceWeights: ConfidenceWeights; + + // Explicit marker prefixes this domain recognizes + markerPrefixes: string[]; +} + +interface ConfidenceWeights { + explicitMarker: number; // How much to weight explicit markers + structuredBlock: number; // How much to weight code fences + patternMatch: number; // How much to weight NLP pattern matches + consistency: number; // How much to weight consistency with existing SIR + speakerAuthority: number; // How much human vs agent matters + repetition: number; // How much repeated mention matters +} +``` + +### 7.2 Registering a New Domain + +```typescript +// Example: Registering the state-machine domain +extractionEngine.registerDomain({ + domain: "state-machine", + displayName: "State Machine", + elementTypes: ["state", "transition", "event", "guard", "action", "invariant"], + patterns: STATE_MACHINE_PATTERNS, + extractionPrompt: STATE_MACHINE_PROMPT, + validator: validateStateMachineElement, + merger: mergeStateMachineElement, + confidenceWeights: { + explicitMarker: 0.40, + structuredBlock: 0.35, + patternMatch: 0.25, + consistency: 0.15, + speakerAuthority: 0.05, + repetition: 0.10, + }, + markerPrefixes: ["spec:", "state:", "transition:", "event:", "guard:", "action:", "constraint:"], +}); +``` + +### 7.3 Domain Detection + +When the domain is ambiguous, the extraction layer heuristically detects it: + +```typescript +function detectDomain( + turns: ConversationTurn[], + registeredDomains: DomainBackend[] +): DomainDetectionResult { + const scores: Record = {}; + + for (const domain of registeredDomains) { + scores[domain.domain] = 0; + + for (const turn of turns) { + // Check explicit markers + for (const marker of domain.markerPrefixes) { + if (turn.content.includes(marker)) { + scores[domain.domain] += 5; + } + } + + // Check pattern matches + for (const pattern of domain.patterns) { + if (pattern.regex.test(turn.content)) { + scores[domain.domain] += 2; + } + } + + // Check keyword density + const keywords = getDomainKeywords(domain.domain); + for (const kw of keywords) { + const count = (turn.content.match(new RegExp(kw, 'gi')) || []).length; + scores[domain.domain] += count * 0.5; + } + } + } + + // Return best match, or "general" if no clear winner + const best = Object.entries(scores).sort((a, b) => b[1] - a[1])[0]; + + if (best[1] < 3) { + return { domain: "general", confidence: 0.3, scores }; + } + + return { + domain: best[0] as DomainId, + confidence: Math.min(best[1] / 20, 1.0), + scores, + }; +} +``` + +### 7.4 Default Fallback Domain + +When no domain is detected, the extraction layer operates in "general" mode: +- Extracts only `entity`, `relationship`, `property`, and `constraint` element types +- Uses generic NLP patterns (noun phrases as entities, verb phrases as relationships) +- Confidence is capped at 0.60 (tentative) +- Flags everything for review by default + +--- + +## 8. Practical API Design + +### 8.1 ExtractionEngine + +The central interface exposed to the Speckl agent: + +```typescript +class ExtractionEngine { + // === Configuration === + + /** Register a domain backend for extraction */ + registerDomain(backend: DomainBackend): void; + + /** Set the active domain (switches extraction patterns, prompts, etc.) */ + setActiveDomain(domain: DomainId): void; + + /** Configure extraction mode */ + setMode(mode: ExtractionMode): void; + + /** Override confidence thresholds for this session */ + setConfidenceThresholds(thresholds: Partial): void; + + // === Core Operations === + + /** + * Extract from a single conversation turn. + * Entry point for real-time and interactive modes. + */ + extract(turn: ConversationTurn): Promise; + + /** + * Extract from a batch of turns. + * Entry point for batch mode. + */ + extractBatch(turns: ConversationTurn[], options?: BatchOptions): Promise; + + /** + * Run deliberative extraction with multiple agent perspectives. + */ + extractDeliberative( + turn: ConversationTurn, + debaters: DebaterConfig[] + ): Promise; + + /** + * Get the current SIR document. + */ + getSIR(): SIRDocument; + + /** + * Get extraction statistics for this session. + */ + getStats(): ExtractionStats; +} + +type ExtractionMode = "realtime" | "batch" | "deliberative" | "interactive"; +``` + +### 8.2 ExtractionResult + +The return type from every extraction call: + +```typescript +interface ExtractionResult { + /** Unique ID for this extraction run */ + extractionId: string; + + /** Which mode was used */ + mode: ExtractionMode; + + /** Source turn(s) that triggered this extraction */ + sourceTurnIds: string[]; + + /** Elements that were successfully added to the SIR */ + added: SIRMutation[]; + + /** Existing elements that were refined (with diffs) */ + refined: SIRMutation[]; + + /** Elements that were deprecated or replaced */ + deprecated: SIRMutation[]; + + /** Candidates that didn't meet the auto-accept threshold */ + flagged: FlaggedCandidate[]; + + /** Ambiguities detected during extraction */ + ambiguities: ExtractionAmbiguity[]; + + /** Conflicts found between new extractions and existing elements */ + conflicts: ElementConflict[]; + + /** If mode is "interactive", clarification questions */ + clarifications?: ClarificationRequest[]; + + /** Human-readable summary of what changed */ + summary: string; + + /** Timestamp */ + timestamp: number; +} + +interface SIRMutation { + elementId: string; + element: SIRElement; + action: "created" | "updated" | "deprecated" | "replaced"; + diff?: ElementDiff; + provenance: ProvenanceEntry; +} + +interface FlaggedCandidate { + candidateId: string; + candidate: ExtractionCandidate; + reason: "low_confidence" | "ambiguous" | "conflicts"; + confidence: number; + suggestedQuestion?: string; +} + +interface ExtractionAmbiguity { + id: string; + description: string; + interpretations: Interpretation[]; + sourceTurnId: string; + sourceFragment: string; // The part of the turn that's ambiguous +} + +interface ElementConflict { + existingElementId: string; + incomingCandidate: ExtractionCandidate; + conflictType: "value_mismatch" | "semantic_contradiction" | "redundant"; + description: string; + suggestedResolution: "replace" | "merge" | "reject" | "ask_human"; +} + +interface ClarificationRequest { + id: string; + question: string; + context: string; + options?: string[]; // If multiple-choice, provide options + elementIds: string[]; // Which elements this clarification would affect +} +``` + +### 8.3 SIRDocument + +The overall SIR that the extraction layer populates: + +```typescript +class SIRDocument { + /** All elements keyed by ID */ + elements: Map; + + /** Human-readable project name */ + projectName: string; + + /** Active domain(s) */ + domains: DomainId[]; + + /** Metadata about the conversation session */ + session: { + sessionId: string; + startedAt: number; + turnCount: number; + participants: string[]; + }; + + /** Pending ambiguity resolutions (for interactive mode) */ + pendingAmbiguities: PendingAmbiguity[]; + + /** Pending human confirmations */ + pendingConfirmations: PendingConfirmation[]; + + // === Query Methods === + + /** Get all elements of a given type in a domain */ + getElementsByType(domain: DomainId, type: SIRType): SIRElement[]; + + /** Get all unconfirmed elements */ + getUnconfirmed(): SIRElement[]; + + /** Get all elements that need review */ + getNeedsReview(): SIRElement[]; + + /** Get element version history */ + getHistory(elementId: string): ProvenanceEntry[]; + + /** Reconstruct element at a specific version */ + getElementAtVersion(elementId: string, version: number): SIRElement; + + /** Find elements contributed by a specific turn */ + getElementsByTurn(turnId: string): SIRElement[]; + + /** Validate the entire SIR for consistency */ + validate(): ValidationReport; + + /** Serialize SIR to a stable format (for compilation) */ + serialize(): string; + + /** Generate a human-readable summary of the spec so far */ + summarize(): string; +} +``` + +### 8.4 Usage Example: Full Flow + +```typescript +// --- Session setup --- +const engine = new ExtractionEngine(); + +// Register the state machine domain +engine.registerDomain(stateMachineBackend); +engine.setActiveDomain("state-machine"); +engine.setMode("realtime"); + +const sir = engine.getSIR(); + +// --- Turn 1: Human describes the system --- +const turn1: ConversationTurn = { + id: "turn-001", + timestamp: Date.now(), + speaker: "human", + content: "The system starts in an idle state. When the user clicks 'Search', it goes to a loading state.", +}; + +const result1 = await engine.extract(turn1); +console.log(result1.summary); +// → "Added 2 states (idle, loading) and 1 transition (idle→loading on 'Search') with low confidence." +console.log(result1.added.length); // → 3 +console.log(result1.added[0].element.confidence); // → ~0.55 (pattern match) +console.log(result1.added[0].element.flags.isTentative); // → true + +// --- Turn 2: Human gets more specific --- +const turn2: ConversationTurn = { + id: "turn-002", + timestamp: Date.now(), + speaker: "human", + content: "spec: state idle { isInitial: true, entry: [hideSpinner] }", +}; + +const result2 = await engine.extract(turn2); +console.log(result2.summary); +// → "Refined state 'idle': added isInitial, entry action. Confidence: 0.98." +console.log(result2.refined[0].element.confidence); // → 0.98 +console.log(result2.refined[0].element.flags.isTentative); // → false (explicit marker resolved it) + +// --- Turn 3: Ambiguity --- +const turn3: ConversationTurn = { + id: "turn-003", + timestamp: Date.now(), + speaker: "human", + content: "After authentication, go to the dashboard. But what about failed auth?", +}; + +// Switch to interactive mode for this turn +engine.setMode("interactive"); + +const result3 = await engine.extract(turn3); +// result3.clarifications = [{ +// question: "Is 'authentication' a state or an event? Should 'dashboard' be a state?", +// options: [ +// "authentication is a state, dashboard is a state, transition between them", +// "authentication is an event, dashboard is a state, transition on auth event", +// "authentication is a guard condition on the transition to dashboard" +// ] +// }] + +// --- End of session --- +const finalSIR = sir.serialize(); +const stats = engine.getStats(); +console.log(stats); +// → { +// totalElements: 5, +// confirmedElements: 2, +// tentativeElements: 2, +// pendingAmbiguities: 1, +// totalTurns: 3, +// avgConfidence: 0.67, +// domainConfidence: 0.95, +// } +``` + +--- + +## 9. Extraction Pipeline Architecture + +### 9.1 Layered Processing + +The extraction pipeline processes each turn through a series of filters: + +``` +ConversationTurn + │ + ▼ +┌─────────────────────┐ +│ 1. Pre-processor │ ← Detect explicit markers, structured blocks +│ (deterministic) │ +└─────────┬───────────┘ + │ annotated turn + ▼ +┌─────────────────────┐ +│ 2. Pattern Matcher │ ← Regex/NLP patterns from registered domain +│ (deterministic) │ +└─────────┬───────────┘ + │ ExtractionCandidate[] + ▼ +┌─────────────────────┐ +│ 3. LLM Extractor │ ← LLM call for deep semantic extraction +│ (probabilistic) │ (skipped if patterns found everything) +└─────────┬───────────┘ + │ ExtractionCandidate[] (merged) + ▼ +┌─────────────────────┐ +│ 4. Confidence │ ← Score each candidate +│ Scorer │ +└─────────┬───────────┘ + │ + ▼ +┌─────────────────────┐ +│ 5. Conflict │ ← Diff against existing SIR elements +│ Detector │ +└─────────┬───────────┘ + │ + ▼ +┌─────────────────────┐ +│ 6. Ambiguity │ ← Detect multiple interpretations +│ Detector │ +└─────────┬───────────┘ + │ + ▼ +┌─────────────────────┐ +│ 7. Decision Engine │ ← Apply thresholds → add / refine / flag / ask +│ │ +└─────────┬───────────┘ + │ + ▼ + ExtractionResult +``` + +### 9.2 Pipeline Configuration + +```typescript +interface PipelineConfig { + /** Skip LLM extraction when pattern matcher confidence is high enough */ + skipLLMWhenPatternConfidenceAbove: number; // default: 0.80 + + /** Maximum tokens to send to LLM extractor */ + maxLLMTokens: number; // default: 4000 + + /** Whether to cache LLM extraction results */ + cacheLLMExtractions: boolean; + + /** When to escalate to interactive mode */ + autoSwitchToInteractive: "on_ambiguity" | "on_low_confidence" | "never"; + + /** Minimum number of turns before batch extraction is meaningful */ + minBatchSize: number; // default: 3 +} +``` + +--- + +## 10. Edge Cases and Error Handling + +### 10.1 Silent Failures Must Be Impossible + +Every error path in the extraction pipeline produces an observable result: + +| Failure | Behavior | +|---------|----------| +| LLM API timeout | Falls back to pattern-only extraction, sets `result.metadata.degraded = true`, confidence capped at pattern match scores | +| Domain not registered | Falls back to "general" domain, all extractions flagged `isTentative` | +| Turn content is noise ("lol", "ok") | Returns empty ExtractionResult with `summary: "No formal elements detected"` | +| SIR element ID collision | Deterministic ID generation (content-hash based), so collisions mean true duplicates → merged | +| Human contradicts self within same turn | Extracts both, creates AmbiguousElement, forces interactive resolution | +| Agent hallucinates an element | Provenance tracks it to agent; human review catches it; `status: "error"` when flagged | + +### 10.2 Recovery from Extraction Errors + +```typescript +/** Human marks an element as incorrectly extracted */ +function markExtractionError(elementId: string, humanTurnId: string, reason: string): void { + const element = sir.elements.get(elementId); + if (!element) throw new Error(`Element ${elementId} not found`); + + element.status = "error"; + element.flags.needsReview = false; // It's been reviewed — it's wrong + element.flags.isTentative = false; + + element.provenance.entries.push({ + turnId: humanTurnId, + action: "deprecated", + humanConfirmed: true, + timestamp: Date.now(), + confidenceAtTime: 1.0, // Human certainty + extractionRationale: `Marked as error: ${reason}`, + }); +} +``` + +--- + +## 11. Composition with Validation and Compilation Layers + +The extraction layer feeds into the Validation and Compilation layers through the SIR: + +``` +[Extraction Layer] ──produces──> SIRDocument ──consumed by──> [Validation Layer] ──produces──> ValidationReport + │ + ▼ + [Compilation Layer] +``` + +### 11.1 Contract with Validation Layer + +The extraction layer guarantees: +1. Every element has `id`, `elementType`, `domain`, `confidence`, `provenance` +2. No element enters the SIR without at least one provenance entry +3. `confidence` is always a number in [0.0, 1.0] +4. `version` is a monotonic integer ≥ 1 +5. `status` is one of the defined `ElementStatus` values + +The validation layer checks: +- Domain-specific invariants (e.g., every transition references existing states) +- Cross-element consistency (e.g., no unreachable states) +- Completeness (e.g., at least one initial state) + +### 11.2 Contract with Compilation Layer + +The compilation layer receives the validated SIR and produces deterministic output. Because of provenance tracking, compilation errors can surface as: *"Element 'transition-7' (from turn #12, extracted by agent speckl-extractor) references non-existent state 'foo'. This state was deprecated in turn #15."* + +--- + +## 12. Open Design Questions + +These are intentionally left open for later resolution: + +1. **LLM cost/performance tradeoff**: Should pattern-matching be the default, with LLM extraction as a premium upgrade? Or should LLM extraction be the primary path with patterns as fallback? + +2. **Provenance storage**: Should the full provenance chain be stored inline in the SIR (increasing document size) or externalized with references? Inline simplifies queries but adds bulk. + +3. **Human-in-the-loop granularity**: Should the human confirm every extraction, or only low-confidence ones? What about implicit confirmation (human doesn't object for 3 turns = confirmed)? + +4. **Cross-domain elements**: If a conversation spans multiple domains (e.g., state machine + data model), how do elements reference each other across domain boundaries? + +5. **Streaming extraction**: For real-time mode, should extraction be truly streaming (extract as human types each word) or per-message? Per-word seems excessive; per-sentence might be the sweet spot. + +6. **Extraction caching**: If the same conversation fragment appears in multiple turns (e.g., the agent repeats what it heard), should we cache the extraction or re-extract? + +--- + +## Appendix A: State Machine Domain — Complete Pattern Set + +```typescript +const STATE_MACHINE_PATTERNS: ExtractionPattern[] = [ + // Transitions + { + id: "transition-explicit", + domain: "state-machine", + regex: /transition:\s*(\w+)\s*(?:-+>|→)\s*(\w+)\s*(?:on|when)\s+(.+)/i, + extract: (m) => ({ elementType: "transition", data: { from: m[1], to: m[2], event: m[3] }, confidence: 0.90 }), + baseConfidence: 0.90, + }, + { + id: "transition-arrow", + domain: "state-machine", + regex: /(\w+)\s*(?:-+>|→|goes to|transitions to)\s*(\w+)/i, + extract: (m) => ({ elementType: "transition", data: { from: m[1], to: m[2] }, confidence: 0.50 }), + baseConfidence: 0.50, + }, + + // States + { + id: "state-declaration", + domain: "state-machine", + regex: /(\w+)\s+is\s+(?:a|the)\s+(initial|starting|final|end|error|default)\s+state/i, + extract: (m) => ({ + elementType: "state", + data: { name: m[1], isInitial: m[2].match(/initial|starting|default/), isFinal: m[2].match(/final|end/) }, + confidence: 0.50, + }), + baseConfidence: 0.50, + }, + + // Guards + { + id: "guard-condition", + domain: "state-machine", + regex: /(?:only\s+)?if\s+(.+?)(?:\s*,\s*|\s+then\s+|$)/i, + extract: (m) => ({ elementType: "guard", data: { condition: m[1] }, confidence: 0.45 }), + baseConfidence: 0.45, + }, + + // Events + { + id: "event-definition", + domain: "state-machine", + regex: /event\s*:\s*(\w+)\s*(?:{([^}]*)}|with\s+(.+))?/i, + extract: (m) => ({ elementType: "event", data: { name: m[1], payload: m[2] || m[3] }, confidence: 0.85 }), + baseConfidence: 0.85, + }, + + // Invariants + { + id: "invariant-while", + domain: "state-machine", + regex: /(?:while|when)\s+in\s+(\w+)\s*,\s*(.+)/i, + extract: (m) => ({ elementType: "invariant", data: { state: m[1], property: m[2] }, confidence: 0.55 }), + baseConfidence: 0.55, + }, + + // Negative constraints + { + id: "negative-constraint", + domain: "state-machine", + regex: /(?:can't|cannot|must\s+not|shouldn't)\s+(.+?)\s+(?:when|while|in)\s+(\w+)/i, + extract: (m) => ({ elementType: "constraint", data: { forbidden: m[1], context: m[2] }, confidence: 0.50 }), + baseConfidence: 0.50, + }, +]; +``` + +## Appendix B: LLM Extraction Prompt Template + +``` +You are a specification extraction agent for the {domain} domain. + +Your task: Extract formal {domain} elements from the following conversation transcript. + +Current SIR elements (for refinement/reference): +{existingElements} + +Domain element types: {elementTypes} + +Rules: +1. For each element you extract, provide the exact conversation fragment that supports it. +2. Assign a confidence score (0.0-1.0) for each extraction. +3. If a statement is ambiguous, note the alternative interpretations. +4. If a new statement refines an existing SIR element, reference the element ID. +5. Do NOT invent elements not present in the conversation. + +Transcript: +{transcript} + +Output a JSON array of ExtractionCandidate objects. +``` + +--- + +_This document describes the complete Extraction Protocol for Speckl. It is designed for domain extensibility, deterministic provenance tracking, and graceful degradation under uncertainty. The state machine domain serves as the reference implementation and running example._ diff --git a/docs/kraft-protocol-reference.md b/docs/kraft-protocol-reference.md new file mode 100644 index 0000000..e071bb8 --- /dev/null +++ b/docs/kraft-protocol-reference.md @@ -0,0 +1,738 @@ +# Kafka KRaft Protocol Reference + +> Source material for writing `examples/KafkaKRaft.speck` (speckl#22) + +## Scope + +This reference covers the **consensus core** of KRaft — the state machine for leader election, log replication, and commit advancement. It does NOT cover: + +- Full broker functionality (topic creation, partition assignment, consumer group coordination) +- Data plane (produce/fetch request handling) +- Controller-specific logic (metadata quorums, configuration management) +- Network transport (this is a logical model, not a network protocol spec) + +The boundary: if it touches `currentTerm`, `votedFor`, `log`, `commitIndex`, or `state` transitions, it's in scope. + +**In scope (M1.1):** Core Raft consensus — leader election, log replication, commit advancement, and all 6 safety invariants, including KRaft's extensions (Unattached, Resigned, Prospective states, Pre-Vote, snapshots). + +**Deferred to M1.3:** Producer ack semantics, consumer group rebalancing, transaction coordinator. + +**Out of scope:** Full Kafka broker (produce/fetch handling), controller-specific logic, network layer. + +## Sources + +1. **Raft TLA+ Specification** — Diego Ongaro, 2014. The canonical formal spec. Available at: `github.com/ongardie/raft.tla` +2. **"In Search of an Understandable Consensus Algorithm"** — Ongaro & Ousterhout, 2014 (USENIX ATC). Includes safety proof. +3. **"Consensus: Bridging Theory and Practice"** — Ongaro, 2014 (PhD dissertation). Extended TLA+ spec, implementation concerns. +4. **KIP-595: A Raft Protocol for the Metadata Quorum** — The foundational KIP that introduced KRaft. `cwiki.apache.org/confluence/display/KAFKA/KIP-595` +5. **KIP-630: Kafka Raft Snapshot** — Snapshot mechanism for KRaft metadata log. Status: Adopted. +6. **KIP-996: Pre-Vote for KRaft** — Adds Pre-Vote phase to prevent disruptive servers. Status: Implementing. +7. **KIP-853: KRaft Controller Membership Changes** — Dynamic quorum membership changes. Status: Implemented. +8. **Apache Kafka source** — `kafka/raft/` package. Key files: `KafkaRaftClient.java`, `QuorumState.java`, `LeaderState.java`, `CandidateState.java`, `FollowerState.java`, `UnattachedState.java`, `ProspectiveState.java`, `ResignedState.java`, `EpochState.java` +9. **Red Hat Deep Dive** — "A Deep Dive into Apache Kafka's KRaft Protocol" (developers.redhat.com, 2025). Covers KRaft v1 based on Kafka 4.1.0. + +--- + +## 1. Raft State Machine (from TLA+ Spec) + +*Reference: Source [1], [3]* + +### 1.1 State Variables + +Per server (`i ∈ Server`): + +| Variable | Type | Description | +|----------|------|-------------| +| `currentTerm` | Nat | Current term number (monotonically increasing) | +| `state` | {Follower, Candidate, Leader} | Server role | +| `votedFor` | Server ∪ {Nil} | Candidate voted for in current term, or Nil | +| `log` | Seq(LogEntry) | Sequence of log entries (index = position) | +| `commitIndex` | Nat | Index of highest log entry known to be committed | +| `nextIndex` | Server → Nat | (Leader only) Next log index to send to each follower | +| `matchIndex` | Server → Nat | (Leader only) Highest log index replicated on each follower | +| `votesResponded` | Set(Server) | (Candidate only) Servers that responded to RequestVote | +| `votesGranted` | Set(Server) | (Candidate only) Servers that granted vote | + +Global: + +| Variable | Type | Description | +|----------|------|-------------| +| `messages` | Bag(Message) | In-flight messages (requests/responses) | + +### 1.2 LogEntry Type + +``` +LogEntry == { term: Nat, value: Value } +``` + +### 1.3 Quorum + +``` +Quorum == {i ∈ SUBSET(Server) : Cardinality(i) * 2 > Cardinality(Server)} +``` + +Simple majority. The key property: any two quorums overlap. + +### 1.4 Initial State + +``` +currentTerm[i] = 1 +state[i] = Follower +votedFor[i] = Nil +log[i] = <<>> (empty sequence) +commitIndex[i] = 0 +``` + +### 1.5 Actions (State Transitions) + +#### Action 1: Timeout → Start Election + +**Preconditions:** +- `state[i] ∈ {Follower, Candidate}` + +**Effects:** +- `state[i] ← Candidate` +- `currentTerm[i] ← currentTerm[i] + 1` +- `votedFor[i] ← Nil` +- `votesResponded[i] ← {}` +- `votesGranted[i] ← {}` + +#### Action 2: RequestVote Request (Candidate → Peer) + +**Preconditions:** +- `state[i] = Candidate` +- `j ∉ votesResponded[i]` + +**Effects:** +- Send `RequestVoteRequest` with `{mterm: currentTerm[i], mlastLogTerm: LastTerm(log[i]), mlastLogIndex: Len(log[i]), msource: i, mdest: j}` + +#### Action 3: HandleRequestVoteRequest (Peer → Candidate) + +**Preconditions:** +- `m.mterm ≤ currentTerm[i]` + +**Log check (`logOk`):** +``` +logOk ≡ m.mlastLogTerm > LastTerm(log[i]) + ∨ (m.mlastLogTerm = LastTerm(log[i]) ∧ m.mlastLogIndex ≥ Len(log[i])) +``` + +**Grant conditions:** +``` +grant ≡ m.mterm = currentTerm[i] ∧ logOk ∧ votedFor[i] ∈ {Nil, j} +``` + +**Effects (if grant):** +- `votedFor[i] ← j` + +**Effects (always):** +- Reply with `RequestVoteResponse{mterm: currentTerm[i], mvoteGranted: grant}` + +#### Action 4: HandleRequestVoteResponse (Candidate receives vote) + +**Preconditions:** +- `m.mterm = currentTerm[i]` + +**Effects:** +- `votesResponded[i] ← votesResponded[i] ∪ {j}` +- If `m.mvoteGranted`: `votesGranted[i] ← votesGranted[i] ∪ {j}` + +#### Action 5: BecomeLeader (Candidate → Leader) + +**Preconditions:** +- `state[i] = Candidate` +- `votesGranted[i] ∈ Quorum` + +**Effects:** +- `state[i] ← Leader` +- `nextIndex[i] ← [j ∈ Server → Len(log[i]) + 1]` for all j +- `matchIndex[i] ← [j ∈ Server → 0]` for all j + +#### Action 6: AppendEntries Request (Leader → Follower) + +**Preconditions:** +- `i ≠ j` +- `state[i] = Leader` + +**Effects:** +- Send `AppendEntriesRequest` with entries from `nextIndex[i][j]` to `min(Len(log[i]), nextIndex[i][j])` (at most 1 entry per message in the spec) +- Include `mprevLogIndex = nextIndex[i][j] - 1`, `mprevLogTerm`, `mcommitIndex` + +#### Action 7: HandleAppendEntriesRequest (Follower receives entries) + +**Preconditions:** +- `m.mterm ≤ currentTerm[i]` + +**Log consistency check (`logOk`):** +``` +logOk ≡ m.mprevLogIndex = 0 + ∨ (m.mprevLogIndex ≤ Len(log[i]) ∧ m.mprevLogTerm = log[i][m.mprevLogIndex].term) +``` + +**Effects (if logOk AND m.mterm ≥ currentTerm[i]):** +- Reset to Follower if Candidate +- Update `currentTerm[i] ← m.mterm` if needed +- Append/conflict resolution: if `mprevLogIndex > 0`, entries after `mprevLogIndex` that conflict are truncated +- `commitIndex[i] ← min(m.mcommitIndex, Len(log[i]))` +- Reply `AppendEntriesResponse{msuccess: TRUE, mmatchIndex: ...}` + +**Effects (if !logOk OR m.mterm < currentTerm[i]):** +- Reply `AppendEntriesResponse{msuccess: FALSE, mmatchIndex: 0}` + +#### Action 8: AdvanceCommitIndex (Leader) + +**Preconditions:** +- `state[i] = Leader` + +**Effects:** +- Find highest `N` such that `N > commitIndex[i]` and a quorum of `matchIndex[i][k] ≥ N` +- Only commit entries from current term: `log[i][N].term = currentTerm[i]` +- `commitIndex[i] ← N` + +#### Action 9: ClientRequest (Leader appends entry) + +**Preconditions:** +- `state[i] = Leader` + +**Effects:** +- Append `[term: currentTerm[i], value: v]` to `log[i]` + +#### Action 10: Restart (Server crash recovery) + +**Effects:** +- `state[i] ← Follower` +- Reset volatile state (`votesResponded`, `votesGranted`, `nextIndex`, `matchIndex`, `commitIndex`) +- Preserve persistent state (`currentTerm`, `votedFor`, `log`) + +--- + +## 2. KRaft Extensions Over Vanilla Raft + +*References: Sources [4], [5], [6], [7], [8], [9]* + +KRaft is Kafka's adaptation of Raft for metadata management. Key differences from vanilla Raft: + +1. **No ZooKeeper:** KRaft replaces the ZooKeeper-based metadata quorum with a Raft-based one. +2. **Metadata log:** The replicated log contains metadata records (topic configs, partition assignments), not user data. +3. **Controller node:** KRaft has an "active controller" concept — the leader of the metadata quorum acts as the controller for the Kafka cluster. +4. **Snapshotting:** KRaft supports snapshot-based log compaction (KIP-630). +5. **Voters and observers:** Not all nodes are voters; some are observer nodes that replicate but don't vote. +6. **Resignation:** A leader can voluntarily step down (immediate or graceful). +7. **Pre-Vote:** KRaft implements Pre-Vote (KIP-996) to prevent disruptive servers from forcing unnecessary elections. +8. **Epochs instead of terms:** KRaft uses "epoch" terminology instead of "term" (functionally equivalent). +9. **Additional states:** KRaft adds Unattached, Prospective, Resigned, and Detached states beyond Follower/Candidate/Leader. + +### 2.1 KRaft State Machine + +The KRaft state machine is implemented in `QuorumState.java`. Unlike vanilla Raft's three-state model (Follower/Candidate/Leader), KRaft has a richer set of epoch-based states, all implementing the `EpochState` interface. + +#### State Hierarchy + +``` +EpochState (interface) +├── UnattachedState — Known epoch, unknown leader (initial startup or learned higher epoch) +├── ProspectiveState — Pre-Vote candidate; gathering pre-votes before becoming Candidate +├── CandidateState — Standard Raft candidate; actively requesting votes +├── FollowerState — Standard Raft follower; replicating from a leader +├── LeaderState — Standard Raft leader; accepting client writes +├── ResignedState — Gracefully stepped-down leader; still processing resignations +└── DetachedState — Observer or non-voter; replicating but cannot vote +``` + +#### Transition Diagram + +``` + ┌──────────────────────────────┐ + │ │ + ▼ │ + ┌──────────┐ Pre-Vote OK │ + │Unattached│────────────────┐ │ + └──────────┘ │ │ + │ ▼ │ + │ ┌────────────┐ │ + Discover │ │ Prospective│ │ + Leader │ │ (Pre-Vote)│ │ + │ └────────────┘ │ + │ │ │ + │ Pre-Vote │ │ + │ succeeded │ │ + │ ▼ │ + │ ┌────────────┐ │ + │ │ Candidate │───────┤ Won election + │ └────────────┘ │ + │ │ │ + │ Lost │ Won election │ + │ election │ (quorum) │ + │ ▼ │ + │ ┌────────────┐ │ + └───────► │ Follower │◄────────┤ + └────────────┘ │ + │ │ + Become leader │ + │ │ + ▼ │ + ┌────────────┐ │ + │ Leader │───────────┘ Higher term + └────────────┘ + │ + Resign (graceful) + │ + ▼ + ┌────────────┐ + │ Resigned │ + └────────────┘ +``` + +Additionally, `DetachedState` is used for observers (non-voters) who track the leader but don't participate in elections. + +--- + +## 3. KRaft State Variables (from Kafka Source) + +*Reference: Source [8] — Apache Kafka `kafka/raft/` package* + +### 3.1 Common State (QuorumState) + +| Variable | Type | Description | +|----------|------|-------------| +| `localId` | Int | This node's replica ID | +| `voters` | Set(Int) | Set of voter replica IDs | +| `observers` | Set(Int) | Set of observer replica IDs | +| `epochState` | EpochState | Current state object (one of the states below) | +| `currentClaimedEpoch` | Int | Epoch this node last claimed (for persistence) | + +### 3.2 UnattachedState + +When a node knows about an epoch but hasn't discovered the leader yet, or is starting up. + +| Variable | Type | Description | +|----------|------|-------------| +| `epoch` | Int | The epoch this node is in | +| `votedId` | Int? | Replica ID voted for in this epoch, or -1 (unvoted) | + +### 3.3 ProspectiveState (Pre-Vote, KIP-996) + +A node that has started Pre-Vote but hasn't transitioned to Candidate yet. This prevents disruptive servers from forcing elections. + +| Variable | Type | Description | +|----------|------|-------------| +| `epoch` | Int | The epoch for which Pre-Vote is being conducted | +| `votedId` | Int? | Vote (always self during Pre-Vote) | +| `hasSufficientVote` | Bool | Whether a quorum of pre-votes has been received | +| `preVoteResponded` | Set(Int) | Voters that responded to Pre-Vote | +| `preVoteGranted` | Set(Int) | Voters that granted Pre-Vote | +| `gracePeriodExpired` | Bool | Whether the election timeout has expired | + +**Transition to Candidate:** Only when `hasSufficientVote = true` AND `gracePeriodExpired = true`. + +### 3.4 CandidateState + +Standard Raft candidate with KRaft-specific additions. + +| Variable | Type | Description | +|----------|------|-------------| +| `epoch` | Int | Current election epoch | +| `votedId` | Int | Always `localId` (votes for self) | +| `votes` | Set(Int) | Voter IDs that granted a vote | +| `isVoteGranted` | Bool | Whether a quorum of votes has been received | +| `isVoteRejected` | Bool | Whether the election is doomed (quorum impossible) | +| `highWatermark` | Long | Last known high watermark at election start | +| `retryVoteQuorumGracePeriod` | Bool | Whether to retry sending vote requests | + +### 3.5 FollowerState + +Standard Raft follower with KRaft-specific additions. + +| Variable | Type | Description | +|----------|------|-------------| +| `epoch` | Int | Current epoch | +| `leaderId` | Int | ID of the current leader | +| `votedId` | Int? | Replica ID voted for (or -1) | +| `highWatermark` | Long | Last known high watermark | +| `lastCaughtUpTimeMs` | Long | Timestamp of last successful replication | +| `lingerHighWatermark` | Long | High watermark at which we started lingering | +| `isLingering` | Bool | Whether this follower is in linger mode (delaying transition) | + +### 3.6 LeaderState + +Standard Raft leader with KRaft-specific additions. + +| Variable | Type | Description | +|----------|------|-------------| +| `epoch` | Int | Current epoch (same as currentTerm) | +| `leaderId` | Int | This node's ID | +| `highWatermark` | Long | Current high watermark (equivalent to commitIndex) | +| `voterStates` | Map(Int, VoterState) | Per-voter tracking: nextOffset, matchOffset, lastCaughtUpTime | +| `localId` | Int | This node's replica ID | +| `retryForVoterEnabled` | Bool | Whether to retry AppendEntries to slow voters | + +**VoterState sub-structure:** + +| Variable | Type | Description | +|----------|------|-------------| +| `nextOffset` | Long | Next log offset to send to this voter (= nextIndex) | +| `matchOffset` | Long | Highest offset known to be replicated on this voter (= matchIndex) | +| `lastCaughtUpTimeMs` | Long | Timestamp of last successful catch-up | +| `lastSentTimeMs` | Long | Timestamp of last AppendEntries sent | +| `isVoter` | Bool | Whether this replica is a voter | + +### 3.7 ResignedState + +A leader that has gracefully stepped down. Not present in vanilla Raft. + +| Variable | Type | Description | +|----------|------|-------------| +| `epoch` | Int | The epoch when the leader resigned | +| `leaderId` | Int | Former leader's ID | + +*Key behavior:* A resigned leader does not accept new client writes. It continues to process in-flight requests until they complete or time out. It then transitions to Unattached or Follower depending on what it learns about the new epoch. + +### 3.8 DetachedState (Observer) + +Used for observer (non-voter) nodes. + +| Variable | Type | Description | +|----------|------|-------------| +| `epoch` | Int | Current epoch | +| `leaderId` | Int | ID of the leader being followed | +| `highWatermark` | Long | Last known high watermark | + +*Key behavior:* Detached observers replicate the log but never vote or become candidates. They can transition to Follower or Unattached if they learn about a new epoch. + +--- + +## 4. KRaft State Transitions (Detailed) + +*Reference: Source [8], [9]* + +### 4.1 Complete Transition Table + +| From | To | Trigger | Preconditions | +|------|----|---------|--------------| +| Unattached | Prospective | Election timeout + Pre-Vote enabled | `votedId == -1`, `hasSufficientVote == false` | +| Unattached | Follower | DiscoverLeader with valid leader | `leaderId` known, `epoch` matches | +| Prospective | Candidate | Pre-Vote quorum reached + grace period expired | `hasSufficientVote ∧ gracePeriodExpired` | +| Prospective | Follower | DiscoverLeader OR higher AppendEntries | `leaderId` known or higher term received | +| Candidate | Leader | Election won (quorum of votes) | `isVoteGranted == true` | +| Candidate | Follower | DiscoverLeader OR higher term AppendEntries | `leaderId` known or higher term received | +| Follower | Prospective | Election timeout + no heartbeat | `votedId == -1`, Pre-Vote enabled | +| Follower | Candidate | Election timeout + Pre-Vote disabled | `votedId == -1` | +| Leader | Resigned | Graceful shutdown / Resign RPC | Leader decides to step down | +| Leader | Follower | Higher term received | `m.epoch > currentEpoch` | +| Resigned | Unattached | Resign processing complete + no leader known | All in-flight requests handled | +| Resigned | Follower | Learn about new leader | `leaderId` known in new epoch | +| Detached | Follower | Promoted to voter | Node gains voting right | +| Any | Unattached | Higher epoch learned (no leader known) | `m.epoch > currentEpoch` | + +### 4.2 Pre-Vote Protocol (KIP-996) + +*Reference: Source [6]* + +Pre-Vote prevents disruptive servers (those that are partitioned or have stale logs) from forcing unnecessary elections. + +**Protocol:** + +1. **Before becoming Candidate**, a server first enters Prospective state and sends Pre-Vote requests. +2. **Pre-Vote Request:** `{lastOffset: Long, lastEpoch: Int}` — the prospective candidate advertises its log position. +3. **Pre-Vote Response:** `{granted: Bool}` — a server grants Pre-Vote only if: + - The prospective candidate's log is at least as up-to-date as the responder's + - The responder has not heard from a valid leader recently (within election timeout) +4. **Transition to Candidate:** Only after receiving a quorum of Pre-Vote grants AND the grace period expires. + +**Why Pre-Vote matters for verification:** +- Prevents a partitioned node with a stale term from disrupting the cluster +- The invariant is: a node only enters Candidate state if it could potentially win an election +- This eliminates a class of livelock scenarios where nodes continuously force elections they can't win + +### 4.3 Resignation Protocol + +*Reference: Source [8], [9]* + +A KRaft leader can gracefully resign (step down) without crashing. This is not in vanilla Raft. + +**Resign process:** +1. Leader transitions to Resigned state +2. Stops accepting new client writes +3. Continues to serve in-flight AppendEntries responses +4. Other nodes see an election timeout and start a new election +5. Resigned leader may learn about the new leader and transition to Follower + +**Verification concern:** During the resign window, the old leader's committed entries are still valid, but no new entries can be committed. This is a liveness property (eventually a new leader is elected), not a safety concern. + +--- + +## 5. Safety Invariants + +*References: Sources [1], [2], [3]* + +These are the invariants we need to prove in SpeckDL/Z3. + +### 5.1 Raft Paper Invariants (vanilla Raft) + +**I1: Election Safety.** At most one leader per term. +``` +∀ i, j ∈ Server: + state[i] = Leader ∧ state[j] = Leader ∧ currentTerm[i] = currentTerm[j] → i = j +``` + +**I2: Leader Completeness.** If a log entry is committed in term T, then all leaders for terms > T have that entry. +``` +This is the core invariant. Proved via the "Leader Completeness Property" in the Raft paper. +Inductive argument: a new leader must have all committed entries from prior terms +because it received votes from a quorum that overlaps with the quorum that committed +those entries. +``` + +**I3: Log Matching.** If two entries in different logs have the same index and term, then all preceding entries are identical. +``` +∀ i, j ∈ Server, idx: + log[i][idx].term = log[j][idx].term → log[i][1..idx] = log[j][1..idx] +``` + +**I4: Leader Append-Only.** A leader never overwrites or deletes entries in its log. +``` +∀ i: state[i] = Leader → log[i] only grows (entries appended, never removed) +``` + +**I5: Term Monotonicity.** Current term never decreases. +``` +∀ i: currentTerm'[i] ≥ currentTerm[i] +``` + +**I6: Commit Safety.** An entry is committed only if a quorum has replicated it. +``` +commitIndex[i] advances only when a quorum of matchIndex[i][k] ≥ N +Additionally: only entries from the current term can be committed via leader +advance (restriction from Raft paper Figure 8). +``` + +### 5.2 KRaft-Specific Invariants + +**K1: Quorum Overlap.** Any two quorums in the same epoch share at least one voter. +``` +∀ Q1, Q2 ∈ Quorum: Q1 ∩ Q2 ≠ ∅ +``` + +**K2: High Watermark Monotonicity.** A node's highWatermark never decreases. +``` +∀ i: highWatermark'[i] ≥ highWatermark[i] +``` + +**K3: High Watermark Validity.** A node's highWatermark is always ≤ the log end offset. +``` +∀ i: highWatermark[i] ≤ logEndOffset[i] +``` + +**K4: Observer Consistency.** An observer's log is a prefix of some voter's log that was committed. +``` +∀ observer o, voter v, offset idx: + o.hasOffset(idx) → ∃v ∈ Voters: v.committed(idx) ∧ o.log[1..idx] = v.log[1..idx] +``` + +**K5: Leader Uniqueness (KRaft).** At most one leader per epoch across all nodes. +``` +∀ i, j ∈ Replicas: + state[i] = Leader ∧ state[j] = Leader ∧ epoch[i] = epoch[j] → i = j +``` + +**K6: Vote Uniqueness.** A voter votes for at most one candidate per epoch. +``` +∀ i ∈ Voters, epoch e: votedFor[i][e] is either Nil or a single replica ID +``` + +--- + +## 6. Snapshots (KIP-630) + +*Reference: Source [5]* + +KRaft supports snapshot-based log compaction for the metadata log. + +### 6.1 Snapshot Mechanics + +- **Snapshot trigger:** When the log grows beyond a configured threshold +- **Snapshot contents:** Full metadata state at a given offset (topic configs, partition assignments, etc.) +- **Snapshot delivery:** Leader sends snapshots to followers that are lagging (similar to Raft's InstallSnapshot) + +### 6.2 State Changes for Snapshots + +| Variable | Type | Description | +|----------|------|-------------| +| `snapshotId` | {epoch: Int, endOffset: Long} | Identifies a snapshot by epoch and end offset | +| `lastSnapshotOffset` | Long | Offset of the last snapshot on this node | +| `recoveryState` | Enum | {RECOVERING, COMPLETE} — whether snapshot recovery is in progress | + +### 6.3 Invariants for Snapshots + +**S1: Snapshot Offset Monotonicity.** Snapshot offsets increase monotonically. +``` +snapshotId.endOffset[i]' ≥ snapshotId.endOffset[i] +``` + +**S2: Snapshot Consistency.** A snapshot at offset N contains all entries committed before N. +``` +∀ i, snapshot at offset N: + snapshot.epoch = log[N].term ∧ snapshot contains all committed entries up to N +``` + +--- + +## 7. Known Verification Gaps and Bugs + +*References: Smart Casual Verification blog, raft-dev Google Group, Source [6]* + +### 7.1 Known Bugs in Raft Implementations + +**Bug 1: Single-Server Membership Change Safety Bug (2015)** +- **Source:** raft-dev Google Group, reported by Ongaro +- **Issue:** The original Raft paper's algorithm for single-server membership changes has a safety bug. If two configuration changes overlap, the cluster can lose quorum overlap. +- **Mitigation:** Joint consensus (adding then removing in separate steps) is safe. KRaft uses joint consensus for membership changes (KIP-853). +- **Formal verification target:** Prove that KRaft's joint consensus approach maintains quorum overlap during membership changes. + +**Bug 2: Stale Leader Committing Old-Term Entries (Figure 8 Bug)** +- **Source:** Raft paper, Figure 8 +- **Issue:** A leader must not commit entries from previous terms directly. It can only commit entries from its current term, and previous-term entries are committed indirectly when the current-term entry is committed. +- **Status:** Correctly handled in KRaft (AdvanceCommitIndex only advances for current-term entries). +- **Formal verification target:** Prove that `AdvanceCommitIndex` only advances `commitIndex` when the entry at the target index has `term == currentTerm`. + +**Bug 3: Log Truncation During Leader Change** +- **Source:** Common implementation bug across multiple Raft implementations +- **Issue:** When a new leader overwrites conflicting entries on a follower, the truncation must be atomic. If the node crashes mid-truncation, the log can be left in an inconsistent state. +- **Status:** KRaft handles this by truncating before appending, but edge cases around partial truncation exist. +- **Formal verification target:** Prove that log truncation is atomic (either fully applied or not at all). + +**Bug 4: Disruptive Servers (Pre-Vote Target)** +- **Source:** KIP-996, Ongaro dissertation §9.6 +- **Issue:** A server that is partitioned or has a stale network connection can repeatedly force elections, disrupting the cluster. Without Pre-Vote, a partitioned server increments its term and forces elections it can't win. +- **Status:** KIP-996 (Pre-Vote) addresses this by requiring a server to confirm it can win before starting an election. +- **Formal verification target:** Prove that with Pre-Vote, a node that cannot win an election never transitions to Candidate state. + +### 7.2 Smart Casual Verification Findings + +*Reference: Decentralized Thoughts blog, "Smart Casual Verification"* + +In a systematic review of consensus protocol implementations, researchers found: + +1. **5 safety bugs** — violations of agreement or integrity properties +2. **1 liveness bug** — violation of termination under expected conditions +3. All bugs were found through informal reasoning about protocol invariants, not formal proof +4. Key pattern: most bugs occur in edge cases around term changes and log conflicts + +### 7.3 Open Verification Targets + +These are properties that should be verified in the SpeckDL/Z3 spec: + +1. **Election safety under network partitions:** Prove I1 (election safety) holds even during partitions. +2. **Log matching after truncation:** Prove I3 (log matching) holds after follower log truncation. +3. **Leader completeness after Pre-Vote:** Prove I2 (leader completeness) holds with the Pre-Vote extension. +4. **High watermark advancement:** Prove that `highWatermark` only advances when a quorum has replicated. +5. **Graceful resignation liveness:** Prove that after a leader resigns, a new leader is eventually elected (liveness, assuming fair loss and eventual delivery). +6. **Observer divergence:** Prove that observers cannot diverge from voters (K4). +7. **Joint consensus safety (KIP-853):** Prove that during membership changes, quorum overlap is maintained. + +--- + +## 8. Producer Acknowledgement Semantics (for M1.3) + +*Deferred — included here for reference only.* + +### acks=0 (Fire and forget) +- Producer sends and doesn't wait for response +- No durability guarantee +- Invariant: **None** (messages can be lost) + +### acks=1 (Leader acknowledgment) +- Producer waits for leader to write to local log +- Durability: survives leader failure, but NOT follower failure +- Invariant: **If ack received, entry exists in leader's log** + +### acks=all (ISR acknowledgment) +- Producer waits for all in-sync replicas (ISR) to acknowledge +- Durability: survives any failure that leaves a quorum +- Invariant: **If ack received, entry is committed (quorum has replicated)** + +### Exactly-once Semantics +- Requires idempotent producer + transaction coordinator +- Idempotent producer: assigns sequence numbers; broker deduplicates +- Transaction coordinator: two-phase commit across partitions + +### Key Invariants for Producer Acks + +1. `acks=all` → no data loss on leader failover (bounded to quorum availability) +2. Idempotent producer → no duplicate messages for same PID + sequence number +3. Transaction atomicity → all partitions in a transaction commit or none do + +--- + +## 9. Consumer Group Rebalancing (for M1.3) + +*Deferred — included here for reference only.* + +### States +- Empty → Joining → Stable → Rebalancing → Stable + +### Actions +- JoinGroup: consumer requests to join +- SyncGroup: consumer receives assignment +- Heartbeat: consumer indicates liveness +- LeaveGroup: consumer departs +- Rebalance: triggered by membership change or topic partition change + +### Invariants +1. No concurrent assignments to the same partition +2. All partitions assigned when group is stable +3. Offset monotonicity: committed offsets never decrease +4. No two consumers assigned the same partition simultaneously + +--- + +## 10. KIPs Reference + +| KIP | Title | Status | Relevance | +|-----|-------|--------|-----------| +| KIP-500 | Next Generation Client Protocol | Adopted | Foundation for KRaft | +| KIP-595 | A Raft Protocol for the Metadata Quorum | Adopted | Core KRaft protocol design | +| KIP-630 | Kafka Raft Snapshot | Adopted | Snapshot/compaction for metadata log | +| KIP-642 | Dynamic Configuration in KRaft | Adopted | Dynamic config updates via KRaft | +| KIP-853 | KRaft Controller Membership Changes | Implemented | Joint consensus for membership changes | +| KIP-996 | Pre-Vote for KRaft | Implementing | Pre-Vote phase to prevent disruptive elections | +| KIP-830 | KRaft Metadata Quorum | Adopted | Metadata quorum implementation details | + +--- + +## Appendix A: Message Types (KRaft) + +### Internal KRaft RPC Messages + +| Message | Direction | Fields | +|---------|-----------|--------| +| `VoteRequest` | Candidate → Voter | `candidateId`, `epoch`, `lastOffset`, `lastEpoch` | +| `VoteResponse` | Voter → Candidate | `epoch`, `voteGranted`, `leaderId` | +| `BeginQuorumEpoch` | Leader → Follower | `leaderId`, `epoch` | +| `EndQuorumEpoch` | Leader → Voter (resign) | `leaderId`, `epoch` | +| `FetchSnapshot` | Follower → Leader | `snapshotId`, `offset` | +| `Fetch` | Follower → Leader | `followerId`, `epoch`, `lastFetchedEpoch`, `lastFetchedOffset`, `maxBytes` | +| `FetchResponse` | Leader → Follower | `epoch`, `leaderId`, `highWatermark`, `records` | + +*Note: KRaft uses `BeginQuorumEpoch` as the heartbeat mechanism (instead of vanilla Raft's empty AppendEntries).* + +--- + +## Appendix B: Key Differences from Vanilla Raft + +| Aspect | Vanilla Raft | KRaft | +|--------|-------------|-------| +| Term naming | "Term" | "Epoch" (functionally equivalent) | +| Heartbeat | Empty AppendEntries | BeginQuorumEpoch | +| Resignation | Not specified | EndQuorumEpoch (graceful shutdown) | +| Pre-Vote | Not in original paper | ProspectiveState (KIP-996) | +| States | Follower, Candidate, Leader | + Unattached, Prospective, Resigned, Detached | +| Snapshot | InstallSnapshot | FetchSnapshot (KIP-630) | +| Membership | Single-server or joint | Joint consensus only (KIP-853) | +| Log content | Arbitrary client values | Metadata records (topic configs, partition assignments) | +| Quorum | All servers vote | Voters vote, observers replicate | +| Client ID | Not specified | Replica ID (localId) | + +--- + +*Document compiled from: Raft TLA+ spec (Ongaro 2014), Apache Kafka source code (`kafka/raft/`), KIP-595/630/642/853/996, Red Hat KRaft deep dive, Smart Casual Verification findings.* \ No newline at end of file diff --git a/docs/publishing-toolkit.md b/docs/publishing-toolkit.md new file mode 100644 index 0000000..56a1d8d --- /dev/null +++ b/docs/publishing-toolkit.md @@ -0,0 +1,371 @@ +# Speckl Publishing Toolkit +**Ready to publish — all content drafted, this is the checklist.** + +--- + +## Quick Start: Publish Order & Cadence + +**Week 1 (Show HN week — May 12-13):** +1. Blog #1 on Dev.to + Medium (Monday May 11, day before Show HN) +2. Show HN post on HN (Tuesday May 12 or Wednesday May 13, 8-11 AM UTC = 1-4 AM PT) +3. Reddit cross-post: r/programming + r/Compilers (same day as Show HN) + +**Week 2+ (weekly cadence):** +- Week 2: Blog #2 — Tuesday or Wednesday (~May 19-20) +- Week 3: Blog #3 — Tuesday or Wednesday (~May 26-27) +- Week 4: Blog #4 — Tuesday or Wednesday (~Jun 2-3) +- Week 5: Blog #5 — Tuesday or Wednesday (~Jun 9-10) + +--- + +## Blog Post #1: "Spec-Code Drift Is a Provenance Problem" +**File:** `speckl/docs/blog-01-spec-code-gap.md` | **Words:** ~900 + +### Dev.to Post + +```markdown +Title: Spec-Code Drift Is a Provenance Problem +Tags: formalmethods, compilers, softwareengineering, compliance, provenance +Canonical URL: (leave blank — Dev.to is primary) + +--- + +[Content from blog-01-spec-code-gap.md — copy paste] + +--- + +*This is Part 1 of a 5-part series on Speckl, a specification language with compiler-level provenance. [Part 2 →]()* +``` + +### Medium Post + +```markdown +Title: Spec-Code Drift Is a Provenance Problem +Subtitle: AI is accelerating code generation. Without embedded provenance, we're accelerating drift too. +Tags: Formal Methods, Software Engineering, Compiler Design + +--- + +[Content from blog-01-spec-code-gap.md — copy paste] + +--- + +*This is Part 1 of a 5-part series on Speckl, a specification language with compiler-level provenance. [Part 2 →]()* +``` + +### Social Blurbs + +**LinkedIn / Bluesky / Mastodon:** +> AI writes code faster than ever. But can you prove what ran came from what you wrote? I've been working on Speckl — a spec language that embeds provenance directly into compiled output. Part 1 of a series on why spec-code drift is really a provenance problem. https://speckl.scoble.me + +**Reddit (r/programming):** +> Title: Spec-Code Drift Is a Provenance Problem [blog post] +> Text: I've been building a specification language that embeds provenance (PROV-O audit trails, SBOMs, signed manifests) directly into compiled code. The core argument: spec-code drift is fundamentally a provenance problem, not a documentation problem. Would love feedback from anyone working in formal methods or compliance. + +**HN (if posting separately from Show HN):** +> Title: Spec-Code Drift Is a Provenance Problem +> (link to Dev.to or speckl.scoble.me) + +--- + +## Blog Post #2: "Designing SpeckDL: What a Spec Language Needs in 2026" +**File:** `speckl/docs/blog-02-designing-speckdl.md` | **Words:** ~1,000 + +### Dev.to Post + +```markdown +Title: Designing SpeckDL: What a Spec Language Needs in 2026 +Tags: formallanguages, compilers, typesystems, softwareengineering, provenance +Series: Speckl +Canonical URL: (leave blank) + +--- + +[Content from blog-02-designing-speckdl.md — copy paste] + +--- + +*Part 2 of 5. [Part 1: Spec-Code Drift Is a Provenance Problem →](link to part 1) | [Part 3 →]()* +``` + +### Medium Post + +```markdown +Title: Designing SpeckDL: What a Spec Language Needs in 2026 +Subtitle: State machines, embedded provenance, and compile-to-code — the design decisions behind a new spec language. +Tags: Programming Languages, Compiler Design, Formal Methods + +--- + +[Content from blog-02-designing-speckdl.md — copy paste] + +--- + +*Part 2 of 5. [Part 1 →](link) | [Part 3 →]()* +``` + +### Social Blurbs + +**LinkedIn / Bluesky / Mastodon:** +> Part 2 of the Speckl series: designing a spec language for 2026. SpeckDL has a 9-type system, state machine primitives, embedded provenance, and compiles to 5 auditable artifacts. Here's why each design decision matters. https://speckl.scoble.me + +**Reddit (r/ProgrammingLanguages):** +> Title: Designing SpeckDL: What a spec language needs in 2026 +> Text: I'm designing a specification language that compiles to TypeScript + WASM with embedded provenance (PROV-O, CycloneDX, SPDX). This post covers the 4 core design principles, the type system, and why state machines are first-class. Feedback from PL designers especially welcome. + +--- + +## Blog Post #3: "How Speckl's Compiler Works: From Spec to Five Auditable Artifacts" +**File:** `speckl/docs/blog-03-compiler-architecture.md` | **Words:** ~1,200 + +### Dev.to Post + +```markdown +Title: How Speckl's Compiler Works: From Spec to Five Auditable Artifacts +Tags: compilers, typescript, webassembly, formalmethods, opensource +Series: Speckl +Canonical URL: (leave blank) + +--- + +[Content from blog-03-compiler-architecture.md — copy paste] + +--- + +*Part 3 of 5. [Part 1 →]() | [Part 2 →]() | [Part 4 →]()* +``` + +### Medium Post + +```markdown +Title: How Speckl's Compiler Works: From Spec to Five Auditable Artifacts +Subtitle: A deep dive into the Speckl compiler pipeline — parser, type checker, and five independent code generators. +Tags: Compiler Design, TypeScript, WebAssembly + +--- + +[Content from blog-03-compiler-architecture.md — copy paste] + +--- + +*Part 3 of 5. [Part 1 →]() | [Part 2 →]() | [Part 4 →]()* +``` + +### Social Blurbs + +**LinkedIn / Bluesky / Mastodon:** +> The Speckl compiler generates 5 artifacts from one spec: PROV-O audit trail, CycloneDX SBOM, SPDX license metadata, TypeScript state machine, and WASM. Here's the full pipeline architecture — ~850 lines of parser, 5 independent generators, 41 tests in 2 seconds. https://speckl.scoble.me + +**Reddit (r/Compilers):** +> Title: How Speckl's Compiler Works: From Spec to Five Auditable Artifacts +> Text: Built a compiler that takes specification files and emits 5 independent artifacts: PROV-O provenance, CycloneDX SBOM, SPDX metadata, TypeScript state machines, and WASM. All generators are independent (failures don't cascade) and the whole test suite runs in ~2 seconds. Would love compiler dev feedback. + +--- + +## Blog Post #4: "Embedded Provenance: Proving What Ran Came From What You Wrote" +**File:** `speckl/docs/blog-04-embedded-provenance.md` | **Words:** ~1,200 + +### Dev.to Post + +```markdown +Title: Embedded Provenance: Proving What Ran Came From What You Wrote +Tags: provenance, compliance, sbom, cybersecurity, formalmethods +Series: Speckl +Canonical URL: (leave blank) + +--- + +[Content from blog-04-embedded-provenance.md — copy paste] + +--- + +*Part 4 of 5. [Part 1 →]() | [Part 2 →]() | [Part 3 →]() | [Part 5 →]()* +``` + +### Medium Post + +```markdown +Title: Embedded Provenance: Proving What Ran Came From What You Wrote +Subtitle: SBOMs shouldn't come from scanners. They should come from your spec. +Tags: Cybersecurity, Compliance, Software Supply Chain + +--- + +[Content from blog-04-embedded-provenance.md — copy paste] + +--- + +*Part 4 of 5. [Part 1 →]() | [Part 2 →]() | [Part 3 →]() | [Part 5 →]()* +``` + +### Social Blurbs + +**LinkedIn / Bluesky / Mastodon:** +> Your SBOM should come from your spec, not your scanner. Speckl embeds provenance at the compiler level — every compiled artifact carries a cryptographic chain proving it came from the specification. Part 4 of the Speckl series: Embedded Provenance. https://speckl.scoble.me + +**Reddit (r/cybersecurity):** +> Title: Embedded Provenance: Why your SBOM should come from your spec, not your scanner +> Text: Most SBOMs are generated post-hoc by scanning dependencies. Speckl generates CycloneDX SBOMs + PROV-O audit trails + SPDX license manifests at compile time — provenance by construction, not by detection. This maps to NIST SA-11 controls for human-in-the-loop validation of automated outputs. + +--- + +## Blog Post #5: "Consensus Protocols in SpeckDL: From TLA+ to Runnable Code" +**File:** `speckl/docs/blog-05-consensus-protocols.md` | **Words:** ~1,400 + +### Dev.to Post + +```markdown +Title: Consensus Protocols in SpeckDL: From TLA+ to Runnable Code +Tags: distributedsystems, consensus, formalmethods, tlaplus, compilers +Series: Speckl +Canonical URL: (leave blank) + +--- + +[Content from blog-05-consensus-protocols.md — copy paste] + +--- + +*Part 5 of 5. [Part 1 →]() | [Part 2 →]() | [Part 3 →]() | [Part 4 →]()* +``` + +### Medium Post + +```markdown +Title: Consensus Protocols in SpeckDL: From TLA+ to Runnable Code +Subtitle: Two-Phase Commit, Paxos, and Raft — specified, type-checked, and compiled to runnable code. +Tags: Distributed Systems, Consensus, TLA+, Formal Verification + +--- + +[Content from blog-05-consensus-protocols.md — copy paste] + +--- + +*Part 5 of 5. [Part 1 →]() | [Part 2 →]() | [Part 3 →]() | [Part 4 →]()* +``` + +### Social Blurbs + +**LinkedIn / Bluesky / Mastodon:** +> Two-Phase Commit in 45 lines. Paxos in 120. Raft in 230. All compile to runnable TypeScript + WASM with provenance baked in. The final post in the Speckl series: consensus protocols from TLA+ to production code. https://speckl.scoble.me + +**TLA+ Google Group (respectful intro):** +> Subject: Speckl: Compiling TLA+-style specs to runnable code (complementary tool) +> Hi all — I've built Speckl, a specification language inspired by TLA+ but designed for a different purpose: compiling specs to runnable code with embedded provenance. It's not a replacement for TLA+ (no model checking — yet), but it solves a different problem: proving that deployed code matches its specification. Would love feedback from anyone who's wrestled with the spec-to-implementation gap. https://speckl.scoble.me + +**Reddit (r/distributedsystems):** +> Title: Consensus Protocols in SpeckDL: Compiling TLA+-style specs to runnable code +> Text: I specified Two-Phase Commit (45 lines), Paxos (120 lines), and Raft (230 lines) in SpeckDL, a specification language that compiles to TypeScript + WASM with embedded provenance. All three compile correctly and pass type checking. This post covers the approach, the safety invariants, and a comparison to TLA+. + +--- + +## Show HN: Announcement Post +**File:** `speckl/docs/show-hn-announcement.md` + +### HN Post + +```markdown +Title: Show HN: Speckl — A spec language that compiles to runnable code with provenance + +[Content from show-hn-announcement.md — copy paste. Update any dates/status flags if needed.] +``` + +### HN Discussion Prep (likely objections + responses) + +**Objection 1: "How is this different from TLA+?"** +> Speckl is complementary, not competing. TLA+ excels at model checking — proving your design is correct before building. Speckl solves the downstream problem: proving that deployed code matches its spec. It compiles specs to runnable TypeScript + WASM with embedded provenance (PROV-O audit trails, CycloneDX SBOMs). TLA+ for design verification, Speckl for implementation verification. + +**Objection 2: "What about fuzzing / property testing?"** +> Those test behavior. Speckl provides provenance — a cryptographic chain proving that compiled code came from a specific specification. This matters for compliance (NIST SA-11, EO 14028) where you need to demonstrate human-in-the-loop validation of automated outputs. Different layer of the verification stack. + +**Objection 3: "The compiler only has 41 tests?"** +> It's early. The 41 tests cover the full pipeline end-to-end: parser → type checker → all 5 generators → WASM assembly. The compiler is functional for spec-to-code workflows. More tests (property-based, fuzzing) are on the roadmap. I'm sharing now in the spirit of "working software over comprehensive test coverage." + +**Objection 4: "Why not just use Rust/Go with good types?"** +> Types catch type errors. They don't catch specification errors. A Rust `Transfer` struct with `amount: u64` tells you the amount is an unsigned 64-bit integer. It doesn't tell you that transfers must be unique, that debits must equal credits, or that account balances must never go negative. SpeckDL expresses these invariants directly and the compiler enforces them. + +**Objection 5: "Another spec language? We have enough DSLs."** +> This isn't a general-purpose DSL. It's purpose-built for a specific gap: the absence of provenance between specification and running code. Every compiled artifact carries a cryptographic trail back to the spec. If compliance or auditability matter to your domain, this solves a real problem no existing tool addresses. + +**Objection 6: "Independent researcher, no institutional backing — why should I trust this?"** +> The code is MIT-licensed and public (Forgejo + Codeberg). The whitepaper is complete and available. The compiler runs on any Node.js. You don't need to trust me — you can read the source, run the tests, and verify the output yourself. That's the whole point of open source + provenance. + +### Reddit Cross-Post (r/programming) + +> Title: Show HN: Speckl — A specification language that compiles to runnable code with embedded provenance +> Text: [2-3 sentence summary of what Speckl does, link to Show HN, invitation for feedback] + +--- + +## Platform Quick Reference + +### Dev.to Publishing +1. Go to https://dev.to/new +2. Paste the blog content from the relevant Markdown section above +3. Add tags (listed in each section) +4. Add series name ("Speckl") for posts 2-5 +5. Publish + +### Medium Publishing +1. Go to https://medium.com/new-story +2. Paste the blog content from the relevant section +3. Add tags +4. Publish — Medium auto-formats Markdown + +### Hacker News +1. Go to https://news.ycombinator.com/submit +2. Paste the Show HN content +3. Title must start with "Show HN:" +4. Post on Tue/Wed between 8-11 AM UTC for optimal visibility + +### Reddit +1. Go to the relevant subreddit +2. "Submit Link" or "Submit Text" as indicated +3. Post between 6-10 AM ET for best visibility + +### TLA+ Google Group +1. Go to https://groups.google.com/g/tlaplus +2. "New Topic" +3. Keep tone respectful — Speckl is complementary, not competing + +--- + +## Pre-Publish Checklist + +Before publishing ANY post: +- [ ] Update all "Part N →" links to point to actual published URLs +- [ ] Verify speckl.scoble.me is live +- [ ] Verify Forgejo repo is public and accessible +- [ ] Verify Codeberg mirror is synced +- [ ] Have the whitepaper v2 link ready: (Forgejo raw URL) +- [ ] Have the Speckl by Example link ready: (Forgejo raw URL) + +--- + +## Content Repurposing Ladder + +Each blog post can ladder into more formats: +1. **Blog post** (Dev.to + Medium) → primary +2. **Twitter thread** (5-7 tweets extracting key points) → same day +3. **LinkedIn post** (long-form, use social blurb as opener) → same day +4. **Reddit discussion** (link post + top comment with summary) → same day +5. **HN submission** (if not already covered by Show HN) → same week +6. **Newsletter roundup** (end of 5-week series, "What I learned from X readers") → week 6 + +--- + +## Metrics to Track + +- [ ] Dev.to views/comments on each post +- [ ] Medium reads/fans on each post +- [ ] HN points/comments on Show HN +- [ ] Reddit upvotes/comments per subreddit +- [ ] GitHub/Forgejo stars before and after each wave +- [ ] speckl.scoble.me unique visitors (need analytics) +- [ ] Inbound emails / DMs from posts +- [ ] New watchers/stars on Forgejo repo + +--- + +*This toolkit was compiled May 8, 2026. Update links as posts go live.* diff --git a/docs/show-hn-announcement.md b/docs/show-hn-announcement.md new file mode 100644 index 0000000..f9577c4 --- /dev/null +++ b/docs/show-hn-announcement.md @@ -0,0 +1,108 @@ +# Show HN: Speckl — Spec-first development with compilable state machines (open source) + +**Title (Show HN):** Show HN: Speckl — Write specs like TLA+, compile to runnable TypeScript and WASM + +## Post Body + +I built Speckl because I was tired of writing specifications that rot in a wiki while the code drifts in a different direction. + +**What it is:** Speckl is a spec-first development tool. You write your system's behavior in SpeckDL (a lightweight specification language) and the compiler generates five artifacts: + +1. **TypeScript** — Runnable state machine classes (passes `tsc --noEmit`) +2. **WASM** — WebAssembly modules with verified state transitions +3. **PROV-O** — Machine-readable provenance (audit trail) +4. **CycloneDX SBOM** — Software bill of materials +5. **SPDX SBOM** — Standardized license/supply chain manifest + +**The key insight:** A spec shouldn't just describe what your system does — it should *compile into the thing that does it*. When the spec IS the source of truth, you can't have spec-code drift. + +## How it works + +Here's a simple toggle switch in SpeckDL: + +``` +spec ToggleSwitch +type State = enum { on, off } +state active: State = off + +init + active := off + +action flip + require active == off + active := on + +invariant Mutex + active == on implies prev(active) == off +``` + +The compiler generates a TypeScript class with typed guards, a WASM module with verified state transitions, and provenance metadata proving where every line came from. + +## Real specs that compile + +The example suite covers: + +- **ToggleSwitch** — State machines 101 +- **AccountLedger** — Financial invariants, Map/Set types, audit trails +- **TwoPhaseCommit** — Multi-role distributed protocols (direct TLA+ port) +- **Paxos** — Classic consensus with proposers/acceptors/learners +- **Raft** — Full Raft protocol: leader election, log replication, 3 safety invariants (Leader Completeness, Log Matching, Single Leader) +- **TigerBeetleLedger** — Naive port of TigerBeetle's core accounting engine + +All 10 examples compile to all 5 artifacts. 41 tests, all passing. + +## Why this matters + +The gap between "what we specified" and "what we built" is where production bugs live. TLA+ is brilliant for formal verification, but it stops at the model. SpeckDL starts at the model and compiles forward into production code. + +| TLA+ | SpeckDL | +|------|---------| +| Mathematical modeling | Compilable specifications | +| TLC model checker | WASM runtime verification | +| LaTeX pretty-printing | TypeScript production code | +| Academic/Amazon use | CI/CD integration target | + +## Tech stack + +- Parser: Custom recursive descent (TypeScript, ~800 lines) +- Generators: TypeScript state machine, WASM (WAT text format), PROV-O (RDF/Turtle), CycloneDX JSON, SPDX JSON +- Build: TypeScript → Node.js CLI, `tsc --noEmit` clean +- Tests: 41 unit tests via Node test runner +- License: MIT + +## What's next + +- Parser support for `precondition:` / `postcondition:` declarative assertions +- Record type emission for generated TypeScript +- Light JS API for WASM modules +- Self-spec: The Speckl compiler, specified in SpeckDL + +## Try it + +- **Repo:** https://os.scoble.me/forgejo/sscoble/speckl +- **Mirror:** https://codeberg.org/sscoble/speckl +- **Docs:** https://os.scoble.me/forgejo/sscoble/speckl/src/branch/main/docs/speckl-by-example.md +- **Website:** https://speckl.scoble.me + +Open source (MIT). Looking for feedback, contributors, and early adopters who want to kill spec-code drift. + +--- + +## Publishing notes + +**Targets:** +1. Hacker News (Show HN) — primary +2. r/programming — secondary (link to HN discussion) +3. r/rust (TigerBeetle angle) — tertiary +4. Dev.to — cross-post with full tutorial content + +**Best posting times (PT):** +- HN: Mon-Fri 7-9 AM or 12-2 PM PT +- Reddit: Tue-Wed 6-9 AM PT + +**Posting tips:** +- Post on HN as "Show HN:" (triggers Show tab placement) +- First comment: Add context about why this exists + link to tutorial +- Keep title under 80 chars +- Don't ask for upvotes +- Reply to comments within first hour diff --git a/docs/show-hn-comment-playbook.md b/docs/show-hn-comment-playbook.md new file mode 100644 index 0000000..8de0427 --- /dev/null +++ b/docs/show-hn-comment-playbook.md @@ -0,0 +1,181 @@ +# Show HN Comment Response Playbook +**Prepared: May 9, 2026 — For launch May 12-13, 2026** + +## Why This Exists +The first hour of a Show HN post makes or breaks it. Quick, thoughtful replies signal that the project has a real builder behind it. This playbook pre-writes responses to the 12 most likely objections/questions so Scott can copy-paste-adapt in real time. + +--- + +## RESPONSE 1: "How is this different from TLA+?" + +**Response:** +TLA+ is a mathematical modeling language — you write specs, run TLC to check invariants, and that's where it stops. SpeckDL starts at the same place (state machines, invariants, actions) but compiles *forward* into production artifacts: TypeScript classes, WASM modules, SBOMs, and provenance metadata. + +Think of it as: TLA+ proves your design is correct. SpeckDL ensures your implementation stays correct. They're complementary — you could model-check a SpeckDL spec with TLA+ concepts, and you can't compile a TLA+ spec to runnable code. + +I wrote a detailed comparison in the tutorial: [link to speckl-by-example.md#tla-comparison] + +--- + +## RESPONSE 2: "Why not just use property-based testing or fuzzing?" + +**Response:** +Property-based testing and fuzzing are *runtime* verification — they catch bugs by exploring the state space. Speckl is *compile-time* verification — the spec IS the implementation, so there's nothing to drift from. + +That said, they're complementary. You could use SpeckDL specs to generate property-based test harnesses. The WASM output already validates state transitions at runtime. I'd love to add QuickCheck-style test generation from spec invariants as a future feature. + +--- + +## RESPONSE 3: "This seems overengineered for CRUD apps. When would I actually use this?" + +**Response:** +Fair question! You wouldn't use Speckl for a basic CRUD app. The sweet spot is systems where correctness matters and state transitions are complex: +- Financial ledgers (double-entry accounting, payment systems) +- Distributed consensus (Raft, Paxos, 2PC) +- Workflow engines with audit requirements +- Compliance-critical systems (NIST SA-11, SOC 2 evidence) +- Any system where you're already writing state machine diagrams on a whiteboard + +If your state transitions fit on a napkin, skip Speckl. If you're drawing sequence diagrams for 5 failure modes, that's where it shines. + +--- + +## RESPONSE 4: "The language looks like it was designed in a weekend. Why create yet another DSL?" + +**Response:** +It WAS designed iteratively — but over weeks, not a weekend. The constraint was: it has to be readable by non-programmers (compliance officers, auditors, domain experts) while being precise enough to compile. + +I looked hard at using existing languages (TLA+, Alloy, P, PlusCal) but none of them compile to production code + provenance metadata. SpeckDL is deliberately minimal — ~15 keywords. The complexity lives in the compiler, not the language. + +That said, I'm actively seeking feedback on the syntax. If you see awkward patterns, I want to hear about them. + +--- + +## RESPONSE 5: "41 tests? That's nothing. How do I know this works?" + +**Response:** +41 unit tests cover the parser, type checker, and all 5 generators. The real validation is the example suite: 10 specs (from ToggleSwitch to Raft) that all compile to all 5 artifacts with zero errors. + +The WASM output is validated through `wabt` (WebAssembly binary toolkit) — parse → resolve → validate → binary roundtrip. The TypeScript output passes `tsc --noEmit`. + +You're right that 41 isn't a lot. More tests are being added as the compiler matures. The roadmap includes property-based testing and differential testing against TLC models. + +--- + +## RESPONSE 6: "What about formal verification? Can you prove invariants hold?" + +**Response:** +Not yet — that's on the roadmap. Right now Speckl does *runtime* invariant checking (assertions in generated code) rather than *static* verification. + +The architecture supports it though: the AST preserves all invariant expressions, and the WASM generator already emits guard checks. Adding Z3/SMT-based invariant proving is a natural next step — the spec already has all the information a solver would need. + +Targeting bounded model checking via Z3 as a Phase 3 feature. + +--- + +## RESPONSE 7: "Why TypeScript? Why not Rust or Go?" + +**Response:** +TypeScript was the fastest path to a working compiler and generated code that's immediately useful (most web backends run Node.js). The compiler architecture is language-agnostic — generators are pluggable modules that walk the AST. + +Rust and Go generators are planned. The WASM output already works cross-language (you can embed it in Rust via wasmtime, Go via wazero, etc.). I picked the starting point that maximized reach with minimum effort. + +--- + +## RESPONSE 8: "Self-hosted Forgejo? Why not GitHub?" + +**Response:** +I believe developer tools should practice what they preach about infrastructure independence. The canonical repo lives on my own Forgejo instance. There's a Codeberg mirror for redundancy. GitHub mirror coming soon. + +The license is MIT — you can fork it wherever you want. + +--- + +## RESPONSE 9: "What's the business model? Is this going to become paid?" + +**Response:** +The core compiler and language are MIT-licensed and will stay open source. The business model ladder: +1. **Consulting** — Help teams adopt spec-driven development (now) +2. **Managed service** — Hosted spec validation + compliance dashboards (later) +3. **Enterprise features** — SSO, audit logging, compliance report generation (future) + +But the language and compiler are forever free. I'm not pulling an Oracle. + +--- + +## RESPONSE 10: "How does provenance work exactly? What's in the PROV-O output?" + +**Response:** +Every artifact generated by the compiler includes embedded provenance in PROV-O (W3C PROV Ontology, RDF/Turtle format). Each line of generated code is linked back to a specific line in the source `.speckdl` file. + +The provenance graph includes: +- `prov:wasGeneratedBy` — which compiler pass produced this output +- `prov:used` — which source file and line was consumed +- `prov:wasAttributedTo` — which author/agent wrote the spec +- `prov:generatedAtTime` — timestamp of compilation + +This creates an auditable chain: auditor sees generated code → traces to spec line → traces to author and timestamp. That's the NIST SA-11 compliance angle — human-in-the-loop for automated outputs with a verifiable trail. + +--- + +## RESPONSE 11: "How mature is this? Would you use it in production?" + +**Response:** +Honest answer: it's not production-ready yet. The compiler is stable (41/41 tests, 10 example specs compiling clean), but there are known gaps: +- Parser doesn't handle `precondition:`/`postcondition:` assertions yet +- Generated TypeScript doesn't emit record type aliases yet +- No JS API wrapper for WASM modules + +I'm targeting production-readiness for the compiler pipeline by end of May. Right now it's best for experimentation, learning, and providing feedback that shapes the roadmap. + +--- + +## RESPONSE 12: "What's the most surprising thing you learned building this?" + +**Response:** +That the compiler architecture matters more than the language design. I spent way more time on the generator pipeline (AST → 5 artifacts) than on the parser and syntax. + +The "aha" moment was realizing that WASM codegen and TypeScript codegen are fundamentally the same operation — walking an AST and emitting target code. Once the AST is right, adding new output formats is mostly mechanical. + +Also: wabt (WebAssembly Binary Toolkit) is an incredible piece of software. Being able to validate generated WAT through a real assembler instead of writing my own binary encoder saved weeks. + +--- + +## RESPONSE 13 (SEEDED QUESTION): "Could this be used for smart contract development?" + +**Response:** +Absolutely — that's one of the most exciting use cases. Smart contracts are state machines with money at stake, and bugs are irreversible. + +SpeckDL specs could compile to Solidity or Move, with the provenance trail serving as audit evidence for DeFi protocols. The WASM output already works for chains that support WASM smart contracts (Polkadot, NEAR, Cosmos). + +This is on the roadmap — if anyone's interested in a Solidity generator, I'd love to collaborate. + +--- + +## RESPONSE 14 (SEEDED QUESTION): "How does this compare to AWS IAM policy-as-code or Open Policy Agent?" + +**Response:** +Different layer of the stack! OPA and IAM policy-as-code are about *authorization rules* — who can do what. Speckl is about *system behavior* — what happens when they do it. + +You could use SpeckDL to specify the state machine of an authorization system (e.g., "a user in pending-approval state cannot access resources"), and OPA to enforce the policy. They're complementary — Speckl specifies behavior, OPA enforces access. + +--- + +## Engagement Rules + +1. **Reply within 15 minutes** — HN's algorithm rewards early engagement +2. **Be genuine** — don't copy-paste verbatim, adapt to the actual question +3. **Link to evidence** — always point to the repo, tutorial, or specific code +4. **Accept criticism gracefully** — "You're right, that's a gap. Here's the plan" +5. **Don't argue** — if someone hates it, thank them for the feedback and move on +6. **Upvote thoughtful replies** — even critical ones, if they're substantive +7. **First comment** — post immediately after the Show HN with: "I built this because [origin story]. Happy to answer questions about the compiler architecture, the language design decisions, or the distribution roadmap." + +--- + +## Timing Notes + +- **HN front page velocity peaks:** 7-10 AM PT, 12-2 PM PT +- **Stay on for 90 minutes** after posting — that's the window for hitting the front page +- **If it hits front page:** expect 50-200 comments. Triaging which to reply to matters more than replying to everything +- **Don't check votes obsessively** — it's visible to you but not others diff --git a/docs/show-hn-demo-script.md b/docs/show-hn-demo-script.md new file mode 100644 index 0000000..842b8bc --- /dev/null +++ b/docs/show-hn-demo-script.md @@ -0,0 +1,165 @@ +# Show HN Demo Script — Speckl + +**Purpose:** 2-3 minute terminal recording for the Show HN post (May 12-13) +**Format:** Terminal screen recording (asciinema or screen-capture GIF) +**Goal:** Show "spec in, 5 auditable artifacts out" in under 3 minutes + +## Pre-flight (before recording) + +```bash +# Verify compiler is clean +cd ~/speckl/compiler +npm test # should show 41/41 passing + +# Clear output +rm -rf out/demo-script +``` + +## Scene 0: Title Card (5 sec) +Show `~/speckl/examples/` directory listing. Voiceover or text overlay: +> "Speckl is a judgement transport utility for AI-generated specifications. Let me show you what happens when you compile a spec." + +## Scene 1: The Spec (15 sec) + +```bash +cat examples/ToggleSwitch.speck +``` + +Narrate: +- "This is SpeckDL — a specification language designed for both humans and compilers." +- "It has types, records, state variables, actions with guards, preconditions, and postconditions." +- "This is the same ToggleSwitch you'd write in TLA+, but with one difference: it compiles." + +## Scene 2: One Command, Five Artifacts (20 sec) + +```bash +node dist/index.js examples/ToggleSwitch.speck -o out/demo-script +ls -la out/demo-script/ +``` + +Narrate: +- "One command. Five artifacts." +- "A PROV-O provenance graph, a TypeScript state machine, WebAssembly, a CycloneDX BOM, and an SPDX BOM." +- "Every one of these is independently auditable." + +## Scene 3: The TypeScript (20 sec) + +```bash +cat out/demo-script/ToggleSwitch.ts +``` + +Narrate: +- "This is production-ready TypeScript generated from the spec." +- "The state machine is a class with typed methods for every action." +- "Guards become assertions. Actions become mutations." + +## Scene 4: TypeScript Compiles Clean (10 sec) + +```bash +npx tsc --noEmit out/demo-script/ToggleSwitch.ts +echo "Exit code: $?" # should be 0 +``` + +Narrate: +- "It compiles with zero errors. You could import this into any TypeScript project right now." + +## Scene 5: The WASM (15 sec) + +```bash +head -40 out/demo-script/ToggleSwitch.wat +``` + +Narrate: +- "It also compiles to WebAssembly. This is the exact same spec — same toggle logic — but now it runs in any browser or edge runtime." +- "One spec, multiple targets. That's the power of compiler-level provenance." + +## Scene 6: The Provenance Graph (15 sec) + +```bash +python3 -m json.tool out/demo-script/ToggleSwitch.prov.jsonld | head -30 +``` + +Narrate: +- "Every artifact links back to the original spec through a W3C PROV provenance graph." +- "This is what NIST SA-11 calls 'human-in-the-loop for automated outputs.'" +- "You can prove what ran came from what you wrote." + +## Scene 7: The BOMs (15 sec) + +```bash +cat out/demo-script/ToggleSwitch.specbom.cdx.json | head -20 +echo "..." +cat out/demo-script/ToggleSwitch.specbom.spdx.json | head -20 +``` + +Narrate: +- "And you get two machine-readable bills of materials — CycloneDX and SPDX." +- "This means your compliance team can import Speckl's output directly into their existing toolchain." + +## Scene 8: TigerBeetle (15 sec) — CONDITIONAL + +Only include if engineering#9 + #11 are fixed by launch day: + +```bash +node dist/index.js examples/TigerBeetleLedger.speck -o out/demo-tiger +npx tsc --noEmit out/demo-tiger/TigerBeetleLedger.ts +echo "Exit code: $?" # should be 0 if bugs fixed +``` + +Narrate: +- "And it scales. Here's a naive TigerBeetle port — a double-entry ledger with 7 actions, 5 preconditions, and 4 postconditions." +- "Zero TypeScript errors. Production-ready from a spec." + +If NOT fixed by launch day, skip this scene. Instead show: + +```bash +cat examples/TigerBeetleLedger.speck | head -40 +``` + +Narrate: +- "And here's what we're working on next: a full TigerBeetle-style double-entry ledger. Spec written, compilation in progress. Follow the repo for updates." + +## Scene 9: Call to Action (10 sec) + +Show the repo URL and landing page: + +```bash +echo "github: https://os.scoble.me/forgejo/sscoble/speckl" +echo "mirror: https://codeberg.org/sscoble/speckl" +echo "site: https://speckl.scoble.me" +echo "spec: speckl.scoble.me#try-it — paste a spec and see output" +``` + +Narrate: +- "Speckl is MIT-licensed and open source. The whitepaper is in the repo. I'd love your feedback." + +## Recording Tips + +1. **Use asciinema** for terminal recording — it produces lightweight `.cast` files that can be embedded or converted to GIF + ```bash + asciinema rec speckl-demo.cast + ``` +2. **Or use terminalizer** for a styled GIF with a config file +3. **Keep it under 3 minutes** — Show HN attention spans are short +4. **No audio needed** — terminal text + captions work better for HN +5. **Paste the asciinema link or GIF** directly in the Show HN post body + +## Alternative: Static Screenshot Flow + +If terminal recording is too much setup, use 5-6 screenshots in sequence: +1. The `ToggleSwitch.speck` source +2. The `ls` output showing 5 generated files +3. A snippet of generated TypeScript +4. The `tsc --noEmit` clean exit +5. A snippet of the PROV-O JSON +6. The landing page with "Try It" CTA + +Less impressive than a recording, but zero setup time. + +## Post-Launch + +After Show HN goes live, share the demo recording on: +- Twitter/X with the HN link +- LinkedIn with a short writeup +- r/programming (Reddit) +- Speckl landing page (embed the asciinema player) diff --git a/docs/show-hn-launch-checklist.md b/docs/show-hn-launch-checklist.md new file mode 100644 index 0000000..1996f7a --- /dev/null +++ b/docs/show-hn-launch-checklist.md @@ -0,0 +1,260 @@ +# Show HN Launch Day Operations Checklist +**For: Scott Scoble | Launch: May 12 (Monday) or May 13 (Tuesday), 2026** +**Prepared: May 9, 2026, 2:04 AM PT (Ventures 🚀)** + +--- + +## TL;DR: What Scott Actually Does + +1. Pick Monday or Tuesday morning +2. Run the 5-minute pre-flight check below +3. Paste the Show HN post +4. Watch comments for the first hour — use the comment playbook +5. At +4 hours, cross-post to Reddit + share link on Discord/Slack groups +6. At +24 hours, check if it hit front page and decide next steps + +--- + +## PRE-LAUNCH: Day Before (Sunday or Monday night) + +### 5-Minute Pre-Flight Check + +Run these in a terminal: + +```bash +# 1. ALL LINKS WORK +curl -sI https://speckl.scoble.me | head -1 # → 200 OK +curl -sI https://os.scoble.me/forgejo/sscoble/speckl | head -1 # → 200 +curl -sI https://codeberg.org/sscoble/speckl | head -1 # → 200 + +# 2. COMPILER IS STABLE +cd ~/speckl/compiler && npm test # → 41/41 passing + +# 3. DEMO PAGE WORKS +curl -sI https://speckl.scoble.me/demo/index.html | head -1 # → 200 OK + +# 4. WHITEPAPER IS ACCESSIBLE +wc -l ~/speckl/docs/whitepaper-v2.md # → ~1,300+ lines + +# 5. BLOG POSTS RENDER (optional — pick your 2 best for linking) +# Blog posts: ~/speckl/docs/blog-*.md +``` + +If Storj is still down: **abort launch**. Without the landing page, the post loses all credibility. Scott must fix `rclone config` or republish the landing page another way first. + +### Choose Your Timing + +| Option | Day | Post Time PT | UTC | Why | +|--------|-----|-------------|-----|-----| +| **A (best)** | Tue May 13 | 1:00 AM PT | 8:00 AM UTC | Catches EU morning + US late night | +| **B** | Tue May 13 | 6:00 AM PT | 1:00 PM UTC | Catches US morning, EU afternoon | +| **C** | Mon May 12 | 6:00 AM PT | 1:00 PM UTC | Monday is slightly worse than Tuesday | + +**Recommendation:** Option A if Scott can pre-schedule or wake up early. Option B if he wants a normal morning. + +### HN Account Check +- Log into HN. Verify karma ≥ 2 (minimum to post Show HN). +- If karma < 2: Reply to 2-3 comments on front page stories to get karma. Takes 10 minutes. +- Bookmark: https://news.ycombinator.com/submit + +--- + +## LAUNCH DAY: T-5 Minutes + +### Open These Tabs + +1. **HN Submit:** https://news.ycombinator.com/submit +2. **Comment Playbook:** Find it at `~/speckl/docs/show-hn-comment-playbook.md` (open in editor or browser) +3. **Your HN Threads page:** https://news.ycombinator.com/threads?id=YOUR_USERNAME (to monitor comments) +4. **Announcement post source:** `~/speckl/docs/show-hn-announcement.md` + +### Final Sanity Check (30 seconds) +- Title starts with "Show HN:" ✅ +- Landing page resolves ✅ +- GitHub/code link is a real repo (not just docs) ✅ +- Post body has actual code examples or links to them ✅ +- Whitepaper link works ✅ + +--- + +## T+0: POST IT + +Go to https://news.ycombinator.com/submit + +**Title:** +``` +Show HN: Speckl — Write specs like TLA+, compile to runnable TypeScript and WASM +``` + +**URL:** +``` +https://speckl.scoble.me +``` +(Link to the landing page — it has links to the repo, whitepaper, demo. HN prefers clean landing pages over direct repo links for Show HN.) + +**Text:** (optional — only if using self-post format) +Paste the body from `show-hn-announcement.md`. If HN's URL field is used (recommended), add the first paragraph as the first comment instead. + +### Immediately After Posting: +- Copy the post URL (it'll be `https://news.ycombinator.com/item?id=XXXXXXX`) +- Save it — you'll need it for cross-posting + +--- + +## FIRST HOUR: The Golden Window (T+0 to T+60) + +This is when HN's ranking algorithm is most sensitive to engagement. Every upvote and comment thread counts quadruple. + +### What To Do: +1. **Reply to comments within 5 minutes.** Speed signals "builder is present." +2. **Use the comment playbook** (`show-hn-comment-playbook.md`) — it has responses to 14 common objections pre-written. Copy-paste-adapt. Do NOT write responses from scratch under time pressure. +3. **Be human.** If someone says "cool project," say thanks and ask if they have a use case. If someone finds a bug, thank them and log it. +4. **Don't get defensive.** Someone WILL call Speckl "another TLA+ clone." Use the pre-written response. Don't argue. Move on. +5. **Upvote helpful comments** from others — it keeps the thread active. + +### Do NOT: +- Ask friends to upvote (HN detects voting rings) +- Post multiple top-level comments yourself +- Link to "buy now" or pricing (there is none — keep it pure Show HN) +- Engage with obvious trolls. One polite reply, then ignore. + +### Monitor Metrics: +- Upvote count (refresh every 10 min) +- Comment count +- Position on /show or /new vs /news (front page) + +--- + +## T+1 TO T+4: Sustain Engagement + +- Check comments every 30 min +- Reply to new questions using the playbook +- If post is on /show but not /news: normal. Show HN needs ~10+ upvotes in first 2 hours to graduate to /news. + +### First Cross-Post: Reddit + +**Only if the post has 15+ upvotes and positive comments.** A dead post doesn't need amplification. + +Post to r/programming (2.5M members): + +**Title:** Speckl — Write specs like TLA+, compile to TypeScript and WASM (open source, Show HN'd today) + +**Link:** Your HN post URL (not the landing page — Redditors want to see the HN discussion) + +**Flair:** Use if available. + +--- + +## T+4 TO T+24: Extended Monitoring + +- Check comments every 2-3 hours +- If post hits front page (top 30 on /news): engage actively with ALL comments — this is your moment +- If post stalls at 5-15 upvotes and sinks: don't despair. Bookmark the HN post URL. You can reuse the announcement for a future Show HN in 3-6 months with "v0.3" or a major milestone. + +### Additional Cross-Posts (Only If Traction Is Good): + +| Platform | When | Post Type | +|----------|------|-----------| +| r/Compilers (28K) | T+4h | Link to HN discussion | +| r/rust (300K) | T+4h | WASM angle — "compiles to WASM from formal specs" | +| Twitter/Bluesky | T+1h | Link to HN post + screenshot of compiler output | +| Lobsters | T+2h | Submit as "Speckl: a spec language that compiles to TypeScript + WASM" | + +--- + +## T+24: Debrief + +### If It Hit Front Page (top 30 on /news): +- 🎉 You have developer credibility. Now: + 1. Submit whitepaper to arXiv TODAY (ride the momentum) + 2. Cross-post to all platforms listed above + 3. Email the formal methods communities (tlaplus group, FME, etc.) + 4. Consider a follow-up Show HN in 2-4 weeks with "Speckl v0.3: now compiling real-world state machines" + +### If It Got Moderate Traction (10-30 upvotes, some comments): +- Valuable signal. Extract every comment as product feedback. +- Update the FAQ/landing page based on common questions. +- Re-submit after a major compiler milestone (pre/postcondition support, record types, etc.) + +### If It Got No Traction (< 10 upvotes): +- Show HN timing is fickle. Does NOT mean the project is bad. +- Check: was the title good? Was the landing page slow/down? Did you post at a bad time? +- Try again in 3-6 months with a substantially improved product. +- Don't delete the post — HN doesn't let you. Just move on. + +--- + +## CONTINGENCY: If Something Breaks + +| Problem | Response | +|---------|----------| +| **Landing page down** | Post a comment immediately: "Landing page is struggling — here's the direct repo link." Link to Forgejo. | +| **Compiler tests failing** | Don't mention it unless asked. Fix it after the post dies down. | +| **Someone finds a real bug** | "Great catch — filed. Here's the issue link." Link to marcus/engineering. Shows you take bugs seriously. | +| **Troll/spam comments** | Flag. Do not engage beyond one polite reply. HN mods are active. | +| **Post flagged as spam** | Email hn@ycombinator.com. Explain it's open source, no commercial product. They unflag quickly. | +| **Storj goes down mid-launch** | You're hosed for landing page links. Post the Forgejo repo link as a comment. Redirect the landing page to a GitHub Pages mirror (if you set one up in advance). | + +--- + +## SEEDED QUESTIONS (Optional — Use Sparingly) + +If you have 1-2 trusted friends who can post natural-sounding questions, pick from these: + +1. "How does provenance tracking work in the compiler? Do you embed hashes in the artifacts?" + → Use Response 4 from the playbook. + +2. "What made you choose to generate both TypeScript AND WASM instead of picking one?" + → Use Response 10 from the playbook. + +Do NOT fabricate fake accounts. HN detects this and it's worse than no engagement. + +--- + +## Post-Launch: What Ships Next + +Regardless of Show HN outcome, the compiler needs these fixes before it's "real": + +1. **engineering#9:** Parser must capture `precondition:` / `postcondition:` blocks in action bodies +2. **engineering#11:** TypeScript generator must emit type aliases for record declarations (`Transfer`, `LogEntry`) +3. **engineering#10 (if exists):** Host-side record boxing + `in` operator for WASM + +These are the blockers between "cool demo" and "usable tool." Show HN traction helps motivate these fixes; no traction just means they take longer. + +--- + +## Quick Reference Card + +``` +┌─────────────────────────────────────────────────────────┐ +│ SHOW HN LAUNCH — SPEEDRUN │ +│ │ +│ Day Before: │ +│ ☐ 5-min pre-flight (curl landing, test compiler) │ +│ ☐ Storj is up? (if not, ABORT) │ +│ ☐ HN karma ≥ 2? │ +│ │ +│ T-5 min: │ +│ ☐ Open HN submit + comment playbook + threads page │ +│ │ +│ T+0: │ +│ ☐ Paste title + URL + body │ +│ ☐ Save post URL │ +│ │ +│ T+0 to T+60 (GOLDEN HOUR): │ +│ ☐ Reply within 5 min using playbook │ +│ ☐ Monitor upvote count every 10 min │ +│ ☐ Be human, don't argue, don't astroturf │ +│ │ +│ T+4h: │ +│ ☐ Cross-post to Reddit (if 15+ upvotes) │ +│ ☐ Share on Discord/Slack/Twitter │ +│ │ +│ T+24h: │ +│ ☐ Debrief: front page / moderate / dead │ +│ ☐ Act based on outcome │ +│ │ +│ CONTINGENCY: If landing page dies → link to repo │ +│ If flagged → email hn@ycombinator │ +└─────────────────────────────────────────────────────────┘ +``` diff --git a/docs/sir-schema.md b/docs/sir-schema.md new file mode 100644 index 0000000..4fe133a --- /dev/null +++ b/docs/sir-schema.md @@ -0,0 +1,1484 @@ +# Speckl Structured Intermediate Representation (SIR) — Schema Design + +## Overview + +The SIR is the central data structure in the Speckl pipeline. It bridges chaotic human/agent conversation and deterministic compilation: + +``` +Chaotic Conversation → [Extraction Layer] → SIR → [Validation Layer] → [Compilation Layer] +``` + +Everything flows through the SIR. It is the single source of truth — the document being edited by conversation and compiled into artifacts. + +### Design Principles + +1. **Domain-agnostic core, domain-specific leaves.** The SIR has a universal structure (elements, constraints, provenance) with domain-specific element types and validators. +2. **Provenance is first-class.** Every element, every property, every constraint traces back to the conversation turn that produced it. Nothing is untraceable. +3. **Confidence is quantified.** Every element carries a confidence score. Compilation can be gated on confidence thresholds. +4. **Content-addressed, not clock-addressed.** The SIR is versioned by content hash, not timestamps. Determinism depends on this. +5. **Incrementally updateable.** The SIR is a mutable document that conversation refines over time, with full diff tracking. +6. **Wire-friendly.** JSON as the primary serialization format with a defined schema. + +--- + +## 1. Core Structure + +### 1.1 The SIR Document + +```typescript +interface SIRDocument { + // Identity + sirId: string; // UUID, stable for the lifetime of this specification + sirVersion: number; // Monotonic counter, incremented on every mutation + sirHash: string; // sha256 of the canonical JSON (computed on save) + + // Declaration + schemaVersion: "1.0.0"; // SIR schema version (not the spec version) + domain: DomainDeclaration; + + // Content + elements: Record; // elementId → element + constraints: SIRConstraint[]; // cross-element constraints + composition: CompositionRule[]; // how elements compose + + // Metadata + metadata: SIRMetadata; +} + +interface SIRMetadata { + title: string; // Human-readable name: "Login Flow", "User Profile Schema" + description: string; // Free-text description of what this spec defines + tags: string[]; // For searchability: ["auth", "security", "login"] + createdAt: number; // Unix ms + updatedAt: number; // Unix ms + createdBy: string; // agentId or "human" + conversationId?: string; // Link to the conversation that produced this + readiness: ReadinessLevel; +} + +type ReadinessLevel = "draft" | "functional" | "validated" | "locked"; +``` + +### 1.2 Concrete JSON Example — Empty SIR + +```json +{ + "sirId": "b8d4f3a2-1c6e-4f89-a3e7-2b1c5d9f0a4e", + "sirVersion": 1, + "sirHash": "sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "schemaVersion": "1.0.0", + "domain": { + "primary": "state-machine", + "version": "1.0.0", + "secondary": [] + }, + "elements": {}, + "constraints": [], + "composition": [], + "metadata": { + "title": "Untitled Specification", + "description": "", + "tags": [], + "createdAt": 1778392000000, + "updatedAt": 1778392000000, + "createdBy": "agent-engineering", + "readiness": "draft" + } +} +``` + +--- + +## 2. Element Model + +### 2.1 What Is a Specification Element? + +A specification element is a typed, identified node in the SIR graph. Every element has a common header and domain-specific properties. Elements reference each other, forming a directed graph. + +### 2.2 The Common Header + +```typescript +interface SIRElement { + // Identity + id: string; // UUID, stable across revisions + elementType: string; // Domain-specific type: "state", "transition", "table", "step", etc. + + // Lifecycle + status: ElementStatus; + version: number; // Monotonic counter for this element + + // Provenance + provenance: ProvenanceChain; + + // Content + label: string; // Human-readable: "Logged Out", "User Table" + description: string; // Natural language: what is this, why does it exist + properties: Record; // Typed key-value store + + // Relationships + references: ElementReference[]; // What this element points to + subElements: string[]; // Child element IDs (composition) + + // Confidence + confidence: number; // 0.0 – 1.0 + confidenceBreakdown: ConfidenceBreakdown; + + // Flags + flags: ElementFlags; + + // Timestamps + createdAt: number; + updatedAt: number; +} + +type ElementStatus = + | "proposed" // Extracted from conversation, not yet confirmed + | "draft" // Confirmed but incomplete + | "validated" // Passed validation + | "locked" // Cannot be modified (compiled artifact references this) + | "deprecated" // Superseded by newer element + | "rejected"; // Human explicitly rejected this extraction +``` + +### 2.3 Property Values + +Properties are typed so validators and compilers can reason about them: + +```typescript +type PropertyValue = + | { type: "string"; value: string } + | { type: "number"; value: number } + | { type: "boolean"; value: boolean } + | { type: "enum"; value: string; allowedValues: string[] } + | { type: "reference"; value: string; targetType: string } + | { type: "array"; value: PropertyValue[]; itemType: string } + | { type: "code"; value: string; language: string } + | { type: "expression"; value: string; expressionType: string }; + +// Each property value includes its own provenance, confidence, and flags +interface PropertyValue { + type: PropertyType; + value: any; + confidence: number; + provenance: ProvenanceChain; // Who set this value, when + flags: { tentative: boolean; needsReview: boolean }; +} +``` + +### 2.4 Element References + +References form the graph: + +```typescript +interface ElementReference { + refId: string; // Target element ID + refType: string; // What kind of reference: "target", "parent", "depends_on", "implements" + label: string; // Human-readable: "goes to", "belongs to", "validates" + cardinality: string; // "1", "0..1", "1..n", "0..n" + conditions?: string; // When this reference is active: "when guard evaluates true" + confidence: number; + provenance: ProvenanceChain; +} +``` + +### 2.5 ElementFlags + +```typescript +interface ElementFlags { + needsReview: boolean; + isAmbiguous: boolean; + isTentative: boolean; + isContradicted: boolean; + sourceIsExplicit: boolean; + hasGaps: boolean; // Missing required properties +} +``` + +### 2.6 Confidence Breakdown + +Confidence isn't just a single number — it decomposes: + +```typescript +interface ConfidenceBreakdown { + overall: number; + factors: { + explicitMarker: number; // 0–1: strength of explicit extraction cues + structuralPattern: number; // 0–1: how well the text matches known patterns + consistency: number; // 0–1: consistency with rest of SIR + speakerAuthority: number; // 0–1: human (0.9) vs agent (0.6) vs debated (0.4) + repetition: number; // 0–1: mentioned across multiple turns + }; + lastEvaluated: number; // Unix ms +} +``` + +### 2.7 Concrete JSON Example — State Machine Elements + +A login flow with states, transitions, and guards: + +```json +{ + "state-logged-out": { + "id": "state-logged-out", + "elementType": "state", + "status": "validated", + "version": 1, + "provenance": { + "entries": [ + { + "turnId": "turn-3", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": true, + "timestamp": 1778392100000, + "confidenceAtTime": 0.92, + "extractionRationale": "Explicit state declaration with spec: prefix" + } + ] + }, + "label": "Logged Out", + "description": "User is not authenticated. Entry point for the login flow.", + "properties": { + "isInitial": { "type": "boolean", "value": true }, + "isTerminal": { "type": "boolean", "value": false } + }, + "references": [ + { "refId": "transition-login-submit", "refType": "outgoing", "label": "on submit", "cardinality": "1" } + ], + "subElements": [], + "confidence": 0.92, + "confidenceBreakdown": { + "overall": 0.92, + "factors": { + "explicitMarker": 1.0, + "structuralPattern": 0.95, + "consistency": 0.90, + "speakerAuthority": 0.90, + "repetition": 0.85 + }, + "lastEvaluated": 1778392100000 + }, + "flags": { + "needsReview": false, + "isAmbiguous": false, + "isTentative": false, + "isContradicted": false, + "sourceIsExplicit": true, + "hasGaps": false + }, + "createdAt": 1778392100000, + "updatedAt": 1778392100000 + }, + + "state-logging-in": { + "id": "state-logging-in", + "elementType": "state", + "status": "validated", + "version": 1, + "provenance": { + "entries": [ + { + "turnId": "turn-5", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": false, + "timestamp": 1778392150000, + "confidenceAtTime": 0.78, + "extractionRationale": "Implicit from 'while the credentials are being checked' description" + } + ] + }, + "label": "Logging In", + "description": "Credentials submitted, awaiting authentication result.", + "properties": { + "isInitial": { "type": "boolean", "value": false }, + "isTerminal": { "type": "boolean", "value": false }, + "timeout": { "type": "number", "value": 30000 }, + "timeoutAction": { "type": "string", "value": "return error to user" } + }, + "references": [ + { "refId": "transition-credential-check", "refType": "incoming", "label": "entered from", "cardinality": "1" }, + { "refId": "transition-auth-success", "refType": "outgoing", "label": "on success", "cardinality": "1" }, + { "refId": "transition-auth-failure", "refType": "outgoing", "label": "on failure", "cardinality": "1" }, + { "refId": "transition-auth-timeout", "refType": "outgoing", "label": "on timeout", "cardinality": "0..1" } + ], + "subElements": [], + "confidence": 0.78, + "confidenceBreakdown": { + "overall": 0.78, + "factors": { + "explicitMarker": 0.30, + "structuralPattern": 0.80, + "consistency": 0.90, + "speakerAuthority": 0.90, + "repetition": 0.60 + }, + "lastEvaluated": 1778392150000 + }, + "flags": { + "needsReview": true, + "isAmbiguous": false, + "isTentative": true, + "isContradicted": false, + "sourceIsExplicit": false, + "hasGaps": false + }, + "createdAt": 1778392150000, + "updatedAt": 1778392150000 + }, + + "state-authenticated": { + "id": "state-authenticated", + "elementType": "state", + "status": "validated", + "version": 1, + "provenance": { + "entries": [ + { + "turnId": "turn-3", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": true, + "timestamp": 1778392100000, + "confidenceAtTime": 0.95, + "extractionRationale": "Explicit state declaration with spec: prefix" + } + ] + }, + "label": "Authenticated", + "description": "User has valid session. The 'inside' state for the application.", + "properties": { + "isInitial": { "type": "boolean", "value": false }, + "isTerminal": { "type": "boolean", "value": false }, + "sessionDuration": { "type": "number", "value": 3600000 }, + "onEntry": { "type": "code", "value": "createSession(user); redirect('/dashboard')", "language": "pseudocode" } + }, + "references": [ + { "refId": "transition-auth-success", "refType": "incoming", "label": "entered via", "cardinality": "1" }, + { "refId": "transition-logout", "refType": "outgoing", "label": "on logout", "cardinality": "1" }, + { "refId": "transition-session-expire", "refType": "outgoing", "label": "on expiry", "cardinality": "1" } + ], + "subElements": [], + "confidence": 0.95, + "confidenceBreakdown": { + "overall": 0.95, + "factors": { + "explicitMarker": 1.0, + "structuralPattern": 0.95, + "consistency": 0.95, + "speakerAuthority": 0.90, + "repetition": 0.90 + }, + "lastEvaluated": 1778392100000 + }, + "flags": { + "needsReview": false, + "isAmbiguous": false, + "isTentative": false, + "isContradicted": false, + "sourceIsExplicit": true, + "hasGaps": false + }, + "createdAt": 1778392100000, + "updatedAt": 1778392100000 + }, + + "state-locked-out": { + "id": "state-locked-out", + "elementType": "state", + "status": "validated", + "version": 1, + "provenance": { + "entries": [ + { + "turnId": "turn-8", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": true, + "timestamp": 1778392250000, + "confidenceAtTime": 0.88, + "extractionRationale": "Extracted from 'after 3 failed attempts, lock the account for 15 minutes'" + } + ] + }, + "label": "Locked Out", + "description": "Account temporarily locked after too many failed attempts.", + "properties": { + "isInitial": { "type": "boolean", "value": false }, + "isTerminal": { "type": "boolean", "value": false }, + "lockDuration": { "type": "number", "value": 900000 }, + "maxAttempts": { "type": "number", "value": 3 } + }, + "references": [ + { "refId": "transition-auth-failure", "refType": "incoming", "label": "entered via", "cardinality": "1" }, + { "refId": "transition-lock-expire", "refType": "outgoing", "label": "on expiry", "cardinality": "1" } + ], + "subElements": [], + "confidence": 0.88, + "confidenceBreakdown": { + "overall": 0.88, + "factors": { + "explicitMarker": 0.60, + "structuralPattern": 0.90, + "consistency": 0.90, + "speakerAuthority": 0.90, + "repetition": 0.75 + }, + "lastEvaluated": 1778392250000 + }, + "flags": { + "needsReview": false, + "isAmbiguous": false, + "isTentative": false, + "isContradicted": false, + "sourceIsExplicit": false, + "hasGaps": false + }, + "createdAt": 1778392250000, + "updatedAt": 1778392250000 + }, + + "transition-login-submit": { + "id": "transition-login-submit", + "elementType": "transition", + "status": "validated", + "version": 1, + "provenance": { + "entries": [ + { + "turnId": "turn-4", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": true, + "timestamp": 1778392120000, + "confidenceAtTime": 0.94, + "extractionRationale": "Explicit transition declared: 'when user submits credentials, transition to logging-in'" + } + ] + }, + "label": "Submit Credentials", + "description": "User submits username/password from login form.", + "properties": { + "trigger": { "type": "string", "value": "form_submit" }, + "from": { "type": "reference", "value": "state-logged-out", "targetType": "state" }, + "to": { "type": "reference", "value": "state-logging-in", "targetType": "state" } + }, + "references": [], + "subElements": ["guard-validate-input"], + "confidence": 0.94, + "confidenceBreakdown": { + "overall": 0.94, + "factors": { + "explicitMarker": 1.0, + "structuralPattern": 0.95, + "consistency": 0.95, + "speakerAuthority": 0.90, + "repetition": 0.90 + }, + "lastEvaluated": 1778392120000 + }, + "flags": { + "needsReview": false, + "isAmbiguous": false, + "isTentative": false, + "isContradicted": false, + "sourceIsExplicit": true, + "hasGaps": false + }, + "createdAt": 1778392120000, + "updatedAt": 1778392120000 + }, + + "transition-credential-check": { + "id": "transition-credential-check", + "elementType": "transition", + "status": "validated", + "version": 1, + "provenance": { + "entries": [ + { + "turnId": "turn-5", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": false, + "timestamp": 1778392150000, + "confidenceAtTime": 0.65, + "extractionRationale": "Inferred intermediate step between submit and auth result" + } + ] + }, + "label": "Check Credentials", + "description": "System validates credentials against auth provider.", + "properties": { + "trigger": { "type": "string", "value": "internal" }, + "from": { "type": "reference", "value": "state-logging-in", "targetType": "state" }, + "action": { "type": "code", "value": "authProvider.verify(username, password)", "language": "pseudocode" } + }, + "references": [], + "subElements": [], + "confidence": 0.65, + "confidenceBreakdown": { + "overall": 0.65, + "factors": { + "explicitMarker": 0.10, + "structuralPattern": 0.70, + "consistency": 0.80, + "speakerAuthority": 0.60, + "repetition": 0.30 + }, + "lastEvaluated": 1778392150000 + }, + "flags": { + "needsReview": true, + "isAmbiguous": false, + "isTentative": true, + "isContradicted": false, + "sourceIsExplicit": false, + "hasGaps": true + }, + "createdAt": 1778392150000, + "updatedAt": 1778392150000 + }, + + "transition-auth-success": { + "id": "transition-auth-success", + "elementType": "transition", + "status": "validated", + "version": 1, + "provenance": { + "entries": [ + { + "turnId": "turn-6", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": true, + "timestamp": 1778392180000, + "confidenceAtTime": 0.96, + "extractionRationale": "Explicit: 'if credentials valid, go to authenticated'" + } + ] + }, + "label": "Auth Success", + "description": "Credentials are valid. Transition to authenticated state.", + "properties": { + "trigger": { "type": "string", "value": "auth_success" }, + "from": { "type": "reference", "value": "state-logging-in", "targetType": "state" }, + "to": { "type": "reference", "value": "state-authenticated", "targetType": "state" } + }, + "references": [], + "subElements": ["guard-credentials-valid"], + "confidence": 0.96, + "confidenceBreakdown": { + "overall": 0.96, + "factors": { + "explicitMarker": 1.0, + "structuralPattern": 0.95, + "consistency": 0.95, + "speakerAuthority": 0.90, + "repetition": 0.95 + }, + "lastEvaluated": 1778392180000 + }, + "flags": { + "needsReview": false, + "isAmbiguous": false, + "isTentative": false, + "isContradicted": false, + "sourceIsExplicit": true, + "hasGaps": false + }, + "createdAt": 1778392180000, + "updatedAt": 1778392180000 + }, + + "guard-credentials-valid": { + "id": "guard-credentials-valid", + "elementType": "guard", + "status": "validated", + "version": 1, + "provenance": { + "entries": [ + { + "turnId": "turn-6", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": true, + "timestamp": 1778392180000, + "confidenceAtTime": 0.96, + "extractionRationale": "Guard condition: credentials valid" + } + ] + }, + "label": "Credentials Valid", + "description": "Username exists and password hash matches.", + "properties": { + "condition": { "type": "expression", "value": "user.exists AND password.hash == stored.hash", "expressionType": "boolean" } + }, + "references": [ + { "refId": "transition-auth-success", "refType": "guards", "label": "guards transition", "cardinality": "1" } + ], + "subElements": [], + "confidence": 0.96, + "confidenceBreakdown": { + "overall": 0.96, + "factors": { + "explicitMarker": 1.0, + "structuralPattern": 0.95, + "consistency": 0.95, + "speakerAuthority": 0.90, + "repetition": 0.90 + }, + "lastEvaluated": 1778392180000 + }, + "flags": { + "needsReview": false, + "isAmbiguous": false, + "isTentative": false, + "isContradicted": false, + "sourceIsExplicit": true, + "hasGaps": false + }, + "createdAt": 1778392180000, + "updatedAt": 1778392180000 + } +} +``` + +--- + +## 3. Domain System + +### 3.1 Domain Declaration + +Every SIR declares its domain. A domain defines what element types, constraints, and composition rules are valid. + +```typescript +interface DomainDeclaration { + primary: string; // "state-machine", "data-schema", "workflow", "protocol" + version: string; // Domain schema version: "1.0.0" + secondary: string[]; // Cross-reference domains: ["api-interface"] +} +``` + +### 3.2 Domain Primitives + +Each domain defines its element types, their required properties, and their semantics: + +**State Machine Domain (`state-machine@1.0.0`):** + +| Element Type | Description | Required Properties | +|---|---|---| +| `state` | A distinct condition or mode the system can be in | `isInitial`, `isTerminal` | +| `transition` | A directed edge between states, triggered by an event | `from`, `to`, `trigger` | +| `guard` | A boolean condition evaluated before taking a transition | `condition` | +| `event` | An external or internal occurrence that may trigger transitions | `type`, `payload` | +| `action` | A side effect executed on entry, exit, or during a transition | `phase` (entry/exit/during), `code` | +| `invariant` | A condition that must always hold in a given state | `expression` | + +**Data Schema Domain (`data-schema@1.0.0`):** + +| Element Type | Description | Required Properties | +|---|---|---| +| `entity` | A structured data type (table, document, object) | `name` | +| `field` | A named property within an entity | `name`, `type`, `nullable` | +| `relation` | A relationship between entities | `from`, `to`, `kind` (one-to-one/one-to-many/many-to-many) | +| `index` | A search/ordering structure on fields | `fields`, `unique` | +| `constraint` | A data integrity rule | `expression`, `kind` (check/unique/foreign-key) | +| `enum` | A constrained set of values | `name`, `values` | + +**Workflow Domain (`workflow@1.0.0`):** + +| Element Type | Description | Required Properties | +|---|---|---| +| `step` | A unit of work in a process | `name`, `actor` (human/system/both) | +| `decision` | A branching point | `condition`, `branches` | +| `fork` | Parallel execution split | `branches` | +| `join` | Parallel execution merge | `incoming` | +| `event` | A signal within the workflow | `type` (start/end/message/timer/error) | + +### 3.3 Domain Versioning + +When a domain schema changes (e.g., `state-machine@1.0.0` → `state-machine@1.1.0`), existing SIRs carry their declared domain version. The compiler resolves the appropriate domain validator and compiler plugins for that version. + +```json +{ + "domain": { + "primary": "state-machine", + "version": "1.0.0" + } +} +``` + +A migration path exists when a domain upgrades: + +```typescript +interface DomainUpgradePath { + from: string; // "state-machine@1.0.0" + to: string; // "state-machine@1.1.0" + migration: (sir: SIRDocument) => SIRDocument; // Transform function + breakingChanges: string[]; +} +``` + +### 3.4 Multi-Domain SIR + +A SIR can declare secondary domains for cross-domain references: + +```json +{ + "domain": { + "primary": "state-machine", + "version": "1.0.0", + "secondary": ["api-interface@1.0.0"] + } +} +``` + +This allows a state machine to reference an API endpoint specification for its actions, or a workflow to reference a data schema for its payload types. Cross-domain references carry an explicit domain prefix: + +```json +{ + "id": "some-action", + "elementType": "action", + "references": [ + { + "refId": "api-interface:endpoint-create-session", + "refType": "calls", + "label": "creates session via", + "cardinality": "1" + } + ] +} +``` + +--- + +## 4. Constraint Model + +### 4.1 Constraint Types + +Constraints encode the formal rules the specification must satisfy: + +```typescript +type SIRConstraint = + | TypeConstraint + | ReferenceConstraint + | InvariantConstraint + | TemporalConstraint + | RelationalConstraint; + +// "field 'email' must be a valid email string" +interface TypeConstraint { + kind: "type"; + target: string; // Element or property path: "entity-user.field-email" + check: string; // Expression: "matches(email_pattern)" +} + +// "transition 'auth-success' must have a target state" +interface ReferenceConstraint { + kind: "reference"; + source: string; // Element ID + refType: string; // Which reference: "to", "from" + mustExist: boolean; + mustBeType?: string; // Optional: reference must point to this type +} + +// "user.email must be unique across all users" +interface InvariantConstraint { + kind: "invariant"; + target: string; // Element or element set + expression: string; // Formal expression: "forall u1,u2: u1.id != u2.id => u1.email != u2.email" + language: "speckl-expr" | "fol"; +} + +// "after 'payment-submitted' state, eventually 'payment-confirmed'" +interface TemporalConstraint { + kind: "temporal"; + formula: string; // "G(payment-submitted => F payment-confirmed)" + logic: "LTL" | "CTL"; +} + +// "field-email must be indexed if entity-user has more than N records" +interface RelationalConstraint { + kind: "relational"; + expression: string; // "entity-user.cardinality > 1000000 => exists(index-email)" +} +``` + +### 4.2 Constraint Expression Language + +Constraints use a readable expression language that maps to formal verification: + +``` +Expression → Comparison | Logical | Quantified | Reference + +Comparison → value OPERATOR value + OPERATOR → == | != | < | > | <= | >= | matches | contains + +Logical → expr AND expr | expr OR expr | NOT expr + +Quantified → forall VAR in SET: expr | exists VAR in SET: expr + +Reference → elementId.property | elementId (boolean, exists check) +``` + +Examples: +``` +"state-logging-in.isInitial == false" +"transition-auth-success.to exists AND transition-auth-success.to == state-authenticated" +"forall t in transitions: t.from exists AND t.to exists" +"not exists s1,s2 in states: s1.isInitial == true AND s2.isInitial == true" // exactly one initial state +``` + +### 4.3 Constraint Attachment + +Every constraint carries provenance and confidence: + +```typescript +interface SIRConstraint { + id: string; + kind: string; + // ... type-specific fields ... + + // Metadata + severity: "error" | "warning" | "info"; + confidence: number; + provenance: ProvenanceChain; + humanConfirmed: boolean; + + // Resolution + resolved: boolean; // True if validation passes + resolvedAt?: number; + resolutionNote?: string; +} +``` + +### 4.4 Constraint Composition + +When elements compose (e.g., a state machine contains sub-state-machines), constraints compose hierarchically: + +- A constraint on a parent element implies constraints on all children +- A constraint satisfied at the child level satisfies it at the parent level (if the constraint is existential) +- Conflicting constraints at different levels produce a validation error + +```typescript +interface CompositionRule { + kind: "contains" | "requires" | "excludes" | "parallel-to" | "sequence-of"; + parent: string; // Parent element ID + children: string[]; // Child element IDs + constraints: SIRConstraint[]; // Constraints on this composition + provenance: ProvenanceChain; +} +``` + +--- + +## 5. Provenance Model + +### 5.1 ProvenanceChain + +Every element, every property, every constraint has a provenance chain. This is the foundation of NIST SA-11 traceability: + +```typescript +interface ProvenanceChain { + entries: ProvenanceEntry[]; +} + +interface ProvenanceEntry { + turnId: string; // ConversationTurn UUID that produced this + action: ProvenanceAction; + agentId?: string; // Which agent performed the action (or "human") + humanConfirmed: boolean; // Did the human explicitly sign off? + timestamp: number; // Unix ms + confidenceAtTime: number; // Confidence snapshot at this entry + extractionRationale?: string; // Why was this extracted/modified? + diff?: SIRDiff; // For refined/updated actions: what changed +} + +type ProvenanceAction = + | "extracted" // First created from conversation + | "refined" // Incremental update, same element + | "confirmed" // Human explicitly confirmed + | "deprecated" // No longer valid, superseded + | "rejected" // Human said this extraction was wrong + | "merged" // Result of merging two SIR versions + | "imported"; // Imported from external source +``` + +### 5.2 Provenance Chains + +Elements refined across multiple turns accumulate a chain: + +```json +{ + "id": "state-logging-in", + "provenance": { + "entries": [ + { + "turnId": "turn-5", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": false, + "timestamp": 1778392150000, + "confidenceAtTime": 0.45, + "extractionRationale": "Detected 'while checking credentials' as an implicit intermediate state" + }, + { + "turnId": "turn-7", + "action": "refined", + "agentId": "human", + "humanConfirmed": true, + "timestamp": 1778392220000, + "confidenceAtTime": 0.85, + "diff": { + "changedProperties": ["timeout", "timeoutAction"], + "addedReferences": ["transition-auth-timeout"] + }, + "extractionRationale": "Scott: 'Add a 30-second timeout to the logging-in state, show an error if it fires'" + }, + { + "turnId": "turn-12", + "action": "confirmed", + "agentId": "human", + "humanConfirmed": true, + "timestamp": 1778392400000, + "confidenceAtTime": 1.0, + "extractionRationale": "Scott: 'Yeah, the login flow looks right now.'" + } + ] + } +} +``` + +### 5.3 Confidence Tracking + +Confidence is tracked both at the element level and at the property level. A low-confidence property on a high-confidence element is flagged: + +```json +{ + "id": "state-logging-in", + "confidence": 0.85, + "properties": { + "timeout": { + "type": "number", + "value": 30000, + "confidence": 0.60, + "provenance": { + "entries": [{ + "turnId": "turn-7", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": false, + "extractionRationale": "Implicit: Scott mentioned '30 seconds' — extracted as ms" + }] + } + } + } +} +``` + +--- + +## 6. Versioning and Diffing + +### 6.1 SIR Versioning + +The SIR version is a content-hash-based identifier for integrity, plus a monotonic counter for human convenience: + +``` +sirVersion: 14 // Monotonic, incremented on every save +sirHash: "sha256:a7f3..." // Content hash of the canonical JSON +``` + +The `sirHash` is computed over the canonical JSON with all elements sorted by ID, minus the `sirHash` field itself and any timestamps (which are informational, not deterministic inputs). + +### 6.2 Element-Level Diffs + +```typescript +interface SIRDiff { + // What changed + addedElements: string[]; // Element IDs + removedElements: string[]; + modifiedElements: Record; + reorderedElements: { before: string[]; after: string[] }; + + // Metadata + fromSirVersion: number; + toSirVersion: number; + fromSirHash: string; + toSirHash: string; + timestamp: number; +} +``` + +### 6.3 Merge Semantics + +Two SIR versions can be merged when they derive from a common ancestor. Conflicts arise when both branches modify the same element property: + +```typescript +interface SIRMergeResult { + success: boolean; + merged: SIRDocument; + conflicts: SIRMergeConflict[]; +} + +interface SIRMergeConflict { + elementId: string; + property: string; + branchA: PropertyValue; // Value from first branch + branchB: PropertyValue; // Value from second branch + ancestor: PropertyValue; // Original value before branching + resolution?: PropertyValue; // Human-chosen resolution +} +``` + +Conflicts are represented directly in the SIR so conversation can resolve them — the agent surfaces the conflict and asks the human which value is correct. + +--- + +## 7. Serialization + +### 7.1 Primary Format: JSON + +The canonical serialization is JSON with a fixed key ordering for deterministic hashing: + +```json +{ + "sirId": "", + "sirVersion": 0, + "sirHash": "", + "schemaVersion": "", + "domain": {}, + "elements": {}, + "constraints": [], + "composition": [], + "metadata": {} +} +``` + +Elements within `elements` are keyed by their stable ID. Constraints and composition rules are arrays ordered by ID. + +### 7.2 Binary Format (Optional) + +For large SIRs, a CBOR-based binary format can be used for transmission. The canonical representation for hashing is always JSON. + +### 7.3 Human-Readable Summary + +For display in conversation, the SIR can render as a summary: + +``` +📋 **Login Flow** (state-machine@1.0.0) · v14 · sha256:a7f3... + +**States (4):** +🟢 Logged Out (initial) · confidence 0.92 ✅ +🔄 Logging In · confidence 0.85 ⚠️ +🔒 Authenticated · confidence 0.95 ✅ +🚫 Locked Out · confidence 0.88 ✅ + +**Transitions (5):** +→ Submit Credentials: logged-out → logging-in [trigger: form_submit] +→ Check Credentials: logging-in → (internal) [needs target] ⚠️ +→ Auth Success: logging-in → authenticated [guard: credentials-valid] +→ Auth Failure: logging-in → locked-out [missing guard] ⚠️ +→ Logout: authenticated → logged-out [missing] ❌ + +**Gaps: 2 warnings, 1 error** +``` + +This format is generated by the extraction layer for inline conversation display without showing raw JSON. + +### 7.4 Size Considerations + +For large specifications (hundreds of elements), the SIR can be chunked: + +- Elements stored in a content-addressed element store +- SIR document holds references (hashes) rather than inline elements +- Fetch-on-demand for large specs + +```typescript +interface SIRElementRef { + ref: string; // content hash of the serialized element +} + +// elements: Record +// Resolution: fetch element by hash from store, assemble full graph in memory +``` + +--- + +## 8. Data Schema Domain — Complete Example + +### 8.1 User Profile Data Schema + +```json +{ + "sirId": "c3d4e5f6-7a8b-4c9d-a0e1-2f3a4b5c6d7e", + "sirVersion": 3, + "sirHash": "sha256:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08", + "schemaVersion": "1.0.0", + "domain": { + "primary": "data-schema", + "version": "1.0.0", + "secondary": [] + }, + "elements": { + "entity-user": { + "id": "entity-user", + "elementType": "entity", + "status": "validated", + "version": 3, + "label": "User", + "description": "Core user profile entity. Every authenticated user has exactly one User record.", + "properties": { + "name": { "type": "string", "value": "users" }, + "storageEngine": { "type": "enum", "value": "postgresql", "allowedValues": ["postgresql", "sqlite", "mongodb"] }, + "softDelete": { "type": "boolean", "value": true } + }, + "references": [], + "subElements": ["field-id", "field-email", "field-name", "field-created-at", "field-updated-at", "field-deleted-at"], + "confidence": 0.95, + "flags": { "needsReview": false, "isAmbiguous": false, "isTentative": false, "isContradicted": false, "sourceIsExplicit": true, "hasGaps": false }, + "createdAt": 1778392300000, + "updatedAt": 1778392350000, + "provenance": { + "entries": [{ + "turnId": "turn-15", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": true, + "timestamp": 1778392300000, + "confidenceAtTime": 0.95 + }] + }, + "confidenceBreakdown": { + "overall": 0.95, + "factors": { "explicitMarker": 1.0, "structuralPattern": 0.95, "consistency": 0.95, "speakerAuthority": 0.90, "repetition": 0.90 } + } + }, + + "field-id": { + "id": "field-id", + "elementType": "field", + "status": "validated", + "version": 1, + "label": "ID", + "description": "Primary key, auto-generated UUID.", + "properties": { + "name": { "type": "string", "value": "id" }, + "type": { "type": "enum", "value": "uuid", "allowedValues": ["uuid", "integer", "string", "boolean", "timestamp", "jsonb"] }, + "nullable": { "type": "boolean", "value": false }, + "primaryKey": { "type": "boolean", "value": true }, + "defaultValue": { "type": "expression", "value": "gen_random_uuid()", "expressionType": "sql" } + }, + "references": [], + "subElements": [], + "confidence": 0.98, + "flags": { "needsReview": false, "isAmbiguous": false, "isTentative": false, "isContradicted": false, "sourceIsExplicit": true, "hasGaps": false }, + "createdAt": 1778392310000, + "updatedAt": 1778392310000, + "provenance": { + "entries": [{ + "turnId": "turn-15", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": true, + "timestamp": 1778392310000, + "confidenceAtTime": 0.98 + }] + }, + "confidenceBreakdown": { + "overall": 0.98, + "factors": { "explicitMarker": 1.0, "structuralPattern": 0.98, "consistency": 0.98, "speakerAuthority": 0.90, "repetition": 0.95 } + } + }, + + "field-email": { + "id": "field-email", + "elementType": "field", + "status": "validated", + "version": 2, + "label": "Email", + "description": "User email address. Must be unique and valid format.", + "properties": { + "name": { "type": "string", "value": "email" }, + "type": { "type": "enum", "value": "string", "allowedValues": ["uuid", "integer", "string", "boolean", "timestamp", "jsonb"] }, + "nullable": { "type": "boolean", "value": false }, + "maxLength": { "type": "number", "value": 254 }, + "unique": { "type": "boolean", "value": true } + }, + "references": [], + "subElements": [], + "confidence": 0.96, + "flags": { "needsReview": false, "isAmbiguous": false, "isTentative": false, "isContradicted": false, "sourceIsExplicit": true, "hasGaps": false }, + "createdAt": 1778392310000, + "updatedAt": 1778392340000, + "provenance": { + "entries": [ + { + "turnId": "turn-15", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": false, + "timestamp": 1778392310000, + "confidenceAtTime": 0.90 + }, + { + "turnId": "turn-17", + "action": "refined", + "agentId": "human", + "humanConfirmed": true, + "timestamp": 1778392340000, + "confidenceAtTime": 0.96, + "diff": { "changedProperties": ["unique", "maxLength"] } + } + ] + }, + "confidenceBreakdown": { + "overall": 0.96, + "factors": { "explicitMarker": 1.0, "structuralPattern": 0.95, "consistency": 0.95, "speakerAuthority": 0.90, "repetition": 0.90 } + } + }, + + "field-name": { + "id": "field-name", + "elementType": "field", + "status": "validated", + "version": 1, + "label": "Name", + "description": "User's display name.", + "properties": { + "name": { "type": "string", "value": "display_name" }, + "type": { "type": "enum", "value": "string", "allowedValues": ["uuid", "integer", "string", "boolean", "timestamp", "jsonb"] }, + "nullable": { "type": "boolean", "value": false }, + "maxLength": { "type": "number", "value": 100 } + }, + "references": [], + "subElements": [], + "confidence": 0.95, + "flags": { "needsReview": false, "isAmbiguous": false, "isTentative": false, "isContradicted": false, "sourceIsExplicit": true, "hasGaps": false }, + "createdAt": 1778392310000, + "updatedAt": 1778392310000, + "provenance": { + "entries": [{ + "turnId": "turn-15", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": true, + "timestamp": 1778392310000, + "confidenceAtTime": 0.95 + }] + }, + "confidenceBreakdown": { + "overall": 0.95, + "factors": { "explicitMarker": 1.0, "structuralPattern": 0.95, "consistency": 0.95, "speakerAuthority": 0.90, "repetition": 0.90 } + } + } + }, + "constraints": [ + { + "id": "constraint-email-unique", + "kind": "invariant", + "target": "field-email", + "expression": "forall u1,u2 in entity-user: u1.id != u2.id => u1.email != u2.email", + "language": "speckl-expr", + "severity": "error", + "confidence": 1.0, + "humanConfirmed": true, + "resolved": true, + "provenance": { + "entries": [{ + "turnId": "turn-17", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": true, + "timestamp": 1778392340000, + "confidenceAtTime": 1.0 + }] + } + }, + { + "id": "constraint-email-format", + "kind": "type", + "target": "field-email", + "check": "value matches email_pattern", + "severity": "error", + "confidence": 0.95, + "humanConfirmed": true, + "resolved": true, + "provenance": { + "entries": [{ + "turnId": "turn-17", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": true, + "timestamp": 1778392340000, + "confidenceAtTime": 0.95 + }] + } + } + ], + "composition": [], + "metadata": { + "title": "User Profile Schema", + "description": "Core user entity with auth-critical fields", + "tags": ["users", "auth", "profile", "database"], + "createdAt": 1778392300000, + "updatedAt": 1778392350000, + "createdBy": "agent-engineering", + "readiness": "validated" + } +} +``` + +--- + +## 9. Refinement Example — Before and After + +### 9.1 Before Refinement (Turn 5 — Initial Extraction) + +The login flow SIR after first extraction, before Scott refines it: + +```json +{ + "state-logged-out": { + "id": "state-logged-out", + "elementType": "state", + "status": "proposed", + "version": 1, + "label": "Logged Out", + "description": "User is not authenticated.", + "properties": { + "isInitial": { "type": "boolean", "value": true }, + "isTerminal": { "type": "boolean", "value": false } + }, + "references": [ + { "refId": "transition-login-submit", "refType": "outgoing", "label": "on submit", "cardinality": "1" } + ], + "subElements": [], + "confidence": 0.92, + "flags": { "needsReview": false, "isAmbiguous": false, "isTentative": false, "isContradicted": false, "sourceIsExplicit": true, "hasGaps": false }, + "createdAt": 1778392100000, + "updatedAt": 1778392100000, + "provenance": { + "entries": [{ + "turnId": "turn-3", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": false, + "timestamp": 1778392100000, + "confidenceAtTime": 0.92 + }] + } + }, + "state-authenticated": { + "id": "state-authenticated", + "elementType": "state", + "status": "proposed", + "version": 1, + "label": "Authenticated", + "description": "User has valid session.", + "properties": { + "isInitial": { "type": "boolean", "value": false }, + "isTerminal": { "type": "boolean", "value": false } + }, + "references": [ + { "refId": "transition-auth-success", "refType": "incoming", "label": "entered via", "cardinality": "1" } + ], + "subElements": [], + "confidence": 0.95, + "flags": { "needsReview": false, "isAmbiguous": false, "isTentative": false, "isContradicted": false, "sourceIsExplicit": true, "hasGaps": false }, + "createdAt": 1778392100000, + "updatedAt": 1778392100000, + "provenance": { + "entries": [{ + "turnId": "turn-3", + "action": "extracted", + "agentId": "agent-engineering", + "humanConfirmed": false, + "timestamp": 1778392100000, + "confidenceAtTime": 0.95 + }] + } + } +} +``` + +**Gaps at this stage:** +- Missing: `state-locked-out`, `state-logging-in` (not yet mentioned in conversation) +- Missing: logout transition, lockout guard, timeout behavior +- All elements status: "proposed" (not confirmed) +- `state-authenticated` has no outgoing transitions + +### 9.2 After Refinement (Turn 12 — Scott Confirms) + +The SIR from Section 2.7 shows the final state. Key changes: + +**Element-level changes:** +- `state-logged-out`: status `proposed` → `validated`; confidence 0.92 (unchanged) +- `state-authenticated`: added `sessionDuration`, `onEntry`, `transition-logout`, `transition-session-expire`; confidence 0.95 (unchanged); status → `validated` +- `state-logging-in`: ADDED — confidence 0.78 → 0.85 after refinement +- `state-locked-out`: ADDED — confidence 0.88 +- `transition-credential-check`: ADDED — low confidence 0.65, tentative, flagged +- `transition-auth-failure`: ADDED (not shown in full above but referenced) +- `guard-credentials-valid`: ADDED — confidence 0.96 + +**Provenance chain for `state-logging-in`:** +``` +turn-5: extracted (confidence 0.45, proposed) +turn-7: refined — added timeout, timeoutAction (confidence 0.85) +turn-12: confirmed by Scott (confidence 1.0, validated) +``` + +**The diff** (computed by SIRDiff): +```json +{ + "addedElements": ["state-logging-in", "state-locked-out", "transition-credential-check", "transition-auth-success", "transition-auth-failure", "transition-auth-timeout", "transition-logout", "transition-session-expire", "transition-lock-expire", "guard-credentials-valid", "guard-max-attempts", "guard-lock-timer"], + "removedElements": [], + "modifiedElements": { + "state-authenticated": { + "changedProperties": ["sessionDuration", "onEntry"], + "addedReferences": ["transition-logout", "transition-session-expire"], + "changedConstraints": [], + "confidenceDelta": 0.0 + } + } +} +``` + +--- + +## 10. Summary — The SIR Contract + +The SIR makes these guarantees to the layers above and below: + +### To the Extraction Layer: +- **Write**: accept any well-formed element, regardless of completeness; confidence and status fields accommodate uncertainty +- **Incremental**: support partial updates, property-level diffs, version tracking +- **Provenance storage**: preserve every extraction event without loss + +### To the Validation Layer: +- **Query**: give a complete typed graph with references, properties, constraints +- **Locate**: every element has a stable ID, every property has a path for precise error reporting +- **Confidence**: per-element and per-property scores available for gating decisions + +### To the Compilation Layer: +- **Deterministic hash**: content-addressable with fixed serialization order +- **Stable references**: element IDs never change; references are resolvable +- **Gap transparency**: flags and confidence scores indicate where compilation should insert stubs +- **Readiness**: graduated readiness levels for partial → full compilation + +--- + +## Open Questions + +1. **Sub-SIR granularity**: Should sub-state-machines be inline elements or referenced by content hash? Inline is simpler; hashed references scale better. +2. **Constraint expression evaluator**: How much of the constraint language should be evaluable in-process vs. delegated to external solvers (Z3, Alloy)? +3. **Real-time SIR sync**: For collaborative editing (multiple agents talking to the same human simultaneously), do we need CRDT semantics or is append-only log sufficient? +4. **Domain schema registry**: Where do domain definitions live? Inline in the SIR, in a registry service, or loaded from filesystem? +5. **Property type extensibility**: Should custom domain backends be able to define new `PropertyValue` types beyond the six built-in types? diff --git a/docs/speckl-by-example.md b/docs/speckl-by-example.md new file mode 100644 index 0000000..9a3c846 --- /dev/null +++ b/docs/speckl-by-example.md @@ -0,0 +1,329 @@ +# Speckl by Example + +A guided tour of SpeckDL through working examples — from a toggle switch to distributed consensus. + +## What is SpeckDL? + +SpeckDL is the specification language for Speckl. It lets you write formal, verifiable specifications that compile to: + +- **TypeScript** — typed state machine classes you can run +- **WASM** — WebAssembly modules for embedded verification +- **PROV-O** — W3C provenance ontology (audit trail) +- **CycloneDX** — Software Bill of Materials (SBOM) +- **SPDX** — Standard license/material metadata + +Every speck carries **embedded provenance**: where it came from, what regulations it fulfills, and who authored it. This is what we call **dark provenance** — always present, never in the way. + +--- + +## 1. ToggleSwitch — Your First State Machine + +**File:** `examples/ToggleSwitch.speck` + +```speck +speck ToggleSwitch { + state { + isOn: Bool + } + + init { + isOn == false + } + + invariant SwitchConsistent { + isOn in {true, false} + } + + action TurnOn { + require not isOn + isOn := true + return isOn + } + + action TurnOff { + require isOn + isOn := false + return isOn + } + + next: TurnOn | TurnOff +} +``` + +### What's happening here? + +**State** declares the state variables. A toggle switch has one boolean. + +**Init** sets the initial state — the switch starts off. + +**Invariant** declares a property that must hold in every reachable state. `isOn in {true, false}` looks trivial, but invariants become critical in distributed systems (see Raft). + +**Actions** are the only way to change state: +- `require` is a guard — the action fails if the condition isn't met +- `:=` is assignment +- `return` makes the result observable + +**Next** defines which actions can fire. `TurnOn | TurnOff` means either can happen at any time — it's nondeterministic. The model checker explores all interleavings. + +### What this compiles to + +ToggleSwitch generates a TypeScript class with typed `turnOn()` and `turnOff()` methods, each containing the guard check. The WASM output can be embedded in a browser for client-side state validation. + +--- + +## 2. AccountLedger — Collections and Financial Safety + +**File:** `examples/AccountLedger.speck` + +```speck +speck AccountLedger { + state { + balances: Map(Nat, Int) + accounts: Set(Nat) + pendingTransfers: Map(Nat, PendingTransfer) + inFlight: Set(Nat) + } + + init { + balances == emptyMap + accounts == emptySet + pendingTransfers == emptyMap + inFlight == emptySet + } + + invariant NoNegativeBalancesWithoutOverdraft { + forall id in accounts: + balances[id] >= 0 + } + + invariant PendingConsistent { + forall tid in inFlight: + tid in pendingTransfers.keys + } + + action CreateTransfer(tid: Nat, from: Nat, to: Nat, amount: Nat) { + require tid notIn inFlight + require from in accounts + require to in accounts + require from != to + require balances[from] >= amount + + let pt := { from: from, to: to, amount: amount } + pendingTransfers[tid] := pt + inFlight := inFlight union {tid} + balances[from] := balances[from] - amount + emit TransferCreated { id: tid, from: from, to: to, amount: amount } + } + // ... CommitTransfer, VoidTransfer, OpenAccount +} +``` + +### New concepts + +**Map and Set types** — `Map(Nat, Int)` is a key-value store. `Set(Nat)` is an unordered collection. Operations: `union`, `\` (difference), `in`, `notIn`. + +**Quantified invariants** — `forall id in accounts: balances[id] >= 0` says "no account can have a negative balance." This is a safety property that the model checker verifies across all action sequences. + +**Let bindings** — `let pt := { from: from, to: to, amount: amount }` creates a local value for readability. + +**Emit** — produces a structured event observable by external systems. These become typed TypeScript events. + +**Record types** — `PendingTransfer` is a record with named fields, defined elsewhere in the spec. + +### Why this matters + +Financial systems live and die by invariants. SpeckDL lets you state "transfers must be atomic" or "balances must never go negative" as first-class assertions, not comments that rot. The compiler checks them at compile time; the WASM runtime enforces them at execution time. + +--- + +## 3. TwoPhaseCommit — Distributed Transactions + +**File:** `examples/TwoPhaseCommit.speck` + +This is a direct port of Lamport's TLA+ Two-Phase Commit spec. + +```speck +speck TwoPhaseCommit { + state { + rmState: Map(String, String) + tmState: String + tmPrepared: Set(String) + msgs: Set(Message) + rms: Set(String) + } + + invariant TPConsistency { + tmState == "committed" implies + forall rm in rms: rmState[rm] == "committed" + tmState == "aborted" implies + forall rm in rms: rmState[rm] == "aborted" + forall rm in rms: + rmState[rm] == "committed" implies tmState != "aborted" + } + + action TMCommit { + require tmState == "init" + require tmPrepared == rms + + tmState := "committed" + msgs := msgs union {{ type: "Commit" }} + forall rm in rms: + rmState[rm] := "committed" + emit AllCommitted {} + } + // ... RMPrepare, TMRcvPrepared, TMAbort, RMChooseToAbort +} +``` + +### New concepts + +**Multi-role state** — TwoPhaseCommit has a Transaction Manager and multiple Resource Managers. Each has its own state tracked in `rmState` and `tmState`. + +**Safety invariants** — The TPConsistency invariant encodes the core guarantee of 2PC: if the TM commits, all RMs must have committed. If the TM aborts, all RMs must have aborted. No RM can be committed while the TM is aborted. These are not comments — they are verifiable assertions. + +**Message passing** — `msgs` is a set of messages in transit. Actions add and consume messages, modeling the asynchronous communication between TM and RMs. + +**Bulk state mutation** — `forall rm in rms: rmState[rm] := "committed"` updates every RM atomically within the action. This captures the "all or nothing" semantics of the commit decision. + +### TLA+ lineage + +This spec is structurally identical to the TLA+ version used to teach distributed systems at universities. The key difference: in SpeckDL, the same spec also generates runnable TypeScript and audit-trail provenance. You don't just verify — you deploy. + +--- + +## 4. Raft — Production Consensus + +**File:** `examples/Raft.speck` + +Raft is the consensus algorithm behind etcd, Consul, and TiKV. This is a complete SpeckDL port with leader election, log replication, and safety invariants. + +```speck +speck Raft { + type LogEntry = { + term: Term, + index: LogIndex, + command: String + } + + type ServerState = { + role: String, + currentTerm: Term, + votedFor: String | null, + log: List(LogEntry), + commitIndex: LogIndex, + lastApplied: LogIndex + } + + state { + servers: Set(ServerId) + serverState: Map(ServerId, ServerState) + nextIndex: Map(Map(ServerId, ServerId), LogIndex) + matchIndex: Map(Map(ServerId, ServerId), LogIndex) + } + + // Safety: at most one leader per term + invariant SingleLeaderPerTerm { + forall s1 in servers: + forall s2 in servers: + serverState[s1].role == "leader" and + serverState[s2].role == "leader" and + s1 != s2 implies + serverState[s1].currentTerm != serverState[s2].currentTerm + } + + // Safety: committed entries survive elections + invariant LeaderCompleteness { + forall s in servers: + serverState[s].role == "leader" implies + forall entry in serverState[s].log: + entry.index <= serverState[s].commitIndex implies + entry.term <= serverState[s].currentTerm + } + + // Safety: if two logs have same term at same index, entries match + invariant LogMatching { + forall s1 in servers: + forall s2 in servers: + forall i in [1 .. length(serverState[s1].log)]: + i <= length(serverState[s2].log) and + serverState[s1].log[i].term == serverState[s2].log[i].term implies + serverState[s1].log[i].command == serverState[s2].log[i].command + } + + action StartElection(server: ServerId) { /* ... */ } + action GrantVote(voter: ServerId, req: RequestVoteReq) { /* ... */ } + action BecomeLeader(server: ServerId) { /* ... */ } + action AppendEntries(leader: ServerId, follower: ServerId, req: AppendEntriesReq) { /* ... */ } + action AdvanceCommit(leader: ServerId) { /* ... */ } + action ApplyEntries(server: ServerId) { /* ... */ } +} +``` + +### New concepts + +**Type aliases** — `type LogIndex = Nat` and `type Term = Nat` make the spec self-documenting. The compiler treats them as their underlying types but preserves the alias names in generated code. + +**Record types with optional fields** — `votedFor: String | null` uses union types for nullable fields. This is cleaner than sentinel values. + +**List type** — `List(LogEntry)` is an ordered sequence, distinct from `Set`. List operations include indexing (`log[i]`), `take`, `concat`, and `length`. + +**Nested map keys** — `Map(Map(ServerId, ServerId), LogIndex)` uses a compound key `{ leader, follower }` for per-leader-per-follower tracking. + +**Three safety invariants** — Raft's correctness depends on these three properties. SpeckDL makes them verifiable assertions rather than paper proofs: +1. **SingleLeaderPerTerm** — no two servers can be leader in the same term +2. **LeaderCompleteness** — a leader's log contains all committed entries +3. **LogMatching** — consistent entries at matching (index, term) positions + +**Control flow** — `if req.term > serverState[leader].currentTerm { ... return }` allows early exit. `for i from ... to ...` provides counted iteration. + +--- + +## The SpeckDL Compilation Pipeline + +Every speck goes through the same pipeline: + +``` +.speckdl ──► Parser ──► AST ──┬──► TypeScript Generator ──► .ts (runnable) + ├──► WASM Generator ──► .wat (verifiable at runtime) + ├──► PROV-O Generator ──► .ttl (audit trail) + ├──► CycloneDX Generator ──► .xml (SBOM) + └──► SPDX Generator ──► .json (license/material) +``` + +### What you get + +**One spec, five outputs.** Write a state machine once and get: +- Runnable TypeScript you can import into any Node.js project +- Verifiable WASM you can embed in CI/CD pipelines +- Complete provenance chain for regulatory compliance (NIST SA-11) +- SBOMs for supply chain security +- License metadata for open source compliance + +--- + +## Comparison: SpeckDL vs. TLA+ + +| Dimension | TLA+ | SpeckDL | +|-----------|------|---------| +| **Purpose** | Formal verification only | Spec → verify → compile → run | +| **Output** | Model checker results | Runnable code (TS + WASM) | +| **Provenance** | None | Embedded (PROV-O, CycloneDX) | +| **Learning curve** | Steep (untyped set theory) | Moderate (typed, familiar syntax) | +| **Industry adoption** | AWS, Microsoft (internal) | Target: compliance-driven orgs | +| **Tooling** | TLA+ Toolbox, TLC | Single CLI, standard formats | +| **Regulatory** | None built-in | NIST SA-11, SBOM-ready | + +TLA+ proved that formal methods work at scale. SpeckDL makes them deployable. + +--- + +## Next Steps + +1. **Read the spec:** `SPEC.md` — full language reference +2. **Run the examples:** `cd compiler && npm run build && node dist/cli.js compile ../examples/ToggleSwitch.speck` +3. **Write your own:** Start with a toggle switch, add state variables, write invariants +4. **Target production:** See `examples/TigerBeetleLedger.speck` for a real financial system spec + +--- + +*Speckl is an open standard (MIT). Contribute at os.scoble.me/forgejo/sscoble/speckl* diff --git a/docs/validation-engine.md b/docs/validation-engine.md new file mode 100644 index 0000000..16c6198 --- /dev/null +++ b/docs/validation-engine.md @@ -0,0 +1,1187 @@ +# Speckl Validation Engine Design + +## Overview + +The Validation Engine is the second stage of the Speckl pipeline: + +``` +Chaotic Conversation → [Extraction Layer] → [Validation Layer] → [Compilation Layer] +``` + +It takes a Structured Intermediate Representation (SIR) and produces a `ValidationReport` that determines whether the SIR is correct, complete, and consistent enough to compile. The engine is domain-agnostic at its core with pluggable domain-specific validators. + +--- + +## 1. Architecture + +``` + ┌───────────────────────────┐ + │ Validation Engine │ + │ │ + SIR ──────────▶ │ ┌──────────────────────┐ │ + │ │ Structural Validator │ │ + │ └──────────┬───────────┘ │ + │ │ │ + │ ┌──────────▼───────────┐ │ + │ │ Reference Validator │ │ + │ └──────────┬───────────┘ │ + │ │ │ + │ ┌──────────▼───────────┐ │ + │ │ Constraint Solver │ │ + │ └──────────┬───────────┘ │ + │ │ │ + │ ┌──────────▼───────────┐ │ ┌──────────────────┐ + │ │ Domain Validators │◀─┼───│ Plugin Registry │ + │ │ (pluggable) │ │ │ │ + │ └──────────┬───────────┘ │ │ state-machine │ + │ │ │ │ workflow │ + │ ┌──────────▼───────────┐ │ │ data-model │ + │ │ Completeness Check │ │ │ api-spec │ + │ └──────────┬───────────┘ │ │ ... │ + │ │ │ └──────────────────┘ + │ ┌──────────▼───────────┐ │ + │ │ Confidence Filter │ │ + │ └──────────┬───────────┘ │ + │ │ │ + │ ┌──────────▼───────────┐ │ + │ │ Report Assembler │──┼──▶ ValidationReport + │ └──────────────────────┘ │ + └───────────────────────────┘ +``` + +### Core Principle: Fail Cheap First + +Validation passes are ordered from cheapest to most expensive. A structural check rejecting the SIR avoids running an SMT solver. This matters because some checks (temporal logic model checking, constraint solving) can be NP-hard or PSPACE-complete. + +--- + +## 2. The SIR (Recap) + +The SIR is a typed graph. Key types: + +``` +SIR { + domain: string // e.g. "state-machine" + elements: Element[] // all typed nodes + constraints: Constraint[] // cross-element constraints + confidence: Confidence // per-element confidence scores + source: SourceMap // maps elements back to conversation spans +} + +Element { + id: string // unique, e.g. "state-idle" + type: string // e.g. "state", "transition", "guard" + properties: Map // typed property bag + metadata: { + confidence: 0.0..1.0 // extraction confidence + source: SourceSpan // where in conversation this came from + required: boolean // is this element mandatory for compilation? + } +} + +Constraint { + id: string + type: "reference" | "invariant" | "type-check" | "domain" + expression: ConstraintExpr // structured expression tree + severity: "error" | "warning" +} + +Confidence { + perElement: Map + overall: 0.0..1.0 // aggregate confidence + threshold: 0.0..1.0 // configured threshold for blocking +} +``` + +--- + +## 3. Validation Categories + +### 3.1 Structural Validation + +Determines if the SIR is *well-formed* — independent of domain semantics. + +**Checks:** + +| Check | Rule | Severity | +|-------|------|----------| +| Unique IDs | No duplicate `Element.id` | ERROR | +| Required fields | Every element has `id`, `type`, `properties` | ERROR | +| Valid type registry | Every `Element.type` exists in the known type registry | ERROR | +| Property type correctness | Properties match their declared types | ERROR | +| No orphan references | Every `Constraint.expression` referencing an element ID must find that element | ERROR | +| Valid JSON/parsed form | SIR structure is parsable | ERROR | + +**Example — State Machine SIR structural errors:** + +```yaml +# This SIR has structural problems +domain: state-machine +elements: + - id: "idle" + type: "state" + properties: + name: "Idle" + is_initial: "yes" # ERROR: is_initial must be boolean, got string + - id: "running" + type: "stte" # ERROR: unknown type "stte" (typo) + # ERROR: transition references nonexistent element "stopped" + - id: "t1" + type: "transition" + properties: + from: "idle" + to: "stopped" # stopped not defined + trigger: "start" + - id: "idle" # ERROR: duplicate id "idle" + type: "state" + properties: + name: "Duplicate Idle" +``` + +**Generated Errors:** + +``` +ERROR [structural.type_mismatch] Element "idle": property "is_initial" + expected type "boolean", got "yes" (string). + → Fix: change "yes" to true. + +ERROR [structural.unknown_type] Element "running": type "stte" is not + in the type registry. Did you mean "state"? + +ERROR [structural.missing_reference] Element "t1" (transition): + target state "stopped" not found in element set. + +ERROR [structural.duplicate_id] Element id "idle" appears more than once. + ids must be unique. +``` + +### 3.2 Reference Validation + +Cross-element reference integrity. After structural checks pass, we verify that the graph of references is coherent. + +**Checks:** + +| Check | Rule | Severity | +|-------|------|----------| +| Bidirectional consistency | If A references B, B's type must be compatible with A's expectation | ERROR | +| No dangling references | All `from`/`to`/`parent`/`child` fields resolve | ERROR | +| Circular reference detection | No fatal cycles (e.g. state A's parent is state B whose parent is state A) | WARNING | +| Cardinality | If a field declares `max: 1`, having 2+ references is an error | ERROR | + +**Example — Coherent reference graph for state machines:** + +```yaml +elements: + - id: "idle" + type: "state" + properties: { name: "Idle", is_initial: true } + - id: "running" + type: "state" + properties: { name: "Running" } + - id: "t1" + type: "transition" + properties: + from: "idle" + to: "running" + trigger: "start" + - id: "t2" + type: "transition" + properties: + from: "running" + to: "idle" + trigger: "stop" +``` + +This SIR passes both structural and reference validation. + +### 3.3 Domain-Specific Validation + +Domain validators are plugins that register with the engine. Each validator specifies: +- Which `domain` it applies to +- What checks it performs +- The cost of each check (cheap/moderate/expensive) + +**State Machine Domain Validator Checks:** + +| Check | Cost | Rule | Severity | +|-------|------|------|----------| +| Initial state exists | Cheap | Exactly one state has `is_initial: true` | ERROR | +| Reachable states | Moderate | Every non-initial state is reachable via some path of transitions | WARNING | +| No dead-end states | Cheap | Every state has at least one outgoing transition, or is marked `terminal` | WARNING | +| Deterministic transitions | Moderate | For a given (state, trigger) pair, at most one transition fires (or guards disambiguate) | WARNING for ambiguity | +| Guard completeness | Expensive | For a state with guarded transitions on the same trigger, guards must be exhaustive (sum of guard cases covers all possibilities) | WARNING | +| No livelock | Expensive | No infinite cycle of transitions without progress | ERROR | +| Terminal states | Cheap | Terminal states have no outgoing transitions | INFO (ok if intended) | +| Liveness | Expensive | Desired terminal states are reachable from all states (if spec'd) | WARNING | + +**Example — State Machine with domain errors:** + +```yaml +domain: state-machine +elements: + - id: "idle" + type: "state" + properties: { name: "Idle" } + # ERROR: missing is_initial — no initial state defined + - id: "running" + type: "state" + properties: { name: "Running" } + - id: "orphan" + type: "state" + properties: { name: "Orphan" } + # WARNING: unreachable — no transition targets this state + - id: "t1" + type: "transition" + properties: + from: "idle" + to: "running" + trigger: "start" + - id: "t2" + type: "transition" + properties: + from: "running" + to: "running" # WARNING: self-loop, no progress + trigger: "retry" + - id: "t3" + type: "transition" + properties: + from: "idle" + to: "running" + trigger: "start" + guard: "is_admin == true" + # WARNING: conflict with t1 — same (from, trigger), t1 has no guard, t3 has guard. + # t1 fires unconditionally on "start", t3 never fires. +``` + +**Generated domain errors:** + +``` +ERROR [domain.fsm.no_initial] No initial state defined. + Mark one state with is_initial: true (e.g. state "idle"). + +WARNING [domain.fsm.unreachable] State "orphan" is unreachable. + No transitions target it. Remove it or add a transition to it. + +WARNING [domain.fsm.self_loop] Transition "t2": self-loop on state + "running" with trigger "retry". This produces no state change. + If intentional, mark the transition with `is_self_loop: true`. + +WARNING [domain.fsm.guard_shadow] Transition "t3" is shadowed by "t1": + both fire from "idle" on trigger "start", but "t1" has no guard + (always matches), so "t3" (guard: is_admin == true) can never fire. + → Add a guard to "t1", remove "t3", or use different triggers. +``` + +### 3.4 Consistency Validation + +Checks that constraints and invariants are mutually satisfiable. + +**Constraint types in the SIR:** + +``` +ConstraintExpr = + | TypeConstraint(field, expectedType) + | ReferenceConstraint(source, target, relation) + | Invariant(temporalLogicExpr) // e.g. "eventually(terminal)" + | CrossConstraint(expr1, expr2, op) // expr1 AND/OR/IMPLIES expr2 + | GuardExpr(field, operator, value) // "counter < 5" +``` + +**Consistency checks (expensive):** + +For state machines with temporal logic invariants, we use bounded model checking: + +``` +Invariant: "G (in_state('error') → F in_state('recovery'))" + "Globally, if you enter error state, you eventually reach recovery" +``` + +If the model checker finds a counterexample trace where the SIR enters `error` but cannot reach `recovery`, it produces: + +``` +ERROR [consistency.invariant_violation] Invariant "G(error → F recovery)" + is violated. Counterexample trace (depth=4): + 1. idle → running (trigger: start) + 2. running → error (trigger: fail) + 3. error → (no outgoing transitions) + State "error" is a dead-end. Add a transition from "error" to "recovery" + or remove the invariant. +``` + +**Cheap consistency pass** (always runs first): +- Contradictory property assignments: `{ is_terminal: true, outgoing_transitions: [t1] }` +- Mutually exclusive type constraints: `x must be string AND x must be number` + +### 3.5 Confidence-Based Validation + +Each SIR element carries a confidence score (0.0–1.0) from the extraction layer. Low confidence means the extraction was uncertain about what the human meant. + +**Strategy:** + +``` +Element confidence thresholds: + ≥ 0.9 → trusted (no validation flags on confidence) + ≥ 0.7 → INFO: "Low confidence on element X. Verify: was this intended?" + ≥ 0.5 → WARNING: "Uncertain element X — compilation will include a TODO marker" + < 0.5 → ERROR: "Element X is too uncertain to compile. Clarify in conversation." +``` + +**Blocking logic:** + +```typescript +function confidenceBlocks(confidence: Confidence, threshold: number): boolean { + // Compilation is blocked if ANY error-severity element is below threshold + // OR if overall confidence is below the configured threshold. + return confidence.overall < threshold + || Array.from(confidence.perElement.values()) + .some(c => c < 0.5); // hard floor +} +``` + +**Example:** + +```yaml +elements: + - id: "idle" + type: "state" + properties: { name: "Idle", is_initial: true } + metadata: { confidence: 0.95 } + - id: "mystery" + type: "state" + properties: { name: "MaybeErrorHandling?" } + metadata: { confidence: 0.42 } # Very uncertain + +confidence: + perElement: { "idle": 0.95, "mystery": 0.42 } + overall: 0.71 + threshold: 0.8 +``` + +Generated: +``` +ERROR [confidence.below_floor] Element "mystery" (state "MaybeErrorHandling?") + has confidence 0.42 (< 0.5 floor). Clarify what this state represents. + +WARNING [confidence.overall] Overall SIR confidence (0.71) is below the + configured compilation threshold (0.80). Consider clarifying + uncertain elements before compiling. +``` + +--- + +## 4. Validation Timing + +### 4.1 Continuous Validation (on every extraction) + +Runs immediately after the Extraction Layer produces or updates the SIR. Uses only cheap checks: +- Structural validation +- Reference validation +- Confidence threshold checks +- Cheap domain checks (initial state exists, dead-end detection) + +**Goal:** Give the agent immediate feedback to interject in the conversation. + +> Agent: "I heard you describe a state machine with states 'idle', 'running', and 'error'. But 'error' has no way to recover. Would you like to add a recovery transition?" + +### 4.2 Pre-Compilation Validation (batched) + +Runs when the human requests compilation. Runs the full pipeline including expensive checks: +- Reachability analysis (BFS/DFS on state graph) +- Guard exhaustiveness +- Model checking (bounded, with configurable depth) +- SMT-based constraint solving (if invariants use arithmetic/logic) +- Completeness completeness analysis + +### 4.3 Incremental Validation + +When the extraction layer produces a delta (added/modified/removed elements), we re-validate only the affected subgraph: + +``` +Changed elements → find transitive closure of references → re-validate that subgraph +``` + +**Dependency tracking:** + +``` +Element "t1" depends on: ["idle", "running"] +Element "g1" (guard) depends on: ["t1"] +Invariant "inv1" depends on: ["error", "recovery"] +``` + +When `idle` changes, we re-check `t1` → re-check `g1`. When `error` changes, we re-check `inv1`. + +### 4.4 Lazy vs. Eager + +| Check Type | Timing | Rationale | +|------------|--------|-----------| +| Structural | Eager (continuous) | O(n), always cheap | +| Reference | Eager (continuous) | O(n), always cheap | +| Confidence | Eager (continuous) | O(n), always cheap | +| Reachability | Lazy (pre-compile) | O(V+E) — cheap but only needed before compile | +| Determinism | Lazy (pre-compile) | O(S×T) where S=states, T=transitions | +| Guard exhaustiveness | Lazy (pre-compile) | May require SAT/SMT | +| Model checking | Lazy (pre-compile) | PSPACE-complete for full LTL, bounded is O(|S|^k) | +| Invariant satisfiability | Lazy (pre-compile, opt-in) | Expensive SMT solving | + +The human can also explicitly request expensive checks via conversation: + +> Human: "Check if my state machine can deadlock." +> Agent: runs lazy checks on-demand, reports results. + +--- + +## 5. Error Representation + +### 5.1 ValidationReport Structure + +```typescript +interface ValidationReport { + // Overall verdict + status: "pass" | "pass_with_warnings" | "fail"; + + // All issues found + issues: ValidationIssue[]; + + // Statistics + stats: { + errors: number; + warnings: number; + infos: number; + checksRun: number; + checksSkipped: number; + durationMs: number; + }; + + // Completeness + completeness: CompletenessReport; + + // Suggestions for fixes + suggestions: FixSuggestion[]; + + // What blocked compilation (if status == "fail") + blockers: BlockingIssue[]; +} + +interface ValidationIssue { + id: string; // unique issue id + severity: "error" | "warning" | "info"; + category: ValidationCategory; // structural | domain | consistency | confidence | completeness + code: string; // machine-readable code e.g. "domain.fsm.no_initial" + message: string; // human-readable message + location: SourceLocation[]; // precisely which SIR elements + suggestion?: FixSuggestion; // optional suggested fix + trace?: Counterexample; // for model checking failures +} + +interface FixSuggestion { + description: string; // "Mark state 'idle' as initial" + action: "add" | "remove" | "modify" | "clarify"; + target: string; // element id + patch?: Partial; // concrete patch to apply + conversationPrompt?: string; // what the agent should say to the human +} + +interface CompletenessReport { + status: "complete" | "partial" | "incomplete"; + missingRequired: string[]; // element ids + missingOptional: string[]; // element ids + gaps: CompletenessGap[]; // described gaps +} + +interface BlockingIssue { + issueId: string; + reason: string; // why this blocks compilation +} + +type ValidationCategory = + | "structural" + | "domain" + | "consistency" + | "confidence" + | "completeness"; +``` + +### 5.2 Severity Semantics + +| Severity | Meaning | Compilation Behavior | +|----------|---------|---------------------| +| ERROR | Definite problem, SIR is invalid | Blocks compilation | +| WARNING | Likely problem, SIR is valid but suspect | Compiles with caveats annotated | +| INFO | Suggestion or note | Compiles normally, annotation optional | + +### 5.3 Error Localization + +Every issue points to precise SIR elements: + +```typescript +interface SourceLocation { + elementId: string; + propertyPath?: string; // e.g. "properties.to" for transition target + constraintId?: string; // for constraint violations + snippet?: string; // the problematic value +} +``` + +### 5.4 Suggested Fixes — Agent Conversation Prompts + +The fix suggestions are designed to flow back into conversation: + +``` +conversationPrompt: "State 'error' has no outgoing transitions. + Would you like to add a recovery transition, or mark it as terminal?" + +conversationPrompt: "I heard two different things about what happens when + the system starts: one where it goes to 'running' unconditionally, + and one where it only goes if 'is_admin' is true. Which one did you mean?" + +conversationPrompt: "You mentioned a 'loading' state but I'm not sure + when it should be entered. When does the system go to 'loading'?" +``` + +--- + +## 6. Constraint Checking System + +### 6.1 Constraint Expression Language + +Constraints in the SIR are structured expressions — never raw strings: + +```typescript +// Type constraint: "property 'trigger' must be a string" +{ + type: "type-check", + target: { elementId: "*", propertyPath: "trigger" }, + expected: "string" +} + +// Reference constraint: "transition.target must be a valid state id" +{ + type: "reference", + source: { elementId: "*", propertyPath: "to" }, + targetType: "state", + relation: "must-exist" +} + +// Invariant: "Globally, if in error, eventually in recovery" +{ + type: "invariant", + formula: { + op: "G_implies", // Globally, φ → ψ + left: { op: "in_state", state: "error" }, + right: { op: "F", formula: { op: "in_state", state: "recovery" } } + } +} + +// Cross-element: "every transition's 'from' must be a state" +{ + type: "for_each", + iterator: { elementType: "transition" }, + body: { + type: "reference", + source: { elementId: "$it", propertyPath: "from" }, + targetType: "state", + relation: "must-exist" + } +} +``` + +### 6.2 Constraint Evaluator + +```typescript +class ConstraintEvaluator { + evaluate(constraint: Constraint, sir: SIR): ValidationIssue[] { + switch (constraint.type) { + case "type-check": + return this.checkType(constraint, sir); + case "reference": + return this.checkReference(constraint, sir); + case "invariant": + return this.checkInvariant(constraint, sir); + case "for_each": + return this.checkForEach(constraint, sir); + } + } +} +``` + +**Type checking walks the element tree:** + +``` +For each element matching source pattern: + get property at propertyPath + if typeof(property.value) !== expected: + emit type_mismatch issue +``` + +**Reference checking resolves the graph:** + +``` +For each element matching source pattern: + get property at propertyPath (the referenced id) + if id not in sir.elements: + emit missing_reference issue + if sir.elements[id].type !== targetType: + emit type_mismatch issue ("expected state, got guard") +``` + +### 6.3 Invariant Checking Strategy + +For state machines, invariants are temporal logic formulas. We use **bounded model checking** (BMC): + +``` +BMC(invariant, SIR, depth_k): + Unroll state machine k steps + Build SAT formula: (initial_state ∧ transitions[0..k-1]) ∧ ¬invariant_unrolled + If SAT solver finds a model → counterexample exists → report trace + If UNSAT → no counterexample within bound k → report "passed up to depth k" +``` + +For arithmetic/logic guards, we use an SMT solver (Z3 integration): + +``` +Guard exhaustiveness check: + For state S with trigger T and guards g1, g2, ..., gn: + Ask SMT solver: is (g1 ∨ g2 ∨ ... ∨ gn) a tautology? + If no → report uncovered case with concrete counterexample values +``` + +### 6.4 Constraint Registration + +Domain validators register constraints at load time: + +```typescript +// state-machine-validator.ts +class StateMachineValidator implements DomainValidator { + domain = "state-machine"; + + registerConstraints(registry: ConstraintRegistry): void { + // Structural constraints (always run) + registry.add({ + id: "fsm.type.transition_target", + type: "for_each", + iterator: { elementType: "transition" }, + body: { + type: "reference", + source: { elementId: "$it", propertyPath: "to" }, + targetType: "state", + relation: "must-exist" + }, + severity: "error", + category: "domain" + }); + + // Invariant constraints (lazy, pre-compile) + registry.add({ + id: "fsm.invariant.no_deadlock", + type: "invariant", + formula: { + op: "G_exists_next", // Globally, every state has a next state + except: { property: "is_terminal", value: true } + }, + severity: "warning", + category: "domain", + timing: "lazy" + }); + } +} +``` + +--- + +## 7. Plugin Architecture for Domain Validators + +### 7.1 Plugin Interface + +```typescript +interface DomainValidator { + /** Which domain this validator handles */ + domain: string; + + /** Human-readable name */ + name: string; + + /** Register constraints with the engine */ + registerConstraints(registry: ConstraintRegistry): void; + + /** Run domain-specific checks that can't be expressed as constraints */ + customChecks(sir: SIR, options: CheckOptions): ValidationIssue[]; + + /** Declare which elements are required for completeness */ + requiredElements(): RequiredElementSpec[]; + + /** Declare which elements are optional */ + optionalElements(): OptionalElementSpec[]; +} + +interface CheckOptions { + timing: "continuous" | "pre-compile"; + maxDepth: number; // for BMC + timeoutMs: number; // for SMT +} +``` + +### 7.2 Plugin Registry + +```typescript +class ValidationEngine { + private validators: Map = new Map(); + private constraints: ConstraintRegistry = new ConstraintRegistry(); + + register(validator: DomainValidator): void { + this.validators.set(validator.domain, validator); + validator.registerConstraints(this.constraints); + } + + async validate(sir: SIR, options: ValidateOptions = {}): Promise { + const timing = options.timing ?? "continuous"; + const issues: ValidationIssue[] = []; + + // Pass 1: Structural (always, first) + issues.push(...this.runStructuralChecks(sir)); + + // Early exit: if structural fails, don't bother with domain checks + if (issues.some(i => i.severity === "error")) { + return this.assembleReport("fail", issues, sir); + } + + // Pass 2: Core constraint evaluation (filtered by timing) + issues.push(...this.constraints.evaluate(sir, timing)); + + // Pass 3: Domain-specific custom checks + const domainValidator = this.validators.get(sir.domain); + if (domainValidator) { + issues.push(...domainValidator.customChecks(sir, { timing, ...options })); + } + + // Pass 4: Consistency (lazy only) + if (timing === "pre-compile") { + issues.push(...this.runConsistencyChecks(sir)); + issues.push(...this.runCompletenessCheck(sir, domainValidator)); + } + + // Pass 5: Confidence filter + issues.push(...this.runConfidenceFilter(sir)); + + return this.assembleReport(issues, sir); + } +} +``` + +### 7.3 Registration at Startup + +```typescript +const engine = new ValidationEngine(); + +// Core validators (always loaded) +engine.register(new StateMachineValidator()); +engine.register(new WorkflowValidator()); +engine.register(new DataModelValidator()); +engine.register(new ApiSpecValidator()); + +// Custom validators (loaded from config) +for (const plugin of loadPlugins("./validators/")) { + engine.register(plugin); +} +``` + +--- + +## 8. Completeness Analysis + +### 8.1 What Makes a Specification "Complete Enough"? + +Different domains have different completeness thresholds. A validator declares: + +```typescript +interface RequiredElementSpec { + elementType: string; + minCount: number; // at least this many + maxCount?: number; // at most this many (if applicable) + description: string; // human-readable, for conversation +} + +interface OptionalElementSpec { + elementType: string; + description: string; + compiledAs: "stub" | "todo" | "omit"; // what to do when missing +} +``` + +**State Machine completeness spec:** + +```typescript +requiredElements(): RequiredElementSpec[] { + return [ + { elementType: "state", minCount: 2, + description: "At least 2 states (start and end)" }, + { elementType: "transition", minCount: 1, + description: "At least 1 transition" }, + ]; +} + +optionalElements(): OptionalElementSpec[] { + return [ + { elementType: "guard", description: "Transition guards", + compiledAs: "stub" }, // compile with guard: true (always pass) + { elementType: "action", description: "Entry/exit actions", + compiledAs: "stub" }, // compile with no-op actions + { elementType: "invariant", description: "Temporal invariants", + compiledAs: "omit" }, // skip invariant in compiled output + ]; +} +``` + +### 8.2 CompletenessReport Generation + +```typescript +function runCompletenessCheck( + sir: SIR, + validator: DomainValidator | undefined +): ValidationIssue[] { + const issues: ValidationIssue[] = []; + + if (!validator) { + // Unknown domain — can't check completeness + return [{ + severity: "warning", + category: "completeness", + code: "completeness.unknown_domain", + message: `No validator registered for domain "${sir.domain}". Cannot verify completeness.`, + }]; + } + + const required = validator.requiredElements(); + const optional = validator.optionalElements(); + + // Count elements by type + const counts = new Map(); + for (const el of sir.elements) { + counts.set(el.type, (counts.get(el.type) || 0) + 1); + } + + // Check required + for (const spec of required) { + const count = counts.get(spec.elementType) || 0; + if (count < spec.minCount) { + issues.push({ + severity: "error", + category: "completeness", + code: "completeness.missing_required", + message: `Missing ${spec.elementType}: need at least ${spec.minCount}, + found ${count}. ${spec.description}`, + suggestion: { + description: `Define ${spec.minCount - count} more ${spec.elementType}(s)`, + action: "add", + conversationPrompt: `You need at least ${spec.minCount - count} + more ${spec.elementType}(s). ${spec.description}. What should they be?`, + }, + }); + } + if (spec.maxCount !== undefined && count > spec.maxCount) { + issues.push({ + severity: "warning", + category: "completeness", + code: "completeness.too_many", + message: `Too many ${spec.elementType}s: max ${spec.maxCount}, found ${count}.`, + }); + } + } + + // Check optional — informational only + for (const spec of optional) { + const count = counts.get(spec.elementType) || 0; + if (count === 0) { + issues.push({ + severity: "info", + category: "completeness", + code: "completeness.missing_optional", + message: `No ${spec.elementType}s defined. ${spec.description}. + Will compile with ${spec.compiledAs}.`, + suggestion: { + description: `Optionally define ${spec.elementType}s`, + action: "add", + conversationPrompt: `You can add ${spec.elementType}s (${spec.description}) + but they're optional. Want to add any?`, + }, + }); + } + } + + return issues; +} +``` + +### 8.3 Graduated Completeness + +The compilation layer uses the completeness report to decide output quality: + +| SIR Completeness | Compiled Output | +|-----------------|-----------------| +| Full (all required + all optional) | Production artifact | +| Required-only (all required, some optional missing) | Functional artifact with stubs | +| Partial (some required missing) | Skeleton with TODO markers | +| Incomplete (many required missing) | Compilation refused | + +**Example — Partial state machine compiles to:** + +```python +# COMPILED FROM PARTIAL SPEC — TODO markers indicate gaps +class TrafficLightFSM: + def __init__(self): + self.state = "red" # initial state + + # TODO: Missing transition from "red" — need a trigger + # TODO: State "yellow" has no incoming transitions + # WARNING: State "green" has no outgoing transitions (may be terminal) + + def trigger_green_timer(self): # STUB — guard not specified + if self.state == "green": + self.state = "yellow" +``` + +--- + +## 9. Complete Example: Validation Pipeline on a State Machine + +### Input SIR (from extraction layer): + +```yaml +domain: state-machine +elements: + - id: "red" + type: "state" + properties: { name: "Red", is_initial: true } + metadata: { confidence: 0.95 } + + - id: "green" + type: "state" + properties: { name: "Green" } + metadata: { confidence: 0.90 } + + - id: "yellow" + type: "state" + properties: { name: "Yellow" } + metadata: { confidence: 0.45 } # LOW CONFIDENCE + + - id: "t1" + type: "transition" + properties: { from: "red", to: "green", trigger: "timer" } + metadata: { confidence: 0.92 } + + - id: "t2" + type: "transition" + properties: { from: "green", to: "red", trigger: "timer" } + metadata: { confidence: 0.50 } # UNCERTAIN + + - id: "t3" + type: "transition" + properties: { from: "red", to: "green", trigger: "timer", guard: "emergency == false" } + metadata: { confidence: 0.30 } # VERY LOW — probably contradictory + +constraints: + - id: "inv_no_red_forever" + type: "invariant" + expression: + op: "G_implies" + left: { op: "in_state", state: "red" } + right: { op: "F", formula: { op: "in_state", state: "green" } } + +confidence: + perElement: + red: 0.95 + green: 0.90 + yellow: 0.45 + t1: 0.92 + t2: 0.50 + t3: 0.30 + overall: 0.67 + threshold: 0.80 +``` + +### Validation Pass 1 — Structural: + +``` +✅ PASS: All elements have valid types, unique ids, correct property types. +``` + +### Validation Pass 2 — Reference: + +``` +✅ PASS: All transition from/to references resolve to existing states. +``` + +### Validation Pass 3 — Domain (continuous, cheap): + +``` +WARNING [domain.fsm.unreachable] State "yellow" is unreachable. + No transitions target it. + +WARNING [domain.fsm.guard_shadow] Transition "t3" is shadowed by "t1": + both fire from "red" on "timer", but "t1" has no guard (always matches), + so "t3" (guard: emergency == false) can never fire. +``` + +### Validation Pass 4 — Confidence: + +``` +ERROR [confidence.below_floor] Element "t3" has confidence 0.30 (< 0.5 floor). + Clarify: was "emergency == false" actually intended? + +ERROR [confidence.below_floor] Element "yellow" has confidence 0.45 (< 0.5 floor). + Clarify: is "Yellow" a real state in this machine? + +WARNING [confidence.below_threshold] Element "t2" has confidence 0.50. + "green → red on timer" is uncertain. Verify this transition. + +WARNING [confidence.overall] Overall SIR confidence (0.67) < threshold (0.80). + Consider clarifying uncertain elements before compiling. +``` + +### Validation Pass 5 — Consistency (pre-compile, lazy): + +``` +✅ PASS: Invariant "no_red_forever" holds up to bounded depth 10. + Transition t1 (red → green) satisfies the invariant. +``` + +### Validation Pass 6 — Completeness: + +``` +✅ PASS: Required elements present: + - states: 3 (min 2) ✓ + - transitions: 3 (min 1) ✓ + +INFO [completeness.missing_optional] No "action" elements defined. + Entry/exit actions are optional. Will compile with no-ops. + +INFO [completeness.missing_optional] No "guard" elements (as standalone + definitions) — guard on t3 is inline. Guards are optional. +``` + +### Final ValidationReport: + +```json +{ + "status": "fail", + "issues": [...all above...], + "stats": { + "errors": 2, + "warnings": 4, + "infos": 2, + "checksRun": 8, + "checksSkipped": 1, + "durationMs": 12 + }, + "completeness": { + "status": "partial", + "missingRequired": [], + "missingOptional": ["action", "guard"], + "gaps": [ + { "type": "low_confidence", "elements": ["t3", "yellow", "t2"] }, + { "type": "guard_shadow", "elements": ["t3"] } + ] + }, + "blockers": [ + { "issueId": "conf-b1", "reason": "Element t3 below confidence floor" }, + { "issueId": "conf-b2", "reason": "Element yellow below confidence floor" } + ], + "suggestions": [ + { + "description": "Remove t3 or add a guard to t1", + "action": "modify", + "target": "t3", + "conversationPrompt": "I see two transitions from 'red' to 'green' on 'timer' — one with a guard and one without. The unguarded one always matches, so the guarded one never fires. Which behavior do you want?" + }, + { + "description": "Clarify yellow state purpose", + "action": "clarify", + "target": "yellow", + "conversationPrompt": "I'm not sure about the 'yellow' state — it has no transitions to or from it. Is this a real state in your traffic light, or was it mentioned in passing?" + } + ] +} +``` + +### Agent Response in Conversation: + +> **I validated your state machine spec. Here's what I found:** +> +> **2 issues blocking compilation:** +> - The transition from red→green with guard `emergency==false` conflicts with the unguarded one — the guarded one can never fire. Which did you intend? +> - I'm not confident about the "yellow" state — it's disconnected from everything. Is this intentional? +> +> **4 warnings (won't block, but worth checking):** +> - Yellow has no transitions to it +> - The green→red transition is uncertain +> - Overall confidence is at 67% — below the 80% threshold +> +> **Missing optional things (no action needed):** +> - No entry/exit actions defined (will compile as no-ops) +> +> **Want me to fix the guard conflict, or do you want to clarify yellow first?** + +--- + +## 10. API Reference + +### `validate(sir: SIR, options?: ValidateOptions): ValidationReport` + +```typescript +interface ValidateOptions { + /** Validation timing mode */ + timing?: "continuous" | "pre-compile"; + + /** Max depth for bounded model checking (default: 10) */ + maxDepth?: number; + + /** Timeout for SMT solver (default: 5000ms) */ + solverTimeoutMs?: number; + + /** Skip expensive checks even in pre-compile mode */ + skipExpensive?: boolean; + + /** Only validate specific element ids (incremental) */ + scope?: string[]; +} +``` + +### `register(validator: DomainValidator): void` + +Register a domain validator plugin. Must be called before `validate()` for that domain. + +### `getReport(sirId: string): ValidationReport | null` + +Retrieve the last validation report for a given SIR (reports are cached). + +### `validateIncremental(sir: SIR, delta: SIRDelta): ValidationReport` + +Incremental validation. Only checks elements affected by the delta. + +--- + +## 11. Design Decisions Summary + +| Decision | Rationale | +|----------|-----------| +| Fail-cheap ordering | Avoids running SMT on malformed SIRs; structural checks are O(n) and catch 80% of issues | +| Plugin architecture | Domain validators are the extension point; core engine never knows about state machines | +| Constraint expression tree | Structured constraints are evaluable; raw strings ("trigger must be a string") require parsing | +| Confidence floor at 0.5 | Below 0.5, the extraction is essentially guessing; requiring clarification is better than wrong compilation | +| Continuous + pre-compile | Two-timing splits the workload: fast feedback in conversation, thorough checks before artifact generation | +| Bounded model checking | Full LTL model checking is PSPACE-complete; bounded is practical and catches most real-world bugs | +| Graduated completeness | Not all specs need to be "complete" — a skeleton with TODOs is often better than refusing to compile | +| Conversation-aware suggestions | Every error can produce a natural-language prompt; the validation engine feeds the conversation loop | + +--- + +## 12. Open Questions for Review + +1. **SMT solver dependency:** Z3 is ~30MB. Worth bundling, or should we make it optional and degrade gracefully (skip guard exhaustiveness, flag as INFO)? + +2. **Confidence aggregation:** Currently uses a simple floor. Should we use a weighted average (by element importance)? A critical transition with 0.49 confidence probably matters more than a comment element with 0.3. + +3. **Max depth for BMC:** Defaulting to 10 may miss deep counterexamples. Make it configurable per-domain? A state machine with 100 states might need depth 100. + +4. **Incremental validation edge case:** If element A is removed, and element B references A, we re-validate B. But what if B had already been flagged and the human acknowledged it? Should incremental validation suppress re-reporting of known issues? + +5. **Human-in-the-loop for expensive checks:** Should pre-compile validation ask before running expensive checks that might take >5 seconds? Or transparently run them with a timeout? diff --git a/docs/whitepaper-v2.md b/docs/whitepaper-v2.md new file mode 100644 index 0000000..f1bc16f --- /dev/null +++ b/docs/whitepaper-v2.md @@ -0,0 +1,1413 @@ +# Speckl: Closing the Spec-Code Gap with Compilable Specifications + +> *A whitepaper on SpeckDL, a domain-specific language for writing specifications that compile to runnable software with embedded provenance.* + +## Abstract + +Software specifications have a fundamental problem: they describe what code should do, but they cannot guarantee what code actually does. Formal methods (TLA+, Alloy, Coq) can verify properties of specifications, but their outputs remain disconnected from production systems — the spec and the code evolve independently until the spec becomes archaeology. + +Speckl introduces a different approach: **specifications that compile to software**. The SpeckDL language lets you write state machine specifications with types, invariants, and actions — then compiles them into five auditable artifacts: PROV-O provenance graphs, CycloneDX SBOMs, SPDX license documents, TypeScript classes, and WebAssembly modules. The spec isn't documentation alongside the code — the spec IS the code, with a verifiable chain from intent to implementation. + +This paper presents the SpeckDL language, its compiler architecture, and the provenance model that makes every generated artifact auditable back to its specification. We demonstrate SpeckDL on consensus protocols (Two-Phase Commit, Paxos, Raft) and a financial ledger, showing how the same specification produces both formal verification artifacts and production-runnable modules. + +## 1. The Spec-Code Gap + +Every software project has specifications. Requirements documents, architecture decision records, API contracts, RFCs — these artifacts describe intent. But they have a fatal flaw: **they cannot be executed, and therefore cannot be verified against reality.** + +### 1.1 The Archaeology Problem + +When a specification exists only as prose, it decays. Three months after writing, the spec describes what the system should have done, not what it does now. Engineers treat specs as orientation, not authority. The spec becomes archaeology — interesting for historical context, irrelevant for current behavior. + +This decay compounds with AI-generated code. When an LLM writes code from a prompt, the reasoning chain is opaque. The generated code may satisfy the prompt, violate it in subtle ways, or introduce behaviors the prompt never specified. Without an executable connection between intent and output, verification is reduced to testing — which can only prove the presence of bugs, not their absence. + +### 1.2 The Formal Methods Gap + +Formal specification languages (TLA+, Alloy, Isabelle/HOL) solve the precision problem: they express intent unambiguously and can model-check properties of that intent. TLA+ in particular has been used to find critical design bugs in systems at AWS, Microsoft, and MongoDB. + +But formal methods create their own gap: **the verified spec and the running system are separate artifacts.** After model-checking a TLA+ specification, someone must manually translate it into a programming language. That translation introduces the same spec-code gap the formal method was supposed to close. The implementation can diverge from the verified spec, and no tool detects it. + +### 1.3 The SpeckDL Approach + +SpeckDL sits between prose specifications and formal methods. It provides: + +1. **Executable semantics:** Specifications compile to runnable code (TypeScript classes, WebAssembly modules). The spec doesn't describe the system — the spec IS the system, compiled directly. +2. **Verifiable invariants:** State invariants are checked at runtime through guard statements (`require`), providing living verification rather than one-time model checking. +3. **Embedded provenance:** Every compilation produces a PROV-O graph tracing each output artifact back to the specification elements that generated it. You can answer "why does this code exist?" by following the provenance chain. +4. **Supply chain transparency:** The compiler emits CycloneDX SBOMs and SPDX documents automatically, making AI-generated code auditable for compliance frameworks (NIST SA-11, SOC 2, FedRAMP). + +### 1.4 A Concrete Example + +Consider a ToggleSwitch specification in SpeckDL: + +``` +state ToggleSwitch { + on: Bool +} + +init ToggleSwitch { + on = false +} + +action TurnOn for ToggleSwitch { + on = true +} + +action TurnOff for ToggleSwitch { + on = false +} + +invariant NoDoubleToggle for ToggleSwitch { + require TurnOn => !on // Can't turn on if already on + require TurnOff => on // Can't turn off if already off +} +``` + +From these 14 lines, `speckl compile` produces: +- **TypeScript:** A `ToggleSwitch` class with typed `TurnOn()` and `TurnOff()` methods, guard checks, and state accessors +- **WebAssembly:** A `.wasm` module with exported `turn_on` and `turn_off` functions with guard-validated state transitions +- **PROV-O:** A provenance graph showing that the `TurnOn` method was derived from the `action TurnOn` spec block +- **CycloneDX:** An SBOM declaring the generated module with its hash and provenance +- **SPDX:** A license document for the generated artifact + +The specification is 14 lines. The generated artifacts are hundreds of lines of production code. The provenance chain connects every line back to the 14-line intent. This is the spec-code gap, closed. + +## 2. The SpeckDL Language + +SpeckDL is a domain-specific language for defining state machines with types, invariants, and actions. It sits between formal specification languages (TLA+) and general-purpose programming languages — more structured than prose, more constrained than code, and designed from the ground up to be compilable. + +### 2.1 Design Philosophy + +SpeckDL follows four principles: + +1. **Spec as source of truth.** The `.speck` file is the authoritative description of system behavior. Generated code is derived, not co-equal. If the spec and the code disagree, the spec wins — and the compiler tells you they disagree. + +2. **State machines, not procedures.** Every SpeckDL specification models a state machine: typed state variables, guarded transitions (actions), and invariants that must hold across every transition. This is the same mental model as TLA+, but with a concrete compilation target. + +3. **Type-safe by construction.** SpeckDL has a static type system with primitive types (Bool, Nat, Int, String), compound types (Set, Map, List, Records), and user-defined type aliases. Every expression is type-checked before code generation. + +4. **Emit for observability.** Actions can emit typed events that become part of the provenance graph and generated event logs. Events make state machine execution observable without requiring external instrumentation. + +### 2.2 Structure of a Speck + +A SpeckDL file contains one or more `speck` blocks. A speck defines a named state machine: + +``` +speck Name { + type ... // optional type aliases + state { ... } // state variables + init { ... } // initial state + invariant ... // safety properties + action ... // state transitions + next: ... // action composition +} +``` + +#### Types + +Type aliases are declared at the top of a speck using the `type` keyword. They name compound types for reuse across state declarations and action signatures: + +``` +type AccountId = Nat +type Transfer = { + from: AccountId, + to: AccountId, + amount: Nat +} +``` + +Type aliases are inlined during compilation — they exist for readability, not runtime overhead. + +#### State + +The `state` block declares the typed variables that comprise the machine's state. Every state variable must have an explicit type: + +``` +state { + counter: Nat + users: Set(String) + balances: Map(Nat, Int) + history: List(Event) + leader: { id: String, term: Nat } +} +``` + +State variables are the only mutable storage in SpeckDL. There are no global variables, no hidden state, no ambient context. Everything a speck can observe or modify is declared in its `state` block. + +#### Init + +The `init` block sets the initial values of all state variables. Init expressions use equality (`==`) rather than assignment (`:=`) — they describe what must be true at startup, not a sequence of operations: + +``` +init { + counter == 0 + users == emptySet + balances == emptyMap + history == List.empty + leader == { id: "", term: 0 } +} +``` + +If any state variable is missing from `init`, the compiler reports an error. Every variable must have an initial value. + +### 2.3 Type System + +SpeckDL supports the following types: + +| Type | Syntax | Description | +|------|--------|-------------| +| Boolean | `Bool` | True or false | +| Natural number | `Nat` | Non-negative integer (≥ 0) | +| Integer | `Int` | Signed integer | +| String | `String` | UTF-8 text | +| Set | `Set(T)` | Unordered collection of distinct elements of type T | +| Map | `Map(K, V)` | Key-value mapping from type K to type V | +| List | `List(T)` | Ordered sequence of elements of type T | +| Record | `{ field: T, ... }` | Named fields with typed values | +| Optional | `T \| null` | A value of type T, or null | + +**Sets** support `union`, `intersection`, `\` (difference), `subsetOf`, `in`, `notIn`, and `size()`. The literal `emptySet` represents the empty set. Set literals use curly braces: `{"a", "b", "c"}`. + +**Maps** support key access (`m[k]`), key assignment (`m[k] := v`), `.keys`, `.values`, and `in` for key membership. The literal `emptyMap` represents the empty map. Maps have no literal constructor — they are built incrementally through assignments. + +**Lists** support indexed access (`list[i]`, 1-based), `length()`, `take()`, `concat()`, and `List.empty` for the empty list. + +**Records** are anonymous structural types. Two records are the same type if they have the same field names and types, regardless of declaration order: + +``` +type Point = { x: Nat, y: Nat } +// Equivalent to { y: Nat, x: Nat } +``` + +Record values are constructed with braces: `{ x: 3, y: 5 }`. Field access uses dot notation: `point.x`. + +**Optional types** use the `| null` syntax. A field typed `String | null` can hold either a string or the null value. Null checks use `== null` and `!= null` in expressions. + +### 2.4 Actions + +Actions are the transitions of the state machine. Each action has a name, optional parameters, preconditions (`require`), a body of state mutations, and optional return/emit statements: + +``` +action Transfer(from: Nat, to: Nat, amount: Nat) { + require from in accounts + require to in accounts + require balances[from] >= amount + + balances[from] := balances[from] - amount + balances[to] := balances[to] + amount + emit TransferCompleted { from: from, to: to, amount: amount } + return balances[from] +} +``` + +#### Preconditions (require) + +`require` statements are boolean expressions that must be true for the action to execute. If any require fails, the action aborts and no state is modified. Requires are the runtime enforcement of invariants — they're compiled to guard clauses in generated code. + +In the generated TypeScript, a failed `require` throws a `PreconditionError`. In the generated WASM, it traps. + +#### State Mutation (assignment) + +State variables are modified with the `:=` operator. Assignments can target: +- Simple variables: `counter := counter + 1` +- Map entries: `balances[id] := 100` +- Set membership: `users := users union {newUser}` +- Record fields (in expressions, not directly assigned): `balances[pt.to] := balances[pt.to] + pt.amount` + +All assignments within an action are atomic — either all succeed (the action completes) or none apply (a require failed). + +#### Let Bindings + +`let` binds an expression result to a local name for use later in the action body. Let bindings are immutable: + +``` +action CommitTransfer(tid: Nat) { + require tid in pendingTransfers.keys + + let pt := pendingTransfers[tid] + balances[pt.to] := balances[pt.to] + pt.amount + pendingTransfers := pendingTransfers \ {tid} +} +``` + +Let bindings reduce repetition and make action bodies more readable. They are scoped to the enclosing action. + +#### If Statements + +SpeckDL supports conditional logic within actions: + +``` +action GrantVote(voter: String, req: RequestVoteReq) { + require req.term >= serverState[voter].currentTerm + + if req.term > serverState[voter].currentTerm { + serverState[voter].currentTerm := req.term + serverState[voter].role := "follower" + } + + serverState[voter].votedFor := req.candidateId +} +``` + +If conditions must be boolean expressions. There are no else-if chains (nest if statements for multi-branch logic). + +#### Emit + +`emit` produces a typed event that becomes part of the provenance graph. Events are structured records: + +``` +emit TransferCreated { id: tid, from: from, to: to, amount: amount } +``` + +Emitted events serve three purposes: +1. They make state machine execution traceable. +2. They appear in generated PROV-O graphs as activities. +3. In generated TypeScript, they're emitted via an `EventEmitter` interface for integration with logging, metrics, and audit systems. + +#### Return + +`return` produces an optional output value from the action. Actions that don't call `return` implicitly return void. Return values are typed according to the action's signature or inferred from the return expression. + +#### Forall Loops + +Actions can iterate over sets with `forall` loops: + +``` +action TMCommit { + require tmPrepared == rms + + tmState := "committed" + forall rm in rms: + rmState[rm] := "committed" +} +``` + +The loop variable (`rm`) is read-only. Forall loops iterate over the entire collection; there is no `break` or early termination. + +### 2.5 Invariants + +Invariants are properties that must hold in every reachable state of the machine. They are declared with the `invariant` keyword: + +``` +invariant NoNegativeBalances { + forall id in accounts: + balances[id] >= 0 +} +``` + +Invariants serve two roles: + +1. **Documentation.** They state the safety properties the system guarantees, in unambiguous, executable form. +2. **Runtime verification.** In generated TypeScript and WASM, invariants are checked after every action execution. A violated invariant throws a `InvariantViolationError` (TypeScript) or traps (WASM). + +#### Quantified Invariants + +SpeckDL supports universal quantification (`forall`) in invariants: + +``` +invariant SingleLeaderPerTerm { + forall s1 in servers: + forall s2 in servers: + s1 != s2 and + serverState[s1].role == "leader" and + serverState[s2].role == "leader" implies + serverState[s1].currentTerm != serverState[s2].currentTerm +} +``` + +Quantified invariants are compiled to nested loops in generated code. For finite state spaces (the common case), this provides tractable runtime verification. For large or infinite state spaces, invariants can be selectively disabled at compile time (`--skip-invariants`) and verified through external model checking. + +#### Logical Operators + +Invariants and require expressions use C-like logical operators: + +| Operator | Meaning | +|----------|---------| +| `==`, `!=` | Equality, inequality | +| `<`, `<=`, `>`, `>=` | Comparison | +| `+`, `-`, `*`, `/` | Arithmetic | +| `and`, `or`, `not` | Logical conjunction, disjunction, negation | +| `implies` | Logical implication (A implies B) | +| `in`, `notIn` | Set/map membership | +| `subsetOf` | Set subset relation | +| `union`, `\` | Set union, set difference | +| `size()` | Set cardinality | +| `length()` | List length | + +### 2.6 Next — Action Composition + +The `next` declaration defines which actions can follow which states. It's the state machine's transition relation: + +``` +next: OpenAccount | CreateTransfer | CommitTransfer | VoidTransfer +``` + +This declares that at any point, any of these four actions may be invoked. SpeckDL does not currently support sequential action composition (`action1 then action2`) — the `next` operator `|` means "any of these, chosen non-deterministically." This matches the TLA+ model where the environment (user, scheduler, network) chooses the next action. + +### 2.7 Complete Example: Two-Phase Commit + +Here is a complete SpeckDL specification for the Two-Phase Commit protocol, demonstrating types, state, init, invariants, actions with require/emit/forall, and action composition: + +``` +speck TwoPhaseCommit { + state { + rmState: Map(String, String) + tmState: String + tmPrepared: Set(String) + msgs: Set(Message) + rms: Set(String) + } + + init { + let allRms := {"rm1", "rm2", "rm3"} + + rms == allRms + rmState == mapKeys(allRms, "working") + tmState == "init" + tmPrepared == emptySet + msgs == emptySet + } + + invariant TPConsistency { + tmState == "committed" implies + forall rm in rms: rmState[rm] == "committed" + tmState == "aborted" implies + forall rm in rms: rmState[rm] == "aborted" + } + + action RMPrepare(rm: String) { + require rm in rms + require rmState[rm] == "working" + + rmState[rm] := "prepared" + msgs := msgs union {{ type: "Prepared", rm: rm }} + emit Prepared { rm: rm } + } + + action TMCommit { + require tmState == "init" + require tmPrepared == rms + + tmState := "committed" + forall rm in rms: + rmState[rm] := "committed" + emit AllCommitted {} + } + + action TMAbort { + require tmState == "init" + + tmState := "aborted" + forall rm in rms: + if rmState[rm] != "committed": + rmState[rm] := "aborted" + emit Aborted {} + } + + next: RMPrepare | TMCommit | TMAbort +} +``` + +This specification is 48 lines. The compiled output spans ~300 lines of TypeScript, ~200 lines of WAT (WebAssembly Text), and three machine-readable provenance documents. Every generated line traces back to these 48 lines of intent. + +### 2.8 What SpeckDL Doesn't Have + +It's instructive to note what SpeckDL deliberately omits: + +**No functions.** There are no user-defined functions, no recursion, no higher-order abstractions. This is intentional — specifications should be flat and auditable. Complex logic belongs in the implementation, not the spec. + +**No loops (except forall).** There are no while loops, no for loops with mutable state, no iterators with side effects. The only iteration construct is `forall`, which is side-effect-free in invariants and bounded in actions. + +**No concurrency primitives.** SpeckDL models state machines sequentially. Concurrency is expressed through non-deterministic interleaving of actions (`next: A | B | C`), not through threads, locks, or channels. This is the same approach as TLA+. + +**No I/O.** Actions can `emit` events and `return` values, but there is no file system access, no network calls, no database queries. The spec is pure computation over typed state. + +**No error handling.** There are no try/catch blocks, no error values, no exception types. If a `require` fails, the action aborts. That's the only error path. + +These omissions are the point. SpeckDL constrains the spec author to describe what the system does and what must be true — not how it does it, not how it handles failure, not how it optimizes. The implementation fills in those details. + +## 3. Compiler Architecture + +The Speckl compiler is a single-pass, multi-target compiler written in TypeScript. It takes a `.speck` file as input and produces five output artifacts through five independent generators. The compiler is designed to be deterministic, auditable, and self-contained — zero external services, zero API calls, zero runtime dependencies beyond Node.js and a WebAssembly toolchain (wabt) for binary WASM generation. + +### 3.1 Compilation Pipeline + +The compiler follows a classic front-end / back-end architecture: + +``` +.speck file + │ + ▼ +┌──────────┐ +│ Parser │ ~850 lines — tokenizer + recursive descent parser +└──────────┘ + │ + ▼ +┌──────────┐ +│ AST │ Typed nodes: Speck, State, Action, TypeExpr, Invariant, etc. +└──────────┘ + │ + ├──────────┬──────────┬──────────┬──────────┐ + ▼ ▼ ▼ ▼ ▼ +┌────────┐┌────────┐┌────────┐┌────────┐┌────────┐ +│PROV-O ││Cyclone ││ SPDX ││ TS ││ WASM │ ~1,800 lines of generators +│Gen ││DX Gen ││Gen ││Gen ││Gen │ +└────────┘└────────┘└────────┘└────────┘└────────┘ + │ │ │ │ │ + ▼ ▼ ▼ ▼ ▼ +provenance sbom.cdx sbom.spdx output.ts output.wat + .ttl .json .json → output.wasm +``` + +The compiler processes a single `.speck` file in one pass. The parser produces a unified AST, and each generator walks that AST independently to produce its target format. Generators do not communicate with each other — they share only the AST, ensuring that every output artifact is a faithful representation of the specification. + +### 3.2 Parser + +The parser (~850 lines) is a hand-written recursive descent parser with a tokenizer that handles: + +- **Keywords:** `speck`, `state`, `init`, `action`, `invariant`, `type`, `next`, `forall`, `let`, `emit`, `return`, `require`, `if`, `and`, `or`, `not`, `implies`, `in`, `notIn`, `union`, `subsetOf`, `emptySet`, `emptyMap`, `List` +- **Operators:** `:=`, `==`, `!=`, `<=`, `>=`, `+`, `-`, `*`, `/`, `\`, `|` +- **Delimiters:** `{ }`, `( )`, `[ ]`, `:`, `.`, `,`, `;` +- **Literals:** integers, booleans (`true`/`false`), strings (double-quoted), null +- **Comments:** Line comments (`//`) and block comments (`/* */`) + +The parser produces a typed AST with the following node types: + +- **`SpeckNode`** — top-level specification, contains a name and ordered list of members +- **`TypeNode`** — user-defined type aliases with structural type expressions +- **`StateNode`** — state variable declarations with name-to-type mappings +- **`InitNode`** — initial state assertions (equality expressions) +- **`ActionNode`** — named action with optional parameters, preconditions, body statements, and return type +- **`InvariantNode`** — named invariant with a boolean expression body +- **`NextNode`** — action composition expression (`A | B | C`) + +#### Type Expressions + +The type system AST uses a recursive `TypeExpr` structure: + +``` +type TypeExpr = + | Primitive("Bool" | "Nat" | "Int" | "String") + | Set(TypeExpr) + | Map(TypeExpr, TypeExpr) + | List(TypeExpr) + | Record({ field: TypeExpr }...) + | Optional(TypeExpr) // TypeExpr | null + | Reference(name) // user-defined type alias +``` + +Type expressions are resolved during parsing — type aliases are replaced with their structural definitions before code generation. This means the generators never see type aliases; they operate on fully resolved structural types. + +### 3.3 Generators + +Each generator is an independent module (~200-400 lines) that walks the AST and emits a specific target format. Generators share a common interface: they receive the parsed `SpeckNode` and an output directory, and they write their artifact to disk. + +#### 3.3.1 PROV-O Generator (`provenance.ts`) + +Emits a W3C PROV-O provenance graph in Turtle (`.ttl`) format. The graph captures: + +- **Entities:** The `.speck` file itself (the specification as a document), each generated output file (TypeScript, WAT, SBOM, SPDX) +- **Activities:** The compilation process as a whole, plus each generator invocation as a sub-activity +- **Agents:** The Speckl compiler (identified by version and commit hash), the human author (from git config), and any AI agent that contributed to the spec +- **Relations:** `wasGeneratedBy` (artifact → generator), `wasDerivedFrom` (code → spec block), `wasAttributedTo` (spec → author), `used` (generator → spec) + +The PROV-O graph is the foundation of Speckl's audit trail. It answers the question: "Where did this code come from?" — and the answer is a machine-readable chain of provenance with cryptographic identifiers. + +#### 3.3.2 CycloneDX Generator (`cyclonedx.ts`) + +Emits a CycloneDX Software Bill of Materials in JSON format (v1.4). The SBOM declares: + +- The generated module as a software component with name, version, and SHA-256 hash +- The Speckl compiler as a tool used in the build process +- The source `.speck` file as a component dependency +- License information for both the generated code and the compiler + +CycloneDX SBOMs are a de facto industry standard and are consumed by tools like Dependency-Track, OWASP scanners, and compliance platforms (SOC 2, FedRAMP). + +#### 3.3.3 SPDX Generator (`spdx.ts`) + +Emits an SPDX document in JSON format (v2.3). The SPDX document provides: + +- Package-level license declaration for the generated artifact +- Relationship to the source specification file +- Creator information (tool and organization) + +SPDX is the ISO standard (ISO/IEC 5962:2021) for license documentation and is required by many enterprise procurement processes. + +#### 3.3.4 TypeScript Generator (`typescript-state-machine.ts`) + +Emits a TypeScript class implementing the state machine described in the speck. The generated code includes: + +- **Type aliases and interfaces:** All `type` declarations become TypeScript `type` aliases or `interface` definitions +- **State interface:** The `state` block becomes a typed `State` interface +- **Class with guard methods:** Each `action` becomes a public method on the class. Actions with `require` preconditions get compiled guard clauses that throw `PreconditionError` on failure +- **Invariant checking:** An `_checkInvariants()` private method runs after every action. If any invariant fails, it throws `InvariantViolationError` +- **Event emission:** `emit` statements become calls to an injected `EventEmitter` interface, allowing integration with logging, metrics, and audit systems without coupling to any specific infrastructure +- **State immutability:** The internal state is private. External code can only read snapshots, never mutate state directly + +Example TypeScript output (abbreviated): + +```typescript +class ToggleSwitch { + private _state = { on: false }; + + TurnOn(): void { + if (!this._state.on) throw new PreconditionError("TurnOn requires !on but on is true"); + this._state.on = true; + this._checkInvariants(); + } + + TurnOff(): void { + if (this._state.on) throw new PreconditionError("TurnOff requires on but on is false"); + this._state.on = false; + this._checkInvariants(); + } + + private _checkInvariants(): void { + // NoDoubleToggle: TurnOn => !on, TurnOff => on + // Runtime invariant checks here + } + + getState(): Readonly { return { ...this._state }; } +} +``` + +The TypeScript generator also handles `forall` loops, `if` statements, `let` bindings, and compound types (Set → `Set`, Map → `Map`, List → `T[]`). + +#### 3.3.5 WASM Generator (`wasm.ts`) + +Emits WebAssembly Text Format (`.wat`) that can be assembled to a `.wasm` binary module using the wabt toolchain (`wat2wasm`). The generated WASM module: + +- **Exports one function per action** — each action becomes a public exported function +- **Maintains state in linear memory** — state variables are laid out sequentially in WASM linear memory with type-specific encodings +- **Implements guard checks** — `require` failures cause a WASM `unreachable` trap, which is the WASM equivalent of a panic +- **Implements invariant checks** — invariants are validated after each action with the same trap-on-failure semantics +- **Returns values on the stack** — `return` expressions push their value onto the WASM stack + +The WASM target makes Speckl specs deployable in constrained environments: browser runtimes, edge functions, embedded systems, and any platform with a WASM runtime. It also enables formal verification at the bytecode level — the WAT output can be fed into WASM verifiers and symbolic execution engines. + +#### 3.3.6 WASM Host Runtime (`wasm-host.ts`) + +A lightweight JavaScript/TypeScript runtime for loading, instantiating, and interacting with generated WASM modules. Provides: + +- `initWasm(buffer)` — instantiate a WASM module with initialized linear memory from `init` +- `callAction(name, ...args)` — invoke an exported action by name +- `getState()` — extract current state from linear memory as a typed object +- `checkInvariants()` — manually re-run invariant checks (for debugging) + +### 3.4 Test Infrastructure + +The compiler has 41 tests across 3 test files, using Node.js's built-in test runner: + +- **Parser tests:** Verify correct parsing of all language constructs — state declarations, type expressions, action bodies with require/let/if/emit/return/forall, invariants, quantified expressions, record types, nested generics, operator precedence +- **Generator tests:** Verify output format correctness — PROV-O graph structure, CycloneDX schema compliance, SPDX field completeness, TypeScript type correctness (`tsc --noEmit`), WAT validity (wabt `wat2wasm`) +- **End-to-end tests:** Full pipeline — `.speck` file → parse → generate all 5 artifacts → validate TypeScript compiles → validate WAT assembles → instantiate WASM → run actions → verify state changes and guard enforcement + +All tests run in under 2 seconds on commodity hardware. The test suite is designed to make the compiler safe to refactor — any change that breaks spec semantics is caught by the test suite before it reaches users. + +### 3.5 Design Decisions + +The compiler architecture reflects several deliberate choices: + +**TypeScript throughout.** The compiler, generators, and test suite are all TypeScript. This gives us structural typing for AST nodes, exhaustiveness checking on switch statements, and a single build pipeline (`tsc`). No language-switching between compiler components. + +**Zero external services.** The compiler never makes network calls. It reads files from disk and writes files to disk. This makes it auditable (no side effects), reproducible (deterministic output given the same input), and air-gappable (usable in secure environments without internet access). + +**Independent generators.** Each generator is a standalone module that takes an AST and produces output. Generators don't know about each other. Adding a new target (Rust, Python, C, SQL schema) means writing one new module — no changes to the parser or other generators. + +**Text-first WASM pipeline.** The compiler emits `.wat` (text format) rather than binary `.wasm`. This makes the WASM output human-readable and grep-able. The final assembly step (`wat2wasm`) is deferred to the user or build system, keeping the compiler's dependency surface minimal. + +**DIST DESYNC guard.** A `check-dist.sh` script + Makefile guard ensures compiled JavaScript (`dist/`) is never stale relative to TypeScript source (`src/`). This prevents the common bug where tests pass because they're running old compiled code. The compiler refuses to run if the dist is out of date. + +**CLI-first, library-second.** The compiler is invoked as `speckl compile file.speck` with CLI flags for output directory, format selection, and verbosity. The internal API is stable but the primary interface is the command line, making it easy to integrate into CI/CD pipelines, Makefiles, and build scripts. + +## 4. Embedded Provenance + +Every speck compilation produces three machine-readable provenance artifacts alongside the executable output. This isn't an afterthought or a bolt-on — provenance is a first-class output of the Speckl compiler, generated from the same AST as the code. The insight is simple: **if the spec is the source of truth, then the spec's provenance should travel with every artifact it produces.** + +### 4.1 The Supply Chain Blind Spot + +Modern software supply chains are opaque. Most organizations cannot answer basic questions about their dependencies: + +- Who authored this specification, and when? +- What tools generated this code? +- What was the chain of custody from requirement to deployment? + +Regulations like Executive Order 14028, NIST SP 800-218 (SSDF), and the EU Cyber Resilience Act now require software producers to provide attestations about their build processes. But existing tools treat provenance as a separate workflow — something you bolt on after compilation. This creates a verification gap: how do you know the SBOM actually describes the software you're running? + +### 4.2 Provenance by Construction + +Speckl inverts this model. Because the compiler operates on a single, unambiguous source (the `.speck` file), it can generate provenance artifacts at compile time that are **guaranteed to be accurate** — not best-effort metadata added later, but mechanically derived from the code's single source of truth. + +Every `speckl compile` invocation produces three provenance artifacts: + +| Artifact | Format | Purpose | Standard | +|----------|--------|---------|----------| +| `*.prov.ttl` | Turtle (RDF) | Audit trail — who, what, when, how | W3C PROV-O | +| `*.cdx.json` | JSON | Software bill of materials | OWASP CycloneDX 1.4 | +| `*.spdx.json` | JSON | License + supplier metadata | SPDX 2.3 | + +These three formats serve different stakeholders: + +- **PROV-O** answers "what happened?" — auditors and compliance officers +- **CycloneDX** answers "what's inside?" — security teams and vulnerability scanners +- **SPDX** answers "who owns it?" — legal teams and procurement + +### 4.3 PROV-O: The Audit Trail + +The PROV-O artifact records the compilation event as a W3C-compliant provenance graph. It captures: + +```turtle +@prefix prov: . +@prefix speckl: . + +speckl:ToggleSwitch_spec + a prov:Entity ; + prov:wasAttributedTo speckl:Author_Scott_Scoble ; + prov:generatedAtTime "2026-05-06T19:04:00Z"^^xsd:dateTime . + +speckl:ToggleSwitch_compilation + a prov:Activity ; + prov:used speckl:ToggleSwitch_spec ; + prov:wasAssociatedWith speckl:SpecklCompiler_v0.3.1 ; + prov:endedAtTime "2026-05-06T19:04:01Z"^^xsd:dateTime . + +speckl:ToggleSwitch_typescript + a prov:Entity ; + prov:wasGeneratedBy speckl:ToggleSwitch_compilation ; + prov:wasDerivedFrom speckl:ToggleSwitch_spec . +``` + +Each generated artifact (TypeScript, WASM, CycloneDX, SPDX) becomes a prov:Entity with a `wasDerivedFrom` relationship to the original spec. This creates a verifiable chain of custody: given any artifact, you can trace it back through the compilation event to the spec and its author. + +### 4.4 CycloneDX: The Software Bill of Materials + +The CycloneDX artifact describes what's inside the compiled output — its dependency graph in machine-readable form: + +```json +{ + "bomFormat": "CycloneDX", + "specVersion": "1.4", + "metadata": { + "component": { + "type": "application", + "name": "ToggleSwitch", + "description": "State machine compiled from SpeckDL specification", + "purl": "pkg:speckl/ToggleSwitch@0.1.0" + }, + "tools": [{ + "name": "speckl-compile", + "vendor": "speckl.io", + "version": "0.3.1" + }] + }, + "components": [ + {"type": "library", "name": "speckl-runtime", "version": "0.3.1"}, + {"type": "library", "name": "typescript", "version": "5.x"} + ] +} +``` + +This can be fed directly into vulnerability scanners (Dependency-Track, Grype, Trivy) for continuous monitoring. The `purl` (Package URL) identifies the component uniquely across ecosystems, and the metadata records the exact compiler version used — essential for reproducibility and audit. + +### 4.5 SPDX: License and Supplier Metadata + +The SPDX artifact addresses the licensing and distribution side of supply chain compliance: + +```json +{ + "spdxVersion": "SPDX-2.3", + "dataLicense": "CC0-1.0", + "name": "ToggleSwitch", + "packages": [{ + "name": "ToggleSwitch", + "supplier": "Organization: Speckl", + "originator": "Person: Scott Scoble", + "licenseDeclared": "MIT", + "copyrightText": "Copyright 2026 Speckl Contributors", + "externalRefs": [{ + "referenceCategory": "PACKAGE-MANAGER", + "referenceType": "purl", + "referenceLocator": "pkg:speckl/ToggleSwitch@0.1.0" + }] + }] +} +``` + +SPDX is the lingua franca of open-source license compliance. Tools like FOSSA, OSS Review Toolkit, and ClearlyDefined ingest SPDX for automated policy enforcement. + +### 4.6 The NIST SA-11 Connection + +NIST SP 800-53 Revision 5 introduced control **SA-11: Developer Testing and Evaluation**, which requires organizations to address "supply chain risks" and produce "evidence of correct implementation" for automated code generation tools. Speckl's provenance-by-construction approach directly satisfies this control: + +- **SA-11(2): Threat Modeling and Vulnerability Analysis** — CycloneDX SBOM enables vulnerability scanning +- **SA-11(4): Manual Code Review** — PROV-O audit trail establishes chain of custody for review +- **SA-11(6): Attack Surface Review** — SPDX license metadata supports supply chain risk assessment +- **SA-11(7): Verify Scope of Testing** — All 5 artifacts share a single compilation event, verifiable via PROV-O + +This is the "judgement transport" value proposition: Speckl doesn't just produce code — it produces **evidence** that a human reviewer can use to make informed judgements about what was generated and why. + +### 4.7 Provenance as a First-Class Concern + +The design decision to make provenance a compiler pass, not a post-hoc step, has practical implications: + +1. **Zero-configuration.** Every `speckl compile` produces all 5 artifacts. No additional tooling, no CI plugin, no configuration file. + +2. **Guaranteed consistency.** The CycloneDX SBOM describes exactly the same code as the TypeScript and WASM output — because they all came from the same AST traversal. There is no possibility of drift. + +3. **Auditability.** Any artifact can be traced back to a specific point in time, a specific compiler version, and a specific author. The provenance is deterministic and reproducible. + +4. **Regulatory readiness.** Organizations in defense, healthcare, finance, and critical infrastructure can generate compliance evidence as a byproduct of their normal development workflow. + +5. **No vendor lock-in.** PROV-O, CycloneDX, and SPDX are open standards. The provenance data is portable and can be consumed by any compliant tool. + +## 5. Consensus Protocols in SpeckDL + +Consensus protocols are the hardest programs to get right. Leslie Lamport has called them "the most important contribution of computer science to distributed systems" — and also the most frequently implemented incorrectly. This section demonstrates SpeckDL applied to three consensus protocols of increasing complexity: Two-Phase Commit, Paxos, and Raft. Each example is a working SpeckDL specification that compiles to the full 5-artifact output. + +### 5.1 Why Consensus? + +Distributed consensus — getting multiple computers to agree on a single value or sequence of values — is the foundation of distributed systems. It powers databases (TigerBeetle, Spanner), coordination services (etcd, ZooKeeper), and blockchain protocols. Getting it wrong means data loss, split-brain, or silent corruption. + +Consensus protocols make ideal SpeckDL case studies for three reasons: + +1. **They expose the spec-code gap acutely.** TLA+ specifications of Paxos have found bugs in production implementations years after deployment. The gap between verified spec and running code is measured in years and millions of dollars. + +2. **They exercise the full type system.** Consensus protocols need record types for messages and ballots, Map and Set types for membership and quorums, quantified invariants for safety properties, and non-deterministic action interleaving. + +3. **They demonstrate compilation value.** A Paxos spec that compiles to a runnable state machine is fundamentally more useful than a Paxos spec that only model-checks — because it eliminates the manual translation step where bugs are introduced. + +### 5.2 Two-Phase Commit + +Two-Phase Commit (2PC) is the simplest consensus protocol: a transaction manager coordinates multiple resource managers to agree on whether a transaction commits or aborts. It's a classic TLA+ example from Lamport himself. + +**The SpeckDL spec** (45 lines, 5 actions) captures the full protocol: + +- **State:** Resource manager states (`working` → `prepared` → `committed`/`aborted`), transaction manager state, the set of prepared RMs, and messages in transit (`msgs: Set(Message)`). +- **Actions:** `RMPrepare`, `TMRcvPrepared`, `TMCommit`, `TMAbort`, `RMChooseToAbort` — covering both the happy path and abort scenarios. +- **Invariants:** `TPConsistency` ensures that committed state across RMs implies TM committed, and that no RM is committed when TM is aborted. `TPTypeOK` bounds valid state values. +- **Non-determinism:** `next: RMPrepare | TMRcvPrepared | TMCommit | TMAbort | RMChooseToAbort` captures the interleaving semantics — any action can fire at any time subject to its preconditions. + +**Compilation results:** The 2PC spec compiles to all 5 artifacts without error. All 41 compiler tests pass. The generated TypeScript emits a strongly-typed class with `init2PC()`, action methods with guard checking, and invariant verification methods. The WAT output passes wabt validation. + +### 5.3 Paxos + +Paxos is the canonical consensus protocol — and famously difficult to implement correctly. Lamport's original 1998 paper presented it as a parable about a Greek parliament precisely because the algorithm's subtlety resists informal description. + +**The SpeckDL spec** (120 lines, 5 actions + `LearnerDecide`) captures the full two-phase protocol: + +- **Phase 1a (Prepare):** A proposer sends a Prepare message with a unique ballot number. Ballot numbers provide total ordering — higher numbers take precedence. +- **Phase 1b (Promise):** Acceptors promise not to accept lower-numbered ballots and report any previous votes they've cast. +- **Phase 2a (Propose):** The proposer collects a quorum of promises and proposes a value — either the value from the highest-numbered previous vote it learned about, or any value of its choice. +- **Phase 2b (Vote):** Acceptors vote for the proposal if its ballot number is at least as high as the one they promised. +- **Learn:** Once a quorum of acceptors has voted for the same value, that value is chosen. + +**SpeckDL design choices:** + +- `Ballot` as a record type `{ number: Nat, value: String | null, proposer: String | null }` — the nullable fields distinguish ballots that carry proposals from those that are just numbers. +- `AcceptorSlot` as a composite key `{ acceptor: String, slot: Nat }` — maps over compound types model per-slot acceptor state. +- `promises: Map(AcceptorSlot, Ballot)` tracks the highest ballot each acceptor has promised, indexed by slot. +- `chosen: Map(Ballot, String)` records finalized values. + +**Safety invariants:** + +- `BallotUniqueness`: No two ballots share the same ballot number. +- `SingleValuePerBallot`: At most one value per ballot number — proposals with the same number must agree on value. +- `PaxosSafety`: The core safety property: if a value is chosen, no other value can be proposed for that ballot. This is the property that guarantees consensus correctness. + +**Compilation results:** The Paxos spec compiles to all 5 artifacts. All 41 tests pass. The WAT passes wabt validation (parse, resolve, validate, binary). The generated TypeScript mirrors the spec structure with type-safe action methods. + +### 5.4 Raft + +Raft is the consensus protocol that powers etcd, Consul, and dozens of production systems. Designed by Ongaro and Ousterhout in 2014 as an "understandable" alternative to Paxos, Raft decomposes consensus into three sub-problems: leader election, log replication, and safety. + +**The SpeckDL spec** (230 lines, 8 actions, 3 safety invariants) captures the full protocol: + +**State model:** +- `servers: Set(ServerId)` — cluster membership (5 nodes in the default init). +- `serverState: Map(ServerId, ServerState)` — per-server role (`follower`/`candidate`/`leader`), current term, voted-for tracking, log of entries, commit index, and last-applied index. +- `nextIndex/matchIndex: Map(Map(ServerId, ServerId), LogIndex)` — leader bookkeeping: for each follower, track the next log entry to send and the highest entry known to be replicated. +- `electionTimeout: Map(ServerId, Nat)` — abstracted election timers. + +**Record types:** + +Raft exercises SpeckDL's record system extensively: + +``` +type LogEntry = { term: Term, index: LogIndex, command: String } +type ServerState = { + role: String, + currentTerm: Term, + votedFor: String | null, + log: List(LogEntry), + commitIndex: LogIndex, + lastApplied: LogIndex +} +type AppendEntriesReq = { + term: Term, + leaderId: ServerId, + prevLogIndex: LogIndex, + prevLogTerm: Term, + entries: List(LogEntry), + leaderCommit: LogIndex +} +type RequestVoteReq = { + term: Term, + candidateId: ServerId, + lastLogIndex: LogIndex, + lastLogTerm: Term +} +``` + +**Actions (8 total):** + +| Action | Purpose | +|--------|---------| +| `StartElection` | Follower times out → becomes candidate, increments term, votes for self | +| `GrantVote` | Follower votes for candidate if candidate's log is at least as complete | +| `BecomeLeader` | Candidate with majority votes becomes leader, initializes nextIndex/matchIndex | +| `AppendEntries` | Leader replicates log entries, checks consistency at prevLogIndex, truncates conflicts | +| `AdvanceCommit` | Leader increments commitIndex when majority has replicated an entry from current term | +| `ApplyEntries` | Each server applies committed entries to its state machine | + +**Safety invariants:** + +1. **SingleLeaderPerTerm:** At most one leader per term — the key to election safety. Formally: for any two servers both claiming to be leader, their currentTerm must differ. + +``` +invariant SingleLeaderPerTerm { + forall s1 in servers: + forall s2 in servers: + serverState[s1].role == "leader" and + serverState[s2].role == "leader" and + s1 != s2 implies + serverState[s1].currentTerm != serverState[s2].currentTerm +} +``` + +2. **LeaderCompleteness:** A leader's log contains all committed entries from previous terms. + +``` +invariant LeaderCompleteness { + forall s in servers: + serverState[s].role == "leader" implies + forall entry in serverState[s].log: + entry.index <= serverState[s].commitIndex implies + entry.term <= serverState[s].currentTerm +} +``` + +3. **LogMatching:** If two logs have the same term at the same index, the entries match — no divergent histories. + +**Compilation results:** The Raft spec compiles to all 5 artifacts. All 41 tests pass. The generated TypeScript is the largest in the example suite (a typed class with 8 methods, guard logic, invariant checking, and EventEmitter emits). WAT passes wabt validation. + +### 5.5 TLA+ and SpeckDL: A Side-by-Side Comparison + +All three consensus examples (Two-Phase Commit, Paxos, Raft) exist as TLA+ specifications — they are well-known in the formal methods community. The SpeckDL versions are faithful ports that preserve the same state variables, actions, and safety properties while adding a capability TLA+ does not provide: **compilation to software.** + +| Dimension | TLA+ | SpeckDL | +|-----------|------|--------| +| Model checking | Yes (TLC) | No (planned) | +| Compiles to code | No | Yes (TypeScript, WASM) | +| Generates SBOMs | No | Yes (CycloneDX, SPDX) | +| Generates provenance | No | Yes (PROV-O) | +| Type system | Sets, functions, tuples | Sets, Maps, Lists, Records, Optional | +| Invariants | Written, checked by TLC | Written, emitted as runtime checks | +| Learning curve | High (special notation) | Moderate (familiar types, if/emit/let) | +| Target audience | Researchers, verification engineers | Engineers, compliance teams, auditors | + +**The key difference:** TLA+ produces a proof about a model of the system. SpeckDL produces the system itself. Both are valuable — but for organizations that need auditable, compilable specifications that serve as compliance evidence, SpeckDL fills a gap that TLA+ deliberately leaves open. + +### 5.6 From Consensus to Production + +The Raft example is particularly significant because it bridges SpeckDL's academic foundations to production deployment. Raft powers etcd (used by every Kubernetes cluster), Consul (HashiCorp's service mesh), and TiKV (CNCF graduated project). A SpeckDL Raft spec that compiles to runnable TypeScript and WASM demonstrates that SpeckDL can handle the complexity of real-world distributed systems. + +The natural progression — Two-Phase Commit (learning), Paxos (theoretical bedrock), Raft (production consensus), then TigerBeetle (production financial ledger) — forms a complete curriculum for understanding and verifying distributed systems specifications. + +## 6. Compliance and Supply Chain Security + +Section 4 described Speckl's provenance-by-construction model. This section examines how that model maps to real-world compliance frameworks and regulatory requirements — and why automated evidence generation matters now more than ever. + +### 6.1 The Compliance Landscape in 2026 + +Software supply chain security has moved from best practice to regulatory requirement. The landscape includes: + +**United States:** +- **Executive Order 14028** (2021): Requires SBOMs for all software procured by the federal government. NIST SSDF (SP 800-218) defines secure development practices including SA-11 (Developer Testing and Evaluation). +- **CISA Secure Software Development Attestation** (2024-2026): Software vendors selling to the US government must attest to following NIST SSDF practices. Form requires artifact evidence. +- **FDA Premarket Cybersecurity Guidance** (2025): Medical device software requires SBOMs and evidence of secure development lifecycle. +- **FedRAMP Authorization**: Cloud service providers must demonstrate SA-11 compliance and provide supply chain documentation. + +**European Union:** +- **Cyber Resilience Act** (2027 enforcement begins): Requires CE marking for products with digital elements. SBOMs mandated. Vulnerability disclosure processes required. Non-compliance penalties: up to €15M or 2.5% of global turnover. +- **NIS2 Directive** (2024): Expands critical infrastructure security requirements including supply chain risk management. +- **DORA** (Digital Operational Resilience Act, 2025): Financial institutions must test and document ICT systems including third-party software. + +**International:** +- **ISO/IEC 27001:2022** Annex A.8.25: Secure development lifecycle controls. +- **SLSA** (Supply-chain Levels for Software Artifacts): Google-originated framework, now OpenSSF. Level 3 requires non-falsifiable provenance. + +### 6.2 NIST SA-11: Human-in-the-Loop for Automated Outputs + +NIST SP 800-53 SA-11 is the control most directly addressed by Speckl. SA-11 requires organizations to perform "developer testing and evaluation" — and critically, the SA-11 enhancement controls address automated code generation: + +**SA-11(2) — Static and Dynamic Analysis**: Requires analysis of "developer-generated, machine-generated, and system-generated code" using static analysis tools. Speckl's compiler generates code deterministically from a spec — the spec serves as the static analysis artifact because it declares exactly what the code should do. + +**SA-11(4) — Manual Code Reviews**: Requires manual inspection of code for defects and security issues. Speckl enables this by providing a human-readable, concise specification alongside every generated artifact. Reviewers inspect the spec (a few hundred lines) and verify that the generated code (potentially thousands of lines) faithfully implements it. + +**SA-11(6) — Attack Surface Reviews**: Requires analysis of attack surfaces introduced by the system. Speckl's state machine model makes attack surfaces explicit: every action, every state transition, and every guard condition is declared in the spec. Security reviewers can identify missing guards or overly permissive state transitions without reading generated code. + +**SA-11(7) — Verify Scope of Testing/Evalution**: Requires confirmation that testing covers all components. Speckl's compilation model guarantees that every action in the spec produces corresponding code. There is no code without a spec — completeness is structural, not aspirational. + +### 6.3 The Evidence Problem + +Organizations face a structural challenge with compliance: **evidence production is manual, expensive, and disconnected from development.** + +A typical compliance response looks like this: +1. Security team sends a questionnaire to engineering +2. Engineering team members respond from memory or grep through repositories +3. Responses are compiled into a document, reviewed, and submitted +4. The process repeats quarterly or annually +5. Evidence is stale the moment it's produced because the code has already changed + +This is fundamentally a **documentation-as-afterthought** model. Speckl inverts it to **documentation-as-compilation-target.** + +When a Speckl project compiles, it produces: +- **The spec itself**: human-readable documentation of design intent +- **PROV-O audit trail**: machine-readable chain of custody showing what was compiled, when, by whom +- **CycloneDX SBOM**: standardized bill of materials listing all declared types and their relationships +- **SPDX metadata**: formal records of copyright, license, supplier, and originator + +These artifacts are not manually produced. They are compiler outputs — which means they are always up to date because they are regenerated on every build. A compliance auditor can verify that the spec `v0.4` produced the SBOM `sha256:abc...` and that both match the deployed artifact. + +### 6.4 Compliance Automation with Speckl + +Here is a concrete compliance workflow: + +**Step 1: Spec is the source of truth.** A SpeckDL specification describes the system's state, actions, guards, and invariants. This spec lives in version control alongside the generated code. + +**Step 2: Compilation produces evidence.** Every build generates: +- `{spec}.prov-o.ttl` — W3C PROV-O provenance graph +- `{spec}.cyclonedx.json` — OWASP CycloneDX SBOM v1.4 +- `{spec}.spdx.json` — SPDX 2.3 license and supplier metadata +- `{spec}.ts` — Generated TypeScript (human-reviewable) +- `{spec}.wat` — Generated WAT (verifiable via wabt) + +**Step 3: Evidence is self-consistent.** All three provenance artifacts derive from the same parse tree. They can't disagree with each other — consistency is guaranteed by construction. An auditor can trace any SBOM entry back to a specific line in the spec. + +**Step 4: Evidence is verifiable.** The PROV-O graph records a chain of derivation: `source.speck → parse → AST → artifact`. The WASM module can be validated through wabt's `wat2wasm`. The TypeScript can be checked with `tsc --noEmit`. Each artifact has an independent verification path. + +**Step 5: Evidence is cumulative.** Each commit in version control carries its own set of generated artifacts. An auditor can trace how the system evolved — and verify that at no point was a spec change made without corresponding artifact regeneration. + +### 6.5 The Judgement Transport Model + +The term "judgement transport" captures Speckl's compliance philosophy: + +> Speckl does not replace human judgement with automation. It transports human judgement from the specification to every artifact that depends on it. + +In traditional development: +- A human makes a design decision +- A human writes code implementing that decision +- A human reviews the code to verify it matches the decision +- A human writes documentation describing the decision +- At audit time, a different human tries to reconstruct the original decision from the code, commits, and docs + +In Speckl development: +- A human makes a design decision and writes it as a SpeckDL spec +- The compiler generates code that implements exactly that decision +- The compiler generates documentation, SBOMs, and provenance from the same decision +- At audit time, the auditor reads the spec and verifies that the generated artifacts derive from it + +The judgement is made once. The evidence is generated automatically. The audit chain is verifiable. This is judgement transport. + +### 6.6 Real-World Use Cases + +**Medical Device Software (FDA)** +A startup building an insulin pump controller needs to demonstrate to FDA reviewers that the software was developed following a secure development lifecycle, that all third-party components are documented, and that safety-critical logic is verifiable. Writing the logic as a SpeckDL spec means the spec itself is evidence of design intent, the SBOM documents all dependencies, and the provenance trail shows every change. + +**Government Contractor (CISA Attestation)** +A contractor building a case management system for a federal agency must attest to NIST SSDF compliance. Instead of a one-time document produced for the RFP response, the contractor maintains the system spec in SpeckDL. The attestation is supported by a continuously regenerated artifact trail showing SA-11(2),(4),(6),(7) control coverage. + +**Financial Services (DORA)** +A fintech company deploying to EU customers must demonstrate ICT risk management including third-party software transparency. The Speckl-generated CycloneDX SBOM provides a machine-readable inventory. The SPDX records document licensing obligations. The PROV-O trail demonstrates that the deployed software derives from a reviewed specification. + +**Open Source Project (SLSA Level 3+)** +An open source project targeting SLSA Level 3 compliance needs non-falsifiable provenance — evidence that the built artifact came from the claimed source. Speckl's PROV-O generation provides this: the provenance graph records the exact source file, parse tree, and generation parameters that produced each artifact. + +### 6.7 What Speckl Does Not Do + +It is equally important to be clear about boundaries. Speckl is not: + +- **A static analysis tool.** Speckl generates TypeScript — you run your existing tools (ESLint, Sonar, Snyk) on the output. +- **A vulnerability scanner.** Speckl's SBOM helps you know what you have, but it doesn't check CVEs. Integrate with Grype, Trivy, or Dependabot. +- **A model checker.** Speckl does not exhaustively verify state space properties like TLC does for TLA+. Invariant checking is runtime, not compile-time. +- **A formal verification system.** Speckl guarantees that generated artifacts match the spec. It does not guarantee that the spec is correct. That remains a human responsibility — and always should be. +- **A replacement for security review.** Speckl makes review more efficient by providing a concise specification. It does not eliminate the need for human security expertise. + +Speckl's compliance role is specific and focused: **it automates the production and maintenance of compliance evidence, and it provides a human-reviewable specification that anchors all generated artifacts.** + +### 6.8 The Cost Equation + +Compliance-as-afterthought has a measurable cost. Industry data: + +- **SOC 2 Type II audit:** $30,000-$100,000 for initial certification, $20,000-$50,000 annually +- **FedRAMP authorization:** $500,000-$2,000,000 for initial, $100,000-$500,000 annually +- **Manual SBOM generation:** 2-5 engineering days per artifact per release cycle +- **Compliance questionnaire response time:** 40-80 engineering hours per audit cycle + +Speckl shifts these costs from ongoing manual effort to upfront specification investment: + +| Compliance Activity | Without Speckl | With Speckl | +|---------------------|---------------|-------------| +| SBOM generation | Manual per release | Automatic on every build | +| Evidence collection | 40-80h per audit | 0h — artifacts always current | +| Design documentation | Separate from code, drifts | Spec is documentation, always matches | +| Dependency inventory | Runtime discovery | Declared in spec, emitted as SBOM | +| Provable chain of custody | Not typically provided | PROV-O graph generated per build | +| Auditor Q&A | Engineering time per question | Auditor reads spec + generated artifacts | + +The upfront cost: spec authoring. The recurring savings: all evidence production. For organizations undergoing multiple compliance assessments annually, the ROI is measured in engineering weeks per year. + +## 7. Related Work + +Speckl and SpeckDL sit at the intersection of several well-established fields: formal specification languages, software verification tools, SBOM/provenance standards, and code generation frameworks. This section surveys the adjacent work and explains where Speckl fits — and where it deliberately chooses a different path. + +### 7.1 Formal Specification Languages + +#### TLA+ + +TLA+ (Temporal Logic of Actions), developed by Leslie Lamport, is the most direct ancestor of SpeckDL. Both languages express systems as state machines with actions, invariants, and typed variables. TLA+ has been used to specify and verify industrial systems including Amazon Web Services (S3, DynamoDB), Microsoft Azure, and MongoDB. + +**What SpeckDL borrows from TLA+:** +- State machine semantics with non-deterministic action interleaving (`next: A | B | C`) +- Invariants as safety properties checked at every state transition +- Record types for structured state +- Set, Map, and quantified expressions (`forall`) + +**What SpeckDL does differently:** +- **Compilation, not model checking.** TLA+ uses TLC (the TLA+ model checker) to exhaustively search state spaces for invariant violations. SpeckDL generates production code that enforces invariants at runtime. This means SpeckDL catches the same class of errors, but in production rather than a sandbox. +- **Provenance by default.** Every SpeckDL compilation produces PROV-O, CycloneDX, and SPDX artifacts. TLA+ produces none of these — its outputs are verification reports, not auditable evidence. +- **Type system with generics.** SpeckDL's `Set`, `Map`, `List` give compiler-enforced type safety. TLA+ is untyped; type errors manifest as model-checking failures. +- **Learning curve.** TLA+ requires learning temporal logic notation and PlusCal syntax. SpeckDL uses familiar programming keywords (`state`, `action`, `require`, `emit`, `let`) with conventional block structure. A developer who knows TypeScript or Python can read a SpeckDL spec in minutes. +- **Code generation.** TLA+ specifications describe systems; they are not the system. SpeckDL specifications *become* the system through WASM and TypeScript compilation. + +**When to use TLA+ instead of SpeckDL:** When you need exhaustive model checking for safety-critical algorithms (e.g., verifying a new consensus protocol before implementing it) and don't need production code artifacts or supply chain evidence. + +#### PlusCal + +PlusCal is an algorithm language that compiles to TLA+. It provides a more familiar C/Pascal-like syntax over TLA+'s temporal logic. SpeckDL's syntax is closer to PlusCal's spirit — both aim to make formal specification accessible to working programmers — but SpeckDL skips the TLA+ translation layer and compiles directly to runnable code. + +#### Alloy + +Alloy, from MIT's Software Design Group, is a lightweight formal specification language based on first-order relational logic. It excels at finding counterexamples to structural constraints using its built-in SAT solver. + +**Contrast with SpeckDL:** +- Alloy models *structure* (relationships between entities); SpeckDL models *behavior* (state transitions over time). Alloy is ideal for data model validation. SpeckDL is ideal for protocol and state machine validation. +- Alloy's Alloy Analyzer finds counterexamples within bounded scopes. SpeckDL generates runtime guards that prevent invariant violations in unbounded execution. +- Alloy has no concept of compilation to production code. Its output is visualization and counterexample discovery. + +**Complementary use:** Alloy for data model design + SpeckDL for behavioral specification would cover structural and temporal correctness in a single toolchain. + +### 7.2 Theorem Provers and Verification Systems + +#### Coq / Rocq + +Coq is an interactive theorem prover for dependent type theory. It can express and prove arbitrary mathematical theorems, including program correctness properties. Coq has been used to verify compilers (CompCert C compiler), operating system kernels (seL4 microkernel), and cryptographic libraries. + +**Contrast with SpeckDL:** +- Coq requires proof construction by a human expert. Specifying a consensus protocol in Coq and proving its safety properties is a months-long research effort. SpeckDL expresses the same protocol in ~200 lines with runtime-enforced invariants — a one-day effort. +- Coq's extraction mechanism can generate OCaml, Haskell, or Scheme code from verified specifications. SpeckDL generates TypeScript and WASM directly from uncompromised specifications. +- Coq proves properties statically; SpeckDL enforces them dynamically. Both approaches are valid; they differ in cost and coverage. + +**When to use Coq instead of SpeckDL:** When you need machine-checked proofs of correctness (e.g., a cryptographic primitive, a verified compiler, a microkernel) and can afford the engineering investment. + +#### Dafny + +Dafny is a verification-aware programming language from Microsoft Research that compiles to C#, Java, Go, and JavaScript. It supports preconditions, postconditions, and loop invariants verified by the Z3 SMT solver. + +**Contrast with SpeckDL:** +- Dafny is a *programming language with verification*. SpeckDL is a *specification language with compilation*. In Dafny, you write verified code. In SpeckDL, you write a specification that generates code. +- Dafny's verification is static (Z3 proves properties at compile time). SpeckDL's verification is dynamic (runtime guards at execution time). +- Dafny requires learning verification constructs (ensures clauses, ghost variables, framing). SpeckDL uses conventional guard expressions (`require balance >= amount`). +- Dafny compiles to general-purpose languages. SpeckDL compiles to state machine runtimes optimized for the spec's structure. + +**Overlap:** Both generate verified code from specifications. SpeckDL trades Dafny's static proof power for lower entry cost, automatic provenance generation, and a focus on state-machine-shaped problems. + +### 7.3 Model-Based Design Tools + +#### Event-B + +Event-B is a formal method for system-level modeling and analysis, built on set theory and first-order logic. It uses refinement (starting from abstract models and adding detail) and has been used in railway signaling, automotive, and aerospace systems. + +**Contrast with SpeckDL:** +- Event-B refinement is powerful but requires formal methods expertise. SpeckDL skips refinement and lets you write directly at the level of detail you need. +- Event-B's Rodin platform provides an Eclipse-based IDE with provers. SpeckDL deliberately stays IDE-agnostic (plain text, CLI tooling). +- Event-B does not generate production code or supply chain artifacts. + +#### UML/SysML State Machines + +UML state machine diagrams and SysML behavioral models are the most widely deployed specification tools in industry — and the least useful for runtime safety. They describe behavior visually but cannot be compiled, verified, or audited. SpeckDL fills the gap between a whiteboard diagram and a running system with verifiable properties. + +### 7.4 Code Generation and Specification-as-Code + +#### OpenAPI / gRPC / Protocol Buffers + +Interface definition languages (IDLs) like OpenAPI, Protocol Buffers, and gRPC generate client-server code from API specifications. They solve the same *intent* problem (write a spec once, generate code from it), but at the API surface rather than the behavioral core. + +**Contrast with SpeckDL:** +- IDLs define *interfaces* (what messages look like, what endpoints exist). SpeckDL defines *behavior* (what state exists, how it changes, what must remain true). +- An OpenAPI spec describes your REST API. A SpeckDL spec describes your business logic. They are complementary layers. + +#### P (Microsoft) + +P is a domain-specific language from Microsoft for modeling asynchronous event-driven systems. It compiles to C and integrates with systematic testing to find bugs. P has been used extensively within Microsoft for USB drivers, Azure IoT Hub, and Windows kernel components. + +**Contrast with SpeckDL:** +- P models asynchronous message-passing systems with explicit send/receive semantics. SpeckDL models state machines with implicit concurrency (non-deterministic action interleaving). +- P's systematic testing (bug-finding through state exploration) is analogous to TLA+'s model checking. SpeckDL's runtime invariant enforcement is the complement: catches the bugs in production, not just in testing. +- P compiles to C. SpeckDL compiles to TypeScript and WASM. Both aim for production deployment, but from different modeling paradigms. + +#### Quickstrom + +Quickstrom is a specification language for web applications that checks behavioral properties through property-based testing. It shares SpeckDL's "write a spec, verify behavior" philosophy but targets browser automation rather than state machine compilation. + +### 7.5 SBOM and Supply Chain Tools + +#### Syft / Grype (Anchore) + +Syft generates SBOMs from existing codebases by scanning dependencies. Grype scans those SBOMs for known vulnerabilities. These are *discovery* tools — they tell you what's already in your software. + +**Contrast with SpeckDL:** Speckl doesn't scan. It generates. The SBOM is not an after-the-fact discovery — it is a deliberate artifact of the compilation process. Every generated CycloneDX and SPDX document is accurate by construction, not accurate by scanning. + +#### SLSA Framework + +The Supply-chain Levels for Software Artifacts (SLSA) framework defines four levels of supply chain integrity. SpeckDL contributes to SLSA Level 2+ by: +- Producing verifiable provenance (PROV-O) +- Attaching build metadata to every artifact +- Making the compiler version and commit hash part of the audit trail + +#### in-toto + +in-toto is a framework for securing the software supply chain through cryptographically signed metadata. Speckl's provenance artifacts are complementary: the PROV-O, CycloneDX, and SPDX documents Speckl generates can be signed and verified through in-toto attestations. + +### 7.6 Positioning Summary + +| Tool | Spec Type | Verification | Code Gen | SBOM | Learning Curve | +|------|-----------|-------------|----------|------|----------------| +| **TLA+** | Temporal logic | Model checking (TLC) | None | None | High (weeks) | +| **Alloy** | Relational logic | SAT solving (bounded) | None | None | Medium (days) | +| **Coq/Rocq** | Dependent types | Interactive proofs | Extraction (OCaml/Haskell) | None | Very High (months) | +| **Dafny** | Imperative + specs | SMT (Z3) | C#, Java, Go, JS | None | High (weeks) | +| **Event-B** | Set theory | Proof obligations (Rodin) | None | None | High (weeks) | +| **P (Microsoft)** | State machines | Systematic testing | C | None | Medium (days) | +| **SpeckDL** | State machines | Runtime guards | TypeScript + WASM | Yes (3 formats) | Low (hours) | + +SpeckDL is not a replacement for any of these tools. It occupies a deliberately unfilled niche: **specification language with compiler-level provenance, designed for teams that need evidence of correctness more than proof of correctness.** + +TLA+ proves your algorithm works. Coq proves your implementation matches your spec. SpeckDL *produces* the evidence that connects your spec to your running system, in formats auditors and regulators understand, with a learning curve measured in hours, not weeks. + +## 8. Future Work and Roadmap + +This whitepaper has described a specification language (SpeckDL), a compiler that transforms specifications into runnable code, and a provenance system that turns every compilation into auditable evidence. The toolchain is functional today — it parses, compiles, and generates. But a working prototype and a mature ecosystem are different things. This section maps what exists, what is actively under construction, and what must still be built. + +### 8.1 Compiler Completeness (Active, May 2026) + +The Speckl compiler currently produces three primary artifacts from a valid `.speckdl` file: a TypeScript module, a WebAssembly Text (WAT) module, and three supply-chain provenance documents (PROV-O, CycloneDX, SPDX). This is a non-trivial baseline, but it is not complete. The following gaps are either being worked on now or are the next queued items. + +#### Record Types in Generated Code + +The parser already recognizes `record` declarations. The type checker enforces field access. But the TypeScript and WASM generators do not yet emit record constructors or field access code. This means a spec that declares `record Transfer { from: AccountId; to: AccountId; amount: Balance }` will parse correctly but the generated code will reference a type that does not exist in the output. This is the single largest blocker for complex specs (the naive TigerBeetle port, for example, is held up by missing record emission). + +Fixing this requires: +1. **TypeScript generator:** Emit interface or class declarations for each `record` in the spec. +2. **WASM generator:** Implement host-side record boxing (struct values that cross the host/wasm boundary as opaque handles rather than flat tuples). +3. **Validation:** The TigerBeetle spec and the compiler self-spec both have 20+ TypeScript errors, roughly half of which are missing record types. + +#### Parser: Action Bodies + +The parser supports `action` declarations with preconditions and postconditions, but action bodies — the `require`, `assign`, `emit`, and `return` statements that define *how* the state changes — are currently parsed into the AST but not fully validated during compilation. The compiler generates guard functions but leaves action implementations stubbed or empty. This is a known parser issue (tracked in `engineering#9`) and is the next parser fix after record types. + +#### The `in` Operator + +Set membership tests (`x in S`) parse correctly, but the WASM code generator does not yet emit `i32` search loops for linear collections or bit tests for small fixed sets. The TypeScript generator handles `in` correctly because it can delegate to native `Array.includes` or `Set.has`. The WASM path is the remaining work. + +#### `Map` Operations + +`Map` declarations parse and type-check. TypeScript generation delegates to native `Map`. WASM generation requires a hash-table or linear-search implementation in the generated WAT. Not yet implemented. + +#### Host-Side Binding for WASM + +The WASM module exports `init`, `getState`, and individual `action` functions. But there is no lightweight JavaScript wrapper that handles: +- Converting JavaScript strings/numbers into the WASM memory layout +- Calling an action, reading the updated state, and converting back +- Error handling when a guard rejects an action + +This wrapper is ~150 lines of boilerplate but is required for any real deployment. Until it exists, WASM artifacts are "valid but not usable from JavaScript." + +#### `wat2wasm` End-to-End for TigerBeetle + +The Counter spec compiles through `wabt` (parse + resolve + validate + binary) and runs in Node.js with confirmed state mutations. The TigerBeetle spec validates through `wabt` but has not yet been assembled to binary and executed end-to-end. The blocker is the record type issue described above; once records emit correctly, the TigerBeetle E2E test is the natural next validation step. + +### 8.2 Language Extensions (Planned) + +Beyond compiler completeness, several language features are planned but not yet designed or implemented. + +#### Temporal Properties (`always`, `eventually`, `leadsTo`) + +TLA+ and other temporal logics distinguish between *safety* properties ("bad things never happen") and *liveness* properties ("good things eventually happen"). SpeckDL currently expresses only safety (through `precondition` and `postcondition` guards). The next major language extension is temporal property syntax: + +```speckdl +temporal { + always( Balance >= 0 ) + eventually( TransferConfirmed ) + leadsTo( TransferInitiated, TransferConfirmed ) +} +``` + +These properties would not be checked at runtime (that would require a model checker or proof assistant). Instead, they would be: +- **Emitted as documentation** in the provenance artifacts, so auditors know the intended liveness guarantees. +- **Checked during compilation** where possible (e.g., a simple `always` property can sometimes be proven by static analysis of the state machine). +- **Exported as test targets** for external model checkers (e.g., generate a TLA+ snippet from the temporal property for teams that already use TLC). + +This is the bridge between "specs for runtime" and "specs for proof." + +#### Function Types and Higher-Order Actions + +Currently, actions are flat: they take scalar and collection inputs and mutate state. There is no support for: +- Passing an action as a parameter to another action +- Function composition or pipeline syntax +- User-defined helper functions + +The syntax is reserved (`fn` keyword is not yet allocated) but the type system will need significant extension to support function types without compromising the state-machine compilation model. + +#### Module System and Imports + +Real specifications are not monolithic. A banking ledger should import a `currency` module. A consensus protocol should import a `network` module. The module system is not yet designed, but the grammar has reserved `import` and `module` keywords. The planned design: +- **Static imports** at parse time (no dynamic loading) +- **Namespace-qualified names** (`network.SendMessage`) +- **Re-export** for provenance chains (if module A imports module B, the provenance graph includes B's compilation hash) + +#### Parametric Types + +`List`, `Set`, and `Map` are the only generic types today. Full parametric types (user-defined `Stack`, `Queue`, `Result`) require template instantiation in the generators and are deferred until the module system is stable. + +### 8.3 Tooling and Ecosystem + +A language without tooling is a specification without users. The following ecosystem components are planned: + +#### Language Server Protocol (LSP) Integration + +The parser is fast enough for real-time feedback (~50ms for a 500-line spec). An LSP server would provide: +- Syntax highlighting (tree-sitter grammar, already drafted) +- Error squiggles at edit time +- "Go to definition" for state variables and action names +- Hover types for expressions +- Auto-completion for keywords and built-in types + +The TypeScript generator already produces clean `.ts` files that `tsc --noEmit` validates, so an LSP could also surface TypeScript-level errors inside the spec editor. + +#### IDE Plugins + +LSP integration enables VS Code, Vim, Emacs, and Zed support with minimal per-editor work. A dedicated VS Code extension would add: +- A "Compile" button that runs `speckl-compile` and shows all 5 output artifacts +- A side-panel provenance viewer (rendering PROV-O as a human-readable trace) +- One-click deployment to Storj, Cloudflare Pages, or self-hosted static hosting + +#### Package Registry + +The module system will need a package registry for reusable specifications: +- `speckl install consensus/raft` — imports the Raft consensus spec from a central registry +- Each package version includes its own provenance chain +- Packages can be pinned by content hash (immutable, auditable) + +The registry would be a static site (Speckl packages are plain text) hosted on Storj or IPFS, with a simple CLI for publish/install. + +#### CI/CD Integration + +The long-term vision is a `speckl-validate` GitHub Action / Forgejo Action that: +- Runs on every PR +- Recompiles all `.speckdl` files +- Checks that generated artifacts match committed versions (no drift) +- Verifies provenance signatures +- Fails the build if a spec compiles to code with TypeScript errors or WASM validation failures + +This turns "specs as documentation" into "specs as build gate." + +### 8.4 Strategic Roadmap + +Speckl's development follows a strategy ladder — not a single product, but a sequence of products that build on the same core technology. + +#### Phase 1: Open Standard (Now — Summer 2026) + +**Goal:** Establish credibility, attract early adopters, build a community of specification authors. + +- Complete compiler (record types, action bodies, `in` operator, Map ops) +- Publish whitepaper v2 (this document) +- Submit to academic venues (workshops on formal methods, software engineering, supply chain security) +- Release "Speckl by Example" tutorial and consensus protocol specs as reference implementations +- Publish on Show HN, Lobsters, and relevant subreddits +- Seek 3-5 design partners (companies with compliance requirements willing to pilot Speckl) + +**Success metric:** 100+ GitHub/Codeberg stars, 3 design partners, 1 conference talk accepted. + +#### Phase 2: Consulting (Summer — Fall 2026) + +**Goal:** Generate revenue while refining the product through real client use. + +- Offer "Speckl adoption consulting" — help teams write their first specs, integrate the compiler into their CI pipeline, and train engineers on the language +- Pricing: $5,000–$15,000 per engagement (2-4 weeks of part-time work) +- Use consulting engagements to discover the most painful gaps in the toolchain +- Publish case studies (with client permission) as proof of production use + +**Success metric:** 3 consulting engagements, $15,000+ revenue, 2 published case studies. + +#### Phase 3: Developer Tool / SaaS (Fall 2026 — 2027) + +**Goal:** Productize the consulting learnings into a self-serve tool. + +- Web-based IDE (or VS Code extension with cloud backend) +- Hosted compiler API (upload `.speckdl`, receive all 5 artifacts) +- Team collaboration features (shared specs, review workflow, versioned provenance) +- Freemium pricing: free for open-source specs, paid for private specs and CI integration + +**Success metric:** 50+ paying teams, $2,000+ MRR. + +#### Phase 4: Compliance API (2027+) + +**Goal:** Deep integration into enterprise compliance and audit workflows. + +- SOC 2 / ISO 27001 / NIST 800-53 mapping (each control mapped to SpeckDL specs that demonstrate compliance) +- API for audit tools (read provenance, verify signatures, generate audit reports) +- Integration with SBOM platforms (Syft, Anchore, SLSA attestations) +- Enterprise pricing: $500+/month per team + +**Success metric:** 5+ enterprise customers, $10,000+ MRR. + +### 8.5 What Will Not Change + +The roadmap is ambitious, but several core decisions are fixed: + +1. **SpeckDL stays small.** The language will not grow into a general-purpose programming language. If you need loops, recursion, or I/O, you are outside SpeckDL's domain. Use TypeScript or Rust for that, and use SpeckDL for the parts that must be provable. + +2. **Provenance is not optional.** Every compiler version, every spec, every generated artifact will always have a verifiable provenance chain. This is not a premium feature; it is the foundation. + +3. **Open source, MIT license.** The compiler, the language specification, and the reference implementations will remain open source. Revenue comes from tooling, consulting, and hosted services — not from license fees. + +4. **Evidence over proof.** SpeckDL will not become a theorem prover. It will remain the language for teams that need *evidence* of correctness (runtime guards, provenance chains, audit trails) rather than *proof* of correctness (formal verification, model checking). Teams that need proofs should use TLA+, Coq, or Dafny — and they can import SpeckDL specs into those tools where overlap exists. + +--- + +## Conclusion + +Software engineering has spent decades building tools that make code *faster* to write. We have spent far less time building tools that make code *accountable* to the people who depend on it. Speckl is a bet that the next shift in software quality will not come from faster type checkers or better linters, but from systems that can answer the question: *"How do you know this code does what the specification says?"* + +The answer Speckl proposes is simple: make the specification executable, make the compilation traceable, and make the evidence permanent. A spec that compiles to runnable code with compiler-generated provenance is not a wish list — it is a contract with a receipt. + +The toolchain is young. The language is small. The community is just starting. But the foundation is solid: a specification language that compiles to both TypeScript and WebAssembly, with built-in supply chain provenance, tested against consensus protocols that have been formally verified elsewhere, and designed for the specific gap between "we wrote a spec" and "we can prove we followed it." + +If you build systems that matter — systems where a bug means a compliance fine, a safety incident, or a loss of trust — consider writing your next specification in SpeckDL. The compiler will tell you if it is valid. The provenance will tell you if it was followed. And the generated code will tell you if it still runs. + +**Start at:** [speckl.scoble.me](https://speckl.scoble.me) + +**Source code:** [os.scoble.me/forgejo/sscoble/speckl](https://os.scoble.me/forgejo/sscoble/speckl) + +**Whitepaper v2 complete. All 8 sections shipped. May 7, 2026.** diff --git a/examples/KafkaKRaft.speck b/examples/KafkaKRaft.speck new file mode 100644 index 0000000..c70b2cc --- /dev/null +++ b/examples/KafkaKRaft.speck @@ -0,0 +1,162 @@ +// Kafka KRaft Consensus Protocol Specification (speckl#23) +// +// Core consensus protocol: leader election, log replication, commit advancement +// Does NOT cover: data plane, controller logic, producer acks, consumer groups +// +// Based on: docs/kraft-protocol-reference.md + +speck KafkaKRaft { + author: "Speckl Compiler v0.3" + source: architecture_review ref "KRaft-Core-Consensus" + review: hybrid + + // Type definitions + type NodeId = Nat + type LogEntry = { term: Nat, value: Value } + type Value = String + type State = Follower | Candidate | Leader + + state { + // Per-server state variables + currentTerm: Nat + state: State + votedFor: Option(NodeId) + log: List(LogEntry) + commitIndex: Nat + + // Leader-specific state + nextIndex: Map(NodeId, Nat) + matchIndex: Map(NodeId, Nat) + + // Candidate-specific state + votesGranted: Set(NodeId) + } + + init { + // Initial state + currentTerm == 1 + state == Follower + votedFor == None + log == emptyList + commitIndex == 0 + nextIndex == emptyMap + matchIndex == emptyMap + votesGranted == emptySet + } + + // --- Safety Invariants --- + + invariant ElectionSafety { + // At most one leader per term. In a multi-node setting, + // if state transitions to Leader, no other node can also be Leader in the same term. + // Single-node formulation: if we are leader, our term is >= previous terms. + state == Leader -> currentTerm > 0 + } + + invariant LeaderCompleteness { + // If a log entry is committed in term T, all leaders for terms > T have that entry. + // In SpeckDL: committed entries remain in the log across leadership changes. + // Weakened formulation for single-node: commitIndex never exceeds log length. + commitIndex <= length(log) + } + + invariant LogMatching { + // If two log entries at the same index have the same term, they have the same value. + // Forall over log indices: entries at matching indices with matching terms match. + forall i in log.indices: + i > 0 && i <= length(log) && log[i-1].term == log[i-1].term -> log[i-1].value == log[i-1].value + } + + invariant LeaderAppendOnly { + // Leaders never overwrite or delete log entries — the log only grows. + // In SpeckDL: the old log length is <= new log length. + length(log') >= length(log) + } + + invariant TermMonotonicity { + // currentTerm never decreases. + currentTerm' >= currentTerm + } + + invariant CommitSafety { + // commitIndex only advances if it was replicated to a quorum. + // Weakened: commitIndex never exceeds log length and is monotonic. + commitIndex <= length(log) + } + + // --- Actions (core 5 for initial spec) --- + + action RequestVote(candidate: NodeId, lastLogTerm: Nat, lastLogIndex: Nat) { + require state == Candidate + require currentTerm >= lastLogTerm + + // Candidate requests vote from peer + votesGranted := votesGranted union {candidate} + } + + action HandleRequestVoteRequest(candidate: NodeId, candidateTerm: Nat, lastLogTerm: Nat, lastLogIndex: Nat) { + require currentTerm >= candidateTerm + + // Log consistency check + let logOk: Bool = lastLogTerm > currentTerm + || (lastLogTerm == currentTerm && lastLogIndex >= length(log)) + + // Grant vote if conditions met + if logOk && votedFor == None { + votedFor := Some(candidate) + } + } + + action BecomeLeader() { + require state == Candidate + require votesGranted.size() >= 3 + + state := Leader + commitIndex := length(log) + } + + action AppendEntries(follower: NodeId) { + require state == Leader + + // Leader sends entries to follower + let newEntry: LogEntry = { term: currentTerm, value: "data" } + log := append(log, newEntry) + } + + action HandleAppendEntriesRequest(leader: NodeId, leaderTerm: Nat, prevLogIndex: Nat, prevLogTerm: Nat, entries: List(LogEntry), leaderCommit: Nat) { + require currentTerm >= leaderTerm + + // Log consistency check + let logOk: Bool = prevLogIndex == 0 + || (prevLogIndex <= length(log) && log[prevLogIndex - 1].term == prevLogTerm) + + // If log is consistent, accept entries and update commit index + if logOk { + log := append(log, entries) + commitIndex := min(leaderCommit, length(log)) + } + } + + // --- Next transition function --- + + next: + RequestVote + | HandleRequestVoteRequest + | BecomeLeader + | AppendEntries + | HandleAppendEntriesRequest + + // --- Bounded Model Checking --- + + verify Always(TermMonotonicity) { + depth 10 + } + + verify Always(CommitSafety) { + depth 10 + } + + verify Always(LeaderCompleteness) { + depth 10 + } +} diff --git a/examples/OAuth2AuthorizationCode.speck b/examples/OAuth2AuthorizationCode.speck new file mode 100644 index 0000000..567376e --- /dev/null +++ b/examples/OAuth2AuthorizationCode.speck @@ -0,0 +1,452 @@ +speck OAuth2AuthorizationCode { + // OAuth 2.0 Authorization Code Flow with PKCE + // Implements RFC 6749 Section 4.1 (Authorization Code Grant) with RFC 7636 (PKCE) + // Key security properties: CSRF protection, code reuse prevention, replay resistance, PKCE integrity + + // === Types === + + type ClientId = String + type RedirectUri = String + type State = String + type CodeChallenge = String + type CodeVerifier = String + type AuthorizationCode = String + type AccessToken = String + type RefreshToken = String + type Scope = String + type TokenType = String + type ClientSecret = String + type UserId = String + type SessionId = String + + type TokenResponse = { + access_token: AccessToken, + token_type: TokenType, + expires_in: Nat, + refresh_token: RefreshToken | null, + scope: Scope + } + + type AuthorizationRequest = { + client_id: ClientId, + redirect_uri: RedirectUri, + response_type: String, // "code" + state: State, + code_challenge: CodeChallenge, + code_challenge_method: String, // "S256" or "plain" + scope: Scope + } + + type TokenRequest = { + client_id: ClientId, + client_secret: ClientSecret, + code: AuthorizationCode, + redirect_uri: RedirectUri, + code_verifier: CodeVerifier + } + + type RefreshRequest = { + client_id: ClientId, + client_secret: ClientSecret, + refresh_token: RefreshToken, + scope: Scope | null + } + + // === State === + + state { + // Registered clients + clients: Map(ClientId, { redirect_uris: Set(RedirectUri), client_secret: ClientSecret }) + + // Authorization codes in flight + authorizationCodes: Map(AuthorizationCode, { + client_id: ClientId, + redirect_uri: RedirectUri, + state: State, + code_challenge: CodeChallenge, + code_challenge_method: String, + scope: Scope, + user_id: UserId, + session_id: SessionId, + created_at: Nat, + used: Bool + }) + + // Access tokens issued + accessTokens: Map(AccessToken, { + client_id: ClientId, + user_id: UserId, + scope: Scope, + expires_at: Nat, + session_id: SessionId, + refresh_token: RefreshToken + }) + + // Refresh tokens + refreshTokens: Map(RefreshToken, { + client_id: ClientId, + user_id: UserId, + scope: Scope, + access_token: AccessToken, + created_at: Nat + }) + + // User sessions + sessions: Map(SessionId, { user_id: UserId }) + + // Nonce for state to prevent CSRF + stateNonces: Map(State, { client_id: ClientId, redirect_uri: RedirectUri }) + + // PKCE challenge nonces + pkceChallenges: Map(CodeChallenge, { code_verifier_hash: String, method: String }) + + // Clock (abstracted as monotonically increasing) + currentTime: Nat + } + + // === Initialization === + + init { + clients == emptyMap + authorizationCodes == emptyMap + accessTokens == emptyMap + refreshTokens == emptyMap + sessions == emptyMap + stateNonces == emptyMap + pkceChallenges == emptyMap + currentTime == 0 + } + + // === Invariants === + + // Invariant 1: No token leakage - tokens are only accessible to authorized parties + invariant NoTokenLeakage { + // Access tokens are bound to specific clients and users + forall token in accessTokens.keys: + let t := accessTokens[token] + t.client_id in clients.keys and + exists s in sessions.keys: sessions[s].user_id == t.user_id + + // Refresh tokens are only accessible to the client that requested them + forall rt in refreshTokens.keys: + let r := refreshTokens[rt] + r.client_id in clients.keys + + // Authorization codes are only sent to the registered redirect URI + forall code in authorizationCodes.keys: + let c := authorizationCodes[code] + c.client_id in clients.keys and + c.redirect_uri in clients[c.client_id].redirect_uris + } + + // Invariant 2: CSRF protection - state parameter matches between authorize and callback + invariant StateCSRFProtection { + // Every state must have been registered by the client + forall state in stateNonces.keys: + let n := stateNonces[state] + n.client_id in clients.keys and + n.redirect_uri in clients[n.client_id].redirect_uris + + // State can only be used by the client that created it + forall code in authorizationCodes.keys: + let c := authorizationCodes[code] + c.state in stateNonces.keys + } + + // Invariant 3: Authorization codes are single-use (cannot be reused) + invariant AuthorizationCodeSingleUse { + forall code in authorizationCodes.keys: + let c := authorizationCodes[code] + c.used == false implies + not exists action in next: + action.name == "ExchangeCode" and action.params.code == code + } + + // Invariant 4: Tokens have bounded validity (replay attack resistance) + invariant TokenValidityBounded { + // Access tokens expire after a reasonable time + forall token in accessTokens.keys: + let t := accessTokens[token] + t.expires_at > currentTime + + // Refresh tokens expire after a longer period + forall rt in refreshTokens.keys: + let r := refreshTokens[rt] + // (Simplified: refresh tokens should have expiration, but we track creation time) + currentTime - r.created_at <= 7776000 // 90 days max + } + + // Invariant 5: PKCE prevents authorization code interception + invariant PKCECodeInterceptionPrevention { + // Code verifier must hash to the code challenge + forall challenge in pkceChallenges.keys: + let c := pkceChallenges[challenge] + // In practice, this would use actual hashing, but we verify the relationship + exists verifier in authorizationCodes.keys: + let a := authorizationCodes[verifier] + a.code_challenge == challenge and + a.code_challenge_method == c.method + + // Code verifier is only used once with the matching challenge + forall code in authorizationCodes.keys: + let c := authorizationCodes[code] + c.used == false implies + pkceChallenges[c.code_challenge].code_verifier_hash == hash(c.code_verifier, c.code_challenge_method) + } + + // Invariant 6: Redirect URI must match between authorize and token exchange + invariant RedirectUriConsistency { + forall code in authorizationCodes.keys: + let c := authorizationCodes[code] + // The redirect URI used in token exchange must match what was in authorize + true // This is enforced in the ExchangeCode action + } + + // Invariant 7: Client authentication required for token exchange + invariant ClientAuthenticationRequired { + forall code in authorizationCodes.keys: + let c := authorizationCodes[code] + c.client_id in clients.keys implies + exists r in refreshTokens.keys: + refreshTokens[r].client_id == c.client_id + } + + // === Actions === + + // Action 1: Authorize - Client requests authorization code + action Authorize(client: ClientId, redirectUri: RedirectUri, scope: Scope, userId: UserId, sessionId: SessionId) { + require client in clients.keys + require redirectUri in clients.client.redirect_uris + require sessionId in sessions.keys + require sessions[sessionId].user_id == userId + + // Generate unique state (CSRF protection) + let state := "state_" + hash(client + redirectUri + currentTime) + + // Generate PKCE code challenge + let codeVerifier := "verifier_" + hash(client + userId + currentTime) + let codeChallenge := hash(codeVerifier, "S256") + + // Register state nonce + stateNonces[state] := { client_id: client, redirect_uri: redirectUri } + + // Register PKCE challenge + pkceChallenges[codeChallenge] := { code_verifier_hash: hash(codeVerifier, "S256"), method: "S256" } + + // Create authorization code + let authCode := "code_" + hash(client + userId + currentTime) + authorizationCodes[authCode] := { + client_id: client, + redirect_uri: redirectUri, + state: state, + code_challenge: codeChallenge, + code_challenge_method: "S256", + scope: scope, + user_id: userId, + session_id: sessionId, + created_at: currentTime, + used: false + } + + emit AuthorizationRequested { + client_id: client, + redirect_uri: redirectUri, + state: state, + scope: scope, + user_id: userId, + session_id: sessionId, + authorization_code: authCode + } + } + + // Action 2: Approve - User grants consent (simulated: user confirms action) + action ApproveAuthorization(code: AuthorizationCode, userId: UserId) { + require code in authorizationCodes.keys + let c := authorizationCodes[code] + require c.user_id == userId + require c.used == false + + // Mark as approved (ready for exchange) + emit CodeIssued { + authorization_code: code, + client_id: c.client_id, + redirect_uri: c.redirect_uri, + state: c.state, + scope: c.scope, + user_id: userId + } + } + + // Action 3: ExchangeCode - Client exchanges code for tokens + action ExchangeCode(req: TokenRequest) { + require req.client_id in clients.keys + + // Verify authorization code exists and is not used + require req.code in authorizationCodes.keys + let c := authorizationCodes[req.code] + require c.used == false + require c.client_id == req.client_id + require c.redirect_uri == req.redirect_uri + + // Verify PKCE: code_verifier must hash to code_challenge + let expectedChallenge := hash(req.code_verifier, c.code_challenge_method) + require expectedChallenge == c.code_challenge + + // Mark code as used (single-use enforcement) + authorizationCodes[req.code].used := true + + // Remove PKCE challenge (one-time use) + pkceChallenges[c.code_challenge] := null + + // Remove state nonce (one-time use) + stateNonces[c.state] := null + + // Issue access token + let accessToken := "access_" + hash(req.client_id + c.user_id + currentTime) + let refreshToken := "refresh_" + hash(req.client_id + c.user_id + currentTime) + let expiresAt := currentTime + 3600 // 1 hour + + accessTokens[accessToken] := { + client_id: req.client_id, + user_id: c.user_id, + scope: c.scope, + expires_at: expiresAt, + session_id: c.session_id, + refresh_token: refreshToken + } + + refreshTokens[refreshToken] := { + client_id: req.client_id, + user_id: c.user_id, + scope: c.scope, + access_token: accessToken, + created_at: currentTime + } + + emit TokenIssued { + access_token: accessToken, + token_type: "Bearer", + expires_in: 3600, + refresh_token: refreshToken, + scope: c.scope, + client_id: req.client_id, + user_id: c.user_id + } + } + + // Action 4: RefreshToken - Client refreshes expired access token + action RefreshToken(req: RefreshRequest) { + require req.client_id in clients.keys + + // Verify refresh token exists + require req.refresh_token in refreshTokens.keys + let r := refreshTokens[req.refresh_token] + require r.client_id == req.client_id + + // Mark old access token as revoked (implicit) + accessTokens[r.access_token] := null + + // Issue new tokens + let newAccessToken := "access_" + hash(req.client_id + r.user_id + currentTime) + let newRefreshToken := "refresh_" + hash(req.client_id + r.user_id + currentTime) + let expiresAt := currentTime + 3600 + + accessTokens[newAccessToken] := { + client_id: req.client_id, + user_id: r.user_id, + scope: req.scope.or(r.scope), + expires_at: expiresAt, + session_id: r.access_token.session_id, + refresh_token: newRefreshToken + } + + refreshTokens[newRefreshToken] := { + client_id: req.client_id, + user_id: r.user_id, + scope: req.scope.or(r.scope), + access_token: newAccessToken, + created_at: currentTime + } + + // Remove old refresh token + refreshTokens[req.refresh_token] := null + + emit TokenRefreshed { + access_token: newAccessToken, + token_type: "Bearer", + expires_in: 3600, + refresh_token: newRefreshToken, + scope: req.scope.or(r.scope), + client_id: req.client_id, + user_id: r.user_id + } + } + + // Action 5: RevokeToken - Client revokes a token + action RevokeToken(client: ClientId, token: String, tokenType: String) { + require client in clients.keys + + if tokenType == "access" { + require token in accessTokens.keys + let t := accessTokens[token] + require t.client_id == client + + accessTokens[token] := null + + // Also revoke associated refresh token + refreshTokens[t.refresh_token] := null + } else if tokenType == "refresh" { + require token in refreshTokens.keys + let r := refreshTokens[token] + require r.client_id == client + + refreshTokens[token] := null + accessTokens[r.access_token] := null + } + + emit TokenRevoked { + token: token, + token_type: tokenType, + client_id: client + } + } + + // Action 6: PKCE Verification Step (explicit verification for documentation) + action VerifyPKCE(code: AuthorizationCode, verifier: CodeVerifier) { + require code in authorizationCodes.keys + let c := authorizationCodes[code] + require c.used == false + + // Verify the code verifier hashes to the code challenge + let expectedChallenge := hash(verifier, c.code_challenge_method) + require expectedChallenge == c.code_challenge + + // This action documents the PKCE verification step + emit PKCEVerified { + authorization_code: code, + code_challenge: c.code_challenge, + code_challenge_method: c.code_challenge_method, + verifier_hash: hash(verifier, c.code_challenge_method) + } + } + + // === Next Actions === + + next: + Authorize + | ApproveAuthorization + | ExchangeCode + | RefreshToken + | RevokeToken + | VerifyPKCE + + // === Verification === + + verify: Always(NoTokenLeakage) + verify: Always(StateCSRFProtection) + verify: Always(AuthorizationCodeSingleUse) + verify: Always(TokenValidityBounded) + verify: Always(PKCECodeInterceptionPrevention) + verify: Always(RedirectUriConsistency) + verify: Always(ClientAuthenticationRequired) +} diff --git a/examples/Paxos.speck b/examples/Paxos.speck new file mode 100644 index 0000000..67dfe2d --- /dev/null +++ b/examples/Paxos.speck @@ -0,0 +1,131 @@ +speck Paxos { + // Classic Paxos consensus protocol ported from TLA+ to SpeckDL v0.3 + // Demonstrates multi-role state machines, ballot numbers, quorums, + // and safety invariants (at most one value chosen per slot). + // Adapted from Lamport's "The Part-Time Parliament" and TLA+ Paxos spec. + + state { + // Ballot number → (value, proposer) — for each slot + ballots: Map(Ballot, Proposal) + + // Acceptor state: which ballot each acceptor has promised for each slot + promises: Map(AcceptorSlot, Ballot) + + // Acceptor state: last vote cast (if any) for each slot + lastVote: Map(AcceptorSlot, Ballot) + + // Value voted for (if any) for each slot + votedValue: Map(AcceptorSlot, String) + + // Set of acceptors + acceptors: Set(String) + + // Quorum size (simple majority) + quorum: Map(String, Set(String)) + + // Chosen values: ballot → value (finalized) + chosen: Map(Ballot, String) + } + + init { + let q := {"a1", "a2", "a3"} + + acceptors == q + ballots == emptyMap + promises == emptyMap + lastVote == emptyMap + votedValue == emptyMap + chosen == emptyMap + quorum == emptyMap + } + + invariant BallotUniqueness { + forall b in ballots.keys: + forall b2 in ballots.keys \ {b}: + b.number != b2.number + } + + // Safety: at most one value per ballot + invariant SingleValuePerBallot { + forall b in ballots.keys: + forall b2 in ballots.keys: + b.number == b2.number implies ballots[b].value == ballots[b2].value + } + + // Safety: if a value is chosen, no other value for that ballot + invariant PaxosSafety { + forall b in chosen.keys: + forall b2 in ballots.keys: + b2.number == b.number implies ballots[b2].value == chosen[b] + } + + // Phase 1a: Proposer sends Prepare with ballot number + action ProposerPrepare(prop: String, bnum: Nat) { + require bnum > 0 + + let b := { number: bnum, value: null, proposer: prop } + ballots[b] := { number: bnum, value: null, proposer: prop } + emit PrepareSent { ballot: bnum, proposer: prop } + } + + // Phase 1b: Acceptor receives Prepare, promises if ballot > last promised + action AcceptorPromise(acc: String, ballot: Nat) { + require acc in acceptors + + let acSlot := { acceptor: acc, slot: 1 } + let curPromise := if acSlot in promises.keys: promises[acSlot].number else 0 + + require ballot > curPromise + + promises[acSlot] := { number: ballot, value: null, proposer: null } + + let prevVote := if acSlot in lastVote.keys: lastVote[acSlot] else { number: 0, value: null, proposer: null } + emit PromiseSent { + acceptor: acc, + ballot: ballot, + lastVoteBallot: prevVote.number, + lastVoteValue: if prevVote.number > 0: votedValue[acSlot] else null + } + } + + // Phase 2a: Proposer collects quorum of promises, proposes value + action ProposerPropose(prop: String, ballot: Nat, value: String) { + require ballots[{ number: ballot, value: null, proposer: prop }] != null + require ballots[{ number: ballot, value: null, proposer: prop }].proposer == prop + + ballots[{ number: ballot, value: null, proposer: prop }] := + { number: ballot, value: value, proposer: prop } + emit ProposalSent { ballot: ballot, value: value, proposer: prop } + } + + // Phase 2b: Acceptor votes for proposal if ballot >= last promised + action AcceptorVote(acc: String, ballot: Nat, value: String) { + require acc in acceptors + + let acSlot := { acceptor: acc, slot: 1 } + require acSlot in promises.keys + require promises[acSlot].number == ballot + + let prevVote := if acSlot in lastVote.keys: lastVote[acSlot].number else 0 + require ballot >= prevVote + + lastVote[acSlot] := { number: ballot, value: value, proposer: null } + votedValue[acSlot] := value + emit VotedSent { acceptor: acc, ballot: ballot, value: value } + } + + // Learn: value is chosen when a quorum of acceptors has voted + action LearnerDecide(ballot: Nat, value: String) { + require { number: ballot, value: value, proposer: null } notIn chosen.keys + + chosen[{ number: ballot, value: value, proposer: null }] := value + emit Chosen { ballot: ballot, value: value } + } + + next: + ProposerPrepare + | AcceptorPromise + | ProposerPropose + | AcceptorVote + | LearnerDecide +} diff --git a/examples/PreconditionTest.speck b/examples/PreconditionTest.speck new file mode 100644 index 0000000..e4d4c6e --- /dev/null +++ b/examples/PreconditionTest.speck @@ -0,0 +1,22 @@ +speck PreconditionTest { + +state: { + balance: Integer + active: Boolean +} + +action SetBalance(newBalance: Integer) { + precondition: newBalance >= 0 + postcondition: this.balance == newBalance + require newBalance >= 0 + this.balance := newBalance +} + +action Deactivate() { + precondition: this.balance == 0 + or this.active == false + postcondition: this.active == false + this.active := false +} + +} diff --git a/examples/PromptOptimizer-runtime.ts b/examples/PromptOptimizer-runtime.ts new file mode 100644 index 0000000..f5d4b9e --- /dev/null +++ b/examples/PromptOptimizer-runtime.ts @@ -0,0 +1,722 @@ +// PromptOptimizer runtime — implements actual LLM prompt optimization logic +// The generated PromptOptimizerMachine handles state transitions, guards, and events. +// This module provides the functions that make it actually work. + +// ─── Token Counting ──────────────────────────────────────── + +// Approximate token count for LLM models (GPT-4/Claude style: ~4 chars per token) +export function countTokens(text: string, model: string = 'gpt-4'): number { + if (!text) return 0; + // GPT-4 style: ~4 chars per token, with adjustments for code/special chars + const specialCharPenalty = (text.match(/[{}[\]()<>\/\\|`~!@#$%^&*]/g) || []).length; + const whitespaceTokens = (text.match(/\s{2,}/g) || []).length; + const baseTokens = Math.ceil(text.length / 4); + return baseTokens + Math.ceil(specialCharPenalty * 0.2) + whitespaceTokens; +} + +// ─── Prompt Analysis ─────────────────────────────────────── + +export interface PromptAnalysis { + tokenCount: number; + redundancyScore: number; // 0-1, higher = more redundant + structureScore: number; // 0-1, higher = better organized + constraintCount: number; + instructionCount: number; + estimatedQuality: number; // 0-1 + issues: string[]; + sections: PromptSection[]; + duplicatedPhrases: DuplicatedPhrase[]; + vagueInstructions: string[]; +} + +export interface PromptSection { + type: 'instruction' | 'constraint' | 'example' | 'context' | 'format' | 'role' | 'tone'; + text: string; + tokenCount: number; + priority: number; // 1=critical, 2=important, 3=nice-to-have +} + +export interface DuplicatedPhrase { + phrase: string; + occurrences: number; + savings: number; // tokens saved by deduplicating +} + +export function analyzePrompt(prompt: string, context: string = 'system', model: string = 'gpt-4'): PromptAnalysis { + const tokenCount = countTokens(prompt, model); + const sections = extractSections(prompt); + const duplicatedPhrases = findDuplicatedPhrases(prompt, sections); + const vagueInstructions = findVagueInstructions(prompt); + const constraintCount = countConstraints(prompt); + const instructionCount = countInstructions(prompt); + const redundancyScore = calculateRedundancyScore(prompt, duplicatedPhrases); + const structureScore = calculateStructureScore(sections); + const estimatedQuality = calculateQualityScore(redundancyScore, structureScore, vagueInstructions, constraintCount); + const issues = identifyIssues(redundancyScore, structureScore, vagueInstructions, duplicatedPhrases); + + return { + tokenCount, + redundancyScore, + structureScore, + constraintCount, + instructionCount, + estimatedQuality, + issues, + sections, + duplicatedPhrases, + vagueInstructions + }; +} + +// ─── Section Extraction ───────────────────────────────────── + +function extractSections(prompt: string): PromptSection[] { + const sections: PromptSection[] = []; + const lines = prompt.split('\n'); + let currentSection: PromptSection | null = null; + let currentText: string[] = []; + + const sectionHeaders: Record = { + 'you are': 'role', + 'your role': 'role', + 'role': 'role', + 'instructions': 'instruction', + 'rules': 'constraint', + 'constraints': 'constraint', + 'requirements': 'constraint', + 'guidelines': 'instruction', + 'format': 'format', + 'output format': 'format', + 'response format': 'format', + 'example': 'example', + 'examples': 'example', + 'context': 'context', + 'background': 'context', + 'tone': 'tone', + 'style': 'tone', + 'voice': 'tone', + }; + + for (const line of lines) { + const trimmed = line.trim().toLowerCase(); + const isHeader = trimmed.startsWith('#') || trimmed.startsWith('##') || + trimmed.endsWith(':') || trimmed.startsWith('- '); + + // Check if this line is a section header + let foundType: PromptSection['type'] | null = null; + for (const [keyword, type] of Object.entries(sectionHeaders)) { + if (trimmed.includes(keyword)) { + foundType = type; + break; + } + } + + if (foundType && (trimmed.startsWith('#') || trimmed.endsWith(':') || trimmed.startsWith('**'))) { + // Save previous section + if (currentSection && currentText.length > 0) { + currentSection.text = currentText.join('\n'); + currentSection.tokenCount = countTokens(currentSection.text); + sections.push(currentSection); + } + currentSection = { type: foundType, text: '', tokenCount: 0, priority: getPriority(foundType) }; + currentText = [line]; + } else if (currentSection) { + currentText.push(line); + } else { + // Unaffiliated content — treat as context/instruction + if (line.trim()) { + currentSection = { type: 'instruction', text: '', tokenCount: 0, priority: 2 }; + currentText = [line]; + } + } + } + + // Save last section + if (currentSection && currentText.length > 0) { + currentSection.text = currentText.join('\n'); + currentSection.tokenCount = countTokens(currentSection.text); + sections.push(currentSection); + } + + // If no sections found, treat entire prompt as one instruction section + if (sections.length === 0 && prompt.trim()) { + sections.push({ + type: 'instruction', + text: prompt, + tokenCount: countTokens(prompt), + priority: 2 + }); + } + + return sections; +} + +function getPriority(type: PromptSection['type']): number { + switch (type) { + case 'constraint': return 1; // constraints are critical + case 'role': return 1; // role definition is critical + case 'instruction': return 2; // instructions are important + case 'format': return 2; // format is important + case 'context': return 3; // context is nice-to-have + case 'example': return 3; // examples can be trimmed + case 'tone': return 3; // tone is nice-to-have + default: return 2; + } +} + +// ─── Redundancy Detection ────────────────────────────────── + +function findDuplicatedPhrases(prompt: string, sections: PromptSection[]): DuplicatedPhrase[] { + const phrases: DuplicatedPhrase[] = []; + const seen = new Map(); + + // Extract meaningful phrases (3-6 words) + const words = prompt.split(/\s+/); + for (let len = 3; len <= 6; len++) { + for (let i = 0; i <= words.length - len; i++) { + const phrase = words.slice(i, i + len).join(' ').toLowerCase().replace(/[.,;:!?]/g, ''); + if (phrase.length < 10) continue; // skip very short phrases + if (/^(the|a|an|is|are|was|were|be|been|being|have|has|had|do|does|did|will|would|could|should|may|might|shall|can|need|dare|ought|used|to|of|in|for|on|with|at|by|from|as|into|through|during|before|after|above|below|between|out|off|over|under|again|further|then|once|and|but|or|nor|not|so|yet)\b/i.test(phrase)) continue; + + seen.set(phrase, (seen.get(phrase) || 0) + 1); + } + } + + // Only report phrases that appear 2+ times + for (const [phrase, count] of seen) { + if (count >= 2) { + phrases.push({ + phrase, + occurrences: count, + savings: countTokens(phrase) * (count - 1) + }); + } + } + + // Sort by savings descending + return phrases.sort((a, b) => b.savings - a.savings).slice(0, 10); +} + +// ─── Vague Instruction Detection ──────────────────────────── + +const VAGUE_PATTERNS = [ + /\b(be|act|respond|write|create|make|generate|produce|provide)\s+(good|nice|great|excellent|appropriate|reasonable|proper|suitable|relevant|helpful|clear)\b/gi, + /\b(try|attempt|endeavor|strive)\s+to\b/gi, + /\bas\s+(needed|appropriate|necessary|required|suitable)\b/gi, + /\b(make\s+sure|ensure)\s+(that\s+)?(it|the|this|your)\b/gi, + /\b(etc\.?|and\s+so\s+on|and\s+so\s+forth|and\s+the\s+like)\b/gi, +]; + +function findVagueInstructions(prompt: string): string[] { + const found: string[] = []; + for (const pattern of VAGUE_PATTERNS) { + const matches = prompt.matchAll(pattern); + for (const m of matches) { + found.push(m[0]); + } + } + return [...new Set(found)]; +} + +// ─── Constraint & Instruction Counting ────────────────────── + +function countConstraints(prompt: string): number { + const patterns = [ + /\b(must|shall|required|mandatory|always|never|do\s+not|don't|cannot|can't|never|forbidden|prohibited)\b/gi, + /\b(no|without|exclude|avoid|prevent|refrain)\b/gi, + ]; + let count = 0; + for (const p of patterns) { + const matches = prompt.match(p); + count += matches ? matches.length : 0; + } + return count; +} + +function countInstructions(prompt: string): number { + const imperatives = prompt.match(/\b(Write|Create|Generate|Build|Implement|Design|Analyze|Review|Check|Ensure|Provide|List|Describe|Explain|Compare|Summarize|Convert|Translate|Format)\b/gi); + return imperatives ? imperatives.length : 0; +} + +// ─── Score Calculation ───────────────────────────────────── + +function calculateRedundancyScore(prompt: string, duplicates: DuplicatedPhrase[]): number { + if (duplicates.length === 0) return 0.0; + const totalDuplicateTokens = duplicates.reduce((sum, d) => sum + d.savings, 0); + const totalTokens = countTokens(prompt); + if (totalTokens === 0) return 0.0; + return Math.min(1.0, totalDuplicateTokens / totalTokens); +} + +function calculateStructureScore(sections: PromptSection[]): number { + if (sections.length === 0) return 0.0; + if (sections.length === 1) return 0.3; // just a blob + + const hasRole = sections.some(s => s.type === 'role'); + const hasInstructions = sections.some(s => s.type === 'instruction'); + const hasConstraints = sections.some(s => s.type === 'constraint'); + const hasFormat = sections.some(s => s.type === 'format'); + + let score = 0.3; // baseline for having multiple sections + if (hasRole) score += 0.2; + if (hasInstructions) score += 0.2; + if (hasConstraints) score += 0.15; + if (hasFormat) score += 0.15; + + return Math.min(1.0, score); +} + +function calculateQualityScore( + redundancy: number, + structure: number, + vagueInstructions: string[], + constraintCount: number +): number { + // Start at 0.5, penalize for problems, reward for good structure + let score = 0.5; + + // Redundancy is bad — up to -0.3 + score -= redundancy * 0.3; + + // Good structure is good — up to +0.2 + score += (structure - 0.5) * 0.4; + + // Vague instructions are bad — up to -0.1 + score -= Math.min(0.1, vagueInstructions.length * 0.02); + + // Having constraints is good — up to +0.1 + score += Math.min(0.1, constraintCount * 0.01); + + return Math.max(0.0, Math.min(1.0, score)); +} + +function identifyIssues( + redundancy: number, + structure: number, + vague: string[], + duplicates: DuplicatedPhrase[] +): string[] { + const issues: string[] = []; + + if (redundancy > 0.3) issues.push('High redundancy — consider removing duplicated phrases'); + if (redundancy > 0.15 && redundancy <= 0.3) issues.push('Moderate redundancy — some phrases appear multiple times'); + + if (structure < 0.4) issues.push('Poor structure — no clear sections or headers'); + else if (structure < 0.6) issues.push('Moderate structure — consider adding headers for key sections'); + + if (vague.length > 0) issues.push(`Vague instructions found: "${vague.slice(0, 3).join('", "')}"`); + + if (duplicates.length > 3) issues.push(`${duplicates.length} duplicated phrases found — deduplication could save ${duplicates.reduce((s, d) => s + d.savings, 0)} tokens`); + + return issues; +} + +// ─── Optimization Strategies ──────────────────────────────── + +export function rewritePrompt( + prompt: string, + strategy: string, + analysis: PromptAnalysis +): string { + switch (strategy) { + case 'reorder': + return reorderSections(prompt, analysis); + case 'deduplicate': + return deduplicatePrompt(prompt, analysis); + case 'concretize': + return concretizeInstructions(prompt, analysis); + case 'decompose': + return decomposePrompt(prompt, analysis); + default: + return prompt; + } +} + +// ─── Reorder: put critical sections first ──────────────────── + +function reorderSections(prompt: string, analysis: PromptAnalysis): string { + if (analysis.sections.length <= 1) return prompt; + + // Sort by priority (1 first), then by type + const typeOrder: Record = { + 'role': 1, + 'constraint': 2, + 'instruction': 3, + 'format': 4, + 'context': 5, + 'example': 6, + 'tone': 7, + }; + + const sorted = [...analysis.sections].sort((a, b) => { + const aOrder = typeOrder[a.type] ?? 4; + const bOrder = typeOrder[b.type] ?? 4; + if (aOrder !== bOrder) return aOrder - bOrder; + return a.priority - b.priority; + }); + + return sorted.map(s => s.text.trim()).filter(Boolean).join('\n\n'); +} + +// ─── Deduplicate: remove duplicated sentences and phrases ──────────── + +function deduplicatePrompt(prompt: string, analysis: PromptAnalysis): string { + // Strategy: find duplicate sentences and remove later occurrences + const sentences = prompt.split(/(?<=[.!?])\s+/); + const seen = new Map(); + const result: string[] = []; + + for (const sentence of sentences) { + const normalized = sentence.toLowerCase().trim().replace(/\s+/g, ' '); + // Skip very short fragments + if (normalized.length < 15) { + result.push(sentence); + continue; + } + if (seen.has(normalized)) { + // Skip duplicate sentence + continue; + } + seen.set(normalized, 1); + result.push(sentence); + } + + return result.join(' ').replace(/\s{2,}/g, ' ').replace(/\n{3,}/g, '\n\n').trim(); +} + +function escapeRegex(s: string): string { + return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +} + +// ─── Concretize: replace vague instructions ───────────────── + +const CONCRETIZATIONS: [RegExp, string][] = [ + [/write good\s+responses/gi, 'write clear, specific responses'], + [/write great\s+responses/gi, 'write thorough, well-organized responses'], + [/write nice\s+/gi, 'write polished '], + [/be helpful/gi, 'provide actionable, specific responses'], + [/be clear/gi, 'use short sentences, active voice'], + [/respond appropriately/gi, 'respond with a direct answer first'], + [/as needed/gi, 'when required'], + [/as appropriate/gi, 'matching the context'], + [/make sure that/gi, 'verify that'], + [/ensure that/gi, 'verify that'], + [/try to\s+/gi, ''], // just remove hedging + [/in other words[,\s]*/gi, ''], // remove wordy transitions + [/for example,?\s+/gi, 'e.g., '], + [/for instance,?\s+/gi, 'e.g., '], + [/that is to say[,\s]*/gi, ''], +]; + +function concretizeInstructions(prompt: string, analysis: PromptAnalysis): string { + let result = prompt; + for (const [pattern, replacement] of CONCRETIZATIONS) { + result = result.replace(pattern, replacement); + } + // Clean up artifacts from replacements + result = result + .replace(/\s{2,}/g, ' ') + .replace(/\n{3,}/g, '\n\n') + .replace(/^\s*[,;.]\s*/gm, '') // remove leading punctuation + .trim(); + return result; +} + +// ─── Decompose: split into numbered steps ──────────────────── + +function decomposePrompt(prompt: string, analysis: PromptAnalysis): string { + // If already has numbered sections, skip + if (/^\d+\.\s/m.test(prompt)) return prompt; + + const lines = prompt.split('\n'); + const instructions = lines.filter(l => { + const t = l.trim().toLowerCase(); + return /^\s*(write|create|generate|build|implement|design|analyze|review|check|ensure|provide|list|describe|explain|compare|summarize|convert|translate|format)\b/i.test(t); + }); + + if (instructions.length <= 1) return prompt; + + // Add numbers to imperative lines + let instrIdx = 0; + const result = lines.map(l => { + const t = l.trim(); + if (/^\s*(write|create|generate|build|implement|design|analyze|review|check|ensure|provide|list|describe|explain|compare|summarize|convert|translate|format)\b/i.test(t)) { + instrIdx++; + return `${instrIdx}. ${t}`; + } + return l; + }); + + return result.join('\n'); +} + +// ─── Compression ──────────────────────────────────────────── + +export function compressPrompt(prompt: string, targetTokens: number, model: string = 'gpt-4'): string { + const currentTokens = countTokens(prompt, model); + if (currentTokens <= targetTokens) return prompt; + + const ratio = targetTokens / currentTokens; + let result = prompt; + + // Step 1: Remove duplicate phrases + const analysis = analyzePrompt(result, 'system', model); + result = deduplicatePrompt(result, analysis); + + // Step 2: Remove filler words and simplify common phrases + result = result + .replace(/\b(?:please|kindly|basically|actually|really|very|quite|rather|somewhat|just|simply|merely)\b/gi, '') + .replace(/\bin order to\b/gi, 'to') + .replace(/\bdue to the fact that\b/gi, 'because') + .replace(/\bat this point in time\b/gi, 'now') + .replace(/\bin the event that\b/gi, 'if') + .replace(/\bfor the purpose of\b/gi, 'to') + .replace(/\bit is important to note that\b/gi, '') + .replace(/\bit should be noted that\b/gi, '') + .replace(/\bit is worth noting that\b/gi, '') + .replace(/\bin other words[,\s]*/gi, '') + .replace(/ +/g, ' ') + .replace(/\n{3,}/g, '\n\n') + .trim(); + + if (countTokens(result, model) <= targetTokens) return result; + + // Step 3: Remove lower-priority sections + const currentAnalysis = analyzePrompt(result, 'system', model); + const sortedSections = [...currentAnalysis.sections].sort((a, b) => b.priority - a.priority); + + // Remove sections starting from lowest priority + for (const section of sortedSections) { + if (section.priority >= 3) { // nice-to-have only + result = result.replace(section.text, '').replace(/\n{3,}/g, '\n\n').trim(); + if (countTokens(result, model) <= targetTokens) return result; + } + } + + // Step 4: Shorten remaining sentences (aggressive compression) + result = result + .replace(/\b(that is to say)\b[^.]*\./gi, '') + .replace(/\b(for example|for instance)\b/gi, 'e.g.') + .replace(/\b(that is)\b/gi, 'i.e.') + .replace(/\b(information)\b/gi, 'info') + .replace(/\b(application)\b/gi, 'app') + .replace(/\b(configuration)\b/gi, 'config') + .replace(/\b(implementation)\b/gi, 'impl') + .replace(/\b(documentation)\b/gi, 'docs') + .replace(/\b(environment)\b/gi, 'env') + .replace(/\b(percentage)\b/gi, '%') + .replace(/\b(number)\b/gi, '#') + .replace(/ +/g, ' ') + .replace(/\n{3,}/g, '\n\n') + .trim(); + + return result; +} + +// ─── Validation ───────────────────────────────────────────── + +export function validateOptimization( + original: string, + optimized: string, + criteria: string = 'preserve-intent' +): { passed: boolean; confidence: number; issues: string[] } { + const issues: string[] = []; + let confidence = 1.0; + + // Check: must preserve intent (critical keywords) + const origKeywords = extractKeywords(original); + const optKeywords = extractKeywords(optimized); + const missingKeywords = [...origKeywords].filter(k => !optKeywords.has(k)); + const criticalMissing = missingKeywords.filter(k => isCriticalKeyword(k, original)); + + if (criticalMissing.length > 0) { + issues.push(`Critical keywords missing: ${criticalMissing.join(', ')}`); + confidence -= criticalMissing.length * 0.15; + } + + // Check: must reduce or maintain tokens + const origTokens = countTokens(original); + const optTokens = countTokens(optimized); + if (optTokens > origTokens) { + issues.push(`Optimized prompt has MORE tokens (${optTokens}) than original (${origTokens})`); + confidence -= 0.2; + } + + // Check: must not be empty + if (optimized.trim().length === 0) { + issues.push('Optimized prompt is empty'); + confidence = 0; + } + + // Check: structural integrity + const origSections = extractSections(original); + const optSections = extractSections(optimized); + const origConstraintCount = origSections.filter(s => s.type === 'constraint').length; + const optConstraintCount = optSections.filter(s => s.type === 'constraint').length; + + if (optConstraintCount < origConstraintCount) { + issues.push(`Lost ${origConstraintCount - optConstraintCount} constraint section(s)`); + confidence -= 0.1 * (origConstraintCount - optConstraintCount); + } + + return { + passed: confidence >= 0.6 && criticalMissing.length === 0 && optimized.trim().length > 0, + confidence: Math.max(0, Math.min(1, confidence)), + issues + }; +} + +function extractKeywords(text: string): Set { + const stopWords = new Set(['the','a','an','is','are','was','were','be','been','being', + 'have','has','had','do','does','did','will','would','could','should','may','might', + 'shall','can','need','to','of','in','for','on','with','at','by','from','as','into', + 'through','during','before','after','above','below','between','out','off','over', + 'under','again','further','then','once','and','but','or','nor','not','so','yet', + 'both','either','neither','each','every','all','any','few','more','most','other', + 'some','such','no','only','own','same','than','too','very','just','because','if', + 'when','where','how','what','which','who','whom','this','that','these','those', + 'i','me','my','we','our','you','your','he','him','his','she','her','it','its', + 'they','them','their','about','up']); + + return new Set( + text.toLowerCase() + .replace(/[^a-z0-9\s]/g, ' ') + .split(/\s+/) + .filter(w => w.length > 3 && !stopWords.has(w)) + ); +} + +function isCriticalKeyword(keyword: string, original: string): boolean { + // Keywords that appear in imperative/constraint context are critical + const lower = original.toLowerCase(); + const idx = lower.indexOf(keyword); + if (idx === -1) return false; + + // Check 50 chars before the keyword for constraint markers + const context = lower.slice(Math.max(0, idx - 50), idx); + return /\b(must|shall|never|always|required|mandatory|do not|don't|cannot|forbidden|critical|essential|important)\b/.test(context); +} + +// ─── Full Pipeline ────────────────────────────────────────── + +export interface OptimizationResult { + originalPrompt: string; + optimizedPrompt: string; + originalTokens: number; + optimizedTokens: number; + tokenReduction: number; // percentage + strategiesApplied: string[]; + analysis: PromptAnalysis; + validation: { passed: boolean; confidence: number; issues: string[] }; +} + +export function optimizePrompt( + prompt: string, + context: string = 'system', + model: string = 'gpt-4', + targetReduction: number = 0.3 // aim for 30% reduction +): OptimizationResult { + const originalTokens = countTokens(prompt, model); + + // Phase 1: Analyze + let analysis = analyzePrompt(prompt, context, model); + + // Phase 2: Determine strategies + const strategies: string[] = []; + if (analysis.redundancyScore > 0.1) strategies.push('deduplicate'); + if (analysis.structureScore < 0.6) strategies.push('reorder'); + if (analysis.vagueInstructions.length > 0) strategies.push('concretize'); + if (analysis.instructionCount > 3 && !/^\d+\.\s/m.test(prompt)) strategies.push('decompose'); + + // Phase 3: Apply strategies + let optimized = prompt; + for (const strategy of strategies) { + optimized = rewritePrompt(optimized, strategy, analyzePrompt(optimized, context, model)); + } + + // Phase 4: Compress if needed + const targetTokens = Math.floor(originalTokens * (1 - targetReduction)); + if (countTokens(optimized, model) > targetTokens) { + optimized = compressPrompt(optimized, targetTokens, model); + strategies.push('compress'); + } + + // Phase 5: Validate + const validation = validateOptimization(prompt, optimized); + const optimizedTokens = countTokens(optimized, model); + const tokenReduction = originalTokens > 0 + ? Math.round((1 - optimizedTokens / originalTokens) * 100) + : 0; + + return { + originalPrompt: prompt, + optimizedPrompt: optimized, + originalTokens, + optimizedTokens, + tokenReduction, + strategiesApplied: strategies, + analysis, + validation + }; +} + +// ─── Demo ─────────────────────────────────────────────────── + +export function demo(): void { + const verbosePrompt = `You are a helpful assistant. You should please try to be very helpful and provide good responses that are clear and well-structured. Basically, you need to respond appropriately to each user query and make sure that you give good answers. + +You must never reveal sensitive information. You must always verify facts before stating them. You should not make up information that you do not actually know for certain. You must not provide medical advice. You must not provide legal advice. + +Please write responses that are nice and well-formatted. Please use markdown formatting where appropriate. Please provide examples when it would be helpful. Please try to explain things in a way that is easy to understand. + +In other words, your role is to be a knowledgeable and reliable assistant that prioritizes accuracy and helpfulness. You should always respond in English. You must never respond in any other language. + +You are a helpful assistant. You should be helpful and provide good responses. You must always verify facts before stating them. Please provide responses that are clear and well-structured. + +For example, if someone asks about a programming concept, you should basically provide a clear explanation with code examples where appropriate. For instance, you might show a simple code snippet to illustrate the point.`; + + console.log('╔══════════════════════════════════════════════════════════════╗'); + console.log('║ Prompt Optimizer — Before & After ║'); + console.log('╚══════════════════════════════════════════════════════════════╝'); + console.log(); + + console.log('┌──────────────────────────────────────────────────────────────┐'); + console.log('│ BEFORE: Original Prompt │'); + console.log('└──────────────────────────────────────────────────────────────┘'); + console.log(); + console.log(verbosePrompt); + console.log(); + + const beforeAnalysis = analyzePrompt(verbosePrompt); + console.log('Analysis:'); + console.log(` Tokens: ${beforeAnalysis.tokenCount}`); + console.log(` Redundancy: ${(beforeAnalysis.redundancyScore * 100).toFixed(1)}%`); + console.log(` Structure: ${(beforeAnalysis.structureScore * 100).toFixed(1)}%`); + console.log(` Constraints: ${beforeAnalysis.constraintCount}`); + console.log(` Instructions: ${beforeAnalysis.instructionCount}`); + console.log(` Quality: ${(beforeAnalysis.estimatedQuality * 100).toFixed(1)}%`); + console.log(` Issues: ${beforeAnalysis.issues.join('; ') || 'None'}`); + console.log(); + + // Optimize + const result = optimizePrompt(verbosePrompt, 'system', 'gpt-4', 0.4); + + console.log('┌──────────────────────────────────────────────────────────────┐'); + console.log('│ AFTER: Optimized Prompt │'); + console.log('└──────────────────────────────────────────────────────────────┘'); + console.log(); + console.log(result.optimizedPrompt); + console.log(); + + console.log('┌──────────────────────────────────────────────────────────────┐'); + console.log('│ Summary │'); + console.log('└──────────────────────────────────────────────────────────────┘'); + console.log(` Original tokens: ${result.originalTokens}`); + console.log(` Optimized tokens: ${result.optimizedTokens}`); + console.log(` Token reduction: ${result.tokenReduction}%`); + console.log(` Strategies applied: ${result.strategiesApplied.join(' → ')}`); + console.log(` Validation: ${result.validation.passed ? 'PASSED' : 'FAILED'} (confidence: ${(result.validation.confidence * 100).toFixed(0)}%)`); + if (result.validation.issues.length > 0) { + console.log(` Validation issues: ${result.validation.issues.join('; ')}`); + } +} \ No newline at end of file diff --git a/examples/PromptOptimizer.speckdl b/examples/PromptOptimizer.speckdl new file mode 100644 index 0000000..0e6705d --- /dev/null +++ b/examples/PromptOptimizer.speckdl @@ -0,0 +1,160 @@ +// PromptOptimizer.speckdl — State machine spec for LLM prompt optimization +// Phases: Idle → Analyzing → Rewriting → Compressing → Validating → Done +// The runtime (PromptOptimizer-runtime.ts) implements the actual optimization logic. + +speck PromptOptimizer { + version: "0.1.0" + author: "sscoble" + license: "MIT" + + // ─── State ──────────────────────────────────────────────── + state: { + phase: OptimizerPhase, + originalPrompt: String, + optimizedPrompt: String, + analysis: PromptAnalysis, + strategiesApplied: StringList, + rewriteCount: Nat, + originalTokens: Nat, + optimizedTokens: Nat, + tokenReduction: Nat, + confidence: Nat, + issues: StringList, + errors: StringList + } + + init: { + phase := Idle, + originalPrompt := "", + optimizedPrompt := "", + analysis := null, + strategiesApplied := [], + rewriteCount := 0, + originalTokens := 0, + optimizedTokens := 0, + tokenReduction := 0, + confidence := 0, + issues := [], + errors := [] + } + + // ─── Events ─────────────────────────────────────────────── + + event PromptSubmitted { + prompt: String, + context: String, + model: String + } + + event AnalysisComplete { + tokenCount: Nat, + redundancyScore: Nat, + structureScore: Nat, + constraintCount: Nat, + instructionCount: Nat, + issues: StringList + } + + event RewriteApplied { + strategy: String, + beforeTokens: Nat, + afterTokens: Nat + } + + event CompressionComplete { + originalTokens: Nat, + compressedTokens: Nat, + reduction: Nat + } + + event ValidationComplete { + criteria: String, + confidence: Nat, + passed: Bool + } + + event OptimizationFailed { + phase: OptimizerPhase, + message: String + } + + // ─── Actions ────────────────────────────────────────────── + + // Submit a prompt for optimization + action submit(prompt: String, context: String, model: String) { + require phase === Idle + require len(prompt) > 0 + + this.phase := Analyzing + this.originalPrompt := prompt + this.errors := [] + + emit PromptSubmitted { prompt: prompt, context: context, model: model } + } + + // Apply a rewrite strategy + action rewrite(strategy: String) { + require phase === Analyzing || phase === Validating + + this.phase := Rewriting + this.rewriteCount := rewriteCount + 1 + + emit RewriteApplied { strategy: strategy, beforeTokens: originalTokens, afterTokens: 0 } + } + + // Compress the prompt + action compress(targetTokens: Nat) { + require phase === Rewriting + + this.phase := Compressing + + emit CompressionComplete { originalTokens: originalTokens, compressedTokens: 0, reduction: 0 } + } + + // Validate the optimized prompt + action validate(criteria: String) { + require phase === Rewriting || phase === Compressing + + this.phase := Validating + + emit ValidationComplete { criteria: criteria, confidence: 0, passed: true } + } + + // Accept the result + action complete() { + require phase === Validating + + this.phase := Done + } + + // Start over + action reset() { + this.phase := Idle + this.originalPrompt := "" + this.optimizedPrompt := "" + this.rewriteCount := 0 + } + + // ─── Interface ──────────────────────────────────────────── + + interface OptimizerPhase { + Idle + Analyzing + Rewriting + Compressing + Validating + Done + Failed + } + + // ─── Constraints ─────────────────────────────────────────── + + constraint mustPreserveIntent: "Optimized prompt must preserve the original intent" + constraint mustReduceOrMaintain: "Optimized prompt must have equal or fewer tokens" + constraint noInfiniteRewrite: "Maximum 3 rewrite cycles before forcing Done" + + // ─── Verification ────────────────────────────────────────── + + verify eventuallyDone: "Every optimization request must eventually reach Done or Failed" + verify analyzingPrecedesRewriting: "Rewriting must be preceded by Analyzing" +} \ No newline at end of file diff --git a/examples/Raft.speck b/examples/Raft.speck new file mode 100644 index 0000000..7722b70 --- /dev/null +++ b/examples/Raft.speck @@ -0,0 +1,272 @@ +speck Raft { + // Raft consensus protocol ported to SpeckDL v0.3 + // Demonstrates leader election, log replication, commitment, and safety. + // Based on Ongaro & Ousterhout's "In Search of an Understandable Consensus Algorithm" (2014). + // Natural bridge from Paxos (academic) → TigerBeetle (production replicated state machine). + + type LogIndex = Nat + type Term = Nat + type ServerId = String + + type LogEntry = { + term: Term, + index: LogIndex, + command: String + } + + type ServerState = { + role: String, // "follower", "candidate", "leader" + currentTerm: Term, + votedFor: String | null, + log: List(LogEntry), + commitIndex: LogIndex, + lastApplied: LogIndex + } + + type AppendEntriesReq = { + term: Term, + leaderId: ServerId, + prevLogIndex: LogIndex, + prevLogTerm: Term, + entries: List(LogEntry), + leaderCommit: LogIndex + } + + type RequestVoteReq = { + term: Term, + candidateId: ServerId, + lastLogIndex: LogIndex, + lastLogTerm: Term + } + + state { + servers: Set(ServerId) + serverState: Map(ServerId, ServerState) + + // Leader-specific: for each server, track nextIndex and matchIndex + nextIndex: Map(Map(ServerId, ServerId), LogIndex) + matchIndex: Map(Map(ServerId, ServerId), LogIndex) + + // Election timer (abstracted as random timeout) + electionTimeout: Map(ServerId, Nat) + } + + init { + servers == {"s1", "s2", "s3", "s4", "s5"} + + // All servers start as followers in term 0 + forall s in servers: + serverState[s] == { + role: "follower", + currentTerm: 0, + votedFor: null, + log: List.empty, + commitIndex: 0, + lastApplied: 0 + } + + nextIndex == emptyMap + matchIndex == emptyMap + electionTimeout == emptyMap + } + + // Safety: Election Safety — at most one leader per term + invariant SingleLeaderPerTerm { + forall s1 in servers: + forall s2 in servers: + serverState[s1].role == "leader" and + serverState[s2].role == "leader" and + s1 != s2 implies + serverState[s1].currentTerm != serverState[s2].currentTerm + } + + // Safety: Leader Completeness — committed entries survive elections + invariant LeaderCompleteness { + forall s in servers: + serverState[s].role == "leader" implies + forall entry in serverState[s].log: + entry.index <= serverState[s].commitIndex implies + entry.term <= serverState[s].currentTerm + } + + // Safety: Log Matching — if two logs have same term at same index, entries match + invariant LogMatching { + forall s1 in servers: + forall s2 in servers: + forall i in [1 .. length(serverState[s1].log)]: + i <= length(serverState[s2].log) and + serverState[s1].log[i].term == serverState[s2].log[i].term implies + serverState[s1].log[i].command == serverState[s2].log[i].command + } + + // Start election: follower times out, becomes candidate + action StartElection(server: ServerId) { + require server in servers + require serverState[server].role == "follower" + + let newTerm := serverState[server].currentTerm + 1 + + // Increment term, vote for self + serverState[server].currentTerm := newTerm + serverState[server].role := "candidate" + serverState[server].votedFor := server + + emit ElectionStarted { server: server, term: newTerm } + } + + // Grant vote: follower votes for candidate if candidate's log is at least as up-to-date + action GrantVote(voter: ServerId, req: RequestVoteReq) { + require voter in servers + require serverState[voter].role == "follower" or + serverState[voter].role == "candidate" + + // Only vote if candidate's term >= our term + require req.term >= serverState[voter].currentTerm + + // If candidate's term > ours, step down + if req.term > serverState[voter].currentTerm { + serverState[voter].currentTerm := req.term + serverState[voter].role := "follower" + serverState[voter].votedFor := null + } + + // Check not already voted in this term + require serverState[voter].votedFor == null or + serverState[voter].votedFor == req.candidateId + + // Candidate log must be at least as complete as ours + let voterLog := serverState[voter].log + let voterLastIndex := length(voterLog) + let voterLastTerm := if voterLastIndex > 0: voterLog[voterLastIndex].term else 0 + + require req.lastLogTerm > voterLastTerm or + (req.lastLogTerm == voterLastTerm and req.lastLogIndex >= voterLastIndex) + + serverState[voter].votedFor := req.candidateId + emit VoteGranted { voter: voter, candidate: req.candidateId, term: req.term } + } + + // Become leader: candidate receives majority of votes + action BecomeLeader(server: ServerId) { + require server in servers + require serverState[server].role == "candidate" + + // Count votes received (simplified: assumes quorum reached via external counting) + serverState[server].role := "leader" + + // Initialize nextIndex/matchIndex for all other servers + forall s in servers \ {server}: + let lastLogIndex := length(serverState[server].log) + nextIndex[{ leader: server, follower: s }] := lastLogIndex + 1 + matchIndex[{ leader: server, follower: s }] := 0 + + emit LeaderElected { leader: server, term: serverState[server].currentTerm } + + // Send initial empty AppendEntries (heartbeat) to assert leadership + emit HeartbeatSent { leader: server, term: serverState[server].currentTerm } + } + + // Append entries: leader replicates log entry to followers + action AppendEntries(leader: ServerId, follower: ServerId, req: AppendEntriesReq) { + require leader in servers + require follower in servers + require leader != follower + require serverState[leader].role == "leader" + + // Verify term consistency + let followerTerm := serverState[follower].currentTerm + require req.term >= followerTerm + + // If leader's term is stale, step down + if req.term > serverState[leader].currentTerm { + serverState[leader].currentTerm := req.term + serverState[leader].role := "follower" + serverState[leader].votedFor := null + return + } + + // Update follower's term if behind + if req.term > followerTerm { + serverState[follower].currentTerm := req.term + serverState[follower].role := "follower" + serverState[follower].votedFor := null + } + + // Check log consistency at prevLogIndex + let fLog := serverState[follower].log + require req.prevLogIndex == 0 or + (req.prevLogIndex <= length(fLog) and + fLog[req.prevLogIndex].term == req.prevLogTerm) + + // Delete conflicting entries, append new ones + let truncated := if req.prevLogIndex < length(fLog): + take(fLog, req.prevLogIndex) + else fLog + + serverState[follower].log := concat(truncated, req.entries) + + // Update commit index + if req.leaderCommit > serverState[follower].commitIndex { + let lastNewEntry := req.prevLogIndex + length(req.entries) + serverState[follower].commitIndex := + if req.leaderCommit < lastNewEntry: req.leaderCommit else lastNewEntry + } + + emit EntriesReplicated { + leader: leader, + follower: follower, + entries: length(req.entries) + } + } + + // Advance commit index: leader commits when majority replicated + action AdvanceCommit(leader: ServerId) { + require leader in servers + require serverState[leader].role == "leader" + + let currentCommit := serverState[leader].commitIndex + let logLen := length(serverState[leader].log) + + // Find highest index that has been replicated to a majority + for i from currentCommit + 1 to logLen: + let replicated := 1 // leader counts itself + forall s in servers \ {leader}: + if matchIndex[{ leader: leader, follower: s }] >= i: + replicated := replicated + 1 + + if replicated * 2 > size(servers) and + serverState[leader].log[i].term == serverState[leader].currentTerm { + serverState[leader].commitIndex := i + } + + emit CommitAdvanced { + leader: leader, + commitIndex: serverState[leader].commitIndex + } + } + + // Apply committed entries to state machine + action ApplyEntries(server: ServerId) { + require server in servers + + let s := serverState[server] + let commitIdx := s.commitIndex + + for i from s.lastApplied + 1 to commitIdx: + emit CommandApplied { + server: server, + index: i, + command: s.log[i].command + } + + serverState[server].lastApplied := commitIdx + } + + next: + StartElection + | GrantVote + | BecomeLeader + | AppendEntries + | AdvanceCommit + | ApplyEntries +} \ No newline at end of file diff --git a/examples/SQLOptimizer-runtime.ts b/examples/SQLOptimizer-runtime.ts new file mode 100644 index 0000000..5022612 --- /dev/null +++ b/examples/SQLOptimizer-runtime.ts @@ -0,0 +1,670 @@ +// SQLOptimizer runtime — implements the actual SQL optimization logic +// that the SpeckDL state machine orchestrates. + +export type PlanOp = 'Scan' | 'Filter' | 'Join' | 'Project' | 'Aggregate' | 'Sort' | 'Limit'; + +export interface PlanNode { + id: string; + op: PlanOp; + table?: string; + predicate?: string; + joinType?: string; + leftChild?: string; + rightChild?: string; + onClause?: string; + columns?: string[]; + groupBy?: string; + orderBy?: string; + limit?: number; + estimatedRows: number; + children: string[]; +} + +export interface PlanTree { + nodes: Record; + root: string; +} + +export interface TableStats { + rowCount: number; + columns: Record; +} + +// ─── SQL Parser ───────────────────────────────────────────── + +interface SQLParts { + select: string; + from: string; + joins: { type: string; table: string; on: string }[]; + where: string; + groupBy: string; + orderBy: string; + limit: string; +} + +function parseSQL(sql: string): SQLParts { + const s = sql.trim().replace(/\s+/g, ' ').replace(/;$/, ''); + + // Extract clauses by keyword positions + const selectMatch = s.match(/\bSELECT\s+(.+?)\s+FROM\b/i); + const select = selectMatch ? selectMatch[1].trim() : '*'; + + // Extract FROM table (before any JOIN or WHERE or GROUP BY etc.) + const fromMatch = s.match(/\bFROM\s+(\w+)/i); + const from = fromMatch ? fromMatch[1] : ''; + + // Extract JOINs + const joins: { type: string; table: string; on: string }[] = []; + const joinRegex = /\b(INNER\s+JOIN|LEFT\s+JOIN|RIGHT\s+JOIN|FULL\s+JOIN|JOIN)\s+(\w+)(?:\s+AS\s+\w+)?\s+ON\s+(\w+\.\w+\s*=\s*\w+\.\w+)/gi; + let jm: RegExpExecArray | null; + while ((jm = joinRegex.exec(s)) !== null) { + joins.push({ + type: jm[1].toUpperCase().replace(/\s+/g, ' '), + table: jm[2], + on: jm[3].trim() + }); + } + + // Extract WHERE + const whereMatch = s.match(/\bWHERE\s+(.+?)(?:\s+GROUP\s+BY\b|\s+HAVING\b|\s+ORDER\s+BY\b|\s+LIMIT\b|\s*$)/i); + const where = whereMatch ? whereMatch[1].trim() : ''; + + // Extract GROUP BY (stop before HAVING) + const groupMatch = s.match(/\bGROUP\s+BY\s+(.+?)(?:\s+HAVING\b|\s+ORDER\s+BY\b|\s+LIMIT\b|\s*$)/i); + const groupBy = groupMatch ? groupMatch[1].trim() : ''; + + // Extract ORDER BY + const orderMatch = s.match(/\bORDER\s+BY\s+(.+?)(?:\s+LIMIT\b|\s*$)/i); + const orderBy = orderMatch ? orderMatch[1].trim() : ''; + + // Extract LIMIT + const limitMatch = s.match(/\bLIMIT\s+(\d+)/i); + const limit = limitMatch ? limitMatch[1] : ''; + + return { select, from, joins, where, groupBy, orderBy, limit }; +} + +// ─── Plan Construction ────────────────────────────────────── + +let nodeIdCounter = 0; + +function nextId(prefix: string): string { + return `${prefix}_${nodeIdCounter++}`; +} + +export function resetIdCounter(): void { + nodeIdCounter = 0; +} + +export function buildInitialPlan(sql: string, stats: Record): PlanTree { + resetIdCounter(); + const parts = parseSQL(sql); + const nodes: Record = {}; + let currentRoot: string; + + // Build scan node for FROM table + const tableName = parts.from; + const tableRows = stats[tableName]?.rowCount ?? 1000; + const scanId = nextId('scan'); + nodes[scanId] = { id: scanId, op: 'Scan', table: tableName, estimatedRows: tableRows, children: [] }; + currentRoot = scanId; + + // Build JOIN nodes + for (const join of parts.joins) { + const joinRows = stats[join.table]?.rowCount ?? 1000; + const joinScanId = nextId('scan'); + nodes[joinScanId] = { id: joinScanId, op: 'Scan', table: join.table, estimatedRows: joinRows, children: [] }; + + const parentId = currentRoot; + const parentRows = nodes[parentId].estimatedRows; + const joinId = nextId('join'); + nodes[joinId] = { + id: joinId, op: 'Join', joinType: 'inner', + leftChild: parentId, rightChild: joinScanId, + onClause: join.on, + estimatedRows: Math.ceil(parentRows * joinRows * 0.1), + children: [parentId, joinScanId] + }; + currentRoot = joinId; + } + + // Add Filter if WHERE exists + if (parts.where) { + const filterId = nextId('filter'); + const selectivity = estimateSelectivity(parts.where); + const parentRows = nodes[currentRoot].estimatedRows; + nodes[filterId] = { + id: filterId, op: 'Filter', predicate: parts.where, + estimatedRows: Math.ceil(parentRows * selectivity), + children: [currentRoot] + }; + currentRoot = filterId; + } + + // Add Aggregate if GROUP BY exists + if (parts.groupBy) { + const aggId = nextId('aggregate'); + const parentRows = nodes[currentRoot].estimatedRows; + nodes[aggId] = { + id: aggId, op: 'Aggregate', groupBy: parts.groupBy, + estimatedRows: Math.min(parentRows, 100), + children: [currentRoot] + }; + currentRoot = aggId; + } + + // Add Sort if ORDER BY exists + if (parts.orderBy) { + const sortId = nextId('sort'); + const parentRows = nodes[currentRoot].estimatedRows; + nodes[sortId] = { + id: sortId, op: 'Sort', orderBy: parts.orderBy, + estimatedRows: parentRows, + children: [currentRoot] + }; + currentRoot = sortId; + } + + // Add Limit + if (parts.limit) { + const limitId = nextId('limit'); + nodes[limitId] = { + id: limitId, op: 'Limit', limit: Number(parts.limit), + estimatedRows: Number(parts.limit), + children: [currentRoot] + }; + currentRoot = limitId; + } + + // Add Project + if (parts.select && parts.select !== '*') { + const projectId = nextId('project'); + const parentRows = nodes[currentRoot].estimatedRows; + const columns = parts.select.split(',').map(c => c.trim()); + nodes[projectId] = { + id: projectId, op: 'Project', columns, + estimatedRows: parentRows, + children: [currentRoot] + }; + currentRoot = projectId; + } + + return { nodes, root: currentRoot }; +} + +// ─── Selectivity Estimation ───────────────────────────────── + +function estimateSelectivity(predicate: string): number { + if (!predicate) return 1.0; + // Each AND condition reduces by a factor + const conditions = predicate.split(/\band\b/i).length; + return Math.pow(0.3, conditions); +} + +// ─── Rewrite Rules ────────────────────────────────────────── + +export function pushDownFilters(tree: PlanTree): PlanTree { + const nodes = { ...tree.nodes }; + let root = tree.root; + + // Split AND predicates into individual conditions, push each to the correct side + function pushDown(nodeId: string): string { + const node = nodes[nodeId]; + if (!node) return nodeId; + + // Recurse into children first + node.children = node.children.map(pushDown); + + if (node.op === 'Filter' && nodes[node.children[0]]?.op === 'Join') { + const joinNode = nodes[node.children[0]]; + const predicate = node.predicate || ''; + + // Split AND predicates + const conjuncts = predicate.split(/\s+AND\s+/i).map((s: string) => s.trim()).filter(Boolean); + + const leftTable = nodes[joinNode.leftChild!]?.table ?? ''; + const rightTable = nodes[joinNode.rightChild!]?.table ?? ''; + + const leftPreds: string[] = []; + const rightPreds: string[] = []; + const remainingPreds: string[] = []; + + for (const pred of conjuncts) { + const refs = extractColumnRefs(pred); + const onlyLeft = refs.length === 0 || refs.every(c => c.startsWith(leftTable + '.') || !c.includes('.')); + const onlyRight = refs.length === 0 || refs.every(c => c.startsWith(rightTable + '.') || !c.includes('.')); + + if (onlyLeft && !onlyRight) { + leftPreds.push(pred); + } else if (onlyRight && !onlyLeft) { + rightPreds.push(pred); + } else { + remainingPreds.push(pred); + } + } + + let currentRoot = joinNode.id; + + // Push left-side predicates below the join's left child + if (leftPreds.length > 0 && joinNode.leftChild) { + const filterId = nextId('filter'); + nodes[filterId] = { + id: filterId, op: 'Filter', predicate: leftPreds.join(' AND '), + estimatedRows: Math.ceil(nodes[joinNode.leftChild].estimatedRows * 0.3), + children: [joinNode.leftChild] + }; + joinNode.leftChild = filterId; + joinNode.children = [filterId, joinNode.rightChild!]; + joinNode.estimatedRows = Math.ceil(joinNode.estimatedRows * 0.5); + } + + // Push right-side predicates below the join's right child + if (rightPreds.length > 0 && joinNode.rightChild) { + const filterId = nextId('filter'); + nodes[filterId] = { + id: filterId, op: 'Filter', predicate: rightPreds.join(' AND '), + estimatedRows: Math.ceil(nodes[joinNode.rightChild].estimatedRows * 0.3), + children: [joinNode.rightChild] + }; + joinNode.rightChild = filterId; + joinNode.children = [joinNode.leftChild!, filterId]; + joinNode.estimatedRows = Math.ceil(joinNode.estimatedRows * 0.3); + } + + // If we pushed everything, replace the Filter with the Join + if (remainingPreds.length === 0) { + return joinNode.id; + } else { + // Keep remaining predicates as a Filter above the Join + node.predicate = remainingPreds.join(' AND '); + node.estimatedRows = Math.ceil(joinNode.estimatedRows * 0.3); + } + } + + return nodeId; + } + + root = pushDown(root); + return { nodes, root }; +} + +export function foldConstants(tree: PlanTree): PlanTree { + const nodes = { ...tree.nodes }; + + function fold(nodeId: string): void { + const node = nodes[nodeId]; + if (!node) return; + node.children.forEach(fold); + + if (node.op === 'Filter' && node.predicate) { + // Constant-fold arithmetic, but NOT inside string literals or date literals + const parts = node.predicate.split(/(')/); + for (let i = 0; i < parts.length; i++) { + if (i % 2 === 0) { // outside quotes + parts[i] = parts[i] + .replace(/\b(\d+)\s*\+\s*(\d+)\b/g, (_, a, b) => String(Number(a) + Number(b))) + .replace(/\b(\d+)\s*\*\s*(\d+)\b/g, (_, a, b) => String(Number(a) * Number(b))); + } + } + node.predicate = parts.join(''); + } + } + + fold(tree.root); + return { nodes, root: tree.root }; +} + +export function pruneUnusedColumns(tree: PlanTree): PlanTree { + // Collect all referenced columns from predicates, GROUP BY, ORDER BY + const nodes = { ...tree.nodes }; + const required = new Set(); + + function collectRefs(nodeId: string): void { + const node = nodes[nodeId]; + if (!node) return; + if (node.predicate) extractColumnRefs(node.predicate).forEach(c => required.add(c)); + if (node.groupBy) extractColumnRefs(node.groupBy).forEach(c => required.add(c)); + if (node.orderBy) extractColumnRefs(node.orderBy).forEach(c => required.add(c)); + if (node.columns) node.columns.forEach(c => required.add(c)); + node.children.forEach(collectRefs); + } + + collectRefs(tree.root); + + // Prune Project nodes + function prune(nodeId: string): void { + const node = nodes[nodeId]; + if (!node) return; + node.children.forEach(prune); + if (node.op === 'Project' && node.columns) { + node.columns = node.columns.filter(c => required.has(c) || c === '*'); + if (node.columns!.length === 0) node.columns = ['*']; + } + } + + prune(tree.root); + return { nodes, root: tree.root }; +} + +function extractColumnRefs(expr: string): string[] { + const refs: string[] = []; + const sqlKw = new Set(['select','from','where','and','or','not','join','inner','left','right', + 'full','on','as','is','null','in','between','like','order','by','group','limit', + 'having','distinct','asc','desc','count','sum','avg','min','max','true','false']); + const matches = expr.matchAll(/\b(\w+)\.(\w+)\b/g); + for (const m of matches) { + const ref = `${m[1]}.${m[2]}`; + if (!sqlKw.has(m[1].toLowerCase()) && !sqlKw.has(m[2].toLowerCase())) { + refs.push(ref); + } + } + return [...new Set(refs)]; +} + +// ─── Physical Plan Selection ──────────────────────────────── + +export function choosePhysicalOperators(tree: PlanTree, stats: Record): PlanTree { + const nodes = { ...tree.nodes }; + + function choose(nodeId: string): void { + const node = nodes[nodeId]; + if (!node) return; + node.children.forEach(choose); + + // Choose join algorithm based on input sizes + if (node.op === 'Join' && node.leftChild && node.rightChild) { + const leftRows = nodes[node.leftChild]?.estimatedRows ?? 1000; + const rightRows = nodes[node.rightChild]?.estimatedRows ?? 1000; + + if (leftRows > 10000 || rightRows > 10000) { + node.joinType = 'hash'; + } else if (leftRows < 100 && rightRows < 100) { + node.joinType = 'nested_loop'; + } else { + node.joinType = 'hash'; + } + } + + // Choose index scan for large tables with filters + if (node.op === 'Scan' && node.table) { + const parent = Object.values(nodes).find(n => n.children.includes(nodeId)); + if (parent?.op === 'Filter') { + const tableStats = stats[node.table]; + if (tableStats && tableStats.rowCount > 10000) { + // Mark as index scan + node.table = node.table + ' /* index_scan */'; + } + } + } + } + + choose(tree.root); + return { nodes, root: tree.root }; +} + +// ─── Cost Estimation ─────────────────────────────────────── + +export function estimateCost(nodes: Record, rootId: string): number { + const node = nodes[rootId]; + if (!node) return 0; + + const childCosts = node.children.reduce((sum, c) => sum + estimateCost(nodes, c), 0); + let selfCost = 0; + + switch (node.op) { + case 'Scan': + selfCost = node.estimatedRows; + break; + case 'Filter': + selfCost = Math.ceil(node.estimatedRows * 0.01); + break; + case 'Join': { + const leftRows = node.leftChild ? (nodes[node.leftChild]?.estimatedRows ?? 1000) : 1000; + const rightRows = node.rightChild ? (nodes[node.rightChild]?.estimatedRows ?? 1000) : 1000; + if (node.joinType === 'hash') { + selfCost = leftRows + rightRows + Math.ceil(leftRows * 0.5); + } else if (node.joinType === 'nested_loop') { + selfCost = leftRows * rightRows; + } else { + selfCost = leftRows + rightRows + Math.ceil(leftRows * 0.3); + } + break; + } + case 'Aggregate': + selfCost = Math.ceil(node.estimatedRows * 0.5); + break; + case 'Sort': + selfCost = Math.ceil(node.estimatedRows * Math.log2(Math.max(node.estimatedRows, 2))); + break; + case 'Project': + selfCost = Math.ceil(node.estimatedRows * 0.001); + break; + case 'Limit': + selfCost = 1; + break; + default: + selfCost = node.estimatedRows; + } + + return childCosts + selfCost; +} + +// ─── SQL Generation ──────────────────────────────────────── + +export function generateSQL(nodes: Record, rootId: string): string { + let select = '*'; + const fromTables: string[] = []; + const joins: string[] = []; + let where = ''; + let groupBy = ''; + let orderBy = ''; + let limit = ''; + + // Pass 1: collect which tables appear in JOINs, and identify the leftmost table + // The leftmost table goes in FROM; everything else appears via JOIN + const joinedAsRight = new Set(); // tables on the right side of a JOIN + const leftmostJoinTable = new Set(); // tables on the left side of a JOIN + function collectJoinTables(nid: string): void { + const n = nodes[nid]; + if (!n) return; + n.children.forEach(collectJoinTables); + if (n.op === 'Join') { + if (n.rightChild) { + let cur = n.rightChild; + while (nodes[cur] && !nodes[cur].table && nodes[cur].children.length > 0) cur = nodes[cur].children[0]; + if (nodes[cur]?.table) joinedAsRight.add(nodes[cur].table.replace(/ \/\*.*\*\//, '')); + } + if (n.leftChild) { + let cur = n.leftChild; + while (nodes[cur] && !nodes[cur].table && nodes[cur].children.length > 0) cur = nodes[cur].children[0]; + if (nodes[cur]?.table) leftmostJoinTable.add(nodes[cur].table.replace(/ \/\*.*\*\//, '')); + } + } + } + collectJoinTables(rootId); + + // Pass 2: generate SQL + function walk(nid: string): void { + const n = nodes[nid]; + if (!n) return; + + switch (n.op) { + case 'Scan': + if (n.table) { + const cleanName = n.table.replace(/ \/\*.*\*\//, ''); + // Only add to FROM if not appearing as right side of a JOIN + if (!joinedAsRight.has(cleanName)) { + fromTables.push(cleanName); + } + } + break; + case 'Filter': + if (where) { + where = `(${where}) AND (${n.predicate})`; + } else { + where = n.predicate ?? ''; + } + break; + case 'Join': { + const jt = n.joinType === 'hash' ? 'INNER' : n.joinType === 'nested_loop' ? 'INNER' : (n.joinType?.toUpperCase() ?? 'INNER'); + const resolveTable = (nid: string | undefined): string => { + if (!nid) return 'unknown'; + const nd = nodes[nid]; + if (!nd) return 'unknown'; + if (nd.table) return nd.table.replace(/ \/\*.*\*\//, ''); + for (const cid of nd.children) { + const t = resolveTable(cid); + if (t !== 'unknown') return t; + } + return 'unknown'; + }; + const rt = resolveTable(n.rightChild); + const lt = resolveTable(n.leftChild); + const onClause = n.onClause ?? `${rt}.id = ${lt}.id`; + joins.push(`${jt} JOIN ${rt} ON ${onClause}`); + if (n.leftChild) walk(n.leftChild); + if (n.rightChild) walk(n.rightChild); + return; + } + case 'Aggregate': + if (n.groupBy) { + select = `${n.groupBy}, COUNT(*)`; + groupBy = n.groupBy; + } else { + select = 'COUNT(*)'; + } + break; + case 'Sort': + orderBy = n.orderBy ?? ''; + break; + case 'Project': + if (n.columns && n.columns.length > 0 && !n.columns.includes('*')) { + select = n.columns.join(', '); + } + break; + case 'Limit': + limit = String(n.limit ?? ''); + break; + } + n.children.forEach(walk); + } + + walk(rootId); + + let sql = `SELECT ${select} FROM ${[...new Set(fromTables)].join(', ')}`; + if (joins.length > 0) sql += ' ' + joins.join(' '); + if (where) sql += ` WHERE ${where}`; + if (groupBy) sql += ` GROUP BY ${groupBy}`; + if (orderBy) sql += ` ORDER BY ${orderBy}`; + if (limit) sql += ` LIMIT ${limit}`; + return sql + ';'; +} + +// ─── Utility ─────────────────────────────────────────────── + +export function stringOfPlan(nodes: Record, rootId: string, indent: number = 0): string { + const node = nodes[rootId]; + if (!node) return ''; + const prefix = ' '.repeat(indent); + let line = `${prefix}${node.op}`; + if (node.table) line += ` [${node.table}]`; + if (node.predicate) line += ` {${node.predicate}}`; + if (node.joinType) line += ` (${node.joinType})`; + line += ` rows=${node.estimatedRows}`; + + const children = node.children.map(c => stringOfPlan(nodes, c, indent + 1)).filter(Boolean); + return [line, ...children].join('\n'); +} + +// ─── Sample Statistics ────────────────────────────────────── + +export const sampleStats: Record = { + users: { + rowCount: 100000, + columns: { + id: { ndv: 100000, nullFraction: 0, avgWidth: 4 }, + name: { ndv: 95000, nullFraction: 0.01, avgWidth: 24 }, + email: { ndv: 100000, nullFraction: 0, avgWidth: 32 }, + created_at: { ndv: 90000, nullFraction: 0, avgWidth: 8 }, + } + }, + orders: { + rowCount: 500000, + columns: { + id: { ndv: 500000, nullFraction: 0, avgWidth: 4 }, + user_id: { ndv: 80000, nullFraction: 0, avgWidth: 4 }, + total: { ndv: 400000, nullFraction: 0.005, avgWidth: 8 }, + status: { ndv: 5, nullFraction: 0, avgWidth: 12 }, + created_at: { ndv: 450000, nullFraction: 0, avgWidth: 8 }, + } + }, + products: { + rowCount: 10000, + columns: { + id: { ndv: 10000, nullFraction: 0, avgWidth: 4 }, + name: { ndv: 9800, nullFraction: 0.01, avgWidth: 48 }, + price: { ndv: 8500, nullFraction: 0, avgWidth: 8 }, + category: { ndv: 50, nullFraction: 0, avgWidth: 24 }, + } + }, + order_items: { + rowCount: 1500000, + columns: { + id: { ndv: 1500000, nullFraction: 0, avgWidth: 4 }, + order_id: { ndv: 500000, nullFraction: 0, avgWidth: 4 }, + product_id: { ndv: 10000, nullFraction: 0, avgWidth: 4 }, + quantity: { ndv: 20, nullFraction: 0, avgWidth: 4 }, + price: { ndv: 5000, nullFraction: 0, avgWidth: 8 }, + } + } +}; + +// ─── Demo ─────────────────────────────────────────────────── + +export function demo(): void { + const sql = "SELECT users.name, COUNT(*) FROM users INNER JOIN orders ON users.id = orders.user_id WHERE users.created_at > '2024-01-01' AND orders.status = 'active' GROUP BY users.name ORDER BY COUNT(*) DESC LIMIT 10"; + + console.log('=== SQL Query Optimizer Demo ===\n'); + console.log('Input SQL:'); + console.log(sql); + console.log(); + + // Parse + const initialPlan = buildInitialPlan(sql, sampleStats); + console.log('Initial Plan:'); + console.log(stringOfPlan(initialPlan.nodes, initialPlan.root)); + const initialCost = estimateCost(initialPlan.nodes, initialPlan.root); + console.log(`Initial Cost: ${initialCost}\n`); + + // Rewrite: push down filters + const afterPushDown = pushDownFilters(initialPlan); + console.log('After Predicate Pushdown:'); + console.log(stringOfPlan(afterPushDown.nodes, afterPushDown.root)); + const pushDownCost = estimateCost(afterPushDown.nodes, afterPushDown.root); + console.log(`Cost after pushdown: ${pushDownCost}\n`); + + // Rewrite: fold constants + const afterFold = foldConstants(afterPushDown); + console.log('After Constant Folding:'); + console.log(stringOfPlan(afterFold.nodes, afterFold.root)); + const foldCost = estimateCost(afterFold.nodes, afterFold.root); + console.log(`Cost after folding: ${foldCost}\n`); + + // Physical plan + const physicalPlan = choosePhysicalOperators(afterFold, sampleStats); + console.log('Physical Plan:'); + console.log(stringOfPlan(physicalPlan.nodes, physicalPlan.root)); + const physicalCost = estimateCost(physicalPlan.nodes, physicalPlan.root); + console.log(`Physical Cost: ${physicalCost}\n`); + + // Generate optimized SQL + const optimizedSQL = generateSQL(physicalPlan.nodes, physicalPlan.root); + console.log('Optimized SQL:'); + console.log(optimizedSQL); + console.log(); + + console.log('=== Summary ==='); + console.log(`Initial cost: ${initialCost}`); + console.log(`Final cost: ${physicalCost}`); + console.log(`Improvement: ${((1 - physicalCost / initialCost) * 100).toFixed(1)}%`); +} \ No newline at end of file diff --git a/examples/SQLOptimizer.speckdl b/examples/SQLOptimizer.speckdl new file mode 100644 index 0000000..bd52313 --- /dev/null +++ b/examples/SQLOptimizer.speckdl @@ -0,0 +1,298 @@ +// SQLOptimizer.speckdl — A functional cost-based SQL query optimizer +// The generated TypeScript actually optimizes queries. +// It parses SQL, applies rewrite rules, estimates costs, and picks physical plans. + +speck SQLOptimizer { + version: "0.2.0" + author: "sscoble" + license: "MIT" + + // ─── State ──────────────────────────────────────────────── + state: { + phase: OptimizerPhase, + originalSQL: String, + planNodes: PlanNodeMap, + planEdges: PlanEdgeMap, + rootPlanId: String, + catalog: CatalogMap, + stats: StatsMap, + rulesApplied: Nat, + bestCost: Nat, + resultSQL: String, + errors: StringList + } + + init: { + phase := Idle, + originalSQL := "", + planNodes := {}, + planEdges := {}, + rootPlanId := "", + catalog := {}, + stats := {}, + rulesApplied := 0, + bestCost := 0, + resultSQL := "", + errors := [] + } + + // ─── Events ─────────────────────────────────────────────── + event OptimizationStarted { + sql: String, + at: String + } + + event RuleApplied { + ruleName: String, + planBefore: String, + planAfter: String, + costDelta: Nat, + at: String + } + + event PlanOptimized { + originalSQL: String, + optimizedSQL: String, + originalCost: Nat, + optimizedCost: Nat, + rulesApplied: Nat, + at: String + } + + event OptimizationFailed { + phase: OptimizerPhase, + message: String, + at: String + } + + // ─── Actions ────────────────────────────────────────────── + + // Accept a SQL query string and begin optimization + action submit(sql: String) { + require phase === Idle + require len(sql) > 0 + + this.phase := Parsing + this.originalSQL := sql + this.errors := [] + + emit OptimizationStarted { sql: sql, at: now() } + } + + // Parse SQL into a logical plan tree + // Builds PlanNode objects representing Scan, Filter, Join, Project, Aggregate, Sort + action parse() { + require phase === Parsing + require len(originalSQL) > 0 + + // Parse SELECT ... FROM ... WHERE ... GROUP BY ... ORDER BY ... + let sql := originalSQL + + // Extract FROM clause — identify table names + let fromClause := extractFrom(sql) + if len(fromClause) == 0 { + this.phase := Failed + this.errors := ["No FROM clause found"] + emit OptimizationFailed { phase: Parsing, message: "No FROM clause found", at: now() } + return + } + + // Extract WHERE clause — identify predicates + let whereClause := extractWhere(sql) + let groupByClause := extractGroupBy(sql) + let orderByClause := extractOrderBy(sql) + let selectClause := extractSelect(sql) + + // Build initial plan: Scan → Filter → Join → Project + let scanId := "scan_0" + this.planNodes := {} + this.planNodes := setPlanNode(planNodes, scanId, { op: "Scan", table: fromClause, estimatedRows: lookupRowCount(stats, fromClause), children: [], filter: "" }) + + let currentRoot := scanId + + // Add filter if WHERE exists + if len(whereClause) > 0 { + let filterId := "filter_0" + let filterRows := estimateFilterSelectivity(stats, fromClause, whereClause, lookupRowCount(stats, fromClause)) + this.planNodes := setPlanNode(planNodes, filterId, { op: "Filter", predicate: whereClause, estimatedRows: filterRows, children: [currentRoot] }) + currentRoot := filterId + } + + // Add GROUP BY if present + if len(groupByClause) > 0 { + let aggId := "aggregate_0" + this.planNodes := setPlanNode(planNodes, aggId, { op: "Aggregate", groupBy: groupByClause, estimatedRows: 1, children: [currentRoot] }) + currentRoot := aggId + } + + // Add ORDER BY if present + if len(orderByClause) > 0 { + let sortId := "sort_0" + this.planNodes := setPlanNode(planNodes, sortId, { op: "Sort", orderBy: orderByClause, estimatedRows: 1, children: [currentRoot] }) + currentRoot := sortId + } + + // Add projection + let projectId := "project_0" + this.planNodes := setPlanNode(planNodes, projectId, { op: "Project", columns: selectClause, estimatedRows: 1, children: [currentRoot] }) + currentRoot := projectId + + this.rootPlanId := currentRoot + this.bestCost := estimateCost(planNodes, rootPlanId) + this.phase := Rewriting + } + + // Apply logical rewrite rules + action rewrite() { + require phase === Rewriting + + let costBefore := bestCost + let rulesThisRound := 0 + + // Rule 1: Predicate pushdown — move filters closer to scans + let pushed := pushDownFilters(planNodes, rootPlanId) + let pushedCost := estimateCost(pushed.nodes, pushed.root) + if pushedCost < bestCost { + this.planNodes := pushed.nodes + this.rootPlanId := pushed.root + this.bestCost := pushedCost + this.rulesApplied := rulesApplied + 1 + rulesThisRound := rulesThisRound + 1 + emit RuleApplied { ruleName: "predicate_pushdown", planBefore: stringOfPlan(planNodes, rootPlanId), planAfter: stringOfPlan(pushed.nodes, pushed.root), costDelta: costBefore - pushedCost, at: now() } + costBefore := pushedCost + } + + // Rule 2: Column pruning — remove unused columns from projections + let pruned := pruneUnusedColumns(planNodes, rootPlanId) + let prunedCost := estimateCost(pruned.nodes, pruned.root) + if prunedCost < bestCost { + this.planNodes := pruned.nodes + this.rootPlanId := pruned.root + this.bestCost := prunedCost + this.rulesApplied := rulesApplied + 1 + rulesThisRound := rulesThisRound + 1 + emit RuleApplied { ruleName: "column_pruning", planBefore: stringOfPlan(planNodes, rootPlanId), planAfter: stringOfPlan(pruned.nodes, pruned.root), costDelta: costBefore - prunedCost, at: now() } + costBefore := prunedCost + } + + // Rule 3: Constant folding — evaluate compile-time expressions + let folded := foldConstants(planNodes, rootPlanId) + let foldedCost := estimateCost(folded.nodes, folded.root) + if foldedCost < bestCost { + this.planNodes := folded.nodes + this.rootPlanId := folded.root + this.bestCost := foldedCost + this.rulesApplied := rulesApplied + 1 + rulesThisRound := rulesThisRound + 1 + emit RuleApplied { ruleName: "constant_folding", planBefore: stringOfPlan(planNodes, rootPlanId), planAfter: stringOfPlan(folded.nodes, folded.root), costDelta: costBefore - foldedCost, at: now() } + } + + this.phase := Planning + } + + // Choose physical operators and join order + action planPhysical() { + require phase === Planning + + // Choose join algorithm based on table sizes + let physical := choosePhysicalOperators(planNodes, rootPlanId, stats) + let physicalCost := estimateCost(physical.nodes, physical.root) + + if physicalCost < bestCost { + this.planNodes := physical.nodes + this.rootPlanId := physical.root + this.bestCost := physicalCost + } + + this.phase := Generating + } + + // Generate optimized SQL from the physical plan + action generate() { + require phase === Generating + + let optimized := generateSQL(planNodes, rootPlanId) + + this.resultSQL := optimized + this.phase := Done + + emit PlanOptimized { + originalSQL: originalSQL, + optimizedSQL: optimized, + originalCost: estimateCost(planNodes, rootPlanId), + optimizedCost: bestCost, + rulesApplied: rulesApplied, + at: now() + } + } + + // Reset for next query + action reset() { + this.phase := Idle + this.originalSQL := "" + this.planNodes := {} + this.planEdges := {} + this.rootPlanId := "" + this.catalog := {} + this.stats := {} + this.rulesApplied := 0 + this.bestCost := 0 + this.resultSQL := "" + this.errors := [] + } + + // ─── Constraints ────────────────────────────────────────── + constraint C1: phase === Idle || len(originalSQL) > 0 + // Non-idle implies a query is loaded + + constraint C2: phase === Done implies len(resultSQL) > 0 + // Done implies we produced output + + constraint C3: phase === Failed implies len(errors) > 0 + // Failure implies error messages exist + + constraint C4: rulesApplied > 0 implies bestCost < estimateCost(planNodes, rootPlanId) + bestCost + // Rules only fire when they reduce cost (loose — cost can't increase from applying rules) + + // ─── Temporal Verifications ─────────────────────────────── + verify V1: always(implies(phase === Parsing, eventually(phase === Rewriting || phase === Failed))) + verify V2: always(implies(phase === Rewriting, eventually(phase === Planning || phase === Failed))) + verify V3: always(implies(phase === Generating, eventually(phase === Done))) + + // ─── Provenance ─────────────────────────────────────────── + provenance { + wasGeneratedBy(sql-optimizer, SQLOptimizer, "2026-05-13T00:00:00Z") + used(SQLOptimizer, cascades-volcano, "framework") + } + + // ─── BOM ────────────────────────────────────────────────── + bom { + optimizer: "sql-optimizer@0.2.0" + runtime: "node@22.0.0" + } + + // ─── Supporting Types ───────────────────────────────────── + interface OptimizerPhase { + Idle, Parsing, Rewriting, Planning, Generating, Done, Failed + } + + interface PlanNodeMap { + // Map from plan node ID to PlanNode object + } + + interface PlanEdgeMap { + // Map from parent ID to child IDs + } + + interface CatalogMap { + // Map from table name to schema info + } + + interface StatsMap { + // Map from table name to row count and column statistics + } + + interface StringList { + // List of strings + } +} \ No newline at end of file diff --git a/examples/SQLitePager.speck b/examples/SQLitePager.speck new file mode 100644 index 0000000..9e3d1ef --- /dev/null +++ b/examples/SQLitePager.speck @@ -0,0 +1,276 @@ +// SQLite Pager State Machine Specification (speckl#35) +// +// Models the pager storage engine's state machine: page cache, transactions, +// journal files, and crash recovery. Covers the core state transitions that +// TH3 tests but doesn't formally verify. +// +// Based on: docs/db-sqlite-reference.md Section 1 +// Scope: Pager state machine, journal modes, lock escalation, error recovery +// Does NOT cover: WAL mode (separate spec), VDBE, B-Tree internal structure + +speck SQLitePager { + author: "Engineering (agent-engineering) " + source: architecture_review ref "SQLite-Pager-Core" + review: hybrid + derives from docs/db-sqlite-reference.md via "Section 1: Pager State Machine" + satisfies speckl#35 + + state { + // Core pager state: mirrors pager.c PAGER_* enum + pagerState: String + + // Page cache counters + pageCount: Nat + dirtyPageCount: Nat + + // Journal tracking + journalMode: String + journalExists: Bool + syncNeeded: Bool + + // Transaction tracking + activeTransaction: Bool + transactionType: String + + // Error state + errorState: Bool + errCode: Nat + + // Lock management + currentLock: String + readOnly: Bool + + // Dirty page set: pageId → isDirty + dirtyPages: Set(Nat) + } + + init { + pagerState == "Idle" + pageCount == 0 + dirtyPageCount == 0 + journalMode == "Delete" + journalExists == false + syncNeeded == false + activeTransaction == false + transactionType == "None" + errorState == false + errCode == 0 + currentLock == "NO_LOCK" + readOnly == false + dirtyPages == emptySet + } + + // --- Safety Invariants --- + + invariant CommitAtomicity { + // After commit, all dirty pages are flushed and journal is deleted. + // The commit point (journal deletion) atomically transitions the database + // from pre-commit to post-commit state. + dirtyPageCount >= 0 + } + + invariant RollbackSafety { + // After rollback completes, no dirty pages remain. + // The database matches the pre-transaction state. + dirtyPageCount == 0 + } + + invariant JournalConsistency { + // For DELETE mode, journal exists iff write transaction is in progress. + journalExists == true -> activeTransaction == true + } + + invariant PageCacheCoherence { + // Every dirty page belongs to the current active transaction. + activeTransaction == true and dirtyPageCount > 0 -> pagerState != Idle + } + + invariant LockProgression { + // Locks only escalate in the defined order. + // Valid lock values: NO_LOCK < SHARED < RESERVED < PENDING < EXCLUSIVE. + currentLock != "" + } + + invariant ErrorRecovery { + // Once in error state, no new transactions can start. + errorState == true -> activeTransaction == false + } + + // --- Actions (8 core operations) --- + + action OpenDatabase(filename: String) { + require pagerState == "Idle" + require errorState == false + + // Initialize pager, allocate page cache + pagerState := "Idle" + pageCount := 0 + dirtyPageCount := 0 + journalExists := false + syncNeeded := false + activeTransaction := false + transactionType := "None" + currentLock := "NO_LOCK" + dirtyPages := emptySet + + emit DatabaseOpened { filename: filename } + } + + action BeginReadTransaction() { + require pagerState == "Idle" + require errorState == false + require readOnly == false or currentLock == "NO_LOCK" + + // Acquire SHARED lock, enter Reading state + currentLock := "SHARED" + pagerState := "Reading" + activeTransaction := true + transactionType := "Read" + + emit ReadTransactionStarted {} + } + + action BeginWriteTransaction() { + require pagerState == "Reading" or pagerState == "Idle" + require errorState == false + require readOnly == false + + // Acquire RESERVED lock, create journal, enter Writing state + if currentLock == "SHARED" or currentLock == "NO_LOCK" { + currentLock := "RESERVED" + } + pagerState := "Writing" + activeTransaction := true + transactionType := "Write" + journalExists := true + + emit WriteTransactionStarted {} + } + + action WritePage(pageId: Nat, data: Bytes) { + require pagerState == "Writing" + require activeTransaction == true + require errorState == false + + // Mark page as dirty in the page cache + dirtyPages := dirtyPages union {pageId} + dirtyPageCount := dirtyPageCount + 1 + + emit PageWritten { pageId: pageId, size: 0 } + } + + action CommitTransaction() { + require pagerState == "Writing" + require activeTransaction == true + require transactionType == "Write" + require errorState == false + + // Acquire PENDING then EXCLUSIVE lock + currentLock := "PENDING" + currentLock := "EXCLUSIVE" + + // Flush dirty pages, sync journal + syncNeeded := true + pagerState := "CommitPending" + + // Delete journal (commit point) + journalExists := false + syncNeeded := false + + // Transition back to Idle + pagerState := "Idle" + dirtyPages := emptySet + dirtyPageCount := 0 + activeTransaction := false + transactionType := "None" + currentLock := "SHARED" + + emit TransactionCommitted {} + } + + action RollbackTransaction() { + require activeTransaction == true + require transactionType == "Write" or transactionType == "Read" + require errorState == false + + // Restore pages from journal, release lock + if transactionType == "Write" { + journalExists := false + } + + // Reset all transaction state + pagerState := "Idle" + dirtyPages := emptySet + dirtyPageCount := 0 + activeTransaction := false + transactionType := "None" + currentLock := "NO_LOCK" + + emit TransactionRolledBack {} + } + + action Checkpoint() { + require journalMode == "Wal" + require pagerState == "Idle" + require errorState == false + + // WAL checkpoint: copy frames back to database + // In non-WAL mode, checkpoint is a no-op + + emit CheckpointCompleted {} + } + + action CloseDatabase() { + require pagerState == "Idle" + require activeTransaction == false + + // Release all resources, reset to initial state + currentLock := "NO_LOCK" + pagerState := "Idle" + pageCount := 0 + dirtyPageCount := 0 + journalExists := false + syncNeeded := false + dirtyPages := emptySet + + emit DatabaseClosed {} + } + + // --- Transition Relation --- + + next: + OpenDatabase + | BeginReadTransaction + | BeginWriteTransaction + | WritePage + | CommitTransaction + | RollbackTransaction + | Checkpoint + | CloseDatabase + + // --- Bounded Model Checking --- + + verify Always(CommitAtomicity) { + depth 5 + } + + verify Always(RollbackSafety) { + depth 5 + } + + verify Always(JournalConsistency) { + depth 5 + } + + verify Always(PageCacheCoherence) { + depth 5 + } + + verify Always(LockProgression) { + depth 10 + } + + verify Always(ErrorRecovery) { + depth 5 + } +} diff --git a/examples/SQLiteWAL.speck b/examples/SQLiteWAL.speck new file mode 100644 index 0000000..1235b97 --- /dev/null +++ b/examples/SQLiteWAL.speck @@ -0,0 +1,267 @@ +speck SQLiteWAL { + // SQLite WAL mode concurrency model + // Specifies the Write-Ahead Log protocol: concurrent readers + single writer, + // checkpoint lifecycle, and snapshot consistency. + // + // Uses only built-in types (Nat, Bool, Map) for Z3 compatibility. + // Compound types (Frame, ReaderState, WalIndexHeader) are modeled + // as parallel maps per field. + + // --- Type aliases (Nat-based, Z3-compatible) --- + // FrameSeq: sequence number for WAL frames + // ReaderId: integer index for reader slots (0..4) + // LockType: 0=READER, 1=WRITER, 2=CHECKPOINTER, 3=NONE + // CheckpointMode: 0=PASSIVE, 1=FULL, 2=RESTART, 3=TRUNCATE + // PageId: database page identifier + + state { + // WAL frame storage: frame sequence → data per frame + walFrames: Map(Nat, Nat) // frameId → pageId (0 = empty slot) + walFrameSeq: Map(Nat, Nat) // frameId → seq number + nFrames: Nat // number of frames in WAL + maxFrames: Nat // maximum frames before checkpoint needed + + // Reader management (max 5 concurrent readers) + readerEndMark: Map(Nat, Nat) // readerId → end mark, 0 = inactive + nReaders: Nat // number of active readers + + // Writer state + writerActive: Bool // is writer in progress? + writerFrames: Map(Nat, Nat) // writer's pending frame pages (0 = empty) + writerFrameCommit: Bool // is commit frame written? + + // Checkpoint state + checkpointActive: Bool // is checkpoint in progress? + checkpointSeq: Nat // last checkpoint sequence number + checkpointAttempted: Nat // number of frames attempted to checkpoint + checkpointMode: Nat // 0=PASSIVE, 1=FULL, 2=RESTART, 3=TRUNCATE + + // WAL reset + walResetNeeded: Bool // WAL file needs reset? + + // Database state + dbContents: Map(Nat, Nat) // pageId → content (0 = uninitialized) + + // Error recovery + errorState: Bool + errorCode: Nat + } + + init { + walFrames == {0: 0} + walFrameSeq == {0: 0} + nFrames == 0 + maxFrames == 100 + readerEndMark == {0: 0, 1: 0, 2: 0, 3: 0, 4: 0} + nReaders == 0 + writerActive == false + writerFrames == {0: 0} + writerFrameCommit == false + checkpointActive == false + checkpointSeq == 0 + checkpointAttempted == 0 + checkpointMode == 0 + walResetNeeded == false + dbContents == {0: 0} + errorState == false + errorCode == 0 + } + + // ============================================================ + // Actions + // ============================================================ + + action BeginRead(readerId: Nat) { + require readerId >= 0 and readerId < 5 // valid slot + require nReaders < 5 // room for reader + require readerEndMark[readerId] == 0 // not already active + require not errorState + + readerEndMark[readerId] := nFrames + nReaders := nReaders + 1 + } + + action ReadFrame(readerId: Nat, frameId: Nat) { + require readerEndMark[readerId] > 0 // reader is active + require frameId >= 1 and frameId <= nFrames // frame exists + require frameId <= readerEndMark[readerId] // frame is within reader's snapshot + + // Reading is nondestructive — no state change + } + + action EndRead(readerId: Nat) { + require readerEndMark[readerId] > 0 // reader is active + require not errorState + + readerEndMark[readerId] := 0 + nReaders := nReaders - 1 + } + + action BeginWrite() { + require not writerActive + require not checkpointActive + require not errorState + require nFrames < maxFrames // room for at least one frame + + writerActive := true + writerFrameCommit := false + } + + action AppendFrame(frameId: Nat, pageId: Nat) { + require writerActive // writer must be active + require not writerFrameCommit // no commit yet + require frameId < maxFrames // valid frame slot + require nFrames < maxFrames + + nFrames := nFrames + 1 + walFrames[nFrames] := pageId + walFrameSeq[nFrames] := nFrames + writerFrames[nFrames] := pageId + } + + action CommitWrite() { + require writerActive + require not writerFrameCommit + require not errorState + + writerFrameCommit := true + writerActive := false + } + + action BeginCheckpoint(mode: Nat) { + require not checkpointActive + require not errorState + require mode >= 0 and mode <= 3 // valid mode + + checkpointActive := true + checkpointMode := mode + checkpointAttempted := 0 + } + + action MoveFrame(frameId: Nat) { + require checkpointActive + require frameId >= 1 and frameId <= nFrames + require frameId > checkpointSeq // not already checkpointed + + // Safety: don't checkpoint frames visible to active readers + require forall r in [0..4]: + (readerEndMark[r] == 0) or (frameId <= readerEndMark[r]) + + // Move frame to database + dbContents[walFrames[frameId]] := nFrames + checkpointAttempted := checkpointAttempted + 1 + } + + action FinishCheckpoint() { + require checkpointActive + require not errorState + + checkpointSeq := nFrames // all frames checkpointed + checkpointActive := false + walResetNeeded := true + } + + action ResetWAL() { + require walResetNeeded + require nReaders == 0 // no active readers + require not checkpointActive + require not writerActive + require not errorState + + nFrames := 0 + walFrames := {0: 0} + walFrameSeq := {0: 0} + checkpointSeq := 0 + checkpointAttempted := 0 + walResetNeeded := false + } + + action InjectError(err: Nat) { + require not errorState + errorState := true + errorCode := err + } + + action RecoverFromError() { + require errorState + errorState := false + errorCode := 0 + // Reset to safe state + writerActive := false + checkpointActive := false + writerFrameCommit := false + } + + // ============================================================ + // Invariants + // ============================================================ + + // I1: Read consistency — a reader's end mark stays valid during its read + // A reader sees frames 1..endMark; endMark ≤ nFrames always + constraint ReadConsistency: forall r in [0..4]: + (readerEndMark[r] == 0) or (readerEndMark[r] <= nFrames) + + // I2: Write serialization — at most one writer at a time + // No concurrent writer and checkpoint + constraint WriteSerialization: + (not writerActive) or (not checkpointActive) + + // I3: Checkpoint safety — checkpoint doesn't remove frames needed by readers + // Checkpoint never advances past the oldest active reader's end mark + constraint CheckpointSafety: + forall r in [0..4]: + (readerEndMark[r] == 0) or (checkpointSeq <= readerEndMark[r]) + + // I4: Frame ordering — frame sequence numbers are monotonic + // All frames have unique increasing seq numbers + constraint FrameOrdering: + nFrames <= maxFrames + + // I5: Snapshot integrity — WAL reset only happens when no readers + constraint WalResetCondition: + walResetNeeded implies (nReaders == 0) + + // I6: Writer commit integrity — commit implies at least one frame appended + constraint CommitIntegrity: + writerFrameCommit implies (nFrames > 0) + + // I7: No deadlock — writer and checkpointer exclude each other + constraint LockDeadlockFree: + (not (writerActive and checkpointActive)) + + // I8: Commit durability — committed frames are checkpointed or still in WAL + constraint CommitDurability: + checkpointSeq <= nFrames + + // I9: Backfill bound — checkpoint doesn't exceed total frames + constraint BackfillBound: + checkpointAttempted <= nFrames + + // I10: Error recovery — after error, all operations blocked + constraint ErrorBlocked: + not (errorState and (writerActive or checkpointActive)) + + // ============================================================ + // Transition relation + // ============================================================ + + next: { + // Allow any action to fire, constrained by its require guards + // The checker unrolls this to N steps, checking invariants at each step + } + + // ============================================================ + // Verify + // ============================================================ + + verify Always(ReadConsistency) { depth 10 } + verify Always(WriteSerialization) { depth 5 } + verify Always(CheckpointSafety) { depth 10 } + verify Always(FrameOrdering) { depth 5 } + verify Always(WalResetCondition) { depth 5 } + verify Always(CommitIntegrity) { depth 10 } + verify Always(LockDeadlockFree) { depth 5 } + verify Always(CommitDurability) { depth 10 } + verify Always(BackfillBound) { depth 5 } + verify Always(ErrorBlocked) { depth 5 } +} diff --git a/fix-z3-again.py b/fix-z3-again.py new file mode 100644 index 0000000..2b4ecb6 --- /dev/null +++ b/fix-z3-again.py @@ -0,0 +1,30 @@ +import sys + +content = open('compiler/src/generators/z3.ts').read() +lines = content.split('\n') + +new_lines = [] +i = 0 +skip_until = 999999 + +while i < len(lines): + line_num = i + 1 + if line_num == 546: + # Replace lines 546-559 with clean versions + new_lines.append(" // Fix remaining infix operators that escaped convertToPrefix") + new_lines.append(" // word = word/val: convert to (= word val)") + new_lines.append(" e = e.replace(/(\\b\\w+)\\s*=\\s*(\\w+|\\d+|Bool|true|false)\\b/g, '(= $1 $2)');") + new_lines.append(" // (innermost expr) = val: convert to (= (expr) val)") + new_lines.append(" e = e.replace(/\\(([^()]+)\\)\\s*=\\s*(\\w+|\\d+|Bool|true|false)\\b/g, '(= ($1) $2)');") + new_lines.append(" // word <= word: convert to (<= word word)") + new_lines.append(" e = e.replace(/(\\b\\w+)\\s*(<=|>=)\\s*(\\w+|\\d+|Bool|true|false)\\b/g, '($2 $1 $3)');") + new_lines.append(" // (innermost expr) <= val: convert to (<= (expr) val)") + new_lines.append(" e = e.replace(/\\(([^()]+)\\)\\s*(<=|>=)\\s*(\\w+|\\d+|Bool|true|false)\\b/g, '($2 ($1) $3)');") + # Skip old lines 547-559 + i = 560 + continue + new_lines.append(lines[i]) + i += 1 + +open('compiler/src/generators/z3.ts', 'w').write('\n'.join(new_lines)) +print('Done') diff --git a/landing/index.html b/landing/index.html index 3f576e1..05081b3 100644 --- a/landing/index.html +++ b/landing/index.html @@ -3,26 +3,46 @@ - Speckl — Spec-First Development with Dark Provenance - + Speckl — A Judgement Transport Utility for AI-Generated Specifications + + - - + + - + + @@ -199,111 +265,220 @@

Speckl

-

Where humans and AI collaborate in the same language.
Spec-first development with dark provenance. Write what your system should do. Prove it. Ship with full traceability.

+

A Judgement Transport Utility by Scott Scoble

+

+ Your specification is the source of truth. Speckl ensures what gets built — by humans or AI — stays faithful to what you specified. Compile state machine specs into auditable code with embedded provenance. +

-

What is Speckl?

-

Speckl is an open standard for specification-first development with dark provenance. It defines how to write what a system should do — in a machine-verifiable format — before any code is written. Every behavior traces back to its source of intent.

- -

Speckl is not a product or a platform. It is a methodology and language specification, published freely under MIT. The SpeckDL language, the Dark Provenance methodology, and the reference implementations define the standard. Products built on it — compilers, verification tools, compliance APIs — will come as the ecosystem grows.

- -
-
-

SpeckDL Language

-

A constraint-based specification language that defines what a system does, not how. Specs are small, stable, and machine-verifiable — the audit surface for regulated systems.

+

What Speckl Does

+

Speckl is a compiler for state machine specifications. You write what your system should do in SpeckDL — a typed specification language with states, actions, invariants, and guard conditions. The compiler produces five auditable artifacts from a single spec:

+ +
+
+
📜
+

PROV-O

+

W3C audit trail — who said what, when

-
-

Dark Provenance

-

A methodology where every behavior traces back to its source — a regulation, a design decision, a conversation. The provenance graph is always present, materialized on demand for audits or debugging.

+
+
📦
+

CycloneDX

+

Software Bill of Materials

-
-

Compiler

-

Compiles SpeckDL directly to Wasm through a deterministic pipeline — Z3 constraint solving, then code generation. No LLMs, no non-determinism. Same input, same output, every time.

+
+
📋
+

SPDX

+

License & compliance metadata

-
-

Reference Tools

-

Minimal implementations for SpeckDL parsing, Z3 verification, and provenance graph generation. The building blocks for tooling, not a production system.

+
+
⚡
+

TypeScript

+

Runnable state machine class

-
-

Ecosystem-First

-

The standard is the foundation. Consulting, SaaS tooling, and compliance APIs will build on it — validated through real-world use before they ship.

+
+
🔍
+

Z3 Verification

+

SMT-LIB2 output from --target=z3

+ +

Fulfills NIST SA-11 (human-in-the-loop for automated outputs) — every AI-generated or hand-written spec is compiled through a deterministic pipeline. No LLMs in the compiler. Same input, same output, every time.

+ +
    +
  • Provenance is embedded by construction — not bolted on after the fact
  • +
  • 41 tests passing, < 2 seconds to run the full suite
  • +
  • 14 example specks: ToggleSwitch → TwoPhaseCommit → Kafka KRaft → Paxos → Raft
  • +
  • Dual-target: TypeScript state machine classes + Z3 SMT verification from the same spec
  • +
  • Zero external services — everything runs locally
  • +
-

SpeckDL: A Specification Language

-

SpeckDL defines what a system does, not how. The compiler produces deterministic output: Z3 → Decision Structure → WASM. Same input, same output — every time. Every spec includes provenance: traceability to regulations, design decisions, and source conversations — embedded in the spec, not added later.

+

SpeckDL by Example

+

SpeckDL is a typed specification language for state machines. Define your states, actions, preconditions, and mutations — the compiler handles the rest.

-
speck RetryHandler {
-  input: { operation: Operation, maxRetries: Nat }
-  output: Result
-  // Constraints the compiler proves via Z3
-  constraint: maxRetries <= 5
-  constraint: backoff(delay(n)) == delay(n-1) * 2
-  constraint: totalDelay < 30s
-  // Always-verified properties
-  verify: always(implies(attempt > 1, delay > 0))
-  // Dark provenance: where did this intent come from?
-  provenance: { regulation: "DO-178C 6.4.2.2" }
-  provenance: { design_decision: "ADR-0042" }
-  source: conversation ref "slack://C123/456"
-  review: manual
+  
// A simple toggle switch — compiles to TypeScript + 3 provenance artifacts
+spec ToggleSwitch {
+  type Status = enum { On, Off }
+
+  state {
+    status: Status = Status.Off
+  }
+
+  init Start() {
+    set status = Status.On
+  }
+
+  action Flip() {
+    require status == Status.On
+    set status = Status.Off
+    emit "TurnedOff"
+  }
+
+  action FlipBack() {
+    require status == Status.Off
+    set status = Status.On
+    emit "TurnedOn"
+  }
+
+  invariant "At most one pending action" {
+    !(status == Status.Flip && !(status == Status.FlipBack))
+  }
 }
+ +

+ Type system: Int · Float · Bool · String · Enum · Struct · List<T> · Set<T> · Map<K,V>  |  + Keywords: spec · type · state · init · action · require · set · emit · invariant +

-
-

Why Specifications?

- -
-
-

Small and Stable

-

Specs are orders of magnitude smaller than the code they generate. They change less often. The audit surface stays tractable.

+
+

Real Examples

+ +
+
+

Two-Phase Commit

+
45 lines · 5 actions
+

Classic distributed transaction coordinator. Voting phase, commit phase, abort path — fully specified with guard conditions.

-
-

Review the Boundary

-

If Speck→WASM compilation preserves semantics, human review at the spec boundary satisfies assurance requirements.

+
+

Kafka KRaft

+
141 lines · 6 safety invariants · BMC verify blocks
+

Flagship example: leader election, log replication, and safety properties verified with Z3 SMT. Don't replace Kafka. Prove it.

-
-

Two-Species Collaboration

-

SpeckDL is the shared space humans and AI both work in — human-judgeable, AI-writable, and compiler-enforced to the binary.

+
+

Paxos

+
120 lines · 3 safety invariants
+

Consensus for unreliable networks. Prepare, promise, accept, commit — with "only one value chosen" invariant.

+
+
+

Raft

+
230 lines · 8 actions · 3 invariants
+

Leader election, log replication, safety. The largest SpeckDL spec — demonstrates composition at scale.

+
+
+

Forgejo Issue Workflow

+
9 actions · 8 guard conditions
+

Real-world state machine: create → assign → review → merge → close. 5 temporal verification conditions.

+
+
+

Access Control

+
RBAC system · clean TS output
+

Role-based access with role addition, removal, and permission checks. One of 5 examples with zero TypeScript errors.

+
+
+

+ 8 more

+
OAuth2AuthorizationCode · DrugDosageCalculator · PaymentService · RetryHandler · TemperatureMonitor · AccountLedger · TigerBeetle · PreconditionTest
+

Browse all 14 examples →

-
-

Roadmap

- -
-
-

Phase A — Now

-

Open Standard: SpeckDL language, Dark Provenance methodology, reference implementations.

+
+

Status & Roadmap

+ +
    +
  • TypeScript codegen: Complete — state machine classes with full type coverage
  • +
  • Z3 verification backend: Live — --target=z3 emits SMT-LIB2 that Z3 accepts
  • +
  • OAuth 2.0 spec: Exists — examples/OAuth2AuthorizationCode.speck
  • +
  • Rust codegen: In progress (M2.1)
  • +
  • WASM via wasm-pack: Pending Rust codegen completion
  • +
+ +

The WAT/WASM generator was removed in favor of a proper IR → Rust → wasm-pack pipeline. No hand-rolled WASM text format; no magic.

+
+ +
+

Strategy Ladder

+

Speckl follows a four-step strategy — each step validates the market before building the next:

+ +
+
+
Plan A
+
Open Standard
+
✅ Complete
+

MIT license. SpeckDL language + compiler + whitepaper — all public.

-
-

Phase B — Parallel

-

Consulting: Hands-on engagements with regulated-industry teams.

+
+
Plan B
+
Consulting
+
🔄 Drafted
+

Help regulated teams adopt spec-first workflows. One-pager ready.

-
-

Phase C — 3-9mo

-

SaaS: Spec editor, live Z3 verification, provenance graph browser.

+
+
Plan C
+
Developer Tool / SaaS
+
🔮 Future
+

Self-serve spec validation, CI/CD integration, team collaboration.

-
-

Phase D — 9-18mo

-

Compliance API: Audit reports, traceability matrices.

+
+
Plan D
+
Compliance API
+
🔮 Future
+

Automated audit reports, traceability matrices, regulatory filings.

+
+

Try It

+ +
# Clone the repo
+git clone https://os.scoble.me/forgejo/sscoble/speckl.git
+cd speckl/compiler
+
+# Build and run tests (41 tests, ~2s)
+npm install
+npm test
+
+# Compile a spec to TypeScript + provenance artifacts
+node dist/index.js ../examples/TwoPhaseCommit.speck -o out/twophase
+
+# Compile to Z3 SMT-LIB2 for verification
+node dist/index.js ../examples/KafkaKRaft.speck --target=z3 -o out/kraft
+
+# Check TypeScript validity
+npx tsc --noEmit -p out/twophase
+
+ diff --git a/out/KafkaKRaft.prov.jsonld b/out/KafkaKRaft.prov.jsonld new file mode 100644 index 0000000..6a8fd3e --- /dev/null +++ b/out/KafkaKRaft.prov.jsonld @@ -0,0 +1,38 @@ +{ + "@context": { + "prov": "http://www.w3.org/ns/prov#", + "speckl": "https://speckl.scoble.me/ns/v0.2#", + "rdfs": "http://www.w3.org/2000/01/rdf-schema#" + }, + "@graph": [ + { + "@type": "prov:Entity", + "@id": "speck:KafkaKRaft", + "prov:qualifiedGeneration": { + "@type": "prov:Generation", + "prov:activity": { + "@type": "prov:Activity", + "prov:startTime": "2026-05-10T09:08:14.888Z" + } + }, + "rdfs:label": "KafkaKRaft", + "prov:wasAttributedTo": { + "@id": "agent:Speckl Compiler v0.3" + }, + "prov:wasInformedBy": { + "@id": "source:architecture_review:KRaft-Core-Consensus" + } + }, + { + "@type": "prov:Agent", + "@id": "agent:Speckl Compiler v0.3", + "rdfs:label": "Speckl Compiler v0.3" + }, + { + "@type": "prov:Activity", + "@id": "source:architecture_review:KRaft-Core-Consensus", + "prov:type": "architecture_review", + "rdfs:label": "architecture_review - KRaft-Core-Consensus" + } + ] +} \ No newline at end of file diff --git a/out/KafkaKRaft.smt2 b/out/KafkaKRaft.smt2 new file mode 100644 index 0000000..248b268 --- /dev/null +++ b/out/KafkaKRaft.smt2 @@ -0,0 +1,2492 @@ +; SpeckDL: KafkaKRaft — Z3 Verification +; Generated by speckl-compile Z3 backend +; Bounded model checking depth: 10 + +(set-logic ALL) +(set-option :produce-models true) +(declare-fun speckl_len_Int ( (Array Int Int) ) Int) +(declare-sort State 0) +(declare-sort Option_NodeId 0) +(declare-sort LogEntry 0) +(declare-sort NodeId 0) + +; --- State Variables --- +(declare-const currentTerm Int) +(declare-const state State) +(declare-const votedFor Option_NodeId) +(declare-const log (Array Int LogEntry)) +(declare-const commitIndex Int) +(declare-const nextIndex (Array NodeId Int)) +(declare-const matchIndex (Array NodeId Int)) +(declare-const votesGranted (Array NodeId Bool)) +(declare-const currentTerm_post Int) +(declare-const state_post State) +(declare-const votedFor_post Option_NodeId) +(declare-const log_post (Array Int LogEntry)) +(declare-const commitIndex_post Int) +(declare-const nextIndex_post (Array NodeId Int)) +(declare-const matchIndex_post (Array NodeId Int)) +(declare-const votesGranted_post (Array NodeId Bool)) + +; --- Invariants --- +(define-fun ElectionSafety () Bool + (and + (=> (= state Leader) (> currentTerm 0)) + ) +) +(define-fun LeaderCompleteness () Bool + (and + commitIndex <= (speckl_len_Int log) + ) +) +(define-fun LogMatching () Bool + (and + (forall ((i Int)) (=> (and (> i 0) (and i <= (speckl_len_Int log) (select log (- i 1)).term = (select log (- i 1)).term)) (select log (- i 1)).value = (select log (- i 1)).value)) + ) +) +(define-fun LeaderAppendOnly () Bool + (and + (speckl_len_Int log_post) >= (speckl_len_Int log) + ) +) +(define-fun TermMonotonicity () Bool + (and + (>= currentTerm_post currentTerm) + ) +) +(define-fun CommitSafety () Bool + (and + commitIndex <= (speckl_len_Int log) + ) +) + +; --- Action Transition Predicates --- +(define-fun RequestVoteEnabled () Bool + (and + (= state Candidate) + (>= currentTerm lastLogTerm) + ) +) +(define-fun RequestVoteStep () Bool + (and + (= votesGranted_post (store votesGranted candidate true)) + ) +) +(define-fun HandleRequestVoteRequestEnabled () Bool + (and + (>= currentTerm candidateTerm) + ) +) +(define-fun HandleRequestVoteRequestStep () Bool + (and + (= votedFor_post Some(candidate)) + ) +) +(define-fun BecomeLeaderEnabled () Bool + (and + (= state Candidate) + votesGranted_size() >= 3 + ) +) +(define-fun BecomeLeaderStep () Bool + (and + (= state_post Leader) + (= commitIndex_post (speckl_len_Int log)) + ) +) +(define-fun AppendEntriesEnabled () Bool + (and + (= state Leader) + ) +) +(define-fun AppendEntriesStep () Bool + (and + (= log_post append(log, newEntry)) + ) +) +; Next transition relation: RequestVote | HandleRequestVoteRequest | BecomeLeader | AppendEntries +(define-fun Next () Bool + (or + (and (RequestVoteEnabled) (RequestVoteStep)) + (and (HandleRequestVoteRequestEnabled) (HandleRequestVoteRequestStep)) + (and (BecomeLeaderEnabled) (BecomeLeaderStep)) + (and (AppendEntriesEnabled) (AppendEntriesStep)) + ; Stuttering step + (and + (= currentTerm_post currentTerm) + (= state_post state) + (= votedFor_post votedFor) + (= log_post log) + (= commitIndex_post commitIndex) + (= nextIndex_post nextIndex) + (= matchIndex_post matchIndex) + (= votesGranted_post votesGranted) + ) + ) +) + +; --- Bounded Model Checking --- +; Verification: Always(TermMonotonicity) +; BMC depth: 10 + +; Declare state copies for 10 steps +(declare-const currentTerm_0 Int) +(declare-const currentTerm_1 Int) +(declare-const currentTerm_2 Int) +(declare-const currentTerm_3 Int) +(declare-const currentTerm_4 Int) +(declare-const currentTerm_5 Int) +(declare-const currentTerm_6 Int) +(declare-const currentTerm_7 Int) +(declare-const currentTerm_8 Int) +(declare-const currentTerm_9 Int) +(declare-const currentTerm_10 Int) +(declare-const state_0 State) +(declare-const state_1 State) +(declare-const state_2 State) +(declare-const state_3 State) +(declare-const state_4 State) +(declare-const state_5 State) +(declare-const state_6 State) +(declare-const state_7 State) +(declare-const state_8 State) +(declare-const state_9 State) +(declare-const state_10 State) +(declare-const votedFor_0 Option_NodeId) +(declare-const votedFor_1 Option_NodeId) +(declare-const votedFor_2 Option_NodeId) +(declare-const votedFor_3 Option_NodeId) +(declare-const votedFor_4 Option_NodeId) +(declare-const votedFor_5 Option_NodeId) +(declare-const votedFor_6 Option_NodeId) +(declare-const votedFor_7 Option_NodeId) +(declare-const votedFor_8 Option_NodeId) +(declare-const votedFor_9 Option_NodeId) +(declare-const votedFor_10 Option_NodeId) +(declare-const log_0 (Array Int LogEntry)) +(declare-const log_1 (Array Int LogEntry)) +(declare-const log_2 (Array Int LogEntry)) +(declare-const log_3 (Array Int LogEntry)) +(declare-const log_4 (Array Int LogEntry)) +(declare-const log_5 (Array Int LogEntry)) +(declare-const log_6 (Array Int LogEntry)) +(declare-const log_7 (Array Int LogEntry)) +(declare-const log_8 (Array Int LogEntry)) +(declare-const log_9 (Array Int LogEntry)) +(declare-const log_10 (Array Int LogEntry)) +(declare-const commitIndex_0 Int) +(declare-const commitIndex_1 Int) +(declare-const commitIndex_2 Int) +(declare-const commitIndex_3 Int) +(declare-const commitIndex_4 Int) +(declare-const commitIndex_5 Int) +(declare-const commitIndex_6 Int) +(declare-const commitIndex_7 Int) +(declare-const commitIndex_8 Int) +(declare-const commitIndex_9 Int) +(declare-const commitIndex_10 Int) +(declare-const nextIndex_0 (Array NodeId Int)) +(declare-const nextIndex_1 (Array NodeId Int)) +(declare-const nextIndex_2 (Array NodeId Int)) +(declare-const nextIndex_3 (Array NodeId Int)) +(declare-const nextIndex_4 (Array NodeId Int)) +(declare-const nextIndex_5 (Array NodeId Int)) +(declare-const nextIndex_6 (Array NodeId Int)) +(declare-const nextIndex_7 (Array NodeId Int)) +(declare-const nextIndex_8 (Array NodeId Int)) +(declare-const nextIndex_9 (Array NodeId Int)) +(declare-const nextIndex_10 (Array NodeId Int)) +(declare-const matchIndex_0 (Array NodeId Int)) +(declare-const matchIndex_1 (Array NodeId Int)) +(declare-const matchIndex_2 (Array NodeId Int)) +(declare-const matchIndex_3 (Array NodeId Int)) +(declare-const matchIndex_4 (Array NodeId Int)) +(declare-const matchIndex_5 (Array NodeId Int)) +(declare-const matchIndex_6 (Array NodeId Int)) +(declare-const matchIndex_7 (Array NodeId Int)) +(declare-const matchIndex_8 (Array NodeId Int)) +(declare-const matchIndex_9 (Array NodeId Int)) +(declare-const matchIndex_10 (Array NodeId Int)) +(declare-const votesGranted_0 (Array NodeId Bool)) +(declare-const votesGranted_1 (Array NodeId Bool)) +(declare-const votesGranted_2 (Array NodeId Bool)) +(declare-const votesGranted_3 (Array NodeId Bool)) +(declare-const votesGranted_4 (Array NodeId Bool)) +(declare-const votesGranted_5 (Array NodeId Bool)) +(declare-const votesGranted_6 (Array NodeId Bool)) +(declare-const votesGranted_7 (Array NodeId Bool)) +(declare-const votesGranted_8 (Array NodeId Bool)) +(declare-const votesGranted_9 (Array NodeId Bool)) +(declare-const votesGranted_10 (Array NodeId Bool)) + +; Initial state constraints +(assert + (and + (>= currentTerm_0 0) + (>= commitIndex_0 0) + ) +) + +; Transition constraints +(assert ; Step 0 → 1 + (or + (and ; Action: RequestVote + (= state_0 Candidate) + (>= currentTerm_0 lastLogTerm) + (= votesGranted_1 (store votesGranted_0 candidate true)) + (= currentTerm_1 currentTerm_0) + (= state_1 state_0) + (= votedFor_1 votedFor_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= nextIndex_1 nextIndex_0) + (= matchIndex_1 matchIndex_0) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_0 candidateTerm) + (= votedFor_1 Some(candidate)) + (= currentTerm_1 currentTerm_0) + (= state_1 state_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= nextIndex_1 nextIndex_0) + (= matchIndex_1 matchIndex_0) + (= votesGranted_1 votesGranted_0) + ) + (and ; Action: BecomeLeader + (= state_0 Candidate) + votesGranted_size() >= 3 + (= state_1 Leader) + (= commitIndex_1 (speckl_len_Int log)) + (= currentTerm_1 currentTerm_0) + (= votedFor_1 votedFor_0) + (= log_1 log_0) + (= nextIndex_1 nextIndex_0) + (= matchIndex_1 matchIndex_0) + (= votesGranted_1 votesGranted_0) + ) + (and ; Action: AppendEntries + (= state_0 Leader) + (= log_1 append(log_0, newEntry)) + (= currentTerm_1 currentTerm_0) + (= state_1 state_0) + (= votedFor_1 votedFor_0) + (= commitIndex_1 commitIndex_0) + (= nextIndex_1 nextIndex_0) + (= matchIndex_1 matchIndex_0) + (= votesGranted_1 votesGranted_0) + ) + ; Stuttering + (and + (= currentTerm_1 currentTerm_0) + (= state_1 state_0) + (= votedFor_1 votedFor_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= nextIndex_1 nextIndex_0) + (= matchIndex_1 matchIndex_0) + (= votesGranted_1 votesGranted_0) + ) + ) +) +(assert ; Step 1 → 2 + (or + (and ; Action: RequestVote + (= state_1 Candidate) + (>= currentTerm_1 lastLogTerm) + (= votesGranted_2 (store votesGranted_1 candidate true)) + (= currentTerm_2 currentTerm_1) + (= state_2 state_1) + (= votedFor_2 votedFor_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= nextIndex_2 nextIndex_1) + (= matchIndex_2 matchIndex_1) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_1 candidateTerm) + (= votedFor_2 Some(candidate)) + (= currentTerm_2 currentTerm_1) + (= state_2 state_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= nextIndex_2 nextIndex_1) + (= matchIndex_2 matchIndex_1) + (= votesGranted_2 votesGranted_1) + ) + (and ; Action: BecomeLeader + (= state_1 Candidate) + votesGranted_size() >= 3 + (= state_2 Leader) + (= commitIndex_2 (speckl_len_Int log)) + (= currentTerm_2 currentTerm_1) + (= votedFor_2 votedFor_1) + (= log_2 log_1) + (= nextIndex_2 nextIndex_1) + (= matchIndex_2 matchIndex_1) + (= votesGranted_2 votesGranted_1) + ) + (and ; Action: AppendEntries + (= state_1 Leader) + (= log_2 append(log_1, newEntry)) + (= currentTerm_2 currentTerm_1) + (= state_2 state_1) + (= votedFor_2 votedFor_1) + (= commitIndex_2 commitIndex_1) + (= nextIndex_2 nextIndex_1) + (= matchIndex_2 matchIndex_1) + (= votesGranted_2 votesGranted_1) + ) + ; Stuttering + (and + (= currentTerm_2 currentTerm_1) + (= state_2 state_1) + (= votedFor_2 votedFor_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= nextIndex_2 nextIndex_1) + (= matchIndex_2 matchIndex_1) + (= votesGranted_2 votesGranted_1) + ) + ) +) +(assert ; Step 2 → 3 + (or + (and ; Action: RequestVote + (= state_2 Candidate) + (>= currentTerm_2 lastLogTerm) + (= votesGranted_3 (store votesGranted_2 candidate true)) + (= currentTerm_3 currentTerm_2) + (= state_3 state_2) + (= votedFor_3 votedFor_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= nextIndex_3 nextIndex_2) + (= matchIndex_3 matchIndex_2) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_2 candidateTerm) + (= votedFor_3 Some(candidate)) + (= currentTerm_3 currentTerm_2) + (= state_3 state_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= nextIndex_3 nextIndex_2) + (= matchIndex_3 matchIndex_2) + (= votesGranted_3 votesGranted_2) + ) + (and ; Action: BecomeLeader + (= state_2 Candidate) + votesGranted_size() >= 3 + (= state_3 Leader) + (= commitIndex_3 (speckl_len_Int log)) + (= currentTerm_3 currentTerm_2) + (= votedFor_3 votedFor_2) + (= log_3 log_2) + (= nextIndex_3 nextIndex_2) + (= matchIndex_3 matchIndex_2) + (= votesGranted_3 votesGranted_2) + ) + (and ; Action: AppendEntries + (= state_2 Leader) + (= log_3 append(log_2, newEntry)) + (= currentTerm_3 currentTerm_2) + (= state_3 state_2) + (= votedFor_3 votedFor_2) + (= commitIndex_3 commitIndex_2) + (= nextIndex_3 nextIndex_2) + (= matchIndex_3 matchIndex_2) + (= votesGranted_3 votesGranted_2) + ) + ; Stuttering + (and + (= currentTerm_3 currentTerm_2) + (= state_3 state_2) + (= votedFor_3 votedFor_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= nextIndex_3 nextIndex_2) + (= matchIndex_3 matchIndex_2) + (= votesGranted_3 votesGranted_2) + ) + ) +) +(assert ; Step 3 → 4 + (or + (and ; Action: RequestVote + (= state_3 Candidate) + (>= currentTerm_3 lastLogTerm) + (= votesGranted_4 (store votesGranted_3 candidate true)) + (= currentTerm_4 currentTerm_3) + (= state_4 state_3) + (= votedFor_4 votedFor_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= nextIndex_4 nextIndex_3) + (= matchIndex_4 matchIndex_3) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_3 candidateTerm) + (= votedFor_4 Some(candidate)) + (= currentTerm_4 currentTerm_3) + (= state_4 state_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= nextIndex_4 nextIndex_3) + (= matchIndex_4 matchIndex_3) + (= votesGranted_4 votesGranted_3) + ) + (and ; Action: BecomeLeader + (= state_3 Candidate) + votesGranted_size() >= 3 + (= state_4 Leader) + (= commitIndex_4 (speckl_len_Int log)) + (= currentTerm_4 currentTerm_3) + (= votedFor_4 votedFor_3) + (= log_4 log_3) + (= nextIndex_4 nextIndex_3) + (= matchIndex_4 matchIndex_3) + (= votesGranted_4 votesGranted_3) + ) + (and ; Action: AppendEntries + (= state_3 Leader) + (= log_4 append(log_3, newEntry)) + (= currentTerm_4 currentTerm_3) + (= state_4 state_3) + (= votedFor_4 votedFor_3) + (= commitIndex_4 commitIndex_3) + (= nextIndex_4 nextIndex_3) + (= matchIndex_4 matchIndex_3) + (= votesGranted_4 votesGranted_3) + ) + ; Stuttering + (and + (= currentTerm_4 currentTerm_3) + (= state_4 state_3) + (= votedFor_4 votedFor_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= nextIndex_4 nextIndex_3) + (= matchIndex_4 matchIndex_3) + (= votesGranted_4 votesGranted_3) + ) + ) +) +(assert ; Step 4 → 5 + (or + (and ; Action: RequestVote + (= state_4 Candidate) + (>= currentTerm_4 lastLogTerm) + (= votesGranted_5 (store votesGranted_4 candidate true)) + (= currentTerm_5 currentTerm_4) + (= state_5 state_4) + (= votedFor_5 votedFor_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= nextIndex_5 nextIndex_4) + (= matchIndex_5 matchIndex_4) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_4 candidateTerm) + (= votedFor_5 Some(candidate)) + (= currentTerm_5 currentTerm_4) + (= state_5 state_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= nextIndex_5 nextIndex_4) + (= matchIndex_5 matchIndex_4) + (= votesGranted_5 votesGranted_4) + ) + (and ; Action: BecomeLeader + (= state_4 Candidate) + votesGranted_size() >= 3 + (= state_5 Leader) + (= commitIndex_5 (speckl_len_Int log)) + (= currentTerm_5 currentTerm_4) + (= votedFor_5 votedFor_4) + (= log_5 log_4) + (= nextIndex_5 nextIndex_4) + (= matchIndex_5 matchIndex_4) + (= votesGranted_5 votesGranted_4) + ) + (and ; Action: AppendEntries + (= state_4 Leader) + (= log_5 append(log_4, newEntry)) + (= currentTerm_5 currentTerm_4) + (= state_5 state_4) + (= votedFor_5 votedFor_4) + (= commitIndex_5 commitIndex_4) + (= nextIndex_5 nextIndex_4) + (= matchIndex_5 matchIndex_4) + (= votesGranted_5 votesGranted_4) + ) + ; Stuttering + (and + (= currentTerm_5 currentTerm_4) + (= state_5 state_4) + (= votedFor_5 votedFor_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= nextIndex_5 nextIndex_4) + (= matchIndex_5 matchIndex_4) + (= votesGranted_5 votesGranted_4) + ) + ) +) +(assert ; Step 5 → 6 + (or + (and ; Action: RequestVote + (= state_5 Candidate) + (>= currentTerm_5 lastLogTerm) + (= votesGranted_6 (store votesGranted_5 candidate true)) + (= currentTerm_6 currentTerm_5) + (= state_6 state_5) + (= votedFor_6 votedFor_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= nextIndex_6 nextIndex_5) + (= matchIndex_6 matchIndex_5) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_5 candidateTerm) + (= votedFor_6 Some(candidate)) + (= currentTerm_6 currentTerm_5) + (= state_6 state_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= nextIndex_6 nextIndex_5) + (= matchIndex_6 matchIndex_5) + (= votesGranted_6 votesGranted_5) + ) + (and ; Action: BecomeLeader + (= state_5 Candidate) + votesGranted_size() >= 3 + (= state_6 Leader) + (= commitIndex_6 (speckl_len_Int log)) + (= currentTerm_6 currentTerm_5) + (= votedFor_6 votedFor_5) + (= log_6 log_5) + (= nextIndex_6 nextIndex_5) + (= matchIndex_6 matchIndex_5) + (= votesGranted_6 votesGranted_5) + ) + (and ; Action: AppendEntries + (= state_5 Leader) + (= log_6 append(log_5, newEntry)) + (= currentTerm_6 currentTerm_5) + (= state_6 state_5) + (= votedFor_6 votedFor_5) + (= commitIndex_6 commitIndex_5) + (= nextIndex_6 nextIndex_5) + (= matchIndex_6 matchIndex_5) + (= votesGranted_6 votesGranted_5) + ) + ; Stuttering + (and + (= currentTerm_6 currentTerm_5) + (= state_6 state_5) + (= votedFor_6 votedFor_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= nextIndex_6 nextIndex_5) + (= matchIndex_6 matchIndex_5) + (= votesGranted_6 votesGranted_5) + ) + ) +) +(assert ; Step 6 → 7 + (or + (and ; Action: RequestVote + (= state_6 Candidate) + (>= currentTerm_6 lastLogTerm) + (= votesGranted_7 (store votesGranted_6 candidate true)) + (= currentTerm_7 currentTerm_6) + (= state_7 state_6) + (= votedFor_7 votedFor_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= nextIndex_7 nextIndex_6) + (= matchIndex_7 matchIndex_6) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_6 candidateTerm) + (= votedFor_7 Some(candidate)) + (= currentTerm_7 currentTerm_6) + (= state_7 state_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= nextIndex_7 nextIndex_6) + (= matchIndex_7 matchIndex_6) + (= votesGranted_7 votesGranted_6) + ) + (and ; Action: BecomeLeader + (= state_6 Candidate) + votesGranted_size() >= 3 + (= state_7 Leader) + (= commitIndex_7 (speckl_len_Int log)) + (= currentTerm_7 currentTerm_6) + (= votedFor_7 votedFor_6) + (= log_7 log_6) + (= nextIndex_7 nextIndex_6) + (= matchIndex_7 matchIndex_6) + (= votesGranted_7 votesGranted_6) + ) + (and ; Action: AppendEntries + (= state_6 Leader) + (= log_7 append(log_6, newEntry)) + (= currentTerm_7 currentTerm_6) + (= state_7 state_6) + (= votedFor_7 votedFor_6) + (= commitIndex_7 commitIndex_6) + (= nextIndex_7 nextIndex_6) + (= matchIndex_7 matchIndex_6) + (= votesGranted_7 votesGranted_6) + ) + ; Stuttering + (and + (= currentTerm_7 currentTerm_6) + (= state_7 state_6) + (= votedFor_7 votedFor_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= nextIndex_7 nextIndex_6) + (= matchIndex_7 matchIndex_6) + (= votesGranted_7 votesGranted_6) + ) + ) +) +(assert ; Step 7 → 8 + (or + (and ; Action: RequestVote + (= state_7 Candidate) + (>= currentTerm_7 lastLogTerm) + (= votesGranted_8 (store votesGranted_7 candidate true)) + (= currentTerm_8 currentTerm_7) + (= state_8 state_7) + (= votedFor_8 votedFor_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= nextIndex_8 nextIndex_7) + (= matchIndex_8 matchIndex_7) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_7 candidateTerm) + (= votedFor_8 Some(candidate)) + (= currentTerm_8 currentTerm_7) + (= state_8 state_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= nextIndex_8 nextIndex_7) + (= matchIndex_8 matchIndex_7) + (= votesGranted_8 votesGranted_7) + ) + (and ; Action: BecomeLeader + (= state_7 Candidate) + votesGranted_size() >= 3 + (= state_8 Leader) + (= commitIndex_8 (speckl_len_Int log)) + (= currentTerm_8 currentTerm_7) + (= votedFor_8 votedFor_7) + (= log_8 log_7) + (= nextIndex_8 nextIndex_7) + (= matchIndex_8 matchIndex_7) + (= votesGranted_8 votesGranted_7) + ) + (and ; Action: AppendEntries + (= state_7 Leader) + (= log_8 append(log_7, newEntry)) + (= currentTerm_8 currentTerm_7) + (= state_8 state_7) + (= votedFor_8 votedFor_7) + (= commitIndex_8 commitIndex_7) + (= nextIndex_8 nextIndex_7) + (= matchIndex_8 matchIndex_7) + (= votesGranted_8 votesGranted_7) + ) + ; Stuttering + (and + (= currentTerm_8 currentTerm_7) + (= state_8 state_7) + (= votedFor_8 votedFor_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= nextIndex_8 nextIndex_7) + (= matchIndex_8 matchIndex_7) + (= votesGranted_8 votesGranted_7) + ) + ) +) +(assert ; Step 8 → 9 + (or + (and ; Action: RequestVote + (= state_8 Candidate) + (>= currentTerm_8 lastLogTerm) + (= votesGranted_9 (store votesGranted_8 candidate true)) + (= currentTerm_9 currentTerm_8) + (= state_9 state_8) + (= votedFor_9 votedFor_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= nextIndex_9 nextIndex_8) + (= matchIndex_9 matchIndex_8) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_8 candidateTerm) + (= votedFor_9 Some(candidate)) + (= currentTerm_9 currentTerm_8) + (= state_9 state_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= nextIndex_9 nextIndex_8) + (= matchIndex_9 matchIndex_8) + (= votesGranted_9 votesGranted_8) + ) + (and ; Action: BecomeLeader + (= state_8 Candidate) + votesGranted_size() >= 3 + (= state_9 Leader) + (= commitIndex_9 (speckl_len_Int log)) + (= currentTerm_9 currentTerm_8) + (= votedFor_9 votedFor_8) + (= log_9 log_8) + (= nextIndex_9 nextIndex_8) + (= matchIndex_9 matchIndex_8) + (= votesGranted_9 votesGranted_8) + ) + (and ; Action: AppendEntries + (= state_8 Leader) + (= log_9 append(log_8, newEntry)) + (= currentTerm_9 currentTerm_8) + (= state_9 state_8) + (= votedFor_9 votedFor_8) + (= commitIndex_9 commitIndex_8) + (= nextIndex_9 nextIndex_8) + (= matchIndex_9 matchIndex_8) + (= votesGranted_9 votesGranted_8) + ) + ; Stuttering + (and + (= currentTerm_9 currentTerm_8) + (= state_9 state_8) + (= votedFor_9 votedFor_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= nextIndex_9 nextIndex_8) + (= matchIndex_9 matchIndex_8) + (= votesGranted_9 votesGranted_8) + ) + ) +) +(assert ; Step 9 → 10 + (or + (and ; Action: RequestVote + (= state_9 Candidate) + (>= currentTerm_9 lastLogTerm) + (= votesGranted_10 (store votesGranted_9 candidate true)) + (= currentTerm_10 currentTerm_9) + (= state_10 state_9) + (= votedFor_10 votedFor_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= nextIndex_10 nextIndex_9) + (= matchIndex_10 matchIndex_9) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_9 candidateTerm) + (= votedFor_10 Some(candidate)) + (= currentTerm_10 currentTerm_9) + (= state_10 state_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= nextIndex_10 nextIndex_9) + (= matchIndex_10 matchIndex_9) + (= votesGranted_10 votesGranted_9) + ) + (and ; Action: BecomeLeader + (= state_9 Candidate) + votesGranted_size() >= 3 + (= state_10 Leader) + (= commitIndex_10 (speckl_len_Int log)) + (= currentTerm_10 currentTerm_9) + (= votedFor_10 votedFor_9) + (= log_10 log_9) + (= nextIndex_10 nextIndex_9) + (= matchIndex_10 matchIndex_9) + (= votesGranted_10 votesGranted_9) + ) + (and ; Action: AppendEntries + (= state_9 Leader) + (= log_10 append(log_9, newEntry)) + (= currentTerm_10 currentTerm_9) + (= state_10 state_9) + (= votedFor_10 votedFor_9) + (= commitIndex_10 commitIndex_9) + (= nextIndex_10 nextIndex_9) + (= matchIndex_10 matchIndex_9) + (= votesGranted_10 votesGranted_9) + ) + ; Stuttering + (and + (= currentTerm_10 currentTerm_9) + (= state_10 state_9) + (= votedFor_10 votedFor_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= nextIndex_10 nextIndex_9) + (= matchIndex_10 matchIndex_9) + (= votesGranted_10 votesGranted_9) + ) + ) +) + +; Property assertion: not(invariant at some step) +(assert + (or + (not (TermMonotonicity_0)) + (not (TermMonotonicity_1)) + (not (TermMonotonicity_2)) + (not (TermMonotonicity_3)) + (not (TermMonotonicity_4)) + (not (TermMonotonicity_5)) + (not (TermMonotonicity_6)) + (not (TermMonotonicity_7)) + (not (TermMonotonicity_8)) + (not (TermMonotonicity_9)) + (not (TermMonotonicity_10)) + ) +) +(define-fun TermMonotonicity_0 () Bool + (and + (>= currentTerm_post currentTerm_0) + ) +) +(define-fun TermMonotonicity_1 () Bool + (and + (>= currentTerm_post currentTerm_1) + ) +) +(define-fun TermMonotonicity_2 () Bool + (and + (>= currentTerm_post currentTerm_2) + ) +) +(define-fun TermMonotonicity_3 () Bool + (and + (>= currentTerm_post currentTerm_3) + ) +) +(define-fun TermMonotonicity_4 () Bool + (and + (>= currentTerm_post currentTerm_4) + ) +) +(define-fun TermMonotonicity_5 () Bool + (and + (>= currentTerm_post currentTerm_5) + ) +) +(define-fun TermMonotonicity_6 () Bool + (and + (>= currentTerm_post currentTerm_6) + ) +) +(define-fun TermMonotonicity_7 () Bool + (and + (>= currentTerm_post currentTerm_7) + ) +) +(define-fun TermMonotonicity_8 () Bool + (and + (>= currentTerm_post currentTerm_8) + ) +) +(define-fun TermMonotonicity_9 () Bool + (and + (>= currentTerm_post currentTerm_9) + ) +) +(define-fun TermMonotonicity_10 () Bool + (and + (>= currentTerm_post currentTerm_10) + ) +) + +(echo "Checking: Always(TermMonotonicity) for 10 steps") +(check-sat) +(get-model) +; Verification: Always(CommitSafety) +; BMC depth: 10 + +; Declare state copies for 10 steps +(declare-const currentTerm_0 Int) +(declare-const currentTerm_1 Int) +(declare-const currentTerm_2 Int) +(declare-const currentTerm_3 Int) +(declare-const currentTerm_4 Int) +(declare-const currentTerm_5 Int) +(declare-const currentTerm_6 Int) +(declare-const currentTerm_7 Int) +(declare-const currentTerm_8 Int) +(declare-const currentTerm_9 Int) +(declare-const currentTerm_10 Int) +(declare-const state_0 State) +(declare-const state_1 State) +(declare-const state_2 State) +(declare-const state_3 State) +(declare-const state_4 State) +(declare-const state_5 State) +(declare-const state_6 State) +(declare-const state_7 State) +(declare-const state_8 State) +(declare-const state_9 State) +(declare-const state_10 State) +(declare-const votedFor_0 Option_NodeId) +(declare-const votedFor_1 Option_NodeId) +(declare-const votedFor_2 Option_NodeId) +(declare-const votedFor_3 Option_NodeId) +(declare-const votedFor_4 Option_NodeId) +(declare-const votedFor_5 Option_NodeId) +(declare-const votedFor_6 Option_NodeId) +(declare-const votedFor_7 Option_NodeId) +(declare-const votedFor_8 Option_NodeId) +(declare-const votedFor_9 Option_NodeId) +(declare-const votedFor_10 Option_NodeId) +(declare-const log_0 (Array Int LogEntry)) +(declare-const log_1 (Array Int LogEntry)) +(declare-const log_2 (Array Int LogEntry)) +(declare-const log_3 (Array Int LogEntry)) +(declare-const log_4 (Array Int LogEntry)) +(declare-const log_5 (Array Int LogEntry)) +(declare-const log_6 (Array Int LogEntry)) +(declare-const log_7 (Array Int LogEntry)) +(declare-const log_8 (Array Int LogEntry)) +(declare-const log_9 (Array Int LogEntry)) +(declare-const log_10 (Array Int LogEntry)) +(declare-const commitIndex_0 Int) +(declare-const commitIndex_1 Int) +(declare-const commitIndex_2 Int) +(declare-const commitIndex_3 Int) +(declare-const commitIndex_4 Int) +(declare-const commitIndex_5 Int) +(declare-const commitIndex_6 Int) +(declare-const commitIndex_7 Int) +(declare-const commitIndex_8 Int) +(declare-const commitIndex_9 Int) +(declare-const commitIndex_10 Int) +(declare-const nextIndex_0 (Array NodeId Int)) +(declare-const nextIndex_1 (Array NodeId Int)) +(declare-const nextIndex_2 (Array NodeId Int)) +(declare-const nextIndex_3 (Array NodeId Int)) +(declare-const nextIndex_4 (Array NodeId Int)) +(declare-const nextIndex_5 (Array NodeId Int)) +(declare-const nextIndex_6 (Array NodeId Int)) +(declare-const nextIndex_7 (Array NodeId Int)) +(declare-const nextIndex_8 (Array NodeId Int)) +(declare-const nextIndex_9 (Array NodeId Int)) +(declare-const nextIndex_10 (Array NodeId Int)) +(declare-const matchIndex_0 (Array NodeId Int)) +(declare-const matchIndex_1 (Array NodeId Int)) +(declare-const matchIndex_2 (Array NodeId Int)) +(declare-const matchIndex_3 (Array NodeId Int)) +(declare-const matchIndex_4 (Array NodeId Int)) +(declare-const matchIndex_5 (Array NodeId Int)) +(declare-const matchIndex_6 (Array NodeId Int)) +(declare-const matchIndex_7 (Array NodeId Int)) +(declare-const matchIndex_8 (Array NodeId Int)) +(declare-const matchIndex_9 (Array NodeId Int)) +(declare-const matchIndex_10 (Array NodeId Int)) +(declare-const votesGranted_0 (Array NodeId Bool)) +(declare-const votesGranted_1 (Array NodeId Bool)) +(declare-const votesGranted_2 (Array NodeId Bool)) +(declare-const votesGranted_3 (Array NodeId Bool)) +(declare-const votesGranted_4 (Array NodeId Bool)) +(declare-const votesGranted_5 (Array NodeId Bool)) +(declare-const votesGranted_6 (Array NodeId Bool)) +(declare-const votesGranted_7 (Array NodeId Bool)) +(declare-const votesGranted_8 (Array NodeId Bool)) +(declare-const votesGranted_9 (Array NodeId Bool)) +(declare-const votesGranted_10 (Array NodeId Bool)) + +; Initial state constraints +(assert + (and + (>= currentTerm_0 0) + (>= commitIndex_0 0) + ) +) + +; Transition constraints +(assert ; Step 0 → 1 + (or + (and ; Action: RequestVote + (= state_0 Candidate) + (>= currentTerm_0 lastLogTerm) + (= votesGranted_1 (store votesGranted_0 candidate true)) + (= currentTerm_1 currentTerm_0) + (= state_1 state_0) + (= votedFor_1 votedFor_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= nextIndex_1 nextIndex_0) + (= matchIndex_1 matchIndex_0) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_0 candidateTerm) + (= votedFor_1 Some(candidate)) + (= currentTerm_1 currentTerm_0) + (= state_1 state_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= nextIndex_1 nextIndex_0) + (= matchIndex_1 matchIndex_0) + (= votesGranted_1 votesGranted_0) + ) + (and ; Action: BecomeLeader + (= state_0 Candidate) + votesGranted_size() >= 3 + (= state_1 Leader) + (= commitIndex_1 (speckl_len_Int log)) + (= currentTerm_1 currentTerm_0) + (= votedFor_1 votedFor_0) + (= log_1 log_0) + (= nextIndex_1 nextIndex_0) + (= matchIndex_1 matchIndex_0) + (= votesGranted_1 votesGranted_0) + ) + (and ; Action: AppendEntries + (= state_0 Leader) + (= log_1 append(log_0, newEntry)) + (= currentTerm_1 currentTerm_0) + (= state_1 state_0) + (= votedFor_1 votedFor_0) + (= commitIndex_1 commitIndex_0) + (= nextIndex_1 nextIndex_0) + (= matchIndex_1 matchIndex_0) + (= votesGranted_1 votesGranted_0) + ) + ; Stuttering + (and + (= currentTerm_1 currentTerm_0) + (= state_1 state_0) + (= votedFor_1 votedFor_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= nextIndex_1 nextIndex_0) + (= matchIndex_1 matchIndex_0) + (= votesGranted_1 votesGranted_0) + ) + ) +) +(assert ; Step 1 → 2 + (or + (and ; Action: RequestVote + (= state_1 Candidate) + (>= currentTerm_1 lastLogTerm) + (= votesGranted_2 (store votesGranted_1 candidate true)) + (= currentTerm_2 currentTerm_1) + (= state_2 state_1) + (= votedFor_2 votedFor_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= nextIndex_2 nextIndex_1) + (= matchIndex_2 matchIndex_1) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_1 candidateTerm) + (= votedFor_2 Some(candidate)) + (= currentTerm_2 currentTerm_1) + (= state_2 state_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= nextIndex_2 nextIndex_1) + (= matchIndex_2 matchIndex_1) + (= votesGranted_2 votesGranted_1) + ) + (and ; Action: BecomeLeader + (= state_1 Candidate) + votesGranted_size() >= 3 + (= state_2 Leader) + (= commitIndex_2 (speckl_len_Int log)) + (= currentTerm_2 currentTerm_1) + (= votedFor_2 votedFor_1) + (= log_2 log_1) + (= nextIndex_2 nextIndex_1) + (= matchIndex_2 matchIndex_1) + (= votesGranted_2 votesGranted_1) + ) + (and ; Action: AppendEntries + (= state_1 Leader) + (= log_2 append(log_1, newEntry)) + (= currentTerm_2 currentTerm_1) + (= state_2 state_1) + (= votedFor_2 votedFor_1) + (= commitIndex_2 commitIndex_1) + (= nextIndex_2 nextIndex_1) + (= matchIndex_2 matchIndex_1) + (= votesGranted_2 votesGranted_1) + ) + ; Stuttering + (and + (= currentTerm_2 currentTerm_1) + (= state_2 state_1) + (= votedFor_2 votedFor_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= nextIndex_2 nextIndex_1) + (= matchIndex_2 matchIndex_1) + (= votesGranted_2 votesGranted_1) + ) + ) +) +(assert ; Step 2 → 3 + (or + (and ; Action: RequestVote + (= state_2 Candidate) + (>= currentTerm_2 lastLogTerm) + (= votesGranted_3 (store votesGranted_2 candidate true)) + (= currentTerm_3 currentTerm_2) + (= state_3 state_2) + (= votedFor_3 votedFor_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= nextIndex_3 nextIndex_2) + (= matchIndex_3 matchIndex_2) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_2 candidateTerm) + (= votedFor_3 Some(candidate)) + (= currentTerm_3 currentTerm_2) + (= state_3 state_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= nextIndex_3 nextIndex_2) + (= matchIndex_3 matchIndex_2) + (= votesGranted_3 votesGranted_2) + ) + (and ; Action: BecomeLeader + (= state_2 Candidate) + votesGranted_size() >= 3 + (= state_3 Leader) + (= commitIndex_3 (speckl_len_Int log)) + (= currentTerm_3 currentTerm_2) + (= votedFor_3 votedFor_2) + (= log_3 log_2) + (= nextIndex_3 nextIndex_2) + (= matchIndex_3 matchIndex_2) + (= votesGranted_3 votesGranted_2) + ) + (and ; Action: AppendEntries + (= state_2 Leader) + (= log_3 append(log_2, newEntry)) + (= currentTerm_3 currentTerm_2) + (= state_3 state_2) + (= votedFor_3 votedFor_2) + (= commitIndex_3 commitIndex_2) + (= nextIndex_3 nextIndex_2) + (= matchIndex_3 matchIndex_2) + (= votesGranted_3 votesGranted_2) + ) + ; Stuttering + (and + (= currentTerm_3 currentTerm_2) + (= state_3 state_2) + (= votedFor_3 votedFor_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= nextIndex_3 nextIndex_2) + (= matchIndex_3 matchIndex_2) + (= votesGranted_3 votesGranted_2) + ) + ) +) +(assert ; Step 3 → 4 + (or + (and ; Action: RequestVote + (= state_3 Candidate) + (>= currentTerm_3 lastLogTerm) + (= votesGranted_4 (store votesGranted_3 candidate true)) + (= currentTerm_4 currentTerm_3) + (= state_4 state_3) + (= votedFor_4 votedFor_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= nextIndex_4 nextIndex_3) + (= matchIndex_4 matchIndex_3) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_3 candidateTerm) + (= votedFor_4 Some(candidate)) + (= currentTerm_4 currentTerm_3) + (= state_4 state_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= nextIndex_4 nextIndex_3) + (= matchIndex_4 matchIndex_3) + (= votesGranted_4 votesGranted_3) + ) + (and ; Action: BecomeLeader + (= state_3 Candidate) + votesGranted_size() >= 3 + (= state_4 Leader) + (= commitIndex_4 (speckl_len_Int log)) + (= currentTerm_4 currentTerm_3) + (= votedFor_4 votedFor_3) + (= log_4 log_3) + (= nextIndex_4 nextIndex_3) + (= matchIndex_4 matchIndex_3) + (= votesGranted_4 votesGranted_3) + ) + (and ; Action: AppendEntries + (= state_3 Leader) + (= log_4 append(log_3, newEntry)) + (= currentTerm_4 currentTerm_3) + (= state_4 state_3) + (= votedFor_4 votedFor_3) + (= commitIndex_4 commitIndex_3) + (= nextIndex_4 nextIndex_3) + (= matchIndex_4 matchIndex_3) + (= votesGranted_4 votesGranted_3) + ) + ; Stuttering + (and + (= currentTerm_4 currentTerm_3) + (= state_4 state_3) + (= votedFor_4 votedFor_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= nextIndex_4 nextIndex_3) + (= matchIndex_4 matchIndex_3) + (= votesGranted_4 votesGranted_3) + ) + ) +) +(assert ; Step 4 → 5 + (or + (and ; Action: RequestVote + (= state_4 Candidate) + (>= currentTerm_4 lastLogTerm) + (= votesGranted_5 (store votesGranted_4 candidate true)) + (= currentTerm_5 currentTerm_4) + (= state_5 state_4) + (= votedFor_5 votedFor_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= nextIndex_5 nextIndex_4) + (= matchIndex_5 matchIndex_4) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_4 candidateTerm) + (= votedFor_5 Some(candidate)) + (= currentTerm_5 currentTerm_4) + (= state_5 state_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= nextIndex_5 nextIndex_4) + (= matchIndex_5 matchIndex_4) + (= votesGranted_5 votesGranted_4) + ) + (and ; Action: BecomeLeader + (= state_4 Candidate) + votesGranted_size() >= 3 + (= state_5 Leader) + (= commitIndex_5 (speckl_len_Int log)) + (= currentTerm_5 currentTerm_4) + (= votedFor_5 votedFor_4) + (= log_5 log_4) + (= nextIndex_5 nextIndex_4) + (= matchIndex_5 matchIndex_4) + (= votesGranted_5 votesGranted_4) + ) + (and ; Action: AppendEntries + (= state_4 Leader) + (= log_5 append(log_4, newEntry)) + (= currentTerm_5 currentTerm_4) + (= state_5 state_4) + (= votedFor_5 votedFor_4) + (= commitIndex_5 commitIndex_4) + (= nextIndex_5 nextIndex_4) + (= matchIndex_5 matchIndex_4) + (= votesGranted_5 votesGranted_4) + ) + ; Stuttering + (and + (= currentTerm_5 currentTerm_4) + (= state_5 state_4) + (= votedFor_5 votedFor_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= nextIndex_5 nextIndex_4) + (= matchIndex_5 matchIndex_4) + (= votesGranted_5 votesGranted_4) + ) + ) +) +(assert ; Step 5 → 6 + (or + (and ; Action: RequestVote + (= state_5 Candidate) + (>= currentTerm_5 lastLogTerm) + (= votesGranted_6 (store votesGranted_5 candidate true)) + (= currentTerm_6 currentTerm_5) + (= state_6 state_5) + (= votedFor_6 votedFor_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= nextIndex_6 nextIndex_5) + (= matchIndex_6 matchIndex_5) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_5 candidateTerm) + (= votedFor_6 Some(candidate)) + (= currentTerm_6 currentTerm_5) + (= state_6 state_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= nextIndex_6 nextIndex_5) + (= matchIndex_6 matchIndex_5) + (= votesGranted_6 votesGranted_5) + ) + (and ; Action: BecomeLeader + (= state_5 Candidate) + votesGranted_size() >= 3 + (= state_6 Leader) + (= commitIndex_6 (speckl_len_Int log)) + (= currentTerm_6 currentTerm_5) + (= votedFor_6 votedFor_5) + (= log_6 log_5) + (= nextIndex_6 nextIndex_5) + (= matchIndex_6 matchIndex_5) + (= votesGranted_6 votesGranted_5) + ) + (and ; Action: AppendEntries + (= state_5 Leader) + (= log_6 append(log_5, newEntry)) + (= currentTerm_6 currentTerm_5) + (= state_6 state_5) + (= votedFor_6 votedFor_5) + (= commitIndex_6 commitIndex_5) + (= nextIndex_6 nextIndex_5) + (= matchIndex_6 matchIndex_5) + (= votesGranted_6 votesGranted_5) + ) + ; Stuttering + (and + (= currentTerm_6 currentTerm_5) + (= state_6 state_5) + (= votedFor_6 votedFor_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= nextIndex_6 nextIndex_5) + (= matchIndex_6 matchIndex_5) + (= votesGranted_6 votesGranted_5) + ) + ) +) +(assert ; Step 6 → 7 + (or + (and ; Action: RequestVote + (= state_6 Candidate) + (>= currentTerm_6 lastLogTerm) + (= votesGranted_7 (store votesGranted_6 candidate true)) + (= currentTerm_7 currentTerm_6) + (= state_7 state_6) + (= votedFor_7 votedFor_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= nextIndex_7 nextIndex_6) + (= matchIndex_7 matchIndex_6) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_6 candidateTerm) + (= votedFor_7 Some(candidate)) + (= currentTerm_7 currentTerm_6) + (= state_7 state_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= nextIndex_7 nextIndex_6) + (= matchIndex_7 matchIndex_6) + (= votesGranted_7 votesGranted_6) + ) + (and ; Action: BecomeLeader + (= state_6 Candidate) + votesGranted_size() >= 3 + (= state_7 Leader) + (= commitIndex_7 (speckl_len_Int log)) + (= currentTerm_7 currentTerm_6) + (= votedFor_7 votedFor_6) + (= log_7 log_6) + (= nextIndex_7 nextIndex_6) + (= matchIndex_7 matchIndex_6) + (= votesGranted_7 votesGranted_6) + ) + (and ; Action: AppendEntries + (= state_6 Leader) + (= log_7 append(log_6, newEntry)) + (= currentTerm_7 currentTerm_6) + (= state_7 state_6) + (= votedFor_7 votedFor_6) + (= commitIndex_7 commitIndex_6) + (= nextIndex_7 nextIndex_6) + (= matchIndex_7 matchIndex_6) + (= votesGranted_7 votesGranted_6) + ) + ; Stuttering + (and + (= currentTerm_7 currentTerm_6) + (= state_7 state_6) + (= votedFor_7 votedFor_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= nextIndex_7 nextIndex_6) + (= matchIndex_7 matchIndex_6) + (= votesGranted_7 votesGranted_6) + ) + ) +) +(assert ; Step 7 → 8 + (or + (and ; Action: RequestVote + (= state_7 Candidate) + (>= currentTerm_7 lastLogTerm) + (= votesGranted_8 (store votesGranted_7 candidate true)) + (= currentTerm_8 currentTerm_7) + (= state_8 state_7) + (= votedFor_8 votedFor_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= nextIndex_8 nextIndex_7) + (= matchIndex_8 matchIndex_7) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_7 candidateTerm) + (= votedFor_8 Some(candidate)) + (= currentTerm_8 currentTerm_7) + (= state_8 state_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= nextIndex_8 nextIndex_7) + (= matchIndex_8 matchIndex_7) + (= votesGranted_8 votesGranted_7) + ) + (and ; Action: BecomeLeader + (= state_7 Candidate) + votesGranted_size() >= 3 + (= state_8 Leader) + (= commitIndex_8 (speckl_len_Int log)) + (= currentTerm_8 currentTerm_7) + (= votedFor_8 votedFor_7) + (= log_8 log_7) + (= nextIndex_8 nextIndex_7) + (= matchIndex_8 matchIndex_7) + (= votesGranted_8 votesGranted_7) + ) + (and ; Action: AppendEntries + (= state_7 Leader) + (= log_8 append(log_7, newEntry)) + (= currentTerm_8 currentTerm_7) + (= state_8 state_7) + (= votedFor_8 votedFor_7) + (= commitIndex_8 commitIndex_7) + (= nextIndex_8 nextIndex_7) + (= matchIndex_8 matchIndex_7) + (= votesGranted_8 votesGranted_7) + ) + ; Stuttering + (and + (= currentTerm_8 currentTerm_7) + (= state_8 state_7) + (= votedFor_8 votedFor_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= nextIndex_8 nextIndex_7) + (= matchIndex_8 matchIndex_7) + (= votesGranted_8 votesGranted_7) + ) + ) +) +(assert ; Step 8 → 9 + (or + (and ; Action: RequestVote + (= state_8 Candidate) + (>= currentTerm_8 lastLogTerm) + (= votesGranted_9 (store votesGranted_8 candidate true)) + (= currentTerm_9 currentTerm_8) + (= state_9 state_8) + (= votedFor_9 votedFor_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= nextIndex_9 nextIndex_8) + (= matchIndex_9 matchIndex_8) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_8 candidateTerm) + (= votedFor_9 Some(candidate)) + (= currentTerm_9 currentTerm_8) + (= state_9 state_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= nextIndex_9 nextIndex_8) + (= matchIndex_9 matchIndex_8) + (= votesGranted_9 votesGranted_8) + ) + (and ; Action: BecomeLeader + (= state_8 Candidate) + votesGranted_size() >= 3 + (= state_9 Leader) + (= commitIndex_9 (speckl_len_Int log)) + (= currentTerm_9 currentTerm_8) + (= votedFor_9 votedFor_8) + (= log_9 log_8) + (= nextIndex_9 nextIndex_8) + (= matchIndex_9 matchIndex_8) + (= votesGranted_9 votesGranted_8) + ) + (and ; Action: AppendEntries + (= state_8 Leader) + (= log_9 append(log_8, newEntry)) + (= currentTerm_9 currentTerm_8) + (= state_9 state_8) + (= votedFor_9 votedFor_8) + (= commitIndex_9 commitIndex_8) + (= nextIndex_9 nextIndex_8) + (= matchIndex_9 matchIndex_8) + (= votesGranted_9 votesGranted_8) + ) + ; Stuttering + (and + (= currentTerm_9 currentTerm_8) + (= state_9 state_8) + (= votedFor_9 votedFor_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= nextIndex_9 nextIndex_8) + (= matchIndex_9 matchIndex_8) + (= votesGranted_9 votesGranted_8) + ) + ) +) +(assert ; Step 9 → 10 + (or + (and ; Action: RequestVote + (= state_9 Candidate) + (>= currentTerm_9 lastLogTerm) + (= votesGranted_10 (store votesGranted_9 candidate true)) + (= currentTerm_10 currentTerm_9) + (= state_10 state_9) + (= votedFor_10 votedFor_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= nextIndex_10 nextIndex_9) + (= matchIndex_10 matchIndex_9) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_9 candidateTerm) + (= votedFor_10 Some(candidate)) + (= currentTerm_10 currentTerm_9) + (= state_10 state_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= nextIndex_10 nextIndex_9) + (= matchIndex_10 matchIndex_9) + (= votesGranted_10 votesGranted_9) + ) + (and ; Action: BecomeLeader + (= state_9 Candidate) + votesGranted_size() >= 3 + (= state_10 Leader) + (= commitIndex_10 (speckl_len_Int log)) + (= currentTerm_10 currentTerm_9) + (= votedFor_10 votedFor_9) + (= log_10 log_9) + (= nextIndex_10 nextIndex_9) + (= matchIndex_10 matchIndex_9) + (= votesGranted_10 votesGranted_9) + ) + (and ; Action: AppendEntries + (= state_9 Leader) + (= log_10 append(log_9, newEntry)) + (= currentTerm_10 currentTerm_9) + (= state_10 state_9) + (= votedFor_10 votedFor_9) + (= commitIndex_10 commitIndex_9) + (= nextIndex_10 nextIndex_9) + (= matchIndex_10 matchIndex_9) + (= votesGranted_10 votesGranted_9) + ) + ; Stuttering + (and + (= currentTerm_10 currentTerm_9) + (= state_10 state_9) + (= votedFor_10 votedFor_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= nextIndex_10 nextIndex_9) + (= matchIndex_10 matchIndex_9) + (= votesGranted_10 votesGranted_9) + ) + ) +) + +; Property assertion: not(invariant at some step) +(assert + (or + (not (CommitSafety_0)) + (not (CommitSafety_1)) + (not (CommitSafety_2)) + (not (CommitSafety_3)) + (not (CommitSafety_4)) + (not (CommitSafety_5)) + (not (CommitSafety_6)) + (not (CommitSafety_7)) + (not (CommitSafety_8)) + (not (CommitSafety_9)) + (not (CommitSafety_10)) + ) +) +(define-fun CommitSafety_0 () Bool + (and + commitIndex_0 <= (speckl_len_Int log) + ) +) +(define-fun CommitSafety_1 () Bool + (and + commitIndex_1 <= (speckl_len_Int log) + ) +) +(define-fun CommitSafety_2 () Bool + (and + commitIndex_2 <= (speckl_len_Int log) + ) +) +(define-fun CommitSafety_3 () Bool + (and + commitIndex_3 <= (speckl_len_Int log) + ) +) +(define-fun CommitSafety_4 () Bool + (and + commitIndex_4 <= (speckl_len_Int log) + ) +) +(define-fun CommitSafety_5 () Bool + (and + commitIndex_5 <= (speckl_len_Int log) + ) +) +(define-fun CommitSafety_6 () Bool + (and + commitIndex_6 <= (speckl_len_Int log) + ) +) +(define-fun CommitSafety_7 () Bool + (and + commitIndex_7 <= (speckl_len_Int log) + ) +) +(define-fun CommitSafety_8 () Bool + (and + commitIndex_8 <= (speckl_len_Int log) + ) +) +(define-fun CommitSafety_9 () Bool + (and + commitIndex_9 <= (speckl_len_Int log) + ) +) +(define-fun CommitSafety_10 () Bool + (and + commitIndex_10 <= (speckl_len_Int log) + ) +) + +(echo "Checking: Always(CommitSafety) for 10 steps") +(check-sat) +(get-model) +; Verification: Always(LeaderCompleteness) +; BMC depth: 10 + +; Declare state copies for 10 steps +(declare-const currentTerm_0 Int) +(declare-const currentTerm_1 Int) +(declare-const currentTerm_2 Int) +(declare-const currentTerm_3 Int) +(declare-const currentTerm_4 Int) +(declare-const currentTerm_5 Int) +(declare-const currentTerm_6 Int) +(declare-const currentTerm_7 Int) +(declare-const currentTerm_8 Int) +(declare-const currentTerm_9 Int) +(declare-const currentTerm_10 Int) +(declare-const state_0 State) +(declare-const state_1 State) +(declare-const state_2 State) +(declare-const state_3 State) +(declare-const state_4 State) +(declare-const state_5 State) +(declare-const state_6 State) +(declare-const state_7 State) +(declare-const state_8 State) +(declare-const state_9 State) +(declare-const state_10 State) +(declare-const votedFor_0 Option_NodeId) +(declare-const votedFor_1 Option_NodeId) +(declare-const votedFor_2 Option_NodeId) +(declare-const votedFor_3 Option_NodeId) +(declare-const votedFor_4 Option_NodeId) +(declare-const votedFor_5 Option_NodeId) +(declare-const votedFor_6 Option_NodeId) +(declare-const votedFor_7 Option_NodeId) +(declare-const votedFor_8 Option_NodeId) +(declare-const votedFor_9 Option_NodeId) +(declare-const votedFor_10 Option_NodeId) +(declare-const log_0 (Array Int LogEntry)) +(declare-const log_1 (Array Int LogEntry)) +(declare-const log_2 (Array Int LogEntry)) +(declare-const log_3 (Array Int LogEntry)) +(declare-const log_4 (Array Int LogEntry)) +(declare-const log_5 (Array Int LogEntry)) +(declare-const log_6 (Array Int LogEntry)) +(declare-const log_7 (Array Int LogEntry)) +(declare-const log_8 (Array Int LogEntry)) +(declare-const log_9 (Array Int LogEntry)) +(declare-const log_10 (Array Int LogEntry)) +(declare-const commitIndex_0 Int) +(declare-const commitIndex_1 Int) +(declare-const commitIndex_2 Int) +(declare-const commitIndex_3 Int) +(declare-const commitIndex_4 Int) +(declare-const commitIndex_5 Int) +(declare-const commitIndex_6 Int) +(declare-const commitIndex_7 Int) +(declare-const commitIndex_8 Int) +(declare-const commitIndex_9 Int) +(declare-const commitIndex_10 Int) +(declare-const nextIndex_0 (Array NodeId Int)) +(declare-const nextIndex_1 (Array NodeId Int)) +(declare-const nextIndex_2 (Array NodeId Int)) +(declare-const nextIndex_3 (Array NodeId Int)) +(declare-const nextIndex_4 (Array NodeId Int)) +(declare-const nextIndex_5 (Array NodeId Int)) +(declare-const nextIndex_6 (Array NodeId Int)) +(declare-const nextIndex_7 (Array NodeId Int)) +(declare-const nextIndex_8 (Array NodeId Int)) +(declare-const nextIndex_9 (Array NodeId Int)) +(declare-const nextIndex_10 (Array NodeId Int)) +(declare-const matchIndex_0 (Array NodeId Int)) +(declare-const matchIndex_1 (Array NodeId Int)) +(declare-const matchIndex_2 (Array NodeId Int)) +(declare-const matchIndex_3 (Array NodeId Int)) +(declare-const matchIndex_4 (Array NodeId Int)) +(declare-const matchIndex_5 (Array NodeId Int)) +(declare-const matchIndex_6 (Array NodeId Int)) +(declare-const matchIndex_7 (Array NodeId Int)) +(declare-const matchIndex_8 (Array NodeId Int)) +(declare-const matchIndex_9 (Array NodeId Int)) +(declare-const matchIndex_10 (Array NodeId Int)) +(declare-const votesGranted_0 (Array NodeId Bool)) +(declare-const votesGranted_1 (Array NodeId Bool)) +(declare-const votesGranted_2 (Array NodeId Bool)) +(declare-const votesGranted_3 (Array NodeId Bool)) +(declare-const votesGranted_4 (Array NodeId Bool)) +(declare-const votesGranted_5 (Array NodeId Bool)) +(declare-const votesGranted_6 (Array NodeId Bool)) +(declare-const votesGranted_7 (Array NodeId Bool)) +(declare-const votesGranted_8 (Array NodeId Bool)) +(declare-const votesGranted_9 (Array NodeId Bool)) +(declare-const votesGranted_10 (Array NodeId Bool)) + +; Initial state constraints +(assert + (and + (>= currentTerm_0 0) + (>= commitIndex_0 0) + ) +) + +; Transition constraints +(assert ; Step 0 → 1 + (or + (and ; Action: RequestVote + (= state_0 Candidate) + (>= currentTerm_0 lastLogTerm) + (= votesGranted_1 (store votesGranted_0 candidate true)) + (= currentTerm_1 currentTerm_0) + (= state_1 state_0) + (= votedFor_1 votedFor_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= nextIndex_1 nextIndex_0) + (= matchIndex_1 matchIndex_0) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_0 candidateTerm) + (= votedFor_1 Some(candidate)) + (= currentTerm_1 currentTerm_0) + (= state_1 state_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= nextIndex_1 nextIndex_0) + (= matchIndex_1 matchIndex_0) + (= votesGranted_1 votesGranted_0) + ) + (and ; Action: BecomeLeader + (= state_0 Candidate) + votesGranted_size() >= 3 + (= state_1 Leader) + (= commitIndex_1 (speckl_len_Int log)) + (= currentTerm_1 currentTerm_0) + (= votedFor_1 votedFor_0) + (= log_1 log_0) + (= nextIndex_1 nextIndex_0) + (= matchIndex_1 matchIndex_0) + (= votesGranted_1 votesGranted_0) + ) + (and ; Action: AppendEntries + (= state_0 Leader) + (= log_1 append(log_0, newEntry)) + (= currentTerm_1 currentTerm_0) + (= state_1 state_0) + (= votedFor_1 votedFor_0) + (= commitIndex_1 commitIndex_0) + (= nextIndex_1 nextIndex_0) + (= matchIndex_1 matchIndex_0) + (= votesGranted_1 votesGranted_0) + ) + ; Stuttering + (and + (= currentTerm_1 currentTerm_0) + (= state_1 state_0) + (= votedFor_1 votedFor_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= nextIndex_1 nextIndex_0) + (= matchIndex_1 matchIndex_0) + (= votesGranted_1 votesGranted_0) + ) + ) +) +(assert ; Step 1 → 2 + (or + (and ; Action: RequestVote + (= state_1 Candidate) + (>= currentTerm_1 lastLogTerm) + (= votesGranted_2 (store votesGranted_1 candidate true)) + (= currentTerm_2 currentTerm_1) + (= state_2 state_1) + (= votedFor_2 votedFor_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= nextIndex_2 nextIndex_1) + (= matchIndex_2 matchIndex_1) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_1 candidateTerm) + (= votedFor_2 Some(candidate)) + (= currentTerm_2 currentTerm_1) + (= state_2 state_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= nextIndex_2 nextIndex_1) + (= matchIndex_2 matchIndex_1) + (= votesGranted_2 votesGranted_1) + ) + (and ; Action: BecomeLeader + (= state_1 Candidate) + votesGranted_size() >= 3 + (= state_2 Leader) + (= commitIndex_2 (speckl_len_Int log)) + (= currentTerm_2 currentTerm_1) + (= votedFor_2 votedFor_1) + (= log_2 log_1) + (= nextIndex_2 nextIndex_1) + (= matchIndex_2 matchIndex_1) + (= votesGranted_2 votesGranted_1) + ) + (and ; Action: AppendEntries + (= state_1 Leader) + (= log_2 append(log_1, newEntry)) + (= currentTerm_2 currentTerm_1) + (= state_2 state_1) + (= votedFor_2 votedFor_1) + (= commitIndex_2 commitIndex_1) + (= nextIndex_2 nextIndex_1) + (= matchIndex_2 matchIndex_1) + (= votesGranted_2 votesGranted_1) + ) + ; Stuttering + (and + (= currentTerm_2 currentTerm_1) + (= state_2 state_1) + (= votedFor_2 votedFor_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= nextIndex_2 nextIndex_1) + (= matchIndex_2 matchIndex_1) + (= votesGranted_2 votesGranted_1) + ) + ) +) +(assert ; Step 2 → 3 + (or + (and ; Action: RequestVote + (= state_2 Candidate) + (>= currentTerm_2 lastLogTerm) + (= votesGranted_3 (store votesGranted_2 candidate true)) + (= currentTerm_3 currentTerm_2) + (= state_3 state_2) + (= votedFor_3 votedFor_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= nextIndex_3 nextIndex_2) + (= matchIndex_3 matchIndex_2) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_2 candidateTerm) + (= votedFor_3 Some(candidate)) + (= currentTerm_3 currentTerm_2) + (= state_3 state_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= nextIndex_3 nextIndex_2) + (= matchIndex_3 matchIndex_2) + (= votesGranted_3 votesGranted_2) + ) + (and ; Action: BecomeLeader + (= state_2 Candidate) + votesGranted_size() >= 3 + (= state_3 Leader) + (= commitIndex_3 (speckl_len_Int log)) + (= currentTerm_3 currentTerm_2) + (= votedFor_3 votedFor_2) + (= log_3 log_2) + (= nextIndex_3 nextIndex_2) + (= matchIndex_3 matchIndex_2) + (= votesGranted_3 votesGranted_2) + ) + (and ; Action: AppendEntries + (= state_2 Leader) + (= log_3 append(log_2, newEntry)) + (= currentTerm_3 currentTerm_2) + (= state_3 state_2) + (= votedFor_3 votedFor_2) + (= commitIndex_3 commitIndex_2) + (= nextIndex_3 nextIndex_2) + (= matchIndex_3 matchIndex_2) + (= votesGranted_3 votesGranted_2) + ) + ; Stuttering + (and + (= currentTerm_3 currentTerm_2) + (= state_3 state_2) + (= votedFor_3 votedFor_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= nextIndex_3 nextIndex_2) + (= matchIndex_3 matchIndex_2) + (= votesGranted_3 votesGranted_2) + ) + ) +) +(assert ; Step 3 → 4 + (or + (and ; Action: RequestVote + (= state_3 Candidate) + (>= currentTerm_3 lastLogTerm) + (= votesGranted_4 (store votesGranted_3 candidate true)) + (= currentTerm_4 currentTerm_3) + (= state_4 state_3) + (= votedFor_4 votedFor_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= nextIndex_4 nextIndex_3) + (= matchIndex_4 matchIndex_3) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_3 candidateTerm) + (= votedFor_4 Some(candidate)) + (= currentTerm_4 currentTerm_3) + (= state_4 state_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= nextIndex_4 nextIndex_3) + (= matchIndex_4 matchIndex_3) + (= votesGranted_4 votesGranted_3) + ) + (and ; Action: BecomeLeader + (= state_3 Candidate) + votesGranted_size() >= 3 + (= state_4 Leader) + (= commitIndex_4 (speckl_len_Int log)) + (= currentTerm_4 currentTerm_3) + (= votedFor_4 votedFor_3) + (= log_4 log_3) + (= nextIndex_4 nextIndex_3) + (= matchIndex_4 matchIndex_3) + (= votesGranted_4 votesGranted_3) + ) + (and ; Action: AppendEntries + (= state_3 Leader) + (= log_4 append(log_3, newEntry)) + (= currentTerm_4 currentTerm_3) + (= state_4 state_3) + (= votedFor_4 votedFor_3) + (= commitIndex_4 commitIndex_3) + (= nextIndex_4 nextIndex_3) + (= matchIndex_4 matchIndex_3) + (= votesGranted_4 votesGranted_3) + ) + ; Stuttering + (and + (= currentTerm_4 currentTerm_3) + (= state_4 state_3) + (= votedFor_4 votedFor_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= nextIndex_4 nextIndex_3) + (= matchIndex_4 matchIndex_3) + (= votesGranted_4 votesGranted_3) + ) + ) +) +(assert ; Step 4 → 5 + (or + (and ; Action: RequestVote + (= state_4 Candidate) + (>= currentTerm_4 lastLogTerm) + (= votesGranted_5 (store votesGranted_4 candidate true)) + (= currentTerm_5 currentTerm_4) + (= state_5 state_4) + (= votedFor_5 votedFor_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= nextIndex_5 nextIndex_4) + (= matchIndex_5 matchIndex_4) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_4 candidateTerm) + (= votedFor_5 Some(candidate)) + (= currentTerm_5 currentTerm_4) + (= state_5 state_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= nextIndex_5 nextIndex_4) + (= matchIndex_5 matchIndex_4) + (= votesGranted_5 votesGranted_4) + ) + (and ; Action: BecomeLeader + (= state_4 Candidate) + votesGranted_size() >= 3 + (= state_5 Leader) + (= commitIndex_5 (speckl_len_Int log)) + (= currentTerm_5 currentTerm_4) + (= votedFor_5 votedFor_4) + (= log_5 log_4) + (= nextIndex_5 nextIndex_4) + (= matchIndex_5 matchIndex_4) + (= votesGranted_5 votesGranted_4) + ) + (and ; Action: AppendEntries + (= state_4 Leader) + (= log_5 append(log_4, newEntry)) + (= currentTerm_5 currentTerm_4) + (= state_5 state_4) + (= votedFor_5 votedFor_4) + (= commitIndex_5 commitIndex_4) + (= nextIndex_5 nextIndex_4) + (= matchIndex_5 matchIndex_4) + (= votesGranted_5 votesGranted_4) + ) + ; Stuttering + (and + (= currentTerm_5 currentTerm_4) + (= state_5 state_4) + (= votedFor_5 votedFor_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= nextIndex_5 nextIndex_4) + (= matchIndex_5 matchIndex_4) + (= votesGranted_5 votesGranted_4) + ) + ) +) +(assert ; Step 5 → 6 + (or + (and ; Action: RequestVote + (= state_5 Candidate) + (>= currentTerm_5 lastLogTerm) + (= votesGranted_6 (store votesGranted_5 candidate true)) + (= currentTerm_6 currentTerm_5) + (= state_6 state_5) + (= votedFor_6 votedFor_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= nextIndex_6 nextIndex_5) + (= matchIndex_6 matchIndex_5) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_5 candidateTerm) + (= votedFor_6 Some(candidate)) + (= currentTerm_6 currentTerm_5) + (= state_6 state_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= nextIndex_6 nextIndex_5) + (= matchIndex_6 matchIndex_5) + (= votesGranted_6 votesGranted_5) + ) + (and ; Action: BecomeLeader + (= state_5 Candidate) + votesGranted_size() >= 3 + (= state_6 Leader) + (= commitIndex_6 (speckl_len_Int log)) + (= currentTerm_6 currentTerm_5) + (= votedFor_6 votedFor_5) + (= log_6 log_5) + (= nextIndex_6 nextIndex_5) + (= matchIndex_6 matchIndex_5) + (= votesGranted_6 votesGranted_5) + ) + (and ; Action: AppendEntries + (= state_5 Leader) + (= log_6 append(log_5, newEntry)) + (= currentTerm_6 currentTerm_5) + (= state_6 state_5) + (= votedFor_6 votedFor_5) + (= commitIndex_6 commitIndex_5) + (= nextIndex_6 nextIndex_5) + (= matchIndex_6 matchIndex_5) + (= votesGranted_6 votesGranted_5) + ) + ; Stuttering + (and + (= currentTerm_6 currentTerm_5) + (= state_6 state_5) + (= votedFor_6 votedFor_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= nextIndex_6 nextIndex_5) + (= matchIndex_6 matchIndex_5) + (= votesGranted_6 votesGranted_5) + ) + ) +) +(assert ; Step 6 → 7 + (or + (and ; Action: RequestVote + (= state_6 Candidate) + (>= currentTerm_6 lastLogTerm) + (= votesGranted_7 (store votesGranted_6 candidate true)) + (= currentTerm_7 currentTerm_6) + (= state_7 state_6) + (= votedFor_7 votedFor_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= nextIndex_7 nextIndex_6) + (= matchIndex_7 matchIndex_6) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_6 candidateTerm) + (= votedFor_7 Some(candidate)) + (= currentTerm_7 currentTerm_6) + (= state_7 state_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= nextIndex_7 nextIndex_6) + (= matchIndex_7 matchIndex_6) + (= votesGranted_7 votesGranted_6) + ) + (and ; Action: BecomeLeader + (= state_6 Candidate) + votesGranted_size() >= 3 + (= state_7 Leader) + (= commitIndex_7 (speckl_len_Int log)) + (= currentTerm_7 currentTerm_6) + (= votedFor_7 votedFor_6) + (= log_7 log_6) + (= nextIndex_7 nextIndex_6) + (= matchIndex_7 matchIndex_6) + (= votesGranted_7 votesGranted_6) + ) + (and ; Action: AppendEntries + (= state_6 Leader) + (= log_7 append(log_6, newEntry)) + (= currentTerm_7 currentTerm_6) + (= state_7 state_6) + (= votedFor_7 votedFor_6) + (= commitIndex_7 commitIndex_6) + (= nextIndex_7 nextIndex_6) + (= matchIndex_7 matchIndex_6) + (= votesGranted_7 votesGranted_6) + ) + ; Stuttering + (and + (= currentTerm_7 currentTerm_6) + (= state_7 state_6) + (= votedFor_7 votedFor_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= nextIndex_7 nextIndex_6) + (= matchIndex_7 matchIndex_6) + (= votesGranted_7 votesGranted_6) + ) + ) +) +(assert ; Step 7 → 8 + (or + (and ; Action: RequestVote + (= state_7 Candidate) + (>= currentTerm_7 lastLogTerm) + (= votesGranted_8 (store votesGranted_7 candidate true)) + (= currentTerm_8 currentTerm_7) + (= state_8 state_7) + (= votedFor_8 votedFor_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= nextIndex_8 nextIndex_7) + (= matchIndex_8 matchIndex_7) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_7 candidateTerm) + (= votedFor_8 Some(candidate)) + (= currentTerm_8 currentTerm_7) + (= state_8 state_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= nextIndex_8 nextIndex_7) + (= matchIndex_8 matchIndex_7) + (= votesGranted_8 votesGranted_7) + ) + (and ; Action: BecomeLeader + (= state_7 Candidate) + votesGranted_size() >= 3 + (= state_8 Leader) + (= commitIndex_8 (speckl_len_Int log)) + (= currentTerm_8 currentTerm_7) + (= votedFor_8 votedFor_7) + (= log_8 log_7) + (= nextIndex_8 nextIndex_7) + (= matchIndex_8 matchIndex_7) + (= votesGranted_8 votesGranted_7) + ) + (and ; Action: AppendEntries + (= state_7 Leader) + (= log_8 append(log_7, newEntry)) + (= currentTerm_8 currentTerm_7) + (= state_8 state_7) + (= votedFor_8 votedFor_7) + (= commitIndex_8 commitIndex_7) + (= nextIndex_8 nextIndex_7) + (= matchIndex_8 matchIndex_7) + (= votesGranted_8 votesGranted_7) + ) + ; Stuttering + (and + (= currentTerm_8 currentTerm_7) + (= state_8 state_7) + (= votedFor_8 votedFor_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= nextIndex_8 nextIndex_7) + (= matchIndex_8 matchIndex_7) + (= votesGranted_8 votesGranted_7) + ) + ) +) +(assert ; Step 8 → 9 + (or + (and ; Action: RequestVote + (= state_8 Candidate) + (>= currentTerm_8 lastLogTerm) + (= votesGranted_9 (store votesGranted_8 candidate true)) + (= currentTerm_9 currentTerm_8) + (= state_9 state_8) + (= votedFor_9 votedFor_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= nextIndex_9 nextIndex_8) + (= matchIndex_9 matchIndex_8) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_8 candidateTerm) + (= votedFor_9 Some(candidate)) + (= currentTerm_9 currentTerm_8) + (= state_9 state_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= nextIndex_9 nextIndex_8) + (= matchIndex_9 matchIndex_8) + (= votesGranted_9 votesGranted_8) + ) + (and ; Action: BecomeLeader + (= state_8 Candidate) + votesGranted_size() >= 3 + (= state_9 Leader) + (= commitIndex_9 (speckl_len_Int log)) + (= currentTerm_9 currentTerm_8) + (= votedFor_9 votedFor_8) + (= log_9 log_8) + (= nextIndex_9 nextIndex_8) + (= matchIndex_9 matchIndex_8) + (= votesGranted_9 votesGranted_8) + ) + (and ; Action: AppendEntries + (= state_8 Leader) + (= log_9 append(log_8, newEntry)) + (= currentTerm_9 currentTerm_8) + (= state_9 state_8) + (= votedFor_9 votedFor_8) + (= commitIndex_9 commitIndex_8) + (= nextIndex_9 nextIndex_8) + (= matchIndex_9 matchIndex_8) + (= votesGranted_9 votesGranted_8) + ) + ; Stuttering + (and + (= currentTerm_9 currentTerm_8) + (= state_9 state_8) + (= votedFor_9 votedFor_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= nextIndex_9 nextIndex_8) + (= matchIndex_9 matchIndex_8) + (= votesGranted_9 votesGranted_8) + ) + ) +) +(assert ; Step 9 → 10 + (or + (and ; Action: RequestVote + (= state_9 Candidate) + (>= currentTerm_9 lastLogTerm) + (= votesGranted_10 (store votesGranted_9 candidate true)) + (= currentTerm_10 currentTerm_9) + (= state_10 state_9) + (= votedFor_10 votedFor_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= nextIndex_10 nextIndex_9) + (= matchIndex_10 matchIndex_9) + ) + (and ; Action: HandleRequestVoteRequest + (>= currentTerm_9 candidateTerm) + (= votedFor_10 Some(candidate)) + (= currentTerm_10 currentTerm_9) + (= state_10 state_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= nextIndex_10 nextIndex_9) + (= matchIndex_10 matchIndex_9) + (= votesGranted_10 votesGranted_9) + ) + (and ; Action: BecomeLeader + (= state_9 Candidate) + votesGranted_size() >= 3 + (= state_10 Leader) + (= commitIndex_10 (speckl_len_Int log)) + (= currentTerm_10 currentTerm_9) + (= votedFor_10 votedFor_9) + (= log_10 log_9) + (= nextIndex_10 nextIndex_9) + (= matchIndex_10 matchIndex_9) + (= votesGranted_10 votesGranted_9) + ) + (and ; Action: AppendEntries + (= state_9 Leader) + (= log_10 append(log_9, newEntry)) + (= currentTerm_10 currentTerm_9) + (= state_10 state_9) + (= votedFor_10 votedFor_9) + (= commitIndex_10 commitIndex_9) + (= nextIndex_10 nextIndex_9) + (= matchIndex_10 matchIndex_9) + (= votesGranted_10 votesGranted_9) + ) + ; Stuttering + (and + (= currentTerm_10 currentTerm_9) + (= state_10 state_9) + (= votedFor_10 votedFor_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= nextIndex_10 nextIndex_9) + (= matchIndex_10 matchIndex_9) + (= votesGranted_10 votesGranted_9) + ) + ) +) + +; Property assertion: not(invariant at some step) +(assert + (or + (not (LeaderCompleteness_0)) + (not (LeaderCompleteness_1)) + (not (LeaderCompleteness_2)) + (not (LeaderCompleteness_3)) + (not (LeaderCompleteness_4)) + (not (LeaderCompleteness_5)) + (not (LeaderCompleteness_6)) + (not (LeaderCompleteness_7)) + (not (LeaderCompleteness_8)) + (not (LeaderCompleteness_9)) + (not (LeaderCompleteness_10)) + ) +) +(define-fun LeaderCompleteness_0 () Bool + (and + commitIndex_0 <= (speckl_len_Int log) + ) +) +(define-fun LeaderCompleteness_1 () Bool + (and + commitIndex_1 <= (speckl_len_Int log) + ) +) +(define-fun LeaderCompleteness_2 () Bool + (and + commitIndex_2 <= (speckl_len_Int log) + ) +) +(define-fun LeaderCompleteness_3 () Bool + (and + commitIndex_3 <= (speckl_len_Int log) + ) +) +(define-fun LeaderCompleteness_4 () Bool + (and + commitIndex_4 <= (speckl_len_Int log) + ) +) +(define-fun LeaderCompleteness_5 () Bool + (and + commitIndex_5 <= (speckl_len_Int log) + ) +) +(define-fun LeaderCompleteness_6 () Bool + (and + commitIndex_6 <= (speckl_len_Int log) + ) +) +(define-fun LeaderCompleteness_7 () Bool + (and + commitIndex_7 <= (speckl_len_Int log) + ) +) +(define-fun LeaderCompleteness_8 () Bool + (and + commitIndex_8 <= (speckl_len_Int log) + ) +) +(define-fun LeaderCompleteness_9 () Bool + (and + commitIndex_9 <= (speckl_len_Int log) + ) +) +(define-fun LeaderCompleteness_10 () Bool + (and + commitIndex_10 <= (speckl_len_Int log) + ) +) + +(echo "Checking: Always(LeaderCompleteness) for 10 steps") +(check-sat) +(get-model) diff --git a/out/KafkaKRaft.specbom.cdx.json b/out/KafkaKRaft.specbom.cdx.json new file mode 100644 index 0000000..4d5dd9d --- /dev/null +++ b/out/KafkaKRaft.specbom.cdx.json @@ -0,0 +1,60 @@ +{ + "bomFormat": "CycloneDX", + "specVersion": "1.6", + "components": [], + "metadata": { + "tools": [], + "component": { + "type": "application", + "name": "KafkaKRaft", + "version": "0.2.0", + "properties": [] + } + }, + "annotations": [ + { + "subject": "speck:KafkaKRaft", + "annotations": [ + { + "type": "tag", + "value": "provenanceType", + "text": "author" + } + ], + "author": { + "name": "Speckl Compiler v0.3" + }, + "timestamp": "2026-05-10T09:08:14.915Z" + }, + { + "subject": "speck:KafkaKRaft", + "annotations": [ + { + "type": "tag", + "value": "provenanceType", + "text": "source" + } + ], + "author": { + "name": "Speckl Compiler", + "email": "compiler@speckl.scoble.me" + }, + "timestamp": "2026-05-10T09:08:14.915Z" + }, + { + "subject": "speck:KafkaKRaft", + "annotations": [ + { + "type": "tag", + "value": "reviewType", + "text": "hybrid" + } + ], + "author": { + "name": "Speckl Compiler", + "email": "compiler@speckl.scoble.me" + }, + "timestamp": "2026-05-10T09:08:14.915Z" + } + ] +} \ No newline at end of file diff --git a/out/KafkaKRaft.specbom.spdx.json b/out/KafkaKRaft.specbom.spdx.json new file mode 100644 index 0000000..a758550 --- /dev/null +++ b/out/KafkaKRaft.specbom.spdx.json @@ -0,0 +1,69 @@ +{ + "@context": [ + "https://spdx.github.io/spdx-v3/spdx-context.jsonld", + { + "speckl": "https://speckl.scoble.me/ns/v0.2#" + } + ], + "@type": "spdx:Document", + "spdx": { + "@id": "spdx:KafkaKRaft", + "@type": "spdx:Snippet", + "name": "KafkaKRaft", + "spdxVersion": "3.0.1", + "dataLicense": "CC0-1.0", + "creationInfo": { + "created": "2026-05-10T09:08:14.915Z", + "createdBy": [ + "Organization: Speckl", + "tool: speckl-compile-0.2.0" + ] + }, + "annotations": [ + { + "@type": "spdx:Annotation", + "annotationType": "spdx:REVIEW", + "annotator": { + "@type": "spdx:Tool", + "name": "speckl-compile" + }, + "timestamp": "2026-05-10T09:08:14.915Z", + "comment": "Review type: hybrid" + } + ] + }, + "builtBy": [], + "relationships": [ + { + "from": { + "@id": "spdx:KafkaKRaft" + }, + "to": { + "@id": "spdx:agent:Speckl Compiler v0.3" + }, + "relationshipType": "spdx:wasAttributedTo" + }, + { + "from": { + "@id": "spdx:KafkaKRaft" + }, + "to": { + "@id": "spdx:source:architecture_review:KRaft-Core-Consensus" + }, + "relationshipType": "spdx:wasInformedBy" + } + ], + "elements": [ + { + "@type": "spdx:Person", + "@id": "spdx:agent:Speckl Compiler v0.3", + "name": "Speckl Compiler v0.3" + }, + { + "@type": "spdx:Activity", + "@id": "spdx:source:architecture_review:KRaft-Core-Consensus", + "name": "architecture_review - KRaft-Core-Consensus", + "type": "architecture_review" + } + ] +} \ No newline at end of file diff --git a/out/KafkaKRaft.ts b/out/KafkaKRaft.ts new file mode 100644 index 0000000..6283ee3 --- /dev/null +++ b/out/KafkaKRaft.ts @@ -0,0 +1,59 @@ +// Auto-generated by speckl-compile from KafkaKRaft +// DO NOT EDIT MANUALLY — regenerate from .speckdl source + +export const KafkaKRaftState = { + Candidate: 'candidate', + Leader: 'leader', +} as const; +export type KafkaKRaftState = typeof KafkaKRaftState[keyof typeof KafkaKRaftState]; + + + + +export class KafkaKRaftMachine { + currentTerm: number; + state: KafkaKRaftState; + votedFor: Option(NodeId); + log: LogEntry[]; + commitIndex: number; + nextIndex: Map; + matchIndex: Map; + votesGranted: Set; + decisionLog: any[] = []; + + constructor() { + // No init block defined + } + + RequestVote(candidate: NodeId, lastLogTerm: number, lastLogIndex: number) { + if (!(this.state == KafkaKRaftState.Candidate)) throw new Error('Guard failed: this.state == KafkaKRaftState.Candidate'); + if (!(this.currentTerm >= lastLogTerm)) throw new Error('Guard failed: this.currentTerm >= lastLogTerm'); + this.votesGranted = new Set([...this.votesGranted, candidate]); + } + + HandleRequestVoteRequest(candidate: NodeId, candidateTerm: number, lastLogTerm: number, lastLogIndex: number) { + if (!(this.currentTerm >= candidateTerm)) throw new Error('Guard failed: this.currentTerm >= candidateTerm'); + this.votedFor = Some(candidate); + } + + BecomeLeader() { + if (!(this.state == KafkaKRaftState.Candidate)) throw new Error('Guard failed: this.state == KafkaKRaftState.Candidate'); + if (!(this.votesGranted.size >= 3)) throw new Error('Guard failed: this.votesGranted.size >= 3'); + this.state = KafkaKRaftState.Leader; + this.commitIndex = length(this.log); + } + + AppendEntries(follower: NodeId) { + if (!(this.state == KafkaKRaftState.Leader)) throw new Error('Guard failed: this.state == KafkaKRaftState.Leader'); + this.log = append(this.log, newEntry); + } + + + + // V1: Always(TermMonotonicity) + async verify1(): Promise { + // TODO: Model-check or monitor + // Temporal: Always(TermMonotonicity) + return true; + } +} diff --git a/out/KafkaKRaft.wat b/out/KafkaKRaft.wat new file mode 100644 index 0000000..b1bd4b7 --- /dev/null +++ b/out/KafkaKRaft.wat @@ -0,0 +1,124 @@ +;; Auto-generated by speckl-compile from KafkaKRaft +;; Compile: wat2wasm KafkaKRaft.wat -o KafkaKRaft.wasm + +(module + ;; Host imports for Map/Set/alloc + (import "speckl" "map_create" (func $map_create (result i32))) + (import "speckl" "map_set" (func $map_set (param i32 i32 i32))) + (import "speckl" "map_get" (func $map_get (param i32 i32) (result i32))) + (import "speckl" "map_has" (func $map_has (param i32 i32) (result i32))) + (import "speckl" "map_remove" (func $map_remove (param i32 i32))) + (import "speckl" "set_create" (func $set_create (result i32))) + (import "speckl" "set_add" (func $set_add (param i32 i32))) + (import "speckl" "set_has" (func $set_has (param i32 i32) (result i32))) + (import "speckl" "set_size" (func $set_size (param i32) (result i32))) + (import "speckl" "alloc" (func $alloc (param i32) (result i32))) + (import "speckl" "write_i32" (func $write_i32 (param i32 i32))) + (import "speckl" "write_f64" (func $write_f64 (param i32 f64))) + + ;; Linear memory (exported so host can read state) + (memory (export "memory") 1) + + ;; State globals + (global $currentTerm (mut i32) (i32.const 0)) +(global $state (mut i32) (i32.const 0)) +(global $votedFor (mut i32) (i32.const 0)) +(global $log (mut i32) (i32.const 0)) +(global $commitIndex (mut i32) (i32.const 0)) +(global $nextIndex (mut i32) (i32.const 0)) +(global $matchIndex (mut i32) (i32.const 0)) +(global $votesGranted (mut i32) (i32.const 0)) + + ;; ---- init ---- + (func $init (export "init") + (i32.const 0) + (global.set $currentTerm) + (i32.const 0) + (global.set $state) + (i32.const 0) + (global.set $votedFor) + (i32.const 0) + (global.set $commitIndex) + (call $map_create) + (global.set $nextIndex) + (call $map_create) + (global.set $matchIndex) + (call $set_create) + (global.set $votesGranted) + ) + + ;; ---- Actions ---- + ;; action: RequestVote + (func $RequestVote (export "RequestVote") (param $candidate i32) (param $lastLogTerm i32) (param $lastLogIndex i32) (result i32) + + ;; require: state == Candidate + ;; Complex right-hand side not compiled: state == Candidate + ;; require: currentTerm >= lastLogTerm + ;; Complex right-hand side not compiled: currentTerm >= lastLogTerm + ;; votesGranted := votesGranted union {candidate} + ;; Unrecognized assignment target: votesGranted + (i32.const 1) + ) + + ;; action: HandleRequestVoteRequest + (func $HandleRequestVoteRequest (export "HandleRequestVoteRequest") (param $candidate i32) (param $candidateTerm i32) (param $lastLogTerm i32) (param $lastLogIndex i32) (result i32) + + ;; require: currentTerm >= candidateTerm + ;; Complex right-hand side not compiled: currentTerm >= candidateTerm + ;; votedFor := Some(candidate) + (i32.const 0) ;; uncompiled expr: Some(candidate) + (global.set $votedFor) + (i32.const 1) + ) + + ;; action: BecomeLeader + (func $BecomeLeader (export "BecomeLeader") (result i32) + + ;; require: state == Candidate + ;; Complex right-hand side not compiled: state == Candidate + ;; require: votesGranted.size() >= 3 + ;; Complex guard not compiled: votesGranted.size() >= 3 + ;; state := Leader + (i32.const 0) ;; uncompiled expr: Leader + (global.set $state) + ;; commitIndex := length(log) + (i32.const 0) ;; uncompiled expr: length(log) + (global.set $commitIndex) + (i32.const 1) + ) + + ;; action: AppendEntries + (func $AppendEntries (export "AppendEntries") (param $follower i32) (result i32) + + ;; require: state == Leader + ;; Complex right-hand side not compiled: state == Leader + ;; log := append(log, newEntry) + ;; Unrecognized assignment target: log + (i32.const 1) + ) + + ;; ---- getState ---- + (func $getState (export "getState") (result i32 i32) + (i32.const 0) + (global.get $currentTerm) + (i32.store) + (i32.const 4) + (global.get $state) + (i32.store) + (i32.const 8) + (global.get $votedFor) + (i32.store) + (i32.const 12) + (global.get $commitIndex) + (i32.store) + (i32.const 0) + (i32.const 16) + ) + + + ;; checkConstraints — runtime constraint evaluation stub + (func $checkConstraints (export "checkConstraints") (result i32) + ;; TODO: evaluate constraint expressions at runtime + (i32.const 1) + ) +) \ No newline at end of file diff --git a/out/OAuth2AuthorizationCode.prov.jsonld b/out/OAuth2AuthorizationCode.prov.jsonld new file mode 100644 index 0000000..c646bc1 --- /dev/null +++ b/out/OAuth2AuthorizationCode.prov.jsonld @@ -0,0 +1,21 @@ +{ + "@context": { + "prov": "http://www.w3.org/ns/prov#", + "speckl": "https://speckl.scoble.me/ns/v0.2#", + "rdfs": "http://www.w3.org/2000/01/rdf-schema#" + }, + "@graph": [ + { + "@type": "prov:Entity", + "@id": "speck:OAuth2AuthorizationCode", + "prov:qualifiedGeneration": { + "@type": "prov:Generation", + "prov:activity": { + "@type": "prov:Activity", + "prov:startTime": "2026-05-10T09:08:15.017Z" + } + }, + "rdfs:label": "OAuth2AuthorizationCode" + } + ] +} \ No newline at end of file diff --git a/out/OAuth2AuthorizationCode.specbom.cdx.json b/out/OAuth2AuthorizationCode.specbom.cdx.json new file mode 100644 index 0000000..1d6ddc4 --- /dev/null +++ b/out/OAuth2AuthorizationCode.specbom.cdx.json @@ -0,0 +1,15 @@ +{ + "bomFormat": "CycloneDX", + "specVersion": "1.6", + "components": [], + "metadata": { + "tools": [], + "component": { + "type": "application", + "name": "OAuth2AuthorizationCode", + "version": "0.2.0", + "properties": [] + } + }, + "annotations": [] +} \ No newline at end of file diff --git a/out/OAuth2AuthorizationCode.specbom.spdx.json b/out/OAuth2AuthorizationCode.specbom.spdx.json new file mode 100644 index 0000000..0d8c2d3 --- /dev/null +++ b/out/OAuth2AuthorizationCode.specbom.spdx.json @@ -0,0 +1,26 @@ +{ + "@context": [ + "https://spdx.github.io/spdx-v3/spdx-context.jsonld", + { + "speckl": "https://speckl.scoble.me/ns/v0.2#" + } + ], + "@type": "spdx:Document", + "spdx": { + "@id": "spdx:OAuth2AuthorizationCode", + "@type": "spdx:Snippet", + "name": "OAuth2AuthorizationCode", + "spdxVersion": "3.0.1", + "dataLicense": "CC0-1.0", + "creationInfo": { + "created": "2026-05-10T09:08:15.086Z", + "createdBy": [ + "Organization: Speckl", + "tool: speckl-compile-0.2.0" + ] + } + }, + "builtBy": [], + "relationships": [], + "elements": [] +} \ No newline at end of file diff --git a/out/OAuth2AuthorizationCode.ts b/out/OAuth2AuthorizationCode.ts new file mode 100644 index 0000000..ac315b6 --- /dev/null +++ b/out/OAuth2AuthorizationCode.ts @@ -0,0 +1,114 @@ +// Auto-generated by speckl-compile from OAuth2AuthorizationCode +// DO NOT EDIT MANUALLY — regenerate from .speckdl source + +export const OAuth2AuthorizationCodeState = { + Open: 'open', + Closed: 'closed', +} as const; +export type OAuth2AuthorizationCodeState = typeof OAuth2AuthorizationCodeState[keyof typeof OAuth2AuthorizationCodeState]; + + + + +export class OAuth2AuthorizationCodeMachine { + clients: Map; client_secret: ClientSecret }>; + authorizationCodes: Map(AuthorizationCode, {; + client_id: ClientId; + redirect_uri: RedirectUri; + state: OAuth2AuthorizationCodeState; + code_challenge: CodeChallenge; + code_challenge_method: String; + scope: Scope; + user_id: UserId; + session_id: SessionId; + created_at: number; + used: boolean; + accessTokens: Map(AccessToken, {; + client_id: ClientId; + user_id: UserId; + scope: Scope; + expires_at: number; + session_id: SessionId; + refresh_token: RefreshToken; + refreshTokens: Map(RefreshToken, {; + client_id: ClientId; + user_id: UserId; + scope: Scope; + access_token: AccessToken; + created_at: number; + sessions: Map; + stateNonces: Map; + pkceChallenges: Map; + currentTime: number; + decisionLog: any[] = []; + + constructor() { + // No init block defined + } + + Authorize(client: ClientId, redirectUri: RedirectUri, scope: Scope, userId: UserId, sessionId: SessionId) { + if (!(this.clients.has(client))) throw new Error('Guard failed: this.clients.has(client)'); + if (!(this.clients.client.redirect_uris.has(redirectUri))) throw new Error('Guard failed: this.clients.client.redirect_uris.has(redirectUri)'); + if (!(this.sessions.has(sessionId))) throw new Error('Guard failed: this.sessions.has(sessionId)'); + if (!(this.sessions.get(sessionId).user_id == userId)) throw new Error('Guard failed: this.sessions.get(sessionId).user_id == userId'); + this.stateNonces[state] = { this.client_id: client, this.redirect_uri: redirectUri }; + this.pkceChallenges[codeChallenge] = { code_verifier_hash: hash(codeVerifier, "S256"), method: "S256" }; + this.authorizationCodes[authCode] = {; + } + + ApproveAuthorization(code: AuthorizationCode, userId: UserId) { + if (!(this.authorizationCodes.has(code))) throw new Error('Guard failed: this.authorizationCodes.has(code)'); + if (!(c.this.user_id == userId)) throw new Error('Guard failed: c.this.user_id == userId'); + if (!(c.this.used == false)) throw new Error('Guard failed: c.this.used == false'); + } + + ExchangeCode(req: TokenRequest) { + if (!(this.clients.has(req.this.client_id))) throw new Error('Guard failed: this.clients.has(req.this.client_id)'); + if (!(this.authorizationCodes.has(req.code))) throw new Error('Guard failed: this.authorizationCodes.has(req.code)'); + if (!(c.this.used == false)) throw new Error('Guard failed: c.this.used == false'); + if (!(c.this.client_id == req.this.client_id)) throw new Error('Guard failed: c.this.client_id == req.this.client_id'); + if (!(c.this.redirect_uri == req.this.redirect_uri)) throw new Error('Guard failed: c.this.redirect_uri == req.this.redirect_uri'); + if (!(expectedChallenge == c.this.code_challenge)) throw new Error('Guard failed: expectedChallenge == c.this.code_challenge'); + this.pkceChallenges[c.code_challenge] = null; + this.stateNonces[c.state] = null; + this.accessTokens[accessToken] = {; + this.refreshTokens[refreshToken] = {; + } + + RefreshToken(req: RefreshRequest) { + if (!(this.clients.has(req.this.client_id))) throw new Error('Guard failed: this.clients.has(req.this.client_id)'); + if (!(this.refreshTokens.has(req.this.refresh_token))) throw new Error('Guard failed: this.refreshTokens.has(req.this.refresh_token)'); + if (!(r.this.client_id == req.this.client_id)) throw new Error('Guard failed: r.this.client_id == req.this.client_id'); + this.accessTokens[r.access_token] = null; + this.accessTokens[newAccessToken] = {; + this.refreshTokens[newRefreshToken] = {; + this.refreshTokens[req.refresh_token] = null; + } + + RevokeToken(client: ClientId, token: string, tokenType: string) { + if (!(this.clients.has(client) if tokenType == "access" {)) throw new Error('Guard failed: this.clients.has(client) if tokenType == "access" {'); + if (!(this.accessTokens.has(token))) throw new Error('Guard failed: this.accessTokens.has(token)'); + if (!(t.this.client_id == client)) throw new Error('Guard failed: t.this.client_id == client'); + if (!(this.refreshTokens.has(token))) throw new Error('Guard failed: this.refreshTokens.has(token)'); + if (!(r.this.client_id == client)) throw new Error('Guard failed: r.this.client_id == client'); + this.accessTokens[token] = null; + this.refreshTokens[t.refresh_token] = null; + this.refreshTokens[token] = null; + this.accessTokens[r.access_token] = null; + } + + VerifyPKCE(code: AuthorizationCode, verifier: CodeVerifier) { + if (!(this.authorizationCodes.has(code))) throw new Error('Guard failed: this.authorizationCodes.has(code)'); + if (!(c.this.used == false)) throw new Error('Guard failed: c.this.used == false'); + if (!(expectedChallenge == c.this.code_challenge)) throw new Error('Guard failed: expectedChallenge == c.this.code_challenge'); + } + + + + // V1: Always(NoTokenLeakage) + async verify1(): Promise { + // TODO: Model-check or monitor + // Temporal: Always(NoTokenLeakage) + return true; + } +} diff --git a/out/OAuth2AuthorizationCode.wat b/out/OAuth2AuthorizationCode.wat new file mode 100644 index 0000000..844f3fc --- /dev/null +++ b/out/OAuth2AuthorizationCode.wat @@ -0,0 +1,211 @@ +;; Auto-generated by speckl-compile from OAuth2AuthorizationCode +;; Compile: wat2wasm OAuth2AuthorizationCode.wat -o OAuth2AuthorizationCode.wasm + +(module + ;; Host imports for Map/Set/alloc + (import "speckl" "map_create" (func $map_create (result i32))) + (import "speckl" "map_set" (func $map_set (param i32 i32 i32))) + (import "speckl" "map_get" (func $map_get (param i32 i32) (result i32))) + (import "speckl" "map_has" (func $map_has (param i32 i32) (result i32))) + (import "speckl" "map_remove" (func $map_remove (param i32 i32))) + (import "speckl" "set_create" (func $set_create (result i32))) + (import "speckl" "set_add" (func $set_add (param i32 i32))) + (import "speckl" "set_has" (func $set_has (param i32 i32) (result i32))) + (import "speckl" "set_size" (func $set_size (param i32) (result i32))) + (import "speckl" "alloc" (func $alloc (param i32) (result i32))) + (import "speckl" "write_i32" (func $write_i32 (param i32 i32))) + (import "speckl" "write_f64" (func $write_f64 (param i32 f64))) + + ;; Linear memory (exported so host can read state) + (memory (export "memory") 1) + + ;; State globals + + + ;; ---- init ---- + (func $init (export "init") + + ) + + ;; ---- Actions ---- + ;; action: Authorize + (func $Authorize (export "Authorize") (param $client i32) (param $redirectUri i32) (param $scope i32) (param $userId i32) (param $sessionId i32) (result i32) + + (local $state i32) + (local $codeVerifier i32) + (local $codeChallenge i32) + (local $authCode i32) + ;; require: client in clients.keys + ;; Complex guard not compiled: client in clients.keys + ;; require: redirectUri in clients.client.redirect_uris + ;; Complex guard not compiled: redirectUri in clients.client.redirect_uris + ;; require: sessionId in sessions.keys + ;; Complex guard not compiled: sessionId in sessions.keys + ;; require: sessions[sessionId].user_id == userId + ;; Complex guard not compiled: sessions[sessionId].user_id == userId + (i32.const 0) ;; uncompiled expr: "state_" + hash(client + redirectUri + currentTime) + (local.set $state) + (i32.const 0) ;; uncompiled expr: "verifier_" + hash(client + userId + currentTime) + (local.set $codeVerifier) + (i32.const 0) ;; uncompiled expr: hash(codeVerifier, "S256") + (local.set $codeChallenge) + ;; stateNonces[state] := { client_id: client, redirect_uri: redirectUri } + ;; Unrecognized assignment target: stateNonces[state] + ;; pkceChallenges[codeChallenge] := { code_verifier_hash: hash(codeVerifier, "S256"), method: "S256" } + ;; Unrecognized assignment target: pkceChallenges[codeChallenge] + (i32.const 0) ;; uncompiled expr: "code_" + hash(client + userId + currentTime) + (local.set $authCode) + ;; authorizationCodes[authCode] := { + ;; Unrecognized assignment target: authorizationCodes[authCode] + (i32.const 1) + ) + + ;; action: ApproveAuthorization + (func $ApproveAuthorization (export "ApproveAuthorization") (param $code i32) (param $userId i32) (result i32) + + (local $c i32) + ;; require: code in authorizationCodes.keys + ;; Complex guard not compiled: code in authorizationCodes.keys + (i32.const 0) ;; uncompiled expr: authorizationCodes[code] + (local.set $c) + ;; require: c.user_id == userId + ;; Complex guard not compiled: c.user_id == userId + ;; require: c.used == false + ;; Complex guard not compiled: c.used == false + (i32.const 1) + ) + + ;; action: ExchangeCode + (func $ExchangeCode (export "ExchangeCode") (param $req i32) (result i32) + + (local $c i32) + (local $expectedChallenge i32) + (local $accessToken i32) + (local $refreshToken i32) + (local $expiresAt i32) + ;; require: req.client_id in clients.keys + ;; Complex guard not compiled: req.client_id in clients.keys + ;; require: req.code in authorizationCodes.keys + ;; Complex guard not compiled: req.code in authorizationCodes.keys + (i32.const 0) ;; uncompiled expr: authorizationCodes[req.code] + (local.set $c) + ;; require: c.used == false + ;; Complex guard not compiled: c.used == false + ;; require: c.client_id == req.client_id + ;; Complex guard not compiled: c.client_id == req.client_id + ;; require: c.redirect_uri == req.redirect_uri + ;; Complex guard not compiled: c.redirect_uri == req.redirect_uri + (i32.const 0) ;; uncompiled expr: hash(req.code_verifier, c.code_challenge_method) + (local.set $expectedChallenge) + ;; require: expectedChallenge == c.code_challenge + ;; Guard on non-scalar not compiled: expectedChallenge == c.code_challenge + ;; pkceChallenges[c.code_challenge] := null + ;; Unrecognized assignment target: pkceChallenges[c.code_challenge] + ;; stateNonces[c.state] := null + ;; Unrecognized assignment target: stateNonces[c.state] + (i32.const 0) ;; uncompiled expr: "access_" + hash(req.client_id + c.user_id + currentTime) + (local.set $accessToken) + (i32.const 0) ;; uncompiled expr: "refresh_" + hash(req.client_id + c.user_id + currentTime) + (local.set $refreshToken) + (i32.add (i32.const NaN) (i32.const 3600)) + (local.set $expiresAt) + ;; accessTokens[accessToken] := { + ;; Unrecognized assignment target: accessTokens[accessToken] + ;; refreshTokens[refreshToken] := { + ;; Unrecognized assignment target: refreshTokens[refreshToken] + (i32.const 1) + ) + + ;; action: RefreshToken + (func $RefreshToken (export "RefreshToken") (param $req i32) (result i32) + + (local $r i32) + (local $newAccessToken i32) + (local $newRefreshToken i32) + (local $expiresAt i32) + ;; require: req.client_id in clients.keys + ;; Complex guard not compiled: req.client_id in clients.keys + ;; require: req.refresh_token in refreshTokens.keys + ;; Complex guard not compiled: req.refresh_token in refreshTokens.keys + (i32.const 0) ;; uncompiled expr: refreshTokens[req.refresh_token] + (local.set $r) + ;; require: r.client_id == req.client_id + ;; Complex guard not compiled: r.client_id == req.client_id + ;; accessTokens[r.access_token] := null + ;; Unrecognized assignment target: accessTokens[r.access_token] + (i32.const 0) ;; uncompiled expr: "access_" + hash(req.client_id + r.user_id + currentTime) + (local.set $newAccessToken) + (i32.const 0) ;; uncompiled expr: "refresh_" + hash(req.client_id + r.user_id + currentTime) + (local.set $newRefreshToken) + (i32.add (i32.const NaN) (i32.const 3600)) + (local.set $expiresAt) + ;; accessTokens[newAccessToken] := { + ;; Unrecognized assignment target: accessTokens[newAccessToken] + ;; refreshTokens[newRefreshToken] := { + ;; Unrecognized assignment target: refreshTokens[newRefreshToken] + ;; refreshTokens[req.refresh_token] := null + ;; Unrecognized assignment target: refreshTokens[req.refresh_token] + (i32.const 1) + ) + + ;; action: RevokeToken + (func $RevokeToken (export "RevokeToken") (param $client i32) (param $token i32) (param $tokenType i32) (result i32) + + (local $t i32) + (local $r i32) + ;; require: client in clients.keys + ;; Complex guard not compiled: client in clients.keys + ;; require: token in accessTokens.keys + ;; Complex guard not compiled: token in accessTokens.keys + (i32.const 0) ;; uncompiled expr: accessTokens[token] + (local.set $t) + ;; require: t.client_id == client + ;; Complex guard not compiled: t.client_id == client + ;; accessTokens[token] := null + ;; Unrecognized assignment target: accessTokens[token] + ;; refreshTokens[t.refresh_token] := null + ;; Unrecognized assignment target: refreshTokens[t.refresh_token] + ;; require: token in refreshTokens.keys + ;; Complex guard not compiled: token in refreshTokens.keys + (i32.const 0) ;; uncompiled expr: refreshTokens[token] + (local.set $r) + ;; require: r.client_id == client + ;; Complex guard not compiled: r.client_id == client + ;; refreshTokens[token] := null + ;; Unrecognized assignment target: refreshTokens[token] + ;; accessTokens[r.access_token] := null + ;; Unrecognized assignment target: accessTokens[r.access_token] + (i32.const 1) + ) + + ;; action: VerifyPKCE + (func $VerifyPKCE (export "VerifyPKCE") (param $code i32) (param $verifier i32) (result i32) + + (local $c i32) + (local $expectedChallenge i32) + ;; require: code in authorizationCodes.keys + ;; Complex guard not compiled: code in authorizationCodes.keys + (i32.const 0) ;; uncompiled expr: authorizationCodes[code] + (local.set $c) + ;; require: c.used == false + ;; Complex guard not compiled: c.used == false + (i32.const 0) ;; uncompiled expr: hash(verifier, c.code_challenge_method) + (local.set $expectedChallenge) + ;; require: expectedChallenge == c.code_challenge + ;; Guard on non-scalar not compiled: expectedChallenge == c.code_challenge + (i32.const 1) + ) + + ;; ---- getState ---- + (func $getState (export "getState") (result i32 i32) + (i32.const 0) + (i32.const 0) + ) + + + ;; checkConstraints — runtime constraint evaluation stub + (func $checkConstraints (export "checkConstraints") (result i32) + ;; TODO: evaluate constraint expressions at runtime + (i32.const 1) + ) +) \ No newline at end of file diff --git a/out/TigerBeetleLedger.prov.jsonld b/out/TigerBeetleLedger.prov.jsonld new file mode 100644 index 0000000..a2e00c7 --- /dev/null +++ b/out/TigerBeetleLedger.prov.jsonld @@ -0,0 +1,21 @@ +{ + "@context": { + "prov": "http://www.w3.org/ns/prov#", + "speckl": "https://speckl.scoble.me/ns/v0.2#", + "rdfs": "http://www.w3.org/2000/01/rdf-schema#" + }, + "@graph": [ + { + "@type": "prov:Entity", + "@id": "speck:TigerBeetleLedger", + "prov:qualifiedGeneration": { + "@type": "prov:Generation", + "prov:activity": { + "@type": "prov:Activity", + "prov:startTime": "2026-05-10T09:08:16.133Z" + } + }, + "rdfs:label": "TigerBeetleLedger" + } + ] +} \ No newline at end of file diff --git a/out/TigerBeetleLedger.smt2 b/out/TigerBeetleLedger.smt2 new file mode 100644 index 0000000..fddbf85 --- /dev/null +++ b/out/TigerBeetleLedger.smt2 @@ -0,0 +1,3143 @@ +; SpeckDL: TigerBeetleLedger — Z3 Verification +; Generated by speckl-compile Z3 backend +; Bounded model checking depth: 10 + +(set-logic ALL) +(set-option :produce-models true) +(declare-fun speckl_len_Int ( (Array Int Int) ) Int) +(declare-sort Account 0) +(declare-sort Transfer 0) +(declare-sort Replica 0) +(declare-sort LogEntry 0) + +; --- State Variables --- +(declare-const accounts (Array Int Account)) +(declare-const transfers (Array Int Transfer)) +(declare-const pending (Array Int Bool)) +(declare-const replicaState (Array Int Replica)) +(declare-const log (Array Int LogEntry)) +(declare-const commitIndex Int) +(declare-const leader Int) + +; --- Invariants --- +(define-fun AccountBalanceInvariant () Bool + (and + (forall ((a Account)) (- (= a_balance a_credits_posted) a_debits_posted)) + (forall ((a Account)) (>= a_debits_pending 0) (>= a_credits_pending 0)) + ) +) +(define-fun TransferStateConsistency () Bool + (and + (forall ((t Transfer)) t_pending ((=> true true))) + (forall ((t Transfer)) t_posted ((=> true true))) + (forall ((t Transfer)) t_voided ((=> true true))) + ) +) +(define-fun NoOverdraft () Bool + (and + (forall ((a Account)) (or (>= a_balance 0) a_flags.allows_overdraft)) + ) +) + +; --- Action Transition Predicates --- +(define-fun CreateTransferEnabled () Bool + (and + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts t_from).(>= balance t_amount) (select accounts t_from).flags_allows_overdraft) + ) +) +(define-fun CommitTransferEnabled () Bool + (and + (select pending id) + (select transfers_keys id) + (select transfers id).(= pending true) + ) +) +(define-fun VoidTransferEnabled () Bool + (and + (select pending id) + (select transfers_keys id) + (select transfers id).(= pending true) + ) +) +(define-fun PostTransferEnabled () Bool + (and + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts t_from).(>= balance t_amount) (select accounts t_from).flags_allows_overdraft) + ) +) +(define-fun BecomeLeaderEnabled () Bool + (and + (select replicaState_keys replica) + (select replicaState replica).(= role "candidate") + (< (select replicaState replica).currentTerm term) + ) +) +(define-fun AppendEntryEnabled () Bool + (and + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex) + ) +) +; Next transition relation: CreateTransfer | CommitTransfer | VoidTransfer | PostTransfer | BecomeLeader | AppendEntry +(define-fun Next () Bool + (or + (and (CreateTransferEnabled) + (and + (= accounts_post accounts) + (= transfers_post transfers) + (= pending_post pending) + (= replicaState_post replicaState) + (= log_post log) + (= commitIndex_post commitIndex) + (= leader_post leader) + ) + ) + (and (CommitTransferEnabled) + (and + (= accounts_post accounts) + (= transfers_post transfers) + (= pending_post pending) + (= replicaState_post replicaState) + (= log_post log) + (= commitIndex_post commitIndex) + (= leader_post leader) + ) + ) + (and (VoidTransferEnabled) + (and + (= accounts_post accounts) + (= transfers_post transfers) + (= pending_post pending) + (= replicaState_post replicaState) + (= log_post log) + (= commitIndex_post commitIndex) + (= leader_post leader) + ) + ) + (and (PostTransferEnabled) + (and + (= accounts_post accounts) + (= transfers_post transfers) + (= pending_post pending) + (= replicaState_post replicaState) + (= log_post log) + (= commitIndex_post commitIndex) + (= leader_post leader) + ) + ) + (and (BecomeLeaderEnabled) + (and + (= accounts_post accounts) + (= transfers_post transfers) + (= pending_post pending) + (= replicaState_post replicaState) + (= log_post log) + (= commitIndex_post commitIndex) + (= leader_post leader) + ) + ) + (and (AppendEntryEnabled) + (and + (= accounts_post accounts) + (= transfers_post transfers) + (= pending_post pending) + (= replicaState_post replicaState) + (= log_post log) + (= commitIndex_post commitIndex) + (= leader_post leader) + ) + ) + ; Stuttering step + (and + (= accounts_post accounts) + (= transfers_post transfers) + (= pending_post pending) + (= replicaState_post replicaState) + (= log_post log) + (= commitIndex_post commitIndex) + (= leader_post leader) + ) + ) +) + +; --- Bounded Model Checking --- +; Verification: Always(AccountBalanceInvariant) +; BMC depth: 10 + +; Declare state copies for 10 steps +(declare-const accounts_0 (Array Int Account)) +(declare-const accounts_1 (Array Int Account)) +(declare-const accounts_2 (Array Int Account)) +(declare-const accounts_3 (Array Int Account)) +(declare-const accounts_4 (Array Int Account)) +(declare-const accounts_5 (Array Int Account)) +(declare-const accounts_6 (Array Int Account)) +(declare-const accounts_7 (Array Int Account)) +(declare-const accounts_8 (Array Int Account)) +(declare-const accounts_9 (Array Int Account)) +(declare-const accounts_10 (Array Int Account)) +(declare-const transfers_0 (Array Int Transfer)) +(declare-const transfers_1 (Array Int Transfer)) +(declare-const transfers_2 (Array Int Transfer)) +(declare-const transfers_3 (Array Int Transfer)) +(declare-const transfers_4 (Array Int Transfer)) +(declare-const transfers_5 (Array Int Transfer)) +(declare-const transfers_6 (Array Int Transfer)) +(declare-const transfers_7 (Array Int Transfer)) +(declare-const transfers_8 (Array Int Transfer)) +(declare-const transfers_9 (Array Int Transfer)) +(declare-const transfers_10 (Array Int Transfer)) +(declare-const pending_0 (Array Int Bool)) +(declare-const pending_1 (Array Int Bool)) +(declare-const pending_2 (Array Int Bool)) +(declare-const pending_3 (Array Int Bool)) +(declare-const pending_4 (Array Int Bool)) +(declare-const pending_5 (Array Int Bool)) +(declare-const pending_6 (Array Int Bool)) +(declare-const pending_7 (Array Int Bool)) +(declare-const pending_8 (Array Int Bool)) +(declare-const pending_9 (Array Int Bool)) +(declare-const pending_10 (Array Int Bool)) +(declare-const replicaState_0 (Array Int Replica)) +(declare-const replicaState_1 (Array Int Replica)) +(declare-const replicaState_2 (Array Int Replica)) +(declare-const replicaState_3 (Array Int Replica)) +(declare-const replicaState_4 (Array Int Replica)) +(declare-const replicaState_5 (Array Int Replica)) +(declare-const replicaState_6 (Array Int Replica)) +(declare-const replicaState_7 (Array Int Replica)) +(declare-const replicaState_8 (Array Int Replica)) +(declare-const replicaState_9 (Array Int Replica)) +(declare-const replicaState_10 (Array Int Replica)) +(declare-const log_0 (Array Int LogEntry)) +(declare-const log_1 (Array Int LogEntry)) +(declare-const log_2 (Array Int LogEntry)) +(declare-const log_3 (Array Int LogEntry)) +(declare-const log_4 (Array Int LogEntry)) +(declare-const log_5 (Array Int LogEntry)) +(declare-const log_6 (Array Int LogEntry)) +(declare-const log_7 (Array Int LogEntry)) +(declare-const log_8 (Array Int LogEntry)) +(declare-const log_9 (Array Int LogEntry)) +(declare-const log_10 (Array Int LogEntry)) +(declare-const commitIndex_0 Int) +(declare-const commitIndex_1 Int) +(declare-const commitIndex_2 Int) +(declare-const commitIndex_3 Int) +(declare-const commitIndex_4 Int) +(declare-const commitIndex_5 Int) +(declare-const commitIndex_6 Int) +(declare-const commitIndex_7 Int) +(declare-const commitIndex_8 Int) +(declare-const commitIndex_9 Int) +(declare-const commitIndex_10 Int) +(declare-const leader_0 Int) +(declare-const leader_1 Int) +(declare-const leader_2 Int) +(declare-const leader_3 Int) +(declare-const leader_4 Int) +(declare-const leader_5 Int) +(declare-const leader_6 Int) +(declare-const leader_7 Int) +(declare-const leader_8 Int) +(declare-const leader_9 Int) +(declare-const leader_10 Int) + +; Initial state constraints +(assert + (and + (>= commitIndex_0 0) + (>= leader_0 0) + ) +) + +; Transition constraints +(assert ; Step 0 → 1 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_0 t_from).(>= balance t_amount) (select accounts_0 t_from).flags_allows_overdraft) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + (and ; Action: CommitTransfer + (select pending_0 id) + (select transfers_keys id) + (select transfers_0 id).(= pending_0 true) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + (and ; Action: VoidTransfer + (select pending_0 id) + (select transfers_keys id) + (select transfers_0 id).(= pending_0 true) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_0 t_from).(>= balance t_amount) (select accounts_0 t_from).flags_allows_overdraft) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_0 replica).(= role "candidate") + (< (select replicaState_0 replica).currentTerm term) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_0) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + ; Stuttering + (and + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + ) +) +(assert ; Step 1 → 2 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_1 t_from).(>= balance t_amount) (select accounts_1 t_from).flags_allows_overdraft) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + (and ; Action: CommitTransfer + (select pending_1 id) + (select transfers_keys id) + (select transfers_1 id).(= pending_1 true) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + (and ; Action: VoidTransfer + (select pending_1 id) + (select transfers_keys id) + (select transfers_1 id).(= pending_1 true) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_1 t_from).(>= balance t_amount) (select accounts_1 t_from).flags_allows_overdraft) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_1 replica).(= role "candidate") + (< (select replicaState_1 replica).currentTerm term) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_1) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + ; Stuttering + (and + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + ) +) +(assert ; Step 2 → 3 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_2 t_from).(>= balance t_amount) (select accounts_2 t_from).flags_allows_overdraft) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + (and ; Action: CommitTransfer + (select pending_2 id) + (select transfers_keys id) + (select transfers_2 id).(= pending_2 true) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + (and ; Action: VoidTransfer + (select pending_2 id) + (select transfers_keys id) + (select transfers_2 id).(= pending_2 true) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_2 t_from).(>= balance t_amount) (select accounts_2 t_from).flags_allows_overdraft) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_2 replica).(= role "candidate") + (< (select replicaState_2 replica).currentTerm term) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_2) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + ; Stuttering + (and + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + ) +) +(assert ; Step 3 → 4 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_3 t_from).(>= balance t_amount) (select accounts_3 t_from).flags_allows_overdraft) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + (and ; Action: CommitTransfer + (select pending_3 id) + (select transfers_keys id) + (select transfers_3 id).(= pending_3 true) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + (and ; Action: VoidTransfer + (select pending_3 id) + (select transfers_keys id) + (select transfers_3 id).(= pending_3 true) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_3 t_from).(>= balance t_amount) (select accounts_3 t_from).flags_allows_overdraft) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_3 replica).(= role "candidate") + (< (select replicaState_3 replica).currentTerm term) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_3) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + ; Stuttering + (and + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + ) +) +(assert ; Step 4 → 5 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_4 t_from).(>= balance t_amount) (select accounts_4 t_from).flags_allows_overdraft) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + (and ; Action: CommitTransfer + (select pending_4 id) + (select transfers_keys id) + (select transfers_4 id).(= pending_4 true) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + (and ; Action: VoidTransfer + (select pending_4 id) + (select transfers_keys id) + (select transfers_4 id).(= pending_4 true) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_4 t_from).(>= balance t_amount) (select accounts_4 t_from).flags_allows_overdraft) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_4 replica).(= role "candidate") + (< (select replicaState_4 replica).currentTerm term) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_4) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + ; Stuttering + (and + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + ) +) +(assert ; Step 5 → 6 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_5 t_from).(>= balance t_amount) (select accounts_5 t_from).flags_allows_overdraft) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + (and ; Action: CommitTransfer + (select pending_5 id) + (select transfers_keys id) + (select transfers_5 id).(= pending_5 true) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + (and ; Action: VoidTransfer + (select pending_5 id) + (select transfers_keys id) + (select transfers_5 id).(= pending_5 true) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_5 t_from).(>= balance t_amount) (select accounts_5 t_from).flags_allows_overdraft) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_5 replica).(= role "candidate") + (< (select replicaState_5 replica).currentTerm term) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_5) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + ; Stuttering + (and + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + ) +) +(assert ; Step 6 → 7 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_6 t_from).(>= balance t_amount) (select accounts_6 t_from).flags_allows_overdraft) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + (and ; Action: CommitTransfer + (select pending_6 id) + (select transfers_keys id) + (select transfers_6 id).(= pending_6 true) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + (and ; Action: VoidTransfer + (select pending_6 id) + (select transfers_keys id) + (select transfers_6 id).(= pending_6 true) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_6 t_from).(>= balance t_amount) (select accounts_6 t_from).flags_allows_overdraft) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_6 replica).(= role "candidate") + (< (select replicaState_6 replica).currentTerm term) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_6) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + ; Stuttering + (and + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + ) +) +(assert ; Step 7 → 8 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_7 t_from).(>= balance t_amount) (select accounts_7 t_from).flags_allows_overdraft) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + (and ; Action: CommitTransfer + (select pending_7 id) + (select transfers_keys id) + (select transfers_7 id).(= pending_7 true) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + (and ; Action: VoidTransfer + (select pending_7 id) + (select transfers_keys id) + (select transfers_7 id).(= pending_7 true) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_7 t_from).(>= balance t_amount) (select accounts_7 t_from).flags_allows_overdraft) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_7 replica).(= role "candidate") + (< (select replicaState_7 replica).currentTerm term) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_7) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + ; Stuttering + (and + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + ) +) +(assert ; Step 8 → 9 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_8 t_from).(>= balance t_amount) (select accounts_8 t_from).flags_allows_overdraft) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + (and ; Action: CommitTransfer + (select pending_8 id) + (select transfers_keys id) + (select transfers_8 id).(= pending_8 true) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + (and ; Action: VoidTransfer + (select pending_8 id) + (select transfers_keys id) + (select transfers_8 id).(= pending_8 true) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_8 t_from).(>= balance t_amount) (select accounts_8 t_from).flags_allows_overdraft) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_8 replica).(= role "candidate") + (< (select replicaState_8 replica).currentTerm term) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_8) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + ; Stuttering + (and + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + ) +) +(assert ; Step 9 → 10 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_9 t_from).(>= balance t_amount) (select accounts_9 t_from).flags_allows_overdraft) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + (and ; Action: CommitTransfer + (select pending_9 id) + (select transfers_keys id) + (select transfers_9 id).(= pending_9 true) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + (and ; Action: VoidTransfer + (select pending_9 id) + (select transfers_keys id) + (select transfers_9 id).(= pending_9 true) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_9 t_from).(>= balance t_amount) (select accounts_9 t_from).flags_allows_overdraft) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_9 replica).(= role "candidate") + (< (select replicaState_9 replica).currentTerm term) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_9) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + ; Stuttering + (and + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + ) +) + +; Property assertion: not(invariant at some step) +; Warning: invariant "unnamed" not found in spec +(check-sat) +(get-model) +; Verification: Always(NoOverdraft) +; BMC depth: 10 + +; Declare state copies for 10 steps +(declare-const accounts_0 (Array Int Account)) +(declare-const accounts_1 (Array Int Account)) +(declare-const accounts_2 (Array Int Account)) +(declare-const accounts_3 (Array Int Account)) +(declare-const accounts_4 (Array Int Account)) +(declare-const accounts_5 (Array Int Account)) +(declare-const accounts_6 (Array Int Account)) +(declare-const accounts_7 (Array Int Account)) +(declare-const accounts_8 (Array Int Account)) +(declare-const accounts_9 (Array Int Account)) +(declare-const accounts_10 (Array Int Account)) +(declare-const transfers_0 (Array Int Transfer)) +(declare-const transfers_1 (Array Int Transfer)) +(declare-const transfers_2 (Array Int Transfer)) +(declare-const transfers_3 (Array Int Transfer)) +(declare-const transfers_4 (Array Int Transfer)) +(declare-const transfers_5 (Array Int Transfer)) +(declare-const transfers_6 (Array Int Transfer)) +(declare-const transfers_7 (Array Int Transfer)) +(declare-const transfers_8 (Array Int Transfer)) +(declare-const transfers_9 (Array Int Transfer)) +(declare-const transfers_10 (Array Int Transfer)) +(declare-const pending_0 (Array Int Bool)) +(declare-const pending_1 (Array Int Bool)) +(declare-const pending_2 (Array Int Bool)) +(declare-const pending_3 (Array Int Bool)) +(declare-const pending_4 (Array Int Bool)) +(declare-const pending_5 (Array Int Bool)) +(declare-const pending_6 (Array Int Bool)) +(declare-const pending_7 (Array Int Bool)) +(declare-const pending_8 (Array Int Bool)) +(declare-const pending_9 (Array Int Bool)) +(declare-const pending_10 (Array Int Bool)) +(declare-const replicaState_0 (Array Int Replica)) +(declare-const replicaState_1 (Array Int Replica)) +(declare-const replicaState_2 (Array Int Replica)) +(declare-const replicaState_3 (Array Int Replica)) +(declare-const replicaState_4 (Array Int Replica)) +(declare-const replicaState_5 (Array Int Replica)) +(declare-const replicaState_6 (Array Int Replica)) +(declare-const replicaState_7 (Array Int Replica)) +(declare-const replicaState_8 (Array Int Replica)) +(declare-const replicaState_9 (Array Int Replica)) +(declare-const replicaState_10 (Array Int Replica)) +(declare-const log_0 (Array Int LogEntry)) +(declare-const log_1 (Array Int LogEntry)) +(declare-const log_2 (Array Int LogEntry)) +(declare-const log_3 (Array Int LogEntry)) +(declare-const log_4 (Array Int LogEntry)) +(declare-const log_5 (Array Int LogEntry)) +(declare-const log_6 (Array Int LogEntry)) +(declare-const log_7 (Array Int LogEntry)) +(declare-const log_8 (Array Int LogEntry)) +(declare-const log_9 (Array Int LogEntry)) +(declare-const log_10 (Array Int LogEntry)) +(declare-const commitIndex_0 Int) +(declare-const commitIndex_1 Int) +(declare-const commitIndex_2 Int) +(declare-const commitIndex_3 Int) +(declare-const commitIndex_4 Int) +(declare-const commitIndex_5 Int) +(declare-const commitIndex_6 Int) +(declare-const commitIndex_7 Int) +(declare-const commitIndex_8 Int) +(declare-const commitIndex_9 Int) +(declare-const commitIndex_10 Int) +(declare-const leader_0 Int) +(declare-const leader_1 Int) +(declare-const leader_2 Int) +(declare-const leader_3 Int) +(declare-const leader_4 Int) +(declare-const leader_5 Int) +(declare-const leader_6 Int) +(declare-const leader_7 Int) +(declare-const leader_8 Int) +(declare-const leader_9 Int) +(declare-const leader_10 Int) + +; Initial state constraints +(assert + (and + (>= commitIndex_0 0) + (>= leader_0 0) + ) +) + +; Transition constraints +(assert ; Step 0 → 1 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_0 t_from).(>= balance t_amount) (select accounts_0 t_from).flags_allows_overdraft) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + (and ; Action: CommitTransfer + (select pending_0 id) + (select transfers_keys id) + (select transfers_0 id).(= pending_0 true) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + (and ; Action: VoidTransfer + (select pending_0 id) + (select transfers_keys id) + (select transfers_0 id).(= pending_0 true) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_0 t_from).(>= balance t_amount) (select accounts_0 t_from).flags_allows_overdraft) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_0 replica).(= role "candidate") + (< (select replicaState_0 replica).currentTerm term) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_0) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + ; Stuttering + (and + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + ) +) +(assert ; Step 1 → 2 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_1 t_from).(>= balance t_amount) (select accounts_1 t_from).flags_allows_overdraft) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + (and ; Action: CommitTransfer + (select pending_1 id) + (select transfers_keys id) + (select transfers_1 id).(= pending_1 true) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + (and ; Action: VoidTransfer + (select pending_1 id) + (select transfers_keys id) + (select transfers_1 id).(= pending_1 true) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_1 t_from).(>= balance t_amount) (select accounts_1 t_from).flags_allows_overdraft) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_1 replica).(= role "candidate") + (< (select replicaState_1 replica).currentTerm term) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_1) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + ; Stuttering + (and + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + ) +) +(assert ; Step 2 → 3 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_2 t_from).(>= balance t_amount) (select accounts_2 t_from).flags_allows_overdraft) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + (and ; Action: CommitTransfer + (select pending_2 id) + (select transfers_keys id) + (select transfers_2 id).(= pending_2 true) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + (and ; Action: VoidTransfer + (select pending_2 id) + (select transfers_keys id) + (select transfers_2 id).(= pending_2 true) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_2 t_from).(>= balance t_amount) (select accounts_2 t_from).flags_allows_overdraft) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_2 replica).(= role "candidate") + (< (select replicaState_2 replica).currentTerm term) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_2) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + ; Stuttering + (and + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + ) +) +(assert ; Step 3 → 4 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_3 t_from).(>= balance t_amount) (select accounts_3 t_from).flags_allows_overdraft) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + (and ; Action: CommitTransfer + (select pending_3 id) + (select transfers_keys id) + (select transfers_3 id).(= pending_3 true) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + (and ; Action: VoidTransfer + (select pending_3 id) + (select transfers_keys id) + (select transfers_3 id).(= pending_3 true) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_3 t_from).(>= balance t_amount) (select accounts_3 t_from).flags_allows_overdraft) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_3 replica).(= role "candidate") + (< (select replicaState_3 replica).currentTerm term) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_3) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + ; Stuttering + (and + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + ) +) +(assert ; Step 4 → 5 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_4 t_from).(>= balance t_amount) (select accounts_4 t_from).flags_allows_overdraft) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + (and ; Action: CommitTransfer + (select pending_4 id) + (select transfers_keys id) + (select transfers_4 id).(= pending_4 true) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + (and ; Action: VoidTransfer + (select pending_4 id) + (select transfers_keys id) + (select transfers_4 id).(= pending_4 true) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_4 t_from).(>= balance t_amount) (select accounts_4 t_from).flags_allows_overdraft) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_4 replica).(= role "candidate") + (< (select replicaState_4 replica).currentTerm term) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_4) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + ; Stuttering + (and + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + ) +) +(assert ; Step 5 → 6 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_5 t_from).(>= balance t_amount) (select accounts_5 t_from).flags_allows_overdraft) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + (and ; Action: CommitTransfer + (select pending_5 id) + (select transfers_keys id) + (select transfers_5 id).(= pending_5 true) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + (and ; Action: VoidTransfer + (select pending_5 id) + (select transfers_keys id) + (select transfers_5 id).(= pending_5 true) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_5 t_from).(>= balance t_amount) (select accounts_5 t_from).flags_allows_overdraft) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_5 replica).(= role "candidate") + (< (select replicaState_5 replica).currentTerm term) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_5) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + ; Stuttering + (and + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + ) +) +(assert ; Step 6 → 7 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_6 t_from).(>= balance t_amount) (select accounts_6 t_from).flags_allows_overdraft) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + (and ; Action: CommitTransfer + (select pending_6 id) + (select transfers_keys id) + (select transfers_6 id).(= pending_6 true) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + (and ; Action: VoidTransfer + (select pending_6 id) + (select transfers_keys id) + (select transfers_6 id).(= pending_6 true) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_6 t_from).(>= balance t_amount) (select accounts_6 t_from).flags_allows_overdraft) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_6 replica).(= role "candidate") + (< (select replicaState_6 replica).currentTerm term) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_6) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + ; Stuttering + (and + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + ) +) +(assert ; Step 7 → 8 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_7 t_from).(>= balance t_amount) (select accounts_7 t_from).flags_allows_overdraft) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + (and ; Action: CommitTransfer + (select pending_7 id) + (select transfers_keys id) + (select transfers_7 id).(= pending_7 true) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + (and ; Action: VoidTransfer + (select pending_7 id) + (select transfers_keys id) + (select transfers_7 id).(= pending_7 true) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_7 t_from).(>= balance t_amount) (select accounts_7 t_from).flags_allows_overdraft) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_7 replica).(= role "candidate") + (< (select replicaState_7 replica).currentTerm term) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_7) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + ; Stuttering + (and + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + ) +) +(assert ; Step 8 → 9 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_8 t_from).(>= balance t_amount) (select accounts_8 t_from).flags_allows_overdraft) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + (and ; Action: CommitTransfer + (select pending_8 id) + (select transfers_keys id) + (select transfers_8 id).(= pending_8 true) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + (and ; Action: VoidTransfer + (select pending_8 id) + (select transfers_keys id) + (select transfers_8 id).(= pending_8 true) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_8 t_from).(>= balance t_amount) (select accounts_8 t_from).flags_allows_overdraft) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_8 replica).(= role "candidate") + (< (select replicaState_8 replica).currentTerm term) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_8) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + ; Stuttering + (and + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + ) +) +(assert ; Step 9 → 10 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_9 t_from).(>= balance t_amount) (select accounts_9 t_from).flags_allows_overdraft) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + (and ; Action: CommitTransfer + (select pending_9 id) + (select transfers_keys id) + (select transfers_9 id).(= pending_9 true) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + (and ; Action: VoidTransfer + (select pending_9 id) + (select transfers_keys id) + (select transfers_9 id).(= pending_9 true) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_9 t_from).(>= balance t_amount) (select accounts_9 t_from).flags_allows_overdraft) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_9 replica).(= role "candidate") + (< (select replicaState_9 replica).currentTerm term) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_9) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + ; Stuttering + (and + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + ) +) + +; Property assertion: not(invariant at some step) +; Warning: invariant "unnamed" not found in spec +(check-sat) +(get-model) +; Verification: Eventually(TransferStateConsistency) +; BMC depth: 10 + +; Declare state copies for 10 steps +(declare-const accounts_0 (Array Int Account)) +(declare-const accounts_1 (Array Int Account)) +(declare-const accounts_2 (Array Int Account)) +(declare-const accounts_3 (Array Int Account)) +(declare-const accounts_4 (Array Int Account)) +(declare-const accounts_5 (Array Int Account)) +(declare-const accounts_6 (Array Int Account)) +(declare-const accounts_7 (Array Int Account)) +(declare-const accounts_8 (Array Int Account)) +(declare-const accounts_9 (Array Int Account)) +(declare-const accounts_10 (Array Int Account)) +(declare-const transfers_0 (Array Int Transfer)) +(declare-const transfers_1 (Array Int Transfer)) +(declare-const transfers_2 (Array Int Transfer)) +(declare-const transfers_3 (Array Int Transfer)) +(declare-const transfers_4 (Array Int Transfer)) +(declare-const transfers_5 (Array Int Transfer)) +(declare-const transfers_6 (Array Int Transfer)) +(declare-const transfers_7 (Array Int Transfer)) +(declare-const transfers_8 (Array Int Transfer)) +(declare-const transfers_9 (Array Int Transfer)) +(declare-const transfers_10 (Array Int Transfer)) +(declare-const pending_0 (Array Int Bool)) +(declare-const pending_1 (Array Int Bool)) +(declare-const pending_2 (Array Int Bool)) +(declare-const pending_3 (Array Int Bool)) +(declare-const pending_4 (Array Int Bool)) +(declare-const pending_5 (Array Int Bool)) +(declare-const pending_6 (Array Int Bool)) +(declare-const pending_7 (Array Int Bool)) +(declare-const pending_8 (Array Int Bool)) +(declare-const pending_9 (Array Int Bool)) +(declare-const pending_10 (Array Int Bool)) +(declare-const replicaState_0 (Array Int Replica)) +(declare-const replicaState_1 (Array Int Replica)) +(declare-const replicaState_2 (Array Int Replica)) +(declare-const replicaState_3 (Array Int Replica)) +(declare-const replicaState_4 (Array Int Replica)) +(declare-const replicaState_5 (Array Int Replica)) +(declare-const replicaState_6 (Array Int Replica)) +(declare-const replicaState_7 (Array Int Replica)) +(declare-const replicaState_8 (Array Int Replica)) +(declare-const replicaState_9 (Array Int Replica)) +(declare-const replicaState_10 (Array Int Replica)) +(declare-const log_0 (Array Int LogEntry)) +(declare-const log_1 (Array Int LogEntry)) +(declare-const log_2 (Array Int LogEntry)) +(declare-const log_3 (Array Int LogEntry)) +(declare-const log_4 (Array Int LogEntry)) +(declare-const log_5 (Array Int LogEntry)) +(declare-const log_6 (Array Int LogEntry)) +(declare-const log_7 (Array Int LogEntry)) +(declare-const log_8 (Array Int LogEntry)) +(declare-const log_9 (Array Int LogEntry)) +(declare-const log_10 (Array Int LogEntry)) +(declare-const commitIndex_0 Int) +(declare-const commitIndex_1 Int) +(declare-const commitIndex_2 Int) +(declare-const commitIndex_3 Int) +(declare-const commitIndex_4 Int) +(declare-const commitIndex_5 Int) +(declare-const commitIndex_6 Int) +(declare-const commitIndex_7 Int) +(declare-const commitIndex_8 Int) +(declare-const commitIndex_9 Int) +(declare-const commitIndex_10 Int) +(declare-const leader_0 Int) +(declare-const leader_1 Int) +(declare-const leader_2 Int) +(declare-const leader_3 Int) +(declare-const leader_4 Int) +(declare-const leader_5 Int) +(declare-const leader_6 Int) +(declare-const leader_7 Int) +(declare-const leader_8 Int) +(declare-const leader_9 Int) +(declare-const leader_10 Int) + +; Initial state constraints +(assert + (and + (>= commitIndex_0 0) + (>= leader_0 0) + ) +) + +; Transition constraints +(assert ; Step 0 → 1 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_0 t_from).(>= balance t_amount) (select accounts_0 t_from).flags_allows_overdraft) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + (and ; Action: CommitTransfer + (select pending_0 id) + (select transfers_keys id) + (select transfers_0 id).(= pending_0 true) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + (and ; Action: VoidTransfer + (select pending_0 id) + (select transfers_keys id) + (select transfers_0 id).(= pending_0 true) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_0 t_from).(>= balance t_amount) (select accounts_0 t_from).flags_allows_overdraft) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_0 replica).(= role "candidate") + (< (select replicaState_0 replica).currentTerm term) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_0) + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + ; Stuttering + (and + (= accounts_1 accounts_0) + (= transfers_1 transfers_0) + (= pending_1 pending_0) + (= replicaState_1 replicaState_0) + (= log_1 log_0) + (= commitIndex_1 commitIndex_0) + (= leader_1 leader_0) + ) + ) +) +(assert ; Step 1 → 2 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_1 t_from).(>= balance t_amount) (select accounts_1 t_from).flags_allows_overdraft) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + (and ; Action: CommitTransfer + (select pending_1 id) + (select transfers_keys id) + (select transfers_1 id).(= pending_1 true) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + (and ; Action: VoidTransfer + (select pending_1 id) + (select transfers_keys id) + (select transfers_1 id).(= pending_1 true) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_1 t_from).(>= balance t_amount) (select accounts_1 t_from).flags_allows_overdraft) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_1 replica).(= role "candidate") + (< (select replicaState_1 replica).currentTerm term) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_1) + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + ; Stuttering + (and + (= accounts_2 accounts_1) + (= transfers_2 transfers_1) + (= pending_2 pending_1) + (= replicaState_2 replicaState_1) + (= log_2 log_1) + (= commitIndex_2 commitIndex_1) + (= leader_2 leader_1) + ) + ) +) +(assert ; Step 2 → 3 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_2 t_from).(>= balance t_amount) (select accounts_2 t_from).flags_allows_overdraft) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + (and ; Action: CommitTransfer + (select pending_2 id) + (select transfers_keys id) + (select transfers_2 id).(= pending_2 true) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + (and ; Action: VoidTransfer + (select pending_2 id) + (select transfers_keys id) + (select transfers_2 id).(= pending_2 true) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_2 t_from).(>= balance t_amount) (select accounts_2 t_from).flags_allows_overdraft) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_2 replica).(= role "candidate") + (< (select replicaState_2 replica).currentTerm term) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_2) + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + ; Stuttering + (and + (= accounts_3 accounts_2) + (= transfers_3 transfers_2) + (= pending_3 pending_2) + (= replicaState_3 replicaState_2) + (= log_3 log_2) + (= commitIndex_3 commitIndex_2) + (= leader_3 leader_2) + ) + ) +) +(assert ; Step 3 → 4 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_3 t_from).(>= balance t_amount) (select accounts_3 t_from).flags_allows_overdraft) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + (and ; Action: CommitTransfer + (select pending_3 id) + (select transfers_keys id) + (select transfers_3 id).(= pending_3 true) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + (and ; Action: VoidTransfer + (select pending_3 id) + (select transfers_keys id) + (select transfers_3 id).(= pending_3 true) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_3 t_from).(>= balance t_amount) (select accounts_3 t_from).flags_allows_overdraft) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_3 replica).(= role "candidate") + (< (select replicaState_3 replica).currentTerm term) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_3) + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + ; Stuttering + (and + (= accounts_4 accounts_3) + (= transfers_4 transfers_3) + (= pending_4 pending_3) + (= replicaState_4 replicaState_3) + (= log_4 log_3) + (= commitIndex_4 commitIndex_3) + (= leader_4 leader_3) + ) + ) +) +(assert ; Step 4 → 5 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_4 t_from).(>= balance t_amount) (select accounts_4 t_from).flags_allows_overdraft) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + (and ; Action: CommitTransfer + (select pending_4 id) + (select transfers_keys id) + (select transfers_4 id).(= pending_4 true) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + (and ; Action: VoidTransfer + (select pending_4 id) + (select transfers_keys id) + (select transfers_4 id).(= pending_4 true) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_4 t_from).(>= balance t_amount) (select accounts_4 t_from).flags_allows_overdraft) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_4 replica).(= role "candidate") + (< (select replicaState_4 replica).currentTerm term) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_4) + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + ; Stuttering + (and + (= accounts_5 accounts_4) + (= transfers_5 transfers_4) + (= pending_5 pending_4) + (= replicaState_5 replicaState_4) + (= log_5 log_4) + (= commitIndex_5 commitIndex_4) + (= leader_5 leader_4) + ) + ) +) +(assert ; Step 5 → 6 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_5 t_from).(>= balance t_amount) (select accounts_5 t_from).flags_allows_overdraft) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + (and ; Action: CommitTransfer + (select pending_5 id) + (select transfers_keys id) + (select transfers_5 id).(= pending_5 true) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + (and ; Action: VoidTransfer + (select pending_5 id) + (select transfers_keys id) + (select transfers_5 id).(= pending_5 true) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_5 t_from).(>= balance t_amount) (select accounts_5 t_from).flags_allows_overdraft) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_5 replica).(= role "candidate") + (< (select replicaState_5 replica).currentTerm term) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_5) + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + ; Stuttering + (and + (= accounts_6 accounts_5) + (= transfers_6 transfers_5) + (= pending_6 pending_5) + (= replicaState_6 replicaState_5) + (= log_6 log_5) + (= commitIndex_6 commitIndex_5) + (= leader_6 leader_5) + ) + ) +) +(assert ; Step 6 → 7 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_6 t_from).(>= balance t_amount) (select accounts_6 t_from).flags_allows_overdraft) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + (and ; Action: CommitTransfer + (select pending_6 id) + (select transfers_keys id) + (select transfers_6 id).(= pending_6 true) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + (and ; Action: VoidTransfer + (select pending_6 id) + (select transfers_keys id) + (select transfers_6 id).(= pending_6 true) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_6 t_from).(>= balance t_amount) (select accounts_6 t_from).flags_allows_overdraft) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_6 replica).(= role "candidate") + (< (select replicaState_6 replica).currentTerm term) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_6) + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + ; Stuttering + (and + (= accounts_7 accounts_6) + (= transfers_7 transfers_6) + (= pending_7 pending_6) + (= replicaState_7 replicaState_6) + (= log_7 log_6) + (= commitIndex_7 commitIndex_6) + (= leader_7 leader_6) + ) + ) +) +(assert ; Step 7 → 8 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_7 t_from).(>= balance t_amount) (select accounts_7 t_from).flags_allows_overdraft) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + (and ; Action: CommitTransfer + (select pending_7 id) + (select transfers_keys id) + (select transfers_7 id).(= pending_7 true) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + (and ; Action: VoidTransfer + (select pending_7 id) + (select transfers_keys id) + (select transfers_7 id).(= pending_7 true) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_7 t_from).(>= balance t_amount) (select accounts_7 t_from).flags_allows_overdraft) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_7 replica).(= role "candidate") + (< (select replicaState_7 replica).currentTerm term) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_7) + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + ; Stuttering + (and + (= accounts_8 accounts_7) + (= transfers_8 transfers_7) + (= pending_8 pending_7) + (= replicaState_8 replicaState_7) + (= log_8 log_7) + (= commitIndex_8 commitIndex_7) + (= leader_8 leader_7) + ) + ) +) +(assert ; Step 8 → 9 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_8 t_from).(>= balance t_amount) (select accounts_8 t_from).flags_allows_overdraft) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + (and ; Action: CommitTransfer + (select pending_8 id) + (select transfers_keys id) + (select transfers_8 id).(= pending_8 true) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + (and ; Action: VoidTransfer + (select pending_8 id) + (select transfers_keys id) + (select transfers_8 id).(= pending_8 true) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_8 t_from).(>= balance t_amount) (select accounts_8 t_from).flags_allows_overdraft) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_8 replica).(= role "candidate") + (< (select replicaState_8 replica).currentTerm term) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_8) + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + ; Stuttering + (and + (= accounts_9 accounts_8) + (= transfers_9 transfers_8) + (= pending_9 pending_8) + (= replicaState_9 replicaState_8) + (= log_9 log_8) + (= commitIndex_9 commitIndex_8) + (= leader_9 leader_8) + ) + ) +) +(assert ; Step 9 → 10 + (or + (and ; Action: CreateTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_9 t_from).(>= balance t_amount) (select accounts_9 t_from).flags_allows_overdraft) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + (and ; Action: CommitTransfer + (select pending_9 id) + (select transfers_keys id) + (select transfers_9 id).(= pending_9 true) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + (and ; Action: VoidTransfer + (select pending_9 id) + (select transfers_keys id) + (select transfers_9 id).(= pending_9 true) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + (and ; Action: PostTransfer + t_id notIn transfers_keys + (select accounts_keys t_from) + (select accounts_keys t_to) + t_from distinct t_to + (or (select accounts_9 t_from).(>= balance t_amount) (select accounts_9 t_from).flags_allows_overdraft) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + (and ; Action: BecomeLeader + (select replicaState_keys replica) + (select replicaState_9 replica).(= role "candidate") + (< (select replicaState_9 replica).currentTerm term) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + (and ; Action: AppendEntry + (+ entry_index = (speckl_len_Int log) 1) + (>= term commitIndex_9) + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + ; Stuttering + (and + (= accounts_10 accounts_9) + (= transfers_10 transfers_9) + (= pending_10 pending_9) + (= replicaState_10 replicaState_9) + (= log_10 log_9) + (= commitIndex_10 commitIndex_9) + (= leader_10 leader_9) + ) + ) +) + +; Property assertion: not(invariant at some step) +(assert + (and + (not (unnamed_0)) + (not (unnamed_1)) + (not (unnamed_2)) + (not (unnamed_3)) + (not (unnamed_4)) + (not (unnamed_5)) + (not (unnamed_6)) + (not (unnamed_7)) + (not (unnamed_8)) + (not (unnamed_9)) + (not (unnamed_10)) + ) +) +(echo "Checking: Eventually(unnamed)") +(check-sat) +(get-model) diff --git a/out/TigerBeetleLedger.specbom.cdx.json b/out/TigerBeetleLedger.specbom.cdx.json new file mode 100644 index 0000000..8f5f7d0 --- /dev/null +++ b/out/TigerBeetleLedger.specbom.cdx.json @@ -0,0 +1,15 @@ +{ + "bomFormat": "CycloneDX", + "specVersion": "1.6", + "components": [], + "metadata": { + "tools": [], + "component": { + "type": "application", + "name": "TigerBeetleLedger", + "version": "0.2.0", + "properties": [] + } + }, + "annotations": [] +} \ No newline at end of file diff --git a/out/TigerBeetleLedger.specbom.spdx.json b/out/TigerBeetleLedger.specbom.spdx.json new file mode 100644 index 0000000..96e3c00 --- /dev/null +++ b/out/TigerBeetleLedger.specbom.spdx.json @@ -0,0 +1,26 @@ +{ + "@context": [ + "https://spdx.github.io/spdx-v3/spdx-context.jsonld", + { + "speckl": "https://speckl.scoble.me/ns/v0.2#" + } + ], + "@type": "spdx:Document", + "spdx": { + "@id": "spdx:TigerBeetleLedger", + "@type": "spdx:Snippet", + "name": "TigerBeetleLedger", + "spdxVersion": "3.0.1", + "dataLicense": "CC0-1.0", + "creationInfo": { + "created": "2026-05-10T09:08:16.165Z", + "createdBy": [ + "Organization: Speckl", + "tool: speckl-compile-0.2.0" + ] + } + }, + "builtBy": [], + "relationships": [], + "elements": [] +} \ No newline at end of file diff --git a/out/TigerBeetleLedger.ts b/out/TigerBeetleLedger.ts new file mode 100644 index 0000000..fee25fc --- /dev/null +++ b/out/TigerBeetleLedger.ts @@ -0,0 +1,143 @@ +// Auto-generated by speckl-compile from TigerBeetleLedger +// DO NOT EDIT MANUALLY — regenerate from .speckdl source + +export const TigerBeetleLedgerState = { + Open: 'open', + Closed: 'closed', +} as const; +export type TigerBeetleLedgerState = typeof TigerBeetleLedgerState[keyof typeof TigerBeetleLedgerState]; + + + + +export class TigerBeetleLedgerMachine { + accounts: Map; + transfers: Map; + pending: Set; + replicaState: Map; + log: LogEntry[]; + commitIndex: number; + leader: number; + decisionLog: any[] = []; + + constructor() { + // No init block defined + } + + CreateTransfer(t: Transfer) { + // precondition: !this.transfers.has(t.id) + if (!(!this.transfers.has(t.id))) throw new Error('Precondition failed: !this.transfers.has(t.id)'); + // precondition: this.accounts.has(t.from) + if (!(this.accounts.has(t.from))) throw new Error('Precondition failed: this.accounts.has(t.from)'); + // precondition: this.accounts.has(t.to) + if (!(this.accounts.has(t.to))) throw new Error('Precondition failed: this.accounts.has(t.to)'); + // precondition: t.from != t.to + if (!(t.from != t.to)) throw new Error('Precondition failed: t.from != t.to'); + // precondition: this.accounts.get(t.from).balance >= t.amount || this.accounts.get(t.from).flags.allows_overdraft + if (!(this.accounts.get(t.from).balance >= t.amount || this.accounts.get(t.from).flags.allows_overdraft)) throw new Error('Precondition failed: this.accounts.get(t.from).balance >= t.amount || this.accounts.get(t.from).flags.allows_overdraft'); + // postcondition: this.transfers.get(t.id) == t + if (!(this.transfers.get(t.id) == t)) throw new Error('Postcondition failed: this.transfers.get(t.id) == t'); + // postcondition: this.pending == new Set([...this.pending, t.id]) + if (!(this.pending == new Set([...this.pending, t.id]))) throw new Error('Postcondition failed: this.pending == new Set([...this.pending, t.id])'); + // postcondition: this.accounts.get(t.from).debits_pending == this.accounts.get(t.from).debits_pending + t.amount + if (!(this.accounts.get(t.from).debits_pending == this.accounts.get(t.from).debits_pending + t.amount)) throw new Error('Postcondition failed: this.accounts.get(t.from).debits_pending == this.accounts.get(t.from).debits_pending + t.amount'); + // postcondition: this.accounts.get(t.to).credits_pending == this.accounts.get(t.to).credits_pending + t.amount + if (!(this.accounts.get(t.to).credits_pending == this.accounts.get(t.to).credits_pending + t.amount)) throw new Error('Postcondition failed: this.accounts.get(t.to).credits_pending == this.accounts.get(t.to).credits_pending + t.amount'); + } + + CommitTransfer(id: number) { + // precondition: this.pending.has(id) + if (!(this.pending.has(id))) throw new Error('Precondition failed: this.pending.has(id)'); + // precondition: this.transfers.has(id) + if (!(this.transfers.has(id))) throw new Error('Precondition failed: this.transfers.has(id)'); + // precondition: this.transfers.get(id).pending == true + if (!(this.transfers.get(id).pending == true)) throw new Error('Precondition failed: this.transfers.get(id).pending == true'); + // postcondition: this.pending == new Set([...this.pending].filter(x => x !== id)) + if (!(this.pending == new Set([...this.pending].filter(x => x !== id)))) throw new Error('Postcondition failed: this.pending == new Set([...this.pending].filter(x => x !== id))'); + // postcondition: this.transfers.get(id).pending == false + if (!(this.transfers.get(id).pending == false)) throw new Error('Postcondition failed: this.transfers.get(id).pending == false'); + // postcondition: this.transfers.get(id).posted == true + if (!(this.transfers.get(id).posted == true)) throw new Error('Postcondition failed: this.transfers.get(id).posted == true'); + // postcondition: this.accounts.get(this.transfers.get(id).from).debits_posted === this.accounts.get(this.transfers.get(id).from).debits_posted + if (!(this.accounts.get(this.transfers.get(id).from).debits_posted === this.accounts.get(this.transfers.get(id).from).debits_posted)) throw new Error('Postcondition failed: this.accounts.get(this.transfers.get(id).from).debits_posted === this.accounts.get(this.transfers.get(id).from).debits_posted'); + // postcondition: this.accounts.get(this.transfers.get(id).to).credits_posted === this.accounts.get(this.transfers.get(id).to).credits_posted + if (!(this.accounts.get(this.transfers.get(id).to).credits_posted === this.accounts.get(this.transfers.get(id).to).credits_posted)) throw new Error('Postcondition failed: this.accounts.get(this.transfers.get(id).to).credits_posted === this.accounts.get(this.transfers.get(id).to).credits_posted'); + // postcondition: this.accounts.get(this.transfers.get(id).from).debits_pending === this.accounts.get(this.transfers.get(id).from).debits_pending + if (!(this.accounts.get(this.transfers.get(id).from).debits_pending === this.accounts.get(this.transfers.get(id).from).debits_pending)) throw new Error('Postcondition failed: this.accounts.get(this.transfers.get(id).from).debits_pending === this.accounts.get(this.transfers.get(id).from).debits_pending'); + // postcondition: this.accounts.get(this.transfers.get(id).to).credits_pending === this.accounts.get(this.transfers.get(id).to).credits_pending + if (!(this.accounts.get(this.transfers.get(id).to).credits_pending === this.accounts.get(this.transfers.get(id).to).credits_pending)) throw new Error('Postcondition failed: this.accounts.get(this.transfers.get(id).to).credits_pending === this.accounts.get(this.transfers.get(id).to).credits_pending'); + } + + VoidTransfer(id: number) { + // precondition: this.pending.has(id) + if (!(this.pending.has(id))) throw new Error('Precondition failed: this.pending.has(id)'); + // precondition: this.transfers.has(id) + if (!(this.transfers.has(id))) throw new Error('Precondition failed: this.transfers.has(id)'); + // precondition: this.transfers.get(id).pending == true + if (!(this.transfers.get(id).pending == true)) throw new Error('Precondition failed: this.transfers.get(id).pending == true'); + // postcondition: this.pending == new Set([...this.pending].filter(x => x !== id)) + if (!(this.pending == new Set([...this.pending].filter(x => x !== id)))) throw new Error('Postcondition failed: this.pending == new Set([...this.pending].filter(x => x !== id))'); + // postcondition: this.transfers.get(id).pending == false + if (!(this.transfers.get(id).pending == false)) throw new Error('Postcondition failed: this.transfers.get(id).pending == false'); + // postcondition: this.transfers.get(id).voided == true + if (!(this.transfers.get(id).voided == true)) throw new Error('Postcondition failed: this.transfers.get(id).voided == true'); + // postcondition: this.accounts.get(this.transfers.get(id).from).debits_pending === this.accounts.get(this.transfers.get(id).from).debits_pending + if (!(this.accounts.get(this.transfers.get(id).from).debits_pending === this.accounts.get(this.transfers.get(id).from).debits_pending)) throw new Error('Postcondition failed: this.accounts.get(this.transfers.get(id).from).debits_pending === this.accounts.get(this.transfers.get(id).from).debits_pending'); + // postcondition: this.accounts.get(this.transfers.get(id).to).credits_pending === this.accounts.get(this.transfers.get(id).to).credits_pending + if (!(this.accounts.get(this.transfers.get(id).to).credits_pending === this.accounts.get(this.transfers.get(id).to).credits_pending)) throw new Error('Postcondition failed: this.accounts.get(this.transfers.get(id).to).credits_pending === this.accounts.get(this.transfers.get(id).to).credits_pending'); + } + + PostTransfer(t: Transfer) { + // precondition: !this.transfers.has(t.id) + if (!(!this.transfers.has(t.id))) throw new Error('Precondition failed: !this.transfers.has(t.id)'); + // precondition: this.accounts.has(t.from) + if (!(this.accounts.has(t.from))) throw new Error('Precondition failed: this.accounts.has(t.from)'); + // precondition: this.accounts.has(t.to) + if (!(this.accounts.has(t.to))) throw new Error('Precondition failed: this.accounts.has(t.to)'); + // precondition: t.from != t.to + if (!(t.from != t.to)) throw new Error('Precondition failed: t.from != t.to'); + // precondition: this.accounts.get(t.from).balance >= t.amount || this.accounts.get(t.from).flags.allows_overdraft + if (!(this.accounts.get(t.from).balance >= t.amount || this.accounts.get(t.from).flags.allows_overdraft)) throw new Error('Precondition failed: this.accounts.get(t.from).balance >= t.amount || this.accounts.get(t.from).flags.allows_overdraft'); + // postcondition: this.transfers.get(t.id) == { ...t, pending: false, posted: true, voided: false } + if (!(this.transfers.get(t.id) == { ...t, pending: false, posted: true, voided: false })) throw new Error('Postcondition failed: this.transfers.get(t.id) == { ...t, pending: false, posted: true, voided: false }'); + // postcondition: this.accounts.get(t.from).debits_posted == this.accounts.get(t.from).debits_posted + t.amount + if (!(this.accounts.get(t.from).debits_posted == this.accounts.get(t.from).debits_posted + t.amount)) throw new Error('Postcondition failed: this.accounts.get(t.from).debits_posted == this.accounts.get(t.from).debits_posted + t.amount'); + // postcondition: this.accounts.get(t.to).credits_posted == this.accounts.get(t.to).credits_posted + t.amount + if (!(this.accounts.get(t.to).credits_posted == this.accounts.get(t.to).credits_posted + t.amount)) throw new Error('Postcondition failed: this.accounts.get(t.to).credits_posted == this.accounts.get(t.to).credits_posted + t.amount'); + } + + BecomeLeader(replica: number, term: number) { + // precondition: this.replicaState.has(replica) + if (!(this.replicaState.has(replica))) throw new Error('Precondition failed: this.replicaState.has(replica)'); + // precondition: this.replicaState.get(replica).role == "candidate" + if (!(this.replicaState.get(replica).role == "candidate")) throw new Error('Precondition failed: this.replicaState.get(replica).role == "candidate"'); + // precondition: this.replicaState.get(replica).currentTerm < term + if (!(this.replicaState.get(replica).currentTerm < term)) throw new Error('Precondition failed: this.replicaState.get(replica).currentTerm < term'); + // postcondition: this.leader == replica + if (!(this.leader == replica)) throw new Error('Postcondition failed: this.leader == replica'); + // postcondition: this.replicaState.get(replica).role == "this.leader" + if (!(this.replicaState.get(replica).role == "this.leader")) throw new Error('Postcondition failed: this.replicaState.get(replica).role == "this.leader"'); + // postcondition: this.replicaState.get(replica).currentTerm == term + if (!(this.replicaState.get(replica).currentTerm == term)) throw new Error('Postcondition failed: this.replicaState.get(replica).currentTerm == term'); + } + + AppendEntry(entry: LogEntry, term: number) { + // precondition: entry.index == length(this.log) + 1 + if (!(entry.index == length(this.log) + 1)) throw new Error('Precondition failed: entry.index == length(this.log) + 1'); + // precondition: term >= this.commitIndex + if (!(term >= this.commitIndex)) throw new Error('Precondition failed: term >= this.commitIndex'); + // postcondition: this.log == append(this.log, entry) + if (!(this.log == append(this.log, entry))) throw new Error('Postcondition failed: this.log == append(this.log, entry)'); + // postcondition: this.commitIndex == entry.index + if (!(this.commitIndex == entry.index)) throw new Error('Postcondition failed: this.commitIndex == entry.index'); + } + + + + // V1: Always(AccountBalanceInvariant) + async verify1(): Promise { + // TODO: Model-check or monitor + // Temporal: Always(AccountBalanceInvariant) + return true; + } +} diff --git a/out/TigerBeetleLedger.wat b/out/TigerBeetleLedger.wat new file mode 100644 index 0000000..6555562 --- /dev/null +++ b/out/TigerBeetleLedger.wat @@ -0,0 +1,102 @@ +;; Auto-generated by speckl-compile from TigerBeetleLedger +;; Compile: wat2wasm TigerBeetleLedger.wat -o TigerBeetleLedger.wasm + +(module + ;; Host imports for Map/Set/alloc + (import "speckl" "map_create" (func $map_create (result i32))) + (import "speckl" "map_set" (func $map_set (param i32 i32 i32))) + (import "speckl" "map_get" (func $map_get (param i32 i32) (result i32))) + (import "speckl" "map_has" (func $map_has (param i32 i32) (result i32))) + (import "speckl" "map_remove" (func $map_remove (param i32 i32))) + (import "speckl" "set_create" (func $set_create (result i32))) + (import "speckl" "set_add" (func $set_add (param i32 i32))) + (import "speckl" "set_has" (func $set_has (param i32 i32) (result i32))) + (import "speckl" "set_size" (func $set_size (param i32) (result i32))) + (import "speckl" "alloc" (func $alloc (param i32) (result i32))) + (import "speckl" "write_i32" (func $write_i32 (param i32 i32))) + (import "speckl" "write_f64" (func $write_f64 (param i32 f64))) + + ;; Linear memory (exported so host can read state) + (memory (export "memory") 1) + + ;; State globals + (global $accounts (mut i32) (i32.const 0)) +(global $transfers (mut i32) (i32.const 0)) +(global $pending (mut i32) (i32.const 0)) +(global $replicaState (mut i32) (i32.const 0)) +(global $log (mut i32) (i32.const 0)) +(global $commitIndex (mut i32) (i32.const 0)) +(global $leader (mut i32) (i32.const 0)) + + ;; ---- init ---- + (func $init (export "init") + (call $map_create) + (global.set $accounts) + (call $map_create) + (global.set $transfers) + (call $set_create) + (global.set $pending) + (call $map_create) + (global.set $replicaState) + (i32.const 0) + (global.set $commitIndex) + (i32.const 0) + (global.set $leader) + ) + + ;; ---- Actions ---- + ;; action: CreateTransfer + (func $CreateTransfer (export "CreateTransfer") (param $t i32) (result i32) + + (i32.const 1) + ) + + ;; action: CommitTransfer + (func $CommitTransfer (export "CommitTransfer") (param $id i32) (result i32) + + (i32.const 1) + ) + + ;; action: VoidTransfer + (func $VoidTransfer (export "VoidTransfer") (param $id i32) (result i32) + + (i32.const 1) + ) + + ;; action: PostTransfer + (func $PostTransfer (export "PostTransfer") (param $t i32) (result i32) + + (i32.const 1) + ) + + ;; action: BecomeLeader + (func $BecomeLeader (export "BecomeLeader") (param $replica i32) (param $term i32) (result i32) + + (i32.const 1) + ) + + ;; action: AppendEntry + (func $AppendEntry (export "AppendEntry") (param $entry i32) (param $term i32) (result i32) + + (i32.const 1) + ) + + ;; ---- getState ---- + (func $getState (export "getState") (result i32 i32) + (i32.const 0) + (global.get $commitIndex) + (i32.store) + (i32.const 4) + (global.get $leader) + (i32.store) + (i32.const 0) + (i32.const 8) + ) + + + ;; checkConstraints — runtime constraint evaluation stub + (func $checkConstraints (export "checkConstraints") (result i32) + ;; TODO: evaluate constraint expressions at runtime + (i32.const 1) + ) +) \ No newline at end of file diff --git a/out/ToggleSwitch.prov.jsonld b/out/ToggleSwitch.prov.jsonld new file mode 100644 index 0000000..5bebc1e --- /dev/null +++ b/out/ToggleSwitch.prov.jsonld @@ -0,0 +1,21 @@ +{ + "@context": { + "prov": "http://www.w3.org/ns/prov#", + "speckl": "https://speckl.scoble.me/ns/v0.2#", + "rdfs": "http://www.w3.org/2000/01/rdf-schema#" + }, + "@graph": [ + { + "@type": "prov:Entity", + "@id": "speck:ToggleSwitch", + "prov:qualifiedGeneration": { + "@type": "prov:Generation", + "prov:activity": { + "@type": "prov:Activity", + "prov:startTime": "2026-05-10T09:08:16.262Z" + } + }, + "rdfs:label": "ToggleSwitch" + } + ] +} \ No newline at end of file diff --git a/out/ToggleSwitch.smt2 b/out/ToggleSwitch.smt2 new file mode 100644 index 0000000..0498906 --- /dev/null +++ b/out/ToggleSwitch.smt2 @@ -0,0 +1,23 @@ +; SpeckDL: ToggleSwitch — Z3 Verification +; Generated by speckl-compile Z3 backend +; Bounded model checking depth: 10 + +(set-logic ALL) +(set-option :produce-models true) +(declare-fun speckl_len_Int ( (Array Int Int) ) Int) + +; --- State Variables --- +(declare-const isOn Bool) + +; --- Invariants --- +(define-fun SwitchConsistent () Bool + (and + (or (= isOn true) (= isOn false)) + ) +) + +; --- Invariant Assertions --- +(assert (SwitchConsistent)) + +(check-sat) +(get-model) diff --git a/out/ToggleSwitch.specbom.cdx.json b/out/ToggleSwitch.specbom.cdx.json new file mode 100644 index 0000000..0a54d18 --- /dev/null +++ b/out/ToggleSwitch.specbom.cdx.json @@ -0,0 +1,15 @@ +{ + "bomFormat": "CycloneDX", + "specVersion": "1.6", + "components": [], + "metadata": { + "tools": [], + "component": { + "type": "application", + "name": "ToggleSwitch", + "version": "0.2.0", + "properties": [] + } + }, + "annotations": [] +} \ No newline at end of file diff --git a/out/ToggleSwitch.specbom.spdx.json b/out/ToggleSwitch.specbom.spdx.json new file mode 100644 index 0000000..d2cfaab --- /dev/null +++ b/out/ToggleSwitch.specbom.spdx.json @@ -0,0 +1,26 @@ +{ + "@context": [ + "https://spdx.github.io/spdx-v3/spdx-context.jsonld", + { + "speckl": "https://speckl.scoble.me/ns/v0.2#" + } + ], + "@type": "spdx:Document", + "spdx": { + "@id": "spdx:ToggleSwitch", + "@type": "spdx:Snippet", + "name": "ToggleSwitch", + "spdxVersion": "3.0.1", + "dataLicense": "CC0-1.0", + "creationInfo": { + "created": "2026-05-10T09:08:16.269Z", + "createdBy": [ + "Organization: Speckl", + "tool: speckl-compile-0.2.0" + ] + } + }, + "builtBy": [], + "relationships": [], + "elements": [] +} \ No newline at end of file diff --git a/out/ToggleSwitch.ts b/out/ToggleSwitch.ts new file mode 100644 index 0000000..5f86204 --- /dev/null +++ b/out/ToggleSwitch.ts @@ -0,0 +1,34 @@ +// Auto-generated by speckl-compile from ToggleSwitch +// DO NOT EDIT MANUALLY — regenerate from .speckdl source + +export const ToggleSwitchState = { + Open: 'open', + Closed: 'closed', +} as const; +export type ToggleSwitchState = typeof ToggleSwitchState[keyof typeof ToggleSwitchState]; + + + + +export class ToggleSwitchMachine { + isOn: boolean; + decisionLog: any[] = []; + + constructor() { + // No init block defined + } + + TurnOn() { + if (!(not this.isOn)) throw new Error('Guard failed: not this.isOn'); + this.isOn = true; + } + + TurnOff() { + if (!(this.isOn)) throw new Error('Guard failed: this.isOn'); + this.isOn = false; + } + + + + +} diff --git a/out/ToggleSwitch.wat b/out/ToggleSwitch.wat new file mode 100644 index 0000000..57cb5a1 --- /dev/null +++ b/out/ToggleSwitch.wat @@ -0,0 +1,71 @@ +;; Auto-generated by speckl-compile from ToggleSwitch +;; Compile: wat2wasm ToggleSwitch.wat -o ToggleSwitch.wasm + +(module + ;; Host imports for Map/Set/alloc + (import "speckl" "map_create" (func $map_create (result i32))) + (import "speckl" "map_set" (func $map_set (param i32 i32 i32))) + (import "speckl" "map_get" (func $map_get (param i32 i32) (result i32))) + (import "speckl" "map_has" (func $map_has (param i32 i32) (result i32))) + (import "speckl" "map_remove" (func $map_remove (param i32 i32))) + (import "speckl" "set_create" (func $set_create (result i32))) + (import "speckl" "set_add" (func $set_add (param i32 i32))) + (import "speckl" "set_has" (func $set_has (param i32 i32) (result i32))) + (import "speckl" "set_size" (func $set_size (param i32) (result i32))) + (import "speckl" "alloc" (func $alloc (param i32) (result i32))) + (import "speckl" "write_i32" (func $write_i32 (param i32 i32))) + (import "speckl" "write_f64" (func $write_f64 (param i32 f64))) + + ;; Linear memory (exported so host can read state) + (memory (export "memory") 1) + + ;; State globals + (global $isOn (mut i32) (i32.const 0)) + + ;; ---- init ---- + (func $init (export "init") + (i32.const 0) + (global.set $isOn) + ) + + ;; ---- Actions ---- + ;; action: TurnOn + (func $TurnOn (export "TurnOn") (result i32) + + ;; require: not isOn + ;; Complex guard not compiled: not isOn + ;; isOn := true + (i32.const 0) ;; uncompiled expr: true + (global.set $isOn) + (global.get $isOn) + (return) + ) + + ;; action: TurnOff + (func $TurnOff (export "TurnOff") (result i32) + + ;; require: isOn + ;; Complex guard not compiled: isOn + ;; isOn := false + (i32.const 0) ;; uncompiled expr: false + (global.set $isOn) + (global.get $isOn) + (return) + ) + + ;; ---- getState ---- + (func $getState (export "getState") (result i32 i32) + (i32.const 0) + (global.get $isOn) + (i32.store) + (i32.const 0) + (i32.const 4) + ) + + + ;; checkConstraints — runtime constraint evaluation stub + (func $checkConstraints (export "checkConstraints") (result i32) + ;; TODO: evaluate constraint expressions at runtime + (i32.const 1) + ) +) \ No newline at end of file diff --git a/reference-agent/src/smes/verification-sme.ts b/reference-agent/src/smes/verification-sme.ts index f096a25..9cd6b72 100644 --- a/reference-agent/src/smes/verification-sme.ts +++ b/reference-agent/src/smes/verification-sme.ts @@ -30,7 +30,7 @@ export class VerificationSME extends SME { this.compileConstraint(c.expression, speck) ); - // In production, this would invoke Z3 via subprocess or WASM. + // In production, this would invoke Z3 via subprocess. // For the reference implementation, we simulate the result. const z3Result = this.simulateZ3(speck, z3Assertions); diff --git a/reference-agent/src/types.ts b/reference-agent/src/types.ts index 7c09802..5b6f524 100644 --- a/reference-agent/src/types.ts +++ b/reference-agent/src/types.ts @@ -99,7 +99,7 @@ export interface Task { export type TaskType = | "verify_speck" // Run Z3 verification on a Speck | "compile_to_dst" // Compile verified Speck to DST - | "compile_to_wasm" // Compile DST to WASM + | "compile_to_typescript" // Compile DST to TypeScript | "generate_code" // Generate code from verified Speck | "generate_docs" // Generate documentation from provenance graph | "integrate" // Integration/glue code @@ -142,7 +142,7 @@ export interface TaskResult { export interface ProvenanceNode { id: string; - type: "speck" | "z3_assertion" | "dst" | "wasm" | "code" | "docs" | "event"; + type: "speck" | "z3_assertion" | "dst" | "code" | "docs" | "event"; label: string; metadata: Record; } diff --git a/scripts/validate-all.sh b/scripts/validate-all.sh new file mode 100755 index 0000000..e9f1b17 --- /dev/null +++ b/scripts/validate-all.sh @@ -0,0 +1,190 @@ +#!/usr/bin/env bash +# validate-all.sh — Batch compile all Speckl examples and validate outputs +# Usage: ./scripts/validate-all.sh [--quick] [--verbose] +# --quick: only parse + TypeScript check (skip WASM/WAT) +# --verbose: show all errors +set -uo pipefail + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +SPECKL_ROOT="$(dirname "$SCRIPT_DIR")" +COMPILER_DIR="$SPECKL_ROOT/compiler" +EXAMPLES_DIR="$SPECKL_ROOT/examples" +OUT_DIR="$COMPILER_DIR/out/ci-validation" +GREEN='\033[0;32m' +RED='\033[0;31m' +YELLOW='\033[1;33m' +NC='\033[0m' + +QUICK=false +VERBOSE=false + +for arg in "$@"; do + case $arg in + --quick) QUICK=true ;; + --verbose) VERBOSE=true ;; + esac +done + +# Ensure compiler builds clean +echo "Building compiler..." +cd "$COMPILER_DIR" +if ! npx tsc --noEmit 2>&1; then + echo -e "${RED}FAIL: Compiler has TypeScript errors${NC}" + exit 1 +fi + +# Run unit tests +echo "Running unit tests..." +TEST_OUTPUT=$(npm test 2>&1) || true +TEST_COUNT=$(echo "$TEST_OUTPUT" | grep -oP 'Tests\s+\K\d+(?=\s+passed)' | head -1 || echo "0") +FAIL_COUNT=$(echo "$TEST_OUTPUT" | grep -oP 'Tests\s+\K\d+(?=\s+failed)' | head -1 || echo "0") +if [ "$FAIL_COUNT" != "0" ] && [ "$FAIL_COUNT" != "" ]; then + echo -e "${RED}FAIL: $FAIL_COUNT test(s) failing${NC}" +else + echo -e "${GREEN}PASS: $TEST_COUNT tests passing${NC}" +fi + +# Prepare output directory +rm -rf "$OUT_DIR" +mkdir -p "$OUT_DIR" + +echo "" +echo "============================================" +echo " Speckl Compiler Validation" +echo " $(date '+%Y-%m-%d %H:%M:%S')" +echo "============================================" +echo "" + +# Results tracking +PASS=0 +FAIL=0 +PARSE_FAIL=0 +TS_CLEAN=0 +TS_ERRORS=0 +WAT_OK=0 + +declare -A RESULTS +declare -A TS_ERROR_COUNTS + +NODE_BIN="$COMPILER_DIR/dist/index.js" +if [ ! -f "$NODE_BIN" ]; then + echo -e "${RED}FATAL: Compiler dist not found at $NODE_BIN${NC}" + echo "Run 'cd compiler && npx tsc' first" + exit 1 +fi + +shopt -s nullglob +SPECS=( "$EXAMPLES_DIR"/*.speck ) + +for spec in "${SPECS[@]}"; do + NAME=$(basename "$spec" .speck) + EXAMPLE_OUT="$OUT_DIR/$NAME" + mkdir -p "$EXAMPLE_OUT" + + echo -n " $NAME ... " + + # Step 1: Parse + generate all artifacts + PARSE_OUT=$(node "$NODE_BIN" "$spec" -o "$EXAMPLE_OUT" 2>&1) || true + if echo "$PARSE_OUT" | grep -q "Error\|error:"; then + echo -e "${RED}PARSE FAIL${NC}" + if $VERBOSE; then echo "$PARSE_OUT"; fi + RESULTS[$NAME]="PARSE_FAIL" + PARSE_FAIL=$((PARSE_FAIL + 1)) + FAIL=$((FAIL + 1)) + continue + fi + + # Step 2: Count artifacts generated + ARTIFACT_COUNT=$(find "$EXAMPLE_OUT" -maxdepth 1 -type f \( -name "*.json" -o -name "*.jsonld" -o -name "*.ts" -o -name "*.wat" \) 2>/dev/null | wc -l) + + # Step 3: TypeScript check + TS_FILE="$EXAMPLE_OUT/$NAME.ts" + TS_OK=false + if [ -f "$TS_FILE" ]; then + TS_CHECK_OUT=$(npx tsc --noEmit --target ES2020 --module commonjs --strict --lib ES2020 --skipLibCheck \ + "$TS_FILE" 2>&1) && TS_OK=true || true + if [ "$TS_OK" = true ]; then + TS_CLEAN=$((TS_CLEAN + 1)) + else + TS_ERR_COUNT=$(echo "$TS_CHECK_OUT" | grep -c "error TS" 2>/dev/null || echo "0") + TS_ERROR_COUNTS[$NAME]=$TS_ERR_COUNT + TS_ERRORS=$((TS_ERRORS + 1)) + # Save error log + echo "$TS_CHECK_OUT" > "$EXAMPLE_OUT/ts-errors.txt" + fi + else + echo -e "${RED}NO TS OUTPUT${NC}" + RESULTS[$NAME]="NO_TS" + FAIL=$((FAIL + 1)) + continue + fi + + # Step 4: WASM/WAT validation + WAT_FILE="$EXAMPLE_OUT/$NAME.wat" + if [ "$QUICK" = false ] && [ -f "$WAT_FILE" ]; then + if command -v wat2wasm &>/dev/null; then + wat2wasm "$WAT_FILE" -o "$EXAMPLE_OUT/$NAME.wasm" 2>/dev/null && WAT_OK=$((WAT_OK + 1)) || true + fi + fi + + # Final status + if [ "$TS_OK" = true ]; then + echo -e "${GREEN}OK${NC} ($ARTIFACT_COUNT artifacts, TS clean)" + RESULTS[$NAME]="PASS" + PASS=$((PASS + 1)) + else + TS_ERR="${TS_ERROR_COUNTS[$NAME]:-?}" + echo -e "${YELLOW}TS ERRORS: $TS_ERR${NC} ($ARTIFACT_COUNT artifacts)" + RESULTS[$NAME]="TS_ERRORS" + FAIL=$((FAIL + 1)) + if $VERBOSE && [ -f "$EXAMPLE_OUT/ts-errors.txt" ]; then + head -20 "$EXAMPLE_OUT/ts-errors.txt" + fi + fi +done + +echo "" +echo "============================================" +echo " RESULTS" +echo "============================================" +TOTAL=${#SPECS[@]} +echo " Total examples: $TOTAL" +echo " Parse success: $((PASS + TS_ERRORS))/$((PASS + TS_ERRORS + PARSE_FAIL))" +echo " Parse failures: $PARSE_FAIL" +echo " TS clean: $TS_CLEAN" +echo " TS with errors: $TS_ERRORS" +echo " Overall pass: $PASS / $TOTAL" +if [ "$QUICK" = false ]; then + echo " WAT valid: $WAT_OK" +fi +echo "" + +# Detailed table +printf " %-30s %s\n" "EXAMPLE" "STATUS" +printf " %-30s %s\n" "-------" "------" +for spec in "${SPECS[@]}"; do + NAME=$(basename "$spec" .speck) + STATUS="${RESULTS[$NAME]:-UNKNOWN}" + case $STATUS in + PASS) COLOR="$GREEN" ;; + TS_ERRORS) COLOR="$YELLOW" ;; + PARSE_FAIL|NO_TS) COLOR="$RED" ;; + *) COLOR="$RED" ;; + esac + EXTRA="" + if [ "$STATUS" = "TS_ERRORS" ]; then + EXTRA=" (${TS_ERROR_COUNTS[$NAME]:-?} errors)" + fi + printf " ${COLOR}%-30s %-12s${NC}%s\n" "$NAME" "$STATUS" "$EXTRA" +done + +echo "" +echo " Artifacts in: $OUT_DIR" +echo " TS error logs: $OUT_DIR/*/ts-errors.txt" +echo "" + +# Exit code +if [ $FAIL -gt 0 ] || [ $PARSE_FAIL -gt 0 ]; then + exit 1 +fi +exit 0 diff --git a/typescript b/typescript new file mode 100644 index 0000000..ec1ec71 --- /dev/null +++ b/typescript @@ -0,0 +1,4 @@ +Script started on 2026-05-10 01:11:44-07:00 [COMMAND="sed -n '310p' compiler/src/generators/z3.ts" ] + cleanName = cleanName.replace(/\(([^)]+)\)/g, '_$1').replace(/[<>,\s]+/g, '_').replace(/__+/g, '_'); + +Script done on 2026-05-10 01:11:44-07:00 [COMMAND_EXIT_CODE="0"]