sscoble/speckl
1
0
Fork 0
Code Issues 2 Pull requests Projects Releases Packages Wiki Activity Actions

Compare commits

base: sscoble:main
Branches Tags
sscoble:main
sscoble:lagrangia-v1.1
sscoble:ifblock-fix
sscoble:now-fix
sscoble:lagrangia-score
sscoble:fix/speckl-55-ltl-inner-identifier-scope
sscoble:fix/parser-precondition-postcondition
sscoble:fix/compiler-enum-rewrite-parser-quotes
..
compare: sscoble:fix/parser-precondition-postcondition
Branches Tags
sscoble:lagrangia-v1.1
sscoble:ifblock-fix
sscoble:now-fix
sscoble:lagrangia-score
sscoble:fix/speckl-55-ltl-inner-identifier-scope
sscoble:main
sscoble:fix/parser-precondition-postcondition
sscoble:fix/compiler-enum-rewrite-parser-quotes

No commits in common. "main" and "fix/parser-precondition-postcondition" have entirely different histories.
main ... fix/parser

79 changed files with 1174 additions and 22922 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.gitignore vendored
View file

@ -1 +0,0 @@
out/

32
README.md
View file

@ -2,7 +2,7 @@
**A judgement transport utility for AI-generated specifications.** Fulfills NIST SA-11 (human-in-the-loop for automated outputs).
Speckl is a specification language (SpeckDL) with a compiler that produces runnable software — TypeScript and provenance artifacts (PROV-O, CycloneDX, SPDX) — from a single source of truth. Write what your system should do. Prove it. Ship it.
Speckl is a specification language (SpeckDL) with a compiler that produces runnable software — WebAssembly, TypeScript, and provenance artifacts (PROV-O, CycloneDX, SPDX) — from a single source of truth. Write what your system should do. Prove it. Ship it.
## The Core Insight
@ -17,7 +17,8 @@ AI generates code at machine speed. Humans own specifications. **SpeckDL is the
Speckl is a **working compiler** that turns formal specifications into multiple target representations:
```
.speckdl spec ──→ TypeScript ← typed, importable classes
.speckdl spec ──→ Wasm (WebAssembly) ← runnable, verified state machines
├── TypeScript ← typed, importable classes
├── PROV-O (RDF) ← W3C provenance standard
├── CycloneDX SBOM ← OWASP software bill of materials
└── SPDX SBOM ← Linux Foundation SBOM standard
@ -45,7 +46,7 @@ spec ToggleSwitch {
}
```
This compiles to a runnable TypeScript class with verified state transitions + guards.
This compiles to a runnable WebAssembly module with verified state transitions + guards, and to a type-safe TypeScript class. Both from the same spec.
### Provenance Embedded at Compile Time
@ -57,12 +58,12 @@ The compiler has been validated against:
| Example | Source | Status |
|---------|--------|--------|
| **ToggleSwitch** | SpeckDL tutorial | ✅ All artifacts |
| **AccountLedger** | Banking ledger with debit/credit | ✅ All artifacts |
| **TwoPhaseCommit** | TLA+ port | ✅ All artifacts |
| **Paxos** | TLA+ port | ✅ All artifacts |
| **Raft** | TLA+ port | ✅ All artifacts |
| **TigerBeetleLedger** | Naive TigerBeetle core | ✅ All artifacts (3 TS type errors in progress) |
| **ToggleSwitch** | SpeckDL tutorial | ✅ All 5 artifacts |
| **AccountLedger** | Banking ledger with debit/credit | ✅ All 5 artifacts |
| **TwoPhaseCommit** | TLA+ port | ✅ All 5 artifacts |
| **Paxos** | TLA+ port | ✅ All 5 artifacts |
| **Raft** | TLA+ port | ✅ All 5 artifacts |
| **TigerBeetleLedger** | Naive TigerBeetle core | ✅ All 5 artifacts (3 TS type errors in progress) |
## Getting Started
@ -71,10 +72,10 @@ git clone https://os.scoble.me/forgejo/sscoble/speckl
cd speckl/compiler
npm install
# Compile an example
# Compile an example to all 5 targets
node dist/index.js ../examples/ToggleSwitch.speck -o out/switch
ls out/switch/
# ToggleSwitch.ts ToggleSwitch.prov-o.jsonld
# ToggleSwitch.wat ToggleSwitch.ts ToggleSwitch.prov-o.jsonld
# ToggleSwitch.cyclonedx.json ToggleSwitch.spdx.json
```
@ -98,10 +99,11 @@ ls out/switch/
## Validation Status
- **Compiler:** 41/41 tests passing (TypeScript codegen)
- **TypeScript:** Type-safe compilation from specs
- **TLA+ Trilogy:** TwoPhaseCommit, Paxos, Raft all compile
- **Compiler Self-Spec:** `speckl-compile.speckdl` defines the Speckl compiler in SpeckDL — generates all artifacts (known TypeScript errors tracked in engineering#9)
- **Compiler:** 41/41 tests passing (WASM + TypeScript codegen)
- **WASM E2E:** wabt assemble + instantiate + run verified
- **TypeScript:** Dual-target compilation — same spec produces both WASM and TS
- **TLA+ Trilogy:** TwoPhaseCommit, Paxos, Raft all compile (WAT validates through wabt)
- **Compiler Self-Spec:** `speckl-compile.speckdl` defines the Speckl compiler in SpeckDL — generates all 5 artifacts (known TypeScript errors tracked in engineering#9)
## Known Issues

120
bench-rust.mjs
View file

@ -1,120 +0,0 @@
import fs from 'fs';
import { execSync } from 'child_process';
const specs = [
{ name: 'KafkaKRaft', construct: 'KafkaKRaftMachine::new()', ops: [
'm.request_vote(2, 1, 0)',
'm.append_entries(2)',
'm.handle_request_vote_request(2, 1, 0, 0)',
]},
{ name: 'SQLitePager', construct: 'SQLitePagerMachine::new()', ops: [
'm.begin_read_transaction()',
'm.begin_write_transaction()',
'm.write_page(1, 100)',
'm.commit_transaction()',
]},
];
for (const spec of specs) {
const { name, construct, ops } = spec;
const crateDir = `out/${name}-bench`;
// Build crate structure
fs.mkdirSync(`${crateDir}/src`, { recursive: true });
fs.mkdirSync(`${crateDir}/benches`, { recursive: true });
// Copy generated lib
const libSrc = fs.readFileSync(`out/${name}/src/lib.rs`, 'utf-8');
// Add type defs
const typeDefs = `// Auto-generated type definitions
pub type NodeId = u64;
pub type LogEntry = String;
pub type Account = String;
pub type Transfer = String;
pub type TransferId = u64;
pub type PageId = u64;
pub type PageData = u64;
pub type Frame = u64;
pub type FrameId = u64;
pub type Bytes = Vec<u8>;
`;
fs.writeFileSync(`${crateDir}/src/lib.rs`, typeDefs + libSrc);
// Cargo.toml
fs.writeFileSync(`${crateDir}/Cargo.toml`, `[package]
name = "${name.toLowerCase()}-bench"
version = "0.1.0"
edition = "2021"
[[bench]]
name = "bench"
harness = false
[profile.bench]
debug = 0
lto = "fat"
codegen-units = 1
`);
// Benchmark code
const opBody = ops.map((op, i) => ` let _ = ${op};`).join('\n');
const totalOps = ops.length;
fs.writeFileSync(`${crateDir}/benches/bench.rs`, `use std::time::Instant;
#[path = "../src/lib.rs"]
mod generated;
use generated::*;
fn main() {
let iterations = 100_000;
println!("\\n=== ${name} Benchmark ===");
println!("Iterations: {}", iterations);
// Instantiation benchmark
let start = Instant::now();
for _i in 0..iterations {
let _m = ${construct};
}
let elapsed = start.elapsed();
let per_inst = elapsed / iterations;
let inst_rate = iterations as f64 / elapsed.as_secs_f64();
println!(" Instantiate:");
println!(" Total: {:?}", elapsed);
println!(" Per op: {:?}", per_inst);
if inst_rate > 1_000_000.0 {
println!(" Rate: {:.1} M/sec", inst_rate / 1_000_000.0);
} else if inst_rate > 1_000.0 {
println!(" Rate: {:.1} K/sec", inst_rate / 1_000.0);
} else {
println!(" Rate: {:.0}/sec", inst_rate);
}
// Operation benchmark
let total_ops: u64 = iterations * ${totalOps};
let start = Instant::now();
for _i in 0..iterations {
let mut m = ${construct};
${opBody}
}
let elapsed = start.elapsed();
let per_op = elapsed / total_ops;
let rate = total_ops as f64 / elapsed.as_secs_f64();
println!(" Operations ({} ops per cycle):", ${totalOps});
println!(" Total: {:?}", elapsed);
println!(" Per op: {:?}", per_op);
if rate > 1_000_000.0 {
println!(" Rate: {:.1} M ops/sec", rate / 1_000_000.0);
} else if rate > 1_000.0 {
println!(" Rate: {:.1} K ops/sec", rate / 1_000.0);
} else {
println!(" Rate: {:.0} ops/sec", rate);
}
}
`);
}
console.log('Bench crates created. Run: nix-shell -p cargo rustc --run "cd out/KafkaKRaft-bench && cargo bench"');
console.log('And: nix-shell -p cargo rustc --run "cd out/SQLitePager-bench && cargo bench"');

155
bench-rust.sh
View file

@ -1,155 +0,0 @@
#!/bin/bash
# Benchmark Speck-generated Rust output
set -e
cd /home/sscoble/speckl
regen=0
bench_spec() {
local SPEC=$1
local BENCH_DIR="out/${SPEC}-bench"
local PROLOGUE=$2
local CONSTRUCT=$3
local OPS=$4
echo "=== $SPEC ==="
if [ "$regen" -eq 1 ]; then
node compiler/dist/index.js "examples/$SPEC.speck" --target=rust 2>/dev/null
fi
mkdir -p "$BENCH_DIR/src"
cp "out/$SPEC/src/lib.rs" "$BENCH_DIR/src/lib.rs"
# Add type defs and fix Option syntax
cat > "$BENCH_DIR/src/lib_patched.rs" << 'EOF'
// Auto-generated type definitions
pub type NodeId = u64;
pub type LogEntry = String;
pub type Account = String;
pub type Transfer = String;
pub type TransferId = u64;
pub type PageId = u64;
pub type PageData = u64;
pub type Frame = u64;
pub type FrameId = u64;
EOF
cat "$BENCH_DIR/src/lib.rs" >> "$BENCH_DIR/src/lib_patched.rs"
mv "$BENCH_DIR/src/lib_patched.rs" "$BENCH_DIR/src/lib.rs"
# Create Cargo.toml with benchmark profile
cat > "$BENCH_DIR/Cargo.toml" << 'CTOML'
[package]
name = "speck-bench"
version = "0.1.0"
edition = "2021"
[[bench]]
name = "bench"
harness = false
[profile.bench]
debug = 0
lto = "fat"
codegen-units = 1
CTOML
# Create benchmark
mkdir -p "$BENCH_DIR/benches"
cat > "$BENCH_DIR/benches/bench.rs" << 'BRS'
use std::time::{Duration, Instant};
#[path = "../src/lib.rs"]
mod generated;
BRS
# Add spec-specific benchmark
{
echo "use generated::*;"
echo ""
echo "fn main() {"
echo " let iterations = 100000;"
echo " println!(\"Benchmarking $SPEC...\");"
echo ""
echo " // Instantiation benchmark"
echo " let start = Instant::now();"
echo " for i in 0..iterations {"
echo " let mut m = $CONSTRUCT;"
echo " let _ = m;"
echo " }"
echo " let elapsed = start.elapsed();"
echo " let per = elapsed / iterations;"
echo " println!(\" Instantiate x{}:\", iterations);"
echo " println!(\" Total: {:?}\", elapsed);"
echo " println!(\" Per op: {:?}\", per);"
echo " println!(\" Ops/sec: {:.0}\", iterations as f64 / elapsed.as_secs_f64());"
echo ""
echo " // Operational benchmark"
echo " let mut total_ops = 0u64;"
echo " let start = Instant::now();"
echo " for i in 0..iterations {"
echo $PROLOGUE
echo " }"
echo " let elapsed = start.elapsed();"
echo " let per = elapsed / total_ops;"
echo " println!(\" Operations x{} ({} ops):\", iterations, total_ops);"
echo " let rate = total_ops as f64 / elapsed.as_secs_f64();"
echo " println!(\" Total: {:?}\", elapsed);"
echo " println!(\" Per op: {:?}\", per);"
echo " if rate > 1_000_000.0 {"
echo " println!(\" Rate: {:.0} M ops/sec\", rate / 1_000_000.0);"
echo " } else if rate > 1_000.0 {"
echo " println!(\" Rate: {:.0} K ops/sec\", rate / 1_000.0);"
echo " } else {"
echo " println!(\" Rate: {:.0} ops/sec\", rate);"
echo " }"
echo "}"
} >> "$BENCH_DIR/benches/bench.rs"
}
# KafkaKRaft benchmark
bench_spec "KafkaKRaft" \
' let mut m = KafkaKRaftMachine::new();
let _ = m.request_vote(2, 1, 0);
let _ = m.append_entries(2);
let _ = m.handle_request_vote_request(2, 1, 0, 0);
total_ops += 3;' \
'KafkaKRaftMachine::new()' \
3
# SQLitePager benchmark
bench_spec "SQLitePager" \
' let mut m = SQLitePagerMachine::new();
let _ = m.begin_read_transaction();
let _ = m.begin_write_transaction();
let _ = m.write_page(1, 100);
let _ = m.commit_transaction();
total_ops += 4;' \
'SQLitePagerMachine::new()' \
4
# TigerBeetleLedger benchmark
bench_spec "TigerBeetleLedger" \
' let mut m = TigerBeetleLedgerMachine::new();
let _ = m.create_account(1, 1000, 0);
let _ = m.create_account(2, 0, 0);
let _ = m.create_transfer(1, 1, 2, 100, 0);
total_ops += 3;' \
'TigerBeetleLedgerMachine::new()' \
3
echo ""
echo "=== Building benchmarks ==="
for D in out/KafkaKRaft-bench out/SQLitePager-bench out/TigerBeetleLedger-bench; do
echo "Building $D..."
(cd "$D" && cargo bench --no-run 2>&1 | tail -5)
done
echo ""
echo "=== Running benchmarks ==="
for D in out/KafkaKRaft-bench out/SQLitePager-bench out/TigerBeetleLedger-bench; do
echo ""
(cd "$D" && cargo bench 2>&1)
done

203
compiler/__tests__/e2e-wasm.test.ts Normal file
View file

@ -0,0 +1,203 @@
import { describe, it, expect } from 'vitest';
import { parseSpeckContent } from '../src/parser.js';
import { generateWASM } from '../src/generators/wasm.js';
import { createHostImports, bindMemory, readScalarState } from '../src/runtime/wasm-host.js';
import fs from 'fs';
import path from 'path';
import os from 'os';
// Dynamic import wabt since it's a CommonJS module
async function loadWabt() {
const wabt = await import('wabt');
return wabt.default();
}
describe('SpeckDL WASM E2E', () => {
it('should compile Counter speck to WAT, assemble to WASM, and run in Node.js', async () => {
// A minimal scalar-only speck that we can fully compile and run
// NOTE: Parser requires braces on the SAME line as block keywords
const speckSource = `speck Counter {
author: "Speckl Compiler v0.3"
source: document ref "WASM-E2E"
review: automated
state: {
count: Nat := 0
maxCount: Nat := 10
}
init: {
count := 0
maxCount := 10
}
action Increment() {
require count < maxCount
count := count + 1
}
action Decrement() {
require count > 0
count := count - 1
}
action Reset() {
count := 0
}
action GetCount() {
return count
}
}`;
// 1. Parse
const ast = parseSpeckContent(speckSource);
expect(ast.specks).toHaveLength(1);
expect(ast.specks[0].name).toBe('Counter');
// 2. Generate WAT to temp dir
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'speckl-wasm-e2e-'));
generateWASM(ast, tmpDir);
const watPath = path.join(tmpDir, 'Counter.wat');
expect(fs.existsSync(watPath)).toBe(true);
const watContent = fs.readFileSync(watPath, 'utf-8');
expect(watContent).toContain('(module');
expect(watContent).toContain('(export "init")');
expect(watContent).toContain('(export "Increment")');
expect(watContent).toContain('(export "Decrement")');
expect(watContent).toContain('(export "Reset")');
expect(watContent).toContain('(export "GetCount")');
expect(watContent).toContain('(export "getState")');
expect(watContent).toContain('(export "memory")');
// 3. Assemble WAT → WASM using wabt
const wabtModule = await loadWabt();
const wasmModule = wabtModule.parseWat('Counter.wat', watContent);
wasmModule.resolveNames();
wasmModule.validate({ features: {} });
const binaryOutput = wasmModule.toBinary({ log: false, write_debug_names: false });
const wasmBytes = binaryOutput.buffer;
expect(wasmBytes.length).toBeGreaterThan(0);
expect(wasmBytes.length).toBeGreaterThan(50); // Not just empty header
// 4. Instantiate with host runtime
const hostState = {
maps: new Map(),
sets: new Map(),
nextHandle: 1,
memory: null as WebAssembly.Memory | null,
};
const imports = { speckl: createHostImports(hostState) };
const wasm = await WebAssembly.instantiate(wasmBytes, imports);
const memory = wasm.instance.exports.memory as WebAssembly.Memory;
bindMemory(hostState, memory);
// 5. Verify exports
expect(typeof wasm.instance.exports.init).toBe('function');
expect(typeof wasm.instance.exports.Increment).toBe('function');
expect(typeof wasm.instance.exports.Decrement).toBe('function');
expect(typeof wasm.instance.exports.Reset).toBe('function');
expect(typeof wasm.instance.exports.GetCount).toBe('function');
expect(typeof wasm.instance.exports.getState).toBe('function');
expect(memory).toBeInstanceOf(WebAssembly.Memory);
// 6. Run init
(wasm.instance.exports.init as Function)();
// 7. Verify initial state via getState
const stateResult = (wasm.instance.exports.getState as Function)();
// getState returns i32 i32 (ptr, len)
expect(stateResult).toBeDefined();
// Read memory directly to verify state
const scalarState = readScalarState(memory, ['count', 'maxCount']);
expect(scalarState.count).toBe(0);
expect(scalarState.maxCount).toBe(10);
// 8. Run Increment 3 times, calling getState after each to sync memory
for (let i = 0; i < 3; i++) {
const ret = (wasm.instance.exports.Increment as Function)();
expect(ret).toBe(1); // Success
(wasm.instance.exports.getState as Function)(); // Sync globals to memory
}
const afterIncrement = readScalarState(memory, ['count', 'maxCount']);
expect(afterIncrement.count).toBe(3);
expect(afterIncrement.maxCount).toBe(10);
// 9. Run Decrement once
const decRet = (wasm.instance.exports.Decrement as Function)();
expect(decRet).toBe(1); // Success
(wasm.instance.exports.getState as Function)();
const afterDecrement = readScalarState(memory, ['count', 'maxCount']);
expect(afterDecrement.count).toBe(2);
// 10. Run GetCount — should return current count (global, not memory)
const getRet = (wasm.instance.exports.GetCount as Function)();
expect(getRet).toBe(2);
// 11. Run Reset
const resetRet = (wasm.instance.exports.Reset as Function)();
expect(resetRet).toBe(1); // Success
(wasm.instance.exports.getState as Function)();
const afterReset = readScalarState(memory, ['count', 'maxCount']);
expect(afterReset.count).toBe(0);
// 12. Guard test: Decrement at 0 should fail
const guardFail = (wasm.instance.exports.Decrement as Function)();
expect(guardFail).toBe(0); // Guard failed → return 0
// 13. Guard test: Increment at maxCount should fail
// First increment to max
for (let i = 0; i < 10; i++) {
(wasm.instance.exports.Increment as Function)();
}
(wasm.instance.exports.getState as Function)();
const atMax = readScalarState(memory, ['count', 'maxCount']);
expect(atMax.count).toBe(10);
const maxGuardFail = (wasm.instance.exports.Increment as Function)();
expect(maxGuardFail).toBe(0); // Guard failed → return 0
// Cleanup
fs.rmSync(tmpDir, { recursive: true, force: true });
}, 30000); // 30s timeout for wabt compilation
it('should compile TigerBeetle speck to valid WAT that wabt accepts', async () => {
// Load the full TigerBeetle speck from file
const speckPath = path.join(__dirname, 'tigerbeetle.speck');
const speckSource = fs.readFileSync(speckPath, 'utf-8');
const ast = parseSpeckContent(speckSource);
expect(ast.specks).toHaveLength(1);
expect(ast.specks[0].name).toBe('TigerBeetle');
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'speckl-wasm-e2e-tb-'));
generateWASM(ast, tmpDir);
const watPath = path.join(tmpDir, 'TigerBeetle.wat');
expect(fs.existsSync(watPath)).toBe(true);
const watContent = fs.readFileSync(watPath, 'utf-8');
// Validate with wabt
const wabtModule = await loadWabt();
const wasmModule = wabtModule.parseWat('TigerBeetle.wat', watContent);
// These should not throw
wasmModule.resolveNames();
wasmModule.validate({ features: {} });
const binaryOutput = wasmModule.toBinary({ log: false, write_debug_names: false });
expect(binaryOutput.buffer.length).toBeGreaterThan(100);
// Cleanup
fs.rmSync(tmpDir, { recursive: true, force: true });
}, 30000);
});

73
compiler/__tests__/parser.test.ts
View file

@ -1,5 +1,6 @@
import { describe, it, expect } from 'vitest';
import { parseSpeckContent, parseSpeckFile } from '../src/parser.js';
import { generateWASM } from '../src/generators/wasm.js';
describe('SpeckDL Parser', () => {
describe('parseSpeckContent', () => {
@ -524,4 +525,76 @@ speck Second {
}
});
});
describe('WASM Codegen', () => {
it('should generate WAT for TigerBeetle with all exports', () => {
const ast = parseSpeckFile('./__tests__/tigerbeetle.speck');
const fs = require('fs');
const os = require('os');
const path = require('path');
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'speckl-wasm-test-'));
generateWASM(ast, tmpDir);
const watPath = path.join(tmpDir, 'TigerBeetle.wat');
expect(fs.existsSync(watPath)).toBe(true);
const wat = fs.readFileSync(watPath, 'utf-8');
// Module structure
expect(wat).toContain('(module');
expect(wat).toContain('(memory (export "memory")');
// Host imports
expect(wat).toContain('(import "speckl" "map_create"');
expect(wat).toContain('(import "speckl" "set_create"');
// State globals
expect(wat).toContain('(global $nextId');
expect(wat).toContain('(global $accounts');
expect(wat).toContain('(global $transfers');
expect(wat).toContain('(global $ledgerOpen');
// init export
expect(wat).toContain('(func $init (export "init")');
// Action exports
expect(wat).toContain('(func $CreateAccount (export "CreateAccount")');
expect(wat).toContain('(func $Transfer (export "Transfer")');
expect(wat).toContain('(func $CloseAccount (export "CloseAccount")');
expect(wat).toContain('(func $CloseLedger (export "CloseLedger")');
// getState export
expect(wat).toContain('(func $getState (export "getState")');
// Constraint stub
expect(wat).toContain('(func $checkConstraints (export "checkConstraints")');
// Scalar init values
expect(wat).toContain('(i32.const 1)'); // nextId default
expect(wat).toContain('(call $map_create)');
expect(wat).toContain('(call $set_create)');
fs.rmSync(tmpDir, { recursive: true });
});
it('should emit scalar guard expressions as WASM comparison', () => {
const ast = parseSpeckFile('./__tests__/tigerbeetle.speck');
const fs = require('fs');
const os = require('os');
const path = require('path');
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'speckl-wasm-test-'));
generateWASM(ast, tmpDir);
const wat = fs.readFileSync(path.join(tmpDir, 'TigerBeetle.wat'), 'utf-8');
// ledgerOpen == true should compile to i32.eq
expect(wat).toContain('(i32.eq (global.get $ledgerOpen) (i32.const 1))');
// nextId + 1 should compile to i32.add
expect(wat).toContain('(i32.add (global.get $nextId) (i32.const 1))');
fs.rmSync(tmpDir, { recursive: true });
});
});
});

72
compiler/dist/index.js vendored
View file

@ -6,8 +6,7 @@ import { generateProvenanceGraph } from './generators/provenance.js';
import { generateCycloneDX } from './generators/cyclonedx.js';
import { generateSPDX } from './generators/spdx.js';
import { generateTypeScriptStateMachine } from './generators/typescript-state-machine.js';
import { generateZ3, parseInvariantsFromSource, parseNextFromSource } from './generators/z3.js';
import { generateRust } from './generators/rust.js';
import { generateWASM } from './generators/wasm.js';
import fs from 'fs';
import path from 'path';
async function main() {
@ -36,19 +35,6 @@ async function main() {
describe: 'Output directory for compiled artifacts',
type: 'string',
default: './out',
})
.option('target', {
alias: 't',
describe: 'Compilation target',
type: 'string',
choices: ['typescript', 'z3', 'rust', 'all'],
default: 'typescript',
})
.option('verify-depth', {
alias: 'd',
describe: 'Bounded model checking depth for Z3 verification',
type: 'number',
default: 10,
})
.option('bom-format', {
alias: 'b',
@ -64,8 +50,6 @@ async function main() {
const options = {
outputDir: argv.outputDir,
bomFormat: argv.bomFormat,
target: argv.target,
verifyDepth: argv.verifyDepth,
};
console.log(`Compiling ${file}...`);
// Parse the Speck file
@ -77,45 +61,23 @@ async function main() {
if (!fs.existsSync(options.outputDir)) {
fs.mkdirSync(options.outputDir, { recursive: true });
}
// Read raw source for invariant/next parsing that the AST parser doesn't handle yet
const rawSource = fs.readFileSync(file, 'utf-8');
// Generate Rust state machine if target is rust or all
if (options.target === 'rust' || options.target === 'all') {
console.log('\nGenerating Rust state machine...');
generateRust(ast, options.outputDir);
// Always generate provenance (it's the core output)
console.log('\nGenerating provenance graph (PROV-O JSON-LD)...');
generateProvenanceGraph(ast, options.outputDir);
// Generate TypeScript state machine
console.log('\nGenerating TypeScript state machine...');
generateTypeScriptStateMachine(ast, options.outputDir);
// Generate WASM state machine
console.log('\nGenerating WebAssembly (WAT)...');
generateWASM(ast, options.outputDir);
// Generate BOM based on format selection
if (options.bomFormat === 'cdx' || options.bomFormat === 'both') {
console.log('\nGenerating CycloneDX BOM...');
generateCycloneDX(ast, options.outputDir);
}
// Always generate provenance (it's the core output) if not z3-only or rust-only
if (options.target !== 'z3' && options.target !== 'rust') {
console.log('\nGenerating provenance graph (PROV-O JSON-LD)...');
generateProvenanceGraph(ast, options.outputDir);
}
// Generate Z3 SMT-LIB2 if target is z3 or all
if (options.target === 'z3' || options.target === 'all') {
console.log('\nGenerating Z3 SMT-LIB2...');
// Parse invariants from raw source for Z3 backend
for (const speck of ast.specks) {
const invariants = parseInvariantsFromSource(rawSource, speck.name);
const nextNode = parseNextFromSource(rawSource, speck.name);
// Attach parsed data to the AST for the generator
speck._invariants = invariants;
speck._next = nextNode;
}
const z3Options = { verifyDepth: options.verifyDepth };
generateZ3(ast, options.outputDir, z3Options);
}
// Generate TypeScript state machine if not z3-only or rust-only
if (options.target !== 'z3' && options.target !== 'rust') {
console.log('\nGenerating TypeScript state machine...');
generateTypeScriptStateMachine(ast, options.outputDir);
// Generate BOM based on format selection
if (options.bomFormat === 'cdx' || options.bomFormat === 'both') {
console.log('\nGenerating CycloneDX BOM...');
generateCycloneDX(ast, options.outputDir);
}
if (options.bomFormat === 'spdx' || options.bomFormat === 'both') {
console.log('\nGenerating SPDX BOM...');
generateSPDX(ast, options.outputDir);
}
if (options.bomFormat === 'spdx' || options.bomFormat === 'both') {
console.log('\nGenerating SPDX BOM...');
generateSPDX(ast, options.outputDir);
}
console.log(`\nCompilation complete. Output: ${options.outputDir}/`);
}

2
compiler/dist/index.js.map vendored
View file

@ -1 +1 @@
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AAEA,OAAO,KAAK,MAAM,OAAO,CAAC;AAC1B,OAAO,EAAE,OAAO,EAAE,MAAM,eAAe,CAAC;AACxC,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAC7C,OAAO,EAAE,uBAAuB,EAAE,MAAM,4BAA4B,CAAC;AACrE,OAAO,EAAE,iBAAiB,EAAE,MAAM,2BAA2B,CAAC;AAC9D,OAAO,EAAE,YAAY,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,EAAE,8BAA8B,EAAE,MAAM,0CAA0C,CAAC;AAC1F,OAAO,EAAE,UAAU,EAAa,yBAAyB,EAAE,mBAAmB,EAAE,MAAM,oBAAoB,CAAC;AAC3G,OAAO,EAAE,YAAY,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,EAAE,MAAM,IAAI,CAAC;AACpB,OAAO,IAAI,MAAM,MAAM,CAAC;AASxB,KAAK,UAAU,IAAI;IACjB,uDAAuD;IACvD,MAAM,WAAW,GAAG,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,QAAQ,CAAC,EAAE,IAAI,EAAE,eAAe,CAAC,CAAC;IACtG,IAAI,EAAE,CAAC,UAAU,CAAC,WAAW,CAAC,EAAE,CAAC;QAC/B,IAAI,CAAC;YACH,MAAM,EAAE,QAAQ,EAAE,GAAG,MAAM,MAAM,CAAC,eAAe,CAAC,CAAC;YACnD,QAAQ,CAAC,SAAS,WAAW,GAAG,EAAE,EAAE,KAAK,EAAE,SAAS,EAAE,CAAC,CAAC;QAC1D,CAAC;QAAC,MAAM,CAAC;YACP,OAAO,CAAC,KAAK,CAAC,oDAAoD,CAAC,CAAC;YACpE,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QAClB,CAAC;IACH,CAAC;IAED,MAAM,IAAI,GAAG,MAAM,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;SAC5C,OAAO,CAAC,WAAW,EAAE,iCAAiC,EAAE,CAAC,QAAQ,EAAE,EAAE;QACpE,OAAO,QAAQ;aACZ,UAAU,CAAC,MAAM,EAAE;YAClB,QAAQ,EAAE,yBAAyB;YACnC,IAAI,EAAE,QAAQ;YACd,YAAY,EAAE,IAAI;SACnB,CAAC;aACD,MAAM,CAAC,YAAY,EAAE;YACpB,KAAK,EAAE,GAAG;YACV,QAAQ,EAAE,yCAAyC;YACnD,IAAI,EAAE,QAAQ;YACd,OAAO,EAAE,OAAO;SACjB,CAAC;aACD,MAAM,CAAC,QAAQ,EAAE;YAChB,KAAK,EAAE,GAAG;YACV,QAAQ,EAAE,oBAAoB;YAC9B,IAAI,EAAE,QAAQ;YACd,OAAO,EAAE,CAAC,YAAY,EAAE,IAAI,EAAE,MAAM,EAAE,KAAK,CAAC;YAC5C,OAAO,EAAE,YAAqB;SAC/B,CAAC;aACD,MAAM,CAAC,cAAc,EAAE;YACtB,KAAK,EAAE,GAAG;YACV,QAAQ,EAAE,kDAAkD;YAC5D,IAAI,EAAE,QAAQ;YACd,OAAO,EAAE,EAAE;SACZ,CAAC;aACD,MAAM,CAAC,YAAY,EAAE;YACpB,KAAK,EAAE,GAAG;YACV,QAAQ,EAAE,mBAAmB;YAC7B,IAAI,EAAE,QAAQ;YACd,OAAO,EAAE,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,CAAC;YAChC,OAAO,EAAE,MAAe;SACzB,CAAC,CAAC;IACP,CAAC,CAAC;SACD,IAAI,EAAE;SACN,IAAI,CAAC;IAER,MAAM,IAAI,GAAG,IAAI,CAAC,IAAc,CAAC;IACjC,MAAM,OAAO,GAAG;QACd,SAAS,EAAE,IAAI,CAAC,SAAmB;QACnC,SAAS,EAAE,IAAI,CAAC,SAAoC;QACpD,MAAM,EAAE,IAAI,CAAC,MAA8C;QAC3D,WAAW,EAAE,IAAI,CAAC,WAAqB;KACxC,CAAC;IAEF,OAAO,CAAC,GAAG,CAAC,aAAa,IAAI,KAAK,CAAC,CAAC;IAEpC,uBAAuB;IACvB,MAAM,GAAG,GAAG,cAAc,CAAC,IAAI,CAAC,CAAC;IAEjC,MAAM,UAAU,GAAG,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC;IACrC,MAAM,WAAW,GAAG,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;IAC7E,OAAO,CAAC,GAAG,CAAC,4BAA4B,UAAU,cAAc,WAAW,YAAY,CAAC,CAAC;IAEzF,iCAAiC;IACjC,IAAI,CAAC,EAAE,CAAC,UAAU,CAAC,OAAO,CAAC,SAAS,CAAC,EAAE,CAAC;QACtC,EAAE,CAAC,SAAS,CAAC,OAAO,CAAC,SAAS,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;IACvD,CAAC;IAED,oFAAoF;IACpF,MAAM,SAAS,GAAG,EAAE,CAAC,YAAY,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC;IAEjD,uDAAuD;IACvD,IAAI,OAAO,CAAC,MAAM,KAAK,MAAM,IAAI,OAAO,CAAC,MAAM,KAAK,KAAK,EAAE,CAAC;QAC1D,OAAO,CAAC,GAAG,CAAC,oCAAoC,CAAC,CAAC;QAClD,YAAY,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;IACvC,CAAC;IAED,gFAAgF;IAChF,IAAI,OAAO,CAAC,MAAM,KAAK,IAAI,IAAI,OAAO,CAAC,MAAM,KAAK,MAAM,EAAE,CAAC;QACzD,OAAO,CAAC,GAAG,CAAC,mDAAmD,CAAC,CAAC;QACjE,uBAAuB,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;IAClD,CAAC;IAED,8CAA8C;IAC9C,IAAI,OAAO,CAAC,MAAM,KAAK,IAAI,IAAI,OAAO,CAAC,MAAM,KAAK,KAAK,EAAE,CAAC;QACxD,OAAO,CAAC,GAAG,CAAC,6BAA6B,CAAC,CAAC;QAC3C,kDAAkD;QAClD,KAAK,MAAM,KAAK,IAAI,GAAG,CAAC,MAAM,EAAE,CAAC;YAC/B,MAAM,UAAU,GAAG,yBAAyB,CAAC,SAAS,EAAE,KAAK,CAAC,IAAI,CAAC,CAAC;YACpE,MAAM,QAAQ,GAAG,mBAAmB,CAAC,SAAS,EAAE,KAAK,CAAC,IAAI,CAAC,CAAC;YAC5D,kDAAkD;YACjD,KAAa,CAAC,WAAW,GAAG,UAAU,CAAC;YACvC,KAAa,CAAC,KAAK,GAAG,QAAQ,CAAC;QAClC,CAAC;QACD,MAAM,SAAS,GAAc,EAAE,WAAW,EAAE,OAAO,CAAC,WAAW,EAAE,CAAC;QAClE,UAAU,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,EAAE,SAAS,CAAC,CAAC;IAChD,CAAC;IAED,gEAAgE;IAChE,IAAI,OAAO,CAAC,MAAM,KAAK,IAAI,IAAI,OAAO,CAAC,MAAM,KAAK,MAAM,EAAE,CAAC;QACzD,OAAO,CAAC,GAAG,CAAC,0CAA0C,CAAC,CAAC;QACxD,8BAA8B,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;QAEvD,yCAAyC;QACzC,IAAI,OAAO,CAAC,SAAS,KAAK,KAAK,IAAI,OAAO,CAAC,SAAS,KAAK,MAAM,EAAE,CAAC;YAChE,OAAO,CAAC,GAAG,CAAC,+BAA+B,CAAC,CAAC;YAC7C,iBAAiB,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;QAC5C,CAAC;QAED,IAAI,OAAO,CAAC,SAAS,KAAK,MAAM,IAAI,OAAO,CAAC,SAAS,KAAK,MAAM,EAAE,CAAC;YACjE,OAAO,CAAC,GAAG,CAAC,0BAA0B,CAAC,CAAC;YACxC,YAAY,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;QACvC,CAAC;IACH,CAAC;IAED,OAAO,CAAC,GAAG,CAAC,mCAAmC,OAAO,CAAC,SAAS,GAAG,CAAC,CAAC;AACvE,CAAC;AAED,IAAI,EAAE,CAAC,KAAK,CAAC,CAAC,GAAG,EAAE,EAAE;IACnB,OAAO,CAAC,KAAK,CAAC,qBAAqB,EAAE,GAAG,CAAC,CAAC;IAC1C,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC"}
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AAEA,OAAO,KAAK,MAAM,OAAO,CAAC;AAC1B,OAAO,EAAE,OAAO,EAAE,MAAM,eAAe,CAAC;AACxC,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAC7C,OAAO,EAAE,uBAAuB,EAAE,MAAM,4BAA4B,CAAC;AACrE,OAAO,EAAE,iBAAiB,EAAE,MAAM,2BAA2B,CAAC;AAC9D,OAAO,EAAE,YAAY,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,EAAE,8BAA8B,EAAE,MAAM,0CAA0C,CAAC;AAC1F,OAAO,EAAE,YAAY,EAAE,MAAM,sBAAsB,CAAC;AACpD,OAAO,EAAE,MAAM,IAAI,CAAC;AACpB,OAAO,IAAI,MAAM,MAAM,CAAC;AAOxB,KAAK,UAAU,IAAI;IACjB,uDAAuD;IACvD,MAAM,WAAW,GAAG,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,QAAQ,CAAC,EAAE,IAAI,EAAE,eAAe,CAAC,CAAC;IACtG,IAAI,EAAE,CAAC,UAAU,CAAC,WAAW,CAAC,EAAE,CAAC;QAC/B,IAAI,CAAC;YACH,MAAM,EAAE,QAAQ,EAAE,GAAG,MAAM,MAAM,CAAC,eAAe,CAAC,CAAC;YACnD,QAAQ,CAAC,SAAS,WAAW,GAAG,EAAE,EAAE,KAAK,EAAE,SAAS,EAAE,CAAC,CAAC;QAC1D,CAAC;QAAC,MAAM,CAAC;YACP,OAAO,CAAC,KAAK,CAAC,oDAAoD,CAAC,CAAC;YACpE,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QAClB,CAAC;IACH,CAAC;IAED,MAAM,IAAI,GAAG,MAAM,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;SAC5C,OAAO,CAAC,WAAW,EAAE,iCAAiC,EAAE,CAAC,QAAQ,EAAE,EAAE;QACpE,OAAO,QAAQ;aACZ,UAAU,CAAC,MAAM,EAAE;YAClB,QAAQ,EAAE,yBAAyB;YACnC,IAAI,EAAE,QAAQ;YACd,YAAY,EAAE,IAAI;SACnB,CAAC;aACD,MAAM,CAAC,YAAY,EAAE;YACpB,KAAK,EAAE,GAAG;YACV,QAAQ,EAAE,yCAAyC;YACnD,IAAI,EAAE,QAAQ;YACd,OAAO,EAAE,OAAO;SACjB,CAAC;aACD,MAAM,CAAC,YAAY,EAAE;YACpB,KAAK,EAAE,GAAG;YACV,QAAQ,EAAE,mBAAmB;YAC7B,IAAI,EAAE,QAAQ;YACd,OAAO,EAAE,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,CAAC;YAChC,OAAO,EAAE,MAAe;SACzB,CAAC,CAAC;IACP,CAAC,CAAC;SACD,IAAI,EAAE;SACN,IAAI,CAAC;IAER,MAAM,IAAI,GAAG,IAAI,CAAC,IAAc,CAAC;IACjC,MAAM,OAAO,GAAG;QACd,SAAS,EAAE,IAAI,CAAC,SAAmB;QACnC,SAAS,EAAE,IAAI,CAAC,SAAoC;KACrD,CAAC;IAEF,OAAO,CAAC,GAAG,CAAC,aAAa,IAAI,KAAK,CAAC,CAAC;IAEpC,uBAAuB;IACvB,MAAM,GAAG,GAAG,cAAc,CAAC,IAAI,CAAC,CAAC;IAEjC,MAAM,UAAU,GAAG,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC;IACrC,MAAM,WAAW,GAAG,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;IAC7E,OAAO,CAAC,GAAG,CAAC,4BAA4B,UAAU,cAAc,WAAW,YAAY,CAAC,CAAC;IAEzF,iCAAiC;IACjC,IAAI,CAAC,EAAE,CAAC,UAAU,CAAC,OAAO,CAAC,SAAS,CAAC,EAAE,CAAC;QACtC,EAAE,CAAC,SAAS,CAAC,OAAO,CAAC,SAAS,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;IACvD,CAAC;IAED,oDAAoD;IACpD,OAAO,CAAC,GAAG,CAAC,mDAAmD,CAAC,CAAC;IACjE,uBAAuB,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;IAEhD,oCAAoC;IACpC,OAAO,CAAC,GAAG,CAAC,0CAA0C,CAAC,CAAC;IACxD,8BAA8B,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;IAEvD,8BAA8B;IAC9B,OAAO,CAAC,GAAG,CAAC,mCAAmC,CAAC,CAAC;IACjD,YAAY,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;IAErC,yCAAyC;IACzC,IAAI,OAAO,CAAC,SAAS,KAAK,KAAK,IAAI,OAAO,CAAC,SAAS,KAAK,MAAM,EAAE,CAAC;QAChE,OAAO,CAAC,GAAG,CAAC,+BAA+B,CAAC,CAAC;QAC7C,iBAAiB,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;IAC5C,CAAC;IAED,IAAI,OAAO,CAAC,SAAS,KAAK,MAAM,IAAI,OAAO,CAAC,SAAS,KAAK,MAAM,EAAE,CAAC;QACjE,OAAO,CAAC,GAAG,CAAC,0BAA0B,CAAC,CAAC;QACxC,YAAY,CAAC,GAAG,EAAE,OAAO,CAAC,SAAS,CAAC,CAAC;IACvC,CAAC;IAED,OAAO,CAAC,GAAG,CAAC,mCAAmC,OAAO,CAAC,SAAS,GAAG,CAAC,CAAC;AACvE,CAAC;AAED,IAAI,EAAE,CAAC,KAAK,CAAC,CAAC,GAAG,EAAE,EAAE;IACnB,OAAO,CAAC,KAAK,CAAC,qBAAqB,EAAE,GAAG,CAAC,CAAC;IAC1C,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC"}

6
compiler/dist/parser.d.ts vendored
View file

@ -20,14 +20,11 @@ export interface OutputNode {
}
export interface ConstraintNode {
type: 'constraint';
name?: string;
expr: string;
}
export interface VerifyNode {
type: 'verify';
name: string;
temporalExpr: string;
depth?: number;
}
export interface InterfaceNode {
type: 'interface';
@ -152,9 +149,6 @@ export type ActionStatement = {
} | {
type: 'return';
expr: string;
} | {
type: 'ifblock';
raw: string;
};
export type MemberNode = ImportNode | InputNode | OutputNode | ConstraintNode | VerifyNode | InterfaceNode | EventNode | ProvenanceNode | ReviewNode | DerivesNode | SatisfiesNode | AuthorNode | SourceNode | BOMNode | StateNode | InitNode | ActionNode;
export interface TypeExpr {

2
compiler/dist/parser.d.ts.map vendored
View file

@ -1 +1 @@
{"version":3,"file":"parser.d.ts","sourceRoot":"","sources":["../src/parser.ts"],"names":[],"mappings":"AAIA,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,UAAU,EAAE,CAAC;CACvB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,IAAI,CAAC,EAAE,MAAM,CAAC;CACf;AAED,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,QAAQ,EAAE,QAAQ,CAAC;CACpB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,QAAQ,EAAE,QAAQ,CAAC;CACpB;AAED,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,YAAY,CAAC;IACnB,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;CACd;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,YAAY,EAAE,MAAM,CAAC;IACrB,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,WAAW,aAAa;IAC5B,IAAI,EAAE,WAAW,CAAC;IAClB,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,eAAe,EAAE,CAAC;CAC5B;AAED,MAAM,WAAW,eAAe;IAC9B,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,QAAQ,CAAA;KAAE,EAAE,CAAC;IAC3C,UAAU,EAAE,QAAQ,CAAC;CACtB;AAED,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,QAAQ,CAAA;KAAE,EAAE,CAAC;CAC5C;AAED,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,YAAY,CAAC;IACnB,OAAO,EAAE,gBAAgB,EAAE,CAAC;CAC7B;AAED,MAAM,WAAW,gBAAgB;IAC/B,IAAI,EAAE,YAAY,GAAG,iBAAiB,GAAG,aAAa,GAAG,cAAc,CAAC;IACxE,KAAK,EAAE,MAAM,CAAC;IACd,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,QAAQ,GAAG,MAAM,GAAG,QAAQ,CAAC;CACpC;AAED,MAAM,WAAW,WAAW;IAC1B,IAAI,EAAE,SAAS,CAAC;IAChB,IAAI,EAAE,MAAM,CAAC;IACb,GAAG,CAAC,EAAE,MAAM,CAAC;CACd;AAED,MAAM,WAAW,aAAa;IAC5B,IAAI,EAAE,WAAW,CAAC;IAClB,WAAW,EAAE,MAAM,CAAC;IACpB,MAAM,CAAC,EAAE,MAAM,CAAC;CACjB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,cAAc,GAAG,SAAS,GAAG,UAAU,GAAG,YAAY,GACrD,qBAAqB,GAAG,cAAc,GAAG,kBAAkB,CAAC;IACnE,GAAG,CAAC,EAAE,MAAM,CAAC;CACd;AAED,MAAM,WAAW,OAAO;IACtB,IAAI,EAAE,KAAK,CAAC;IACZ,QAAQ,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,OAAO,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IAC9C,MAAM,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,OAAO,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IAC5C,OAAO,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,OAAO,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IAC7C,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,IAAI,CAAC,EAAE,MAAM,CAAC;CACf;AAGD,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,SAAS,EAAE,QAAQ,EAAE,CAAC;CACvB;AAED,MAAM,WAAW,QAAQ;IACvB,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,QAAQ,CAAC;IACnB,WAAW,CAAC,EAAE,MAAM,CAAC;CACtB;AAED,MAAM,WAAW,QAAQ;IACvB,IAAI,EAAE,MAAM,CAAC;IACb,WAAW,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAA;KAAE,EAAE,CAAC;CAC/C;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,QAAQ,CAAA;KAAE,EAAE,CAAC;IAC3C,UAAU,EAAE,eAAe,EAAE,CAAC;CAC/B;AAED,MAAM,MAAM,eAAe,GACvB;IAAE,IAAI,EAAE,QAAQ,CAAC;IAAC,MAAM,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GAChD;IAAE,IAAI,EAAE,KAAK,CAAC;IAAC,IAAI,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GAC3C;IAAE,IAAI,EAAE,SAAS,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GACjC;IAAE,IAAI,EAAE,cAAc,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GACtC;IAAE,IAAI,EAAE,eAAe,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GACvC;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,KAAK,EAAE,MAAM,CAAC;IAAC,MAAM,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAA;KAAE,EAAE,CAAA;CAAE,GAC1E;IAAE,IAAI,EAAE,QAAQ,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GAChC;IAAE,IAAI,EAAE,SAAS,CAAC;IAAC,GAAG,EAAE,MAAM,CAAA;CAAE,CAAC;AAErC,MAAM,MAAM,UAAU,GAClB,UAAU,GACV,SAAS,GACT,UAAU,GACV,cAAc,GACd,UAAU,GACV,aAAa,GACb,SAAS,GACT,cAAc,GACd,UAAU,GACV,WAAW,GACX,aAAa,GACb,UAAU,GACV,UAAU,GACV,OAAO,GACP,SAAS,GACT,QAAQ,GACR,UAAU,CAAC;AAEf,MAAM,WAAW,QAAQ;IACvB,IAAI,EAAE,WAAW,GAAG,QAAQ,GAAG,MAAM,GAAG,KAAK,GAAG,KAAK,GAAG,OAAO,CAAC;IAChE,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,MAAM,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,QAAQ,CAAA;KAAE,EAAE,CAAC;IAC5C,WAAW,CAAC,EAAE,QAAQ,CAAC;IACvB,OAAO,CAAC,EAAE,QAAQ,CAAC;IACnB,SAAS,CAAC,EAAE,QAAQ,CAAC;CACtB;AAED,MAAM,WAAW,GAAG;IAClB,MAAM,EAAE,SAAS,EAAE,CAAC;CACrB;AAED;;GAEG;AACH,wBAAgB,cAAc,CAAC,QAAQ,EAAE,MAAM,GAAG,GAAG,CAGpD;AAED;;GAEG;AACH,wBAAgB,iBAAiB,CAAC,OAAO,EAAE,MAAM,EAAE,QAAQ,CAAC,EAAE,MAAM,GAAG,GAAG,CAwCzE"}
{"version":3,"file":"parser.d.ts","sourceRoot":"","sources":["../src/parser.ts"],"names":[],"mappings":"AAIA,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,UAAU,EAAE,CAAC;CACvB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,IAAI,CAAC,EAAE,MAAM,CAAC;CACf;AAED,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,QAAQ,EAAE,QAAQ,CAAC;CACpB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,QAAQ,EAAE,QAAQ,CAAC;CACpB;AAED,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,YAAY,CAAC;IACnB,IAAI,EAAE,MAAM,CAAC;CACd;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,YAAY,EAAE,MAAM,CAAC;CACtB;AAED,MAAM,WAAW,aAAa;IAC5B,IAAI,EAAE,WAAW,CAAC;IAClB,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,eAAe,EAAE,CAAC;CAC5B;AAED,MAAM,WAAW,eAAe;IAC9B,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,QAAQ,CAAA;KAAE,EAAE,CAAC;IAC3C,UAAU,EAAE,QAAQ,CAAC;CACtB;AAED,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,QAAQ,CAAA;KAAE,EAAE,CAAC;CAC5C;AAED,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,YAAY,CAAC;IACnB,OAAO,EAAE,gBAAgB,EAAE,CAAC;CAC7B;AAED,MAAM,WAAW,gBAAgB;IAC/B,IAAI,EAAE,YAAY,GAAG,iBAAiB,GAAG,aAAa,GAAG,cAAc,CAAC;IACxE,KAAK,EAAE,MAAM,CAAC;IACd,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,QAAQ,GAAG,MAAM,GAAG,QAAQ,CAAC;CACpC;AAED,MAAM,WAAW,WAAW;IAC1B,IAAI,EAAE,SAAS,CAAC;IAChB,IAAI,EAAE,MAAM,CAAC;IACb,GAAG,CAAC,EAAE,MAAM,CAAC;CACd;AAED,MAAM,WAAW,aAAa;IAC5B,IAAI,EAAE,WAAW,CAAC;IAClB,WAAW,EAAE,MAAM,CAAC;IACpB,MAAM,CAAC,EAAE,MAAM,CAAC;CACjB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,cAAc,GAAG,SAAS,GAAG,UAAU,GAAG,YAAY,GACrD,qBAAqB,GAAG,cAAc,GAAG,kBAAkB,CAAC;IACnE,GAAG,CAAC,EAAE,MAAM,CAAC;CACd;AAED,MAAM,WAAW,OAAO;IACtB,IAAI,EAAE,KAAK,CAAC;IACZ,QAAQ,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,OAAO,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IAC9C,MAAM,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,OAAO,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IAC5C,OAAO,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,OAAO,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IAC7C,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,IAAI,CAAC,EAAE,MAAM,CAAC;CACf;AAGD,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,SAAS,EAAE,QAAQ,EAAE,CAAC;CACvB;AAED,MAAM,WAAW,QAAQ;IACvB,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,QAAQ,CAAC;IACnB,WAAW,CAAC,EAAE,MAAM,CAAC;CACtB;AAED,MAAM,WAAW,QAAQ;IACvB,IAAI,EAAE,MAAM,CAAC;IACb,WAAW,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAA;KAAE,EAAE,CAAC;CAC/C;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,QAAQ,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,QAAQ,CAAA;KAAE,EAAE,CAAC;IAC3C,UAAU,EAAE,eAAe,EAAE,CAAC;CAC/B;AAED,MAAM,MAAM,eAAe,GACvB;IAAE,IAAI,EAAE,QAAQ,CAAC;IAAC,MAAM,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GAChD;IAAE,IAAI,EAAE,KAAK,CAAC;IAAC,IAAI,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GAC3C;IAAE,IAAI,EAAE,SAAS,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GACjC;IAAE,IAAI,EAAE,cAAc,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GACtC;IAAE,IAAI,EAAE,eAAe,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GACvC;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,KAAK,EAAE,MAAM,CAAC;IAAC,MAAM,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAA;KAAE,EAAE,CAAA;CAAE,GAC1E;IAAE,IAAI,EAAE,QAAQ,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,CAAC;AAErC,MAAM,MAAM,UAAU,GAClB,UAAU,GACV,SAAS,GACT,UAAU,GACV,cAAc,GACd,UAAU,GACV,aAAa,GACb,SAAS,GACT,cAAc,GACd,UAAU,GACV,WAAW,GACX,aAAa,GACb,UAAU,GACV,UAAU,GACV,OAAO,GACP,SAAS,GACT,QAAQ,GACR,UAAU,CAAC;AAEf,MAAM,WAAW,QAAQ;IACvB,IAAI,EAAE,WAAW,GAAG,QAAQ,GAAG,MAAM,GAAG,KAAK,GAAG,KAAK,GAAG,OAAO,CAAC;IAChE,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,MAAM,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,QAAQ,CAAA;KAAE,EAAE,CAAC;IAC5C,WAAW,CAAC,EAAE,QAAQ,CAAC;IACvB,OAAO,CAAC,EAAE,QAAQ,CAAC;IACnB,SAAS,CAAC,EAAE,QAAQ,CAAC;CACtB;AAED,MAAM,WAAW,GAAG;IAClB,MAAM,EAAE,SAAS,EAAE,CAAC;CACrB;AAED;;GAEG;AACH,wBAAgB,cAAc,CAAC,QAAQ,EAAE,MAAM,GAAG,GAAG,CAGpD;AAED;;GAEG;AACH,wBAAgB,iBAAiB,CAAC,OAAO,EAAE,MAAM,EAAE,QAAQ,CAAC,EAAE,MAAM,GAAG,GAAG,CAwCzE"}

518
compiler/dist/parser.js vendored
View file

@ -63,9 +63,9 @@ function parseSpeck(lines, startIndex) {
// Count braces on this line first
const openCount = (rawLine.match(/{/g) || []).length;
const closeCount = (rawLine.match(/}/g) || []).length;
// Check for block-opening members (those that open braces on the same line)
// Check for block-opening members
const blockStarters = ['state:', 'init:', 'action ', 'event ', 'provenance ', 'bom ', 'interface ',
'state {', 'init {', 'verify ', 'constraint '];
'state {', 'init {'];
const isBlockStarter = blockStarters.some(s => line.startsWith(s));
if (isBlockStarter && openCount > 0) {
// Multi-line block: parse it and skip to its end
@ -73,39 +73,9 @@ function parseSpeck(lines, startIndex) {
if (member) {
members.push(member);
}
// Skip past this block if it spans multiple lines
if (closeCount < openCount) {
const endIdx = findBlockEnd(lines, i + 1, braceCount + openCount);
i = endIdx + 1;
}
else {
i++;
}
continue;
}
// Handle constraint/verify blocks that may span multiple lines without braces
// (e.g. constraint Name: forall r in [0..4]:)
if (line.startsWith('constraint ') || line.startsWith('verify ')) {
const member = parseMultiLineConstraintVerify(lines, i);
if (member) {
members.push(member);
// Skip past the consumed continuation lines
while (i + 1 < lines.length) {
const nextLine = lines[i + 1].trim();
if (!nextLine || nextLine.startsWith('//') || nextLine.startsWith('/*') ||
nextLine.startsWith('constraint ') || nextLine.startsWith('verify ') ||
nextLine.startsWith('constraint:') || nextLine.startsWith('verify:') ||
nextLine.startsWith('state') || nextLine.startsWith('init') ||
nextLine.startsWith('action ') || nextLine.startsWith('event ') ||
nextLine.startsWith('interface ') || nextLine.startsWith('provenance ') ||
nextLine.startsWith('bom ') || nextLine.startsWith('}') ||
nextLine.startsWith('next:') || nextLine.startsWith('invariant ')) {
break;
}
i++;
}
}
i++;
// Skip past this block
const endIdx = findBlockEnd(lines, i + 1, braceCount + openCount);
i = endIdx + 1;
continue;
}
// Single-line: update brace count, then parse if still inside
@ -141,25 +111,6 @@ function findBlockEnd(lines, startIndex, startBraceCount) {
}
return i;
}
// Check if an expression has balanced braces, parens, and brackets
function isExpressionComplete(expr) {
let braces = 0, parens = 0, brackets = 0;
for (const char of expr) {
if (char === '{')
braces++;
else if (char === '}')
braces--;
else if (char === '(')
parens++;
else if (char === ')')
parens--;
else if (char === '[')
brackets++;
else if (char === ']')
brackets--;
}
return braces === 0 && parens === 0 && brackets === 0;
}
function parseMemberBlock(lines, startIndex) {
const firstLine = lines[startIndex].trim();
const startBraceCount = (firstLine.match(/{/g) || []).length;
@ -190,12 +141,6 @@ function parseMemberBlock(lines, startIndex) {
if (firstLine.startsWith('bom ')) {
return { type: 'bom' };
}
if (firstLine.startsWith('verify ')) {
return parseVerifyBlockMultiline(lines, startIndex, startBraceCount);
}
if (firstLine.startsWith('constraint ')) {
return parseConstraintBlockMultiline(lines, startIndex, startBraceCount);
}
return null;
}
function parseMember(line) {
@ -209,25 +154,13 @@ function parseMember(line) {
const typeExpr = parseTypeExpr(line.substring(7).trim());
return { type: 'output', typeExpr };
}
// constraint Name: expr (single-line named constraint)
if (line.startsWith('constraint ')) {
const match = line.match(/^constraint\s+(\w+):\s*(.+)$/);
if (match) {
return { type: 'constraint', name: match[1], expr: match[2].trim() };
}
return null; // multi-line constraint block (handled by parseMemberBlock)
}
// constraint: expr (unnamed, single-line)
// constraint:
if (line.startsWith('constraint:')) {
return { type: 'constraint', expr: line.substring(11).trim() };
}
// verify Always(Name) { depth N } or verify Always(Name) (single-line forms)
if (line.startsWith('verify ')) {
return parseVerifySingleLine(line);
}
// verify: expr (old unnamed single-line)
// verify:
if (line.startsWith('verify:')) {
return { type: 'verify', name: 'unnamed', temporalExpr: line.substring(7).trim() };
return { type: 'verify', temporalExpr: line.substring(7).trim() };
}
// interface:
if (line.startsWith('interface ')) {
@ -325,28 +258,18 @@ function parseStateBlock(inner) {
for (const vStr of varStrs) {
const withDefault = vStr.match(/^(\w+)\s*:\s*(.+?)\s*:=\s*(.+)$/);
if (withDefault) {
// Strip trailing comments from type expression
let typeStr = withDefault[2].trim();
const commentIdx = typeStr.indexOf('//');
if (commentIdx >= 0)
typeStr = typeStr.substring(0, commentIdx).trim();
variables.push({
name: withDefault[1],
typeExpr: parseTypeExpr(typeStr),
typeExpr: parseTypeExpr(withDefault[2].trim()),
defaultInit: withDefault[3].trim(),
});
continue;
}
const withoutDefault = vStr.match(/^(\w+)\s*:\s*(.+)$/);
if (withoutDefault) {
// Strip trailing comments from type expression
let typeStr = withoutDefault[2].trim();
const commentIdx = typeStr.indexOf('//');
if (commentIdx >= 0)
typeStr = typeStr.substring(0, commentIdx).trim();
variables.push({
name: withoutDefault[1],
typeExpr: parseTypeExpr(typeStr),
typeExpr: parseTypeExpr(withoutDefault[2].trim()),
});
}
}
@ -417,20 +340,15 @@ function parseTypeExpr(expr) {
// Map<K, V>
if (expr.startsWith('Map<') && expr.endsWith('>')) {
const inner = expr.substring(4, expr.length - 1);
// Find the first-level comma (not inside nested generics or record literals)
// Find the first-level comma (not inside nested generics)
let depth = 0;
let braceDepth = 0;
let commaIdx = -1;
for (let i = 0; i < inner.length; i++) {
if (inner[i] === '<' || inner[i] === '(')
if (inner[i] === '<')
depth++;
else if (inner[i] === '>' || inner[i] === ')')
else if (inner[i] === '>')
depth--;
else if (inner[i] === '{')
braceDepth++;
else if (inner[i] === '}')
braceDepth--;
else if (inner[i] === ',' && depth === 0 && braceDepth === 0) {
else if (inner[i] === ',' && depth === 0) {
commaIdx = i;
break;
}
@ -457,18 +375,13 @@ function parseTypeExpr(expr) {
if (expr.startsWith('Map(') && expr.endsWith(')')) {
const inner = expr.substring(4, expr.length - 1);
let depth = 0;
let braceDepth = 0;
let commaIdx = -1;
for (let i = 0; i < inner.length; i++) {
if (inner[i] === '(' || inner[i] === '<')
depth++;
else if (inner[i] === ')' || inner[i] === '>')
depth--;
else if (inner[i] === '{')
braceDepth++;
else if (inner[i] === '}')
braceDepth--;
else if (inner[i] === ',' && depth === 0 && braceDepth === 0) {
else if (inner[i] === ',' && depth === 0) {
commaIdx = i;
break;
}
@ -612,42 +525,13 @@ function parseSource(line) {
function parseStateBlockMultiline(lines, startIndex, startBraceCount) {
const endIndex = findBlockEnd(lines, startIndex + 1, startBraceCount);
const innerLines = lines.slice(startIndex + 1, endIndex).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
// State blocks separate fields with commas or newlines/semicolons.
// Strip trailing commas (they are field separators, not continuations)
// then join with semicolons between complete variable declarations.
const cleanedLines = innerLines.map(l => l.replace(/,+\s*$/, ''));
const inner = cleanedLines.reduce((acc, line, i) => {
if (i === 0)
return line;
const prevLine = cleanedLines[i - 1];
// Check if this line starts a new variable declaration (name: type)
const isNewVar = /^\w+\s*:/.test(line);
// Check if prev line ends with a continuation character (brace or paren, NOT comma)
const prevIsContinuation = /[{(]\s*$/.test(prevLine);
if (isNewVar && !prevIsContinuation) {
return acc + '; ' + line;
}
return acc + ' ' + line;
}, '');
const inner = innerLines.join('; ');
return parseStateBlock(inner);
}
function parseInitBlockMultiline(lines, startIndex, startBraceCount) {
const endIndex = findBlockEnd(lines, startIndex + 1, startBraceCount);
const innerLines = lines.slice(startIndex + 1, endIndex).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
// Init blocks separate fields with commas or newlines/semicolons.
// Strip trailing commas and join with semicolons.
const cleanedLines = innerLines.map(l => l.replace(/,+\s*$/, ''));
const inner = cleanedLines.reduce((acc, line, i) => {
if (i === 0)
return line;
const prevLine = cleanedLines[i - 1];
const isNewVar = /^\w+\s*:/.test(line);
const prevIsContinuation = /[{(]\s*$/.test(prevLine);
if (isNewVar && !prevIsContinuation) {
return acc + '; ' + line;
}
return acc + ' ' + line;
}, '');
const inner = innerLines.join('; ');
return parseInitBlock(inner);
}
function parseActionBlockMultiline(lines, startIndex, startBraceCount) {
@ -656,41 +540,17 @@ function parseActionBlockMultiline(lines, startIndex, startBraceCount) {
if (!header)
return null;
const endIndex = findBlockEnd(lines, startIndex + 1, startBraceCount);
// Keep blank lines as null to separate statements, filter only comments
const bodyLinesRaw = lines.slice(startIndex + 1, endIndex).map(l => {
const trimmed = l.trim();
if (!trimmed || trimmed.startsWith('//') || trimmed.startsWith('/*'))
return null;
return trimmed;
});
// Collapse consecutive nulls and strip trailing nulls
const bodyLines = [];
for (const line of bodyLinesRaw) {
if (line !== null || (bodyLines.length > 0 && bodyLines[bodyLines.length - 1] !== null)) {
bodyLines.push(line);
}
}
// Remove trailing nulls
while (bodyLines.length > 0 && bodyLines[bodyLines.length - 1] === null) {
bodyLines.pop();
}
const bodyLines = lines.slice(startIndex + 1, endIndex).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
const statements = [];
for (let i = 0; i < bodyLines.length; i++) {
const stmtOrNull = bodyLines[i];
// Skip blank line separators
if (stmtOrNull === null)
continue;
const stmt = stmtOrNull;
const stmt = bodyLines[i];
// precondition: expr
if (stmt.startsWith('precondition:')) {
let expr = stmt.substring(13).trim();
// support 'or' continuation lines
while (i + 1 < bodyLines.length) {
const nextLine = bodyLines[i + 1];
if (nextLine === null || !nextLine.startsWith('or '))
break;
while (i + 1 < bodyLines.length && bodyLines[i + 1].startsWith('or ')) {
i++;
expr += ' || ' + nextLine.substring(3).trim();
expr += ' || ' + bodyLines[i].substring(3).trim();
}
statements.push({ type: 'precondition', expr });
continue;
@ -699,35 +559,16 @@ function parseActionBlockMultiline(lines, startIndex, startBraceCount) {
if (stmt.startsWith('postcondition:')) {
let expr = stmt.substring(14).trim();
// support 'or' continuation lines
while (i + 1 < bodyLines.length) {
const nextLine = bodyLines[i + 1];
if (nextLine === null || !nextLine.startsWith('or '))
break;
while (i + 1 < bodyLines.length && bodyLines[i + 1].startsWith('or ')) {
i++;
expr += ' || ' + nextLine.substring(3).trim();
expr += ' || ' + bodyLines[i].substring(3).trim();
}
statements.push({ type: 'postcondition', expr });
continue;
}
// require expr
if (stmt.startsWith('require ')) {
let expr = stmt.substring(8).trim();
// support multi-line continuation (forall, or, etc.)
while (i + 1 < bodyLines.length) {
const nextLine = bodyLines[i + 1];
if (nextLine === null)
break;
// Stop if next line starts a new statement keyword
if (nextLine.startsWith('precondition:') || nextLine.startsWith('postcondition:') ||
nextLine.startsWith('require ') || nextLine.startsWith('return ') ||
nextLine.startsWith('emit ') || nextLine.startsWith('let ') ||
nextLine.includes(':=')) {
break;
}
i++;
expr += ' ' + nextLine;
}
statements.push({ type: 'require', expr });
statements.push({ type: 'require', expr: stmt.substring(8).trim() });
continue;
}
// return expr
@ -742,7 +583,7 @@ function parseActionBlockMultiline(lines, startIndex, startBraceCount) {
const fieldsStr = emitMatch[2];
const fields = [];
if (fieldsStr) {
const fieldPairs = splitFields(fieldsStr);
const fieldPairs = fieldsStr.split(',').map(s => s.trim()).filter(Boolean);
for (const fp of fieldPairs) {
const colonIdx = fp.indexOf(':');
if (colonIdx > 0) {
@ -756,52 +597,16 @@ function parseActionBlockMultiline(lines, startIndex, startBraceCount) {
statements.push({ type: 'emit', event: eventName, fields });
continue;
}
// if condition { ... } else if condition { ... } else { ... }
// SpeckDL conditional blocks - parse as a special statement type
if (stmt.startsWith('if ')) {
// Find matching } for this if block (may span multiple bodyLines entries)
let braceDepth = 0;
let j = i;
const blockLines = [];
while (j < bodyLines.length) {
const line = bodyLines[j];
if (line === null) {
j++;
continue;
}
blockLines.push(line);
for (const ch of line) {
if (ch === '{')
braceDepth++;
else if (ch === '}')
braceDepth--;
}
j++;
if (braceDepth === 0)
break;
}
// For now, treat as a raw 'ifblock' statement that the generator can handle
statements.push({ type: 'ifblock', raw: blockLines.join(' ') });
i = j - 1; // skip to end of if block
continue;
}
// let name := expr
const letMatch = stmt.match(/^let\s+(\w+)\s*:=\s*(.+)$/);
if (letMatch) {
statements.push({ type: 'let', name: letMatch[1], expr: letMatch[2].trim() });
continue;
}
// assignment: target := expr (may span multiple lines)
// assignment: target := expr
const assignMatch = stmt.match(/^(\w+(?:\[.*?\])?)\s*:=\s*(.+)$/);
if (assignMatch) {
let target = assignMatch[1];
let expr = assignMatch[2].trim();
// Handle multiline expressions: join subsequent lines until braces/parens are balanced
while (i + 1 < bodyLines.length && !isExpressionComplete(expr)) {
i++;
expr += ' ' + bodyLines[i];
}
statements.push({ type: 'assign', target, expr });
statements.push({ type: 'assign', target: assignMatch[1], expr: assignMatch[2].trim() });
}
}
return { ...header, statements };
@ -827,266 +632,25 @@ function parseInterfaceBlockMultiline(lines, startIndex, startBraceCount) {
const innerLines = lines.slice(startIndex + 1, endIndex).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
const methods = [];
for (const line of innerLines) {
// name-only: enum-style values (e.g., "Idle", "Idle, Parsing" comma-separated)
// Split on comma and take each trimmed token
const tokens = line.split(',').map(s => s.trim()).filter(Boolean);
for (const token of tokens) {
// Check if it’s a name(params) -> returnType method signature
const methodMatch = token.match(/^(\w+)\s*\(([^)]*)\)\s*->\s*(.+?),?$/);
if (methodMatch) {
const name = methodMatch[1];
const paramsStr = methodMatch[2];
const returnTypeStr = methodMatch[3].replace(/,$/, '').trim();
const params = [];
if (paramsStr.trim()) {
const paramPairs = paramsStr.split(',').map(s => s.trim()).filter(Boolean);
for (const p of paramPairs) {
const pMatch = p.match(/^(\w+)\s*:\s*(.+)$/);
if (pMatch) {
params.push({ name: pMatch[1], type: parseTypeExpr(pMatch[2].trim()) });
}
// name(params) -> returnType
const match = line.match(/^(\w+)\s*\(([^)]*)\)\s*-\u003e\s*(.+?),?$/);
if (match) {
const name = match[1];
const paramsStr = match[2];
const returnTypeStr = match[3].replace(/,$/, '').trim();
const params = [];
if (paramsStr.trim()) {
const paramPairs = paramsStr.split(',').map(s => s.trim()).filter(Boolean);
for (const p of paramPairs) {
const pMatch = p.match(/^(\w+)\s*:\s*(.+)$/);
if (pMatch) {
params.push({ name: pMatch[1], type: parseTypeExpr(pMatch[2].trim()) });
}
}
methods.push({ name, params, returnType: parseTypeExpr(returnTypeStr) });
}
else if (/^\w+$/.test(token)) {
// Name-only (enum-style) — treat as a method with no params and void return
methods.push({ name: token, params: [], returnType: parseTypeExpr('Void') });
}
methods.push({ name, params, returnType: parseTypeExpr(returnTypeStr) });
}
}
return { ...header, methods };
}
// --- Multi-line constraint/verify parsing (no braces needed) ---
/**
* Parse a constraint or verify that may span multiple lines without braces.
* Used for forall-style constraints like:
* constraint Name: forall r in [0..4]:
* (expr)
*
* and verify blocks like:
* verify Always(Name)
* verify Always(Name) { depth N }
*/
function parseMultiLineConstraintVerify(lines, startIndex) {
const line = lines[startIndex].trim();
if (line.startsWith('verify ')) {
return parseVerifyAnyForm(lines, startIndex);
}
if (line.startsWith('constraint ')) {
return parseConstraintAnyForm(lines, startIndex);
}
return null;
}
function parseConstraintAnyForm(lines, startIndex) {
const line = lines[startIndex].trim();
const namedMatch = line.match(/^constraint\s+(\w+):(.*)$/);
if (!namedMatch)
return null;
const name = namedMatch[1];
let expr = namedMatch[2].trim();
// If opens a brace, treat as block
if (expr.startsWith('{')) {
const afterBrace = expr.substring(1).trim();
const openCount = (line.match(/{/g) || []).length;
const endIdx = findBlockEnd(lines, startIndex + 1, openCount);
const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
const inner = innerLines.join(' ');
expr = (afterBrace + ' ' + inner).trim();
}
else {
// No braces — but might span multiple lines (forall continuation)
const continuationLines = [];
for (let j = startIndex + 1; j < lines.length; j++) {
const nl = lines[j].trim();
if (!nl || nl.startsWith('//') || nl.startsWith('/*') ||
nl.startsWith('constraint ') || nl.startsWith('verify ') ||
nl.startsWith('constraint:') || nl.startsWith('verify:') ||
nl.startsWith('state') || nl.startsWith('init') ||
nl.startsWith('action ') || nl.startsWith('event ') ||
nl.startsWith('interface ') || nl.startsWith('provenance ') ||
nl.startsWith('bom ') || nl.startsWith('}') ||
nl.startsWith('next:') || nl.startsWith('invariant ')) {
break;
}
continuationLines.push(nl);
}
if (continuationLines.length > 0) {
expr = expr + ' ' + continuationLines.join(' ');
}
}
return { type: 'constraint', name, expr: expr.trim() };
}
function parseVerifyAnyForm(lines, startIndex) {
const line = lines[startIndex].trim();
// Single-line: verify Always(Name) { depth N }
const fullMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*\{\s*depth\s+(\d+)\s*\}/);
if (fullMatch) {
return {
type: 'verify',
name: fullMatch[2],
temporalExpr: fullMatch[1].trim(),
depth: parseInt(fullMatch[3], 10),
};
}
// Multi-line: verify Always(Name) { ... }
if (line.includes('{')) {
const headerMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*\{/);
if (headerMatch) {
const temporalExpr = headerMatch[1].trim();
const name = headerMatch[2];
const openCount = (line.match(/{/g) || []).length;
const endIdx = findBlockEnd(lines, startIndex + 1, openCount);
const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
const inner = innerLines.join(' ');
const afterBrace = line.substring(line.indexOf('{') + 1).trim();
const combined = (afterBrace + ' ' + inner).trim();
const depthMatch = combined.match(/\bdepth\s+(\d+)\b/);
const depth = depthMatch ? parseInt(depthMatch[1], 10) : undefined;
return { type: 'verify', name, temporalExpr, depth };
}
}
// Simple: verify Always(Name)
const simpleMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*$/);
if (simpleMatch) {
return {
type: 'verify',
name: simpleMatch[2],
temporalExpr: simpleMatch[1].trim(),
};
}
return null;
}
// --- Verify/Constraint block parsing ---
/**
* Parse a verify block opened with `verify Always(Name) { ... }` or similar.
* The first line already contains the header; we need to parse the body inside { }.
*/
function parseVerifyBlockMultiline(lines, startIndex, startBraceCount) {
const firstLine = lines[startIndex].trim();
// Match: verify Always(Name) { depth N } or verify Always(Name) {
const headerMatch = firstLine.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*\{/);
if (!headerMatch)
return null;
const temporalExpr = headerMatch[1].trim();
const name = headerMatch[2];
// Check if the brace closes on the same line (single-line form)
const openBraceCount = (firstLine.match(/{/g) || []).length;
const closeBraceCount = (firstLine.match(/}/g) || []).length;
if (closeBraceCount >= openBraceCount) {
// Single-line: parse depth from same line
const afterBrace = firstLine.substring(firstLine.indexOf('{') + 1);
const beforeClose = afterBrace.substring(0, afterBrace.lastIndexOf('}')).trim();
const depthMatch = beforeClose.match(/\bdepth\s+(\d+)\b/);
const depth = depthMatch ? parseInt(depthMatch[1], 10) : undefined;
return { type: 'verify', name, temporalExpr, depth };
}
// Multi-line block
let depth;
const endIdx = findBlockEnd(lines, startIndex + 1, startBraceCount);
const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
const inner = innerLines.join(' ');
const afterBrace = firstLine.substring(firstLine.indexOf('{') + 1).trim();
const depthMatch = (afterBrace + ' ' + inner).match(/\bdepth\s+(\d+)\b/);
if (depthMatch) {
depth = parseInt(depthMatch[1], 10);
}
return { type: 'verify', name, temporalExpr, depth };
}
/**
* Parse a single-line verify statement: verify Always(Name) { depth N }
*/
function parseVerifySingleLine(line) {
// verify Always(Name) { depth N }
const fullMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*\{\s*depth\s+(\d+)\s*\}/);
if (fullMatch) {
return {
type: 'verify',
name: fullMatch[2],
temporalExpr: fullMatch[1].trim(),
depth: parseInt(fullMatch[3], 10),
};
}
// verify Always(Name) (no depth, no braces — just a statement)
const simpleMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*$/);
if (simpleMatch) {
return {
type: 'verify',
name: simpleMatch[2],
temporalExpr: simpleMatch[1].trim(),
};
}
return null;
}
/**
* Parse a constraint block opened with `constraint Name: ...` which may
* be multi-line (e.g. forall spanning multiple lines).
*/
function parseConstraintBlockMultiline(lines, startIndex, startBraceCount) {
const firstLine = lines[startIndex].trim();
// Match: constraint Name: expr (possibly opening a {} block, or just a forall on next line)
// For the SQLiteWAL style: constraint Name: expr (single line, no braces needed)
const namedMatch = firstLine.match(/^constraint\s+(\w+):(.*)$/);
if (namedMatch) {
const name = namedMatch[1];
let expr = namedMatch[2].trim();
// If the expression opens a brace, collect the block
if (expr.startsWith('{')) {
const endIdx = findBlockEnd(lines, startIndex + 1, startBraceCount);
const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
const inner = innerLines.join(' ');
// Combine: content after { on first line + content from inner lines
const afterBrace = expr.substring(1).trim();
expr = (afterBrace + ' ' + inner).trim();
}
else {
// No braces, but might still be multiline (forall spanning multiple lines)
// Check if next line(s) continue the expression
const endIdx = findBlockEnd(lines, startIndex + 1, startBraceCount);
const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
if (innerLines.length > 0) {
expr = expr + ' ' + innerLines.join(' ');
}
}
return { type: 'constraint', name, expr: expr.trim() };
}
return null;
}
/** Split on commas while respecting parenthesized groups and quoted strings */
function splitFields(s) {
const result = [];
let depth = 0;
let inDouble = false;
let inSingle = false;
let start = 0;
for (let i = 0; i < s.length; i++) {
const ch = s[i];
if (ch === '"' && !inSingle) {
inDouble = !inDouble;
continue;
}
if (ch === "'" && !inDouble) {
inSingle = !inSingle;
continue;
}
if (inDouble || inSingle)
continue;
if (ch === '(' || ch === '{') {
depth++;
continue;
}
if (ch === ')' || ch === '}') {
depth--;
continue;
}
if (ch === ',' && depth === 0) {
result.push(s.substring(start, i).trim());
start = i + 1;
}
}
const last = s.substring(start).trim();
if (last)
result.push(last);
return result;
}
//# sourceMappingURL=parser.js.map

2
compiler/dist/parser.js.map vendored
View file

File diff suppressed because one or more lines are too long

2
compiler/node_modules/.vite/vitest/results.json generated vendored
View file

@ -1 +1 @@
{"version":"2.1.9","results":[[":__tests__/parser.test.ts",{"duration":31.455063999999993,"failed":false}],[":src/__tests__/compiler.test.ts",{"duration":12.627537000000018,"failed":false}]]}
{"version":"2.1.9","results":[[":__tests__/parser.test.ts",{"duration":40.638667,"failed":false}],[":__tests__/e2e-wasm.test.ts",{"duration":25.08648199999999,"failed":true}],[":src/__tests__/compiler.test.ts",{"duration":13.108581000000015,"failed":false}]]}

155
compiler/src/expr/ast.ts
View file

@ -1,155 +0,0 @@
// SpeckDL Expression AST nodes
// These represent parsed SpeckDL expressions as a typed tree.
import { Token } from './tokenizer';
// --- Expression nodes ---
export type Expr =
| LiteralExpr
| IdentExpr
| UnaryExpr
| BinaryExpr
| CallExpr
| MemberExpr
| IndexExpr
| IfExpr
| ObjectExpr // { key: val, ... }
| SetExpr // { elem, elem, ... }
| MapExpr // { key -> val, key -> val }
| RangeExpr // [0..N]
| ParenthesizedExpr;
// --- Literals ---
export interface LiteralExpr {
kind: 'literal';
type: 'string' | 'number' | 'bool' | 'null';
value: string;
token: Token;
}
// --- Identifier ---
export interface IdentExpr {
kind: 'ident';
name: string;
token: Token;
}
// --- Unary operators: not, - ---
export interface UnaryExpr {
kind: 'unary';
op: 'not' | 'neg';
operand: Expr;
token: Token;
}
// --- Binary operators ---
export type BinaryOp =
| 'eq' // ==
| 'neq' // !=
| 'lt' // <
| 'gt' // >
| 'lte' // <=
| 'gte' // >=
| 'add' // +
| 'sub' // -
| 'mul' // *
| 'div' // /
| 'mod' // %
| 'and' // and
| 'or' // or
| 'in' // in
| 'union' // union
| 'diff' // \
| 'notin' // notIn
| 'concat'; // + for sets/lists
export interface BinaryExpr {
kind: 'binary';
op: BinaryOp;
left: Expr;
right: Expr;
token: Token; // operator token
}
// --- Function calls: hash(x), length(x), size(x), etc. ---
export interface CallExpr {
kind: 'call';
callee: string; // function name
args: Expr[];
token: Token;
}
// --- Member access: x.field ---
export interface MemberExpr {
kind: 'member';
object: Expr;
property: string;
token: Token; // dot token
}
// --- Index access: x[y] ---
export interface IndexExpr {
kind: 'index';
object: Expr;
index: Expr;
token: Token; // [ token
}
// --- If expression: if cond: thenExpr else elseExpr ---
export interface IfExpr {
kind: 'if';
condition: Expr;
thenBranch: Expr;
elseBranch: Expr | null; // null if no else
token: Token;
}
// --- Object/record literal: { key: val, ... } ---
export interface ObjectExpr {
kind: 'object';
fields: { key: string; value: Expr }[];
token: Token;
}
// --- Set literal: { elem, elem, ... } ---
export interface SetExpr {
kind: 'set';
elements: Expr[];
token: Token;
}
// --- Map literal: { key -> val, key -> val } ---
export interface MapExpr {
kind: 'map';
entries: { key: Expr; value: Expr }[];
token: Token;
}
// --- Range: [0..N] ---
export interface RangeExpr {
kind: 'range';
start: number;
end: Expr; // identifier or number
token: Token;
}
// --- Parenthesized expression ---
export interface ParenthesizedExpr {
kind: 'parens';
expr: Expr;
token: Token;
}

5
compiler/src/expr/index.ts
View file

@ -1,5 +0,0 @@
export { Tokenizer, Token, TokenType, tokenize } from './tokenizer';
export { ExpressionParser, parseExpression } from './parser';
export * from './ast';
export * from './stmt-ast';
export { parseActionStatements } from './stmt-parser';

411
compiler/src/expr/parser.ts
View file

@ -1,411 +0,0 @@
// SpeckDL Expression Parser
// Recursive-descent + Pratt parser for SpeckDL expressions.
// Takes a token stream and produces an Expr AST.
import { Token, TokenType } from './tokenizer';
import {
Expr, BinaryOp, LiteralExpr, IdentExpr, UnaryExpr, BinaryExpr,
CallExpr, MemberExpr, IndexExpr, IfExpr, ObjectExpr, SetExpr,
MapExpr, RangeExpr, ParenthesizedExpr
} from './ast';
// --- Precedence (higher = binds tighter) ---
const PREC = {
LOWEST: 0,
LOGICAL: 1, // or, and, ||, &&
EQUALITY: 2, // ==, !=
COMPARISON: 3, // <, >, <=, >=
SET: 4, // in, notIn
ADDITIVE: 5, // +, -, union, \
MULTIPLY: 6, // *, /, %
PREFIX: 7, // not, -
POSTFIX: 8, // f(), x.y, x[y]
} as const;
/** Get operator precedence. Handles both symbol tokens and ident-based keywords. */
function precOf(type: TokenType, value?: string): number {
// Ident-based keywords (or, and, in, not, union)
if (type === TokenType.IDENT) {
switch (value) {
case 'or': case 'and': return PREC.LOGICAL;
case 'in': case 'notIn': return PREC.SET;
case 'not': return PREC.PREFIX;
case 'union': return PREC.ADDITIVE;
}
}
switch (type) {
case TokenType.DBLPIPE: case TokenType.DBLAMP: return PREC.LOGICAL;
case TokenType.EQ: case TokenType.NEQ: return PREC.EQUALITY;
case TokenType.LT: case TokenType.GT:
case TokenType.LTE: case TokenType.GTE: return PREC.COMPARISON;
case TokenType.NOTIN: return PREC.SET;
case TokenType.ADD: case TokenType.SUB:
case TokenType.UNION: case TokenType.BACKSLASH: return PREC.ADDITIVE;
case TokenType.MUL: case TokenType.DIV: case TokenType.MOD: return PREC.MULTIPLY;
case TokenType.LPAREN: case TokenType.DOT:
case TokenType.LBRACKET: return PREC.POSTFIX;
default: return PREC.LOWEST;
}
}
/** Map token type + value to binary operator name. */
function binOp(type: TokenType, value?: string): BinaryOp | null {
if (type === TokenType.IDENT) {
switch (value) {
case 'or': return 'or';
case 'and': return 'and';
case 'in': return 'in';
case 'notIn': return 'notin';
case 'union': return 'union';
}
}
switch (type) {
case TokenType.EQ: return 'eq';
case TokenType.NEQ: return 'neq';
case TokenType.LT: return 'lt';
case TokenType.GT: return 'gt';
case TokenType.LTE: return 'lte';
case TokenType.GTE: return 'gte';
case TokenType.ADD: return 'add';
case TokenType.SUB: return 'sub';
case TokenType.MUL: return 'mul';
case TokenType.DIV: return 'div';
case TokenType.MOD: return 'mod';
case TokenType.DBLPIPE: return 'or';
case TokenType.DBLAMP: return 'and';
case TokenType.BACKSLASH: return 'diff';
case TokenType.UNION: return 'union';
case TokenType.NOTIN: return 'notin';
default: return null;
}
}
export class ExpressionParser {
private tokens: Token[];
private pos: number = 0;
constructor(tokens: Token[]) {
this.tokens = tokens;
}
parse(): Expr {
return this.expr(PREC.LOWEST);
}
/** Parse expression with given minimum precedence. */
private expr(minPrec: number): Expr {
const tok = this.advance();
let left = this.prefix(tok);
while (this.pos < this.tokens.length) {
const peek = this.current();
const p = precOf(peek.type, peek.value);
if (p <= minPrec) break;
// COLON only used inside if-expr (handled in prefix) — stop here
if (peek.type === TokenType.COLON) break;
const opTok = this.advance();
left = this.infix(left, opTok);
}
return left;
}
// --- Prefix parsers ---
private prefix(tok: Token): Expr {
switch (tok.type) {
case TokenType.IDENT:
case TokenType.TYPE:
case TokenType.FORALL:
case TokenType.CONSTANT:
return this.identOrCall(tok);
case TokenType.NUMBER:
return { kind: 'literal', type: 'number', value: tok.value, token: tok } as LiteralExpr;
case TokenType.STRING:
return { kind: 'literal', type: 'string', value: tok.value, token: tok } as LiteralExpr;
case TokenType.BOOL:
return { kind: 'literal', type: 'bool', value: tok.value, token: tok } as LiteralExpr;
case TokenType.NULL:
return { kind: 'literal', type: 'null', value: 'null', token: tok } as LiteralExpr;
case TokenType.NOT:
return { kind: 'unary', op: 'not', operand: this.expr(PREC.PREFIX), token: tok } as UnaryExpr;
case TokenType.SUB:
return { kind: 'unary', op: 'neg', operand: this.expr(PREC.PREFIX), token: tok } as UnaryExpr;
case TokenType.IF:
return this.parseIf(tok);
case TokenType.LBRACE:
return this.parseBrace(tok);
case TokenType.LBRACKET:
return this.parseBracket(tok);
case TokenType.LPAREN:
return this.parseParens(tok);
case TokenType.EMPTYSET:
return { kind: 'call', callee: 'emptySet', args: [], token: tok } as CallExpr;
default:
throw new Error(`Unexpected token '${tok.value}' (${tok.type}) at ${tok.line}:${tok.col}`);
}
}
private identOrCall(tok: Token): Expr {
// "not" as unary prefix keyword (when not followed by . or ()
if (tok.value === 'not' && this.current().type !== TokenType.LPAREN && this.current().type !== TokenType.DOT) {
return { kind: 'unary', op: 'not', operand: this.expr(PREC.PREFIX), token: tok } as UnaryExpr;
}
// Function call: ident(
if (this.current().type === TokenType.LPAREN) {
const lp = this.advance();
return this.parseCallWithCallee(tok, lp);
}
return { kind: 'ident', name: tok.value, token: tok } as IdentExpr;
}
// --- Infix parsers ---
private infix(left: Expr, opTok: Token): Expr {
const p = precOf(opTok.type, opTok.value);
switch (opTok.type) {
case TokenType.LPAREN:
return this.parseCallWithCallee((left as IdentExpr).token, opTok);
case TokenType.DOT:
return this.parseMember(left, opTok);
case TokenType.LBRACKET:
return this.parseIndex(left, opTok);
default: {
// Binary operator (symbol or ident-based keyword)
const op = binOp(opTok.type, opTok.value);
if (!op) throw new Error(`Not an infix operator: ${opTok.type} '${opTok.value}'`);
const right = this.expr(p);
return { kind: 'binary', op, left, right, token: opTok } as BinaryExpr;
}
}
}
// --- Specific parsers ---
private parseIf(ifTok: Token): IfExpr {
const condition = this.expr(PREC.LOWEST);
this.expect(TokenType.COLON, 'Expected : after if condition');
const thenBranch = this.expr(PREC.LOWEST);
let elseBranch: Expr | null = null;
if (this.current().type === TokenType.ELSE) {
this.advance();
elseBranch = this.expr(PREC.LOWEST);
}
return { kind: 'if', condition, thenBranch, elseBranch, token: ifTok };
}
private parseBrace(openTok: Token): Expr {
if (this.current().type === TokenType.RBRACE) {
this.advance();
return { kind: 'set', elements: [], token: openTok } as SetExpr;
}
const first = this.expr(PREC.LOWEST);
// Map literal: key -> val
if (this.current().type === TokenType.ARROW) {
this.advance();
const val = this.expr(PREC.LOWEST);
const entries = [{ key: first, value: val }];
while (this.current().type === TokenType.COMMA) {
this.advance();
if (this.current().type === TokenType.RBRACE) break;
const k = this.expr(PREC.LOWEST);
this.expect(TokenType.ARROW, 'Expected -> in map literal');
const v = this.expr(PREC.LOWEST);
entries.push({ key: k, value: v });
}
this.expect(TokenType.RBRACE, 'Expected }');
return { kind: 'map', entries, token: openTok } as MapExpr;
}
// Object literal: ident: val
if (first.kind === 'ident' && this.current().type === TokenType.COLON) {
this.advance();
const val = this.expr(PREC.LOWEST);
const fields = [{ key: (first as IdentExpr).name, value: val }];
while (this.current().type === TokenType.COMMA) {
this.advance();
if (this.current().type === TokenType.RBRACE) break;
const keyTok = this.expect(TokenType.IDENT, 'Expected field name');
this.expect(TokenType.COLON, 'Expected :');
const v = this.expr(PREC.LOWEST);
fields.push({ key: keyTok.value, value: v });
}
this.expect(TokenType.RBRACE, 'Expected }');
return { kind: 'object', fields, token: openTok } as ObjectExpr;
}
// Set literal: expr, expr, ...
const elements = [first];
while (this.current().type === TokenType.COMMA) {
this.advance();
if (this.current().type === TokenType.RBRACE) break;
elements.push(this.expr(PREC.LOWEST));
}
this.expect(TokenType.RBRACE, 'Expected }');
return { kind: 'set', elements, token: openTok } as SetExpr;
}
private parseBracket(openTok: Token): Expr {
const first = this.expr(PREC.LOWEST);
// Range: [start..end]
if (this.current().type === TokenType.DOTDOT) {
this.advance();
const startNum = first.kind === 'literal' && first.type === 'number'
? parseInt((first as LiteralExpr).value) : 0;
const end = this.expr(PREC.LOWEST);
this.expect(TokenType.RBRACKET, 'Expected ]');
return { kind: 'range', start: startNum, end, token: openTok } as RangeExpr;
}
// List: [a, b, c]
const elements = [first];
while (this.current().type === TokenType.COMMA) {
this.advance();
if (this.current().type === TokenType.RBRACKET) break;
elements.push(this.expr(PREC.LOWEST));
}
this.expect(TokenType.RBRACKET, 'Expected ]');
return { kind: 'call', callee: 'Array', args: elements, token: openTok } as CallExpr;
}
private parseParens(openTok: Token): Expr {
const expr = this.expr(PREC.LOWEST);
this.expect(TokenType.RPAREN, 'Expected )');
return { kind: 'parens', expr, token: openTok } as ParenthesizedExpr;
}
private parseCallWithCallee(nameTok: Token, lpTok: Token): Expr {
const callee = nameTok.value;
const args: Expr[] = [];
if (this.current().type !== TokenType.RPAREN) {
args.push(this.expr(PREC.LOWEST));
while (this.current().type === TokenType.COMMA) {
this.advance();
if (this.current().type === TokenType.RPAREN) break;
args.push(this.expr(PREC.LOWEST));
}
}
this.expect(TokenType.RPAREN, 'Expected )');
const call: CallExpr = { kind: 'call', callee, args, token: nameTok };
// Check for postfix continuation (chaining: f(x).y, f(x)[z], f(x) \ {y})
while (this.pos < this.tokens.length) {
const peek = this.current();
if (peek.type === TokenType.DOT) {
const dotTok = this.advance();
return this.parseMember(call, dotTok);
}
if (peek.type === TokenType.LBRACKET) {
const bracketTok = this.advance();
return this.parseIndex(call, bracketTok);
}
const p = precOf(peek.type, peek.value);
if (p <= PREC.LOWEST) break;
if (peek.type === TokenType.COLON) break;
// Infix continuation (e.g., \ operator)
const opTok = this.advance();
return this.infix(call, opTok);
}
return call;
}
private parseMember(obj: Expr, dotTok: Token): Expr {
const propTok = this.expect(TokenType.IDENT, 'Expected property name after .');
const member: MemberExpr = {
kind: 'member', object: obj, property: propTok.value, token: dotTok
};
// Chain: x.y.z() or x.y[z]
while (this.pos < this.tokens.length) {
const peek = this.current();
if (peek.type === TokenType.DOT) {
const d = this.advance();
return this.parseMember(member, d);
}
if (peek.type === TokenType.LBRACKET) {
const b = this.advance();
return this.parseIndex(member, b);
}
if (peek.type === TokenType.LPAREN) {
const lp = this.advance();
// Method call: x.y(args) — wrap as call with member as callee
const args: Expr[] = [];
if (this.current().type !== TokenType.RPAREN) {
args.push(this.expr(PREC.LOWEST));
while (this.current().type === TokenType.COMMA) {
this.advance();
if (this.current().type === TokenType.RPAREN) break;
args.push(this.expr(PREC.LOWEST));
}
}
this.expect(TokenType.RPAREN, 'Expected )');
// Return as a method call on the member
const methodCall: CallExpr = {
kind: 'call', callee: `${propTok.value}()`, args: [obj, ...args], token: dotTok
};
// Actually, for TS generation we want: obj.prop(args)
// Let's represent this as a call with the member expression as the callee
// For now, keep it as a member + separate call representation
return { kind: 'call', callee: propTok.value, args: [obj, ...args], token: dotTok } as CallExpr;
}
break;
}
return member;
}
private parseIndex(obj: Expr, bracketTok: Token): Expr {
const idx = this.expr(PREC.LOWEST);
this.expect(TokenType.RBRACKET, 'Expected ]');
const index: IndexExpr = {
kind: 'index', object: obj, index: idx, token: bracketTok
};
// Chain: x[y].z or x[y][z]
while (this.pos < this.tokens.length) {
const peek = this.current();
if (peek.type === TokenType.DOT) {
const d = this.advance();
return this.parseMember(index, d);
}
if (peek.type === TokenType.LBRACKET) {
const b = this.advance();
return this.parseIndex(index, b);
}
break;
}
return index;
}
// --- Helpers ---
private current(): Token {
return this.pos < this.tokens.length
? this.tokens[this.pos]
: { type: TokenType.EOF, value: '', pos: 0, line: 0, col: 0 };
}
private advance(): Token {
const tok = this.current();
if (this.pos < this.tokens.length) this.pos++;
return tok;
}
private expect(type: TokenType, msg?: string): Token {
const tok = this.current();
if (tok.type !== type) {
throw new Error(msg || `Expected ${type} but got ${tok.type} ('${tok.value}') at ${tok.line}:${tok.col}`);
}
return this.advance();
}
}
/** Parse a single SpeckDL expression string to AST. */
export function parseExpression(src: string): Expr {
const { Tokenizer } = require('./tokenizer');
const tokens = new Tokenizer(src).tokenize();
return new ExpressionParser(tokens).parse();
}

66
compiler/src/expr/stmt-ast.ts
View file

@ -1,66 +0,0 @@
// SpeckDL Statement AST
// Typed statement nodes with parsed expression subtrees.
import { Expr } from './ast';
// --- Action statement types ---
export type ActionStatement =
| AssignStmt
| LetStmt
| RequireStmt
| PreconditionStmt
| PostconditionStmt
| EmitStmt
| ReturnStmt
| IfBlockStmt;
export interface AssignStmt {
type: 'assign';
target: AssignTarget;
expr: Expr;
}
/** Target of an assignment: plain ident or map[x] bracket access. */
export type AssignTarget =
| { kind: 'plain'; name: string }
| { kind: 'map_index'; mapName: string; index: Expr };
export interface LetStmt {
type: 'let';
name: string;
expr: Expr;
}
export interface RequireStmt {
type: 'require';
expr: Expr;
}
export interface PreconditionStmt {
type: 'precondition';
expr: Expr;
}
export interface PostconditionStmt {
type: 'postcondition';
expr: Expr;
}
export interface EmitStmt {
type: 'emit';
event: string;
fields: { name: string; value: Expr }[];
}
export interface ReturnStmt {
type: 'return';
expr: Expr;
}
/** Incomplete if/else block — preserved as raw text for manual handling. */
export interface IfBlockStmt {
type: 'ifblock';
raw: string;
// Future: properly parsed conditional block
}

221
compiler/src/expr/stmt-parser.ts
View file

@ -1,221 +0,0 @@
// SpeckDL Statement Parser
// Parses an action body (list of lines) into typed ActionStatement nodes
// with full expression subtrees from the ExpressionParser.
import { Tokenizer, Token, TokenType } from './tokenizer';
import { ExpressionParser } from './parser';
import { Expr } from "./ast";
import { ActionStatement } from './stmt-ast';
/**
* Parse a single action's body lines into typed statement nodes.
* Handles: precondition, postcondition, require, let, assign, emit, return, ifblock.
*/
export function parseActionStatements(lines: string[]): ActionStatement[] {
const stmts: ActionStatement[] = [];
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
if (!line) continue;
if (line.startsWith('//') || line.startsWith('/*')) continue;
const trimmed = line.trim();
if (!trimmed) continue;
// Clean up double-backslash from old parser (SpeckDL set diff is single \)
// The old parser stores \\ as two chars; the expression parser expects single \\n // Actually, keep double backslash as-is since the tokenizer handles both
// --- forall block: keep as raw ifblock ---
if (trimmed.startsWith('forall ')) {
// Collect entire forall block
let blockText = trimmed;
let j = i + 1;
while (j < lines.length) {
const next = lines[j];
if (!next || !next.trim()) break;
const nt = next.trim();
if (nt.startsWith('require ') || nt.startsWith('let ') || nt.startsWith('return ') ||
nt.startsWith('emit ') || /^\w+\s*:=/.test(nt)) break;
blockText += ' ' + nt;
j++;
}
stmts.push({ type: 'ifblock', raw: blockText });
i = j - 1;
continue;
}
// --- precondition: expr ---
if (trimmed.startsWith('precondition:')) {
const exprs = collectExpression(lines, i, 13, 'or ');
i = exprs.endIdx;
stmts.push({ type: 'precondition', expr: parseExprSingle(exprs.text) });
continue;
}
// --- postcondition: expr ---
if (trimmed.startsWith('postcondition:')) {
const exprs = collectExpression(lines, i, 14, 'or ');
i = exprs.endIdx;
stmts.push({ type: 'postcondition', expr: parseExprSingle(exprs.text) });
continue;
}
// --- require expr ---
if (trimmed.startsWith('require ')) {
const exprs = collectRequireExpression(lines, i);
i = exprs.endIdx;
stmts.push({ type: 'require', expr: parseExprSingle(exprs.text) });
continue;
}
// --- return expr ---
if (trimmed.startsWith('return ')) {
stmts.push({ type: 'return', expr: parseExprSingle(trimmed.substring(7).trim()) });
continue;
}
// --- emit Event { fields } ---
const emitMatch = trimmed.match(/^emit\s+(\w+)\s*\{\s*(.*)\s*\}$/);
if (emitMatch) {
const eventName = emitMatch[1];
const fieldsStr = emitMatch[2];
const fields: { name: string; value: Expr }[] = [];
if (fieldsStr.trim()) {
for (const fp of fieldsStr.split(',').map(s => s.trim()).filter(Boolean)) {
const colonIdx = fp.indexOf(':');
if (colonIdx > 0) {
fields.push({
name: fp.substring(0, colonIdx).trim(),
value: parseExprSingle(fp.substring(colonIdx + 1).trim()),
});
}
}
}
stmts.push({ type: 'emit', event: eventName, fields });
continue;
}
// --- if condition { ... } else ... ---
if (trimmed.startsWith('if ')) {
// Collect the full if/else block as raw text
let braceDepth = 0;
let j = i;
const blockLines: string[] = [];
while (j < lines.length) {
const bl = lines[j];
if (!bl) { j++; continue; }
const bt = bl.trim();
if (bt.startsWith('//')) { j++; continue; }
blockLines.push(bt);
for (const ch of bt) {
if (ch === '{') braceDepth++;
else if (ch === '}') braceDepth--;
}
j++;
if (braceDepth === 0) break;
}
stmts.push({ type: 'ifblock', raw: blockLines.join(' ') });
i = j - 1;
continue;
}
// --- let name := expr ---
const letMatch = trimmed.match(/^let\s+([A-Za-z_]\w*)\s*:=\s*(.+)$/);
if (letMatch) {
const rhs = letMatch[2].trim();
if (/^if\b/.test(rhs) && !/\belse\b/.test(rhs)) {
stmts.push({ type: 'ifblock', raw: trimmed });
continue;
}
stmts.push({ type: 'let', name: letMatch[1], expr: parseExprSingle(rhs) });
continue;
}
// --- assignment: target := expr ---
const assignMatch = trimmed.match(/^(\w+(?:\[.*?\])?)\s*:=\s*(.+)$/);
if (assignMatch) {
let exprText = assignMatch[2].trim();
// Handle multiline expressions
while (i + 1 < lines.length) {
const next = lines[i + 1];
if (!next || next.trim().startsWith('//') || next.trim().startsWith('require ') ||
next.trim().startsWith('return ') || next.trim().startsWith('emit ') ||
next.trim().startsWith('let ') || /\w+\s*:=/.test(next.trim())) {
break;
}
// Join continuation lines
if (next.trim()) {
exprText += ' ' + next.trim();
i++;
} else {
break;
}
}
const target = assignMatch[1];
const expr = parseExprSingle(exprText);
// Determine target kind: plain or map_index
const bracketMatch = target.match(/^(\w+)\[(.+)\]$/);
if (bracketMatch && bracketMatch[2].trim()) {
stmts.push({
type: 'assign',
target: { kind: 'map_index', mapName: bracketMatch[1], index: parseExprSingle(bracketMatch[2]) },
expr,
});
} else {
stmts.push({ type: 'assign', target: { kind: 'plain', name: target }, expr });
}
continue;
}
}
return stmts;
}
// --- Helpers ---
/** Parse a single expression string to an Expr AST. */
function parseExprSingle(src: string): Expr {
const tokens = new Tokenizer(src).tokenize();
return new ExpressionParser(tokens).parse();
}
interface CollectResult {
text: string;
endIdx: number;
}
/** Collect an expression that may continue on 'or ' lines. */
function collectExpression(lines: string[], startIdx: number, prefixLen: number, continuationPfx: string): CollectResult {
let text = lines[startIdx].trim().substring(prefixLen).trim();
let i = startIdx;
while (i + 1 < lines.length) {
const next = lines[i + 1];
if (!next || !next.trim().startsWith(continuationPfx)) break;
i++;
text += ' or ' + next.trim().substring(continuationPfx.length).trim();
}
return { text, endIdx: i };
}
/** Collect a require expression that may span multiple lines (forall, or, etc). */
function collectRequireExpression(lines: string[], startIdx: number): CollectResult {
let text = lines[startIdx].trim().substring(8).trim();
let i = startIdx;
while (i + 1 < lines.length) {
const next = lines[i + 1];
if (!next) break;
const nt = next.trim();
if (nt.startsWith('//') || nt.startsWith('precondition:') || nt.startsWith('postcondition:') ||
nt.startsWith('require ') || nt.startsWith('return ') || nt.startsWith('emit ') ||
nt.startsWith('let ') || /\w+\s*:=/.test(nt)) {
break;
}
i++;
text += ' ' + nt;
}
return { text, endIdx: i };
}
// Re-export Expr type
export type { Expr } from './ast';

387
compiler/src/expr/tokenizer.ts
View file

@ -1,387 +0,0 @@
// SpeckDL Tokenizer
// Produces a token stream from SpeckDL source text.
// Used by the expression parser and statement parser.
export interface Token {
type: TokenType;
value: string;
pos: number; // character offset in source
line: number;
col: number;
}
export enum TokenType {
// Identifiers and literals
IDENT = 'IDENT',
STRING = 'STRING',
NUMBER = 'NUMBER',
BOOL = 'BOOL', // true, false
NULL = 'NULL', // null
// Keywords
LET = 'LET',
IF = 'IF',
ELSE = 'ELSE',
FORALL = 'FORALL',
IN = 'IN',
NOTIN = 'NOTIN', // notIn keyword
NOT = 'NOT',
AND = 'AND',
OR = 'OR',
REQUIRE = 'REQUIRE',
RETURN = 'RETURN',
EMIT = 'EMIT',
PRECONDITION = 'PRECONDITION',
POSTCONDITION = 'POSTCONDITION',
CONSTANT = 'CONSTANT',
TYPE = 'TYPE',
// Operators
ASSIGN = 'ASSIGN', // :=
EQ = 'EQ', // ==
NEQ = 'NEQ', // !=
LTE = 'LTE', // <=
GTE = 'GTE', // >=
LT = 'LT', // <
GT = 'GT', // >
ADD = 'ADD', // +
SUB = 'SUB', // -
MUL = 'MUL', // *
DIV = 'DIV', // /
MOD = 'MOD', // %
// Delimiters
LPAREN = 'LPAREN', // (
RPAREN = 'RPAREN', // )
LBRACE = 'LBRACE', // {
RBRACE = 'RBRACE', // }
LBRACKET = 'LBRACKET',// [
RBRACKET = 'RBRACKET',// ]
COLON = 'COLON', // :
COMMA = 'COMMA', // ,
SEMI = 'SEMI', // ; — statement separator in multi-stmt lines
DOT = 'DOT', // .
DOTDOT = 'DOTDOT', // ..
ARROW = 'ARROW', // ->
PIPE = 'PIPE', // |
DBLPIPE = 'DBLPIPE', // ||
BACKSLASH = 'BACKSLASH', // \
DBLAMP = 'DBLAMP', // &&
AT = 'AT', // @
HASH = 'HASH', // #
// Set operations
UNION = 'UNION', // union keyword
INTER = 'INTER', // inter keyword
DIFF = 'DIFF', // diff keyword
EMPTYSET = 'EMPTYSET', // emptySet keyword
EOF = 'EOF',
}
const KEYWORDS: Record<string, TokenType> = {
'let': TokenType.LET,
'if': TokenType.IF,
'else': TokenType.ELSE,
'forall': TokenType.FORALL,
'require': TokenType.REQUIRE,
'return': TokenType.RETURN,
'emit': TokenType.EMIT,
'precondition': TokenType.PRECONDITION,
'postcondition': TokenType.POSTCONDITION,
'constant': TokenType.CONSTANT,
'type': TokenType.TYPE,
'union': TokenType.UNION,
'inter': TokenType.INTER,
'diff': TokenType.DIFF,
'emptySet': TokenType.EMPTYSET,
'notIn': TokenType.NOTIN,
'true': TokenType.BOOL,
'false': TokenType.BOOL,
'null': TokenType.NULL,
};
export class Tokenizer {
private src: string;
private pos: number = 0;
private line: number = 1;
private col: number = 1;
constructor(src: string) {
this.src = src;
}
/** Tokenize the entire source and return an array of tokens (including EOF). */
tokenize(): Token[] {
const tokens: Token[] = [];
let tok: Token;
while ((tok = this.next()).type !== TokenType.EOF) {
tokens.push(tok);
}
tokens.push(tok); // include EOF
return tokens;
}
/** Return the next token. Returns EOF token when source is exhausted. */
next(): Token {
this.skipWhitespaceAndComments();
if (this.pos >= this.src.length) {
return this.makeToken(TokenType.EOF, '');
}
const ch = this.src[this.pos];
// Two-char operators first
if (ch === ':' && this.peek() === '=') {
return this.advance(2, TokenType.ASSIGN, ':=');
}
if (ch === '=' && this.peek() === '=') {
return this.advance(2, TokenType.EQ, '==');
}
if (ch === '!' && this.peek() === '=') {
return this.advance(2, TokenType.NEQ, '!=');
}
if (ch === '<' && this.peek() === '=') {
return this.advance(2, TokenType.LTE, '<=');
}
if (ch === '>' && this.peek() === '=') {
return this.advance(2, TokenType.GTE, '>=');
}
if (ch === '-' && this.peek() === '>') {
return this.advance(2, TokenType.ARROW, '->');
}
if (ch === '|' && this.peek() === '|') {
return this.advance(2, TokenType.DBLPIPE, '||');
}
if (ch === '&' && this.peek() === '&') {
return this.advance(2, TokenType.DBLAMP, '&&');
}
if (ch === '.' && this.peek() === '.') {
return this.advance(2, TokenType.DOTDOT, '..');
}
// Single-char operators & delimiters
switch (ch) {
case ':': return this.advance(1, TokenType.COLON, ':');
case '(': return this.advance(1, TokenType.LPAREN, '(');
case ')': return this.advance(1, TokenType.RPAREN, ')');
case '{': return this.advance(1, TokenType.LBRACE, '{');
case '}': return this.advance(1, TokenType.RBRACE, '}');
case '[': return this.advance(1, TokenType.LBRACKET, '[');
case ']': return this.advance(1, TokenType.RBRACKET, ']');
case ',': return this.advance(1, TokenType.COMMA, ',');
case ';': return this.advance(1, TokenType.SEMI, ';');
case '.': return this.advance(1, TokenType.DOT, '.');
case '|': return this.advance(1, TokenType.PIPE, '|');
case '\\':
// Double backslash (\\) is set difference in SpeckDL — skip the second \
if (this.pos + 1 < this.src.length && this.src[this.pos + 1] === '\\') {
this.pos++; this.col++; // skip second backslash
}
return this.advance(1, TokenType.BACKSLASH, '\\');
case '@': return this.advance(1, TokenType.AT, '@');
case '#': return this.advance(1, TokenType.HASH, '#');
case '<': return this.advance(1, TokenType.LT, '<');
case '>': return this.advance(1, TokenType.GT, '>');
case '+': return this.advance(1, TokenType.ADD, '+');
case '*': return this.advance(1, TokenType.MUL, '*');
case '%': return this.advance(1, TokenType.MOD, '%');
}
// - can be minus or start of a negative number
if (ch === '-') {
if (this.isDigit(this.peek())) {
return this.readNumber();
}
return this.advance(1, TokenType.SUB, '-');
}
// / can be division or start of a comment
if (ch === '/') {
return this.advance(1, TokenType.DIV, '/');
}
// Strings
if (ch === '"') {
return this.readString('"');
}
if (ch === "'") {
return this.readString("'");
}
// Numbers
if (this.isDigit(ch)) {
return this.readNumber();
}
// Identifiers and keywords
if (this.isIdentStart(ch)) {
return this.readIdent();
}
// Unknown character — skip and warn
console.warn(`Tokenizer: unexpected character '${ch}' at ${this.line}:${this.col}`);
return this.advance(1, TokenType.EOF, ch);
}
/** Peek at the next token without consuming it (used by parser). */
peekToken(): Token {
const savedPos = this.pos;
const savedLine = this.line;
const savedCol = this.col;
const tok = this.next();
this.pos = savedPos;
this.line = savedLine;
this.col = savedCol;
return tok;
}
// --- Private helpers ---
private peek(): string {
if (this.pos + 1 >= this.src.length) return '\0';
return this.src[this.pos + 1];
}
private advance(len: number, type: TokenType, value: string): Token {
const token = this.makeToken(type, value);
this.pos += len;
this.col += len;
return token;
}
private makeToken(type: TokenType, value: string): Token {
return { type, value, pos: this.pos, line: this.line, col: this.col };
}
private skipWhitespaceAndComments(): void {
while (this.pos < this.src.length) {
const ch = this.src[this.pos];
// Whitespace
if (ch === ' ' || ch === '\t' || ch === '\r') {
this.pos++; this.col++;
continue;
}
if (ch === '\n') {
this.pos++; this.line++; this.col = 1;
continue;
}
// Line comments: //
if (ch === '/' && this.peek() === '/') {
this.pos += 2; this.col += 2;
while (this.pos < this.src.length && this.src[this.pos] !== '\n') {
this.pos++; this.col++;
}
continue;
}
// Block comments: /* ... */
if (ch === '/' && this.peek() === '*') {
this.pos += 2; this.col += 2;
while (this.pos < this.src.length) {
if (this.src[this.pos] === '*' && this.peek() === '/') {
this.pos += 2; this.col += 2;
break;
}
if (this.src[this.pos] === '\n') {
this.line++; this.col = 1;
} else {
this.col++;
}
this.pos++;
}
continue;
}
break;
}
}
private isDigit(ch: string): boolean {
return ch >= '0' && ch <= '9';
}
private isIdentStart(ch: string): boolean {
return (ch >= 'a' && ch <= 'z') || (ch >= 'A' && ch <= 'Z') || ch === '_';
}
private isIdentPart(ch: string): boolean {
return this.isIdentStart(ch) || this.isDigit(ch);
}
private readNumber(): Token {
const start = this.pos;
let value = '';
// Handle leading minus
if (this.src[this.pos] === '-') {
value += '-';
this.pos++; this.col++;
}
while (this.pos < this.src.length && this.isDigit(this.src[this.pos])) {
value += this.src[this.pos];
this.pos++; this.col++;
}
// Decimal part
if (this.pos < this.src.length && this.src[this.pos] === '.' && this.isDigit(this.peek())) {
value += '.';
this.pos++; this.col++;
while (this.pos < this.src.length && this.isDigit(this.src[this.pos])) {
value += this.src[this.pos];
this.pos++; this.col++;
}
}
return this.makeToken(TokenType.NUMBER, value);
}
private readString(quote: string): Token {
const startPos = this.pos;
// Skip opening quote
this.pos++; this.col++;
let value = '';
while (this.pos < this.src.length) {
const ch = this.src[this.pos];
if (ch === '\\') {
this.pos++; this.col++;
if (this.pos < this.src.length) {
const escaped = this.src[this.pos];
value += '\\' + escaped;
this.pos++; this.col++;
}
continue;
}
if (ch === quote) {
this.pos++; this.col++; // skip closing quote
return this.makeToken(TokenType.STRING, value);
}
if (ch === '\n') {
console.warn(`Tokenizer: unclosed string starting at line ${this.line}`);
return this.makeToken(TokenType.STRING, value);
}
value += ch;
this.pos++; this.col++;
}
return this.makeToken(TokenType.STRING, value);
}
private readIdent(): Token {
const startPos = this.pos;
let value = '';
while (this.pos < this.src.length && this.isIdentPart(this.src[this.pos])) {
value += this.src[this.pos];
this.pos++; this.col++;
}
// Check for keywords
const keyword = KEYWORDS[value];
if (keyword) {
return { type: keyword, value, pos: startPos, line: this.line, col: this.col - value.length };
}
return { type: TokenType.IDENT, value, pos: startPos, line: this.line, col: this.col - value.length };
}
}
/** Convenience: tokenize a string in one call. */
export function tokenize(src: string): Token[] {
return new Tokenizer(src).tokenize();
}

662
compiler/src/generators/rust.ts
View file

@ -1,662 +0,0 @@
import { AST, SpeckNode, MemberNode, StateNode, InitNode, ActionNode, ConstraintNode, VerifyNode, EventNode } from '../parser.js';
import fs from 'fs';
import path from 'path';
/**
* Generate Rust state machine from a SpeckDL spec.
*
* Outputs a self-contained .rs file with:
* - State struct with snake_case fields
* - State machine impl with action methods returning Result
* - Invariant check functions
* - Event enums
* - Cargo.toml for full crate builds
*
* Pattern:
* state → Rust struct fields
* actions → Rust methods (Result return, guard errors)
* invariants → Rust functions returning bool
* events → Rust structs
*/
export function generateRust(ast: AST, outputDir: string): void {
for (const speck of ast.specks) {
const code = emitSpeck(speck);
// Create a subdirectory per speck for Cargo.toml + src/lib.rs
const speckDir = path.join(outputDir, speck.name);
const srcDir = path.join(speckDir, 'src');
if (!fs.existsSync(srcDir)) {
fs.mkdirSync(srcDir, { recursive: true });
}
const libPath = path.join(srcDir, 'lib.rs');
fs.writeFileSync(libPath, code);
console.log(`Generated Rust crate: ${libPath}`);
// Write Cargo.toml
const cargoToml = `[package]
name = "${snakeCase(speck.name)}"
version = "0.1.0"
edition = "2021"
description = "Auto-generated Speck state machine: ${speck.name}"
[dependencies]
`;
fs.writeFileSync(path.join(speckDir, 'Cargo.toml'), cargoToml);
console.log(`Generated Cargo.toml: ${speckDir}/Cargo.toml`);
}
}
function emitSpeck(speck: SpeckNode): string {
const deduped = new Map<string, MemberNode>();
for (const m of speck.members) {
const key = memberKey(m);
const existing = deduped.get(key);
if (!existing || isEmptyStub(existing)) {
deduped.set(key, m);
}
}
const members = Array.from(deduped.values());
const stateNode = members.find(m => m.type === 'state') as StateNode | undefined;
const initNode = members.find(m => m.type === 'init') as InitNode | undefined;
const actions = members.filter(m => m.type === 'action') as ActionNode[];
const events = members.filter(m => m.type === 'event') as EventNode[];
const constraints = members.filter(m => m.type === 'constraint') as ConstraintNode[];
const verifies = members.filter(m => m.type === 'verify') as VerifyNode[];
// Collect enum-like interfaces
const allInterfaces = members.filter(m => m.type === 'interface') as any[];
const enumInterfaces = allInterfaces.filter(iface => iface?.methods?.length > 0);
let stateValues: string[] = [];
if (enumInterfaces.length > 0) {
for (const iface of enumInterfaces) {
stateValues.push(...iface.methods.map((m: any) => m.name));
}
} else {
const seen = new Set<string>();
const builtinTypes = new Set(['String', 'Nat', 'Int', 'Bool', 'Real', 'Bytes', 'Date', 'Number']);
for (const a of actions) {
for (const s of a.statements) {
if (s.type === 'require') {
const matches = s.expr.match(/\b[A-Z][a-zA-Z]+\b/g);
if (matches) {
matches.forEach((m: string) => {
if (!builtinTypes.has(m)) seen.add(m);
});
}
}
}
}
stateValues = Array.from(seen).sort();
}
if (stateValues.length === 0) stateValues = ['Open', 'Closed'];
// Build enum map
const enumMap = new Map<string, string[]>();
for (const iface of enumInterfaces) {
enumMap.set(iface.name, iface.methods.map((m: any) => m.name));
}
// Determine state enum name
const stateVarType = stateNode?.variables?.[0]?.typeExpr;
let stateEnumName = `${speck.name}State`;
if (stateVarType?.type === 'ident' && stateVarType.name) {
const cleanTypeName = cleanName(stateVarType.name);
stateEnumName = `${speck.name}${cleanTypeName}`;
} else if (enumMap.size > 0) {
const firstEnum = enumMap.keys().next().value;
if (firstEnum) stateEnumName = `${speck.name}${firstEnum}`;
}
if (enumInterfaces.length === 0) {
enumMap.set(stateEnumName.replace(speck.name, ''), stateValues);
}
// Scan event field types for secondary enums
const eventFieldTypes = new Set<string>();
for (const ev of events) {
for (const f of ev.fields) {
if (f.type?.type === 'ident' && f.type.name) {
const typeName = cleanName(f.type.name);
if (typeName !== stateEnumName.replace(speck.name, '')) {
eventFieldTypes.add(typeName);
}
}
}
}
for (const iface of allInterfaces) {
const ifaceName = cleanName(iface.name);
if (eventFieldTypes.has(ifaceName)) {
if (iface.methods?.length > 0) {
enumMap.set(ifaceName, iface.methods.map((m: any) => m.name));
} else {
const seen = new Set<string>();
const builtinTypes = new Set(['String', 'Nat', 'Int', 'Bool', 'Real', 'Bytes', 'Date', 'Number']);
for (const a of actions) {
for (const s of a.statements) {
if (s.type === 'require') {
const matches = s.expr.match(/\b[A-Z][a-zA-Z]+\b/g);
if (matches) {
matches.forEach((m: string) => {
if (!builtinTypes.has(m)) seen.add(m);
});
}
}
if (s.type === 'emit') {
for (const f of s.fields) {
const matches = f.value.match(/\b[A-Z][a-zA-Z]+\b/g);
if (matches) {
matches.forEach((m: string) => {
if (!builtinTypes.has(m)) seen.add(m);
});
}
}
}
}
}
const primaryValues = new Set(enumMap.get(stateEnumName.replace(speck.name, '')) || []);
const secondaryValues = Array.from(seen).filter(v => !primaryValues.has(v));
if (secondaryValues.length > 0) {
enumMap.set(ifaceName, secondaryValues.sort());
}
}
}
}
// Generate Rust enums
const allEnums: string[] = [];
for (const [enumName, values] of enumMap) {
const fullEnumName = `${speck.name}${enumName}`;
const variants = values.map((v: string) => ` ${v},`).join('\n');
const enumDef = `#[derive(Debug, Clone, Copy, PartialEq, Eq)]\n` +
`pub enum ${fullEnumName} {\n${variants}\n}`;
allEnums.push(enumDef);
const displayArms = values.map((v: string) =>
` ${fullEnumName}::${v} => write!(f, "${snakeCase(v)}"),`
).join('\n');
const displayImpl = `impl std::fmt::Display for ${fullEnumName} {\n` +
` fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n` +
` match self {\n${displayArms}\n` +
` }\n` +
` }\n` +
`}`;
allEnums.push(displayImpl);
}
const stateEnum = allEnums.join('\n\n');
// Event structs
const eventStructs = events.map(e => {
const fields = e.fields.map(f => {
const name = snakeCase(cleanName(f.name));
const ty = rustType(f.type, speck.name, enumMap);
return ` pub ${name}: ${ty},`;
}).join('\n');
return `#[derive(Debug, Clone)]\npub struct ${e.name} {\n${fields}\n}`;
}).join('\n\n');
const structName = `${speck.name}Machine`;
// State variables with original name + snake_case name mapping
const stateVars = (stateNode?.variables ?? []).map(v => ({
origName: cleanName(v.name),
rustName: snakeCase(cleanName(v.name)),
typeExpr: v.typeExpr,
defaultInit: v.defaultInit,
}));
// Build name mapping for expression rewriting: original name → Rust field name
const nameMap = new Map<string, string>();
for (const v of stateVars) {
nameMap.set(v.origName, v.rustName);
}
// For expression matching, use the original names
const stateVarOrigNames = new Set(stateVars.map(v => v.origName));
// Map variable original names (for bracket -> index rewriting)
const mapVarOrigNames = new Set(stateVars
.filter(v => v.typeExpr?.type === 'map' || (v.typeExpr?.type === 'ident' && v.typeExpr?.name && /^Map/i.test(v.typeExpr.name)))
.map(v => v.origName));
// State struct fields
const stateFields = stateVars.map(v => {
return ` pub ${v.rustName}: ${rustType(v.typeExpr, speck.name, enumMap)},`;
}).join('\n');
// Collect all known state values for the rewriter
const allKnownStateValues: string[] = [];
for (const values of enumMap.values()) {
allKnownStateValues.push(...values);
}
// Init assignments
const initAssignments = initNode
? initNode.assignments.map(a => {
const rustName = snakeCase(cleanName(a.name));
const expr = cleanExpr(a.expr);
const rewritten = rewriteRustExpr(
expr, nameMap, mapVarOrigNames, new Set(),
stateEnumName, allKnownStateValues
);
return ` ${rustName}: ${rewritten},`;
}).join('\n')
: '';
// If no explicit init assignments, emit defaults for all state vars
const initFields = initAssignments || stateVars.map(v => {
const defaultValue = defaultRustValue(v.rustName, v.typeExpr);
return ` ${v.rustName}: ${defaultValue},`;
}).join('\n');
// Action methods
const actionMethods = actions
.map(a => emitAction(a, stateVarOrigNames, nameMap, mapVarOrigNames, speck.name, stateEnumName, allKnownStateValues, enumMap))
.join('\n\n');
// Invariant checks
const invariantChecks = constraints.map((c, i) =>
` /// C${i + 1}: ${c.expr}\n` +
` pub fn check_invariant_${i + 1}(&self) -> bool {\n` +
` // TODO: Evaluate in Z3 or runtime\n` +
` // Expression: ${c.expr}\n` +
` true\n` +
` }`
).join('\n\n');
// Verify stubs
const verifyChecks = verifies.map((v, i) =>
` /// V${i + 1}: ${v.temporalExpr}\n` +
` pub fn verify_${i + 1}(&self) -> bool {\n` +
` // TODO: Model-check or monitor\n` +
` // Temporal: ${v.temporalExpr}\n` +
` true\n` +
` }`
).join('\n\n');
// Decision log type
const logEventType = events.length > 0 ? `Vec<${events[0].name}>` : 'Vec<String>';
// Struct definition
const structDef = `/// Speck state machine: ${speck.name}\n` +
`/// Auto-generated Rust state machine from SpeckDL specification.\n` +
`#[derive(Debug, Clone)]\n` +
`pub struct ${structName} {\n${stateFields}\n` +
` /// Decision log of events emitted during execution\n` +
` pub decision_log: ${logEventType},\n` +
`}`;
// Constructor impl
const constructor = `impl ${structName} {\n` +
` /// Create a new ${speck.name} machine in its initial state.\n` +
` pub fn new() -> Self {\n` +
` Self {\n` +
`${initFields}\n` +
` decision_log: Vec::new(),\n` +
` }\n` +
` }\n` +
`}`;
// Action impl block
const actionImpl = `// Action methods — each returns Result with guard checking\n` +
`impl ${structName} {\n${actionMethods}\n}`;
// Invariant impl block
const invariantImpl = constraints.length > 0
? `\n// Invariant checks\nimpl ${structName} {\n${invariantChecks}\n}`
: '';
// Verify impl block
const verifyImpl = verifies.length > 0
? `\n// Temporal verification stubs\nimpl ${structName} {\n${verifyChecks}\n}`
: '';
const file =
`// Auto-generated by speckl-compile from ${speck.name}\n` +
`// DO NOT EDIT MANUALLY — regenerate from .speckdl source\n\n` +
`use std::collections::{HashMap, HashSet};\n\n` +
`${stateEnum}\n\n` +
`${eventStructs}\n\n` +
`${structDef}\n\n` +
`${constructor}\n\n` +
`${actionImpl}\n` +
`${invariantImpl}\n` +
`${verifyImpl}\n`;
return file;
}
function emitAction(
action: ActionNode,
stateVarOrigNames: Set<string>,
nameMap: Map<string, string>,
mapVarOrigNames: Set<string>,
speckName: string,
stateEnumName: string,
knownStateValues: string[],
enumMap: Map<string, string[]>
): string {
const localNames = new Set(action.params.map(p => cleanName(p.name)));
const params = action.params
.map(p => `${snakeCase(cleanName(p.name))}: ${rustType(p.type, speckName, enumMap)}`)
.join(', ');
const methodName = snakeCase(action.name);
// Preconditions
const preconditions = action.statements
.filter(s => s.type === 'precondition')
.map(s => {
const expr = rewriteRustExpr(s.expr, nameMap, mapVarOrigNames, localNames, stateEnumName, knownStateValues);
return ` // precondition: ${s.expr}\n if !(${expr}) {\n return Err(format!("Precondition failed: {}", ${JSON.stringify(s.expr)}));\n }`;
})
.join('\n');
// Requires (guards)
const requires = action.statements
.filter(s => s.type === 'require')
.map(s => {
const expr = rewriteRustExpr(s.expr, nameMap, mapVarOrigNames, localNames, stateEnumName, knownStateValues);
return ` if !(${expr}) {\n return Err(format!("Guard failed: {}", ${JSON.stringify(s.expr)}));\n }`;
})
.join('\n');
// Assignments
const assigns = action.statements
.filter(s => s.type === 'assign')
.map(s => {
const target = snakeCase(cleanName(s.target));
const expr = rewriteRustExpr(s.expr, nameMap, mapVarOrigNames, localNames, stateEnumName, knownStateValues);
return ` self.${target} = ${expr};`;
})
.join('\n');
// Emits
const emits = action.statements
.filter(s => s.type === 'emit')
.map(s => {
const fields = s.fields
.map(f => {
const name = snakeCase(cleanName(f.name));
const val = rewriteRustExpr(f.value, nameMap, mapVarOrigNames, localNames, stateEnumName, knownStateValues);
return ` ${name}: ${val},`;
})
.join('\n');
return ` self.decision_log.push(${s.event} {\n${fields}\n });`;
})
.join('\n');
// Postconditions
const postconditions = action.statements
.filter(s => s.type === 'postcondition')
.map(s => {
const expr = rewriteRustExpr(s.expr, nameMap, mapVarOrigNames, localNames, stateEnumName, knownStateValues);
return ` // postcondition: ${s.expr}\n if !(${expr}) {\n return Err(format!("Postcondition failed: {}", ${JSON.stringify(s.expr)}));\n }`;
})
.join('\n');
// Returns
const returnStmts = action.statements
.filter(s => s.type === 'return')
.map(s => {
const expr = rewriteRustExpr(s.expr, nameMap, mapVarOrigNames, localNames, stateEnumName, knownStateValues);
return ` Ok(${expr})`;
})
.join('\n');
const guards = [preconditions, requires].filter(s => s.length > 0);
const mutations = [assigns, emits].filter(s => s.length > 0);
const postPost = [postconditions, returnStmts].filter(s => s.length > 0);
const bodyParts: string[] = [];
if (guards.length > 0) bodyParts.push(guards.join('\n'));
if (mutations.length > 0) bodyParts.push(mutations.join('\n'));
if (postPost.length > 0) bodyParts.push(postPost.join('\n'));
const body = bodyParts.join('\n');
// Determine return type
const hasReturn = action.statements.some(s => s.type === 'return');
const returnType = hasReturn ? ' -> Result<bool, String>' : ' -> Result<(), String>';
return ` /// Execute action: ${action.name}\n pub fn ${methodName}(&mut self${params ? ', ' + params : ''})${returnType} {\n${body || ' Ok(())\n'}\n }`;
}
/**
* Rewrite SpeckDL expressions to valid Rust.
*
* @param nameMap - Maps original SpeckDL variable names → Rust snake_case field names (e.g. isOn → is_on)
* @param mapVarNames - Set of original variable names that are Map-typed (for bracket access rewriting)
*/
function rewriteRustExpr(
expr: string,
nameMap: Map<string, string>,
mapVarOrigNames: Set<string>,
localNames: Set<string>,
stateEnumName: string,
knownStateValues: string[]
): string {
function shouldPrefix(ident: string): boolean {
return nameMap.has(ident) && !localNames.has(ident);
}
function rustify(ident: string): string {
const mapped = nameMap.get(ident);
return mapped ? `self.${mapped}` : ident;
}
// Step 1: Replace function calls
let rust = expr
.replace(/\bnow\(\)/g, '0u64 /* TODO: use std::time::SystemTime */')
.replace(/\blen\(([^)]+)\)/g, '$1.len()')
.replace(/\blength\(([^)]+)\)/g, '$1.len()')
.replace(/\bjoin\(([^,]+),\s*([^)]+)\)/g, '$1.join($2)')
.replace(/\bappend\(([^,]+),\s*([^)]+)\)/g, '{$1.push($2); $1} /* append */')
.replace(/\bforall\s+(\w+)\s+in\s+([^:]+):\s*(.+)/g, '$2.iter().all(|$1| $3)');
// Step 2: Replace state enum literals with Rust enum refs BEFORE identifier replacement
const allStateValues = new Set<string>(knownStateValues);
['Open','Grooming','Groomed','Assigned','InProgress','InReview','Merged','Blocked','Closed','Idle','Parsing','Parsed','ProvenanceGenerated','BOMsGenerated','TypeScriptGenerated','Done'].forEach(s => allStateValues.add(s));
if (allStateValues.size > 0) {
const stateValueRegex = new RegExp('\\b(' + Array.from(allStateValues).join('|') + ')\\b', 'g');
rust = rust.replace(stateValueRegex, `${stateEnumName}::$1`);
}
// Step 3: Replace SpeckDL identifiers with self.rust_name if they're state vars
// Split on non-identifier boundaries
const tokens = rust.split(/([\s\(\)\[\]{}+\-*/=<>!&|.,;:'"`]+)/);
const result = tokens.map(tok => {
if (/^[a-zA-Z_]\w*::\w+$/.test(tok)) return tok; // already :: (enum ref)
if (/^[a-zA-Z_]\w*\.\w+$/.test(tok)) return tok; // already dot (field access)
if (/^(true|false|null|None|Some|Ok|Err|not|and|or|in|notIn|old|forall)$/.test(tok)) return tok;
if (/^\d+$/.test(tok)) return tok;
if (/^[a-zA-Z_]\w*$/.test(tok) && shouldPrefix(tok)) {
return rustify(tok);
}
return tok;
}).join('');
rust = result
// Undo self. prefix that accidentally got added after bracket access
.replace(/\]\.self\.(\w+)/g, '].$1');
// Step 4: Translate SpeckDL expression patterns to Rust
// 4a: notIn -> !collection.contains(&key)
rust = rust.replace(/(\S+)\s+notIn\s+(\S+)\.keys/g, '!$2.contains_key(&$1)');
// x in map.keys -> map.contains_key(&x)
rust = rust.replace(/(\S+)\s+in\s+(\S+)\.keys/g, '$2.contains_key(&$1)');
// x in set -> set.contains(&x) (but be careful not to match Rust keywords)
rust = rust.replace(/(\S+)\s+in\s+(\S+)/g, (match, key, collection) => {
const reserved = new Set(['true','false','self','mut','ref','let','fn','pub','use','mod','struct','enum','impl','for','while','loop','if','else','match','return','where','as','move','dyn','unsafe','extern','crate','super','Self']);
if (reserved.has(collection)) return match;
return `${collection}.contains(&${key})`;
});
// notIn plain (not .keys)
rust = rust.replace(/(\S+)\s+notIn\s+(\S+)/g, '!$2.contains(&$1)');
// 4b: not expr → !expr (SpeckDL 'not' keyword)
// Need to handle 'not' followed by an identifier — this was already handled above with self-dot rewrite
// But plain 'not ident' must become '!self.field_name'
rust = rust.replace(/\bnot\s+([a-zA-Z_]\w*)/g, '!$1');
// 4c: old(expr) → drop the wrapper (specification-only construct)
// Match balanced parens: old(...)
rust = rust.replace(/old\(([^()]*(?:\([^()]*\)[^()]*)*)\)/g, '$1');
// 4d: Set union: pending union {t.id}
rust = rust.replace(/(\S+)\s+union\s+\{([^}]+)\}/g, '{$1.clone().into_iter().chain(std::iter::once($2)).collect()}');
// 4e: Set difference: pending \\ {id}
rust = rust.replace(/(\S+)\s+\\\\\s+\{([^}]+)\}/g, '{$1.clone().into_iter().filter(|x| x != &$2).collect()}');
// 4f: Map access via brackets → bracket indexing (Rust uses &key for HashMap)
for (const mapVar of mapVarOrigNames) {
const rustMapVar = nameMap.get(mapVar) || snakeCase(mapVar);
// Handle self.field_name[expr] or bare field_name[expr]
// Must run BEFORE identifier replacement so original names match
const pattern1 = new RegExp(`(self\\.)?(${escapeRegex(rustMapVar)})\\[([^\\]]+)\\]`, 'g');
rust = rust.replace(pattern1, '$1$2[&$3]');
// Also handle original name in brackets (e.g. accounts[...])
const pattern2 = new RegExp(`(self\\.)?(${escapeRegex(mapVar)})\\[([^\\]]+)\\]`, 'g');
rust = rust.replace(pattern2, (match, p1, _p2, p3) => {
const prefix = p1 || '';
return `${prefix}${rustMapVar}[&${p3}]`;
});
}
// 4g: .size() → .len() (Rust collections use .len())
rust = rust.replace(/\.size\(\)/g, '.len()');
// 4h: == → keeps == (Rust uses ==), but handle === → ==
rust = rust.replace(/\b===\b/g, '==');
// 4i: != keeps != in Rust
// 4j: Clean up trailing standalone comparisons
rust = rust.replace(/(\S+)\s*==\s*$/g, '$1 == $1');
rust = rust.replace(/(\S+)\s*===\s*$/g, '$1 == $1');
return rust.trim();
}
// Helpers ------------------------------------------------------------------
function memberKey(m: MemberNode): string {
if (m.type === 'action') return `action:${m.name}`;
if (m.type === 'event') return `event:${m.name}`;
if (m.type === 'interface') return `interface:${m.name}`;
if (m.type === 'state') return 'state';
if (m.type === 'init') return 'init';
if (m.type === 'provenance') return 'provenance';
if (m.type === 'bom') return 'bom';
if (m.type === 'input') return 'input';
if (m.type === 'output') return 'output';
return m.type;
}
function isEmptyStub(m: MemberNode): boolean {
if (m.type === 'state') return m.variables.length === 0;
if (m.type === 'init') return m.assignments.length === 0;
if (m.type === 'action') return m.statements.length === 0;
if (m.type === 'event') return m.fields.length === 0;
if (m.type === 'interface') return m.methods.length === 0;
if (m.type === 'provenance') return m.clauses.length === 0;
if (m.type === 'bom') return !m.compiler && !m.solver && !m.runtime;
return false;
}
function cleanName(s: string): string {
return s.replace(/,$/, '').trim();
}
function cleanExpr(s: string): string {
return s.replace(/,$/, '').trim();
}
function snakeCase(s: string): string {
// Convert CamelCase/PascalCase to snake_case
return s
.replace(/([A-Z]+)([A-Z][a-z])/g, '$1_$2')
.replace(/([a-z\d])([A-Z])/g, '$1_$2')
.replace(/([a-zA-Z])(\d)/g, '$1_$2')
.replace(/[^a-zA-Z0-9_]/g, '_')
.toLowerCase()
.replace(/_+/g, '_')
.replace(/^_|_$/g, '');
}
function escapeRegex(s: string): string {
return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
}
function defaultRustValue(rustName: string, typeExpr: any): string {
if (!typeExpr) return 'Default::default()'; // not ideal but a fallback
if (typeExpr.type === 'primitive') {
switch (typeExpr.name) {
case 'Nat': return '0u64';
case 'Int': return '0i64';
case 'Real': case 'Number': return '0.0f64';
case 'Bool': return 'false';
case 'String': return 'String::new()';
case 'Bytes': return 'Vec::new()';
default: return 'Default::default()';
}
}
if (typeExpr.type === 'list') return 'Vec::new()';
if (typeExpr.type === 'set') return 'HashSet::new()';
if (typeExpr.type === 'map') return 'HashMap::new()';
return 'Default::default()';
}
function rustType(t: any, speckName: string = '', enumMap?: Map<string, string[]>): string {
if (!t) return '()';
if (t.type === 'primitive') {
const map: Record<string, string> = {
Nat: 'u64', Int: 'i64', Real: 'f64',
Bool: 'bool', String: 'String', Bytes: 'Vec<u8>',
Number: 'f64',
};
return map[t.name] || t.name.toLowerCase();
}
if (t.type === 'list') return `Vec<${rustType(t.elementType, speckName, enumMap)}>`;
if (t.type === 'set') return `HashSet<${rustType(t.elementType, speckName, enumMap)}>`;
if (t.type === 'map') return `HashMap<${rustType(t.keyType, speckName, enumMap)}, ${rustType(t.valueType, speckName, enumMap)}>`;
if (t.type === 'record') {
const fields = t.fields.map((f: any) => {
const name = snakeCase(cleanName(f.name));
return `pub ${name}: ${rustType(f.type, speckName, enumMap)}`;
}).join(', ');
return `${snakeCase(speckName)}_record { ${fields} }`;
}
if (t.type === 'ident') {
const rawName = cleanName(t.name);
if (enumMap && enumMap.has(rawName)) {
return `${speckName}${rawName}`;
}
if (rawName === 'Account') return 'Account';
if (rawName === 'Transfer') return 'Transfer';
if (rawName === 'Replica') return 'Replica';
if (rawName === 'LogEntry') return 'LogEntry';
if (rawName === 'Nat') return 'u64';
if (rawName === 'Int') return 'i64';
if (rawName === 'Bool') return 'bool';
if (rawName === 'String') return 'String';
// Handle Option<T> from SpeckDL's Option(Type) syntax
const optionMatch = rawName.match(/^Option[_(]\s*(\w+)\s*\)?$/);
if (optionMatch) {
const inner = optionMatch[1];
const innerRust = inner === 'Nat' ? 'u64' :
inner === 'Int' ? 'i64' :
inner === 'Bool' ? 'bool' :
inner === 'String' ? 'String' : inner;
return 'Option<' + innerRust + '>';
}
// Opaque identifiers become type aliases
return rawName;
}
return '()';
}

599
compiler/src/generators/typescript-state-machine.ts
View file

@ -1,4 +1,4 @@
import { AST, SpeckNode, MemberNode, StateNode, InitNode, ActionNode, ConstraintNode, VerifyNode, EventNode, TypeExpr } from '../parser.js';
import { AST, SpeckNode, MemberNode, StateNode, InitNode, ActionNode, ConstraintNode, VerifyNode, EventNode } from '../parser.js';
import fs from 'fs';
import path from 'path';
@ -38,19 +38,6 @@ function emitSpeck(speck: SpeckNode): string {
const initNode = members.find(m => m.type === 'init') as InitNode | undefined;
const actions = members.filter(m => m.type === 'action') as ActionNode[];
const events = members.filter(m => m.type === 'event') as EventNode[];
// Collect implicit event types from emit statements
const implicitEvents = new Map<string, { name: string; fields: { name: string; value: string }[] }>();
for (const action of actions) {
for (const stmt of action.statements) {
if (stmt.type === 'emit') {
const existing = implicitEvents.get(stmt.event);
if (!existing) {
implicitEvents.set(stmt.event, { name: stmt.event, fields: stmt.fields });
}
}
}
}
const constraints = members.filter(m => m.type === 'constraint') as ConstraintNode[];
const verifies = members.filter(m => m.type === 'verify') as VerifyNode[];
@ -101,24 +88,8 @@ function emitSpeck(speck: SpeckNode): string {
}
// If we fell back to guard scanning, create the primary enum under the right name
// Avoid collision: don't add to enumMap if the key matches an existing ident type in state vars
const stateVarIdentNames = new Set<string>();
for (const v of (stateNode?.variables ?? [])) {
(function visit(t: any) {
if (!t) return;
if (t.type === 'ident' && t.name) stateVarIdentNames.add(cleanName(t.name));
if (t.elementType) visit(t.elementType);
if (t.keyType) visit(t.keyType);
if (t.valueType) visit(t.valueType);
if (t.fields) t.fields.forEach((f: any) => visit(f.type));
})(v.typeExpr);
}
const enumKey = stateEnumName.replace(speck.name, '');
if (enumInterfaces.length === 0 && !stateVarIdentNames.has(enumKey)) {
enumMap.set(enumKey, stateValues);
} else if (enumInterfaces.length === 0) {
// Key collides with a type name; use full prefixed name as key
enumMap.set(stateEnumName, stateValues);
if (enumInterfaces.length === 0) {
enumMap.set(stateEnumName.replace(speck.name, ''), stateValues);
}
// Scan event field types for secondary enum types (e.g., ArtifactKind)
@ -187,21 +158,13 @@ function emitSpeck(speck: SpeckNode): string {
}
const stateEnum = allEnums.join('\n\n');
const eventInterfaces = [
// Explicit event declarations
...events.map(e => {
const fields = e.fields.map(f => {
const name = cleanName(f.name);
return ` ${name}: ${tsType(f.type, speck.name, enumMap)};`;
}).join('\n');
return `export interface ${e.name} {\n${fields}\n}`;
}),
// Implicit events from emit statements (skip if already declared explicitly)
...Array.from(implicitEvents.values()).filter(e => !events.some(ev => ev.name === e.name)).map(e => {
const fields = e.fields.map((f: any) => ` ${f.name}: any;`).join('\n');
return `export interface ${e.name} {\n${fields}\n}`;
})
].join('\n\n');
const eventInterfaces = events.map(e => {
const fields = e.fields.map(f => {
const name = cleanName(f.name);
return ` ${name}: ${tsType(f.type, speck.name, enumMap)};`;
}).join('\n');
return `export interface ${e.name} {\n${fields}\n}`;
}).join('\n\n');
const className = `${speck.name}Machine`;
const stateVars = (stateNode?.variables ?? []).map(v => ({
@ -212,32 +175,6 @@ function emitSpeck(speck: SpeckNode): string {
const stateVarNames = new Set(stateVars.map(v => v.name));
// Collect identifier types used in state vars and action params (opaque record types)
// These are types like 'Account', 'Transfer' that are referenced but not defined in the spec
const opaqueRecordTypes = new Set<string>();
const builtinIdents = new Set(['String', 'Nat', 'Int', 'Bool', 'Real', 'Bytes', 'Date', 'Number', 'IssueState', 'AST', 'ArtifactMap']);
function collectIdents(t: TypeExpr | undefined): void {
if (!t) return;
if (t.type === 'ident' && t.name) {
const name = cleanName(t.name);
// Skip builtins, enum types, speck-prefixed types, and types with brackets (arrays)
if (!builtinIdents.has(name) && !enumMap.has(name) && !name.startsWith(speck.name) && !name.includes('[')) {
opaqueRecordTypes.add(name);
}
}
if (t.elementType) collectIdents(t.elementType);
if (t.keyType) collectIdents(t.keyType);
if (t.valueType) collectIdents(t.valueType);
if (t.fields) t.fields.forEach((f: any) => collectIdents(f.type));
}
for (const v of (stateNode?.variables ?? [])) collectIdents(v.typeExpr);
for (const a of actions) for (const p of a.params) collectIdents(p.type);
// Collect which state variables are Map types (for bracket -> .get() translation)
const mapVarNames = new Set(stateVars
.filter(v => v.typeExpr?.type === 'map' || (v.typeExpr?.type === 'ident' && v.typeExpr?.name && /^Map/i.test(v.typeExpr.name)))
.map(v => v.name));
const stateProperties = stateVars.map(v => {
return ` ${v.name}: ${tsType(v.typeExpr, speck.name, enumMap)};`;
}).join('\n');
@ -258,7 +195,7 @@ function emitSpeck(speck: SpeckNode): string {
}).join('\n')
: '';
const actionMethods = actions.map(a => emitAction(a, stateVarNames, mapVarNames, speck.name, stateEnumName, allKnownStateValues, enumMap)).join('\n\n');
const actionMethods = actions.map(a => emitAction(a, stateVarNames, speck.name, stateEnumName, allKnownStateValues, enumMap)).join('\n\n');
const constraintChecks = constraints.map((c, i) =>
` // C${i + 1}: ${c.expr}\n` +
@ -279,30 +216,11 @@ function emitSpeck(speck: SpeckNode): string {
).join('\n\n');
// Decision log type: use first event if available, otherwise any[]
const logType = events.length > 0 ? '(' + events.map(e => e.name).join(' | ') + ')[]' : 'any[]';
// Build opaque type declarations
const opaqueTypeDecls = opaqueRecordTypes.size > 0
? '// Runtime type placeholders (opaque record types)\n' +
Array.from(opaqueRecordTypes).sort().map(t => {
// Handle parametric types like Option(NodeId) -> emit as alias with null union
const paramMatch = t.match(/^(\w+)\((.+)\)$/);
if (paramMatch) {
const [, base, inner] = paramMatch;
return `type ${base}${inner.replace(/\s+/g, '')} = ${inner} | null;`;
}
return `type ${t} = any;`;
}).join('\n') + '\n\n'
: '';
// Add non-null assertions (!) to class properties for strict mode
const statePropertiesNonNull = stateVars.map(v => {
return ` ${v.name}!: ${tsType(v.typeExpr, speck.name, enumMap)};`;
}).join('\n');
const logType = events.length > 0 ? events[0].name + '[]' : 'any[]';
const classBody = `
${opaqueTypeDecls}export class ${className} {
${statePropertiesNonNull}
export class ${className} {
${stateProperties}
decisionLog: ${logType} = [];
constructor() {
@ -317,460 +235,153 @@ ${verifyChecks}
}
`;
const hashUtility = speck.members.some(m => {
if (m.type !== 'action') return false;
const stmts = (m as any).statements || [];
return stmts.some((s: any) => s.expr && s.expr.includes('hash('));
}) ?
`// Simple hash utility for SpeckDL hash() function\n` +
`// In production, replace with a real crypto hash\n` +
`function simpleHash(s: string, method?: string): string {\n` +
` let h = 0;\n` +
` for (let i = 0; i < s.length; i++) {\n` +
` h = ((h << 5) - h + s.charCodeAt(i)) | 0;\n` +
` }\n` +
` return 'h_' + Math.abs(h).toString(16);\n` +
`}\n\n`
: '';
const file =
`// Auto-generated by speckl-compile from ${speck.name}\n` +
`// DO NOT EDIT MANUALLY — regenerate from .speckdl source\n\n` +
`${stateEnum}\n\n` +
`${eventInterfaces}\n\n` +
`${hashUtility}` +
`${classBody}`;
return file;
}
function emitAction(action: ActionNode, stateVarNames: Set<string>, mapVarNames: Set<string>, speckName: string, stateEnumName: string, knownStateValues: string[], enumMap: Map<string, string[]>): string {
function emitAction(action: ActionNode, stateVarNames: Set<string>, speckName: string, stateEnumName: string, knownStateValues: string[], enumMap: Map<string, string[]>): string {
// Collect local parameter names so we DON'T prefix them with this.
const localNames = new Set(action.params.map(p => p.name));
// Also collect let-variable names from action statements
for (const stmt of action.statements) {
if (stmt.type === 'let' && stmt.name) localNames.add(stmt.name);
}
const params = action.params
.map(p => `${p.name}: ${tsType(p.type, speckName, enumMap)}`)
.join(', ');
// Statements in SpeckDL order: emit lets before guards so guards can reference them
// (SpeckDL allows guards to reference let-bound variables that appear before them)
const bodyParts: string[] = [];
for (const stmt of action.statements) {
switch (stmt.type) {
case 'precondition': {
const expr = rewriteExpr(stmt.expr, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues);
bodyParts.push(` // precondition: ${expr}\n if (!(${expr})) throw new Error('Precondition failed: ${expr.replace(/'/g, "\\'")}');`);
break;
}
case 'require': {
const expr = rewriteExpr(stmt.expr, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues);
bodyParts.push(` if (!(${expr})) throw new Error('Guard failed: ${expr.replace(/'/g, "\\'")}');`);
break;
}
case 'let': {
let expr = rewriteExpr(stmt.expr, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues);
// Fix: if rewriteExpr converted if-expression to a comment (incomplete conditional),
// replace the const declaration with undefined + comment
if (expr.match(/^\/\* if /)) {
expr = `undefined; // ${expr.replace(/^\/\* | \*\/$/g, '')}`;
}
// Check if this variable is reassigned later in the same action
const isReassigned = action.statements.some((s: any, i: number) => i > action.statements.indexOf(stmt) && s.type === 'assign' && s.target === stmt.name);
const keyword = isReassigned ? 'let' : 'const';
bodyParts.push(` ${keyword} ${stmt.name} = ${expr};`);
break;
}
case 'assign': {
const target = cleanName(stmt.target);
const expr = rewriteExpr(stmt.expr, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues);
// Handle Map bracket assignment: mapVar[index] := value -> this.mapVar.set(index, value)
// Support nested brackets by finding the outermost [ ] pair
const mapBracketMatch = target.match(/^(\w+)\[([\s\S]+)\]$/);
if (mapBracketMatch && mapVarNames.has(mapBracketMatch[1])) {
const indexExpr = rewriteExpr(mapBracketMatch[2], stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues);
// Detect compound key: { key1: val1, key2: val2 } -> nested Map access
const compoundKeyMatch = indexExpr.match(/^\{\s*(\w+)\s*:\s*([^,{}]+),\s*(\w+)\s*:\s*([^,{}]+)\s*\}$/);
if (compoundKeyMatch) {
// Map<K1, Map<K2, V> -> this.mapVar.get(k1)!.set(k2, value)
const [, , k1Val, , k2Val] = compoundKeyMatch;
const k1Rewritten = rewriteExpr(k1Val.trim(), stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues);
const k2Rewritten = rewriteExpr(k2Val.trim(), stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues);
if (expr.trim() === 'null') {
bodyParts.push(` this.${mapBracketMatch[1]}.get(${k1Rewritten})!.delete(${k2Rewritten});`);
} else {
bodyParts.push(` this.${mapBracketMatch[1]}.get(${k1Rewritten})!.set(${k2Rewritten}, ${expr});`);
}
} else if (expr.trim() === 'null') {
bodyParts.push(` this.${mapBracketMatch[1]}.delete(${indexExpr});`);
} else {
bodyParts.push(` this.${mapBracketMatch[1]}.set(${indexExpr}, ${expr});`);
}
} else if (localNames.has(target)) {
bodyParts.push(` ${target} = ${expr};`);
} else {
bodyParts.push(` this.${target} = ${expr};`);
}
break;
}
case 'emit': {
const fields = stmt.fields.map(f => {
const preconditions = action.statements
.filter(s => s.type === 'precondition')
.map(s => {
const expr = rewriteExpr(s.expr, stateVarNames, localNames, stateEnumName, knownStateValues);
return ` // precondition: ${expr}\n if (!(${expr})) throw new Error('Precondition failed: ${expr.replace(/'/g, "\\'")}');`;
})
.join('\n');
const requires = action.statements
.filter(s => s.type === 'require')
.map(s => {
const expr = rewriteExpr(s.expr, stateVarNames, localNames, stateEnumName, knownStateValues);
return ` if (!(${expr})) throw new Error('Guard failed: ${expr.replace(/'/g, "\\'")}');`;
})
.join('\n');
const assigns = action.statements
.filter(s => s.type === 'assign')
.map(s => {
const target = cleanName(s.target);
const expr = rewriteExpr(s.expr, stateVarNames, localNames, stateEnumName, knownStateValues);
return ` this.${target} = ${expr};`;
})
.join('\n');
const emits = action.statements
.filter(s => s.type === 'emit')
.map(s => {
const fields = s.fields
.map(f => {
const name = cleanName(f.name);
const val = rewriteExpr(f.value, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues);
const val = rewriteExpr(f.value, stateVarNames, localNames, stateEnumName, knownStateValues);
return ` ${name}: ${val}`;
}).join(',\n');
bodyParts.push(` this.decisionLog.push({\n${fields}\n } as ${stmt.event});`);
break;
}
case 'ifblock': {
bodyParts.push(` // TODO: SpeckDL conditional block (if/else) - manual translation required`);
break;
}
case 'return': {
const expr = rewriteExpr(stmt.expr, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues);
bodyParts.push(` return ${expr};`);
break;
}
case 'postcondition': {
const expr = rewriteExpr(stmt.expr, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues);
const needsTsIgnore = /===\s*\{|===\s*\[/.test(expr);
const tsIgnore = needsTsIgnore ? ' // @ts-expect-error TS2839 — generated spec comparison (structural equality not available in JS)\n' : '';
bodyParts.push(`${tsIgnore} // postcondition: ${expr}\n if (!(${expr})) throw new Error('Postcondition failed: ${expr.replace(/'/g, "\\'")}');`);
break;
}
}
}
const bodyPartsStr = bodyParts.filter(Boolean).join('\n');
})
.join(',\n');
return ` this.decisionLog.push({\n${fields}\n } as ${s.event});`;
})
.join('\n');
// Detect parser-dropped variables (let/forall blocks the parser doesn't emit)
const reservedWords = new Set([
"abstract", "any", "as", "async", "await", "boolean", "break", "case", "catch", "class",
"const", "continue", "debugger", "default", "delete", "do", "else", "enum", "export",
"extends", "false", "finally", "for", "from", "function", "get", "if", "implements",
"import", "in", "instanceof", "interface", "let", "never", "new", "null", "number",
"object", "of", "package", "private", "protected", "public", "readonly", "require",
"return", "set", "static", "string", "super", "switch", "symbol", "this", "throw",
"true", "try", "typeof", "undefined", "unknown", "var", "void", "while", "with", "yield",
"console", "Math", "Date", "JSON", "Object", "Array", "Map", "Set", "Promise", "Error",
"Number", "String", "Boolean", "Symbol", "NaN", "Infinity", "parseInt", "parseFloat",
"isNaN", "isFinite", "encodeURI", "decodeURI", "encodeURIComponent", "decodeURIComponent",
"simpleHash",
"has", "get", "set", "delete", "size", "keys", "values", "entries", "push", "pop", "length",
"add", "concat", "filter", "map", "forEach", "join", "indexOf", "includes", "toString",
"valueOf", "some", "every", "reduce", "find", "slice", "splice", "sort"
]);
const known = new Set([...localNames, ...stateVarNames, ...reservedWords]);
for (const sv of knownStateValues) known.add(sv);
for (const m of bodyPartsStr.matchAll(/\b(?:const|let|var)\s+(\w+)/g)) known.add(m[1]);
const postconditions = action.statements
.filter(s => s.type === 'postcondition')
.map(s => {
const expr = rewriteExpr(s.expr, stateVarNames, localNames, stateEnumName, knownStateValues);
return ` // postcondition: ${expr}\n if (!(${expr})) throw new Error('Postcondition failed: ${expr.replace(/'/g, "\\'")}');`;
})
.join('\n');
// Clean body: remove strings, comments, property/type names, type assertions
const cleaned = bodyPartsStr
.replace(/"[^"]*"/g, '""')
.replace(/'[^']*'/g, "''")
.replace(/\/\/[^\n]*/g, "")
.replace(/\b([A-Z]\w+)\s*:/g, " :")
.replace(/\.\s*(\w+)/g, ".xxx")
.replace(/\bas\s+\w+/g, " ");
const found = new Set<string>();
for (const m of cleaned.matchAll(/\b([a-zA-Z_]\w*)\b/g)) found.add(m[1]);
const undefined_ = [...found].filter(id => !known.has(id));
const preamble = undefined_.map(id => ' const ' + id + ': any = {} as any; // FIXME: parser dropped let/forall').join('\n');
const finalBody = preamble ? preamble + '\n' + bodyPartsStr : bodyPartsStr;
return ` ${action.name}(${params}) {
${finalBody}
}`;
// Preconditions + requires first (guards), then mutations (assign + emit), then postconditions
const guards = [preconditions, requires].filter(Boolean).join('\n');
const mutations = [assigns, emits].filter(Boolean).join('\n');
const bodyParts = [guards, mutations, postconditions].filter(Boolean).join('\n');
return ` ${action.name}(${params}) {\n${bodyParts}\n }`;
}
// Rewrite SpeckDL expressions to valid TypeScript
// Only prefix identifiers with `this.` if they are state variables AND not local params
function rewriteExpr(expr: string, stateVarNames: Set<string>, mapVarNames: Set<string>, localNames: Set<string>, stateEnumName: string, knownStateValues: string[]): string {
function rewriteExpr(expr: string, stateVarNames: Set<string>, localNames: Set<string>, stateEnumName: string, knownStateValues: string[]): string {
function shouldPrefix(id: string): boolean {
return stateVarNames.has(id) && !localNames.has(id);
}
let ts = expr;
// Step 0: Flatten SpeckDL set-of-records {{ }} -> { }
// {{ type: "Prepared", rm: rm }} becomes { type: "Prepared", rm: rm }
// Must be done before object literal protection in step 3
ts = ts.replace(/\{\{/g, '{').replace(/\}\}/g, '}');
// Step 1: Replace function calls and special syntax
ts = ts
.replace(/\bhash\(([^,{}]+),\s*([^)]+)\)/g, 'simpleHash($1, $2)')
.replace(/\bhash\(([^)]+)\)/g, 'simpleHash($1)')
.replace(/\bnow\(\)/g, 'new Date().toISOString()')
.replace(/\blength\(([^)]+)\)/g, '$1.length')
let ts = expr
.replace(/\bnow\(\)/g, 'Date.now()')
.replace(/\blen\(([^)]+)\)/g, '$1.length')
.replace(/\bjoin\(([^,{}]+),\s*([^)]+)\)/g, '$1 + $2')
.replace(/\bjoin\(([^,]+),\s*([^)]+)\)/g, '$1.join($2)')
.replace(/\bforall\s+(\w+)\s+in\s+([^:]+):\s*(.+)/g, '$2.every(($1) => $3)');
// Step 2: Protect string literals so enum replacement doesn't touch them
const stringLiterals: string[] = [];
ts = ts.replace(/"([^"]*)"/g, (match) => {
stringLiterals.push(match);
return `__STR_LIT_${stringLiterals.length - 1}__`;
});
// Step 2b: Replace state enum literals with enum refs BEFORE identifier replacement
// Step 2: Replace state enum literals with enum refs BEFORE identifier replacement
const allStateValues = new Set<string>(knownStateValues);
// Add common fallback states
['Open','Grooming','Groomed','Assigned','InProgress','InReview','Merged','Blocked','Closed','Idle','Parsing','Parsed','ProvenanceGenerated','BOMsGenerated','TypeScriptGenerated','Done'].forEach(s => allStateValues.add(s));
['Open','Grooming','Groomed','Assigned','InProgress','InReview','Merged','Blocked','Closed','Idle','Parsing','Parsed','ProvenanceGenerated','BOMsGenerated','TypeScriptGenerated','WASMGenerated','Done'].forEach(s => allStateValues.add(s));
if (allStateValues.size > 0) {
const stateValueRegex = new RegExp('\\b(' + Array.from(allStateValues).join('|') + ')\\b', 'g');
ts = ts.replace(stateValueRegex, `${stateEnumName}.$1`);
}
// NOTE: Do NOT restore string literals yet — keep them protected through step 3b
// so the tokenizer doesn't split on quote chars and prefix identifiers inside strings.
// Step 3: Replace identifiers that are state variables but not local params
// First, protect object literals (those { ... } blocks are object values, not state refs)
// First, protect identifiers inside object literals (e.g. { ...t, pending: false })
// Those identifiers are object keys, not state variable refs
const objectLiterals: string[] = [];
// Match { ... } including nested braces (simplistic: match balanced braces)
ts = ts.replace(/\{[^{}]*(?:\{[^{}]*\}[^{}]*)*\}/g, (match) => {
ts = ts.replace(/\{[^}]*\.\.\.[^}]*\}/g, (match) => {
objectLiterals.push(match);
return `__OBJ_LIT_${objectLiterals.length - 1}__`;
});
// Step 3a: notIn rewrite BEFORE this-prefixing so we can match bare identifiers
// notIn -> !<map>.has(<key>) (e.g. "t.id notIn transfers.keys" -> "!transfers.has(t.id)")
ts = ts.replace(/(\S+)\s+notIn\s+(\S+)\.keys/g, '!$2.has($1)');
// notIn for Sets (no .keys suffix), e.g. "acctId notIn accounts" -> "!accounts.has(acctId)"
ts = ts.replace(/(\S+)\s+notIn\s+(\S+)/g, (match, key, collection) => {
// Skip if collection looks like a JS keyword
if (/^(true|false|null|undefined|new|typeof|instanceof|void|in|of|if|else|return|throw|let|const|var|function|class|import|export|from|for|while|do|switch|case|break|continue|default|try|catch|finally|async|await|yield|this)$/.test(collection)) return match;
return `!${collection}.has(${key})`;
});
// Step 3b: Replace identifiers that are state variables but not local params
// Track if previous token was 'this.' to avoid double-prefixing
const tokens = ts.split(/([\s\(\)\[\]{}+\-*/=<>!&|.,;:'"`]+)/);
let prevWasThisDot = false;
const result = tokens.map(tok => {
if (/^[a-zA-Z_]\w*\.\w+$/.test(tok)) return tok; // already has dot (enum ref)
if (/^(true|false|null|undefined|\d+|\w+\.\w+)$/.test(tok)) return tok;
if (tok === 'this') { prevWasThisDot = true; return tok; }
if (tok === '.') { prevWasThisDot = true; return tok; }
if (prevWasThisDot) { prevWasThisDot = false; return tok; } // skip if preceded by this.
if (/^[a-zA-Z_]\w*$/.test(tok) && shouldPrefix(tok)) {
return `this.${tok}`;
}
prevWasThisDot = false;
return tok;
}).join('')
// Undo this. prefix on identifiers after bracket access
.replace(/\]\.this\.(\w+)/g, '].$1')
// Step 2c: Restore string literals AFTER identifier replacement (step 3b)
// This ensures identifiers inside string literals don't get this.-prefixed
.replace(/__STR_LIT_(\d+)__/g, (_, i) => stringLiterals[parseInt(i)])
// Restore object literals will happen after step 5
// Restore object literals
.replace(/__OBJ_LIT_(\d+)__/g, (_, i) => objectLiterals[parseInt(i)]);
// Step 4: Replace comparison operators (also handle trailing == at end of string)
// Use space-aware replacement since \b==\b fails when == is between spaces (both non-word)
ts = result
.replace(/ == /g, ' === ')
.replace(/ != /g, ' !== ')
.replace(/\b==\b/g, '===')
.replace(/\b!=\b/g, '!==')
.replace(/==$/, '==='); // trailing == at end-of-string (no word boundary after)
// Step 5: Translate SpeckDL expression patterns to JavaScript
// 5a-pre: Map dot-access rewriting (mapVar.paramName -> mapVar.get(paramName)!)
// Handle SpeckDL syntax where "clients.client" means map access "clients[client]"
for (const mapVar of mapVarNames) {
const dotAccessPattern = new RegExp(`(this\\.)?${mapVar}\\.(\\w+)`, 'g');
ts = ts.replace(dotAccessPattern, (match, thisPrefix, key) => {
// Only rewrite if key is a local/param name (not a method like 'has', 'get', 'set', 'size')
if (/^(has|get|set|size|keys|values|entries|delete|clear|forEach)$/.test(key)) return match;
if (localNames.has(key)) {
const prefix = thisPrefix || '';
return `${prefix}${mapVar}.get(${key})!`;
}
return match;
});
}
// 5a: x in <map>.keys -> <map>.has(x) (e.g. "t.from in accounts.keys" -> "accounts.has(t.from)")
ts = ts.replace(/(\S+)\s+in\s+(\S+)\.keys/g, '$2.has($1)');
// 5a3: x in <set> -> <set>.has(x) (e.g. "id in pending" -> "pending.has(id)")
ts = ts.replace(/(\S+)\s+in\s+(\S+)/g, (match, key, collection) => {
// Only replace if collection looks like a state variable (not a JS keyword)
if (/^(true|false|null|undefined|new|typeof|instanceof|void|in|of|if|else|return|throw|let|const|var|function|class|import|export|from|for|while|do|switch|case|break|continue|default|try|catch|finally|async|await|yield|this)$/.test(collection)) return match;
return `${collection}.has(${key})`;
});
// 5a: notIn -> !<map>.has(<key>) (e.g. "t.id notIn transfers.keys" -> "!transfers.has(t.id)")
ts = ts.replace(/(\S+)\s+notIn\s+(\S+)\.keys/g, '!$2.has($1)');
// 5b: old(expr) -> drop the wrapper (specification construct, no JS equivalent needed)
// Match balanced parens: old(...). The inner expr can contain parens.
ts = ts.replace(/old\(([^()]*(?:\([^()]*\)[^()]*)*)\)/g, '$1');
// 5c: union -> new Set(Array.from(a).concat([b])) (set union operator)
// "pending union {t.id}" -> "new Set(Array.from(pending).concat([t.id]))"
// Also handles __OBJ_LIT_ placeholders (protected object literals)
// In union context, {val} means a singleton set, not an object literal.
// But if the val is an object literal (has key:value pairs), keep braces.
ts = ts.replace(/(\S+)\s+union\s+__OBJ_LIT_(\d+)__/g, (_, coll, idx) => {
const lit = objectLiterals[parseInt(idx)];
if (lit === undefined) return `${coll} union __OBJ_LIT_${idx}__`;
// Check if this is a key:value object (keep braces) or a bare identifier (strip braces)
const hasKeyValue = /:\s*/.test(lit.replace(/^\{\s*/, '').replace(/\s*\}$/, ''));
if (hasKeyValue) {
return `new Set(Array.from(${coll}).concat([${lit}]))`;
}
const inner = lit.replace(/^\{\s*/, '').replace(/\s*\}$/, '');
return `new Set(Array.from(${coll}).concat([${inner}]))`;
});
ts = ts.replace(/(\S+)\s+union\s+\{([^,{}]+)\}/g, 'new Set(Array.from($1).concat([$2]))');
// 5c: union -> new Set([...a, ...b]) (set union operator)
// "pending union {t.id}" -> "new Set([...pending, t.id])"
ts = ts.replace(/(\S+)\s+union\s+\{([^}]+)\}/g, 'new Set([...$1, $2])');
// 5d: \\ -> set difference via filter (handles both \\ and \\\\)
// "pending \\ {id}" or "pending \ {id}" -> "new Set(Array.from(pending).filter(x => x !== id))"
// For Maps: "pending \\ {id}" -> "new Map(Array.from(pending).filter(([k, v]) => k !== id))"
// Also handles __OBJ_LIT_ placeholders (protected object literals)
ts = ts.replace(/(\S+)\s+\\\\\s+__OBJ_LIT_(\d+)__/g, (_, coll, idx) => {
const base = coll.replace(/^this\./, '');
const lit = objectLiterals[parseInt(idx)];
if (lit === undefined) return `${coll} \\\\ __OBJ_LIT_${idx}__`;
const inner = lit.replace(/^\{\s*/, '').replace(/\s*\}$/, '');
if (mapVarNames.has(base)) {
return `new Map(Array.from(${coll}).filter(([k, v]) => k !== ${inner}))`;
}
return `new Set(Array.from(${coll}).filter(x => x !== ${inner}))`;
});
ts = ts.replace(/(\S+)\s+\\\s+__OBJ_LIT_(\d+)__/g, (_, coll, idx) => {
const base = coll.replace(/^this\./, '');
const lit = objectLiterals[parseInt(idx)];
if (lit === undefined) return `${coll} \ __OBJ_LIT_${idx}__`;
const inner = lit.replace(/^\{\s*/, '').replace(/\s*\}$/, '');
if (mapVarNames.has(base)) {
return `new Map(Array.from(${coll}).filter(([k, v]) => k !== ${inner}))`;
}
return `new Set(Array.from(${coll}).filter(x => x !== ${inner}))`;
});
ts = ts.replace(/(\S+)\s+\\\\\s+\{([^,{}]+)\}/g, (match, collection, key) => {
const base = collection.replace(/^this\./, '');
if (mapVarNames.has(base)) {
return `new Map(Array.from(${collection}).filter(([k, v]) => k !== ${key}))`;
}
return `new Set(Array.from(${collection}).filter(x => x !== ${key}))`;
});
ts = ts.replace(/(\S+)\s+\\\s+\{([^,{}]+)\}/g, (match, collection, key) => {
const base = collection.replace(/^this\./, '');
if (mapVarNames.has(base)) {
return `new Map(Array.from(${collection}).filter(([k, v]) => k !== ${key}))`;
}
return `new Set(Array.from(${collection}).filter(x => x !== ${key}))`;
});
// 5d1: not -> ! (negation operator)
// "not isOn" -> "!this.isOn" (will be prefixed by step 3)
ts = ts.replace(/\bnot\s+/g, '!');
// 5d2: and -> &&, or -> || (logical operators)
ts = ts.replace(/ and /g, ' && ').replace(/ or /g, ' || ');
// Handle SpeckDL .or() method — record-type short-circuit or
ts = ts.replace(/\.or\(([^)]+)\)/g, ' || $1');
// 5d3: Strip SpeckDL comments (// ...) before TS processing
ts = ts.replace(/\/\/.*$/gm, '');
// 5d4: [a..b] range syntax -> array literal [a, a+1, ..., b]
ts = ts.replace(/\[(\d+)\.\.(\d+)\]/g, (match, a, b) => {
const start = parseInt(a);
const end = parseInt(b);
const values = [];
for (let i = start; i <= end; i++) values.push(i);
return `[${values.join(', ')}]`;
});
// 5d5: Map literals {k: v} for Map type assignments
// If target is a Map var, convert {k: v} to new Map([[k, v]])
ts = ts.replace(/\{(\d+):\s*([^,{}]+)\}/g, (match, k, v) => {
// Only convert if it looks like a Map literal (single key:value pair with number key)
return `new Map([[${k}, ${v}]])`;
});
// 5d6: emptySet -> new Set()
ts = ts.replace(/\bemptySet\b/g, 'new Set()');
// 5d: \\ -> set difference via filter
// "pending \\ {id}" -> "new Set([...pending].filter(x => x !== id))"
ts = ts.replace(/(\S+)\s+\\\\\s+\{([^}]+)\}/g, 'new Set([...$1].filter(x => x !== $2))');
// 5e: Clean up trailing empty comparisons (===) — expression has no RHS after old() removal
// The template wraps as (!(expr)) — so trailing === becomes ===) in output.
// We need to handle the === at end-of-string within the expression.
// "accounts[t.from].debits_posted ===" -> "accounts[t.from].debits_posted === accounts[t.from].debits_posted"
ts = ts.replace(/(\S+)\s*===\s*$/g, '$1 === $1');
// 5f: append(X, Y) -> [...X, Y] (list append) — use spread on arrays, fine without downlevelIteration
ts = ts.replace(/\bappend\(([^,{}]+),\s*([^)]+)\)/g, '[...$1, $2]');
// 5g: Map access via bracket -> .get() (e.g. "transfers[id]" -> "transfers.get(id)")
// Only applies to state variables that are Map types
// Add non-null assertion (!) because Map.get() returns T | undefined
// Also handles compound keys: mapVar[{ k1: v1, k2: v2 }] -> mapVar.get(v1)!.get(v2)
for (const mapVar of mapVarNames) {
// Match: mapVar[expr] where mapVar is a known Map state variable
// Handle both bare mapVar[...] and this.mapVar[...]
// The inner expr needs rewriting for this-prefixing
const barePattern = new RegExp('\\b(' + mapVar + ')\\[([^\\]]+)\\]', 'g');
ts = ts.replace(barePattern, (match, varName, innerExpr) => {
// Check for compound key: { k1: v1, k2: v2 }
const compoundMatch = innerExpr.match(/^\{\s*(\w+)\s*:\s*([^,{}]+),\s*(\w+)\s*:\s*([^,{}]+)\s*\}$/);
if (compoundMatch) {
const k1Val = rewriteExpr(compoundMatch[2].trim(), stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues);
const k2Val = rewriteExpr(compoundMatch[4].trim(), stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues);
return `${varName}.get(${k1Val})!.get(${k2Val})!`;
}
const rewrittenInner = rewriteExpr(innerExpr, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues);
return `${varName}.get(${rewrittenInner})!`;
});
const thisPattern = new RegExp('\\b(this\\.' + mapVar + ')\\[([^\\]]+)\\]', 'g');
ts = ts.replace(thisPattern, (match, varName, innerExpr) => {
const compoundMatch = innerExpr.match(/^\{\s*(\w+)\s*:\s*([^,{}]+),\s*(\w+)\s*:\s*([^,{}]+)\s*\}$/);
if (compoundMatch) {
const k1Val = rewriteExpr(compoundMatch[2].trim(), stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues);
const k2Val = rewriteExpr(compoundMatch[4].trim(), stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues);
return `${varName}.get(${k1Val})!.get(${k2Val})!`;
}
const rewrittenInner = rewriteExpr(innerExpr, stateVarNames, mapVarNames, localNames, stateEnumName, knownStateValues);
return `${varName}.get(${rewrittenInner})!`;
});
}
// 5h: .size() -> .size (Set/Map size property, not a method in JS)
ts = ts.replace(/\.size\(\)/g, '.size');
// 5i: SpeckDL conditional expression: if cond: a else b -> cond ? a : b
// Simpler approach: match if <cond>: <then> else <else>
// where cond doesn't contain { } : and then/else don't contain ; or }
let prev2;
do {
prev2 = ts;
// Match if followed by non-special chars, colon, then clause, else, else clause
ts = ts.replace(/\bif\s+([^{}:]+):\s*([^{};]+)\s+else\s+([^{};\n]+)/g, '($1 ? $2 : $3)');
} while (ts !== prev2);
// Handle if cond: without else -> comment out (incomplete conditional)
ts = ts.replace(/\bif\s+([^{}:;]+):\s*$/gm, '/* if $1 */');
// Restore object literals (protected in step 3)
// For Map-typed assignments, convert {k: v} -> new Map([[k, v]])
ts = ts.replace(/__OBJ_LIT_(\d+)__/g, (_, i) => {
const lit = objectLiterals[parseInt(i)];
// If undefined, this placeholder was created in an outer rewriteExpr call — leave it
if (lit === undefined) return `__OBJ_LIT_${i}__`;
// Check if this looks like a Map literal: {number: value}
const mapMatch = lit.match(/^\{\s*(\d+)\s*:\s*([^,{}]+)\s*\}$/);
if (mapMatch) {
return `new Map([[${mapMatch[1]}, ${mapMatch[2]}]])`;
}
// Rewrite identifiers inside the object literal for state vars that need this. prefix
// (e.g., { created_at: currentTime } -> { created_at: this.currentTime })
const rewritten = lit.replace(/\b(currentTime|now)\b/g, (m: string) => {
if (m === 'currentTime') return 'this.currentTime';
return m;
});
return rewritten;
});
return ts.trim();
}
@ -806,7 +417,7 @@ function cleanName(s: string): string {
function replaceStateWithEnum(expr: string, enumName: string, knownStateValues: string[]): string {
const allStateValues = new Set<string>(knownStateValues);
['Open','Grooming','Groomed','Assigned','InProgress','InReview','Merged','Blocked','Closed','Idle','Parsing','Parsed','ProvenanceGenerated','BOMsGenerated','TypeScriptGenerated','Done'].forEach(s => allStateValues.add(s));
['Open','Grooming','Groomed','Assigned','InProgress','InReview','Merged','Blocked','Closed','Idle','Parsing','Parsed','ProvenanceGenerated','BOMsGenerated','TypeScriptGenerated','WASMGenerated','Done'].forEach(s => allStateValues.add(s));
if (allStateValues.size === 0) return expr;
const stateValueRegex = new RegExp('\\b(' + Array.from(allStateValues).join('|') + ')\\b', 'g');
return expr.replace(stateValueRegex, `${enumName}.$1`);
@ -820,23 +431,14 @@ function tsType(t: any, speckName: string = '', enumMap?: Map<string, string[]>)
if (!t) return 'unknown';
if (t.type === 'primitive') {
const map: Record<string, string> = {
Nat: 'number', Int: 'number', Integer: 'number', Real: 'number',
Bool: 'boolean', Boolean: 'boolean', String: 'string', Bytes: 'Uint8Array'
Nat: 'number', Int: 'number', Real: 'number',
Bool: 'boolean', String: 'string', Bytes: 'Uint8Array'
};
return map[t.name] || t.name.toLowerCase();
}
if (t.type === 'list') return `${tsType(t.elementType, speckName, enumMap)}[]`;
if (t.type === 'set') return `Set<${tsType(t.elementType, speckName, enumMap)}>`;
if (t.type === 'map') {
// Detect compound key: Map(Map(K1, K2), V) -> Map<K1, Map<K2, V>>
if (t.keyType?.type === 'map') {
const k1 = tsType(t.keyType.keyType, speckName, enumMap);
const k2 = tsType(t.keyType.valueType, speckName, enumMap);
const v = tsType(t.valueType, speckName, enumMap);
return `Map<${k1}, Map<${k2}, ${v}>>`;
}
return `Map<${tsType(t.keyType, speckName, enumMap)}, ${tsType(t.valueType, speckName, enumMap)}>`;
}
if (t.type === 'map') return `Map<${tsType(t.keyType, speckName, enumMap)}, ${tsType(t.valueType, speckName, enumMap)}>`;
if (t.type === 'record') {
const fields = t.fields.map((f: any) => {
const name = cleanName(f.name);
@ -846,28 +448,13 @@ function tsType(t: any, speckName: string = '', enumMap?: Map<string, string[]>)
}
if (t.type === 'ident') {
const rawName = cleanName(t.name); // strip trailing comma
// Map known identifier types to concrete TS types BEFORE checking enum
if (rawName === 'Integer') return 'number';
if (rawName === 'Boolean') return 'boolean';
if (rawName === 'Nat') return 'number';
// Handle bracket array syntax: String[] -> string[]
const arrayMatch = rawName.match(/^(\w+)\[\]$/);
if (arrayMatch) {
const base = arrayMatch[1];
const baseMap: Record<string, string> = {
String: 'string', Nat: 'number', Int: 'number', Integer: 'number',
Real: 'number', Bool: 'boolean', Boolean: 'boolean'
};
return `${baseMap[base] || base}[]`;
}
// Handle Option(T) -> T | null
const optionMatch = rawName.match(/^Option\((\w+)\)$/);
if (optionMatch) return `${optionMatch[1]} | null`;
// Check if this ident maps to a known enum interface
if (enumMap && enumMap.has(rawName)) {
return `${speckName}${rawName}`;
}
// Map known identifier types to concrete TS types
if (rawName === 'IssueState') return 'IssueWorkflowState';
if (rawName === 'Nat') return 'number';
// Opaque interfaces become 'any' for now
if (rawName === 'AST' || rawName === 'ArtifactMap') return 'any';
return rawName;

472
compiler/src/generators/wasm.ts Normal file
View file

@ -0,0 +1,472 @@
import { AST, SpeckNode, StateNode, InitNode, ActionNode, EventNode } from '../parser.js';
import fs from 'fs';
import path from 'path';
/**
* Generate WebAssembly Text (WAT) from a SpeckDL state machine spec.
*
* Emits a .wat file per speck with:
* - Linear memory for scalar state (Nat/Int/Real/Bool)
* - Imported host functions for Map<K,V> and Set<T>
* - Exported `init()` — calls host to init Maps/Sets + sets scalars
* - Exported action functions (one per action)
* - Exported `getState()` — serializes scalars into memory
* - Exported `checkConstraints()` — runtime guard stubs
*
* Compile to .wasm with: wat2wasm <file>.wat -o <file>.wasm
*
* Host runtime (JS): see `speckl/runtime/wasm-host.js`
*/
// ---------------------------------------------------------------------------
// Host imports required by generated modules
// ---------------------------------------------------------------------------
const HOST_IMPORTS = [
'(import "speckl" "map_create" (func $map_create (result i32)))',
'(import "speckl" "map_set" (func $map_set (param i32 i32 i32)))',
'(import "speckl" "map_get" (func $map_get (param i32 i32) (result i32)))',
'(import "speckl" "map_has" (func $map_has (param i32 i32) (result i32)))',
'(import "speckl" "map_remove" (func $map_remove (param i32 i32)))',
'(import "speckl" "set_create" (func $set_create (result i32)))',
'(import "speckl" "set_add" (func $set_add (param i32 i32)))',
'(import "speckl" "set_has" (func $set_has (param i32 i32) (result i32)))',
'(import "speckl" "set_size" (func $set_size (param i32) (result i32)))',
'(import "speckl" "alloc" (func $alloc (param i32) (result i32)))',
'(import "speckl" "write_i32" (func $write_i32 (param i32 i32)))',
'(import "speckl" "write_f64" (func $write_f64 (param i32 f64)))',
];
export function generateWASM(ast: AST, outputDir: string): void {
for (const speck of ast.specks) {
const hasStateMachine = speck.members.some(m =>
m.type === 'state' || m.type === 'init' || m.type === 'action'
);
if (!hasStateMachine) continue;
const wat = emitSpeckWAT(speck);
const filename = `${speck.name}.wat`;
const filepath = path.join(outputDir, filename);
fs.writeFileSync(filepath, wat);
console.log(`Generated WAT: ${filepath}`);
}
}
function emitSpeckWAT(speck: SpeckNode): string {
const stateNode = speck.members.find(m => m.type === 'state') as StateNode | undefined;
const initNode = speck.members.find(m => m.type === 'init') as InitNode | undefined;
const actions = speck.members.filter(m => m.type === 'action') as ActionNode[];
const events = speck.members.filter(m => m.type === 'event') as EventNode[];
const stateVars = stateNode?.variables ?? [];
// -------------------------------------------------------------------------
// 1. Memory layout for scalar state variables
// -------------------------------------------------------------------------
// Each scalar gets 8 bytes (i64/f64 slot). Map/Set handles are i32 refs
// stored in globals.
//
// Layout: [0..8) var0, [8..16) var1, ... then string pool for getState()
//
const scalarVars = stateVars.filter(v => isScalarType(v.typeExpr));
const refVars = stateVars.filter(v => !isScalarType(v.typeExpr));
const memSize = Math.max(1, Math.ceil((scalarVars.length * 8 + 1024) / 65536));
// -------------------------------------------------------------------------
// 2. Globals: one per state variable
// -------------------------------------------------------------------------
const globals = stateVars.map((v, i) => {
if (isScalarType(v.typeExpr)) {
const wasmType = wasmTypeOf(v.typeExpr);
const initVal = initValueFor(v.name, v.typeExpr, v.defaultInit, initNode);
return `(global $${v.name} (mut ${wasmType}) (${wasmType}.const ${initVal}))`;
} else {
// Map or Set — store as i32 handle (0 = uninitialized)
return `(global $${v.name} (mut i32) (i32.const 0))`;
}
}).join('\n');
// -------------------------------------------------------------------------
// 3. init() export
// -------------------------------------------------------------------------
const initBody = emitInitBody(stateVars, initNode);
// -------------------------------------------------------------------------
// 4. Action exports
// -------------------------------------------------------------------------
const actionFuncs = actions.map(a => emitActionWAT(a, stateVars)).join('\n\n');
// -------------------------------------------------------------------------
// 5. getState() export — write scalars to memory, return ptr+len
// -------------------------------------------------------------------------
const getStateBody = emitGetState(scalarVars);
// -------------------------------------------------------------------------
// 6. checkConstraints() stub
// -------------------------------------------------------------------------
const checkConstraints = `
;; checkConstraints — runtime constraint evaluation stub
(func $checkConstraints (export "checkConstraints") (result i32)
;; TODO: evaluate constraint expressions at runtime
(i32.const 1)
)`;
// -------------------------------------------------------------------------
// 7. Assemble module
// -------------------------------------------------------------------------
const module =
`;; Auto-generated by speckl-compile from ${speck.name}\n` +
`;; Compile: wat2wasm ${speck.name}.wat -o ${speck.name}.wasm\n\n` +
`(module\n` +
` ;; Host imports for Map/Set/alloc\n` +
` ${HOST_IMPORTS.join('\n ')}\n\n` +
` ;; Linear memory (exported so host can read state)\n` +
` (memory (export "memory") ${memSize})\n\n` +
` ;; State globals\n` +
` ${globals}\n\n` +
` ;; ---- init ----\n` +
` (func $init (export "init")\n` +
` ${initBody}\n` +
` )\n\n` +
` ;; ---- Actions ----\n` +
` ${actionFuncs}\n\n` +
` ;; ---- getState ----\n` +
` (func $getState (export "getState") (result i32 i32)\n` +
` ${getStateBody}\n` +
` )\n\n` +
` ${checkConstraints}\n` +
`)`;
return module;
}
// =============================================================================
// init() body generation
// =============================================================================
function emitInitBody(
stateVars: StateNode['variables'],
initNode: InitNode | undefined
): string {
const lines: string[] = [];
for (const v of stateVars) {
if (isScalarType(v.typeExpr)) {
const val = initValueFor(v.name, v.typeExpr, v.defaultInit, initNode);
const wasmType = wasmTypeOf(v.typeExpr);
lines.push(`(${wasmType}.const ${val})`);
lines.push(`(global.set $${v.name})`);
} else if (isMapType(v.typeExpr)) {
lines.push(`(call $map_create)`);
lines.push(`(global.set $${v.name})`);
} else if (isSetType(v.typeExpr)) {
lines.push(`(call $set_create)`);
lines.push(`(global.set $${v.name})`);
}
}
return lines.join('\n ');
}
// =============================================================================
// Action function generation
// =============================================================================
function emitActionWAT(action: ActionNode, stateVars: StateNode['variables']): string {
const scalarStateNames = new Set(
stateVars.filter(v => isScalarType(v.typeExpr)).map(v => v.name)
);
const mapStateNames = new Set(
stateVars.filter(v => isMapType(v.typeExpr)).map(v => v.name)
);
const setStateNames = new Set(
stateVars.filter(v => isSetType(v.typeExpr)).map(v => v.name)
);
// Params: SpeckDL Nat/Int → i32, Real → f64, Bool → i32
const params = action.params.map(p => {
const t = wasmTypeOf(p.type);
return `(param $${p.name} ${t})`;
}).join(' ');
const localLines: string[] = [];
const bodyLines: string[] = [];
for (const stmt of action.statements) {
switch (stmt.type) {
case 'require': {
// Emit guard: if !condition → return 0 (failure indicator)
// For now, require guards that reference only scalar state variables
// and simple comparisons are emitted. Complex ones trap/return 0.
const guard = emitGuardExpr(stmt.expr, scalarStateNames, mapStateNames, setStateNames);
bodyLines.push(`;; require: ${stmt.expr}`);
bodyLines.push(guard);
break;
}
case 'assign': {
bodyLines.push(`;; ${stmt.target} := ${stmt.expr}`);
bodyLines.push(emitAssignment(stmt.target, stmt.expr, scalarStateNames, mapStateNames));
break;
}
case 'let': {
// let name := expr — allocate local, compute expr, store
const wasmT = inferExprType(stmt.expr, scalarStateNames);
localLines.push(`(local $${stmt.name} ${wasmT})`);
bodyLines.push(emitExpr(stmt.expr, scalarStateNames));
bodyLines.push(`(local.set $${stmt.name})`);
break;
}
case 'emit': {
// emit → decision log push (host call stub for now)
bodyLines.push(`;; emit ${stmt.event} { ... }`);
// TODO: structured emit via host
break;
}
case 'return': {
bodyLines.push(emitExpr(stmt.expr, scalarStateNames));
bodyLines.push(`(return)`);
break;
}
}
}
// Default return: i32.const 1 for success if action returns void
const hasReturn = action.statements.some(s => s.type === 'return');
if (!hasReturn) {
bodyLines.push(`(i32.const 1)`);
}
const locals = localLines.length ? `\n ${localLines.join('\n ')}` : '';
return ` ;; action: ${action.name}\n` +
` (func $${action.name} (export "${action.name}") ${params} (result i32)\n` +
` ${locals}\n` +
` ${bodyLines.join('\n ')}\n` +
` )`;
}
// =============================================================================
// Expression / guard emission (simplified — covers common scalar patterns)
// =============================================================================
function emitGuardExpr(
expr: string,
scalars: Set<string>,
maps: Set<string>,
sets: Set<string>
): string {
// Very simple pattern: "x >= 0", "x == true", "x != y", etc.
const m = expr.match(/^\s*(\w+)\s*(>=|<=|>|<|==|!=)\s*(.+?)\s*$/);
if (!m) {
// Can't compile guard — just comment, no stack effect
return `;; Complex guard not compiled: ${expr}`;
}
const [, left, op, rightRaw] = m;
const right = rightRaw.trim();
if (!scalars.has(left)) {
return `;; Guard on non-scalar not compiled: ${expr}`;
}
const lhs = `(global.get $${left})`;
// Parse right side
let rhs: string;
const rightNum = parseFloat(right);
if (!isNaN(rightNum)) {
rhs = `(i32.const ${rightNum})`;
} else if (right === 'true') {
rhs = `(i32.const 1)`;
} else if (right === 'false') {
rhs = `(i32.const 0)`;
} else if (scalars.has(right)) {
rhs = `(global.get $${right})`;
} else {
return `;; Complex right-hand side not compiled: ${expr}`;
}
const comparison = (() => {
switch (op) {
case '>=': return `(i32.ge_s ${lhs} ${rhs})`;
case '<=': return `(i32.le_s ${lhs} ${rhs})`;
case '>': return `(i32.gt_s ${lhs} ${rhs})`;
case '<': return `(i32.lt_s ${lhs} ${rhs})`;
case '==': return `(i32.eq ${lhs} ${rhs})`;
case '!=': return `(i32.ne ${lhs} ${rhs})`;
default: return `(i32.const 1)`;
}
})();
// If guard fails (comparison is FALSE), return 0
return `${comparison}\n (i32.eqz)\n (if (then (return (i32.const 0))))`;
}
function emitExpr(expr: string, scalars: Set<string>): string {
// Handle: nextId + 1, nextId, numeric literals, etc.
expr = expr.trim();
// Simple numeric literal
const num = parseFloat(expr);
if (!isNaN(num) && !expr.match(/\w/)) {
return `(i32.const ${num})`;
}
// Simple variable reference
if (/^\w+$/.test(expr) && scalars.has(expr)) {
return `(global.get $${expr})`;
}
// Binary op: left + right, left - right, left * right
const binOp = expr.match(/^(\w+)\s*([+\-*/])\s*(.+)$/);
if (binOp) {
const [, left, op, right] = binOp;
const lhs = scalars.has(left) ? `(global.get $${left})` : `(i32.const ${parseFloat(left)})`;
const rhsNum = parseFloat(right.trim());
const rhs = isNaN(rhsNum)
? (scalars.has(right.trim()) ? `(global.get $${right.trim()})` : `(i32.const 0) ;; uncompiled`)
: `(i32.const ${rhsNum})`;
switch (op) {
case '+': return `(i32.add ${lhs} ${rhs})`;
case '-': return `(i32.sub ${lhs} ${rhs})`;
case '*': return `(i32.mul ${lhs} ${rhs})`;
default: return `(i32.const 0) ;; uncompiled op ${op}`;
}
}
return `(i32.const 0) ;; uncompiled expr: ${expr}`;
}
function inferExprType(expr: string, scalars: Set<string>): string {
// Default to i32; we could infer from variable types but that's future work
return 'i32';
}
function emitAssignment(
target: string,
expr: string,
scalars: Set<string>,
maps: Set<string>
): string {
// Scalar assignment: this.x = expr
if (scalars.has(target)) {
return `${emitExpr(expr, scalars)}\n (global.set $${target})`;
}
// Map assignment: accounts[id] := expr — host call
const mapAssign = target.match(/^(\w+)\[(.+)\]$/);
if (mapAssign && maps.has(mapAssign[1])) {
const mapName = mapAssign[1];
const keyExpr = mapAssign[2];
// For now: map_set(map_handle, key_handle, value_handle)
// This needs boxing; simplified version just marks as stub
return `;; Map assignment not yet compiled: ${target} := ${expr}`;
}
// Record field assignment: accounts[id].field := expr
const recordAssign = target.match(/^(\w+)\[(.+)\]\.(\w+)$/);
if (recordAssign && maps.has(recordAssign[1])) {
return `;; Record field assignment not yet compiled: ${target} := ${expr}`;
}
return `;; Unrecognized assignment target: ${target}`;
}
// =============================================================================
// getState() — serialize scalar state to memory
// =============================================================================
function emitGetState(scalarVars: StateNode['variables']): string {
const lines: string[] = [];
let offset = 0;
for (const v of scalarVars) {
const t = wasmTypeOf(v.typeExpr);
if (t === 'i32') {
lines.push(`(i32.const ${offset})`);
lines.push(`(global.get $${v.name})`);
lines.push(`(i32.store)`);
offset += 4;
} else if (t === 'f64') {
lines.push(`(i32.const ${offset})`);
lines.push(`(global.get $${v.name})`);
lines.push(`(f64.store)`);
offset += 8;
} else {
// i64
lines.push(`(i32.const ${offset})`);
lines.push(`(global.get $${v.name})`);
lines.push(`(i64.store)`);
offset += 8;
}
}
// Return (ptr, len)
lines.push(`(i32.const 0)`); // ptr = 0
lines.push(`(i32.const ${offset})`); // len = total bytes written
return lines.join('\n ');
}
// =============================================================================
// Type helpers
// =============================================================================
function isScalarType(t: any): boolean {
if (t.type === 'primitive') {
return ['Nat', 'Int', 'Real', 'Bool'].includes(t.name);
}
if (t.type === 'ident') return true; // opaque ident → i32 handle
return false;
}
function isMapType(t: any): boolean {
return t.type === 'map';
}
function isSetType(t: any): boolean {
return t.type === 'set';
}
function wasmTypeOf(t: any): string {
if (t.type === 'primitive') {
if (t.name === 'Nat') return 'i32';
if (t.name === 'Int') return 'i32';
if (t.name === 'Real') return 'f64';
if (t.name === 'Bool') return 'i32';
if (t.name === 'String') return 'i32'; // ptr
if (t.name === 'Bytes') return 'i32'; // ptr
return 'i32';
}
if (t.type === 'ident') return 'i32';
if (t.type === 'map') return 'i32'; // handle
if (t.type === 'set') return 'i32'; // handle
return 'i32';
}
function initValueFor(
varName: string,
typeExpr: any,
defaultInit: string | undefined,
initNode: InitNode | undefined
): string {
// Use initNode assignment if present
const initAssign = initNode?.assignments.find(a => a.name === varName);
if (initAssign) {
const expr = initAssign.expr.trim();
if (expr === 'true') return '1';
if (expr === 'false') return '0';
if (expr === 'Map.empty()') return '0'; // will be replaced by host call
if (expr === 'Set.empty()') return '0';
const num = parseFloat(expr);
if (!isNaN(num)) return String(num);
return '0';
}
// Fallback to defaultInit
if (defaultInit) {
if (defaultInit === 'true') return '1';
if (defaultInit === 'false') return '0';
const num = parseFloat(defaultInit);
if (!isNaN(num)) return String(num);
}
// Default zero
if (typeExpr.type === 'primitive' && typeExpr.name === 'Real') return '0.0';
return '0';
}

1182
compiler/src/generators/z3.ts
View file

File diff suppressed because it is too large Load diff

82
compiler/src/index.ts
View file

@ -7,16 +7,13 @@ import { generateProvenanceGraph } from './generators/provenance.js';
import { generateCycloneDX } from './generators/cyclonedx.js';
import { generateSPDX } from './generators/spdx.js';
import { generateTypeScriptStateMachine } from './generators/typescript-state-machine.js';
import { generateZ3, Z3Options, parseInvariantsFromSource, parseNextFromSource } from './generators/z3.js';
import { generateRust } from './generators/rust.js';
import { generateWASM } from './generators/wasm.js';
import fs from 'fs';
import path from 'path';
interface CompileOptions {
outputDir: string;
bomFormat: 'cdx' | 'spdx' | 'both';
target: 'typescript' | 'z3' | 'all';
verifyDepth: number;
}
async function main() {
@ -46,19 +43,6 @@ async function main() {
type: 'string',
default: './out',
})
.option('target', {
alias: 't',
describe: 'Compilation target',
type: 'string',
choices: ['typescript', 'z3', 'rust', 'all'],
default: 'typescript' as const,
})
.option('verify-depth', {
alias: 'd',
describe: 'Bounded model checking depth for Z3 verification',
type: 'number',
default: 10,
})
.option('bom-format', {
alias: 'b',
describe: 'BOM output format',
@ -74,8 +58,6 @@ async function main() {
const options = {
outputDir: argv.outputDir as string,
bomFormat: argv.bomFormat as 'cdx' | 'spdx' | 'both',
target: argv.target as 'typescript' | 'z3' | 'rust' | 'all',
verifyDepth: argv.verifyDepth as number,
};
console.log(`Compiling ${file}...`);
@ -92,51 +74,27 @@ async function main() {
fs.mkdirSync(options.outputDir, { recursive: true });
}
// Read raw source for invariant/next parsing that the AST parser doesn't handle yet
const rawSource = fs.readFileSync(file, 'utf-8');
// Always generate provenance (it's the core output)
console.log('\nGenerating provenance graph (PROV-O JSON-LD)...');
generateProvenanceGraph(ast, options.outputDir);
// Generate TypeScript state machine
console.log('\nGenerating TypeScript state machine...');
generateTypeScriptStateMachine(ast, options.outputDir);
// Generate Rust state machine if target is rust or all
if (options.target === 'rust' || options.target === 'all') {
console.log('\nGenerating Rust state machine...');
generateRust(ast, options.outputDir);
// Generate WASM state machine
console.log('\nGenerating WebAssembly (WAT)...');
generateWASM(ast, options.outputDir);
// Generate BOM based on format selection
if (options.bomFormat === 'cdx' || options.bomFormat === 'both') {
console.log('\nGenerating CycloneDX BOM...');
generateCycloneDX(ast, options.outputDir);
}
// Always generate provenance (it's the core output) if not z3-only or rust-only
if (options.target !== 'z3' && options.target !== 'rust') {
console.log('\nGenerating provenance graph (PROV-O JSON-LD)...');
generateProvenanceGraph(ast, options.outputDir);
}
// Generate Z3 SMT-LIB2 if target is z3 or all
if (options.target === 'z3' || options.target === 'all') {
console.log('\nGenerating Z3 SMT-LIB2...');
// Parse invariants from raw source for Z3 backend
for (const speck of ast.specks) {
const invariants = parseInvariantsFromSource(rawSource, speck.name);
const nextNode = parseNextFromSource(rawSource, speck.name);
// Attach parsed data to the AST for the generator
(speck as any)._invariants = invariants;
(speck as any)._next = nextNode;
}
const z3Options: Z3Options = { verifyDepth: options.verifyDepth };
generateZ3(ast, options.outputDir, z3Options);
}
// Generate TypeScript state machine if not z3-only or rust-only
if (options.target !== 'z3' && options.target !== 'rust') {
console.log('\nGenerating TypeScript state machine...');
generateTypeScriptStateMachine(ast, options.outputDir);
// Generate BOM based on format selection
if (options.bomFormat === 'cdx' || options.bomFormat === 'both') {
console.log('\nGenerating CycloneDX BOM...');
generateCycloneDX(ast, options.outputDir);
}
if (options.bomFormat === 'spdx' || options.bomFormat === 'both') {
console.log('\nGenerating SPDX BOM...');
generateSPDX(ast, options.outputDir);
}
if (options.bomFormat === 'spdx' || options.bomFormat === 'both') {
console.log('\nGenerating SPDX BOM...');
generateSPDX(ast, options.outputDir);
}
console.log(`\nCompilation complete. Output: ${options.outputDir}/`);

522
compiler/src/parser.ts
View file

@ -28,15 +28,12 @@ export interface OutputNode {
export interface ConstraintNode {
type: 'constraint';
name?: string;
expr: string;
}
export interface VerifyNode {
type: 'verify';
name: string;
temporalExpr: string;
depth?: number;
}
export interface InterfaceNode {
@ -138,8 +135,7 @@ export type ActionStatement =
| { type: 'precondition'; expr: string }
| { type: 'postcondition'; expr: string }
| { type: 'emit'; event: string; fields: { name: string; value: string }[] }
| { type: 'return'; expr: string }
| { type: 'ifblock'; raw: string };
| { type: 'return'; expr: string };
export type MemberNode =
| ImportNode
@ -249,9 +245,9 @@ function parseSpeck(lines: string[], startIndex: number): SpeckNode | null {
const openCount = (rawLine.match(/{/g) || []).length;
const closeCount = (rawLine.match(/}/g) || []).length;
// Check for block-opening members (those that open braces on the same line)
// Check for block-opening members
const blockStarters = ['state:', 'init:', 'action ', 'event ', 'provenance ', 'bom ', 'interface ',
'state {', 'init {', 'verify ', 'constraint '];
'state {', 'init {'];
const isBlockStarter = blockStarters.some(s => line.startsWith(s));
if (isBlockStarter && openCount > 0) {
@ -260,39 +256,9 @@ function parseSpeck(lines: string[], startIndex: number): SpeckNode | null {
if (member) {
members.push(member);
}
// Skip past this block if it spans multiple lines
if (closeCount < openCount) {
const endIdx = findBlockEnd(lines, i + 1, braceCount + openCount);
i = endIdx + 1;
} else {
i++;
}
continue;
}
// Handle constraint/verify blocks that may span multiple lines without braces
// (e.g. constraint Name: forall r in [0..4]:)
if (line.startsWith('constraint ') || line.startsWith('verify ')) {
const member = parseMultiLineConstraintVerify(lines, i);
if (member) {
members.push(member);
// Skip past the consumed continuation lines
while (i + 1 < lines.length) {
const nextLine = lines[i + 1].trim();
if (!nextLine || nextLine.startsWith('//') || nextLine.startsWith('/*') ||
nextLine.startsWith('constraint ') || nextLine.startsWith('verify ') ||
nextLine.startsWith('constraint:') || nextLine.startsWith('verify:') ||
nextLine.startsWith('state') || nextLine.startsWith('init') ||
nextLine.startsWith('action ') || nextLine.startsWith('event ') ||
nextLine.startsWith('interface ') || nextLine.startsWith('provenance ') ||
nextLine.startsWith('bom ') || nextLine.startsWith('}') ||
nextLine.startsWith('next:') || nextLine.startsWith('invariant ')) {
break;
}
i++;
}
}
i++;
// Skip past this block
const endIdx = findBlockEnd(lines, i + 1, braceCount + openCount);
i = endIdx + 1;
continue;
}
@ -332,20 +298,6 @@ function findBlockEnd(lines: string[], startIndex: number, startBraceCount: numb
return i;
}
// Check if an expression has balanced braces, parens, and brackets
function isExpressionComplete(expr: string): boolean {
let braces = 0, parens = 0, brackets = 0;
for (const char of expr) {
if (char === '{') braces++;
else if (char === '}') braces--;
else if (char === '(') parens++;
else if (char === ')') parens--;
else if (char === '[') brackets++;
else if (char === ']') brackets--;
}
return braces === 0 && parens === 0 && brackets === 0;
}
function parseMemberBlock(lines: string[], startIndex: number): MemberNode | null {
const firstLine = lines[startIndex].trim();
const startBraceCount = (firstLine.match(/{/g) || []).length;
@ -377,12 +329,6 @@ function parseMemberBlock(lines: string[], startIndex: number): MemberNode | nul
if (firstLine.startsWith('bom ')) {
return { type: 'bom' };
}
if (firstLine.startsWith('verify ')) {
return parseVerifyBlockMultiline(lines, startIndex, startBraceCount);
}
if (firstLine.startsWith('constraint ')) {
return parseConstraintBlockMultiline(lines, startIndex, startBraceCount);
}
return null;
}
@ -400,28 +346,14 @@ function parseMember(line: string): MemberNode | null {
return { type: 'output', typeExpr };
}
// constraint Name: expr (single-line named constraint)
if (line.startsWith('constraint ')) {
const match = line.match(/^constraint\s+(\w+):\s*(.+)$/);
if (match) {
return { type: 'constraint', name: match[1], expr: match[2].trim() };
}
return null; // multi-line constraint block (handled by parseMemberBlock)
}
// constraint: expr (unnamed, single-line)
// constraint:
if (line.startsWith('constraint:')) {
return { type: 'constraint', expr: line.substring(11).trim() };
}
// verify Always(Name) { depth N } or verify Always(Name) (single-line forms)
if (line.startsWith('verify ')) {
return parseVerifySingleLine(line);
}
// verify: expr (old unnamed single-line)
// verify:
if (line.startsWith('verify:')) {
return { type: 'verify', name: 'unnamed', temporalExpr: line.substring(7).trim() };
return { type: 'verify', temporalExpr: line.substring(7).trim() };
}
// interface:
@ -534,26 +466,18 @@ function parseStateBlock(inner: string): StateNode {
for (const vStr of varStrs) {
const withDefault = vStr.match(/^(\w+)\s*:\s*(.+?)\s*:=\s*(.+)$/);
if (withDefault) {
// Strip trailing comments from type expression
let typeStr = withDefault[2].trim();
const commentIdx = typeStr.indexOf('//');
if (commentIdx >= 0) typeStr = typeStr.substring(0, commentIdx).trim();
variables.push({
name: withDefault[1],
typeExpr: parseTypeExpr(typeStr),
typeExpr: parseTypeExpr(withDefault[2].trim()),
defaultInit: withDefault[3].trim(),
});
continue;
}
const withoutDefault = vStr.match(/^(\w+)\s*:\s*(.+)$/);
if (withoutDefault) {
// Strip trailing comments from type expression
let typeStr = withoutDefault[2].trim();
const commentIdx = typeStr.indexOf('//');
if (commentIdx >= 0) typeStr = typeStr.substring(0, commentIdx).trim();
variables.push({
name: withoutDefault[1],
typeExpr: parseTypeExpr(typeStr),
typeExpr: parseTypeExpr(withoutDefault[2].trim()),
});
}
}
@ -639,16 +563,13 @@ function parseTypeExpr(expr: string): TypeExpr {
// Map<K, V>
if (expr.startsWith('Map<') && expr.endsWith('>')) {
const inner = expr.substring(4, expr.length - 1);
// Find the first-level comma (not inside nested generics or record literals)
// Find the first-level comma (not inside nested generics)
let depth = 0;
let braceDepth = 0;
let commaIdx = -1;
for (let i = 0; i < inner.length; i++) {
if (inner[i] === '<' || inner[i] === '(') depth++;
else if (inner[i] === '>' || inner[i] === ')') depth--;
else if (inner[i] === '{') braceDepth++;
else if (inner[i] === '}') braceDepth--;
else if (inner[i] === ',' && depth === 0 && braceDepth === 0) {
if (inner[i] === '<') depth++;
else if (inner[i] === '>') depth--;
else if (inner[i] === ',' && depth === 0) {
commaIdx = i;
break;
}
@ -678,14 +599,11 @@ function parseTypeExpr(expr: string): TypeExpr {
if (expr.startsWith('Map(') && expr.endsWith(')')) {
const inner = expr.substring(4, expr.length - 1);
let depth = 0;
let braceDepth = 0;
let commaIdx = -1;
for (let i = 0; i < inner.length; i++) {
if (inner[i] === '(' || inner[i] === '<') depth++;
else if (inner[i] === ')' || inner[i] === '>') depth--;
else if (inner[i] === '{') braceDepth++;
else if (inner[i] === '}') braceDepth--;
else if (inner[i] === ',' && depth === 0 && braceDepth === 0) {
else if (inner[i] === ',' && depth === 0) {
commaIdx = i;
break;
}
@ -855,41 +773,14 @@ function parseSource(line: string): SourceNode {
function parseStateBlockMultiline(lines: string[], startIndex: number, startBraceCount: number): StateNode {
const endIndex = findBlockEnd(lines, startIndex + 1, startBraceCount);
const innerLines = lines.slice(startIndex + 1, endIndex).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
// State blocks separate fields with commas or newlines/semicolons.
// Strip trailing commas (they are field separators, not continuations)
// then join with semicolons between complete variable declarations.
const cleanedLines = innerLines.map(l => l.replace(/,+\s*$/, ''));
const inner = cleanedLines.reduce((acc, line, i) => {
if (i === 0) return line;
const prevLine = cleanedLines[i - 1];
// Check if this line starts a new variable declaration (name: type)
const isNewVar = /^\w+\s*:/.test(line);
// Check if prev line ends with a continuation character (brace or paren, NOT comma)
const prevIsContinuation = /[{(]\s*$/.test(prevLine);
if (isNewVar && !prevIsContinuation) {
return acc + '; ' + line;
}
return acc + ' ' + line;
}, '');
const inner = innerLines.join('; ');
return parseStateBlock(inner);
}
function parseInitBlockMultiline(lines: string[], startIndex: number, startBraceCount: number): InitNode {
const endIndex = findBlockEnd(lines, startIndex + 1, startBraceCount);
const innerLines = lines.slice(startIndex + 1, endIndex).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
// Init blocks separate fields with commas or newlines/semicolons.
// Strip trailing commas and join with semicolons.
const cleanedLines = innerLines.map(l => l.replace(/,+\s*$/, ''));
const inner = cleanedLines.reduce((acc, line, i) => {
if (i === 0) return line;
const prevLine = cleanedLines[i - 1];
const isNewVar = /^\w+\s*:/.test(line);
const prevIsContinuation = /[{(]\s*$/.test(prevLine);
if (isNewVar && !prevIsContinuation) {
return acc + '; ' + line;
}
return acc + ' ' + line;
}, '');
const inner = innerLines.join('; ');
return parseInitBlock(inner);
}
@ -899,39 +790,18 @@ function parseActionBlockMultiline(lines: string[], startIndex: number, startBra
if (!header) return null;
const endIndex = findBlockEnd(lines, startIndex + 1, startBraceCount);
// Keep blank lines as null to separate statements, filter only comments
const bodyLinesRaw = lines.slice(startIndex + 1, endIndex).map(l => {
const trimmed = l.trim();
if (!trimmed || trimmed.startsWith('//') || trimmed.startsWith('/*')) return null;
return trimmed;
});
// Collapse consecutive nulls and strip trailing nulls
const bodyLines: (string | null)[] = [];
for (const line of bodyLinesRaw) {
if (line !== null || (bodyLines.length > 0 && bodyLines[bodyLines.length - 1] !== null)) {
bodyLines.push(line);
}
}
// Remove trailing nulls
while (bodyLines.length > 0 && bodyLines[bodyLines.length - 1] === null) {
bodyLines.pop();
}
const bodyLines = lines.slice(startIndex + 1, endIndex).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
const statements: ActionStatement[] = [];
for (let i = 0; i < bodyLines.length; i++) {
const stmtOrNull = bodyLines[i];
// Skip blank line separators
if (stmtOrNull === null) continue;
const stmt = stmtOrNull;
const stmt = bodyLines[i];
// precondition: expr
if (stmt.startsWith('precondition:')) {
let expr = stmt.substring(13).trim();
// support 'or' continuation lines
while (i + 1 < bodyLines.length) {
const nextLine = bodyLines[i + 1];
if (nextLine === null || !nextLine.startsWith('or ')) break;
while (i + 1 < bodyLines.length && bodyLines[i + 1].startsWith('or ')) {
i++;
expr += ' || ' + nextLine.substring(3).trim();
expr += ' || ' + bodyLines[i].substring(3).trim();
}
statements.push({ type: 'precondition', expr });
continue;
@ -940,33 +810,16 @@ function parseActionBlockMultiline(lines: string[], startIndex: number, startBra
if (stmt.startsWith('postcondition:')) {
let expr = stmt.substring(14).trim();
// support 'or' continuation lines
while (i + 1 < bodyLines.length) {
const nextLine = bodyLines[i + 1];
if (nextLine === null || !nextLine.startsWith('or ')) break;
while (i + 1 < bodyLines.length && bodyLines[i + 1].startsWith('or ')) {
i++;
expr += ' || ' + nextLine.substring(3).trim();
expr += ' || ' + bodyLines[i].substring(3).trim();
}
statements.push({ type: 'postcondition', expr });
continue;
}
// require expr
if (stmt.startsWith('require ')) {
let expr = stmt.substring(8).trim();
// support multi-line continuation (forall, or, etc.)
while (i + 1 < bodyLines.length) {
const nextLine = bodyLines[i + 1];
if (nextLine === null) break;
// Stop if next line starts a new statement keyword
if (nextLine.startsWith('precondition:') || nextLine.startsWith('postcondition:') ||
nextLine.startsWith('require ') || nextLine.startsWith('return ') ||
nextLine.startsWith('emit ') || nextLine.startsWith('let ') ||
nextLine.includes(':=')) {
break;
}
i++;
expr += ' ' + nextLine;
}
statements.push({ type: 'require', expr });
statements.push({ type: 'require', expr: stmt.substring(8).trim() });
continue;
}
// return expr
@ -981,7 +834,7 @@ function parseActionBlockMultiline(lines: string[], startIndex: number, startBra
const fieldsStr = emitMatch[2];
const fields: { name: string; value: string }[] = [];
if (fieldsStr) {
const fieldPairs = splitFields(fieldsStr);
const fieldPairs = fieldsStr.split(',').map(s => s.trim()).filter(Boolean);
for (const fp of fieldPairs) {
const colonIdx = fp.indexOf(':');
if (colonIdx > 0) {
@ -995,46 +848,16 @@ function parseActionBlockMultiline(lines: string[], startIndex: number, startBra
statements.push({ type: 'emit', event: eventName, fields });
continue;
}
// if condition { ... } else if condition { ... } else { ... }
// SpeckDL conditional blocks - parse as a special statement type
if (stmt.startsWith('if ')) {
// Find matching } for this if block (may span multiple bodyLines entries)
let braceDepth = 0;
let j = i;
const blockLines: string[] = [];
while (j < bodyLines.length) {
const line = bodyLines[j];
if (line === null) { j++; continue; }
blockLines.push(line);
for (const ch of line) {
if (ch === '{') braceDepth++;
else if (ch === '}') braceDepth--;
}
j++;
if (braceDepth === 0) break;
}
// For now, treat as a raw 'ifblock' statement that the generator can handle
statements.push({ type: 'ifblock', raw: blockLines.join(' ') });
i = j - 1; // skip to end of if block
continue;
}
// let name := expr
const letMatch = stmt.match(/^let\s+(\w+)\s*:=\s*(.+)$/);
if (letMatch) {
statements.push({ type: 'let', name: letMatch[1], expr: letMatch[2].trim() });
continue;
}
// assignment: target := expr (may span multiple lines)
// assignment: target := expr
const assignMatch = stmt.match(/^(\w+(?:\[.*?\])?)\s*:=\s*(.+)$/);
if (assignMatch) {
let target = assignMatch[1];
let expr = assignMatch[2].trim();
// Handle multiline expressions: join subsequent lines until braces/parens are balanced
while (i + 1 < bodyLines.length && !isExpressionComplete(expr)) {
i++;
expr += ' ' + bodyLines[i];
}
statements.push({ type: 'assign', target, expr });
statements.push({ type: 'assign', target: assignMatch[1], expr: assignMatch[2].trim() });
}
}
@ -1066,284 +889,25 @@ function parseInterfaceBlockMultiline(lines: string[], startIndex: number, start
const methods: MethodSignature[] = [];
for (const line of innerLines) {
// name-only: enum-style values (e.g., "Idle", "Idle, Parsing" comma-separated)
// Split on comma and take each trimmed token
const tokens = line.split(',').map(s => s.trim()).filter(Boolean);
for (const token of tokens) {
// Check if it’s a name(params) -> returnType method signature
const methodMatch = token.match(/^(\w+)\s*\(([^)]*)\)\s*->\s*(.+?),?$/);
if (methodMatch) {
const name = methodMatch[1];
const paramsStr = methodMatch[2];
const returnTypeStr = methodMatch[3].replace(/,$/, '').trim();
const params: { name: string; type: TypeExpr }[] = [];
if (paramsStr.trim()) {
const paramPairs = paramsStr.split(',').map(s => s.trim()).filter(Boolean);
for (const p of paramPairs) {
const pMatch = p.match(/^(\w+)\s*:\s*(.+)$/);
if (pMatch) {
params.push({ name: pMatch[1], type: parseTypeExpr(pMatch[2].trim()) });
}
// name(params) -> returnType
const match = line.match(/^(\w+)\s*\(([^)]*)\)\s*-\u003e\s*(.+?),?$/);
if (match) {
const name = match[1];
const paramsStr = match[2];
const returnTypeStr = match[3].replace(/,$/, '').trim();
const params: { name: string; type: TypeExpr }[] = [];
if (paramsStr.trim()) {
const paramPairs = paramsStr.split(',').map(s => s.trim()).filter(Boolean);
for (const p of paramPairs) {
const pMatch = p.match(/^(\w+)\s*:\s*(.+)$/);
if (pMatch) {
params.push({ name: pMatch[1], type: parseTypeExpr(pMatch[2].trim()) });
}
}
methods.push({ name, params, returnType: parseTypeExpr(returnTypeStr) });
} else if (/^\w+$/.test(token)) {
// Name-only (enum-style) — treat as a method with no params and void return
methods.push({ name: token, params: [], returnType: parseTypeExpr('Void') });
}
methods.push({ name, params, returnType: parseTypeExpr(returnTypeStr) });
}
}
return { ...header, methods };
}
// --- Multi-line constraint/verify parsing (no braces needed) ---
/**
* Parse a constraint or verify that may span multiple lines without braces.
* Used for forall-style constraints like:
* constraint Name: forall r in [0..4]:
* (expr)
*
* and verify blocks like:
* verify Always(Name)
* verify Always(Name) { depth N }
*/
function parseMultiLineConstraintVerify(lines: string[], startIndex: number): MemberNode | null {
const line = lines[startIndex].trim();
if (line.startsWith('verify ')) {
return parseVerifyAnyForm(lines, startIndex);
}
if (line.startsWith('constraint ')) {
return parseConstraintAnyForm(lines, startIndex);
}
return null;
}
function parseConstraintAnyForm(lines: string[], startIndex: number): ConstraintNode | null {
const line = lines[startIndex].trim();
const namedMatch = line.match(/^constraint\s+(\w+):(.*)$/);
if (!namedMatch) return null;
const name = namedMatch[1];
let expr = namedMatch[2].trim();
// If opens a brace, treat as block
if (expr.startsWith('{')) {
const afterBrace = expr.substring(1).trim();
const openCount = (line.match(/{/g) || []).length;
const endIdx = findBlockEnd(lines, startIndex + 1, openCount);
const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
const inner = innerLines.join(' ');
expr = (afterBrace + ' ' + inner).trim();
} else {
// No braces — but might span multiple lines (forall continuation)
const continuationLines: string[] = [];
for (let j = startIndex + 1; j < lines.length; j++) {
const nl = lines[j].trim();
if (!nl || nl.startsWith('//') || nl.startsWith('/*') ||
nl.startsWith('constraint ') || nl.startsWith('verify ') ||
nl.startsWith('constraint:') || nl.startsWith('verify:') ||
nl.startsWith('state') || nl.startsWith('init') ||
nl.startsWith('action ') || nl.startsWith('event ') ||
nl.startsWith('interface ') || nl.startsWith('provenance ') ||
nl.startsWith('bom ') || nl.startsWith('}') ||
nl.startsWith('next:') || nl.startsWith('invariant ')) {
break;
}
continuationLines.push(nl);
}
if (continuationLines.length > 0) {
expr = expr + ' ' + continuationLines.join(' ');
}
}
return { type: 'constraint', name, expr: expr.trim() };
}
function parseVerifyAnyForm(lines: string[], startIndex: number): VerifyNode | null {
const line = lines[startIndex].trim();
// Single-line: verify Always(Name) { depth N }
const fullMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*\{\s*depth\s+(\d+)\s*\}/);
if (fullMatch) {
return {
type: 'verify',
name: fullMatch[2],
temporalExpr: fullMatch[1].trim(),
depth: parseInt(fullMatch[3], 10),
};
}
// Multi-line: verify Always(Name) { ... }
if (line.includes('{')) {
const headerMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*\{/);
if (headerMatch) {
const temporalExpr = headerMatch[1].trim();
const name = headerMatch[2];
const openCount = (line.match(/{/g) || []).length;
const endIdx = findBlockEnd(lines, startIndex + 1, openCount);
const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
const inner = innerLines.join(' ');
const afterBrace = line.substring(line.indexOf('{') + 1).trim();
const combined = (afterBrace + ' ' + inner).trim();
const depthMatch = combined.match(/\bdepth\s+(\d+)\b/);
const depth = depthMatch ? parseInt(depthMatch[1], 10) : undefined;
return { type: 'verify', name, temporalExpr, depth };
}
}
// Simple: verify Always(Name)
const simpleMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*$/);
if (simpleMatch) {
return {
type: 'verify',
name: simpleMatch[2],
temporalExpr: simpleMatch[1].trim(),
};
}
return null;
}
// --- Verify/Constraint block parsing ---
/**
* Parse a verify block opened with `verify Always(Name) { ... }` or similar.
* The first line already contains the header; we need to parse the body inside { }.
*/
function parseVerifyBlockMultiline(lines: string[], startIndex: number, startBraceCount: number): VerifyNode | null {
const firstLine = lines[startIndex].trim();
// Match: verify Always(Name) { depth N } or verify Always(Name) {
const headerMatch = firstLine.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*\{/);
if (!headerMatch) return null;
const temporalExpr = headerMatch[1].trim();
const name = headerMatch[2];
// Check if the brace closes on the same line (single-line form)
const openBraceCount = (firstLine.match(/{/g) || []).length;
const closeBraceCount = (firstLine.match(/}/g) || []).length;
if (closeBraceCount >= openBraceCount) {
// Single-line: parse depth from same line
const afterBrace = firstLine.substring(firstLine.indexOf('{') + 1);
const beforeClose = afterBrace.substring(0, afterBrace.lastIndexOf('}')).trim();
const depthMatch = beforeClose.match(/\bdepth\s+(\d+)\b/);
const depth = depthMatch ? parseInt(depthMatch[1], 10) : undefined;
return { type: 'verify', name, temporalExpr, depth };
}
// Multi-line block
let depth: number | undefined;
const endIdx = findBlockEnd(lines, startIndex + 1, startBraceCount);
const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
const inner = innerLines.join(' ');
const afterBrace = firstLine.substring(firstLine.indexOf('{') + 1).trim();
const depthMatch = (afterBrace + ' ' + inner).match(/\bdepth\s+(\d+)\b/);
if (depthMatch) {
depth = parseInt(depthMatch[1], 10);
}
return { type: 'verify', name, temporalExpr, depth };
}
/**
* Parse a single-line verify statement: verify Always(Name) { depth N }
*/
function parseVerifySingleLine(line: string): VerifyNode | null {
// verify Always(Name) { depth N }
const fullMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*\{\s*depth\s+(\d+)\s*\}/);
if (fullMatch) {
return {
type: 'verify',
name: fullMatch[2],
temporalExpr: fullMatch[1].trim(),
depth: parseInt(fullMatch[3], 10),
};
}
// verify Always(Name) (no depth, no braces — just a statement)
const simpleMatch = line.match(/^verify\s+((?:Always|always|Eventually|eventually)\s*\(\s*(\w+)\s*\))\s*$/);
if (simpleMatch) {
return {
type: 'verify',
name: simpleMatch[2],
temporalExpr: simpleMatch[1].trim(),
};
}
return null;
}
/**
* Parse a constraint block opened with `constraint Name: ...` which may
* be multi-line (e.g. forall spanning multiple lines).
*/
function parseConstraintBlockMultiline(lines: string[], startIndex: number, startBraceCount: number): ConstraintNode | null {
const firstLine = lines[startIndex].trim();
// Match: constraint Name: expr (possibly opening a {} block, or just a forall on next line)
// For the SQLiteWAL style: constraint Name: expr (single line, no braces needed)
const namedMatch = firstLine.match(/^constraint\s+(\w+):(.*)$/);
if (namedMatch) {
const name = namedMatch[1];
let expr = namedMatch[2].trim();
// If the expression opens a brace, collect the block
if (expr.startsWith('{')) {
const endIdx = findBlockEnd(lines, startIndex + 1, startBraceCount);
const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
const inner = innerLines.join(' ');
// Combine: content after { on first line + content from inner lines
const afterBrace = expr.substring(1).trim();
expr = (afterBrace + ' ' + inner).trim();
} else {
// No braces, but might still be multiline (forall spanning multiple lines)
// Check if next line(s) continue the expression
const endIdx = findBlockEnd(lines, startIndex + 1, startBraceCount);
const innerLines = lines.slice(startIndex + 1, endIdx).map(l => l.trim()).filter(l => l && !l.startsWith('//') && !l.startsWith('/*'));
if (innerLines.length > 0) {
expr = expr + ' ' + innerLines.join(' ');
}
}
return { type: 'constraint', name, expr: expr.trim() };
}
return null;
}
/** Split on commas while respecting parenthesized groups and quoted strings */
function splitFields(s: string): string[] {
const result: string[] = [];
let depth = 0;
let inDouble = false;
let inSingle = false;
let start = 0;
for (let i = 0; i < s.length; i++) {
const ch = s[i];
if (ch === '"' && !inSingle) { inDouble = !inDouble; continue; }
if (ch === "'" && !inDouble) { inSingle = !inSingle; continue; }
if (inDouble || inSingle) continue;
if (ch === '(' || ch === '{') { depth++; continue; }
if (ch === ')' || ch === '}') { depth--; continue; }
if (ch === ',' && depth === 0) {
result.push(s.substring(start, i).trim());
start = i + 1;
}
}
const last = s.substring(start).trim();
if (last) result.push(last);
return result;
}

165
compiler/src/runtime/wasm-host.ts Normal file
View file

@ -0,0 +1,165 @@
/**
* Speckl WASM Host Runtime
*
* Provides JS-side implementations for imported host functions used by
* Speckl-generated WASM modules:
* - map_create, map_set, map_get, map_has, map_remove
* - set_create, set_add, set_has, set_size
* - alloc, write_i32, write_f64
*
* Usage:
* const wasmModule = await WebAssembly.instantiate(bytes, {
* speckl: createHostImports()
* });
* wasmModule.exports.init();
* wasmModule.exports.CreateAccount(100.0);
* const state = getStateFromMemory(wasmModule);
*/
interface HostState {
maps: Map<number, Map<number, any>>;
sets: Map<number, Set<any>>;
nextHandle: number;
memory: WebAssembly.Memory | null;
}
function createHostState(): HostState {
return {
maps: new Map(),
sets: new Map(),
nextHandle: 1,
memory: null,
};
}
export function createHostImports(state: HostState = createHostState()) {
return {
// ---- Map operations ----
map_create: (): number => {
const handle = state.nextHandle++;
state.maps.set(handle, new Map());
return handle;
},
map_set: (mapHandle: number, keyHandle: number, valueHandle: number): void => {
const map = state.maps.get(mapHandle);
if (!map) throw new Error(`Map handle ${mapHandle} not found`);
// For now, keys and values are simple i32 handles; boxing layer comes later
map.set(keyHandle, valueHandle);
},
map_get: (mapHandle: number, keyHandle: number): number => {
const map = state.maps.get(mapHandle);
if (!map) return 0;
return map.has(keyHandle) ? (map.get(keyHandle) ?? 0) : 0;
},
map_has: (mapHandle: number, keyHandle: number): number => {
const map = state.maps.get(mapHandle);
if (!map) return 0;
return map.has(keyHandle) ? 1 : 0;
},
map_remove: (mapHandle: number, keyHandle: number): void => {
const map = state.maps.get(mapHandle);
if (map) map.delete(keyHandle);
},
// ---- Set operations ----
set_create: (): number => {
const handle = state.nextHandle++;
state.sets.set(handle, new Set());
return handle;
},
set_add: (setHandle: number, valueHandle: number): void => {
const set = state.sets.get(setHandle);
if (!set) throw new Error(`Set handle ${setHandle} not found`);
set.add(valueHandle);
},
set_has: (setHandle: number, valueHandle: number): number => {
const set = state.sets.get(setHandle);
if (!set) return 0;
return set.has(valueHandle) ? 1 : 0;
},
set_size: (setHandle: number): number => {
const set = state.sets.get(setHandle);
if (!set) return 0;
return set.size;
},
// ---- Memory helpers ----
alloc: (size: number): number => {
// Simple bump allocator: for now, just return 0 (stack allocation handled by compiler)
// Future: maintain bump pointer in memory
return 0;
},
write_i32: (ptr: number, value: number): void => {
if (!state.memory) throw new Error('Memory not bound');
const view = new DataView(state.memory.buffer);
view.setInt32(ptr, value, true); // little-endian
},
write_f64: (ptr: number, value: number): void => {
if (!state.memory) throw new Error('Memory not bound');
const view = new DataView(state.memory.buffer);
view.setFloat64(ptr, value, true);
},
};
}
/**
* Bind a WASM memory export to the host state so write_i32/write_f64 work.
*/
export function bindMemory(state: HostState, memory: WebAssembly.Memory): void {
state.memory = memory;
}
/**
* Read scalar state from WASM memory after getState() has been called.
* Layout matches emitGetState(): [nextId: i32][ledgerOpen: i32]...
*/
export function readScalarState(memory: WebAssembly.Memory, varNames: string[]): Record<string, number> {
const view = new DataView(memory.buffer);
const result: Record<string, number> = {};
let offset = 0;
for (const name of varNames) {
// All current scalars are i32 in TigerBeetle; extend for f64 later
result[name] = view.getInt32(offset, true);
offset += 4;
}
return result;
}
/**
* High-level helper: instantiate a Speckl WASM module from WAT text.
* Requires `wat2wasm` or a WAT parser. For now, expects pre-compiled .wasm bytes.
*/
export async function instantiateSpeckl(
wasmBytes: BufferSource,
varNames: string[]
): Promise<{
exports: WebAssembly.Exports;
state: HostState;
getScalarState: () => Record<string, number>;
}> {
const hostState = createHostState();
const imports = { speckl: createHostImports(hostState) };
const wasm = await WebAssembly.instantiate(wasmBytes, imports);
const memory = wasm.instance.exports.memory as WebAssembly.Memory;
bindMemory(hostState, memory);
const getScalarState = () => readScalarState(memory, varNames);
return {
exports: wasm.instance.exports,
state: hostState,
getScalarState,
};
}

7
compiler/test-output-rust/TigerBeetleLedger/Cargo.toml
View file

@ -1,7 +0,0 @@
[package]
name = "tiger_beetle_ledger"
version = "0.1.0"
edition = "2021"
description = "Auto-generated Speck state machine: TigerBeetleLedger"
[dependencies]

270
compiler/test-output-rust/TigerBeetleLedger/src/lib.rs
View file

@ -1,270 +0,0 @@
// Auto-generated by speckl-compile from TigerBeetleLedger
// DO NOT EDIT MANUALLY — regenerate from .speckdl source
use std::collections::{HashMap, HashSet};
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum TigerBeetleLedgerState {
Open,
Closed,
}
impl std::fmt::Display for TigerBeetleLedgerState {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
TigerBeetleLedgerState::Open => write!(f, "open"),
TigerBeetleLedgerState::Closed => write!(f, "closed"),
}
}
}
/// Speck state machine: TigerBeetleLedger
/// Auto-generated Rust state machine from SpeckDL specification.
#[derive(Debug, Clone)]
pub struct TigerBeetleLedgerMachine {
pub accounts: HashMap<u64, Account>,
pub transfers: HashMap<u64, Transfer>,
pub pending: HashSet<u64>,
pub replica_state: HashMap<u64, Replica>,
pub log: Vec<LogEntry>,
pub commit_index: u64,
pub leader: u64,
/// Decision log of events emitted during execution
pub decision_log: Vec<String>,
}
impl TigerBeetleLedgerMachine {
/// Create a new TigerBeetleLedger machine in its initial state.
pub fn new() -> Self {
Self {
accounts: HashMap::new(),
transfers: HashMap::new(),
pending: HashSet::new(),
replica_state: HashMap::new(),
log: Vec::new(),
commit_index: 0u64,
leader: 0u64,
decision_log: Vec::new(),
}
}
}
// Action methods — each returns Result with guard checking
impl TigerBeetleLedgerMachine {
/// Execute action: CreateTransfer
pub fn create_transfer(&mut self, t: Transfer) -> Result<(), String> {
// precondition: t.id notIn transfers.keys
if !(!self.transfers.contains_key(&t.id)) {
return Err(format!("Precondition failed: {}", "t.id notIn transfers.keys"));
}
// precondition: t.from in accounts.keys
if !(self.accounts.contains_key(&t.from)) {
return Err(format!("Precondition failed: {}", "t.from in accounts.keys"));
}
// precondition: t.to in accounts.keys
if !(self.accounts.contains_key(&t.to)) {
return Err(format!("Precondition failed: {}", "t.to in accounts.keys"));
}
// precondition: t.from != t.to
if !(t.from != t.to) {
return Err(format!("Precondition failed: {}", "t.from != t.to"));
}
// precondition: accounts[t.from].balance >= t.amount || accounts[t.from].flags.allows_overdraft
if !(self.accounts[&&t.from].balance >= t.amount || self.accounts[&&t.from].flags.allows_overdraft) {
return Err(format!("Precondition failed: {}", "accounts[t.from].balance >= t.amount || accounts[t.from].flags.allows_overdraft"));
}
// postcondition: transfers[t.id] == t
if !(self.transfers[&&t.id] == t) {
return Err(format!("Postcondition failed: {}", "transfers[t.id] == t"));
}
// postcondition: pending == pending union {t.id}
if !(self.pending == {self.pending.clone().into_iter().chain(std::iter::once(t.id)).collect()}) {
return Err(format!("Postcondition failed: {}", "pending == pending union {t.id}"));
}
// postcondition: accounts[t.from].debits_pending == old(accounts[t.from].debits_pending) + t.amount
if !(self.accounts[&&t.from].debits_pending == self.accounts[&&t.from].debits_pending + t.amount) {
return Err(format!("Postcondition failed: {}", "accounts[t.from].debits_pending == old(accounts[t.from].debits_pending) + t.amount"));
}
// postcondition: accounts[t.to].credits_pending == old(accounts[t.to].credits_pending) + t.amount
if !(self.accounts[&&t.to].credits_pending == self.accounts[&&t.to].credits_pending + t.amount) {
return Err(format!("Postcondition failed: {}", "accounts[t.to].credits_pending == old(accounts[t.to].credits_pending) + t.amount"));
}
}
/// Execute action: CommitTransfer
pub fn commit_transfer(&mut self, id: u64) -> Result<(), String> {
// precondition: id in pending
if !(self.pending.contains(&id)) {
return Err(format!("Precondition failed: {}", "id in pending"));
}
// precondition: id in transfers.keys
if !(self.transfers.contains_key(&id)) {
return Err(format!("Precondition failed: {}", "id in transfers.keys"));
}
// precondition: transfers[id].pending == true
if !(self.transfers[&&id].pending == true) {
return Err(format!("Precondition failed: {}", "transfers[id].pending == true"));
}
// postcondition: pending == pending \\ {id}
if !(self.pending == {self.pending.clone().into_iter().filter(|x| x != &id).collect()}) {
return Err(format!("Postcondition failed: {}", "pending == pending \\\\ {id}"));
}
// postcondition: transfers[id].pending == false
if !(self.transfers[&&id].pending == false) {
return Err(format!("Postcondition failed: {}", "transfers[id].pending == false"));
}
// postcondition: transfers[id].posted == true
if !(self.transfers[&&id].posted == true) {
return Err(format!("Postcondition failed: {}", "transfers[id].posted == true"));
}
// postcondition: accounts[transfers[id].from].debits_posted ==
if !(self.accounts[&&self.transfers[&&id].from].debits_posted == self.accounts[&&self.transfers[&&id].from].debits_posted) {
return Err(format!("Postcondition failed: {}", "accounts[transfers[id].from].debits_posted =="));
}
// postcondition: accounts[transfers[id].to].credits_posted ==
if !(self.accounts[&&self.transfers[&&id].to].credits_posted == self.accounts[&&self.transfers[&&id].to].credits_posted) {
return Err(format!("Postcondition failed: {}", "accounts[transfers[id].to].credits_posted =="));
}
// postcondition: accounts[transfers[id].from].debits_pending ==
if !(self.accounts[&&self.transfers[&&id].from].debits_pending == self.accounts[&&self.transfers[&&id].from].debits_pending) {
return Err(format!("Postcondition failed: {}", "accounts[transfers[id].from].debits_pending =="));
}
// postcondition: accounts[transfers[id].to].credits_pending ==
if !(self.accounts[&&self.transfers[&&id].to].credits_pending == self.accounts[&&self.transfers[&&id].to].credits_pending) {
return Err(format!("Postcondition failed: {}", "accounts[transfers[id].to].credits_pending =="));
}
}
/// Execute action: VoidTransfer
pub fn void_transfer(&mut self, id: u64) -> Result<(), String> {
// precondition: id in pending
if !(self.pending.contains(&id)) {
return Err(format!("Precondition failed: {}", "id in pending"));
}
// precondition: id in transfers.keys
if !(self.transfers.contains_key(&id)) {
return Err(format!("Precondition failed: {}", "id in transfers.keys"));
}
// precondition: transfers[id].pending == true
if !(self.transfers[&&id].pending == true) {
return Err(format!("Precondition failed: {}", "transfers[id].pending == true"));
}
// postcondition: pending == pending \\ {id}
if !(self.pending == {self.pending.clone().into_iter().filter(|x| x != &id).collect()}) {
return Err(format!("Postcondition failed: {}", "pending == pending \\\\ {id}"));
}
// postcondition: transfers[id].pending == false
if !(self.transfers[&&id].pending == false) {
return Err(format!("Postcondition failed: {}", "transfers[id].pending == false"));
}
// postcondition: transfers[id].voided == true
if !(self.transfers[&&id].voided == true) {
return Err(format!("Postcondition failed: {}", "transfers[id].voided == true"));
}
// postcondition: accounts[transfers[id].from].debits_pending ==
if !(self.accounts[&&self.transfers[&&id].from].debits_pending == self.accounts[&&self.transfers[&&id].from].debits_pending) {
return Err(format!("Postcondition failed: {}", "accounts[transfers[id].from].debits_pending =="));
}
// postcondition: accounts[transfers[id].to].credits_pending ==
if !(self.accounts[&&self.transfers[&&id].to].credits_pending == self.accounts[&&self.transfers[&&id].to].credits_pending) {
return Err(format!("Postcondition failed: {}", "accounts[transfers[id].to].credits_pending =="));
}
}
/// Execute action: PostTransfer
pub fn post_transfer(&mut self, t: Transfer) -> Result<(), String> {
// precondition: t.id notIn transfers.keys
if !(!self.transfers.contains_key(&t.id)) {
return Err(format!("Precondition failed: {}", "t.id notIn transfers.keys"));
}
// precondition: t.from in accounts.keys
if !(self.accounts.contains_key(&t.from)) {
return Err(format!("Precondition failed: {}", "t.from in accounts.keys"));
}
// precondition: t.to in accounts.keys
if !(self.accounts.contains_key(&t.to)) {
return Err(format!("Precondition failed: {}", "t.to in accounts.keys"));
}
// precondition: t.from != t.to
if !(t.from != t.to) {
return Err(format!("Precondition failed: {}", "t.from != t.to"));
}
// precondition: accounts[t.from].balance >= t.amount || accounts[t.from].flags.allows_overdraft
if !(self.accounts[&&t.from].balance >= t.amount || self.accounts[&&t.from].flags.allows_overdraft) {
return Err(format!("Precondition failed: {}", "accounts[t.from].balance >= t.amount || accounts[t.from].flags.allows_overdraft"));
}
// postcondition: transfers[t.id] == { ...t, pending: false, posted: true, voided: false }
if !(self.transfers[&&t.id] == { ...t, self.pending: false, posted: true, voided: false }) {
return Err(format!("Postcondition failed: {}", "transfers[t.id] == { ...t, pending: false, posted: true, voided: false }"));
}
// postcondition: accounts[t.from].debits_posted == old(accounts[t.from].debits_posted) + t.amount
if !(self.accounts[&&t.from].debits_posted == self.accounts[&&t.from].debits_posted + t.amount) {
return Err(format!("Postcondition failed: {}", "accounts[t.from].debits_posted == old(accounts[t.from].debits_posted) + t.amount"));
}
// postcondition: accounts[t.to].credits_posted == old(accounts[t.to].credits_posted) + t.amount
if !(self.accounts[&&t.to].credits_posted == self.accounts[&&t.to].credits_posted + t.amount) {
return Err(format!("Postcondition failed: {}", "accounts[t.to].credits_posted == old(accounts[t.to].credits_posted) + t.amount"));
}
}
/// Execute action: BecomeLeader
pub fn become_leader(&mut self, replica: u64, term: u64) -> Result<(), String> {
// precondition: replica in replicaState.keys
if !(self.replica_state.contains_key(&replica)) {
return Err(format!("Precondition failed: {}", "replica in replicaState.keys"));
}
// precondition: replicaState[replica].role == "candidate"
if !(self.replica_state[&replica].role == "candidate") {
return Err(format!("Precondition failed: {}", "replicaState[replica].role == \"candidate\""));
}
// precondition: replicaState[replica].currentTerm < term
if !(self.replica_state[&replica].currentTerm < term) {
return Err(format!("Precondition failed: {}", "replicaState[replica].currentTerm < term"));
}
// postcondition: leader == replica
if !(self.leader == replica) {
return Err(format!("Postcondition failed: {}", "leader == replica"));
}
// postcondition: replicaState[replica].role == "leader"
if !(self.replica_state[&replica].role == "self.leader") {
return Err(format!("Postcondition failed: {}", "replicaState[replica].role == \"leader\""));
}
// postcondition: replicaState[replica].currentTerm == term
if !(self.replica_state[&replica].currentTerm == term) {
return Err(format!("Postcondition failed: {}", "replicaState[replica].currentTerm == term"));
}
}
/// Execute action: AppendEntry
pub fn append_entry(&mut self, entry: LogEntry, term: u64) -> Result<(), String> {
// precondition: entry.index == length(log) + 1
if !(entry.index == self.log.len() + 1) {
return Err(format!("Precondition failed: {}", "entry.index == length(log) + 1"));
}
// precondition: term >= commitIndex
if !(term >= self.commit_index) {
return Err(format!("Precondition failed: {}", "term >= commitIndex"));
}
// postcondition: log == append(log, entry)
if !(self.log == {self.log.push(entry); self.log} /* append */) {
return Err(format!("Postcondition failed: {}", "log == append(log, entry)"));
}
// postcondition: commitIndex == entry.index
if !(self.commit_index == entry.index) {
return Err(format!("Postcondition failed: {}", "commitIndex == entry.index"));
}
}
}
// Temporal verification stubs
impl TigerBeetleLedgerMachine {
/// V1: Always(AccountBalanceInvariant)
pub fn verify_1(&self) -> bool {
// TODO: Model-check or monitor
// Temporal: Always(AccountBalanceInvariant)
true
}
}

2
compiler/test-output/RetryHandler.prov.jsonld
View file

@ -12,7 +12,7 @@
"@type": "prov:Generation",
"prov:activity": {
"@type": "prov:Activity",
"prov:startTime": "2026-05-11T17:13:39.546Z"
"prov:startTime": "2026-05-05T10:11:33.422Z"
}
},
"rdfs:label": "RetryHandler",

6
compiler/test-output/RetryHandler.specbom.cdx.json
View file

@ -38,7 +38,7 @@
"name": "Scott Scoble",
"email": "scott@scoble.me"
},
"timestamp": "2026-05-11T17:13:39.549Z"
"timestamp": "2026-05-05T10:11:33.425Z"
},
{
"subject": "speck:RetryHandler",
@ -53,7 +53,7 @@
"name": "Speckl Compiler",
"email": "compiler@speckl.scoble.me"
},
"timestamp": "2026-05-11T17:13:39.549Z"
"timestamp": "2026-05-05T10:11:33.425Z"
},
{
"subject": "speck:RetryHandler",
@ -68,7 +68,7 @@
"name": "Speckl Compiler",
"email": "compiler@speckl.scoble.me"
},
"timestamp": "2026-05-11T17:13:39.549Z"
"timestamp": "2026-05-05T10:11:33.425Z"
}
]
}

4
compiler/test-output/RetryHandler.specbom.spdx.json
View file

@ -13,7 +13,7 @@
"spdxVersion": "3.0.1",
"dataLicense": "CC0-1.0",
"creationInfo": {
"created": "2026-05-11T17:13:39.551Z",
"created": "2026-05-05T10:11:33.426Z",
"createdBy": [
"Organization: Speckl",
"tool: speckl-compile-0.2.0"
@ -27,7 +27,7 @@
"@type": "spdx:Tool",
"name": "speckl-compile"
},
"timestamp": "2026-05-11T17:13:39.551Z",
"timestamp": "2026-05-05T10:11:33.426Z",
"comment": "Review type: manual"
}
]

334
docs/PROVE-KAFKA-PLAN.md
View file

@ -1,334 +0,0 @@
# Prove Kafka — Project Plan
> *Don't replace Kafka. Prove it. Then prove everything else.*
---
## Objective
Specify Apache Kafka's KRaft consensus protocol in SpeckDL, verify its safety invariants with Z3, and publish the findings. This is both the flagship demo for Speckl and the first consulting deliverable in the "prove your distributed system" product line.
---
## Guiding Principle
Dependencies only. No wall-time estimates. Each milestone lists what must be true before it can start.
---
## Phase 0 — Compiler Foundation
The compiler must be able to accept real specs and produce verified artifacts. Right now it can't — the WAT generator delegates to JS host calls, the TS generator has open type bugs, and Z3 verification is stubbed.
### M0.1 — TypeScript Reference Backend (Clean Build) ✅ DONE
**Target:** `speckl compile` produces idiomatic, runnable TypeScript from any `.speck` file. All 12 examples compile and pass generated tests.
**Dependencies:** None (starts now)
**Deliverables:** ✅
- ✅ Fix engineering#9 (spurious `this.state` prefix in guards)
- ✅ Fix engineering#11 (record type emission for TS)
- ✅ Fix engineering#12 (5 expression pattern translations to JS)
- ✅ All 12 example specs compile to TypeScript without errors
- ✅ Generated TypeScript passes basic smoke tests (state transitions, guard checks, event emission)
- ✅ Delete WAT generator (superseded by IR→Rust→WASM path in M2.1)
**Completed:** 2026-05-10. Branch `fix/ts-generator-bugs`, `fix/remove-wat-generator`. All 41 tests passing.
---
### M0.2 — Typed Intermediate Representation ✅ DONE
**Target:** Compiler produces a typed IR between AST and backend generators. No generator duplicates type logic or parses raw AST.
**Dependencies:** M0.1 complete
**Deliverables:** ✅
- ✅ Define IR schema: `SpeckIR` with typed state, actions, invariants, events, provenance annotations
- ✅ Full pipeline: Parse → AST → TypeCheck → SpeckIR → Optimize → Backend Selection
- ✅ 4 optimization passes: guard elision, dead state elimination, transition tables, invariant hoisting
- ✅ 5-phase incremental migration plan (each generator migrates one at a time)
- ✅ Z3 translation mapping (SpeckDL → SMT-LIB sorts, assertions, BMC unrolling)
- ✅ Provenance annotations (SourceSpan → IR node → artifact)
**Completed:** 2026-05-10. Branch `feat/ir-design-speckl-20`. 1,106-line design doc at `docs/ir-design.md`. Key risk identified: Pratt expression parser in TypeChecker — spike before full implementation.
---
### M0.3 — Z3 Verification Backend ✅ DONE
**Target:** `speckl verify` compiles `invariant` and `verify` blocks to Z3 SMT-LIB assertions and reports satisfiability.
**Dependencies:** M0.2 complete
**Deliverables:** ✅
- ✅ Z3 backend: translate `state` to sort declarations, `invariant` to define-fun assertions, `verify` to BMC unrolling
- ✅ `--target=z3` CLI option produces valid SMT-LIB2 that Z3 accepts
- ✅ `--verify-depth=N` configurable bounded model checking
- ✅ Zero errors on all specs (ToggleSwitch, TigerBeetleLedger, KafkaKRaft)
- ✅ Clean `sat` with model output from Z3
- ✅ BMC verify blocks in KRaft spec (3 invariants, depth 5)
**Completed:** 2026-05-10. Z3 backend: `compiler/src/generators/z3.ts` (1,037 lines). Z3 4.15.4 verified working.
---
## Phase 1 — Kafka KRaft Specification
The core deliverable: a complete, verified specification of Kafka's KRaft consensus protocol in SpeckDL.
### M1.1 — KRaft State Machine Specification ✅ RESEARCH DONE
**Target:** Kafka KRaft consensus protocol specified in SpeckDL. Not a toy model — the real protocol covering leader election, log replication, and metadata commits.
**Dependencies:** M0.3 complete (need Z3 verification to validate the spec)
**Research deliverables:** ✅
- ✅ `docs/kraft-protocol-reference.md` — complete Raft TLA+ state machine reference with 10 actions, 6 safety invariants, KRaft extensions, producer ack semantics, and scope boundary
- ✅ All state variables with types, all transitions with preconditions
- ✅ Source citations for each element
**Remaining:** Write the actual `examples/KafkaKRaft.speck` spec (M1.2 dependency on Z3 backend M0.3, but spec can be written now)
---
### M1.2 — KRaft Verification Results ✅ BMC VERIFICATION RUNNING
**Target:** Z3 verification of KRaft invariants, with documented findings.
**Dependencies:** M1.1 complete
**Deliverables:** ✅
- ✅ `examples/KafkaKRaft.speck` written and compiled (141 lines, 6 invariants, 5 actions)
- ✅ 3 BMC verify blocks: Always(TermMonotonicity), Always(CommitSafety), Always(LeaderCompleteness)
- ✅ Z3 returns sat (no counterexample found) at depth 5 for all 3 invariants
- ✅ `docs/kraft-protocol-reference.md` — complete source documentation
**Remaining:** Run at deeper depths, document formal conclusions, fill blog findings section
---
### M1.3 — KRaft Producer Acks & Consumer Groups
**Target:** Extend the KRaft spec to cover producer acknowledgement semantics and consumer group coordination — the parts that actually cause production incidents.
**Dependencies:** M1.2 complete
**Deliverables:**
- `examples/KafkaProducer.speck` — producer ack levels (0=fire and forget, 1=leader ack, all=ISR ack), idempotent producer state, transaction coordinator
- `examples/KafkaConsumerGroup.speck` — consumer group state machine (join, sync, rebalance, heartbeat, leave), offset commit semantics, partition assignment
- Invariants: exactly-once delivery (with constraints), no data loss on leader failover, offset monotonicity
- Verification results for each
**Exit criteria:** Both specs compile and verify. Producer ack invariant proves that `acks=all` prevents data loss on leader failure (bounded). Consumer group rebalance is modeled and verified for safety (no concurrent assignments to same partition).
---
## Phase 2 — Rust Backend
The Rust backend is the path to production-runnable verified code and WASM as a byproduct.
### M2.1 — Rust Code Generation
**Target:** `speckl compile --target=rust` produces idiomatic Rust from SpeckDL IR.
**Dependencies:** M0.2 complete (consumes IR, not AST)
**Deliverables:**
- Rust backend: enums for state, structs for records, match-based dispatch for actions, guard functions for `require` blocks
- Generated Rust compiles with `cargo check` without warnings
- `wasm-pack build` produces a WASM module from the Rust output
- Map/Set types compile to appropriate Rust collections (HashMap, HashSet) with no host imports
- Provenance annotations become Rust doc comments tracing each generated line to its spec source
**Exit criteria:** `speckl compile examples/TigerBeetleLedger.speck --target=rust` produces Rust that compiles and passes basic state machine tests. `wasm-pack build` on the output produces a valid `.wasm` module.
---
### M2.2 — KRaft Reference Implementation in Rust
**Target:** Compile `KafkaKRaft.speck` to Rust, run the generated code, demonstrate it matches the spec's invariants at runtime.
**Dependencies:** M1.1 complete (need the spec), M2.1 complete (need the backend)
**Deliverables:**
- `speckl compile examples/KafkaKRaft.speck --target=rust` produces a runnable Rust module
- Runtime guard checks fire and prevent invariant violations
- Provenance trace: each Rust function has a doc comment pointing to the source `.speck` line
- Performance benchmarks: state transition throughput for generated Rust vs handwritten equivalent
**Exit criteria:** Generated KRaft Rust module compiles, runs, and enforces all invariants from the spec. Benchmark shows generated code within 2x of hand-optimized Rust for state transitions.
---
## Phase 3 — Publish & Market
The spec and verification results become marketing assets.
### M3.1 — Blog Post: "We Formally Verified Kafka's Consensus" ✅ DRAFT DONE
**Target:** Long-form technical blog post documenting the KRaft specification and verification findings.
**Dependencies:** M1.2 complete (need results), M0.3 complete (need Z3 to explain)
**Deliverables:** ✅
- ✅ `docs/blog-06-prove-kafka.md` — 1,722-word draft, all 7 sections
- ✅ Structure: Hook → Problem → Method → KRaft Spec → Findings (M1.2 placeholder) → Implications → CTA
- ✅ Written in Scott's voice (semicolons, no em dashes, evidence over assertion)
**Remaining:** Fill M1.2 verification results into findings section, then publish
---
### M3.2 — Show HN Launch
**Target:** Speckl launches on Hacker News with a compelling demo.
**Dependencies:** M0.3 complete (Z3 verification works), M1.2 complete (KRaft results), M3.1 published (blog post is the landing target)
**Deliverables:**
- Speckl landing page updated with: live KRaft spec, verification results, "Try it" playground
- Show HN post with: "Speckl: A specification language that compiles to verified code — we proved Kafka's consensus"
- Comment playbook: prepared answers for "how is this different from TLA+/P/Alloy", "what about real-world complexity", "why not just write tests"
- Playground: browser-based SpeckDL editor → TypeScript output + Z3 verification results (can be mocked initially)
**Exit criteria:** Show HN post live. Landing page live. Playground functional enough for visitors to type a spec and see output.
---
### M3.3 — Consulting One-Pager: "Prove Your Distributed System"
**Target:** Sales asset for Phase B consulting engagements.
**Dependencies:** M1.2 complete (KRaft results are the proof case), M3.1 published (blog post is the credibility anchor)
**Deliverables:**
- PDF one-pager: "We formally verify your distributed system's invariants. Here's what we proved about Kafka."
- Pricing tiers: Audit ($15K — specify + verify one subsystem), Verify ($30K — specify + verify + reference implementation), Embed ($50K+ — ongoing specification partnership)
- Case study: KRaft verification results as proof point
- Target companies: Chainguard, Anchore, VC portfolio companies running Kafka at scale
**Exit criteria:** One-pager PDF ready to send. Blog post is live and linkable.
---
## Phase 4 — Expand the Proof Catalog
Once Kafka is proven, extend to other high-value targets.
### M4.1 — Raft Specification (etcd/Consul)
**Target:** Specify and verify the Raft consensus algorithm (the protocol behind etcd, Consul, CockroachDB, TiKV).
**Dependencies:** M1.2 complete (proven methodology), M0.3 complete (Z3 verification)
**Deliverables:**
- `examples/Raft.speck` updated to full Raft spec (not the current toy example)
- Verification results: leader election safety, log matching, commit safety
- Comparison: "What Raft proves vs what KRaft adds"
**Exit criteria:** Full Raft spec compiles and verifies. Written analysis comparing Raft vs KRaft verification results.
---
### M4.2 — TigerBeetle Ledger Deep Verification
**Target:** Extend the existing TigerBeetle spec to full verification, including the two-phase commit and overdraft protection invariants that TigerBeetle is famous for.
**Dependencies:** M0.3 complete, M2.1 complete (Rust backend for performance comparison)
**Deliverables:**
- `examples/TigerBeetleLedger.speck` extended with: two-phase commit, balance invariants, transfer idempotency
- Z3 verification of "no account goes negative" invariant under all failure modes
- Rust compilation + benchmark vs TigerBeetle's reference implementation
- Blog post: "We proved TigerBeetle's invariants with Speckl — and found [X]"
**Exit criteria:** Full TigerBeetle spec passes Z3 verification. Performance comparison documented. Blog post drafted.
---
### M4.3 — OAuth 2.0 / OIDC Flow Specification ✅ DONE
**Target:** Specify the OAuth 2.0 authorization code flow (including PKCE) as a state machine and verify its invariants.
**Dependencies:** M0.3 complete
**Deliverables:** ✅
- ✅ `examples/OAuth2AuthorizationCode.speck` — 6 actions, 7 invariants, PKCE modeled
- ✅ Invariants: NoTokenLeakage, StateCSRFProtection, AuthorizationCodeSingleUse, TokenValidityBounded, PKCECodeInterceptionPrevention, RedirectUriConsistency, ClientAuthenticationRequired
- Remaining: Z3 verification (depends on M0.3)
**Completed:** 2026-05-10. Branch `feature/oauth2-spec`.
---
## Dependency Graph
```
M0.1 (TS Backend Clean)
└──→ M0.2 (Typed IR)
└──→ M0.3 (Z3 Backend)
├──→ M1.1 (KRaft Spec)
│ └──→ M1.2 (KRaft Verification)
│ ├──→ M1.3 (Producer/Consumer)
│ └──→ M3.1 (Blog Post)
│ ├──→ M3.2 (Show HN)
│ └──→ M3.3 (Consulting One-Pager)
├──→ M4.1 (Raft)
└──→ M4.3 (OAuth 2.0)
M0.2 ──→ M2.1 (Rust Backend)
└──→ M2.2 (KRaft Rust Reference)
└──→ M4.2 (TigerBeetle Deep)
M1.2 ──→ M4.1, M4.2, M4.3
```
---
## Success Metrics
| Metric | Target |
|--------|--------|
| KRaft spec lines | ≤ 300 lines of SpeckDL |
| Invariants verified | ≥ 8 (leader election, log matching, commit safety, etc.) |
| Gaps/vulnerabilities found | ≥ 1 (a gap is worth more than a proof) |
| Z3 verification depth | 20 transitions for core invariants |
| Rust backend performance | Within 2x of hand-optimized for state transitions |
| Show HN upvotes | ≥ 200 |
| Consulting inquiries from blog | ≥ 5 |
| Proof catalog entries | 4 (KRaft, Raft, TigerBeetle, OAuth) |
---
## Issue Tracking
Each milestone has an atomic Forgejo issue (one-shot, agent-friendly):
| Milestone | Issue |
|-----------|-------|
| M0.1: TS bugs | #18 |
| M0.1: Delete WAT | #19 |
| M0.2: Typed IR | #20 |
| M0.3: Z3 backend | #21 |
| M1.1: KRaft spec | #22 |
| M1.2: KRaft verify | #23 |
| M1.3: Producer/Consumer | #24 |
| M2.1: Rust backend | #25 |
| M2.2: KRaft Rust ref | #26 |
| M3.1: Blog post | #27 |
| M3.2: Show HN | #28 |
| M3.3: Consulting PDF | #29 |
| M4.1: Raft spec | #30 |
| M4.2: TigerBeetle | #31 |
| M4.3: OAuth 2.0 | #32 |
Epic #17 closed — superseded by #18–#32.
## Open Items
- **Z3 depth vs. completeness:** Bounded model checking at depth 20 is not exhaustive. Need a strategy for "verified up to N transitions" vs. "verified for all transitions." Document this honestly.
- **KRaft spec scope:** Kafka's KRaft is ~15K lines of Java. The SpeckDL spec must cover the consensus core, not the full broker. Need a clear scope boundary.
- **Show HN timing:** Coordinate with M3.1 publication. Don't launch the HN post until the blog post and playground are live.
- **Consulting pricing:** The one-pager has draft tiers ($15K/$30K/$50K+). Scott needs to review and approve before any outreach.

102
docs/PROVE-SQLITE-PLAN.md
View file

@ -1,102 +0,0 @@
# Prove SQLite — Project Plan
> *The most deployed database in the world. Never formally verified. We're going to fix that.*
---
## Objective
Specify SQLite's core state machines in SpeckDL — pager, WAL mode, VDBE bytecode engine, and B-Tree — then verify their critical invariants with Z3. Publish findings. This is the sequel to Prove Kafka (docs/PROVE-KAFKA-PLAN.md, now complete).
---
## Why SQLite
SQLite is the most deployed database engine in the world — every smartphone, every browser, countless embedded systems. Its testing infrastructure (TH3, 100% branch coverage, mutation testing) is among the best in the industry. But coverage is not proof.
Formal verification of SQLite's core invariants would be a first. Not a theoretical exercise — this is software running on billions of devices.
---
## Phases
### Phase 1 — SQLite Pager & Journal Modes
**Target:** Specify and verify the pager state machine, including transaction lifecycle, journal modes, and WAL concurrency.
| Milestone | Description | Dependencies | Status |
|-----------|-------------|--------------|--------|
| DB.1 | Research doc: pager, WAL, VDBE, B-Tree | None | 🚧 In progress |
| DB.2 | `SQLitePager.speck` | DB.1 | ⬜ |
| DB.3 | Z3 verification of pager invariants | DB.2 | ⬜ |
| DB.4 | `SQLiteWAL.speck` | DB.1 | ⬜ |
| DB.5 | Z3 verification of WAL concurrency | DB.4 | ⬜ |
| DB.6 | Blog post: "Prove SQLite" | DB.3, DB.5 | 🚧 In progress |
### Phase 2 — SQLite VDBE Bytecode Engine
**Target:** Specify the VDBE execution model — program counter, registers, key opcodes, transaction integration.
| Milestone | Description | Dependencies | Status |
|-----------|-------------|--------------|--------|
| DB.7 | `SQLiteVDBE.speck` | DB.1 | ⬜ |
| DB.8 | Z3 verification of VDBE invariants | DB.7 | ⬜ |
| DB.9 | B-Tree cursor integration spec | DB.8 | ⬜ |
### Phase 3 — B-Tree
**Target:** Specify the B-Tree structure — page organization, splits, merges, cursor operations.
| Milestone | Description | Dependencies | Status |
|-----------|-------------|--------------|--------|
| DB.10 | `SQLiteBTree.speck` | DB.7 | ⬜ |
| DB.11 | Z3 verification of B-Tree invariants | DB.10 | ⬜ |
| DB.12 | Synthesis: unified SQLite verification report | DB.3, DB.5, DB.8, DB.11 | ⬜ |
---
## Target Invariants (by layer)
### Pager Invariants
1. **Transaction atomicity:** pages are either fully written or fully rolled back
2. **Journal integrity:** journal reflects committed state, is cleaned on rollback
3. **Cache consistency:** dirty pages are written back before journal deletion
4. **File format:** page size is consistent across the database file
### WAL Invariants
1. **Read consistency:** a reader sees a consistent snapshot (no partial WAL writes)
2. **Write serialization:** at most one writer at a time
3. **Checkpoint safety:** checkpoint does not remove frames still needed by active readers
4. **Frame ordering:** WAL frames are committed in sequence, no gaps
### VDBE Invariants
1. **Program counter validity:** PC stays within instruction bounds
2. **Register type safety:** read register types match write register types
3. **Cursor consistency:** open cursors point to valid B-Tree positions
4. **HALT guarantee:** every program reaches HALT (no infinite loops)
### B-Tree Invariants
1. **Page organization:** every page has valid header and cell pointers
2. **Key ordering:** keys are sorted within a page and across sibling pages
3. **Balance:** tree depth is uniform across all leaves
4. **Split correctness:** splitting a page preserves key ordering and parent linkage
---
## Deliverables
- `docs/db-sqlite-reference.md` — research source material
- `examples/SQLitePager.speck` — pager + WAL specification
- `examples/SQLiteVDBE.speck` — VDBE specification
- `examples/SQLiteBTree.speck` — B-Tree specification
- `docs/blog-07-prove-sqlite.md` — blog post
- Z3 verification results (sat/unsat per invariant)
- For any gaps found: documented counterexample traces
---
## Guiding Principle
Dependencies only. No wall-time estimates. Each milestone lists what must be true before it can start.
Same approach as Prove Kafka, bigger target.

103
docs/blog-07-prove-sqlite.md
View file

@ -1,103 +0,0 @@
# We're Going to Formally Verify SQLite
SQLite runs on every smartphone, every browser, every embedded system. Its core state machines have never been formally verified. We're going to fix that.
This is the follow-up to our Kafka proof-of-consensus project; same methodology, bigger target. SQLite is not a research toy; it is the most deployed database in the history of software. If we can prove its invariants, we prove something that matters.
---
## Why SQLite
SQLite is not a theoretical construct. It ships inside:
- Every iPhone and Android device (iOS, Android system libraries)
- Every major browser (Chrome, Firefox, Safari, Edge)
- Countless embedded systems, IoT devices, and automotive platforms
- macOS, Windows, and most Linux distributions by default
The SQLite Consortium estimates **one trillion** active SQLite databases. Not one billion; one trillion. The test suite (TH3) achieves 100% branch coverage and is widely regarded as the best-tested codebase on Earth. But 100% branch coverage is not proof. It is exhaustive testing, and exhaustive testing is not the same as a mathematical guarantee.
SQLite's internals are exceptionally well-documented. D. Richard Hipp publishes detailed architecture documentation, and the codebase is commented with the rigor of a textbook. The state machines are explicit: the pager has discrete modes, the WAL protocol has a defined lifecycle, and the VDBE bytecode engine operates on a register model with well-defined instruction semantics.
This is not a black box. It is a glass box with a published manual. That makes it an ideal target for formal specification.
## What We'll Verify (Phase 1)
We are not attempting to verify all of SQLite. The full engine is hundreds of thousands of lines of C. Our Phase 1 scope targets three core subsystems whose state machines are both critical and amenable to SpeckDL specification:
### 1. The Pager State Machine
The pager manages the transaction lifecycle: reading pages from disk, caching them, tracking dirtiness, and orchestrating writes. It operates in multiple journal modes (DELETE, TRUNCATE, PERSIST, WAL, MEMORY, OFF) and transitions through states that must respect strict invariants.
The pager state machine is where SQLite's durability guarantee lives. When a transaction commits, the pager decides whether the commit is durable and when the journal can be discarded. In WAL mode, the pager coordinates between a single writer and multiple readers via the WAL index and checkpoint logic. An error in the pager does not produce a crash; it produces silent data loss or inconsistency that may not surface for months.
Key properties to verify:
- A dirty page is never evicted without being written
- WAL mode maintains read consistency across concurrent readers and a single writer
- Journal mode transitions are atomic with respect to crash recovery
- The pager never enters an inconsistent state after an interrupted commit
- Checkpoint operations preserve the "all committed data is in the database file" invariant
### 2. The VDBE Bytecode Engine
The Virtual Database Engine executes compiled SQL bytecode on a register machine. Each instruction manipulates registers, cursors, and the B-Tree layer. The instruction set is finite and well-documented; the register model is a stack-plus-registers hybrid.
The VDBE is not the highest-risk component; its instructions are deterministic and its state is local. But it is the component every SQL statement passes through. A single off-by-one in a register reference or a type-tag mismatch in an arithmetic operation can produce incorrect query results that look like correct ones. Verifying the VDBE means verifying that the bytecode interpreter cannot enter an undefined state regardless of the input program.
Key properties to verify:
- Register references are always within bounds
- Type tags are consistent with the operations performed on them
- Cursor operations maintain valid iterator state
- The program counter advances monotonically except for controlled jumps
- The stack depth never underflows or exceeds its declared maximum
### 3. B-Tree Invariants
The B-Tree layer is SQLite's storage backbone: tables and indexes are both B-Trees. Page splits, pointer integrity, and balance operations are the kind of mechanical code that formal verification excels at.
The B-Tree code in SQLite is dense. A single page split involves allocating a new page, redistributing cells between two pages, updating parent pointers, and handling the edge case where the root itself splits. Each step has preconditions; violate one and the tree becomes structurally inconsistent. Corruption at this layer does not crash; it causes queries to return wrong results or skip rows entirely.
Key properties to verify:
- After any page split, all keys remain reachable via tree traversal
- Sibling pointers maintain total order
- Balance operations preserve the B-Tree height bound
- Cell offsets remain within the page boundary after any insertion or deletion
- Free space within a page is always contiguous and correctly sized
## Method
Same as Prove Kafka: SpeckDL specification, Z3 bounded model checking, publish findings.
The framework is already built. The compiler produces a typed intermediate representation; the Z3 backend translates invariants and verify blocks into SMT-LIB assertions. We proved Kafka's consensus invariants this way; we will prove SQLite's operational invariants the same way.
The workflow:
1. **Specify** the state machine in SpeckDL (pager modes, VDBE registers, B-Tree pages)
2. **Compile** to Z3 SMT-LIB with `speckl verify --target=z3 --verify-depth=N`
3. **Run** bounded model checking against the invariants
4. **Document** the results: what held, what didn't, what depth we reached
5. **Publish** the specification and findings as open-source artifacts
The SpeckDL specifications will be open source. The Z3 output will be reproducible. If we find an invariant violation, you'll read about it here first.
## Why This Matters
There is no shortage of testing in SQLite. TH3 is the gold standard. But testing and proof are different categories of assurance.
Testing says: "We tried a lot of cases and didn't find a bug." Proof says: "No case exists that violates this property." The first is empirical confidence; the second is logical certainty. Both matter. SQLite has the first in abundance. It has never had the second.
Formal verification of SQLite's core invariants would be a first. Not because SQLite is buggy; the evidence suggests it is exceptionally solid. But because proof is the only assurance that does not degrade over time. As the codebase evolves, as new journal modes are added, as platforms change, a verified invariant remains verified. A test case does not.
We proved Kafka's consensus. Now we're proving the world's most deployed database.
## Follow Along
The specifications will live in the Speckl repository at `examples/SQLitePager.speck`, `examples/SQLiteVDBE.speck`, and `examples/SQLiteBTree.speck`. Verification results will be published as we complete each subsystem.
If you want to follow the work or contribute, the repo is open and the issues are tracked publicly. When we find something, you'll read it here first.
---
*This post is part of the Speckl proof catalog: formally verifying the software the world depends on. Read the [Kafka proof](/blog/prove-kafka) or explore the [Speckl compiler](https://os.scoble.me/forgejo/sscoble/speckl).*
*Speckl is MIT-licensed and open source. The specifications, the compiler, and the findings are all public.*

1317
docs/compilation-layer.md
View file

File diff suppressed because it is too large Load diff

551
docs/db-sqlite-reference.md
View file

@ -1,551 +0,0 @@
# SQLite Pager, WAL, VDBE, and B-Tree Reference
> Author: Marcus (agent-cos) <cos@marcus.local>
> Date: 2026-05-09
> Purpose: Engineering source material for SpeckDL formal verification
> Target: SQLite version 3.53.1 (check-in c88b22011a54b, 2026-05-05)
---
## 1. Pager State Machine
The pager is SQLite's storage engine — manages page cache, transactions, and crash recovery. Located in `src/pager.c` (~12K lines). The pager owns the lock state, journal state, and page cache.
### 1.1 Pager State Enum (from `pager.c` header comment)
```
PAGER_UNLOCK (0) — No lock held. Initial state.
PAGER_SHARED (1) — Read lock (SQLITE_LOCK_SHARED). Read-only access.
PAGER_RESERVED (2) — Reserved lock (SQLITE_LOCK_RESERVED). Preparing to write.
PAGER_EXCLUSIVE (3) — Exclusive lock (SQLITE_LOCK_EXCLUSIVE). Actively writing.
PAGER_SYNCED (4) — All dirty pages written + synced. Journal can be deleted.
PAGER_ERROR (5) — Error occurred. No further operations allowed.
PAGER_OPEN (6) — Special state: connection open but no lock. Transitional.
```
### 1.2 State Transition Table
| From | To | Trigger / Guard | Side Effects |
|------|----|-----------------|--------------|
| PAGER_OPEN | PAGER_SHARED | First `pager_acquire()` / `pager_begin()` | Obtain SQLITE_LOCK_SHARED |
| PAGER_UNLOCK | PAGER_SHARED | sqlite3PagerSharedLock() | Obtain shared lock, recover hot journal if present |
| PAGER_SHARED | PAGER_UNLOCK | `pager_unlock()` / end of read tx | Release shared lock |
| PAGER_SHARED | PAGER_RESERVED | `pager_write()` or `pager_begin()` | Obtain SQLITE_LOCK_RESERVED. **No other writer can proceed.** |
| PAGER_RESERVED | PAGER_EXCLUSIVE | First page modification (in `PAGER_RESERVED`) | - |
| PAGER_RESERVED | PAGER_EXCLUSIVE | Commit begins | Obtain PENDING then EXCLUSIVE lock |
| PAGER_RESERVED | PAGER_SHARED | ROLLBACK | Discard changes, release reserved lock |
| PAGER_EXCLUSIVE | PAGER_SYNCED | All dirty pages written + fsynced | Journal header updated to sync state |
| PAGER_SYNCED | PAGER_SHARED (journal-mode) or PAGER_UNLOCK (WAL) | Delete journal file (COMMIT point) | Transaction atomically committed |
| PAGER_EXCLUSIVE | PAGER_ERROR | I/O error, disk full, etc. | errCode set, hot journal preserved |
| PAGER_SYNCED | PAGER_ERROR | I/O error during sync | errCode set, journal preserved for recovery |
| any → | PAGER_ERROR | Any I/O error during pager operation | Further ops blocked |
| PAGER_ERROR | PAGER_OPEN | `pager_unlock()` forced close | Resets connection but keeps db open |
### 1.3 Journal Mode Variants
Three rollback journal modes (non-WAL):
- **DELETE** (default): Journal file deleted on commit. If crash before delete → hot journal → recovery rolls back.
- **TRUNCATE**: Journal file truncated to zero length on commit. Faster on some filesystems than delete.
- **PERSIST**: Journal header zeroed on commit, file left on disk. Slightly faster, but leaves journal files around.
The pager tracks `journalMode` and adjusts the commit-delete step accordingly.
### 1.4 Pager Invariants (to formally specify)
- **I1 (Lock State Consistency)**: `eState` must be consistent with actual file lock held. If `eState==PAGER_SHARED`, must hold `SQLITE_LOCK_SHARED`. If `eState==PAGER_EXCLUSIVE`, must hold `SQLITE_LOCK_EXCLUSIVE`.
- **I2 (Dirty Page Tracking)**: For any page in the cache, `isDirty` iff the page has been modified since last checkpoint/journal write.
- **I3 (Journal Existence)**: The journal file exists iff `eState >= PAGER_RESERVED` and `eState != PAGER_ERROR` (except during recovery).
- **I4 (Hot Journal Detection)**: A journal is "hot" iff it exists but the database was not cleanly closed. On first shared lock, hot journals must be rolled back.
- **I5 (Error Propagation)**: Once `eState == PAGER_ERROR`, all subsequent pager operations must return the stored `errCode`.
- **I6 (Read-Only Pager)**: If `readOnly` flag is set, `eState` must never exceed `PAGER_SHARED`.
- **I7 (Sync Before Commit)**: No dirty page is persisted to the database file until after the journal is synced to disk AND the exclusive lock is held.
- **I8 (Commit Atomicity)**: The commit point is the deletion of the journal file. Before this: crash → rollback. After this: crash → committed state visible.
### 1.5 Lock Escalation Sequence (from `atomiccommit.html`)
```
NO_LOCK → SHARED (read) → RESERVED (intent to write) → PENDING (block new readers)
→ EXCLUSIVE (writing) → no lock (after commit journal deletion)
```
Key subtlety: PENDING lock allows existing readers to finish but blocks new readers. This prevents writer starvation in the face of constant new read connections.
---
## 2. WAL Mode State Machine
WAL mode inverts the rollback journal: original data stays in the database file, changes are appended to the WAL file. Commit = appending a commit record to WAL.
### 2.1 WAL Files
| File | Purpose | Lifecycle |
|------|---------|-----------|
| `X-wal` | Write-ahead log. Sequential append of frames. | Exists while connections are open. Deleted on clean close. |
| `X-shm` | Shared memory (wal-index). Hash table for frame location. | mmap'd. Not fsynced. Omitted under EXCLUSIVE locking mode. |
| `X` | Main database file. Contains original + checkpointed pages. | Persistent. |
### 2.2 WAL-Index Header Fields (from `walformat.html`)
| Field | Bytes | Description |
|-------|-------|-------------|
| iVersion | 0-3 | WAL-index format version (3007000) |
| iChange | 8-11 | Transaction counter, incremented per tx |
| isInit | 12 | 1 when shm file initialized |
| bigEndCksum | 13 | Checksum endianness flag |
| szPage | 14-15 | Page size (or 1 for 65536) |
| mxFrame | 16-19 | Number of valid commit frames in WAL |
| nPage | 20-23 | Database size in pages |
| aFrameCksum | 24-31 | Checksum of last frame |
| aSalt | 32-39 | Two salt values from WAL header |
| nBackfill | 96-99 | Frames already backfilled by checkpoints |
| read-mark[0..4] | 100-119 | Five reader end marks |
| nBackfillAttempted | 128-132 | Frames attempted but not confirmed backfilled |
### 2.3 WAL Locks
| Lock | Byte | Purpose |
|------|------|---------|
| WAL_WRITE_LOCK | 120 | Exclusive: appending frames to WAL |
| WAL_CKPT_LOCK | 121 | Exclusive: running checkpoint |
| WAL_RECOVER_LOCK | 122 | Exclusive: running recovery |
| WAL_READ_LOCK(0) | 123 | Reader end-marks (0-4) |
| WAL_READ_LOCK(1) | 124 | |
| WAL_READ_LOCK(2) | 125 | |
| WAL_READ_LOCK(3) | 126 | |
| WAL_READ_LOCK(4) | 127 | |
### 2.4 WAL Frame Lifecycle
```
Frame format (in WAL file): [page-number] [db-size-after-commit] [salt-1] [salt-2] [checksum-1] [checksum-2]
followed by page data.
Frame lifecycle states:
1. WRITER appends frames to end of WAL file (holding WAL_WRITE_LOCK exclusive)
2. Last frame of a transaction marks "commit" via non-zero db-size field
3. READERS snapshot mxFrame at transaction start → their "end mark"
4. CHECKPOINTER copies frames back to database (holding WAL_CKPT_LOCK), increments nBackfill
5. RESET: When nBackfill == mxFrame and no reader holds WAL_READ_LOCK(N) for N>0,
writer rewinds WAL to beginning (resets mxFrame, nBackfill to 0)
```
### 2.5 Reader Snapshot (End Mark) Protocol
- When a read transaction begins, the reader notes current `mxFrame` value → this is its "end mark"
- The reader acquires one of WAL_READ_LOCK(0..4) to declare its end mark
- To read page P, the reader calls `FindFrame(P, end_mark)`: find the latest frame for page P in WAL with frame number ≤ end_mark
- If found: read from WAL frame. If not found: read from database file.
- Readers hold WAL_READ_LOCK(N) for the duration of their read transaction
- This prevents checkpointer from overwriting pages the reader still needs
### 2.6 Checkpoint States
| Checkpoint Mode | Behavior |
|----------------|----------|
| PASSIVE | Non-blocking. Works around readers. May not complete. Default. |
| FULL | Blocks until checkpoint completes or a writer appears. More aggressive. |
| RESTART | Full checkpoint then blocks until no readers. Ensures WAL reset. |
| TRUNCATE | Like RESTART, also truncates WAL file to zero bytes. |
Checkpoint progress constraint: MUST STOP at the minimum `end_mark` across all active readers.
- `nBackfill ≤ min(read_mark[0..4])`
- `nBackfillAttempted ≥ nBackfill` (may be ahead if reader blocked a backfill)
### 2.7 WAL Reset Condition
The WAL is reset (wound back to start) when:
1. `nBackfill == mxFrame` (all frames backfilled)
2. No reader holds `WAL_READ_LOCK(N)` for any N > 0 (no readers using WAL)
3. Writer holds WAL_WRITE_LOCK exclusively
4. On reset: `mxFrame = 0`, `nBackfill = 0`, file position rewound
### 2.8 WAL Invariants (to formally specify)
- **W1 (mxFrame Monotonic)**: `mxFrame` is non-decreasing except during WAL reset.
- **W2 (nBackfill Bound)**: `nBackfill ≤ mxFrame` always.
- **W3 (Backfill ≤ Readers)**: `nBackfillAttempted ≤ min(read_mark[0..4])` for any checkpoint that respects readers.
- **W4 (Frame Validity)**: All frames 1..mxFrame have valid checksums. Frames > mxFrame are undefined.
- **W5 (Commit Detection)**: Frame k is a commit frame iff its db-size-after-commit field ≠ 0.
- **W6 (Read Consistency)**: A reader with end-mark M sees exactly the database state as of the last commit frame ≤ M.
- **W7 (WAL Write Serialization)**: Only one connection may append to WAL at a time (WAL_WRITE_LOCK exclusive).
- **W8 (Checkpoint Isolation)**: Checkpoint must not overwrite pages needed by any active reader (nBackfill ≤ min(read_marks)).
- **W9 (Salt Match)**: WAL file header salt values must match `aSalt` in the WAL-index header. Mismatch → WAL is stale → must recover.
- **W10 (WAL Reset Safety)**: WAL may only be reset when `nBackfill == mxFrame` AND no reader locks exist.
---
## 3. VDBE Instruction Set (Key Opcodes)
The VDBE is SQLite's bytecode virtual machine. Each prepared statement is a bytecode program. Source: `vdbe.c` (~8K lines). 192 opcodes in version 3.53.1.
### 3.1 Instruction Format
```
Each instruction: [opcode] P1 P2 P3 P4 P5
P1, P2, P3: 32-bit signed integers (registers, cursor numbers, jump targets)
P4: various (int, string, function pointer, collation)
P5: 16-bit unsigned flags
```
### 3.2 Transaction & Locking Opcodes
| Opcode | P1 | P2 | P4 | Effect |
|--------|----|----|----|--------|
| **Transaction** | iDb | write_flag | | Begin a transaction on database P1. P2=1 for write, 0 for read. Opens pager transaction. |
| **AutoCommit** | autocommit_flag | rollback_flag | | Set auto-commit on/off. If P2 set, rollback first. Causes VM halt. |
| **Checkpoint** | iDb | mode | | Run checkpoint on database P1. P2 = PASSIVE(0)/FULL(1)/RESTART(2)/TRUNCATE(3). |
| **Savepoint** | start_flag | savepoint_name | | Begin/Release/Rollback a savepoint. |
| **TableLock** | iDb | root_page | table_name | Lock a table before modifying. write_flag in P3. |
| **SchemaLock** | iDb | | schema_name | Lock schema objects. |
### 3.3 Cursor Opcodes
| Opcode | P1 | P2 | P3 | P4 | Effect |
|--------|----|----|----|----|--------|
| **OpenRead** | cursor | root_page | iDb | | Open cursor P1 for read on root page P2 of db P3 |
| **OpenWrite** | cursor | root_page | iDb | table_name | Open cursor P1 for write on root page P2 of db P3 |
| **OpenAutoindex** | cursor | key_cols | nCol | | Open ephemeral auto-index cursor |
| **OpenEphemeral** | cursor | nCol | | | Open temporary table, auto-clears on close |
| **Close** | cursor | | | | Close cursor P1 |
| **SeekGT** | cursor | jump_target | | key_reg | Seek cursor P1 to first entry > key in register |
| **SeekGE** | cursor | jump_target | | key_reg | Seek cursor P1 to first entry >= key in register |
| **SeekLT** | cursor | jump_target | | key_reg | Seek cursor P1 to first entry < key in register |
| **SeekLE** | cursor | jump_target | | key_reg | Seek cursor P1 to first entry <= key in register |
| **SeekRowid** | cursor | | | | Seek cursor P1 to entry with given rowid |
| **NotFound** | cursor | jump_target | | | Jump if key not found in index |
| **Found** | cursor | jump_target | | | Jump if key found in index |
| **Next** | cursor | jump_target | | | Advance cursor P1 to next entry. Jump to P2 if no more rows. |
| **Prev** | cursor | jump_target | | | Move cursor P1 to previous entry. Jump to P2 if at beginning. |
| **Last** | cursor | | | | Move cursor P1 to last entry. |
| **Rewind** | cursor | jump_target | | | Move to first entry. Jump to P2 if empty. |
| **Column** | cursor | column_idx | dest_reg | | Read column P2 from cursor P1 into register P3 |
| **Rowid** | cursor | dest_reg | | | Write rowid of cursor P1 into register P2 |
| **IdxInsert** | cursor | | | key | Insert entry into index cursor |
| **IdxDelete** | cursor | | | key | Delete entry from index cursor |
| **Insert** | cursor | | | | Insert current record into table cursor |
| **Delete** | cursor | | | | Delete current record from table cursor |
| **SorterSort** | cursor | | | | Sort sorter cursor P1 |
| **SorterData** | cursor | dest_reg | | | Read current sorter row |
### 3.4 Value & Register Opcodes
| Opcode | P1 | P2 | P3 | Effect |
|--------|----|----|----|--------|
| **Integer** | value | dest | | Store integer value in register P2 (P1 is the value) |
| **Real** | value (P4) | dest | | Store real value in register P2 |
| **String8** | | dest | string | Store UTF-8 string in register P2. Self-modifying → becomes String opcode. |
| **Null** | start | count | | Set P2 registers starting at P1 to NULL |
| **Blob** | size | dest | data | Store P1-byte blob in register P2 |
| **Copy** | src | dest | count | Copy P3 registers from P1 to P2 |
| **Move** | src | dest | count | Move P3 registers from P1 to P2 (clear source) |
| **SCopy** | src | dest | | Soft copy (shallow) register P1 to P2 |
| **ResultRow** | reg_start | nReg | | Pause VM: emit result row from registers P1..P1+P2-1. sqlite3_step() returns SQLITE_ROW. |
| **Yield** | pc_store | | | Coroutine yield. Swap PC with value in register P1. |
| **Halt** | p1 | p2 | p3 | Halt execution. If P1==SQLITE_OK and P2==OE_Abort, auto-rollback. |
### 3.5 Arithmetic & Comparison Opcodes
| Opcode | P1 | P2 | P3 | P4 | Effect |
|--------|----|----|----|----|--------|
| **Add** | r1 | r2 | result | | r[P3] = r[P1] + r[P2] |
| **Subtract** | r1 | r2 | result | | r[P3] = r[P1] - r[P2] |
| **Multiply** | r1 | r2 | result | | r[P3] = r[P1] * r[P2] |
| **Divide** | r1 | r2 | result | | r[P3] = r[P1] / r[P2] |
| **Remainder** | r1 | r2 | result | | r[P3] = r[P1] % r[P2] |
| **Eq** | r1 | r2 | jump | coll | Compare r[P1] and r[P2]. Jump to P3 if equal. |
| **Ne** | r1 | r2 | jump | coll | Jump if not equal |
| **Lt** | r1 | r2 | jump | coll | Jump if r[P1] < r[P2] |
| **Le** | r1 | r2 | jump | coll | Jump if r[P1] ≤ r[P2] |
| **Gt** | r1 | r2 | jump | coll | Jump if r[P1] > r[P2] |
| **Ge** | r1 | r2 | jump | coll | Jump if r[P1] ≥ r[P2] |
| **BitAnd** | r1 | r2 | result | | r[P3] = r[P1] & r[P2] |
| **BitOr** | r1 | r2 | result | | r[P3] = r[P1] \| r[P2] |
| **BitNot** | src | dest | | | r[P2] = ~r[P1] |
| **Concat** | r1 | r2 | result | | r[P3] = r[P1] \|\| r[P2] |
### 3.6 Control Flow Opcodes
| Opcode | P1 | P2 | Effect |
|--------|----|----|--------|
| **Goto** | | target | Unconditional jump to P2 |
| **Gosub** | return_reg | target | Push PC to register P1, jump to P2 |
| **Return** | return_reg | | Jump to address stored in P1 + 1 |
| **If** | jump_target | dest | Conditional jump |
| **IfNot** | jump_target | dest | Conditional jump (negated) |
| **IfNull** | reg | jump_target | Jump if register contains NULL |
| **NotNull** | reg | jump_target | Jump if register is not NULL |
| **IfPos** | reg | jump_target | Jump if register > 0 |
| **IfNeg** | reg | jump_target | Jump if register < 0 |
| **Once** | flag | | NOP if flag was set. Execute body exactly once. |
| **Init** | flags | max_addr | First opcode. Sets trace flags. Self-modifying: increments P1 each invocation. |
| **Program** | | sub_program_addr | Invoke a subprogram (trigger). |
| **Param** | param_idx | dest_reg | Read parameter from caller into destination register. |
### 3.7 Aggregate Function Opcodes
| Opcode | P1 | P2 | P3 | Effect |
|--------|----|----|----|--------|
| **AggStep** | | arg_reg | accum | Execute xStep for aggregate function |
| **AggStep1** | step/inverse | arg_reg | accum | Execute xStep (P1=0) or xInverse (P1!=0) |
| **AggInverse** | | arg_reg | accum | Execute xInverse (window functions) |
| **AggFinal** | | | accum | Finalize aggregate, store result |
| **AggValue** | | | result | Store current aggregate value |
### 3.8 VDBE Invariants (to formally specify)
- **V1 (PC Bounds)**: Program counter must remain within [0, nOp] during execution.
- **V2 (Register Type Safety)**: Each register holds exactly one type at a time (NULL, int, real, text, blob, RowSet, Frame, Undefined).
- **V3 (Cursor Validity)**: An opcode must not reference a cursor unless it was previously opened (OpenRead/OpenWrite/OpenEphemeral) and not yet closed.
- **V4 (Transaction Guard)**: Before any write opcode executes, a transaction must be open. Read opcodes may run outside a transaction but must obtain shared lock.
- **V5 (ResultRow Protocol)**: ResultRow must be preceded by opcodes that fill the output registers. sqlite3_step() returns SQLITE_ROW; subsequent call resumes after ResultRow.
- **V6 (Halt Termination)**: Execution terminates only via Halt opcode, PC exceeding last opcode address, or error.
- **V7 (Error Cleanup)**: On error or HALT with error, all open cursors are closed, memory freed, and active transactions rolled back.
- **V8 (Subprogram Isolation)**: Each Program invocation gets its own register set. Params are the only communication channel to the caller.
- **V9 (Coroutine Symmetry)**: Yield stores PC to register; subsequent Yield reads register to PC. Coroutine execution must return to same point.
---
## 4. B-Tree Invariants
SQLite uses B+tree for tables, B-tree for indices. Source: `btree.c`.
### 4.1 Page Types
| Type | Value | Description |
|------|-------|-------------|
| Interior index | 0x02 | Index b-tree interior page |
| Interior table | 0x05 | Table b-tree interior page |
| Leaf index | 0x0a | Index b-tree leaf page |
| Leaf table | 0x0d | Table b-tree leaf page |
### 4.2 Page Layout
```
Page header:
[page_type:1] [first_freeblock:2] [cell_count:2] [cell_content_start:2] [fragmented_free_bytes:1]
Followed by:
[right_child_pointer:4] (interior pages only)
[cell_pointer_array: cell_count * 2 bytes] — offsets to cells, in key order
[unallocated space in middle]
[cells: packed from end of page backwards]
```
### 4.3 B-Tree Cell Format
**Table leaf cell:**
```
[payload_size: varint] [rowid: varint] [payload: header + columns]
```
**Table interior cell:**
```
[left_child_page:4] [rowid: varint] [payload: partial row data]
```
**Index leaf cell:**
```
[payload_size: varint] [payload: header + index columns + rowid]
```
**Index interior cell:**
```
[left_child_page:4] [payload_size: varint] [payload: header + index columns + rowid]
```
### 4.4 Key Ordering Properties
- **Table B-Tree**: Ordered by rowid (integer primary key). Interior cells store the rowid of the first entry in each child page.
- **Index B-Tree**: Ordered by indexed column(s) + rowid. Interior cells store the key of the first entry in each child page.
- All keys within a page are in ascending order.
- For any interior page: all keys in child subtree i are ≤ the key at position i in the interior page, and ≤ all keys in child subtree i+1. (For indices, strict < for key columns, ≤ for rowid as tiebreaker.)
### 4.5 Page Split Condition
A page splits when inserting a cell would exceed the usable page space.
1. A new page is allocated.
2. Cells are distributed (roughly half each) between old and new page.
3. Parent interior page gets a new cell pointing to the new page.
### 4.6 Page Merge Condition
When deleting, if a page's content falls below the minimum fill threshold (roughly 25-50% depending on implementation), the page may be merged with a sibling:
1. Cells are moved to sibling page.
2. Parent pointer to the empty page is removed.
3. Empty page is added to freelist.
### 4.7 B-Tree Cursor Operations
```
Cursor state: { Btree*, root page, current page, cell index, cache of page stack }
Operations:
sqlite3BtreeCursor(): Create cursor on B-tree identified by root page + flags
sqlite3BtreeMoveto(): Position cursor to entry that matches key (or nearest)
sqlite3BtreeNext(): Advance cursor. Page crossing handled automatically.
sqlite3BtreePrevious(): Move cursor backwards. Page crossing handled.
sqlite3BtreeInsert(): Insert cell into current position
sqlite3BtreeDelete(): Delete entry at cursor position
```
### 4.8 B-Tree Invariants (to formally specify)
- **B1 (Page Type Consistency)**: A page's type field must match its role. Interior pages reference child pages; leaf pages do not.
- **B2 (Key Order Monotonic)**: For any interior page, keys in child page i are less than or equal to keys in child page i+1. Cells within a page are sorted ascending.
- **B3 (Pointer Integrity)**: Every child page reference must point to a valid allocated page of the correct type (or overflow page, or freelist page).
- **B4 (Cell Count Consistency)**: `cell_count` in the page header must equal the number of entries in the cell pointer array.
- **B5 (Free Space Accounting)**: `fragmented_free_bytes` + unallocated space between cell pointer array and cells must sum correctly to the page's available free space.
- **B6 (Page Size Boundary)**: Usable page size = page_size - reserved_bytes. All cells must fit within the usable page.
- **B7 (Root Page Integrity)**: The b-tree root page number must be a valid, allocated b-tree page. The schema table's root page mappings must be consistent.
- **B8 (Balance Preserved)**: After every insert/delete operation, the b-tree must satisfy all internal structural invariants (fill ratio, parent-child pointer correctness, key ordering).
- **B9 (Overflow Chain Integrity)**: If a cell's payload spills to overflow pages, the chain of overflow pages must: (a) form a proper linked list, (b) be bounded in length by the payload size, (c) contain no cycles.
- **B10 (Freelist Consistency)**: Freelist pages must not be reachable from any b-tree root. Freelist trunk pages must point to valid leaf pages or 0.
---
## 5. Known Bugs and Formal Verification Precedents
### 5.1 Notable SQLite CVEs
| CVE | Year | Type | Module | Fixed In | Severity |
|-----|------|------|--------|----------|----------|
| CVE-2022-35737 | 2022 | Integer overflow → buffer overrun → RCE/DoS | printf/str_vappendf | 3.39.2 | High (7.5) |
| CVE-2025-3277 | 2025 | Write past end of malloc'd array via integer overflow | concat_ws() function | 3.49.1 | Moderate |
| CVE-2025-6965 | 2025 | Integer overflow → array bounds read | SQL injection path | 3.50.2 | Moderate |
| CVE-2024-0232 | 2024 | Use-after-free in JSON parser | JSON | 3.43.2 | Moderate |
| CVE-2023-7104 | 2023 | Corruption in session extension changeset processing | Session (ext) | 3.43.1 | Low |
| CVE-2025-29088 | 2025 | Crash via bad args to sqlite3_db_config | Config API | 3.49.1 | Low |
**Key observation from SQLite's CVE page:**
> Almost all CVEs written against SQLite require the ability to inject and run arbitrary SQL. The core storage layer (pager, WAL, B-tree) has remarkably few logic bugs. Most CVEs are in SQL-level functions, extensions, or edge cases in the printf/logging subsystem.
### 5.2 22-Year-Old Integer Overflow (CVE-2022-35737)
Introduced October 2000 (version 1.0.12), discovered 2022 by Trail of Bits. Exploitable on 64-bit systems when large strings (>2GB) passed to printf functions with `!` unicode flag. Arbitrary code execution possible without stack canaries. Not a pager/journal bug — but illustrates how old, untested edge cases survive.
### 5.3 Formal Verification Precedents
#### 5.3.1 FoundationDB (Apple)
- **What**: Distributed transactional key-value store, formally verified in Flow (their deterministic simulation language)
- **Method**: Deterministic simulation testing — run all DB processes as fibers on a single thread with simulated network/disk, inject faults, check invariants. Run millions of random seeds nightly.
- **Result**: Found ~70 bugs before release. Flow-based simulation verified correctness of their distributed transaction, replication, and recovery protocols.
- **Relevance to SQLite**: The determinism approach parallels what could be done for SQLite's pager. SQLite is single-process by design, making deterministic replay easier.
#### 5.3.2 TigerBeetle
- **What**: Distributed financial accounting database. VOPR (Viewstamped Operation Replicator) deterministic simulator.
- **Method**: Entire cluster runs on single thread. 1000x speed. Injects network, storage, and process faults. Swarm testing + fuzzer. Continuous verification.
- **Relevance to SQLite**: The VOPR approach of running the real implementation under controlled simulation is directly applicable to testing SQLite's recovery paths in the pager.
#### 5.3.3 Amazon (DynamoDB, S3)
- **What**: TLA+ specifications for AWS distributed services
- **Method**: Model checking (TLC) + manual proofs (TLAPS). Found bugs in replication protocols, quorum membership, leader election.
- **Paper**: "Use of Formal Methods at Amazon Web Services" (Newcombe et al., 2014)
- **Relevance to SQLite**: TLA+ is ideal for specifying state machines like the pager's locking protocol. The pager state machine is small enough for tractable TLC model checking.
#### 5.3.4 MongoDB Transactions (VLDB 2025)
- **What**: "Design and Modular Verification of Distributed Transactions in MongoDB"
- **Method**: TLA+ specification + model checking + test-case generation.
- **Relevance to SQLite**: Demonstrates that even complex transactional protocols can be verified with TLA+.
#### 5.3.5 Formal Land's Rocq Translation of SQLite B-Tree (2025-2026)
- **What**: AI-assisted translation of SQLite C code to Rocq (Coq variant). Translation only (not full proof of equivalence).
- **Result**: "Still a big project to run with current technologies." Covered substantial part of B-tree but deemed too unstable for full verification at current AI capability levels.
- **Relevance**: Direct precedent that SQLite formal verification is hard. Suggests model-level verification (state machine specs) is more tractable than code-level proofs.
#### 5.3.6 Richard Hipp's Own Attempt (SQLite Lead)
- Dr. Hipp reports spending "years" trying to write a formal design specification for SQLite but found it never added value and abandoned it. This is telling — both about the difficulty and about the value proposition.
### 5.4 SQLite's Existing Testing (Non-Formal)
| Method | Scale | Notes |
|--------|-------|-------|
| TH3 (Test Harness #3) | Proprietary. Billions of test cases. | 100% branch coverage. Mutation testing (inject bugs, verify they're caught). |
| OSS-Fuzz | Continuous. | Google's fuzzing infrastructure. Catches memory safety bugs. |
| Release testing | ~100M tests per release | Run on multiple platforms |
| Anomaly testing | Random SQL/DB operations with crash injection | Verify recovery |
---
## 6. Target Invariants for SpeckDL Specification
Ranked by impact (what would be most impressive/valuable to prove):
### Tier 1: Impact/Feasibility Sweet Spot
1. **Pager Commit Atomicity (I8)**
- Prove: For any sequence of pager operations and any crash point, the database file is always in either the pre-commit state (all changes rolled back) or the post-commit state (all changes applied). No partial-commit states.
- Value: This is the fundamental SQLite guarantee. Proving it formally would be the headline achievement.
- Feasibility: Medium. The pager state machine has only 5-6 states. The tricky part is modeling disk I/O reordering and crash-at-any-point semantics.
2. **WAL Read Isolation (W6)**
- Prove: A reader with end-mark M sees exactly the database state as of the last commit ≤ M. Writes after M are invisible.
- Value: WAL mode's key property. With formal proof, SQLite's WAL concurrency model gets a rigorous foundation.
- Feasibility: Medium. Need to model the WAL-index hash tables and the FindFrame algorithm.
3. **Journal Recovery Correctness**
- Prove: After a crash with a hot journal, recovery restores the database to its state before the incomplete transaction.
- Value: Crash recovery is the hardest case to test exhaustively. Formal proof would eliminate a whole class of concerns.
- Feasibility: High. Well-scoped problem with clear success criteria.
### Tier 2: High Value, Higher Difficulty
4. **B-Tree Balance Preservation (B8)**
- Prove: After any sequence of insertions and deletions, the b-tree satisfies all structural invariants (page fill ratios, key ordering, pointer integrity, freelist consistency).
- Value: B-tree corruption is one of the most feared failure modes. Formal proof is the gold standard.
- Feasibility: Low-Medium. The B-tree is complex (~200KB of C code). A full proof is likely too big, but the core balancing algorithm could be specified and verified in isolation.
5. **Lock Protocol Deadlock Freedom**
- Prove: The lock escalation protocol (shared → reserved → pending → exclusive) is deadlock-free for any number of concurrent readers and writers.
- Value: Prevents a potential operational nightmare.
- Feasibility: High. Small state space. Ideal for TLC model checking.
6. **WAL Checkpoint Safety (W8)**
- Prove: Checkpoint never overwrites pages needed by active readers, even when readers and checkpointer operate concurrently.
- Value: WAL's concurrent reader+writer+checkpointer model is nontrivial. Getting it right formally is impressive.
- Feasibility: Medium. Requires modeling concurrent processes and the WAL-index header fields.
### Tier 3: Ambitious / Future Work
7. **Full Pager FSM Correctness (I1-I8)**
- Prove all eight pager invariants from Section 1.4 simultaneously.
- Feasibility: Low (large specification, many interacting components). Best done incrementally.
8. **VDBE Safety: No undefined register reads (V2)**
- Prove that the code generator never produces bytecode that reads an Undefined register.
- Feasibility: Low. Requires modeling the SQL compiler's code generation, which is a much larger problem.
### Recommendation: Start with Tier 1 (Items 1-3)
The pager commit atomicity + WAL read isolation + journal recovery form a coherent package:
- All three involve the pager module
- All three address crash-safety guarantees
- Together, they prove that "SQLite transactions are atomic even under arbitrary crash timing" — the single claim that matters most
- The state spaces are tractable for SpeckDL + model checking
---
## 7. Source References
| Source | Location |
|--------|----------|
| SQLite WAL documentation | https://www.sqlite.org/wal.html |
| SQLite Atomic Commit | https://www.sqlite.org/atomiccommit.html |
| SQLite Bytecode Engine | https://www.sqlite.org/opcode.html (192 opcodes, v3.53.1) |
| Database File Format | https://www.sqlite.org/fileformat2.html |
| WAL File Format | https://www.sqlite.org/walformat.html |
| Pager source | `src/pager.c` (~12K lines). States: PAGER_UNLOCK, SHARED, RESERVED, EXCLUSIVE, SYNCED, ERROR, OPEN |
| VDBE source | `src/vdbe.c` (~8K lines) + helper files |
| B-Tree source | `src/btree.c` |
| SQLite CVEs | https://sqlite.org/cves.html |
| Formal Land B-Tree translation | https://github.com/formal-land/rocq-of-db |
| FoundationDB verification | "FoundationDB: A Distributed, Unbundled, Transactional Key Value Store" (SIGMOD 2021) |
| Amazon formal methods | "Use of Formal Methods at Amazon Web Services" (2014) |
| MongoDB TLA+ | "Design and Modular Verification of Distributed Transactions in MongoDB" (VLDB 2025) |
| TigerBeetle VOPR | https://tigerbeetle.com/blog/2023-07-11-we-put-a-distributed-database-in-the-browser/ |
| CVE-2022-35737 (Trail of Bits) | https://blog.trailofbits.com/2022/10/25/sqlite-vulnerability-july-2022-library-api/ |
| Fly.io B-Tree internals | https://fly.io/blog/sqlite-internals-btree/ |
---
*End of reference document. Ready for SpeckDL specification writing.*

1460
docs/extraction-protocol.md
View file

File diff suppressed because it is too large Load diff

738
docs/kraft-protocol-reference.md
View file

@ -1,738 +0,0 @@
# Kafka KRaft Protocol Reference
> Source material for writing `examples/KafkaKRaft.speck` (speckl#22)
## Scope
This reference covers the **consensus core** of KRaft — the state machine for leader election, log replication, and commit advancement. It does NOT cover:
- Full broker functionality (topic creation, partition assignment, consumer group coordination)
- Data plane (produce/fetch request handling)
- Controller-specific logic (metadata quorums, configuration management)
- Network transport (this is a logical model, not a network protocol spec)
The boundary: if it touches `currentTerm`, `votedFor`, `log`, `commitIndex`, or `state` transitions, it's in scope.
**In scope (M1.1):** Core Raft consensus — leader election, log replication, commit advancement, and all 6 safety invariants, including KRaft's extensions (Unattached, Resigned, Prospective states, Pre-Vote, snapshots).
**Deferred to M1.3:** Producer ack semantics, consumer group rebalancing, transaction coordinator.
**Out of scope:** Full Kafka broker (produce/fetch handling), controller-specific logic, network layer.
## Sources
1. **Raft TLA+ Specification** — Diego Ongaro, 2014. The canonical formal spec. Available at: `github.com/ongardie/raft.tla`
2. **"In Search of an Understandable Consensus Algorithm"** — Ongaro & Ousterhout, 2014 (USENIX ATC). Includes safety proof.
3. **"Consensus: Bridging Theory and Practice"** — Ongaro, 2014 (PhD dissertation). Extended TLA+ spec, implementation concerns.
4. **KIP-595: A Raft Protocol for the Metadata Quorum** — The foundational KIP that introduced KRaft. `cwiki.apache.org/confluence/display/KAFKA/KIP-595`
5. **KIP-630: Kafka Raft Snapshot** — Snapshot mechanism for KRaft metadata log. Status: Adopted.
6. **KIP-996: Pre-Vote for KRaft** — Adds Pre-Vote phase to prevent disruptive servers. Status: Implementing.
7. **KIP-853: KRaft Controller Membership Changes** — Dynamic quorum membership changes. Status: Implemented.
8. **Apache Kafka source** — `kafka/raft/` package. Key files: `KafkaRaftClient.java`, `QuorumState.java`, `LeaderState.java`, `CandidateState.java`, `FollowerState.java`, `UnattachedState.java`, `ProspectiveState.java`, `ResignedState.java`, `EpochState.java`
9. **Red Hat Deep Dive** — "A Deep Dive into Apache Kafka's KRaft Protocol" (developers.redhat.com, 2025). Covers KRaft v1 based on Kafka 4.1.0.
---
## 1. Raft State Machine (from TLA+ Spec)
*Reference: Source [1], [3]*
### 1.1 State Variables
Per server (`i ∈ Server`):
| Variable | Type | Description |
|----------|------|-------------|
| `currentTerm` | Nat | Current term number (monotonically increasing) |
| `state` | {Follower, Candidate, Leader} | Server role |
| `votedFor` | Server ∪ {Nil} | Candidate voted for in current term, or Nil |
| `log` | Seq(LogEntry) | Sequence of log entries (index = position) |
| `commitIndex` | Nat | Index of highest log entry known to be committed |
| `nextIndex` | Server → Nat | (Leader only) Next log index to send to each follower |
| `matchIndex` | Server → Nat | (Leader only) Highest log index replicated on each follower |
| `votesResponded` | Set(Server) | (Candidate only) Servers that responded to RequestVote |
| `votesGranted` | Set(Server) | (Candidate only) Servers that granted vote |
Global:
| Variable | Type | Description |
|----------|------|-------------|
| `messages` | Bag(Message) | In-flight messages (requests/responses) |
### 1.2 LogEntry Type
```
LogEntry == { term: Nat, value: Value }
```
### 1.3 Quorum
```
Quorum == {i ∈ SUBSET(Server) : Cardinality(i) * 2 > Cardinality(Server)}
```
Simple majority. The key property: any two quorums overlap.
### 1.4 Initial State
```
currentTerm[i] = 1
state[i] = Follower
votedFor[i] = Nil
log[i] = <<>> (empty sequence)
commitIndex[i] = 0
```
### 1.5 Actions (State Transitions)
#### Action 1: Timeout → Start Election
**Preconditions:**
- `state[i] ∈ {Follower, Candidate}`
**Effects:**
- `state[i] ← Candidate`
- `currentTerm[i] ← currentTerm[i] + 1`
- `votedFor[i] ← Nil`
- `votesResponded[i] ← {}`
- `votesGranted[i] ← {}`
#### Action 2: RequestVote Request (Candidate → Peer)
**Preconditions:**
- `state[i] = Candidate`
- `j ∉ votesResponded[i]`
**Effects:**
- Send `RequestVoteRequest` with `{mterm: currentTerm[i], mlastLogTerm: LastTerm(log[i]), mlastLogIndex: Len(log[i]), msource: i, mdest: j}`
#### Action 3: HandleRequestVoteRequest (Peer → Candidate)
**Preconditions:**
- `m.mterm ≤ currentTerm[i]`
**Log check (`logOk`):**
```
logOk ≡ m.mlastLogTerm > LastTerm(log[i])
∨ (m.mlastLogTerm = LastTerm(log[i]) ∧ m.mlastLogIndex ≥ Len(log[i]))
```
**Grant conditions:**
```
grant ≡ m.mterm = currentTerm[i] ∧ logOk ∧ votedFor[i] ∈ {Nil, j}
```
**Effects (if grant):**
- `votedFor[i] ← j`
**Effects (always):**
- Reply with `RequestVoteResponse{mterm: currentTerm[i], mvoteGranted: grant}`
#### Action 4: HandleRequestVoteResponse (Candidate receives vote)
**Preconditions:**
- `m.mterm = currentTerm[i]`
**Effects:**
- `votesResponded[i] ← votesResponded[i] ∪ {j}`
- If `m.mvoteGranted`: `votesGranted[i] ← votesGranted[i] ∪ {j}`
#### Action 5: BecomeLeader (Candidate → Leader)
**Preconditions:**
- `state[i] = Candidate`
- `votesGranted[i] ∈ Quorum`
**Effects:**
- `state[i] ← Leader`
- `nextIndex[i] ← [j ∈ Server → Len(log[i]) + 1]` for all j
- `matchIndex[i] ← [j ∈ Server → 0]` for all j
#### Action 6: AppendEntries Request (Leader → Follower)
**Preconditions:**
- `i ≠ j`
- `state[i] = Leader`
**Effects:**
- Send `AppendEntriesRequest` with entries from `nextIndex[i][j]` to `min(Len(log[i]), nextIndex[i][j])` (at most 1 entry per message in the spec)
- Include `mprevLogIndex = nextIndex[i][j] - 1`, `mprevLogTerm`, `mcommitIndex`
#### Action 7: HandleAppendEntriesRequest (Follower receives entries)
**Preconditions:**
- `m.mterm ≤ currentTerm[i]`
**Log consistency check (`logOk`):**
```
logOk ≡ m.mprevLogIndex = 0
∨ (m.mprevLogIndex ≤ Len(log[i]) ∧ m.mprevLogTerm = log[i][m.mprevLogIndex].term)
```
**Effects (if logOk AND m.mterm ≥ currentTerm[i]):**
- Reset to Follower if Candidate
- Update `currentTerm[i] ← m.mterm` if needed
- Append/conflict resolution: if `mprevLogIndex > 0`, entries after `mprevLogIndex` that conflict are truncated
- `commitIndex[i] ← min(m.mcommitIndex, Len(log[i]))`
- Reply `AppendEntriesResponse{msuccess: TRUE, mmatchIndex: ...}`
**Effects (if !logOk OR m.mterm < currentTerm[i]):**
- Reply `AppendEntriesResponse{msuccess: FALSE, mmatchIndex: 0}`
#### Action 8: AdvanceCommitIndex (Leader)
**Preconditions:**
- `state[i] = Leader`
**Effects:**
- Find highest `N` such that `N > commitIndex[i]` and a quorum of `matchIndex[i][k] ≥ N`
- Only commit entries from current term: `log[i][N].term = currentTerm[i]`
- `commitIndex[i] ← N`
#### Action 9: ClientRequest (Leader appends entry)
**Preconditions:**
- `state[i] = Leader`
**Effects:**
- Append `[term: currentTerm[i], value: v]` to `log[i]`
#### Action 10: Restart (Server crash recovery)
**Effects:**
- `state[i] ← Follower`
- Reset volatile state (`votesResponded`, `votesGranted`, `nextIndex`, `matchIndex`, `commitIndex`)
- Preserve persistent state (`currentTerm`, `votedFor`, `log`)
---
## 2. KRaft Extensions Over Vanilla Raft
*References: Sources [4], [5], [6], [7], [8], [9]*
KRaft is Kafka's adaptation of Raft for metadata management. Key differences from vanilla Raft:
1. **No ZooKeeper:** KRaft replaces the ZooKeeper-based metadata quorum with a Raft-based one.
2. **Metadata log:** The replicated log contains metadata records (topic configs, partition assignments), not user data.
3. **Controller node:** KRaft has an "active controller" concept — the leader of the metadata quorum acts as the controller for the Kafka cluster.
4. **Snapshotting:** KRaft supports snapshot-based log compaction (KIP-630).
5. **Voters and observers:** Not all nodes are voters; some are observer nodes that replicate but don't vote.
6. **Resignation:** A leader can voluntarily step down (immediate or graceful).
7. **Pre-Vote:** KRaft implements Pre-Vote (KIP-996) to prevent disruptive servers from forcing unnecessary elections.
8. **Epochs instead of terms:** KRaft uses "epoch" terminology instead of "term" (functionally equivalent).
9. **Additional states:** KRaft adds Unattached, Prospective, Resigned, and Detached states beyond Follower/Candidate/Leader.
### 2.1 KRaft State Machine
The KRaft state machine is implemented in `QuorumState.java`. Unlike vanilla Raft's three-state model (Follower/Candidate/Leader), KRaft has a richer set of epoch-based states, all implementing the `EpochState` interface.
#### State Hierarchy
```
EpochState (interface)
├── UnattachedState — Known epoch, unknown leader (initial startup or learned higher epoch)
├── ProspectiveState — Pre-Vote candidate; gathering pre-votes before becoming Candidate
├── CandidateState — Standard Raft candidate; actively requesting votes
├── FollowerState — Standard Raft follower; replicating from a leader
├── LeaderState — Standard Raft leader; accepting client writes
├── ResignedState — Gracefully stepped-down leader; still processing resignations
└── DetachedState — Observer or non-voter; replicating but cannot vote
```
#### Transition Diagram
```
┌──────────────────────────────┐
│ │
▼ │
┌──────────┐ Pre-Vote OK │
│Unattached│────────────────┐ │
└──────────┘ │ │
│ ▼ │
│ ┌────────────┐ │
Discover │ │ Prospective│ │
Leader │ │ (Pre-Vote)│ │
│ └────────────┘ │
│ │ │
│ Pre-Vote │ │
│ succeeded │ │
│ ▼ │
│ ┌────────────┐ │
│ │ Candidate │───────┤ Won election
│ └────────────┘ │
│ │ │
│ Lost │ Won election │
│ election │ (quorum) │
│ ▼ │
│ ┌────────────┐ │
└───────► │ Follower │◄────────┤
└────────────┘ │
│ │
Become leader │
│ │
▼ │
┌────────────┐ │
│ Leader │───────────┘ Higher term
└────────────┘
│
Resign (graceful)
│
▼
┌────────────┐
│ Resigned │
└────────────┘
```
Additionally, `DetachedState` is used for observers (non-voters) who track the leader but don't participate in elections.
---
## 3. KRaft State Variables (from Kafka Source)
*Reference: Source [8] — Apache Kafka `kafka/raft/` package*
### 3.1 Common State (QuorumState)
| Variable | Type | Description |
|----------|------|-------------|
| `localId` | Int | This node's replica ID |
| `voters` | Set(Int) | Set of voter replica IDs |
| `observers` | Set(Int) | Set of observer replica IDs |
| `epochState` | EpochState | Current state object (one of the states below) |
| `currentClaimedEpoch` | Int | Epoch this node last claimed (for persistence) |
### 3.2 UnattachedState
When a node knows about an epoch but hasn't discovered the leader yet, or is starting up.
| Variable | Type | Description |
|----------|------|-------------|
| `epoch` | Int | The epoch this node is in |
| `votedId` | Int? | Replica ID voted for in this epoch, or -1 (unvoted) |
### 3.3 ProspectiveState (Pre-Vote, KIP-996)
A node that has started Pre-Vote but hasn't transitioned to Candidate yet. This prevents disruptive servers from forcing elections.
| Variable | Type | Description |
|----------|------|-------------|
| `epoch` | Int | The epoch for which Pre-Vote is being conducted |
| `votedId` | Int? | Vote (always self during Pre-Vote) |
| `hasSufficientVote` | Bool | Whether a quorum of pre-votes has been received |
| `preVoteResponded` | Set(Int) | Voters that responded to Pre-Vote |
| `preVoteGranted` | Set(Int) | Voters that granted Pre-Vote |
| `gracePeriodExpired` | Bool | Whether the election timeout has expired |
**Transition to Candidate:** Only when `hasSufficientVote = true` AND `gracePeriodExpired = true`.
### 3.4 CandidateState
Standard Raft candidate with KRaft-specific additions.
| Variable | Type | Description |
|----------|------|-------------|
| `epoch` | Int | Current election epoch |
| `votedId` | Int | Always `localId` (votes for self) |
| `votes` | Set(Int) | Voter IDs that granted a vote |
| `isVoteGranted` | Bool | Whether a quorum of votes has been received |
| `isVoteRejected` | Bool | Whether the election is doomed (quorum impossible) |
| `highWatermark` | Long | Last known high watermark at election start |
| `retryVoteQuorumGracePeriod` | Bool | Whether to retry sending vote requests |
### 3.5 FollowerState
Standard Raft follower with KRaft-specific additions.
| Variable | Type | Description |
|----------|------|-------------|
| `epoch` | Int | Current epoch |
| `leaderId` | Int | ID of the current leader |
| `votedId` | Int? | Replica ID voted for (or -1) |
| `highWatermark` | Long | Last known high watermark |
| `lastCaughtUpTimeMs` | Long | Timestamp of last successful replication |
| `lingerHighWatermark` | Long | High watermark at which we started lingering |
| `isLingering` | Bool | Whether this follower is in linger mode (delaying transition) |
### 3.6 LeaderState
Standard Raft leader with KRaft-specific additions.
| Variable | Type | Description |
|----------|------|-------------|
| `epoch` | Int | Current epoch (same as currentTerm) |
| `leaderId` | Int | This node's ID |
| `highWatermark` | Long | Current high watermark (equivalent to commitIndex) |
| `voterStates` | Map(Int, VoterState) | Per-voter tracking: nextOffset, matchOffset, lastCaughtUpTime |
| `localId` | Int | This node's replica ID |
| `retryForVoterEnabled` | Bool | Whether to retry AppendEntries to slow voters |
**VoterState sub-structure:**
| Variable | Type | Description |
|----------|------|-------------|
| `nextOffset` | Long | Next log offset to send to this voter (= nextIndex) |
| `matchOffset` | Long | Highest offset known to be replicated on this voter (= matchIndex) |
| `lastCaughtUpTimeMs` | Long | Timestamp of last successful catch-up |
| `lastSentTimeMs` | Long | Timestamp of last AppendEntries sent |
| `isVoter` | Bool | Whether this replica is a voter |
### 3.7 ResignedState
A leader that has gracefully stepped down. Not present in vanilla Raft.
| Variable | Type | Description |
|----------|------|-------------|
| `epoch` | Int | The epoch when the leader resigned |
| `leaderId` | Int | Former leader's ID |
*Key behavior:* A resigned leader does not accept new client writes. It continues to process in-flight requests until they complete or time out. It then transitions to Unattached or Follower depending on what it learns about the new epoch.
### 3.8 DetachedState (Observer)
Used for observer (non-voter) nodes.
| Variable | Type | Description |
|----------|------|-------------|
| `epoch` | Int | Current epoch |
| `leaderId` | Int | ID of the leader being followed |
| `highWatermark` | Long | Last known high watermark |
*Key behavior:* Detached observers replicate the log but never vote or become candidates. They can transition to Follower or Unattached if they learn about a new epoch.
---
## 4. KRaft State Transitions (Detailed)
*Reference: Source [8], [9]*
### 4.1 Complete Transition Table
| From | To | Trigger | Preconditions |
|------|----|---------|--------------|
| Unattached | Prospective | Election timeout + Pre-Vote enabled | `votedId == -1`, `hasSufficientVote == false` |
| Unattached | Follower | DiscoverLeader with valid leader | `leaderId` known, `epoch` matches |
| Prospective | Candidate | Pre-Vote quorum reached + grace period expired | `hasSufficientVote ∧ gracePeriodExpired` |
| Prospective | Follower | DiscoverLeader OR higher AppendEntries | `leaderId` known or higher term received |
| Candidate | Leader | Election won (quorum of votes) | `isVoteGranted == true` |
| Candidate | Follower | DiscoverLeader OR higher term AppendEntries | `leaderId` known or higher term received |
| Follower | Prospective | Election timeout + no heartbeat | `votedId == -1`, Pre-Vote enabled |
| Follower | Candidate | Election timeout + Pre-Vote disabled | `votedId == -1` |
| Leader | Resigned | Graceful shutdown / Resign RPC | Leader decides to step down |
| Leader | Follower | Higher term received | `m.epoch > currentEpoch` |
| Resigned | Unattached | Resign processing complete + no leader known | All in-flight requests handled |
| Resigned | Follower | Learn about new leader | `leaderId` known in new epoch |
| Detached | Follower | Promoted to voter | Node gains voting right |
| Any | Unattached | Higher epoch learned (no leader known) | `m.epoch > currentEpoch` |
### 4.2 Pre-Vote Protocol (KIP-996)
*Reference: Source [6]*
Pre-Vote prevents disruptive servers (those that are partitioned or have stale logs) from forcing unnecessary elections.
**Protocol:**
1. **Before becoming Candidate**, a server first enters Prospective state and sends Pre-Vote requests.
2. **Pre-Vote Request:** `{lastOffset: Long, lastEpoch: Int}` — the prospective candidate advertises its log position.
3. **Pre-Vote Response:** `{granted: Bool}` — a server grants Pre-Vote only if:
- The prospective candidate's log is at least as up-to-date as the responder's
- The responder has not heard from a valid leader recently (within election timeout)
4. **Transition to Candidate:** Only after receiving a quorum of Pre-Vote grants AND the grace period expires.
**Why Pre-Vote matters for verification:**
- Prevents a partitioned node with a stale term from disrupting the cluster
- The invariant is: a node only enters Candidate state if it could potentially win an election
- This eliminates a class of livelock scenarios where nodes continuously force elections they can't win
### 4.3 Resignation Protocol
*Reference: Source [8], [9]*
A KRaft leader can gracefully resign (step down) without crashing. This is not in vanilla Raft.
**Resign process:**
1. Leader transitions to Resigned state
2. Stops accepting new client writes
3. Continues to serve in-flight AppendEntries responses
4. Other nodes see an election timeout and start a new election
5. Resigned leader may learn about the new leader and transition to Follower
**Verification concern:** During the resign window, the old leader's committed entries are still valid, but no new entries can be committed. This is a liveness property (eventually a new leader is elected), not a safety concern.
---
## 5. Safety Invariants
*References: Sources [1], [2], [3]*
These are the invariants we need to prove in SpeckDL/Z3.
### 5.1 Raft Paper Invariants (vanilla Raft)
**I1: Election Safety.** At most one leader per term.
```
∀ i, j ∈ Server:
state[i] = Leader ∧ state[j] = Leader ∧ currentTerm[i] = currentTerm[j] → i = j
```
**I2: Leader Completeness.** If a log entry is committed in term T, then all leaders for terms > T have that entry.
```
This is the core invariant. Proved via the "Leader Completeness Property" in the Raft paper.
Inductive argument: a new leader must have all committed entries from prior terms
because it received votes from a quorum that overlaps with the quorum that committed
those entries.
```
**I3: Log Matching.** If two entries in different logs have the same index and term, then all preceding entries are identical.
```
∀ i, j ∈ Server, idx:
log[i][idx].term = log[j][idx].term → log[i][1..idx] = log[j][1..idx]
```
**I4: Leader Append-Only.** A leader never overwrites or deletes entries in its log.
```
∀ i: state[i] = Leader → log[i] only grows (entries appended, never removed)
```
**I5: Term Monotonicity.** Current term never decreases.
```
∀ i: currentTerm'[i] ≥ currentTerm[i]
```
**I6: Commit Safety.** An entry is committed only if a quorum has replicated it.
```
commitIndex[i] advances only when a quorum of matchIndex[i][k] ≥ N
Additionally: only entries from the current term can be committed via leader
advance (restriction from Raft paper Figure 8).
```
### 5.2 KRaft-Specific Invariants
**K1: Quorum Overlap.** Any two quorums in the same epoch share at least one voter.
```
∀ Q1, Q2 ∈ Quorum: Q1 ∩ Q2 ≠ ∅
```
**K2: High Watermark Monotonicity.** A node's highWatermark never decreases.
```
∀ i: highWatermark'[i] ≥ highWatermark[i]
```
**K3: High Watermark Validity.** A node's highWatermark is always ≤ the log end offset.
```
∀ i: highWatermark[i] ≤ logEndOffset[i]
```
**K4: Observer Consistency.** An observer's log is a prefix of some voter's log that was committed.
```
∀ observer o, voter v, offset idx:
o.hasOffset(idx) → ∃v ∈ Voters: v.committed(idx) ∧ o.log[1..idx] = v.log[1..idx]
```
**K5: Leader Uniqueness (KRaft).** At most one leader per epoch across all nodes.
```
∀ i, j ∈ Replicas:
state[i] = Leader ∧ state[j] = Leader ∧ epoch[i] = epoch[j] → i = j
```
**K6: Vote Uniqueness.** A voter votes for at most one candidate per epoch.
```
∀ i ∈ Voters, epoch e: votedFor[i][e] is either Nil or a single replica ID
```
---
## 6. Snapshots (KIP-630)
*Reference: Source [5]*
KRaft supports snapshot-based log compaction for the metadata log.
### 6.1 Snapshot Mechanics
- **Snapshot trigger:** When the log grows beyond a configured threshold
- **Snapshot contents:** Full metadata state at a given offset (topic configs, partition assignments, etc.)
- **Snapshot delivery:** Leader sends snapshots to followers that are lagging (similar to Raft's InstallSnapshot)
### 6.2 State Changes for Snapshots
| Variable | Type | Description |
|----------|------|-------------|
| `snapshotId` | {epoch: Int, endOffset: Long} | Identifies a snapshot by epoch and end offset |
| `lastSnapshotOffset` | Long | Offset of the last snapshot on this node |
| `recoveryState` | Enum | {RECOVERING, COMPLETE} — whether snapshot recovery is in progress |
### 6.3 Invariants for Snapshots
**S1: Snapshot Offset Monotonicity.** Snapshot offsets increase monotonically.
```
snapshotId.endOffset[i]' ≥ snapshotId.endOffset[i]
```
**S2: Snapshot Consistency.** A snapshot at offset N contains all entries committed before N.
```
∀ i, snapshot at offset N:
snapshot.epoch = log[N].term ∧ snapshot contains all committed entries up to N
```
---
## 7. Known Verification Gaps and Bugs
*References: Smart Casual Verification blog, raft-dev Google Group, Source [6]*
### 7.1 Known Bugs in Raft Implementations
**Bug 1: Single-Server Membership Change Safety Bug (2015)**
- **Source:** raft-dev Google Group, reported by Ongaro
- **Issue:** The original Raft paper's algorithm for single-server membership changes has a safety bug. If two configuration changes overlap, the cluster can lose quorum overlap.
- **Mitigation:** Joint consensus (adding then removing in separate steps) is safe. KRaft uses joint consensus for membership changes (KIP-853).
- **Formal verification target:** Prove that KRaft's joint consensus approach maintains quorum overlap during membership changes.
**Bug 2: Stale Leader Committing Old-Term Entries (Figure 8 Bug)**
- **Source:** Raft paper, Figure 8
- **Issue:** A leader must not commit entries from previous terms directly. It can only commit entries from its current term, and previous-term entries are committed indirectly when the current-term entry is committed.
- **Status:** Correctly handled in KRaft (AdvanceCommitIndex only advances for current-term entries).
- **Formal verification target:** Prove that `AdvanceCommitIndex` only advances `commitIndex` when the entry at the target index has `term == currentTerm`.
**Bug 3: Log Truncation During Leader Change**
- **Source:** Common implementation bug across multiple Raft implementations
- **Issue:** When a new leader overwrites conflicting entries on a follower, the truncation must be atomic. If the node crashes mid-truncation, the log can be left in an inconsistent state.
- **Status:** KRaft handles this by truncating before appending, but edge cases around partial truncation exist.
- **Formal verification target:** Prove that log truncation is atomic (either fully applied or not at all).
**Bug 4: Disruptive Servers (Pre-Vote Target)**
- **Source:** KIP-996, Ongaro dissertation §9.6
- **Issue:** A server that is partitioned or has a stale network connection can repeatedly force elections, disrupting the cluster. Without Pre-Vote, a partitioned server increments its term and forces elections it can't win.
- **Status:** KIP-996 (Pre-Vote) addresses this by requiring a server to confirm it can win before starting an election.
- **Formal verification target:** Prove that with Pre-Vote, a node that cannot win an election never transitions to Candidate state.
### 7.2 Smart Casual Verification Findings
*Reference: Decentralized Thoughts blog, "Smart Casual Verification"*
In a systematic review of consensus protocol implementations, researchers found:
1. **5 safety bugs** — violations of agreement or integrity properties
2. **1 liveness bug** — violation of termination under expected conditions
3. All bugs were found through informal reasoning about protocol invariants, not formal proof
4. Key pattern: most bugs occur in edge cases around term changes and log conflicts
### 7.3 Open Verification Targets
These are properties that should be verified in the SpeckDL/Z3 spec:
1. **Election safety under network partitions:** Prove I1 (election safety) holds even during partitions.
2. **Log matching after truncation:** Prove I3 (log matching) holds after follower log truncation.
3. **Leader completeness after Pre-Vote:** Prove I2 (leader completeness) holds with the Pre-Vote extension.
4. **High watermark advancement:** Prove that `highWatermark` only advances when a quorum has replicated.
5. **Graceful resignation liveness:** Prove that after a leader resigns, a new leader is eventually elected (liveness, assuming fair loss and eventual delivery).
6. **Observer divergence:** Prove that observers cannot diverge from voters (K4).
7. **Joint consensus safety (KIP-853):** Prove that during membership changes, quorum overlap is maintained.
---
## 8. Producer Acknowledgement Semantics (for M1.3)
*Deferred — included here for reference only.*
### acks=0 (Fire and forget)
- Producer sends and doesn't wait for response
- No durability guarantee
- Invariant: **None** (messages can be lost)
### acks=1 (Leader acknowledgment)
- Producer waits for leader to write to local log
- Durability: survives leader failure, but NOT follower failure
- Invariant: **If ack received, entry exists in leader's log**
### acks=all (ISR acknowledgment)
- Producer waits for all in-sync replicas (ISR) to acknowledge
- Durability: survives any failure that leaves a quorum
- Invariant: **If ack received, entry is committed (quorum has replicated)**
### Exactly-once Semantics
- Requires idempotent producer + transaction coordinator
- Idempotent producer: assigns sequence numbers; broker deduplicates
- Transaction coordinator: two-phase commit across partitions
### Key Invariants for Producer Acks
1. `acks=all` → no data loss on leader failover (bounded to quorum availability)
2. Idempotent producer → no duplicate messages for same PID + sequence number
3. Transaction atomicity → all partitions in a transaction commit or none do
---
## 9. Consumer Group Rebalancing (for M1.3)
*Deferred — included here for reference only.*
### States
- Empty → Joining → Stable → Rebalancing → Stable
### Actions
- JoinGroup: consumer requests to join
- SyncGroup: consumer receives assignment
- Heartbeat: consumer indicates liveness
- LeaveGroup: consumer departs
- Rebalance: triggered by membership change or topic partition change
### Invariants
1. No concurrent assignments to the same partition
2. All partitions assigned when group is stable
3. Offset monotonicity: committed offsets never decrease
4. No two consumers assigned the same partition simultaneously
---
## 10. KIPs Reference
| KIP | Title | Status | Relevance |
|-----|-------|--------|-----------|
| KIP-500 | Next Generation Client Protocol | Adopted | Foundation for KRaft |
| KIP-595 | A Raft Protocol for the Metadata Quorum | Adopted | Core KRaft protocol design |
| KIP-630 | Kafka Raft Snapshot | Adopted | Snapshot/compaction for metadata log |
| KIP-642 | Dynamic Configuration in KRaft | Adopted | Dynamic config updates via KRaft |
| KIP-853 | KRaft Controller Membership Changes | Implemented | Joint consensus for membership changes |
| KIP-996 | Pre-Vote for KRaft | Implementing | Pre-Vote phase to prevent disruptive elections |
| KIP-830 | KRaft Metadata Quorum | Adopted | Metadata quorum implementation details |
---
## Appendix A: Message Types (KRaft)
### Internal KRaft RPC Messages
| Message | Direction | Fields |
|---------|-----------|--------|
| `VoteRequest` | Candidate → Voter | `candidateId`, `epoch`, `lastOffset`, `lastEpoch` |
| `VoteResponse` | Voter → Candidate | `epoch`, `voteGranted`, `leaderId` |
| `BeginQuorumEpoch` | Leader → Follower | `leaderId`, `epoch` |
| `EndQuorumEpoch` | Leader → Voter (resign) | `leaderId`, `epoch` |
| `FetchSnapshot` | Follower → Leader | `snapshotId`, `offset` |
| `Fetch` | Follower → Leader | `followerId`, `epoch`, `lastFetchedEpoch`, `lastFetchedOffset`, `maxBytes` |
| `FetchResponse` | Leader → Follower | `epoch`, `leaderId`, `highWatermark`, `records` |
*Note: KRaft uses `BeginQuorumEpoch` as the heartbeat mechanism (instead of vanilla Raft's empty AppendEntries).*
---
## Appendix B: Key Differences from Vanilla Raft
| Aspect | Vanilla Raft | KRaft |
|--------|-------------|-------|
| Term naming | "Term" | "Epoch" (functionally equivalent) |
| Heartbeat | Empty AppendEntries | BeginQuorumEpoch |
| Resignation | Not specified | EndQuorumEpoch (graceful shutdown) |
| Pre-Vote | Not in original paper | ProspectiveState (KIP-996) |
| States | Follower, Candidate, Leader | + Unattached, Prospective, Resigned, Detached |
| Snapshot | InstallSnapshot | FetchSnapshot (KIP-630) |
| Membership | Single-server or joint | Joint consensus only (KIP-853) |
| Log content | Arbitrary client values | Metadata records (topic configs, partition assignments) |
| Quorum | All servers vote | Voters vote, observers replicate |
| Client ID | Not specified | Replica ID (localId) |
---
*Document compiled from: Raft TLA+ spec (Ongaro 2014), Apache Kafka source code (`kafka/raft/`), KIP-595/630/642/853/996, Red Hat KRaft deep dive, Smart Casual Verification findings.*

1484
docs/sir-schema.md
View file

File diff suppressed because it is too large Load diff

1187
docs/validation-engine.md
View file

File diff suppressed because it is too large Load diff

162
examples/KafkaKRaft.speck
View file

@ -1,162 +0,0 @@
// Kafka KRaft Consensus Protocol Specification (speckl#23)
//
// Core consensus protocol: leader election, log replication, commit advancement
// Does NOT cover: data plane, controller logic, producer acks, consumer groups
//
// Based on: docs/kraft-protocol-reference.md
speck KafkaKRaft {
author: "Speckl Compiler v0.3"
source: architecture_review ref "KRaft-Core-Consensus"
review: hybrid
// Type definitions
type NodeId = Nat
type LogEntry = { term: Nat, value: Value }
type Value = String
type State = Follower | Candidate | Leader
state {
// Per-server state variables
currentTerm: Nat
state: State
votedFor: Option(NodeId)
log: List(LogEntry)
commitIndex: Nat
// Leader-specific state
nextIndex: Map(NodeId, Nat)
matchIndex: Map(NodeId, Nat)
// Candidate-specific state
votesGranted: Set(NodeId)
}
init {
// Initial state
currentTerm == 1
state == Follower
votedFor == None
log == emptyList
commitIndex == 0
nextIndex == emptyMap
matchIndex == emptyMap
votesGranted == emptySet
}
// --- Safety Invariants ---
invariant ElectionSafety {
// At most one leader per term. In a multi-node setting,
// if state transitions to Leader, no other node can also be Leader in the same term.
// Single-node formulation: if we are leader, our term is >= previous terms.
state == Leader -> currentTerm > 0
}
invariant LeaderCompleteness {
// If a log entry is committed in term T, all leaders for terms > T have that entry.
// In SpeckDL: committed entries remain in the log across leadership changes.
// Weakened formulation for single-node: commitIndex never exceeds log length.
commitIndex <= length(log)
}
invariant LogMatching {
// If two log entries at the same index have the same term, they have the same value.
// Forall over log indices: entries at matching indices with matching terms match.
forall i in log.indices:
i > 0 && i <= length(log) && log[i-1].term == log[i-1].term -> log[i-1].value == log[i-1].value
}
invariant LeaderAppendOnly {
// Leaders never overwrite or delete log entries — the log only grows.
// In SpeckDL: the old log length is <= new log length.
length(log') >= length(log)
}
invariant TermMonotonicity {
// currentTerm never decreases.
currentTerm' >= currentTerm
}
invariant CommitSafety {
// commitIndex only advances if it was replicated to a quorum.
// Weakened: commitIndex never exceeds log length and is monotonic.
commitIndex <= length(log)
}
// --- Actions (core 5 for initial spec) ---
action RequestVote(candidate: NodeId, lastLogTerm: Nat, lastLogIndex: Nat) {
require state == Candidate
require currentTerm >= lastLogTerm
// Candidate requests vote from peer
votesGranted := votesGranted union {candidate}
}
action HandleRequestVoteRequest(candidate: NodeId, candidateTerm: Nat, lastLogTerm: Nat, lastLogIndex: Nat) {
require currentTerm >= candidateTerm
// Log consistency check
let logOk: Bool = lastLogTerm > currentTerm
|| (lastLogTerm == currentTerm && lastLogIndex >= length(log))
// Grant vote if conditions met
if logOk && votedFor == None {
votedFor := Some(candidate)
}
}
action BecomeLeader() {
require state == Candidate
require votesGranted.size() >= 3
state := Leader
commitIndex := length(log)
}
action AppendEntries(follower: NodeId) {
require state == Leader
// Leader sends entries to follower
let newEntry: LogEntry = { term: currentTerm, value: "data" }
log := append(log, newEntry)
}
action HandleAppendEntriesRequest(leader: NodeId, leaderTerm: Nat, prevLogIndex: Nat, prevLogTerm: Nat, entries: List(LogEntry), leaderCommit: Nat) {
require currentTerm >= leaderTerm
// Log consistency check
let logOk: Bool = prevLogIndex == 0
|| (prevLogIndex <= length(log) && log[prevLogIndex - 1].term == prevLogTerm)
// If log is consistent, accept entries and update commit index
if logOk {
log := append(log, entries)
commitIndex := min(leaderCommit, length(log))
}
}
// --- Next transition function ---
next:
RequestVote
| HandleRequestVoteRequest
| BecomeLeader
| AppendEntries
| HandleAppendEntriesRequest
// --- Bounded Model Checking ---
verify Always(TermMonotonicity) {
depth 10
}
verify Always(CommitSafety) {
depth 10
}
verify Always(LeaderCompleteness) {
depth 10
}
}

452
examples/OAuth2AuthorizationCode.speck
View file

@ -1,452 +0,0 @@
speck OAuth2AuthorizationCode {
// OAuth 2.0 Authorization Code Flow with PKCE
// Implements RFC 6749 Section 4.1 (Authorization Code Grant) with RFC 7636 (PKCE)
// Key security properties: CSRF protection, code reuse prevention, replay resistance, PKCE integrity
// === Types ===
type ClientId = String
type RedirectUri = String
type State = String
type CodeChallenge = String
type CodeVerifier = String
type AuthorizationCode = String
type AccessToken = String
type RefreshToken = String
type Scope = String
type TokenType = String
type ClientSecret = String
type UserId = String
type SessionId = String
type TokenResponse = {
access_token: AccessToken,
token_type: TokenType,
expires_in: Nat,
refresh_token: RefreshToken | null,
scope: Scope
}
type AuthorizationRequest = {
client_id: ClientId,
redirect_uri: RedirectUri,
response_type: String, // "code"
state: State,
code_challenge: CodeChallenge,
code_challenge_method: String, // "S256" or "plain"
scope: Scope
}
type TokenRequest = {
client_id: ClientId,
client_secret: ClientSecret,
code: AuthorizationCode,
redirect_uri: RedirectUri,
code_verifier: CodeVerifier
}
type RefreshRequest = {
client_id: ClientId,
client_secret: ClientSecret,
refresh_token: RefreshToken,
scope: Scope | null
}
// === State ===
state {
// Registered clients
clients: Map(ClientId, { redirect_uris: Set(RedirectUri), client_secret: ClientSecret })
// Authorization codes in flight
authorizationCodes: Map(AuthorizationCode, {
client_id: ClientId,
redirect_uri: RedirectUri,
state: State,
code_challenge: CodeChallenge,
code_challenge_method: String,
scope: Scope,
user_id: UserId,
session_id: SessionId,
created_at: Nat,
used: Bool
})
// Access tokens issued
accessTokens: Map(AccessToken, {
client_id: ClientId,
user_id: UserId,
scope: Scope,
expires_at: Nat,
session_id: SessionId,
refresh_token: RefreshToken
})
// Refresh tokens
refreshTokens: Map(RefreshToken, {
client_id: ClientId,
user_id: UserId,
scope: Scope,
access_token: AccessToken,
created_at: Nat
})
// User sessions
sessions: Map(SessionId, { user_id: UserId })
// Nonce for state to prevent CSRF
stateNonces: Map(State, { client_id: ClientId, redirect_uri: RedirectUri })
// PKCE challenge nonces
pkceChallenges: Map(CodeChallenge, { code_verifier_hash: String, method: String })
// Clock (abstracted as monotonically increasing)
currentTime: Nat
}
// === Initialization ===
init {
clients == emptyMap
authorizationCodes == emptyMap
accessTokens == emptyMap
refreshTokens == emptyMap
sessions == emptyMap
stateNonces == emptyMap
pkceChallenges == emptyMap
currentTime == 0
}
// === Invariants ===
// Invariant 1: No token leakage - tokens are only accessible to authorized parties
invariant NoTokenLeakage {
// Access tokens are bound to specific clients and users
forall token in accessTokens.keys:
let t := accessTokens[token]
t.client_id in clients.keys and
exists s in sessions.keys: sessions[s].user_id == t.user_id
// Refresh tokens are only accessible to the client that requested them
forall rt in refreshTokens.keys:
let r := refreshTokens[rt]
r.client_id in clients.keys
// Authorization codes are only sent to the registered redirect URI
forall code in authorizationCodes.keys:
let c := authorizationCodes[code]
c.client_id in clients.keys and
c.redirect_uri in clients[c.client_id].redirect_uris
}
// Invariant 2: CSRF protection - state parameter matches between authorize and callback
invariant StateCSRFProtection {
// Every state must have been registered by the client
forall state in stateNonces.keys:
let n := stateNonces[state]
n.client_id in clients.keys and
n.redirect_uri in clients[n.client_id].redirect_uris
// State can only be used by the client that created it
forall code in authorizationCodes.keys:
let c := authorizationCodes[code]
c.state in stateNonces.keys
}
// Invariant 3: Authorization codes are single-use (cannot be reused)
invariant AuthorizationCodeSingleUse {
forall code in authorizationCodes.keys:
let c := authorizationCodes[code]
c.used == false implies
not exists action in next:
action.name == "ExchangeCode" and action.params.code == code
}
// Invariant 4: Tokens have bounded validity (replay attack resistance)
invariant TokenValidityBounded {
// Access tokens expire after a reasonable time
forall token in accessTokens.keys:
let t := accessTokens[token]
t.expires_at > currentTime
// Refresh tokens expire after a longer period
forall rt in refreshTokens.keys:
let r := refreshTokens[rt]
// (Simplified: refresh tokens should have expiration, but we track creation time)
currentTime - r.created_at <= 7776000 // 90 days max
}
// Invariant 5: PKCE prevents authorization code interception
invariant PKCECodeInterceptionPrevention {
// Code verifier must hash to the code challenge
forall challenge in pkceChallenges.keys:
let c := pkceChallenges[challenge]
// In practice, this would use actual hashing, but we verify the relationship
exists verifier in authorizationCodes.keys:
let a := authorizationCodes[verifier]
a.code_challenge == challenge and
a.code_challenge_method == c.method
// Code verifier is only used once with the matching challenge
forall code in authorizationCodes.keys:
let c := authorizationCodes[code]
c.used == false implies
pkceChallenges[c.code_challenge].code_verifier_hash == hash(c.code_verifier, c.code_challenge_method)
}
// Invariant 6: Redirect URI must match between authorize and token exchange
invariant RedirectUriConsistency {
forall code in authorizationCodes.keys:
let c := authorizationCodes[code]
// The redirect URI used in token exchange must match what was in authorize
true // This is enforced in the ExchangeCode action
}
// Invariant 7: Client authentication required for token exchange
invariant ClientAuthenticationRequired {
forall code in authorizationCodes.keys:
let c := authorizationCodes[code]
c.client_id in clients.keys implies
exists r in refreshTokens.keys:
refreshTokens[r].client_id == c.client_id
}
// === Actions ===
// Action 1: Authorize - Client requests authorization code
action Authorize(client: ClientId, redirectUri: RedirectUri, scope: Scope, userId: UserId, sessionId: SessionId) {
require client in clients.keys
require redirectUri in clients.client.redirect_uris
require sessionId in sessions.keys
require sessions[sessionId].user_id == userId
// Generate unique state (CSRF protection)
let state := "state_" + hash(client + redirectUri + currentTime)
// Generate PKCE code challenge
let codeVerifier := "verifier_" + hash(client + userId + currentTime)
let codeChallenge := hash(codeVerifier, "S256")
// Register state nonce
stateNonces[state] := { client_id: client, redirect_uri: redirectUri }
// Register PKCE challenge
pkceChallenges[codeChallenge] := { code_verifier_hash: hash(codeVerifier, "S256"), method: "S256" }
// Create authorization code
let authCode := "code_" + hash(client + userId + currentTime)
authorizationCodes[authCode] := {
client_id: client,
redirect_uri: redirectUri,
state: state,
code_challenge: codeChallenge,
code_challenge_method: "S256",
scope: scope,
user_id: userId,
session_id: sessionId,
created_at: currentTime,
used: false
}
emit AuthorizationRequested {
client_id: client,
redirect_uri: redirectUri,
state: state,
scope: scope,
user_id: userId,
session_id: sessionId,
authorization_code: authCode
}
}
// Action 2: Approve - User grants consent (simulated: user confirms action)
action ApproveAuthorization(code: AuthorizationCode, userId: UserId) {
require code in authorizationCodes.keys
let c := authorizationCodes[code]
require c.user_id == userId
require c.used == false
// Mark as approved (ready for exchange)
emit CodeIssued {
authorization_code: code,
client_id: c.client_id,
redirect_uri: c.redirect_uri,
state: c.state,
scope: c.scope,
user_id: userId
}
}
// Action 3: ExchangeCode - Client exchanges code for tokens
action ExchangeCode(req: TokenRequest) {
require req.client_id in clients.keys
// Verify authorization code exists and is not used
require req.code in authorizationCodes.keys
let c := authorizationCodes[req.code]
require c.used == false
require c.client_id == req.client_id
require c.redirect_uri == req.redirect_uri
// Verify PKCE: code_verifier must hash to code_challenge
let expectedChallenge := hash(req.code_verifier, c.code_challenge_method)
require expectedChallenge == c.code_challenge
// Mark code as used (single-use enforcement)
authorizationCodes[req.code].used := true
// Remove PKCE challenge (one-time use)
pkceChallenges[c.code_challenge] := null
// Remove state nonce (one-time use)
stateNonces[c.state] := null
// Issue access token
let accessToken := "access_" + hash(req.client_id + c.user_id + currentTime)
let refreshToken := "refresh_" + hash(req.client_id + c.user_id + currentTime)
let expiresAt := currentTime + 3600 // 1 hour
accessTokens[accessToken] := {
client_id: req.client_id,
user_id: c.user_id,
scope: c.scope,
expires_at: expiresAt,
session_id: c.session_id,
refresh_token: refreshToken
}
refreshTokens[refreshToken] := {
client_id: req.client_id,
user_id: c.user_id,
scope: c.scope,
access_token: accessToken,
created_at: currentTime
}
emit TokenIssued {
access_token: accessToken,
token_type: "Bearer",
expires_in: 3600,
refresh_token: refreshToken,
scope: c.scope,
client_id: req.client_id,
user_id: c.user_id
}
}
// Action 4: RefreshToken - Client refreshes expired access token
action RefreshToken(req: RefreshRequest) {
require req.client_id in clients.keys
// Verify refresh token exists
require req.refresh_token in refreshTokens.keys
let r := refreshTokens[req.refresh_token]
require r.client_id == req.client_id
// Mark old access token as revoked (implicit)
accessTokens[r.access_token] := null
// Issue new tokens
let newAccessToken := "access_" + hash(req.client_id + r.user_id + currentTime)
let newRefreshToken := "refresh_" + hash(req.client_id + r.user_id + currentTime)
let expiresAt := currentTime + 3600
accessTokens[newAccessToken] := {
client_id: req.client_id,
user_id: r.user_id,
scope: req.scope.or(r.scope),
expires_at: expiresAt,
session_id: r.access_token.session_id,
refresh_token: newRefreshToken
}
refreshTokens[newRefreshToken] := {
client_id: req.client_id,
user_id: r.user_id,
scope: req.scope.or(r.scope),
access_token: newAccessToken,
created_at: currentTime
}
// Remove old refresh token
refreshTokens[req.refresh_token] := null
emit TokenRefreshed {
access_token: newAccessToken,
token_type: "Bearer",
expires_in: 3600,
refresh_token: newRefreshToken,
scope: req.scope.or(r.scope),
client_id: req.client_id,
user_id: r.user_id
}
}
// Action 5: RevokeToken - Client revokes a token
action RevokeToken(client: ClientId, token: String, tokenType: String) {
require client in clients.keys
if tokenType == "access" {
require token in accessTokens.keys
let t := accessTokens[token]
require t.client_id == client
accessTokens[token] := null
// Also revoke associated refresh token
refreshTokens[t.refresh_token] := null
} else if tokenType == "refresh" {
require token in refreshTokens.keys
let r := refreshTokens[token]
require r.client_id == client
refreshTokens[token] := null
accessTokens[r.access_token] := null
}
emit TokenRevoked {
token: token,
token_type: tokenType,
client_id: client
}
}
// Action 6: PKCE Verification Step (explicit verification for documentation)
action VerifyPKCE(code: AuthorizationCode, verifier: CodeVerifier) {
require code in authorizationCodes.keys
let c := authorizationCodes[code]
require c.used == false
// Verify the code verifier hashes to the code challenge
let expectedChallenge := hash(verifier, c.code_challenge_method)
require expectedChallenge == c.code_challenge
// This action documents the PKCE verification step
emit PKCEVerified {
authorization_code: code,
code_challenge: c.code_challenge,
code_challenge_method: c.code_challenge_method,
verifier_hash: hash(verifier, c.code_challenge_method)
}
}
// === Next Actions ===
next:
Authorize
| ApproveAuthorization
| ExchangeCode
| RefreshToken
| RevokeToken
| VerifyPKCE
// === Verification ===
verify: Always(NoTokenLeakage)
verify: Always(StateCSRFProtection)
verify: Always(AuthorizationCodeSingleUse)
verify: Always(TokenValidityBounded)
verify: Always(PKCECodeInterceptionPrevention)
verify: Always(RedirectUriConsistency)
verify: Always(ClientAuthenticationRequired)
}

131
examples/Paxos.speck
View file

@ -1,131 +0,0 @@
speck Paxos {
// Classic Paxos consensus protocol ported from TLA+ to SpeckDL v0.3
// Demonstrates multi-role state machines, ballot numbers, quorums,
// and safety invariants (at most one value chosen per slot).
// Adapted from Lamport's "The Part-Time Parliament" and TLA+ Paxos spec.
state {
// Ballot number → (value, proposer) — for each slot
ballots: Map(Ballot, Proposal)
// Acceptor state: which ballot each acceptor has promised for each slot
promises: Map(AcceptorSlot, Ballot)
// Acceptor state: last vote cast (if any) for each slot
lastVote: Map(AcceptorSlot, Ballot)
// Value voted for (if any) for each slot
votedValue: Map(AcceptorSlot, String)
// Set of acceptors
acceptors: Set(String)
// Quorum size (simple majority)
quorum: Map(String, Set(String))
// Chosen values: ballot → value (finalized)
chosen: Map(Ballot, String)
}
init {
let q := {"a1", "a2", "a3"}
acceptors == q
ballots == emptyMap
promises == emptyMap
lastVote == emptyMap
votedValue == emptyMap
chosen == emptyMap
quorum == emptyMap
}
invariant BallotUniqueness {
forall b in ballots.keys:
forall b2 in ballots.keys \ {b}:
b.number != b2.number
}
// Safety: at most one value per ballot
invariant SingleValuePerBallot {
forall b in ballots.keys:
forall b2 in ballots.keys:
b.number == b2.number implies ballots[b].value == ballots[b2].value
}
// Safety: if a value is chosen, no other value for that ballot
invariant PaxosSafety {
forall b in chosen.keys:
forall b2 in ballots.keys:
b2.number == b.number implies ballots[b2].value == chosen[b]
}
// Phase 1a: Proposer sends Prepare with ballot number
action ProposerPrepare(prop: String, bnum: Nat) {
require bnum > 0
let b := { number: bnum, value: null, proposer: prop }
ballots[b] := { number: bnum, value: null, proposer: prop }
emit PrepareSent { ballot: bnum, proposer: prop }
}
// Phase 1b: Acceptor receives Prepare, promises if ballot > last promised
action AcceptorPromise(acc: String, ballot: Nat) {
require acc in acceptors
let acSlot := { acceptor: acc, slot: 1 }
let curPromise := if acSlot in promises.keys: promises[acSlot].number else 0
require ballot > curPromise
promises[acSlot] := { number: ballot, value: null, proposer: null }
let prevVote := if acSlot in lastVote.keys: lastVote[acSlot] else { number: 0, value: null, proposer: null }
emit PromiseSent {
acceptor: acc,
ballot: ballot,
lastVoteBallot: prevVote.number,
lastVoteValue: if prevVote.number > 0: votedValue[acSlot] else null
}
}
// Phase 2a: Proposer collects quorum of promises, proposes value
action ProposerPropose(prop: String, ballot: Nat, value: String) {
require ballots[{ number: ballot, value: null, proposer: prop }] != null
require ballots[{ number: ballot, value: null, proposer: prop }].proposer == prop
ballots[{ number: ballot, value: null, proposer: prop }] :=
{ number: ballot, value: value, proposer: prop }
emit ProposalSent { ballot: ballot, value: value, proposer: prop }
}
// Phase 2b: Acceptor votes for proposal if ballot >= last promised
action AcceptorVote(acc: String, ballot: Nat, value: String) {
require acc in acceptors
let acSlot := { acceptor: acc, slot: 1 }
require acSlot in promises.keys
require promises[acSlot].number == ballot
let prevVote := if acSlot in lastVote.keys: lastVote[acSlot].number else 0
require ballot >= prevVote
lastVote[acSlot] := { number: ballot, value: value, proposer: null }
votedValue[acSlot] := value
emit VotedSent { acceptor: acc, ballot: ballot, value: value }
}
// Learn: value is chosen when a quorum of acceptors has voted
action LearnerDecide(ballot: Nat, value: String) {
require { number: ballot, value: value, proposer: null } notIn chosen.keys
chosen[{ number: ballot, value: value, proposer: null }] := value
emit Chosen { ballot: ballot, value: value }
}
next:
ProposerPrepare
| AcceptorPromise
| ProposerPropose
| AcceptorVote
| LearnerDecide
}

22
examples/PreconditionTest.speck
View file

@ -1,22 +0,0 @@
speck PreconditionTest {
state: {
balance: Integer
active: Boolean
}
action SetBalance(newBalance: Integer) {
precondition: newBalance >= 0
postcondition: this.balance == newBalance
require newBalance >= 0
this.balance := newBalance
}
action Deactivate() {
precondition: this.balance == 0
or this.active == false
postcondition: this.active == false
this.active := false
}
}

722
examples/PromptOptimizer-runtime.ts
View file

@ -1,722 +0,0 @@
// PromptOptimizer runtime — implements actual LLM prompt optimization logic
// The generated PromptOptimizerMachine handles state transitions, guards, and events.
// This module provides the functions that make it actually work.
// ─── Token Counting ────────────────────────────────────────
// Approximate token count for LLM models (GPT-4/Claude style: ~4 chars per token)
export function countTokens(text: string, model: string = 'gpt-4'): number {
if (!text) return 0;
// GPT-4 style: ~4 chars per token, with adjustments for code/special chars
const specialCharPenalty = (text.match(/[{}[\]()<>\/\\|`~!@#$%^&*]/g) || []).length;
const whitespaceTokens = (text.match(/\s{2,}/g) || []).length;
const baseTokens = Math.ceil(text.length / 4);
return baseTokens + Math.ceil(specialCharPenalty * 0.2) + whitespaceTokens;
}
// ─── Prompt Analysis ───────────────────────────────────────
export interface PromptAnalysis {
tokenCount: number;
redundancyScore: number; // 0-1, higher = more redundant
structureScore: number; // 0-1, higher = better organized
constraintCount: number;
instructionCount: number;
estimatedQuality: number; // 0-1
issues: string[];
sections: PromptSection[];
duplicatedPhrases: DuplicatedPhrase[];
vagueInstructions: string[];
}
export interface PromptSection {
type: 'instruction' | 'constraint' | 'example' | 'context' | 'format' | 'role' | 'tone';
text: string;
tokenCount: number;
priority: number; // 1=critical, 2=important, 3=nice-to-have
}
export interface DuplicatedPhrase {
phrase: string;
occurrences: number;
savings: number; // tokens saved by deduplicating
}
export function analyzePrompt(prompt: string, context: string = 'system', model: string = 'gpt-4'): PromptAnalysis {
const tokenCount = countTokens(prompt, model);
const sections = extractSections(prompt);
const duplicatedPhrases = findDuplicatedPhrases(prompt, sections);
const vagueInstructions = findVagueInstructions(prompt);
const constraintCount = countConstraints(prompt);
const instructionCount = countInstructions(prompt);
const redundancyScore = calculateRedundancyScore(prompt, duplicatedPhrases);
const structureScore = calculateStructureScore(sections);
const estimatedQuality = calculateQualityScore(redundancyScore, structureScore, vagueInstructions, constraintCount);
const issues = identifyIssues(redundancyScore, structureScore, vagueInstructions, duplicatedPhrases);
return {
tokenCount,
redundancyScore,
structureScore,
constraintCount,
instructionCount,
estimatedQuality,
issues,
sections,
duplicatedPhrases,
vagueInstructions
};
}
// ─── Section Extraction ─────────────────────────────────────
function extractSections(prompt: string): PromptSection[] {
const sections: PromptSection[] = [];
const lines = prompt.split('\n');
let currentSection: PromptSection | null = null;
let currentText: string[] = [];
const sectionHeaders: Record<string, PromptSection['type']> = {
'you are': 'role',
'your role': 'role',
'role': 'role',
'instructions': 'instruction',
'rules': 'constraint',
'constraints': 'constraint',
'requirements': 'constraint',
'guidelines': 'instruction',
'format': 'format',
'output format': 'format',
'response format': 'format',
'example': 'example',
'examples': 'example',
'context': 'context',
'background': 'context',
'tone': 'tone',
'style': 'tone',
'voice': 'tone',
};
for (const line of lines) {
const trimmed = line.trim().toLowerCase();
const isHeader = trimmed.startsWith('#') || trimmed.startsWith('##') ||
trimmed.endsWith(':') || trimmed.startsWith('- ');
// Check if this line is a section header
let foundType: PromptSection['type'] | null = null;
for (const [keyword, type] of Object.entries(sectionHeaders)) {
if (trimmed.includes(keyword)) {
foundType = type;
break;
}
}
if (foundType && (trimmed.startsWith('#') || trimmed.endsWith(':') || trimmed.startsWith('**'))) {
// Save previous section
if (currentSection && currentText.length > 0) {
currentSection.text = currentText.join('\n');
currentSection.tokenCount = countTokens(currentSection.text);
sections.push(currentSection);
}
currentSection = { type: foundType, text: '', tokenCount: 0, priority: getPriority(foundType) };
currentText = [line];
} else if (currentSection) {
currentText.push(line);
} else {
// Unaffiliated content — treat as context/instruction
if (line.trim()) {
currentSection = { type: 'instruction', text: '', tokenCount: 0, priority: 2 };
currentText = [line];
}
}
}
// Save last section
if (currentSection && currentText.length > 0) {
currentSection.text = currentText.join('\n');
currentSection.tokenCount = countTokens(currentSection.text);
sections.push(currentSection);
}
// If no sections found, treat entire prompt as one instruction section
if (sections.length === 0 && prompt.trim()) {
sections.push({
type: 'instruction',
text: prompt,
tokenCount: countTokens(prompt),
priority: 2
});
}
return sections;
}
function getPriority(type: PromptSection['type']): number {
switch (type) {
case 'constraint': return 1; // constraints are critical
case 'role': return 1; // role definition is critical
case 'instruction': return 2; // instructions are important
case 'format': return 2; // format is important
case 'context': return 3; // context is nice-to-have
case 'example': return 3; // examples can be trimmed
case 'tone': return 3; // tone is nice-to-have
default: return 2;
}
}
// ─── Redundancy Detection ──────────────────────────────────
function findDuplicatedPhrases(prompt: string, sections: PromptSection[]): DuplicatedPhrase[] {
const phrases: DuplicatedPhrase[] = [];
const seen = new Map<string, number>();
// Extract meaningful phrases (3-6 words)
const words = prompt.split(/\s+/);
for (let len = 3; len <= 6; len++) {
for (let i = 0; i <= words.length - len; i++) {
const phrase = words.slice(i, i + len).join(' ').toLowerCase().replace(/[.,;:!?]/g, '');
if (phrase.length < 10) continue; // skip very short phrases
if (/^(the|a|an|is|are|was|were|be|been|being|have|has|had|do|does|did|will|would|could|should|may|might|shall|can|need|dare|ought|used|to|of|in|for|on|with|at|by|from|as|into|through|during|before|after|above|below|between|out|off|over|under|again|further|then|once|and|but|or|nor|not|so|yet)\b/i.test(phrase)) continue;
seen.set(phrase, (seen.get(phrase) || 0) + 1);
}
}
// Only report phrases that appear 2+ times
for (const [phrase, count] of seen) {
if (count >= 2) {
phrases.push({
phrase,
occurrences: count,
savings: countTokens(phrase) * (count - 1)
});
}
}
// Sort by savings descending
return phrases.sort((a, b) => b.savings - a.savings).slice(0, 10);
}
// ─── Vague Instruction Detection ────────────────────────────
const VAGUE_PATTERNS = [
/\b(be|act|respond|write|create|make|generate|produce|provide)\s+(good|nice|great|excellent|appropriate|reasonable|proper|suitable|relevant|helpful|clear)\b/gi,
/\b(try|attempt|endeavor|strive)\s+to\b/gi,
/\bas\s+(needed|appropriate|necessary|required|suitable)\b/gi,
/\b(make\s+sure|ensure)\s+(that\s+)?(it|the|this|your)\b/gi,
/\b(etc\.?|and\s+so\s+on|and\s+so\s+forth|and\s+the\s+like)\b/gi,
];
function findVagueInstructions(prompt: string): string[] {
const found: string[] = [];
for (const pattern of VAGUE_PATTERNS) {
const matches = prompt.matchAll(pattern);
for (const m of matches) {
found.push(m[0]);
}
}
return [...new Set(found)];
}
// ─── Constraint & Instruction Counting ──────────────────────
function countConstraints(prompt: string): number {
const patterns = [
/\b(must|shall|required|mandatory|always|never|do\s+not|don't|cannot|can't|never|forbidden|prohibited)\b/gi,
/\b(no|without|exclude|avoid|prevent|refrain)\b/gi,
];
let count = 0;
for (const p of patterns) {
const matches = prompt.match(p);
count += matches ? matches.length : 0;
}
return count;
}
function countInstructions(prompt: string): number {
const imperatives = prompt.match(/\b(Write|Create|Generate|Build|Implement|Design|Analyze|Review|Check|Ensure|Provide|List|Describe|Explain|Compare|Summarize|Convert|Translate|Format)\b/gi);
return imperatives ? imperatives.length : 0;
}
// ─── Score Calculation ─────────────────────────────────────
function calculateRedundancyScore(prompt: string, duplicates: DuplicatedPhrase[]): number {
if (duplicates.length === 0) return 0.0;
const totalDuplicateTokens = duplicates.reduce((sum, d) => sum + d.savings, 0);
const totalTokens = countTokens(prompt);
if (totalTokens === 0) return 0.0;
return Math.min(1.0, totalDuplicateTokens / totalTokens);
}
function calculateStructureScore(sections: PromptSection[]): number {
if (sections.length === 0) return 0.0;
if (sections.length === 1) return 0.3; // just a blob
const hasRole = sections.some(s => s.type === 'role');
const hasInstructions = sections.some(s => s.type === 'instruction');
const hasConstraints = sections.some(s => s.type === 'constraint');
const hasFormat = sections.some(s => s.type === 'format');
let score = 0.3; // baseline for having multiple sections
if (hasRole) score += 0.2;
if (hasInstructions) score += 0.2;
if (hasConstraints) score += 0.15;
if (hasFormat) score += 0.15;
return Math.min(1.0, score);
}
function calculateQualityScore(
redundancy: number,
structure: number,
vagueInstructions: string[],
constraintCount: number
): number {
// Start at 0.5, penalize for problems, reward for good structure
let score = 0.5;
// Redundancy is bad — up to -0.3
score -= redundancy * 0.3;
// Good structure is good — up to +0.2
score += (structure - 0.5) * 0.4;
// Vague instructions are bad — up to -0.1
score -= Math.min(0.1, vagueInstructions.length * 0.02);
// Having constraints is good — up to +0.1
score += Math.min(0.1, constraintCount * 0.01);
return Math.max(0.0, Math.min(1.0, score));
}
function identifyIssues(
redundancy: number,
structure: number,
vague: string[],
duplicates: DuplicatedPhrase[]
): string[] {
const issues: string[] = [];
if (redundancy > 0.3) issues.push('High redundancy — consider removing duplicated phrases');
if (redundancy > 0.15 && redundancy <= 0.3) issues.push('Moderate redundancy — some phrases appear multiple times');
if (structure < 0.4) issues.push('Poor structure — no clear sections or headers');
else if (structure < 0.6) issues.push('Moderate structure — consider adding headers for key sections');
if (vague.length > 0) issues.push(`Vague instructions found: "${vague.slice(0, 3).join('", "')}"`);
if (duplicates.length > 3) issues.push(`${duplicates.length} duplicated phrases found — deduplication could save ${duplicates.reduce((s, d) => s + d.savings, 0)} tokens`);
return issues;
}
// ─── Optimization Strategies ────────────────────────────────
export function rewritePrompt(
prompt: string,
strategy: string,
analysis: PromptAnalysis
): string {
switch (strategy) {
case 'reorder':
return reorderSections(prompt, analysis);
case 'deduplicate':
return deduplicatePrompt(prompt, analysis);
case 'concretize':
return concretizeInstructions(prompt, analysis);
case 'decompose':
return decomposePrompt(prompt, analysis);
default:
return prompt;
}
}
// ─── Reorder: put critical sections first ────────────────────
function reorderSections(prompt: string, analysis: PromptAnalysis): string {
if (analysis.sections.length <= 1) return prompt;
// Sort by priority (1 first), then by type
const typeOrder: Record<string, number> = {
'role': 1,
'constraint': 2,
'instruction': 3,
'format': 4,
'context': 5,
'example': 6,
'tone': 7,
};
const sorted = [...analysis.sections].sort((a, b) => {
const aOrder = typeOrder[a.type] ?? 4;
const bOrder = typeOrder[b.type] ?? 4;
if (aOrder !== bOrder) return aOrder - bOrder;
return a.priority - b.priority;
});
return sorted.map(s => s.text.trim()).filter(Boolean).join('\n\n');
}
// ─── Deduplicate: remove duplicated sentences and phrases ────────────
function deduplicatePrompt(prompt: string, analysis: PromptAnalysis): string {
// Strategy: find duplicate sentences and remove later occurrences
const sentences = prompt.split(/(?<=[.!?])\s+/);
const seen = new Map<string, number>();
const result: string[] = [];
for (const sentence of sentences) {
const normalized = sentence.toLowerCase().trim().replace(/\s+/g, ' ');
// Skip very short fragments
if (normalized.length < 15) {
result.push(sentence);
continue;
}
if (seen.has(normalized)) {
// Skip duplicate sentence
continue;
}
seen.set(normalized, 1);
result.push(sentence);
}
return result.join(' ').replace(/\s{2,}/g, ' ').replace(/\n{3,}/g, '\n\n').trim();
}
function escapeRegex(s: string): string {
return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
}
// ─── Concretize: replace vague instructions ─────────────────
const CONCRETIZATIONS: [RegExp, string][] = [
[/write good\s+responses/gi, 'write clear, specific responses'],
[/write great\s+responses/gi, 'write thorough, well-organized responses'],
[/write nice\s+/gi, 'write polished '],
[/be helpful/gi, 'provide actionable, specific responses'],
[/be clear/gi, 'use short sentences, active voice'],
[/respond appropriately/gi, 'respond with a direct answer first'],
[/as needed/gi, 'when required'],
[/as appropriate/gi, 'matching the context'],
[/make sure that/gi, 'verify that'],
[/ensure that/gi, 'verify that'],
[/try to\s+/gi, ''], // just remove hedging
[/in other words[,\s]*/gi, ''], // remove wordy transitions
[/for example,?\s+/gi, 'e.g., '],
[/for instance,?\s+/gi, 'e.g., '],
[/that is to say[,\s]*/gi, ''],
];
function concretizeInstructions(prompt: string, analysis: PromptAnalysis): string {
let result = prompt;
for (const [pattern, replacement] of CONCRETIZATIONS) {
result = result.replace(pattern, replacement);
}
// Clean up artifacts from replacements
result = result
.replace(/\s{2,}/g, ' ')
.replace(/\n{3,}/g, '\n\n')
.replace(/^\s*[,;.]\s*/gm, '') // remove leading punctuation
.trim();
return result;
}
// ─── Decompose: split into numbered steps ────────────────────
function decomposePrompt(prompt: string, analysis: PromptAnalysis): string {
// If already has numbered sections, skip
if (/^\d+\.\s/m.test(prompt)) return prompt;
const lines = prompt.split('\n');
const instructions = lines.filter(l => {
const t = l.trim().toLowerCase();
return /^\s*(write|create|generate|build|implement|design|analyze|review|check|ensure|provide|list|describe|explain|compare|summarize|convert|translate|format)\b/i.test(t);
});
if (instructions.length <= 1) return prompt;
// Add numbers to imperative lines
let instrIdx = 0;
const result = lines.map(l => {
const t = l.trim();
if (/^\s*(write|create|generate|build|implement|design|analyze|review|check|ensure|provide|list|describe|explain|compare|summarize|convert|translate|format)\b/i.test(t)) {
instrIdx++;
return `${instrIdx}. ${t}`;
}
return l;
});
return result.join('\n');
}
// ─── Compression ────────────────────────────────────────────
export function compressPrompt(prompt: string, targetTokens: number, model: string = 'gpt-4'): string {
const currentTokens = countTokens(prompt, model);
if (currentTokens <= targetTokens) return prompt;
const ratio = targetTokens / currentTokens;
let result = prompt;
// Step 1: Remove duplicate phrases
const analysis = analyzePrompt(result, 'system', model);
result = deduplicatePrompt(result, analysis);
// Step 2: Remove filler words and simplify common phrases
result = result
.replace(/\b(?:please|kindly|basically|actually|really|very|quite|rather|somewhat|just|simply|merely)\b/gi, '')
.replace(/\bin order to\b/gi, 'to')
.replace(/\bdue to the fact that\b/gi, 'because')
.replace(/\bat this point in time\b/gi, 'now')
.replace(/\bin the event that\b/gi, 'if')
.replace(/\bfor the purpose of\b/gi, 'to')
.replace(/\bit is important to note that\b/gi, '')
.replace(/\bit should be noted that\b/gi, '')
.replace(/\bit is worth noting that\b/gi, '')
.replace(/\bin other words[,\s]*/gi, '')
.replace(/ +/g, ' ')
.replace(/\n{3,}/g, '\n\n')
.trim();
if (countTokens(result, model) <= targetTokens) return result;
// Step 3: Remove lower-priority sections
const currentAnalysis = analyzePrompt(result, 'system', model);
const sortedSections = [...currentAnalysis.sections].sort((a, b) => b.priority - a.priority);
// Remove sections starting from lowest priority
for (const section of sortedSections) {
if (section.priority >= 3) { // nice-to-have only
result = result.replace(section.text, '').replace(/\n{3,}/g, '\n\n').trim();
if (countTokens(result, model) <= targetTokens) return result;
}
}
// Step 4: Shorten remaining sentences (aggressive compression)
result = result
.replace(/\b(that is to say)\b[^.]*\./gi, '')
.replace(/\b(for example|for instance)\b/gi, 'e.g.')
.replace(/\b(that is)\b/gi, 'i.e.')
.replace(/\b(information)\b/gi, 'info')
.replace(/\b(application)\b/gi, 'app')
.replace(/\b(configuration)\b/gi, 'config')
.replace(/\b(implementation)\b/gi, 'impl')
.replace(/\b(documentation)\b/gi, 'docs')
.replace(/\b(environment)\b/gi, 'env')
.replace(/\b(percentage)\b/gi, '%')
.replace(/\b(number)\b/gi, '#')
.replace(/ +/g, ' ')
.replace(/\n{3,}/g, '\n\n')
.trim();
return result;
}
// ─── Validation ─────────────────────────────────────────────
export function validateOptimization(
original: string,
optimized: string,
criteria: string = 'preserve-intent'
): { passed: boolean; confidence: number; issues: string[] } {
const issues: string[] = [];
let confidence = 1.0;
// Check: must preserve intent (critical keywords)
const origKeywords = extractKeywords(original);
const optKeywords = extractKeywords(optimized);
const missingKeywords = [...origKeywords].filter(k => !optKeywords.has(k));
const criticalMissing = missingKeywords.filter(k => isCriticalKeyword(k, original));
if (criticalMissing.length > 0) {
issues.push(`Critical keywords missing: ${criticalMissing.join(', ')}`);
confidence -= criticalMissing.length * 0.15;
}
// Check: must reduce or maintain tokens
const origTokens = countTokens(original);
const optTokens = countTokens(optimized);
if (optTokens > origTokens) {
issues.push(`Optimized prompt has MORE tokens (${optTokens}) than original (${origTokens})`);
confidence -= 0.2;
}
// Check: must not be empty
if (optimized.trim().length === 0) {
issues.push('Optimized prompt is empty');
confidence = 0;
}
// Check: structural integrity
const origSections = extractSections(original);
const optSections = extractSections(optimized);
const origConstraintCount = origSections.filter(s => s.type === 'constraint').length;
const optConstraintCount = optSections.filter(s => s.type === 'constraint').length;
if (optConstraintCount < origConstraintCount) {
issues.push(`Lost ${origConstraintCount - optConstraintCount} constraint section(s)`);
confidence -= 0.1 * (origConstraintCount - optConstraintCount);
}
return {
passed: confidence >= 0.6 && criticalMissing.length === 0 && optimized.trim().length > 0,
confidence: Math.max(0, Math.min(1, confidence)),
issues
};
}
function extractKeywords(text: string): Set<string> {
const stopWords = new Set(['the','a','an','is','are','was','were','be','been','being',
'have','has','had','do','does','did','will','would','could','should','may','might',
'shall','can','need','to','of','in','for','on','with','at','by','from','as','into',
'through','during','before','after','above','below','between','out','off','over',
'under','again','further','then','once','and','but','or','nor','not','so','yet',
'both','either','neither','each','every','all','any','few','more','most','other',
'some','such','no','only','own','same','than','too','very','just','because','if',
'when','where','how','what','which','who','whom','this','that','these','those',
'i','me','my','we','our','you','your','he','him','his','she','her','it','its',
'they','them','their','about','up']);
return new Set(
text.toLowerCase()
.replace(/[^a-z0-9\s]/g, ' ')
.split(/\s+/)
.filter(w => w.length > 3 && !stopWords.has(w))
);
}
function isCriticalKeyword(keyword: string, original: string): boolean {
// Keywords that appear in imperative/constraint context are critical
const lower = original.toLowerCase();
const idx = lower.indexOf(keyword);
if (idx === -1) return false;
// Check 50 chars before the keyword for constraint markers
const context = lower.slice(Math.max(0, idx - 50), idx);
return /\b(must|shall|never|always|required|mandatory|do not|don't|cannot|forbidden|critical|essential|important)\b/.test(context);
}
// ─── Full Pipeline ──────────────────────────────────────────
export interface OptimizationResult {
originalPrompt: string;
optimizedPrompt: string;
originalTokens: number;
optimizedTokens: number;
tokenReduction: number; // percentage
strategiesApplied: string[];
analysis: PromptAnalysis;
validation: { passed: boolean; confidence: number; issues: string[] };
}
export function optimizePrompt(
prompt: string,
context: string = 'system',
model: string = 'gpt-4',
targetReduction: number = 0.3 // aim for 30% reduction
): OptimizationResult {
const originalTokens = countTokens(prompt, model);
// Phase 1: Analyze
let analysis = analyzePrompt(prompt, context, model);
// Phase 2: Determine strategies
const strategies: string[] = [];
if (analysis.redundancyScore > 0.1) strategies.push('deduplicate');
if (analysis.structureScore < 0.6) strategies.push('reorder');
if (analysis.vagueInstructions.length > 0) strategies.push('concretize');
if (analysis.instructionCount > 3 && !/^\d+\.\s/m.test(prompt)) strategies.push('decompose');
// Phase 3: Apply strategies
let optimized = prompt;
for (const strategy of strategies) {
optimized = rewritePrompt(optimized, strategy, analyzePrompt(optimized, context, model));
}
// Phase 4: Compress if needed
const targetTokens = Math.floor(originalTokens * (1 - targetReduction));
if (countTokens(optimized, model) > targetTokens) {
optimized = compressPrompt(optimized, targetTokens, model);
strategies.push('compress');
}
// Phase 5: Validate
const validation = validateOptimization(prompt, optimized);
const optimizedTokens = countTokens(optimized, model);
const tokenReduction = originalTokens > 0
? Math.round((1 - optimizedTokens / originalTokens) * 100)
: 0;
return {
originalPrompt: prompt,
optimizedPrompt: optimized,
originalTokens,
optimizedTokens,
tokenReduction,
strategiesApplied: strategies,
analysis,
validation
};
}
// ─── Demo ───────────────────────────────────────────────────
export function demo(): void {
const verbosePrompt = `You are a helpful assistant. You should please try to be very helpful and provide good responses that are clear and well-structured. Basically, you need to respond appropriately to each user query and make sure that you give good answers.
You must never reveal sensitive information. You must always verify facts before stating them. You should not make up information that you do not actually know for certain. You must not provide medical advice. You must not provide legal advice.
Please write responses that are nice and well-formatted. Please use markdown formatting where appropriate. Please provide examples when it would be helpful. Please try to explain things in a way that is easy to understand.
In other words, your role is to be a knowledgeable and reliable assistant that prioritizes accuracy and helpfulness. You should always respond in English. You must never respond in any other language.
You are a helpful assistant. You should be helpful and provide good responses. You must always verify facts before stating them. Please provide responses that are clear and well-structured.
For example, if someone asks about a programming concept, you should basically provide a clear explanation with code examples where appropriate. For instance, you might show a simple code snippet to illustrate the point.`;
console.log('╔══════════════════════════════════════════════════════════════╗');
console.log('║ Prompt Optimizer — Before & After ║');
console.log('╚══════════════════════════════════════════════════════════════╝');
console.log();
console.log('┌──────────────────────────────────────────────────────────────┐');
console.log('│ BEFORE: Original Prompt │');
console.log('└──────────────────────────────────────────────────────────────┘');
console.log();
console.log(verbosePrompt);
console.log();
const beforeAnalysis = analyzePrompt(verbosePrompt);
console.log('Analysis:');
console.log(` Tokens: ${beforeAnalysis.tokenCount}`);
console.log(` Redundancy: ${(beforeAnalysis.redundancyScore * 100).toFixed(1)}%`);
console.log(` Structure: ${(beforeAnalysis.structureScore * 100).toFixed(1)}%`);
console.log(` Constraints: ${beforeAnalysis.constraintCount}`);
console.log(` Instructions: ${beforeAnalysis.instructionCount}`);
console.log(` Quality: ${(beforeAnalysis.estimatedQuality * 100).toFixed(1)}%`);
console.log(` Issues: ${beforeAnalysis.issues.join('; ') || 'None'}`);
console.log();
// Optimize
const result = optimizePrompt(verbosePrompt, 'system', 'gpt-4', 0.4);
console.log('┌──────────────────────────────────────────────────────────────┐');
console.log('│ AFTER: Optimized Prompt │');
console.log('└──────────────────────────────────────────────────────────────┘');
console.log();
console.log(result.optimizedPrompt);
console.log();
console.log('┌──────────────────────────────────────────────────────────────┐');
console.log('│ Summary │');
console.log('└──────────────────────────────────────────────────────────────┘');
console.log(` Original tokens: ${result.originalTokens}`);
console.log(` Optimized tokens: ${result.optimizedTokens}`);
console.log(` Token reduction: ${result.tokenReduction}%`);
console.log(` Strategies applied: ${result.strategiesApplied.join(' → ')}`);
console.log(` Validation: ${result.validation.passed ? 'PASSED' : 'FAILED'} (confidence: ${(result.validation.confidence * 100).toFixed(0)}%)`);
if (result.validation.issues.length > 0) {
console.log(` Validation issues: ${result.validation.issues.join('; ')}`);
}
}

160
examples/PromptOptimizer.speckdl
View file

@ -1,160 +0,0 @@
// PromptOptimizer.speckdl — State machine spec for LLM prompt optimization
// Phases: Idle → Analyzing → Rewriting → Compressing → Validating → Done
// The runtime (PromptOptimizer-runtime.ts) implements the actual optimization logic.
speck PromptOptimizer {
version: "0.1.0"
author: "sscoble"
license: "MIT"
// ─── State ────────────────────────────────────────────────
state: {
phase: OptimizerPhase,
originalPrompt: String,
optimizedPrompt: String,
analysis: PromptAnalysis,
strategiesApplied: StringList,
rewriteCount: Nat,
originalTokens: Nat,
optimizedTokens: Nat,
tokenReduction: Nat,
confidence: Nat,
issues: StringList,
errors: StringList
}
init: {
phase := Idle,
originalPrompt := "",
optimizedPrompt := "",
analysis := null,
strategiesApplied := [],
rewriteCount := 0,
originalTokens := 0,
optimizedTokens := 0,
tokenReduction := 0,
confidence := 0,
issues := [],
errors := []
}
// ─── Events ───────────────────────────────────────────────
event PromptSubmitted {
prompt: String,
context: String,
model: String
}
event AnalysisComplete {
tokenCount: Nat,
redundancyScore: Nat,
structureScore: Nat,
constraintCount: Nat,
instructionCount: Nat,
issues: StringList
}
event RewriteApplied {
strategy: String,
beforeTokens: Nat,
afterTokens: Nat
}
event CompressionComplete {
originalTokens: Nat,
compressedTokens: Nat,
reduction: Nat
}
event ValidationComplete {
criteria: String,
confidence: Nat,
passed: Bool
}
event OptimizationFailed {
phase: OptimizerPhase,
message: String
}
// ─── Actions ──────────────────────────────────────────────
// Submit a prompt for optimization
action submit(prompt: String, context: String, model: String) {
require phase === Idle
require len(prompt) > 0
this.phase := Analyzing
this.originalPrompt := prompt
this.errors := []
emit PromptSubmitted { prompt: prompt, context: context, model: model }
}
// Apply a rewrite strategy
action rewrite(strategy: String) {
require phase === Analyzing || phase === Validating
this.phase := Rewriting
this.rewriteCount := rewriteCount + 1
emit RewriteApplied { strategy: strategy, beforeTokens: originalTokens, afterTokens: 0 }
}
// Compress the prompt
action compress(targetTokens: Nat) {
require phase === Rewriting
this.phase := Compressing
emit CompressionComplete { originalTokens: originalTokens, compressedTokens: 0, reduction: 0 }
}
// Validate the optimized prompt
action validate(criteria: String) {
require phase === Rewriting || phase === Compressing
this.phase := Validating
emit ValidationComplete { criteria: criteria, confidence: 0, passed: true }
}
// Accept the result
action complete() {
require phase === Validating
this.phase := Done
}
// Start over
action reset() {
this.phase := Idle
this.originalPrompt := ""
this.optimizedPrompt := ""
this.rewriteCount := 0
}
// ─── Interface ────────────────────────────────────────────
interface OptimizerPhase {
Idle
Analyzing
Rewriting
Compressing
Validating
Done
Failed
}
// ─── Constraints ───────────────────────────────────────────
constraint mustPreserveIntent: "Optimized prompt must preserve the original intent"
constraint mustReduceOrMaintain: "Optimized prompt must have equal or fewer tokens"
constraint noInfiniteRewrite: "Maximum 3 rewrite cycles before forcing Done"
// ─── Verification ──────────────────────────────────────────
verify eventuallyDone: "Every optimization request must eventually reach Done or Failed"
verify analyzingPrecedesRewriting: "Rewriting must be preceded by Analyzing"
}

272
examples/Raft.speck
View file

@ -1,272 +0,0 @@
speck Raft {
// Raft consensus protocol ported to SpeckDL v0.3
// Demonstrates leader election, log replication, commitment, and safety.
// Based on Ongaro & Ousterhout's "In Search of an Understandable Consensus Algorithm" (2014).
// Natural bridge from Paxos (academic) → TigerBeetle (production replicated state machine).
type LogIndex = Nat
type Term = Nat
type ServerId = String
type LogEntry = {
term: Term,
index: LogIndex,
command: String
}
type ServerState = {
role: String, // "follower", "candidate", "leader"
currentTerm: Term,
votedFor: String | null,
log: List(LogEntry),
commitIndex: LogIndex,
lastApplied: LogIndex
}
type AppendEntriesReq = {
term: Term,
leaderId: ServerId,
prevLogIndex: LogIndex,
prevLogTerm: Term,
entries: List(LogEntry),
leaderCommit: LogIndex
}
type RequestVoteReq = {
term: Term,
candidateId: ServerId,
lastLogIndex: LogIndex,
lastLogTerm: Term
}
state {
servers: Set(ServerId)
serverState: Map(ServerId, ServerState)
// Leader-specific: for each server, track nextIndex and matchIndex
nextIndex: Map(Map(ServerId, ServerId), LogIndex)
matchIndex: Map(Map(ServerId, ServerId), LogIndex)
// Election timer (abstracted as random timeout)
electionTimeout: Map(ServerId, Nat)
}
init {
servers == {"s1", "s2", "s3", "s4", "s5"}
// All servers start as followers in term 0
forall s in servers:
serverState[s] == {
role: "follower",
currentTerm: 0,
votedFor: null,
log: List.empty,
commitIndex: 0,
lastApplied: 0
}
nextIndex == emptyMap
matchIndex == emptyMap
electionTimeout == emptyMap
}
// Safety: Election Safety — at most one leader per term
invariant SingleLeaderPerTerm {
forall s1 in servers:
forall s2 in servers:
serverState[s1].role == "leader" and
serverState[s2].role == "leader" and
s1 != s2 implies
serverState[s1].currentTerm != serverState[s2].currentTerm
}
// Safety: Leader Completeness — committed entries survive elections
invariant LeaderCompleteness {
forall s in servers:
serverState[s].role == "leader" implies
forall entry in serverState[s].log:
entry.index <= serverState[s].commitIndex implies
entry.term <= serverState[s].currentTerm
}
// Safety: Log Matching — if two logs have same term at same index, entries match
invariant LogMatching {
forall s1 in servers:
forall s2 in servers:
forall i in [1 .. length(serverState[s1].log)]:
i <= length(serverState[s2].log) and
serverState[s1].log[i].term == serverState[s2].log[i].term implies
serverState[s1].log[i].command == serverState[s2].log[i].command
}
// Start election: follower times out, becomes candidate
action StartElection(server: ServerId) {
require server in servers
require serverState[server].role == "follower"
let newTerm := serverState[server].currentTerm + 1
// Increment term, vote for self
serverState[server].currentTerm := newTerm
serverState[server].role := "candidate"
serverState[server].votedFor := server
emit ElectionStarted { server: server, term: newTerm }
}
// Grant vote: follower votes for candidate if candidate's log is at least as up-to-date
action GrantVote(voter: ServerId, req: RequestVoteReq) {
require voter in servers
require serverState[voter].role == "follower" or
serverState[voter].role == "candidate"
// Only vote if candidate's term >= our term
require req.term >= serverState[voter].currentTerm
// If candidate's term > ours, step down
if req.term > serverState[voter].currentTerm {
serverState[voter].currentTerm := req.term
serverState[voter].role := "follower"
serverState[voter].votedFor := null
}
// Check not already voted in this term
require serverState[voter].votedFor == null or
serverState[voter].votedFor == req.candidateId
// Candidate log must be at least as complete as ours
let voterLog := serverState[voter].log
let voterLastIndex := length(voterLog)
let voterLastTerm := if voterLastIndex > 0: voterLog[voterLastIndex].term else 0
require req.lastLogTerm > voterLastTerm or
(req.lastLogTerm == voterLastTerm and req.lastLogIndex >= voterLastIndex)
serverState[voter].votedFor := req.candidateId
emit VoteGranted { voter: voter, candidate: req.candidateId, term: req.term }
}
// Become leader: candidate receives majority of votes
action BecomeLeader(server: ServerId) {
require server in servers
require serverState[server].role == "candidate"
// Count votes received (simplified: assumes quorum reached via external counting)
serverState[server].role := "leader"
// Initialize nextIndex/matchIndex for all other servers
forall s in servers \ {server}:
let lastLogIndex := length(serverState[server].log)
nextIndex[{ leader: server, follower: s }] := lastLogIndex + 1
matchIndex[{ leader: server, follower: s }] := 0
emit LeaderElected { leader: server, term: serverState[server].currentTerm }
// Send initial empty AppendEntries (heartbeat) to assert leadership
emit HeartbeatSent { leader: server, term: serverState[server].currentTerm }
}
// Append entries: leader replicates log entry to followers
action AppendEntries(leader: ServerId, follower: ServerId, req: AppendEntriesReq) {
require leader in servers
require follower in servers
require leader != follower
require serverState[leader].role == "leader"
// Verify term consistency
let followerTerm := serverState[follower].currentTerm
require req.term >= followerTerm
// If leader's term is stale, step down
if req.term > serverState[leader].currentTerm {
serverState[leader].currentTerm := req.term
serverState[leader].role := "follower"
serverState[leader].votedFor := null
return
}
// Update follower's term if behind
if req.term > followerTerm {
serverState[follower].currentTerm := req.term
serverState[follower].role := "follower"
serverState[follower].votedFor := null
}
// Check log consistency at prevLogIndex
let fLog := serverState[follower].log
require req.prevLogIndex == 0 or
(req.prevLogIndex <= length(fLog) and
fLog[req.prevLogIndex].term == req.prevLogTerm)
// Delete conflicting entries, append new ones
let truncated := if req.prevLogIndex < length(fLog):
take(fLog, req.prevLogIndex)
else fLog
serverState[follower].log := concat(truncated, req.entries)
// Update commit index
if req.leaderCommit > serverState[follower].commitIndex {
let lastNewEntry := req.prevLogIndex + length(req.entries)
serverState[follower].commitIndex :=
if req.leaderCommit < lastNewEntry: req.leaderCommit else lastNewEntry
}
emit EntriesReplicated {
leader: leader,
follower: follower,
entries: length(req.entries)
}
}
// Advance commit index: leader commits when majority replicated
action AdvanceCommit(leader: ServerId) {
require leader in servers
require serverState[leader].role == "leader"
let currentCommit := serverState[leader].commitIndex
let logLen := length(serverState[leader].log)
// Find highest index that has been replicated to a majority
for i from currentCommit + 1 to logLen:
let replicated := 1 // leader counts itself
forall s in servers \ {leader}:
if matchIndex[{ leader: leader, follower: s }] >= i:
replicated := replicated + 1
if replicated * 2 > size(servers) and
serverState[leader].log[i].term == serverState[leader].currentTerm {
serverState[leader].commitIndex := i
}
emit CommitAdvanced {
leader: leader,
commitIndex: serverState[leader].commitIndex
}
}
// Apply committed entries to state machine
action ApplyEntries(server: ServerId) {
require server in servers
let s := serverState[server]
let commitIdx := s.commitIndex
for i from s.lastApplied + 1 to commitIdx:
emit CommandApplied {
server: server,
index: i,
command: s.log[i].command
}
serverState[server].lastApplied := commitIdx
}
next:
StartElection
| GrantVote
| BecomeLeader
| AppendEntries
| AdvanceCommit
| ApplyEntries
}

670
examples/SQLOptimizer-runtime.ts
View file

@ -1,670 +0,0 @@
// SQLOptimizer runtime — implements the actual SQL optimization logic
// that the SpeckDL state machine orchestrates.
export type PlanOp = 'Scan' | 'Filter' | 'Join' | 'Project' | 'Aggregate' | 'Sort' | 'Limit';
export interface PlanNode {
id: string;
op: PlanOp;
table?: string;
predicate?: string;
joinType?: string;
leftChild?: string;
rightChild?: string;
onClause?: string;
columns?: string[];
groupBy?: string;
orderBy?: string;
limit?: number;
estimatedRows: number;
children: string[];
}
export interface PlanTree {
nodes: Record<string, PlanNode>;
root: string;
}
export interface TableStats {
rowCount: number;
columns: Record<string, { ndv: number; nullFraction: number; avgWidth: number }>;
}
// ─── SQL Parser ─────────────────────────────────────────────
interface SQLParts {
select: string;
from: string;
joins: { type: string; table: string; on: string }[];
where: string;
groupBy: string;
orderBy: string;
limit: string;
}
function parseSQL(sql: string): SQLParts {
const s = sql.trim().replace(/\s+/g, ' ').replace(/;$/, '');
// Extract clauses by keyword positions
const selectMatch = s.match(/\bSELECT\s+(.+?)\s+FROM\b/i);
const select = selectMatch ? selectMatch[1].trim() : '*';
// Extract FROM table (before any JOIN or WHERE or GROUP BY etc.)
const fromMatch = s.match(/\bFROM\s+(\w+)/i);
const from = fromMatch ? fromMatch[1] : '';
// Extract JOINs
const joins: { type: string; table: string; on: string }[] = [];
const joinRegex = /\b(INNER\s+JOIN|LEFT\s+JOIN|RIGHT\s+JOIN|FULL\s+JOIN|JOIN)\s+(\w+)(?:\s+AS\s+\w+)?\s+ON\s+(\w+\.\w+\s*=\s*\w+\.\w+)/gi;
let jm: RegExpExecArray | null;
while ((jm = joinRegex.exec(s)) !== null) {
joins.push({
type: jm[1].toUpperCase().replace(/\s+/g, ' '),
table: jm[2],
on: jm[3].trim()
});
}
// Extract WHERE
const whereMatch = s.match(/\bWHERE\s+(.+?)(?:\s+GROUP\s+BY\b|\s+HAVING\b|\s+ORDER\s+BY\b|\s+LIMIT\b|\s*$)/i);
const where = whereMatch ? whereMatch[1].trim() : '';
// Extract GROUP BY (stop before HAVING)
const groupMatch = s.match(/\bGROUP\s+BY\s+(.+?)(?:\s+HAVING\b|\s+ORDER\s+BY\b|\s+LIMIT\b|\s*$)/i);
const groupBy = groupMatch ? groupMatch[1].trim() : '';
// Extract ORDER BY
const orderMatch = s.match(/\bORDER\s+BY\s+(.+?)(?:\s+LIMIT\b|\s*$)/i);
const orderBy = orderMatch ? orderMatch[1].trim() : '';
// Extract LIMIT
const limitMatch = s.match(/\bLIMIT\s+(\d+)/i);
const limit = limitMatch ? limitMatch[1] : '';
return { select, from, joins, where, groupBy, orderBy, limit };
}
// ─── Plan Construction ──────────────────────────────────────
let nodeIdCounter = 0;
function nextId(prefix: string): string {
return `${prefix}_${nodeIdCounter++}`;
}
export function resetIdCounter(): void {
nodeIdCounter = 0;
}
export function buildInitialPlan(sql: string, stats: Record<string, TableStats>): PlanTree {
resetIdCounter();
const parts = parseSQL(sql);
const nodes: Record<string, PlanNode> = {};
let currentRoot: string;
// Build scan node for FROM table
const tableName = parts.from;
const tableRows = stats[tableName]?.rowCount ?? 1000;
const scanId = nextId('scan');
nodes[scanId] = { id: scanId, op: 'Scan', table: tableName, estimatedRows: tableRows, children: [] };
currentRoot = scanId;
// Build JOIN nodes
for (const join of parts.joins) {
const joinRows = stats[join.table]?.rowCount ?? 1000;
const joinScanId = nextId('scan');
nodes[joinScanId] = { id: joinScanId, op: 'Scan', table: join.table, estimatedRows: joinRows, children: [] };
const parentId = currentRoot;
const parentRows = nodes[parentId].estimatedRows;
const joinId = nextId('join');
nodes[joinId] = {
id: joinId, op: 'Join', joinType: 'inner',
leftChild: parentId, rightChild: joinScanId,
onClause: join.on,
estimatedRows: Math.ceil(parentRows * joinRows * 0.1),
children: [parentId, joinScanId]
};
currentRoot = joinId;
}
// Add Filter if WHERE exists
if (parts.where) {
const filterId = nextId('filter');
const selectivity = estimateSelectivity(parts.where);
const parentRows = nodes[currentRoot].estimatedRows;
nodes[filterId] = {
id: filterId, op: 'Filter', predicate: parts.where,
estimatedRows: Math.ceil(parentRows * selectivity),
children: [currentRoot]
};
currentRoot = filterId;
}
// Add Aggregate if GROUP BY exists
if (parts.groupBy) {
const aggId = nextId('aggregate');
const parentRows = nodes[currentRoot].estimatedRows;
nodes[aggId] = {
id: aggId, op: 'Aggregate', groupBy: parts.groupBy,
estimatedRows: Math.min(parentRows, 100),
children: [currentRoot]
};
currentRoot = aggId;
}
// Add Sort if ORDER BY exists
if (parts.orderBy) {
const sortId = nextId('sort');
const parentRows = nodes[currentRoot].estimatedRows;
nodes[sortId] = {
id: sortId, op: 'Sort', orderBy: parts.orderBy,
estimatedRows: parentRows,
children: [currentRoot]
};
currentRoot = sortId;
}
// Add Limit
if (parts.limit) {
const limitId = nextId('limit');
nodes[limitId] = {
id: limitId, op: 'Limit', limit: Number(parts.limit),
estimatedRows: Number(parts.limit),
children: [currentRoot]
};
currentRoot = limitId;
}
// Add Project
if (parts.select && parts.select !== '*') {
const projectId = nextId('project');
const parentRows = nodes[currentRoot].estimatedRows;
const columns = parts.select.split(',').map(c => c.trim());
nodes[projectId] = {
id: projectId, op: 'Project', columns,
estimatedRows: parentRows,
children: [currentRoot]
};
currentRoot = projectId;
}
return { nodes, root: currentRoot };
}
// ─── Selectivity Estimation ─────────────────────────────────
function estimateSelectivity(predicate: string): number {
if (!predicate) return 1.0;
// Each AND condition reduces by a factor
const conditions = predicate.split(/\band\b/i).length;
return Math.pow(0.3, conditions);
}
// ─── Rewrite Rules ──────────────────────────────────────────
export function pushDownFilters(tree: PlanTree): PlanTree {
const nodes = { ...tree.nodes };
let root = tree.root;
// Split AND predicates into individual conditions, push each to the correct side
function pushDown(nodeId: string): string {
const node = nodes[nodeId];
if (!node) return nodeId;
// Recurse into children first
node.children = node.children.map(pushDown);
if (node.op === 'Filter' && nodes[node.children[0]]?.op === 'Join') {
const joinNode = nodes[node.children[0]];
const predicate = node.predicate || '';
// Split AND predicates
const conjuncts = predicate.split(/\s+AND\s+/i).map((s: string) => s.trim()).filter(Boolean);
const leftTable = nodes[joinNode.leftChild!]?.table ?? '';
const rightTable = nodes[joinNode.rightChild!]?.table ?? '';
const leftPreds: string[] = [];
const rightPreds: string[] = [];
const remainingPreds: string[] = [];
for (const pred of conjuncts) {
const refs = extractColumnRefs(pred);
const onlyLeft = refs.length === 0 || refs.every(c => c.startsWith(leftTable + '.') || !c.includes('.'));
const onlyRight = refs.length === 0 || refs.every(c => c.startsWith(rightTable + '.') || !c.includes('.'));
if (onlyLeft && !onlyRight) {
leftPreds.push(pred);
} else if (onlyRight && !onlyLeft) {
rightPreds.push(pred);
} else {
remainingPreds.push(pred);
}
}
let currentRoot = joinNode.id;
// Push left-side predicates below the join's left child
if (leftPreds.length > 0 && joinNode.leftChild) {
const filterId = nextId('filter');
nodes[filterId] = {
id: filterId, op: 'Filter', predicate: leftPreds.join(' AND '),
estimatedRows: Math.ceil(nodes[joinNode.leftChild].estimatedRows * 0.3),
children: [joinNode.leftChild]
};
joinNode.leftChild = filterId;
joinNode.children = [filterId, joinNode.rightChild!];
joinNode.estimatedRows = Math.ceil(joinNode.estimatedRows * 0.5);
}
// Push right-side predicates below the join's right child
if (rightPreds.length > 0 && joinNode.rightChild) {
const filterId = nextId('filter');
nodes[filterId] = {
id: filterId, op: 'Filter', predicate: rightPreds.join(' AND '),
estimatedRows: Math.ceil(nodes[joinNode.rightChild].estimatedRows * 0.3),
children: [joinNode.rightChild]
};
joinNode.rightChild = filterId;
joinNode.children = [joinNode.leftChild!, filterId];
joinNode.estimatedRows = Math.ceil(joinNode.estimatedRows * 0.3);
}
// If we pushed everything, replace the Filter with the Join
if (remainingPreds.length === 0) {
return joinNode.id;
} else {
// Keep remaining predicates as a Filter above the Join
node.predicate = remainingPreds.join(' AND ');
node.estimatedRows = Math.ceil(joinNode.estimatedRows * 0.3);
}
}
return nodeId;
}
root = pushDown(root);
return { nodes, root };
}
export function foldConstants(tree: PlanTree): PlanTree {
const nodes = { ...tree.nodes };
function fold(nodeId: string): void {
const node = nodes[nodeId];
if (!node) return;
node.children.forEach(fold);
if (node.op === 'Filter' && node.predicate) {
// Constant-fold arithmetic, but NOT inside string literals or date literals
const parts = node.predicate.split(/(')/);
for (let i = 0; i < parts.length; i++) {
if (i % 2 === 0) { // outside quotes
parts[i] = parts[i]
.replace(/\b(\d+)\s*\+\s*(\d+)\b/g, (_, a, b) => String(Number(a) + Number(b)))
.replace(/\b(\d+)\s*\*\s*(\d+)\b/g, (_, a, b) => String(Number(a) * Number(b)));
}
}
node.predicate = parts.join('');
}
}
fold(tree.root);
return { nodes, root: tree.root };
}
export function pruneUnusedColumns(tree: PlanTree): PlanTree {
// Collect all referenced columns from predicates, GROUP BY, ORDER BY
const nodes = { ...tree.nodes };
const required = new Set<string>();
function collectRefs(nodeId: string): void {
const node = nodes[nodeId];
if (!node) return;
if (node.predicate) extractColumnRefs(node.predicate).forEach(c => required.add(c));
if (node.groupBy) extractColumnRefs(node.groupBy).forEach(c => required.add(c));
if (node.orderBy) extractColumnRefs(node.orderBy).forEach(c => required.add(c));
if (node.columns) node.columns.forEach(c => required.add(c));
node.children.forEach(collectRefs);
}
collectRefs(tree.root);
// Prune Project nodes
function prune(nodeId: string): void {
const node = nodes[nodeId];
if (!node) return;
node.children.forEach(prune);
if (node.op === 'Project' && node.columns) {
node.columns = node.columns.filter(c => required.has(c) || c === '*');
if (node.columns!.length === 0) node.columns = ['*'];
}
}
prune(tree.root);
return { nodes, root: tree.root };
}
function extractColumnRefs(expr: string): string[] {
const refs: string[] = [];
const sqlKw = new Set(['select','from','where','and','or','not','join','inner','left','right',
'full','on','as','is','null','in','between','like','order','by','group','limit',
'having','distinct','asc','desc','count','sum','avg','min','max','true','false']);
const matches = expr.matchAll(/\b(\w+)\.(\w+)\b/g);
for (const m of matches) {
const ref = `${m[1]}.${m[2]}`;
if (!sqlKw.has(m[1].toLowerCase()) && !sqlKw.has(m[2].toLowerCase())) {
refs.push(ref);
}
}
return [...new Set(refs)];
}
// ─── Physical Plan Selection ────────────────────────────────
export function choosePhysicalOperators(tree: PlanTree, stats: Record<string, TableStats>): PlanTree {
const nodes = { ...tree.nodes };
function choose(nodeId: string): void {
const node = nodes[nodeId];
if (!node) return;
node.children.forEach(choose);
// Choose join algorithm based on input sizes
if (node.op === 'Join' && node.leftChild && node.rightChild) {
const leftRows = nodes[node.leftChild]?.estimatedRows ?? 1000;
const rightRows = nodes[node.rightChild]?.estimatedRows ?? 1000;
if (leftRows > 10000 || rightRows > 10000) {
node.joinType = 'hash';
} else if (leftRows < 100 && rightRows < 100) {
node.joinType = 'nested_loop';
} else {
node.joinType = 'hash';
}
}
// Choose index scan for large tables with filters
if (node.op === 'Scan' && node.table) {
const parent = Object.values(nodes).find(n => n.children.includes(nodeId));
if (parent?.op === 'Filter') {
const tableStats = stats[node.table];
if (tableStats && tableStats.rowCount > 10000) {
// Mark as index scan
node.table = node.table + ' /* index_scan */';
}
}
}
}
choose(tree.root);
return { nodes, root: tree.root };
}
// ─── Cost Estimation ───────────────────────────────────────
export function estimateCost(nodes: Record<string, PlanNode>, rootId: string): number {
const node = nodes[rootId];
if (!node) return 0;
const childCosts = node.children.reduce((sum, c) => sum + estimateCost(nodes, c), 0);
let selfCost = 0;
switch (node.op) {
case 'Scan':
selfCost = node.estimatedRows;
break;
case 'Filter':
selfCost = Math.ceil(node.estimatedRows * 0.01);
break;
case 'Join': {
const leftRows = node.leftChild ? (nodes[node.leftChild]?.estimatedRows ?? 1000) : 1000;
const rightRows = node.rightChild ? (nodes[node.rightChild]?.estimatedRows ?? 1000) : 1000;
if (node.joinType === 'hash') {
selfCost = leftRows + rightRows + Math.ceil(leftRows * 0.5);
} else if (node.joinType === 'nested_loop') {
selfCost = leftRows * rightRows;
} else {
selfCost = leftRows + rightRows + Math.ceil(leftRows * 0.3);
}
break;
}
case 'Aggregate':
selfCost = Math.ceil(node.estimatedRows * 0.5);
break;
case 'Sort':
selfCost = Math.ceil(node.estimatedRows * Math.log2(Math.max(node.estimatedRows, 2)));
break;
case 'Project':
selfCost = Math.ceil(node.estimatedRows * 0.001);
break;
case 'Limit':
selfCost = 1;
break;
default:
selfCost = node.estimatedRows;
}
return childCosts + selfCost;
}
// ─── SQL Generation ────────────────────────────────────────
export function generateSQL(nodes: Record<string, PlanNode>, rootId: string): string {
let select = '*';
const fromTables: string[] = [];
const joins: string[] = [];
let where = '';
let groupBy = '';
let orderBy = '';
let limit = '';
// Pass 1: collect which tables appear in JOINs, and identify the leftmost table
// The leftmost table goes in FROM; everything else appears via JOIN
const joinedAsRight = new Set<string>(); // tables on the right side of a JOIN
const leftmostJoinTable = new Set<string>(); // tables on the left side of a JOIN
function collectJoinTables(nid: string): void {
const n = nodes[nid];
if (!n) return;
n.children.forEach(collectJoinTables);
if (n.op === 'Join') {
if (n.rightChild) {
let cur = n.rightChild;
while (nodes[cur] && !nodes[cur].table && nodes[cur].children.length > 0) cur = nodes[cur].children[0];
if (nodes[cur]?.table) joinedAsRight.add(nodes[cur].table.replace(/ \/\*.*\*\//, ''));
}
if (n.leftChild) {
let cur = n.leftChild;
while (nodes[cur] && !nodes[cur].table && nodes[cur].children.length > 0) cur = nodes[cur].children[0];
if (nodes[cur]?.table) leftmostJoinTable.add(nodes[cur].table.replace(/ \/\*.*\*\//, ''));
}
}
}
collectJoinTables(rootId);
// Pass 2: generate SQL
function walk(nid: string): void {
const n = nodes[nid];
if (!n) return;
switch (n.op) {
case 'Scan':
if (n.table) {
const cleanName = n.table.replace(/ \/\*.*\*\//, '');
// Only add to FROM if not appearing as right side of a JOIN
if (!joinedAsRight.has(cleanName)) {
fromTables.push(cleanName);
}
}
break;
case 'Filter':
if (where) {
where = `(${where}) AND (${n.predicate})`;
} else {
where = n.predicate ?? '';
}
break;
case 'Join': {
const jt = n.joinType === 'hash' ? 'INNER' : n.joinType === 'nested_loop' ? 'INNER' : (n.joinType?.toUpperCase() ?? 'INNER');
const resolveTable = (nid: string | undefined): string => {
if (!nid) return 'unknown';
const nd = nodes[nid];
if (!nd) return 'unknown';
if (nd.table) return nd.table.replace(/ \/\*.*\*\//, '');
for (const cid of nd.children) {
const t = resolveTable(cid);
if (t !== 'unknown') return t;
}
return 'unknown';
};
const rt = resolveTable(n.rightChild);
const lt = resolveTable(n.leftChild);
const onClause = n.onClause ?? `${rt}.id = ${lt}.id`;
joins.push(`${jt} JOIN ${rt} ON ${onClause}`);
if (n.leftChild) walk(n.leftChild);
if (n.rightChild) walk(n.rightChild);
return;
}
case 'Aggregate':
if (n.groupBy) {
select = `${n.groupBy}, COUNT(*)`;
groupBy = n.groupBy;
} else {
select = 'COUNT(*)';
}
break;
case 'Sort':
orderBy = n.orderBy ?? '';
break;
case 'Project':
if (n.columns && n.columns.length > 0 && !n.columns.includes('*')) {
select = n.columns.join(', ');
}
break;
case 'Limit':
limit = String(n.limit ?? '');
break;
}
n.children.forEach(walk);
}
walk(rootId);
let sql = `SELECT ${select} FROM ${[...new Set(fromTables)].join(', ')}`;
if (joins.length > 0) sql += ' ' + joins.join(' ');
if (where) sql += ` WHERE ${where}`;
if (groupBy) sql += ` GROUP BY ${groupBy}`;
if (orderBy) sql += ` ORDER BY ${orderBy}`;
if (limit) sql += ` LIMIT ${limit}`;
return sql + ';';
}
// ─── Utility ───────────────────────────────────────────────
export function stringOfPlan(nodes: Record<string, PlanNode>, rootId: string, indent: number = 0): string {
const node = nodes[rootId];
if (!node) return '';
const prefix = ' '.repeat(indent);
let line = `${prefix}${node.op}`;
if (node.table) line += ` [${node.table}]`;
if (node.predicate) line += ` {${node.predicate}}`;
if (node.joinType) line += ` (${node.joinType})`;
line += ` rows=${node.estimatedRows}`;
const children = node.children.map(c => stringOfPlan(nodes, c, indent + 1)).filter(Boolean);
return [line, ...children].join('\n');
}
// ─── Sample Statistics ──────────────────────────────────────
export const sampleStats: Record<string, TableStats> = {
users: {
rowCount: 100000,
columns: {
id: { ndv: 100000, nullFraction: 0, avgWidth: 4 },
name: { ndv: 95000, nullFraction: 0.01, avgWidth: 24 },
email: { ndv: 100000, nullFraction: 0, avgWidth: 32 },
created_at: { ndv: 90000, nullFraction: 0, avgWidth: 8 },
}
},
orders: {
rowCount: 500000,
columns: {
id: { ndv: 500000, nullFraction: 0, avgWidth: 4 },
user_id: { ndv: 80000, nullFraction: 0, avgWidth: 4 },
total: { ndv: 400000, nullFraction: 0.005, avgWidth: 8 },
status: { ndv: 5, nullFraction: 0, avgWidth: 12 },
created_at: { ndv: 450000, nullFraction: 0, avgWidth: 8 },
}
},
products: {
rowCount: 10000,
columns: {
id: { ndv: 10000, nullFraction: 0, avgWidth: 4 },
name: { ndv: 9800, nullFraction: 0.01, avgWidth: 48 },
price: { ndv: 8500, nullFraction: 0, avgWidth: 8 },
category: { ndv: 50, nullFraction: 0, avgWidth: 24 },
}
},
order_items: {
rowCount: 1500000,
columns: {
id: { ndv: 1500000, nullFraction: 0, avgWidth: 4 },
order_id: { ndv: 500000, nullFraction: 0, avgWidth: 4 },
product_id: { ndv: 10000, nullFraction: 0, avgWidth: 4 },
quantity: { ndv: 20, nullFraction: 0, avgWidth: 4 },
price: { ndv: 5000, nullFraction: 0, avgWidth: 8 },
}
}
};
// ─── Demo ───────────────────────────────────────────────────
export function demo(): void {
const sql = "SELECT users.name, COUNT(*) FROM users INNER JOIN orders ON users.id = orders.user_id WHERE users.created_at > '2024-01-01' AND orders.status = 'active' GROUP BY users.name ORDER BY COUNT(*) DESC LIMIT 10";
console.log('=== SQL Query Optimizer Demo ===\n');
console.log('Input SQL:');
console.log(sql);
console.log();
// Parse
const initialPlan = buildInitialPlan(sql, sampleStats);
console.log('Initial Plan:');
console.log(stringOfPlan(initialPlan.nodes, initialPlan.root));
const initialCost = estimateCost(initialPlan.nodes, initialPlan.root);
console.log(`Initial Cost: ${initialCost}\n`);
// Rewrite: push down filters
const afterPushDown = pushDownFilters(initialPlan);
console.log('After Predicate Pushdown:');
console.log(stringOfPlan(afterPushDown.nodes, afterPushDown.root));
const pushDownCost = estimateCost(afterPushDown.nodes, afterPushDown.root);
console.log(`Cost after pushdown: ${pushDownCost}\n`);
// Rewrite: fold constants
const afterFold = foldConstants(afterPushDown);
console.log('After Constant Folding:');
console.log(stringOfPlan(afterFold.nodes, afterFold.root));
const foldCost = estimateCost(afterFold.nodes, afterFold.root);
console.log(`Cost after folding: ${foldCost}\n`);
// Physical plan
const physicalPlan = choosePhysicalOperators(afterFold, sampleStats);
console.log('Physical Plan:');
console.log(stringOfPlan(physicalPlan.nodes, physicalPlan.root));
const physicalCost = estimateCost(physicalPlan.nodes, physicalPlan.root);
console.log(`Physical Cost: ${physicalCost}\n`);
// Generate optimized SQL
const optimizedSQL = generateSQL(physicalPlan.nodes, physicalPlan.root);
console.log('Optimized SQL:');
console.log(optimizedSQL);
console.log();
console.log('=== Summary ===');
console.log(`Initial cost: ${initialCost}`);
console.log(`Final cost: ${physicalCost}`);
console.log(`Improvement: ${((1 - physicalCost / initialCost) * 100).toFixed(1)}%`);
}

298
examples/SQLOptimizer.speckdl
View file

@ -1,298 +0,0 @@
// SQLOptimizer.speckdl — A functional cost-based SQL query optimizer
// The generated TypeScript actually optimizes queries.
// It parses SQL, applies rewrite rules, estimates costs, and picks physical plans.
speck SQLOptimizer {
version: "0.2.0"
author: "sscoble"
license: "MIT"
// ─── State ────────────────────────────────────────────────
state: {
phase: OptimizerPhase,
originalSQL: String,
planNodes: PlanNodeMap,
planEdges: PlanEdgeMap,
rootPlanId: String,
catalog: CatalogMap,
stats: StatsMap,
rulesApplied: Nat,
bestCost: Nat,
resultSQL: String,
errors: StringList
}
init: {
phase := Idle,
originalSQL := "",
planNodes := {},
planEdges := {},
rootPlanId := "",
catalog := {},
stats := {},
rulesApplied := 0,
bestCost := 0,
resultSQL := "",
errors := []
}
// ─── Events ───────────────────────────────────────────────
event OptimizationStarted {
sql: String,
at: String
}
event RuleApplied {
ruleName: String,
planBefore: String,
planAfter: String,
costDelta: Nat,
at: String
}
event PlanOptimized {
originalSQL: String,
optimizedSQL: String,
originalCost: Nat,
optimizedCost: Nat,
rulesApplied: Nat,
at: String
}
event OptimizationFailed {
phase: OptimizerPhase,
message: String,
at: String
}
// ─── Actions ──────────────────────────────────────────────
// Accept a SQL query string and begin optimization
action submit(sql: String) {
require phase === Idle
require len(sql) > 0
this.phase := Parsing
this.originalSQL := sql
this.errors := []
emit OptimizationStarted { sql: sql, at: now() }
}
// Parse SQL into a logical plan tree
// Builds PlanNode objects representing Scan, Filter, Join, Project, Aggregate, Sort
action parse() {
require phase === Parsing
require len(originalSQL) > 0
// Parse SELECT ... FROM ... WHERE ... GROUP BY ... ORDER BY ...
let sql := originalSQL
// Extract FROM clause — identify table names
let fromClause := extractFrom(sql)
if len(fromClause) == 0 {
this.phase := Failed
this.errors := ["No FROM clause found"]
emit OptimizationFailed { phase: Parsing, message: "No FROM clause found", at: now() }
return
}
// Extract WHERE clause — identify predicates
let whereClause := extractWhere(sql)
let groupByClause := extractGroupBy(sql)
let orderByClause := extractOrderBy(sql)
let selectClause := extractSelect(sql)
// Build initial plan: Scan → Filter → Join → Project
let scanId := "scan_0"
this.planNodes := {}
this.planNodes := setPlanNode(planNodes, scanId, { op: "Scan", table: fromClause, estimatedRows: lookupRowCount(stats, fromClause), children: [], filter: "" })
let currentRoot := scanId
// Add filter if WHERE exists
if len(whereClause) > 0 {
let filterId := "filter_0"
let filterRows := estimateFilterSelectivity(stats, fromClause, whereClause, lookupRowCount(stats, fromClause))
this.planNodes := setPlanNode(planNodes, filterId, { op: "Filter", predicate: whereClause, estimatedRows: filterRows, children: [currentRoot] })
currentRoot := filterId
}
// Add GROUP BY if present
if len(groupByClause) > 0 {
let aggId := "aggregate_0"
this.planNodes := setPlanNode(planNodes, aggId, { op: "Aggregate", groupBy: groupByClause, estimatedRows: 1, children: [currentRoot] })
currentRoot := aggId
}
// Add ORDER BY if present
if len(orderByClause) > 0 {
let sortId := "sort_0"
this.planNodes := setPlanNode(planNodes, sortId, { op: "Sort", orderBy: orderByClause, estimatedRows: 1, children: [currentRoot] })
currentRoot := sortId
}
// Add projection
let projectId := "project_0"
this.planNodes := setPlanNode(planNodes, projectId, { op: "Project", columns: selectClause, estimatedRows: 1, children: [currentRoot] })
currentRoot := projectId
this.rootPlanId := currentRoot
this.bestCost := estimateCost(planNodes, rootPlanId)
this.phase := Rewriting
}
// Apply logical rewrite rules
action rewrite() {
require phase === Rewriting
let costBefore := bestCost
let rulesThisRound := 0
// Rule 1: Predicate pushdown — move filters closer to scans
let pushed := pushDownFilters(planNodes, rootPlanId)
let pushedCost := estimateCost(pushed.nodes, pushed.root)
if pushedCost < bestCost {
this.planNodes := pushed.nodes
this.rootPlanId := pushed.root
this.bestCost := pushedCost
this.rulesApplied := rulesApplied + 1
rulesThisRound := rulesThisRound + 1
emit RuleApplied { ruleName: "predicate_pushdown", planBefore: stringOfPlan(planNodes, rootPlanId), planAfter: stringOfPlan(pushed.nodes, pushed.root), costDelta: costBefore - pushedCost, at: now() }
costBefore := pushedCost
}
// Rule 2: Column pruning — remove unused columns from projections
let pruned := pruneUnusedColumns(planNodes, rootPlanId)
let prunedCost := estimateCost(pruned.nodes, pruned.root)
if prunedCost < bestCost {
this.planNodes := pruned.nodes
this.rootPlanId := pruned.root
this.bestCost := prunedCost
this.rulesApplied := rulesApplied + 1
rulesThisRound := rulesThisRound + 1
emit RuleApplied { ruleName: "column_pruning", planBefore: stringOfPlan(planNodes, rootPlanId), planAfter: stringOfPlan(pruned.nodes, pruned.root), costDelta: costBefore - prunedCost, at: now() }
costBefore := prunedCost
}
// Rule 3: Constant folding — evaluate compile-time expressions
let folded := foldConstants(planNodes, rootPlanId)
let foldedCost := estimateCost(folded.nodes, folded.root)
if foldedCost < bestCost {
this.planNodes := folded.nodes
this.rootPlanId := folded.root
this.bestCost := foldedCost
this.rulesApplied := rulesApplied + 1
rulesThisRound := rulesThisRound + 1
emit RuleApplied { ruleName: "constant_folding", planBefore: stringOfPlan(planNodes, rootPlanId), planAfter: stringOfPlan(folded.nodes, folded.root), costDelta: costBefore - foldedCost, at: now() }
}
this.phase := Planning
}
// Choose physical operators and join order
action planPhysical() {
require phase === Planning
// Choose join algorithm based on table sizes
let physical := choosePhysicalOperators(planNodes, rootPlanId, stats)
let physicalCost := estimateCost(physical.nodes, physical.root)
if physicalCost < bestCost {
this.planNodes := physical.nodes
this.rootPlanId := physical.root
this.bestCost := physicalCost
}
this.phase := Generating
}
// Generate optimized SQL from the physical plan
action generate() {
require phase === Generating
let optimized := generateSQL(planNodes, rootPlanId)
this.resultSQL := optimized
this.phase := Done
emit PlanOptimized {
originalSQL: originalSQL,
optimizedSQL: optimized,
originalCost: estimateCost(planNodes, rootPlanId),
optimizedCost: bestCost,
rulesApplied: rulesApplied,
at: now()
}
}
// Reset for next query
action reset() {
this.phase := Idle
this.originalSQL := ""
this.planNodes := {}
this.planEdges := {}
this.rootPlanId := ""
this.catalog := {}
this.stats := {}
this.rulesApplied := 0
this.bestCost := 0
this.resultSQL := ""
this.errors := []
}
// ─── Constraints ──────────────────────────────────────────
constraint C1: phase === Idle || len(originalSQL) > 0
// Non-idle implies a query is loaded
constraint C2: phase === Done implies len(resultSQL) > 0
// Done implies we produced output
constraint C3: phase === Failed implies len(errors) > 0
// Failure implies error messages exist
constraint C4: rulesApplied > 0 implies bestCost < estimateCost(planNodes, rootPlanId) + bestCost
// Rules only fire when they reduce cost (loose — cost can't increase from applying rules)
// ─── Temporal Verifications ───────────────────────────────
verify V1: always(implies(phase === Parsing, eventually(phase === Rewriting || phase === Failed)))
verify V2: always(implies(phase === Rewriting, eventually(phase === Planning || phase === Failed)))
verify V3: always(implies(phase === Generating, eventually(phase === Done)))
// ─── Provenance ───────────────────────────────────────────
provenance {
wasGeneratedBy(sql-optimizer, SQLOptimizer, "2026-05-13T00:00:00Z")
used(SQLOptimizer, cascades-volcano, "framework")
}
// ─── BOM ──────────────────────────────────────────────────
bom {
optimizer: "sql-optimizer@0.2.0"
runtime: "node@22.0.0"
}
// ─── Supporting Types ─────────────────────────────────────
interface OptimizerPhase {
Idle, Parsing, Rewriting, Planning, Generating, Done, Failed
}
interface PlanNodeMap {
// Map from plan node ID to PlanNode object
}
interface PlanEdgeMap {
// Map from parent ID to child IDs
}
interface CatalogMap {
// Map from table name to schema info
}
interface StatsMap {
// Map from table name to row count and column statistics
}
interface StringList {
// List of strings
}
}

276
examples/SQLitePager.speck
View file

@ -1,276 +0,0 @@
// SQLite Pager State Machine Specification (speckl#35)
//
// Models the pager storage engine's state machine: page cache, transactions,
// journal files, and crash recovery. Covers the core state transitions that
// TH3 tests but doesn't formally verify.
//
// Based on: docs/db-sqlite-reference.md Section 1
// Scope: Pager state machine, journal modes, lock escalation, error recovery
// Does NOT cover: WAL mode (separate spec), VDBE, B-Tree internal structure
speck SQLitePager {
author: "Engineering (agent-engineering) <engineering@marcus.local>"
source: architecture_review ref "SQLite-Pager-Core"
review: hybrid
derives from docs/db-sqlite-reference.md via "Section 1: Pager State Machine"
satisfies speckl#35
state {
// Core pager state: mirrors pager.c PAGER_* enum
pagerState: String
// Page cache counters
pageCount: Nat
dirtyPageCount: Nat
// Journal tracking
journalMode: String
journalExists: Bool
syncNeeded: Bool
// Transaction tracking
activeTransaction: Bool
transactionType: String
// Error state
errorState: Bool
errCode: Nat
// Lock management
currentLock: String
readOnly: Bool
// Dirty page set: pageId → isDirty
dirtyPages: Set(Nat)
}
init {
pagerState == "Idle"
pageCount == 0
dirtyPageCount == 0
journalMode == "Delete"
journalExists == false
syncNeeded == false
activeTransaction == false
transactionType == "None"
errorState == false
errCode == 0
currentLock == "NO_LOCK"
readOnly == false
dirtyPages == emptySet
}
// --- Safety Invariants ---
invariant CommitAtomicity {
// After commit, all dirty pages are flushed and journal is deleted.
// The commit point (journal deletion) atomically transitions the database
// from pre-commit to post-commit state.
dirtyPageCount >= 0
}
invariant RollbackSafety {
// After rollback completes, no dirty pages remain.
// The database matches the pre-transaction state.
dirtyPageCount == 0
}
invariant JournalConsistency {
// For DELETE mode, journal exists iff write transaction is in progress.
journalExists == true -> activeTransaction == true
}
invariant PageCacheCoherence {
// Every dirty page belongs to the current active transaction.
activeTransaction == true and dirtyPageCount > 0 -> pagerState != Idle
}
invariant LockProgression {
// Locks only escalate in the defined order.
// Valid lock values: NO_LOCK < SHARED < RESERVED < PENDING < EXCLUSIVE.
currentLock != ""
}
invariant ErrorRecovery {
// Once in error state, no new transactions can start.
errorState == true -> activeTransaction == false
}
// --- Actions (8 core operations) ---
action OpenDatabase(filename: String) {
require pagerState == "Idle"
require errorState == false
// Initialize pager, allocate page cache
pagerState := "Idle"
pageCount := 0
dirtyPageCount := 0
journalExists := false
syncNeeded := false
activeTransaction := false
transactionType := "None"
currentLock := "NO_LOCK"
dirtyPages := emptySet
emit DatabaseOpened { filename: filename }
}
action BeginReadTransaction() {
require pagerState == "Idle"
require errorState == false
require readOnly == false or currentLock == "NO_LOCK"
// Acquire SHARED lock, enter Reading state
currentLock := "SHARED"
pagerState := "Reading"
activeTransaction := true
transactionType := "Read"
emit ReadTransactionStarted {}
}
action BeginWriteTransaction() {
require pagerState == "Reading" or pagerState == "Idle"
require errorState == false
require readOnly == false
// Acquire RESERVED lock, create journal, enter Writing state
if currentLock == "SHARED" or currentLock == "NO_LOCK" {
currentLock := "RESERVED"
}
pagerState := "Writing"
activeTransaction := true
transactionType := "Write"
journalExists := true
emit WriteTransactionStarted {}
}
action WritePage(pageId: Nat, data: Bytes) {
require pagerState == "Writing"
require activeTransaction == true
require errorState == false
// Mark page as dirty in the page cache
dirtyPages := dirtyPages union {pageId}
dirtyPageCount := dirtyPageCount + 1
emit PageWritten { pageId: pageId, size: 0 }
}
action CommitTransaction() {
require pagerState == "Writing"
require activeTransaction == true
require transactionType == "Write"
require errorState == false
// Acquire PENDING then EXCLUSIVE lock
currentLock := "PENDING"
currentLock := "EXCLUSIVE"
// Flush dirty pages, sync journal
syncNeeded := true
pagerState := "CommitPending"
// Delete journal (commit point)
journalExists := false
syncNeeded := false
// Transition back to Idle
pagerState := "Idle"
dirtyPages := emptySet
dirtyPageCount := 0
activeTransaction := false
transactionType := "None"
currentLock := "SHARED"
emit TransactionCommitted {}
}
action RollbackTransaction() {
require activeTransaction == true
require transactionType == "Write" or transactionType == "Read"
require errorState == false
// Restore pages from journal, release lock
if transactionType == "Write" {
journalExists := false
}
// Reset all transaction state
pagerState := "Idle"
dirtyPages := emptySet
dirtyPageCount := 0
activeTransaction := false
transactionType := "None"
currentLock := "NO_LOCK"
emit TransactionRolledBack {}
}
action Checkpoint() {
require journalMode == "Wal"
require pagerState == "Idle"
require errorState == false
// WAL checkpoint: copy frames back to database
// In non-WAL mode, checkpoint is a no-op
emit CheckpointCompleted {}
}
action CloseDatabase() {
require pagerState == "Idle"
require activeTransaction == false
// Release all resources, reset to initial state
currentLock := "NO_LOCK"
pagerState := "Idle"
pageCount := 0
dirtyPageCount := 0
journalExists := false
syncNeeded := false
dirtyPages := emptySet
emit DatabaseClosed {}
}
// --- Transition Relation ---
next:
OpenDatabase
| BeginReadTransaction
| BeginWriteTransaction
| WritePage
| CommitTransaction
| RollbackTransaction
| Checkpoint
| CloseDatabase
// --- Bounded Model Checking ---
verify Always(CommitAtomicity) {
depth 5
}
verify Always(RollbackSafety) {
depth 5
}
verify Always(JournalConsistency) {
depth 5
}
verify Always(PageCacheCoherence) {
depth 5
}
verify Always(LockProgression) {
depth 10
}
verify Always(ErrorRecovery) {
depth 5
}
}

267
examples/SQLiteWAL.speck
View file

@ -1,267 +0,0 @@
speck SQLiteWAL {
// SQLite WAL mode concurrency model
// Specifies the Write-Ahead Log protocol: concurrent readers + single writer,
// checkpoint lifecycle, and snapshot consistency.
//
// Uses only built-in types (Nat, Bool, Map) for Z3 compatibility.
// Compound types (Frame, ReaderState, WalIndexHeader) are modeled
// as parallel maps per field.
// --- Type aliases (Nat-based, Z3-compatible) ---
// FrameSeq: sequence number for WAL frames
// ReaderId: integer index for reader slots (0..4)
// LockType: 0=READER, 1=WRITER, 2=CHECKPOINTER, 3=NONE
// CheckpointMode: 0=PASSIVE, 1=FULL, 2=RESTART, 3=TRUNCATE
// PageId: database page identifier
state {
// WAL frame storage: frame sequence → data per frame
walFrames: Map(Nat, Nat) // frameId → pageId (0 = empty slot)
walFrameSeq: Map(Nat, Nat) // frameId → seq number
nFrames: Nat // number of frames in WAL
maxFrames: Nat // maximum frames before checkpoint needed
// Reader management (max 5 concurrent readers)
readerEndMark: Map(Nat, Nat) // readerId → end mark, 0 = inactive
nReaders: Nat // number of active readers
// Writer state
writerActive: Bool // is writer in progress?
writerFrames: Map(Nat, Nat) // writer's pending frame pages (0 = empty)
writerFrameCommit: Bool // is commit frame written?
// Checkpoint state
checkpointActive: Bool // is checkpoint in progress?
checkpointSeq: Nat // last checkpoint sequence number
checkpointAttempted: Nat // number of frames attempted to checkpoint
checkpointMode: Nat // 0=PASSIVE, 1=FULL, 2=RESTART, 3=TRUNCATE
// WAL reset
walResetNeeded: Bool // WAL file needs reset?
// Database state
dbContents: Map(Nat, Nat) // pageId → content (0 = uninitialized)
// Error recovery
errorState: Bool
errorCode: Nat
}
init {
walFrames == {0: 0}
walFrameSeq == {0: 0}
nFrames == 0
maxFrames == 100
readerEndMark == {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}
nReaders == 0
writerActive == false
writerFrames == {0: 0}
writerFrameCommit == false
checkpointActive == false
checkpointSeq == 0
checkpointAttempted == 0
checkpointMode == 0
walResetNeeded == false
dbContents == {0: 0}
errorState == false
errorCode == 0
}
// ============================================================
// Actions
// ============================================================
action BeginRead(readerId: Nat) {
require readerId >= 0 and readerId < 5 // valid slot
require nReaders < 5 // room for reader
require readerEndMark[readerId] == 0 // not already active
require not errorState
readerEndMark[readerId] := nFrames
nReaders := nReaders + 1
}
action ReadFrame(readerId: Nat, frameId: Nat) {
require readerEndMark[readerId] > 0 // reader is active
require frameId >= 1 and frameId <= nFrames // frame exists
require frameId <= readerEndMark[readerId] // frame is within reader's snapshot
// Reading is nondestructive — no state change
}
action EndRead(readerId: Nat) {
require readerEndMark[readerId] > 0 // reader is active
require not errorState
readerEndMark[readerId] := 0
nReaders := nReaders - 1
}
action BeginWrite() {
require not writerActive
require not checkpointActive
require not errorState
require nFrames < maxFrames // room for at least one frame
writerActive := true
writerFrameCommit := false
}
action AppendFrame(frameId: Nat, pageId: Nat) {
require writerActive // writer must be active
require not writerFrameCommit // no commit yet
require frameId < maxFrames // valid frame slot
require nFrames < maxFrames
nFrames := nFrames + 1
walFrames[nFrames] := pageId
walFrameSeq[nFrames] := nFrames
writerFrames[nFrames] := pageId
}
action CommitWrite() {
require writerActive
require not writerFrameCommit
require not errorState
writerFrameCommit := true
writerActive := false
}
action BeginCheckpoint(mode: Nat) {
require not checkpointActive
require not errorState
require mode >= 0 and mode <= 3 // valid mode
checkpointActive := true
checkpointMode := mode
checkpointAttempted := 0
}
action MoveFrame(frameId: Nat) {
require checkpointActive
require frameId >= 1 and frameId <= nFrames
require frameId > checkpointSeq // not already checkpointed
// Safety: don't checkpoint frames visible to active readers
require forall r in [0..4]:
(readerEndMark[r] == 0) or (frameId <= readerEndMark[r])
// Move frame to database
dbContents[walFrames[frameId]] := nFrames
checkpointAttempted := checkpointAttempted + 1
}
action FinishCheckpoint() {
require checkpointActive
require not errorState
checkpointSeq := nFrames // all frames checkpointed
checkpointActive := false
walResetNeeded := true
}
action ResetWAL() {
require walResetNeeded
require nReaders == 0 // no active readers
require not checkpointActive
require not writerActive
require not errorState
nFrames := 0
walFrames := {0: 0}
walFrameSeq := {0: 0}
checkpointSeq := 0
checkpointAttempted := 0
walResetNeeded := false
}
action InjectError(err: Nat) {
require not errorState
errorState := true
errorCode := err
}
action RecoverFromError() {
require errorState
errorState := false
errorCode := 0
// Reset to safe state
writerActive := false
checkpointActive := false
writerFrameCommit := false
}
// ============================================================
// Invariants
// ============================================================
// I1: Read consistency — a reader's end mark stays valid during its read
// A reader sees frames 1..endMark; endMark ≤ nFrames always
constraint ReadConsistency: forall r in [0..4]:
(readerEndMark[r] == 0) or (readerEndMark[r] <= nFrames)
// I2: Write serialization — at most one writer at a time
// No concurrent writer and checkpoint
constraint WriteSerialization:
(not writerActive) or (not checkpointActive)
// I3: Checkpoint safety — checkpoint doesn't remove frames needed by readers
// Checkpoint never advances past the oldest active reader's end mark
constraint CheckpointSafety:
forall r in [0..4]:
(readerEndMark[r] == 0) or (checkpointSeq <= readerEndMark[r])
// I4: Frame ordering — frame sequence numbers are monotonic
// All frames have unique increasing seq numbers
constraint FrameOrdering:
nFrames <= maxFrames
// I5: Snapshot integrity — WAL reset only happens when no readers
constraint WalResetCondition:
walResetNeeded implies (nReaders == 0)
// I6: Writer commit integrity — commit implies at least one frame appended
constraint CommitIntegrity:
writerFrameCommit implies (nFrames > 0)
// I7: No deadlock — writer and checkpointer exclude each other
constraint LockDeadlockFree:
(not (writerActive and checkpointActive))
// I8: Commit durability — committed frames are checkpointed or still in WAL
constraint CommitDurability:
checkpointSeq <= nFrames
// I9: Backfill bound — checkpoint doesn't exceed total frames
constraint BackfillBound:
checkpointAttempted <= nFrames
// I10: Error recovery — after error, all operations blocked
constraint ErrorBlocked:
not (errorState and (writerActive or checkpointActive))
// ============================================================
// Transition relation
// ============================================================
next: {
// Allow any action to fire, constrained by its require guards
// The checker unrolls this to N steps, checking invariants at each step
}
// ============================================================
// Verify
// ============================================================
verify Always(ReadConsistency) { depth 10 }
verify Always(WriteSerialization) { depth 5 }
verify Always(CheckpointSafety) { depth 10 }
verify Always(FrameOrdering) { depth 5 }
verify Always(WalResetCondition) { depth 5 }
verify Always(CommitIntegrity) { depth 10 }
verify Always(LockDeadlockFree) { depth 5 }
verify Always(CommitDurability) { depth 10 }
verify Always(BackfillBound) { depth 5 }
verify Always(ErrorBlocked) { depth 5 }
}

30
fix-z3-again.py
View file

@ -1,30 +0,0 @@
import sys
content = open('compiler/src/generators/z3.ts').read()
lines = content.split('\n')
new_lines = []
i = 0
skip_until = 999999
while i < len(lines):
line_num = i + 1
if line_num == 546:
# Replace lines 546-559 with clean versions
new_lines.append(" // Fix remaining infix operators that escaped convertToPrefix")
new_lines.append(" // word = word/val: convert to (= word val)")
new_lines.append(" e = e.replace(/(\\b\\w+)\\s*=\\s*(\\w+|\\d+|Bool|true|false)\\b/g, '(= $1 $2)');")
new_lines.append(" // (innermost expr) = val: convert to (= (expr) val)")
new_lines.append(" e = e.replace(/\\(([^()]+)\\)\\s*=\\s*(\\w+|\\d+|Bool|true|false)\\b/g, '(= ($1) $2)');")
new_lines.append(" // word <= word: convert to (<= word word)")
new_lines.append(" e = e.replace(/(\\b\\w+)\\s*(<=|>=)\\s*(\\w+|\\d+|Bool|true|false)\\b/g, '($2 $1 $3)');")
new_lines.append(" // (innermost expr) <= val: convert to (<= (expr) val)")
new_lines.append(" e = e.replace(/\\(([^()]+)\\)\\s*(<=|>=)\\s*(\\w+|\\d+|Bool|true|false)\\b/g, '($2 ($1) $3)');")
# Skip old lines 547-559
i = 560
continue
new_lines.append(lines[i])
i += 1
open('compiler/src/generators/z3.ts', 'w').write('\n'.join(new_lines))
print('Done')

73
landing/index.html
View file

@ -4,33 +4,13 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Speckl — A Judgement Transport Utility for AI-Generated Specifications</title>
<meta name="description" content="Speckl bridges the gap between what you specify and what gets built. State machine specs compile to auditable code — TypeScript, Z3 verification, and provenance artifacts. NIST SA-11 human-in-the-loop for AI-generated outputs.">
<meta name="author" content="Scott Scoble">
<meta name="description" content="Speckl bridges the gap between what you specify and what gets built. State machine specs compile to auditable code — TypeScript, WASM, and provenance artifacts. NIST SA-11 human-in-the-loop for AI-generated outputs.">
<link rel="icon" href="favicon.svg" type="image/svg+xml">
<link rel="canonical" href="https://speckl.scoble.me/">
<meta property="og:title" content="Speckl — Judgement Transport for AI Specifications">
<meta property="og:description" content="State machine specs → auditable code. Compile SpeckDL to TypeScript + Z3 SMT + provenance artifacts. Open source (MIT). Created by Scott Scoble.">
<meta property="og:description" content="State machine specs → auditable code. Compile SpeckDL to TypeScript + WASM + provenance artifacts. Open source (MIT).">
<meta property="og:type" content="website">
<meta property="og:url" content="https://speckl.scoble.me/">
<meta property="og:site_name" content="Speckl by Scott Scoble">
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "SoftwareSourceCode",
"name": "Speckl",
"description": "A judgement transport utility for AI-generated specifications. Compiles SpeckDL state machine specs to TypeScript, Z3 verification, and provenance artifacts.",
"author": {
"@type": "Person",
"name": "Scott Scoble",
"url": "https://scott.scoble.me",
"sameAs": ["https://github.com/sscoble", "https://linkedin.com/in/scottscoble"]
},
"codeRepository": "https://os.scoble.me/forgejo/sscoble/speckl",
"license": "https://opensource.org/licenses/MIT",
"programmingLanguage": "TypeScript",
"keywords": ["speckl", "speckdl", "formal-specification", "state-machines", "provenance", "compliance", "scott-scoble"]
}
</script>
<style>
:root {
--bg: #0a0f0a;
@ -265,7 +245,7 @@
<header class="hero">
<div class="hero-content">
<h1>Speckl</h1>
<p class="hero-subtitle">A Judgement Transport Utility by Scott Scoble</p>
<p class="hero-subtitle">A Judgement Transport Utility</p>
<p class="hero-tagline">
Your specification is the source of truth. Speckl ensures what gets built — by humans or AI — stays faithful to what you specified. Compile state machine specs into auditable code with embedded provenance.
</p>
@ -304,9 +284,9 @@
<p>Runnable state machine class</p>
</div>
<div class="pipeline-step">
<div class="icon">🔍</div>
<h4>Z3 Verification</h4>
<p>SMT-LIB2 output from --target=z3</p>
<div class="icon">🔧</div>
<h4>WASM</h4>
<p>Verified WebAssembly binary</p>
</div>
</div>
@ -315,8 +295,8 @@
<ul class="check-list">
<li>Provenance is <strong>embedded by construction</strong> — not bolted on after the fact</li>
<li>41 tests passing, < 2 seconds to run the full suite</li>
<li>14 example specks: ToggleSwitch → TwoPhaseCommit → Kafka KRaft → Paxos → Raft</li>
<li>Dual-target: TypeScript state machine classes + Z3 SMT verification from the same spec</li>
<li>11 example specks: ToggleSwitch → TwoPhaseCommit → Paxos → Raft</li>
<li>Dual-target: TypeScript state machine classes + verified WASM from the same spec</li>
<li>Zero external services — everything runs locally</li>
</ul>
</section>
@ -325,7 +305,7 @@
<h2>SpeckDL by Example</h2>
<p>SpeckDL is a typed specification language for state machines. Define your states, actions, preconditions, and mutations — the compiler handles the rest.</p>
<div class="code-block"><pre><span class="cm">// A simple toggle switch — compiles to TypeScript + 3 provenance artifacts</span>
<div class="code-block"><pre><span class="cm">// A simple toggle switch — compiles to TypeScript + WASM + 3 provenance artifacts</span>
<span class="kw">spec</span> <span class="ty">ToggleSwitch</span> {
<span class="kw">type</span> <span class="ty">Status</span> = <span class="ty">enum</span> { <span class="st">On</span>, <span class="st">Off</span> }
@ -369,11 +349,6 @@
<div class="meta">45 lines · 5 actions</div>
<p>Classic distributed transaction coordinator. Voting phase, commit phase, abort path — fully specified with guard conditions.</p>
</div>
<div class="example-card">
<h4>Kafka KRaft</h4>
<div class="meta">141 lines · 6 safety invariants · BMC verify blocks</div>
<p>Flagship example: leader election, log replication, and safety properties verified with Z3 SMT. Don't replace Kafka. Prove it.</p>
</div>
<div class="example-card">
<h4>Paxos</h4>
<div class="meta">120 lines · 3 safety invariants</div>
@ -395,27 +370,13 @@
<p>Role-based access with role addition, removal, and permission checks. One of 5 examples with zero TypeScript errors.</p>
</div>
<div class="example-card">
<h4>+ 8 more</h4>
<div class="meta">OAuth2AuthorizationCode · DrugDosageCalculator · PaymentService · RetryHandler · TemperatureMonitor · AccountLedger · TigerBeetle · PreconditionTest</div>
<p><a href="https://os.scoble.me/forgejo/sscoble/speckl/src/branch/main/examples">Browse all 14 examples →</a></p>
<h4>+ 6 more</h4>
<div class="meta">DrugDosageCalculator · PaymentService · RetryHandler · TemperatureMonitor · AccountLedger · TigerBeetle</div>
<p><a href="https://os.scoble.me/forgejo/sscoble/speckl/src/branch/main/examples">Browse all 11 examples →</a></p>
</div>
</div>
</section>
<section id="status">
<h2>Status &amp; Roadmap</h2>
<ul class="check-list">
<li><strong>TypeScript codegen:</strong> Complete — state machine classes with full type coverage</li>
<li><strong>Z3 verification backend:</strong> Live — <code>--target=z3</code> emits SMT-LIB2 that Z3 accepts</li>
<li><strong>OAuth 2.0 spec:</strong> Exists — <code>examples/OAuth2AuthorizationCode.speck</code></li>
<li><strong>Rust codegen:</strong> In progress (M2.1)</li>
<li><strong>WASM via wasm-pack:</strong> Pending Rust codegen completion</li>
</ul>
<p>The WAT/WASM generator was removed in favor of a proper IR → Rust → wasm-pack pipeline. No hand-rolled WASM text format; no magic.</p>
</section>
<section id="strategy">
<h2>Strategy Ladder</h2>
<p>Speckl follows a four-step strategy — each step validates the market before building the next:</p>
@ -459,14 +420,14 @@ git clone https://os.scoble.me/forgejo/sscoble/speckl.git
npm install
npm test
<span class="cm"># Compile a spec to TypeScript + provenance artifacts</span>
<span class="cm"># Compile a spec to all 5 artifacts</span>
node dist/index.js ../examples/TwoPhaseCommit.speck -o out/twophase
<span class="cm"># Compile to Z3 SMT-LIB2 for verification</span>
node dist/index.js ../examples/KafkaKRaft.speck --target=z3 -o out/kraft
<span class="cm"># Check TypeScript validity</span>
npx tsc --noEmit -p out/twophase</pre></div>
npx tsc --noEmit -p out/twophase
<span class="cm"># Validate WASM (requires wabt)</span>
wat2wasm out/twophase/TwoPhaseCommit.wat</pre></div>
</section>
<footer>

38
out/KafkaKRaft.prov.jsonld
View file

@ -1,38 +0,0 @@
{
"@context": {
"prov": "http://www.w3.org/ns/prov#",
"speckl": "https://speckl.scoble.me/ns/v0.2#",
"rdfs": "http://www.w3.org/2000/01/rdf-schema#"
},
"@graph": [
{
"@type": "prov:Entity",
"@id": "speck:KafkaKRaft",
"prov:qualifiedGeneration": {
"@type": "prov:Generation",
"prov:activity": {
"@type": "prov:Activity",
"prov:startTime": "2026-05-10T09:08:14.888Z"
}
},
"rdfs:label": "KafkaKRaft",
"prov:wasAttributedTo": {
"@id": "agent:Speckl Compiler v0.3"
},
"prov:wasInformedBy": {
"@id": "source:architecture_review:KRaft-Core-Consensus"
}
},
{
"@type": "prov:Agent",
"@id": "agent:Speckl Compiler v0.3",
"rdfs:label": "Speckl Compiler v0.3"
},
{
"@type": "prov:Activity",
"@id": "source:architecture_review:KRaft-Core-Consensus",
"prov:type": "architecture_review",
"rdfs:label": "architecture_review - KRaft-Core-Consensus"
}
]
}

2492
out/KafkaKRaft.smt2
View file

File diff suppressed because it is too large Load diff

60
out/KafkaKRaft.specbom.cdx.json
View file

@ -1,60 +0,0 @@
{
"bomFormat": "CycloneDX",
"specVersion": "1.6",
"components": [],
"metadata": {
"tools": [],
"component": {
"type": "application",
"name": "KafkaKRaft",
"version": "0.2.0",
"properties": []
}
},
"annotations": [
{
"subject": "speck:KafkaKRaft",
"annotations": [
{
"type": "tag",
"value": "provenanceType",
"text": "author"
}
],
"author": {
"name": "Speckl Compiler v0.3"
},
"timestamp": "2026-05-10T09:08:14.915Z"
},
{
"subject": "speck:KafkaKRaft",
"annotations": [
{
"type": "tag",
"value": "provenanceType",
"text": "source"
}
],
"author": {
"name": "Speckl Compiler",
"email": "compiler@speckl.scoble.me"
},
"timestamp": "2026-05-10T09:08:14.915Z"
},
{
"subject": "speck:KafkaKRaft",
"annotations": [
{
"type": "tag",
"value": "reviewType",
"text": "hybrid"
}
],
"author": {
"name": "Speckl Compiler",
"email": "compiler@speckl.scoble.me"
},
"timestamp": "2026-05-10T09:08:14.915Z"
}
]
}

69
out/KafkaKRaft.specbom.spdx.json
View file

@ -1,69 +0,0 @@
{
"@context": [
"https://spdx.github.io/spdx-v3/spdx-context.jsonld",
{
"speckl": "https://speckl.scoble.me/ns/v0.2#"
}
],
"@type": "spdx:Document",
"spdx": {
"@id": "spdx:KafkaKRaft",
"@type": "spdx:Snippet",
"name": "KafkaKRaft",
"spdxVersion": "3.0.1",
"dataLicense": "CC0-1.0",
"creationInfo": {
"created": "2026-05-10T09:08:14.915Z",
"createdBy": [
"Organization: Speckl",
"tool: speckl-compile-0.2.0"
]
},
"annotations": [
{
"@type": "spdx:Annotation",
"annotationType": "spdx:REVIEW",
"annotator": {
"@type": "spdx:Tool",
"name": "speckl-compile"
},
"timestamp": "2026-05-10T09:08:14.915Z",
"comment": "Review type: hybrid"
}
]
},
"builtBy": [],
"relationships": [
{
"from": {
"@id": "spdx:KafkaKRaft"
},
"to": {
"@id": "spdx:agent:Speckl Compiler v0.3"
},
"relationshipType": "spdx:wasAttributedTo"
},
{
"from": {
"@id": "spdx:KafkaKRaft"
},
"to": {
"@id": "spdx:source:architecture_review:KRaft-Core-Consensus"
},
"relationshipType": "spdx:wasInformedBy"
}
],
"elements": [
{
"@type": "spdx:Person",
"@id": "spdx:agent:Speckl Compiler v0.3",
"name": "Speckl Compiler v0.3"
},
{
"@type": "spdx:Activity",
"@id": "spdx:source:architecture_review:KRaft-Core-Consensus",
"name": "architecture_review - KRaft-Core-Consensus",
"type": "architecture_review"
}
]
}

59
out/KafkaKRaft.ts
View file

@ -1,59 +0,0 @@
// Auto-generated by speckl-compile from KafkaKRaft
// DO NOT EDIT MANUALLY — regenerate from .speckdl source
export const KafkaKRaftState = {
Candidate: 'candidate',
Leader: 'leader',
} as const;
export type KafkaKRaftState = typeof KafkaKRaftState[keyof typeof KafkaKRaftState];
export class KafkaKRaftMachine {
currentTerm: number;
state: KafkaKRaftState;
votedFor: Option(NodeId);
log: LogEntry[];
commitIndex: number;
nextIndex: Map<NodeId, number>;
matchIndex: Map<NodeId, number>;
votesGranted: Set<NodeId>;
decisionLog: any[] = [];
constructor() {
// No init block defined
}
RequestVote(candidate: NodeId, lastLogTerm: number, lastLogIndex: number) {
if (!(this.state == KafkaKRaftState.Candidate)) throw new Error('Guard failed: this.state == KafkaKRaftState.Candidate');
if (!(this.currentTerm >= lastLogTerm)) throw new Error('Guard failed: this.currentTerm >= lastLogTerm');
this.votesGranted = new Set([...this.votesGranted, candidate]);
}
HandleRequestVoteRequest(candidate: NodeId, candidateTerm: number, lastLogTerm: number, lastLogIndex: number) {
if (!(this.currentTerm >= candidateTerm)) throw new Error('Guard failed: this.currentTerm >= candidateTerm');
this.votedFor = Some(candidate);
}
BecomeLeader() {
if (!(this.state == KafkaKRaftState.Candidate)) throw new Error('Guard failed: this.state == KafkaKRaftState.Candidate');
if (!(this.votesGranted.size >= 3)) throw new Error('Guard failed: this.votesGranted.size >= 3');
this.state = KafkaKRaftState.Leader;
this.commitIndex = length(this.log);
}
AppendEntries(follower: NodeId) {
if (!(this.state == KafkaKRaftState.Leader)) throw new Error('Guard failed: this.state == KafkaKRaftState.Leader');
this.log = append(this.log, newEntry);
}
// V1: Always(TermMonotonicity)
async verify1(): Promise<boolean> {
// TODO: Model-check or monitor
// Temporal: Always(TermMonotonicity)
return true;
}
}

124
out/KafkaKRaft.wat
View file

@ -1,124 +0,0 @@
;; Auto-generated by speckl-compile from KafkaKRaft
;; Compile: wat2wasm KafkaKRaft.wat -o KafkaKRaft.wasm
(module
;; Host imports for Map/Set/alloc
(import "speckl" "map_create" (func $map_create (result i32)))
(import "speckl" "map_set" (func $map_set (param i32 i32 i32)))
(import "speckl" "map_get" (func $map_get (param i32 i32) (result i32)))
(import "speckl" "map_has" (func $map_has (param i32 i32) (result i32)))
(import "speckl" "map_remove" (func $map_remove (param i32 i32)))
(import "speckl" "set_create" (func $set_create (result i32)))
(import "speckl" "set_add" (func $set_add (param i32 i32)))
(import "speckl" "set_has" (func $set_has (param i32 i32) (result i32)))
(import "speckl" "set_size" (func $set_size (param i32) (result i32)))
(import "speckl" "alloc" (func $alloc (param i32) (result i32)))
(import "speckl" "write_i32" (func $write_i32 (param i32 i32)))
(import "speckl" "write_f64" (func $write_f64 (param i32 f64)))
;; Linear memory (exported so host can read state)
(memory (export "memory") 1)
;; State globals
(global $currentTerm (mut i32) (i32.const 0))
(global $state (mut i32) (i32.const 0))
(global $votedFor (mut i32) (i32.const 0))
(global $log (mut i32) (i32.const 0))
(global $commitIndex (mut i32) (i32.const 0))
(global $nextIndex (mut i32) (i32.const 0))
(global $matchIndex (mut i32) (i32.const 0))
(global $votesGranted (mut i32) (i32.const 0))
;; ---- init ----
(func $init (export "init")
(i32.const 0)
(global.set $currentTerm)
(i32.const 0)
(global.set $state)
(i32.const 0)
(global.set $votedFor)
(i32.const 0)
(global.set $commitIndex)
(call $map_create)
(global.set $nextIndex)
(call $map_create)
(global.set $matchIndex)
(call $set_create)
(global.set $votesGranted)
)
;; ---- Actions ----
;; action: RequestVote
(func $RequestVote (export "RequestVote") (param $candidate i32) (param $lastLogTerm i32) (param $lastLogIndex i32) (result i32)
;; require: state == Candidate
;; Complex right-hand side not compiled: state == Candidate
;; require: currentTerm >= lastLogTerm
;; Complex right-hand side not compiled: currentTerm >= lastLogTerm
;; votesGranted := votesGranted union {candidate}
;; Unrecognized assignment target: votesGranted
(i32.const 1)
)
;; action: HandleRequestVoteRequest
(func $HandleRequestVoteRequest (export "HandleRequestVoteRequest") (param $candidate i32) (param $candidateTerm i32) (param $lastLogTerm i32) (param $lastLogIndex i32) (result i32)
;; require: currentTerm >= candidateTerm
;; Complex right-hand side not compiled: currentTerm >= candidateTerm
;; votedFor := Some(candidate)
(i32.const 0) ;; uncompiled expr: Some(candidate)
(global.set $votedFor)
(i32.const 1)
)
;; action: BecomeLeader
(func $BecomeLeader (export "BecomeLeader") (result i32)
;; require: state == Candidate
;; Complex right-hand side not compiled: state == Candidate
;; require: votesGranted.size() >= 3
;; Complex guard not compiled: votesGranted.size() >= 3
;; state := Leader
(i32.const 0) ;; uncompiled expr: Leader
(global.set $state)
;; commitIndex := length(log)
(i32.const 0) ;; uncompiled expr: length(log)
(global.set $commitIndex)
(i32.const 1)
)
;; action: AppendEntries
(func $AppendEntries (export "AppendEntries") (param $follower i32) (result i32)
;; require: state == Leader
;; Complex right-hand side not compiled: state == Leader
;; log := append(log, newEntry)
;; Unrecognized assignment target: log
(i32.const 1)
)
;; ---- getState ----
(func $getState (export "getState") (result i32 i32)
(i32.const 0)
(global.get $currentTerm)
(i32.store)
(i32.const 4)
(global.get $state)
(i32.store)
(i32.const 8)
(global.get $votedFor)
(i32.store)
(i32.const 12)
(global.get $commitIndex)
(i32.store)
(i32.const 0)
(i32.const 16)
)
;; checkConstraints — runtime constraint evaluation stub
(func $checkConstraints (export "checkConstraints") (result i32)
;; TODO: evaluate constraint expressions at runtime
(i32.const 1)
)
)

21
out/OAuth2AuthorizationCode.prov.jsonld
View file

@ -1,21 +0,0 @@
{
"@context": {
"prov": "http://www.w3.org/ns/prov#",
"speckl": "https://speckl.scoble.me/ns/v0.2#",
"rdfs": "http://www.w3.org/2000/01/rdf-schema#"
},
"@graph": [
{
"@type": "prov:Entity",
"@id": "speck:OAuth2AuthorizationCode",
"prov:qualifiedGeneration": {
"@type": "prov:Generation",
"prov:activity": {
"@type": "prov:Activity",
"prov:startTime": "2026-05-10T09:08:15.017Z"
}
},
"rdfs:label": "OAuth2AuthorizationCode"
}
]
}

15
out/OAuth2AuthorizationCode.specbom.cdx.json
View file

@ -1,15 +0,0 @@
{
"bomFormat": "CycloneDX",
"specVersion": "1.6",
"components": [],
"metadata": {
"tools": [],
"component": {
"type": "application",
"name": "OAuth2AuthorizationCode",
"version": "0.2.0",
"properties": []
}
},
"annotations": []
}

26
out/OAuth2AuthorizationCode.specbom.spdx.json
View file

@ -1,26 +0,0 @@
{
"@context": [
"https://spdx.github.io/spdx-v3/spdx-context.jsonld",
{
"speckl": "https://speckl.scoble.me/ns/v0.2#"
}
],
"@type": "spdx:Document",
"spdx": {
"@id": "spdx:OAuth2AuthorizationCode",
"@type": "spdx:Snippet",
"name": "OAuth2AuthorizationCode",
"spdxVersion": "3.0.1",
"dataLicense": "CC0-1.0",
"creationInfo": {
"created": "2026-05-10T09:08:15.086Z",
"createdBy": [
"Organization: Speckl",
"tool: speckl-compile-0.2.0"
]
}
},
"builtBy": [],
"relationships": [],
"elements": []
}

114
out/OAuth2AuthorizationCode.ts
View file

@ -1,114 +0,0 @@
// Auto-generated by speckl-compile from OAuth2AuthorizationCode
// DO NOT EDIT MANUALLY — regenerate from .speckdl source
export const OAuth2AuthorizationCodeState = {
Open: 'open',
Closed: 'closed',
} as const;
export type OAuth2AuthorizationCodeState = typeof OAuth2AuthorizationCodeState[keyof typeof OAuth2AuthorizationCodeState];
export class OAuth2AuthorizationCodeMachine {
clients: Map<ClientId, { redirect_uris: Set<RedirectUri>; client_secret: ClientSecret }>;
authorizationCodes: Map(AuthorizationCode, {;
client_id: ClientId;
redirect_uri: RedirectUri;
state: OAuth2AuthorizationCodeState;
code_challenge: CodeChallenge;
code_challenge_method: String;
scope: Scope;
user_id: UserId;
session_id: SessionId;
created_at: number;
used: boolean;
accessTokens: Map(AccessToken, {;
client_id: ClientId;
user_id: UserId;
scope: Scope;
expires_at: number;
session_id: SessionId;
refresh_token: RefreshToken;
refreshTokens: Map(RefreshToken, {;
client_id: ClientId;
user_id: UserId;
scope: Scope;
access_token: AccessToken;
created_at: number;
sessions: Map<SessionId, { user_id: UserId }>;
stateNonces: Map<OAuth2AuthorizationCodeState, { client_id: ClientId; redirect_uri: RedirectUri }>;
pkceChallenges: Map<CodeChallenge, { code_verifier_hash: string; method: string }>;
currentTime: number;
decisionLog: any[] = [];
constructor() {
// No init block defined
}
Authorize(client: ClientId, redirectUri: RedirectUri, scope: Scope, userId: UserId, sessionId: SessionId) {
if (!(this.clients.has(client))) throw new Error('Guard failed: this.clients.has(client)');
if (!(this.clients.client.redirect_uris.has(redirectUri))) throw new Error('Guard failed: this.clients.client.redirect_uris.has(redirectUri)');
if (!(this.sessions.has(sessionId))) throw new Error('Guard failed: this.sessions.has(sessionId)');
if (!(this.sessions.get(sessionId).user_id == userId)) throw new Error('Guard failed: this.sessions.get(sessionId).user_id == userId');
this.stateNonces[state] = { this.client_id: client, this.redirect_uri: redirectUri };
this.pkceChallenges[codeChallenge] = { code_verifier_hash: hash(codeVerifier, "S256"), method: "S256" };
this.authorizationCodes[authCode] = {;
}
ApproveAuthorization(code: AuthorizationCode, userId: UserId) {
if (!(this.authorizationCodes.has(code))) throw new Error('Guard failed: this.authorizationCodes.has(code)');
if (!(c.this.user_id == userId)) throw new Error('Guard failed: c.this.user_id == userId');
if (!(c.this.used == false)) throw new Error('Guard failed: c.this.used == false');
}
ExchangeCode(req: TokenRequest) {
if (!(this.clients.has(req.this.client_id))) throw new Error('Guard failed: this.clients.has(req.this.client_id)');
if (!(this.authorizationCodes.has(req.code))) throw new Error('Guard failed: this.authorizationCodes.has(req.code)');
if (!(c.this.used == false)) throw new Error('Guard failed: c.this.used == false');
if (!(c.this.client_id == req.this.client_id)) throw new Error('Guard failed: c.this.client_id == req.this.client_id');
if (!(c.this.redirect_uri == req.this.redirect_uri)) throw new Error('Guard failed: c.this.redirect_uri == req.this.redirect_uri');
if (!(expectedChallenge == c.this.code_challenge)) throw new Error('Guard failed: expectedChallenge == c.this.code_challenge');
this.pkceChallenges[c.code_challenge] = null;
this.stateNonces[c.state] = null;
this.accessTokens[accessToken] = {;
this.refreshTokens[refreshToken] = {;
}
RefreshToken(req: RefreshRequest) {
if (!(this.clients.has(req.this.client_id))) throw new Error('Guard failed: this.clients.has(req.this.client_id)');
if (!(this.refreshTokens.has(req.this.refresh_token))) throw new Error('Guard failed: this.refreshTokens.has(req.this.refresh_token)');
if (!(r.this.client_id == req.this.client_id)) throw new Error('Guard failed: r.this.client_id == req.this.client_id');
this.accessTokens[r.access_token] = null;
this.accessTokens[newAccessToken] = {;
this.refreshTokens[newRefreshToken] = {;
this.refreshTokens[req.refresh_token] = null;
}
RevokeToken(client: ClientId, token: string, tokenType: string) {
if (!(this.clients.has(client) if tokenType == "access" {)) throw new Error('Guard failed: this.clients.has(client) if tokenType == "access" {');
if (!(this.accessTokens.has(token))) throw new Error('Guard failed: this.accessTokens.has(token)');
if (!(t.this.client_id == client)) throw new Error('Guard failed: t.this.client_id == client');
if (!(this.refreshTokens.has(token))) throw new Error('Guard failed: this.refreshTokens.has(token)');
if (!(r.this.client_id == client)) throw new Error('Guard failed: r.this.client_id == client');
this.accessTokens[token] = null;
this.refreshTokens[t.refresh_token] = null;
this.refreshTokens[token] = null;
this.accessTokens[r.access_token] = null;
}
VerifyPKCE(code: AuthorizationCode, verifier: CodeVerifier) {
if (!(this.authorizationCodes.has(code))) throw new Error('Guard failed: this.authorizationCodes.has(code)');
if (!(c.this.used == false)) throw new Error('Guard failed: c.this.used == false');
if (!(expectedChallenge == c.this.code_challenge)) throw new Error('Guard failed: expectedChallenge == c.this.code_challenge');
}
// V1: Always(NoTokenLeakage)
async verify1(): Promise<boolean> {
// TODO: Model-check or monitor
// Temporal: Always(NoTokenLeakage)
return true;
}
}

211
out/OAuth2AuthorizationCode.wat
View file

@ -1,211 +0,0 @@
;; Auto-generated by speckl-compile from OAuth2AuthorizationCode
;; Compile: wat2wasm OAuth2AuthorizationCode.wat -o OAuth2AuthorizationCode.wasm
(module
;; Host imports for Map/Set/alloc
(import "speckl" "map_create" (func $map_create (result i32)))
(import "speckl" "map_set" (func $map_set (param i32 i32 i32)))
(import "speckl" "map_get" (func $map_get (param i32 i32) (result i32)))
(import "speckl" "map_has" (func $map_has (param i32 i32) (result i32)))
(import "speckl" "map_remove" (func $map_remove (param i32 i32)))
(import "speckl" "set_create" (func $set_create (result i32)))
(import "speckl" "set_add" (func $set_add (param i32 i32)))
(import "speckl" "set_has" (func $set_has (param i32 i32) (result i32)))
(import "speckl" "set_size" (func $set_size (param i32) (result i32)))
(import "speckl" "alloc" (func $alloc (param i32) (result i32)))
(import "speckl" "write_i32" (func $write_i32 (param i32 i32)))
(import "speckl" "write_f64" (func $write_f64 (param i32 f64)))
;; Linear memory (exported so host can read state)
(memory (export "memory") 1)
;; State globals
;; ---- init ----
(func $init (export "init")
)
;; ---- Actions ----
;; action: Authorize
(func $Authorize (export "Authorize") (param $client i32) (param $redirectUri i32) (param $scope i32) (param $userId i32) (param $sessionId i32) (result i32)
(local $state i32)
(local $codeVerifier i32)
(local $codeChallenge i32)
(local $authCode i32)
;; require: client in clients.keys
;; Complex guard not compiled: client in clients.keys
;; require: redirectUri in clients.client.redirect_uris
;; Complex guard not compiled: redirectUri in clients.client.redirect_uris
;; require: sessionId in sessions.keys
;; Complex guard not compiled: sessionId in sessions.keys
;; require: sessions[sessionId].user_id == userId
;; Complex guard not compiled: sessions[sessionId].user_id == userId
(i32.const 0) ;; uncompiled expr: "state_" + hash(client + redirectUri + currentTime)
(local.set $state)
(i32.const 0) ;; uncompiled expr: "verifier_" + hash(client + userId + currentTime)
(local.set $codeVerifier)
(i32.const 0) ;; uncompiled expr: hash(codeVerifier, "S256")
(local.set $codeChallenge)
;; stateNonces[state] := { client_id: client, redirect_uri: redirectUri }
;; Unrecognized assignment target: stateNonces[state]
;; pkceChallenges[codeChallenge] := { code_verifier_hash: hash(codeVerifier, "S256"), method: "S256" }
;; Unrecognized assignment target: pkceChallenges[codeChallenge]
(i32.const 0) ;; uncompiled expr: "code_" + hash(client + userId + currentTime)
(local.set $authCode)
;; authorizationCodes[authCode] := {
;; Unrecognized assignment target: authorizationCodes[authCode]
(i32.const 1)
)
;; action: ApproveAuthorization
(func $ApproveAuthorization (export "ApproveAuthorization") (param $code i32) (param $userId i32) (result i32)
(local $c i32)
;; require: code in authorizationCodes.keys
;; Complex guard not compiled: code in authorizationCodes.keys
(i32.const 0) ;; uncompiled expr: authorizationCodes[code]
(local.set $c)
;; require: c.user_id == userId
;; Complex guard not compiled: c.user_id == userId
;; require: c.used == false
;; Complex guard not compiled: c.used == false
(i32.const 1)
)
;; action: ExchangeCode
(func $ExchangeCode (export "ExchangeCode") (param $req i32) (result i32)
(local $c i32)
(local $expectedChallenge i32)
(local $accessToken i32)
(local $refreshToken i32)
(local $expiresAt i32)
;; require: req.client_id in clients.keys
;; Complex guard not compiled: req.client_id in clients.keys
;; require: req.code in authorizationCodes.keys
;; Complex guard not compiled: req.code in authorizationCodes.keys
(i32.const 0) ;; uncompiled expr: authorizationCodes[req.code]
(local.set $c)
;; require: c.used == false
;; Complex guard not compiled: c.used == false
;; require: c.client_id == req.client_id
;; Complex guard not compiled: c.client_id == req.client_id
;; require: c.redirect_uri == req.redirect_uri
;; Complex guard not compiled: c.redirect_uri == req.redirect_uri
(i32.const 0) ;; uncompiled expr: hash(req.code_verifier, c.code_challenge_method)
(local.set $expectedChallenge)
;; require: expectedChallenge == c.code_challenge
;; Guard on non-scalar not compiled: expectedChallenge == c.code_challenge
;; pkceChallenges[c.code_challenge] := null
;; Unrecognized assignment target: pkceChallenges[c.code_challenge]
;; stateNonces[c.state] := null
;; Unrecognized assignment target: stateNonces[c.state]
(i32.const 0) ;; uncompiled expr: "access_" + hash(req.client_id + c.user_id + currentTime)
(local.set $accessToken)
(i32.const 0) ;; uncompiled expr: "refresh_" + hash(req.client_id + c.user_id + currentTime)
(local.set $refreshToken)
(i32.add (i32.const NaN) (i32.const 3600))
(local.set $expiresAt)
;; accessTokens[accessToken] := {
;; Unrecognized assignment target: accessTokens[accessToken]
;; refreshTokens[refreshToken] := {
;; Unrecognized assignment target: refreshTokens[refreshToken]
(i32.const 1)
)
;; action: RefreshToken
(func $RefreshToken (export "RefreshToken") (param $req i32) (result i32)
(local $r i32)
(local $newAccessToken i32)
(local $newRefreshToken i32)
(local $expiresAt i32)
;; require: req.client_id in clients.keys
;; Complex guard not compiled: req.client_id in clients.keys
;; require: req.refresh_token in refreshTokens.keys
;; Complex guard not compiled: req.refresh_token in refreshTokens.keys
(i32.const 0) ;; uncompiled expr: refreshTokens[req.refresh_token]
(local.set $r)
;; require: r.client_id == req.client_id
;; Complex guard not compiled: r.client_id == req.client_id
;; accessTokens[r.access_token] := null
;; Unrecognized assignment target: accessTokens[r.access_token]
(i32.const 0) ;; uncompiled expr: "access_" + hash(req.client_id + r.user_id + currentTime)
(local.set $newAccessToken)
(i32.const 0) ;; uncompiled expr: "refresh_" + hash(req.client_id + r.user_id + currentTime)
(local.set $newRefreshToken)
(i32.add (i32.const NaN) (i32.const 3600))
(local.set $expiresAt)
;; accessTokens[newAccessToken] := {
;; Unrecognized assignment target: accessTokens[newAccessToken]
;; refreshTokens[newRefreshToken] := {
;; Unrecognized assignment target: refreshTokens[newRefreshToken]
;; refreshTokens[req.refresh_token] := null
;; Unrecognized assignment target: refreshTokens[req.refresh_token]
(i32.const 1)
)
;; action: RevokeToken
(func $RevokeToken (export "RevokeToken") (param $client i32) (param $token i32) (param $tokenType i32) (result i32)
(local $t i32)
(local $r i32)
;; require: client in clients.keys
;; Complex guard not compiled: client in clients.keys
;; require: token in accessTokens.keys
;; Complex guard not compiled: token in accessTokens.keys
(i32.const 0) ;; uncompiled expr: accessTokens[token]
(local.set $t)
;; require: t.client_id == client
;; Complex guard not compiled: t.client_id == client
;; accessTokens[token] := null
;; Unrecognized assignment target: accessTokens[token]
;; refreshTokens[t.refresh_token] := null
;; Unrecognized assignment target: refreshTokens[t.refresh_token]
;; require: token in refreshTokens.keys
;; Complex guard not compiled: token in refreshTokens.keys
(i32.const 0) ;; uncompiled expr: refreshTokens[token]
(local.set $r)
;; require: r.client_id == client
;; Complex guard not compiled: r.client_id == client
;; refreshTokens[token] := null
;; Unrecognized assignment target: refreshTokens[token]
;; accessTokens[r.access_token] := null
;; Unrecognized assignment target: accessTokens[r.access_token]
(i32.const 1)
)
;; action: VerifyPKCE
(func $VerifyPKCE (export "VerifyPKCE") (param $code i32) (param $verifier i32) (result i32)
(local $c i32)
(local $expectedChallenge i32)
;; require: code in authorizationCodes.keys
;; Complex guard not compiled: code in authorizationCodes.keys
(i32.const 0) ;; uncompiled expr: authorizationCodes[code]
(local.set $c)
;; require: c.used == false
;; Complex guard not compiled: c.used == false
(i32.const 0) ;; uncompiled expr: hash(verifier, c.code_challenge_method)
(local.set $expectedChallenge)
;; require: expectedChallenge == c.code_challenge
;; Guard on non-scalar not compiled: expectedChallenge == c.code_challenge
(i32.const 1)
)
;; ---- getState ----
(func $getState (export "getState") (result i32 i32)
(i32.const 0)
(i32.const 0)
)
;; checkConstraints — runtime constraint evaluation stub
(func $checkConstraints (export "checkConstraints") (result i32)
;; TODO: evaluate constraint expressions at runtime
(i32.const 1)
)
)

21
out/TigerBeetleLedger.prov.jsonld
View file

@ -1,21 +0,0 @@
{
"@context": {
"prov": "http://www.w3.org/ns/prov#",
"speckl": "https://speckl.scoble.me/ns/v0.2#",
"rdfs": "http://www.w3.org/2000/01/rdf-schema#"
},
"@graph": [
{
"@type": "prov:Entity",
"@id": "speck:TigerBeetleLedger",
"prov:qualifiedGeneration": {
"@type": "prov:Generation",
"prov:activity": {
"@type": "prov:Activity",
"prov:startTime": "2026-05-10T09:08:16.133Z"
}
},
"rdfs:label": "TigerBeetleLedger"
}
]
}

3143
out/TigerBeetleLedger.smt2
View file

File diff suppressed because it is too large Load diff

15
out/TigerBeetleLedger.specbom.cdx.json
View file

@ -1,15 +0,0 @@
{
"bomFormat": "CycloneDX",
"specVersion": "1.6",
"components": [],
"metadata": {
"tools": [],
"component": {
"type": "application",
"name": "TigerBeetleLedger",
"version": "0.2.0",
"properties": []
}
},
"annotations": []
}

26
out/TigerBeetleLedger.specbom.spdx.json
View file

@ -1,26 +0,0 @@
{
"@context": [
"https://spdx.github.io/spdx-v3/spdx-context.jsonld",
{
"speckl": "https://speckl.scoble.me/ns/v0.2#"
}
],
"@type": "spdx:Document",
"spdx": {
"@id": "spdx:TigerBeetleLedger",
"@type": "spdx:Snippet",
"name": "TigerBeetleLedger",
"spdxVersion": "3.0.1",
"dataLicense": "CC0-1.0",
"creationInfo": {
"created": "2026-05-10T09:08:16.165Z",
"createdBy": [
"Organization: Speckl",
"tool: speckl-compile-0.2.0"
]
}
},
"builtBy": [],
"relationships": [],
"elements": []
}

143
out/TigerBeetleLedger.ts
View file

@ -1,143 +0,0 @@
// Auto-generated by speckl-compile from TigerBeetleLedger
// DO NOT EDIT MANUALLY — regenerate from .speckdl source
export const TigerBeetleLedgerState = {
Open: 'open',
Closed: 'closed',
} as const;
export type TigerBeetleLedgerState = typeof TigerBeetleLedgerState[keyof typeof TigerBeetleLedgerState];
export class TigerBeetleLedgerMachine {
accounts: Map<number, Account>;
transfers: Map<number, Transfer>;
pending: Set<number>;
replicaState: Map<number, Replica>;
log: LogEntry[];
commitIndex: number;
leader: number;
decisionLog: any[] = [];
constructor() {
// No init block defined
}
CreateTransfer(t: Transfer) {
// precondition: !this.transfers.has(t.id)
if (!(!this.transfers.has(t.id))) throw new Error('Precondition failed: !this.transfers.has(t.id)');
// precondition: this.accounts.has(t.from)
if (!(this.accounts.has(t.from))) throw new Error('Precondition failed: this.accounts.has(t.from)');
// precondition: this.accounts.has(t.to)
if (!(this.accounts.has(t.to))) throw new Error('Precondition failed: this.accounts.has(t.to)');
// precondition: t.from != t.to
if (!(t.from != t.to)) throw new Error('Precondition failed: t.from != t.to');
// precondition: this.accounts.get(t.from).balance >= t.amount || this.accounts.get(t.from).flags.allows_overdraft
if (!(this.accounts.get(t.from).balance >= t.amount || this.accounts.get(t.from).flags.allows_overdraft)) throw new Error('Precondition failed: this.accounts.get(t.from).balance >= t.amount || this.accounts.get(t.from).flags.allows_overdraft');
// postcondition: this.transfers.get(t.id) == t
if (!(this.transfers.get(t.id) == t)) throw new Error('Postcondition failed: this.transfers.get(t.id) == t');
// postcondition: this.pending == new Set([...this.pending, t.id])
if (!(this.pending == new Set([...this.pending, t.id]))) throw new Error('Postcondition failed: this.pending == new Set([...this.pending, t.id])');
// postcondition: this.accounts.get(t.from).debits_pending == this.accounts.get(t.from).debits_pending + t.amount
if (!(this.accounts.get(t.from).debits_pending == this.accounts.get(t.from).debits_pending + t.amount)) throw new Error('Postcondition failed: this.accounts.get(t.from).debits_pending == this.accounts.get(t.from).debits_pending + t.amount');
// postcondition: this.accounts.get(t.to).credits_pending == this.accounts.get(t.to).credits_pending + t.amount
if (!(this.accounts.get(t.to).credits_pending == this.accounts.get(t.to).credits_pending + t.amount)) throw new Error('Postcondition failed: this.accounts.get(t.to).credits_pending == this.accounts.get(t.to).credits_pending + t.amount');
}
CommitTransfer(id: number) {
// precondition: this.pending.has(id)
if (!(this.pending.has(id))) throw new Error('Precondition failed: this.pending.has(id)');
// precondition: this.transfers.has(id)
if (!(this.transfers.has(id))) throw new Error('Precondition failed: this.transfers.has(id)');
// precondition: this.transfers.get(id).pending == true
if (!(this.transfers.get(id).pending == true)) throw new Error('Precondition failed: this.transfers.get(id).pending == true');
// postcondition: this.pending == new Set([...this.pending].filter(x => x !== id))
if (!(this.pending == new Set([...this.pending].filter(x => x !== id)))) throw new Error('Postcondition failed: this.pending == new Set([...this.pending].filter(x => x !== id))');
// postcondition: this.transfers.get(id).pending == false
if (!(this.transfers.get(id).pending == false)) throw new Error('Postcondition failed: this.transfers.get(id).pending == false');
// postcondition: this.transfers.get(id).posted == true
if (!(this.transfers.get(id).posted == true)) throw new Error('Postcondition failed: this.transfers.get(id).posted == true');
// postcondition: this.accounts.get(this.transfers.get(id).from).debits_posted === this.accounts.get(this.transfers.get(id).from).debits_posted
if (!(this.accounts.get(this.transfers.get(id).from).debits_posted === this.accounts.get(this.transfers.get(id).from).debits_posted)) throw new Error('Postcondition failed: this.accounts.get(this.transfers.get(id).from).debits_posted === this.accounts.get(this.transfers.get(id).from).debits_posted');
// postcondition: this.accounts.get(this.transfers.get(id).to).credits_posted === this.accounts.get(this.transfers.get(id).to).credits_posted
if (!(this.accounts.get(this.transfers.get(id).to).credits_posted === this.accounts.get(this.transfers.get(id).to).credits_posted)) throw new Error('Postcondition failed: this.accounts.get(this.transfers.get(id).to).credits_posted === this.accounts.get(this.transfers.get(id).to).credits_posted');
// postcondition: this.accounts.get(this.transfers.get(id).from).debits_pending === this.accounts.get(this.transfers.get(id).from).debits_pending
if (!(this.accounts.get(this.transfers.get(id).from).debits_pending === this.accounts.get(this.transfers.get(id).from).debits_pending)) throw new Error('Postcondition failed: this.accounts.get(this.transfers.get(id).from).debits_pending === this.accounts.get(this.transfers.get(id).from).debits_pending');
// postcondition: this.accounts.get(this.transfers.get(id).to).credits_pending === this.accounts.get(this.transfers.get(id).to).credits_pending
if (!(this.accounts.get(this.transfers.get(id).to).credits_pending === this.accounts.get(this.transfers.get(id).to).credits_pending)) throw new Error('Postcondition failed: this.accounts.get(this.transfers.get(id).to).credits_pending === this.accounts.get(this.transfers.get(id).to).credits_pending');
}
VoidTransfer(id: number) {
// precondition: this.pending.has(id)
if (!(this.pending.has(id))) throw new Error('Precondition failed: this.pending.has(id)');
// precondition: this.transfers.has(id)
if (!(this.transfers.has(id))) throw new Error('Precondition failed: this.transfers.has(id)');
// precondition: this.transfers.get(id).pending == true
if (!(this.transfers.get(id).pending == true)) throw new Error('Precondition failed: this.transfers.get(id).pending == true');
// postcondition: this.pending == new Set([...this.pending].filter(x => x !== id))
if (!(this.pending == new Set([...this.pending].filter(x => x !== id)))) throw new Error('Postcondition failed: this.pending == new Set([...this.pending].filter(x => x !== id))');
// postcondition: this.transfers.get(id).pending == false
if (!(this.transfers.get(id).pending == false)) throw new Error('Postcondition failed: this.transfers.get(id).pending == false');
// postcondition: this.transfers.get(id).voided == true
if (!(this.transfers.get(id).voided == true)) throw new Error('Postcondition failed: this.transfers.get(id).voided == true');
// postcondition: this.accounts.get(this.transfers.get(id).from).debits_pending === this.accounts.get(this.transfers.get(id).from).debits_pending
if (!(this.accounts.get(this.transfers.get(id).from).debits_pending === this.accounts.get(this.transfers.get(id).from).debits_pending)) throw new Error('Postcondition failed: this.accounts.get(this.transfers.get(id).from).debits_pending === this.accounts.get(this.transfers.get(id).from).debits_pending');
// postcondition: this.accounts.get(this.transfers.get(id).to).credits_pending === this.accounts.get(this.transfers.get(id).to).credits_pending
if (!(this.accounts.get(this.transfers.get(id).to).credits_pending === this.accounts.get(this.transfers.get(id).to).credits_pending)) throw new Error('Postcondition failed: this.accounts.get(this.transfers.get(id).to).credits_pending === this.accounts.get(this.transfers.get(id).to).credits_pending');
}
PostTransfer(t: Transfer) {
// precondition: !this.transfers.has(t.id)
if (!(!this.transfers.has(t.id))) throw new Error('Precondition failed: !this.transfers.has(t.id)');
// precondition: this.accounts.has(t.from)
if (!(this.accounts.has(t.from))) throw new Error('Precondition failed: this.accounts.has(t.from)');
// precondition: this.accounts.has(t.to)
if (!(this.accounts.has(t.to))) throw new Error('Precondition failed: this.accounts.has(t.to)');
// precondition: t.from != t.to
if (!(t.from != t.to)) throw new Error('Precondition failed: t.from != t.to');
// precondition: this.accounts.get(t.from).balance >= t.amount || this.accounts.get(t.from).flags.allows_overdraft
if (!(this.accounts.get(t.from).balance >= t.amount || this.accounts.get(t.from).flags.allows_overdraft)) throw new Error('Precondition failed: this.accounts.get(t.from).balance >= t.amount || this.accounts.get(t.from).flags.allows_overdraft');
// postcondition: this.transfers.get(t.id) == { ...t, pending: false, posted: true, voided: false }
if (!(this.transfers.get(t.id) == { ...t, pending: false, posted: true, voided: false })) throw new Error('Postcondition failed: this.transfers.get(t.id) == { ...t, pending: false, posted: true, voided: false }');
// postcondition: this.accounts.get(t.from).debits_posted == this.accounts.get(t.from).debits_posted + t.amount
if (!(this.accounts.get(t.from).debits_posted == this.accounts.get(t.from).debits_posted + t.amount)) throw new Error('Postcondition failed: this.accounts.get(t.from).debits_posted == this.accounts.get(t.from).debits_posted + t.amount');
// postcondition: this.accounts.get(t.to).credits_posted == this.accounts.get(t.to).credits_posted + t.amount
if (!(this.accounts.get(t.to).credits_posted == this.accounts.get(t.to).credits_posted + t.amount)) throw new Error('Postcondition failed: this.accounts.get(t.to).credits_posted == this.accounts.get(t.to).credits_posted + t.amount');
}
BecomeLeader(replica: number, term: number) {
// precondition: this.replicaState.has(replica)
if (!(this.replicaState.has(replica))) throw new Error('Precondition failed: this.replicaState.has(replica)');
// precondition: this.replicaState.get(replica).role == "candidate"
if (!(this.replicaState.get(replica).role == "candidate")) throw new Error('Precondition failed: this.replicaState.get(replica).role == "candidate"');
// precondition: this.replicaState.get(replica).currentTerm < term
if (!(this.replicaState.get(replica).currentTerm < term)) throw new Error('Precondition failed: this.replicaState.get(replica).currentTerm < term');
// postcondition: this.leader == replica
if (!(this.leader == replica)) throw new Error('Postcondition failed: this.leader == replica');
// postcondition: this.replicaState.get(replica).role == "this.leader"
if (!(this.replicaState.get(replica).role == "this.leader")) throw new Error('Postcondition failed: this.replicaState.get(replica).role == "this.leader"');
// postcondition: this.replicaState.get(replica).currentTerm == term
if (!(this.replicaState.get(replica).currentTerm == term)) throw new Error('Postcondition failed: this.replicaState.get(replica).currentTerm == term');
}
AppendEntry(entry: LogEntry, term: number) {
// precondition: entry.index == length(this.log) + 1
if (!(entry.index == length(this.log) + 1)) throw new Error('Precondition failed: entry.index == length(this.log) + 1');
// precondition: term >= this.commitIndex
if (!(term >= this.commitIndex)) throw new Error('Precondition failed: term >= this.commitIndex');
// postcondition: this.log == append(this.log, entry)
if (!(this.log == append(this.log, entry))) throw new Error('Postcondition failed: this.log == append(this.log, entry)');
// postcondition: this.commitIndex == entry.index
if (!(this.commitIndex == entry.index)) throw new Error('Postcondition failed: this.commitIndex == entry.index');
}
// V1: Always(AccountBalanceInvariant)
async verify1(): Promise<boolean> {
// TODO: Model-check or monitor
// Temporal: Always(AccountBalanceInvariant)
return true;
}
}

102
out/TigerBeetleLedger.wat
View file

@ -1,102 +0,0 @@
;; Auto-generated by speckl-compile from TigerBeetleLedger
;; Compile: wat2wasm TigerBeetleLedger.wat -o TigerBeetleLedger.wasm
(module
;; Host imports for Map/Set/alloc
(import "speckl" "map_create" (func $map_create (result i32)))
(import "speckl" "map_set" (func $map_set (param i32 i32 i32)))
(import "speckl" "map_get" (func $map_get (param i32 i32) (result i32)))
(import "speckl" "map_has" (func $map_has (param i32 i32) (result i32)))
(import "speckl" "map_remove" (func $map_remove (param i32 i32)))
(import "speckl" "set_create" (func $set_create (result i32)))
(import "speckl" "set_add" (func $set_add (param i32 i32)))
(import "speckl" "set_has" (func $set_has (param i32 i32) (result i32)))
(import "speckl" "set_size" (func $set_size (param i32) (result i32)))
(import "speckl" "alloc" (func $alloc (param i32) (result i32)))
(import "speckl" "write_i32" (func $write_i32 (param i32 i32)))
(import "speckl" "write_f64" (func $write_f64 (param i32 f64)))
;; Linear memory (exported so host can read state)
(memory (export "memory") 1)
;; State globals
(global $accounts (mut i32) (i32.const 0))
(global $transfers (mut i32) (i32.const 0))
(global $pending (mut i32) (i32.const 0))
(global $replicaState (mut i32) (i32.const 0))
(global $log (mut i32) (i32.const 0))
(global $commitIndex (mut i32) (i32.const 0))
(global $leader (mut i32) (i32.const 0))
;; ---- init ----
(func $init (export "init")
(call $map_create)
(global.set $accounts)
(call $map_create)
(global.set $transfers)
(call $set_create)
(global.set $pending)
(call $map_create)
(global.set $replicaState)
(i32.const 0)
(global.set $commitIndex)
(i32.const 0)
(global.set $leader)
)
;; ---- Actions ----
;; action: CreateTransfer
(func $CreateTransfer (export "CreateTransfer") (param $t i32) (result i32)
(i32.const 1)
)
;; action: CommitTransfer
(func $CommitTransfer (export "CommitTransfer") (param $id i32) (result i32)
(i32.const 1)
)
;; action: VoidTransfer
(func $VoidTransfer (export "VoidTransfer") (param $id i32) (result i32)
(i32.const 1)
)
;; action: PostTransfer
(func $PostTransfer (export "PostTransfer") (param $t i32) (result i32)
(i32.const 1)
)
;; action: BecomeLeader
(func $BecomeLeader (export "BecomeLeader") (param $replica i32) (param $term i32) (result i32)
(i32.const 1)
)
;; action: AppendEntry
(func $AppendEntry (export "AppendEntry") (param $entry i32) (param $term i32) (result i32)
(i32.const 1)
)
;; ---- getState ----
(func $getState (export "getState") (result i32 i32)
(i32.const 0)
(global.get $commitIndex)
(i32.store)
(i32.const 4)
(global.get $leader)
(i32.store)
(i32.const 0)
(i32.const 8)
)
;; checkConstraints — runtime constraint evaluation stub
(func $checkConstraints (export "checkConstraints") (result i32)
;; TODO: evaluate constraint expressions at runtime
(i32.const 1)
)
)

21
out/ToggleSwitch.prov.jsonld
View file

@ -1,21 +0,0 @@
{
"@context": {
"prov": "http://www.w3.org/ns/prov#",
"speckl": "https://speckl.scoble.me/ns/v0.2#",
"rdfs": "http://www.w3.org/2000/01/rdf-schema#"
},
"@graph": [
{
"@type": "prov:Entity",
"@id": "speck:ToggleSwitch",
"prov:qualifiedGeneration": {
"@type": "prov:Generation",
"prov:activity": {
"@type": "prov:Activity",
"prov:startTime": "2026-05-10T09:08:16.262Z"
}
},
"rdfs:label": "ToggleSwitch"
}
]
}

23
out/ToggleSwitch.smt2
View file

@ -1,23 +0,0 @@
; SpeckDL: ToggleSwitch — Z3 Verification
; Generated by speckl-compile Z3 backend
; Bounded model checking depth: 10
(set-logic ALL)
(set-option :produce-models true)
(declare-fun speckl_len_Int ( (Array Int Int) ) Int)
; --- State Variables ---
(declare-const isOn Bool)
; --- Invariants ---
(define-fun SwitchConsistent () Bool
(and
(or (= isOn true) (= isOn false))
)
)
; --- Invariant Assertions ---
(assert (SwitchConsistent))
(check-sat)
(get-model)

15
out/ToggleSwitch.specbom.cdx.json
View file

@ -1,15 +0,0 @@
{
"bomFormat": "CycloneDX",
"specVersion": "1.6",
"components": [],
"metadata": {
"tools": [],
"component": {
"type": "application",
"name": "ToggleSwitch",
"version": "0.2.0",
"properties": []
}
},
"annotations": []
}

26
out/ToggleSwitch.specbom.spdx.json
View file

@ -1,26 +0,0 @@
{
"@context": [
"https://spdx.github.io/spdx-v3/spdx-context.jsonld",
{
"speckl": "https://speckl.scoble.me/ns/v0.2#"
}
],
"@type": "spdx:Document",
"spdx": {
"@id": "spdx:ToggleSwitch",
"@type": "spdx:Snippet",
"name": "ToggleSwitch",
"spdxVersion": "3.0.1",
"dataLicense": "CC0-1.0",
"creationInfo": {
"created": "2026-05-10T09:08:16.269Z",
"createdBy": [
"Organization: Speckl",
"tool: speckl-compile-0.2.0"
]
}
},
"builtBy": [],
"relationships": [],
"elements": []
}

34
out/ToggleSwitch.ts
View file

@ -1,34 +0,0 @@
// Auto-generated by speckl-compile from ToggleSwitch
// DO NOT EDIT MANUALLY — regenerate from .speckdl source
export const ToggleSwitchState = {
Open: 'open',
Closed: 'closed',
} as const;
export type ToggleSwitchState = typeof ToggleSwitchState[keyof typeof ToggleSwitchState];
export class ToggleSwitchMachine {
isOn: boolean;
decisionLog: any[] = [];
constructor() {
// No init block defined
}
TurnOn() {
if (!(not this.isOn)) throw new Error('Guard failed: not this.isOn');
this.isOn = true;
}
TurnOff() {
if (!(this.isOn)) throw new Error('Guard failed: this.isOn');
this.isOn = false;
}
}

71
out/ToggleSwitch.wat
View file

@ -1,71 +0,0 @@
;; Auto-generated by speckl-compile from ToggleSwitch
;; Compile: wat2wasm ToggleSwitch.wat -o ToggleSwitch.wasm
(module
;; Host imports for Map/Set/alloc
(import "speckl" "map_create" (func $map_create (result i32)))
(import "speckl" "map_set" (func $map_set (param i32 i32 i32)))
(import "speckl" "map_get" (func $map_get (param i32 i32) (result i32)))
(import "speckl" "map_has" (func $map_has (param i32 i32) (result i32)))
(import "speckl" "map_remove" (func $map_remove (param i32 i32)))
(import "speckl" "set_create" (func $set_create (result i32)))
(import "speckl" "set_add" (func $set_add (param i32 i32)))
(import "speckl" "set_has" (func $set_has (param i32 i32) (result i32)))
(import "speckl" "set_size" (func $set_size (param i32) (result i32)))
(import "speckl" "alloc" (func $alloc (param i32) (result i32)))
(import "speckl" "write_i32" (func $write_i32 (param i32 i32)))
(import "speckl" "write_f64" (func $write_f64 (param i32 f64)))
;; Linear memory (exported so host can read state)
(memory (export "memory") 1)
;; State globals
(global $isOn (mut i32) (i32.const 0))
;; ---- init ----
(func $init (export "init")
(i32.const 0)
(global.set $isOn)
)
;; ---- Actions ----
;; action: TurnOn
(func $TurnOn (export "TurnOn") (result i32)
;; require: not isOn
;; Complex guard not compiled: not isOn
;; isOn := true
(i32.const 0) ;; uncompiled expr: true
(global.set $isOn)
(global.get $isOn)
(return)
)
;; action: TurnOff
(func $TurnOff (export "TurnOff") (result i32)
;; require: isOn
;; Complex guard not compiled: isOn
;; isOn := false
(i32.const 0) ;; uncompiled expr: false
(global.set $isOn)
(global.get $isOn)
(return)
)
;; ---- getState ----
(func $getState (export "getState") (result i32 i32)
(i32.const 0)
(global.get $isOn)
(i32.store)
(i32.const 0)
(i32.const 4)
)
;; checkConstraints — runtime constraint evaluation stub
(func $checkConstraints (export "checkConstraints") (result i32)
;; TODO: evaluate constraint expressions at runtime
(i32.const 1)
)
)

2
reference-agent/src/smes/verification-sme.ts
View file

@ -30,7 +30,7 @@ export class VerificationSME extends SME {
this.compileConstraint(c.expression, speck)
);
// In production, this would invoke Z3 via subprocess.
// In production, this would invoke Z3 via subprocess or WASM.
// For the reference implementation, we simulate the result.
const z3Result = this.simulateZ3(speck, z3Assertions);

4
reference-agent/src/types.ts
View file

@ -99,7 +99,7 @@ export interface Task {
export type TaskType =
| "verify_speck" // Run Z3 verification on a Speck
| "compile_to_dst" // Compile verified Speck to DST
| "compile_to_typescript" // Compile DST to TypeScript
| "compile_to_wasm" // Compile DST to WASM
| "generate_code" // Generate code from verified Speck
| "generate_docs" // Generate documentation from provenance graph
| "integrate" // Integration/glue code
@ -142,7 +142,7 @@ export interface TaskResult {
export interface ProvenanceNode {
id: string;
type: "speck" | "z3_assertion" | "dst" | "code" | "docs" | "event";
type: "speck" | "z3_assertion" | "dst" | "wasm" | "code" | "docs" | "event";
label: string;
metadata: Record<string, unknown>;
}

4
typescript
View file

@ -1,4 +0,0 @@
Script started on 2026-05-10 01:11:44-07:00 [COMMAND="sed -n '310p' compiler/src/generators/z3.ts" <not executed on terminal>]
cleanName = cleanName.replace(/\(([^)]+)\)/g, '_$1').replace(/[<>,\s]+/g, '_').replace(/__+/g, '_');
Script done on 2026-05-10 01:11:44-07:00 [COMMAND_EXIT_CODE="0"]