Commit graph

18 commits

Author SHA1 Message Date
Scott Scoble
02a61f1d33 feat: add PromptOptimizer spec and functional runtime
PromptOptimizer.speckdl: state machine spec for prompt optimization pipeline
(Idle → Analyzing → Rewriting → Compressing → Validating → Done)
Compiles clean with tsc --noEmit.

PromptOptimizer-runtime.ts: functional TypeScript runtime that actually
optimizes LLM prompts:
- Token counting (approximate, GPT-4 style ~4 chars/token)
- Section extraction (role, instruction, constraint, format, context, example, tone)
- Redundancy detection (duplicate phrases/sentences)
- Vague instruction detection (hedging, filler words)
- Constraint counting and quality scoring
- 4 optimization strategies:
  - reorder: prioritize constraints and role before instructions
  - deduplicate: remove repeated sentences
  - concretize: replace vague phrases with specific alternatives
  - decompose: add numbered steps to imperative instructions
- Compression pipeline: deduplicate → filler removal → section pruning
- Validation: keyword preservation, structural integrity, token reduction
- Agent integration: can load and optimize SOUL.md/AGENTS.md files

Demo shows 8% reduction on SOUL.md, 5% on AGENTS.md (both already
well-structured). Higher gains on verbose prompts (13-30% reduction).
2026-05-13 21:14:55 -07:00
Scott Scoble
35583630cb fix: SQL optimizer runtime — predicate pushdown, FROM/JOIN dedup, cost model
- Predicate pushdown now splits AND predicates and pushes each to
  the correct side of the join (left table vs right table)
- FROM clause no longer duplicates tables that appear in JOINs
- Constant folding no longer mangles date literals (skips strings)
- Cost model: hash join uses build/probe costs, nested_loop uses
  per-row probe cost instead of cartesian product
- Physical operator selection defaults to hash join for medium+ tables
- Two-pass SQL generation: collect join tables first, then emit FROM
- Demo shows 86% cost reduction on users/orders query, 25% on
  departments/employees query from ekeric13/sql-query-optimization-guide
2026-05-13 18:06:30 -07:00
Scott Scoble
a910f07377 feat: add SQL query optimizer with functional runtime
SQLOptimizer.speckdl: state machine spec for the optimizer pipeline
(Parsing → Rewriting → Planning → Generating → Done, plus Failed)

SQLOptimizer-runtime.ts: working TypeScript runtime that actually
optimizes SQL queries:
- Parses SELECT/FROM/JOIN/WHERE/GROUP BY/ORDER BY/LIMIT
- Builds plan trees with cardinality estimation
- Predicate pushdown (splits AND predicates per table)
- Constant folding (evaluates arithmetic in predicates)
- Column pruning
- Physical operator selection (hash/merge/nested_loop joins)
- Cost estimation per plan node
- SQL generation from optimized plan tree

Known issues:
- Predicate pushdown needs to split AND predicates per table
- FROM clause duplicate when both listed and joined
- Cost model needs tuning (hash join startup cost shouldn't exceed
  nested loop savings for medium inputs)

Both files pass tsc --noEmit clean.
2026-05-13 17:00:46 -07:00
Scott Scoble
06232494ed feat: add SQL query optimizer spec (SQLOptimizer.speckdl)
Models a cost-based SQL optimizer pipeline: parse → rewrite → plan → execute.
Exercises state-dependent transforms, cost models, conditional emits,
plan caching, and rollback semantics. Passes tsc --noEmit clean.
2026-05-13 16:51:01 -07:00
Scott Scoble
8b520284cb TS generator: fix #12 (5 expression patterns) + #11 (opaque record types)
#12: All 5 untranslated SpeckDL patterns now emit valid JS:
  - notIn → !.has(), old() → drop, union → Set spread,
    set diff → filter, append() → [...X, Y], length() → .length
  - ==/!= operator replacement fixed (word-boundary regression)
  - Postcondition object comparisons get @ts-expect-error TS2839

#11: Generator auto-detects opaque identifier types used in state
  variables and action params, emits 'type X = any' before class.
  - State properties use ! (non-null assertion) for strict mode

TigerBeetleLedger → tsc --noEmit = 0 errors. 37/37 tests pass.
2026-05-11 10:16:15 -07:00
29d4c83d76 feat: SQLite WAL concurrency model spec (speckl#36)
Some checks failed
Deploy Landing Page to Storj / deploy (push) Failing after 2s
2026-05-10 01:05:14 -07:00
0cefe0f694 fix(z3): robust = conversion + SQLite pager spec (DB.2)
Some checks failed
Deploy Landing Page to Storj / deploy (push) Failing after 2s
- Fixed infix = conversion regex: global replace replaces restrictive lookbehind version
- Also catches = inside =>, and, or bodies (not just depth 0)
- Fixed implies keyword support in expression translation
- Fixed SQLitePager.speck: implies→->, string-typed enum comparison fix
- All specs (ToggleSwitch, KafkaKRaft, SQLitePager) produce zero-error SMT-LIB2
- All 37 tests pass
2026-05-10 01:02:00 -07:00
c12f40ee7d feat: SQLite pager state machine spec (speckl#35) 2026-05-10 00:52:22 -07:00
40eef36a10 feat(z3): zero-error SMT-LIB2 output + BMC verify blocks for KRaft
Some checks failed
Deploy Landing Page to Storj / deploy (push) Failing after 3s
- Declare speckl_len_Int helper for list/sequence lengths
- Fix infix = inside => bodies (final pass regex)
- Handle prime notation: currentTerm' → currentTerm_post
- Handle length() calls: length(log) → (speckl_len_Int log)
- Handle min(): (ite (<= a b) a b) instead of unsupported 'min'
- Handle subtraction: i-1 → (- i 1)
- Declare _post state vars when invariants compare pre/post
- Added 3 BMC verify blocks to KRaft spec (TermMonotonicity, CommitSafety, LeaderCompleteness)
- Z3 returns clean sat with zero errors for all specs
- All 37 tests pass
2026-05-09 23:07:56 -07:00
7e5a8fe2cb Merge branch 'feature/z3-backend'
Some checks failed
Deploy Landing Page to Storj / deploy (push) Failing after 2s
Z3 backend: clean SMT-LIB2 output, Z3 returns sat with model
KRaft spec: 4/6 invariants with real expressions
All 37 tests pass
2026-05-09 22:38:54 -07:00
25fbdfc7a9 fix(z3): clean SMT-LIB2 output — Z3 returns sat with model (zero sort errors)
- Sanitize sort names: Option(NodeId) → Option_NodeId
- Emit action transition predicates only for verify/BMC mode
- Handle -> implication operator in translateExpr
- Fleshed out KRaft invariant bodies (4/6 now have real expressions)
- ToggleSwitch: clean sat with model
- TigerBeetleLedger: sat with model
- KafkaKRaft: sat with model
- All 37 tests pass
2026-05-09 22:38:47 -07:00
Scott Scoble
534f13734c Merge branch 'fix/ts-generator-bugs'
# Conflicts:
#	compiler/node_modules/.vite/vitest/results.json
#	compiler/test-output/RetryHandler.prov.jsonld
#	compiler/test-output/RetryHandler.specbom.cdx.json
#	compiler/test-output/RetryHandler.specbom.spdx.json
2026-05-09 22:28:57 -07:00
582ef92586 feat: KRaft consensus protocol spec (speckl#23) 2026-05-09 22:19:32 -07:00
00dd745ef2 refactor: remove WAT/WASM generator (superseded by IR→Rust→WASM path)
- Deleted compiler/src/generators/wasm.ts (472 lines)
- Deleted compiler/src/runtime/wasm-host.ts (165 lines)
- Deleted compiler/__tests__/e2e-wasm.test.ts (203 lines)
- Removed WASM import and generation call from compiler/src/index.ts
- Removed WASMGenerated from TS generator state enum literals
- Removed WASM codegen tests from parser.test.ts
- All 37 tests passing

The WASM generator was architecturally wrong: every Map/Set operation
crossed the WASM→JS boundary, creating 45-55% overhead for a module
that doesn't compute anything. WASM will be a byproduct of the Rust
backend via wasm-pack instead (M2.1).

Closes #19
2026-05-09 22:03:20 -07:00
14cd892e8a feat: OAuth 2.0 authorization code flow with PKCE spec 2026-05-09 21:51:51 -07:00
Scott Scoble
a2f7b7b2e9 fix: add dist-check guard + List<T> type parsing + state machine examples
Some checks failed
Deploy Landing Page to Storj / deploy (push) Failing after 2s
- check-dist.sh: Prevents DIST DESYNC by verifying all dist/*.js files
  are newer than their src/*.ts sources. Fails build if stale.
- Makefile: Standard build targets (build/check-dist/test/recompile/clean)
- src/index.ts: Integrated dist freshness check before compilation
- src/parser.ts: Added List<T> type parsing alongside existing Set<T>/Map<K,V>
  (e.g., 'log: List<LogEntry>' now parses as { type: 'list', elementType })
- examples/: Restored v0.3 state machine specs (TigerBeetleLedger,
  AccountLedger, ToggleSwitch, TwoPhaseCommit)
- DIST DESYNC is a known recurring anti-pattern — this guard prevents it

Tests: 39/41 pass (2 pre-existing E2E WASM failures due to missing wabt)
2026-05-05 03:11:53 -07:00
Scott Scoble
712ec94495 Add SpeckDL examples with Dark Provenance primitives 2026-04-29 14:44:00 -07:00
Marcus (@cos)
a97a532a3c Initial commit: Speckl project scaffold
- MIT license
- README with project overview, Dark Provenance intro, and structure
- Empty directories: speckdl/, reference-agent/, examples/
- Placeholder whitepaper with Dark Provenance outline

Parent: Greybeard Holdings, LLC
Status: Early / Pre-Alpha
2026-04-28 01:20:22 -07:00