Four defects found by dogfooding Speckl against the v0.3 product-layer
spec (FederatedMeetupProduct.speckdl, 74 members):
D1 — CRITICAL: List<T> fields in interface records emitted empty type
Root cause: interface record fields called speckTypeToProto(f.type.name)
which returns '' for List/Map/Set types (their .name is undefined).
Event and state fields had inline List/Map dispatch, but interfaces did not.
Fix: extracted typeExprToProto() shared function that handles all TypeExpr
variants (primitive, ident, list, map, set). All four field-emitting code
paths (interfaces, events, state, transitions) now use it.
D2 — MODERATE: Transition type discriminator field silently dropped
Root cause: parser stores the type field in t.typeDiscriminator (string)
rather than in t.fields array. Generator only emitted t.fields.
Fix: emit typeDiscriminator as 'type' field at number 1, offset other
fields by 1 to avoid collision.
D4 — LOW: Enum variant name collisions across enums in same proto scope
Proto3 requires all enum values unique within enclosing scope, not just
the enum. PENDING in Role and PENDING in OrderStatus collided.
Fix: prefix all variants with upperSnake(enumName) — the proto3 convention.
This also matches the handwritten proto's naming (TRANSITION_TYPE_*).
D5 — LOW: Date in interface fields did not trigger timestamp import
The usesTimestamp check only scanned events and state, not interfaces.
Fix: added interface member check to the timestamp import scan.
D3 (deferred): rc_ typo in spec source silently swallowed by parser.
Parser bug, not generator bug. Filed for future cycle.
Verified: 56/56 tests pass, both FederatedMeetup.speckdl and
FederatedMeetupProduct.speckdl compile to protoc-clean output.
End-to-end compile of FederatedMeetup.speckdl (with the cycle-65
Transition envelope added) surfaced a real bug: the speckdl uses
default-value syntax on event fields:
initial_mesh_peers: List<InitialMeshPeer> = []
The default is a Speckl semantic — a state-machine init value —
but the .proto wire format has no place for it. Without stripping,
the type expression 'List<InitialMeshPeer> = []' was emitted
verbatim as the field type, producing broken proto3 syntax that
protoc rejected.
Fix: parseEventBlockMultiline now strips '= <expr>' suffix from
the type expression before calling parseTypeExpr. The default value
is dropped (it's a Speckl semantic only; the wire format carries
no default).
Tests: 1 new parser test covers the federated-meetup
'initial_mesh_peers: List<InitialMeshPeer> = []' shape. All 56
tests pass (55 baseline + 1 new). tsc build clean.
End-to-end verified: with this fix and the cycle-65 federated-
meetup speckdl changes, the Speckl-generated .proto now compiles
cleanly with 'protoc' (previously rejected).
The .proto generator now recognizes two new top-level speck members
and emits real proto3 syntax for them:
oneof Name { oneof is NOT emitted at top level
variant_field: VariantType (it lives inside the transition's
} payload block — see below)
transition Name { emitted as:
type: <EnumName> message Name {
payload: <OneofName> <regular fields>
hlc: Bytes oneof payload {
} VariantType variant_field = 10;
...
}
}
The and fields are special-cased: type records the
discriminator enum reference (so future type-aware code can find it),
payload records the sibling oneof block to inline. Neither is emitted
as a regular proto field.
The oneof variants reference the suffixed message name when the
speck has 'event_suffix: "Payload"' (or any suffix). This makes the
generated variants match the consumer's Go types in the federated-
meetup case: 'CreateGroupPayload create_group = 10' is the type
binding the consumer's apply switch expects.
End-to-end verified: a speck with
event_suffix: "Payload"
oneof TransitionPayload {
create_group: CreateGroup
add_steward: AddSteward
}
transition Transition {
type: TransitionType
payload: TransitionPayload
hlc: Bytes
}
event CreateGroup { name: String }
event AddSteward { new_steward: String }
emits the right proto3:
message CreateGroupPayload { ... }
message AddStewardPayload { ... }
message Transition {
bytes hlc = 1;
oneof payload {
CreateGroupPayload create_group = 10;
AddStewardPayload add_steward = 11;
}
}
All 55 tests pass. tsc build clean.
The federated-meetup speckdl (specs/FederatedMeetup.speckdl) does
not yet declare the Transition envelope — it uses 13 separate
event blocks. Adding the transition+oneof envelope to the real
spec is the next step (cycle 65).
End-to-end compile of FederatedMeetup.speckdl with the new
'state as StateSnapshot' header (cycle 63) surfaced that the
parser's main loop and parseMemberBlock dispatcher were both
missing 'state as ' as a recognized block-starter prefix. The
state-block parser (parseStateBlockMultiline) already extracts
the 'as' clause correctly (cycle 59), but the call site never
reached it: the block fell through the 'state:' / 'state {'
checks and was parsed as a regular member line, producing no
StateNode at all.
Fix: add 'state as ' to the blockStarters list in the main
parse loop, and add a parallel dispatch in parseMemberBlock
that routes 'state as ' headers to parseStateBlockMultiline.
The dispatcher change is symmetric with the existing 'state:'
and 'state {' forms.
Tests: 1 new parser test confirms the dispatch works for
'state as StateSnapshot { ... }'. All 55 tests pass (54
baseline + 1 new). tsc build clean.
End-to-end verified: running generateProtobuf against the
federated-meetup speck with 'state as StateSnapshot' now
emits 'message StateSnapshot { ... }' (correct), where
pre-fix the StateNode was lost entirely.
End-to-end compile of FederatedMeetup.speckdl surfaced a real
generator input bug: inline line comments on field declarations
were bleeding into the parsed type expression. A field like
mesh_ip: Bytes // 4 bytes (IPv4 overlay)
produced a TypeExpr whose name was literally
'Bytes // 4 bytes (IPv4 overlay)'
which the .proto generator emitted verbatim:
bytes // 4 bytes (IPv4 overlay) mesh_ip = 2;
— broken syntax (comment in the type position).
Fix: both parseEventBlockMultiline and parseInterfaceBlockMultiline
now strip '// ...' suffixes from each inner line BEFORE matching the
'name: type' regex or splitting on commas. The state-block parser
already had this; cycle 62 brings event and interface to parity.
Tests: 2 new parser tests covering event and interface blocks, both
with the FederatedMeetup-style inline comment. All 54 tests pass
(52 baseline + 2 new). tsc build clean.
End-to-end verified: running generateProtobuf against the real
FederatedMeetup.speckdl now emits clean 'bytes mesh_ip = 2;' for
the InitialMeshPeer interface, where cycle 62's pre-fix output
was the broken 'Bytes // 4 bytes (IPv4 overlay) mesh_ip = 2;'.
This unblocks the next phase of the federated-meetup Speckl
migration: the consumer Go code can now bind to the generated
proto without a half-broken field in the way.
Adds 'event_suffix: "Payload"' as a top-of-file directive. When set,
the .proto generator appends the suffix to every 'event' block's
emitted message name. Default (no directive) preserves the existing
'event CreateGroup' → 'message CreateGroup' behavior.
Why: federated-meetup's handwritten proto names every transition
payload CreateGroupPayload, AddStewardPayload, etc. (the convention
that the wrapper message carries a 'Payload' suffix and the event
type lives in 'oneof payload { ... }'). Without this directive, the
Speckl-generated proto would emit 'message CreateGroup' and the
consumer Go code would fail to find the types it imports.
The test fixture uses three tests:
- directive parsed and stored on SpeckNode
- directive absent → eventSuffix undefined
- both quoted ("Payload") and bare (Payload) values accepted
The reset of (parseSpeck as any)._protoPackage / _goPackage /
_eventSuffix at the start of parseSpeck fixes a pre-existing
static-field-hack leak where a previous test's metadata would
persist into the next test's parse. Documented as a known
anti-pattern in spec-compiler-dogfood SKILL.md; full cleanup
to a local-object pattern is tracked separately. The minimum
fix here is 'reset at start of parse' which restores test
isolation without refactoring the entire metadata-collection
path.
End-to-end verified: running generateProtobuf against a speck
with event_suffix: "Payload" emits 'message CreateGroupPayload'
and 'message AddStewardPayload' (correct).
All 52 tests pass. tsc build clean.
Reference: .hermes/skills/spec-compiler-dogfood/references/
speckl-federated-meetup-protobuf-migration.md — the migration
unblocks when every generated message name matches what the
consumer's Go imports expect.
Adds a regression test that confirms a bare-identifier interface (no
methods with parens, no fields with colons) parses as kind: 'enum'
with the variants on its methods[] array. The .proto generator already
emits these correctly as 'enum X { X_UNSPECIFIED = 0; FOO = 1; ... }'
- confirmed end-to-end by running generateProtobuf against the
federated-meetup CustodyTier shape.
This was a verification cycle rather than a feature cycle: the compiler
already does the right thing for enums, but without a regression test
a future refactor of classifyInterfaces or parseInterface could break
this silently. The test pins the behavior so a PR that breaks enum
emission will fail CI.
All 49 tests pass (48 + 1 new). tsc build clean.
Reference: federated-meetup.speckdl CustodyTier shape
interface CustodyTier {
ColdHardware
HotSigning
ThresholdSplit
BackupOnly
Compromised
}
Adds 'state as <MessageName> { ... }' syntax for the .proto generator.
When the 'as' clause is present, the generator emits
message <MessageName> { ... }
instead of the default
message <SpeckName>State { ... }.
Why: the federated-meetup consumer (internal/types/types.go and friends)
imports 'pb.StateSnapshot', not 'pb.FederatedMeetupState'. The previous
default of namespacing by speck name broke the migration: the generated
proto would have compiled cleanly in protoc but the consuming Go code
would not have built. With this override, the Speckl-generated proto
emits the exact message name the consumer expects.
The default behavior (no 'as' clause) is unchanged, so existing specs
that relied on the old convention (e.g. tests, examples) still work.
Tests: 2 new parser tests, both green. All 48 tests pass. tsc build clean.
Reference: .hermes/skills/spec-compiler-dogfood/references/
speckl-federated-meetup-protobuf-migration.md
— the build break last session was caused by the missing override.
Adds 'transition Name { ... }' as a top-level member of a speck
block. The AST node is TransitionNode with fields[] plus optional
payloadOneof and typeDiscriminator references. Two field names
have special meaning:
payload: <OneofName> → the generator will emit the referenced
oneof's variants inline as
'oneof payload { ... }' inside the
message body (i.e. payload becomes a
proto oneof, not a regular field)
type: <EnumName> → the discriminator enum for the variants
All other fields are emitted as ordinary proto message fields.
This is the shape federated-meetup needs for its Transition message:
message Transition {
TransitionType type = 1;
StateRoot prior_state = 2;
oneof payload {
CreateGroupPayload create_group = 10;
...
}
Multisig steward_signatures = 31;
bytes hlc = 32;
fixed32 branch_id = 33;
google.protobuf.Timestamp signed_at = 30;
}
The next cycle (64) will wire the .proto generator to emit this shape
from a TransitionNode. Cycle 58 is parser-only — 3 new tests cover
regular fields, payload+type references, and trailing punctuation
tolerance. All 46 tests (39 baseline + 4 oneof + 3 transition) pass.
tsc build clean.
Adds 'oneof Name { field: Type, ... }' as a top-level member of a
speck block, alongside interface, event, service, etc. The AST node
is OneofNode with a variants array of { fieldName, typeName }. The
parser supports multi-line blocks, comments, and trailing commas /
semicolons. Generator integration is a follow-up cycle (proto/TS).
Motivation: federated-meetup's Transition message needs a oneof for
the 15+ payload variants (create_group, add_steward, ...). Without
oneof syntax, the consumer Go code has to hand-roll
'Transition_AddHostPeer' wrapper messages for each event type.
With oneof, the generated .proto emits a real proto3 oneof and the
generated Go bindings have a single 'isXXX()' discriminator API
that matches the consumer's expectations.
Tests: 4 new parser tests covering happy path, trailing punctuation,
comments, and coexistence with other member types. All 43 tests
(39 baseline + 4 new) pass; tsc build clean.
Reference: spec-compiler-dogfood skill, references/speckl-federated-
meetup-protobuf-migration.md — the prior build break was caused by
proto message-shape mismatches between the generated .proto and the
consumer Go code. oneof syntax is the first of 5 compiler extensions
needed to make Speckl-generated proto bindings a drop-in for the
handwritten proto/federated_meetup/v1/{state,rpc}.proto.
The .proto backend previously emitted:
package speckl.<name>.v1;
option go_package = "github.com/sscoble/<name>/proto/.../v1;...v1";
hard-coded. Both fields are now overridable via the SpeckNode's
protoPackage / goPackage fields (populated by top-of-file metadata in
the .speckdl; see upcoming cycle 57 for the related oneof syntax which
will also be reading from the same SpeckNode).
For federated-meetup the overrides are:
proto_package: "federated_meetup.v1"
go_package: "github.com/sscoble/federated-meetup/proto/...;federated_meetupv1"
The consumer Go code (internal/types/types.go, internal/group/) imports
"federated_meetup/v1" — without this override, the generated proto
would emit package "speckl.federated_meetup.v1" and the bindings
would not match the imports, breaking the consuming project's build.
Picks up a half-applied change left in the working tree from the
v0.6 review-v2 session. dist/ is gitignored; tsc build was clean
and all 39 baseline parser tests pass.
Multi-line `input: { ... }` and `output: { ... }` blocks fell through
to parseMember which returned `{ type: 'ident', name: '{' }` instead of
a record with fields. Broke RetryHandler.speck and similar real specks.
Fix: add `input:`/`output:` to blockStarters, dispatch to new
parseInputOutputBlock() that brace-counts across newlines, strips
line comments, and delegates to parseTypeExpr's record branch.
CoS-verified-live-close (cycle 126 pattern): subagent hit max_iterations
on the brace-count edge case. CoS applied focused fix + 2 new tests.
39/39 vitest pass.
Test evidence:
- RetryHandler.speck: input typeExpr.type='record' 4 fields
- 37 prior tests + 2 new = 39 passing
- Add 'verify:' to the parseSpeck dispatch condition so multi-line verify:
blocks reach the multi-line parser
- New parseVerifyColonBlock helper collects continuation lines using the
same break-condition list as the existing skip-loop, with depth-balanced
counter for (), [], {}
- Joins the parts with single spaces and returns a VerifyNode with a
well-formed temporalExpr
- Closes sscoble/speckl#54 — unblocks B2 PR stack (#47 + #48 already closed)
- Pre-existing LTL inner-identifier scoping bug remains in #55 (out of scope)
- Adds <meta name='author' content='Scott Scoble'> to landing page head
- Adds JSON-LD SoftwareSourceCode structured data with author, repo, license, keywords
- Updates hero subtitle to 'A Judgement Transport Utility by Scott Scoble'
- Updates OG description with 'Created by Scott Scoble'
- Updates OG site_name to 'Speckl by Scott Scoble'
- Keywords include 'scott-scoble' for name association
- Predicate pushdown now splits AND predicates and pushes each to
the correct side of the join (left table vs right table)
- FROM clause no longer duplicates tables that appear in JOINs
- Constant folding no longer mangles date literals (skips strings)
- Cost model: hash join uses build/probe costs, nested_loop uses
per-row probe cost instead of cartesian product
- Physical operator selection defaults to hash join for medium+ tables
- Two-pass SQL generation: collect join tables first, then emit FROM
- Demo shows 86% cost reduction on users/orders query, 25% on
departments/employees query from ekeric13/sql-query-optimization-guide
SQLOptimizer.speckdl: state machine spec for the optimizer pipeline
(Parsing → Rewriting → Planning → Generating → Done, plus Failed)
SQLOptimizer-runtime.ts: working TypeScript runtime that actually
optimizes SQL queries:
- Parses SELECT/FROM/JOIN/WHERE/GROUP BY/ORDER BY/LIMIT
- Builds plan trees with cardinality estimation
- Predicate pushdown (splits AND predicates per table)
- Constant folding (evaluates arithmetic in predicates)
- Column pruning
- Physical operator selection (hash/merge/nested_loop joins)
- Cost estimation per plan node
- SQL generation from optimized plan tree
Known issues:
- Predicate pushdown needs to split AND predicates per table
- FROM clause duplicate when both listed and joined
- Cost model needs tuning (hash join startup cost shouldn't exceed
nested loop savings for medium inputs)
Both files pass tsc --noEmit clean.
Parser fixes:
- Fixed multiline state/init block parsing: trailing commas no longer
treated as continuation markers, merging all state vars into one
- Added splitFields() function for comma-aware emit field splitting
that respects parenthesized groups and quoted strings
- Fixed interface block parsing: enum-style interfaces (name-only
values like 'Idle, Parsing, Parsed, ...') now correctly parsed as
methods instead of being skipped (methods.length was 0)
TypeScript generator fixes:
- String[] array type syntax now correctly maps to string[] in output
- collectIdents filters out names containing brackets
- Duplicate event interfaces deduplicated (implicit emit events
filtered when explicit events already declare them)
- join(X, Y) translates to X + Y (string concatenation) instead of
X.join(Y) which is wrong for string args
- now() generates new Date().toISOString() instead of Date.now()
to match String/string type annotations
- decisionLog type uses proper union: (A | B | C)[] instead of A[]
- String literals protected from this.-prefixing during identifier
replacement (fixes 'leader' inside quotes becoming 'this.leader')
- Removed debug console.log from enum generation
Both TigerBeetleLedger.speck and speckl-compile.speckdl now generate
TypeScript that passes tsc --noEmit with zero errors.
Closes#9
- Add braceDepth tracking to Map/Set type parsing for inline records
- Fix multiline state block joining to preserve type expressions
- Fix multiline object literal parsing in assignments
- Add smart set-difference for Maps vs Sets
- Fix bare notIn for Sets
- Add non-null assertion (!) to Map.get() calls
- Add implicit event interface generation from emit statements
- Protect object literals from this. prefix injection
The four design docs from the GP architecture research thread. These define Speckl's
general-purpose architecture: conversation → SIR → deterministic compile, with domain
plugins for state machines, data schemas, and workflows.
- sir-schema.md (48KB): Central IR data structure with ~20 interfaces, 8 domain element tables
- extraction-protocol.md (47KB): How conversation populates the SIR
- validation-engine.md (39KB): SIR validation before compilation
- compilation-layer.md (43KB): How validated SIRs compile to deterministic artifacts
Phase strategy: build GP core now, ship state machines first, layer data schemas
and workflows on the same architecture (no rewrites).
- Declare speckl_len_Int helper for list/sequence lengths
- Fix infix = inside => bodies (final pass regex)
- Handle prime notation: currentTerm' → currentTerm_post
- Handle length() calls: length(log) → (speckl_len_Int log)
- Handle min(): (ite (<= a b) a b) instead of unsupported 'min'
- Handle subtraction: i-1 → (- i 1)
- Declare _post state vars when invariants compare pre/post
- Added 3 BMC verify blocks to KRaft spec (TermMonotonicity, CommitSafety, LeaderCompleteness)
- Z3 returns clean sat with zero errors for all specs
- All 37 tests pass
- Sanitize sort names: Option(NodeId) → Option_NodeId
- Emit action transition predicates only for verify/BMC mode
- Handle -> implication operator in translateExpr
- Fleshed out KRaft invariant bodies (4/6 now have real expressions)
- ToggleSwitch: clean sat with model
- TigerBeetleLedger: sat with model
- KafkaKRaft: sat with model
- All 37 tests pass
- x in someSet → someSet.has(x)
- x in someMap.keys → someMap.has(x)
- x notIn someMap.keys → !someMap.has(x)
- m[k] → m.get(k) for Map state variables
- s.size() → s.size (property, not method)
- Track Map-typed state variables via mapVarNames for accurate .get() translation
- Thread mapVarNames through emitAction and rewriteExpr
All 41 tests passing. TigerBeetleLedger now generates correct TypeScript
with .get() for Map access and .has() for Set membership.
Part of M0.1: Fix TS generator bugs (speckl#18)
- Deleted compiler/src/generators/wasm.ts (472 lines)
- Deleted compiler/src/runtime/wasm-host.ts (165 lines)
- Deleted compiler/__tests__/e2e-wasm.test.ts (203 lines)
- Removed WASM import and generation call from compiler/src/index.ts
- Removed WASMGenerated from TS generator state enum literals
- Removed WASM codegen tests from parser.test.ts
- All 37 tests passing
The WASM generator was architecturally wrong: every Map/Set operation
crossed the WASM→JS boundary, creating 45-55% overhead for a module
that doesn't compute anything. WASM will be a byproduct of the Rust
backend via wasm-pack instead (M2.1).
Closes#19
