RFC-0030: Fuzz infrastructure catch-up — multi-RFC corpus migration

  • Status: Accepted (Execution committed 2026-05-18)
  • Author: Mark Truluck mark.truluck@cogiton.com
  • Created: 2026-05-18
  • Prerequisite to: Frame 4.1.2 RC (or whichever release this lands in)

Framing. RFC-0029 catalogued the fuzz infrastructure status and recommended treating it as developer-side preventative infra not blocking RC. The 2026-05-18 pre-RC validation pass exposed that the fuzz corpus has accumulated multi-RFC staleness over months — the persist generator emits a contract three RFC amendments behind (RFC-0012 + RFC-0015), per-language renderers in langs.py bake the old persist API across 40+ sites, and multiple shell-phase generators emit drivers that bypass the RFC-0015 factory contract. The matrix has been the working release signal during this period; the fuzz harness silently rotted. This RFC is the catch-up plan. Estimated 1-2 weeks focused work; explicitly scoped to RC-block until done.

What “stale” actually means here

The fuzz harness has two layers:

  1. Generators (fuzz/diff_harness/gen_*_pure.py for Phase 2-7; fuzz/gen_*.py for Phase 8-21) — produce abstract Frame source per case. Several emit pre-RFC syntax.
  2. Per-language renderers (fuzz/diff_harness/langs.py, 4001 LOC) — render each Frame case into language-specific runnable programs (instantiate, drive, save/restore, classify trace). Many renderers call the old persist API.

Both layers need updating. The 2026-05-18 pre-RC investigation confirmed three concrete staleness classes:

Class 1: RFC-0013 — @@target lang → @@[target("lang")]

The hard cut shipped in framec via E804. All 7 diff-harness generators emit the bare form, but run_fuzz.py rewrites it to the attribute form at runtime, so this class is functionally masked — generators are stale but the harness compensates. Recommend cleaning the generators anyway (consistency, removes the rewrite shim).

Class 2: RFC-0012 + RFC-0015 — persist contract

Two amendments compounded:

  • RFC-0012 amendment: @@[persist] alone is insufficient; must declare @@[save] / @@[load].
  • RFC-0015 hard-cut: operations: block form deprecated. Module-level attributes mandatory: 
    @@[persist(<type>)]
@@[save(<method-name>)]
@@[load(<method-name>)]
@@system Foo { ... }

    Plus the API shift: Foo.restore_state(blob) classmethod → inst.restore_state(blob) instance method (caller creates the instance via factory).

Impact:

  • gen_persist_pure.py emits the pre-amendment form (@@persist bare).
  • langs.py calls Foo.restore_state(blob) in 40+ places across every backend’s persist renderer.

Class 3: RFC-0015 factory bypass in shell-phase drivers

Shell-phase generators (gen_perm.py, gen_ctrl_flow.py, etc.) embed a per-language driver that does _inst = Foo() (bare constructor). RFC-0015 made the factory the only valid construction form — Foo() skips the $> dispatch and state-vars never get populated.

Surfaced as 3 cases in Phase 10 (perm) + 12 (ctrl_flow) failing with KeyError: 'scache' at runtime when handler bodies read state-vars from an uninitialized compartment.

Likely affects all shell-phase generators with per-language driver templates — needs survey.

Execution plan

Six discrete tracks, sequential or parallel as the surface allows. Estimated effort assumes one focused worker; explicitly not the developer-side smoke-tier work that already runs in 2 minutes.

Track A — Persist phase end-to-end (4-8 hr)

  1. Update gen_persist_pure.py to emit RFC-0015 contract (@@[persist(<type>)] + module-level @@[save]/@@[load]). ✅ Generator emit updated 2026-05-18.
  2. Update langs.py persist renderer for all 17 backends: Foo.restore_state(blob) → factory-construct + instance call. The persist-blob type is per-target (Python str, Rust String, GDScript PackedByteArray, etc.); add a per-Lang field for it.
  3. Regenerate cases/persist/ with new generator.
  4. Run python3 diff_harness/run_fuzz.py --cases cases/persist --langs <each> per backend; verify 17/17 pass-or-skip green.
  5. Land Track A’s commits as a single PR-ish unit; clean handoff point.

Track B — Multisys phase (~1 hr)

  1. Update gen_multisys_pure.py to emit @@[main] on the lead system (RFC-0021 requirement for multi-system modules).
  2. Regenerate cases/multisys/.
  3. Verify diff-harness pass.

Track C — Shell-phase driver factory migration (3-6 hr)

  1. Audit all gen_*.py shell-phase generators for _inst = Foo() pattern in driver templates.
  2. Replace with the per-language factory form. Per the cases_persist_x/ reference: Frame-level @@SystemName() syntax (compiled to per-backend factory automatically) is the cleanest path — works in any backend without per-language driver branches.
  3. Regenerate affected cases_* directories.
  4. Verify state-var-using cases now compile + run clean.

Track D — Other-phase renderer survey (4-8 hr)

langs.py is 4001 LOC of per-backend renderers. Tracks A-C address the known issues; this track is the systematic survey: for each phase (selfcall, hsm, operations, async, multisys, nested, perm, etc.), verify the per-backend renderer matches current Frame API. Specifically check:

  • Construction (factory vs bare ctor)
  • Dispatch (instance method invocation shape)
  • Trace emission (matches TRACE_FORMAT.md v1)
  • Persist (if applicable — only persist + persist_x phases use it)

Open dedicated sub-tasks for each renderer divergence found.

Track E — Regen all 35k cases (~30 min)

After Tracks A-D land, regenerate the full corpus:

cd fuzz/
for gen in diff_harness/gen_*_pure.py gen_*.py; do
  python3 "$gen"
done

Spot-check the per-phase counts match FUZZ_PLAN.md expectations.

Track F — Full fuzz re-run + surfaced-defect fixing (open-ended)

FRAMEC=... ./run_all.sh --tier=full --lang=all should now run all phases. Real framec defects that surface — fix in framec (separate commits) and re-run. Estimated yield based on historical ratio: 3-8 defects over the corpus regeneration. Each defect adds ~30-60 min to investigate + fix + verify, plus matrix re-run for regression confirmation.

Tracking

Each track lands as its own commit (or commit cluster). After all six green, run_all.sh --tier=full --lang=all should report every phase PASS — that’s the RC bar for fuzz coverage.

Roadmap: #172 was closed by RFC-0029 as “the work was done.” This RFC reopens that question — the work was built but the corpus

  • renderers have rotted such that “the fuzz works” is no longer true. Closing this RFC = the fuzz works again.

Drawbacks

  • 1-2 weeks blocks the RC. Real time cost. Acceptable per Mark’s 2026-05-18 decision; documented here so the cost is visible.
  • The matrix has been the working RC signal during the rot period. Fixing fuzz doesn’t catch issues that have already shipped; it restores the ability to catch future regressions that the matrix misses by construction (the matrix is curated; fuzz is generated permutation).
  • Renderer migration is mechanical but error-prone. 40+ per- language sites in langs.py; each migration must preserve trace-line semantics. Risk of introducing rendering bugs that produce false-positive divergences. Mitigation: run each track’s cases through the diff-harness BEFORE moving to the next; bisect any divergence to the renderer change.
  • No proactive guard for future RFC rot. Without a “matrix + fuzz must both pass before merge” CI policy, the corpus will rot again as soon as the next breaking RFC ships. Separate concern, separate RFC (cf. RFC-0029 § CI integration question).

Unresolved questions

  • Should the per-language renderer migration be one big commit or 17 small commits? Big-bang is easier to review for the pattern (factory + instance call); per-lang is easier to bisect if a divergence surfaces. Lean per-lang.

  • run_fuzz.py’s @@target rewrite shim — keep or remove? Cleaning Track A’s generator emission means the shim is no longer load-bearing. Removing it would catch future generator staleness immediately rather than silently masking it. Recommend remove as part of Track A close-out.

  • Smoke tier vs full tier as the RC bar. Smoke is ~2 min; full is ~45 min. Recommend smoke for ongoing pre-PR gate, full for release candidates only. Decide in RFC-0029 § CI integration follow-up.

References

  • RFC-0029 — fuzz infrastructure status pre-this-RFC
  • framec-test-env/fuzz/FUZZ_PLAN.md — phase catalog
  • framec-test-env/fuzz/diff_harness/langs.py — 4001-LOC renderer file; primary surface for Tracks A and D
  • framec-test-env/fuzz/diff_harness/gen_persist_pure.py — Track A generator (already updated 2026-05-18)
  • Roadmap #172 (closed by RFC-0029, effectively reopened by this)