RFC-0051 — Structured handler-body lowering for transforming backends

  • Status: Draft
  • Author: Mark Truluck
  • Scope: Replace the Erlang handler-body text-reparse pipeline (brace→case → SSA Data-threading → gen_statem tuple injection → early-exit nesting → separator joining, each re-scanning the previous stage’s emitted output) with a single structural lowering over a control-flow IR assembled from the scanned handler body. Erlang-first; the same IR generalizes to any backend that must transform native control flow (Lua). No effect on passthrough backends.
  • Companions: RFC-0042 (@@fsm — already has the IfAst/Block shape this reuses), RFC-0046 (@@:self.field), RFC-0050 (statement syntax).
  • Supersedes: a deleted earlier RFC-0051 draft that wrongly proposed parsing if/else into the front-end as a Frame statement. That framing is rejected here (§3); if/else stays native.
  • Tracking: framec#123 (text-oracle elimination), framec#119, framec#125.

1. Purpose

The Erlang backend lowers a handler body through a pipeline of text passes that reparse one another’s output:

spliced body
  → erlang_lower_native_if   (native `if … ->` → brace `if`)
  → erlang_transform_blocks  (brace `{ }` → `case … of`)        [now an FSM]
  → erlang_nest_early_exits  (re-scan `; false -> ok` to nest trailing code)
  → body_processor           (re-scan lines to thread `Data` SSA + wrap cases)
  → case_arms                (re-scan arms to inject `{keep_state,…}` tuples)
  → erlang_smart_join        (re-scan to place `,`/`;`/newline)

Stages 3–6 recover control flow and data flow from emitted text. This is the largest instance of the anti-pattern framec#123 exists to remove, and it is incomplete by construction: every stage enumerates a fixed set of textual shapes, and an unenumerated shape is silently mishandled. The bugs found this arc are all the same defect wearing different masks:

  • framec#119 — a nested fall-through arm returned bare ok instead of a gen_statem tuple (bad_return_from_state_function), because tuple injection only descended one text level.
  • framec#125 — an else if arm that mutates plus trailing code that mutates: Data not threaded through the non-terminal case (; false ->,, wrong DataN).
  • The nested early-exit crash — nest_early_exits mis-placed an outer trailing statement into the inner case as a duplicate ; false -> arm.
  • The over-folding bug — the early-exit nesting folds every no-else if + trailing into the false arm, even when the body does not exit; the trailing must run unconditionally. (Pre-existing in nest_early_exits; faithfully reproduced when it was ported into the parser.)
  • 37_aggregator — sequential non-terminal mutating ifs: the Data-wrap emitted a spurious __ReturnVal = case … / ; false -> ok,.

Each fix shifted the defect to the next stage (patching the fold exposed body_processor; a “does this body exit?” heuristic that sniffs self.result for "frame_transition__" is itself the disease). No sequence of text patches converges, because no stage has the structure it is reasoning about.

This RFC replaces stages 1–6 with: assemble a control-flow IR from the already- scanned handler body, then emit gen_statem Erlang from it in one structural pass. Early-exit, Data SSA threading, and tuple injection become properties of the structure, not pattern matches on text.

2. Why only Erlang (and Lua) deviate at all

The seven Frame statements are universal — -> $S, => $^, push$, pop$, $.x =/@@:self.x =, @@:return, @@:(e) — and a handler body is everywhere “the seven, plus native code.” What differs is how a backend expands them:

  • The 16 imperative/passthrough backends expand in place. They are mutable and statement-based: @@:self.x = 5 → self.x = 5; -> $S → …; return;; native if c { … } is already valid target syntax and passes through untouched. The segment scanner finds the seven statements, expands each where it sits, and the surrounding native code is the user’s own valid code.

  • Erlang cannot expand in place. It is immutable and expression-based, so the same seven statements reshape the surrounding code: @@:self.x = 5 becomes a new binding Data1 = Data#data{x = 5} that every later read must thread; -> $S must become the clause’s tail expression {next_state,…}, not a mid-body jump; and native if c { … } has no Erlang form — it must become case … of. A transition emits return on every imperative backend (frame_expansion/transition.rs), so if c { -> $S } trailing is naturally correct there; Erlang has no early return and must restructure.

So two distinct deviations are tangled together, and this RFC separates them:

  1. Intrinsic and unavoidable — Erlang reshapes code around the seven statements. Forced by the target. Not a defect.
  2. Accreted and fixable — that reshaping is done by re-parsing emitted text across six stages. This is what makes Erlang look unlike every other backend, and it is the entire bug surface.

Lua is the same kind of backend — it transforms the same native braces ({ } → do/then…end) and already parses them (OutputBlockParserFsm). So “backends that must rewrite native control flow” = {Erlang, Lua}; “backends that pass it through” = the rest. Only the first set needs the structure this RFC defines.

3. What this is — and explicitly is not

  • It is NOT if/else becoming the eighth Frame statement. Frame does not own its semantics, does not validate it, and makes no portability guarantee about it. The condition and the leaf statements remain native text, emitted verbatim into whatever control-flow form the target uses. The seven statements are special precisely because they have no native corollary; if/else has one and stays native.
  • It is NOT front-end parsing of native control flow. The front-end / segmenter continues to emit if/else as opaque NativeCode. Nothing about @@system parsing changes, so no passthrough backend is affected.
  • It is NOT a full native AST. Conditions and expressions are opaque text leaves. The only @@:self.field / $.x accesses inside them are already Frame constructs (RFC-0046) that framec recognizes; the structural walk rewrites them to the live DataN#data.field at their position.
  • It IS a control-flow skeleton — If / Block nodes wrapping the seven Frame statements (as typed nodes) and native text leaves — assembled at codegen for the backends that must transform control flow, so they reshape from structure instead of from their own emitted text.

The distinction is exactly the one the segmenter already lives by: framec parses the shape of a program (where @@system blocks are, where the seven statements are) without interpreting native code. The control-flow shape (if/else nesting) is the same kind of shape — needed only because some targets cannot take the native shape verbatim.

4. The IR 

HandlerIR = Block

Block     = Vec<Node>

Node =
  | Frame(FrameStmt)              // one of the seven — a typed node, NOT text
  | If { cond: Text, then: Block, els: Option<Block> }
  | Native(Text)                  // opaque native leaf: expression / call / the
                                  //   RHS of an assignment, with @@:self.field /
                                  //   $.x accesses marked for position-aware rewrite
  • FrameStmt is the existing structured form of the seven (Transition, Forward, StackPush, StackPop, the assign/return/context nodes). Critically, a transition is a node, so “does this branch exit?” is matches!(last, Frame(Transition|Forward)) — a structural fact, never a substring scan.
  • If.cond is text (emitted verbatim as case (<cond>) of), then/els are nested blocks. else if is els: Some([If{…}]) (right-nested), the shape IfAst already uses for @@fsm.
  • No Loop node: RFC-0050 §2.5 admits no Frame-level while/for/loop.

This is deliberately the same shape as IfAst/Block already built for @@fsm (RFC-0042). §5 reuses scanning, not the front-end grammar; §10 notes a later unification.

5. Production — assemble from the scan, never reparse output

The IR is built from artifacts that already exist, with no second pass over generated code:

  • HandlerEntry.body_statements: Vec<Statement> already holds the seven Frame statements as typed nodes interleaved with NativeCode(String) chunks, in source order (e.g. NativeCode("if c {"), Transition, NativeCode("} more")).
  • OutputBlockLexerFsm (the shared, string/comment-safe brace tokenizer Lua and Erlang already use via lex_blocks) tokenizes the NativeCode text into IF/ELSE/LBRACE/RBRACE/….

The builder walks body_statements once, maintaining a stack of open Blocks:

  • a NativeCode chunk is lexed; if <cond> { pushes an If (cond = the text between IF and the block {), } else { / } else if { switch/extend the arm, } pops; any non-structural text becomes a Native leaf;
  • a Frame statement is appended as a Frame node to the current block.

The result is the If/Block tree with Frame statements as nodes and native text as leaves. The brace discipline already proven for #122 (a {/} is a block delimiter only at a line-structural position; tuple/map braces are leaves) carries over directly — it is the same lexer.

This is a single scan-driven parse, the textbook front half of a compiler: lexer (FSM) → parser → IR. There is no inspection of emitted Erlang anywhere.

6. The Erlang structural emitter

One recursive walk of the IR emits gen_statem Erlang, threading two pieces of context functionally: the live Data SSA name and the live return value.

  • Block — emit nodes in order, threading Data. The block’s tail position determines whether its last node must yield a gen_statem return tuple.
  • Frame(VarAssign{field, rhs}) — Data{n+1} = Data{n}#data{field = rhs'}, where rhs' has @@:self.f rewritten to Data{n}#data.f (the live binding). Advances the Data name.
  • Frame(Transition) — frame_transition__(target, Data{n}, …, From, Ret) in tail position (it already returns {next_state,…}).
  • Native(text) — emit verbatim, with @@:self.f rewrites; never a return.

  • If { cond, then, els } — the core, and the three behaviors that were three buggy text stages become three structural cases:

    1. Terminal if (early exit). then unconditionally exits — i.e. its last node is a Frame(Transition|Forward), a structural test. With no els and trailing nodes after the If in the parent Block, the trailing is the false arm: case (cond) of true -> <then> ; false -> <trailing> end. The over-folding bug is impossible: a non-exiting then does not satisfy the predicate, so its trailing runs unconditionally.
    2. Non-terminal if that mutates. then/els bind Data but do not exit, and code follows. The If must yield the merged Data: Data{n+1} = case (cond) of true -> <thenData>; false -> <elseData/Data{n}> end, and the walk continues from Data{n+1}. #125 and 37_aggregator are impossible: the merge is explicit, not a text wrap.
    3. Pure control if in tail position — both arms end in returns; the case is itself the clause’s tail.
  • Return-tuple completeness. Because the walk knows tail position, every leaf on every path emits exactly one status tuple (frame_transition__ / {keep_state, Data{n}, [{reply, From, Ret}]}). #119 is impossible by construction.

frame_transition__/7, the #data{} record, expression_to_string (Erlang arm), and the __ReturnVal convention are reused unchanged — only their placement moves from “recovered from text” to “known from structure.”

7. Correctness is structural, not enumerated

Each bug this arc found maps to a property the IR makes true by construction:

Bug Text-pipeline cause Structural guarantee
#119 fall-through ok tuple injection descends one text level every leaf path yields a tuple (§6)
over-folding folds any no-else if+trailing fold only when then is a transition node (§6.1)
nested early-exit crash ; false -> ok re-scan mis-nests nesting is the tree, not a re-scan
#125 / 37_aggregator Data not threaded through non-terminal case explicit SSA merge at If joins (§6.2)

There is no enumeration of arm shapes to be incomplete, and no self.result sniffing. The same read_after / nested / 37_aggregator inputs that the text pipeline gets wrong are correct here because the lowering reasons about the program, not its printout.

8. What is deleted

  • erlang_system/blocks.rs: the early-exit fold inside ErlangBlockParserFsm, erlang_nest_early_exits (already gone), and erlang_lower_native_if (native if becomes an If node directly — §5 lexes if … -> as readily as if … {).
  • erlang_system/body_processor.rs (~1,746 lines): the line classifier and SSA Data text-threading; the CaseFrame wrap. The SSA logic becomes the §6 walk.
  • erlang_system/case_arms.rs (~430 lines): analyze_case_arms, rewrite_mixed_case_arms, erlang_inject_orphan_reply_tuples. Tuple placement becomes §6.
  • erlang_smart_join shrinks to formatting only; the bracket core stays the RFC-0035 paren FSM.

Net: ~2,100 lines of text-reparse → one structural emitter, with the scanning already shared (OutputBlockLexerFsm).

9. Migration — phased, behind a seam, TDD-gated

The single substitution point is erlang_system.rs where the spliced body is handed to the text pipeline; the IR path slots in there.

  • Phase 0 — contract. Lock the behavior as executable tests before code: the bug repros (read_after, nested, else-if + trailing/#125, 37_aggregator, basic/exiting early-exit) as behavioral fixtures asserting runtime results, plus the 322 existing Erlang fixtures as the regression net. These define “done.”
  • Phase 1 — IR + builder. §4 types + the §5 scan-driven builder. Unit-test the IR shape on the Phase-0 inputs (assert tree structure, not text).
  • Phase 2 — emitter behind a fallback seam. §6 emitter; route a handler through it only when its body is fully representable, else fall back to the text pipeline. Land with zero regression; grow coverage handler-shape by handler-shape (transitions, forward, push/pop, persist, async, HSM cascades).
  • Phase 3 — cut over + delete. Make the IR path the only path; delete §8.
  • Phase 4 — Lua. Re-point Lua’s do/then…end transform at the same IR (it transforms the same braces). Optional, later.

10. Risks and mitigations

Risk Mitigation
gen_statem surface is large (transitions, forward, push$/pop$, persist, async, HSM enter/exit cascades, self-call guards) — the emitter must cover all the text pipeline does Phase-2 fallback seam: unsupported shapes use the old path until the emitter covers them; never a regression. The 322-fixture matrix is the coverage oracle.
Building the IR from interleaved body_statements + lexed NativeCode mis-aligns spans The statements are already source-ordered; the builder is a single linear walk, unit-tested on the Phase-0 inputs.
Re-deriving an If shape duplicates @@fsm’s IfAst Reuse the IfAst/Block types; §5 only adds a scan-driven builder for @@system bodies. A later RFC may unify the producers.
Behavioral drift on the 322 fixtures Per-handler cutover + before/after generated-Erlang diff where behavior is unchanged; full matrix 17/17 each phase.

11. Validation

  • Phase-0 behavioral fixtures green (the bug repros — currently failing or latently wrong under the text pipeline).
  • Erlang matrix (322 + the new fixtures) and full 17-language matrix 17/17.
  • Unit tests on the IR builder (tree shape) and the emitter (Data SSA names, tuple completeness, early-exit placement, non-terminal case merge).
  • cargo test + clippy + fmt clean; generated-Erlang byte-diff reviewed where behavior is unchanged.

12. Out of scope

  • The passthrough backends — unchanged; they never see the IR.
  • Loops — none (RFC-0050 §2.5).
  • Front-end grammar changes for @@system — none; if/else stays native, parsed for shape at codegen only.

13. Definition of done

  1. The Erlang handler-body lowering is one structural pass over the §4 IR; body_processor.rs, case_arms.rs, and the blocks.rs early-exit/native-if logic are deleted.
  2. The Phase-0 repros (read_after, nested, #125, 37_aggregator, early-exit) pass behaviorally; #119 and #125 close.
  3. Erlang matrix green; full matrix 17/17; unit tests + clippy + fmt clean.
  4. No code path inspects emitted Erlang to recover structure.