Step 6 — Closures and Upvalues (Sketch, Deferred to cp-12)

Why This Is a Step at All

cp-07 deliberately rejects programs that capture a local from an enclosing function:

fn outer(a) {
    fn inner() { return a; }   // ❌ compile error in cp-07
    return inner();
}

with a clear message pointing to cp-12. This step explains why the restriction exists and what the implementation will look like when we lift it.

The Problem

A function value can outlive the stack frame in which it was defined:

fn make_counter() {
    var n = 0;
    fn step() { n = n + 1; return n; }
    return step;            // ← step references `n` AFTER make_counter returns
}

let c = make_counter();
print c(); print c(); print c();   // 1 2 3

When make_counter returns, its stack frame is destroyed — yet step still needs n. The variable has escaped from the stack.

Possible Solutions

cp-12 will implement the upvalue approach because:

• It keeps non-capturing locals as cheap stack slots (no boxing tax).
• It scales to mutable captures without aliasing footguns.
• It is what Lua 5.x, Lox, and many embedded VMs do — well-documented.

Sketch of the Upvalue Mechanism

Add three new value-level concepts:

struct Upvalue {
    Value* location;     // points into the stack while OPEN
    Value  closed;       // takes ownership when the slot is closed
    bool   isOpen;
    Upvalue* next;       // intrusive list, head per VM, kept sorted by stack address
};

struct Closure {
    FunctionPtr           fn;
    std::vector<Upvalue*> upvalues;   // resolved by index in bytecode
};

A Value::Fn evolves into Value::Closure carrying a shared_ptr<Closure>. Closure holds the function plus a vector of upvalues, one per captured variable.

New Opcodes (cp-12)

Compile Side

When the compiler sees a reference inside inner to a name declared in outer's locals:

1. Walk outwards through states_ until it finds the name.
2. In every intermediate function, add an upvalue entry whose isLocal=true in the immediately-surrounding function and isLocal=false deeper out.
3. Replace the bytecode with GetUpvalue idx / SetUpvalue idx.

When the compiler emits a Closure for a function with k upvalues, it emits k (isLocal, index) pairs following the opcode. At runtime, the VM reads these and either:

• (isLocal=true) captures the surrounding frame's slot directly (calls captureUpvalue(&stack_[frame.slotBase + index])), or
• (isLocal=false) copies one of the enclosing closure's upvalues (captureUpvalue(enclosing->upvalues[index])).

Run Side

captureUpvalue(loc) walks the open-upvalue list, returns an existing one if some closure already captured the same address, otherwise allocates a new Upvalue{loc, …, isOpen=true} and links it in sorted order.

When a local goes out of scope (or a frame returns), the VM emits CloseUpvalue / scans for any open upvalues at addresses ≥ the popping threshold and closes them — copying the value into closed and re-pointing location at &closed. From that moment on, the closure transparently sees the value through the heap copy.

The result: captured locals only pay the heap cost when they are actually captured, and only once per (variable × set of capturing closures).

Why Defer All This?

• The cp-07 lab is large enough already.
• Most of the interesting engineering — slot allocation, dispatch, frame management — is independent of closures and easier to learn in isolation.
• cp-12's JIT motivates closures: a closure becomes a useful unit for inlining and specialisation.

What cp-07 Actually Does

Op::GetUpvalue / Op::SetUpvalue / Op::CloseUpvalue exist in the opcode table (so cp-12 can drop in changes without renumbering) but the VM throws a runtime error if it ever executes one:

case Op::GetUpvalue:
case Op::SetUpvalue:
case Op::CloseUpvalue:
    throw RuntimeError(currentLine(),
        "upvalues not supported in cp-07 (see cp-12)");

And the compiler refuses to emit them — the isOuterLocal helper in steps/03 detects the attempted capture and emits a friendly diagnostic at compile time, well before the user sees any opaque runtime error.

Pitfalls (for cp-12)

• Forgetting to sort open-upvalues by stack address. The close operation relies on stopping at the first upvalue below the threshold.
• Double-closing. An upvalue captured by N closures must close exactly once; the open list dedupes by address.
• Calling convention coupling. The Closure opcode's variable-length operand encoding is awkward to disassemble; budget extra time on the disassembler.