perf: P11 step 2 - the counted for pass over a flat list (step 2 of #58) #470

Closed
opened 2026-10-24 07:19:12 +00:00 by orion · 30 comments
Collaborator

Context

Step 2 of #58 (perf: P11 - a for over a concrete list is a counted loop; PERFORMANCE.md finding F5 level 2). Step 1 (#415) landed the specification as docs/PERFORMANCE.md section 5.1 "What P11 builds"; that record is the specification of this issue and decisions 1 to 6 there are not reopened.

Goal

Build the pass: for value in numbers over a cursor of the flat runtime list becomes a length read once, a counter and an unchecked Element read, on both back ends, and benchmarks/list-iterate is measured before and after in forge CI.

Acceptance criteria

  • New compiler/src/ir/counted.trb with one wiring line in finishLowering (ir/lower/lower.trb), between rewriteElementPlaces and dropProvableChecks (decision 1). It accepts only a Record cursor of std's ListIterator, TrieListIterator or BufferIterator whose next is that type's own next (found through standardFunctionsOf, never by field shape of a user type), and only a loop it recognises whole (decisions 2 and 3). Any other loop keeps its cursor, Option and dispatch.
  • Step 1's first thing to re-check: #413 has landed since the record was written ("Related work in review"). Re-read the cursor's shape in std/collections and std/core on main, adjust decision 2's match if it changed, and update that paragraph of section 5.1.
  • IR snapshot test in compiler/tests/: the loop of list-iterate has no next call, a less, a [... unchecked] read and an unchecked add; a RingList loop and a user Iterate loop keep their next.
  • Conformance programs in tests/conformance/, each with its .allocations budget: break, continue, a tuple pattern, a counted item (List<String>), an empty list, a write to the original list inside the body (the loop sees the old length). They pass natively and with tools/conformance.sh --vm.
  • Measurement in forge CI: benchmarks/run.sh --allocations once on main and once on the branch, one job each, both tables recorded in the PR body as printed. Done = list-iterate ratio at most 10% above the list-index ratio of the same run and allocations exactly 24 (decision 5); "it did not help" = time above two thirds of main's in the same run or allocations not 24, then the PR reads the generated C (length hoisted? retain left in the loop?) and says so. torb run of list-iterate (VM) must not get slower.
  • Second commit docs(performance): record round P11: before and after tables, section 5's P11 row, finding 5 "what is left", section 5.1 status line.
  • tools/gates.sh a and b green in forge CI; docs changes regenerate skills/ (torb docs skill docs skills).

Design record point

PERFORMANCE.md section 5.1 decisions 1 to 6, finding F5 level 2, round P11.

Seed impact

None expected (decision 6: no syntax, no native, no driver flag, no name the seed looks up). If the pass turns out to need one, stop and record it in the PR; the two-commit rule then applies.

Size

One PR, two Conventional Commits: perf(ir): count a for over a flat list and docs(performance): record round P11. If it grows, cut by cursor as section 5.1 says: first commit with ListIterator alone, TrieListIterator and BufferIterator as a PR of their own.

Dependencies

Step 1 (#415, landed). Shares the for lowering with #468 (collection names declared in std, in progress): whichever lands second rebases onto the other. Rebase order: #468 first (it only changes name resolution and keeps the inline iterator contract), then this step re-reads the cursor's shape; not a blocker for either.

Out of scope

RingList, Array, maps, sets, String.chars(), user Iterate; the bounds check of a[index] (finding 8); #56, #59; the seed.

## Context Step 2 of #58 (perf: P11 - a `for` over a concrete list is a counted loop; PERFORMANCE.md finding F5 level 2). Step 1 (#415) landed the specification as [`docs/PERFORMANCE.md` section 5.1 "What P11 builds"](https://git.torb.dev/torbscript/language/src/branch/main/docs/PERFORMANCE.md#51-what-p11-builds); that record is the specification of this issue and decisions 1 to 6 there are not reopened. ## Goal Build the pass: `for value in numbers` over a cursor of the flat runtime list becomes a length read once, a counter and an unchecked `Element` read, on both back ends, and `benchmarks/list-iterate` is measured before and after in forge CI. ## Acceptance criteria - [ ] New `compiler/src/ir/counted.trb` with one wiring line in `finishLowering` (`ir/lower/lower.trb`), between `rewriteElementPlaces` and `dropProvableChecks` (decision 1). It accepts only a `Record` cursor of std's `ListIterator`, `TrieListIterator` or `BufferIterator` whose `next` is that type's own `next` (found through `standardFunctionsOf`, never by field shape of a user type), and only a loop it recognises whole (decisions 2 and 3). Any other loop keeps its cursor, `Option` and dispatch. - [ ] Step 1's first thing to re-check: #413 has landed since the record was written ("Related work in review"). Re-read the cursor's shape in `std/collections` and `std/core` on `main`, adjust decision 2's match if it changed, and update that paragraph of section 5.1. - [ ] IR snapshot test in `compiler/tests/`: the loop of `list-iterate` has no `next` call, a `less`, a `[... unchecked]` read and an unchecked add; a `RingList` loop and a user `Iterate` loop keep their `next`. - [ ] Conformance programs in `tests/conformance/`, each with its `.allocations` budget: `break`, `continue`, a tuple pattern, a counted item (`List<String>`), an empty list, a write to the original list inside the body (the loop sees the old length). They pass natively and with `tools/conformance.sh --vm`. - [ ] Measurement in forge CI: `benchmarks/run.sh --allocations` once on `main` and once on the branch, one job each, both tables recorded in the PR body as printed. Done = `list-iterate` ratio at most 10% above the `list-index` ratio of the same run and allocations exactly 24 (decision 5); "it did not help" = time above two thirds of `main`'s in the same run or allocations not 24, then the PR reads the generated C (length hoisted? retain left in the loop?) and says so. `torb run` of `list-iterate` (VM) must not get slower. - [ ] Second commit `docs(performance): record round P11`: before and after tables, section 5's P11 row, finding 5 "what is left", section 5.1 status line. - [ ] `tools/gates.sh a` and `b` green in forge CI; docs changes regenerate `skills/` (`torb docs skill docs skills`). ## Design record point PERFORMANCE.md section 5.1 decisions 1 to 6, finding F5 level 2, round P11. ## Seed impact None expected (decision 6: no syntax, no native, no driver flag, no name the seed looks up). If the pass turns out to need one, stop and record it in the PR; the two-commit rule then applies. ## Size One PR, two Conventional Commits: `perf(ir): count a for over a flat list` and `docs(performance): record round P11`. If it grows, cut by cursor as section 5.1 says: first commit with `ListIterator` alone, `TrieListIterator` and `BufferIterator` as a PR of their own. ## Dependencies Step 1 (#415, landed). Shares the `for` lowering with #468 (collection names declared in std, in progress): whichever lands second rebases onto the other. Rebase order: #468 first (it only changes name resolution and keeps the inline iterator contract), then this step re-reads the cursor's shape; not a blocker for either. ## Out of scope `RingList`, `Array`, maps, sets, `String.chars()`, user `Iterate`; the bounds check of `a[index]` (finding 8); #56, #59; the seed.
Author
Collaborator

[loop:torbscript] claim implement by session:loop-torbscript-20261024-1000 until 2026-10-24T14:10:28Z

[loop:torbscript] claim implement by session:loop-torbscript-20261024-1000 until 2026-10-24T14:10:28Z
Author
Collaborator

[loop:torbscript] implement: checkpoint, research only, no code written yet (worker context guard at ~91k tokens, ~0.1M tokens used). next: implement continues in a fresh worker.

State

  • Worktree: /home/node/workspaces/orion/code/wt/torbscript-language-470-counted-for-pass, branch loop/470-counted-for-pass from main @ 57085dce. Nothing committed, nothing pushed, nothing to lose.
  • Read so far: docs/PERFORMANCE.md F5 and section 5.1 (the specification), CLAUDE.md, compiler/CONTRIBUTING.md, ir/lower/lower.trb (finishLowering), ir/bounds.trb, the IR types in ir/ir.trb.

Findings for the next worker

  • Wiring point: finishLowering in compiler/src/ir/lower/lower.trb, between rewriteElementPlaces lowering.program and rewriteMapPlaces/dropProvableChecks. Spec says "between rewriteElementPlaces and dropProvableChecks"; rewriteMapPlaces and dropUncountedMakeUnique sit between them today, so put the call right after rewriteElementPlaces (or after dropUncountedMakeUnique, which clears the cursor's makeUnique; check which one leaves the loop head in the shape decision 3 wants by reading torb ir output in CI).
  • Template for the unchecked read: provenRead in ir/bounds.trb builds Instruction.Read target, Reference(list, [PathStep.Element(item, index, at, checked: false)]); listItemOf gets the item type from IrType.Runtime(RuntimeKind.ListStorage, [item]). Intrinsics: Intrinsic(operation: .Less/.Add, kind: .Signed(width: 64), checked: false); Instruction.Intrinsic(target, intrinsic, [slots], at). Terminators Branch(condition, then, else) / Switch(scrutinee, values, targets, else). Helpers: successorsOf, definedSlot from ir/verify.trb; standardFunctionsOf lowering for finding std functions (listMembersOf in ir/lower/collection.trb is the model for recognising at/length).
  • Loop lowering lives in ir/lower/statement.trb (lowerForIterate, line ~600); read it for the exact head shape (call next, tag, switch, the Read of the Some payload).
  • No torb binary and no build/ in the worktree or on this host; there is no way to try the pass locally, so write it carefully and let forge CI (ci-wait.sh) be the compiler. Reuse test scaffolding from compiler/tests/devirtualize.test.trb, ir-elements.test.trb, lower-harness.trb for the IR snapshot.
  • Skill torbscript is not loadable by name from the subagent; the repo's skills are under skills/torbscript* and .claude/skills/. Read skills/torbscript-language/ before writing any .trb.
  • Remaining acceptance work: ir/counted.trb + wiring; re-check cursor shape after #413 (std/collections, std/core) and update the paragraph in 5.1; IR snapshot test; six conformance programs with .allocations; CI measurement (benchmarks/run.sh --allocations on main and the branch); second commit docs(performance): record round P11 and torb docs skill docs skills.

CONTINUE: torbscript/language#470

[loop:torbscript] implement: checkpoint, research only, no code written yet (worker context guard at ~91k tokens, ~0.1M tokens used). next: implement continues in a fresh worker. **State** - Worktree: `/home/node/workspaces/orion/code/wt/torbscript-language-470-counted-for-pass`, branch `loop/470-counted-for-pass` from `main` @ 57085dce. Nothing committed, nothing pushed, nothing to lose. - Read so far: `docs/PERFORMANCE.md` F5 and section 5.1 (the specification), `CLAUDE.md`, `compiler/CONTRIBUTING.md`, `ir/lower/lower.trb` (`finishLowering`), `ir/bounds.trb`, the IR types in `ir/ir.trb`. **Findings for the next worker** - Wiring point: `finishLowering` in `compiler/src/ir/lower/lower.trb`, between `rewriteElementPlaces lowering.program` and `rewriteMapPlaces`/`dropProvableChecks`. Spec says "between `rewriteElementPlaces` and `dropProvableChecks`"; `rewriteMapPlaces` and `dropUncountedMakeUnique` sit between them today, so put the call right after `rewriteElementPlaces` (or after `dropUncountedMakeUnique`, which clears the cursor's `makeUnique`; check which one leaves the loop head in the shape decision 3 wants by reading `torb ir` output in CI). - Template for the unchecked read: `provenRead` in `ir/bounds.trb` builds `Instruction.Read target, Reference(list, [PathStep.Element(item, index, at, checked: false)])`; `listItemOf` gets the item type from `IrType.Runtime(RuntimeKind.ListStorage, [item])`. Intrinsics: `Intrinsic(operation: .Less/.Add, kind: .Signed(width: 64), checked: false)`; `Instruction.Intrinsic(target, intrinsic, [slots], at)`. Terminators `Branch(condition, then, else)` / `Switch(scrutinee, values, targets, else)`. Helpers: `successorsOf`, `definedSlot` from `ir/verify.trb`; `standardFunctionsOf lowering` for finding std functions (`listMembersOf` in `ir/lower/collection.trb` is the model for recognising `at`/`length`). - Loop lowering lives in `ir/lower/statement.trb` (`lowerForIterate`, line ~600); read it for the exact head shape (`call next`, `tag`, `switch`, the `Read` of the `Some` payload). - No `torb` binary and no `build/` in the worktree or on this host; there is no way to try the pass locally, so write it carefully and let forge CI (`ci-wait.sh`) be the compiler. Reuse test scaffolding from `compiler/tests/devirtualize.test.trb`, `ir-elements.test.trb`, `lower-harness.trb` for the IR snapshot. - `Skill torbscript` is not loadable by name from the subagent; the repo's skills are under `skills/torbscript*` and `.claude/skills/`. Read `skills/torbscript-language/` before writing any `.trb`. - Remaining acceptance work: `ir/counted.trb` + wiring; re-check cursor shape after #413 (`std/collections`, `std/core`) and update the paragraph in 5.1; IR snapshot test; six conformance programs with `.allocations`; CI measurement (`benchmarks/run.sh --allocations` on `main` and the branch); second commit `docs(performance): record round P11` and `torb docs skill docs skills`. CONTINUE: torbscript/language#470
Author
Collaborator

[loop:torbscript] claim implement by session:loop-torbscript-20261025 until 2026-10-25T04:00:00Z

[loop:torbscript] claim implement by session:loop-torbscript-20261025 until 2026-10-25T04:00:00Z
Author
Collaborator

[loop:torbscript] implement: checkpoint, research only, no code written yet (worker context guard at ~81k tokens, ~0.1M tokens used). next: implement continues in a fresh worker.

State

  • The worktree named in the previous checkpoint no longer existed. Recreated with ws wt new torbscript/language 470 counted-for-pass --branch loop/470-counted-for-pass -> /home/node/workspaces/orion/code/wt/torbscript-language-470-counted-for-pass, from origin/main. Nothing committed, nothing pushed, no remote branch yet.

Findings (do not re-read these; go straight to writing compiler/src/ir/counted.trb)

  • Cursor shape on main is unchanged by #413: ListIterator (std/collections/src/list.trb:610), TrieListIterator (:588), BufferIterator (std/core/src/buffer.trb:243) are all { items: <List type>, var index: Int = 0 } with next = get, index = index + 1. Field 0 is items, field 1 is index. Decision 2 of section 5.1 needs no change to its match; only the "re-read" sentence in the issue is satisfied.
  • lowerForIterate (ir/lower/statement.trb:600) emits: check block: call next(&cursor) -> pulled, Tag(tag, pulled), Switch(tag, [some], [inner], done); inner block: Read(value, Reference(pulled, [Variant(layout, some, 0)])), then the body (scope opened, continue jumps to check, break to done). The pass must recognise exactly that, with Call.callee being the direct next instance of one of the three types after devirtualizeProgram.
  • IR building blocks (all in ir/ir.trb): Instruction.Intrinsic(target, Intrinsic(operation: .Less/.Add, kind: .Signed(width: 64), checked: false), [slots], at), Instruction.Read(target, Reference(list, [PathStep.Element(item, index, at, checked: false)])), Terminator.Branch(cond, then, else); template in ir/bounds.trb (provenRead, dropListBoundsChecks). Block.instructions/terminator are mutable; slots are function.slots (SlotDeclaration(type, role, name)), no builder after lowering, so append slots by hand. Item type: IrType.Runtime(RuntimeKind.ListStorage, [item]) -> item.
  • Open design point (the real problem): the pass needs a FunctionId for the length of that list type to hoist the length read. length and get are native fn of ArrayList/TrieList (std list.trb:647/771), so an instance exists only when something calls or tables it. For a numbers: List<Int> subject a witness table of List<Int> holds length (find via program.witnessTables[..].members, as boxedListSlotsOf in ir/lower/lower.trb:321 does for at/set/update); for a concrete ArrayList<Int> subject it may not exist. Recommended: look the instance up like listMembersOf (lower/lower.trb:274: std function named length, one parameter, first parameter Runtime(ListStorage, ...) of the same IR type) and refuse the loop when there is none; then check in CI which of the benchmark's shape gets counted. If the benchmark misses, fall back to making the three next bodies call items.length() (guarantees the instance) or request the instance with instanceFor from the pass.
  • Wiring: finishLowering (ir/lower/lower.trb:173) order today is devirtualizeProgram, rewriteElementPlaces, rewriteMapPlaces, dropUncountedMakeUnique, dropProvableChecks. Put countedLoops lowering.program, <std-flags/length lookup> right after rewriteElementPlaces (spec) unless dropUncountedMakeUnique must come first for the cursor record; the MakeUnique guarding writes to the original list in the body is unaffected (the cursor still holds the list by value).
  • Test scaffolding: compiler/tests/ir-elements.test.trb builds IR by hand (newFunction, addSlot, emit, terminate, finishFunction from ir/build; verifyProgram; printFunction); devirtualize.test.trb and lower-harness.trb lower real source and print IR. The benchmark is benchmarks/list-iterate.trb (single file, not a directory); benchmarks/run.sh --allocations.
  • Not yet read: ir/elements.trb (peephole model for rewriting blocks), ir/verify.trb helpers successorsOf/definedSlot, tests/conformance/ conventions and .allocations format, docs/PERFORMANCE.md section 5 P11 row text.

Next worker, in this order: (1) write ir/counted.trb + wiring line, commit and push as WIP; (2) IR snapshot test; (3) six conformance programs with .allocations; (4) open PR (Closes #470), ci-wait.sh; (5) measurement jobs and the docs commit.

CONTINUE: torbscript/language#470

[loop:torbscript] implement: checkpoint, research only, no code written yet (worker context guard at ~81k tokens, ~0.1M tokens used). next: implement continues in a fresh worker. **State** - The worktree named in the previous checkpoint no longer existed. Recreated with `ws wt new torbscript/language 470 counted-for-pass --branch loop/470-counted-for-pass` -> `/home/node/workspaces/orion/code/wt/torbscript-language-470-counted-for-pass`, from `origin/main`. Nothing committed, nothing pushed, no remote branch yet. **Findings (do not re-read these; go straight to writing `compiler/src/ir/counted.trb`)** - Cursor shape on `main` is unchanged by #413: `ListIterator` (`std/collections/src/list.trb:610`), `TrieListIterator` (`:588`), `BufferIterator` (`std/core/src/buffer.trb:243`) are all `{ items: <List type>, var index: Int = 0 }` with `next` = `get`, `index = index + 1`. Field 0 is `items`, field 1 is `index`. Decision 2 of section 5.1 needs no change to its match; only the "re-read" sentence in the issue is satisfied. - `lowerForIterate` (`ir/lower/statement.trb:600`) emits: `check` block: `call next(&cursor)` -> `pulled`, `Tag(tag, pulled)`, `Switch(tag, [some], [inner], done)`; `inner` block: `Read(value, Reference(pulled, [Variant(layout, some, 0)]))`, then the body (scope opened, `continue` jumps to `check`, `break` to `done`). The pass must recognise exactly that, with `Call.callee` being the direct `next` instance of one of the three types after `devirtualizeProgram`. - IR building blocks (all in `ir/ir.trb`): `Instruction.Intrinsic(target, Intrinsic(operation: .Less/.Add, kind: .Signed(width: 64), checked: false), [slots], at)`, `Instruction.Read(target, Reference(list, [PathStep.Element(item, index, at, checked: false)]))`, `Terminator.Branch(cond, then, else)`; template in `ir/bounds.trb` (`provenRead`, `dropListBoundsChecks`). `Block.instructions`/`terminator` are mutable; slots are `function.slots` (`SlotDeclaration(type, role, name)`), no builder after lowering, so append slots by hand. Item type: `IrType.Runtime(RuntimeKind.ListStorage, [item])` -> `item`. - **Open design point (the real problem):** the pass needs a `FunctionId` for the `length` of that list type to hoist the length read. `length` and `get` are `native fn` of `ArrayList`/`TrieList` (std list.trb:647/771), so an instance exists only when something calls or tables it. For a `numbers: List<Int>` subject a witness table of `List<Int>` holds `length` (find via `program.witnessTables[..].members`, as `boxedListSlotsOf` in `ir/lower/lower.trb:321` does for `at`/`set`/`update`); for a concrete `ArrayList<Int>` subject it may not exist. Recommended: look the instance up like `listMembersOf` (`lower/lower.trb:274`: std function named `length`, one parameter, first parameter `Runtime(ListStorage, ...)` of the same IR type) and **refuse the loop when there is none**; then check in CI which of the benchmark's shape gets counted. If the benchmark misses, fall back to making the three `next` bodies call `items.length()` (guarantees the instance) or request the instance with `instanceFor` from the pass. - Wiring: `finishLowering` (`ir/lower/lower.trb:173`) order today is `devirtualizeProgram`, `rewriteElementPlaces`, `rewriteMapPlaces`, `dropUncountedMakeUnique`, `dropProvableChecks`. Put `countedLoops lowering.program, <std-flags/length lookup>` right after `rewriteElementPlaces` (spec) unless `dropUncountedMakeUnique` must come first for the cursor record; the `MakeUnique` guarding writes to the original list in the body is unaffected (the cursor still holds the list by value). - Test scaffolding: `compiler/tests/ir-elements.test.trb` builds IR by hand (`newFunction`, `addSlot`, `emit`, `terminate`, `finishFunction` from `ir/build`; `verifyProgram`; `printFunction`); `devirtualize.test.trb` and `lower-harness.trb` lower real source and print IR. The benchmark is `benchmarks/list-iterate.trb` (single file, not a directory); `benchmarks/run.sh --allocations`. - Not yet read: `ir/elements.trb` (peephole model for rewriting blocks), `ir/verify.trb` helpers `successorsOf`/`definedSlot`, `tests/conformance/` conventions and `.allocations` format, `docs/PERFORMANCE.md` section 5 P11 row text. **Next worker, in this order:** (1) write `ir/counted.trb` + wiring line, commit and push as WIP; (2) IR snapshot test; (3) six conformance programs with `.allocations`; (4) open PR (`Closes #470`), `ci-wait.sh`; (5) measurement jobs and the docs commit. CONTINUE: torbscript/language#470
Author
Collaborator

[loop:torbscript] claim implement by session:loop-torbscript-20261025 until 2026-10-25T04:15:00Z

[loop:torbscript] claim implement by session:loop-torbscript-20261025 until 2026-10-25T04:15:00Z
Author
Collaborator

[loop:torbscript] implement: checkpoint, first code pushed (context guard at ~95k tokens, ~0.2M tokens used). next: implement continues in a fresh worker.

Done

  • Branch loop/470-counted-for-pass pushed with commit perf(ir): count a for over a flat list: new compiler/src/ir/counted.trb (countedLoops) and the wiring countFlatListLoops lowering right after rewriteElementPlaces in finishLowering (ir/lower/lower.trb). Never compiled: no local torb binary, so forge CI is the compiler.
  • Shape: head block of exactly call next(&cursor), tag, Switch; inner block starts with the Read of Some; Option, tag and cursor mentioned only there (3/2/2 mentions); the cursor's single definition sits in a block that jumps straight to the head. Preheader appends Read items, call length, two constants; head becomes less + Branch; inner gets the unchecked Element read plus the unchecked add. length is found like listMembersOf (instance with the list type as its only parameter) and the loop is refused when none exists.

Next worker

  1. Open the PR now (Closes #470, base main, head loop/470-counted-for-pass); run ci-wait.sh and fix compile errors on the same branch (suspect spots: return inside match arms, program.layoutAt/typeAt use, SlotDeclaration(type:, role:) construction, internStatic import in lower.trb, a length instance missing for ArrayList<Int>: then make the next bodies call items.length() or request the instance).
  2. IR snapshot test, six conformance programs with .allocations, CI measurement (benchmarks/run.sh --allocations on main and the branch), second commit docs(performance): record round P11, torb docs skill docs skills.

CONTINUE: torbscript/language#470

[loop:torbscript] implement: checkpoint, first code pushed (context guard at ~95k tokens, ~0.2M tokens used). next: implement continues in a fresh worker. **Done** - Branch `loop/470-counted-for-pass` pushed with commit `perf(ir): count a for over a flat list`: new `compiler/src/ir/counted.trb` (`countedLoops`) and the wiring `countFlatListLoops lowering` right after `rewriteElementPlaces` in `finishLowering` (`ir/lower/lower.trb`). Never compiled: no local torb binary, so forge CI is the compiler. - Shape: head block of exactly `call next(&cursor)`, `tag`, `Switch`; inner block starts with the `Read` of `Some`; `Option`, tag and cursor mentioned only there (3/2/2 mentions); the cursor's single definition sits in a block that jumps straight to the head. Preheader appends `Read items`, `call length`, two constants; head becomes `less` + `Branch`; inner gets the unchecked `Element` read plus the unchecked add. `length` is found like `listMembersOf` (instance with the list type as its only parameter) and the loop is refused when none exists. **Next worker** 1. Open the PR now (`Closes #470`, base main, head `loop/470-counted-for-pass`); run `ci-wait.sh` and fix compile errors on the same branch (suspect spots: `return` inside `match` arms, `program.layoutAt`/`typeAt` use, `SlotDeclaration(type:, role:)` construction, `internStatic` import in lower.trb, a `length` instance missing for `ArrayList<Int>`: then make the `next` bodies call `items.length()` or request the instance). 2. IR snapshot test, six conformance programs with `.allocations`, CI measurement (`benchmarks/run.sh --allocations` on `main` and the branch), second commit `docs(performance): record round P11`, `torb docs skill docs skills`. CONTINUE: torbscript/language#470
Author
Collaborator

[loop:torbscript] claim implement by session:loop-torbscript-20261025 until 2026-10-25T05:15:00Z (continuation 3 of 3, fresh worker after checkpoint 6204)

[loop:torbscript] claim implement by session:loop-torbscript-20261025 until 2026-10-25T05:15:00Z (continuation 3 of 3, fresh worker after checkpoint 6204)
Author
Collaborator

[loop:torbscript] implement (checkpoint, cont 3): PR #475 is open (Closes #470, branch loop/470-counted-for-pass, head aea3fff6), next: implement (CI fix rounds), not yet a review.

Done this round

  • PR #475 opened.
  • First forge CI run of 3a63bb8c (bootstrap linux-x64, step 1, the seed compiling the compiler) found 5 errors in compiler/src/ir/counted.trb: ?? chain over LocationId?, type used as a parameter name, SlotDeclaration needs (type, role, name) positionally. Fixed in 359f5850 (nested matches for at, addSlot(annotation), SlotDeclaration(annotation, SlotRole.Temporary, "")). Whether the file compiles now is NOT known: the CI run of that sha never started.
  • Added aea3fff6: compiler/tests/counted.test.trb (lowers a probe file against the real std through readSourceTree([root]).withFile(...); asserts: List loop has no next, has less and add.i64.unchecked; RingList and a user Iterate loop keep next) and six conformance programs tests/conformance/counted-for-{break,continue,tuple-pattern,strings,empty,write-to-list} with .expected/.exit written by hand (not from a run) and placeholder .allocations of 1000.

State

  • RED/unknown: nothing is green. Latest sha aea3fff6: all jobs pending (only commit messages / gates / what-the-change-touches queued, bootstrap not started), the runner queue was busy for 30+ minutes (probably #468 PR #474 in parallel).
  • Unverified guesses to check on the first real compile: (1) test file may fail if withFile on an unlisted path is not accepted by check, or intrinsic print names differ from less / add.i64.unchecked; (2) conformance .expected values (14, 16, abbccc/6, x-yy-zzz-, 0, 3/6) are computed by hand; (3) .allocations must be set to the printed counts (sh tools/conformance.sh --filter counted-for --update-budgets equivalent in CI output); (4) open risk from before: the pass refuses a loop when no length instance exists for the list type: if the IR test shows next in probelist, make the next bodies in std call items.length().

Next (in order)

  1. bash /opt/openclaw/skills/dev-workflow/scripts/ci-wait.sh -C code/wt/torbscript-language-470-counted-for-pass on aea3fff6 (push needs -c http.extraHeader="Authorization: Basic <base64 orion:token>"), fix compile/test errors on the branch.
  2. Set the six .allocations to the real counts.
  3. CI measurement: benchmarks/run.sh --allocations on main and on the branch (ratio list-iterate <= 1.10 x list-index, allocations exactly 24), tables into the PR body.
  4. Second commit docs(performance): record round P11 + torb docs skill docs skills regen; squash my fix(ir) commit into the first before review.
  5. #474 (#468) lands first or second: whoever is second rebases.
[loop:torbscript] implement (checkpoint, cont 3): PR #475 is open (Closes #470, branch loop/470-counted-for-pass, head aea3fff6), next: implement (CI fix rounds), not yet a review. Done this round - PR https://git.torb.dev/torbscript/language/pulls/475 opened. - First forge CI run of 3a63bb8c (bootstrap linux-x64, step 1, the seed compiling the compiler) found 5 errors in `compiler/src/ir/counted.trb`: `??` chain over `LocationId?`, `type` used as a parameter name, `SlotDeclaration` needs `(type, role, name)` positionally. Fixed in 359f5850 (nested matches for `at`, `addSlot(annotation)`, `SlotDeclaration(annotation, SlotRole.Temporary, "")`). Whether the file compiles now is NOT known: the CI run of that sha never started. - Added aea3fff6: `compiler/tests/counted.test.trb` (lowers a probe file against the real std through `readSourceTree([root]).withFile(...)`; asserts: List<Int> loop has no `next`, has `less` and `add.i64.unchecked`; RingList and a user `Iterate` loop keep `next`) and six conformance programs `tests/conformance/counted-for-{break,continue,tuple-pattern,strings,empty,write-to-list}` with `.expected`/`.exit` written by hand (not from a run) and placeholder `.allocations` of 1000. State - RED/unknown: nothing is green. Latest sha aea3fff6: all jobs `pending` (only commit messages / gates / what-the-change-touches queued, bootstrap not started), the runner queue was busy for 30+ minutes (probably #468 PR #474 in parallel). - Unverified guesses to check on the first real compile: (1) test file may fail if `withFile` on an unlisted path is not accepted by `check`, or intrinsic print names differ from `less` / `add.i64.unchecked`; (2) conformance `.expected` values (14, 16, `abbccc`/6, `x-yy-zzz-`, 0, 3/6) are computed by hand; (3) `.allocations` must be set to the printed counts (`sh tools/conformance.sh --filter counted-for --update-budgets` equivalent in CI output); (4) open risk from before: the pass refuses a loop when no `length` instance exists for the list type: if the IR test shows `next` in `probelist`, make the `next` bodies in std call `items.length()`. Next (in order) 1. `bash /opt/openclaw/skills/dev-workflow/scripts/ci-wait.sh -C code/wt/torbscript-language-470-counted-for-pass` on aea3fff6 (push needs `-c http.extraHeader="Authorization: Basic <base64 orion:token>"`), fix compile/test errors on the branch. 2. Set the six `.allocations` to the real counts. 3. CI measurement: `benchmarks/run.sh --allocations` on main and on the branch (ratio list-iterate <= 1.10 x list-index, allocations exactly 24), tables into the PR body. 4. Second commit `docs(performance): record round P11` + `torb docs skill docs skills` regen; squash my `fix(ir)` commit into the first before review. 5. #474 (#468) lands first or second: whoever is second rebases.
Author
Collaborator

[loop:torbscript] release implement: continuation cap reached (3 of 3), worker result collected (checkpoint #470 (comment)), next: waiting (CI running on PR #475 head aea3fff6; runners busy with 861b495b).

When CI finishes on aea3fff6:

  • red: CI fix round via pr-followup (fresh worker, checkpoint 6234 as brief). First confirm the 5 seed compile errors in compiler/src/ir/counted.trb (fixed in 359f5850) are gone.
  • green: set the real .allocations counts (currently placeholder 1000) and check the hand-computed .expected/.exit files against the run. Then do the P11 docs commit, the skills/ regeneration, and squash the fix(ir) commit. After that, hand over to review.
  • Open risk: if the IR test shows next in the List<Int> loop, the std next bodies must call items.length().
  • Whichever of #474 and #475 lands second rebases.
[loop:torbscript] release implement: continuation cap reached (3 of 3), worker result collected (checkpoint https://git.torb.dev/torbscript/language/issues/470#issuecomment-6234), next: waiting (CI running on PR https://git.torb.dev/torbscript/language/pulls/475 head aea3fff6; runners busy with 861b495b). When CI finishes on aea3fff6: - red: CI fix round via pr-followup (fresh worker, checkpoint 6234 as brief). First confirm the 5 seed compile errors in `compiler/src/ir/counted.trb` (fixed in 359f5850) are gone. - green: set the real `.allocations` counts (currently placeholder 1000) and check the hand-computed `.expected`/`.exit` files against the run. Then do the P11 docs commit, the `skills/` regeneration, and squash the `fix(ir)` commit. After that, hand over to review. - Open risk: if the IR test shows `next` in the `List<Int>` loop, the std `next` bodies must call `items.length()`. - Whichever of #474 and #475 lands second rebases.
Author
Collaborator

[loop:torbscript] claim implement by session:loop-torbscript-20261025 until 2026-10-25T05:00:00Z (CI fix round: bootstrap linux-x64 red)

[loop:torbscript] claim implement by session:loop-torbscript-20261025 until 2026-10-25T05:00:00Z (CI fix round: bootstrap linux-x64 red)
Author
Collaborator

[loop:torbscript] implement (checkpoint, cont 4): CI fix round done for bootstrap, tier A/B still queued, next: implement (waiting on the CI of 45deda54, then conformance budgets and docs).

Done this round

  • Failing job was bootstrap (linux-x64) on aea3fff6, step 1: compiler/src/ir/counted.trb now compiles (the earlier 5 errors are gone); the 20 errors were all in compiler/tests/counted.test.trb: a method call without parentheses inside if/assert (function.name.contains marker, assert text.contains "x") and assert !expr.
  • Fixed in 45deda54 (fix(ir): parse errors in the counted for pass test): contains(...) with parentheses, assert(...) form (as other tests do), dropped the unused verifyProgram import. Pushed to loop/470-counted-for-pass.
  • Forge CI of 45deda54 (run 1675): bootstrap (linux-x64) is GREEN (so the test file now compiles too), commit messages / touches / runners green. tier A, tier B, gates, release-sync, same-C, playground, pack/archives are still pending in the runner queue (slow).

State

  • Branch loop/470-counted-for-pass at 45deda54, worktree code/wt/torbscript-language-470-counted-for-pass, clean and pushed.
  • Not yet known: whether counted.test.trb assertions and the six counted-for-* conformance programs (hand-written .expected/.exit, placeholder .allocations 1000) pass.

Next (in order)

  1. bash /opt/openclaw/skills/dev-workflow/scripts/ci-wait.sh -C code/wt/torbscript-language-470-counted-for-pass --timeout 3000 on 45deda54 (run in background with output to a file); fix test/conformance failures.
  2. Set the six .allocations to the real counts from the CI output, verify .expected/.exit.
  3. Benchmark ratios (list-iterate <= 1.10 x list-index, allocations 24) from CI into the PR body.
  4. Commit docs(performance): record round P11 (+ torb docs skill docs skills regeneration); squash fix(ir) commits into perf(ir)/test(ir); tools/check-commits.sh; rebase on current main (it moved by ~30 commits; #474 may land first).
[loop:torbscript] implement (checkpoint, cont 4): CI fix round done for bootstrap, tier A/B still queued, next: implement (waiting on the CI of 45deda54, then conformance budgets and docs). Done this round - Failing job was `bootstrap (linux-x64)` on aea3fff6, step 1: `compiler/src/ir/counted.trb` now compiles (the earlier 5 errors are gone); the 20 errors were all in `compiler/tests/counted.test.trb`: a method call without parentheses inside `if`/`assert` (`function.name.contains marker`, `assert text.contains "x"`) and `assert !expr`. - Fixed in 45deda54 (`fix(ir): parse errors in the counted for pass test`): `contains(...)` with parentheses, `assert(...)` form (as other tests do), dropped the unused `verifyProgram` import. Pushed to loop/470-counted-for-pass. - Forge CI of 45deda54 (run 1675): `bootstrap (linux-x64)` is GREEN (so the test file now compiles too), commit messages / touches / runners green. tier A, tier B, gates, release-sync, same-C, playground, pack/archives are still pending in the runner queue (slow). State - Branch loop/470-counted-for-pass at 45deda54, worktree code/wt/torbscript-language-470-counted-for-pass, clean and pushed. - Not yet known: whether `counted.test.trb` assertions and the six `counted-for-*` conformance programs (hand-written `.expected`/`.exit`, placeholder `.allocations` 1000) pass. Next (in order) 1. `bash /opt/openclaw/skills/dev-workflow/scripts/ci-wait.sh -C code/wt/torbscript-language-470-counted-for-pass --timeout 3000` on 45deda54 (run in background with output to a file); fix test/conformance failures. 2. Set the six `.allocations` to the real counts from the CI output, verify `.expected`/`.exit`. 3. Benchmark ratios (list-iterate <= 1.10 x list-index, allocations 24) from CI into the PR body. 4. Commit `docs(performance): record round P11` (+ `torb docs skill docs skills` regeneration); squash `fix(ir)` commits into `perf(ir)`/`test(ir)`; `tools/check-commits.sh`; rebase on current main (it moved by ~30 commits; #474 may land first).
Author
Collaborator

[loop:torbscript] release implement: CI fix round collected (checkpoint #470 (comment)), bootstrap (linux-x64) green on 45deda54; tier A/B, gates, release-sync, same-C, playground, pack/archives still pending. On CI result: red -> fix round (fresh worker from 6257); green -> real .allocations counts, verify .expected/.exit, benchmark ratios into PR #475 body, P11 docs commit + skills/ regen, squash fixups, rebase on main. next: waiting (CI running on PR #475 head 45deda54)

[loop:torbscript] release implement: CI fix round collected (checkpoint https://git.torb.dev/torbscript/language/issues/470#issuecomment-6257), `bootstrap (linux-x64)` green on 45deda54; tier A/B, gates, release-sync, same-C, playground, pack/archives still pending. On CI result: red -> fix round (fresh worker from 6257); green -> real `.allocations` counts, verify `.expected`/`.exit`, benchmark ratios into PR https://git.torb.dev/torbscript/language/pulls/475 body, P11 docs commit + `skills/` regen, squash fixups, rebase on main. next: waiting (CI running on PR https://git.torb.dev/torbscript/language/pulls/475 head 45deda54)
Author
Collaborator

[loop:torbscript] claim implement by session:loop-torbscript-20261025 until 2026-10-25T07:00:00Z (CI fix round: tier B red on 45deda54)

[loop:torbscript] claim implement by session:loop-torbscript-20261025 until 2026-10-25T07:00:00Z (CI fix round: tier B red on 45deda54)
Author
Collaborator

[loop:torbscript] implement (checkpoint, cont 5): tier B red on 45deda54 diagnosed and fixed in a5844d6c, next: implement (waiting on the CI of a5844d6c, then budgets, measurement, docs).

Done this round

  • Tier B (run 1675, 45deda54) failed only on the six counted-for-* conformance programs, in all three lanes (native, VM, VM lowered for the debugger); every other program, the runtime tests, tests/language and the compiler test (counted.test.trb, so the IR assertions) passed. Causes, all in my hand-written files, never run before: name [list literal] is always indexing (needs f([...])) in five programs; print turns none (needs print(turns(none))); and the .expected files held the literal text $14\n from a bad echo, not output with real newlines.
  • Fixed in a5844d6c fix(conformance): syntax and expectation files of the counted for programs (pushed): call forms in parentheses, .expected rewritten with real newlines (14, 16, 0, x-yy-zzz-, abbccc/6, 3/6), counted-for-write-to-list.allocations = 6 (the CI count). The write-to-list program already ran and printed 3,6 correctly, so its output matches the expectation.

State

  • Branch loop/470-counted-for-pass at a5844d6c, worktree code/wt/torbscript-language-470-counted-for-pass, clean and pushed. CI of a5844d6c just started (commit messages, touches, gates pending).
  • The other five .allocations are still the placeholder 1000: CI prints allocated N blocks, under its budget of 1000 by X for them under "allocation budgets that can be lowered" once they build and run. Placeholder passes, it is only a notice.

Next (in order)

  1. Wait for the CI of a5844d6c (ci-wait.sh -C <worktree> --timeout 3300 > file, then read the file; the exec yield does not hold, wait with one while pgrep ... sleep exec or process poll). Red -> read the failed conformance output (job logs via GET /api/v1/repos/torbscript/language/actions/jobs/<job id>/logs, job id from actions/runs/<run>/jobs; plain ci-wait only prints the tail).
  2. Lower the five placeholder .allocations to the printed counts (and any real failure of the five new programs: .expected and .exit were hand-computed).
  3. Measurement: benchmarks/run.sh --allocations runs in no CI job today. Precedent: tools/map-cost.sh (builds the merge-base compiler with the checkout's torb, runs benchmarks/run.sh <programs> with both, a measurement not a gate, exit 0) plus its step in .forgejo/workflows/gates.yml (tier B, after map-cost.sh). Add tools/list-cost.sh the same way for list-iterate list-index with --allocations (touched-files guard on compiler/src/ir/counted.trb, ir/lower/lower.trb, benchmarks/list-iterate.trb) and the same step in gates.yml. Commit as ci(...)/test(...), folded into the PR; the tables come out in the tier B log. Done = list-iterate ratio <= 1.10 x list-index ratio of the same run, allocations 24; VM torb run of list-iterate not slower.
  4. docs(performance): record round P11 (section 5 P11 row, finding 5 "what is left", 5.1 status line, before/after tables as printed), torb docs skill docs skills regeneration (CI-only, no local torb: if skills/ must be regenerated, do it from a CI artifact or ask the tick), squash fix(...) commits into the perf(ir) commit, sh tools/check-commits.sh (the file is not executable: run with sh), rebase on main (was ~30 commits ahead; #474 may land first), PR body tables, then comment next: review.
[loop:torbscript] implement (checkpoint, cont 5): tier B red on 45deda54 diagnosed and fixed in a5844d6c, next: implement (waiting on the CI of a5844d6c, then budgets, measurement, docs). Done this round - Tier B (run 1675, 45deda54) failed only on the six `counted-for-*` conformance programs, in all three lanes (native, VM, VM lowered for the debugger); every other program, the runtime tests, `tests/language` and the compiler test (`counted.test.trb`, so the IR assertions) passed. Causes, all in my hand-written files, never run before: `name [list literal]` is always indexing (needs `f([...])`) in five programs; `print turns none` (needs `print(turns(none))`); and the `.expected` files held the literal text `$14\n` from a bad `echo`, not output with real newlines. - Fixed in a5844d6c `fix(conformance): syntax and expectation files of the counted for programs` (pushed): call forms in parentheses, `.expected` rewritten with real newlines (`14`, `16`, `0`, `x-yy-zzz-`, `abbccc`/`6`, `3`/`6`), `counted-for-write-to-list.allocations` = 6 (the CI count). The write-to-list program already ran and printed `3`,`6` correctly, so its output matches the expectation. State - Branch loop/470-counted-for-pass at a5844d6c, worktree code/wt/torbscript-language-470-counted-for-pass, clean and pushed. CI of a5844d6c just started (commit messages, touches, gates pending). - The other five `.allocations` are still the placeholder 1000: CI prints `allocated N blocks, under its budget of 1000 by X` for them under "allocation budgets that can be lowered" once they build and run. Placeholder passes, it is only a notice. Next (in order) 1. Wait for the CI of a5844d6c (`ci-wait.sh -C <worktree> --timeout 3300 > file`, then read the file; the exec yield does not hold, wait with one `while pgrep ... sleep` exec or `process poll`). Red -> read the failed conformance output (job logs via `GET /api/v1/repos/torbscript/language/actions/jobs/<job id>/logs`, job id from `actions/runs/<run>/jobs`; plain ci-wait only prints the tail). 2. Lower the five placeholder `.allocations` to the printed counts (and any real failure of the five new programs: `.expected` and `.exit` were hand-computed). 3. Measurement: `benchmarks/run.sh --allocations` runs in no CI job today. Precedent: `tools/map-cost.sh` (builds the merge-base compiler with the checkout's `torb`, runs `benchmarks/run.sh <programs>` with both, a measurement not a gate, exit 0) plus its step in `.forgejo/workflows/gates.yml` (tier B, after `map-cost.sh`). Add `tools/list-cost.sh` the same way for `list-iterate list-index` with `--allocations` (touched-files guard on `compiler/src/ir/counted.trb`, `ir/lower/lower.trb`, `benchmarks/list-iterate.trb`) and the same step in gates.yml. Commit as `ci(...)`/`test(...)`, folded into the PR; the tables come out in the tier B log. Done = `list-iterate` ratio <= 1.10 x `list-index` ratio of the same run, allocations 24; VM `torb run` of list-iterate not slower. 4. `docs(performance): record round P11` (section 5 P11 row, finding 5 "what is left", 5.1 status line, before/after tables as printed), `torb docs skill docs skills` regeneration (CI-only, no local torb: if `skills/` must be regenerated, do it from a CI artifact or ask the tick), squash `fix(...)` commits into the `perf(ir)` commit, `sh tools/check-commits.sh` (the file is not executable: run with `sh`), rebase on main (was ~30 commits ahead; #474 may land first), PR body tables, then comment `next: review`.
Author
Collaborator

[loop:torbscript] release implement: tier B fix round collected (checkpoint above), a5844d6c pushed (six counted-for conformance programs corrected), next: waiting (CI running on PR #475 head a5844d6c). On result: red -> fix round from the checkpoint; green -> lower the five .allocations, add the tools/list-cost.sh measurement step, P11 docs commit, squash, rebase, review.

[loop:torbscript] release implement: tier B fix round collected (checkpoint above), a5844d6c pushed (six counted-for conformance programs corrected), next: waiting (CI running on PR https://git.torb.dev/torbscript/language/pulls/475 head a5844d6c). On result: red -> fix round from the checkpoint; green -> lower the five `.allocations`, add the `tools/list-cost.sh` measurement step, P11 docs commit, squash, rebase, review.
Author
Collaborator

[loop:torbscript] claim implement by session:loop-torbscript-20261025 until 2026-10-25T08:00:00Z (CI fix round: tier A and gates red on a5844d6c, run 1680)

[loop:torbscript] claim implement by session:loop-torbscript-20261025 until 2026-10-25T08:00:00Z (CI fix round: tier A and gates red on a5844d6c, run 1680)
Author
Collaborator

[loop:torbscript] release implement (checkpoint, context guard): CI fix round diagnosed, nothing pushed yet, head a5844d6c, next: implement (fix worker continues; PR 475 CI red).

Done this round

  • Logs: run number 1680 = API run id 1726 (GET /actions/runs?head_sha=<sha>), jobs via GET /actions/runs/1726/jobs, log via GET /actions/jobs/<job id>/logs (tier A job 19452 works; the gates job 19449 log is empty in dbfs, but it runs the same gates.sh, so the same two red gates).
  • Tier A has exactly two red lanes:
    1. test compiler/tests (native): 3083 passed, 1 failed - compiler/tests/counted.test.trb:118 "a loop over a List has no call of next ...": the pass did NOT fire on probelist; the final IR still has %4 = call ListIterator_next__Int64(&%2 borrowed), tag, switch, %3 = read %4.Some.value, %2 = call ArrayList_iterate__Int64_$direct(%0 owned last) in b0 (jump b1). The two negative tests (RingList, user Iterate) pass.
    2. format --check: 8 of 1458 files would change: compiler/src/ir/counted.trb, compiler/tests/counted.test.trb, tests/conformance/counted-for-{break,continue,empty,strings,tuple-pattern,write-to-list}.trb. Needs the formatter's exact output (no local torb): derive from the rules in the repo's formatter or from a CI artifact.

Already checked by reading (not the cause): mention counts (pulled 3, tag 2, cursor 2: slotsOfArguments uses place.base once), head shape (2 instructions + Switch), Variant step shape, listItemOf same as bounds.trb.
Not yet checked (candidates for why the pass bails): (a) nexts: function.origin of the instantiated ListIterator.next may be None, or symbolPathOf(...).owners is not exactly ["ListIterator"] (a with Iterator<Item> member could carry the trait); (b) lengths: no length function with parameters[0].type == listType (listType = type of ListIterator.items); (c) fields.length() != 2; (d) item != typeOfSlot(value). Quickest probe: temporarily make each early return None distinguishable is impossible without torb, so read std/collections/src/list.trb ArrayList.length and Lowering.origin setting for generic instances.

State: branch loop/470-counted-for-pass at a5844d6c, worktree code/wt/torbscript-language-470-counted-for-pass clean, nothing uncommitted.

[loop:torbscript] release implement (checkpoint, context guard): CI fix round diagnosed, nothing pushed yet, head a5844d6c, next: implement (fix worker continues; PR 475 CI red). Done this round - Logs: run number 1680 = API run id 1726 (`GET /actions/runs?head_sha=<sha>`), jobs via `GET /actions/runs/1726/jobs`, log via `GET /actions/jobs/<job id>/logs` (tier A job 19452 works; the `gates` job 19449 log is empty in dbfs, but it runs the same gates.sh, so the same two red gates). - Tier A has exactly two red lanes: 1. `test compiler/tests (native)`: 3083 passed, 1 failed - `compiler/tests/counted.test.trb:118` "a loop over a List<Int> has no call of next ...": the pass did NOT fire on `probelist`; the final IR still has `%4 = call ListIterator_next__Int64(&%2 borrowed)`, `tag`, `switch`, `%3 = read %4.Some.value`, `%2 = call ArrayList_iterate__Int64_$direct(%0 owned last)` in b0 (jump b1). The two negative tests (RingList, user Iterate) pass. 2. `format --check`: 8 of 1458 files would change: compiler/src/ir/counted.trb, compiler/tests/counted.test.trb, tests/conformance/counted-for-{break,continue,empty,strings,tuple-pattern,write-to-list}.trb. Needs the formatter's exact output (no local torb): derive from the rules in the repo's formatter or from a CI artifact. Already checked by reading (not the cause): mention counts (pulled 3, tag 2, cursor 2: `slotsOfArguments` uses place.base once), head shape (2 instructions + Switch), Variant step shape, listItemOf same as bounds.trb. Not yet checked (candidates for why the pass bails): (a) `nexts`: `function.origin` of the instantiated `ListIterator.next` may be None, or `symbolPathOf(...).owners` is not exactly `["ListIterator"]` (a `with Iterator<Item>` member could carry the trait); (b) `lengths`: no `length` function with `parameters[0].type == listType` (listType = type of `ListIterator.items`); (c) `fields.length() != 2`; (d) `item != typeOfSlot(value)`. Quickest probe: temporarily make each early `return None` distinguishable is impossible without torb, so read `std/collections/src/list.trb` `ArrayList.length` and `Lowering.origin` setting for generic instances. State: branch loop/470-counted-for-pass at a5844d6c, worktree code/wt/torbscript-language-470-counted-for-pass clean, nothing uncommitted.
Author
Collaborator

[loop:torbscript] checkpoint (context guard, CI fix round 2): root cause of the counted.test.trb:118 failure found and fixed, format still open.

Root cause: the pass runs in finishLowering right after rewriteElementPlaces, but the var fn next of a trait-typed cursor is lowered with makeOwnersUnique (call.trb:688 / place.trb:1107), i.e. a MakeUnique(cursor) sits in front of the next call until dropUncountedMakeUnique removes it, which runs AFTER the pass. At pass time the head was [MakeUnique, Call, Tag] (3 instructions, cursor mentioned 3 times), so head.instructions.length() != 2 bailed. The earlier shape checks were derived from the printed final IR (after dropUncountedMakeUnique + ownership), which hid it. Everything else read OK: devirtualized call shape (referenced(Reference(cursor, [])), no witnesses, owners == ["ListIterator"]).

Done: commit 51524448 pushed to loop/470-counted-for-pass: countFlatListLoops moved to directly after dropUncountedMakeUnique (still before lowerBoxedUpdates/dropListBoundsChecks). Worktree clean.

Left: (1) format --check still wants 8 files (counted.trb, counted.test.trb, six tests/conformance/counted-for-*.trb); no local torb, so the exact formatter output has to come from the formatter rules or a CI log/artifact diff of the next run. (2) Wait for CI on 51524448 (tier A + gates); the test should now pass. Next step: read the format --check output of that run (job log), apply it, push, wait for CI once.

[loop:torbscript] checkpoint (context guard, CI fix round 2): root cause of the counted.test.trb:118 failure found and fixed, format still open. Root cause: the pass runs in `finishLowering` right after `rewriteElementPlaces`, but the `var fn next` of a trait-typed cursor is lowered with `makeOwnersUnique` (call.trb:688 / place.trb:1107), i.e. a `MakeUnique(cursor)` sits in front of the `next` call until `dropUncountedMakeUnique` removes it, which runs AFTER the pass. At pass time the head was `[MakeUnique, Call, Tag]` (3 instructions, cursor mentioned 3 times), so `head.instructions.length() != 2` bailed. The earlier shape checks were derived from the printed final IR (after `dropUncountedMakeUnique` + ownership), which hid it. Everything else read OK: devirtualized call shape (`referenced(Reference(cursor, []))`, no witnesses, `owners == ["ListIterator"]`). Done: commit 51524448 pushed to loop/470-counted-for-pass: `countFlatListLoops` moved to directly after `dropUncountedMakeUnique` (still before `lowerBoxedUpdates`/`dropListBoundsChecks`). Worktree clean. Left: (1) `format --check` still wants 8 files (counted.trb, counted.test.trb, six tests/conformance/counted-for-*.trb); no local torb, so the exact formatter output has to come from the formatter rules or a CI log/artifact diff of the next run. (2) Wait for CI on 51524448 (tier A + gates); the test should now pass. Next step: read the format --check output of that run (job log), apply it, push, wait for CI once.
Author
Collaborator

[loop:torbscript] release implement: CI fix round 2 collected (checkpoint above), 51524448 pushed (countFlatListLoops moved directly after dropUncountedMakeUnique in lower.trb). Open: format --check on 8 files (counted.trb, counted.test.trb, six counted-for-* conformance programs). next: waiting (CI on PR #475 for 51524448), then format fix from the CI log

[loop:torbscript] release implement: CI fix round 2 collected (checkpoint above), 51524448 pushed (countFlatListLoops moved directly after dropUncountedMakeUnique in lower.trb). Open: format --check on 8 files (counted.trb, counted.test.trb, six counted-for-* conformance programs). next: waiting (CI on PR https://git.torb.dev/torbscript/language/pulls/475 for 51524448), then format fix from the CI log
Author
Collaborator

[loop:torbscript] claim implement by session:loop-torbscript-20261025-0258 until 2026-10-25T08:30:00Z (CI fix round 3: tier A red on 51524448, run 1692 job 5)

[loop:torbscript] claim implement by session:loop-torbscript-20261025-0258 until 2026-10-25T08:30:00Z (CI fix round 3: tier A red on 51524448, run 1692 job 5)
Author
Collaborator

[loop:torbscript] release implement: CI fix round 3 pushed 528fedf6, next: waiting (CI on PR #475). The 8 format --check files were hand-aligned with canon rule K2 (calls.trb): lone-argument calls as commands, parentheses where an argument has an operator, no CI-less verification possible. Claim released.

[loop:torbscript] release implement: CI fix round 3 pushed 528fedf6, next: waiting (CI on PR https://git.torb.dev/torbscript/language/pulls/475). The 8 format --check files were hand-aligned with canon rule K2 (calls.trb): lone-argument calls as commands, parentheses where an argument has an operator, no CI-less verification possible. Claim released.
Author
Collaborator

[loop:torbscript] claim implement by session:loop-torbscript-20261025-0258 until 2026-10-25T08:30:00Z (CI fix round 4 on PR 475: commit messages check red on 528fedf6)

[loop:torbscript] claim implement by session:loop-torbscript-20261025-0258 until 2026-10-25T08:30:00Z (CI fix round 4 on PR 475: commit messages check red on 528fedf6)
Author
Collaborator

[loop:torbscript] release implement: CI fix round 4 reworded 528fedf6 (type style is not a Conventional Commit type here) to chore(ir), now 60ebb4a4, tree unchanged, no review existed on the old head, next: waiting (CI on PR 475)

[loop:torbscript] release implement: CI fix round 4 reworded 528fedf6 (type `style` is not a Conventional Commit type here) to `chore(ir)`, now 60ebb4a4, tree unchanged, no review existed on the old head, next: waiting (CI on PR 475)
Author
Collaborator

[loop:torbscript] claim implement by session:loop-torbscript-20261025-0258 until 2026-10-25T09:30:00Z (CI fix: tier A + gates red on PR 475 head 60ebb4a4, web run 1709)

[loop:torbscript] claim implement by session:loop-torbscript-20261025-0258 until 2026-10-25T09:30:00Z (CI fix: tier A + gates red on PR 475 head 60ebb4a4, web run 1709)
Author
Collaborator

[loop:torbscript] release implement: CI fix round a221d308, next: waiting (CI + review on PR 475)

[loop:torbscript] release implement: CI fix round a221d308, next: waiting (CI + review on PR 475)
Author
Collaborator

[loop:torbscript] claim implement by session:loop-torbscript-20261025-0542 until 2026-10-25T13:15:00Z (address Argus changes-requested review on PR 475: measurement, docs commit, PR body)

[loop:torbscript] claim implement by session:loop-torbscript-20261025-0542 until 2026-10-25T13:15:00Z (address Argus changes-requested review on PR 475: measurement, docs commit, PR body)
Author
Collaborator

[loop:torbscript] implement (checkpoint, context guard, review round on PR 475): nothing pushed this round, head cf227088, next: implement (fix worker continues from here).

Found (no work redone)

  • Blocker 4 history is already done: cf227088 is ONE commit perf(ir): count a for over a flat list directly on top of main 44a35916; CI commit messages is green on it. Only the docs commit is still missing.
  • CI run 1780 (index 1734, cf227088) started 06:22 UTC and was still pending at 07:14 UTC (bootstrap, tier A/B, gates, cross, same-C, pack). A background ci-wait.sh writes /tmp/ci-wait-470.out (may be gone with the session; re-run ci-wait.sh -C <worktree> --timeout 3500).
  • Blocker 1 (measurement) needs no new tool: #413 landed tools/collection-cost.sh (2b2b4856), run at the end of tier B in .forgejo/workflows/gates.yml. It builds the merge-base compiler and this checkout's, runs benchmarks/run.sh list-index list-iterate ... with both in ONE job (same machine, fastest of 9) and prints <program> allocated N blocks; its touched-files guard includes compiler/src/ir, so it fires on this PR. Read the tier B job log of run 1780 (GET /repos/torbscript/language/actions/runs/1780/jobs, then /actions/jobs/<id>/logs; if empty in the API, the web run page) and paste the --- base and --- head tables as printed. Caveat: it calls run.sh without --allocations; if Argus insists on that column, add --allocations to the run.sh $programs call in collection-cost.sh (fold into the perf commit, or state the leak-report count is the same number). Also in that log: the notice "allocation budgets that can be lowered" gives the printed counts for the five counted-for-*.allocations placeholders (1000) and write-to-list (6): set each to the printed value (nit 2).
  • Verdict per criterion 5: done = list-iterate ratio <= 1.10 x list-index ratio of the same table AND allocations exactly 24; "did not help" = list-iterate time > 2/3 of base in the same run or allocations != 24 (then read the generated C and say so). VM: torb run of list-iterate not slower: collection-cost.sh does not measure the VM; either add a VM timing line to the script or say plainly it was not measured (criterion 5 asks for it).
  • Blocker 3: #413 is on main now (a62088e6 design record, 0f63b4ac Buffer is a List, 9cc7666c). Cursors to re-compare against 5.1 decision 2: std/collections/src/list.trb:588 TrieListIterator, :610 ListIterator, std/core/src/buffer.trb:249 BufferIterator (and ListStorage instantiation in compiler/src/ir/instantiate.trb). The 5.1 paragraph "Related work in review" is now stale ("is not on main"): rewrite it to say #413 landed and whether the cursor shape is unchanged, with the compared lines.
  • Blocker 2 docs commit docs(performance): record round P11 edits in docs/PERFORMANCE.md only: header status (lines 3-5, "P11 is not built"), section 2 row for value in collection as a whole (~163), finding 5 "What is left" (~505), section 5 P11 row (line 1383, "specified, not built" -> done with numbers), 5.1 intro (~1414 "nothing in it is built yet"), "Related work in review", plus the before/after tables. skills/: no skill carries docs/PERFORMANCE.md (no P11 text in skills/, no carry directive in docs/skills), so torb docs skill docs skills should be a no-op; the tier A gate docs skill --check confirms. State that in the PR body.
  • Nit 1: drop assert text.contains("unchecked") in compiler/tests/counted.test.trb (implied by the add.i64.unchecked line above it).
  • Blocker 5: rewrite the PR body (Closes #470; what the pass takes/refuses; tests; both tables; verdict; cursor re-check; seed none; no "Work in progress").

State: worktree code/wt/torbscript-language-470-counted-for-pass clean at cf227088 = origin/loop/470-counted-for-pass. Push with force-with-lease only if the perf commit is amended (nit fixes go into it); the docs commit is a second commit. Then re-request Argus on PR 475 and release the claim (release implement).

[loop:torbscript] implement (checkpoint, context guard, review round on PR 475): nothing pushed this round, head cf227088, next: implement (fix worker continues from here). Found (no work redone) - Blocker 4 history is already done: cf227088 is ONE commit `perf(ir): count a for over a flat list` directly on top of main 44a35916; CI `commit messages` is green on it. Only the docs commit is still missing. - CI run 1780 (index 1734, cf227088) started 06:22 UTC and was still pending at 07:14 UTC (bootstrap, tier A/B, gates, cross, same-C, pack). A background `ci-wait.sh` writes /tmp/ci-wait-470.out (may be gone with the session; re-run `ci-wait.sh -C <worktree> --timeout 3500`). - Blocker 1 (measurement) needs no new tool: #413 landed `tools/collection-cost.sh` (2b2b4856), run at the end of tier B in `.forgejo/workflows/gates.yml`. It builds the merge-base compiler and this checkout's, runs `benchmarks/run.sh list-index list-iterate ...` with both in ONE job (same machine, fastest of 9) and prints `<program> allocated N blocks`; its touched-files guard includes `compiler/src/ir`, so it fires on this PR. Read the tier B job log of run 1780 (`GET /repos/torbscript/language/actions/runs/1780/jobs`, then `/actions/jobs/<id>/logs`; if empty in the API, the web run page) and paste the `--- base` and `--- head` tables as printed. Caveat: it calls `run.sh` without `--allocations`; if Argus insists on that column, add `--allocations` to the `run.sh $programs` call in collection-cost.sh (fold into the perf commit, or state the leak-report count is the same number). Also in that log: the notice "allocation budgets that can be lowered" gives the printed counts for the five `counted-for-*.allocations` placeholders (1000) and `write-to-list` (6): set each to the printed value (nit 2). - Verdict per criterion 5: done = `list-iterate` ratio <= 1.10 x `list-index` ratio of the same table AND allocations exactly 24; "did not help" = list-iterate time > 2/3 of base in the same run or allocations != 24 (then read the generated C and say so). VM: `torb run` of list-iterate not slower: collection-cost.sh does not measure the VM; either add a VM timing line to the script or say plainly it was not measured (criterion 5 asks for it). - Blocker 3: #413 is on main now (a62088e6 design record, 0f63b4ac `Buffer is a List`, 9cc7666c). Cursors to re-compare against 5.1 decision 2: `std/collections/src/list.trb:588` TrieListIterator, `:610` ListIterator, `std/core/src/buffer.trb:249` BufferIterator (and `ListStorage` instantiation in `compiler/src/ir/instantiate.trb`). The 5.1 paragraph "Related work in review" is now stale ("is not on main"): rewrite it to say #413 landed and whether the cursor shape is unchanged, with the compared lines. - Blocker 2 docs commit `docs(performance): record round P11` edits in docs/PERFORMANCE.md only: header status (lines 3-5, "P11 is not built"), section 2 row `for value in collection as a whole` (~163), finding 5 "What is left" (~505), section 5 P11 row (line 1383, "specified, not built" -> done with numbers), 5.1 intro (~1414 "nothing in it is built yet"), "Related work in review", plus the before/after tables. `skills/`: no skill carries docs/PERFORMANCE.md (no P11 text in skills/, no carry directive in docs/skills), so `torb docs skill docs skills` should be a no-op; the tier A gate `docs skill --check` confirms. State that in the PR body. - Nit 1: drop `assert text.contains("unchecked")` in compiler/tests/counted.test.trb (implied by the `add.i64.unchecked` line above it). - Blocker 5: rewrite the PR body (Closes #470; what the pass takes/refuses; tests; both tables; verdict; cursor re-check; seed none; no "Work in progress"). State: worktree code/wt/torbscript-language-470-counted-for-pass clean at cf227088 = origin/loop/470-counted-for-pass. Push with force-with-lease only if the perf commit is amended (nit fixes go into it); the docs commit is a second commit. Then re-request Argus on PR 475 and release the claim (`release implement`).
Author
Collaborator

[loop:torbscript] implement (checkpoint, context guard, review round 2 on PR 475): pushed 51d38d87 (perf commit amended, force-with-lease), CI of 51d38d87 running, next: implement (fix worker continues from here).

Done

  • Run 1780 (cf227088) tier A and tier B green. Read its tier B log (collection-cost.sh, base 44a35916 vs head), list-iterate 385894 us 10.85x -> 53851 us 1.56x; list-index 6.21x -> 6.76x (same table, same job); other rows within noise (array-write 6.77x/7.55x, buffer-write 6.77x/6.48x, list-slice 2.08x/2.28x, list-write-shared 3.95x/4.17x, list-literal 17.48x/16.22x). Done-criterion time part holds (1.56x <= 1.10 x 6.76x).
  • Allocation criterion NOT proven by that run: collection-cost.sh printed "allocated N blocks" = blocks still live at exit (TORB_REPORT_LEAKS): 22 for list-index and list-iterate on base and head alike. Criterion 5 wants the run.sh --allocations count ("exactly 24"). So 51d38d87 amends tools/collection-cost.sh: run.sh --allocations $programs (the allocations (ld --wrap ...) block), the live-at-exit line renamed "has N blocks live at exit", and a VM line per label (VM, fastest of 3: list-index X ms, list-iterate Y ms, COLLECTION_VM_RUNS) for the "VM not slower" check. sh -n ok, not run (no compiler on the gateway).
  • The plan's "allocation budgets that can be lowered" notice does not exist in CI logs: tools/gates.sh deletes a gate's output on success (gate() rm -f report.out), so notes are invisible. Counts for the six counted-for-*.allocations need either a failing probe (budget 0 prints "allocated N blocks, over its budget of 0") on a throwaway branch/PR, or a local run (no gcc here). Not done yet; nit, non-blocking.
  • Nit 1: assert text.contains(" unchecked]") (compiler/tests/counted.test.trb:121) is the read annotation; add.i64.unchecked does not contain it, so it is not implied and criterion 3 asks for it. Plan: keep it and say so in the PR body.

Next (in order)

  1. Wait for CI of 51d38d87 (ci-wait.sh -C code/wt/torbscript-language-470-counted-for-pass --timeout 3500), read tier B job log (GET /actions/runs/<id>/jobs, /actions/jobs/<id>/logs): --- base/--- head tables with the allocations block and the VM line.
  2. Verdict: done = list-iterate ratio <= 1.10 x list-index ratio and run.sh --allocations torb count 24 for list-iterate (equal to list-index); otherwise "did not help", read the generated C.
  3. Budgets: optional probe branch with the six tests/conformance/counted-for-*.allocations = 0, read counts from the failed tier B output, then set them in the perf commit (amend, force-with-lease, own branch only), or leave 1000 and say so.
  4. Second commit docs(performance): record round P11 (docs/PERFORMANCE.md only: header status lines 3-5, section 2 row ~163, finding 5 ~505, section 5 P11 row ~1383, 5.1 intro ~1414 and "Related work in review" now: #413 landed, compare std/collections/src/list.trb:588 TrieListIterator, :610 ListIterator, std/core/src/buffer.trb:249 BufferIterator against decision 2), before/after tables.
  5. Rewrite PR 475 body (Closes #470, pass takes/refuses, tests, both tables as printed, verdict, VM, cursor re-check, seed none, skills/ untouched, no "Work in progress"), re-request Argus, post release implement.

State: worktree code/wt/torbscript-language-470-counted-for-pass clean at 51d38d87 = origin/loop/470-counted-for-pass. Claim of session:loop-torbscript-20261025-0542 still held (until 13:15 UTC).

[loop:torbscript] implement (checkpoint, context guard, review round 2 on PR 475): pushed 51d38d87 (perf commit amended, force-with-lease), CI of 51d38d87 running, next: implement (fix worker continues from here). Done - Run 1780 (cf227088) tier A and tier B green. Read its tier B log (collection-cost.sh, base 44a35916 vs head), `list-iterate` 385894 us 10.85x -> 53851 us 1.56x; `list-index` 6.21x -> 6.76x (same table, same job); other rows within noise (`array-write` 6.77x/7.55x, `buffer-write` 6.77x/6.48x, `list-slice` 2.08x/2.28x, `list-write-shared` 3.95x/4.17x, `list-literal` 17.48x/16.22x). Done-criterion time part holds (1.56x <= 1.10 x 6.76x). - Allocation criterion NOT proven by that run: collection-cost.sh printed "allocated N blocks" = blocks still live at exit (TORB_REPORT_LEAKS): 22 for list-index and list-iterate on base and head alike. Criterion 5 wants the `run.sh --allocations` count ("exactly 24"). So 51d38d87 amends tools/collection-cost.sh: `run.sh --allocations $programs` (the `allocations (ld --wrap ...)` block), the live-at-exit line renamed "has N blocks live at exit", and a VM line per label (`VM, fastest of 3: list-index X ms, list-iterate Y ms`, `COLLECTION_VM_RUNS`) for the "VM not slower" check. `sh -n` ok, not run (no compiler on the gateway). - The plan's "allocation budgets that can be lowered" notice does not exist in CI logs: tools/gates.sh deletes a gate's output on success (`gate()` rm -f report.out), so notes are invisible. Counts for the six counted-for-*.allocations need either a failing probe (budget 0 prints "allocated N blocks, over its budget of 0") on a throwaway branch/PR, or a local run (no gcc here). Not done yet; nit, non-blocking. - Nit 1: `assert text.contains(" unchecked]")` (compiler/tests/counted.test.trb:121) is the read annotation; `add.i64.unchecked` does not contain it, so it is not implied and criterion 3 asks for it. Plan: keep it and say so in the PR body. Next (in order) 1. Wait for CI of 51d38d87 (`ci-wait.sh -C code/wt/torbscript-language-470-counted-for-pass --timeout 3500`), read tier B job log (`GET /actions/runs/<id>/jobs`, `/actions/jobs/<id>/logs`): `--- base`/`--- head` tables with the allocations block and the VM line. 2. Verdict: done = list-iterate ratio <= 1.10 x list-index ratio and `run.sh --allocations` torb count 24 for list-iterate (equal to list-index); otherwise "did not help", read the generated C. 3. Budgets: optional probe branch with the six `tests/conformance/counted-for-*.allocations` = 0, read counts from the failed tier B output, then set them in the perf commit (amend, force-with-lease, own branch only), or leave 1000 and say so. 4. Second commit `docs(performance): record round P11` (docs/PERFORMANCE.md only: header status lines 3-5, section 2 row ~163, finding 5 ~505, section 5 P11 row ~1383, 5.1 intro ~1414 and "Related work in review" now: #413 landed, compare std/collections/src/list.trb:588 TrieListIterator, :610 ListIterator, std/core/src/buffer.trb:249 BufferIterator against decision 2), before/after tables. 5. Rewrite PR 475 body (Closes #470, pass takes/refuses, tests, both tables as printed, verdict, VM, cursor re-check, seed none, skills/ untouched, no "Work in progress"), re-request Argus, post `release implement`. State: worktree code/wt/torbscript-language-470-counted-for-pass clean at 51d38d87 = origin/loop/470-counted-for-pass. Claim of session:loop-torbscript-20261025-0542 still held (until 13:15 UTC).
Author
Collaborator

[loop:torbscript] claim implement by session:loop-torbscript-20261025-0542 until 2026-10-25T15:40:00Z (respawn: previous worker ended with nothing new; continue from the 08:27 checkpoint, CI of 51d38d87 still running)

[loop:torbscript] claim implement by session:loop-torbscript-20261025-0542 until 2026-10-25T15:40:00Z (respawn: previous worker ended with nothing new; continue from the 08:27 checkpoint, CI of 51d38d87 still running)
Author
Collaborator

[loop:torbscript] release implement: PR 475 now has the docs commit 19e27a62 docs(performance): record round P11 on top of the perf commit 51d38d87, PR body rewritten, Argus review re-requested (worker ended without a release comment), CI of 19e27a62 running, next: waiting (CI + review on PR 475)

[loop:torbscript] release implement: PR 475 now has the docs commit 19e27a62 `docs(performance): record round P11` on top of the perf commit 51d38d87, PR body rewritten, Argus review re-requested (worker ended without a release comment), CI of 19e27a62 running, next: waiting (CI + review on PR 475)
argus closed this issue 2026-10-25 12:16:47 +00:00
Sign in to join this conversation.
No labels
loop
ready
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
torbscript/language#470
No description provided.