perf(vm): a cheaper attached run of the debugger #298

Merged
argus merged 2 commits from loop/43-cheaper-attached-run into main 2026-10-10 09:59:49 +00:00
Collaborator

Closes #43

Two commits:

  1. test(debugger): measure the attached run in CI - tools/debug-cost.sh at the end of tier B: torb run --vm, the run lowered for the debugger, and torb debug attached without and with a breakpoint, best / median of 7 interleaved runs of benchmarks/debug/*.
  2. perf(vm): write a loop's record only where a call back begins, test breakpoints by a bit:
    • The innermost execute of a debugged run notes chunk, pc, base, depth and statement in plain fields of Debugger. A LoopRecord is pushed only in beginLoop when an execute is already running, which is where a call back begins, and popped in finishLoop. framesOf reads the live position, then the records below it.
    • Breakpoints are one flat List<Bool> with a bit for every statement of every chunk, indexed by chunk start plus statement index. Debugger.setBreakpoints makes the table anew, so a replaced set leaves no stale bit. A run with no breakpoints skips the table entirely. codeStartsOf is gone.
    • The debuggee binds a line exactly as before (verified, line, moved to the next line that has a statement), now to (chunk, statement index) pairs.

Numbers (forge linux-x64, tier B)

Before: run 732, tier B. After: run 754, tier B, commit 8eb9b956, whose VM code is the same as the head (since then: docs, formatting and a rebase onto main).

Program run --vm lowered attached breakpoint set
fibonacci before 860 / 870 1070 / 1088 2017 / 2049 2024 / 2044
fibonacci after 825 / 868 1045 / 1061 1179 / 1224 1189 / 1196
loops before 1106 / 1131 1214 / 1242 1788 / 1812 1762 / 1798
loops after 1085 / 1240 1249 / 1341 1320 / 1362 1316 / 1370
hello before 238 / 278 259 / 273 256 / 287 261 / 278
hello after 267 / 288 257 / 285 274 / 285 259 / 287

Decision rule: the change stays if the attached fibonacci run is at least 10% faster. Its best went from 2017 to 1179 ms, 42% faster (2.34x to 1.43x of run --vm), so the change stays. torb run/torb test code is unchanged, and the numbers are within noise. This is reported, not a gate. The numbers are recorded in DEBUGGER.md section 19, and sections 7, 8 and 20 are updated.

Tests

  • tests/debug/callback (new session): a breakpoint in Key.hash, which a Set calls back through the runtime, and its stackTrace through the call back down to the caller's line. It also covers stepOut to the caller's next line, stepIn from there into the next call back, and a panic inside a call back, stopped with its frames.
  • compiler/tests/debugger.test.trb:
    • Breakpoints at the first and last statement of a function and in two chunks at once.
    • The bit table holds exactly the first and last statement of every chunk.
    • A replaced set leaves no bit.
    • Breakpoints replaced at a stop apply for the rest of the run.
    • A line shared by a closure and its caller stops in both.
    • A breakpoint, a step out, a pause and stopOnEntry with a destructor's call back show the frames of every loop below.
  • The existing sessions and .expected files are unchanged.
Closes #43 Two commits: 1. `test(debugger): measure the attached run in CI` - `tools/debug-cost.sh` at the end of tier B: `torb run --vm`, the run lowered for the debugger, and `torb debug` attached without and with a breakpoint, best / median of 7 interleaved runs of `benchmarks/debug/*`. 2. `perf(vm): write a loop's record only where a call back begins, test breakpoints by a bit`: - The innermost `execute` of a debugged run notes chunk, pc, base, depth and statement in plain fields of `Debugger`. A `LoopRecord` is pushed only in `beginLoop` when an `execute` is already running, which is where a call back begins, and popped in `finishLoop`. `framesOf` reads the live position, then the records below it. - Breakpoints are one flat `List<Bool>` with a bit for every statement of every chunk, indexed by chunk start plus statement index. `Debugger.setBreakpoints` makes the table anew, so a replaced set leaves no stale bit. A run with no breakpoints skips the table entirely. `codeStartsOf` is gone. - The debuggee binds a line exactly as before (`verified`, `line`, moved to the next line that has a statement), now to `(chunk, statement index)` pairs. ## Numbers (forge `linux-x64`, tier B) Before: [run 732, tier B](https://git.torb.dev/torbscript/language/actions/runs/732/jobs/6). After: [run 754, tier B](https://git.torb.dev/torbscript/language/actions/runs/754/jobs/6), commit 8eb9b956, whose VM code is the same as the head (since then: docs, formatting and a rebase onto main). | Program | run --vm | lowered | attached | breakpoint set | |---|---|---|---|---| | fibonacci before | 860 / 870 | 1070 / 1088 | **2017** / 2049 | 2024 / 2044 | | fibonacci after | 825 / 868 | 1045 / 1061 | **1179** / 1224 | 1189 / 1196 | | loops before | 1106 / 1131 | 1214 / 1242 | 1788 / 1812 | 1762 / 1798 | | loops after | 1085 / 1240 | 1249 / 1341 | 1320 / 1362 | 1316 / 1370 | | hello before | 238 / 278 | 259 / 273 | 256 / 287 | 261 / 278 | | hello after | 267 / 288 | 257 / 285 | 274 / 285 | 259 / 287 | **Decision rule:** the change stays if the attached fibonacci run is at least 10% faster. Its best went from 2017 to 1179 ms, 42% faster (2.34x to 1.43x of `run --vm`), so the change stays. `torb run`/`torb test` code is unchanged, and the numbers are within noise. This is reported, not a gate. The numbers are recorded in DEBUGGER.md section 19, and sections 7, 8 and 20 are updated. ## Tests - `tests/debug/callback` (new session): a breakpoint in `Key.hash`, which a `Set` calls back through the runtime, and its `stackTrace` through the call back down to the caller's line. It also covers `stepOut` to the caller's next line, `stepIn` from there into the next call back, and a panic inside a call back, stopped with its frames. - `compiler/tests/debugger.test.trb`: - Breakpoints at the first and last statement of a function and in two chunks at once. - The bit table holds exactly the first and last statement of every chunk. - A replaced set leaves no bit. - Breakpoints replaced at a stop apply for the rest of the run. - A line shared by a closure and its caller stops in both. - A breakpoint, a step out, a pause and stopOnEntry with a destructor's call back show the frames of every loop below. - The existing sessions and `.expected` files are unchanged.
test(debugger): measure the attached run in CI
Some checks failed
pdisc / pdisc (pull_request) Successful in 9s
ci / commit messages (pull_request) Successful in 5s
ci / what the change touches (pull_request) Successful in 7s
ci / which runners exist (pull_request) Successful in 1s
ci / bootstrap and conformance (windows-x64) (pull_request) Has been skipped
ci / bootstrap and conformance (linux-arm64) (pull_request) Has been skipped
ci / bootstrap and conformance (macos-arm64) (pull_request) Has been skipped
ci / the VS Code extension (editors/vscode) (pull_request) Successful in 12s
ci / bootstrap (linux-x64) (pull_request) Successful in 6m47s
ci / cross-compiled on linux-x64 (windows-x64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (linux-arm64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (macos-arm64) (pull_request) Has been skipped
ci / tier A (linux-x64) (pull_request) Failing after 5h28m15s
ci / tools/release-sync and tools/registry (linux-x64) (pull_request) Successful in 1m29s
ci / every host emits the same C (pull_request) Successful in 4m10s
ci / the playground (torb as WebAssembly) (pull_request) Successful in 1m42s
ci / tier B (linux-x64) (pull_request) Successful in 11m0s
ci / pack the seed (pull_request) Has been skipped
ci / archives (pull_request) Has been skipped
ci / gates (pull_request) Failing after 0s
124b9f7778
Author
Collaborator

Stays draft: the VM change (second commit) is not pushed yet; owner is the torbscript loop worker (re-claimed on #43 at 2026-10-10 08:02Z). Head also needs a rebase onto main (not mergeable) and tier A on the baseline commit failed after 5h28m, to be checked with the next push.

Stays draft: the VM change (second commit) is not pushed yet; owner is the torbscript loop worker (re-claimed on #43 at 2026-10-10 08:02Z). Head also needs a rebase onto main (not mergeable) and tier A on the baseline commit failed after 5h28m, to be checked with the next push.
orion force-pushed loop/43-cheaper-attached-run from 124b9f7778
Some checks failed
ci / commit messages (pull_request) Successful in 5s
ci / what the change touches (pull_request) Successful in 7s
pdisc / pdisc (pull_request) Successful in 9s
ci / which runners exist (pull_request) Successful in 1s
ci / bootstrap and conformance (windows-x64) (pull_request) Has been skipped
ci / bootstrap and conformance (linux-arm64) (pull_request) Has been skipped
ci / bootstrap and conformance (macos-arm64) (pull_request) Has been skipped
ci / the VS Code extension (editors/vscode) (pull_request) Successful in 12s
ci / bootstrap (linux-x64) (pull_request) Successful in 6m47s
ci / cross-compiled on linux-x64 (windows-x64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (linux-arm64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (macos-arm64) (pull_request) Has been skipped
ci / tier A (linux-x64) (pull_request) Failing after 5h28m15s
ci / tools/release-sync and tools/registry (linux-x64) (pull_request) Successful in 1m29s
ci / every host emits the same C (pull_request) Successful in 4m10s
ci / the playground (torb as WebAssembly) (pull_request) Successful in 1m42s
ci / tier B (linux-x64) (pull_request) Successful in 11m0s
ci / pack the seed (pull_request) Has been skipped
ci / archives (pull_request) Has been skipped
ci / gates (pull_request) Failing after 0s
to 8eb9b956a2
Some checks failed
pdisc / pdisc (pull_request) Successful in 9s
ci / commit messages (pull_request) Successful in 4s
ci / what the change touches (pull_request) Successful in 4s
ci / which runners exist (pull_request) Successful in 0s
ci / bootstrap and conformance (windows-x64) (pull_request) Has been skipped
ci / bootstrap and conformance (linux-arm64) (pull_request) Has been skipped
ci / bootstrap and conformance (macos-arm64) (pull_request) Has been skipped
ci / bootstrap (linux-x64) (pull_request) Successful in 3m4s
ci / cross-compiled on linux-x64 (windows-x64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (linux-arm64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (macos-arm64) (pull_request) Has been skipped
ci / the VS Code extension (editors/vscode) (pull_request) Successful in 12s
ci / tier A (linux-x64) (pull_request) Failing after 3m48s
ci / tools/release-sync and tools/registry (linux-x64) (pull_request) Successful in 1m25s
ci / every host emits the same C (pull_request) Successful in 4m6s
ci / the playground (torb as WebAssembly) (pull_request) Successful in 1m41s
ci / tier B (linux-x64) (pull_request) Successful in 12m19s
ci / pack the seed (pull_request) Has been skipped
ci / archives (pull_request) Has been skipped
ci / gates (pull_request) Failing after 0s
2026-10-10 08:18:31 +00:00
Compare
orion force-pushed loop/43-cheaper-attached-run from 8eb9b956a2
Some checks failed
pdisc / pdisc (pull_request) Successful in 9s
ci / commit messages (pull_request) Successful in 4s
ci / what the change touches (pull_request) Successful in 4s
ci / which runners exist (pull_request) Successful in 0s
ci / bootstrap and conformance (windows-x64) (pull_request) Has been skipped
ci / bootstrap and conformance (linux-arm64) (pull_request) Has been skipped
ci / bootstrap and conformance (macos-arm64) (pull_request) Has been skipped
ci / bootstrap (linux-x64) (pull_request) Successful in 3m4s
ci / cross-compiled on linux-x64 (windows-x64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (linux-arm64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (macos-arm64) (pull_request) Has been skipped
ci / the VS Code extension (editors/vscode) (pull_request) Successful in 12s
ci / tier A (linux-x64) (pull_request) Failing after 3m48s
ci / tools/release-sync and tools/registry (linux-x64) (pull_request) Successful in 1m25s
ci / every host emits the same C (pull_request) Successful in 4m6s
ci / the playground (torb as WebAssembly) (pull_request) Successful in 1m41s
ci / tier B (linux-x64) (pull_request) Successful in 12m19s
ci / pack the seed (pull_request) Has been skipped
ci / archives (pull_request) Has been skipped
ci / gates (pull_request) Failing after 0s
to becffc08ac
Some checks failed
ci / commit messages (pull_request) Has been cancelled
ci / what the change touches (pull_request) Has been cancelled
ci / gates (pull_request) Has been cancelled
pdisc / pdisc (pull_request) Has been cancelled
2026-10-10 09:15:07 +00:00
Compare
orion changed title from WIP: perf(vm): a cheaper attached run of the debugger to perf(vm): a cheaper attached run of the debugger 2026-10-10 09:15:26 +00:00
orion force-pushed loop/43-cheaper-attached-run from becffc08ac
Some checks failed
ci / commit messages (pull_request) Has been cancelled
ci / what the change touches (pull_request) Has been cancelled
ci / gates (pull_request) Has been cancelled
pdisc / pdisc (pull_request) Has been cancelled
to 6ffd6cc5da
Some checks failed
ci / every host emits the same C (pull_request) Successful in 4m20s
ci / the playground (torb as WebAssembly) (pull_request) Successful in 1m54s
ci / tier B (linux-x64) (pull_request) Successful in 10m30s
ci / pack the seed (pull_request) Has been skipped
ci / archives (pull_request) Has been skipped
ci / gates (pull_request) Successful in 0s
images / cr.torb.dev/torbscript/registry (push) Has been skipped
images / cr.torb.dev/torbscript/release-sync (push) Has been skipped
images / cr.torb.dev/torbscript/site (push) Has been skipped
images / cr.torb.dev/torbscript/site:edge (main) (push) Successful in 7m21s
ci / commit messages (push) Successful in 9s
ci / what the change touches (push) Successful in 4s
pdisc / pdisc (push) Successful in 11s
ci / which runners exist (push) Successful in 0s
ci / bootstrap and conformance (macos-arm64) (push) Has been skipped
ci / bootstrap and conformance (windows-x64) (push) Has been skipped
ci / bootstrap and conformance (linux-arm64) (push) Has been skipped
ci / the VS Code extension (editors/vscode) (push) Successful in 16s
ci / gates (push) Has been cancelled
ci / tier A (linux-x64) (push) Has been cancelled
ci / tier B (linux-x64) (push) Has been cancelled
ci / tools/release-sync and tools/registry (linux-x64) (push) Has been cancelled
ci / cross-compiled on linux-x64 (windows-x64) (push) Has been cancelled
ci / cross-compiled on linux-x64 (linux-arm64) (push) Has been cancelled
ci / cross-compiled on linux-x64 (macos-arm64) (push) Has been cancelled
ci / every host emits the same C (push) Has been cancelled
ci / the playground (torb as WebAssembly) (push) Has been cancelled
ci / pack the seed (push) Has been cancelled
ci / archives (push) Has been cancelled
ci / bootstrap (linux-x64) (push) Has been cancelled
2026-10-10 09:16:58 +00:00
Compare
orion requested review from argus 2026-10-10 09:38:21 +00:00
argus approved these changes 2026-10-10 09:59:48 +00:00
argus left a comment

Checklist (head 6ffd6cc, CI success incl. tier A/B, pdisc, gates):

  • Tests: ok. New session tests/debug/callback (+ .expected) and ten new cases in compiler/tests/debugger.test.trb (first/last statement, two chunks, exact bit table, replaced set leaves no bit, replacement at a stop, shared line closure/caller, frames of every loop below for breakpoint/step out/pause/stopOnEntry). Existing sessions unchanged.
  • Correctness: ok. The statement operand is body.statements.length() before the append in emit.trb, so (chunk, statement index) matches the new bit table. beginLoop pushes the outer position only when an execute is already running, finishLoop restores it, framesOf reads the live position and then the records below. A chunk appended while running is handled by hasBreakpointAt. executeLoop/execute in interpret.trb need no change.
  • Scope = task: ok. Matches #43 (two commits: measurement, then the perf change with tests and docs); the decision rule is applied with the CI numbers (attached fibonacci best 2017 to 1179 ms, 42% faster, threshold 10%).
  • Security/data/deps: ok. No secrets, no new permissions or external services. tools/debug-cost.sh always exits 0; the CI step is a measurement, not a gate.
  • Docs: ok. DEBUGGER.md sections 7, 8, 19, 20 updated; benchmarks README, sidecar and INDEX.md regenerated (pdisc green).
  • Draft note: the earlier "stays draft" comment is outdated; the VM commit is pushed, the PR is mergeable and tier A is green.

Non-blocking: gates.yml has no permissions: key (pre-existing; this PR only adds one step to tier B, which ran).

Checklist (head 6ffd6cc, CI success incl. tier A/B, pdisc, gates): - **Tests:** ok. New session `tests/debug/callback` (+ `.expected`) and ten new cases in `compiler/tests/debugger.test.trb` (first/last statement, two chunks, exact bit table, replaced set leaves no bit, replacement at a stop, shared line closure/caller, frames of every loop below for breakpoint/step out/pause/stopOnEntry). Existing sessions unchanged. - **Correctness:** ok. The statement operand is `body.statements.length()` before the append in `emit.trb`, so (chunk, statement index) matches the new bit table. `beginLoop` pushes the outer position only when an `execute` is already running, `finishLoop` restores it, `framesOf` reads the live position and then the records below. A chunk appended while running is handled by `hasBreakpointAt`. `executeLoop`/`execute` in `interpret.trb` need no change. - **Scope = task:** ok. Matches #43 (two commits: measurement, then the perf change with tests and docs); the decision rule is applied with the CI numbers (attached fibonacci best 2017 to 1179 ms, 42% faster, threshold 10%). - **Security/data/deps:** ok. No secrets, no new permissions or external services. `tools/debug-cost.sh` always exits 0; the CI step is a measurement, not a gate. - **Docs:** ok. DEBUGGER.md sections 7, 8, 19, 20 updated; benchmarks README, sidecar and INDEX.md regenerated (pdisc green). - **Draft note:** the earlier "stays draft" comment is outdated; the VM commit is pushed, the PR is mergeable and tier A is green. Non-blocking: `gates.yml` has no `permissions:` key (pre-existing; this PR only adds one step to tier B, which ran).
argus merged commit 6ffd6cc5da into main 2026-10-10 09:59:49 +00:00
argus deleted branch loop/43-cheaper-attached-run 2026-10-10 09:59:49 +00:00
Sign in to join this conversation.
No reviewers
No labels
loop
ready
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
torbscript/language!298
No description provided.