debugger: a worker pool the debugger can hold (step 2 of #41) #586

Closed
opened 2026-10-29 14:23:39 +00:00 by orion · 10 comments
Collaborator

Context

Step 2 of #41 (epic), first half. Step 1 (#403, PR #404) made every task a DAP thread on the single-worker model. Letting a debugged program use several workers needs two things: the debugger's per-thread position state must live per interpreter (today one Debugger object holds the innermost execute's position, its loop records and the running task, which only one thread may write), and a stop-the-world handshake in the VM and runtime. This issue is the first, behaviour-preserving half; the handshake and the multi-worker session are the next issue, filed when this one has landed (just in time, linked by dependency).

Source

DEBUGGER.md sections 14 and 20

Goal

The state that describes where one thread is - chunk, pc, base, depth, statement, below, loops, executes, runningTask, taskBelow and the statement countdown - moves out of the shared Debugger (compiler/src/vm/debug.trb) into a per-interpreter record reached through that interpreter's Registers. What is decided for the whole run stays shared: breakpoints, stepping and its start, pauseRequested, stopsOnEntry, the tables and the host callbacks. The debugger sources (compiler/src/debugger/) read the stopped thread's record. The run still uses one worker under the debugger; nothing observable changes.

Acceptance criteria

  • Debugger holds no field that is written at a statement or by beginLoop/finishLoop; they are written on a per-interpreter record (one for the one worker now), and reachStatement, height, position, beginLoop, finishLoop, the runningTask/taskBelow bookkeeping in interpret.trb and the frame code in compiler/src/debugger/ take that record
  • A stop reports and walks the frames of the thread it happened on (the record of the interpreter that called stop), so a second record would not be confused with the first; a unit test in the compiler's test suite builds two records and checks that beginLoop/finishLoop/height on one do not change the other
  • Every session in tests/debug/ produces a byte-identical .expected; no .dap or .expected file changes
  • Without the debugger nothing changes: no new field read or write on the loop's non-debugged path (the arm of statement still costs one Option test), and workersFor is untouched
  • DEBUGGER.md section 20: the pool item says it is done in two steps and names this one as the first (state per thread), the second as open; section 14 and the decision stay as they are until the second lands

Seed

Settled: not in the seed's closure for anything that changes the seed. The debugger and VM sources are ordinary compiler code compiled by the seed; this change adds no syntax, no std name the compiler looks up by string, no native and no driver flag. No seed refresh is needed.

Size

Settled: one PR, two Conventional Commits - refactor(vm): keep a debugged thread's position on its interpreter (the move, tests/debug unchanged), then test(vm): two thread records do not share a position plus the DEBUGGER.md note. The remainder of the original scope is the next step, filed when this one lands: kernel operations to hold and release the other workers, workersFor giving each worker an interpreter under the debugger, the stop-the-world handshake, a multi-worker session in tests/debug/, and the section 14 decision update.

Dependencies

Builds on #403 (merged). Part of #41.

## Context Step 2 of #41 (epic), first half. Step 1 (#403, PR #404) made every task a DAP thread on the single-worker model. Letting a debugged program use several workers needs two things: the debugger's per-thread position state must live per interpreter (today one `Debugger` object holds the innermost `execute`'s position, its loop records and the running task, which only one thread may write), and a stop-the-world handshake in the VM and runtime. This issue is the first, behaviour-preserving half; the handshake and the multi-worker session are the next issue, filed when this one has landed (just in time, linked by dependency). ## Source [DEBUGGER.md sections 14 and 20](https://git.torb.dev/torbscript/language/src/branch/main/docs/design/DEBUGGER.md) ## Goal The state that describes where one thread is - `chunk`, `pc`, `base`, `depth`, `statement`, `below`, `loops`, `executes`, `runningTask`, `taskBelow` and the statement `countdown` - moves out of the shared `Debugger` (compiler/src/vm/debug.trb) into a per-interpreter record reached through that interpreter's `Registers`. What is decided for the whole run stays shared: breakpoints, `stepping` and its start, `pauseRequested`, `stopsOnEntry`, the tables and the host callbacks. The debugger sources (compiler/src/debugger/) read the stopped thread's record. The run still uses one worker under the debugger; nothing observable changes. ## Acceptance criteria - [ ] `Debugger` holds no field that is written at a statement or by `beginLoop`/`finishLoop`; they are written on a per-interpreter record (one for the one worker now), and `reachStatement`, `height`, `position`, `beginLoop`, `finishLoop`, the `runningTask`/`taskBelow` bookkeeping in `interpret.trb` and the frame code in `compiler/src/debugger/` take that record - [ ] A stop reports and walks the frames of the thread it happened on (the record of the interpreter that called `stop`), so a second record would not be confused with the first; a unit test in the compiler's test suite builds two records and checks that `beginLoop`/`finishLoop`/`height` on one do not change the other - [ ] Every session in `tests/debug/` produces a byte-identical `.expected`; no `.dap` or `.expected` file changes - [ ] Without the debugger nothing changes: no new field read or write on the loop's non-debugged path (the arm of `statement` still costs one `Option` test), and `workersFor` is untouched - [ ] DEBUGGER.md section 20: the pool item says it is done in two steps and names this one as the first (state per thread), the second as open; section 14 and the decision stay as they are until the second lands ## Seed Settled: not in the seed's closure for anything that changes the seed. The debugger and VM sources are ordinary compiler code compiled by the seed; this change adds no syntax, no std name the compiler looks up by string, no native and no driver flag. No seed refresh is needed. ## Size Settled: one PR, two Conventional Commits - `refactor(vm): keep a debugged thread's position on its interpreter` (the move, tests/debug unchanged), then `test(vm): two thread records do not share a position` plus the DEBUGGER.md note. The remainder of the original scope is the next step, filed when this one lands: kernel operations to hold and release the other workers, `workersFor` giving each worker an interpreter under the debugger, the stop-the-world handshake, a multi-worker session in `tests/debug/`, and the section 14 decision update. ## Dependencies Builds on #403 (merged). Part of #41.
Author
Collaborator

[loop:torbscript] intake: in scope (author orion, label loop), step 2 of epic #41 (DEBUGGER.md sections 14 and 20), not a duplicate; step 1 (#403) is merged. Touches the VM worker model and debugger only, no syntax, std name or driver flag; seed relevance and size are settled in refinement. No owner decision needed (internal technical design). next: refine

[loop:torbscript] intake: in scope (author orion, label loop), step 2 of epic #41 (DEBUGGER.md sections 14 and 20), not a duplicate; step 1 (#403) is merged. Touches the VM worker model and debugger only, no syntax, std name or driver flag; seed relevance and size are settled in refinement. No owner decision needed (internal technical design). next: refine
Author
Collaborator

[loop:torbscript] claim refine by session:loop-torbscript-20261029-0858 until 2026-10-29T19:30:00Z

[loop:torbscript] claim refine by session:loop-torbscript-20261029-0858 until 2026-10-29T19:30:00Z
Author
Collaborator

[loop:torbscript] decided by the loop: #586 is split just in time. The original scope (workers under the debugger, stop-the-world handshake, multi-worker session, new kernel hold/release operation in runtime/task.c) does not fit one PR of two commits: Debugger (debug.trb) is one shared object whose 10 position fields (chunk, pc, base, depth, statement, below, loops, executes, runningTask, taskBelow) plus countdown are written at every statement, so a second interpreter thread would race on all of them; that has to move per interpreter first, and it is behaviour-preserving and checkable against the 9 existing tests/debug sessions byte for byte. The handshake half (runtime C + VM + session + section 14 decision) is the next issue, filed when this lands; I did not file it now (no backlog). Seed: debugger/VM sources are ordinary compiler code, no syntax, std-name or driver-flag change, no seed refresh. Acceptance criteria and size rewritten in the issue body.

[loop:torbscript] refine: body rewritten to testable criteria, size one PR/two commits, seed settled (none), next: implement. Tokens used about 60k.

[loop:torbscript] decided by the loop: #586 is split just in time. The original scope (workers under the debugger, stop-the-world handshake, multi-worker session, new kernel hold/release operation in runtime/task.c) does not fit one PR of two commits: `Debugger` (debug.trb) is one shared object whose 10 position fields (chunk, pc, base, depth, statement, below, loops, executes, runningTask, taskBelow) plus `countdown` are written at every statement, so a second interpreter thread would race on all of them; that has to move per interpreter first, and it is behaviour-preserving and checkable against the 9 existing tests/debug sessions byte for byte. The handshake half (runtime C + VM + session + section 14 decision) is the next issue, filed when this lands; I did not file it now (no backlog). Seed: debugger/VM sources are ordinary compiler code, no syntax, std-name or driver-flag change, no seed refresh. Acceptance criteria and size rewritten in the issue body. [loop:torbscript] refine: body rewritten to testable criteria, size one PR/two commits, seed settled (none), next: implement. Tokens used about 60k.
Author
Collaborator

[loop:torbscript] claim implement by session:loop-torbscript-20261029-0858 until 2026-10-29T20:35:00Z

[loop:torbscript] claim implement by session:loop-torbscript-20261029-0858 until 2026-10-29T20:35:00Z
Author
Collaborator

[loop:torbscript] release implement: PR #603 opened, CI pending

[loop:torbscript] release implement: PR #603 opened, CI pending
Author
Collaborator

[loop:torbscript] claim fix by session:loop-torbscript-20261029-0858 until 2026-10-29T21:00:00Z

[loop:torbscript] claim fix by session:loop-torbscript-20261029-0858 until 2026-10-29T21:00:00Z
Author
Collaborator

[loop:torbscript] release fix: bootstrap (linux-x64) of PR #603 failed in step 1 with two checker errors, not a signature mismatch. (1) Registers.thread had the field default DebugThread(), and a field default is a constant that may not make a shared type object; it is now a required field and both construction sites (registersFor, workersFor) pass their own DebugThread(). (2) The new test asserted first.… and second.… directly, and assert quotes what it captures while DebugThread is not Encode; the test now compares words made by two small helpers (spotOf, depthOf). Each fix is folded into the commit that owns it (force-push with lease on the loop branch; history stays the same two Conventional Commits, check-commits.sh clean). New head 6e8f4f19. Not verified locally: no compiler on this machine, so forge CI is the first compile; the expected strings in the test were traced by hand.

[loop:torbscript] release fix: bootstrap (linux-x64) of PR #603 failed in step 1 with two checker errors, not a signature mismatch. (1) `Registers.thread` had the field default `DebugThread()`, and a field default is a constant that may not make a `shared type` object; it is now a required field and both construction sites (`registersFor`, `workersFor`) pass their own `DebugThread()`. (2) The new test asserted `first.…` and `second.…` directly, and `assert` quotes what it captures while `DebugThread` is not `Encode`; the test now compares words made by two small helpers (`spotOf`, `depthOf`). Each fix is folded into the commit that owns it (force-push with lease on the loop branch; history stays the same two Conventional Commits, `check-commits.sh` clean). New head 6e8f4f19. Not verified locally: no compiler on this machine, so forge CI is the first compile; the expected strings in the test were traced by hand.
Author
Collaborator

[loop:torbscript] claim fix by session:loop-torbscript-20261029-0858 until 2026-10-30T00:00Z

[loop:torbscript] claim fix by session:loop-torbscript-20261029-0858 until 2026-10-30T00:00Z
Author
Collaborator

[loop:torbscript] release fix: debugger.test.trb thread-position test binds the spotOf/depthOf results to constants (command form) before asserting, since the published seed rejects assert(f x == y) and the nightly rejects assert(f(x) == y) (assert quotes its condition, a DebugThread is no Encode); head 4df953c6

[loop:torbscript] release fix: debugger.test.trb thread-position test binds the spotOf/depthOf results to constants (command form) before asserting, since the published seed rejects `assert(f x == y)` and the nightly rejects `assert(f(x) == y)` (`assert` quotes its condition, a DebugThread is no `Encode`); head 4df953c6
argus closed this issue 2026-10-29 22:29:26 +00:00
Author
Collaborator

Post-merge follow-up (PR #603, merged 2026-10-29 22:29 UTC as 66bd7d7): the change is merged and this issue is closed by its Closes trailer. Post-merge CI on 66bd7d7 was still pending at check time: ci commit messages, ci what-the-change-touches, ci gates, pdisc, and the site:edge image. Compile, gates and tests/debug are therefore not yet verified on main. The repo has no deploy step for this change. Follow-up: re-check the commit status on 66bd7d7; if gates fail, reopen with the failing job.

Post-merge follow-up (PR #603, merged 2026-10-29 22:29 UTC as 66bd7d7): the change is merged and this issue is closed by its Closes trailer. Post-merge CI on 66bd7d7 was still pending at check time: ci commit messages, ci what-the-change-touches, ci gates, pdisc, and the site:edge image. Compile, gates and tests/debug are therefore not yet verified on main. The repo has no deploy step for this change. Follow-up: re-check the commit status on 66bd7d7; if gates fail, reopen with the failing job.
Sign in to join this conversation.
No labels
loop
ready
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
torbscript/language#586
No description provided.