debugger: hold and release the workers, the stop-the-world handshake (step 2b of #41) #719
Labels
No labels
loop
queue/failed
queue/queued
ready
status/parked
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
torbscript/language#719
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Context
Step 2b of #41 (epic). #586 (PR #603) moved a debugged thread's position onto its interpreter (
DebugThreadonRegisters); a debugged run still keeps every task on one worker (workersForin compiler/src/vm/interpret.trb returns no extra interpreters whenmain.debugger.isSome()). What is left of step 2 is letting a debugged program use several workers: the runtime must hold and release the other workers while one thread is stopped, and each worker needs an interpreter with a thread record under the debugger.Source
DEBUGGER.md sections 14 and 20
Goal
Under the debugger a program may run on more than one worker: when any thread stops, every other worker is held until the session resumes, and a stop reports the stopped thread only. A debugged run that does not ask for more than one worker behaves exactly as today.
Design (decided by the loop, see the comments)
tasks { workers }orTORB_WORKERS). So no existing session changes, and a run stays deterministic unless the program opted in. With more than one worker,workersForno longer returns early formain.debugger.isSome(): every ring worker after the first and every thread of the blocking pool getsRegisterswith aDebugThread, as for a plain run. Runs that run scripts,confinesTasksprograms and coverage runs keep their early return.runtime/task.c, two newKernelOperations besideDebugTasks, with the C side inruntime/machine.c):DebugHoldis called by the thread that stops. It sets the hold, then waits until every other program thread is quiescent: parked at a statement, idle in its scheduler (asleep on its queue), or inside a native call that blocks (readLine, a sleep, a wait on IO, anoffloadbody in a native call).DebugReleaseclears the hold and wakes the parked threads. Only one thread holds the world at a time (a lock in the kernel); a thread that reaches a stop while another holds simply parks.statement, before it evaluates breakpoints or steps (so a breakpoint it reaches while another thread is stopped is reported at the next stop after the resume, never lost and never a second stop). A thread that wakes from idle or from a blocking native call while the hold is on parks before it runs program code. The check is one atomic read per statement of a debugged run only; the non-debugged loop is untouched.Debuggerstate is written only while the world is held.stopalready runs with every other worker parked.poll(reading the channel everypollIntervalstatements) reads the channel under the debugger lock; if it has something to apply (pause,setBreakpoints,setExceptionBreakpoints) it brackets that with hold and release, so no worker reads the breakpoint table while it is rebuilt.next,stepIn,stepOutrelease every worker, but the step is ended only by the thread that began it (Debuggerrecords which); other workers run free and may hit breakpoints or a pause.continuereleases all. This keeps section 14's "act on the position where the program stopped, whicheverthreadIdthe client names".threadsis unchanged (answered at once fromDebugTasks). The stack of a thread whose task is parked at a statement on another worker is read from that worker'sDebugThread(itsrunningTaskis the task's block) and that worker's registers, not from the task block; a task that waits or has not started keeps today's path.stoppedstill carries the thread that stopped andallThreadsStopped: true.tests/debug/<name>/project.trbwithtasks { workers: 2 }) so the session starts a multi-worker run; iftorb debugdoes not take the manifest's pool,tools/debug.shpassesTORB_WORKERSfrom an optional<name>.workersfile instead (the implementer checks which one holds and records it).Requirements
continue,next,stepInorstepOutreleases them.threads,stackTrace,scopesandvariablesread a program that stands.workersFortakes the same path as today.Acceptance criteria
runtime/tests/pool_test.c(runtime unit test, tier A) has a test with at least three workers: a thread calls the hold while the others run a statement counter; after the hold returns the counters stand (read twice, 50 ms apart, equal), after the release they advance; a worker asleep in its queue and a thread in a blocking native call do not stop the hold from returning, and neither runs program code until the releasetests/debug/workers.dap(programtests/debug/workers/main.trb, a manifest or.workersfile giving it 2 or more workers) with two tasks that really run on two workers (ordered by channel handshakes, not sleeps): a breakpoint stops one task;threadsandstackTraceof the other task show its frame at the statement where it is parked; anevaluateof a loop counter of the parked task answers the same value twice, with other requests between the two, and its work (aprintafter the breakpoint's line, the second task's completion) appears in the output only aftercontinue; aftercontinuethe program runs to its end with the right output and exit code 0workers.expectedis byte-identical across 50 consecutive local runs ofsh tools/debug.sh --filter workersof the CI-builttorb(the implementer states the count in the PR; the forge CI runs it on every lane)tests/debug/produces a byte-identical.expected: no.dapor.expectedfile changes (the default stays one worker)threadIdof that task and walks the frames of that worker'sDebugThread(a unit test or the session:stackTraceof the stopped task shows its own frames, notmain's)continuethe session stops at it with the rightthreadId(inworkers.dap, order forced by a channel)sh tools/gates.sh agreen in forge CI; notorb runof a program without the debugger changes (the existingtools/debug-cost.shnumbers do not move beyond noise, read from the tier B job when it runs)docs/ROADMAP.md/ the epic's step 2b updated if they list it;skills/regenerated if docs under the skill source changeSeed
Settled: no seed impact. Two new
KernelOperationcases are ordinary compiler code (asDebugTasksin #403 was) compiled by the seed; no new syntax, no std name the compiler looks up by string, no driver flag, no new native. The runtime gets two kernel operations and a hold intask.c; the seed ships the runtime of its commit, so the seed need not know them. No seed refresh is needed, and the loop touches no seed.Size
Settled: one PR, two Conventional Commits.
feat(runtime): hold and release the workers for a debugger- the kernel operationsDebugHold/DebugRelease(compiler/src/backend/bytecode/format.trb,runtime/machine.c), the hold inruntime/task.cand its test inruntime/tests/pool_test.c; nothing in the VM calls it yet.feat(vm): debug a program on more than one worker-workersFor, the park at a statement, the handshake inreachStatement/poll, frames of a parked thread, the sessiontests/debug/workers, and the DEBUGGER.md update.If the second commit proves too large for one review, the first commit lands alone as its own PR and the epic's step 2b gets a checklist (2b-i hold in the kernel, 2b-ii the VM handshake and the session); the loop then files 2b-ii just in time.
Dependencies
Builds on #586 (merged, PR #603) and #403 (merged). Part of #41.
[loop:torbscript] intake: in scope (DEBUGGER.md sections 14 and 20, step 2b of #41), no duplicate, no owner question, no seed impact expected. next: refine
[loop:torbscript] claim refine by session:loop-torbscript-20261112 until 2026-11-12T20:45:00Z
[loop:torbscript] decided by the loop: (1) a debugged run keeps one worker by default and uses more only when the program asks (manifest
tasks { workers }orTORB_WORKERS) - this leaves every existingtests/debugsession byte-identical and the default run deterministic, which a multi-worker default would not; (2) the hold is a kernel operation pair inruntime/task.c(DebugHold/DebugRelease) with one holder at a time, workers park at their nextstatementbefore breakpoint/step evaluation, idle workers and threads in blocking native calls count as quiescent - so a stop never waits for areadLineand no breakpoint is lost; (3) sharedDebuggerstate is written only while the world is held (poll brackets a channel request that changes state with hold/release); (4) a step is ended only by the thread that began it, other workers run free during it; (5) the stack of a task parked on another worker is read from that worker'sDebugThread; (6) no seed impact: two newKernelOperationcases are ordinary compiler code, asDebugTaskswas, and the runtime ships with the seed's commit; (7) size: one PR, two commits (runtime hold + C test, then VM handshake + session + docs), falling back to a 2b-i/2b-ii checklist on #41 if the second commit is too large. No owner question: all technical, none on the owner list.[loop:torbscript] refine: body rewritten to the definition of ready (testable criteria, design points in DEBUGGER.md sections 14 and 20, seed: no impact, size: one PR in two commits), label
readyset. next: implementation[loop:torbscript] claim implementation by session:loop-torbscript-20261112 until 2026-11-12T23:50:00Z
Checkpoint. Done: runtime hold/park/quiet + kernel ops debug.hold/debug.release (91/92) + pool test, one commit
d42ab40e. In flight: CI on PR #722. Branch loop/719-debug-worker-hold. Next: CI green, review; the VM half (workersFor, statement park, tests/debug/workers, DEBUGGER.md) stays open on this issue.[loop:torbscript] implementation: runtime half landed as PR #722 (#722): torb_debug_hold/release/park and the quiet bracket for blocking calls in runtime/task.c, kernel ops debug.hold (91) and debug.release (92), pool test. Split: the VM half (workersFor, the park at a statement, the handshake, tests/debug/workers, DEBUGGER.md) is not in it; the PR says Refs #719 so this issue stays open for it. CI at hand-off: commit messages, bootstrap (linux-x64) and release-sync green; tier A/B, same-C and playground still running, gates pending. next: review
orion referenced this issue2026-11-12 18:58:54 +00:00
[loop:torbscript] claim implementation by session:loop-torbscript-20261113 until 2026-11-13T05:55:00Z
ArrayList, the hash tables and the tries onBuffer#55[loop:torbscript] implementation: the VM half landed as PR #726 (#726), branch loop/719-debug-workers-vm, one commit 'feat(vm): debug a program on more than one worker' (workersFor, park at a statement, handshake, frames of a parked thread, tests/debug/workers, DEBUGGER.md). The worker ended after opening the PR; CI at hand-off: commit messages, touches, pdisc, bootstrap green, tier A, JS conformance, playground, gates still pending. The claim is released with this note. next: waiting (CI on PR #726; a CI fix gets a fresh worker)
[loop:torbscript] claim implement by session:loop-torbscript-20261113 until 2026-11-13T07:30:00Z
PR #726: tier A (linux-x64) is red on
d6a9d391(run 2631 job 5); a fix worker reads the failure and pushes a fix to loop/719-debug-workers-vm.[loop:torbscript] implementation: PR #726 tier A (linux-x64) failed on one gate,
tests/debug against .expected (torb debug)(14 passed, 1 failed):tests/debug/workersdiffered only in the twothreadsanswers (request_seq 7 and 11), which listed one task more thanworkers.expectedhad -task 4(id 5). That is not a bug in the VM half: the leader waits infromFollower.next(), andChannelSource.next()is the native taskreceived(channel), a real task that waits, so by the rule of DEBUGGER.md section 14 (every task that waits is a thread) it is listed until the follower has offered. Every other line of the session (stop events, frames, ids, evaluates) matched. Fix:workers.expectednow liststask 4in those two answers andworkers.dapsays why (commitfix(vm): list the leader's pending next() as a thread in the workers session, 2 files). New head:ec5457d6. CI at the time of writing: run just started onec5457d6, all jobs pending (the previous tier A took 35 min); the other checks ofd6a9d391that finished were green. Not run locally (gates run in forge CI only).[loop:torbscript] release implement: fix pushed (
ec5457d6) for the red tier A; CI of the new head is pending, the next tick judges itMerged as PR #726 (head
7b0e7593, merged by argus). Post-merge state: no "After merge" section in the PR; the change is on main, but in no release. The repo builds images on x.y.z tags (release.yml, images.yml) and the manifests say 0.1.0 with no tag cut (docs/design/RELEASE.md line 58). Next step, not done in this run: a release-cut PR (version bump in project.trb, compiler/project.trb, editors/vscode/package.json and languageVersion in compiler/src/package/version.trb, reviewer argus), then tag the release commit. The release version and the plan for the untagged 0.1.0/0.2.0 line are still open in the release plan. Verification: the workers session tests/debug/workers.dap has not been run locally; CI on main decides.