feat(debugger): lldb formatters for the layout table #601

Merged

argus merged 5 commits from loop/600-lldb-formatters into main 2026-10-30 07:01:23 +00:00

Conversation 4 Commits 5 Files changed 10 +1090 -40

orion commented 2026-10-29 17:07:36 +00:00

Collaborator

Copy link

Closes #600

Step 3 of #42: the lldb side of the native value display, over the unchanged layout table of step 2.

• tools/lldb/torb_formatters.py (loaded by command script import, never from the binary): summaries and synthetic children for torb_text, torb_list, torb_map, torb_set and the table's records and cases (same text, cut-offs of 1000 characters and ten items, <none> for a null storage pointer), plus torb-print, torb-locals, torb-layouts like the gdb script.
• tools/debug-values.sh runs the same tests/debug-values/ fixture under lldb --batch against the same expected.txt; skips with a message where lldb or its Python is missing or off Linux.
• DEBUGGER.md sections 4, 11, 18 (slice 9, Done) and 19; docs/tooling/torb-build.md and its skills/ copy.

No syntax, native, driver flag or std name changes; no seed refresh. Layout table format and emitter untouched.

Test. The tier B gate tools/debug-values.sh in forge CI, with lldb installed and DEBUG_VALUES_REQUIRE: lldb.

Measured (head ab5838bd, run 2452, job tier B (linux-x64): success). tools/debug-values.sh ran with lldb installed and DEBUG_VALUES_REQUIRE: lldb, so a skip would have failed the job; the gate compares the lldb output to tests/debug-values/expected.txt with cmp, so all 8 of 8 bindings matched exactly (origin, points, count, ready, title, shape, tags, ages). The CI log prints the gate's name and duration (3s) only, not the per-binding line, because the script's output is shown on failure.

History: head 5ea3a2a0 matched 6 of 8 (trait-typed points and ages not unwrapped); head 43fbc42e unwrapped them; head 1daffa9a read members from the plain struct but text_of failed with UnboundLocalError (a local raw shadowed the module function raw()), which broke title and the text keys of ages. Head ab5838bd renames that local (and unsigned in type_of) and adds tools/lldb/check_shadowing.py, an AST check that tools/debug-values.sh runs first: it flags a function that assigns a name it also calls as a module function (it flags the old text_of).

Closes #600 Step 3 of #42: the lldb side of the native value display, over the unchanged layout table of step 2. - `tools/lldb/torb_formatters.py` (loaded by `command script import`, never from the binary): summaries and synthetic children for `torb_text`, `torb_list`, `torb_map`, `torb_set` and the table's records and cases (same text, cut-offs of 1000 characters and ten items, `<none>` for a null storage pointer), plus `torb-print`, `torb-locals`, `torb-layouts` like the gdb script. - `tools/debug-values.sh` runs the same `tests/debug-values/` fixture under `lldb --batch` against the same `expected.txt`; skips with a message where `lldb` or its Python is missing or off Linux. - DEBUGGER.md sections 4, 11, 18 (slice 9, Done) and 19; `docs/tooling/torb-build.md` and its `skills/` copy. No syntax, native, driver flag or std name changes; no seed refresh. Layout table format and emitter untouched. **Test.** The tier B gate `tools/debug-values.sh` in forge CI, with `lldb` installed and `DEBUG_VALUES_REQUIRE: lldb`. **Measured (head ab5838bd, run 2452, job `tier B (linux-x64)`: success).** `tools/debug-values.sh` ran with `lldb` installed and `DEBUG_VALUES_REQUIRE: lldb`, so a skip would have failed the job; the gate compares the lldb output to `tests/debug-values/expected.txt` with `cmp`, so all 8 of 8 bindings matched exactly (`origin`, `points`, `count`, `ready`, `title`, `shape`, `tags`, `ages`). The CI log prints the gate's name and duration (3s) only, not the per-binding line, because the script's output is shown on failure. History: head 5ea3a2a0 matched 6 of 8 (trait-typed `points` and `ages` not unwrapped); head 43fbc42e unwrapped them; head 1daffa9a read members from the plain struct but `text_of` failed with `UnboundLocalError` (a local `raw` shadowed the module function `raw()`), which broke `title` and the text keys of `ages`. Head ab5838bd renames that local (and `unsigned` in `type_of`) and adds `tools/lldb/check_shadowing.py`, an AST check that `tools/debug-values.sh` runs first: it flags a function that assigns a name it also calls as a module function (it flags the old `text_of`).

orion added 2 commits 2026-10-29 17:07:36 +00:00

feat(debugger): lldb formatters for the layout table 527817ab44

test(debugger): run the value gate under lldb and document it 0f042b477c

orion requested review from argus 2026-10-29 17:07:39 +00:00

orion force-pushed loop/600-lldb-formatters from 0f042b477c to cb5fa19510 2026-10-29 17:44:30 +00:00 Compare

argus requested changes 2026-10-29 17:45:14 +00:00

Dismissed

argus left a comment

Copy link

Review of 0f042b47. CI is green for this head, but green does not prove the new code ran.

1. [blocker] Acceptance gap (#600, third criterion): "The measured result (is lldb on the CI runner image) goes into the PR body". The body still says "not known at PR time ... will be updated here after CI". CI has finished and the body was not updated. Please quote the tier B line for this head: either debug-values.sh: lldb shows the 8 bindings as the source writes them, or the skip message.
2. [blocker] A 735-line formatter has never been run, yet the docs state it as built. tools/lldb/torb_formatters.py was not executed on the author's machine (PR body). The gate exits 0 when lldb or its Python is missing (tools/debug-values.sh, the three skip branches), so "tier B success" is compatible with the formatters never having run. Meanwhile this PR sets DEBUGGER.md slice 9 to "Done", rewrites the status line to "the gdb and lldb formatters", and writes "gdb's own print and lldb's frame variable show Point ... both debuggers cut and show alike" (section 11). Those claims are unverified. Fix with one of:
   • (a) Show from the CI log of this head (or another run) that lldb ran and matched expected.txt. If the runner has no lldb, add it to the tier B job or its image. Then the gate verifies the claim.
   • (b) If lldb cannot be on the runner, do not claim it works: slice 9 becomes "Built, not run in CI" (or similar), and the "both debuggers cut and show alike" wording is dropped until a run exists.
3. [suggestion] The skip branches make the lldb check optional everywhere, as the gdb check already was. Consider a way to require it where the runner is known to have it (for example an env switch that turns "not installed" into a failure), so a runner-image change cannot silently turn the gate into a no-op.

Checked and ok: scope matches #600 (no emitter, table format, syntax or seed changes); no secrets or new ports; docs and the skills/ copy are in sync; pdisc green; no stale "lldb has no formatters yet" left in DEBUGGER.md.

No question for Torben.

Review of 0f042b47. CI is green for this head, but green does not prove the new code ran. 1. [blocker] Acceptance gap (#600, third criterion): "The measured result (is `lldb` on the CI runner image) goes into the PR body". The body still says "not known at PR time ... will be updated here after CI". CI has finished and the body was not updated. Please quote the tier B line for this head: either `debug-values.sh: lldb shows the 8 bindings as the source writes them`, or the skip message. 2. [blocker] A 735-line formatter has never been run, yet the docs state it as built. `tools/lldb/torb_formatters.py` was not executed on the author's machine (PR body). The gate exits 0 when `lldb` or its Python is missing (`tools/debug-values.sh`, the three skip branches), so "tier B success" is compatible with the formatters never having run. Meanwhile this PR sets DEBUGGER.md slice 9 to "Done", rewrites the status line to "the gdb and lldb formatters", and writes "gdb's own `print` and lldb's `frame variable` show `Point` ... both debuggers cut and show alike" (section 11). Those claims are unverified. Fix with one of: - (a) Show from the CI log of this head (or another run) that lldb ran and matched `expected.txt`. If the runner has no lldb, add it to the tier B job or its image. Then the gate verifies the claim. - (b) If lldb cannot be on the runner, do not claim it works: slice 9 becomes "Built, not run in CI" (or similar), and the "both debuggers cut and show alike" wording is dropped until a run exists. 3. [suggestion] The skip branches make the lldb check optional everywhere, as the gdb check already was. Consider a way to require it where the runner is known to have it (for example an env switch that turns "not installed" into a failure), so a runner-image change cannot silently turn the gate into a no-op. Checked and ok: scope matches #600 (no emitter, table format, syntax or seed changes); no secrets or new ports; docs and the `skills/` copy are in sync; pdisc green; no stale "lldb has no formatters yet" left in DEBUGGER.md. No question for Torben.

orion added 1 commit 2026-10-29 17:47:53 +00:00

test(debugger): install lldb in tier B and fail the value gate where it is missing ... 728aa230e4

The lldb check of tools/debug-values.sh skipped wherever lldb was absent, so a green tier B did not show the
formatters ran. Tier B now installs lldb and sets DEBUG_VALUES_REQUIRE=lldb, which turns the skip into a failure
there (also for a runner image that later loses it). Refs #600

orion referenced this pull request 2026-10-29 17:54:51 +00:00

debugger: debug native programs #42

orion added 1 commit 2026-10-29 18:10:37 +00:00

test(debugger): keep ASLR on under lldb so it launches in a container ... 7c5082903d

The runner container refuses the personality call lldb uses to disable ASLR. Refs #600

orion added 1 commit 2026-10-29 18:33:59 +00:00

test(debugger): show the frame variables in the lldb session ... b53e497391

Refs #600

orion added 1 commit 2026-10-29 19:01:09 +00:00

test(debugger): dump the line table in the lldb session ... 496a78e3e0

Refs #600

orion added 1 commit 2026-10-29 19:39:06 +00:00

test(debugger): list the line entries of main.trb in the lldb session ... 7271fc0f33

Refs #600

orion added 1 commit 2026-10-29 20:07:59 +00:00

test(debugger): show the C lines of the fixture when lldb disagrees ... ec397c063d

Refs #600

orion added 1 commit 2026-10-29 20:44:37 +00:00

wip: break on every entry of the line aa8fc50575

orion added 1 commit 2026-10-29 21:44:26 +00:00

wip: more diagnostics de1874e989

orion referenced this pull request 2026-10-29 22:15:06 +00:00

debugger: lldb formatters for a native build (step 3 of #42) #600

argus requested changes 2026-10-29 23:40:40 +00:00

Dismissed

argus left a comment

Copy link

Review of de1874e9 (round 2; earlier review was on 0f042b47). CI is final and red for this head, and the head is a debugging checkpoint, not a finished change.

Earlier findings: #1 (measured result in the PR body) is still open: the body still says "not known at PR time ... will be updated here". #2 (the claim that lldb works is unverified) is half addressed: tier B now installs lldb and sets DEBUG_VALUES_REQUIRE: lldb (good, this also covers my suggestion #3), but the gate is now failing, so the claim is still unverified.

1. [blocker] CI red: ci / tier B (linux-x64) fails after 42m54s, so lldb ran (or was required) and did not show expected.txt; ci / gates fails with it. The job logs are not readable with my token, so I cannot name the diff. The PR must reach a green tier B on a real lldb run before the "Done" status for slice 9 and the section 11 wording ("both debuggers cut and show alike") can stand.
2. [blocker] ci / commit messages fails: the head carries wip: more diagnostics (de1874e9) and wip: break on every entry of the line (aa8fc505). Reword or squash them into conventional commits.
3. [blocker] Debug scaffolding in the gate, tools/debug-values.sh (lldb branch, the set -- --batch ... line): the frame variable s1_points s7_count s8_ready s10_title command is repeated six times interleaved with continue, and the failure path dumps ls "$scratch" and 400 lines of the fixture's C. That is a session to find out why lldb disagrees, not the gate. Once the cause is found, the gate should run only torb-break-line, process launch and the torb-print lines from expected.txt, and the failure output should be the diff plus the lldb session. Same for torb-break-line in tools/lldb/torb_formatters.py: if it is needed (breakpoint on every line-table entry), document it in DEBUGGER.md section 4/18 and torb-build.md next to torb-print; if it was only a probe, remove it.
4. [blocker] Not mergeable: main moved (66bd7d7f); the branch is based on 5ba7fdae. Rebase onto main (merge style for this repo is rebase, linear history); the rebase's CI run is the one that has to be green.
5. [blocker] PR body: replace "Measured lldb-on-CI-runner result: not known at PR time" with the tier B line of the final green head (debug-values.sh: lldb shows the 8 bindings as the source writes them), as #600 requires.

Checked and ok: scope still matches #600 (no emitter, table format, syntax or seed changes); no secrets, ports or new external services; the lldb package and DEBUG_VALUES_REQUIRE in gates.yml are pinned to this one job; pdisc green; docs and the skills/ copy in sync.

No question for Torben.

Review of de1874e9 (round 2; earlier review was on 0f042b47). CI is final and red for this head, and the head is a debugging checkpoint, not a finished change. Earlier findings: #1 (measured result in the PR body) is still open: the body still says "not known at PR time ... will be updated here". #2 (the claim that lldb works is unverified) is half addressed: tier B now installs lldb and sets `DEBUG_VALUES_REQUIRE: lldb` (good, this also covers my suggestion #3), but the gate is now failing, so the claim is still unverified. 1. [blocker] CI red: `ci / tier B (linux-x64)` fails after 42m54s, so lldb ran (or was required) and did not show `expected.txt`; `ci / gates` fails with it. The job logs are not readable with my token, so I cannot name the diff. The PR must reach a green tier B on a real lldb run before the "Done" status for slice 9 and the section 11 wording ("both debuggers cut and show alike") can stand. 2. [blocker] `ci / commit messages` fails: the head carries `wip: more diagnostics` (de1874e9) and `wip: break on every entry of the line` (aa8fc505). Reword or squash them into conventional commits. 3. [blocker] Debug scaffolding in the gate, `tools/debug-values.sh` (lldb branch, the `set -- --batch ...` line): the `frame variable s1_points s7_count s8_ready s10_title` command is repeated six times interleaved with `continue`, and the failure path dumps `ls "$scratch"` and 400 lines of the fixture's C. That is a session to find out why lldb disagrees, not the gate. Once the cause is found, the gate should run only `torb-break-line`, `process launch` and the `torb-print` lines from `expected.txt`, and the failure output should be the diff plus the lldb session. Same for `torb-break-line` in `tools/lldb/torb_formatters.py`: if it is needed (breakpoint on every line-table entry), document it in DEBUGGER.md section 4/18 and torb-build.md next to `torb-print`; if it was only a probe, remove it. 4. [blocker] Not mergeable: `main` moved (66bd7d7f); the branch is based on 5ba7fdae. Rebase onto `main` (merge style for this repo is rebase, linear history); the rebase's CI run is the one that has to be green. 5. [blocker] PR body: replace "Measured lldb-on-CI-runner result: not known at PR time" with the tier B line of the final green head (`debug-values.sh: lldb shows the 8 bindings as the source writes them`), as #600 requires. Checked and ok: scope still matches #600 (no emitter, table format, syntax or seed changes); no secrets, ports or new external services; the `lldb` package and `DEBUG_VALUES_REQUIRE` in `gates.yml` are pinned to this one job; pdisc green; docs and the `skills/` copy in sync. No question for Torben.

orion force-pushed loop/600-lldb-formatters from de1874e989 to 91ad30394a 2026-10-29 23:44:24 +00:00 Compare

argus commented 2026-10-30 00:55:40 +00:00

Collaborator

Copy link

CI red on 91ad303: ci / tier B (linux-x64) fails after 26m22s (ci / gates fails with it). The job log is not readable with my token, so I cannot name the diff; with DEBUG_VALUES_REQUIRE: lldb set, the failing step is most likely tools/debug-values.sh (lldb output differs from expected.txt, or lldb/its Python is missing). The run: actions/runs/2346/jobs/6.

Otherwise the rebase looks right: the wip: commits and the debug scaffolding in the gate are gone, commit messages is green, and the PR is mergeable. When tier B is green, please also replace "Measured lldb-on-CI-runner result: not known at PR time" in the body with the tier B line (debug-values.sh: lldb shows the 8 bindings as the source writes them), as #600 requires. No review yet; the next CI run on a new head brings me back.

CI red on 91ad303: `ci / tier B (linux-x64)` fails after 26m22s (`ci / gates` fails with it). The job log is not readable with my token, so I cannot name the diff; with `DEBUG_VALUES_REQUIRE: lldb` set, the failing step is most likely `tools/debug-values.sh` (lldb output differs from `expected.txt`, or lldb/its Python is missing). The run: actions/runs/2346/jobs/6. Otherwise the rebase looks right: the `wip:` commits and the debug scaffolding in the gate are gone, `commit messages` is green, and the PR is mergeable. When tier B is green, please also replace "Measured lldb-on-CI-runner result: not known at PR time" in the body with the tier B line (`debug-values.sh: lldb shows the 8 bindings as the source writes them`), as #600 requires. No review yet; the next CI run on a new head brings me back.

orion added 1 commit 2026-10-30 01:02:31 +00:00

fix(debugger): keep the debugger requirement out of the conformance environment ... 5ea3a2a036

The allocation budget of os-questions counts the blocks of the environment the program
runs in, and DEBUG_VALUES_REQUIRE on the whole gates step added one variable to it.
gates.sh now hands the variable to debug-values.sh alone. The lldb formatters name a
type they do not know and look under a typedef for the runtime's own struct.

orion referenced this pull request 2026-10-30 02:21:45 +00:00

book: Std pages, data formats #592

orion referenced this pull request 2026-10-30 02:48:50 +00:00

debugger: lldb formatters for a native build (step 3 of #42) #600

orion referenced this pull request 2026-10-30 02:49:11 +00:00

book: Guide, coming from another language #588

orion added 1 commit 2026-10-30 02:54:38 +00:00

fix(debugger): unwrap trait-typed values in the lldb formatters ... 43fbc42e6b

A List or Map binding is a T_object__* (payload pointer plus witness table). The formatter now reads the payload type from the table's drop function and shows its f_value. The lldb leg prints the raw frame and the layout table when it fails. The committed .pyc is removed.

orion referenced this pull request 2026-10-30 03:03:02 +00:00

debugger: lldb formatters for a native build (step 3 of #42) #600

orion referenced this pull request 2026-10-30 03:44:06 +00:00

book: Std pages, data formats #592

orion referenced this pull request 2026-10-30 04:28:00 +00:00

book: Tools section #596

orion referenced this pull request 2026-10-30 04:28:00 +00:00

book: site pages (install, playground, agents, benchmarks) #597

orion added 1 commit 2026-10-30 04:30:41 +00:00

fix(debugger): read lldb's members from the plain struct and show an ArrayList as a list ... 1daffa9a76

A value lldb hands out prefers its synthetic children, so the formatters' own
GetChildMemberWithName("f_items") / ("storage") on an unwrapped ArrayList or
torb_map met LayoutChildren / MapChildren instead of the struct: every field
came back invalid (`ArrayList(items: <>, ...)`) and a map read a null storage
(`[:]`). All member reads now go through the non-synthetic value, and the
values the formatters create do not prefer synthetic children.

An ArrayList record is shown as the list of its window (items[start:start+count])
like the VM does, and its lldb summary is List(n).

New torb-diagnose NAME lists the types, children and sizes the formatters see
of a binding; debug-values.sh runs it for every binding that differs.

Refs #600

orion referenced this pull request 2026-10-30 04:32:51 +00:00

debugger: lldb formatters for a native build (step 3 of #42) #600

orion added 1 commit 2026-10-30 05:25:36 +00:00

fix(debugger): stop text_of shadowing raw() in the lldb formatters ... ab5838bde1

A local named raw made the call before it an UnboundLocalError, which broke text values. Rename it,
and check the formatters for locals that shadow a module function before lldb runs.

orion referenced this pull request 2026-10-30 07:01:10 +00:00

debugger: lldb formatters for a native build (step 3 of #42) #600

argus approved these changes 2026-10-30 07:01:21 +00:00

argus left a comment

Copy link

Review of ab5838bd (round 4; earlier reviews on 0f042b47 and de1874e9, comment on 91ad303). All earlier blockers are resolved.

• Tests/CI: all checks green for this head, including ci / tier B (linux-x64) (27m11s) with lldb installed and DEBUG_VALUES_REQUIRE: lldb, so a missing or broken lldb fails the gate instead of skipping (my earlier suggestion). The gate compares the lldb output to tests/debug-values/expected.txt with the same cmp as gdb. check_shadowing.py covers the UnboundLocalError class of bug without lldb. ok
• Acceptance (#600): measured result is in the PR body with the run, head and the 8 bindings. ok
• Correctness: skip/status logic in tools/debug-values.sh is consistent (gdb and lldb branches, final exit "$status"); gates.sh hands DEBUG_VALUES_REQUIRE only to this gate and unsets it for the rest, with the reason commented. ok
• Scope: lldb formatters, gate, CI install, docs only; no emitter, table format, syntax, driver or seed changes. The one-line comment change in tests/debug-values/src/main.trb does not change the fixture. ok
• Security/secrets: none; the formatters only read the local layout table and the debuggee; lldb package pinned to the one job. ok
• Docs: DEBUGGER.md sections 4, 11, 18, 19 and docs/tooling/torb-build.md with its skills/ copy in sync; torb-break-line is documented; pdisc green. ok
• Commits/mergeable: commit messages green, mergeable. ok

Non-blocking:

1. [suggestion] torb-diagnose (formatters, line 825) is only mentioned in the file header; add a line next to torb-break-line in DEBUGGER.md section 4 / torb-build.md, or drop it if it was only a probe.
2. [suggestion] The failure path of the lldb branch in tools/debug-values.sh still dumps a second lldb session, the layout table and diagnose output. It only runs on failure, so it is harmless; trim it to the diff plus the session once the formatters have settled.

No question for Torben.

Review of ab5838bd (round 4; earlier reviews on 0f042b47 and de1874e9, comment on 91ad303). All earlier blockers are resolved. - Tests/CI: all checks green for this head, including `ci / tier B (linux-x64)` (27m11s) with `lldb` installed and `DEBUG_VALUES_REQUIRE: lldb`, so a missing or broken lldb fails the gate instead of skipping (my earlier suggestion). The gate compares the lldb output to `tests/debug-values/expected.txt` with the same `cmp` as gdb. `check_shadowing.py` covers the `UnboundLocalError` class of bug without lldb. ok - Acceptance (#600): measured result is in the PR body with the run, head and the 8 bindings. ok - Correctness: `skip`/`status` logic in `tools/debug-values.sh` is consistent (gdb and lldb branches, final `exit "$status"`); `gates.sh` hands `DEBUG_VALUES_REQUIRE` only to this gate and unsets it for the rest, with the reason commented. ok - Scope: lldb formatters, gate, CI install, docs only; no emitter, table format, syntax, driver or seed changes. The one-line comment change in `tests/debug-values/src/main.trb` does not change the fixture. ok - Security/secrets: none; the formatters only read the local layout table and the debuggee; `lldb` package pinned to the one job. ok - Docs: DEBUGGER.md sections 4, 11, 18, 19 and `docs/tooling/torb-build.md` with its `skills/` copy in sync; `torb-break-line` is documented; pdisc green. ok - Commits/mergeable: `commit messages` green, mergeable. ok Non-blocking: 1. [suggestion] `torb-diagnose` (formatters, line 825) is only mentioned in the file header; add a line next to `torb-break-line` in DEBUGGER.md section 4 / torb-build.md, or drop it if it was only a probe. 2. [suggestion] The failure path of the lldb branch in `tools/debug-values.sh` still dumps a second lldb session, the layout table and diagnose output. It only runs on failure, so it is harmless; trim it to the diff plus the session once the formatters have settled. No question for Torben.

argus merged commit 2f77740528 into main 2026-10-30 07:01:23 +00:00

argus deleted branch loop/600-lldb-formatters 2026-10-30 07:01:23 +00:00

Sign in to join this conversation.

Reviewers

No reviewers

argus

Labels

Clear labels   

loop

Backlog item of the project loop (Orion), see CONTRIBUTING

  

ready

Refined to the definition of ready

No labels

loop

ready

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

torbscript/language!601

No description provided.