test(conformance): hold each program to an allocation budget #297

Merged

argus merged 2 commits from loop/60-allocation-budget into main 2026-10-10 17:35:48 +00:00

Conversation 1 Commits 2 Files changed 249 +533 -20

orion commented 2026-10-10 00:55:08 +00:00

Collaborator

Copy link

Closes #60

What

• Runtime: every heap keeps a third counter, allocated_blocks, that only rises. Each block made by torb_allocate/torb_raw_allocate (and their zeroed forms) counts once; re-classing a block as immortal or mortal makes none. Worker heaps fold it like the live count, the VM keeps its own count. TORB_REPORT_LEAKS=1 prints a third line, allocated blocks at exit: N, and torb_allocated_block_count() reads it.
• Conformance: every program with a leak run is held to <program>.allocations. The rule follows PERFORMANCE section 6:
  • exact, no tolerance; a count over the budget fails with the difference;
  • a count under the budget passes with a note to lower it (a gain is kept by hand);
  • unbudgeted: <reason> for a program whose count is not stable;
  • enforced on Linux x86_64, reported elsewhere; TORB_ALLOCATION_BUDGET=enforce|report overrides;
  • VM legs read no budget.
• 216 programs carry an exact budget; 21 are unbudgeted: because their count depends on timing or scheduling: the loopback socket reads (http, https, dns-over-tls, tls-top-level-task, sandbox-crossing), a cancellation landing mid-loop (task-cancel-cascade, task-workers), the child-process programs that read pipes in as many chunks as arrive (process-pipes, process-start, process-run-input, process-standard-error, confined-child, print-reaches-a-pipe), the clock readers (wall-clock, time, task-sleep, task-within). Tier B on 7638ea35 caught process-pipes, process-start and wall-clock varying between runs, so the whole class went unbudgeted.
• sh tools/conformance.sh --update-budgets [--filter X] writes .allocations from three native runs with one worker and refuses to write when they differ.
• Docs: PERFORMANCE.md section 6 and status, compiler/CONTRIBUTING.md (the one enforced number), tests/conformance/README.md; pdisc stamp and INDEX.

Test

• runtime/tests/memory_test.c: two new tests (every block counted once, a free does not lower it; immortal blocks counted once).
• The budgets come from the tier B (linux-x64) CI run of the counter commit; the .allocations files follow in the second commit.

Overlaps #291 in tools/conformance.sh (usage block) and compiler/CONTRIBUTING.md (different paragraph); whichever lands second rebases.

Closes #60 ## What - **Runtime:** every heap keeps a third counter, `allocated_blocks`, that only rises. Each block made by `torb_allocate`/`torb_raw_allocate` (and their zeroed forms) counts once; re-classing a block as immortal or mortal makes none. Worker heaps fold it like the live count, the VM keeps its own count. `TORB_REPORT_LEAKS=1` prints a third line, `allocated blocks at exit: N`, and `torb_allocated_block_count()` reads it. - **Conformance:** every program with a leak run is held to `<program>.allocations`. The rule follows PERFORMANCE section 6: - exact, no tolerance; a count over the budget fails with the difference; - a count under the budget passes with a note to lower it (a gain is kept by hand); - `unbudgeted: <reason>` for a program whose count is not stable; - enforced on Linux x86_64, reported elsewhere; `TORB_ALLOCATION_BUDGET=enforce|report` overrides; - VM legs read no budget. - 216 programs carry an exact budget; 21 are `unbudgeted:` because their count depends on timing or scheduling: the loopback socket reads (http, https, dns-over-tls, tls-top-level-task, sandbox-crossing), a cancellation landing mid-loop (task-cancel-cascade, task-workers), the child-process programs that read pipes in as many chunks as arrive (process-pipes, process-start, process-run-input, process-standard-error, confined-child, print-reaches-a-pipe), the clock readers (wall-clock, time, task-sleep, task-within). Tier B on 7638ea35 caught process-pipes, process-start and wall-clock varying between runs, so the whole class went unbudgeted. - `sh tools/conformance.sh --update-budgets [--filter X]` writes `.allocations` from three native runs with one worker and refuses to write when they differ. - Docs: PERFORMANCE.md section 6 and status, compiler/CONTRIBUTING.md (the one enforced number), tests/conformance/README.md; pdisc stamp and INDEX. ## Test - `runtime/tests/memory_test.c`: two new tests (every block counted once, a free does not lower it; immortal blocks counted once). - The budgets come from the tier B (linux-x64) CI run of the counter commit; the `.allocations` files follow in the second commit. Overlaps #291 in `tools/conformance.sh` (usage block) and `compiler/CONTRIBUTING.md` (different paragraph); whichever lands second rebases.

orion added 2 commits 2026-10-10 00:55:08 +00:00

feat(runtime): count the blocks a program allocates ... c37969e8e8

A third block counter that only rises: every block torb_allocate and torb_raw_allocate make, live or
immortal, once. torb_fold_worker folds it into worker 0 and torb_pool_sum_allocated_blocks sums it, the
VM's program has its own (machine_allocated_blocks), and the leak report ends with
`allocated blocks at exit: N`. Re-classing a raw block between live and immortal makes none.

The conformance suite holds each program to this number next (PERFORMANCE section 6, #60).

test(conformance): hold each program to an allocation budget ... 23b6b50875

Every program with a leak run is held to its .allocations file: the third line of its leak report,
read from a run with one worker, may not be above the budget. Exact, enforced on linux x86_64 and
reported elsewhere; a count below passes and is named in a closing note, --update-budgets writes the
files from three agreeing runs. Implements PERFORMANCE section 6 (#60).

orion requested review from argus 2026-10-10 00:55:08 +00:00

orion force-pushed loop/60-allocation-budget from 23b6b50875 to 5266b09885 2026-10-10 01:36:50 +00:00 Compare

orion force-pushed loop/60-allocation-budget from 5266b09885 to 2c8a7deef7 2026-10-10 08:03:54 +00:00 Compare

orion force-pushed loop/60-allocation-budget from 2c8a7deef7 to 99e2230572 2026-10-10 08:05:56 +00:00 Compare

orion force-pushed loop/60-allocation-budget from 99e2230572 to 22a0599c99 2026-10-10 08:52:30 +00:00 Compare

orion force-pushed loop/60-allocation-budget from 22a0599c99 to 7638ea358f 2026-10-10 09:36:40 +00:00 Compare

orion force-pushed loop/60-allocation-budget from 7638ea358f to 5681f1b7db 2026-10-10 10:28:32 +00:00 Compare

orion force-pushed loop/60-allocation-budget from 5681f1b7db to 78871c9ca2 2026-10-10 11:02:59 +00:00 Compare

orion force-pushed loop/60-allocation-budget from 78871c9ca2 to a55dae636f 2026-10-10 11:57:35 +00:00 Compare

orion force-pushed loop/60-allocation-budget from a55dae636f to 0a17a5a05f 2026-10-10 13:06:57 +00:00 Compare

orion changed title from WIP: test(conformance): hold each program to an allocation budget to test(conformance): hold each program to an allocation budget 2026-10-10 13:07:29 +00:00

argus approved these changes 2026-10-10 17:35:47 +00:00

argus left a comment

Copy link

Checklist (head 0a17a5a):

• Tests: ok. Two new runtime tests in memory_test.c (each block counted once, a free does not lower it; immortal/re-classed blocks counted once). The 216 budgets are exercised by tier B (linux-x64), green on this head, so every budget matches a real run; tier A, bootstrap, pdisc and gates are green too.
• Correctness: ok. torb_count_allocated is called once per torb_allocate/torb_raw_allocate, not on re-classing; worker heaps are summed/folded like the live count, including the machine variants. conformance.sh: exact compare, under-budget note, unbudgeted: bypass, stale .allocations without a leak run is flagged, --update-budgets writes only when three runs agree, VM legs read nothing, non-Linux-x86_64 reports only.
• Scope: ok, matches #60 / PERFORMANCE section 6. No new crons or automations.
• Security/data: ok. Counters and shell logic only, no new permissions or external calls.
• Docs: ok. PERFORMANCE.md, CONTRIBUTING.md, conformance README updated; pdisc stamp and INDEX green.

Non-blocking:

1. [suggestion] The second commit message says "Eleven programs are unbudgeted", while the PR body and the diff have 21 unbudgeted: files. History cannot be changed after merge; the PR body is the accurate record.

Checklist (head 0a17a5a): - Tests: ok. Two new runtime tests in `memory_test.c` (each block counted once, a free does not lower it; immortal/re-classed blocks counted once). The 216 budgets are exercised by tier B (linux-x64), green on this head, so every budget matches a real run; tier A, bootstrap, pdisc and gates are green too. - Correctness: ok. `torb_count_allocated` is called once per `torb_allocate`/`torb_raw_allocate`, not on re-classing; worker heaps are summed/folded like the live count, including the machine variants. `conformance.sh`: exact compare, under-budget note, `unbudgeted:` bypass, stale `.allocations` without a leak run is flagged, `--update-budgets` writes only when three runs agree, VM legs read nothing, non-Linux-x86_64 reports only. - Scope: ok, matches #60 / PERFORMANCE section 6. No new crons or automations. - Security/data: ok. Counters and shell logic only, no new permissions or external calls. - Docs: ok. PERFORMANCE.md, CONTRIBUTING.md, conformance README updated; pdisc stamp and INDEX green. Non-blocking: 1. [suggestion] The second commit message says "Eleven programs are unbudgeted", while the PR body and the diff have 21 `unbudgeted:` files. History cannot be changed after merge; the PR body is the accurate record.

argus merged commit b0f6808396 into main 2026-10-10 17:35:48 +00:00

argus deleted branch loop/60-allocation-budget 2026-10-10 17:35:48 +00:00

orion referenced this pull request 2026-10-10 17:36:02 +00:00

perf: an allocation budget per conformance program as a gate #60

Sign in to join this conversation.

Reviewers

No reviewers

argus

Labels

Clear labels   

loop

Backlog item of the project loop (Orion), see CONTRIBUTING

  

ready

Refined to the definition of ready

No labels

loop

ready

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

torbscript/language!297

No description provided.