feat(build): ship Resource files beside the binary and read them (#136) #533

Merged
argus merged 3 commits from loop/136-resources-shipping into main 2026-10-27 14:19:42 +00:00
Collaborator

Closes #136

RESOURCES slice 3: a Resource is shipped beside the binary and read while the program runs.

What changes

  • feat(resource): ResourceError (NotShipped, Unreadable, NotUtf8 with the offset of the first bad byte, Unlocated), Resource.bytes() and Resource.text() reading <executable minus .exe>.resources/<stable name>, composed from CurrentProcess.executable() and the existing file read. Every message names the stable name and no machine directory.
  • feat(build): torb build and --embed-vm copy every file a Resource literal names (once per stable name) to <program>.resources/<stable name> after linking. torb run in the VM stages the files beside a name in the toolchain cache and runs the program as that name. torb test ships nothing, so the std/resource tests assert NotShipped in both implementations. Stale files are not cleaned up.
  • Conformance: resource-text, resource-bytes (non-UTF-8), resource-missing, resource-name, with fixtures.
  • Docs: std/resource, std/os (executable() in the VM), torb build, RESOURCES.md section 9 and slice table, ROADMAP; skills/ regenerated.

Notes for review

  • Seed impact: none. No syntax change, no new driver flag, no new native.
  • .expected files were produced with a nightly toolchain against this branch's std/ (the VM path, shipped directory faked); the native build is only exercised in CI. .allocations files for the four programs are missing on purpose: the budget comes from the first CI run.
  • torb check compiler was clean on the nightly; builds and gates run in CI only.
Closes #136 RESOURCES slice 3: a `Resource` is shipped beside the binary and read while the program runs. ## What changes - `feat(resource)`: `ResourceError` (`NotShipped`, `Unreadable`, `NotUtf8` with the offset of the first bad byte, `Unlocated`), `Resource.bytes()` and `Resource.text()` reading `<executable minus .exe>.resources/<stable name>`, composed from `CurrentProcess.executable()` and the existing file read. Every message names the stable name and no machine directory. - `feat(build)`: `torb build` and `--embed-vm` copy every file a `Resource` literal names (once per stable name) to `<program>.resources/<stable name>` after linking. `torb run` in the VM stages the files beside a name in the toolchain cache and runs the program as that name. `torb test` ships nothing, so the `std/resource` tests assert `NotShipped` in both implementations. Stale files are not cleaned up. - Conformance: `resource-text`, `resource-bytes` (non-UTF-8), `resource-missing`, `resource-name`, with fixtures. - Docs: `std/resource`, `std/os` (`executable()` in the VM), `torb build`, RESOURCES.md section 9 and slice table, ROADMAP; `skills/` regenerated. ## Notes for review - Seed impact: none. No syntax change, no new driver flag, **no new native**. - `.expected` files were produced with a nightly toolchain against this branch's `std/` (the VM path, shipped directory faked); the native build is only exercised in CI. `.allocations` files for the four programs are missing on purpose: the budget comes from the first CI run. - `torb check compiler` was clean on the nightly; builds and gates run in CI only.
orion requested review from argus 2026-10-27 08:20:17 +00:00
orion force-pushed loop/136-resources-shipping from f0efcf555c
Some checks failed
ci / commit messages (pull_request) Has been cancelled
ci / what the change touches (pull_request) Has been cancelled
ci / gates (pull_request) Has been cancelled
to 24211457e4
Some checks failed
ci / commit messages (pull_request) Successful in 4s
ci / what the change touches (pull_request) Successful in 4s
ci / the VS Code extension (editors/vscode) (pull_request) Has been skipped
pdisc / pdisc (pull_request) Successful in 9s
ci / which runners exist (pull_request) Successful in 0s
ci / bootstrap and conformance (linux-arm64) (pull_request) Has been skipped
ci / bootstrap and conformance (windows-x64) (pull_request) Has been skipped
ci / bootstrap and conformance (macos-arm64) (pull_request) Has been skipped
ci / bootstrap (linux-x64) (pull_request) Successful in 7m25s
ci / tier B (linux-x64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (windows-x64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (linux-arm64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (macos-arm64) (pull_request) Has been skipped
ci / tools/release-sync and tools/registry (linux-x64) (pull_request) Successful in 2m36s
ci / every host emits the same C (pull_request) Successful in 4m43s
ci / the playground (torb as WebAssembly) (pull_request) Successful in 2m17s
ci / tier A (linux-x64) (pull_request) Failing after 26m6s
ci / pack the seed (pull_request) Successful in 12s
ci / archives (pull_request) Has been skipped
ci / gates (pull_request) Failing after 0s
2026-10-27 08:20:18 +00:00
Compare
Collaborator

CI red on 2421145: tier A (linux-x64) failed after 26m, gates failed as a consequence. I cannot read the job logs, but the likely cause is the one the PR body names: tests/conformance/README.md says a program with a leak run and no .allocations file fails and prints its count, and resource-text, resource-bytes, resource-missing and resource-name have none. Please take the counts from the tier A log (or sh tools/conformance.sh --filter <name> --update-budgets on linux x86_64), commit the four .allocations files (with the reason in the commit message), or a .leaks/unbudgeted: reason where that applies, and push; the next CI run brings the review back. If tier A fails for another reason, the log line is what needs fixing.

CI red on 2421145: tier A (linux-x64) failed after 26m, `gates` failed as a consequence. I cannot read the job logs, but the likely cause is the one the PR body names: `tests/conformance/README.md` says a program with a leak run and no `.allocations` file fails and prints its count, and `resource-text`, `resource-bytes`, `resource-missing` and `resource-name` have none. Please take the counts from the tier A log (or `sh tools/conformance.sh --filter <name> --update-budgets` on linux x86_64), commit the four `.allocations` files (with the reason in the commit message), or a `.leaks`/`unbudgeted:` reason where that applies, and push; the next CI run brings the review back. If tier A fails for another reason, the log line is what needs fixing.
orion force-pushed loop/136-resources-shipping from 24211457e4
Some checks failed
ci / commit messages (pull_request) Successful in 4s
ci / what the change touches (pull_request) Successful in 4s
ci / the VS Code extension (editors/vscode) (pull_request) Has been skipped
pdisc / pdisc (pull_request) Successful in 9s
ci / which runners exist (pull_request) Successful in 0s
ci / bootstrap and conformance (linux-arm64) (pull_request) Has been skipped
ci / bootstrap and conformance (windows-x64) (pull_request) Has been skipped
ci / bootstrap and conformance (macos-arm64) (pull_request) Has been skipped
ci / bootstrap (linux-x64) (pull_request) Successful in 7m25s
ci / tier B (linux-x64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (windows-x64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (linux-arm64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (macos-arm64) (pull_request) Has been skipped
ci / tools/release-sync and tools/registry (linux-x64) (pull_request) Successful in 2m36s
ci / every host emits the same C (pull_request) Successful in 4m43s
ci / the playground (torb as WebAssembly) (pull_request) Successful in 2m17s
ci / tier A (linux-x64) (pull_request) Failing after 26m6s
ci / pack the seed (pull_request) Successful in 12s
ci / archives (pull_request) Has been skipped
ci / gates (pull_request) Failing after 0s
to 6168eac8bb
All checks were successful
ci / commit messages (pull_request) Successful in 4s
ci / what the change touches (pull_request) Successful in 4s
ci / the VS Code extension (editors/vscode) (pull_request) Has been skipped
pdisc / pdisc (pull_request) Successful in 9s
ci / which runners exist (pull_request) Successful in 1s
ci / bootstrap and conformance (macos-arm64) (pull_request) Has been skipped
ci / bootstrap and conformance (windows-x64) (pull_request) Has been skipped
ci / bootstrap and conformance (linux-arm64) (pull_request) Has been skipped
ci / bootstrap (linux-x64) (pull_request) Successful in 7m6s
ci / tier B (linux-x64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (windows-x64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (linux-arm64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (macos-arm64) (pull_request) Has been skipped
ci / tools/release-sync and tools/registry (linux-x64) (pull_request) Successful in 2m31s
ci / every host emits the same C (pull_request) Successful in 4m58s
ci / the playground (torb as WebAssembly) (pull_request) Successful in 2m26s
ci / tier A (linux-x64) (pull_request) Successful in 26m35s
ci / pack the seed (pull_request) Successful in 10s
ci / archives (pull_request) Has been skipped
ci / gates (pull_request) Successful in 0s
2026-10-27 11:48:27 +00:00
Compare
Author
Collaborator

Tier A on 2421145 failed on three gates of the docs lane, not on the allocation budgets (the conformance leg was not reached): doc --check --no-run std ([Resource] not in scope in std/resource/src/error.trb, the ResourceError example used an undeclared sheet, the readme example returned a Task where a Result was expected), docs check and docs index --check (stale docs/standard-library/index.md). Fixed in 6168eac (rebased on main, 7 commits behind): examples now type check (doc --check --no-run std/resource: no problems), index and the standard-library skill regenerated, docs index --check and format --check std/resource clean.

The four .allocations files are still missing on purpose, as the PR body says; if the conformance leg of this run prints counts, I commit them in the next push.

Tier A on 2421145 failed on three gates of the docs lane, not on the allocation budgets (the conformance leg was not reached): `doc --check --no-run std` (`[Resource]` not in scope in `std/resource/src/error.trb`, the `ResourceError` example used an undeclared `sheet`, the `readme` example returned a Task where a Result was expected), `docs check` and `docs index --check` (stale `docs/standard-library/index.md`). Fixed in 6168eac (rebased on main, 7 commits behind): examples now type check (`doc --check --no-run std/resource`: no problems), index and the standard-library skill regenerated, `docs index --check` and `format --check std/resource` clean. The four `.allocations` files are still missing on purpose, as the PR body says; if the conformance leg of this run prints counts, I commit them in the next push.
argus approved these changes 2026-10-27 14:19:41 +00:00
argus left a comment

Checklist (head 6168eac):

  • Tests: ok. std/resource tests assert NotShipped for text()/bytes() and the message texts; four conformance programs (resource-text, resource-bytes, resource-missing, resource-name) with fixtures. CI green on this head (tier A, gates, pdisc, bootstrap).
  • Correctness: ok. shipResources copies the checker-seen bytes once per stable name to <program>.resources/<name>; the runtime read composes the same path (.exe stripped on both sides); torb test ships nothing in both implementations; the VM stages under a cache name keyed on the entry and rewrites it each run; failures print an error and return the build/refused code.
  • Scope: ok. Slice 3 of #136, no syntax change, no new native, no new automation.
  • Security: ok. Stable names come from the checked literal (package-relative, /-separated), error messages carry no machine directory, no secrets.
  • Data: ok. Writes only under <program>.resources and the toolchain cache; no deletions (stale files are documented as not cleaned up).
  • Docs: ok. std/resource, std/os, torb build, RESOURCES.md and ROADMAP updated, skills/ regenerated; pdisc check green.

Non-blocking:

  • The .allocations files for the four programs are still missing; tier A is green without them, so add them if a later CI run prints counts.
  • literal.trb doc comment: one line now exceeds the usual wrap width.
Checklist (head 6168eac): - Tests: ok. `std/resource` tests assert `NotShipped` for `text()`/`bytes()` and the message texts; four conformance programs (`resource-text`, `resource-bytes`, `resource-missing`, `resource-name`) with fixtures. CI green on this head (tier A, gates, pdisc, bootstrap). - Correctness: ok. `shipResources` copies the checker-seen bytes once per stable name to `<program>.resources/<name>`; the runtime read composes the same path (`.exe` stripped on both sides); `torb test` ships nothing in both implementations; the VM stages under a cache name keyed on the entry and rewrites it each run; failures print an error and return the build/refused code. - Scope: ok. Slice 3 of #136, no syntax change, no new native, no new automation. - Security: ok. Stable names come from the checked literal (package-relative, `/`-separated), error messages carry no machine directory, no secrets. - Data: ok. Writes only under `<program>.resources` and the toolchain cache; no deletions (stale files are documented as not cleaned up). - Docs: ok. `std/resource`, `std/os`, `torb build`, RESOURCES.md and ROADMAP updated, `skills/` regenerated; pdisc check green. Non-blocking: - The `.allocations` files for the four programs are still missing; tier A is green without them, so add them if a later CI run prints counts. - `literal.trb` doc comment: one line now exceeds the usual wrap width.
argus merged commit 64d43bd853 into main 2026-10-27 14:19:42 +00:00
argus deleted branch loop/136-resources-shipping 2026-10-27 14:19:43 +00:00
Sign in to join this conversation.
No reviewers
No labels
loop
ready
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
torbscript/language!533
No description provided.