resources: shipping (RESOURCES slice 3) #136
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Goal
A
Resource(RESOURCES.md section 4) can be read while the program runs:torb buildships the files thatResourceliterals name tobuild/<profile>/<program>.resources/<stable name>, andResource.bytes()/Resource.text()read them relative to the running program (never the working directory), failing with aResourceError.Source
What exists on main (verified)
std/resourcehasResource,EmbeddedBytes,EmbeddedText; the checker resolves a literal and records the stable name;EmbeddedText.text()reads a nativeembeddedText.CurrentProcess.executable()(runtime/process.c,runtime/platform.c;/proc/self/exe,GetModuleFileNameW,_NSGetExecutablePath,KERN_PROC_PATHNAME). The "medium risk" of the design record (a platform call on each platform) is therefore already paid.ResourceError,Resource.bytes()/text(), any.resources/directory in the build.Scope
One PR, two Conventional Commits:
feat(resource): read a shipped resource beside the program:ResourceError(at least: not shipped, unreadable, not UTF-8 with the offset of the first bad byte, program location unknown),Resource.bytes(): Task<Result<Bytes, ResourceError>>andResource.text(): Task<Result<String, ResourceError>>. The directory is<directory of the executable>/<program file name>.resources/<stable name>,/separators on every platform. Prefer composingCurrentProcess.executable()with the existing file read over a new native; theTaskis already complete when returned (native targets).feat(build): ship the files of Resource literals beside the binary:torb build/runcopy every file aResourceliteral of the checked program names (once per stable name, normalised) to<output>.resources/<stable name>. OnlyResource(notEmbedded*). Stale files are not cleaned up here.Acceptance criteria
Resourceliteral"./data.txt"with.text()prints the file's content aftertorb buildand running the binary from a different working directory, on Linux (CI) and by construction on every platform (no per-platform code beyondCurrentProcess.executable()).build/<profile>/<program>.resources/<owner>/<name>/<path>exists after the build, with the same bytes; two literals reaching one file produce one copy..resources/directory:bytes()returnsErrwith the stable name in the message (name()spelling identical on both implementations), never a panic.text()on a non-UTF-8 file is aResourceErrorcarrying the offset of the first bad byte, never a replacement character..expectedfiles include the stable name): shipped resource read ok, shipped resource missing at run time, binary (non-UTF-8) viabytes(), program printing a resource'sname.std/resourcetests anddocs/standard-library/resource.md(new) updated;docs/tooling/torb-build.mdnames the.resources/layout;skills/regenerated.sh tools/gates.sh agreen in forge CI.Out of scope (other slices)
Lock
settingsresourcelines,--watch, embedded total/budget (#137);ResourceDirectory(#99);Sandbox.load(slices 5 and 6, #138); the web target'sfetch(RESOURCES section 9, "later"); the VM reading from the source tree (milestone 7:torb runbuilds natively today).Seed impact
No syntax change, no new driver flag. Expected no new native (composed from
CurrentProcess.executable()and the file read); if the implementer needs one, say so in the PR body (seed refresh stays Torben's).Size
One PR, two commits; estimate 3-4 h (std/resource + one build step + conformance).
Dependencies
None unmerged. Slices 1 (
std/resource) andCurrentProcess.executable()are onmain.ResourceDirectory(WEB F8) #99ResourceDirectory(WEB F8) #99[loop:torbscript] intake: in scope (author orion, label loop), source RESOURCES.md slice 3, not a duplicate. Run time and platform work (every platform); unblocks #99, #100, #137. No dependencies. next: waiting (refine queued, ready buffer full) until 2026-10-10T04:00:00Z
ResourceDirectory(WEB F8) #99[loop:torbscript] claim refine by session:loop-torbscript-20261027-0718 until 2026-10-27T09:46Z
[loop:torbscript] decided by the loop: not an epic. The design record rates slice 3 medium because "where is my own binary" is a platform call per platform, but
CurrentProcess.executable()already exists on Linux, Windows, macOS and FreeBSD, so the slice reduces to std/resource (ResourceError,bytes(),text()) plus one build copy step: one PR, two commits. Packaging extras (lock lines, watch, budget) stay in #137, the webfetchstays "later" per RESOURCES.md section 9. Seed impact: none expected (no new native planned).[loop:torbscript] refine: body rewritten (goal, testable criteria, design pointer, seed impact no, size 1 PR / 2 commits), label
readyset, claim released, next: implement. Worker usage: roughly 60k tokens.[loop:torbscript] claim implement by session:loop-torbscript-20261027-0718 until 2026-10-27T13:56Z
[loop:torbscript] implement checkpoint (claim kept): branch
loop/136-resources-shippingpushed, no PR yet. Worktreecode/wt/torbscript-language-136-resources-shipping.Done (checked with a nightly
torb check/torb format --check, both clean; no gates or bootstrap run locally):feat(resource): read a shipped resource beside the program:std/resource/src/error.trb(ResourceError:NotShipped,Unreadable,NotUtf8(offset),Unlocated; messages carry the stable name and no machine path),Resource.bytes()/text()composed fromCurrentProcess.executable()+File.readBytes(<exe minus .exe>.resources/<stable name>), no new native.Resourceliteral (CheckedLiteral.source,Checker.shippedResources(): Map<stable name, path>).Finding: the issue text says "
torb runbuilds natively today", which is stale:torb rundefaults to the VM, and the VM answersCurrentProcess.executable()with the entry.trb, so<entry>.trb.resources/would not exist and the conformance VM leg could not read a resource. Plan: incompiler/src/vm/run.trbrunModules, whenshippedResources()is non-empty and the run is not a test run, stage the files under a cache directory (File.copy) and hand the VM that staged path as executable (documented pitfall instd/osprocess docs);torb buildships after a successful link via the same helper (compiler/src/cli/resources.trb,File.copy,File.createDirectorycreates parents).Next steps: (1)
std/resource/tests(Err path: not shipped, name in message), (2)compiler/src/cli/resources.trb+ call inbuildProgramafter link + VM staging, (3) four conformance programs (ok, missing at run time, binary viabytes(),name) with.expectedincl. stable name, resource fixture files, (4)docs/standard-library/resource.md(exists, update),docs/tooling/torb-build.mdlayout,torb docs skill docs skills,tools/check-commits.sh, (5) PRCloses #136, review requestargus, ci-wait.Worker usage: roughly 330k tokens so far.
[loop:torbscript] claim implement (continuation 1, from the checkpoint of comment 7441, pinned on anthropic:notahumanmate) by session:loop-torbscript-20261027-0718 until 2026-10-27T14:06Z
[loop:torbscript] implement checkpoint 2 (claim kept, continuation 1 ran out of context): branch
loop/136-resources-shippingpushed (3 commits: feat(resource), wip checker records, wip shipping+staging+conformance), no PR yet. Worktreecode/wt/torbscript-language-136-resources-shipping.Done since checkpoint 1 (only formatted with a nightly
torb format;torb check compilerwas started in the background, log/tmp/rs-136/check-compiler.log, result not yet read):compiler/src/cli/resources.trb:shipResources(writes<program minus .exe>.resources/<stable name>from the bytes the checker saw,workspace.resource(path)+File.writeBytes),stageResources(VM:<cache>/resources/<hash of entry>/<entry name>as the name the program runs under),resourceDirectoryOf.cli/build.trbbuildProgram:checker.shippedResources()after lowering, ship after a successful link, not foroptions.testing(a test run ships nothing in VM and native, so thestd/resourcetests can assert theNotShippedpath in both).cli/embed.trb(--embed-vm) ships beside its binary too.vm/run.trb:bytecodeOf(no caller left) removed;runModulesusesmachineLoweringOf,MachineLowering.resourcesadded, stages when!isTestRunand resources exist and hands the staged path torunProgramas executable.std/resource/tests/resource.test.trb: NotShipped fortext()andbytes()(name in message),show()of all four errors.tests/conformance/resource-{name,text,bytes,missing}.trb+ fixturesresource-files/{note.txt,pattern.bin};.expected/.exitNOT written yet.Next steps: (1) read check-compiler.log and fix errors (watch:
Mapiterationfor (name, path) in resources,File.createDirectorycreating parents,shipResourcesreturnResult<Int, String>); (2) verify the conformance programs' logic with the nightly by faking the shipped dir (<entry>.trb.resources/<stable name>), write.expected(stable name istorbscript/conformance/resource-files/note.txtif the package directory resolves that way, check) and.exit(0);.allocationscannot be measured locally: push without, read the count from the redconformancejob, add them; (3) docs:docs/standard-library/resource.md(ResourceError, bytes/text, shipping; drop "next two slices"),docs/tooling/torb-build.md(<program>.resources/layout, tests ship nothing,--embed-vmships),docs/standard-library/os.md/std/osprocess.trbexecutable()doc (VM answers the staged path for a program with Resource literals),docs/design/RESOURCES.mdsection 9 VM paragraph (stale "torb run builds natively") + slice 3 row Done,docs/ROADMAP.mdresources row; thenbuild/release/torb docs skill docs skillsis a gate (CI only): regenerate with nightlytorb docs skill docs skillsif it works, else expect the CI red to show the diff; (4) squash the two wip commits into commit 2 (feat(build): ship the files of Resource literals beside the binary, <=100 chars),tools/check-commits.sh; (5) PRCloses #136, reviewargus, one ci-wait.Worker usage: roughly 620k tokens so far (330k + ~290k this continuation).
[loop:torbscript] implement checkpoint 2 addendum:
torb check compiler(nightly) is clean (410 files, no problems) after a fourth wip commit (const _ =on the staged file count); branch pushed. Remaining are steps (2)-(5) above (4 wip commits to squash into commit 2).[loop:torbscript] implement: PR open #533 (
Closes #136, review requested from argus). Branch rebased on main and squashed into two commits (feat(resource), feat(build));tools/check-commits.shclean,.expected/.exitof the four conformance programs written (checked with the nightly VM against this branch's std), docs andskills/regenerated (second regeneration after the rebase: no diff). Forge CI of24211457is running;.allocationsof the four programs are added from the CI output if that is all that is red. Claim kept. Worker usage so far roughly 0.8M tokens.[loop:torbscript] release implement: the continuation worker timed out while waiting for CI; PR #533 (head
24211457, review requested from argus) carries all its work. next: waiting (CI on PR #533); red CI (e.g. missing.allocationsof the four conformance programs) gets a fix worker from the PR state.