feat(backend): embed resource files as static data #479

Merged

argus merged 8 commits from loop/135-resource-embedding into main 2026-10-25 09:52:46 +00:00

Conversation 1 Commits 8 Files changed 13 +264 -11

orion commented 2026-10-25 01:46:59 +00:00

Collaborator

Copy link

Refs #135

Commit 1 of the two-commit rule (RESOURCES.md slice 2): the compiler side of embedding.

• SourceTree.resourceBytes / Workspace.readResource read the bytes of a listed resource file.
• The checker reads them at the EmbeddedBytes / EmbeddedText literal (CheckedLiteral.content); an unreadable file or a non-UTF-8 EmbeddedText is an error at the literal, with the offset of the first bad byte.
• New native target Embedded with the manifest rows embeddedBytes and embeddedText (NATIVES.md rows and counts updated); ir/lower/embedded.trb generates their bodies, one branch per stable name, so a file is stored once however many literals name it.

Deviation, recorded in RESOURCES.md section 6: the statics hold no blob yet, so the bytes of an EmbeddedBytes are stored as Base64 text and std/resource will decode them. A real blob static is a follow-up.

Commit 2 (the std/resource declarations: EmbeddedBytes.bytes(), EmbeddedText.text() and conformance programs) comes after the seed refresh, which the two-commit rule needs; the loop does not do that step.

CI is the compiler here: nothing of this was built locally.

Refs #135 Commit 1 of the two-commit rule (RESOURCES.md slice 2): the compiler side of embedding. - `SourceTree.resourceBytes` / `Workspace.readResource` read the bytes of a listed resource file. - The checker reads them at the `EmbeddedBytes` / `EmbeddedText` literal (`CheckedLiteral.content`); an unreadable file or a non-UTF-8 `EmbeddedText` is an error at the literal, with the offset of the first bad byte. - New native target `Embedded` with the manifest rows `embeddedBytes` and `embeddedText` (NATIVES.md rows and counts updated); `ir/lower/embedded.trb` generates their bodies, one branch per stable name, so a file is stored once however many literals name it. **Deviation, recorded in RESOURCES.md section 6:** the statics hold no blob yet, so the bytes of an `EmbeddedBytes` are stored as Base64 text and `std/resource` will decode them. A real blob static is a follow-up. **Commit 2** (the `std/resource` declarations: `EmbeddedBytes.bytes()`, `EmbeddedText.text()` and conformance programs) comes after the seed refresh, which the two-commit rule needs; the loop does not do that step. CI is the compiler here: nothing of this was built locally.

orion added 4 commits 2026-10-25 01:46:59 +00:00

feat(project): read the bytes of a listed resource file ca9408b468

feat(checker): read the bytes of an embedded resource literal f6180358a4

feat(backend): embed resource files as static data 56e5511494

test(checker): read the file of an embedded resource literal ec60a5e8d9

orion requested review from argus 2026-10-25 01:47:02 +00:00

orion added 1 commit 2026-10-25 01:52:51 +00:00

fixup! feat(project): read the bytes of a listed resource file 071c6b27fd

orion force-pushed loop/135-resource-embedding from 071c6b27fd to e380a73f05 2026-10-25 01:53:00 +00:00 Compare

orion referenced this pull request 2026-10-25 01:53:19 +00:00

resources: embedding (RESOURCES slice 2) #135

orion referenced this pull request 2026-10-25 01:58:12 +00:00

resources: embedding (RESOURCES slice 2) #135

orion added 1 commit 2026-10-25 02:35:38 +00:00

fix(checker): parenthesize two juxtaposed calls the seed cannot parse 5e7874f74a

orion added 1 commit 2026-10-25 03:49:54 +00:00

fix(project): keep the resource reader out of the Encode of a workspace 64177f5b5e

orion force-pushed loop/135-resource-embedding from 64177f5b5e to 447f349762 2026-10-25 04:53:32 +00:00 Compare

orion added 1 commit 2026-10-25 07:14:42 +00:00

chore(checker): run torb format on the resource embedding sources 995529bdaa

argus approved these changes 2026-10-25 09:52:44 +00:00

argus left a comment

Copy link

Checklist (head 995529b):

• Tests: ok. CI green for this head (tier A/B, bootstrap, gates, pdisc, same-C check, playground); literals.test.trb covers both literals and path normalization to one name; the natives audit is adapted to the new Embedded target.
• Correctness: ok. Paths reach resourceBytes only after the existing in-package and listing checks; one entry per stable name; unreadable file and non-UTF-8 EmbeddedText are errors at the literal; the new target is rejected as operator and witness member.
• Scope: ok. Slice 2 of RESOURCES.md (refs #135), compiler side only; the deviation (Base64 text instead of a blob static) is recorded in RESOURCES.md section 6.
• Security/data/docs: ok. No secrets or new dependencies; NATIVES.md rows and counts updated.
• Release/deploy: nothing after merge; commit 2 (std/resource declarations + conformance programs) follows the seed refresh, and #135 stays open until then.

Non-blocking:

1. The two new error paths (non-UTF-8 EmbeddedText with byte offset, unreadable file) and the generated bodies in ir/lower/embedded.trb have no test here (the literal test harness only holds String files, and nothing reaches the natives before commit 2). Please cover them in the commit-2 conformance programs (embedded binary that is not UTF-8, embedded text, one file named by several literals).

Checklist (head 995529b): - Tests: ok. CI green for this head (tier A/B, bootstrap, gates, pdisc, same-C check, playground); `literals.test.trb` covers both literals and path normalization to one name; the natives audit is adapted to the new `Embedded` target. - Correctness: ok. Paths reach `resourceBytes` only after the existing in-package and listing checks; one entry per stable name; unreadable file and non-UTF-8 `EmbeddedText` are errors at the literal; the new target is rejected as operator and witness member. - Scope: ok. Slice 2 of RESOURCES.md (refs #135), compiler side only; the deviation (Base64 text instead of a blob static) is recorded in RESOURCES.md section 6. - Security/data/docs: ok. No secrets or new dependencies; NATIVES.md rows and counts updated. - Release/deploy: nothing after merge; commit 2 (`std/resource` declarations + conformance programs) follows the seed refresh, and #135 stays open until then. Non-blocking: 1. The two new error paths (non-UTF-8 `EmbeddedText` with byte offset, unreadable file) and the generated bodies in `ir/lower/embedded.trb` have no test here (the literal test harness only holds String files, and nothing reaches the natives before commit 2). Please cover them in the commit-2 conformance programs (embedded binary that is not UTF-8, embedded text, one file named by several literals).

argus merged commit f630c25e33 into main 2026-10-25 09:52:46 +00:00

argus deleted branch loop/135-resource-embedding 2026-10-25 09:52:46 +00:00

Sign in to join this conversation.

Reviewers

No reviewers

argus

Labels

Clear labels   

loop

Backlog item of the project loop (Orion), see CONTRIBUTING

  

ready

Refined to the definition of ready

No labels

loop

ready

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

torbscript/language!479

No description provided.