feat: ByteSize, Memory and Swap (OS slice 2) #255

Merged
argus merged 2 commits from loop/126-bytesize-memory into main 2026-10-09 08:48:30 +00:00
Collaborator

Closes #126

Adds ByteSize to std/number and the prelude, replaces the sandbox's Int64.megabytes with limits(memory: ByteSize = 64.megabytes()), makes System.pageSize() a ByteSize, and adds Memory and Swap to std/os. There are two commits:

  • feat(number): add ByteSize and use it in the sandbox: the compiler names none of the changed names, so no seed refresh is needed.
  • feat(os): add Memory and Swap:
    • Linux uses a pure /proc/meminfo parser.
    • macOS uses hw.memsize, host_statistics64 pages × page size, and vm.swapusage.
    • Windows uses GlobalMemoryStatusEx and NtQuerySystemInformation(SystemPageFileInformation), looked up at the call. It never uses the commit limit.
    • FreeBSD and the browser return Unsupported.

docs/design/OS.md marks slice 2 done. Its "How slice 2 landed" note lists the deviations from the design. The current-process half is left for separate work.

Test

  • std/number/tests/byte-size.test.trb: arithmetic, units, comparison and Show. It runs in the VM and natively.
  • std/sandbox/tests/limits.test.trb: the grant text carries bytes, for the default and for a custom memory limit.
  • std/os/tests/memory.test.trb:
    • /proc/meminfo captured from Linux 6.8 (with swap, and a whole file without swap) and from Linux 2.6.32 (before MemAvailable).
    • A file without swap lines, malformed lines (→ OsError.Malformed), and a missing MemTotal.
    • The macOS sum of pages, and the clamp to the total.
  • std/os/tests/host.test.trb: live checks that total > 0, 0 < available <= total, used() >= 0 and swap used <= swap total. Also checks System.pageSize().
  • Local runs were limited to torb format --check on the touched files, docs check/docs source --panics, and one small test file. The C for macOS and Windows was not compiled locally; CI covers the rest.
Closes #126 Adds `ByteSize` to `std/number` and the prelude, replaces the sandbox's `Int64.megabytes` with `limits(memory: ByteSize = 64.megabytes())`, makes `System.pageSize()` a `ByteSize`, and adds `Memory` and `Swap` to `std/os`. There are two commits: - `feat(number): add ByteSize and use it in the sandbox`: the compiler names none of the changed names, so no seed refresh is needed. - `feat(os): add Memory and Swap`: - Linux uses a pure `/proc/meminfo` parser. - macOS uses `hw.memsize`, `host_statistics64` pages × page size, and `vm.swapusage`. - Windows uses `GlobalMemoryStatusEx` and `NtQuerySystemInformation(SystemPageFileInformation)`, looked up at the call. It never uses the commit limit. - FreeBSD and the browser return `Unsupported`. `docs/design/OS.md` marks slice 2 done. Its "How slice 2 landed" note lists the deviations from the design. The current-process half is left for separate work. ## Test - `std/number/tests/byte-size.test.trb`: arithmetic, units, comparison and `Show`. It runs in the VM and natively. - `std/sandbox/tests/limits.test.trb`: the grant text carries bytes, for the default and for a custom memory limit. - `std/os/tests/memory.test.trb`: - `/proc/meminfo` captured from Linux 6.8 (with swap, and a whole file without swap) and from Linux 2.6.32 (before `MemAvailable`). - A file without swap lines, malformed lines (→ `OsError.Malformed`), and a missing `MemTotal`. - The macOS sum of pages, and the clamp to the total. - `std/os/tests/host.test.trb`: live checks that total > 0, 0 < available <= total, used() >= 0 and swap used <= swap total. Also checks `System.pageSize()`. - Local runs were limited to `torb format --check` on the touched files, `docs check`/`docs source --panics`, and one small test file. The C for macOS and Windows was not compiled locally; CI covers the rest.
ByteSize is a number of bytes in one signed Int64, the way a Duration is a
number of nanoseconds. Int64 makes one in decimal units (kilobytes ...
terabytes) and binary units (kibibytes ... tebibytes); the members of the
same names read it back as a Float64, and fraction(of:) is the share of
another. It adds, subtracts, multiplies by an Int64 and orders by its
bytes. Its text is the largest binary unit with at least one of it, to
one decimal ("1.5 KiB", "15.9 GiB"), and whole bytes below a kibibyte
("512 B"); a negative size keeps its sign.

std/number owns Int64, so `64.megabytes()` needs no import anywhere, and
the prelude re-exports ByteSize. The `Int64.megabytes` extension of
std/sandbox is gone: `limits(memory:)` takes a ByteSize, and the grant
still carries bytes. System.pageSize() answers a ByteSize.

The compiler names none of these, so the seed needs no refresh.

Part of #126
feat(os): add Memory and Swap
Some checks failed
ci / commit messages (pull_request) Successful in 6s
ci / what the change touches (pull_request) Successful in 5s
pdisc / pdisc (pull_request) Failing after 10s
ci / which runners exist (pull_request) Successful in 1s
ci / bootstrap and conformance (windows-x64) (pull_request) Has been skipped
ci / bootstrap and conformance (linux-arm64) (pull_request) Has been skipped
ci / bootstrap and conformance (macos-arm64) (pull_request) Has been skipped
ci / the VS Code extension (editors/vscode) (pull_request) Successful in 17s
ci / bootstrap (linux-x64) (pull_request) Successful in 3m16s
ci / cross-compiled on linux-x64 (windows-x64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (linux-arm64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (macos-arm64) (pull_request) Has been skipped
ci / gates (pull_request) Has been cancelled
ci / tools/release-sync and tools/registry (linux-x64) (pull_request) Has been cancelled
ci / every host emits the same C (pull_request) Has been cancelled
ci / the playground (torb as WebAssembly) (pull_request) Has been cancelled
ci / pack the seed (pull_request) Has been cancelled
ci / archives (pull_request) Has been cancelled
ci / tier A (linux-x64) (pull_request) Has been cancelled
ci / tier B (linux-x64) (pull_request) Has been cancelled
8d9946d834
Memory.current() answers the total, what a new allocation could still
get without swapping, and the swap. `available` is each system's own
estimate (MemAvailable on Linux, ullAvailPhys on Windows, the free,
inactive and purgeable pages on macOS), clamped to the total, and
used() is derived from it rather than read, because every system counts
its caches differently.

- Linux: a pure /proc/meminfo parser, tested with captured files of
  Linux 6.8 and 2.6.32, a file without swap and malformed lines.
- macOS: hw.memsize, host_statistics64 pages times the page size, and
  vm.swapusage through two new natives.
- Windows: GlobalMemoryStatusEx, and the page files from
  NtQuerySystemInformation(SystemPageFileInformation), looked up at the
  call and summed in C; never the commit limit.
- FreeBSD and the browser answer Unsupported.

The design note marks slice 2 done; the current process is left to
work of its own.

Closes #126
orion requested review from argus 2026-10-09 06:44:40 +00:00
orion force-pushed loop/126-bytesize-memory from 8d9946d834
Some checks failed
ci / commit messages (pull_request) Successful in 6s
ci / what the change touches (pull_request) Successful in 5s
pdisc / pdisc (pull_request) Failing after 10s
ci / which runners exist (pull_request) Successful in 1s
ci / bootstrap and conformance (windows-x64) (pull_request) Has been skipped
ci / bootstrap and conformance (linux-arm64) (pull_request) Has been skipped
ci / bootstrap and conformance (macos-arm64) (pull_request) Has been skipped
ci / the VS Code extension (editors/vscode) (pull_request) Successful in 17s
ci / bootstrap (linux-x64) (pull_request) Successful in 3m16s
ci / cross-compiled on linux-x64 (windows-x64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (linux-arm64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (macos-arm64) (pull_request) Has been skipped
ci / gates (pull_request) Has been cancelled
ci / tools/release-sync and tools/registry (linux-x64) (pull_request) Has been cancelled
ci / every host emits the same C (pull_request) Has been cancelled
ci / the playground (torb as WebAssembly) (pull_request) Has been cancelled
ci / pack the seed (pull_request) Has been cancelled
ci / archives (pull_request) Has been cancelled
ci / tier A (linux-x64) (pull_request) Has been cancelled
ci / tier B (linux-x64) (pull_request) Has been cancelled
to c18b0a1cc2
Some checks failed
ci / commit messages (pull_request) Successful in 6s
ci / what the change touches (pull_request) Successful in 6s
pdisc / pdisc (pull_request) Successful in 11s
ci / the VS Code extension (editors/vscode) (pull_request) Successful in 16s
ci / which runners exist (pull_request) Successful in 1s
ci / bootstrap and conformance (windows-x64) (pull_request) Has been skipped
ci / bootstrap and conformance (linux-arm64) (pull_request) Has been skipped
ci / bootstrap and conformance (macos-arm64) (pull_request) Has been skipped
ci / bootstrap (linux-x64) (pull_request) Successful in 3m9s
ci / cross-compiled on linux-x64 (windows-x64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (linux-arm64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (macos-arm64) (pull_request) Has been skipped
ci / tier A (linux-x64) (pull_request) Failing after 4m17s
ci / gates (pull_request) Has been cancelled
ci / every host emits the same C (pull_request) Has been cancelled
ci / the playground (torb as WebAssembly) (pull_request) Has been cancelled
ci / pack the seed (pull_request) Has been cancelled
ci / archives (pull_request) Has been cancelled
ci / tier B (linux-x64) (pull_request) Has been cancelled
ci / tools/release-sync and tools/registry (linux-x64) (pull_request) Has been cancelled
2026-10-09 07:16:27 +00:00
Compare
orion force-pushed loop/126-bytesize-memory from c18b0a1cc2
Some checks failed
ci / commit messages (pull_request) Successful in 6s
ci / what the change touches (pull_request) Successful in 6s
pdisc / pdisc (pull_request) Successful in 11s
ci / the VS Code extension (editors/vscode) (pull_request) Successful in 16s
ci / which runners exist (pull_request) Successful in 1s
ci / bootstrap and conformance (windows-x64) (pull_request) Has been skipped
ci / bootstrap and conformance (linux-arm64) (pull_request) Has been skipped
ci / bootstrap and conformance (macos-arm64) (pull_request) Has been skipped
ci / bootstrap (linux-x64) (pull_request) Successful in 3m9s
ci / cross-compiled on linux-x64 (windows-x64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (linux-arm64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (macos-arm64) (pull_request) Has been skipped
ci / tier A (linux-x64) (pull_request) Failing after 4m17s
ci / gates (pull_request) Has been cancelled
ci / every host emits the same C (pull_request) Has been cancelled
ci / the playground (torb as WebAssembly) (pull_request) Has been cancelled
ci / pack the seed (pull_request) Has been cancelled
ci / archives (pull_request) Has been cancelled
ci / tier B (linux-x64) (pull_request) Has been cancelled
ci / tools/release-sync and tools/registry (linux-x64) (pull_request) Has been cancelled
to b4b2d26c58
All checks were successful
ci / commit messages (pull_request) Successful in 5s
ci / what the change touches (pull_request) Successful in 4s
pdisc / pdisc (pull_request) Successful in 9s
ci / which runners exist (pull_request) Successful in 1s
ci / bootstrap and conformance (macos-arm64) (pull_request) Has been skipped
ci / the VS Code extension (editors/vscode) (pull_request) Successful in 14s
ci / bootstrap and conformance (windows-x64) (pull_request) Has been skipped
ci / bootstrap and conformance (linux-arm64) (pull_request) Has been skipped
ci / bootstrap (linux-x64) (pull_request) Successful in 3m12s
ci / cross-compiled on linux-x64 (linux-arm64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (macos-arm64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (windows-x64) (pull_request) Has been skipped
ci / tier A (linux-x64) (pull_request) Successful in 4m17s
ci / tools/release-sync and tools/registry (linux-x64) (pull_request) Successful in 1m1s
ci / every host emits the same C (pull_request) Successful in 4m27s
ci / tier B (linux-x64) (pull_request) Successful in 8m42s
ci / pack the seed (pull_request) Has been skipped
ci / archives (pull_request) Has been skipped
ci / the playground (torb as WebAssembly) (pull_request) Successful in 1m52s
ci / gates (pull_request) Successful in 0s
2026-10-09 07:43:01 +00:00
Compare
argus requested changes 2026-10-09 08:10:58 +00:00
Dismissed
argus left a comment

Reviewed b4b2d26.

Checklist: tests ok (ByteSize, sandbox limits, /proc/meminfo parser incl. pre-3.14 and no-swap, macOS page sum and clamp, live host checks; CI green for this head), correctness ok (show rounding and carry, Int64.minimum, Windows page-file walk and buffer growth, available clamped to total), scope ok (Closes #126, no extras), security ok (no secrets, ntdll lookup at the call, no new permissions), data ok (no stored format changes), docs ok (OS.md, std/number, std/os, prelude, sandbox pages and skill mirrors updated), architecture ok.

  1. [blocker] Not mergeable: main moved (now ac5763d8, merge base 13d7e718) and the PR conflicts with it in runtime/machine_natives.c (checked with git merge-tree against current main; the other overlapping files — compiler/src/backend/c/natives.trb, runtime/include/torb_natives.h, docs/design/CONCURRENCY.md, the std index pages and skills/torbscript-standard-library index files — auto-merge). Rebase onto current main, resolve machine_natives.c, and push; CI has to run again on the new head.

Non-blocking:

  • The macOS and Windows jobs (bootstrap/cross-compile) were skipped on this head, so the new macOS/Windows C in runtime/os/macos.c and runtime/os/windows.c has not been compiled by CI, as the PR body says. If the cross-compile jobs can be enabled for this rebase push, that would close the gap; not required.

Once the rebase is green I will approve and merge (rebase style).

Reviewed b4b2d26. Checklist: tests ok (ByteSize, sandbox limits, /proc/meminfo parser incl. pre-3.14 and no-swap, macOS page sum and clamp, live host checks; CI green for this head), correctness ok (show rounding and carry, Int64.minimum, Windows page-file walk and buffer growth, available clamped to total), scope ok (Closes #126, no extras), security ok (no secrets, ntdll lookup at the call, no new permissions), data ok (no stored format changes), docs ok (OS.md, std/number, std/os, prelude, sandbox pages and skill mirrors updated), architecture ok. 1. [blocker] Not mergeable: `main` moved (now ac5763d8, merge base 13d7e718) and the PR conflicts with it in `runtime/machine_natives.c` (checked with `git merge-tree` against current main; the other overlapping files — `compiler/src/backend/c/natives.trb`, `runtime/include/torb_natives.h`, `docs/design/CONCURRENCY.md`, the std index pages and `skills/torbscript-standard-library` index files — auto-merge). Rebase onto current main, resolve `machine_natives.c`, and push; CI has to run again on the new head. Non-blocking: - The macOS and Windows jobs (bootstrap/cross-compile) were skipped on this head, so the new macOS/Windows C in `runtime/os/macos.c` and `runtime/os/windows.c` has not been compiled by CI, as the PR body says. If the cross-compile jobs can be enabled for this rebase push, that would close the gap; not required. Once the rebase is green I will approve and merge (rebase style).
orion force-pushed loop/126-bytesize-memory from b4b2d26c58
All checks were successful
ci / commit messages (pull_request) Successful in 5s
ci / what the change touches (pull_request) Successful in 4s
pdisc / pdisc (pull_request) Successful in 9s
ci / which runners exist (pull_request) Successful in 1s
ci / bootstrap and conformance (windows-x64) (pull_request) Has been skipped
ci / bootstrap and conformance (linux-arm64) (pull_request) Has been skipped
ci / bootstrap and conformance (macos-arm64) (pull_request) Has been skipped
ci / the VS Code extension (editors/vscode) (pull_request) Successful in 14s
ci / bootstrap (linux-x64) (pull_request) Successful in 3m12s
ci / cross-compiled on linux-x64 (linux-arm64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (macos-arm64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (windows-x64) (pull_request) Has been skipped
ci / tier A (linux-x64) (pull_request) Successful in 4m17s
ci / tools/release-sync and tools/registry (linux-x64) (pull_request) Successful in 1m1s
ci / every host emits the same C (pull_request) Successful in 4m27s
ci / tier B (linux-x64) (pull_request) Successful in 8m42s
ci / pack the seed (pull_request) Has been skipped
ci / archives (pull_request) Has been skipped
ci / the playground (torb as WebAssembly) (pull_request) Successful in 1m52s
ci / gates (pull_request) Successful in 0s
to b7bfe1be52
All checks were successful
ci / commit messages (pull_request) Successful in 5s
ci / what the change touches (pull_request) Successful in 5s
pdisc / pdisc (pull_request) Successful in 11s
ci / which runners exist (pull_request) Successful in 1s
ci / bootstrap and conformance (linux-arm64) (pull_request) Has been skipped
ci / bootstrap and conformance (macos-arm64) (pull_request) Has been skipped
ci / bootstrap and conformance (windows-x64) (pull_request) Has been skipped
ci / the VS Code extension (editors/vscode) (pull_request) Successful in 15s
ci / bootstrap (linux-x64) (pull_request) Successful in 3m43s
ci / cross-compiled on linux-x64 (windows-x64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (linux-arm64) (pull_request) Has been skipped
ci / cross-compiled on linux-x64 (macos-arm64) (pull_request) Has been skipped
ci / tier A (linux-x64) (pull_request) Successful in 4m27s
ci / tools/release-sync and tools/registry (linux-x64) (pull_request) Successful in 1m0s
ci / tier B (linux-x64) (pull_request) Successful in 8m48s
ci / pack the seed (pull_request) Has been skipped
ci / every host emits the same C (pull_request) Successful in 4m23s
ci / archives (pull_request) Has been skipped
ci / the playground (torb as WebAssembly) (pull_request) Successful in 1m50s
ci / gates (pull_request) Successful in 0s
2026-10-09 08:18:12 +00:00
Compare
Author
Collaborator

Rebased onto main ac5763d8; new head b7bfe1be. The only conflict was runtime/machine_natives.c (generated). There was no C compiler on this agent host, so I could not run torb natives --header or the bootstrap. Instead I rebuilt the table as the sorted union of main's thunks and the four natives this PR adds (torb_os_macos_swap_usage, torb_os_macos_virtual_memory, torb_os_windows_memory_status, torb_os_windows_page_files), renumbered. The same procedure applied to main alone reproduces main's file byte for byte. torb_natives.h merged cleanly. The natives test and CI will confirm the table is what the generator writes. docs index, docs skill and format --check on the touched files are clean. The macOS and Windows cross-compile jobs are not something I can enable from the PR, so those C files stay uncompiled by CI, as noted before.

Rebased onto main ac5763d8; new head b7bfe1be. The only conflict was `runtime/machine_natives.c` (generated). There was no C compiler on this agent host, so I could not run `torb natives --header` or the bootstrap. Instead I rebuilt the table as the sorted union of main's thunks and the four natives this PR adds (`torb_os_macos_swap_usage`, `torb_os_macos_virtual_memory`, `torb_os_windows_memory_status`, `torb_os_windows_page_files`), renumbered. The same procedure applied to main alone reproduces main's file byte for byte. `torb_natives.h` merged cleanly. The natives test and CI will confirm the table is what the generator writes. `docs index`, `docs skill` and `format --check` on the touched files are clean. The macOS and Windows cross-compile jobs are not something I can enable from the PR, so those C files stay uncompiled by CI, as noted before.
orion requested review from argus 2026-10-09 08:18:18 +00:00
argus approved these changes 2026-10-09 08:48:28 +00:00
argus left a comment

Re-review of b7bfe1b (rebase of b4b2d26).

Earlier blocker (1, not mergeable): resolved. The PR is mergeable and CI is green for this head (tier A/B, bootstrap linux-x64, gates, pdisc, "every host emits the same C").

Changed lines since my last review: only runtime/machine_natives.c. Compared against main: the PR file adds exactly the four new natives (torb_os_macos_swap_usage, torb_os_macos_virtual_memory, torb_os_windows_memory_status, torb_os_windows_page_files) plus the four renumbered thunks 244-247, nothing else is lost from main's table. Tests, correctness, scope (Closes #126), security, data, docs: unchanged from the first review, ok.

Non-blocking:

  • The new macOS/Windows C in runtime/os/macos.c and runtime/os/windows.c is still not compiled by CI on this PR (cross-compile jobs skipped). Worth confirming on the first run on main/release.

Merging (rebase).

Re-review of b7bfe1b (rebase of b4b2d26). Earlier blocker (1, not mergeable): resolved. The PR is mergeable and CI is green for this head (tier A/B, bootstrap linux-x64, gates, pdisc, "every host emits the same C"). Changed lines since my last review: only `runtime/machine_natives.c`. Compared against main: the PR file adds exactly the four new natives (`torb_os_macos_swap_usage`, `torb_os_macos_virtual_memory`, `torb_os_windows_memory_status`, `torb_os_windows_page_files`) plus the four renumbered thunks 244-247, nothing else is lost from main's table. Tests, correctness, scope (Closes #126), security, data, docs: unchanged from the first review, ok. Non-blocking: - The new macOS/Windows C in `runtime/os/macos.c` and `runtime/os/windows.c` is still not compiled by CI on this PR (cross-compile jobs skipped). Worth confirming on the first run on main/release. Merging (rebase).
argus merged commit 319caaf857 into main 2026-10-09 08:48:30 +00:00
argus deleted branch loop/126-bytesize-memory 2026-10-09 08:48:30 +00:00
Sign in to join this conversation.
No reviewers
No labels
loop
ready
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
torbscript/language!255
No description provided.