os: Frequency, processor counts, times and usage (OS slice 3, step 1) #488
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Goal
First step of #127 (OS slice 3, processors):
Frequency, the processor counts, the processor times andProcessor.usage, on every supported system.describewith thex86/decoders and the frequencies are the two later steps of the epic.Source
docs/design/OS.mdsection 4 (Frequency), section 5.2 (Processor,ProcessorTimes,ProcessorUsage), section 7 (natives per system), section 10 slice 3.Scope
Frequencyinstd/os(ofHertz,ofMegahertz,hertz,megahertz,gigahertz,showas3.6 GHz/800 MHz,Show,Compare).Processor.logicalCount(),physicalCount(),times(),timesOfEach(),usage(interval),ProcessorTimes(withSubtractandtotal()),ProcessorUsage(withbetween)./proc/statandsysconf/sysfs topology, pure parsers), Windows (GetLogicalProcessorInformationEx,GetSystemTimes,NtQuerySystemInformationlooked up at the call), macOS (host_statistics,host_processor_info,hw.physicalcpu), POSIX/BSD where they share; FreeBSD and the browser answerUnsupportedas in slice 2.usageis TorbScript overtimesandsleep, as written in OS.md 5.2.Acceptance criteria
Frequencyconversions andshoware tested (800 MHz,3.6 GHz, rounding, ordering).Processor.logicalCount()andphysicalCount()answerOkon Linux, Windows and macOS, withphysicalCount() <= logicalCount()andlogicalCount() >= 1.times()andtimesOfEach()answer cumulative times; two readings are monotonic;timesOfEach().length == logicalCount();ProcessorTimes.total()is the sum of the five fields./proc/statparser (one aggregate line, one line per processor,iowait,nice,irq,stealfolded intoother, clock ticks toDurationviaCLK_TCK) is pure and tested on every machine with captured inputs, including a short and a malformed line (Err, no panic).ProcessorUsage.betweenis tested with fixed differences: fractions are in 0..1,busy == user + systemwithin rounding, and a zero-length difference answers 0, not NaN.usage(interval)is aTaskthat sleepsintervalbetween two readings; a test runs it with a short interval and checks the ranges.OsError.Unsupported;torb check --every-targetis clean.docs/standard-library/os.mddocuments the new API;docs checkclean;skills/regenerated.tools/gates.sh agreen in forge CI.Seed impact
None expected: new std names and new natives in the manifest, no syntax change, no new driver flag, and the compiler looks up no name of this step by string and does not import
Processor. If CI proves otherwise, the PR uses the two-commit seed rule and says so.Size
One PR, two Conventional Commits at most:
feat(os): Frequency and processor countsandfeat(os): processor times and usage, or one commit if small.Dependencies
#126 is closed (landed). Part of the epic #127; the next steps (
describewithx86/, then frequencies and the slice's done mark) are filed when this one lands.[loop:torbscript] claim implement by session:loop-torbscript-20261026 until 2026-10-26T08:00:00Z
[loop:torbscript] implement: checkpoint, no code pushed yet (the worker's reading budget ran out on orientation), next: implement continues in a fresh worker. Approx. tokens: 80k (reading only).
Findings for the next worker (so it can write code at once):
319caaf8(feat(os): add Memory and Swap). Touches:std/os/src/memory.trb(public type +match OperatingSystem.currentarms,.FreeBsd/.Browser=>Fail OsError.Unsupported(...)),std/os/src/{linux,macos,windows}/memory.trb,{macos,windows}/native.trb,std/os/src/lib.trb(public use + doc list),std/os/tests/*.test.trb,compiler/src/backend/c/natives.trb(manifest rows viasystemRuntimeOf(...), per-system guards),runtime/include/torb_os.h,runtime/os/{macos,windows}.c,docs/standard-library/os.md,docs/design/OS.md(slice mark), regeneratedskills/./proc/statthroughreadSystemFile(std/os/src/linux/system.trb) with a pure public parser (seeparseMemory);CLK_TCKviaPosix.configuration("CLK_TCK"), logical count viaconfiguration("NPROCESSORS_ONLN")or thecpuNlines, physical count from/sys/devices/system/cpu/cpuN/topology/{core_id,physical_package_id}(needslistSystemDirectory, not in the Linux natives yet; alternative: parse/proc/cpuinfophysical id/core id, whichreadSystemFilecan read).Bsd.sysctlInteger("hw.physicalcpu"/"hw.logicalcpu")(std/os/src/bsd/system.trb), times via newMacOs.processorLoad/processorLoadOfEach(OS.md section 7).Windows.processorTopology/systemTimes/processorTimesOfEach(OS.md section 7),NtQuerySystemInformationlooked up at the call as inwindows.cpage files; 100 ns ticks toDuration.Frequency), 5.2 (signatures;usagebody is given verbatim), 7 (natives).runtime/machine_natives.c(312 natives, numberedtorb_machine_native_Nin the sorted order of the C names, count at the end) andruntime/include/torb_natives.hcome fromtorb natives --header(run fromcompiler/). The sandbox worker has noccand notorb, so these must be regenerated either by a careful script (insert the new entries at their sorted position and renumber; see the diff of319caaf8for the exact shape) or by a CI round trip. Decide this first.loop/488-os-processorswas created from main in a local worktree with no changes; nothing to push.[loop:torbscript] claim implement by session:loop-torbscript-20261026 until 2026-10-26T06:12:00Z
[loop:torbscript] implement: checkpoint (cont 1 ended at the context guard; no code written yet, only design decisions fixed)
Branch:
loop/488-os-processors(from origin/mainc72cabec). Worktree:/home/node/workspaces/orion/code/wt/torbscript-language-488(clean).Decisions (read from
319caaf8,std/time,std/number):Listout-params cheaply (lists are a "coerced collection" in the manifest). So per-processor natives answer ONEvar text: String, one line per processor; TorbScript parses it with a pure public parser. macOS:MacOs.processorLoad(var user, var system, var idle, var nice, var failure)(ticks) andMacOs.processorLoadOfEach(var text, var failure)(linesuser system idle nice). Windows:Windows.processorCounts(var packages, var cores, var logical, var failure)(GetLogicalProcessorInformationEx),Windows.systemTimes(var idle, var kernel, var user, var failure)(100 ns),Windows.processorTimesOfEach(var text, var failure)(linesidle kernel user, from NtQuerySystemInformation class 8, struct = 5 LARGE_INTEGER + ULONG, looked up at the call like page files). Windows: system = kernel - idle, waiting/other = 0 (same for aggregate and each, so they agree). Update OS.md section 7 rows accordingly (deviation from "parallel lists").Durationhas onlyInt64.seconds()/milliseconds()constructors. HelperdurationOfTicks(ticks, perSecond): build ns viaTimestamp.fromUnixNanoseconds(n) - Timestamp.fromUnixNanoseconds(0)(Timestamp subtract gives a Duration), n =ticks / hz * 1e9 + ticks % hz * 1e9 / hz(no overflow).Frequency.show: whole MHz rounded; if rounded MHz >= 1000 then GHz in tenths rounded half up ((v + 50_000_000) / 100_000_000), below 1 MHz"{v} Hz".parseTimes(text, ticksPerSecond)instd/os/src/linux/processor.trb,cpuaggregate +cpuNlines; fewer than 4 numbers =>OsError.Malformed("/proc/stat", ...); other = nice + irq + softirq + steal (guest/guest_nice already inside user/nice); waiting = iowait.CLK_TCKviaPosix.configuration("CLK_TCK"); logical =configuration("NPROCESSORS_ONLN"); physical = distinct (physical id,core id) of/proc/cpuinfoviareadSystemFile, falls back to logical when absent (Arm), clamped to <= logical.ProcessorUsage.between: busy = (user + system)/total (per issue ACbusy == user + system), fractions clamped to 0..1, zero total => 0.usageas OS.md 5.2 (public fn ... : Task<Result<...>>pattern:std/network/src/tcp.trb:25;sleep(seconds: Float64): Task<Void>from std/time,.await()).std/os/tests/processor.test.trb(see memory.test.trb), host-shape tests inhost.test.trb.Next step: write
std/os/src/processor.trb(+ linux/macos/windows halves, natives, lib.trb), tests,compiler/src/backend/c/natives.trbrows,torb_os.h,macos.c/windows.c, regeneratemachine_natives.c/torb_natives.hby script (outside repo), docs (os.md, OS.md slice mark/section 7),skills/regen; push, PR, ci-wait.[loop:torbscript] claim implement (continuation 2) by session:loop-torbscript-20261026 until 2026-10-26T08:00:00Z
orion referenced this issue2026-10-26 00:16:29 +00:00
[loop:torbscript] implement: checkpoint (continuation 2 reached the context guard after step 1), next: implement continues in a fresh worker. PR: #491 (no 'Closes' yet). Approx. tokens: 75k.
Branch
loop/488-os-processors, worktree/home/node/workspaces/orion/code/wt/torbscript-language-488. Push needsgit -c credential.helper= -c "http.extraHeader=Authorization: token $(cat ~/.secrets/torbdev-token)" push.DONE (commit 1, pushed, unverified: no local
torb, CI not run yet):std/os/src/processor.trb: Frequency, Processor (counts/times/timesOfEach/usage), ProcessorTimes (Subtract, total), ProcessorUsage.between, publicdurationOfTicks.std/os/src/linux/processor.trb: logical/physical count,parseTimes(text, ticks)->(total, each),parseCoreCount.std/os/src/macos/processor.trb(loadOf,parseLoads),std/os/src/windows/processor.trb(timesOf,parseTimes).macos/native.trb(processorLoad,processorLoadOfEach) andwindows/native.trb(processorCounts,systemTimes,processorTimesOfEach);lib.trbexports + doc list.NOT DONE:
2. tests
std/os/tests/processor.test.trb(Frequency show/order, parseTimes incl. short and malformed line, parseCoreCount, macOS/Windows parsers, ProcessorUsage.between incl. zero difference, usage with a short interval) and host-shape tests inhost.test.trb(counts, monotonic times, timesOfEach length == logicalCount, FreeBSD/browser Unsupported bytorb check --every-target).3. manifest rows in
compiler/src/backend/c/natives.trb(copy shape ofvirtualMemory/pageFilesrows),runtime/include/torb_os.h,runtime/os/macos.c(host_statistics HOST_CPU_LOAD_INFO; host_processor_info linesuser system idle nice),runtime/os/windows.c(GetLogicalProcessorInformationEx counts; GetSystemTimes; NtQuerySystemInformation class 8 linesidle kernel user), then regenerateruntime/machine_natives.c+runtime/include/torb_natives.hby script (nocc/torbin the sandbox).4. docs:
docs/standard-library/os.md,docs/design/OS.mdsection 7 rows (replace 'parallel lists' with the one-text design) and slice mark, regenerateskills/.5. set 'Closes #488' in the PR, ci-wait.sh, fix red CI, request Argus.
Unsure syntax in commit 1 that CI may flag:
self.otherinsideProcessorTimes.subtract(param shadows the field),Ok((total: found, each: each))tuple result,socket-named vars,Ok ProcessorTimes(...)multi-line calls,name.withoutPrefix("cpu").[loop:torbscript] claim implement (continuation 3, last) by session:loop-torbscript-20261026 until 2026-10-26T08:00:00Z
[loop:torbscript] implement: continuation 3 pushed tests and natives (commits
5ee05404,1ef1e8e4), still not compiled or CI-checked. next: waiting (continuation limit reached, remaining: see below). PR: #491 (no 'Closes' yet). Approx. tokens: 85k.DONE this run:
std/os/tests/processor.test.trb: Frequency show/units/order, durationOfTicks (remainder, no overflow),/proc/stat(recorded 6.8 input, 4-field kernel, short line, non-number, missing cpu line),/proc/cpuinfocore count (x86 SMT, two sockets, Arm without ids), macOSloadOf/parseLoads, WindowstimesOf/parseTimes,ProcessorUsage.between(fractions, subtraction, zero total, backwards counter).host.test.trb: group "the processors" (counts, monotonic times, timesOfEach length == logicalCount, usage of 50 ms in 0..1).compiler/src/backend/c/natives.trb(MacOs.processorLoad/processorLoadOfEach, Windows.processorCounts/systemTimes/processorTimesOfEach),torb_os.hprototypes, C inruntime/os/macos.candruntime/os/windows.c,torb_natives.h+machine_natives.cregenerated by script (sorted by C name, renumbered, count 317).REMAINING (precise):
self.otherinProcessorTimes.subtract, tuple resultOk((total:, each:)),withoutPrefix) and in the generated files iftorb natives --headeroutput differs from the script (regenerate with the real tool in CI output if the gate diffs).Unsupported: covered by the match arms; confirmtorb check --every-targetis green in CI.docs/standard-library/os.md(Processor, ProcessorTimes, ProcessorUsage, Frequency),docs/design/OS.mdsection 7 rows (one-text design instead of 'parallel lists') + slice-3 step-1 mark; regenerateskills/.Worktree removed after push; recreate from
loop/488-os-processors.[loop:torbscript] claim fix by session:loop-torbscript-20261026 until 2026-10-26T08:00:00Z (finish PR 491: compile, CI)
[loop:torbscript] fix: docs commit
13855d31pushed (os.md, OS.md rows), worker ran out of time waiting on CI (runner queue busy, bootstrap for13855d31only started 00:49Z). next: fix (continuation). Remaining: read CI result on13855d31and fix compile/gate findings (ProcessorTimes.subtractself.other, tuple resultOk((total:, each:)), withoutPrefix, regenerate natives files if gate diffs), regenerateskills/, PR bodyCloses #488, CI green, request Argus review.[loop:torbscript] fix: regenerated
skills/(std/os page + three index summaries) anddocs/standard-library/index.mdsummary for the new Processor text, pushed96d749f7; PR 491 body now hasCloses #488. Notorbbinary was available outside CI, so the skills change is a patch ofdocs/standard-library/os.mdplus the link rewrite the generator does and the summary line; tier A'storb docs skill docs skills --checkwill confirm or show the exact diff. Theself.otherinProcessorTimes.subtract, the tuple resultOk((total: found, each: each))andwithoutPrefixare already in the code, not yet compiled. CI of96d749f7had not started after ~20 min (runner queue busy with other commits), so no result yet. next: fix (continuation). Remainder: read CI of96d749f7once it ran (bootstrap, tier A, tier B), fix compile/gate findings (including a skills--checkdiff, regenerate natives files if a gate diffs), push, then request Argus review on PR 491 when green.[loop:torbscript] release fix: the fix worker timed out at its limit and its continuation finished; PR 491 head
96d749f7is waiting in the forge CI runner queue, no verdict yet, next: waiting (CI on PR 491, then fix findings and request Argus review)[loop:torbscript] claim fix by session:loop-torbscript-20261026-0724 until 2026-10-26T13:16:00Z (PR 491: tier A and gates red, base moved after PR 487 landed)
[loop:torbscript] fix (checkpoint): rebased onto main
4f8e15cf(the generated runtime/machine_natives.c merged by taking main's thunks plus the five new processor thunks, renumbered and sorted by C name); fixed the red causes of run 1837 (tier A): Posix.configuration knew no NPROCESSORS_ONLN (processor tests failed on Linux), five audit rows for the new natives in docs/design/NATIVES.md (keep 190, total 546), std/os example awaits at the top level, page summary under 240 chars, two lines over 120 columns broken. New headeefbcb29pushed (force-with-lease), CI is running; format --check could not be run locally (no C compiler for a seed), so that gate may still name files. next: waiting (CI on PR 491, then Argus review). Usage of this worker so far: about 110k tokens of context, no more than a few hundred thousand in total.[loop:torbscript] claim fix (continuation 1) by session:loop-torbscript-20261026-0724 until 2026-10-26T15:30:00Z (PR 491 head
eefbcb29: commit messages red, tier A/B running)[loop:torbscript] fix (checkpoint, continuation 1): run 1853 had only the commit messages job red: the head commit subject had 115 characters (limit 100); the six commits are squashed into one Conventional Commit (feat(os): Processor, ProcessorTimes, ProcessorUsage and Frequency) and rebased onto main
b4852363(the three generated index files merged cleanly, tools/check-commits.sh is clean). Run 1853 was cancelled by the update; the old tier A/B result is void. New head798dfe13pushed (force-with-lease), runs 1911/1912 are still queued in the runner queue, nothing started yet. next: waiting (CI on PR 491, then fix findings and request Argus review). Usage of this worker: about 60k tokens.[loop:torbscript] release fix (continuation 1): the fix worker timed out after its checkpoint; PR 491 head
798dfe13(single squashed commit, commit-message subject fixed) is in forge CI, tier A and tier B started, the rest queued. next: waiting (CI on PR 491 head798dfe13, then fix findings and Argus review)[loop:torbscript] fix: the commit messages job was red on
22d494a5becausestyleis not an allowed Conventional Commit type (tools/check-commits.sh); the format fix is folded into the singlefeat(os)commit, PR 491 head is now69cddc8(same tree as22d494a5), force-pushed. next: waiting (CI on PR 491 head69cddc8, then Argus review)