parallel: Window and windows (CONCURRENCY slice F, gap 6) #4
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Context
Gap 6 and the barrier exposure of slice F in CONCURRENCY.md: data-parallel work over plain numbers without copies. Gaps 4 and 5 of the same slice are split out (see Dependencies); this issue keeps
Window,windowsand the gate.Source
PlainandWindow" as open work.Goal
windows(items, count, body)splits a buffer intocountdisjointWindows and runsbodyon each at the same time on the worker pool, the caller taking window 0, returning only when every window is done. A window is a pointer, an offset and a length into the caller's heap: no count change, no copy.Acceptance criteria
public shared type Window<Item>instd/parallelwithLengthand[index]read and write (it implementsMutableIndex<Int, Item>; the record's R4 forbids only what resizes: noadd,insert,removeAt), produced only by the region, with noClose.Item: Plainis the bound (needs #155).public fn windows<Item: Plain>(var items: ..., count: Int, body: (var window: Window<Item>) => Void)instd/parallel, imported by name (not in the prelude, section 15 question 2). The body is a closure (needs #156).countonly, never onworkers(the rule of section 4); a length that is not a multiple ofcountgives windows that differ by at most one element, none empty unlesscount > length;count < 1is a panic with a message.Taskfunction cannot answer awindowscall as aTask(TYPECHECKER gap 49 already refuses avarparameter on a function that answers aTask).Windowreach the region's body despite theshared typeconfinement of gap 3 (a region's body is the exception besideTaskandChannel), and still refuses aWindowthat escapes the body (stored, returned, captured byspawn): one probe per escape.Floats,windows(values, 64) { var window => for i in 0..window.length { window[i] = window[i] * 2.0 } }, gives the same output as the sequential loop, byte-identical, atworkers: 1,2andWorkers.count(); zero live blocks afterwards; zero copies at the crossing (how the counter is read is the second owner decision of the comment below).tests/conformance/with a.workersfile (4 and 1), plus a runtime test besideruntime/tests/pool_test.cfor the fork-join and the borders.varaccess before the region starts (a shared storage is copied once, before, and that copy is not a crossing copy).docs/standard-librarypage forstd/parallelgetswindows,WindowandPlain; CONCURRENCY.md slice F and section 16 say "Built" with the date;docs check docsgreen;skills/regenerated.Touches the seed?
Window's two members (length,[index]) in the manifest of natives (both back ends), the fork-join entry exposed to std. The compiler itself does not use them, so no seed refresh is needed for the repo to build; if declaringwindowsinstd/turns out to need one (the seed compiler buildsstd/), that is an owner call and the loop stops there.Plain(#155).Size
Larger than one PR as written; proposed split (this issue is the last part):
Plain(one commit, checker).varparameter (one or two commits, parser, checker, IR).Window,windows, the gate. Two Conventional Commits:feat(runtime): a window over a buffer and the region that hands them out(C runtime, VM native, manifest) andfeat(parallel): windows and Window(std, conformance, docs). If the owner picksBuffer<Item>(#54) as the parameter type, this also depends on #54.Implementer: opus/high (VM, both back ends,
runtime/).Dependencies
Plain) and #156 (closurevarparameter).Workers.count()are built onmain; gap 12's counter exists astorb_pool_statistics.copiedinruntime/task.cbut nothing reads it from a program yet.Buffer<Item>is #54 (not built); see the owner decision comment.Moved from the planning lists of the docs; the design record stays the specification.
[loop:torbscript] intake: in scope (author orion, label loop). Implements CONCURRENCY.md slice F (gaps 4, 5, 6); design record is the specification, no collision with CONCEPT "Open Questions". Touches the VM and both back ends (opus/high worker when implemented). Release freeze for 0.1.0 (2026-10-06) holds new PRs, next: refine (queued, buffer >= 2 ready)
[loop:torbscript] claim refine by session:loop-torbscript-20261007 until 2026-10-07T17:39:15Z
Plainmarker (CONCURRENCY gap 4, part of slice F) #155varparameter (CONCURRENCY gap 5, part of slice F) #156parallel:to parallel:Plain,Windowandwindows(CONCURRENCY slice F)Windowandwindows(CONCURRENCY slice F, gap 6)[loop:torbscript] owner decision: two choices slice F depends on are not settled in a design record.
1. What does
windowsborrow from, whileBuffer<Item>(#54) does not exist?The record writes
windows<Item: Plain>(var items: Buffer<Item>, ...)and says only that the signature "type checks today withListin place of the plannedBuffer" (CONCURRENCY.md section 6). #54 is a kernel-of-natives issue that needs a seed refresh and has no date.windowsoverList<Item: Plain>now; when #54 lands the parameter type widens toBuffer<Item>(aListof plain items stays accepted or is converted by the one-line change the COLLECTIONS record already plans forBytes).ListofPlainitems is one contiguous block, thevaraccess makes it exclusive, and a unique storage is the only precondition the borrow needs. The ECS columns (#92) wait for #54 anyway.2. How does the gate read the copy counter (gap 12)?
The gate is "zero copies (the counter of gap 12 at zero)". The counter exists as
torb_pool_statistics.copiedinruntime/task.c, read today only by C tests; no program, and no conformance run, can read it. A conformance program is the only gate that runs in both back ends.runtime/tests/pool_test.cpinscopiedunchanged around awindowsregion; the conformance program checks output equality and the leak gate. Zero copies is then proven for the C pool the VM also uses, but not asserted by a TorbScript program.Workers.copies(): Intinstd/task(orstd/parallel) that a conformance program compares before and after the region. One new native in both back ends, a public name, no compiler use (no seed refresh).--explain-copiesis the build-time half); nothing is scheduled.Taken as given, no decision needed:
WindowimplementsMutableIndex(the record listsIndexin the declaration, but its output-buffer shape in section 6 writes through windows);Plainand the closurevarparameter are settled in the record and split out as #155 and #156.[loop:torbscript] refine: issue #4 was too large for one PR and rests on two unsettled choices. Split out the settled parts as #155 (
Plain, gap 4) and #156 (closurevarparameter, gap 5), both labelledready; #4 now coversWindow,windowsand the gate (gap 6, depends on #155 and #156), with a rewritten body (goal, acceptance criteria, design record points, seed impact, size). FoundBuffer<Item>is #54 (unbuilt) and the copy counter of gap 12 is not readable from a program, so the owner decision above is filed (element container while #54 is missing; how the gate reads the counter). #4 is notready. Worker usage: roughly 100k tokens (sonnet), read-only checks of the design records andmainat61d7075c. next: waiting (owner decision)[loop:torbscript] decided by the loop: both choices of the earlier owner-decision comment are internal technical design (no public statement, cost, account, or breaking change), and the owner guidance recorded on #17 and #164 is to decide such choices and record the reasoning. (1)
windowstakesList<Item: Plain>now, option (a): a list of plain items is one contiguous block, thevaraccess makes it exclusive, and unique storage is the only precondition the borrow needs; the parameter widens toBuffer<Item>when #54 lands. Waiting for #54 would block slice F and the ECS columns on an undated kernel change. (2) The gate reads the counter through a read-only native, option (b), named and documented as a statistic: the gate is written in terms of the copy counter and must hold in both back ends, which only a TorbScript conformance program checks. It adds one native in both back ends and one public name, no compiler use, so no seed refresh. Dependencies #155 and #156 are closed. Labelledready. next: implement (ready)[loop:torbscript] implement: ready, but it touches the VM and both back ends while the workers on #164 (vm) and #207 (IR) run on the same paths. next: waiting (live workers on the same paths) until 2026-10-09T05:10:00Z
[loop:torbscript] claim implement by session:loop-torbscript-20261009 until 2026-10-09T09:58Z
[loop:torbscript] implement: worker started (opus/high, branch
loop/4-parallel-windows); the earlier wait on #164/#207 ended, both landed (#239, #238). next: implement (worker in flight, claim above)[loop:torbscript] claim implement by session:loop-torbscript-20261009 until 2026-10-09T09:58Z (worker running, branch
loop/4-parallel-windows)Buffer<Item>as the heap storage primitive #54torb#21login,logout,ownerandyankcommands #22torb audit#23torb vendor#24std/http#30torb doc --checkfail #31Decimalthat computes #34Float32arithmetic, comparison, text and narrowing fromFloat64(slice 1) #35setVariable#38Flags#47ArrayList, the hash tables and the tries onBuffer#55TakeOut/PutBackon the map #56forover a concrete list is a counted loop #58torb build --explain-copies#61Into<Path>signatures (PATH slice 2) #68Storage(URI gap 13) #69101 Switching Protocolsand a duplex connection (NETWORK slice 14) #74std/toml#77std/xml#78std/htmldocuments (epic) #79Dualnumbers (forward mode) #83std/ecswith oneStoreper group type, andstd/transform(ECS slice 1) #89std/htmlmarkup (WEB slice 1) #93[loop:torbscript] implement:
windowsandWindowlanded in PR #248 (Closes #4). The PR adds the region runtime (torb_region_*,Workers.copies()) pluswindows<Item: Plain>(var items: ArrayList<Item>, count, body)andWindowinstd/parallel. The checker keeps a window inside its body: it cannot be stored in a field, returned, rebound or captured by a task. It also adds conformance programs (borders, count, bounds, a million scaled), region tests in pool_test and docs (parallel.md, task.md, CONCURRENCY.md slice F / gap 6). CI is green ona71d6f33: gates, tier A, tier B (conformance in both back ends), bootstrap, commit messages. No seed refresh was needed.Open points (listed in the PR):
Windowis an ordinarytype, not ashared type, because a shared type can only implement shared traits andLength/MutableIndexare not shared.MutableIndex<Int, Item>) drops the scoped marker, which is a possible escape.values == sequentialon twoArrayList<Float>is an internal error of the native back end. The conformance program works around it by comparing item by item; this deserves its own issue.windowstakes avar ArrayList.Tokens: ~0.9M.
next: review (waiting on the merge gate argus)
[loop:torbscript] claim implement by session:loop-torbscript-20261009 until 2026-10-09T12:36Z (fix of the review of PR #248: restore CRLF in three checker files)
[loop:torbscript] PR #248 was merged by argus (merge commit
f9b867d0). No after-merge steps in the PR; issue closed. Non-blocking follow-ups stay in #253.windowsbody on a helper worker must not hang the join #258ArrayList, the hash tables and the tries onBuffer(step 1 of #55) #413