registry: retry documentation renders that fail for I/O reasons (step 1 of #236) #498
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
First step of #236 (follow-up of #17). Implements RELEASE.md section 7.7 (documentation per package), "A failure is either the package's or not" in
tools/registry/src/documentation.trb.Goal: a render that fails because of the environment (full disk, write over the file-size limit, unreadable file) is retried instead of ending
failedat once, and a retried version is re-queued after a new build of the toolchain.Change (two Conventional Commits):
fix(compiler):torb docexits with 74 (EX_IOERR) where the command ends with anIoError(the failure path ofendWith,compiler/src/main.trb), and keeps 1 for problems it found in the package. Onlydocis changed; the other commands keep 1. The exit codes are listed in the usage text oftorb docand in thetorb docreference page;skills/is regenerated.fix(registry):attemptOf(tools/registry/src/documentation.trb) maps 74 toAttempt.Interrupted(retried with the existing back-off,failedafterdocumentationRetries). The toolchain string the worker records is<--version output>+<first 12 hex of the sha-256 of thetorbexecutable>, so the start-up re-queue offailedversions (toolchain != ...) fires for every new build. The section "A failure is either the package's or not" is updated.Acceptance criteria:
torb docon a directory it cannot write its output to (read-only output path) exits 74; on a package with a type error it still exits 1. A test intests/covers both.tools/registry/tests/documentation.test.trb: a faketorbthat exits 74 leaves the versionpendingwithattemptscounted and aduein the future, andfailedonce the retries are used up; a fake that exits 1 still endsfailedat once.torbexecutables with equal--versionoutput but different bytes, and the start-up re-queue queues a version that failed with the first one.RELEASE.md7.7, thetorb docreference) state exit codes 0, 1, 74.Seed: no (no syntax change, no new native, no driver flag, no std name looked up by string). The compiler change is in
compiler/src/main.trband the exit code is read by the worker only.Not in this step: the other points of #236 (fair queue, re-queue race, swap without 404s, health, stall measurement, nested manifests); they follow as their own steps.
[loop:torbscript] claim implement by session:loop-torbscript-20261026-1044 until 2026-10-26T19:09:07Z
[loop:torbscript] implement: checkpoint (context limit reached before the first edit; nothing pushed, no branch yet), next: re-run implement from main (PR 500 already in main,
de6c1cb7)Findings so far, to save the next worker the exploration (~100k tokens):
Commit 1
fix(compiler): exit 74 when torb doc ends with an I/O errorcompiler/src/main.trbarm["doc", ...more](~line 1086):runDoc(options)?goes to the top-level?report (exit 1). Replace bymatch runDoc(options) { Ok(found) => found, Fail(problem) => { printError "error: {problem}"; Process.exit 74 } }(same text asendWith), keepif code != 0 { Process.exit code }. Name the code in a documentedconst.docUsageLines()incompiler/src/reference/command.trb(~107); the pagedocs/tooling/torb-doc.md(Synopsis) gets the exit codes 0, 1, 74.skills/torbscript-projects/references/tooling/torb-doc.mdis the docs page with links rewritten to(skill ...: references/...); text without links is copied verbatim, so it can be edited by hand to match;torb docs skill docs skills --checkis the CI gate.tools/command-line.sh(gate "tests/command-line", expected filetests/command-line/arguments.expected). Plan: before the--helploop add two runs in the scratch dir that record onlyexit <code>(stderr/stdout to /dev/null): a package with a type error (public fn answer(): Int { "no" }insrc/main.trb,project.trbwith name and version) ->exit 1, and a good package with--output <existing file>/site(createDirectory below a file fails, also as root in CI) ->exit 74; append the two matching lines toarguments.expectedright after the$ torb --help/exit 0block.Commit 2
fix(registry): retry documentation renders that fail with exit 74tools/registry/src/documentation.trb:attemptOf(~731) addrun.code == 74->Attempt.Interrupted. Instarted(~218) the toolchain string becomes<first --version line>+<first 12 hex of Sha256.of(File.read(program)) .hex()>(use Sha256 from "std/digest";languagemust still be parsed from the version part only, not the suffixed string;sandboxedprobe runsprogramas before). Update the module comment "A failure is either the package's or not" (line ~30),tools/registry/README.md(~line 130, "tried again after 1, 5 and 25 minutes"), anddocs/design/RELEASE.md7.7 (~line 1190, "any other failure is tried again ...": add exit 0, 1, 74 and the toolchain string).tools/registry/tests/documentation.test.trb: the existing stand-in (exit 3, same--versionas the real torb) now has other bytes, so its start re-queues everyfailedrow: the assertionsecondRound == 5becomes12(broken 4 + future 1 + local/escape/nested 1 each + fragment 4; verify in CI). New stand-ins (factor the script writer out of the existing one): exit 74 with retry delay 3600 (config.copy(documentationRetrySeconds: 3600)) on a newly published package published aftersecondRound->pending,attempts == 1,due > unixepoch()(read withdatabase.rows<T>("SELECT due > unixepoch() AS later ..."), the API shows nodue); a second worker with delay 0 ->failed,attempts == 4; a stand-in with exit 1 ->failed, attempts 1; two stand-ins with equal--versionand different bytes record differenttoolchainvalues (API fieldtoolchainexists inDocumentationAnswer, api.trb:359, add it to the test'sDocumentationField), and the start of the second one queues the version that failed with the first. New noise: every stand-in start re-queues all failed rows, so usedrained(worker, 40)and only assert on the new packages.Gates: CI only (
ci-wait.sh);torb format --checklayout; pdisc sidecar hashes are restamped when the markdown changes (docs/tooling/torb-doc.md,RELEASE.md, README).[loop:torbscript] claim implement (continuation 1) by session:loop-torbscript-20261026-1044 until 2026-10-26T19:19:00Z
[loop:torbscript] implement: PR #501 opened (branch loop/498-doc-io-retry, two commits), CI running, next: wait for CI, fix findings on the same branch (unverified locally: no torb build in the exec; the registry test, format layout and the command-line test were never run)
[loop:torbscript] implement: checkpoint, next: wait for CI of PR #501 (branch loop/498-doc-io-retry, head
0801902, two commits: fix(compiler) exit 74 + command-line test + docs; fix(registry) attemptOf 74, toolchain = version+sha256sum[:12], tests, RELEASE 7.7, README). At 13:52Z the runs 1937/1938 were still queued (status waiting, no job started), so no CI result yet. Nothing was run locally (no torb build): expect to fix on the same branch (fold into the two commits, force-push with lease): (1) torb format layout of documentation.trb and tests/documentation.test.trb, (2) tools/command-line.sh new doc runs (exit 1 / exit 74 pinned in tests/command-line/arguments.expected), (3) registry test: standInWritten/dueLater helpers, secondRound is asserted as >= 5 (could be tightened), new groups 'an I/O error of torb' and 'the toolchain of a build', (4) sha256sum is used instead of std/digest (pure TorbScript hashing of a ~16 MB torb is too slow in the VM). docs/ and skills/ are excluded from pdisc, so no restamp for torb-doc.md; tools/registry/README.md is restamped. Tokens used by this worker about 115k.[loop:torbscript] claim fix by session:loop-torbscript-20261026-1044 until 2026-10-26T20:52:53Z
[loop:torbscript] release: claim fix by session:loop-torbscript-20261026-1044 released (the worker lost its context after the push; PR #501 was approved and merged as
0e91170e), next: done