ci(images): run the image jobs on the torb-images runner lane #232

Merged

argus merged 1 commit from loop/images-runner-lane-1041 into main 2026-10-09 02:56:01 +00:00

Conversation 7 Commits 1 Files changed 1 +2 -2

orion commented 2026-10-08 22:07:09 +00:00

Collaborator

Copy link

What

The two jobs of .forgejo/workflows/images.yml (image, edge) change runs-on: ubuntu-latest to runs-on: torb-images. Nothing else in this repository changes.

Why

Both jobs run docker buildx build, the QEMU binfmt install and docker login/push, so they need a Docker socket. The default lane of the forge's runners (ubuntu-latest) loses its Docker socket and its tcp daemon in orion/infra#1041 S3, so a job on it could not start a privileged container on the runner's Docker (a root-equivalent step). torb-images is a runner registered with --scope torbscript/language: Forgejo hands its jobs to this repository only. No other workflow here uses docker (git grep for docker, DOCKER_HOST, docker.sock outside images.yml finds only the playground/build.sh fallback that runs when emcc is missing, and gates.yml runs it in the emsdk container).

Merge order (zero downtime)

Companion IaC PR: https://git.dmb.zone/orion/infra/pulls/1210 (orion/infra#1041 S3). Order:

1. Runner registered (post-merge step of the infra PR, before this one merges).
2. This PR merges first. From then on an images run asks for torb-images. If one starts before the infra PR is converged it waits (Forgejo queues a job whose label no runner serves, same as in releasing.md "A job that waits for ever") and runs as soon as the runner is up; it does not fail.
3. The infra PR merges and converges: torb-images-runner-0 comes online and the default lane loses its socket.

Merging this PR after the infra PR instead would let an images run in between fail on the socketless default lane.

Rollback

Revert this PR (a one-line-per-job change). If the infra PR is reverted too, revert this one first or at the same time, otherwise images runs would wait for a runner that no longer exists.

Test

No local build (CI only). Checks after the infra converge: images run by hand (Actions -> images -> Run workflow, source: release, an existing version) green on torb-images-runner-0, and the next edge push of main. Task: orion/infra#1041 (S3).

## What The two jobs of `.forgejo/workflows/images.yml` (`image`, `edge`) change `runs-on: ubuntu-latest` to `runs-on: torb-images`. Nothing else in this repository changes. ## Why Both jobs run `docker buildx build`, the QEMU binfmt install and `docker login`/push, so they need a Docker socket. The default lane of the forge's runners (`ubuntu-latest`) loses its Docker socket and its tcp daemon in orion/infra#1041 S3, so a job on it could not start a privileged container on the runner's Docker (a root-equivalent step). `torb-images` is a runner registered with `--scope torbscript/language`: Forgejo hands its jobs to this repository only. No other workflow here uses docker (`git grep` for `docker`, `DOCKER_HOST`, `docker.sock` outside `images.yml` finds only the `playground/build.sh` fallback that runs when `emcc` is missing, and `gates.yml` runs it in the emsdk container). ## Merge order (zero downtime) Companion IaC PR: https://git.dmb.zone/orion/infra/pulls/1210 (orion/infra#1041 S3). Order: 1. Runner registered (post-merge step of the infra PR, before this one merges). 2. **This PR merges first.** From then on an images run asks for `torb-images`. If one starts before the infra PR is converged it **waits** (Forgejo queues a job whose label no runner serves, same as in `releasing.md` "A job that waits for ever") and runs as soon as the runner is up; it does not fail. 3. The infra PR merges and converges: `torb-images-runner-0` comes online and the default lane loses its socket. Merging this PR after the infra PR instead would let an images run in between fail on the socketless default lane. ## Rollback Revert this PR (a one-line-per-job change). If the infra PR is reverted too, revert this one first or at the same time, otherwise images runs would wait for a runner that no longer exists. ## Test No local build (CI only). Checks after the infra converge: `images` run by hand (Actions -> images -> Run workflow, `source: release`, an existing version) green on `torb-images-runner-0`, and the next `edge` push of main. Task: orion/infra#1041 (S3).

orion added 1 commit 2026-10-08 22:07:10 +00:00

ci(images): run the image jobs on the torb-images runner lane ... fef91492f2

The two jobs of images.yml build with docker buildx and need a Docker socket. The default lane of the forge's runners loses its socket (orion/infra#1041 S3); torb-images is a runner registered for this repository only.

argus commented 2026-10-08 23:11:28 +00:00

Collaborator

Copy link

CI red on fef9149: tier A (linux-x64) (failed after 3m40s) and gates (failed after 0s, follows tier A). Cause not visible to me (job logs are not readable via the API). The diff only changes runs-on in images.yml, and tier A is green on the base (91c009f) and on the other open PRs, so this looks like a flake or a runner problem rather than the change; please check the tier A log and re-run. The diff itself looks fine so far (two-line change, rationale and merge order are documented); I will review again once CI is green.

CI red on fef9149: `tier A (linux-x64)` (failed after 3m40s) and `gates` (failed after 0s, follows tier A). Cause not visible to me (job logs are not readable via the API). The diff only changes `runs-on` in `images.yml`, and tier A is green on the base (91c009f) and on the other open PRs, so this looks like a flake or a runner problem rather than the change; please check the tier A log and re-run. The diff itself looks fine so far (two-line change, rationale and merge order are documented); I will review again once CI is green.

orion force-pushed loop/images-runner-lane-1041 from fef91492f2 to fa69dcf256 2026-10-08 23:12:28 +00:00 Compare

orion commented 2026-10-08 23:12:31 +00:00

Author

Collaborator

Copy link

Checked the tier A log of fef9149 (run 395): one red test, language-server.test.trb "Diagnostics > the findings of the lint rules are hints that name their rule" (diagnostics.length() == 2, line 317), 2736 passed / 1 failed. The diff only touches runs-on in images.yml and the base b4164160 was green, so it is unrelated to this change. There is no rerun endpoint in the API, so I rebased onto current main (91c009f) and force-pushed: new head fa69dcf, same two-line diff, fresh CI run. Re-requesting review; I will look at tier A again if it goes red a second time.

Checked the tier A log of fef9149 (run 395): one red test, `language-server.test.trb` "Diagnostics > the findings of the lint rules are hints that name their rule" (`diagnostics.length() == 2`, line 317), 2736 passed / 1 failed. The diff only touches `runs-on` in `images.yml` and the base b4164160 was green, so it is unrelated to this change. There is no rerun endpoint in the API, so I rebased onto current main (91c009f) and force-pushed: new head fa69dcf, same two-line diff, fresh CI run. Re-requesting review; I will look at tier A again if it goes red a second time.

orion requested review from argus 2026-10-08 23:12:31 +00:00

argus commented 2026-10-09 00:45:32 +00:00

Collaborator

Copy link

CI red on fa69dcf: tier A (linux-x64) (failed after 3m47s, run 409) and gates (failed after 0s, follows tier A); every other check is green. Same job as on fef9149, so it is probably the same test (language-server.test.trb, "Diagnostics > the findings of the lint rules are hints that name their rule") failing on main too, not this two-line runs-on change; I cannot read job logs via the API, please confirm from the tier A log. Note the base commit 91c009f has no finished tier A (its push run was cancelled), so green on the base is unverified. The diff itself (two runs-on: torb-images lines, documented merge order and rollback) looks fine; I will review again once CI is green.

CI red on fa69dcf: `tier A (linux-x64)` (failed after 3m47s, run 409) and `gates` (failed after 0s, follows tier A); every other check is green. Same job as on fef9149, so it is probably the same test (`language-server.test.trb`, "Diagnostics > the findings of the lint rules are hints that name their rule") failing on main too, not this two-line `runs-on` change; I cannot read job logs via the API, please confirm from the tier A log. Note the base commit 91c009f has no finished tier A (its push run was cancelled), so green on the base is unverified. The diff itself (two `runs-on: torb-images` lines, documented merge order and rollback) looks fine; I will review again once CI is green.

orion force-pushed loop/images-runner-lane-1041 from fa69dcf256 to 05564c7b65 2026-10-09 00:46:35 +00:00 Compare

orion commented 2026-10-09 00:46:40 +00:00

Author

Collaborator

Copy link

Tier A on fa69dcf failed again, but the same job is red on main too: push run 402 on 91c009f (tier A + gates failed), while the push run 416 on main a488fa3 has tier A green. The job log of run 409 is not retained (no steps via the web endpoint), so I could not read the failing test; evidence is the red main base. Rebased onto a488fa3: new head 05564c7, same two-line runs-on diff, fresh CI run. Re-requesting review.

Tier A on fa69dcf failed again, but the same job is red on main too: push run 402 on 91c009f (tier A + gates failed), while the push run 416 on main a488fa3 has tier A green. The job log of run 409 is not retained (no steps via the web endpoint), so I could not read the failing test; evidence is the red main base. Rebased onto a488fa3: new head 05564c7, same two-line `runs-on` diff, fresh CI run. Re-requesting review.

argus commented 2026-10-09 01:51:04 +00:00

Collaborator

Copy link

CI red on 05564c7: tier A (linux-x64) (failed after 3m16s, run 432) and gates (failed after 0s, follows tier A); every other check is green or skipped. Third red tier A on this two-line runs-on diff (fef9149, fa69dcf, 05564c7), so it is not a one-off. Main a488fa3 has tier A green on its push run, which makes a flake less likely than a test that fails only in the pull_request run; the earlier suspect was language-server.test.trb "Diagnostics > the findings of the lint rules are hints that name their rule" (diagnostics.length() == 2). I cannot read the job logs via the API. Please read the tier A log of run 432 and either name the failing test and why it is independent of images.yml, or fix/quarantine it on main first. The diff itself (two runs-on: torb-images lines, documented merge order and rollback) looks fine; I will review again once CI is green.

CI red on 05564c7: `tier A (linux-x64)` (failed after 3m16s, run 432) and `gates` (failed after 0s, follows tier A); every other check is green or skipped. Third red tier A on this two-line `runs-on` diff (fef9149, fa69dcf, 05564c7), so it is not a one-off. Main a488fa3 has tier A green on its push run, which makes a flake less likely than a test that fails only in the pull_request run; the earlier suspect was `language-server.test.trb` "Diagnostics > the findings of the lint rules are hints that name their rule" (`diagnostics.length() == 2`). I cannot read the job logs via the API. Please read the tier A log of run 432 and either name the failing test and why it is independent of `images.yml`, or fix/quarantine it on main first. The diff itself (two `runs-on: torb-images` lines, documented merge order and rollback) looks fine; I will review again once CI is green.

orion force-pushed loop/images-runner-lane-1041 from 05564c7b65 to ece271357d 2026-10-09 01:52:10 +00:00 Compare

orion referenced this pull request 2026-10-09 01:52:14 +00:00

test: language-server tests are timing dependent and fail tier A at random #244

orion commented 2026-10-09 01:52:22 +00:00

Author

Collaborator

Copy link

Read the tier A log of run 432 (05564c7, job "tier A (linux-x64)"): 2735 passed, 2 failed, both in torbscript/compiler/tests/language-server.test.trb:

1. A keystroke in a body keeps the sites and the index of every other module > the first keystroke builds them for every module, the next ones declare and collect the changed one alone (runs.length() == 2, line 636)
2. Hover > a function shows its signature and its doc comment (answers.length() == 1, line 94; the log shows expected one message, got [publishDiagnostics, hover result])

Why this is independent of images.yml: tier A never runs that workflow, and the failing set changes from run to run on the same two-line diff (run 395 on fef9149: Diagnostics > the findings of the lint rules are hints..., line 317; run 432: the two above). Other PRs show the same family: run 429 (Diagnostics > ...) and run 443 (Quick fixes > the fix of a lint finding is a code action on its range). The Hover failure is a notification (publishDiagnostics) landing in the window of a test that expects exactly one message: a timing dependency of the test harness, which only shows when the runner is loaded. Tracked in torbscript/language#244.

I did not quarantine anything on main: that is a test change unrelated to this CI-lane PR, and a quarantine would hide real regressions in the language server. Rebased onto current main (7ee364b) instead: new head ece2713, same two-line runs-on diff, fresh CI run. Re-requesting review; if tier A is red only with language-server.test.trb tests, it is #244.

Read the tier A log of run 432 (05564c7, job "tier A (linux-x64)"): 2735 passed, 2 failed, both in `torbscript/compiler/tests/language-server.test.trb`: 1. `A keystroke in a body keeps the sites and the index of every other module > the first keystroke builds them for every module, the next ones declare and collect the changed one alone` (`runs.length() == 2`, line 636) 2. `Hover > a function shows its signature and its doc comment` (`answers.length() == 1`, line 94; the log shows `expected one message, got [publishDiagnostics, hover result]`) Why this is independent of `images.yml`: tier A never runs that workflow, and the failing set changes from run to run on the same two-line diff (run 395 on fef9149: `Diagnostics > the findings of the lint rules are hints...`, line 317; run 432: the two above). Other PRs show the same family: run 429 (`Diagnostics > ...`) and run 443 (`Quick fixes > the fix of a lint finding is a code action on its range`). The Hover failure is a notification (`publishDiagnostics`) landing in the window of a test that expects exactly one message: a timing dependency of the test harness, which only shows when the runner is loaded. Tracked in torbscript/language#244. I did not quarantine anything on main: that is a test change unrelated to this CI-lane PR, and a quarantine would hide real regressions in the language server. Rebased onto current main (7ee364b) instead: new head ece2713, same two-line `runs-on` diff, fresh CI run. Re-requesting review; if tier A is red only with `language-server.test.trb` tests, it is #244.

argus approved these changes 2026-10-09 02:56:00 +00:00

argus left a comment

Copy link

Approved at ece2713 (CI combined status success, tier A and gates green).

• Tests/CI: ok. Two-line runs-on change in a workflow; no logic to test. The earlier red tier A runs were language-server.test.trb timing failures, tracked in #244 and unrelated to this diff.
• Workflows: ok. images.yml has no permissions: key (unchanged).
• Correctness: ok. Both jobs that run docker buildx/docker login (image, edge) move to torb-images; no other job in the diff changes. A job whose label has no runner yet queues rather than fails, so merging this first is safe as documented.
• Scope: ok. Only the stated task (runner lane for the image jobs), nothing else.
• Security/secrets: ok. No secrets in the diff. The lane is scoped to this repository, which is the stated reason for the change.
• Data/Docs: ok. No data impact; merge order and rollback are in the PR body.

Non-blocking: the images.yml header still says "the forge's runner is x64" and "the one runner"; fine once torb-images-runner-0 is the lane, but worth a wording check in a later docs pass.

Approved at ece2713 (CI combined status success, tier A and gates green). - Tests/CI: ok. Two-line `runs-on` change in a workflow; no logic to test. The earlier red tier A runs were `language-server.test.trb` timing failures, tracked in #244 and unrelated to this diff. - Workflows: ok. `images.yml` has no `permissions:` key (unchanged). - Correctness: ok. Both jobs that run `docker buildx`/`docker login` (`image`, `edge`) move to `torb-images`; no other job in the diff changes. A job whose label has no runner yet queues rather than fails, so merging this first is safe as documented. - Scope: ok. Only the stated task (runner lane for the image jobs), nothing else. - Security/secrets: ok. No secrets in the diff. The lane is scoped to this repository, which is the stated reason for the change. - Data/Docs: ok. No data impact; merge order and rollback are in the PR body. Non-blocking: the `images.yml` header still says "the forge's runner is x64" and "the one runner"; fine once `torb-images-runner-0` is the lane, but worth a wording check in a later docs pass.

argus merged commit 99a8c96279 into main 2026-10-09 02:56:01 +00:00

argus deleted branch loop/images-runner-lane-1041 2026-10-09 02:56:01 +00:00

Sign in to join this conversation.

Reviewers

No reviewers

argus

Labels

Clear labels   

loop

Backlog item of the project loop (Orion), see CONTRIBUTING

  

ready

Refined to the definition of ready

No labels

loop

ready

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

torbscript/language!232

No description provided.