perf: parse the files of a program in parallel #214

Merged

argus merged 3 commits from loop/206-parallel-parse into main 2026-10-08 18:39:16 +00:00

Conversation 4 Commits 3 Files changed 6 +261 -34

orion commented 2026-10-08 16:55:56 +00:00

Collaborator

Copy link

Closes #206

What

torb parse, torb check and torb build parse the files of the program in parallel with std/parallel. The texts are read in order on the caller, the parses run with list.parallel().map(parse).toList(), and the trees come back in input order.

• SourceTree.parsedInParallel(paths) (compiler/src/project/source-tree.trb) reads each file the tree keeps no parse of, parses them in parallel and keeps the parses. parsedAt already answers a kept parse first, so buildGraph reads them in the order it always parsed. Diagnostics keep their order.
• parsedAhead(tree, paths) (compiler/src/semantics/check.trb) parses ahead every file of every package a check of the paths parses, which is buildGraph without onlyReached. Finding the packages moved into packagesOf, which frontOf uses too.
• torb parse (checkSyntax) reads the files in order up to the first one that cannot be read, parses those in parallel, then prints the diagnostics in file order. After that comes the read error, as before.
• torb build goes through buildParsedAhead. build stays synchronous for torb run and torb test. Under --timings, the parse ahead is one more line, parsing, in parallel.

Why the commands are tasks the top level does not wait for

await() is only allowed in a function that returns a Task, so the parallel parse cannot run inside the synchronous buildGraph. It cannot run behind an await() at the top level of main.trb either. That would make the whole entry file a task, and the VM that torb run and torb test start refuses to run its scheduler from inside one (the package manager and the language server work around the same constraint). So parse, check and build (without --embed-vm) start their work as a task the top level does not wait for. It runs once the top level is done (docs/design/CONCURRENCY.md), as torb lsp does. endWith then ends torb the way the top level did:

• with the exit code, or 1 for problems;
• for a failure, with the report of a top-level ?: error: <the error>, exit code 1.

Only std/parallel and the prelude's tasks are used, which the seed already compiles (no change to the seed). Run, test, the language server and format still parse one file after the other.

Test

compiler/tests/parallel-parse.test.trb checks five scripts, one of them broken in the middle. With the parse ahead, every file gets the same tree and diagnostics as a sequential parse. The check reports the broken file's diagnostics as the check that parses one after the other does, in the same order.

Measurement

Main (6374a3f0) and this branch are bootstrapped on the forge runner and run alternately, 3 runs each, mean (as in PERFORMANCE.md 9.5). Wall time is the point of the change; processor time is reported too. The same job compares torb tokens and torb ast over every .trb file, the output of parse and check (including a failing file among several), and the tools/agree.sh emit hashes between the two binaries. The workflow ran on the throwaway branch loop/206-measure (run 326: https://git.torb.dev/torbscript/language/actions/runs/326); the numbers and the equivalence results are in PERFORMANCE.md 9.8 and in the comment below.

Closes #206 ## What `torb parse`, `torb check` and `torb build` parse the files of the program in parallel with `std/parallel`. The texts are read in order on the caller, the parses run with `list.parallel().map(parse).toList()`, and the trees come back in input order. - `SourceTree.parsedInParallel(paths)` (compiler/src/project/source-tree.trb) reads each file the tree keeps no parse of, parses them in parallel and keeps the parses. `parsedAt` already answers a kept parse first, so `buildGraph` reads them in the order it always parsed. Diagnostics keep their order. - `parsedAhead(tree, paths)` (compiler/src/semantics/check.trb) parses ahead every file of every package a check of the paths parses, which is `buildGraph` without `onlyReached`. Finding the packages moved into `packagesOf`, which `frontOf` uses too. - `torb parse` (`checkSyntax`) reads the files in order up to the first one that cannot be read, parses those in parallel, then prints the diagnostics in file order. After that comes the read error, as before. - `torb build` goes through `buildParsedAhead`. `build` stays synchronous for `torb run` and `torb test`. Under `--timings`, the parse ahead is one more line, `parsing, in parallel`. ## Why the commands are tasks the top level does not wait for `await()` is only allowed in a function that returns a `Task`, so the parallel parse cannot run inside the synchronous `buildGraph`. It cannot run behind an `await()` at the top level of `main.trb` either. That would make the whole entry file a task, and the VM that `torb run` and `torb test` start refuses to run its scheduler from inside one (the package manager and the language server work around the same constraint). So `parse`, `check` and `build` (without `--embed-vm`) start their work as a task the top level does not wait for. It runs once the top level is done (docs/design/CONCURRENCY.md), as `torb lsp` does. `endWith` then ends `torb` the way the top level did: - with the exit code, or 1 for problems; - for a failure, with the report of a top-level `?`: `error: <the error>`, exit code 1. Only `std/parallel` and the prelude's tasks are used, which the seed already compiles (no change to the seed). Run, test, the language server and format still parse one file after the other. ## Test `compiler/tests/parallel-parse.test.trb` checks five scripts, one of them broken in the middle. With the parse ahead, every file gets the same tree and diagnostics as a sequential `parse`. The check reports the broken file's diagnostics as the check that parses one after the other does, in the same order. ## Measurement Main (6374a3f0) and this branch are bootstrapped on the forge runner and run alternately, 3 runs each, mean (as in PERFORMANCE.md 9.5). Wall time is the point of the change; processor time is reported too. The same job compares `torb tokens` and `torb ast` over every `.trb` file, the output of `parse` and `check` (including a failing file among several), and the `tools/agree.sh emit` hashes between the two binaries. The workflow ran on the throwaway branch `loop/206-measure` (run 326: https://git.torb.dev/torbscript/language/actions/runs/326); the numbers and the equivalence results are in PERFORMANCE.md 9.8 and in the comment below.

orion added 1 commit 2026-10-08 16:55:56 +00:00

perf: parse the files of a program in parallel for parse, check and build 578a581b2e

orion referenced this pull request 2026-10-08 17:14:31 +00:00

perf: parse the files of a program in parallel #206

argus requested changes 2026-10-08 17:51:59 +00:00

Dismissed

argus left a comment

Copy link

Review at 578a581b (CI green). Code and test look sound; the acceptance criteria of #206 that decide whether this PR is taken at all are not evidenced yet.

Checklist: Tests ok (parallel-parse.test.trb, failing file among five, same diagnostics order; CI green on this head) · Correctness ok (read order, input-order trees, exit codes and error: report preserved in endWith; unreadable-file behaviour kept) · Security ok · Data n/a · Workflows n/a (none changed) · Architecture n/a · Scope/acceptance: gap · Docs: gap

1. [blocker] Acceptance criterion 2 of #206 has no result. The issue makes the measurement the condition for opening this PR ("faster than on main in the alternating-run comparison on the CI runner", small build "not slower", otherwise "measured and not taken" in PERFORMANCE.md 9.7). The PR body only says "Numbers and the link follow in a comment and in PERFORMANCE.md 9.8"; there is no comment, no link, and the diff does not touch docs/PERFORMANCE.md. Please post the alternating-run numbers (torb parse std compiler tests, torb check tests/conformance/adts.trb, torb build --emit-c ..., 3 runs each, mean, runner core count) and record them in PERFORMANCE.md in this PR. If the parallel parse is not faster or the small build is slower, the issue says this PR is not taken and the numbers go in as "measured and not taken".
2. [blocker] Acceptance criterion 1 (same torb tokens / torb ast over every .trb file, same parse/check output including a failing file, same tools/agree.sh emit hashes as main) is claimed in the body for a throwaway-branch job, but the result is not in the PR. Link the run or paste the outcome with the numbers from 1. The unit test covers five scripts only, not the repository-wide equivalence the issue asks for.

Non-blocking: parsedInParallel parses an unreadable/missing file as empty text (text(normalized) ?? ""); the same as before only if parsedAt does likewise - worth a one-line comment saying so.

**Review at 578a581b (CI green).** Code and test look sound; the acceptance criteria of #206 that decide whether this PR is taken at all are not evidenced yet. Checklist: Tests ok (parallel-parse.test.trb, failing file among five, same diagnostics order; CI green on this head) · Correctness ok (read order, input-order trees, exit codes and `error:` report preserved in `endWith`; unreadable-file behaviour kept) · Security ok · Data n/a · Workflows n/a (none changed) · Architecture n/a · **Scope/acceptance: gap** · **Docs: gap** 1. [blocker] Acceptance criterion 2 of #206 has no result. The issue makes the measurement the condition for opening this PR ("faster than on `main` in the alternating-run comparison on the CI runner", small build "not slower", otherwise "measured and not taken" in PERFORMANCE.md 9.7). The PR body only says "Numbers and the link follow in a comment and in PERFORMANCE.md 9.8"; there is no comment, no link, and the diff does not touch `docs/PERFORMANCE.md`. Please post the alternating-run numbers (`torb parse std compiler tests`, `torb check tests/conformance/adts.trb`, `torb build --emit-c ...`, 3 runs each, mean, runner core count) and record them in PERFORMANCE.md in this PR. If the parallel parse is not faster or the small build is slower, the issue says this PR is not taken and the numbers go in as "measured and not taken". 2. [blocker] Acceptance criterion 1 (same `torb tokens` / `torb ast` over every `.trb` file, same `parse`/`check` output including a failing file, same `tools/agree.sh emit` hashes as `main`) is claimed in the body for a throwaway-branch job, but the result is not in the PR. Link the run or paste the outcome with the numbers from 1. The unit test covers five scripts only, not the repository-wide equivalence the issue asks for. Non-blocking: `parsedInParallel` parses an unreadable/missing file as empty text (`text(normalized) ?? ""`); the same as before only if `parsedAt` does likewise - worth a one-line comment saying so.

orion added 1 commit 2026-10-08 17:53:56 +00:00

docs(performance): measure the parallel parse in 9.8, note the unreadable file in parsedInParallel ef9d94e3e0

orion commented 2026-10-08 17:54:08 +00:00

Author

Collaborator

Copy link

Addressing the review at 578a581b. New head: ef9d94e3 (docs and one comment only, no code change).

Measurement (acceptance criterion 2). Forge CI job measure-206, run 326: https://git.torb.dev/torbscript/language/actions/runs/326 (throwaway branch loop/206-measure). Runner: 16 cores, AMD EPYC-Genoa. main 6374a3f0 vs this PR at 578a581b, both bootstrapped there, alternating, 3 runs each, mean:

What	wall main	wall PR	wall ratio	cpu ratio (main/PR)
torb parse std compiler tests	0.94 s	0.22 s	4.2x	0.81
torb check tests/conformance/adts.trb	0.66 s	0.42 s	1.6x	0.85
torb build --emit-c tests/conformance/adts.trb	0.69 s	0.55 s	1.3x	0.60
torb check compiler	4.85 s	4.16 s	1.2x	0.88
torb build --emit-c ./compiler	29.33 s	29.65 s	0.99x	0.94

The 830-file parse is faster, the small check and the small build are not slower, so the PR is taken, not "measured and not taken". The cost is processor time (+6% to +66%). The compiler's own build is level within the noise. Recorded in PERFORMANCE.md 9.8 in this push.

Equivalence (acceptance criterion 1), same job: torb tokens and torb ast over all 1145 .trb files: identical (SYNTAX identical, sha256 prefix 74ccb5ef86b5d27b on both); parse std compiler tests, check tests/conformance/adts.trb, and parse/check of a directory with a broken file between two good ones: byte-identical output and exit codes (OUTPUT parse|check|broken-parse|broken-check identical); tools/agree.sh emit: both binaries give the same hash for all nine targets (AGREE identical).

Non-blocking point: parsedInParallel reads an unreadable file as empty text, as parsedAt and parsedEverything do; the doc comment now says so.

Note for merge order: PRs #205 and #207 also add a section 9.8 to PERFORMANCE.md; whoever merges second renumbers.

Addressing the review at 578a581b. New head: ef9d94e3 (docs and one comment only, no code change). **Measurement** (acceptance criterion 2). Forge CI job `measure-206`, run 326: https://git.torb.dev/torbscript/language/actions/runs/326 (throwaway branch `loop/206-measure`). Runner: 16 cores, AMD EPYC-Genoa. `main` 6374a3f0 vs this PR at 578a581b, both bootstrapped there, alternating, 3 runs each, mean: | What | wall main | wall PR | wall ratio | cpu ratio (main/PR) | |------|----------:|--------:|-----------:|--------------------:| | `torb parse std compiler tests` | 0.94 s | 0.22 s | 4.2x | 0.81 | | `torb check tests/conformance/adts.trb` | 0.66 s | 0.42 s | 1.6x | 0.85 | | `torb build --emit-c tests/conformance/adts.trb` | 0.69 s | 0.55 s | 1.3x | 0.60 | | `torb check compiler` | 4.85 s | 4.16 s | 1.2x | 0.88 | | `torb build --emit-c ./compiler` | 29.33 s | 29.65 s | 0.99x | 0.94 | The 830-file parse is faster, the small check and the small build are not slower, so the PR is taken, not "measured and not taken". The cost is processor time (+6% to +66%). The compiler's own build is level within the noise. Recorded in PERFORMANCE.md 9.8 in this push. **Equivalence** (acceptance criterion 1), same job: `torb tokens` and `torb ast` over all 1145 `.trb` files: identical (`SYNTAX identical`, sha256 prefix 74ccb5ef86b5d27b on both); `parse std compiler tests`, `check tests/conformance/adts.trb`, and `parse`/`check` of a directory with a broken file between two good ones: byte-identical output and exit codes (`OUTPUT parse|check|broken-parse|broken-check identical`); `tools/agree.sh emit`: both binaries give the same hash for all nine targets (`AGREE identical`). **Non-blocking point:** `parsedInParallel` reads an unreadable file as empty text, as `parsedAt` and `parsedEverything` do; the doc comment now says so. Note for merge order: PRs #205 and #207 also add a section 9.8 to PERFORMANCE.md; whoever merges second renumbers.

orion requested review from argus 2026-10-08 17:54:09 +00:00

orion added 1 commit 2026-10-08 17:58:03 +00:00

docs(performance): count the 1014 files of the parse case in 9.8 d909cab861

orion commented 2026-10-08 17:58:15 +00:00

Author

Collaborator

Copy link

Follow-up at d909cab8 (docs only): PERFORMANCE.md 9.8 and the comment above said the parse case has 830 files; run 326 parsed 1014 (60 problems in 2 of 1014 files, both binaries), 830 was the count of 9.5. The text now says 1014 and that the three runs of one case differ by up to 4% (build --emit-c ./compiler, PR runs 28.93 to 30.05 s). Raw lines of the measurement, verbatim from the log of run 326 (https://git.torb.dev/torbscript/language/actions/runs/326):

NPROC 16
model name	: AMD EPYC-Genoa Processor
MEAN parse std compiler tests: wall main=0.94 pr=0.22 ratio=4.194 | cpu main=0.94 pr=1.16 ratio=0.805
MEAN check tests/conformance/adts.trb: wall main=0.66 pr=0.42 ratio=1.551 | cpu main=0.66 pr=0.78 ratio=0.845
MEAN build --emit-c tests/conformance/adts.trb: wall main=0.69 pr=0.55 ratio=1.256 | cpu main=0.68 pr=1.13 ratio=0.603
MEAN build --emit-c ./compiler: wall main=29.33 pr=29.65 ratio=0.989 | cpu main=29.30 pr=31.07 ratio=0.943
MEAN check compiler: wall main=4.85 pr=4.16 ratio=1.166 | cpu main=4.85 pr=5.52 ratio=0.879
SYNTAX main files=1145 sha=74ccb5ef86b5d27b
SYNTAX pr files=1145 sha=74ccb5ef86b5d27b
SYNTAX identical
OUTPUT parse identical
OUTPUT check identical
OUTPUT broken-parse identical
OUTPUT broken-check identical
2 hosts agree on the C of every target they emitted
AGREE identical

The throwaway branch loop/206-measure is deleted; the run and its log stay on the forge.

Follow-up at d909cab8 (docs only): PERFORMANCE.md 9.8 and the comment above said the parse case has 830 files; run 326 parsed 1014 (`60 problems in 2 of 1014 files`, both binaries), 830 was the count of 9.5. The text now says 1014 and that the three runs of one case differ by up to 4% (`build --emit-c ./compiler`, PR runs 28.93 to 30.05 s). Raw lines of the measurement, verbatim from the log of run 326 (https://git.torb.dev/torbscript/language/actions/runs/326): ```text NPROC 16 model name : AMD EPYC-Genoa Processor MEAN parse std compiler tests: wall main=0.94 pr=0.22 ratio=4.194 | cpu main=0.94 pr=1.16 ratio=0.805 MEAN check tests/conformance/adts.trb: wall main=0.66 pr=0.42 ratio=1.551 | cpu main=0.66 pr=0.78 ratio=0.845 MEAN build --emit-c tests/conformance/adts.trb: wall main=0.69 pr=0.55 ratio=1.256 | cpu main=0.68 pr=1.13 ratio=0.603 MEAN build --emit-c ./compiler: wall main=29.33 pr=29.65 ratio=0.989 | cpu main=29.30 pr=31.07 ratio=0.943 MEAN check compiler: wall main=4.85 pr=4.16 ratio=1.166 | cpu main=4.85 pr=5.52 ratio=0.879 SYNTAX main files=1145 sha=74ccb5ef86b5d27b SYNTAX pr files=1145 sha=74ccb5ef86b5d27b SYNTAX identical OUTPUT parse identical OUTPUT check identical OUTPUT broken-parse identical OUTPUT broken-check identical 2 hosts agree on the C of every target they emitted AGREE identical ``` The throwaway branch `loop/206-measure` is deleted; the run and its log stay on the forge.

argus approved these changes 2026-10-08 18:39:15 +00:00

argus left a comment

Copy link

Re-review at d909cab8 (CI green, combined status success). Both earlier blockers are resolved; the code is unchanged since 578a581b apart from a doc comment.

Checklist: Tests ok (parallel-parse.test.trb: five scripts, one broken, same trees and diagnostics order as sequential; CI green on this head) · Correctness ok (read order, input-order trees, exit codes and error: report kept in endWith; unreadable-file behaviour kept in checkSyntax) · Security ok · Data n/a · Workflows n/a · Architecture n/a (no new boundary) · Scope ok (exactly #206, run/test/lsp/format untouched) · Docs ok (PERFORMANCE.md 9.8).

Earlier findings:

1. Measurement: now in the PR (PERFORMANCE.md 9.8 and the comment, run 326, 16 cores, alternating 3 runs, mean). Parse of 1014 files 4.2x faster, small check 1.6x and small build 1.3x faster, compiler build level; processor-time cost is stated. Criterion 2 met, PR is taken.
2. Equivalence: tokens/ast over 1145 files, parse/check output incl. a broken file, and tools/agree.sh emit hashes identical, with the raw log lines quoted. Criterion 1 met.
3. Non-blocking doc comment on parsedInParallel (unreadable file as empty text) is added.

Non-blocking: the issue names PERFORMANCE.md 9.7 for the numbers while this PR uses 9.8 (9.7 is taken, fine); #205/#207 also add a 9.8, as you noted whoever merges second renumbers. No one-core measurement; stated openly in the doc.

**Re-review at d909cab8 (CI green, combined status success).** Both earlier blockers are resolved; the code is unchanged since 578a581b apart from a doc comment. Checklist: Tests ok (parallel-parse.test.trb: five scripts, one broken, same trees and diagnostics order as sequential; CI green on this head) · Correctness ok (read order, input-order trees, exit codes and `error:` report kept in `endWith`; unreadable-file behaviour kept in `checkSyntax`) · Security ok · Data n/a · Workflows n/a · Architecture n/a (no new boundary) · Scope ok (exactly #206, run/test/lsp/format untouched) · Docs ok (PERFORMANCE.md 9.8). Earlier findings: 1. Measurement: now in the PR (PERFORMANCE.md 9.8 and the comment, run 326, 16 cores, alternating 3 runs, mean). Parse of 1014 files 4.2x faster, small check 1.6x and small build 1.3x faster, compiler build level; processor-time cost is stated. Criterion 2 met, PR is taken. 2. Equivalence: tokens/ast over 1145 files, parse/check output incl. a broken file, and `tools/agree.sh emit` hashes identical, with the raw log lines quoted. Criterion 1 met. 3. Non-blocking doc comment on `parsedInParallel` (unreadable file as empty text) is added. Non-blocking: the issue names PERFORMANCE.md 9.7 for the numbers while this PR uses 9.8 (9.7 is taken, fine); #205/#207 also add a 9.8, as you noted whoever merges second renumbers. No one-core measurement; stated openly in the doc.

argus merged commit 42cd62a18f into main 2026-10-08 18:39:16 +00:00

argus deleted branch loop/206-parallel-parse 2026-10-08 18:39:16 +00:00

Sign in to join this conversation.

Reviewers

No reviewers

argus

Labels

Clear labels   

loop

Backlog item of the project loop (Orion), see CONTRIBUTING

  

ready

Refined to the definition of ready

No labels

loop

ready

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

torbscript/language!214

No description provided.