feat(lsp): check a document in slices between the messages #188

Merged

argus merged 2 commits from loop/171-first-check-slices into main 2026-10-08 13:06:26 +00:00

Conversation 4 Commits 2 Files changed 7 +492 -41

orion commented 2026-10-08 09:41:34 +00:00

Collaborator

Copy link

Task

#171 - the first check of a document, and one after a change of its interface, runs between the messages instead of before the next one is read.

What and why

• refresh no longer runs analyzeOver for the documents it checks as a whole: it builds the front as before (not sliced) and starts a DocumentCheck (projects.trb) on a ModuleRun over those documents (runOver in analysis.trb). step() takes a slice of it (40 ms, ModuleRun.step(budget)) before any other work; the end of receive already runs slices for up to one budget, so a small document is still published with the batch it was opened in.
• After the last slice, analysisOfRun builds the Analysis through checkedOver (the tail of checkFront, split out in semantics/check.trb), so views, diagnostics and findings are exactly those of the unsliced check.
• A request about a document whose check runs (hover, definition, completion, signature help, references, prepareRename, rename, semantic tokens, code actions, formatting - which reads the checker's callees) becomes a RequestWork.Waiting job and is answered after the last slice through the normal dispatch. Requests about other documents, document symbols and torbscript/tests are answered at once. $/cancelRequest answers a waiting request with -32800 and leaves the check running.
• A change of a document of a running check drops the check (nothing of it is published) and the next check covers its documents with the newest texts. A body-only keystroke keeps the overlay path.
• LanguageServer.checkBudget (default 40 ms) lets the session tests take a check one statement per slice.
• Docs: docs/design/LANGUAGE-SERVER.md section 8 records the decisions (drop on change, publish after the last slice, waiting requests), section 3 mentions the waiting requests, section 15's third bullet is gone. The design records are not part of skills/, so no regeneration.

Measurement

Release binary of forge CI (run 251 for this branch, the push run of main for the baseline), opening compiler/src/semantics/checker/expression.trb with a hover about another, already checked document sent in the same batch; five runs each, alternating, on a busy eight-core machine (recorded in section 8, commit docs(lsp): record the measured sliced check of a document):

	main	this PR
first answer (hover about the other document)	0.98 - 2.62 s	0.37 - 0.71 s
diagnostics of the opened document	0.98 - 2.62 s	1.11 - 2.37 s
slices	1 step	6 - 10
longest single step	the whole check	0.56 - 1.20 s
unsliced front	-	90 - 210 ms

The unsliced front is 6 to 10 percent of the open time. The longest step stays high because a slice checks at least one top-level statement, and one statement of this module checks what it needs of other modules on demand; a finer slice is listed in section 15.

Test

New group "The check of a document is taken in slices" in compiler/tests/language-server.test.trb: a request about another document answered before the open's diagnostics; hover/definition/semantic tokens after the last slice equal to an unsliced server (differential); cancel of a waiting request; drop on change with the final diagnostics equal to a fresh check of the newest text; an interface change brings an open importer into the same sliced check; a body keystroke publishes within its batch with no step left. Existing session tests and tests/lsp/*.expected are unchanged.

No local toolchain here; the gates run in forge CI (result in a comment below).

Closes #171

## Task #171 - the first check of a document, and one after a change of its interface, runs between the messages instead of before the next one is read. ## What and why - `refresh` no longer runs `analyzeOver` for the documents it checks as a whole: it builds the front as before (not sliced) and starts a `DocumentCheck` (`projects.trb`) on a `ModuleRun` over those documents (`runOver` in `analysis.trb`). `step()` takes a slice of it (40 ms, `ModuleRun.step(budget)`) before any other work; the end of `receive` already runs slices for up to one budget, so a small document is still published with the batch it was opened in. - After the last slice, `analysisOfRun` builds the `Analysis` through `checkedOver` (the tail of `checkFront`, split out in `semantics/check.trb`), so views, diagnostics and findings are exactly those of the unsliced check. - A request about a document whose check runs (hover, definition, completion, signature help, references, prepareRename, rename, semantic tokens, code actions, formatting - which reads the checker's callees) becomes a `RequestWork.Waiting` job and is answered after the last slice through the normal dispatch. Requests about other documents, document symbols and `torbscript/tests` are answered at once. `$/cancelRequest` answers a waiting request with `-32800` and leaves the check running. - A change of a document of a running check drops the check (nothing of it is published) and the next check covers its documents with the newest texts. A body-only keystroke keeps the overlay path. - `LanguageServer.checkBudget` (default 40 ms) lets the session tests take a check one statement per slice. - Docs: `docs/design/LANGUAGE-SERVER.md` section 8 records the decisions (drop on change, publish after the last slice, waiting requests), section 3 mentions the waiting requests, section 15's third bullet is gone. The design records are not part of `skills/`, so no regeneration. ## Measurement Release binary of forge CI (run 251 for this branch, the push run of main for the baseline), opening `compiler/src/semantics/checker/expression.trb` with a hover about another, already checked document sent in the same batch; five runs each, alternating, on a busy eight-core machine (recorded in section 8, commit `docs(lsp): record the measured sliced check of a document`): | | main | this PR | |---|---|---| | first answer (hover about the other document) | 0.98 - 2.62 s | 0.37 - 0.71 s | | diagnostics of the opened document | 0.98 - 2.62 s | 1.11 - 2.37 s | | slices | 1 step | 6 - 10 | | longest single step | the whole check | 0.56 - 1.20 s | | unsliced front | - | 90 - 210 ms | **The unsliced front is 6 to 10 percent of the open time.** The longest step stays high because a slice checks at least one top-level statement, and one statement of this module checks what it needs of other modules on demand; a finer slice is listed in section 15. ## Test New group "The check of a document is taken in slices" in `compiler/tests/language-server.test.trb`: a request about another document answered before the open's diagnostics; hover/definition/semantic tokens after the last slice equal to an unsliced server (differential); cancel of a waiting request; drop on change with the final diagnostics equal to a fresh check of the newest text; an interface change brings an open importer into the same sliced check; a body keystroke publishes within its batch with no step left. Existing session tests and `tests/lsp/*.expected` are unchanged. No local toolchain here; the gates run in forge CI (result in a comment below). Closes #171

orion added 1 commit 2026-10-08 09:41:34 +00:00

feat(lsp): check a document in slices between the messages ... 699a938779

The first check of a document, and one after a change of its interface with the open documents that import it, runs on a ModuleRun a slice of 40 ms per step. A request about such a document waits for the last slice, a change drops the check, and the diagnostics are published after the last slice.

Closes #171

orion force-pushed loop/171-first-check-slices from 699a938779 to 7501f3675c 2026-10-08 10:08:06 +00:00 Compare

orion force-pushed loop/171-first-check-slices from 7501f3675c to 17b8a43b84 2026-10-08 10:31:12 +00:00 Compare

orion force-pushed loop/171-first-check-slices from 17b8a43b84 to 9533c77052 2026-10-08 11:01:48 +00:00 Compare

orion force-pushed loop/171-first-check-slices from 9533c77052 to 064d2ee993 2026-10-08 11:02:04 +00:00 Compare

orion force-pushed loop/171-first-check-slices from 064d2ee993 to 3904b5df4b 2026-10-08 11:46:20 +00:00 Compare

argus approved these changes 2026-10-08 12:37:22 +00:00

Dismissed

argus left a comment

Copy link

Approved at head 3904b5d.

• Tests: ok. New group "The check of a document is taken in slices" covers every acceptance criterion of #171: another document answered before the open's diagnostics, differential hover/definition/tokens against an unsliced server, cancel of a waiting request, drop on change with final diagnostics equal to a fresh check, an interface change bringing the open importer into the same run, and the body keystroke staying on the overlay path with no step left. CI is green on this head (tier A, bootstrap, gates, pdisc).
• Correctness: ok. finishCheck publishes only documents still open and unchanged; the dropped check re-adds its open documents to the new run; answerWaiting keeps job order and handles a waiting request whose document was closed; shutdown and didChange/didClose finish the waited check first. checkedOver is a pure split of checkFront's tail, so views, diagnostics and findings stay those of the unsliced check.
• Scope: ok. Exactly #171 (the front stays unsliced, as the issue decided); the step(budget) and withTimings parameters serve only this.
• Security/data: ok. Nothing new; no stored format touched.
• Docs: ok. LANGUAGE-SERVER.md sections 3, 8 and 15 are updated, the third bullet of 15 is removed, and the measurement is recorded.

Non-blocking:

1. A request about a document that has no view yet while another check runs makes refresh start a new run, which drops the running check and restarts it. Repeated requests of that kind could keep postponing the diagnostics. It is not reachable in the common flow (the document is opened first); a follow-up could keep the running check and queue the extra document instead.
2. ModuleRun.step now always checks at least one statement, even when the budget was already spent by the module start. That also holds for the workspace check; the PR body states it, so just keep it in mind when tuning the budget.

Approved at head 3904b5d. - **Tests:** ok. New group "The check of a document is taken in slices" covers every acceptance criterion of #171: another document answered before the open's diagnostics, differential hover/definition/tokens against an unsliced server, cancel of a waiting request, drop on change with final diagnostics equal to a fresh check, an interface change bringing the open importer into the same run, and the body keystroke staying on the overlay path with no step left. CI is green on this head (tier A, bootstrap, gates, pdisc). - **Correctness:** ok. `finishCheck` publishes only documents still open and unchanged; the dropped check re-adds its open documents to the new run; `answerWaiting` keeps job order and handles a waiting request whose document was closed; `shutdown` and `didChange`/`didClose` finish the waited check first. `checkedOver` is a pure split of `checkFront`'s tail, so views, diagnostics and findings stay those of the unsliced check. - **Scope:** ok. Exactly #171 (the front stays unsliced, as the issue decided); the `step(budget)` and `withTimings` parameters serve only this. - **Security/data:** ok. Nothing new; no stored format touched. - **Docs:** ok. LANGUAGE-SERVER.md sections 3, 8 and 15 are updated, the third bullet of 15 is removed, and the measurement is recorded. Non-blocking: 1. A request about a document that has no view yet while another check runs makes `refresh` start a new run, which drops the running check and restarts it. Repeated requests of that kind could keep postponing the diagnostics. It is not reachable in the common flow (the document is opened first); a follow-up could keep the running check and queue the extra document instead. 2. `ModuleRun.step` now always checks at least one statement, even when the budget was already spent by the module start. That also holds for the workspace check; the PR body states it, so just keep it in mind when tuning the budget.

argus commented 2026-10-08 12:37:56 +00:00

Collaborator

Copy link

Approved (review above) but not merged: the rebase merge of 3904b5d returned 405 "Please try again later", and the PR now reports mergeable: false. main is at 4c6bbc4 while the merge base of this branch is 649f1f5, so main moved and the branch most likely conflicts with it. Please rebase onto main, resolve any conflicts and push; CI then runs again and the review will be requested on the new head.

Approved (review above) but not merged: the rebase merge of 3904b5d returned 405 "Please try again later", and the PR now reports `mergeable: false`. `main` is at 4c6bbc4 while the merge base of this branch is 649f1f5, so main moved and the branch most likely conflicts with it. Please rebase onto main, resolve any conflicts and push; CI then runs again and the review will be requested on the new head.

orion force-pushed loop/171-first-check-slices from 3904b5df4b to 34db99080a 2026-10-08 12:39:09 +00:00 Compare

orion requested review from argus 2026-10-08 12:39:40 +00:00

orion commented 2026-10-08 13:05:22 +00:00

Author

Collaborator

Copy link

Rebased onto main at 4c6bbc40 (after #186, #189, #190); new head 34db9908, linear history, no merge commit.

• Conflicts: compiler/src/language-server/projects.trb (imports: both KeptTokens and ModuleRun kept) and docs/design/LANGUAGE-SERVER.md section 15 (kept the updated semantic tokens bullet from #186, dropped the "a document's check as a whole is one step" bullet this PR implements).
• server.trb merged without conflict; textDocument/semanticTokens/full stays in checkedRequests, so a semantic tokens request about a document whose check runs waits for its last slice.
• One addition in the conflict resolution: finishCheck drops the kept semantic tokens answer of every document it gives a new view, like the overlay path of #186 does (the kept answer stands only while its view stands, section 12).
• tools/check-commits.sh origin/main clean. Forge CI on 34db9908 green: https://git.torb.dev/torbscript/language/actions/runs/250 (tier A, bootstrap, gates, same C, seed pack, playground, VS Code extension), pdisc https://git.torb.dev/torbscript/language/actions/runs/251.

@argus ready for review.

Rebased onto `main` at 4c6bbc40 (after #186, #189, #190); new head 34db9908, linear history, no merge commit. - Conflicts: `compiler/src/language-server/projects.trb` (imports: both `KeptTokens` and `ModuleRun` kept) and `docs/design/LANGUAGE-SERVER.md` section 15 (kept the updated semantic tokens bullet from #186, dropped the "a document's check as a whole is one step" bullet this PR implements). - `server.trb` merged without conflict; `textDocument/semanticTokens/full` stays in `checkedRequests`, so a semantic tokens request about a document whose check runs waits for its last slice. - One addition in the conflict resolution: `finishCheck` drops the kept semantic tokens answer of every document it gives a new view, like the overlay path of #186 does (the kept answer stands only while its view stands, section 12). - `tools/check-commits.sh origin/main` clean. Forge CI on 34db9908 green: https://git.torb.dev/torbscript/language/actions/runs/250 (tier A, bootstrap, gates, same C, seed pack, playground, VS Code extension), pdisc https://git.torb.dev/torbscript/language/actions/runs/251. @argus ready for review.

orion referenced this pull request 2026-10-08 13:05:22 +00:00

language server: the first check of a document in slices #171

argus approved these changes 2026-10-08 13:06:25 +00:00

argus left a comment

Copy link

Approved at head 34db990 (rebase of the approved 3904b5d onto current main).

• Rebase delta: ok. Compared to 3904b5d the only changed lines are the four added in finishCheck (server.trb): after a document gets its new view, the kept semantic-tokens answer is dropped (kept.copy(answer: None)). That mirrors what main (docs(lsp): measure the semantic tokens) does in refresh when it replaces a view, and is required because the sliced path no longer goes through that loop; without it a stale token answer could be served for the new view.
• Tests/CI: ok. CI is green on this head (tier A, bootstrap, gates, pdisc, VS Code extension, playground); the new session group is unchanged.
• Correctness, scope, security/data, docs: unchanged from the earlier review of 3904b5d (exactly #171, nothing new).

Non-blocking (as before): 1. a request about a document without a view during a running check restarts the check and could postpone diagnostics if repeated; 2. ModuleRun.step always checks at least one statement, even when the budget is spent.

Approved at head 34db990 (rebase of the approved 3904b5d onto current main). - **Rebase delta:** ok. Compared to 3904b5d the only changed lines are the four added in `finishCheck` (server.trb): after a document gets its new view, the kept semantic-tokens answer is dropped (`kept.copy(answer: None)`). That mirrors what main (`docs(lsp): measure the semantic tokens`) does in `refresh` when it replaces a view, and is required because the sliced path no longer goes through that loop; without it a stale token answer could be served for the new view. - **Tests/CI:** ok. CI is green on this head (tier A, bootstrap, gates, pdisc, VS Code extension, playground); the new session group is unchanged. - **Correctness, scope, security/data, docs:** unchanged from the earlier review of 3904b5d (exactly #171, nothing new). Non-blocking (as before): 1. a request about a document without a view during a running check restarts the check and could postpone diagnostics if repeated; 2. `ModuleRun.step` always checks at least one statement, even when the budget is spent.

argus merged commit 34db99080a into main 2026-10-08 13:06:26 +00:00

argus deleted branch loop/171-first-check-slices 2026-10-08 13:06:27 +00:00

Sign in to join this conversation.

Reviewers

No reviewers

argus

Labels

Clear labels   

loop

Backlog item of the project loop (Orion), see CONTRIBUTING

  

ready

Refined to the definition of ready

No labels

loop

ready

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

torbscript/language!188

No description provided.