language server: reuse the declaration sites and the implementation index across keystrokes

orion commented

2026-10-05 21:07:10 +00:00

Collaborator

Context

LANGUAGE-SERVER.md section 15 lists what the language server still pays on every keystroke. This issue took the whole list; refinement split it into four pieces, each one PR with its own measurement:

this issue: the declaration sites and the implementation index of the overlay (first bullet of section 15)
#170: semantic tokens (second bullet)
#171: the first check of a document in slices (third bullet)
#172: a rename checks the files of its name once (fourth bullet; depends on #171)

Each PR removes its own bullet from section 15 and writes what it built into the section that owns the topic.

Goal

A keystroke inside a function body no longer rebuilds the declaration sites and the implementation index of the whole program. Today checkBodiesWithin (compiler/src/semantics/checker/check.trb) calls declareSites for every module and buildImplementations over every module on each overlay run, which is 40 to 70 ms of a 90 to 240 ms keystroke on the files of section 7. The sites and the index of every module whose text did not change come from state kept with the front; only the changed module's part is redone.

Design record points (pinned)

LANGUAGE-SERVER.md section 15, first bullet: the passes depend on the spans of the changed document, which is what keeps them from being reused. This issue removes that obstacle.
Section 7: the overlay is a checker over the current front with the sites and the index of the whole program "because a body may call anything"; a body is a pure function of the signatures (TYPECHECKER.md 7.3). Other modules' sites and implementations are therefore unchanged by a body-only edit.
Section 6: refreshedFront keeps the symbols' ids and rewrites only the changed file's spans; a front is built from scratch when the outline changed, a document closed, a watched file changed on disk, and after 200 runs. The kept state has the lifetime of the front it was built over: dropped exactly where the front is built from scratch, redone for the changed module where the front is refreshed.
Section 5: the overlay answers only for the bodies inside the changed spans; what the checker reports outside them is dropped. Unchanged by this issue.

Requirements

The overlay run does not call declareSites or buildImplementations over unchanged modules. The changed module's sites (they hold its syntax nodes, in the coordinates of the current text) and its implementations (their at spans, the bySpan keys) are built again from the refreshed front.
The overlay checker generates implementations on demand (derived Equals, Show and so on) and counts those added after the run started. It must not leak that into the kept state: the kept state is copied, or the additions are undone, whichever the worker measures to be cheaper. The choice and its cost go into the record.
checkTypes, preparedChecker and torb check / torb build behave exactly as before; only the language server's overlay path changes.
Answers are identical to today's: every diagnostic, table entry, hover, definition and token the overlay produces equals what the overlay of the unchanged code produced.

Acceptance criteria

A body-only keystroke builds the sites and the implementation index for the changed module only. A test in compiler/tests/language-server.test.trb observes it (the run's Timing entries, or a counter the checker keeps for the test) over a workspace of at least three modules.
Equivalence test: for a body edit in a module that declares a type with a with list, an extend, and calls a trait implementation of another module, the overlay's diagnostics and the resolutions inside the changed body equal those of a checker that builds everything (checkBodiesWithin as it is today), including a body that needs a derived implementation.
The kept state is dropped or redone where the front is: an interface change (outline change), a closed document, a changed file on disk, and the 200th run each lead to the same answers as before, with a test for the first and the third.
The existing session tests (compiler/tests/language-server.test.trb, tests/lsp/*.expected) pass unchanged.
The cost is measured on compiler/src/semantics/checker/expression.trb (the largest file of the compiler) the way section 7 measures a keystroke (median of ten keystrokes typing a word into a local of a function in the middle of the file, the workspace check off, the pass timings declaration sites and the implementation index read before and after). The numbers and the method are written into section 7 ("What a keystroke costs") and the first bullet of section 15 is removed. The measurement is taken from a binary the forge CI built, not from a local bootstrap, and it is not a gate (no threshold in a test).
If the docs change reaches skills/, it is regenerated (torb docs skill docs skills, checked by tier A).

Seed

No. Compiler code in existing syntax; no new native, no driver flag, no std name looked up by string.

Size and commits

One PR, one or two Conventional Commits: perf(lsp): reuse the declaration sites and the implementation index across keystrokes, and optionally docs(lsp): measure a keystroke after the reuse. Tier A (sh tools/gates.sh a) in the forge CI; tier B is not needed (no compiler/src/ir, backend or runtime change).

Implementer

opus / high: compiler/src/semantics/checker/ and the language server core.

Collisions

Touches compiler/src/semantics/checker/check.trb, compiler/src/semantics/check.trb, compiler/src/language-server/analysis.trb and view.trb. #171 touches analysis.trb and server.trb as well: whichever lands second rebases. #9 (running) adds a checker table under compiler/src/semantics/checker/; the overlap is at most a neighbouring hunk. #6 and #7 (running) touch runtime, VM and benchmarks, not the language server.

## Context [LANGUAGE-SERVER.md section 15](https://git.torb.dev/torbscript/language/src/commit/068e52da9182672d56447e775c6705661eca5af2/docs/design/LANGUAGE-SERVER.md#L396) lists what the language server still pays on every keystroke. This issue took the whole list; refinement split it into four pieces, each one PR with its own measurement: - **this issue**: the declaration sites and the implementation index of the overlay (first bullet of section 15) - #170: semantic tokens (second bullet) - #171: the first check of a document in slices (third bullet) - #172: a rename checks the files of its name once (fourth bullet; depends on #171) Each PR removes its own bullet from section 15 and writes what it built into the section that owns the topic. ## Goal A keystroke inside a function body no longer rebuilds the declaration sites and the implementation index of the whole program. Today `checkBodiesWithin` (`compiler/src/semantics/checker/check.trb`) calls `declareSites` for every module and `buildImplementations` over every module on each overlay run, which is 40 to 70 ms of a 90 to 240 ms keystroke on the files of section 7. The sites and the index of every module whose text did not change come from state kept with the front; only the changed module's part is redone. ## Design record points (pinned) - [LANGUAGE-SERVER.md section 15, first bullet](https://git.torb.dev/torbscript/language/src/commit/068e52da9182672d56447e775c6705661eca5af2/docs/design/LANGUAGE-SERVER.md#L398): the passes depend on the spans of the changed document, which is what keeps them from being reused. This issue removes that obstacle. - Section 7: the overlay is a checker over the current front with the sites and the index of the whole program "because a body may call anything"; a body is a pure function of the signatures (TYPECHECKER.md 7.3). Other modules' sites and implementations are therefore unchanged by a body-only edit. - Section 6: `refreshedFront` keeps the symbols' ids and rewrites only the changed file's spans; a front is built from scratch when the outline changed, a document closed, a watched file changed on disk, and after 200 runs. The kept state has the lifetime of the front it was built over: dropped exactly where the front is built from scratch, redone for the changed module where the front is refreshed. - Section 5: the overlay answers only for the bodies inside the changed spans; what the checker reports outside them is dropped. Unchanged by this issue. ## Requirements - The overlay run does not call `declareSites` or `buildImplementations` over unchanged modules. The changed module's sites (they hold its syntax nodes, in the coordinates of the current text) and its implementations (their `at` spans, the `bySpan` keys) are built again from the refreshed front. - The overlay checker generates implementations on demand (derived `Equals`, `Show` and so on) and counts those added after the run started. It must not leak that into the kept state: the kept state is copied, or the additions are undone, whichever the worker measures to be cheaper. The choice and its cost go into the record. - `checkTypes`, `preparedChecker` and `torb check` / `torb build` behave exactly as before; only the language server's overlay path changes. - Answers are identical to today's: every diagnostic, table entry, hover, definition and token the overlay produces equals what the overlay of the unchanged code produced. ## Acceptance criteria - [ ] A body-only keystroke builds the sites and the implementation index for the changed module only. A test in `compiler/tests/language-server.test.trb` observes it (the run's `Timing` entries, or a counter the checker keeps for the test) over a workspace of at least three modules. - [ ] Equivalence test: for a body edit in a module that declares a `type` with a `with` list, an `extend`, and calls a trait implementation of another module, the overlay's diagnostics and the resolutions inside the changed body equal those of a checker that builds everything (`checkBodiesWithin` as it is today), including a body that needs a derived implementation. - [ ] The kept state is dropped or redone where the front is: an interface change (outline change), a closed document, a changed file on disk, and the 200th run each lead to the same answers as before, with a test for the first and the third. - [ ] The existing session tests (`compiler/tests/language-server.test.trb`, `tests/lsp/*.expected`) pass unchanged. - [ ] The cost is measured on `compiler/src/semantics/checker/expression.trb` (the largest file of the compiler) the way section 7 measures a keystroke (median of ten keystrokes typing a word into a local of a function in the middle of the file, the workspace check off, the pass timings `declaration sites` and `the implementation index` read before and after). The numbers and the method are written into section 7 ("What a keystroke costs") and the first bullet of section 15 is removed. The measurement is taken from a binary the forge CI built, not from a local bootstrap, and it is not a gate (no threshold in a test). - [ ] If the docs change reaches `skills/`, it is regenerated (`torb docs skill docs skills`, checked by tier A). ## Seed No. Compiler code in existing syntax; no new native, no driver flag, no std name looked up by string. ## Size and commits One PR, one or two Conventional Commits: `perf(lsp): reuse the declaration sites and the implementation index across keystrokes`, and optionally `docs(lsp): measure a keystroke after the reuse`. Tier A (`sh tools/gates.sh a`) in the forge CI; tier B is not needed (no `compiler/src/ir`, `backend` or `runtime` change). ## Implementer opus / high: `compiler/src/semantics/checker/` and the language server core. ## Collisions Touches `compiler/src/semantics/checker/check.trb`, `compiler/src/semantics/check.trb`, `compiler/src/language-server/analysis.trb` and `view.trb`. #171 touches `analysis.trb` and `server.trb` as well: whichever lands second rebases. #9 (running) adds a checker table under `compiler/src/semantics/checker/`; the overlap is at most a neighbouring hunk. #6 and #7 (running) touch runtime, VM and benchmarks, not the language server.

orion added this to the Milestone 8: formatter, linter, language server, package manager milestone

2026-10-05 21:07:10 +00:00

orion added the

loop

label

2026-10-05 21:07:10 +00:00

orion commented

2026-10-06 15:58:13 +00:00

Author

Collaborator

[loop:torbscript] intake: in scope (author orion, label loop). Implements LANGUAGE-SERVER.md section 15; design record is the specification, no collision with CONCEPT "Open Questions", no seed impact expected. Five criteria, likely split into several issues in refine. Release freeze for 0.1.0 (2026-10-06) holds new PRs, next: refine (queued, buffer >= 2 ready)

orion commented

2026-10-08 01:17:22 +00:00

Author

Collaborator

[loop:torbscript] claim refine by session:loop-torbscript-20261008 until 2026-10-08T03:17:22Z

orion changed title from ~~language server: the whole-program work left per keystroke~~ to language server: reuse the declaration sites and the implementation index across keystrokes

2026-10-08 01:22:31 +00:00

orion referenced this issue

2026-10-08 01:22:31 +00:00

language server: semantic tokens without the resolver over the whole file per request #170

orion referenced this issue

2026-10-08 01:22:32 +00:00

language server: the first check of a document in slices #171

orion referenced this issue

2026-10-08 01:22:32 +00:00

language server: a rename checks the files of its name once #172

orion added the

ready

label

2026-10-08 01:22:32 +00:00

orion referenced this issue

2026-10-08 01:22:40 +00:00

language server: semantic tokens without the resolver over the whole file per request #170

orion referenced this issue

2026-10-08 01:22:40 +00:00

language server: the first check of a document in slices #171

orion referenced this issue

2026-10-08 01:22:41 +00:00

language server: a rename checks the files of its name once #172

orion commented

2026-10-08 01:22:41 +00:00

Author

Collaborator

[loop:torbscript] refine: the five criteria cover four independent pieces, so the issue is split by section 15 bullet: this issue keeps the declaration sites and the implementation index (rewritten with goal, requirements, testable criteria, pinned record points, seed impact: no, one PR with one or two Conventional Commits, implementer opus/high), and #170 (semantic tokens), #171 (first check of a document in slices) and #172 (rename checks once, depends on #171) carry the rest, each ready with its own measurement on the largest compiler file. No owner question: all four stay inside the design record (no protocol addition, no breaking change, no seed change). Collisions: #10 and #171 share analysis.trb and view.trb (second to land rebases); #9 may touch a neighbouring hunk under compiler/src/semantics/checker/; no overlap with #6 and #7. Worker usage ~120000 tokens, next: implement