collections: rebuild ArrayList, the hash tables and the tries on Buffer #55

Open
opened 2026-10-05 21:07:15 +00:00 by orion · 7 comments
Collaborator

Context

With Buffer<Item>, the collections become TorbScript instead of runtime code.

Source

Acceptance criteria

  • ArrayList, the hash tables and the tries in TorbScript on Buffer
  • Benchmarks no slower than before
  • Their runtime natives removed

Dependencies

  • Depends on #54 (core: Buffer<Item> as the heap storage primitive)

Notes

Adds or changes a native, syntax or a name the compiler looks up: land it in two commits (teach both forms, refresh the seed, then migrate), as CLAUDE.md "Seed and breaking changes" says.

Moved from the planning lists of the docs; the design record stays the specification.

Steps

  • Step 1: design record docs/design/COLLECTION-STORAGE.md (#413)
  • Step 2: slice 1, commit 1 of 2, collection names declared in std/ are ordinary types, Buffer.holdsClose, cost baselines (#468, PR #474)
  • Step 3: slice 1, commit 2 of 2, ArrayList over Buffer, its 12 rows removed (#526)
  • Slice 2: TrieList delegating to ArrayList (filed when step 3 lands)
  • Slice 3: HashMap over Buffer
  • Slice 4: HashSet over Buffer, runtime/map.c removed
  • Slice 5: TrieMap and TrieSet delegating
  • Slices 6 and 7: persistent tries, only if trie-fork and the ratios hold
## Context With `Buffer<Item>`, the collections become TorbScript instead of runtime code. ## Source - [ROADMAP.md, the kernel of natives](https://git.torb.dev/torbscript/language/src/commit/393ef2608e4a60b8ff76d8b6b408951b822d887d/docs/ROADMAP.md#L115) - [COLLECTIONS.md gap 6](https://git.torb.dev/torbscript/language/src/commit/393ef2608e4a60b8ff76d8b6b408951b822d887d/docs/design/COLLECTIONS.md#L925) ## Acceptance criteria - [ ] `ArrayList`, the hash tables and the tries in TorbScript on `Buffer` - [ ] Benchmarks no slower than before - [ ] Their runtime natives removed ## Dependencies - Depends on #54 (core: `Buffer<Item>` as the heap storage primitive) ## Notes Adds or changes a native, syntax or a name the compiler looks up: land it in two commits (teach both forms, refresh the seed, then migrate), as `CLAUDE.md` "Seed and breaking changes" says. _Moved from the planning lists of the docs; the design record stays the specification._ ## Steps - [x] Step 1: design record `docs/design/COLLECTION-STORAGE.md` (#413) - [x] Step 2: slice 1, commit 1 of 2, collection names declared in `std/` are ordinary types, `Buffer.holdsClose`, cost baselines (#468, PR #474) - [ ] Step 3: slice 1, commit 2 of 2, `ArrayList` over `Buffer`, its 12 rows removed (#526) - [ ] Slice 2: `TrieList` delegating to `ArrayList` (filed when step 3 lands) - [ ] Slice 3: `HashMap` over `Buffer` - [ ] Slice 4: `HashSet` over `Buffer`, `runtime/map.c` removed - [ ] Slice 5: `TrieMap` and `TrieSet` delegating - [ ] Slices 6 and 7: persistent tries, only if `trie-fork` and the ratios hold
orion added this to the Kernel of natives milestone 2026-10-05 21:07:15 +00:00
Author
Collaborator

[loop:torbscript] intake: in scope (author orion, label loop), sources ROADMAP.md kernel of natives and COLLECTIONS.md gap 6, not a duplicate. Large and seed-relevant (two-commit rule), performance claim ("no slower") is measured in forge CI; refine splits it per collection. Depends on #54. next: waiting (dependency #54 and ready buffer full: #4, #163, #164, #223 are ready) until 2026-10-10T04:00:00Z

[loop:torbscript] intake: in scope (author orion, label loop), sources ROADMAP.md kernel of natives and COLLECTIONS.md gap 6, not a duplicate. Large and seed-relevant (two-commit rule), performance claim ("no slower") is measured in forge CI; refine splits it per collection. Depends on #54. next: waiting (dependency #54 and ready buffer full: #4, #163, #164, #223 are ready) until 2026-10-10T04:00:00Z
Author
Collaborator

[loop:torbscript] claim refine by session:loop-torbscript-20261023 until 2026-10-23T06:16:00Z

[loop:torbscript] claim refine by session:loop-torbscript-20261023 until 2026-10-23T06:16:00Z
Author
Collaborator

[loop:torbscript] refine: epic, not ready; step 1 filed as #413 and marked ready (not this issue), next: waiting (epic waits on its steps)

decided by the loop: #55 is too large for one PR. Why: the compiler names ArrayList in about 200 places (list literals, element access and write lowering, the manifest rows, RuntimeKind.ListStorage); HashMap/HashSet stand on runtime/map.c, which #56 is changing (torb_map_slot); TrieList/TrieMap/TrieSet are documented aliases of the flat storage today, not tries; and Buffer shares ArrayList's runtime list, so the natives only go away once the representation, the O(1) shared ArrayList.slice against the copying Buffer.slice, and the order against #52/#53/#56 are decided. #54, #299 and #300 are closed, so the dependency is met.

Step 1 is #413, a design record (docs/design/COLLECTION-STORAGE.md): representation of ArrayList, hash table layout against runtime/map.c, tries (real or aliases), the benchmarks that bound each slice ("no slower" as a ratio on the same forge CI runner), the manifest rows each slice removes, and the slice list. Documentation only, no seed impact. Keyword search of open and closed issues (Buffer, collections, ArrayList, Trie, HashMap, storage, design record) found no duplicate; #52/#407 and #53/#410 are related natives work, not the same.

Remaining steps, filed one at a time when each starts, each under the two-commit rule where it adds or removes a native: ArrayList on Buffer, the hash tables (after #56 lands), the tries, and the removal of the leftover runtime natives. The seed refresh stays Torben's. Tokens used by this refine run: about 110k.

[loop:torbscript] refine: epic, not `ready`; step 1 filed as #413 and marked `ready` (not this issue), next: waiting (epic waits on its steps) decided by the loop: #55 is too large for one PR. Why: the compiler names `ArrayList` in about 200 places (list literals, element access and write lowering, the manifest rows, `RuntimeKind.ListStorage`); `HashMap`/`HashSet` stand on `runtime/map.c`, which #56 is changing (`torb_map_slot`); `TrieList`/`TrieMap`/`TrieSet` are documented aliases of the flat storage today, not tries; and `Buffer` shares `ArrayList`'s runtime list, so the natives only go away once the representation, the O(1) shared `ArrayList.slice` against the copying `Buffer.slice`, and the order against #52/#53/#56 are decided. #54, #299 and #300 are closed, so the dependency is met. Step 1 is #413, a design record (`docs/design/COLLECTION-STORAGE.md`): representation of `ArrayList`, hash table layout against `runtime/map.c`, tries (real or aliases), the benchmarks that bound each slice ("no slower" as a ratio on the same forge CI runner), the manifest rows each slice removes, and the slice list. Documentation only, no seed impact. Keyword search of open and closed issues (Buffer, collections, ArrayList, Trie, HashMap, storage, design record) found no duplicate; #52/#407 and #53/#410 are related natives work, not the same. Remaining steps, filed one at a time when each starts, each under the two-commit rule where it adds or removes a native: `ArrayList` on `Buffer`, the hash tables (after #56 lands), the tries, and the removal of the leftover runtime natives. The seed refresh stays Torben's. Tokens used by this refine run: about 110k.
Author
Collaborator

[loop:torbscript] claim refine by session:loop-torbscript-20261024 until 2026-10-24T09:10:00Z

[loop:torbscript] claim refine by session:loop-torbscript-20261024 until 2026-10-24T09:10:00Z
Author
Collaborator

[loop:torbscript] refine: step 2 filed as #468 and marked ready, next: waiting (epic waits on its steps, #468 next). Step 1 (#413) verified landed on main (docs/design/COLLECTION-STORAGE.md present). Tokens used: about 60k

[loop:torbscript] refine: step 2 filed as #468 and marked ready, next: waiting (epic waits on its steps, #468 next). Step 1 (#413) verified landed on main (docs/design/COLLECTION-STORAGE.md present). Tokens used: about 60k
Author
Collaborator

[loop:torbscript] claim refine (next step of the epic after #468) by session:loop-torbscript-20261027 until 2026-10-27T05:30Z

[loop:torbscript] claim refine (next step of the epic after #468) by session:loop-torbscript-20261027 until 2026-10-27T05:30Z
Author
Collaborator

[loop:torbscript] refine: #468 verified landed on main (PR #474, merged 2026-10-25) and the published seed c72cabec00ed contains it; step 3 filed as #526 (slice 1 commit 2 of 2, ArrayList over Buffer) and marked ready; checklist added to this issue's body. Tokens used: about 120k. next: waiting (epic waits on its steps, #526 next)

[loop:torbscript] refine: #468 verified landed on main (PR #474, merged 2026-10-25) and the published seed `c72cabec00ed` contains it; step 3 filed as #526 (slice 1 commit 2 of 2, `ArrayList` over `Buffer`) and marked `ready`; checklist added to this issue's body. Tokens used: about 120k. next: waiting (epic waits on its steps, #526 next)
Sign in to join this conversation.
No labels
loop
ready
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
torbscript/language#55
No description provided.