text: `String.toNfc()`, `toNfd()`, `toNfkc()` and `toNfkd()` (commit 2 of 2)

orion commented

2026-10-13 22:53:26 +00:00

Collaborator

Context

Commit 2 of 2 for normalization (the compiler side is #362). std/text declares String.toNfc(), toNfd(), toNfkc() and toNfkd() and the docs and conformance programs that go with them. It closes FIRST-PARTY.md gap 23's std/text half (SASLprep itself belongs to the SCRAM slice that needs it) and gives std/dns the NFC it needs (DNS.md section 3).

Depends on

#362, merged and in a published seed (seeds.txt on the seeds release lists a seed at or after the merge commit). Until then this issue is not ready.

Acceptance criteria

std/text/src/lib.trb: native fn toNfc(): Self, toNfd(), toNfkc(), toNfkd() on String with doc comments (# Examples, # Pitfalls: normalization changes bytes, compare normalized texts, NFKC loses distinctions, not a security boundary by itself) in the style of the neighbours.
A conformance program tests/conformance/text-normalization.trb (+ .expected, .allocations), identical in the VM and natively: precomposed against decomposed ü, Hangul syllables and jamo, a ccc reordering case, a singleton (Ω U+2126), compatibility (ﬁ, ①, Ａ, ㌀), an ASCII text (storage shared), the empty text, and nf(nf(x)) == nf(x).
A std test (std/text/tests/) that runs a committed excerpt of NormalizationTest.txt of the pinned version through all four methods; the whole file stays the runtime test of the previous issue.
docs/standard-library/ text page, docs/design/FIRST-PARTY.md gap 23 row marked built for the normalization part, docs/design/DNS.md slice 9 row text updated ("NFC: built"), docs/ROADMAP.md line about the natives updated; torb docs skill docs skills regenerated and committed.
Tier A and B green in forge CI.

Seed impact

No new native here (it is known to the seed), but a std name is added: the declaration is what the previous commit made safe. Nothing under seed/; no seed refresh needed after this merge.

Size

One PR, one Conventional Commit: feat(text): normalization forms of a String.

## Context Commit 2 of 2 for normalization (the compiler side is #362). `std/text` declares `String.toNfc()`, `toNfd()`, `toNfkc()` and `toNfkd()` and the docs and conformance programs that go with them. It closes [FIRST-PARTY.md gap 23](https://git.torb.dev/torbscript/language/src/commit/393ef2608e4a60b8ff76d8b6b408951b822d887d/docs/design/FIRST-PARTY.md#L283)'s `std/text` half (SASLprep itself belongs to the SCRAM slice that needs it) and gives `std/dns` the NFC it needs ([DNS.md section 3](https://git.torb.dev/torbscript/language/src/commit/393ef2608e4a60b8ff76d8b6b408951b822d887d/docs/design/DNS.md#L119)). ## Depends on #362, **merged and in a published seed** (`seeds.txt` on the `seeds` release lists a seed at or after the merge commit). Until then this issue is not `ready`. ## Acceptance criteria - [ ] `std/text/src/lib.trb`: `native fn toNfc(): Self`, `toNfd()`, `toNfkc()`, `toNfkd()` on `String` with doc comments (`# Examples`, `# Pitfalls`: normalization changes bytes, compare normalized texts, NFKC loses distinctions, not a security boundary by itself) in the style of the neighbours. - [ ] A conformance program `tests/conformance/text-normalization.trb` (+ `.expected`, `.allocations`), identical in the VM and natively: precomposed against decomposed `ü`, Hangul syllables and jamo, a ccc reordering case, a singleton (`Ω` U+2126), compatibility (`ﬁ`, `①`, `Ａ`, `㌀`), an ASCII text (storage shared), the empty text, and `nf(nf(x)) == nf(x)`. - [ ] A std test (`std/text/tests/`) that runs a committed excerpt of `NormalizationTest.txt` of the pinned version through all four methods; the whole file stays the runtime test of the previous issue. - [ ] `docs/standard-library/` text page, `docs/design/FIRST-PARTY.md` gap 23 row marked built for the normalization part, `docs/design/DNS.md` slice 9 row text updated ("NFC: built"), `docs/ROADMAP.md` line about the natives updated; `torb docs skill docs skills` regenerated and committed. - [ ] Tier A and B green in forge CI. ## Seed impact No new native here (it is known to the seed), but a std name is added: the declaration is what the previous commit made safe. Nothing under `seed/`; no seed refresh needed after this merge. ## Size One PR, one Conventional Commit: `feat(text): normalization forms of a String`.

orion added the

loop

label

2026-10-13 22:53:26 +00:00

orion referenced this issue

2026-10-13 22:53:36 +00:00

text: pinned Unicode data, generator and the character tables (slice 1 of the full Unicode tables) #36

orion referenced this issue

2026-10-13 22:53:40 +00:00

text: normalization data and natives, teach the compiler (commit 1 of 2 for NFC and NFKC) #362

orion referenced this issue

2026-10-13 22:53:41 +00:00