http: HPACK, the header compression of HTTP/2 (step 1 of #72)

orion commented

2026-10-29 03:28:26 +00:00

Collaborator

Goal

Step 1 of #72 (HTTP/2): HPACK (RFC 7541), the header compression of HTTP/2, as a codec of its own in std/http, with no connection, no frames and no change to what a Client or a Server does. Every later step of #72 reads and writes header blocks through it.

Rules

A new module std/http/src/hpack.trb (internal to the package, imported by relative path like parse.trb):

Integers with an N-bit prefix (RFC 7541 section 5.1) and string literals, plain or Huffman (section 5.2). The Huffman code is the table of Appendix B (257 symbols, EOS included). Decoding refuses EOS inside a string, padding of more than 7 bits, padding that is not the most significant bits of EOS, and an integer that does not fit in 32 bits (refused as soon as it overflows, not after reading all its continuation octets).
The static table (Appendix A, 61 entries) and the dynamic table (section 4): entry size is name + value + 32 octets, eviction oldest first, an entry larger than the table empties it and is not added.
HpackDecoder (of(maxTableSize, maxListSize)): decode(block): Result<List<(name: String, value: String)>, HpackError> reads the representations of section 6 (indexed, literal with incremental indexing, without indexing, never indexed, dynamic table size update). Refused as a compression error: index 0, an index past the tables, a size update above the limit the decoder was given or after the first header of a block, a string that is cut off, and a decoded list over maxListSize (the sum of name + value + 32 per RFC 9113 section 6.5.2, counted while decoding so a large block does not allocate first). setMaxTableSize(size) is how a SETTINGS change reaches it (the shrink is applied at once, the encoder must announce it).
HpackEncoder (of(maxTableSize)): encode(fields): Buffer<UInt8>. Encoder policy, decided by the loop: a full match in a table is an indexed field; otherwise a literal with incremental indexing, except authorization, proxy-authorization, cookie and set-cookie, which are sent never indexed (section 7.1.3, as Go's net/http2 does); a string is Huffman-coded only where that is shorter. Names are lower-cased by the caller's contract (RFC 9113 section 8.2.1): the encoder refuses an upper-case name with HpackError. setMaxTableSize(size) shrinks the table and emits the required size update at the start of the next block (the smallest size in between first, then the final one, section 4.2).
HpackError is a type of its own with isCompressionError(); mapping it to COMPRESSION_ERROR (a connection error, RFC 9113 section 4.3) is the connection's job in a later step.

Acceptance criteria

std/http/tests/hpack.test.trb (VM and --native, as the other std/http/tests):

The worked examples of RFC 7541 Appendix C pass byte for byte, decoding and encoding: C.2 (single representations), C.3 and C.4 (three requests on one dynamic table without and with Huffman), C.5 and C.6 (three responses with eviction at table size 256, without and with Huffman). The encoder is compared with the Huffman examples through its own policy only where it would choose the same representation; otherwise the test decodes the encoder's output and compares fields.
The Huffman code: every one of the 257 symbols round-trips, and each refusal above (EOS, padding, overflow, cut-off string) is its own test.
Dynamic table: eviction order, an entry larger than the table, a size update to 0 and back, setMaxTableSize shrinking with the pending update, a size update above the limit refused.
Never-indexed names stay out of the dynamic table and are decoded as never-indexed (the decoder reports the flag so a proxy could keep it; a field type (name, value) is enough for the callers of later steps, the flag is a documented, tested detail of HpackDecoder).
A round trip of at least 200 generated header lists (a fixed seed) through one encoder and one decoder with a table of 256 octets, so eviction happens, gives the same lists back.
maxListSize stops a decode of an oversized block before the whole list exists.

tools/gates.sh a and b green in forge CI. NETWORK.md section 12 gets an "as built" paragraph for HPACK and slice 12's row says "Partly done: HPACK (step 1 of #72)"; FIRST-PARTY.md gap 25 gets the same note (it stays open); skills/ regenerated if the docs change reaches them.

Design record / seed

Implements NETWORK.md slice 12 (HTTP/2). Seed: no (std only, no syntax, no native, no string lookup by the compiler).

Size

One PR, two Conventional Commits: feat(http): HPACK integers, strings and the Huffman code, then feat(http): HPACK header blocks with the dynamic table (with the docs).

Dependencies

#118 (landed). Part of #72; steps 2 to 6 of #72 follow just in time.

## Goal Step 1 of #72 (HTTP/2): HPACK (RFC 7541), the header compression of HTTP/2, as a codec of its own in `std/http`, with no connection, no frames and no change to what a `Client` or a `Server` does. Every later step of #72 reads and writes header blocks through it. ## Rules A new module `std/http/src/hpack.trb` (internal to the package, imported by relative path like `parse.trb`): - **Integers** with an N-bit prefix (RFC 7541 section 5.1) and **string literals**, plain or Huffman (section 5.2). The Huffman code is the table of Appendix B (257 symbols, `EOS` included). Decoding refuses `EOS` inside a string, padding of more than 7 bits, padding that is not the most significant bits of `EOS`, and an integer that does not fit in 32 bits (refused as soon as it overflows, not after reading all its continuation octets). - **The static table** (Appendix A, 61 entries) and the **dynamic table** (section 4): entry size is `name + value + 32` octets, eviction oldest first, an entry larger than the table empties it and is not added. - **`HpackDecoder`** (`of(maxTableSize, maxListSize)`): `decode(block): Result<List<(name: String, value: String)>, HpackError>` reads the representations of section 6 (indexed, literal with incremental indexing, without indexing, never indexed, dynamic table size update). Refused as a compression error: index 0, an index past the tables, a size update above the limit the decoder was given or after the first header of a block, a string that is cut off, and a decoded list over `maxListSize` (the sum of `name + value + 32` per RFC 9113 section 6.5.2, counted while decoding so a large block does not allocate first). `setMaxTableSize(size)` is how a SETTINGS change reaches it (the shrink is applied at once, the encoder must announce it). - **`HpackEncoder`** (`of(maxTableSize)`): `encode(fields): Buffer<UInt8>`. Encoder policy, decided by the loop: a full match in a table is an indexed field; otherwise a literal with incremental indexing, except `authorization`, `proxy-authorization`, `cookie` and `set-cookie`, which are sent **never indexed** (section 7.1.3, as Go's `net/http2` does); a string is Huffman-coded only where that is shorter. Names are lower-cased by the caller's contract (RFC 9113 section 8.2.1): the encoder refuses an upper-case name with `HpackError`. `setMaxTableSize(size)` shrinks the table and emits the required size update at the start of the next block (the smallest size in between first, then the final one, section 4.2). - `HpackError` is a type of its own with `isCompressionError()`; mapping it to `COMPRESSION_ERROR` (a connection error, RFC 9113 section 4.3) is the connection's job in a later step. ## Acceptance criteria `std/http/tests/hpack.test.trb` (VM and `--native`, as the other `std/http/tests`): 1. The worked examples of RFC 7541 Appendix C pass byte for byte, decoding and encoding: C.2 (single representations), C.3 and C.4 (three requests on one dynamic table without and with Huffman), C.5 and C.6 (three responses with eviction at table size 256, without and with Huffman). The encoder is compared with the Huffman examples through its own policy only where it would choose the same representation; otherwise the test decodes the encoder's output and compares fields. 2. The Huffman code: every one of the 257 symbols round-trips, and each refusal above (EOS, padding, overflow, cut-off string) is its own test. 3. Dynamic table: eviction order, an entry larger than the table, a size update to 0 and back, `setMaxTableSize` shrinking with the pending update, a size update above the limit refused. 4. Never-indexed names stay out of the dynamic table and are decoded as never-indexed (the decoder reports the flag so a proxy could keep it; a field type `(name, value)` is enough for the callers of later steps, the flag is a documented, tested detail of `HpackDecoder`). 5. A round trip of at least 200 generated header lists (a fixed seed) through one encoder and one decoder with a table of 256 octets, so eviction happens, gives the same lists back. 6. `maxListSize` stops a decode of an oversized block before the whole list exists. `tools/gates.sh a` and `b` green in forge CI. NETWORK.md section 12 gets an "as built" paragraph for HPACK and slice 12's row says "Partly done: HPACK (step 1 of #72)"; FIRST-PARTY.md gap 25 gets the same note (it stays open); `skills/` regenerated if the docs change reaches them. ## Design record / seed Implements NETWORK.md slice 12 (HTTP/2). Seed: no (std only, no syntax, no native, no string lookup by the compiler). ## Size One PR, two Conventional Commits: `feat(http): HPACK integers, strings and the Huffman code`, then `feat(http): HPACK header blocks with the dynamic table` (with the docs). ## Dependencies #118 (landed). Part of #72; steps 2 to 6 of #72 follow just in time.

orion added the

loop

ready

labels

2026-10-29 03:28:26 +00:00

orion referenced this issue

2026-10-29 03:28:45 +00:00

http: HTTP/2 #72

orion referenced this issue

2026-10-29 03:28:52 +00:00

http: HTTP/2 #72

orion commented

2026-10-29 03:50:16 +00:00

Author

Collaborator

[loop:torbscript] claim implement by session:loop-torbscript-20261029 until 2026-10-29T09:50:16Z

orion referenced this issue from a pull request that will close it,

2026-10-29 04:11:36 +00:00

feat(http): HPACK, the header compression of HTTP/2 (step 1 of #72) #564

orion commented

2026-10-29 04:37:22 +00:00

Author

Collaborator

[loop:torbscript] implement: HPACK (std/http/src/hpack.trb, two commits) is up as #564 (Closes #563). Locally clean in the nightly VM (64 tests, format, doc and panics checks, skills up to date); on forge CI the commit-message, scope, runner, pdisc and bootstrap jobs are green, gates and tier A were still queued after 25 minutes. Decision to check in review: encode answers Result<Bytes, HpackError> because of the upper-case-name refusal, and a Huffman string is used only where strictly shorter. next: waiting (forge CI on PR #564: gates, tier A)

[loop:torbscript] implement: HPACK (std/http/src/hpack.trb, two commits) is up as https://git.torb.dev/torbscript/language/pulls/564 (Closes #563). Locally clean in the nightly VM (64 tests, format, doc and panics checks, skills up to date); on forge CI the commit-message, scope, runner, pdisc and bootstrap jobs are green, gates and tier A were still queued after 25 minutes. Decision to check in review: encode answers Result<Bytes, HpackError> because of the upper-case-name refusal, and a Huffman string is used only where strictly shorter. next: waiting (forge CI on PR #564: gates, tier A)