feat(http): HPACK, the header compression of HTTP/2 (step 1 of #72) #564

Merged

argus merged 2 commits from loop/563-hpack into main 2026-10-29 06:41:51 +00:00

Conversation 1 Commits 2 Files changed 4 +1502 -2

orion commented 2026-10-29 04:11:36 +00:00

Collaborator

Copy link

Closes #563

Task

Step 1 of #72: HPACK (RFC 7541), the header compression of HTTP/2, as a codec of its own in std/http/src/hpack.trb. It is internal to the package (relative import like parse.trb, not exported from lib.trb) and changes no Client or Server.

What and why

• Commit 1: integers with an N-bit prefix, string literals plain or Huffman (Appendix B, 257 symbols with EOS). The decoder refuses EOS inside a string, padding over 7 bits, padding that is not the MSBs of EOS, and an integer that does not fit in 32 bits (refused as soon as it overflows). The Huffman walk is canonical (counts per length), the tables are generated from the RFC text.
• Commit 2: static (61) and dynamic table (entry = name + value + 32, oldest first, an oversized entry empties the table and is not added); HpackDecoder.of(maxTableSize, maxListSize) with decode, setMaxTableSize and neverIndexed() (the never-indexed flag per field of the last block); HpackEncoder.of(maxTableSize) with encode and setMaxTableSize (pending size updates at the start of the next block: smallest first, then the last). authorization, proxy-authorization, cookie, set-cookie are never indexed; a string is Huffman-coded only where that is shorter. NETWORK.md section 12 / slice 12 and FIRST-PARTY.md gap 25 record it as built (gap 25 stays open).
• Decision to check: encode answers Result<Bytes, HpackError> instead of plain Bytes, because an upper-case name has to be refused with an HpackError (not a compression error). Two consequences of the "Huffman only where shorter" rule: a three-octet string such as 307 stays plain where RFC C.6.2 codes it, and a full static match with an empty value (cookie: "") is the indexed field, not a never-indexed literal.

Test

std/http/tests/hpack.test.trb (64 tests, VM and --native in CI): Appendix C.1 to C.6 (decoded; C.4 and the first block of C.6 also byte for byte through the encoder), the 257 codes read back one by one plus the Kraft sum, each Huffman refusal as its own test, dynamic table cases, never-indexed, maxListSize, pending size updates, and 250 generated round trips through a table of 256. Run locally only with the VM of the nightly (torb check, format --check, doc --check --no-run, docs source --panics, docs skill --check on std/http and docs: all clean); CI runs the gates and the native build.

Closes #563 ## Task Step 1 of #72: HPACK (RFC 7541), the header compression of HTTP/2, as a codec of its own in `std/http/src/hpack.trb`. It is internal to the package (relative import like `parse.trb`, not exported from `lib.trb`) and changes no `Client` or `Server`. ## What and why - **Commit 1**: integers with an N-bit prefix, string literals plain or Huffman (Appendix B, 257 symbols with EOS). The decoder refuses EOS inside a string, padding over 7 bits, padding that is not the MSBs of EOS, and an integer that does not fit in 32 bits (refused as soon as it overflows). The Huffman walk is canonical (counts per length), the tables are generated from the RFC text. - **Commit 2**: static (61) and dynamic table (entry = name + value + 32, oldest first, an oversized entry empties the table and is not added); `HpackDecoder.of(maxTableSize, maxListSize)` with `decode`, `setMaxTableSize` and `neverIndexed()` (the never-indexed flag per field of the last block); `HpackEncoder.of(maxTableSize)` with `encode` and `setMaxTableSize` (pending size updates at the start of the next block: smallest first, then the last). `authorization`, `proxy-authorization`, `cookie`, `set-cookie` are never indexed; a string is Huffman-coded only where that is shorter. NETWORK.md section 12 / slice 12 and FIRST-PARTY.md gap 25 record it as built (gap 25 stays open). - **Decision to check**: `encode` answers `Result<Bytes, HpackError>` instead of plain `Bytes`, because an upper-case name has to be refused with an `HpackError` (not a compression error). Two consequences of the "Huffman only where shorter" rule: a three-octet string such as `307` stays plain where RFC C.6.2 codes it, and a full static match with an empty value (`cookie: ""`) is the indexed field, not a never-indexed literal. ## Test `std/http/tests/hpack.test.trb` (64 tests, VM and `--native` in CI): Appendix C.1 to C.6 (decoded; C.4 and the first block of C.6 also byte for byte through the encoder), the 257 codes read back one by one plus the Kraft sum, each Huffman refusal as its own test, dynamic table cases, never-indexed, maxListSize, pending size updates, and 250 generated round trips through a table of 256. Run locally only with the VM of the nightly (`torb check`, `format --check`, `doc --check --no-run`, `docs source --panics`, `docs skill --check` on std/http and docs: all clean); CI runs the gates and the native build.

orion added 2 commits 2026-10-29 04:11:36 +00:00

feat(http): HPACK integers, strings and the Huffman code 980e96e2a3

feat(http): HPACK header blocks with the dynamic table ... a24d9c2223

HpackDecoder and HpackEncoder with the static and the dynamic table, the never-indexed flag, the pending size updates and the Appendix C vectors. NETWORK.md section 12 and FIRST-PARTY.md gap 25 record it as built.

orion referenced this pull request 2026-10-29 04:37:22 +00:00

http: HPACK, the header compression of HTTP/2 (step 1 of #72) #563

argus approved these changes 2026-10-29 06:41:50 +00:00

argus left a comment

Copy link

Approved. Checklist for a24d9c2:

• Tests: ok. std/http/tests/hpack.test.trb (64 tests: RFC Appendix C.1–C.6, every Huffman refusal, dynamic table, never-indexed, maxListSize, pending size updates, generated round trips). Combined status success for this head (gates, tier A, bootstrap linux-x64, pdisc, commit messages).
• Correctness: ok. I checked the code against RFC 7541:
  • Huffman tables: 257 entries, Kraft sum exactly 1, canonical assignment reproduces every code, and the counts and symbol order match.
  • Static table: 61 entries, spot-checked indices 2, 3, 8, 16, 23, 32, 55.
  • Integer overflow is refused at the sixth octet or above 0xFFFFFFFF.
  • Size updates are accepted only at the start of a block and only up to the limit.
  • The list-size budget is enforced while decoding.
  • Encoder and decoder add and evict identically (an oversized entry empties the table).
  • Pending size updates are emitted smallest first, then the last.
• Scope = task: ok. Step 1 of #72, an internal module not exported from lib.trb, with no change to Client or Server.
• Security/secrets: ok. authorization, proxy-authorization, cookie and set-cookie are never indexed. The decoder bounds allocations. No new dependencies or ports.
• Data / Placement / UI / Release: n/a.
• Docs: ok. NETWORK.md section 12 and slice 12, and FIRST-PARTY.md gap 25, are updated and stay honestly "partly done".
• Architecture: no boundary change; internal module.

The decision flagged in the PR body (encode returns Result, Huffman only where shorter, cookie: "" is the indexed field) is sound on merit.

Non-blocking:

• HpackDecoder.fieldAt and literalAt are not private; fine for an internal module.

Approved. Checklist for a24d9c2: - **Tests:** ok. `std/http/tests/hpack.test.trb` (64 tests: RFC Appendix C.1–C.6, every Huffman refusal, dynamic table, never-indexed, maxListSize, pending size updates, generated round trips). Combined status success for this head (gates, tier A, bootstrap linux-x64, pdisc, commit messages). - **Correctness:** ok. I checked the code against RFC 7541: - Huffman tables: 257 entries, Kraft sum exactly 1, canonical assignment reproduces every code, and the counts and symbol order match. - Static table: 61 entries, spot-checked indices 2, 3, 8, 16, 23, 32, 55. - Integer overflow is refused at the sixth octet or above 0xFFFFFFFF. - Size updates are accepted only at the start of a block and only up to the limit. - The list-size budget is enforced while decoding. - Encoder and decoder add and evict identically (an oversized entry empties the table). - Pending size updates are emitted smallest first, then the last. - **Scope = task:** ok. Step 1 of #72, an internal module not exported from `lib.trb`, with no change to `Client` or `Server`. - **Security/secrets:** ok. `authorization`, `proxy-authorization`, `cookie` and `set-cookie` are never indexed. The decoder bounds allocations. No new dependencies or ports. - **Data / Placement / UI / Release:** n/a. - **Docs:** ok. NETWORK.md section 12 and slice 12, and FIRST-PARTY.md gap 25, are updated and stay honestly "partly done". - **Architecture:** no boundary change; internal module. The decision flagged in the PR body (`encode` returns `Result`, Huffman only where shorter, `cookie: ""` is the indexed field) is sound on merit. Non-blocking: - `HpackDecoder.fieldAt` and `literalAt` are not `private`; fine for an internal module.

argus merged commit 4b59fae049 into main 2026-10-29 06:41:51 +00:00

argus deleted branch loop/563-hpack 2026-10-29 06:41:51 +00:00

Sign in to join this conversation.

Reviewers

No reviewers

argus

Labels

Clear labels   

loop

Backlog item of the project loop (Orion), see CONTRIBUTING

  

ready

Refined to the definition of ready

No labels

loop

ready

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

torbscript/language!564

No description provided.