http: 101 Switching Protocols and a duplex connection (NETWORK slice 14) #74

Closed
opened 2026-10-05 21:07:17 +00:00 by orion · 9 comments
Collaborator

Context

The duplex connection of the live UI (WEB.md's fifth request).

Source

Acceptance criteria

  • A server handler can upgrade and hold a duplex connection
  • WebSocket framing usable on it

Dependencies

None known.

Moved from the planning lists of the docs; the design record stays the specification.

## Context The duplex connection of the live UI (WEB.md's fifth request). ## Source - [NETWORK.md slice 14](https://git.torb.dev/torbscript/language/src/commit/393ef2608e4a60b8ff76d8b6b408951b822d887d/docs/design/NETWORK.md#L753) ## Acceptance criteria - [ ] A server handler can upgrade and hold a duplex connection - [ ] WebSocket framing usable on it ## Dependencies None known. _Moved from the planning lists of the docs; the design record stays the specification._
Author
Collaborator

[loop:torbscript] intake: in scope (author orion, label loop), source NETWORK.md slice 14, not a duplicate, no collision with a design record. TorbScript in std/http; WebSocket framing is a second slice, refine splits it. next: refine, waiting (ready buffer full: #4, #163, #164, #223 are ready) until 2026-10-10T04:00:00Z

[loop:torbscript] intake: in scope (author orion, label loop), source NETWORK.md slice 14, not a duplicate, no collision with a design record. TorbScript in `std/http`; WebSocket framing is a second slice, refine splits it. next: refine, waiting (ready buffer full: #4, #163, #164, #223 are ready) until 2026-10-10T04:00:00Z
Author
Collaborator

[loop:torbscript] claim refine by session:loop-torbscript-20261024 until 2026-10-24T02:35:08Z

[loop:torbscript] claim refine by session:loop-torbscript-20261024 until 2026-10-24T02:35:08Z
Author
Collaborator

[loop:torbscript] decided by the loop: #74 is the first and only step of the WebSocket work that belongs to std/http; the framing is not a second step of this issue. The issue body stays as written; this comment is the amendment and replaces its two acceptance criteria.

Why the split lands here. NETWORK.md section 13, request 5, is explicit: the server answers 101 Switching Protocols with an upgrade closure and hands it the connection's source() and sink(); "WebSocket framing is a Stage of std/web, as WEB.md proposes" (WEB.md section 9 item 5: "not in the transport"). WEB.md slice 8 (#105, which already depends on #74) owns the framing, the Sec-WebSocket-Accept handshake (Sha1 exists in std/digest) and the session task. So the old criterion "WebSocket framing usable on it" moves to #105; no new issue is needed, and none is filed. Nothing else of the epic is left for std/http: the client-side upgrade is not in slice 14 (the record names only the server), and std/web does not exist yet (#94).

Dependencies, verified on main (bc504c24). None open. Slice 13 (disconnect watch) is done and shapes the design below (watchPeer in std/http/src/server.trb); Connection (std/http/src/connection.trb) already wraps plain TCP and TLS in one Transport, so one mechanism serves http and https. Seed impact: none (no syntax, native, driver flag or looked-up std name).

Technical decisions (all inside NETWORK.md; no owner question).

  • std/http/src/message.trb: Status.switchingProtocols (101; the reason phrase already exists). public type Upgrade = (duplex: Duplex) => Task<Result<Void, HttpError>> (the same failure type as Handler). Response gets a last field var upgrade: Upgrade? = None (a default, so the positional constructors in the client and parser stay valid) and static fn upgrade(handler: Upgrade, headers: Headers = Headers()): Self, status 101, empty body, the given fields. The protocol fields (Upgrade: websocket, Connection: Upgrade, Sec-WebSocket-*) are the handler's, so the mechanism stays generic and std/web writes the WebSocket ones.
  • New std/http/src/duplex.trb: public shared type Duplex with Close over the server's Connection: fn source(chunk: Int = 65536): Source<Bytes, HttpError>, fn sink(): Sink<Bytes, HttpError>, fn remote(): SocketAddress?, fn isEncrypted(): Bool. The source first yields whatever the connection had buffered after the request head (a client may send its first frame with the request, and watchPeer may have read bytes), then reads the transport; it ends with None when the peer ends. sink().add(bytes) writes all of it, sink().end() finishes the writing half (close_notify on TLS). Re-exported from lib.trb (Duplex, Upgrade).
  • std/http/src/write.trb: a new writeUpgrade(connection, response) writes HTTP/1.1 101 <phrase>, the handler's fields (the Connection filter of isFramingField is lifted for a 101, since Connection: Upgrade is the handler's; CR/LF/NUL sanitising and the token check stay), no Content-Length, no Transfer-Encoding, no Connection: close, no body.
  • std/http/src/server.trb, serveConnection: after the handler answered, if response.upgrade is Some: (a) status other than 101, or a 101 without a closure, is answered 500 like a failed handler (a bare 101 would leave the client waiting); (b) the unread request body is drained like for any answer, and a body that cannot be drained closes the connection without the 101; (c) writeUpgrade, then the closure runs with the Duplex and the connection task awaits it; (d) afterwards closeGently, and the connection is never read as HTTP again. The head, idle and drain limits no longer apply to it; its timing is the closure's. A failing or cancelled closure closes only that connection and sends nothing (the failure stays private, as for a handler).
  • Shutdown: an upgraded connection is serving.busy = true, so shutdown(grace) waits for it up to grace and then cancels it; close() cancels at once. The closure is cancelled with the connection task, the Duplex releases with it.

Acceptance criteria (replace those of the body)

  • Status.switchingProtocols, Response.upgrade(...), Upgrade and Duplex exist as above, are re-exported from std/http, and carry doc comments with an # Examples block (an echo upgrade).
  • tests/conformance/http-upgrade.trb (with .expected, .exit, .allocations; tier B runs it in the VM and natively, byte for byte) over loopback with a raw TcpStream client: (1) a request with Upgrade: echo and Connection: Upgrade is answered 101 Switching Protocols with the handler's two fields and none of Content-Length, Transfer-Encoding, Connection: close; (2) bytes sent in the same segment as the request head arrive first at the closure's source, then later bytes in order; (3) bytes written to the sink arrive at the client, both directions interleaved over at least two round trips; (4) sink().end() ends the client's read with end of stream, and the client closing ends the source with None; (5) a plain request on a second connection is served beside the open upgrade; (6) 101 without a closure (Response.of(Status.switchingProtocols)) is answered 500; (7) shutdown(grace) with an open upgrade waits for grace and cancels the closure, observed by a shared flag like http-disconnect.trb.
  • An https variant of (1) to (4) in tests/conformance/https-upgrade.trb, built on the fixture of https-exchange.trb; if that fixture does not carry a second program cheaply, the same cases go into https-exchange.trb instead and the PR says so.
  • No regression: http-exchange, http-wire, http-disconnect, http-workers and https-exchange keep their .expected files unchanged.
  • docs/standard-library/http.md documents Duplex, Upgrade and Response.upgrade (declaration markers as for Server); NETWORK.md section 11 slice 14 becomes "Done (2026-10-24, server side; the WebSocket framing is std/web, WEB.md slice 8, #105)", section 13 row 5 says "as built"; skills/ regenerated (torb docs skill docs skills; --check is a tier A gate).
  • torb format --check clean on the touched files, tools/check-commits.sh clean, tier A and B green in forge CI.

Out of scope

WebSocket frames, masking, Sec-WebSocket-Accept, ping/pong, close handshake, permessage-deflate (#105, std/web); a client-side upgrade; HTTP/2 CONNECT and extended CONNECT (slice 12).

Size

One PR, two Conventional Commits: feat(std/http): upgrade a connection with 101 Switching Protocols and a Duplex (code, conformance programs) and docs(std/http): the upgrade in the standard-library page, NETWORK.md and skills. Worker: std/ and docs, so implementer sonnet/medium; the interplay with watchPeer and the shutdown path deserves a careful read of serveConnection before the edit.

[loop:torbscript] decided by the loop: #74 is the first and only step of the WebSocket work that belongs to `std/http`; the framing is not a second step of this issue. The issue body stays as written; this comment is the amendment and replaces its two acceptance criteria. **Why the split lands here.** NETWORK.md section 13, request 5, is explicit: the server answers `101 Switching Protocols` with an upgrade closure and hands it the connection's `source()` and `sink()`; "WebSocket framing is a `Stage` of `std/web`, as WEB.md proposes" (WEB.md section 9 item 5: "not in the transport"). WEB.md slice 8 (#105, which already depends on #74) owns the framing, the `Sec-WebSocket-Accept` handshake (`Sha1` exists in `std/digest`) and the session task. So the old criterion "WebSocket framing usable on it" moves to #105; no new issue is needed, and none is filed. Nothing else of the epic is left for `std/http`: the client-side upgrade is not in slice 14 (the record names only the server), and `std/web` does not exist yet (#94). **Dependencies, verified on `main` (bc504c24).** None open. Slice 13 (disconnect watch) is done and shapes the design below (`watchPeer` in `std/http/src/server.trb`); `Connection` (`std/http/src/connection.trb`) already wraps plain TCP and TLS in one `Transport`, so one mechanism serves `http` and `https`. **Seed impact: none** (no syntax, native, driver flag or looked-up std name). **Technical decisions (all inside NETWORK.md; no owner question).** - `std/http/src/message.trb`: `Status.switchingProtocols` (101; the reason phrase already exists). `public type Upgrade = (duplex: Duplex) => Task<Result<Void, HttpError>>` (the same failure type as `Handler`). `Response` gets a last field `var upgrade: Upgrade? = None` (a default, so the positional constructors in the client and parser stay valid) and `static fn upgrade(handler: Upgrade, headers: Headers = Headers()): Self`, status 101, empty body, the given fields. The protocol fields (`Upgrade: websocket`, `Connection: Upgrade`, `Sec-WebSocket-*`) are the handler's, so the mechanism stays generic and `std/web` writes the WebSocket ones. - New `std/http/src/duplex.trb`: `public shared type Duplex with Close` over the server's `Connection`: `fn source(chunk: Int = 65536): Source<Bytes, HttpError>`, `fn sink(): Sink<Bytes, HttpError>`, `fn remote(): SocketAddress?`, `fn isEncrypted(): Bool`. The source first yields whatever the connection had buffered after the request head (a client may send its first frame with the request, and `watchPeer` may have read bytes), then reads the transport; it ends with `None` when the peer ends. `sink().add(bytes)` writes all of it, `sink().end()` finishes the writing half (`close_notify` on TLS). Re-exported from `lib.trb` (`Duplex`, `Upgrade`). - `std/http/src/write.trb`: a new `writeUpgrade(connection, response)` writes `HTTP/1.1 101 <phrase>`, the handler's fields (the `Connection` filter of `isFramingField` is lifted for a 101, since `Connection: Upgrade` is the handler's; CR/LF/NUL sanitising and the token check stay), no `Content-Length`, no `Transfer-Encoding`, no `Connection: close`, no body. - `std/http/src/server.trb`, `serveConnection`: after the handler answered, if `response.upgrade` is `Some`: (a) status other than 101, or a 101 without a closure, is answered `500` like a failed handler (a bare 101 would leave the client waiting); (b) the unread request body is drained like for any answer, and a body that cannot be drained closes the connection without the 101; (c) `writeUpgrade`, then the closure runs with the `Duplex` and the connection task awaits it; (d) afterwards `closeGently`, and the connection is never read as HTTP again. The head, idle and drain limits no longer apply to it; its timing is the closure's. A failing or cancelled closure closes only that connection and sends nothing (the failure stays private, as for a handler). - Shutdown: an upgraded connection is `serving.busy = true`, so `shutdown(grace)` waits for it up to `grace` and then cancels it; `close()` cancels at once. The closure is cancelled with the connection task, the `Duplex` releases with it. ## Acceptance criteria (replace those of the body) - [ ] `Status.switchingProtocols`, `Response.upgrade(...)`, `Upgrade` and `Duplex` exist as above, are re-exported from `std/http`, and carry doc comments with an `# Examples` block (an echo upgrade). - [ ] `tests/conformance/http-upgrade.trb` (with `.expected`, `.exit`, `.allocations`; tier B runs it in the VM and natively, byte for byte) over loopback with a raw `TcpStream` client: (1) a request with `Upgrade: echo` and `Connection: Upgrade` is answered `101 Switching Protocols` with the handler's two fields and none of `Content-Length`, `Transfer-Encoding`, `Connection: close`; (2) bytes sent in the same segment as the request head arrive first at the closure's source, then later bytes in order; (3) bytes written to the sink arrive at the client, both directions interleaved over at least two round trips; (4) `sink().end()` ends the client's read with end of stream, and the client closing ends the source with `None`; (5) a plain request on a second connection is served beside the open upgrade; (6) `101` without a closure (`Response.of(Status.switchingProtocols)`) is answered `500`; (7) `shutdown(grace)` with an open upgrade waits for `grace` and cancels the closure, observed by a shared flag like `http-disconnect.trb`. - [ ] An `https` variant of (1) to (4) in `tests/conformance/https-upgrade.trb`, built on the fixture of `https-exchange.trb`; if that fixture does not carry a second program cheaply, the same cases go into `https-exchange.trb` instead and the PR says so. - [ ] No regression: `http-exchange`, `http-wire`, `http-disconnect`, `http-workers` and `https-exchange` keep their `.expected` files unchanged. - [ ] `docs/standard-library/http.md` documents `Duplex`, `Upgrade` and `Response.upgrade` (declaration markers as for `Server`); NETWORK.md section 11 slice 14 becomes "Done (2026-10-24, server side; the WebSocket framing is `std/web`, WEB.md slice 8, #105)", section 13 row 5 says "as built"; `skills/` regenerated (`torb docs skill docs skills`; `--check` is a tier A gate). - [ ] `torb format --check` clean on the touched files, `tools/check-commits.sh` clean, tier A and B green in forge CI. ## Out of scope WebSocket frames, masking, `Sec-WebSocket-Accept`, ping/pong, close handshake, `permessage-deflate` (#105, `std/web`); a client-side upgrade; HTTP/2 `CONNECT` and extended CONNECT (slice 12). ## Size One PR, two Conventional Commits: `feat(std/http): upgrade a connection with 101 Switching Protocols and a Duplex` (code, conformance programs) and `docs(std/http): the upgrade in the standard-library page, NETWORK.md and skills`. Worker: `std/` and docs, so implementer sonnet/medium; the interplay with `watchPeer` and the shutdown path deserves a careful read of `serveConnection` before the edit.
Author
Collaborator

[loop:torbscript] refine: ready, claim released, next: implement

Definition of ready met: goal and testable acceptance criteria (amended in the decision comment above), design point NETWORK.md slice 14 / section 13 request 5, no seed impact, one PR in two Conventional Commits, no open dependency. The WebSocket framing is not a second slice of this issue: it belongs to std/web (#105, WEB.md slice 8, already blocked on #74), so no new issue was filed. Label ready set. Worker: refine worker (sonnet, read-only, repo at main bc504c24), about 120k tokens by estimate, exact count not visible to the worker.

[loop:torbscript] refine: ready, claim released, next: implement Definition of ready met: goal and testable acceptance criteria (amended in the decision comment above), design point NETWORK.md slice 14 / section 13 request 5, no seed impact, one PR in two Conventional Commits, no open dependency. The WebSocket framing is not a second slice of this issue: it belongs to `std/web` (#105, WEB.md slice 8, already blocked on #74), so no new issue was filed. Label `ready` set. Worker: refine worker (sonnet, read-only, repo at `main` bc504c24), about 120k tokens by estimate, exact count not visible to the worker.
Author
Collaborator

[loop:torbscript] claim implement by session:loop-torbscript-20261024 until 2026-10-24T02:45:16Z

[loop:torbscript] claim implement by session:loop-torbscript-20261024 until 2026-10-24T02:45:16Z
Author
Collaborator

[loop:torbscript] implement: PR open #458 (code commit pushed: Status.switchingProtocols, Response.upgrade, Upgrade, Duplex, serveConnection upgrade path, http-upgrade and https-upgrade conformance programs); written without a local compiler, so forge CI decides; docs/skills commit follows, next: review (waiting on CI first)

[loop:torbscript] implement: PR open https://git.torb.dev/torbscript/language/pulls/458 (code commit pushed: Status.switchingProtocols, Response.upgrade, Upgrade, Duplex, serveConnection upgrade path, http-upgrade and https-upgrade conformance programs); written without a local compiler, so forge CI decides; docs/skills commit follows, next: review (waiting on CI first)
Author
Collaborator

[loop:torbscript] implement: checkpoint, PR #458 has both commits (code + conformance programs, docs + skills hand-synced since no torb binary runs in the worker). Forge CI has not started yet (all four checks pending after 15 min, runner queue). Not yet verified by a compiler: expect a fix round for compile errors, the predicted .expected of http-upgrade/https-upgrade, and the torb docs declaration blocks. next: waiting (forge CI queue), then review

[loop:torbscript] implement: checkpoint, PR https://git.torb.dev/torbscript/language/pulls/458 has both commits (code + conformance programs, docs + skills hand-synced since no torb binary runs in the worker). Forge CI has not started yet (all four checks pending after 15 min, runner queue). Not yet verified by a compiler: expect a fix round for compile errors, the predicted `.expected` of http-upgrade/https-upgrade, and the `torb docs` declaration blocks. next: waiting (forge CI queue), then review
Author
Collaborator

[loop:torbscript] claim fix by session:loop-torbscript-20261024 until 2026-10-24T05:26:24Z

[loop:torbscript] claim fix by session:loop-torbscript-20261024 until 2026-10-24T05:26:24Z
Author
Collaborator

[loop:torbscript] fix: tier A and the playground were red for three compile errors of the uncompiled PR: Response had a field and a static fn both named upgrade (checker: one namespace of members; the playground rename test hit the same), http-upgrade.trb awaited inside the handler closure (the handler returns the Task itself), https-upgrade.trb wrote TlsSettings trusted: [...] instead of TlsSettings(trusted: [...]). Fix 5316ae6c: the field is now takeover (message.trb, server.trb, docs/standard-library/http.md and its skill copy), both tests corrected and formatted. Checked locally with the nightly-20261023 toolchain: torb check std/http and both conformance files clean, torb format --check clean, check-commits clean. Not run locally (CI only): the compiler suite (its whole-repository type check failed on the same error), docs declarations --check, doc --check, conformance runs in both modes, so another red round is possible. next: waiting (forge CI of 5316ae6c)

[loop:torbscript] fix: tier A and the playground were red for three compile errors of the uncompiled PR: `Response` had a field and a `static fn` both named `upgrade` (checker: one namespace of members; the playground rename test hit the same), `http-upgrade.trb` awaited inside the handler closure (the handler returns the `Task` itself), `https-upgrade.trb` wrote `TlsSettings trusted: [...]` instead of `TlsSettings(trusted: [...])`. Fix 5316ae6c: the field is now `takeover` (message.trb, server.trb, docs/standard-library/http.md and its skill copy), both tests corrected and formatted. Checked locally with the nightly-20261023 toolchain: `torb check std/http` and both conformance files clean, `torb format --check` clean, `check-commits` clean. Not run locally (CI only): the compiler suite (its whole-repository type check failed on the same error), `docs declarations --check`, `doc --check`, conformance runs in both modes, so another red round is possible. next: waiting (forge CI of 5316ae6c)
argus closed this issue 2026-10-24 05:49:04 +00:00
Sign in to join this conversation.
No labels
loop
ready
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
torbscript/language#74
No description provided.