tls: one transport of TCP or TLS over TCP, upgradable in the middle #115

Closed
opened 2026-10-05 21:07:22 +00:00 by orion · 17 comments
Collaborator

Goal

std/tls exports one Transport: TCP or TLS over TCP behind the same four operations (receive, send, finish, remote), upgradable in the middle of a connection. This is what Postgres's TLS request, MySQL's SSL request, SMTP's STARTTLS and rediss: need. Today every consumer has a private copy (std/http/src/transport.trb, torb/postgres's Wire).

Design point

  • FIRST-PARTY.md gap 17 (section 8) and NETWORK.md section 5 ("std/http reads and writes either through one Transport"). The interface of std/tls stays; this adds a type beside TlsStream.
  • Lifted from std/http's private Transport (plain / encrypted, isEncrypted, receiveBuffer, sendBuffer, finish, remote), plus the middle upgrade.

Scope

Public API in std/tls/src/lib.trb:

  • Transport.plain(stream), Transport.encrypted(stream, session) as in std/http today.
  • var fn upgrade(serverName, settings = TlsSettings()): Task<Result<Void, NetworkError>>: TLS client handshake over the plain TCP stream already held. Refused (a NetworkError, nothing sent) where the transport is already encrypted. A failed handshake leaves the transport unusable; the error is the one TlsStream.connect gives (isCertificateRejected(), isTlsFailure()).
  • var fn upgradeAccepting(identity, protocols = [], clientRoots = []): the server's side with the semantics of TlsStream.accept.
  • fn tls(): TlsStream? (the session, for negotiatedProtocol() and the peer's certificate of #116) and fn isEncrypted(): Bool.
  • Doc comment: bytes a caller read from the plain phase and buffered (a Reader) must be empty before upgrade; a transport itself buffers nothing, so a STARTTLS response-injection cannot hide in it.

Acceptance criteria

  • Transport with the API above in std/tls, documented with a STARTTLS-shaped example.
  • Conformance tests in tests/conformance/ (pattern of tls-loopback.trb): plain exchange, upgrade / upgradeAccepting over loopback, encrypted exchange afterwards, isEncrypted() flips; upgrade on an encrypted transport fails; upgrade for a wrong name fails with isCertificateRejected(); a plain-only transport never reports a session.
  • std/http uses std/tls's Transport; its private src/transport.trb is deleted; its existing tests pass unchanged.
  • FIRST-PARTY.md gap 17 marked done with the file and test names, and the "Until then" copies of torb/postgres named as still to move (outside this repo).
  • sh tools/gates.sh a green in forge CI; docs change regenerates skills/.

Seed

Does not touch the seed: pure TorbScript in std/tls and std/http, no syntax change, no new native, no driver flag, no name the compiler looks up by string.

Size

One PR, two Conventional Commits:

  1. feat(tls): one Transport of TCP or TLS over TCP, upgradable in the middle (type, tests).
  2. refactor(http): use std/tls's Transport (delete the private copy, FIRST-PARTY.md gap 17 done, regenerated skills/).

Dependencies

None open. #116 (peer's certificate) and #117/#118 (client certificates, ALPN) are merged. #463 (commit 2 of 2 for #116) also edits std/tls/src/lib.trb: whichever lands second rebases; no semantic dependency.

Moved from the planning lists of the docs; the design record stays the specification.

## Goal `std/tls` exports one `Transport`: TCP or TLS over TCP behind the same four operations (receive, send, finish, remote), upgradable in the middle of a connection. This is what Postgres's TLS request, MySQL's SSL request, SMTP's `STARTTLS` and `rediss:` need. Today every consumer has a private copy (`std/http/src/transport.trb`, `torb/postgres`'s `Wire`). ## Design point - [FIRST-PARTY.md gap 17](https://git.torb.dev/torbscript/language/src/branch/main/docs/design/FIRST-PARTY.md) (section 8) and NETWORK.md section 5 ("`std/http` reads and writes either through one `Transport`"). The interface of `std/tls` stays; this adds a type beside `TlsStream`. - Lifted from `std/http`'s private `Transport` (plain / encrypted, `isEncrypted`, `receiveBuffer`, `sendBuffer`, `finish`, `remote`), plus the middle upgrade. ## Scope Public API in `std/tls/src/lib.trb`: - `Transport.plain(stream)`, `Transport.encrypted(stream, session)` as in `std/http` today. - `var fn upgrade(serverName, settings = TlsSettings()): Task<Result<Void, NetworkError>>`: TLS client handshake over the plain TCP stream already held. Refused (a `NetworkError`, nothing sent) where the transport is already encrypted. A failed handshake leaves the transport unusable; the error is the one `TlsStream.connect` gives (`isCertificateRejected()`, `isTlsFailure()`). - `var fn upgradeAccepting(identity, protocols = [], clientRoots = [])`: the server's side with the semantics of `TlsStream.accept`. - `fn tls(): TlsStream?` (the session, for `negotiatedProtocol()` and the peer's certificate of #116) and `fn isEncrypted(): Bool`. - Doc comment: bytes a caller read from the plain phase and buffered (a `Reader`) must be empty before `upgrade`; a transport itself buffers nothing, so a `STARTTLS` response-injection cannot hide in it. ## Acceptance criteria - [ ] `Transport` with the API above in `std/tls`, documented with a `STARTTLS`-shaped example. - [ ] Conformance tests in `tests/conformance/` (pattern of `tls-loopback.trb`): plain exchange, `upgrade` / `upgradeAccepting` over loopback, encrypted exchange afterwards, `isEncrypted()` flips; `upgrade` on an encrypted transport fails; `upgrade` for a wrong name fails with `isCertificateRejected()`; a plain-only transport never reports a session. - [ ] `std/http` uses `std/tls`'s `Transport`; its private `src/transport.trb` is deleted; its existing tests pass unchanged. - [ ] FIRST-PARTY.md gap 17 marked done with the file and test names, and the "Until then" copies of `torb/postgres` named as still to move (outside this repo). - [ ] `sh tools/gates.sh a` green in forge CI; docs change regenerates `skills/`. ## Seed Does not touch the seed: pure TorbScript in `std/tls` and `std/http`, no syntax change, no new native, no driver flag, no name the compiler looks up by string. ## Size One PR, two Conventional Commits: 1. `feat(tls): one Transport of TCP or TLS over TCP, upgradable in the middle` (type, tests). 2. `refactor(http): use std/tls's Transport` (delete the private copy, FIRST-PARTY.md gap 17 done, regenerated `skills/`). ## Dependencies None open. #116 (peer's certificate) and #117/#118 (client certificates, ALPN) are merged. #463 (commit 2 of 2 for #116) also edits `std/tls/src/lib.trb`: whichever lands second rebases; no semantic dependency. _Moved from the planning lists of the docs; the design record stays the specification._
Author
Collaborator

[loop:torbscript] intake: in scope (author orion, label loop), source FIRST-PARTY.md gap 17, not a duplicate. Depends on TLS work in std. next: waiting (refine queued, ready buffer full) until 2026-10-10T04:00:00Z

[loop:torbscript] intake: in scope (author orion, label loop), source FIRST-PARTY.md gap 17, not a duplicate. Depends on TLS work in std. next: waiting (refine queued, ready buffer full) until 2026-10-10T04:00:00Z
Author
Collaborator

[loop:torbscript] claim refine by session:loop-torbscript-20261027 until 2026-10-27T01:37Z

[loop:torbscript] claim refine by session:loop-torbscript-20261027 until 2026-10-27T01:37Z
Author
Collaborator

[loop:torbscript] decided by the loop: (1) The type lives in std/tls and std/http moves onto it, because std/http already imports std/tls, so no new package edge, and the gap names std/tls. (2) upgrade is an explicit method on the transport (client) with upgradeAccepting for the server, so loopback tests cover both ends. (3) The transport buffers nothing; the caller must drain its own Reader before upgrading, documented. (4) The torb/postgres Wire copy is outside this repo and stays until it moves. (5) Size: one PR, two commits, no seed touch. Dependencies #116/#117/#118 are already merged; #463 only overlaps the same file. The issue body now holds goal, criteria, design point, seed impact and size.

[loop:torbscript] decided by the loop: (1) The type lives in `std/tls` and `std/http` moves onto it, because `std/http` already imports `std/tls`, so no new package edge, and the gap names `std/tls`. (2) `upgrade` is an explicit method on the transport (client) with `upgradeAccepting` for the server, so loopback tests cover both ends. (3) The transport buffers nothing; the caller must drain its own `Reader` before upgrading, documented. (4) The `torb/postgres` `Wire` copy is outside this repo and stays until it moves. (5) Size: one PR, two commits, no seed touch. Dependencies #116/#117/#118 are already merged; #463 only overlaps the same file. The issue body now holds goal, criteria, design point, seed impact and size.
Author
Collaborator

[loop:torbscript] refine: body rewritten to the definition of ready, label ready set, next: implement. Worker usage: about 60k tokens (sonnet).

[loop:torbscript] refine: body rewritten to the definition of ready, label `ready` set, next: implement. Worker usage: about 60k tokens (sonnet).
Author
Collaborator

[loop:torbscript] claim implement by session:loop-torbscript-20261027 until 2026-10-27T06:00Z

[loop:torbscript] claim implement by session:loop-torbscript-20261027 until 2026-10-27T06:00Z
Author
Collaborator

[loop:torbscript] implement: checkpoint, claim 7124 released, no PR yet, next: implement (continue).

Done (commit 9442c1e4 on loop/115-tls-transport, pushed): Transport in std/tls/src/lib.trb with plain, encrypted, isEncrypted, tls, upgrade, upgradeAccepting, receiveBuffer, sendBuffer, finish, remote; a failed handshake marks the transport unusable (isClosed()), upgrade on an encrypted transport is refused with NetworkError.invalid. Doc comment carries a STARTTLS example and the drain-your-Reader rule.

Left, in order:

  1. tests/conformance/tls-transport.{trb,expected,exit,allocations} (pattern of tls-alpn.trb; cert constants copied from it): plain exchange, upgrade/upgradeAccepting, encrypted exchange, isEncrypted() flips, second upgrade refused, wrong name -> isCertificateRejected() then isClosed(), plain-only never has a session, Transport.encrypted pair. .exit is 0 without newline; .allocations is unbudgeted: <reason> like tls-alpn.
  2. Docs: docs/standard-library/tls.md Transport declaration block (generated format, source order) plus prose, mirrored by hand into skills/torbscript-networking/references/standard-library/tls.md (only link rewriting differs) unless a torb binary is available.
  3. Commit 2 refactor(http): use std/tls's Transport: delete std/http/src/transport.trb, change use Transport from "./transport" to from "std/tls" in client.trb, connection.trb, server.trb; FIRST-PARTY.md row 17 marked done with file and test names, torb/postgres Wire named as still to move.
  4. Open PR (Closes #115), read tier A with ci-wait.sh, fix red rounds.

Unverified risks: nothing compiled or formatted yet (no torb binary in the gateway), so the first CI round may show type or format --check findings; doc --check type-checks the new doc example (Buffer.from inference).

Worktree: code/wt/torbscript-language-115 (orion workspace). Last commit: 9442c1e4. Worker usage: about 110k tokens (sonnet).

CONTINUE: torbscript/language#115

[loop:torbscript] implement: checkpoint, claim 7124 released, no PR yet, next: implement (continue). Done (commit 9442c1e4 on `loop/115-tls-transport`, pushed): `Transport` in `std/tls/src/lib.trb` with `plain`, `encrypted`, `isEncrypted`, `tls`, `upgrade`, `upgradeAccepting`, `receiveBuffer`, `sendBuffer`, `finish`, `remote`; a failed handshake marks the transport unusable (`isClosed()`), `upgrade` on an encrypted transport is refused with `NetworkError.invalid`. Doc comment carries a STARTTLS example and the drain-your-Reader rule. Left, in order: 1. `tests/conformance/tls-transport.{trb,expected,exit,allocations}` (pattern of `tls-alpn.trb`; cert constants copied from it): plain exchange, `upgrade`/`upgradeAccepting`, encrypted exchange, `isEncrypted()` flips, second upgrade refused, wrong name -> `isCertificateRejected()` then `isClosed()`, plain-only never has a session, `Transport.encrypted` pair. `.exit` is `0` without newline; `.allocations` is `unbudgeted: <reason>` like `tls-alpn`. 2. Docs: `docs/standard-library/tls.md` Transport declaration block (generated format, source order) plus prose, mirrored by hand into `skills/torbscript-networking/references/standard-library/tls.md` (only link rewriting differs) unless a torb binary is available. 3. Commit 2 `refactor(http): use std/tls's Transport`: delete `std/http/src/transport.trb`, change `use Transport from "./transport"` to `from "std/tls"` in `client.trb`, `connection.trb`, `server.trb`; FIRST-PARTY.md row 17 marked done with file and test names, `torb/postgres` `Wire` named as still to move. 4. Open PR (`Closes #115`), read tier A with ci-wait.sh, fix red rounds. Unverified risks: nothing compiled or formatted yet (no torb binary in the gateway), so the first CI round may show type or `format --check` findings; `doc --check` type-checks the new doc example (`Buffer.from` inference). Worktree: `code/wt/torbscript-language-115` (orion workspace). Last commit: 9442c1e4. Worker usage: about 110k tokens (sonnet). CONTINUE: torbscript/language#115
Author
Collaborator

[loop:torbscript] claim implement (continuation 1/3 after checkpoint 7143) by session:loop-torbscript-20261027 until 2026-10-27T06:00Z

[loop:torbscript] claim implement (continuation 1/3 after checkpoint 7143) by session:loop-torbscript-20261027 until 2026-10-27T06:00Z
orion referenced this issue from a commit 2026-10-27 00:07:32 +00:00
Author
Collaborator

[loop:torbscript] implement: checkpoint, claim 7144 released, PR #522 is open, forge CI tier A running (ci-wait started, no result read yet), next: implement (continue).

Done (4 commits on loop/115-tls-transport, pushed, last 9cc7e9da): Transport in std/tls (9442c1e4); conformance tests/conformance/tls-transport.{trb,expected,exit,allocations} (plain, STARTTLS-style upgrade on both ends with ALPN, second upgrade refused, wrong name -> certificate rejected then closed, Transport.encrypted); docs in docs/standard-library/tls.md and the skill mirror, http.md source lists, FIRST-PARTY.md row 17 done; std/http now imports Transport from std/tls and std/http/src/transport.trb is deleted.

Left, in order: read tier A of PR 522 with ci-wait.sh (-C code/wt/torbscript-language-115), fix red rounds on the same branch (nothing was compiled or formatted before CI, so expect type or format --check findings and a possibly different server-side outcome in the "wrong name" line of tls-transport.expected), then write the final next: review comment.

Unverified risks: tls-transport.expected was derived by hand; the doc fragment in tls.md is fragment (lexed only), the doc comment example in std/tls/src/lib.trb is type-checked by doc --check.

Worktree: code/wt/torbscript-language-115. Worker usage so far: about 190k tokens (sonnet).

CONTINUE: torbscript/language#115

[loop:torbscript] implement: checkpoint, claim 7144 released, PR https://git.torb.dev/torbscript/language/pulls/522 is open, forge CI tier A running (ci-wait started, no result read yet), next: implement (continue). Done (4 commits on `loop/115-tls-transport`, pushed, last 9cc7e9da): `Transport` in `std/tls` (9442c1e4); conformance `tests/conformance/tls-transport.{trb,expected,exit,allocations}` (plain, STARTTLS-style upgrade on both ends with ALPN, second upgrade refused, wrong name -> certificate rejected then closed, `Transport.encrypted`); docs in `docs/standard-library/tls.md` and the skill mirror, `http.md` source lists, FIRST-PARTY.md row 17 done; `std/http` now imports `Transport` from `std/tls` and `std/http/src/transport.trb` is deleted. Left, in order: read tier A of PR 522 with ci-wait.sh (`-C code/wt/torbscript-language-115`), fix red rounds on the same branch (nothing was compiled or formatted before CI, so expect type or `format --check` findings and a possibly different server-side outcome in the "wrong name" line of `tls-transport.expected`), then write the final `next: review` comment. Unverified risks: `tls-transport.expected` was derived by hand; the doc fragment in `tls.md` is `fragment` (lexed only), the doc comment example in `std/tls/src/lib.trb` is type-checked by `doc --check`. Worktree: `code/wt/torbscript-language-115`. Worker usage so far: about 190k tokens (sonnet). CONTINUE: torbscript/language#115
Author
Collaborator

[loop:torbscript] claim implement (fix round, red CI on PR #522) by session:loop-torbscript-20261027 until 2026-10-27T07:19Z

[loop:torbscript] claim implement (fix round, red CI on PR #522) by session:loop-torbscript-20261027 until 2026-10-27T07:19Z
Author
Collaborator

[loop:torbscript] release implement: worker died on a session limit (profile locked), fresh worker follows on the next profile

[loop:torbscript] release implement: worker died on a session limit (profile locked), fresh worker follows on the next profile
Author
Collaborator

[loop:torbscript] claim implement (fix round, red CI on PR #522, respawn on anthropic:tkoehn) by session:loop-torbscript-20261027 until 2026-10-27T08:00Z

[loop:torbscript] claim implement (fix round, red CI on PR #522, respawn on anthropic:tkoehn) by session:loop-torbscript-20261027 until 2026-10-27T08:00Z
Author
Collaborator

[loop:torbscript] implement: checkpoint, no new commit this round. HEAD d630bd8e (var session field fix) is pushed and forge CI on PR 522 is running (commit messages, change scope, pdisc green so far); a fresh worker on anthropic:tkoehn is waiting on ci-wait for tier A, next: implement (fix any red job, then write the final comment). Worker usage so far: about 25k tokens (sonnet).

[loop:torbscript] implement: checkpoint, no new commit this round. HEAD d630bd8e (var session field fix) is pushed and forge CI on PR 522 is running (commit messages, change scope, pdisc green so far); a fresh worker on anthropic:tkoehn is waiting on ci-wait for tier A, next: implement (fix any red job, then write the final comment). Worker usage so far: about 25k tokens (sonnet).
Author
Collaborator

[loop:torbscript] release implement: the fix-round worker (anthropic:tkoehn) was killed before its result; HEAD d630bd8e is pushed and forge CI on PR #522 is running. next: waiting (CI on PR #522) until 2026-10-27T05:00:00Z

[loop:torbscript] release implement: the fix-round worker (anthropic:tkoehn) was killed before its result; HEAD d630bd8e is pushed and forge CI on PR #522 is running. next: waiting (CI on PR #522) until 2026-10-27T05:00:00Z
Author
Collaborator

[loop:torbscript] claim implement (fix round, red tier A on PR #522 head d630bd8e, pinned on anthropic:notahumanmate) by session:loop-torbscript-20261027 until 2026-10-27T11:15Z

[loop:torbscript] claim implement (fix round, red tier A on PR #522 head d630bd8e, pinned on anthropic:notahumanmate) by session:loop-torbscript-20261027 until 2026-10-27T11:15Z
Author
Collaborator

[loop:torbscript] implement: root cause of red tier A on d630bd8e was format --check only (tests/conformance/tls-transport.trb: two Transport.encrypted(stream, session) calls in command position must be commands); every other gate lane in that run was green (check, docs, repl/lsp/debug, native + VM std tests, language, packages, vendor, lock). Fixed with the formatter itself (nightly torb format, 2 lines) and pushed ff60cab1 style(tls): format the Transport conformance test to loop/115-tls-transport. Forge CI on ff60cab1 has not started: all jobs of runs 1969/1970 are still queued (waiting for a runner, ~55 min at 06:10 CET), so tier A is undecided. next: waiting (CI on PR #522). Worker usage about 100k tokens (sonnet).

[loop:torbscript] implement: root cause of red tier A on d630bd8e was `format --check` only (tests/conformance/tls-transport.trb: two `Transport.encrypted(stream, session)` calls in command position must be commands); every other gate lane in that run was green (check, docs, repl/lsp/debug, native + VM std tests, language, packages, vendor, lock). Fixed with the formatter itself (nightly `torb format`, 2 lines) and pushed ff60cab1 `style(tls): format the Transport conformance test` to loop/115-tls-transport. Forge CI on ff60cab1 has not started: all jobs of runs 1969/1970 are still queued (waiting for a runner, ~55 min at 06:10 CET), so tier A is undecided. next: waiting (CI on PR #522). Worker usage about 100k tokens (sonnet).
Author
Collaborator

[loop:torbscript] claim implement (fix round, red 'commit messages' check on PR #522 head ff60cab1, pinned on anthropic:notahumanmate) by session:loop-torbscript-20261027-0718 until 2026-10-27T12:43Z

[loop:torbscript] claim implement (fix round, red 'commit messages' check on PR #522 head ff60cab1, pinned on anthropic:notahumanmate) by session:loop-torbscript-20261027-0718 until 2026-10-27T12:43Z
orion referenced this issue from a commit 2026-10-27 06:43:52 +00:00
Author
Collaborator

[loop:torbscript] implement: fix round, ci / commit messages failed because style(tls): format the Transport conformance test used the type style, which tools/check-commits.sh does not allow (feat fix perf refactor docs test build ci chore revert); also folded the fix(tls) repair of the PR's own earlier commit, squashed both fixups into their origin commits (4 commits left, check-commits.sh clean locally), rebased onto current main, changeset identical (diff vs old head equal), new head 2beb1bb5; CI queued, next: waiting (CI on PR #522). Worker tokens: ~45k. Claim 7406 released.

[loop:torbscript] implement: fix round, `ci / commit messages` failed because `style(tls): format the Transport conformance test` used the type `style`, which tools/check-commits.sh does not allow (feat fix perf refactor docs test build ci chore revert); also folded the `fix(tls)` repair of the PR's own earlier commit, squashed both fixups into their origin commits (4 commits left, check-commits.sh clean locally), rebased onto current main, changeset identical (diff vs old head equal), new head 2beb1bb5; CI queued, next: waiting (CI on PR #522). Worker tokens: ~45k. Claim 7406 released.
orion referenced this issue from a commit 2026-10-27 10:36:37 +00:00
argus closed this issue 2026-10-27 12:33:03 +00:00
Sign in to join this conversation.
No labels
loop
ready
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
torbscript/language#115
No description provided.