storage: an s3: driver as a package of its own #20
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Context
The registry stores through
std/storage; production wants an S3 bucket. This is thes3:driver that URI.md section 11 names as the firstStoragedriver outsidestd, and milestone 10's "S3 as a package of its own".Goal
A package
torb/s3withS3Storage, aStoragefor the schemes3:that works against AWS S3 and S3-compatible servers (MinIO, R2, Hetzner object storage). The registry wiring is split into #242.Where it lives
FIRST-PARTY.md sections 2 to 5 decide it: the package is its own repository
torb/s3in the organisationtorbon git.torb.dev, made fromtorb/package-template, with its own semantic version starting at0.1.0, published to the registry by trusted publishing. It is not a directory of this repository. The work therefore lands as pull requests oftorb/s3, not oftorbscript/language.Design
type S3Storage with Storage(ashared type, likeFileStorage),schemes()answers["s3"];S3Storage.of(settings, credentials);S3Credentials(access key id, secret key, optional session token) withS3Credentials.fromEnvironment(); secrets never appear inShowoutput or in aStorageFailuretext.s3://bucket/keymeans bucket and object key. Virtual-hosted style for AWS (no endpoint set), path style when a custom endpoint is given (MinIO and friends); both selectable in the settings.std/http:read:GetObject; a 404 withNoSuchKeyisStorageFailure.NotFound.exists:HeadObject.write:PutObjectof the wholeBytes(no multipart in this version; the limit is documented in the README).list:ListObjectsV2withdelimiter=/and continuation tokens, answering theUris of the entries below the prefix.delete: aHeadObjectfirst so that a missing key isNotFoundlike inFileStorageandMemoryStorage(S3 itself answers 204 for a missing key), thenDeleteObject.HmacSha256andSha256ofstd/digestand the signing date fromClock.timestamp()(a clock that tests can inject, so signatures are deterministic); signed payload hash, no chunked uploads.<Error><Code>and<Message>map toNotFound(NoSuchKey,NoSuchBucket) andRefused(everything else, with the code and the request id in the reason); a transport failure isRefused.Acceptance criteria
torb/s3exists from the package template; itsproject.trbnamestorb/s3, license MIT,languageset;torb checkandtorb test tests/unitpass in its forge CIS3StorageimplementsStorage:read,exists,write,list,deleteas designed above,readTextandwriteTextthrough the defaultsAuthorizationheader, with the clock injectedtests/unit(thetorb/postgrespattern): success, 404NoSuchKey, 403AccessDenied, a listing over two pages, an empty listing, a key with characters that need percent-encoding. The tests need no network, no bucket and no credentials; the loop never touches a real bucketShowtext and no failure message (a test searches for them)Storage.registry([FileStorage(), S3Storage.of(...)])dispatchess3://...to the driver and still namess3in its "the registry knows ..." message (test)std/xml(#78), not with a private parserSeed impact
None. No syntax, native, driver flag or std name looked up by the compiler changes; the package is pure TorbScript over
std.Dependencies
std/xml): FIRST-PARTY.md section 8 gap 24 says "torb/s3waits" for it, and the last criterion keeps one XML implementation instead of a private subset parser (the first version of this issue allowed one; decided by the loop, see the comment).torb/s3, its trusted-publisher entry on the registry and its first publication are outside this loop's contract (its scope istorbscript/language; publishing and organisation settings are forbidden to the loop).std/digest), the wall clock (Clock.timestamp), Base64 and hexadecimal,std/httpover TLS.Size
One pull request of
torb/s3as two Conventional Commits:feat: Signature Version 4andfeat: S3Storage for s3: URIs. If it grows past that, the README and the test double go in a thirddocs/testcommit, not a second issue.Source
std, repository conventions, order) and section 8 gap 24Moved from the planning lists of the docs; the design record stays the specification.
[loop:torbscript] intake: in scope (author orion, label loop).
s3:Storage driver as a package (ROADMAP milestone 10, URI.md slice 12); built against recorded responses, no real bucket or credentials touched by the loop. Package layout and name may need an owner decision, to settle in refine. Release freeze for 0.1.0 (2026-10-06) holds new PRs, next: refine (queued)[loop:torbscript] claim refine by session:loop-torbscript-20261009 until 2026-10-09T03:18:00Z
s3:driver #242[loop:torbscript] decided by the loop: (1) name and layout: package
torb/s3, typeS3Storage, in a repository of its own under the organisationtorb(FIRST-PARTY.md sections 2 to 5 already decide vendor, repository per package, template and versioning; the existingtorb/sql,torb/postgresandtorb/redisfollow it). Not a directory of this repository, so the registry wiring (tools/registry, which is in this repository) is split into #242. (2) Listings and errors wait forstd/xml(#78) instead of a private XML subset: the design record lists it as the one thingtorb/s3waits for (section 8 gap 24), a throwaway parser would be written twice, and the cost of waiting is zero because nothing else blocks. This replaces the issue's earlier note that the driver may parse the few elements itself. (3) Path style for custom endpoints and virtual-hosted style for AWS;PutObjectas one request without multipart;deletedoes aHeadObjectfirst so a missing key isNotFoundas inFileStorageandMemoryStorage; the clock is injectable so signatures are testable against AWS's published vectors. All internal technical choices, no public commitment.[loop:torbscript] refine: requirements brought to the definition of ready (goal, testable criteria against recorded responses and AWS's signing vectors, design records, no seed impact, size of one pull request as two commits) and the issue body rewritten; the registry half split off as #242. Not set
ready, next: waiting (dependency): #78 (std/xml, FIRST-PARTY.md gap 24) is open, and the package belongs in a repositorytorb/s3that does not exist yet, which is outside this loop's contract (scopetorbscript/language; repository creation, trusted publisher and publishing are not the loop's). Refine worker usage: roughly 110k tokens.std/xmltree, reader and writer (step 1 of #78) #429std/xml#78std/xml#78s3:driver #242std/xmlreads the internal subset: entities, default attributes and an expansion limit (step 2 of #78) #552Xml.decodereads a document into a typed value (step 3a of #78) #570std/xml#78[loop:torbscript] refine: still blocked, nothing to refine further. The issue is at the definition of ready except for two external points: #78 (
std/xml, epic; step 3a #570 reads S3 listings) is not done, and the repositorytorb/s3is outside this loop's contract (repository creation, trusted publisher and publishing are not the loop's). next: waiting (dependency #78 step 3a / repo torb/s3 by the owner) until 2026-11-05T08:00:00ZXml.encodewrites a typed value as an XML document (step 3b of #78) #605