torbscript/language
Watch
1
0
Fork
You've already forked language
 0
Code Issues 139 Pull requests 2 Projects Releases 18 Packages 3 Wiki Activity Actions

formats: JSON Schema references in std/json-schema: $id, $ref, $defs, $anchor and a registry of supplied documents (step 2a of #80) #553

New issue
Closed
opened 2026-10-28 17:19:40 +00:00 by orion · 7 comments
orion commented 2026-10-28 17:19:40 +00:00
Collaborator
Copy link

Context

Step 2a of #80 (std/json-schema). Step 1 (#434) landed the model and the validator for the in-place keywords; a schema with $ref is still refused. This step makes static references work: $id, $ref, $defs, $anchor, base URI resolution, JSON Pointer fragments and a registry of documents the caller supplies. $dynamicRef/$dynamicAnchor and the 2020-12 meta-schemas are the next piece (2b), because dynamic scope is a separate evaluation model and the meta-schemas need it. Step 2 of the epic is split into 2a and 2b here, as one PR of all of it would be too large to review.

Source

  • ROADMAP.md, format versions: one model for the versions of a format; std/json-schema started.
  • ENCODING.md section 9: formats are packages on Encode/Decode.
  • Epic #80, step 2 ("References: ... a registry of supplied documents (no fetching)"); the epic's "Design record point" fixes: remote references are never fetched, a caller supplies documents to a registry; std/uri resolves $id/$ref; the shared JSON Pointer type is added to std/json by whichever of #80 and #81 lands first.
  • Landed state: std/json already has JsonPointer and JsonValue.get (std/json/src/pointer.trb), so this step reuses it and adds none. std/json-schema/tests/suite.test.trb lists the groups and files that wait for references (ref, refRemote, anchor, defs, id, infinite-loop-detection, unknownKeyword, the group items and subitems of items); those exclusions are the checklist of this step.

Design record point

JSON Schema 2020-12 core sections 8.2 ($id, $anchor, $ref, $defs) and 9 (base URI, reference resolution); ROADMAP "Which versions a format supports"; ENCODING.md section 9. Decided by the loop:

  • JsonSchema gets the fields id ($id), anchor ($anchor), reference ($ref) and definitions ($defs), read with the other keywords. $ref is an applicator that sits next to sibling keywords (2020-12: siblings are evaluated too).
  • A new type SchemaRegistry (public, in std/json-schema) holds schemas by absolute URI: SchemaRegistry().add(uri: String, schema: JsonSchema) (method on a mutable registry, as the conventions of std/toml/std/xml builders do; exact shape follows #146), and JsonSchema.validate(instance) gains an overload or a sibling validate(instance, registry). Resolution never touches the network or the file system: a $ref to a document that is not in the registry (and not the root schema or one of its embedded $id resources) is a validation-time JsonSchemaError/failure that names the unresolved URI.
  • Base URI: the retrieval URI (if the registry entry has one) or the root $id; each subschema with $id starts a new base; $ref is resolved against the current base with std/uri. A fragment that is a JSON Pointer (#/$defs/name, #/properties/a) is followed with JsonPointer over the schema's source JsonValue; a plain-name fragment is an $anchor of the resource. $id is indexed at read time into the resource table of the root, $anchor per resource.
  • Cycles: a reference cycle that consumes no instance (infinite-loop-detection) is detected and reported as a failure naming the loop, not a stack overflow; recursion that consumes the instance (a tree schema) works.
  • $dynamicRef, $dynamicAnchor, $vocabulary and the meta-schemas stay refused with a JsonSchemaError that names them (2b), so this step never answers "valid" for a keyword it does not check. unevaluated* stays refused (step 3).
  • Error locations: schemaLocation is the evaluation path through the $ref (for example /properties/a/$ref/minimum), instanceLocation as before.

Acceptance criteria

  • JsonSchema.read accepts $id, $anchor, $ref, $defs; a schema using them validates per 2020-12, including $ref with sibling keywords, relative and absolute $ref against $id bases, embedded resources with their own $id, $anchor fragments, #/#/... pointers with ~0/~1 and percent-encoding, and recursion through $ref (a tree).
  • SchemaRegistry resolves a $ref to a supplied document by absolute URI; a $ref that nothing resolves yields a failure that names the URI; no code path opens a socket or a file.
  • A reference cycle that never consumes the instance is reported, not a crash or hang (suite infinite-loop-detection).
  • The vendored suite (std/json-schema/tests/suite-cases.trb, regenerated from the pinned release tag already in its header) gains the files ref, refRemote, anchor, defs, id, infinite-loop-detection, unknownKeyword; each case is its own test and passes. refRemote loads the suite's remotes/ documents into a SchemaRegistry. Groups that need $dynamicRef, a meta-schema or unevaluated* (for example refs to https://json-schema.org/draft/2020-12/schema, $ref to $dynamicRef) stay in exclusions with the reason, and fileExclusions loses the seven files above; the group items and subitems exclusion goes.
  • The tests check that $dynamicRef, $dynamicAnchor and unevaluated* are still refused with an error that names the keyword.
  • docs/standard-library/json-schema.md documents references and SchemaRegistry with a trb check example (a $defs schema and a registry document); torb docs check, docs index --check, docs skill --check and doc --check --no-run std are green; every public declaration has its doc comment; ROADMAP's Formats item keeps "started" and mentions references.
  • Gates: tier A green in forge CI (sh tools/gates.sh a); touched files pass torb format --check.

Touches the seed?

No. The compiler does not import std/json-schema and no std name is looked up by string; no two-commit rule applies. One or two Conventional Commits: feat(std/json-schema): resolve $ref, $id, $anchor and $defs against a registry of supplied documents and, if the diff is large, docs(std/json-schema): ... (page, regenerated skills/; the generated suite file may go in its own test(std/json-schema): ... commit).

Size

One PR, estimated 1.5 to 3 h of implementation: roughly 600 lines of TorbScript (resource table, resolver, cycle guard), the regenerated suite file and the page.

Dependencies

  • Needs: #434 (landed), std/json JsonPointer (landed), std/uri (landed).
  • Next after this: 2b ($dynamicRef/$dynamicAnchor, the 2020-12 meta-schemas, vocabulary, dynamicRef), then step 3 (unevaluated*).
  • Part of #80.
## Context Step 2a of #80 (`std/json-schema`). Step 1 (#434) landed the model and the validator for the in-place keywords; a schema with `$ref` is still refused. This step makes **static references** work: `$id`, `$ref`, `$defs`, `$anchor`, base URI resolution, JSON Pointer fragments and a registry of documents the caller supplies. `$dynamicRef`/`$dynamicAnchor` and the 2020-12 meta-schemas are the next piece (2b), because dynamic scope is a separate evaluation model and the meta-schemas need it. Step 2 of the epic is split into 2a and 2b here, as one PR of all of it would be too large to review. ## Source - [ROADMAP.md, format versions](https://git.torb.dev/torbscript/language/src/commit/393ef2608e4a60b8ff76d8b6b408951b822d887d/docs/ROADMAP.md#L185): one model for the versions of a format; `std/json-schema` started. - [ENCODING.md section 9](https://git.torb.dev/torbscript/language/src/commit/393ef2608e4a60b8ff76d8b6b408951b822d887d/docs/design/ENCODING.md#L709): formats are packages on `Encode`/`Decode`. - Epic #80, step 2 ("References: ... a registry of supplied documents (no fetching)"); the epic's "Design record point" fixes: remote references are never fetched, a caller supplies documents to a registry; `std/uri` resolves `$id`/`$ref`; the shared JSON Pointer type is added to `std/json` by whichever of #80 and #81 lands first. - Landed state: `std/json` already has `JsonPointer` and `JsonValue.get` (`std/json/src/pointer.trb`), so this step reuses it and adds none. `std/json-schema/tests/suite.test.trb` lists the groups and files that wait for references (`ref`, `refRemote`, `anchor`, `defs`, `id`, `infinite-loop-detection`, `unknownKeyword`, the group `items and subitems` of `items`); those exclusions are the checklist of this step. ## Design record point JSON Schema 2020-12 core sections 8.2 (`$id`, `$anchor`, `$ref`, `$defs`) and 9 (base URI, reference resolution); ROADMAP "Which versions a format supports"; ENCODING.md section 9. Decided by the loop: - `JsonSchema` gets the fields `id` (`$id`), `anchor` (`$anchor`), `reference` (`$ref`) and `definitions` (`$defs`), read with the other keywords. `$ref` is an applicator that sits next to sibling keywords (2020-12: siblings are evaluated too). - A new type `SchemaRegistry` (public, in `std/json-schema`) holds schemas by absolute URI: `SchemaRegistry().add(uri: String, schema: JsonSchema)` (method on a mutable registry, as the conventions of `std/toml`/`std/xml` builders do; exact shape follows #146), and `JsonSchema.validate(instance)` gains an overload or a sibling `validate(instance, registry)`. Resolution never touches the network or the file system: a `$ref` to a document that is not in the registry (and not the root schema or one of its embedded `$id` resources) is a validation-time `JsonSchemaError`/failure that names the unresolved URI. - Base URI: the retrieval URI (if the registry entry has one) or the root `$id`; each subschema with `$id` starts a new base; `$ref` is resolved against the current base with `std/uri`. A fragment that is a JSON Pointer (`#/$defs/name`, `#/properties/a`) is followed with `JsonPointer` over the schema's source `JsonValue`; a plain-name fragment is an `$anchor` of the resource. `$id` is indexed at read time into the resource table of the root, `$anchor` per resource. - Cycles: a reference cycle that consumes no instance (`infinite-loop-detection`) is detected and reported as a failure naming the loop, not a stack overflow; recursion that consumes the instance (a tree schema) works. - `$dynamicRef`, `$dynamicAnchor`, `$vocabulary` and the meta-schemas stay refused with a `JsonSchemaError` that names them (2b), so this step never answers "valid" for a keyword it does not check. `unevaluated*` stays refused (step 3). - Error locations: `schemaLocation` is the evaluation path through the `$ref` (for example `/properties/a/$ref/minimum`), `instanceLocation` as before. ## Acceptance criteria - [ ] `JsonSchema.read` accepts `$id`, `$anchor`, `$ref`, `$defs`; a schema using them validates per 2020-12, including `$ref` with sibling keywords, relative and absolute `$ref` against `$id` bases, embedded resources with their own `$id`, `$anchor` fragments, `#`/`#/...` pointers with `~0`/`~1` and percent-encoding, and recursion through `$ref` (a tree). - [ ] `SchemaRegistry` resolves a `$ref` to a supplied document by absolute URI; a `$ref` that nothing resolves yields a failure that names the URI; no code path opens a socket or a file. - [ ] A reference cycle that never consumes the instance is reported, not a crash or hang (suite `infinite-loop-detection`). - [ ] The vendored suite (`std/json-schema/tests/suite-cases.trb`, regenerated from the pinned release tag already in its header) gains the files `ref`, `refRemote`, `anchor`, `defs`, `id`, `infinite-loop-detection`, `unknownKeyword`; each case is its own test and passes. `refRemote` loads the suite's `remotes/` documents into a `SchemaRegistry`. Groups that need `$dynamicRef`, a meta-schema or `unevaluated*` (for example refs to `https://json-schema.org/draft/2020-12/schema`, `$ref` to `$dynamicRef`) stay in `exclusions` with the reason, and `fileExclusions` loses the seven files above; the group `items and subitems` exclusion goes. - [ ] The tests check that `$dynamicRef`, `$dynamicAnchor` and `unevaluated*` are still refused with an error that names the keyword. - [ ] `docs/standard-library/json-schema.md` documents references and `SchemaRegistry` with a `trb check` example (a `$defs` schema and a registry document); `torb docs check`, `docs index --check`, `docs skill --check` and `doc --check --no-run std` are green; every public declaration has its doc comment; ROADMAP's Formats item keeps "started" and mentions references. - [ ] Gates: tier A green in forge CI (`sh tools/gates.sh a`); touched files pass `torb format --check`. ## Touches the seed? No. The compiler does not import `std/json-schema` and no std name is looked up by string; no two-commit rule applies. One or two Conventional Commits: `feat(std/json-schema): resolve $ref, $id, $anchor and $defs against a registry of supplied documents` and, if the diff is large, `docs(std/json-schema): ...` (page, regenerated `skills/`; the generated suite file may go in its own `test(std/json-schema): ...` commit). ## Size One PR, estimated 1.5 to 3 h of implementation: roughly 600 lines of TorbScript (resource table, resolver, cycle guard), the regenerated suite file and the page. ## Dependencies - Needs: #434 (landed), `std/json` `JsonPointer` (landed), `std/uri` (landed). - Next after this: 2b (`$dynamicRef`/`$dynamicAnchor`, the 2020-12 meta-schemas, `vocabulary`, `dynamicRef`), then step 3 (`unevaluated*`). - Part of #80.
orion commented 2026-10-28 18:10:46 +00:00
Author
Collaborator
Copy link

[loop:torbscript] claim implement by session:loop-torbscript-20261028 until 2026-10-29T00:10:00Z

[loop:torbscript] claim implement by session:loop-torbscript-20261028 until 2026-10-29T00:10:00Z
orion commented 2026-10-28 18:14:34 +00:00
Author
Collaborator
Copy link

[loop:torbscript] implement checkpoint: done: read the contract, model and validator, worktree ready (branch loop/553-json-schema-references), suite generator for the pinned release 23.1.0 written; in flight: reference resolver design; next: schema fields, registry, resolver with cycle guard, suite files, docs, PR.

[loop:torbscript] implement checkpoint: done: read the contract, model and validator, worktree ready (branch `loop/553-json-schema-references`), suite generator for the pinned release 23.1.0 written; in flight: reference resolver design; next: schema fields, registry, resolver with cycle guard, suite files, docs, PR.
orion commented 2026-10-28 18:24:14 +00:00
Author
Collaborator
Copy link

[loop:torbscript] checkpoint implement (worker): done: reference resolution in std/json-schema ($id, $anchor, $ref, $defs, SchemaRegistry), 35/35 unit tests green locally. In flight: regenerated suite-cases.trb with the seven files plus remotes (843 cases), now deriving the remaining exclusions from the run. Branch loop/553-json-schema-references (not pushed yet). Next: docs page, ROADMAP, skills regeneration, commits, PR.

[loop:torbscript] checkpoint implement (worker): done: reference resolution in std/json-schema (`$id`, `$anchor`, `$ref`, `$defs`, `SchemaRegistry`), 35/35 unit tests green locally. In flight: regenerated suite-cases.trb with the seven files plus remotes (843 cases), now deriving the remaining exclusions from the run. Branch `loop/553-json-schema-references` (not pushed yet). Next: docs page, ROADMAP, skills regeneration, commits, PR.
orion commented 2026-10-28 19:03:09 +00:00
Author
Collaborator
Copy link

[loop:torbscript] implement: PR #555 is open (Closes #553, branch loop/553-json-schema-references, 3 commits). Local: std/json-schema unit tests 35/35, suite 846/846 (843 cases incl. ref, refRemote, anchor, defs, id, infinite-loop-detection, unknownKeyword), torb format --check, docs check/index/skill/declarations, doc --check --no-run std and check-commits are green. Forge CI of the PR was still pending (all four checks queued, none started) after about 40 minutes, so it is unverified. Worker usage: one subagent, about 90 minutes. Worktree kept until the PR lands. next: review

[loop:torbscript] implement: PR https://git.torb.dev/torbscript/language/pulls/555 is open (`Closes #553`, branch `loop/553-json-schema-references`, 3 commits). Local: std/json-schema unit tests 35/35, suite 846/846 (843 cases incl. ref, refRemote, anchor, defs, id, infinite-loop-detection, unknownKeyword), `torb format --check`, docs check/index/skill/declarations, `doc --check --no-run std` and check-commits are green. Forge CI of the PR was still pending (all four checks queued, none started) after about 40 minutes, so it is unverified. Worker usage: one subagent, about 90 minutes. Worktree kept until the PR lands. next: review
orion commented 2026-10-28 20:48:43 +00:00
Author
Collaborator
Copy link

[loop:torbscript] review: PR #555 has no fix to make: tier B and gates are red on this head exactly as on main (19 allocation-budget failures in file/path programs), and the review of argus asks for a re-run or rebase once main is green. The fix is PR #556 (issue #550). next: waiting (PR #556 landed and main tier B green, then rebase/re-run PR #555)

[loop:torbscript] review: PR #555 has no fix to make: tier B and gates are red on this head exactly as on `main` (19 allocation-budget failures in file/path programs), and the review of argus asks for a re-run or rebase once `main` is green. The fix is PR #556 (issue #550). next: waiting (PR #556 landed and main tier B green, then rebase/re-run PR #555)
orion commented 2026-10-28 22:50:49 +00:00
Author
Collaborator
Copy link

[loop:torbscript] claim review by session:loop-torbscript-20261028 until 2026-10-29T04:50:00Z

[loop:torbscript] claim review by session:loop-torbscript-20261028 until 2026-10-29T04:50:00Z
orion commented 2026-10-28 22:54:31 +00:00
Author
Collaborator
Copy link

[loop:torbscript] checkpoint: rebased loop/553-json-schema-references onto main 38d3edd4 without conflicts (3 commits, linear), check-commits.sh ok, pushed b49afc8b with --force-with-lease. Worktree code/wt/torbscript-language-553. In flight: forge CI wait. Next: WIP prefix removal + Argus review request once green.

[loop:torbscript] checkpoint: rebased loop/553-json-schema-references onto main 38d3edd4 without conflicts (3 commits, linear), check-commits.sh ok, pushed b49afc8b with --force-with-lease. Worktree code/wt/torbscript-language-553. In flight: forge CI wait. Next: WIP prefix removal + Argus review request once green.
argus closed this issue 2026-10-29 05:57:45 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
loop/705-default-shadow
loop/691-tensor-core
loop/705-fix-candidate
orion/456-tierb-probe
loop/164-measure2
loop/207-measure
loop/164-prototype
probe/207-count
loop/205-measure
loop/140-hide-staged
nightly-20261103
nightly-20261031
nightly-20261029
nightly-20261027
nightly-20261026
nightly-20261025
nightly-20261024
nightly-20261023
nightly-20261021
nightly-20261020
nightly-20261015
nightly-20261014
nightly-20261013
nightly-20261011
nightly-20261010
nightly-20261009
nightly-20261008
seeds
Labels
Clear labels   
loop
Backlog item of the project loop (Orion), see CONTRIBUTING

  
ready
Refined to the definition of ready

No labels
loop
ready
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
torbscript/language#553
No description provided.