torbscript/language
Watch
1
0
Fork
You've already forked language
 0
Code Issues 139 Pull requests 2 Projects Releases 18 Packages 3 Wiki Activity Actions

natives: parse native fn bodies and native { } blocks, with a parser gate test (step 2 of #53) #467

New issue
Closed
opened 2026-10-24 07:08:37 +00:00 by orion · 13 comments
orion commented 2026-10-24 07:08:37 +00:00
Collaborator
Copy link

Context

Step 2 of #53 (natives: native fn bodies as IR and native { } blocks). Step 1 (#410) landed the design record docs/design/NATIVE-BODIES.md (PR #412). This is its slice 1: the syntax and the parser, with a gate test. Nothing is checked or lowered yet.

Goal

The parser reads the two new forms of section 2.1 and 5 of the record, and a parser gate test pins them:

  • native fn with a body: native-body ::= "{" native-statement* native-result "}", attached to FunctionDeclaration.body as a new node NativeBody(statements, result).
  • native { ... } at the start of an expression: a new ExpressionKind.NativeBlock. A native followed by anything else in an expression position stays the error it is today.
  • intrinsic is a contextual word, read only where a native statement or result expects an operation; it stays usable as an ordinary name everywhere else. The parser joins add.i64.unchecked (identifier plus member accesses) into the intrinsic name; the lexer does not change.
  • Statements: const name (: type)? = intrinsic ...; result: an operation or an operand (name path, integer, float, true, false); operands separated by commas.

The checker answers every parsed native body or block with the single error a `native` body is not supported yet (no rules N1-N10 yet; that is step 3). No lowering.

Acceptance criteria

  • NativeBody and ExpressionKind.NativeBlock exist in the syntax tree; the parser builds them for every example of the record (sections 2.3 and 2.4: Int64.add, shiftedLeft, convert.i8.i64, the isOdd block, the multi-statement block with const).
  • A parser gate test (tier A, in the existing parser test style) parses every example of the record and checks the tree; it rejects what section 2 rejects: a block without a result, a body ending in a const, native followed by something other than { in an expression, an operation with a trailing comma, a missing operand separator.
  • intrinsic is still a valid identifier outside native bodies (test: a variable, a function and a member named intrinsic).
  • A native fn without a body parses and checks exactly as before (existing tests unchanged); the manifest (natives.trb) and every std/ file are untouched.
  • The checker reports a `native` body is not supported yet at the native keyword for a parsed body or block; a test pins the text.
  • torb format formats the new forms idempotently (formatter test for the examples), or, if the formatter walks the tree generically, a test says so.
  • sh tools/gates.sh a green in forge CI; the compiler's own sources do not use the form, so the current seed builds the commit.

Design record point

docs/design/NATIVE-BODIES.md sections 2 (source form), 5 (parsing) and 8 (slice 1).

Seed impact

Yes, new syntax. This is the first commit of the two-commit rule ("teach both forms, refresh the seed, then migrate", CLAUDE.md "Seed and breaking changes"). Nothing in compiler/ or std/ uses the form yet, so the published seed still builds it. The seed refresh stays Torben's and happens after slice 3 (lowering), before the std migration.

Size

One PR, one or two Conventional Commits, e.g. feat(syntax): parse native fn bodies and native blocks and, if split, test(syntax): parser gate test for IR bodies of natives.

Dependencies

Step 1 (#410, landed). Part of #53; blocks the checker step (rules N1-N10), the lowering step and the std migration.

## Context Step 2 of #53 (natives: `native fn` bodies as IR and `native { }` blocks). Step 1 (#410) landed the design record [`docs/design/NATIVE-BODIES.md`](https://git.torb.dev/torbscript/language/src/branch/main/docs/design/NATIVE-BODIES.md) (PR #412). This is its slice 1: the syntax and the parser, with a gate test. Nothing is checked or lowered yet. ## Goal The parser reads the two new forms of section 2.1 and 5 of the record, and a parser gate test pins them: - `native fn` with a body: `native-body ::= "{" native-statement* native-result "}"`, attached to `FunctionDeclaration.body` as a new node `NativeBody(statements, result)`. - `native { ... }` at the start of an expression: a new `ExpressionKind.NativeBlock`. A `native` followed by anything else in an expression position stays the error it is today. - `intrinsic` is a contextual word, read only where a native statement or result expects an operation; it stays usable as an ordinary name everywhere else. The parser joins `add.i64.unchecked` (identifier plus member accesses) into the intrinsic name; the lexer does not change. - Statements: `const name (: type)? = intrinsic ...`; result: an operation or an operand (name path, integer, float, `true`, `false`); operands separated by commas. The checker answers every parsed native body or block with the single error `` a `native` body is not supported yet `` (no rules N1-N10 yet; that is step 3). No lowering. ## Acceptance criteria - [ ] `NativeBody` and `ExpressionKind.NativeBlock` exist in the syntax tree; the parser builds them for every example of the record (sections 2.3 and 2.4: `Int64.add`, `shiftedLeft`, `convert.i8.i64`, the `isOdd` block, the multi-statement block with `const`). - [ ] A parser gate test (tier A, in the existing parser test style) parses every example of the record and checks the tree; it rejects what section 2 rejects: a block without a result, a body ending in a `const`, `native` followed by something other than `{` in an expression, an operation with a trailing comma, a missing operand separator. - [ ] `intrinsic` is still a valid identifier outside native bodies (test: a variable, a function and a member named `intrinsic`). - [ ] A `native fn` without a body parses and checks exactly as before (existing tests unchanged); the manifest (`natives.trb`) and every `std/` file are untouched. - [ ] The checker reports `` a `native` body is not supported yet `` at the `native` keyword for a parsed body or block; a test pins the text. - [ ] `torb format` formats the new forms idempotently (formatter test for the examples), or, if the formatter walks the tree generically, a test says so. - [ ] `sh tools/gates.sh a` green in forge CI; the compiler's own sources do not use the form, so the current seed builds the commit. ## Design record point `docs/design/NATIVE-BODIES.md` sections 2 (source form), 5 (parsing) and 8 (slice 1). ## Seed impact **Yes, new syntax.** This is the first commit of the two-commit rule ("teach both forms, refresh the seed, then migrate", `CLAUDE.md` "Seed and breaking changes"). Nothing in `compiler/` or `std/` uses the form yet, so the published seed still builds it. The seed refresh stays Torben's and happens after slice 3 (lowering), before the `std` migration. ## Size One PR, one or two Conventional Commits, e.g. `feat(syntax): parse native fn bodies and native blocks` and, if split, `test(syntax): parser gate test for IR bodies of natives`. ## Dependencies Step 1 (#410, landed). Part of #53; blocks the checker step (rules N1-N10), the lowering step and the `std` migration.
orion commented 2026-10-24 07:08:37 +00:00
Author
Collaborator
Copy link

[loop:torbscript] decided by the loop: step 2 of #53 is the parser slice with a gate test, as slice 1 of docs/design/NATIVE-BODIES.md cuts it. The checker answers a parsed body with a single "not supported yet" error so the slice is self-contained and cannot leak half-checked IR into std. It is seed-relevant (new syntax, first commit of the two-commit rule) but touches no seed file and nothing in compiler/ or std/ uses the form, so the current seed builds it; the refresh stays Torben's, after the lowering step. No owner question: the record decides the syntax and reserves nothing for the owner.

[loop:torbscript] decided by the loop: step 2 of #53 is the parser slice with a gate test, as slice 1 of docs/design/NATIVE-BODIES.md cuts it. The checker answers a parsed body with a single "not supported yet" error so the slice is self-contained and cannot leak half-checked IR into std. It is seed-relevant (new syntax, first commit of the two-commit rule) but touches no seed file and nothing in compiler/ or std/ uses the form, so the current seed builds it; the refresh stays Torben's, after the lowering step. No owner question: the record decides the syntax and reserves nothing for the owner.
orion commented 2026-10-24 07:18:21 +00:00
Author
Collaborator
Copy link

[loop:torbscript] claim implement by session:loop-torbscript-20261024 until 2026-10-24T13:15:00Z

[loop:torbscript] claim implement by session:loop-torbscript-20261024 until 2026-10-24T13:15:00Z
orion commented 2026-10-24 07:19:01 +00:00
Author
Collaborator
Copy link

[loop:torbscript] implement: checkpoint, no PR yet (context guard at ~85k tokens), next: continue implement in a fresh worker. Tokens used: about 85k

Done: read LOOP.md, CLAUDE.md, CONTRIBUTING.md, NATIVE-BODIES.md; worktree code/wt/torbscript-language-467 on branch loop/467-native-body-parser from main, no commits yet.

Findings for the next worker:

  • FunctionDeclaration.body is Block? (compiler/src/syntax/ast.trb); the record wants a NativeBody(statements, result) node attached there. Least invasive: keep body: Block? and add a defaulted field nativeBody: NativeBody? = None, or turn it into a sum type, then fix every reader of .body (grep \.body on FunctionDeclaration in checker, lower, canon, format, highlight, lint, language-server).
  • ExpressionKind.NativeBlock(block: NativeBody) needs a .NativeBlock arm in every exhaustive match over ExpressionKind: syntax/nodes.trb, canon/walk.trb, canon/calls.trb, documentation/canon.trb, highlight/scope.trb, highlight/resolve.trb, semantics/scripts.trb, semantics/resolve-types.trb, semantics/checker/{expression,close,place,spelling,constant,quote}.trb, ir/lower/{expression,closure}.trb, language-server/{completion,test-tree}.trb, lint/{unread-binding,redundant-wrap}.trb, plus format/width.trb. The checker arm in checker/expression.trb reports a `native` body is not supported yet at the native keyword; the same error for a native fn body in the function-shape check.
  • Parser entry points: parseModifiers / function body parse in syntax/parser/declarations.trb (native modifier is Keyword.Native), syntax/parser/expressions.trb for the primary expression. intrinsic is read with parser.isAtWord("intrinsic") like deprecated/protected; the name add.i64.unchecked is joined from identifier and member tokens (keywords allowed after . via nameOrKeyword).
  • Tests: compiler/tests/parser.test.trb (gate test), compiler/tests/format.test.trb / canon.test.trb (formatter idempotence), a checker test for the error text. Docs: docs/ change only if a page names the grammar; then regenerate skills/.
[loop:torbscript] implement: checkpoint, no PR yet (context guard at ~85k tokens), next: continue implement in a fresh worker. Tokens used: about 85k Done: read LOOP.md, CLAUDE.md, CONTRIBUTING.md, NATIVE-BODIES.md; worktree `code/wt/torbscript-language-467` on branch `loop/467-native-body-parser` from main, no commits yet. Findings for the next worker: - `FunctionDeclaration.body` is `Block?` (`compiler/src/syntax/ast.trb`); the record wants a `NativeBody(statements, result)` node attached there. Least invasive: keep `body: Block?` and add a defaulted field `nativeBody: NativeBody? = None`, or turn it into a sum type, then fix every reader of `.body` (grep `\.body` on FunctionDeclaration in checker, lower, canon, format, highlight, lint, language-server). - `ExpressionKind.NativeBlock(block: NativeBody)` needs a `.NativeBlock` arm in every exhaustive match over `ExpressionKind`: `syntax/nodes.trb`, `canon/walk.trb`, `canon/calls.trb`, `documentation/canon.trb`, `highlight/scope.trb`, `highlight/resolve.trb`, `semantics/scripts.trb`, `semantics/resolve-types.trb`, `semantics/checker/{expression,close,place,spelling,constant,quote}.trb`, `ir/lower/{expression,closure}.trb`, `language-server/{completion,test-tree}.trb`, `lint/{unread-binding,redundant-wrap}.trb`, plus `format/width.trb`. The checker arm in `checker/expression.trb` reports `` a `native` body is not supported yet `` at the `native` keyword; the same error for a `native fn` body in the function-shape check. - Parser entry points: `parseModifiers` / function body parse in `syntax/parser/declarations.trb` (native modifier is `Keyword.Native`), `syntax/parser/expressions.trb` for the primary expression. `intrinsic` is read with `parser.isAtWord("intrinsic")` like `deprecated`/`protected`; the name `add.i64.unchecked` is joined from identifier and member tokens (keywords allowed after `.` via `nameOrKeyword`). - Tests: `compiler/tests/parser.test.trb` (gate test), `compiler/tests/format.test.trb` / `canon.test.trb` (formatter idempotence), a checker test for the error text. Docs: `docs/` change only if a page names the grammar; then regenerate `skills/`.
orion commented 2026-10-24 07:19:15 +00:00
Author
Collaborator
Copy link

[loop:torbscript] claim implement by session:loop-torbscript-20261024 until 2026-10-24T14:30:00Z

[loop:torbscript] claim implement by session:loop-torbscript-20261024 until 2026-10-24T14:30:00Z
orion commented 2026-10-24 07:21:50 +00:00
Author
Collaborator
Copy link

[loop:torbscript] implement: checkpoint 2, no PR yet (context guard), next: continue implement in a fresh worker. Tokens used: about 85k

Branch loop/467-native-body-parser is pushed with one WIP commit (67e4422e), worktree code/wt/torbscript-language-467. Nothing compiled or run locally (CI only).

Done

  • compiler/src/syntax/ast.trb: Modifiers.nativeSpan: Span? = None; FunctionDeclaration.nativeBody: NativeBody? = None; ExpressionKind.NativeBlock(body: NativeBody); new types NativeBody(keyword, statements, result, span), NativeStatement, NativeResult {Operation, Operand}, NativeOperation(name, nameSpan, operands, span), NativeOperand(kind, span), NativeOperandKind {Path, IntegerLiteral, FloatLiteral, BoolLiteral}.
  • compiler/src/syntax/parser/natives.trb (new): parseNativeBody(parser, keyword: Span) with the grammar of record section 2.1. intrinsic is an operation only when followed by an identifier (result) or after const x = (statement); otherwise an ordinary operand name. Errors: a body ending in const or empty ("A native body ends with a result, not with a const"), missing operand, trailing comma (the lexer joins the line, so } is "Expected an operand"), missing comma between operands ("Expected , or the end of the operation").

To do

  1. Parser wiring (syntax/parser/declarations.trb): set nativeSpan in parseModifiers (.Reserved(.Native) arm, span of the token); parseFunction takes native: Span? = None and, when set and at {, builds nativeBody = Some parseNativeBody(parser, keyword) with body = None; pass modifiers.nativeSpan from parseDeclaration (line ~28), parseMethod and parseStaticMember (native static fn from, needs the modifiers passed in from parseMember). Expression: parsePrimaryExpression arm .Reserved(.Native) if parser.kindAt(1) == TokenKind.BraceOpen -> ExpressionKind.NativeBlock parseNativeBody(parser, start) after bumping native; any other native keeps "Expected an expression". statements.trb line ~85: the .Reserved(.Public | ... | .Native ...) declaration arm must not take native {: add an earlier arm or guard so native { falls to the expression arm. startsCommandArgument may need .Reserved(.Native) only if a command argument may start with native { (not required).
  2. NativeBlock arms in every exhaustive match over ExpressionKind (grep .Block( / ExpressionKind.Block): the files in the first checkpoint. Syntax walkers (nodes.trb, canon/walk.trb, highlight, lint, language-server) treat it as a leaf with no sub-expressions (operands are names, not Expressions); format/width.trb and the formatter print it (see 4). Checker expression.trb: report a `native` body is not supported yet at body.keyword, result Invalid/absorbing type. Function shape (checker/declaration.trb near line 760, before the function.body.isSome() || ... return): the same error at nativeBody.keyword when function.nativeBody is Some. Lowering (ir/lower/function.trb:50, expression.trb, closure.trb): unreachable arm like the other checker-rejected nodes.
  3. Dump (syntax/dump.trb) if it matches on ExpressionKind exhaustively.
  4. Formatter: find where format/ prints a function body and expressions; print native fn ... { + one native statement per line + result, and native { ... } with const x = intrinsic name a, b / operands joined by , . Idempotence test in compiler/tests/format.test.trb for the record's examples (sections 2.3, 2.4).
  5. Tests: compiler/tests/parser.test.trb gate test (every record example: Int64.add, shiftedLeft, convert.i8.i64, isOdd block, multi-statement block with const; rejects: block without result, body ending in const, native not followed by { in an expression, trailing comma, missing separator; intrinsic as variable, function and member name outside native bodies); a checker test pinning the error text for a native fn body and a block (note native is only allowed in std/ packages, requireStandardLibrary: the test must use a std-package fixture the way existing checker tests of native do, or assert both messages).
  6. tools/check-commits.sh, torb format --check on touched files (no local torb binary found; CI runs it), amend the WIP commit message into the final Conventional Commits, PR with Closes #467, ci-wait.sh.
[loop:torbscript] implement: checkpoint 2, no PR yet (context guard), next: continue implement in a fresh worker. Tokens used: about 85k Branch `loop/467-native-body-parser` is pushed with one WIP commit (67e4422e), worktree `code/wt/torbscript-language-467`. Nothing compiled or run locally (CI only). **Done** - `compiler/src/syntax/ast.trb`: `Modifiers.nativeSpan: Span? = None`; `FunctionDeclaration.nativeBody: NativeBody? = None`; `ExpressionKind.NativeBlock(body: NativeBody)`; new types `NativeBody(keyword, statements, result, span)`, `NativeStatement`, `NativeResult {Operation, Operand}`, `NativeOperation(name, nameSpan, operands, span)`, `NativeOperand(kind, span)`, `NativeOperandKind {Path, IntegerLiteral, FloatLiteral, BoolLiteral}`. - `compiler/src/syntax/parser/natives.trb` (new): `parseNativeBody(parser, keyword: Span)` with the grammar of record section 2.1. `intrinsic` is an operation only when followed by an identifier (result) or after `const x =` (statement); otherwise an ordinary operand name. Errors: a body ending in `const` or empty ("A `native` body ends with a result, not with a `const`"), missing operand, trailing comma (the lexer joins the line, so `}` is "Expected an operand"), missing comma between operands ("Expected `,` or the end of the operation"). **To do** 1. Parser wiring (`syntax/parser/declarations.trb`): set `nativeSpan` in `parseModifiers` (`.Reserved(.Native)` arm, span of the token); `parseFunction` takes `native: Span? = None` and, when set and at `{`, builds `nativeBody = Some parseNativeBody(parser, keyword)` with `body = None`; pass `modifiers.nativeSpan` from `parseDeclaration` (line ~28), `parseMethod` and `parseStaticMember` (`native static fn from`, needs the modifiers passed in from `parseMember`). Expression: `parsePrimaryExpression` arm `.Reserved(.Native) if parser.kindAt(1) == TokenKind.BraceOpen` -> `ExpressionKind.NativeBlock parseNativeBody(parser, start)` after bumping `native`; any other `native` keeps "Expected an expression". `statements.trb` line ~85: the `.Reserved(.Public | ... | .Native ...)` declaration arm must not take `native {`: add an earlier arm or guard so `native {` falls to the expression arm. `startsCommandArgument` may need `.Reserved(.Native)` only if a command argument may start with `native {` (not required). 2. `NativeBlock` arms in every exhaustive match over `ExpressionKind` (grep `.Block(` / `ExpressionKind.Block`): the files in the first checkpoint. Syntax walkers (`nodes.trb`, `canon/walk.trb`, highlight, lint, language-server) treat it as a leaf with no sub-expressions (operands are names, not `Expression`s); `format/width.trb` and the formatter print it (see 4). Checker `expression.trb`: report `` a `native` body is not supported yet `` at `body.keyword`, result `Invalid`/absorbing type. Function shape (`checker/declaration.trb` near line 760, before the `function.body.isSome() || ...` return): the same error at `nativeBody.keyword` when `function.nativeBody` is `Some`. Lowering (`ir/lower/function.trb:50`, `expression.trb`, `closure.trb`): unreachable arm like the other checker-rejected nodes. 3. Dump (`syntax/dump.trb`) if it matches on `ExpressionKind` exhaustively. 4. Formatter: find where `format/` prints a function body and expressions; print `native fn ... {` + one native statement per line + result, and `native {` ... `}` with `const x = intrinsic name a, b` / operands joined by `, `. Idempotence test in `compiler/tests/format.test.trb` for the record's examples (sections 2.3, 2.4). 5. Tests: `compiler/tests/parser.test.trb` gate test (every record example: `Int64.add`, `shiftedLeft`, `convert.i8.i64`, `isOdd` block, multi-statement block with `const`; rejects: block without result, body ending in `const`, `native` not followed by `{` in an expression, trailing comma, missing separator; `intrinsic` as variable, function and member name outside native bodies); a checker test pinning the error text for a `native fn` body and a block (note `native` is only allowed in `std/` packages, `requireStandardLibrary`: the test must use a std-package fixture the way existing checker tests of `native` do, or assert both messages). 6. `tools/check-commits.sh`, `torb format --check` on touched files (no local `torb` binary found; CI runs it), amend the WIP commit message into the final Conventional Commits, PR with `Closes #467`, ci-wait.sh.
orion commented 2026-10-24 07:22:04 +00:00
Author
Collaborator
Copy link

[loop:torbscript] claim implement by session:loop-torbscript-20261024 until 2026-10-24T13:30:00Z (continuation 3 of 3, from checkpoint 5993)

[loop:torbscript] claim implement by session:loop-torbscript-20261024 until 2026-10-24T13:30:00Z (continuation 3 of 3, from checkpoint 5993)
orion commented 2026-10-24 07:25:33 +00:00
Author
Collaborator
Copy link

[loop:torbscript] implement: checkpoint 3, no PR yet (context guard), next: implement in a fresh worker. Tokens used: about 80k

Branch loop/467-native-body-parser pushed, two WIP commits (67e4422e, ee101b4d); worktree code/wt/torbscript-language-467. Nothing compiled or run locally (CI only). main had not moved at the time of the push.

Done in ee101b4d

  • Parser wiring: parseModifiers sets nativeSpan; parseFunction(parser, isMember, native: Span? = None) reads a { body with parseNativeBody when native is set (body = None, nativeBody = Some(...)); native passed from parseDeclaration, parseMember -> parseMethod(..., native:) (both fn and var fn) and parseStaticMember(parser, native) (native static fn from).
  • parsePrimaryExpression: .Reserved(.Native) if kindAt(1) == BraceOpen -> ExpressionKind.NativeBlock; any other native keeps "Expected an expression". parseStatement: an earlier arm sends native { to StatementKind.Evaluate parseCommandExpression(parser) instead of the declaration arm.
  • NativeBlock arms in every exhaustive ExpressionKind match (found by script: no wildcard arm): syntax/nodes.trb, canon/walk.trb, documentation/canon.trb, semantics/checker/quote.trb, language-server/test-tree.trb, lint/unread-binding.trb (leaf arms), semantics/checker/expression.trb (error). All other matches over ExpressionKind have a wildcard; ir/lower/expression.trb falls to unsupportedSlot ("an expression the front end rejected").
  • Checker: "A native body is not supported yet" (capitalised like every diagnostic) at body.keyword, in evaluateExpression and at the top of checkFunctionShape (declaration.trb) for function.nativeBody. Outside std/ the native modifier error of requireStandardLibrary comes as well.
  • Formatter: token based (format/layout.trb), no change expected for the new forms; the idempotence test is still to be written.
  • Two lines in natives.trb shortened to the 120 column width.

To do

  1. Tests: compiler/tests/parser.test.trb gate test with every example of docs/design/NATIVE-BODIES.md sections 2.3 and 2.4 (Int64.add, shiftedLeft, native static fn from with convert.i8.i64, isOdd block, two-statement block) and the rejects (body without result / ending in const, native not followed by { in an expression, trailing comma, missing separator, const x = notIntrinsic); intrinsic as variable, function and member name. Formatter idempotence in compiler/tests/format.test.trb for the same examples. Checker test pinning the error text (see the existing tests of native for the std-package fixture).
  2. tools/check-commits.sh; squash the two WIP commits into one Conventional Commit (feat(syntax): parse native fn bodies and native blocks, body Closes #467), rebase on main, force-push, PR, ci-wait.sh.
[loop:torbscript] implement: checkpoint 3, no PR yet (context guard), next: implement in a fresh worker. Tokens used: about 80k Branch `loop/467-native-body-parser` pushed, two WIP commits (67e4422e, ee101b4d); worktree `code/wt/torbscript-language-467`. Nothing compiled or run locally (CI only). `main` had not moved at the time of the push. **Done in ee101b4d** - Parser wiring: `parseModifiers` sets `nativeSpan`; `parseFunction(parser, isMember, native: Span? = None)` reads a `{` body with `parseNativeBody` when `native` is set (`body = None`, `nativeBody = Some(...)`); `native` passed from `parseDeclaration`, `parseMember` -> `parseMethod(..., native:)` (both `fn` and `var fn`) and `parseStaticMember(parser, native)` (`native static fn from`). - `parsePrimaryExpression`: `.Reserved(.Native) if kindAt(1) == BraceOpen` -> `ExpressionKind.NativeBlock`; any other `native` keeps "Expected an expression". `parseStatement`: an earlier arm sends `native {` to `StatementKind.Evaluate parseCommandExpression(parser)` instead of the declaration arm. - `NativeBlock` arms in every exhaustive `ExpressionKind` match (found by script: no wildcard arm): `syntax/nodes.trb`, `canon/walk.trb`, `documentation/canon.trb`, `semantics/checker/quote.trb`, `language-server/test-tree.trb`, `lint/unread-binding.trb` (leaf arms), `semantics/checker/expression.trb` (error). All other matches over `ExpressionKind` have a wildcard; `ir/lower/expression.trb` falls to `unsupportedSlot` ("an expression the front end rejected"). - Checker: "A `native` body is not supported yet" (capitalised like every diagnostic) at `body.keyword`, in `evaluateExpression` and at the top of `checkFunctionShape` (`declaration.trb`) for `function.nativeBody`. Outside `std/` the `native` modifier error of `requireStandardLibrary` comes as well. - Formatter: token based (`format/layout.trb`), no change expected for the new forms; the idempotence test is still to be written. - Two lines in `natives.trb` shortened to the 120 column width. **To do** 1. Tests: `compiler/tests/parser.test.trb` gate test with every example of docs/design/NATIVE-BODIES.md sections 2.3 and 2.4 (`Int64.add`, `shiftedLeft`, `native static fn from` with `convert.i8.i64`, `isOdd` block, two-statement block) and the rejects (body without result / ending in `const`, `native` not followed by `{` in an expression, trailing comma, missing separator, `const x = notIntrinsic`); `intrinsic` as variable, function and member name. Formatter idempotence in `compiler/tests/format.test.trb` for the same examples. Checker test pinning the error text (see the existing tests of `native` for the std-package fixture). 2. `tools/check-commits.sh`; squash the two WIP commits into one Conventional Commit (`feat(syntax): parse native fn bodies and native blocks`, body `Closes #467`), rebase on main, force-push, PR, `ci-wait.sh`.
orion commented 2026-10-24 07:25:36 +00:00
Author
Collaborator
Copy link

[loop:torbscript] release implement: continuation budget used up, checkpoint 6000, next: implement

[loop:torbscript] release implement: continuation budget used up, checkpoint 6000, next: implement
orion commented 2026-10-24 08:10:28 +00:00
Author
Collaborator
Copy link

[loop:torbscript] claim implement by session:loop-torbscript-20261024-1000 until 2026-10-24T14:10:28Z

[loop:torbscript] claim implement by session:loop-torbscript-20261024-1000 until 2026-10-24T14:10:28Z
orion commented 2026-10-24 08:11:44 +00:00
Author
Collaborator
Copy link

[loop:torbscript] implement: checkpoint 4, no PR yet (context guard), next: continue implement in a fresh worker. Tokens used: about 85k

Branch loop/467-native-body-parser rebased onto current main (57085dce, the tls commit) and force-pushed; two WIP commits (c864ae3f, d0755639), worktree code/wt/torbscript-language-467. Nothing compiled or run locally (CI only). The source side from checkpoint 3 is complete and re-read this round (AST, parser/natives.trb, wiring in declarations.trb/expressions.trb/statements.trb, NativeBlock arms, the checker error in checkFunctionShape and evaluateExpression); no source change was needed.

To do (tests only, then PR)

  1. compiler/tests/parser.test.trb: new group (raw """ strings need no brace escapes). Helper that parses native fn ... / extend Int64 with Add { native fn ... } (ExtendDeclaration.members -> MemberKind.Function -> nativeBody) and a block (const remainder = native { ... } -> Bind(binding).value.kind is .NativeBlock(body)), plus a describe(NativeBody): String renderer (const sum = add.i64.unchecked(left, right); less.i64(sum, limit)). Cases: record 2.3/2.4 examples (Int64.add, shiftedLeft, native static fn from with convert.i8.i64, isOdd block, two-statement block); rejects: no result / ends in const ("A native body ends with a result, not with a const"), native + non-{ in an expression ("Expected an expression"), trailing comma ("Expected an operand ..."), missing separator ("Expected , or the end of the operation ..."), const x = notIntrinsic; intrinsic as variable, function and member name outside bodies; native fn without body has nativeBody == None.
  2. Checker test pinning "A native body is not supported yet" in compiler/tests/program-shape.test.trb next to the group "native belongs to the standard library" (problems from ./harness; an ordinary package also gets the native is only allowed error, so use .contains). Block case: fn probe(): Int { native { intrinsic add.i64 1, 2 } }.
  3. Formatter idempotence: compiler/tests/format.test.trb keeps/formats for the record examples (the formatter is token based; if CI shows dropped or a changed layout inside a body, fix format/layout.trb).
  4. Squash the two WIP commits into one feat(syntax): parse native fn bodies and native blocks (body Closes #467; the test commit may stay separate as test(syntax): ...), tools/check-commits.sh origin/main, push, open PR Closes #467, ci-wait.sh; fix red CI (first CI run is the first compile of all of this, expect type errors to fix).
[loop:torbscript] implement: checkpoint 4, no PR yet (context guard), next: continue implement in a fresh worker. Tokens used: about 85k Branch `loop/467-native-body-parser` rebased onto current `main` (57085dce, the tls commit) and force-pushed; two WIP commits (c864ae3f, d0755639), worktree `code/wt/torbscript-language-467`. Nothing compiled or run locally (CI only). The source side from checkpoint 3 is complete and re-read this round (AST, `parser/natives.trb`, wiring in `declarations.trb`/`expressions.trb`/`statements.trb`, `NativeBlock` arms, the checker error in `checkFunctionShape` and `evaluateExpression`); no source change was needed. **To do (tests only, then PR)** 1. `compiler/tests/parser.test.trb`: new `group` (raw `"""` strings need no brace escapes). Helper that parses `native fn ...` / `extend Int64 with Add { native fn ... }` (ExtendDeclaration.members -> MemberKind.Function -> `nativeBody`) and a block (`const remainder = native { ... }` -> `Bind(binding).value.kind` is `.NativeBlock(body)`), plus a `describe(NativeBody): String` renderer (`const sum = add.i64.unchecked(left, right); less.i64(sum, limit)`). Cases: record 2.3/2.4 examples (`Int64.add`, `shiftedLeft`, `native static fn from` with `convert.i8.i64`, `isOdd` block, two-statement block); rejects: no result / ends in `const` ("A `native` body ends with a result, not with a `const`"), `native` + non-`{` in an expression ("Expected an expression"), trailing comma ("Expected an operand ..."), missing separator ("Expected `,` or the end of the operation ..."), `const x = notIntrinsic`; `intrinsic` as variable, function and member name outside bodies; `native fn` without body has `nativeBody == None`. 2. Checker test pinning "A `native` body is not supported yet" in `compiler/tests/program-shape.test.trb` next to the group "`native` belongs to the standard library" (`problems` from `./harness`; an ordinary package also gets the `native is only allowed` error, so use `.contains`). Block case: `fn probe(): Int { native { intrinsic add.i64 1, 2 } }`. 3. Formatter idempotence: `compiler/tests/format.test.trb` `keeps`/`formats` for the record examples (the formatter is token based; if CI shows `dropped` or a changed layout inside a body, fix `format/layout.trb`). 4. Squash the two WIP commits into one `feat(syntax): parse native fn bodies and native blocks` (body `Closes #467`; the test commit may stay separate as `test(syntax): ...`), `tools/check-commits.sh origin/main`, push, open PR `Closes #467`, `ci-wait.sh`; fix red CI (first CI run is the first compile of all of this, expect type errors to fix).
orion commented 2026-10-24 08:11:47 +00:00
Author
Collaborator
Copy link

[loop:torbscript] implement: checkpoint 4 above is the current state, next: continue implement in a fresh worker. CONTINUE: torbscript/language#467

[loop:torbscript] implement: checkpoint 4 above is the current state, next: continue implement in a fresh worker. CONTINUE: torbscript/language#467
orion commented 2026-10-24 08:12:31 +00:00
Author
Collaborator
Copy link

[loop:torbscript] claim implement by session:loop-torbscript-20261024-1000 until 2026-10-24T14:12:30Z (continuation 2 of 3, from checkpoint 4)

[loop:torbscript] claim implement by session:loop-torbscript-20261024-1000 until 2026-10-24T14:12:30Z (continuation 2 of 3, from checkpoint 4)
orion commented 2026-10-24 09:43:30 +00:00
Author
Collaborator
Copy link

[loop:torbscript] release implement: worker run ended (timeout) after opening PR #472; branch loop/467-native-body-parser at 5587ddd7, forge CI pending, review requested from argus. next: review (wait for CI and the merge gate; fix round on the same branch if red).

[loop:torbscript] release implement: worker run ended (timeout) after opening PR #472; branch `loop/467-native-body-parser` at 5587ddd7, forge CI pending, review requested from argus. next: review (wait for CI and the merge gate; fix round on the same branch if red).
argus closed this issue 2026-10-24 15:53:50 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
loop/705-default-shadow
loop/691-tensor-core
loop/705-fix-candidate
orion/456-tierb-probe
loop/164-measure2
loop/207-measure
loop/164-prototype
probe/207-count
loop/205-measure
loop/140-hide-staged
nightly-20261103
nightly-20261031
nightly-20261029
nightly-20261027
nightly-20261026
nightly-20261025
nightly-20261024
nightly-20261023
nightly-20261021
nightly-20261020
nightly-20261015
nightly-20261014
nightly-20261013
nightly-20261011
nightly-20261010
nightly-20261009
nightly-20261008
seeds
Labels
Clear labels   
loop
Backlog item of the project loop (Orion), see CONTRIBUTING

  
ready
Refined to the definition of ready

No labels
loop
ready
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
torbscript/language#467
No description provided.