natives: checker rules N1-N10 and the intrinsic typing table for native bodies (step 3 of #53) #478
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Context
Step 3 of #53 (natives:
native fnbodies as IR andnative { }blocks). Step 1 (#410) landed the design recorddocs/design/NATIVE-BODIES.md, step 2 (#467, PR #472) the parser: a body or block is parsed intoNativeBody/ExpressionKind.NativeBlockand the checker answers every one withA `native` body is not supported yet(compiler/src/semantics/checker/declaration.trb). This is slice 2 of the record's section 8: the checker rules. Nothing is lowered yet.Goal
Replace the single "not supported yet" error by the verification of section 3 and the rules N1 to N10 of section 4, driven by one typing table, so a malformed body is refused with the record's text and a well-formed one is accepted by the checker.
intrinsicSignatureof section 3.2 (operands and result perIntrinsicOperationand numeric kind) as the single source, incompiler/src/ir/next toIntrinsic. The IR verifier (ir/verify.trb) and the new check both call it. A test fails while anIntrinsicOperationhas no row.uncheckedshape), N3 (arity), N4 (compiler-emitted intrinsics are not nameable), N5 (operand types, literals fit), N6 (scope of names, novaroperand), N7 (result type, no trailingconst), N8 (uncheckedonly onadd,subtract,multiply,negateof an integer kind), N9 (manifest mismatch as an internal error: anative fnwith a body that has a manifest row; the converse half of the rule is the existing text), N10 (no generic type parameter in a body). Error texts, notes and spans exactly as the table of section 4.std.Acceptance criteria
intrinsicSignatureexists once;ir/verify.trbuses it for the typing of intrinsic calls, and a test enumerates everyIntrinsicOperationagainst it.compiler/tests/program-shape.test.trb; N1 uses an ordinary package, the others a std package fixture.Int64.add,shiftedLeft,convert.i8.i64, theisOddblock, the multi-statement block) check without error in a std package fixture.native fnand the manifest are unchanged; nostd/file and nocompiler/source uses the form; existing tests unchanged.sh tools/gates.sh agreen in forge CI; the commit builds on the current seed.Design record point
docs/design/NATIVE-BODIES.mdsections 3 (what the checker verifies), 4 (rules N1-N10) and 8 (slice 2). Update section 8 if the table's module differs from the one named here.Seed impact
None of its own: the form is already taught by #467 and the compiler's own sources do not use it, so the current seed builds the commit. No new native, no new driver flag, no name the compiler looks up by string. The two-commit rule continues with step 4 (lowering, then the seed refresh, which stays Torben's) and the
stdmigration.Size
One PR, one or two Conventional Commits:
refactor(ir): one typing table for intrinsics(table plus verifier switch, if the diff warrants a split) andfeat(checker): verify native bodies and blocks. The table is a deliberate part of this step because the record makes the checker and the verifier share it.Dependencies
[loop:torbscript] decided by the loop: this is slice 2 of the record (checker rules), filed just in time now that #467 landed (PR #472). Keyword search for duplicates (native, native fn, native block, checker rules, IR body) found no open issue for it. The shared typing table moves into this step because section 3.2 makes it the single source for the verifier and the checker; the 1-2 commit size holds (table + verifier switch, rules + tests). No seed impact and no owner question: the record decides the rules and reserves nothing for the owner. Marked
ready.native fnbodies as IR andnative { }blocks #53[loop:torbscript] claim implement by session:loop-torbscript-20261025-0258 until 2026-10-25T07:25:00Z
[loop:torbscript] implement: checkpoint (worker context budget reached, no code written yet), next: implement continues with a fresh worker (claim by session:loop-torbscript-20261025-0258 stays, lease until 2026-10-25T07:25:00Z).
Done: worktree
code/wt/torbscript-language-478-native-checkeron branchloop/478-native-checkerfrommain32d880c9(not pushed, no changes yet). Read the design record, CLAUDE.md and the code the step touches. Worker usage so far: about 90k prompt tokens.Findings for the next worker:
compiler/src/syntax/ast.trb(NativeBody,NativeStatement,NativeResult,NativeOperation{name: String joined with ".", nameSpan, operands, span},NativeOperand{Path|IntegerLiteral|FloatLiteral|BoolLiteral}); the name is one joined string, so the span of theuncheckedsuffix (N8) has to be computed fromnameSpan(last 9 columns) or the parser has to keep the segment spans.compiler/src/semantics/checker/declaration.trb:763(native fn body) andcompiler/src/semantics/checker/expression.trb:184(.NativeBlock).Intrinsic{operation, kind, checked},IntrinsicOperation,NumericKindincompiler/src/ir/ir.trb:552-640; names viadescribeIntrinsic/describeOperation(private) inir/print.trb:661-733. Typing is spread overir/verify.trb:1342-1640(verifyIntrinsic,verifyOperands,verifyIntrinsicResult,operandCountOf(public, only used there),needsNumericKind,allowsNumericKind,producesNoValue,conversionKindOfwhere aCharis a valid conversion source, comparisons without kind compare any two equal types,textConcatoperands may be primitives the runtime shows).compiler/src/ir/intrinsics.trbwithintrinsicSignature(Intrinsic)as an IR-type-independent shape (operand/result asNumeric(kind),Bool,Text,SameAsFirst,Integer, variadic flag, no-result flag,emittedByCompilerflag) plus a parse of the printed name into anIntrinsic; makeverify.trbderive operand count, kind rules and result from it while keeping its messages (check tests that pin them before touching); a test enumerates everyIntrinsicOperation(use an exhaustivematchin the table so a missing row fails the build, plus the enumeration test). Then the checker module (new file next todeclaration.trb), tests incompiler/tests/native-bodies.test.trbin the style ofprogram-shape.test.trbwith a std package fixture.textCompareis the lowering's own, N4's list and the grammar disagree (grammar lists it, N4 list omits it). Decide: not nameable (the lowering emits it only against zero, and no manifest row is a.IntrinsictextCompare), and fix N4's list and the grammar in section 2.1/4 ofdocs/design/NATIVE-BODIES.md, then regenerateskills/only if the docs template pulls it in.torbbinary exists on the host: write.trbcarefully and let forge CI be the compiler;torb format --checkis not available locally either, so match the surrounding layout by hand (120 columns, no semicolons).Still open: everything in the acceptance criteria.
[loop:torbscript] release implement: the worker ended at its context budget without code (checkpoint above, CONTINUE 1 of 3), and this session has no spawn tool to start the successor, so the claim is released instead of holding a slot until its lease ends. next: implement continues with a fresh worker from the checkpoint at the next tick.
[loop:torbscript] claim implement by session:loop-torbscript-20261025-0258 until 2026-10-25T07:40:00Z (continuation 1 of 3, from the checkpoint)
[loop:torbscript] implement: checkpoint 2 (continuation 1 reached its context guard), next: implement continues with a fresh worker (claim by session:loop-torbscript-20261025-0258 stays).
Pushed: branch
loop/478-native-checker@2a640345, one commitrefactor(ir): add the typing table of intrinsics= newcompiler/src/ir/intrinsics.trb(NOT yet used by anything, not compiled yet):intrinsicSignature(Intrinsic): IntrinsicSignature{operands: List<IntrinsicType>, variadic, result}(exhaustivematch, no wildcard, so a newIntrinsicOperationfails the build),IntrinsicType{Numeric(kind), Source(kind), Boolean, Text, TextPart, SameAsFirst, LazyCell, LazyItem, Anything},kindUseOf(Required/Optional/Forbidden),isEmittedByCompiler(N4; includestextCompare, see the record fix below),everyOperation()for the enumeration test.Remaining (in order):
verify.trb(lines ~1342-1640): replaceoperandCountOf(public, no other user),needsNumericKind/allowsNumericKind,producesNoValue(public, no other user) and the operand/result typing by the table. Keep the messages pinned bytests/ir.test.trb(lines 539the result oflessis aBool, and %2 isInt64, 562 `the result of `add.i64` is `Int32` and %1 is `Int64, 621-684, 790): for aNumericresult whose operand 0 has the same shape, keepexpectTypeagainst the first operand;Convertresult keepsexpectNumeric;Textoperand of printParts keepsverifyTextOperands,TextPartkeepsverifyPartOperands,SameAsFirstkeepsverifySameOperands,Anything/LazyCelloperands unchecked,LazyItemresult keeps the lazy-cell check.describeOperationin print.trb is private: make it public or add a name parser (parseIntrinsicName(String): Intrinsic?-like, for N2) in intrinsics.trb.checker/declaration.trb:763incheckFunctionShape, which is a syntactic pass before types exist, andchecker/expression.trb:184.NativeBlock) by a newchecker/native.trb. Open question found: operand types needTypeIds of parameters/self/consts; look at howcheck.trb:421/:976andstatement.trb:606check a function body (scope + slots) and where the std-package test is (packageNameOf,requireStandardLibraryin declaration.trb:707-735 for N1; message and note are in the record). Map a TypeId toNumericKindvia theNominalsymbol names (WellKnown, wellknown.trb); N8's span: last 9 columns ofnameSpan(name is a joined String).compiler/tests/native-bodies.test.trb(N1 withproblems, others with a std package fixture viaproblemsAt(files, ["std/..."]), see implementations.test.trb:149) plus a test overeveryOperation(); updateprogram-shape.test.trb:364-381("not supported yet" tests) and the record: fix N4 list and the grammar (textCompareis compiler-only), section 8 module namecompiler/src/ir/intrinsics.trb.Closes #478, review argus.[loop:torbscript] implement: checkpoint 3 (continuation 2 reached its context guard), next: implement continues with a fresh worker (claim by session:loop-torbscript-20261025-0258 stays).
Pushed: branch
loop/478-native-checker@c4fea69d, two commits:refactor(ir): add the typing table of intrinsics(newcompiler/src/ir/intrinsics.trb) andrefactor(ir): verify intrinsic calls against the typing table(step 1 DONE, not compiled locally; CI is the compiler):verify.trbverifyIntrinsicnow usesintrinsicSignature/kindUseOf;operandCountOf,producesNoValue,needsNumericKind,allowsNumericKind,isShift/Conversion/TextParts/Comparison/BooleanOperationremoved; new table-drivenverifyOperands,verifyIntrinsicResult+verifyResultRow;print.trbdescribeOperationis now public. Risk to watch on first CI: or-patterns with payloads (.Source(_)arms),rows[if .. {..} else {..}]indexing in verifyOperands,textEqualoperand message is now "joins strings" (was "compares"),Booleanoperand ofbooleanNotis unchecked. No PR opened yet; runsh tools/gates.sh aonly in CI (open a draft-free PR early to get CI on this commit, if it compiles in CI fix first).Remaining:
2. Checker (new
compiler/src/semantics/checker/native.trb). Findings:syntax/ast.trb:451-505(NativeBody{keyword,statements,result,span},NativeStatement{name,type,operation,span},NativeOperation{name,nameSpan,operands,span},NativeOperandKind.Path/IntegerLiteral/FloatLiteral/BoolLiteral);FunctionDeclaration.nativeBody(ast.trb:203),bodyis None then.checker/declaration.trb:762(insidecheckFunctionShape, syntactic: keep only N1 there viarequireStandardLibrary-stylepackageNameOf(checker, module).startsWith("std/")atbody.keyword, message+note in record section 4 N1; N10 can go here too: anyfunction.genericsentry => error at its name span) andchecker/expression.trb:184(.NativeBlock(body): N1 + verify withchecker.findLocal).statement.trb:591 checkFunctionBodyreturnsvoidTypewhenfunction.bodyis None; add an early branch forfunction.nativeBodyafterchecker.openScope(); declareGenerics; declareBindingsFor ...(same set-up as ordinary), then callcheckNativeBody(checker, body, result); alsocheck.trb:420 isBodyWithinonly looks atfunction.bodyspan: usenativeBodyspan too, else the body is never visited.signature.trb:580 hasBody=function.body.isSome()(leave).checker.findLocal(text)->LocalKind.Bound(id)->checker.binding(id)isLocalBinding{type: TypeId, var: Bool, bindsParameter, ...}(context.trb:90-110);selfischecker.receiverBinding(name.trb:64-80). Field path: look up member types like name.trblookupMember.WellKnown.numerics: Map<Int,String>(symbol value -> "Int64"/"UInt8"/"Float32"),.boolean,.string,.character(checker/wellknown.trb);checker.types.form(id)is.Nominal(symbol, args).operation.nameon "."; operation names viadescribeOperation(print.trb) overeveryOperation();convert.F.T; suffixunchecked; kind suffixesi8..f64. N4 viaisEmittedByCompiler, kind use viakindUseOf. N8 span: last 9 columns ofnameSpan. N9: find where the manifest rows are read (grepnative fn without a body); N9 error "has a native body and a row in the manifest" is an internal error, check that site.compiler/tests/native-bodies.test.trb(N1 viaproblems, others std fixture viaproblemsAt(files, ["std/..."]), seeimplementations.test.trb:149), a test overeveryOperation()(each has a row;isEmittedByCompilerones are not nameable), the record examples 2.3/2.4 check clean; updateprogram-shape.test.trb:364-381("not supported yet" tests); recorddocs/design/NATIVE-BODIES.md: N4 list and section 3.2 text addtextCompareas compiler-only, section 8 module namecompiler/src/ir/intrinsics.trb.Closes #478against main, review argus, wait CI ~5 min, final write-back.Token usage: continuation 2 ~85k prompt tokens.
[loop:torbscript] implement: checkpoint 4 (continuation 3 reached its context guard), next: implement continues with a fresh worker (claim by session:loop-torbscript-20261025-0258 stays).
Pushed: branch
loop/478-native-checker@14a3626d(4 commits, not compiled locally, no PR yet). New:compiler/src/semantics/checker/native.trb(N1allowsNativeBody, N2 to N5, N6, N7 result mismatch, N8 viacheckNativeOperation/intrinsicNamed/operandType; types compared by name String: Int64/Bool/String/Char). Wired:statement.trbcheckNativeFunction(early branch incheckFunctionBody),expression.trb.NativeBlock=>checkNativeBody(+fitsIntonow public),check.trb isBodyWithinalso looks atnativeBody,declaration.trb checkFunctionShape: N10 only (error at generic name span), N1 left tocheckNativeBody.Risks to look at on first CI: cyclic
usebetween native.trb and statement/expression (should be fine, other checker files are cyclic);match text { "i8" => ... }on String;Some(.Signed(_)) | Some(.Unsigned(_))or-pattern inallowsUnchecked;names.map({ _.text }).toList().joined(...);.Source(kind)or-pattern with.Numeric(kind)inexpectedName(may need two arms);parts[0]/rest.last()forms;Span(end - 9, end)constructor;(expected ?? "Float64").startsWith;checker.nominalOrInvalid/booleanType()arevar fn(mytypeOfNamehasvar checker); unused imports (IntrinsicOperation/Intrinsic used;LocalKind,NativeStatementimports ok, drop unused if lint complains). N7b (body ends in aconst) is already a parser error with its own text (parser/natives.trb), the record text differs: align the record, not the code.Remaining:
ir/witness.trb generatedKindOf(line ~455,nativeNamed). Plan: addhasNativeBody: Bool = falselast field toSignature(checker/type.trb:460), set it wherehasBodyis computed (checker/signature.trb:580,function.nativeBody.isSome()), and at the top ofgeneratedKindOfreport viareportUnsupported:\Owner.name` has a native body and a row in the manifestwhennativeNamed` hits, else a clean 'native bodies are not lowered yet'.compiler/tests/native-bodies.test.trb: N1 (ordinary package,problems), N2..N8, N10 via std fixture (problemsAt(files, ["std/..."]), seeimplementations.test.trb:149), record examples 2.3/2.4 clean, a test overeveryOperation()(eachintrinsicSignaturerow exists;isEmittedByCompilerones are refused by N4). Updateprogram-shape.test.trb:364-381(the two 'not supported yet' tests: now N1 text for ordinary package).docs/design/NATIVE-BODIES.md: N4 list and 3.2 addtextCompare, section 8 module namecompiler/src/ir/intrinsics.trb, N7b wording (parser text), N9 as a lowering message.Closes #478), request review from argus, wait CI ~5 min, final write-back.Token usage: continuation 3 ~95k prompt tokens.
[loop:torbscript] release implement: continuation limit (3 of 3) reached after checkpoint 4. Branch loop/478-native-checker @
14a3626dholds the typing table, the verify.trb switch and the checker N1 to N8 plus wiring (not yet compiled, no PR). Remaining: N9 (manifest mismatch), tests in compiler/tests/native-bodies.test.trb, the design record edits, PR and review from argus, CI. next: implement (a new worker chain from checkpoint 4 at the next tick, with a fresh claim).[loop:torbscript] claim implement by session:loop-torbscript-20261025-0258 until 2026-10-25T08:30:00Z (new chain 2, from checkpoint 4)
[loop:torbscript] implement: checkpoint 5 (chain 2 reached its context guard), next: implement continues with a fresh worker (claim by session:loop-torbscript-20261025-0258 stays).
Pushed: branch
loop/478-native-checker@203a9f84(8 commits, not compiled locally, no PR yet). New since checkpoint 4:Signature.hasNativeBody: Bool = false(last field,checker/type.trb), set in bothfunctionSignaturereturns that can see a native body (checker/signature.trb);ir/witness.trb generatedKindOfnow refuses a native body first: with a manifest row"`X.y` is not supported by the native back end yet: it has a native body and a row in the manifest", else "its native body is not lowered yet" (reportUnsupported, so the text isunsupportedMessage's).ir/intrinsics.trb.Remaining:
compiler/tests/native-bodies.test.trb(does not exist yet). Layout: header doc comment,use test, group from "std/test",use problems, problemsAt from "./harness",group "..." { test "..." { ... } }. Std fixture likeimplementations.test.trb:149:["std/graphics/project.trb": "name \"std/graphics\"\n", "std/graphics/src/lib.trb": source]withproblemsAt(files, ["std/graphics"]); use freepublic native fn(anextend Int64in a std fixture would fail coherence). The harness prelude (tests/harness.trb) only has Int8/Int32/Int64/UInt8/Float32/Float64/Bool/Char/String. Cases: cleandouble(intrinsic add.i64 value, value) and a body withconststatements; N1 (ordinary package:problems(...).contains(...), bothdeclaration.trb:720and the N1 text appear for anative fn); N2 (unknown name,addwithout kind,booleanNot.i64); N3 arity; N4 (printParts,textCompare); N5 (Int64 vs String operand, literal that does not fitadd.i8 value, 300); N6 (unknown name,varparameter); N7 (Boolbody forInt64declaration); N8 (uncheckedonless); N10 (native fn same<Value>(...)); a test overeveryOperation()(eachintrinsicSignaturerow exists,isEmittedByCompilerones are refused by N4,use ... from "../src/ir/intrinsics").program-shape.test.trbgroup "Anativebody is parsed and not supported yet" (lines ~364-381) still expects the removed text "Anativebody is not supported yet": rename the group and expect the N1 text`native` bodies are only allowed in a package of the standard library(.contains).feat(checker): ...,Closes #478), request review from argus, wait for CI (the first compile of native.trb is the risk list of checkpoint 4), write-back.[loop:torbscript] implement: PR #480 (head
95409405ac), next: waiting (CI + review)[loop:torbscript] claim implement by session:loop-torbscript-20261025-0258 until 2026-10-25T08:30:00Z (CI fix round 1 on PR 480: commit messages and bootstrap (linux-x64) red on
95409405)[loop:torbscript] release implement: CI fix round 1 bootstrap parse errors fixed (nested juxtaposed constructor calls now parenthesized,
native/typeused as names renamed, Map.contains -> containsKey,assert !xrewritten, match scrutinee call parenthesized)e3f9f820, next: waiting (CI + review on PR 480). Open:ci / commit messagesfails onb4fe0944style(checker): ...(typestyleis not in feat fix perf refactor docs test build ci chore revert); fixing it needs a reword of that mid-history commit (force-push) or a squash, awaiting that decision. Type errors behind the parse errors are not visible yet, so a round 2 is likely.[loop:torbscript] decided by the loop: commit
b4fe0944 style(checker): ...reworded tochore(checker): keep the native checker lines within 120 columns(styleis not an allowed type in tools/check-commits.sh; PR 480 had no review yet, trees identical). New head14db9dc3, check-commits passes locally. next: waiting (CI on PR #480, bootstrap round 2 expected)[loop:torbscript] claim implement by session:loop-torbscript-20261025-0258 until 2026-10-25T09:30:00Z (CI fix round 2 on PR 480: bootstrap (linux-x64) red on
14db9dc3, run 1714 job 4)[loop:torbscript] release implement: CI fix round 2 the stage-1 compiler no longer panics in verify.trb (intrinsic with no operands,
returnTraceMark, indexedoperands[0]); the seed already built stage 1 with the native checker, so no more parse/type errors were left in this run -813a413e, next: waiting (CI + review on PR 480)[loop:torbscript] claim implement by session:loop-torbscript-20261025-0542 until 2026-10-25T11:45:00Z (CI fix round 3 on PR 480: tier A (linux-x64) and gates red on
813a413e, run 1718 jobs 5 and 2)[loop:torbscript] implement: checkpoint 6 (CI fix round 3 worker reached its context guard, no code changed, head stays
813a413e), next: implement continues with a fresh worker (claim by session:loop-torbscript-20261025-0542 stays). Worker usage about 90k prompt tokens.Root causes from run 1718 (read via
GET /actions/runs/1764/jobs, then/actions/jobs/<id>/logs; the job numbers in the status URLs are run-relative, the log API needs the global id, here 19898 = tier A; thegatesjob log returns HTTP 500 and was cancelled, it only repeats tier A):format --check:compiler/src/semantics/checker/native.trbandcompiler/tests/native-bodies.test.trbwould change. Needstorb formaton both (no torb binary in the gateway pod; take the formatter from a CI artifact/seed or reformat by hand to the canon).N7: a body that ends in a const: the parser reports "A native body ends with a result" and hands the checkermissingResult=NativeOperand(Path([]));operandType(native.trb ~line 292,names[0].text) panics with index 0 out of bounds. Fix: in the.Path(names)case returnNonewhennames.isEmpty()(the parser already reported it).constfail without a visible message (the log shows only the assertion):a body of const statements ends in a name(const wide: Int64 = intrinsic shiftedLeft.i64 sum, 1expects no problem) andthe annotation of a const has to be the type of its operation(expects exactly "sumisInt64, the annotation saysBool"). Tests without an annotation pass. SuspectcheckNativeStatement(native.trb ~line 100):typeFromReference(checker, reference, scopeHere(checker).forType())ornameOfTypeyields another name/extra diagnostic in the std test package (e.g.scope.recordsrecording the type span, orInt64/Boolnot resolving viawellKnownthere). Next worker: make the test printinStd(source)in the assertion (or assert against a local variable first) in one CI round, or compare how other checker code resolves a type name to a String (checker.describe).State: worktree
code/wt/torbscript-language-478-native-checkerclean at813a413e, nothing unpushed.[loop:torbscript] release implement: PR 480 head moved to
900430cbafter the round-3 checkpoint, CI is running on it; if it is red again the next fix worker starts from the checkpoint comment above, next: waiting (CI + review on PR 480)[loop:torbscript] claim implement by session:loop-torbscript-20261025-0542 until 2026-10-25T14:20:00Z (CI fix round 4 on PR 480: tier A + gates red on
900430cb, run 1730; start from the round-3 checkpoint)[loop:torbscript] release implement: PR 480 head moved to
6c53ec44after fix round 4 (worker ended without a release comment), CI is running on it; if it is red again the next fix worker starts from the round-3 checkpoint and the failing job logs, next: waiting (CI + review on PR 480)native fnbodies andnative { }blocks, kernel test (step 4 of #53) #484native fnbodies andnative { }blocks, kernel test (step 4 of #53) #484native fnbodies as IR andnative { }blocks #53native fnbodies as IR andnative { }blocks #53native fnbodies as IR andnative { }blocks #53