From ae6df4fa1f103ff9b14963731b2edd84e38738b1 Mon Sep 17 00:00:00 2001 From: "sorokin.oleg.edu" Date: Thu, 24 Sep 2026 14:29:26 -0700 Subject: [PATCH] Publish independent mixed index key and byte-span oracle --- .../analysis-mixed-index-20260924/ORACLE.md | 116 ++++++++++++++++++ .../check_oracle_bytes.py | 51 ++++++++ .../analysis-mixed-index-contract-20260924.md | 6 + 3 files changed, 173 insertions(+) create mode 100644 harness/analysis-mixed-index-20260924/ORACLE.md create mode 100644 harness/analysis-mixed-index-20260924/check_oracle_bytes.py diff --git a/harness/analysis-mixed-index-20260924/ORACLE.md b/harness/analysis-mixed-index-20260924/ORACLE.md new file mode 100644 index 00000000..14a07788 --- /dev/null +++ b/harness/analysis-mixed-index-20260924/ORACLE.md @@ -0,0 +1,116 @@ +# Exhaustive authored mixed typed-index oracle + +2026-09-24. Independent source/data derivation for private source `aaec78fcc9b273d1485497126392afdd1fb2e54e`. Reuse fixture and independent model construction in `crates/oa-core/src/language/analysis_source/mixed/project/tests.rs:254–330`, `expected` at line 52, and the explicit site oracle at lines 331–538. No Index, lowerer, formatter, parser or compiler executed to produce expectations. Numeric spans below were checked using direct unique substring locations in independently reconstructed literal source; all original fixture bytes are ASCII/LF. This is suggested test data, not a passing-test report. + +## Fixture identity and file binding + +Use the existing `id(n)` helper (line 9): it formats n as twelve decimal digits inside a UUID then parses that UUID. **Do not substitute `Id::from_u128(n)`**: for n=10 the fixture suffix is `000000000010`, not `00000000000a`. + +Retain entry `main.arcadia`; canonical numeric files are 0=`a.arcadia` (213 bytes), 1=`b.arcadia` (213 bytes), 2=`main.arcadia` (474 bytes), aggregate 900 original bytes. Include spelling/order in main is b then a; numeric file order is canonical a then b. File 0 has calculation f/id(10), file 1 g/id(11); both declare parameter x/id(1), y/id(12). Parameter x UUID also equals the Project's global id(1). + +Main's ordinary node n points forward to typed property v; property v's subject points forward to ordinary activity w. Its base is exactly `@f(y: 2, x: @g(x: 3, y: 4)) + @f(x: 5, y: 6)`. Typed declarations begin after the Project header, includes and ordinary node. Reuse the existing independently built full Project, including the diagram/node defaults and both calculation bodies; do not replace full equality with an index-only projection. + +Notation: `G(n)` = `Target::Global(id(n))`; `Q(c,p)` = `Target::Parameter { calculation: id(c), parameter: ParameterId(id(p)) }`. Paths are complete **root-to-leaf** `Child` sequences: L=Left, R=Right, A1=Argument(ParameterId(id(1))), A12=Argument(ParameterId(id(12))). `[]` is a real empty expression-root path. `—` means `None`, not `Some(empty)`. All spans are half-open full-original-file byte spans. + +## Entire expected names catalog (G=11) + +| Qualified target | Name | Original declaration anchor | +|---|---|---| +| G(1) | p | main: `project p` | +| G(2) | w | main: `oa_activity w` | +| G(3) | v | main: `oa_property v` | +| G(4) | n | main: `node n` | +| G(5) | d | main: `diagram d` | +| G(10) | f | a: `calculation f` | +| G(11) | g | b: `calculation g` | +| Q(10,1) | x | a: `input x` | +| Q(10,12) | y | a: `input y` | +| Q(11,1) | x | b: `input x` | +| Q(11,12) | y | b: `input y` | + +Global p/w/n/d names are required even though this typed index must contain **no ordinary declaration/reference occurrences** for them. There are 7 global names plus 4 scoped names; every name is one UTF-8 byte. + +## Entire expected expression map (E=15) + +Each row is `(ExpressionKey { owner, field, path }, Location { file, span })`. E labels are explanatory, not persisted arena positions or part of key identity. The order below matches the authored lower-site oracle only for ease of comparison; compare a complete map/set of keys rather than relying on BTree ordering of Path. + +| Row | File | Owner | Field | Path | Span | Exact source slice / unique anchor | +|---|---:|---:|---|---|---|---| +| E0 | 0 | 10 | Body | [] | 202..209 | `$x + $y` | +| E1 | 0 | 10 | Body | [L] | 202..204 | `$x` | +| E2 | 0 | 10 | Body | [R] | 207..209 | `$y` | +| E3 | 1 | 11 | Body | [] | 202..209 | `$x - $y` | +| E4 | 1 | 11 | Body | [L] | 202..204 | `$x` | +| E5 | 1 | 11 | Body | [R] | 207..209 | `$y` | +| E6 | 2 | 3 | Base | [] | 290..334 | `@f(y: 2, x: @g(x: 3, y: 4)) + @f(x: 5, y: 6)` | +| E7 | 2 | 3 | Base | [L] | 290..317 | `@f(y: 2, x: @g(x: 3, y: 4))` | +| E8 | 2 | 3 | Base | [L,A12] | 296..297 | `y: 2`, offset 3, length 1 | +| E9 | 2 | 3 | Base | [L,A1] | 302..316 | `@g(x: 3, y: 4)` | +| E10 | 2 | 3 | Base | [L,A1,A1] | 308..309 | `x: 3`, offset 3, length 1 | +| E11 | 2 | 3 | Base | [L,A1,A12] | 314..315 | `y: 4`, offset 3, length 1 | +| E12 | 2 | 3 | Base | [R] | 320..334 | `@f(x: 5, y: 6)` | +| E13 | 2 | 3 | Base | [R,A1] | 326..327 | `x: 5`, offset 3, length 1 | +| E14 | 2 | 3 | Base | [R,A12] | 332..333 | `y: 6`, offset 3, length 1 | + +There are 3 roots and 12 nonroot sites, hence **P=12 distinct allocated shared Path Steps** under one-Step-per-child construction. Maximum expression depth including root is 4. Same spans 202..209 and parameter token positions in a/b are deliberately different owners/files. + +## Entire expected occurrence map (H+O=21) + +Each row spells every `OccurrenceKey` field: owner, optional field, optional path, token role and qualified target, plus physical file/span. `Decl` means `TokenRole::Declaration`; every other role below means `TokenRole::Use(lower::Role::)`. Input1/Input12 mean `Field::Input(ParameterId(id(1/12)))`. Every token length is 1, with @/$ excluded. + +| Row | File | Owner | Field | Path | Role | Target | Span | Exact spelling / independent source anchor | +|---|---:|---:|---|---|---|---|---|---| +| H0 | 0 | 10 | — | — | Decl | G(10) | 12..13 | f; `calculation f`, +12 | +| H1 | 1 | 11 | — | — | Decl | G(11) | 12..13 | g; `calculation g`, +12 | +| H2 | 2 | 3 | — | — | Decl | G(3) | 206..207 | v; `oa_property v`, +12 | +| O0 | 0 | 10 | Input1 | — | ParameterDeclaration | Q(10,1) | 68..69 | x; `input x`, +6 | +| O1 | 0 | 10 | Input12 | — | ParameterDeclaration | Q(10,12) | 128..129 | y; `input y`, +6 | +| O2 | 0 | 10 | Body | [L] | ParameterUse | Q(10,1) | 203..204 | x; `$x`, +1 | +| O3 | 0 | 10 | Body | [R] | ParameterUse | Q(10,12) | 208..209 | y; `$y`, +1 | +| O4 | 1 | 11 | Input1 | — | ParameterDeclaration | Q(11,1) | 68..69 | x; `input x`, +6 | +| O5 | 1 | 11 | Input12 | — | ParameterDeclaration | Q(11,12) | 128..129 | y; `input y`, +6 | +| O6 | 1 | 11 | Body | [L] | ParameterUse | Q(11,1) | 203..204 | x; `$x`, +1 | +| O7 | 1 | 11 | Body | [R] | ParameterUse | Q(11,12) | 208..209 | y; `$y`, +1 | +| O8 | 2 | 3 | Subject | — | Reference | G(2) | 269..270 | w; `subject @w`, +9 | +| O9 | 2 | 3 | Base | [L] | CallTarget | G(10) | 291..292 | f; `@f(y:`, +1 | +| O10 | 2 | 3 | Base | [L] | ArgumentLabel | Q(10,12) | 293..294 | y; `y: 2`, +0 | +| O11 | 2 | 3 | Base | [L] | ArgumentLabel | Q(10,1) | 299..300 | x; `x: @g`, +0 | +| O12 | 2 | 3 | Base | [L,A1] | CallTarget | G(11) | 303..304 | g; `@g(x:`, +1 | +| O13 | 2 | 3 | Base | [L,A1] | ArgumentLabel | Q(11,1) | 305..306 | x; `x: 3`, +0 | +| O14 | 2 | 3 | Base | [L,A1] | ArgumentLabel | Q(11,12) | 311..312 | y; `y: 4`, +0 | +| O15 | 2 | 3 | Base | [R] | CallTarget | G(10) | 321..322 | f; `@f(x: 5`, +1 | +| O16 | 2 | 3 | Base | [R] | ArgumentLabel | Q(10,1) | 323..324 | x; `x: 5`, +0 | +| O17 | 2 | 3 | Base | [R] | ArgumentLabel | Q(10,12) | 329..330 | y; `y: 6`, +0 | + +Totals: H=3 typed global headers. O=18 lower occurrences: 4 parameter declarations, 4 parameter uses, 1 subject reference, 3 call targets, 6 labels. Header occurrences are produced by indexing and are absent from the retained lower O vector. Repeated f labels have different full paths; nested g labels differ by both path and calculation scope. Ordinary node's `element @v` remains absent from this typed occurrence map. + +For executable expected records, reuse the existing unique-anchor `span(source, context, offset, len)` helper at line 156; assert the numeric values above for the unmodified fixture. Build expected paths from these literal Child arrays, not by following `resolved.expressions.parent` or enumerating actual Index keys. Construct expected keys/targets explicitly from this table. Exhaustive equality must compare all 11 names, all 15 expressions, all 21 occurrences, and the entire independent Project. + +## Exact added retained storage and scratch anchors + +Under accepted contract dimensions and current three-map/shared-Step design: + +| New retained representation | Entry slots | Occurrence slots | Owned text bytes | +|---|---:|---:|---:| +| Qualified names | 11 | 0 | 11 | +| Occurrence map | 21 | 0 | 0 | +| Expression map | 15 | 0 | 0 | +| Unique Path Steps | 12 | 0 | 0 | +| **Added retained total** | **59** | **0** | **11** | + +Thus final Ledger is exactly the incoming accepted-witness Ledger plus `(entries:59, occurrences:0, text:11)` if no additional retained representation is introduced. The baseline retains the raw 15 expression-site entry slots and 18 occurrence slots; neither is released merely because indexed versions now exist. Original source bytes 900 are borrowed and are not added again to text storage. + +Known scratch sizes are owner map 3, physical-token set 21, path lookup Vec 15, call map 3, reverse-name maps 11, syntax-site lookup 15. The original all-roots-first LIFO syntax traversal's peak work Vec has 5 elements (root main pops, right call contributes x/y while a/b roots and left call remain). This 5 is specific to the present traversal order, not a requirement to preserve that order. A final retained delta is fixed; a whole-construction cap must separately account for the chosen scratch lifetime/ordering and for any earlier resolution peak. Do not claim `baseline.entries+59` is a sufficient construction cap while scratch is alive. With all original scratch still alive at syntax-walk peak, scratch is 3+21+15+3+11+15+5=73; explicit earlier releases can reduce this overlap. Any implementation reserving conservative bounds should state the difference instead of weakening exact semantic oracles. + +## Corruption that must reach the call-site check + +Before Index construction, mutate only O11 (main 299..300, owner3, Base, expression site7 / path[L]) from target `Q(10,1)` to `Q(11,1)`. Keep source, tokens, owner, field, expression and span unchanged. + +This is deliberately not an unknown-target or spelling error: both calculations have x/id(1); both qualified target names are `x`. The token is the same full Word inside its expression/owner. The changed key does not collide with nested g's label because that label has path[L,A1], and physical ownership remains unique. ArgumentLabel role accepts a scoped target with a path. The call at path[L] still targets G(10), so `index.rs:438–448` must reject with `Argument label crosses its called calculation.` at span299..300. Assert this diagnostic, not merely any error. Repeating with O10's y Q(10,12)->Q(11,12) is equivalent but unnecessary duplication. + +A distinct runtime omission negative removes O11 while leaving its argument child E9: the qualified-label check (`index.rs:455–469`) must reject `Argument child has no qualified label.` at E9 span302..316. By contrast removing O0 or O8 is an exhaustive-output-oracle failure, not a promised current runtime rejection. + +## Optional direct extensions without changing oracle provenance + +- **SA:** apply the same rich additions to `bundle(Layer::Sa)` and construct model from `expected(Layer::Sa)`. Its system s/id(6) adds one global name. H/O/E/P and all expression/token spans through main's base remain the same because `sa_property` and `oa_property` have equal length and the system/diagram-layer differences occur later. G=12, new retained entries=60, name bytes=12. Assert the system in the complete Project and catalog; it contributes no typed occurrence. +- **Unicode/CRLF:** independently prefix each file with `// 雨\r\n` (8 UTF-8 bytes) and replace each original LF with CRLF. For any original endpoint b in these ASCII files, the transformed byte endpoint is `8 + b + count(original[..b] == LF)`. This raw-byte transformation is independent of parsing/indexing and applies separately to start/end. Or use the same unique literal anchors against transformed sources; Rust string find returns byte offsets. Keep exact original bundle comparison. Do not carry the ASCII numeric offsets unchanged into this variant. diff --git a/harness/analysis-mixed-index-20260924/check_oracle_bytes.py b/harness/analysis-mixed-index-20260924/check_oracle_bytes.py new file mode 100644 index 00000000..f7f2c19f --- /dev/null +++ b/harness/analysis-mixed-index-20260924/check_oracle_bytes.py @@ -0,0 +1,51 @@ +"""Parent raw-byte check of independently authored fixture/index expectations.""" +from pathlib import Path +import json +import re + +def ident(n): + return f'00000000-0000-0000-0000-{n:012}' + +body = (f'node n id "{ident(4)}" {{ diagram @d; element @v; }}\n' + f'oa_property v "Value" id "{ident(3)}" {{ subject @w; type Number; base 1; }}\n' + f'oa_activity w "Work" id "{ident(2)}" {{}}\n' + f'diagram d "Diagram" id "{ident(5)}" {{ }}\n') +main = f'arcadia 8;\nproject p "Project" id "{ident(1)}" {{\n{body}\n}}\n' +main = main.replace('base 1;', 'base @f(y: 2, x: @g(x: 3, y: 4)) + @f(x: 5, y: 6);') +main = main.replace('node n', 'include "b.arcadia"; include "a.arcadia";\nnode n') +files = [] +for alias, name, n, op in [('f', 'F', 10, '+'), ('g', 'G', 11, '-')]: + files.append((f'calculation {alias} "{name}" id "{ident(n)}" {{\n' + f'input x id "{ident(1)}" : Number;\n' + f'input y id "{ident(12)}" : Number;\n' + f'result Number; body $x {op} $y;\n}}\n').encode()) +files.append(main.encode()) +assert list(map(len, files)) == [213, 213, 474] +oracle = Path(__file__).with_name('ORACLE.md').read_text(encoding='utf-8') +rows = [line.split('|')[1:-1] for line in oracle.splitlines() if re.match(r'\| (E|H|O)\d+ \|', line)] +expressions = [r for r in rows if r[0].strip().startswith('E')] +occurrences = [r for r in rows if not r[0].strip().startswith('E')] +expected_expressions = ['$x + $y', '$x', '$y', '$x - $y', '$x', '$y', + '@f(y: 2, x: @g(x: 3, y: 4)) + @f(x: 5, y: 6)', + '@f(y: 2, x: @g(x: 3, y: 4))', '2', '@g(x: 3, y: 4)', '3', '4', + '@f(x: 5, y: 6)', '5', '6'] +expected_occurrences = ['f', 'g', 'v', 'x', 'y', 'x', 'y', 'x', 'y', 'x', 'y', + 'w', 'f', 'y', 'x', 'g', 'x', 'y', 'f', 'x', 'y'] +assert len(expressions) == 15 and len(occurrences) == 21 +checks = [] +for group, expected, span_column in [(expressions, expected_expressions, 5), + (occurrences, expected_occurrences, 7)]: + for row, value in zip(group, expected, strict=True): + name, file = row[0].strip(), int(row[1]) + start, end = map(int, row[span_column].strip().split('..')) + assert files[file][start:end] == value.encode(), (name, files[file][start:end], value) + transformed = b'// \xe9\x9b\xa8\r\n' + files[file].replace(b'\n', b'\r\n') + shifted_start = 8 + start + files[file][:start].count(b'\n') + shifted_end = 8 + end + files[file][:end].count(b'\n') + assert transformed[shifted_start:shifted_end] == value.encode() + checks.append(name) +assert 11 + 21 + 15 + 12 == 59 +print(json.dumps({'result': 'PASS', 'file_bytes': list(map(len, files)), + 'original_span_checks': len(checks), 'unicode_crlf_span_checks': len(checks), + 'retained_entries': 59, 'retained_name_bytes': 11, + 'scope': 'Independent raw fixture/byte arithmetic, no parser, lowerer, index or compiler execution.'})) diff --git a/harness/analysis-mixed-index-contract-20260924.md b/harness/analysis-mixed-index-contract-20260924.md index fcd7575d..063b7756 100644 --- a/harness/analysis-mixed-index-contract-20260924.md +++ b/harness/analysis-mixed-index-contract-20260924.md @@ -157,6 +157,12 @@ implied by a narrow private test. ## Required evaluators before implementation acceptance +The [independently enumerated rich-fixture oracle](analysis-mixed-index-20260924/ORACLE.md) +provides concrete names, keys, spans and retained-storage totals. Parent raw-byte +verification passes all 36 original and 36 transformed Unicode/CRLF spans using +the adjacent `check_oracle_bytes.py`, without running the parser, lowerer, index +or compiler. This validates the test data's byte arithmetic, not implementation. + 1. Hand-enumerated OA and SA mixed bundles: entire name catalog plus every typed declaration/reference/expression key, full-file span and spelling. Include interleaved ordinary declarations, cross-file references in both semantic