diff --git a/harness/DECISIONS.md b/harness/DECISIONS.md index 00cbbe9f..5ea3ee3a 100644 --- a/harness/DECISIONS.md +++ b/harness/DECISIONS.md @@ -549,3 +549,17 @@ domain and completeness-proof ambiguities before adoption. This is a bounded implementation decision, not runtime acceptance or full editing parity. Revisit if a concrete verifier invariant prevents reuse or multiple retained owners become necessary; no generic public indexing framework is justified now. + +## 2026-09-24 — index ordinary authored names before rename composition + +Adopt the [ordinary occurrence successor](analysis-ordinary-index-contract-20260924.md) +after source mapping and independent critique. Retain the accepted typed owner +and add a physical-token-keyed ordinary map from sealed original positions. +This isolates complete coverage/ownership from later edit intent and avoids a +new structural path arena or parser. A fused streaming rename planner could +save map storage, but would combine two unverified boundaries; revisit after +coverage is established and a real consumer demonstrates the benefit. +The contract explicitly handles synthetic brace-containing roots, complete +reference extents and the existing ordinary depth policy. It grants read-only +name-occurrence authority only. Rename composition and complete editing parity +remain separate required capabilities. diff --git a/harness/STATE.md b/harness/STATE.md index 3c70a977..1e26cf89 100644 --- a/harness/STATE.md +++ b/harness/STATE.md @@ -234,6 +234,9 @@ The [source review](analysis-mixed-index-20260924/source-review.md) records thre repaired diagnostic findings; its reviewed production hashes are unchanged. The new owner keeps legacy updater authority separate. Ordinary occurrence indexing, complete mixed editing and public source8 remain required. +The [ordinary occurrence successor contract](analysis-ordinary-index-contract-20260924.md) +uses retained parser positions and physical token keys, with a separate consumed +read-only owner; it is an accepted design, not an implemented capability. The preceding Project [Windows evidence](experiments/analysis-mixed-project-20260924/README.md) distinguishes runtime source **71cadfa0cd0e112caa29588d2494753301f3cb6c** from the formatting-only aaec successor. Strict core all-target Clippy, actual listing diff --git a/harness/analysis-ordinary-index-20260924/critique.md b/harness/analysis-ordinary-index-20260924/critique.md new file mode 100644 index 00000000..3ca7a6ca --- /dev/null +++ b/harness/analysis-ordinary-index-20260924/critique.md @@ -0,0 +1,47 @@ +# Ordinary occurrence successor: independent design critique + +2026-09-24. Source-only review of `ORDINARY-SUCCESSOR-MAP.md` against clean private `93eb6117f95353d25d594d59780018f197c51672` in `.local/as3batch`. No source edits, Cargo, runtime claims or Gitea work. Parent owns adoption. Source paths below are relative to `crates/oa-core/src/`. + +**Recommendation: proceed with the proposed consumed, read-only authored-name owner and physical-token ordinary keys. No fundamental design flaw found.** The map correctly separates this dependency from editing and full graphical/textual/API parity. The next contract should resolve the few boundaries below; none requires a generic indexing framework or structural ordinary path arena. + +## Completeness and provenance + +The retained projection is sufficient for the declared authored-name set. `Ast::resolve` resolves every `Reference` through global aliases and rejects references in unsupported field roles (`language.rs:761–820`); ordinary construction recursively resolves every declaration field, with the explicit Trace baseline exception (`:1684–1693`). `semantic_source.rs:13–31` refuses all `@alias` inside historical semantic payloads, while other baseline fields use ordinary resolution (`:87–100`). Therefore walking every retained `EditKind::Reference` covers admitted ordinary authored alias uses, including ordinary→typed targets; do not introduce a second field-name/schema selector or exclude every field named baseline. The existing active-step accounting selector cannot establish this completeness. + +The typed family already includes typed→ordinary references. Its qualified parameter names must not enter the ordinary reverse-name table: an ordinary `@x` binds a global x even when a calculation parameter is also named x or reuses its UUID. Implicit lexical parenthood has no editable name token. Project/ordinary/typed headers together must account for every global exactly once; scoped parameter declarations remain typed occurrences. Model-reference enumeration is an unsuitable substitute because defaults/implicit parents/historical IDs can have no authored alias token. + +Keep completeness explicitly grounded in sealed projection emission plus independent complete-set tests. Matching token text does not prove an omitted projection record exists. Rechecking an AST/position parallel walk is an optional stronger invariant, not needed to call the immutable emitted set complete under the already accepted trust model. Do not promise arbitrary coordinated corruption rejection. + +## Required containment clarification + +The phrase field/declaration containment needs a precise exception. `EditDeclaration.fields` is a **synthetic Record** whose span is `body.start - 1 .. body.end + 1` (`language.rs:1129–1135`), including braces. It is deliberately not contained in the body interior. A blanket check that every walked EditValue lies inside `EditDeclaration.body` would reject valid declarations. Validate that synthetic root against the declaration span; apply body/field containment to actual field values and Reference leaves. Start traversal at its Record children if that makes the distinction simpler. + +Nested declarations live in the flat ordinary declaration preorder; the parent's `fields` map does not contain child declaration fields (`language.rs:1479–1534`). Pair each retained position with its corresponding declaration occurrence and use that owner, rather than choosing the broadest containing declaration or scanning its entire byte range. Parent and child declaration spans overlap legitimately. This matters for both exact ownership and future movement semantics. + +## Project header and reference token contract + +Project `SourceHeader.span.start` is exactly the `project` token, after version syntax and any trivia (`language.rs:1437–1471`). A binary search of retained significant tokens, equality with whole Word `project`, and the immediately next whole Word alias gives a sufficient narrow projection; require the span inside that same header and compare its target/name to the retained header Project and complete global catalog. Retain canonical entry-file ownership. No new header parser, substring scanner or manufactured numeric source map is justified. + +For each ordinary Reference, require the retained full extent to begin at the whole `@` punctuation token, then require the next significant token to be a Word with the projected alias and global catalog spelling, and require **that Word's end to equal the Reference extent's end**. The parser consumes exactly that pair, including arbitrary intervening trivia (`language.rs:1333–1336`), and the position projection preserves the full consumed extent (`:1068–1084`). Index only the Word span for replacement. This explicit equality prevents an ambiguous oversized extent from being treated as authenticated just because it contains a matching Word. + +The proposed observable tuple includes full Reference extent, but the minimal value sketch lists only owner/role/target. Settle that API detail: retaining the full extent as an inline optional Span is inexpensive and useful for the promised diagnostics/oracle; it adds no logical container slot or owned text. Alternatively keep it construction-only and stop promising it in the returned view. Whole-token validation plus physical duplicate detection is sufficient for overlap safety in rename-only output; do not weaken that requirement to substring spelling checks. + +## Storage, frame policy and simpler alternatives + +The proposed retained delta **1 + D + A entry slots, zero occurrence slots, zero copied text bytes** is sound for one physical-keyed map with inline owner/role/target/spans. Existing typed maps, raw arenas, original syntax and warnings remain charged. Account the reverse global-name table, typed physical lookup and all actual iterative work entries concurrently. A header-coverage set or AST↔position verification table, if added, is extra scratch; it is not free because the source was previously admitted. + +Do not add `Budget::frame(walk_depth)` merely to bound the ordinary traversal. The parser's ordinary value policy is already a separate fixed depth64 (`language.rs:1291–1298`), and semantic declaration nesting uses its own flat/open stack. A new mixed-frame interpretation could reject already accepted ordinary input under a tight typed-frame profile. The proposed iterative reference walk needs only aggregate entry-slot reservations for its live work Vec; it creates no persistent structural path or new recursion. Preserve all usage fields for this successor unless a separately justified depth operation is deliberately specified. + +Physical keys are the simpler correct choice here: the owner is immutable and later emission consumes original spans. They are snapshot identities, not semantic IDs across edits. Adding field-name Strings and persistent ordinary paths would raise memory and cleanup costs without improving rename completeness; retained EditValue trees remain available for later scalar/structural work. + +A plausible smaller alternative is to stream these same sealed ordinary occurrences directly into a later rename planner instead of retaining an N-entry map. That avoids retaining unchanged occurrences while allocating changed edit records. However it fuses discovery, validation and edit intent and makes the complete read-only boundary less independently inspectable. Given the currently unresolved ordinary ownership coverage, the proposed materialized map is justified as a bounded, testable step. Keep the underlying walker private and simple so a future consumer need not duplicate traversal; do not introduce an additional generalized producer/visitor framework now. + +## Falsifiers worth carrying into the next contract + +- Nested parent/child ordinary elements, each with explicit references, including one explicit parent and one implicit parent: no synthetic-root containment failure, no double indexing through the parent's overlapping span, no invented token for the implicit edge. +- `@ /* alias-like @noise */ real_alias` with the same spelling in prose/enums/includes: only the final whole alias Word is indexed, exact full extent remains available, and no prose token appears in the output set. +- Ordinary global x plus scoped parameter x sharing or reusing a UUID: the ordinary reference targets only Global(x); typed `$x` and call labels keep calculation scope. +- A tiny mixed bundle's complete global header set equals the global name-catalog set exactly, while its repeated references remain separate physical records. Include canonical file order opposite include traversal and identical offsets in two files. +- Exact `baseline + N + live scratch` boundary and copied-text delta zero, including late failure/drop; depth64 ordinary values remain under the established policy without borrowing the typed structural-frame metric. + +The map's later rename-only scope is a useful subsequent milestone, provided its complete-target diff gate, one inherited authority/profile, complete edit membership, single original-coordinate emission, actual-output admission and whole-Project equality remain mandatory. It must not claim ordinary scalar edits, typed subtree changes, parameter insertion/deletion/reorder, repaired deletion/reidentity, file movement, missing-ID stabilization, public migration, history/checked-save or usable UI/API parity. Those user requirements remain the completion criteria beyond this dependency; no alternative final product is implied by accepting it. diff --git a/harness/analysis-ordinary-index-20260924/source-map.md b/harness/analysis-ordinary-index-20260924/source-map.md new file mode 100644 index 00000000..c8f5e487 --- /dev/null +++ b/harness/analysis-ordinary-index-20260924/source-map.md @@ -0,0 +1,121 @@ +# Ordinary original-coordinate occurrence successor: source map + +2026-09-24. Local-only bounded research by `/root/mixed_index_source_review` (GPT-6 Astra). Parent owns contract adoption, implementation, integration and publication. No source edits, Cargo, runtime checks, or posts. This report does not reopen the accepted typed index. + +## Recommendation and evidence version + +The next useful dependency is a **private, consumed owner of the complete authored name-occurrence set**: retain accepted `IndexedMixedProject`, add ordinary/project alias headers and every ordinary authored `@alias` leaf, and keep both occurrence families bound to its original complete bundle and inherited ledger. Do this before adapting edit production. It can reuse the retained ordinary position trees without another parser, ordinary decoder, formatter, or new generic indexing framework. + +Then implement a precisely gated **global/scoped rename-only** original-coordinate composition route using this owner and one independently supplied complete target Project. Ordinary scalar changes, insertions, type changes and structural moves are later explicit operation families. This sequencing avoids promising that the existing general ordinary producer already obeys a new mixed one-pass protocol. It is a bounded dependency toward full parity, not an alternative end state. + +All source references below are relative to the repository at `.local/as3batch`, clean branch `codex/analysis-mixed-index-956atf-20260924`, commit **`93eb6117f95353d25d594d59780018f197c51672`**. References are to `crates/oa-core/src/` unless prefixed `tests/`. Governing root documents read: + +- `harness/analysis-mixed-index-contract-20260924.md`, SHA-256 `A344C69068DCE09A9B03BD3002020C87AFC563C7B4CC859522DDA0D4048CD3FB`. +- `harness/analysis-mixed-project-contract-20260923.md`, SHA-256 `D10D3EB32122FDF0E6570962C98871D6907E9FE59DBD6ECCF00DF64082E7382F`, especially retained syntax/site handoff and aggregate accounting. +- `harness/analysis-original-span-review-20260923.md`, SHA-256 `F7200342EA280EF96E80914E336AF0203AEAB530A5255E57D10AA8E763DBE05F`, especially exact diff gates, producer normalization, full-snapshot authority and combined output checks. + +Recommendations below are proposals for parent adoption. Source facts and inherited requirements are identified separately. + +## Source-grounded seam map + +| Existing seam | Verified capability and implication | +| --- | --- | +| `language/analysis_source/mixed.rs:39`, `:49` | `ParsedMixedSyntax` owns the exact bundle borrow, caller limits, canonical `files`, Budget and metadata; each `FileSyntax` retains full-file significant tokens, ordinary `SourceUnit`, typed declarations, `positions: Vec<(Id, EditDeclaration)>`, and lexical-owner indices. No remapping or duplicate source table is needed. | +| `language/analysis_source/mixed.rs:316`, `:332`, `:357` | Ordinary positions are already projected once from that file's retained tokens and original AST after aggregate metadata preflight; the complete result retains those positions. Calling `edit_declarations` again would discard this advantage and charge/work outside the witnessed path. | +| `language.rs:926`, `:943` | `EditDeclaration` has full-file span/body/header-role spans, lexical owner, and a recursive `EditValue` tree. `EditKind` distinguishes Reference, HistoricalUuid, Atom, Record and List. This is enough to enumerate ordinary name tokens and preserve original nesting without reconstructing the generic model schema. | +| `language.rs:1068`, `:1076`, `:1100` | Projected Reference/HistoricalUuid spans start at the marker and end at the second significant token; ordinary declaration header maps already expose the alias token. Stabilized IDs are required. A Reference's entire span is not the replacement span: it can contain `@`, whitespace and comments. | +| `language.rs:895`, `:1437` | `SourceHeader` records the project/header/body/description but no header-role token map. Project alias is the one missing small position projection. Its original header start and complete token stream are already retained. | +| `language.rs:761`, `:782`, `:799` | Admitted ordinary `Ast::Reference` resolves through the global alias catalog; quoted strings in reference fields are refused. Arrays and records recurse through the same resolver. Ordinary live name uses are not guessed from arbitrary strings or UUID-looking text. | +| `language/semantic_source.rs:13`, `:34`, `:87` | Semantic Trace baselines reject `@alias` and require historical `uuid "..."` in typed reference slots. Those UUID leaves are not current alias references. Other baseline fields pass through ordinary resolution; the occurrence walker should follow actual `EditKind`, not blindly exclude every field named baseline. | +| `language/analysis_source/mixed/project.rs:38`, `:43`, `:83`, `:91` | The accepted typed index consumes its resolved witness; the output retains original bundle, canonical files and complete qualified names. A private child module beside this owner can borrow retained fields during construction without exposing replaceable provenance. It need not create a legacy numeric-map Index. | +| `language/analysis_source/mixed/project.rs:405` | `active_references` only meters selected steps/occurrence/contextual-chain arrays. **It is not a completeness selector for alias occurrences.** Scalar parents, view references, route endpoints, artifact owners and trace endpoints also need rename occurrences. Reusing this hook would omit valid ordinary uses. | +| `source_project.rs:93`, `:417` | Public `SourceIndex` catalogs declaration/body/field/include locations and lexical ownership, not all name/reference tokens. Its fields are public. It is neither the missing occurrence index nor a trustworthy mixed constructor input. | +| `source_project.rs:640`, `:647`, `:735` | The public updater recompiles mutable public state, then reparses every original file with `edit_declarations`. It formats old/new declarations and parses those formatted texts to produce a generic syntax diff. This route cannot simply be called on private mixed source8. | +| `source_project.rs:1043`, `:1075` | `SyntaxUpdate::value` already demonstrates correct preservation for Reference→Reference: replace the last significant alias token only. Reuse that behavior, using retained full-file tokens; do not copy its per-fragment `significant()` rescanning into the new index. | +| `source_project.rs:1104`, `:918` | Other syntax-kind changes erase individual significant tokens and insert replacement text at the same original start, retaining trivia. Their insertion/deletion pairs do not obey a blanket no-shared-boundary rule without producer-local normalization. This is why rename-only is a precise first scope. | +| `source_project.rs:539` | Existing `apply_edits` groups every equal `(start,end)` span by concatenating replacement strings, then applies backwards. That permissiveness is not an acceptable cross-producer authority rule for duplicate nonempty replacements. | +| `source_project.rs:868`; `source_project/source_structure.rs:34`, `:61` | Ordinary scalar edits are applied before the structural updater reparses their result. Those structural spans are in later coordinates. Preserve this public behavior until a separate verified integration replaces it; it is outside the original-coordinate one-pass proof. | +| `language/analysis_source/update.rs:78`, `:118`, `:169`, `:227` | Existing typed Plan/Composition cleanly separate complete name/edit membership validation from standalone output length checks, but still require legacy `Index` and caller numeric sources/Budget. Their algorithm is useful; their nominal input authority is deliberately incompatible with mixed input. | +| `source_project.rs:877` | Existing final acceptance recompiles actual emitted bytes and compares the entire resulting Project to intent. The private mixed route needs the equivalent complete parse/resolve/admission/equality boundary, with no public source8 dispatch implied. | + +The retained scanner in `language/analysis_source/lex.rs:18` charges raw tokens, including trivia, and retains significant token spans. Reference indexing can therefore binary-search the original `@` token, require the immediately following significant token to be a Word with exact catalog spelling, and preserve all intervening comments without allocating a new token stream. + +## Proposed next-index contract + +### Scope and ownership + +Consume the accepted typed-index owner into another private nominal wrapper, or use an equivalent internal consumed construction step that retains all its fields. Name its capability precisely: complete **authored name occurrences**, not complete scalar/structural editing authority. No public or sibling-fabricable input accepting a Project, token table, occurrence list, source remap or replacement Budget. Do not clone the existing catalog or typed maps; move/retain them once. Avoid self-references: new records own IDs, roles and spans; temporary reverse catalog maps borrow names only during construction. + +The new ordinary set consists of: + +1. The project alias header, once. +2. Each ordinary declaration alias header, once, including elements, relations, diagrams, nodes/placements, artifacts and traces actually represented by those declarations. +3. Every `EditKind::Reference` in every ordinary declaration field tree, including nested records/lists and repeated equal targets. Resolve each spelling to `Target::Global` through the already admitted complete catalog, which includes typed globals. Ordinary references never resolve through parameter-only name entries. + +It excludes display names, descriptions, comments, string literals, enum words, include path strings, declaration identity strings and `HistoricalUuid` leaves. Implicit lexical parenthood is a semantic relation with **no alias token**; do not manufacture a reference for it. Explicit `parent @name` is an ordinary reference and must be indexed. Current mixed syntax permits nested ordinary declarations but explicitly refuses typed headers nested in ordinary bodies (`mixed/tests.rs:448`); do not widen that grammar as part of indexing. + +### Minimal construction/verification + +- Iterate canonical files and their retained ordinary positions. Check the declaration position/ID/header against the corresponding sealed ordinary declaration rather than resolving owner from a coincident offset or the outermost enclosing span. Nested parent declaration spans overlap child spans, so broad range scanning is not a valid ownership rule. +- Use the existing ordinary header alias span. Project header needs only a narrow retained-token projection anchored at `SourceHeader.span.start`: validate `project`, the following whole alias token and its equality to the retained Project alias. There is no need to call `edit_header(source_slice)`, which lexes again (`language.rs:951`), or build a second complete header grammar. A shared borrowed token-role helper is reasonable only if it remains small and preserves ordinary parser behavior. +- Walk `EditValue` Record/List iteratively, reserving each live walk slot. An encountered Reference must have valid UTF-8 full-file range, marker `@`, precisely the expected following whole Word alias token and canonical target spelling. Store/edit the alias token span alone. Track the owning declaration and retain its original file. Require field/declaration containment where justified; do not confuse overlapping declaration bodies with reference ownership. +- Reject duplicate physical token ownership across the combined typed and ordinary sets, including ordinary headers. Count the project header and every global declaration exactly once across the two producer families. A reverse global-name map excludes scoped parameters and borrows the existing names. +- Continue using explicit safe fallback diagnostics for an unowned/invalid numeric file or malformed span. No panic-prone slicing during error transport. Failure consumes/drops the whole attempted owner and exposes no usable partial index. + +Completeness is obtained by exhaustive traversal of the sealed retained parser projection and independently authored output oracles. The existing parser/projection is the trusted emission source. A new reference map does not, by itself, prove arbitrary damage that consistently removes the same reference from retained AST and position tree. If runtime detection of arbitrary projection omissions is required, specify a separate AST↔position walk and account for it; do not silently claim that a token-spelling check proves it. + +### Key representation: consequential but bounded choice + +| Candidate | Tradeoff against requirements | +| --- | --- | +| One ordinary map keyed by canonical `(file, alias-token start, alias-token end)`, values contain declaration owner, Header/Reference role and global target | Recommended for this snapshot-bound rename dependency. Repeated references remain distinct; deterministic physical ordering is immediate; keys need no copied field strings or new path tree. Identity is explicitly ephemeral to this exact immutable snapshot, never a persistent model/command identity. Retained EditValue trees remain the richer structural source for later operations. | +| Owned structural ordinary keys with record-field/list-index path, comparable to typed paths | Better for structural diagnostics and future field editing, but adds a new edge type, persistent-path storage/drop policy, field-name ownership and duplicated tree metadata before a consumer requires it. Reconsider when a concrete scalar/subtree editing contract needs stable structural keys; list indices still do not supply semantic row identity across edits. | +| Reuse generic `SyntaxUpdate` as the ordinary index or run public `update_bundle` | Produces broad changes but reparses/formats, lacks retained occurrence authority and includes later-coordinate structural processing. Rejected as the next private original-coordinate seam. | +| Scan all original Word tokens for catalog names | Superficially small but changes prose/enums/include/header roles incorrectly, cannot establish owner and cannot distinguish historical semantics. Reject. | + +Do not force ordinary roles/keys into typed `Field`/`Child` enums. Two sealed occurrence families can share checked source/token lookup and physical-edit validation without a public generic indexing framework. Any helper extraction must leave the accepted typed semantics intact. + +### Aggregate storage proposal + +Continue the retained mixed Ledger/Budget; accepted typed maps and raw arenas remain live. If the recommended one-map representation has D ordinary declarations and A ordinary Reference leaves, its additional retained entry count is **`N = 1 + D + A`**, zero new lowerer-occurrence slots and zero new owned-name bytes. This formula assumes inline owner/role/target/span records and no copied structural path. Any extra retained map/vector must be added explicitly. + +Scratch must include the borrowed reverse global-name map Gg, an efficient cross-producer physical-token lookup for T existing typed locations if used, live iterative walk slots W, and any owner/header/AST↔position validation table actually allocated. A design can use the new map itself for ordinary duplicate-token detection and a T-slot scratch set for typed collision lookup; there is no need to retain a second ordinary physical set. Then a conservative simple peak is baseline plus `N + Gg + T + W`; actual prefix/phase peaks must be hand-counted for cap tests. This is a proposal, not a measured cap. + +Reserve before every growth/copy; release scratch only after its allocation is dropped. Borrowed names require map entry slots but no copied text charge. New index records belong to entry slots under the existing contract; do not recharge the old `Budget.reference` parser/active-step counters or lowerer occurrence domain. Do not call `Project::identities()` merely to build another name Vec. If structural keys are chosen instead, inventory their owned field strings and unique path nodes before accepting the alternative. + +The ordinary tree was already admitted under its existing depth policy; an iterative reference-only walk need not introduce a new frame interpretation. Preserve inherited frame limits, never widen them, and do not confuse live work-slot reservations with `Usage.deepest_frame`. If an explicit depth check is added, specify whether it uses original ordinary depth or a structural path depth; `Budget::frame` may update the high-water mark, but index scratch does not change parsed byte/token/entry/reference counts. The existing baseline, raw typed arenas and typed persistent Path reservations must survive unchanged throughout this successor. + +## Subsequent rename composition seam + +This section establishes dependencies, not permission to broaden the next index patch. + +1. **Single intent:** derive global aliases and calculation-scoped parameter names by stable identity from one complete target Project and the complete source owner. Validate final global/scoped namespaces simultaneously, permitting swaps. Do not accept independent ordinary and typed final-name maps that can disagree. +2. **Exact diff gate:** for the first rename route require equality of every old/new Project part except the expressly allowed aliases/parameter names. Global IDs/categories, parameter IDs/order/types, all expressions/bindings/subjects, arrays/order, geometry, names/descriptions, baselines, ownership and all other fields remain equal. Do not silently ignore a scalar change because no occurrence changed. An explicit gate is narrower and clearer than the phrase topology-preserving. The comparison strategy should be chosen before implementation; copying/masking a whole model has allocation implications, while borrowed structural comparison needs complete field coverage. +3. **One owner/profile:** keep the complete entry, canonical paths, file set, bytes, caller limits, complete old model and index authority bound together. A batch from a different owner cannot be installed merely because touched token bytes or numeric keys match. Bind producer objects to this owner rather than weakening legacy `Index` typing. The existing borrowed `OriginalEdit` is only an inspection record (`update.rs:292`), not transferable authority. +4. **Small shared algorithms:** reuse/extract private final-name validation, token-membership and edit-order/completeness logic from `update.rs:35`, `:79`, `:117` behind authenticated wrappers when implementing composition. Keep legacy `Plan` behavior and its standalone output preflight. The mixed route must use aggregate live plan storage against its retained ledger, not simply inherit old per-map checks or build a temporary numeric-source `BTreeMap` copy. +5. **Original edits only:** both families produce alias-token replacements in original file coordinates. This rename-only set has nonempty whole-token ranges and needs no zero-width insertions. Reject duplicates/overlaps across producers; every changed target's entire authenticated occurrence set must be present, including ordinary→typed and typed→ordinary references. Preserve all other bytes exactly. +6. **Combined preflight/output:** count retained plan records, owned replacement names and scratch before allocation; then compute complete combined per-file and aggregate output lengths, including unchanged files, before constructing output. A shrink may compensate output growth, but cannot erase live source/index/plan storage or prior parse/numeric costs. Emit each file once from original bytes in ascending span order. No `Plan::apply` typed-first intermediate followed by ordinary offsets. +7. **Actual result acceptance:** parse/resolve/admit the emitted complete bundle under the retained policy and compare the entire Project to the independent target. Return/install nothing on failure. New output gets its own bounded parse pass; do not reset original usage to hide redundant work. Public source8, checked-save/history and GUI activation are later gates. + +When adding ordinary scalar operations, define exactly allowed leaf/header forms and a reviewed same-producer normalization for type changes and ordered insertion groups. `SyntaxUpdate` has comment-preserving behavior worth retaining, but its current compound edits require that extra contract. Never relax duplicate/overlap rules globally to accommodate one producer. When adding structural movement/deletion, redesign composition of its ownership pieces in original coordinates or provide a separately verified transformation; the current reparse-after-scalar phase is not evidence for one-pass original-coordinate correctness. + +## Required independent evaluators and concrete risks + +For the proposed **read-only ordinary occurrence index**: + +- Hand-enumerate every ordinary/project header and every Reference tuple `(canonical file, owner, role, global target, full reference extent, alias-token span, spelling)` in tiny OA and SA mixed bundles. Include interleaved typed roots, nested ordinary declarations, explicit and implicit parents, reversed include traversal versus canonical file order, coincident offsets across files, Unicode/CRLF and same-spelling comments/prose. Compare complete sets, not selected counts. Expectations cannot come from builder, formatter or lowerer. +- Extend the current independent rich fixture (`mixed/project/tests.rs:253`): its `node n` references ordinary diagram d and typed property v, while typed property v refers to ordinary work w. Enumerate ordinary project/node/work/diagram headers and references in addition to the already accepted typed maps. Shared parameter UUID/global identity remains distinct and cannot enter the ordinary reverse alias map. +- Add category/shape coverage beyond that small fixture: relation endpoints and nested contextual chains, diagram routes/view options, artifact owner, live trace endpoints, nested records/list repeats, and semantic historical UUIDs that coincide with a renamed live target. The latter must remain absent from alias occurrences. Include paths and display/prose alias text stay untouched. +- Prove `@ /* comment */ alias` indexes only the alias token and preserves marker/trivia; test wrong marker, non-Word token, partial alias range, wrong name/global target, duplicate physical token, owner/file mismatch and UTF-8-split/out-of-range locations reaching intended checks. Distinguish per-record corruption checks from sealed-emission completeness assumptions. +- Hand-count retained N and peak scratch with accepted typed/maps/syntax baseline still present. Exact cap and cap-minus-one must isolate index growth, not an earlier parse. Parse bytes/tokens/entries/references/Exact work remain unchanged; any frame metric change must have an explicit permitted reason. Success and representative late failure/drop run on the existing small-stack harness. +- Preserve exact bundle pointer/bytes/entry/file keys, entire Project, warnings and static admission metadata; no mutable authority or legacy updater conversion escapes. Run only affected existing suites after implementation; retain explicit inherited failure dispositions. + +For **later rename composition**, separately require exact authored whole-output oracles: simultaneous global/scoped swaps; repeated/reordered typed calls; both reference directions and untouched unrelated files; project/ordinary/typed header renames; same-name no-op; comments/prose/historical UUID unchanged; mixed growing and shrinking replacements with exact combined output cap; stale entry/path/unrelated file; cross-owner batch pairing; missing/duplicate edits and forged physical positions; target containing any unsupported change; and atomic late failure. A parseable/admissible damaged result must fail whole-Project equality, so earlier parse errors do not stand in for the equality gate. + +Existing public regression anchors (same commit, under `crates/oa-core/tests/`) are `source_project.rs:496` cross-file alias uses, `:843` exact CRLF/scalar/header preservation, `:877` comments after `@`, `:926` kind changes, `:999` scalar/record conversion, `:1030` reference reorder comments; `source_identity_updates.rs:37` origin ownership and `:81` identity swaps; `source_nested_updates.rs:49` nested token preservation and `:244` nested identity/alias swap plus cross-file movement. These demonstrate existing behavior to preserve, not proof that the new private route already supports those broad edits. + +## Remaining choices and completion boundary + +Parent should settle ordinary key identity and the exact read-only output surface before implementation. The recommended physical-token key is enough for rename completeness and does not claim semantic identity across edits. Decide whether runtime AST↔position omission verification adds justified value beyond sealed origin plus independent oracles. Either choice must state its guarantee and charge its scratch; it must not be inferred from a misleading test name. + +Once this index is accepted, the highest-value next milestone is the precisely gated complete-target rename compositor with actual emitted-bundle admission/equality. Scalar editing, typed subtrees, parameter insertion/deletion/reorder, repaired deletion, reidentity, file ownership/movement, missing-ID stabilization/reindexing, migration, history/checked-save and GUI/API parity remain required. Neither the index nor a private rename milestone completes source8 or the user-owned project specification. diff --git a/harness/analysis-ordinary-index-contract-20260924.md b/harness/analysis-ordinary-index-contract-20260924.md new file mode 100644 index 00000000..f9105136 --- /dev/null +++ b/harness/analysis-ordinary-index-contract-20260924.md @@ -0,0 +1,147 @@ +# Complete authored-name occurrences: private successor contract + +Accepted bounded design, 2026-09-24, based on private source +`93eb6117f95353d25d594d59780018f197c51672`, after parent source inspection, +[seam research](analysis-ordinary-index-20260924/source-map.md) and +[independent critique](analysis-ordinary-index-20260924/critique.md). +Issue 7 owns work status. This does not alter prerelease source `715036e`. + +The user requires complete graphical/text/API parity. This dependency adds +ordinary name occurrences to the accepted typed index, followed by a separately +specified rename compositor. It does not redefine parity as renaming, activate +public source8 or claim scalar/structural editing support. + +## Chosen capability and alternatives + +Consume `IndexedMixedProject` into an opaque private owner that retains it and +one ordinary occurrence map. The owner provides read-only views of original +bundle/files, complete Project, warnings, existing typed maps and new ordinary +records. No mutable replacement state, legacy Index/updater conversion, public +factory from loose parts or independently transferable edit authority. + +Use physical keys `(canonical file, alias-token start, alias-token end)` for the +new map. Each inline value contains owning declaration ID, global target ID, +Header/Reference role and the original full occurrence extent. For headers the +extent is the alias token; for references it includes `@` and intervening trivia. +These are snapshot keys, not persistent semantic identities. Preserve the richer +retained position trees for subsequent structural work. Do not force ordinary +keys into typed expression `Field`/`Child` types. + +This materialized map makes complete coverage inspectable before edit authority +is added. A fused traversal/rename planner could avoid retained map slots, but +would combine still-uncertain ownership coverage with intent/diff/output logic. +Revisit that tradeoff only after evidence demonstrates a useful simplification. +Owned structural key paths would duplicate retained metadata without a current +consumer and introduce extra string/path/drop accounting; defer them. Whole-word +source scans and public `SyntaxUpdate` are unsuitable: the former cannot identify +reference roles; the latter reparses, formats and later applies structural edits +in transformed coordinates. + +## Complete scope + +Add exactly one project alias header, each ordinary declaration alias header, +and every actual `EditKind::Reference` leaf in every ordinary declaration field +tree. Include scalar and nested/list references, repeated targets, routes/view +options, artifact/trace endpoints and references from ordinary syntax into typed +globals. Resolve only through `Target::Global` names in the existing complete +catalog; scoped calculation parameters never enter this reverse lookup. + +Exclude comments, descriptions, display names, string literals, enum words, +include paths, identity strings and `HistoricalUuid` leaves. Do not exclude a +whole field by name merely because some fields use historical references. +Implicit lexical parenting has no authored alias token; explicit `parent @x` +does. Preserve the existing restriction against typed declarations nested in +ordinary bodies. Do not use `active_references`: it only meters selected ordered +model-step references and is not an alias occurrence selector. + +Typed occurrences stay unchanged. Across both families, every global declaration +header appears once and no physical alias token has two owners. Parameters keep +their existing calculation-scoped identity even when UUIDs coincide with globals. + +## Construction and verification + +Reuse sealed `FileSyntax.positions`, ordinary AST and original significant +tokens. No reparse, fragment lexer, source mask, model copy or formatter. + +1. Iterate canonical files and their ordinary declaration/position pairs. Check + matching IDs, owner and header containment against that declaration occurrence. + Nested declaration spans overlap: their parent range cannot determine ownership. +2. Project header lacks a role map. Project only its alias from retained tokens + anchored at `SourceHeader.span.start`: require `project`, the following complete + Word token, containment in that header and exact admitted project spelling. + Ordinary header alias spans already exist; verify complete Word identity and + the corresponding global declaration before insertion. +3. Walk Record/List values iteratively. The synthetic root `fields` span includes + braces (`body.start-1..body.end+1`); do not require it inside the body interior. + Actual field/value/reference spans must remain within their justified parent + and owning declaration. Nested declarations are separate flat occurrences, + not children discovered through broad span scans. +4. For a Reference, require a valid original UTF-8 extent and a significant `@` + token at its start. The next significant token must be a whole Word whose end + equals the retained Reference end, spelling equals the retained reference, + and global catalog target agrees. Store only that Word as the alias location, + retaining the full extent inline. Comments/whitespace between marker and Word + remain outside replacement authority. +5. Reject duplicate physical alias tokens across ordinary and typed families, + repeated declaration headers, wrong owners/files/targets, partial or malformed + tokens, and absent global headers. Full-token validation makes separate + overlapping partial-token ownership invalid before insertion. +6. Failure drops the consumed attempt with no partial owner. Preserve known + canonical file attribution. Invalid file keys or invalid UTF-8 spans use the + explicit safe diagnostic fallback already established by the typed constructor. + +Completeness relies on exhaustive traversal of the sealed parser projection, +checked against independent full-set oracles. Do not add another AST/position +emitter merely to claim runtime detection of arbitrary coordinated corruption. +Per-record validation does not prove arbitrary omissions from trusted emission; +name this boundary accurately in tests and documentation. + +## Storage and depth + +Retain the entire inherited ledger, typed maps/paths, raw arenas, syntax and +warnings. With D ordinary declarations and A reference leaves, the one-map design +adds exactly **N = 1 + D + A entry slots**, zero lowerer occurrence slots and zero +owned-name bytes. IDs, roles and two spans are inline. Added retained containers +or copied strings would require a revised explicit accounting decision. + +Count the borrowed reverse global map G, typed physical-location lookup T and +all live iterative work slots W as scratch entries. A header-seen flag can live +in each reverse-map value; avoid an uncounted extra set. New-map membership +handles ordinary duplicates. Conservative coexistence is inherited baseline plus +`N + G + T + W`; tests must hand-count actual phase peaks, not assume this upper +bound is always simultaneously reached. Reserve before growth, and release only +after the corresponding allocation is dropped. + +Do not recharge parsed bytes/tokens/entries/references, Exact work, raw lowerer +occurrences or model-step counters. The ordinary tree already satisfies its +depth-64 parser policy. This iterative reference walk introduces no new +`Budget::frame` interpretation or raised limit; parsed Usage remains unchanged. +Workspace stack/lifecycle guarantees remain bounded by actual tested cases. + +## Acceptance before subsequent edits + +Independently enumerate complete header/reference records for small OA and SA +mixed bundles, including both reference directions, nested ordinary declarations, +explicit/implicit parents, noncanonical include traversal, coincident cross-file +offsets, Unicode/CRLF, marker comments and alias-like prose. Extend the accepted +rich fixture and separately cover ordinary categories absent from it: relation +endpoints, contextual chains, routes/view options, artifacts and traces, including +historical UUIDs equal to live identities. Expected maps must not derive from the +index, formatter or lowerer. + +Require targeted corruptions to reach the intended marker/token/target/owner/file, +physical uniqueness and safe diagnostic checks. Hand-count retained and peak +storage with typed baseline still live; isolate exact-cap/cap-minus-one failures +from earlier parsing. Preserve complete Project, bundle identity/bytes/entry, +canonical file keys, warnings, static admission metadata and parse/numeric usage. +Exercise success and late failure on the existing bounded-stack harness. Retain +all prior typed-index outcomes and explicitly report inherited full-suite/format +failures. No public API or source8 activation is part of this change. + +The next separate contract must gate a complete independent target Project to +allowed global/scoped renames only, validate final namespaces simultaneously, +produce both families' complete nonoverlapping edits in original coordinates, +preflight combined storage/output including untouched files, emit each file once, +then parse/resolve/admit actual output and compare the whole Project to intent. +Scalar changes, structural moves, insertion/removal, reidentity, stabilization, +migration, history/checked save and UI/API parity remain required later work.