ARCADIA-Studio/harness/experiments/text-diagnostics-20260913/reviews/diagnostic-span-research.md

Diagnostic span research: source before #12 implementation

Read-only static inspection by diagnostic_input, 2026-09-13. Parent supplied branch codex/text-diagnostics-9e8r6t, base b4b860f (tested code 2df7); this review did not run Git, builds, tests, UI, or network. Source root is C:/ar-text-9e8r6t; references below are relative to it. Hash inventory records the actual inspected bytes before the separately authorized provenance repair. Existing test assertions are source evidence, not new execution evidence. Scope is diagnostic production, range semantics and publication authority; renderer geometry is not established here.

Verified producer contract

• oa-core/src/language.rs:9–21,105–110 (under crates/ throughout): Diagnostic severity/code/message plus optional model ID/field, one-based line and Unicode-scalar column; Span is a half-open UTF-8 byte interval into the exact analyzed source. Byte offsets are authoritative for geometry. Scalar columns are neither byte columns, grapheme counts, nor visual columns. Tabs, combining marks and wide characters prohibit multiplying column by a fixed width.
• validation.rs:7–11: only Error and Warning exist. Related locations do not create a third severity. source_project.rs:74–83: primary path + Diagnostic; related is only Vec<SourceLocation { path, span }>, with no independent related message, code, severity or line/column.
• language.rs:214–300,989–1017,1264–1276: the real tokenizer advances complete Unicode scalars or ASCII runs; strict parsing reports token start, and appends a sentinel at len..len. diagnostic at324–342 clamps the offset to byte length, counts LF for lines and scalars after the last LF for column, and covers the next scalar if one exists. Consequently an unknown @missing resolution diagnostic points at @, not the entire alias. Invalid/unterminated quoted strings point at their opening quote, not the entire tolerant error token. EOF is a genuine empty range. The offset clamp does not repair an interior UTF-8 offset: producers rely on token/AST boundaries.
• language.rs:526–565: single-file language lint maps model issues to one next scalar at field/declaration offset. Do not assume its span size describes the App's bundle lint.
• source_project.rs:603–657: bundle model diagnostics use the full indexed field VALUE span if a matching field exists, else the full declaration, else project header. language.rs:1338–1366,1525–1559 makes field ranges from AST values and completes declaration ranges at the closing brace. Nested declarations and records/lists can produce large multiline ranges. No implicit token narrowing is encoded in Diagnostic. unallocated_leaf warning (validation.rs:340–350) falls back to a declaration when there is no authored allocation field.
• source_project.rs:454–474: duplicate alias or UUID emits duplicate_declaration on the later full declaration and adds the earlier declaration as related; project identity collision may point back to the project header. This is the only related.push production site found in core. Declaration ordering follows include traversal, not a presentation sort. Other located/parser wrappers examined produce empty related lists.
• source_project.rs:165–200: global/path/budget diagnostics often explicitly use Span::default() and remain 0..0; located failures otherwise preserve supplied include/declaration extents. Missing includes/cycles are located at the including directive, not at a nonexistent target file. Include span covers include through ; (language.rs:1565–1578). source_files.rs:216–238,398–408,455–489,504–510,534–545 similarly produces raw include locations and a zero-width entry pending_save location. Its end clamp does not check UTF-8 boundaries; final consumers should validate rather than use clipping to repair malformed public data.
• source_project.rs:12–13,132–164: LogicalPath is a portable slash-separated key, validated at bundle boundaries; includes normalize bounded ./.. components and reject unsafe paths. Match loaded file identity exactly, never by basename, OS case folding or a span belonging to another file. A failure can name a path not loaded; it belongs in Problems but cannot be underlined in an unrelated active buffer.
• source_project.rs:273–300: lint_bundle diagnoses exact supplied draft bytes with compile_once. compile_bundle can insert UUIDs, then recompile/reindex the changed returned bundle and perform schema upgrade. Compiled diagnostics are tied to compiled.bundle, not the caller's original input. Migration/structure helpers wrap existing diagnostics (source_migration.rs:31,45, source_structure.rs:350–355); these are not additional range conventions.
• CRLF: byte ranges retain both bytes. Line labels count LF only. No producer normalizes offsets to LF-only input. A standalone CR is treated as tokenizer whitespace but does not increment the diagnostic line count. This is not proof the editor uses identical paragraph splitting: a renderer adapter must demonstrate its mapping, including mixed line endings, instead of deriving positions from labels.

Current App lifecycle

app/source_symbols.rs:48–52 defines current source key from workspace session UUID and workspace revision. source_workspace.rs:292–300 switches active file without changing that snapshot: active path and current editor instance/layout are additional hover authority. Detached buffers remain available while source_workspace.rs:396–443 sends only reachable files to bundle lint; absence of a marker is not evidence that a detached file was validated.

app.rs:665–681 centralizes publication: every replacement/clear allocates a new generation and stamps current source key through text_workspace.rs:247–254. A navigation-session reset only republishes retained warnings if their pre-reset publication was current; presentation reset otherwise removes publication. clear_source_presentation retains worker ownership while clearing UI/query authority. Accepted source/model changes use sync_source_with_alias_editor (689–697) to install accepted compiled bytes before publishing their diagnostics.

Text edits at2982ff advance source version/workspace revision and immediately publish empty before debounced lint. Undo/Redo similarly clear before lint. Linted (3129–3142) releases only its matching running job, installs only the current source version, and starts pending work. start_pending_lint (4030–4052) analyzes a cloned workspace bundle; task result itself carries source version, not SnapshotKey. Session/source mutations must therefore retain their existing version invalidations; overlay checks are additional defenses, not substitutes. Pure caret movement/scroll should not invalidate diagnostic content.

Open success (3834–3865) constructs a Document, installs its compiled bundle into Workspace, then publishes compiled diagnostics. Repair Open (3866–3884) installs a new raw Workspace and publishes repair diagnostics; last accepted model may remain unchanged. IncludesLoaded (3104–3118) merges newly discovered files without overwriting existing draft buffers, publishes disk diagnostics, then requests raw lint. Recovery filters pending_save and republishes (3054–3072). Saved/destination changes use the centralized session reset. These paths require source provenance, not merely a fresh generation allocated when arbitrary diagnostics arrive.

Existing Problems navigation admissions (text_workspace.rs:95–119,379ff,568–585,613–619) require matching publication, source key, Problems visibility, drawer visit, loaded target and valid UTF-8 bounds. These are useful range checks. Problems drawer visit is inappropriate as the lifetime of editor underlines, which must also work with that drawer hidden. Editor hover is read-only and must not call source_interaction, reconstruct Content, steal focus/selection, or create navigation history.

Concrete provenance finding before implementation

source_files.rs:528–531 stores compile_bundle(...).diagnostics into DiskProject while its bundle retains original bytes. Missing UUIDs can shift these warning ranges. source_workspace.rs:363–381 preserves existing buffers on discovery; app.rs:3112–3115 then stamps disk diagnostics as current, prior to asynchronous raw lint. A shifted range may still be valid/in bounds, so generation + bounds checks alone do not detect this. This is a concrete static wrong-publication path, not newly reproduced runtime evidence.

The existing core test advisory_lint_locations_refer_to_raw_drafts_before_identity_stabilization (tests/source_project.rs:207–237) demonstrates that two no-ID functions have distinct raw versus compiled warning offsets. Use its actual fixture through DiskProject discovery and the App IncludesLoaded message. Also test a reply containing bytes different from a retained buffer: normal load_includes passes workspace overlays and checks source version, but an admission test must establish what metadata/byte equality the reply guarantees rather than relying on an arbitrary minted current generation. Parent authorized a separate bounded producer/admission repair after this report; these findings/hashes intentionally remain pre-fix evidence.

Proposed range and hover invariants (decisions pending parent synthesis)

1. Admit a marker only from a diagnostic publication proven to describe the current workspace snapshot, for the exact active loaded path; require start <= end <= source.len() and both UTF-8 boundaries. Reject malformed ranges rather than clamp, wrap, search for similar text, borrow another file's line/column or silently repair bytes. Preserve the raw diagnostic for Problems even if a visual range is inadmissible.
2. Retain exact primary range semantics. Split it into visible visual-line fragments using actual editor text shaping/layout, not a scalar column approximation. Clip drawing/hit tests to the editor's content viewport, including horizontal scrolling, wrapping and DPI. A half-open end at the next line's column0 must not underline that next line's text. Newline bytes are not visible glyphs. Empty ranges, EOF and empty lines need a small anchored visible/hittable marker; do not underline an unrelated previous character to manufacture width. The marker is an affordance, not a modified source span.
3. Primary spans receive their owning Error/Warning styling. Related spans may optionally expose the owning message as explicitly related context; they must not imply an independent validation failure on the earlier valid declaration. Smallest clear policy is primary underlines plus related context in hover/Problems until separate related marking semantics are accepted. This is a product decision, not a producer constraint.
4. Overlapping primary ranges should expose every applicable diagnostic deterministically (severity then stable publication ordinal is sufficient), with explicit count if presentation is bounded. A nested broad declaration warning must not swallow a narrow error's hit target. End offsets are excluded except for explicitly empty anchor hit areas. Hover text should preserve message/code/severity and source context; no model mutation or automatic reveal.
5. Cache geometry/hover against publication generation + snapshot + active path/editor identity + layout/viewport revision. Clear/recompute on publication replacement/clear, source edit, file/session replacement, leave, scrolling/reflow or an occluding popup/modal. A delayed tooltip timer/message must revalidate its captured authority and pointer hit; A→B→A must not revive an old hover. Local passive hover should not advance the global interaction clock on every mouse move or cancel a valid pending navigation merely by appearing.
6. Stable current publication/layout should preserve editing, selection, focus and scroll. Underline drawing must not replace the source editor widget or its Content. Any optional click-to-navigate is a separate command needing existing navigation/modal admissions; passive visibility cannot authorize activation.

Discriminating fixtures and required evidence

Fixture / primary source basis	Required check (not run here)
Existing no-ID two-function raw lint test at core tests/source_project.rs207	Raw warning ranges slice exact declarations; disk/App publication does not borrow stabilized offsets. Add retained-buffer mismatch and delayed obsolete IncludesLoaded controls.
Existing child Unicode unknown parent at241; cyclic-parent variant262	Byte start at @missing, scalar label after Cyrillic comment/CJK title; unknown reference is one-scalar @, validated cyclic-parent issue may cover full field value. Check exact start AND end, not just line.
Existing duplicate-alias/UUID/project-ID test280	Primary in b.arcadia, related in a.arcadia/project header; both exact full spans, no marker in the wrong active file; hide/reopen Problems unrelated to marker visibility.
Existing missing/invalid include disk test tests/source_files.rs59; includes test source_project.rs310	Underline including directive through semicolon, keep missing file message, no phantom target buffer. Use unique temporary root for actual disk producer.
Real parser: arcadia 5; project p "P" { and empty input	Error sentinel at len..len; empty source0..0; visible zero-width anchor/hit target without slicing past EOF.
Real lexer: valid prefix followed by unexpected non-ASCII scalar, unterminated quoted string	Full UTF-8 scalar range for invalid scalar; opening quote only for unterminated string. Add combining sequence and tabs before error to falsify fixed-width column math.
Real unallocated sa_function with body/newlines and no allocation; cyclic parent field	Multiline declaration and field-value ranges; CRLF, mixed endings, trailing newline and end-at-next-line0; drawing excludes unrelated line. Parent must confirm exact real diagnostic outputs during tests.
Real pending-save journal discovery / low ProjectLimits	Genuine zero-width entry/global markers, severity Error, remains current; do not label as malformed just because start==end.
Synthetic malformed public SourceDiagnostic alongside real valid diagnostic	Reversed/out-of-bounds/interior-UTF8 ranges omitted safely, valid neighbor remains; explicitly synthetic robustness test, not invented producer behavior.
Synthetic overlapping ranges plus real broad warning	All hover messages reachable and stable; narrow range not swallowed; tests distinguish primary from related context.
App publication replacement/edit, file A→B→A, reset, delayed lint, popup/rename/busy state	Old hover hidden/inert; no model/source/history/reveal/focus/selection/scroll changes; current stationary-pointer hit recomputed with current layout.

Producer tests establish source-byte semantics; mounted widget tests must exercise actual shaping, event cursor and clipping, and actual message/operation queues without tree-index guesses. Native acceptance must inspect long/multiline diagnostics, Unicode/tab geometry, scroll/wrap/resizing, EOF/empty buffer, overlapping hover, typing/selection/Find, and occlusion in realistic Text workflows. Existing #13 evidence may be reused for unchanged components but does not establish new underline or tooltip behavior.

Source-before-fix SHA-256 inventory

• crates/oa-core/src/language.rs (105312 bytes): 34cbb144a111e6f0ddadfca7d40636a8e3323fe65ea8920fd2402f0352b41f57
• crates/oa-core/src/validation.rs (112792 bytes): 7c1b4ae5e4ebba13e99af9158ccde138dfc5e9a7afb580e76fba72d0b64ba74e
• crates/oa-core/src/source_project.rs (61843 bytes): 1394370bcc5908873b830c2c11312e6e1611af41f46c4f43ca0a6bcb713bb466
• crates/oa-core/src/source_files.rs (74803 bytes): 669491dbddb8e8e6556161426dffa1a49ac1c826d31020bff0956d81a4d84683
• crates/oa-core/tests/source_project.rs (45910 bytes): cef58adb16aa4d2527ddc0628aec3e26f0b42b4be70014a7e360fafdac13a942
• crates/oa-core/tests/source_files.rs (13168 bytes): f0429a0b89a443458ea9af7083c1e4e7411417a7e6b47e618bfbb9266dc164f9
• crates/oa-desktop/src/source_workspace.rs (24090 bytes): 388c71f63829c0c76526a1700ebf5f240c86c467a65aae4fca4cd6c6a88c468b
• crates/oa-desktop/src/app.rs (223370 bytes): 4f4c1bb5f832601a072b5cd406a75731982e8b9955ee6af39a93f124496938f9
• crates/oa-desktop/src/app/source_symbols.rs (21421 bytes): d81914a553114d85452d973a1edb175843d27154b3f9b9fc4cbc5049d423abe4
• crates/oa-desktop/src/app/text_workspace.rs (21658 bytes): 723ccc8b31c0e2ea8d34841359bf5df086ef866b2bf5c65eefce276c20015bb0
• crates/oa-desktop/src/document.rs (32600 bytes): e87d7ce456bcab1dab55249021eaa8056f41b888723d143428486e31cb9ca4b1