ARCADIA-Studio/harness/release-source-integration.md

# Source workspace integration contract

Status: implementation in progress after the accepted routing checkpoint. This
is the next vertical slice of the first release, not release completion.

## Observable outcome

A system engineer opens an entry `.arcadia` file and edits its included files
independently. Switching files retains invalid drafts and cursor positions.
Located project diagnostics navigate across files. Apply adopts the entire
validated source/model pair atomically. Graphical edits preserve comments and
file ownership. Undo/Redo restores exact accepted source, including comment-only
edits. Save All writes the project with conflict detection and recoverable disk
failure handling; reopen reproduces the accepted model and source.

## Architecture and sequence

1. Add `prepare_command(project, command)` to the shared semantic command engine.
   It returns a validated candidate and selected identity without mutating input
   or executing creation twice. Existing semantic Session uses the same helper.
2. Desktop `Document` owns one history of accepted `CompiledSourceProject`
   snapshots. Command preparation followed by `update_bundle` succeeds before
   adoption. Source-only accepted edits advance revision and history. Separate
   draft history remains an editor concern; do not synchronize two accepted
   model/source undo stacks.
3. Keep filesystem code outside pure `source_project`. Discover includes with
   `inspect_includes`, enforce aggregate budgets and portable/contained physical
   paths, retain exact original disk bytes before identity stabilization.
4. Integrate file navigation, full-bundle asynchronous lint, Apply, Open and Save
   into the existing Iced application. Results carry document identity and draft
   generation. The active text control renders one file; inactive drafts persist.
5. Extend production syntax through the same compiler/index/updater, separately
   from this integration. Never copy the experimental lowering/writer wholesale.
6. Verify real native two-file editing, failure repair, graphical rename, exact
   Undo, Save All and reopen; obtain independent correctness and UX review.

## Failure and persistence decisions

- Dirty text in any file blocks graphical mutation until explicitly applied or
  reverted. A failed compile/writeback leaves model, source, revision and history
  unchanged. Diagnostics retain original per-file spans and related locations.
- Successful lint is advisory: only explicit Apply adopts inserted identities.
- Syntax/missing-include errors retain loaded editable documents. Removing an
  include detaches its buffer; it does not delete a file. Compile reachable files
  while retaining detached dirty content visibly and guarding close/discard.
- Check all dependency baselines before saving, including unchanged files.
  Stage exact bytes, keep recovery information, replace the entry last and roll
  back ordinary partial failure. Interrupted recovery must not overwrite a newly
  observed external edit. This is recoverability, not filesystem-wide atomic
  visibility or compare-and-swap with other editors.
- Source Save As retains relative file layout and reviews destination collisions.
  JSON remains whole-model interchange and cannot erase source layout/history.
- Recovery/staging directories use the reserved `.arcadia-save-*` prefix. Ignore
  `.arcadia-save-*/` when tracking a source project in Git; these contain local
  save recovery data, not model declarations.
- Source writeback consumes the command engine's complete original-to-candidate
  identity correspondence. Changing UUID/alias or swapping UUIDs preserves the
  authored declaration file and associated reference-list comments. Inferring
  correspondence from surviving UUIDs or aliases is insufficient. Full model
  replacement uses common UUIDs as its explicit correspondence boundary.

## Production syntax direction

Retain flat syntax. Nested elements express semantic parent only; allocation
stays explicit. Contextual input/output shorthand creates real SA function ports,
with item lists and an optional ordinary field body. FE shorthand references real
ports and infers redundant function endpoints after resolution. Ordered steps
blocks cover existing scenario/process/chain/Branch semantics. Typed value bodies
remain ordinary fields; includes stay document-level. No new global reserved
aliases. The parser must expose real spans and lexical ownership, never fake
spans for inferred values. Structural edits compose around the existing
UUID-aware scalar/list updater.

Contiguous leading and same-line trailing comments travel with their declaration;
blank-separated comments stay in the containing gap. Deleted syntax leaves its
comments in the old body with separation so they cannot become the next record's
documentation. Preserve authored flat placement; prefer owner-local placement
for new children. Equal explicit parent and nesting may coexist; contradictions
fail. Shorthand-derived contracts/endpoints have one definition. Valid edits that
outgrow shorthand expand that declaration locally instead of rejecting the edit.

## Acceptance evidence required

- Creation selects the same UUID persisted in source; rejection is fully atomic.
- Comment-only Apply/Undo/Redo and history branching restore exact bytes.
- Two-file errors and stale lint/discovery cannot change newer drafts or models.
- IDs stabilize once while disk conflict baselines retain original bytes.
- Item rationale comments follow UUIDs through reorder and alias swaps.
- Nested create/delete/reparent and occurrence reorder/move preserve identity,
  comments, allocations and unrelated files; empty steps blocks remain editable.
- External edits/deletions/new-file collisions reject Save All before loss;
  injected partial failures and interrupted recovery preserve coherent content.
- Default/minimum-size native editing and save/reopen are observed, not inferred
  from compilation. Windows/Linux builds and focused regressions pass.

## Native integration follow-up

On Windows build SHA256 `3210042E61E95748386A57AD67120506F01A9EDB0A74DB40E36F94896C4068EC`
(commit `d364d3c`), root reopened `ui-source-project/native-source-trial.arcadia`
through the native picker. The 21-file project reported 41 objects, 25 links,
zero checks and all changes saved. Both entry rationale and included
`oa/participants.arcadia` rationale were visible in the real editor.

Root renamed Vessel in distress graphically, followed Properties → Source to
the same declaration in its owning file, and inspected the retained comment,
alias and UUID. The expected OA/SA name-drift warning appeared. Model Undo
restored the name/source and removed that warning, returning to all changes saved.

Root inserted an invalid declaration in the included file, switched to the entry,
and observed the inactive file's path/line diagnostic and dirty marker. Apply
rejected it while preserving the accepted 41-object model and the draft.
Architecture displayed a Source drafts pending notice instead of editable
properties. Open source drafts and the diagnostic link returned to the correct
file and selected the invalid line. Deleting it and Apply succeeded; Model Undo
restored the original saved source. Interrupted-journal and minimum-size source
acceptance are still underway; full release completion is not implied.

The independent recovery fixture generator used production core APIs to validate
four isolated originals and recover copies. Exact paths/contracts are in
`D:/CodexBuilds/Capella2-release1-01a06e2e/ui-source-recovery/README.md`.
Root opened `interrupted-mixed/project.arcadia`, inspected the AFTER included
bytes alongside the BEFORE entry, then recovered through the actual UI. The
result was Recovery baseline, two objects, one allocation and zero checks;
BEFORE comment/name bytes returned and the journal retired. The original UI put
recovery below the file list and displayed an internal journal path without
explaining rollback. Root replaced that with a prominent contextual banner.

Banner build SHA256 `91C03202C495DD074DFF328419E215A5C3BE935B15EE88BBA90467565D874C22`
passed native inspection at 1440×900 and exactly 1100×700 client. Save As to the
missing-entry fixture stopped before overwriting its pending transaction and
opened the banner. Root typed a new source comment; the banner remained. Recover
removed the partial first-save files/journal while preserving the current
41-object model and the newly typed draft. Save All then succeeded at the same
path without another picker, writing all 21 files. The app reported all changes
saved, 41 objects, 25 links and zero checks. Disk inspection confirmed the new
entry comment exactly.
The build succeeded after adding the omitted tooltip import. A later Clippy run
encountered the concurrent parser worker's incomplete Declaration initializer;
rerun once parser/writeback interfaces settle rather than treating it as a
verified lint pass.

At 1100×700, root opened `included-syntax-error/project.arcadia`. The included
`model.arcadia:4:17` diagnostic remained visible from the entry file, while the
previous 41-object graphical model remained accepted. Clicking the diagnostic
opened and selected the invalid line. Root inserted the missing description
value, retained the inline comment, and applied all files. The repaired model
adopted atomically as Recovery baseline, two objects, one allocation and zero
checks. Save All succeeded. Native Open reopened the same project with all
changes saved; the selected activity's property showed Saved activity rationale,
and Source navigation showed the original UUID, alias and retained comments.
The banner build then closed normally (session 29002, exit 0); its runtime log
was empty. This completes these Windows source recovery/minimum-size trials;
new nesting and REST implementations still need their own integrated evidence.

## Production nested ownership and REST integration (2026-09-06)

The existing source3 parser now accepts nested element declarations with real
lexical-owner/body spans. The structural writer relocates declaration pieces
after scalar edits, carrying attached comments and original record provenance.
The six-file `examples/source-project/project.arcadia` fixture compiles with
stable IDs, zero semantic diagnostics and three warning-free initial views.

Initial Windows integration passed 416 workspace tests. Root's final pass after
comment, provenance and example follow-ups passed 422; Linux passed 425 (including
platform-specific filesystem checks). Strict Windows/Linux workspace Clippy,
formatting, and optimized Linux desktop/server builds passed. Linux desktop
SHA256 is `C794426BD91980B2384B92884176542B2B081FA7B4ADE8ADA5D7EECEB72D6351`;
server SHA256 is `F7EFAAF3FA42D33D7CA77633BFDC9B9C87B57C558C3D4B1C71D508BF80293992`.
Both normative reST documents rendered through strict Docutils after fixing an
API section underline. Independent parser probes passed 14
malformed cases, 308 token mutations, budgets/spans/stabilization and 600 levels
on a 128 KiB stack. Independent writer probes passed all 125 valid parent
forests on four elements, whitespace/CRLF/compact variants, deletion/UUID reuse,
unchanged-file fidelity and budget atomicity. No material implementation finding
remains from those reviews. Source-preservation wording was qualified for
explicit nested reparenting. A fresh agent spawn hit the thread limit, so
reviewers examined artifacts implemented by another worker.

Review/reproduction fixed two material defects: equal-model delete/recreate
commands skipped provenance-aware source updates in desktop/server, and a
comment after an owner's opening brace followed a moved child. The latter
failed its focused reproducer before the fix and all 12 nested updater tests
passed afterward. All 12 Document tests passed, including exact nested source
Undo/Redo, rejected Apply and reopen.

Root operated the actual server at loopback port 18788: imported a nested
three-file project (201), applied a comment-only revision 2 with an unchanged
model, reparented a child across files with its rationale at revision 3, rejected
the original ETag (412), stopped and restarted the service, and verified exact
model/source/ETag preservation plus conditional source GET (304). Trial records
are in `D:/CodexBuilds/Capella2-release1-01a06e2e/source-project-service-trial`.
Runtime logs contain only the expected listening messages. The restarted server
was stopped; fresh process inventory confirmed it is no longer running.

Windows native build SHA256
`51002E90C7C75880FF054931343ADD48DE9970A29D9DDD294EB749E3C665F49C`
opened a separate copy of the six-file example. Root inspected OA allocation,
SA boundary/ports and functional containment views, changed Assess report's
semantic parent to Assure response quality in Properties, and applied it. The
outline and diagram adopted the new hierarchy; the performer remained Rescue
coordination system and Report input stayed owned by Assess report. Existing
coordinates remain authored: Arrange plus Fit produced a readable updated view.
Source navigated directly to `sa/assurance.arcadia` and displayed the relocated
function, port, UUIDs and leading/trailing rationale. Undo arrangement then Undo
reparent restored the original files and all-changes-saved state. Both Redo
operations succeeded. Save All wrote six files; disk inspection confirmed the
new declaration ownership and unchanged allocation. Native Open reopened the
saved project with 22 objects, 13 links, zero checks and all changes saved; the
SA allocation view retained the same participant/function ports and exchanges.
Minimum-size follow-up, independent native review and actual rebuilt Linux
nesting trial are still pending.