ARCADIA-Studio/harness/release-source-integration.md

Source workspace integration contract

Status: implementation in progress after the accepted routing checkpoint. This is the next vertical slice of the first release, not release completion.

Observable outcome

A system engineer opens an entry .arcadia file and edits its included files independently. Switching files retains invalid drafts and cursor positions. Located project diagnostics navigate across files. Apply adopts the entire validated source/model pair atomically. Graphical edits preserve comments and file ownership. Undo/Redo restores exact accepted source, including comment-only edits. Save All writes the project with conflict detection and recoverable disk failure handling; reopen reproduces the accepted model and source.

Architecture and sequence

1. Add prepare_command(project, command) to the shared semantic command engine. It returns a validated candidate and selected identity without mutating input or executing creation twice. Existing semantic Session uses the same helper.
2. Desktop Document owns one history of accepted CompiledSourceProject snapshots. Command preparation followed by update_bundle succeeds before adoption. Source-only accepted edits advance revision and history. Separate draft history remains an editor concern; do not synchronize two accepted model/source undo stacks.
3. Keep filesystem code outside pure source_project. Discover includes with inspect_includes, enforce aggregate budgets and portable/contained physical paths, retain exact original disk bytes before identity stabilization.
4. Integrate file navigation, full-bundle asynchronous lint, Apply, Open and Save into the existing Iced application. Results carry document identity and draft generation. The active text control renders one file; inactive drafts persist.
5. Extend production syntax through the same compiler/index/updater, separately from this integration. Never copy the experimental lowering/writer wholesale.
6. Verify real native two-file editing, failure repair, graphical rename, exact Undo, Save All and reopen; obtain independent correctness and UX review.

Failure and persistence decisions

• Dirty text in any file blocks graphical mutation until explicitly applied or reverted. A failed compile/writeback leaves model, source, revision and history unchanged. Diagnostics retain original per-file spans and related locations.
• Successful lint is advisory: only explicit Apply adopts inserted identities.
• Syntax/missing-include errors retain loaded editable documents. Removing an include detaches its buffer; it does not delete a file. Compile reachable files while retaining detached dirty content visibly and guarding close/discard.
• Check all dependency baselines before saving, including unchanged files. Stage exact bytes, keep recovery information, replace the entry last and roll back ordinary partial failure. Interrupted recovery must not overwrite a newly observed external edit. This is recoverability, not filesystem-wide atomic visibility or compare-and-swap with other editors.
• Source Save As retains relative file layout and reviews destination collisions. JSON remains whole-model interchange and cannot erase source layout/history.
• Recovery/staging directories use the reserved .arcadia-save-* prefix. Ignore .arcadia-save-*/ when tracking a source project in Git; these contain local save recovery data, not model declarations.
• Source writeback consumes the command engine's complete original-to-candidate identity correspondence. Changing UUID/alias or swapping UUIDs preserves the authored declaration file and associated reference-list comments. Inferring correspondence from surviving UUIDs or aliases is insufficient. Full model replacement uses common UUIDs as its explicit correspondence boundary.

Production syntax direction

Retain flat syntax. Nested elements express semantic parent only; allocation stays explicit. Contextual input/output shorthand creates real SA function ports, with item lists and an optional ordinary field body. FE shorthand references real ports and infers redundant function endpoints after resolution. Ordered steps blocks cover existing scenario/process/chain/Branch semantics. Typed value bodies remain ordinary fields; includes stay document-level. No new global reserved aliases. The parser must expose real spans and lexical ownership, never fake spans for inferred values. Structural edits compose around the existing UUID-aware scalar/list updater.

Contiguous leading and same-line trailing comments travel with their declaration; blank-separated comments stay in the containing gap. Deleted syntax leaves its comments in the old body with separation so they cannot become the next record's documentation. Preserve authored flat placement; prefer owner-local placement for new children. Equal explicit parent and nesting may coexist; contradictions fail. Shorthand-derived contracts/endpoints have one definition. Valid edits that outgrow shorthand expand that declaration locally instead of rejecting the edit.

Acceptance evidence required

• Creation selects the same UUID persisted in source; rejection is fully atomic.
• Comment-only Apply/Undo/Redo and history branching restore exact bytes.
• Two-file errors and stale lint/discovery cannot change newer drafts or models.
• IDs stabilize once while disk conflict baselines retain original bytes.
• Item rationale comments follow UUIDs through reorder and alias swaps.
• Nested create/delete/reparent and occurrence reorder/move preserve identity, comments, allocations and unrelated files; empty steps blocks remain editable.
• External edits/deletions/new-file collisions reject Save All before loss; injected partial failures and interrupted recovery preserve coherent content.
• Default/minimum-size native editing and save/reopen are observed, not inferred from compilation. Windows/Linux builds and focused regressions pass.

Native integration follow-up

On Windows build SHA256 3210042E61E95748386A57AD67120506F01A9EDB0A74DB40E36F94896C4068EC (commit d364d3c), root reopened ui-source-project/native-source-trial.arcadia through the native picker. The 21-file project reported 41 objects, 25 links, zero checks and all changes saved. Both entry rationale and included oa/participants.arcadia rationale were visible in the real editor.

Root renamed Vessel in distress graphically, followed Properties → Source to the same declaration in its owning file, and inspected the retained comment, alias and UUID. The expected OA/SA name-drift warning appeared. Model Undo restored the name/source and removed that warning, returning to all changes saved.

Root inserted an invalid declaration in the included file, switched to the entry, and observed the inactive file's path/line diagnostic and dirty marker. Apply rejected it while preserving the accepted 41-object model and the draft. Architecture displayed a Source drafts pending notice instead of editable properties. Open source drafts and the diagnostic link returned to the correct file and selected the invalid line. Deleting it and Apply succeeded; Model Undo restored the original saved source. Interrupted-journal and minimum-size source acceptance are still underway; full release completion is not implied.

The independent recovery fixture generator used production core APIs to validate four isolated originals and recover copies. Exact paths/contracts are in D:/CodexBuilds/Capella2-release1-01a06e2e/ui-source-recovery/README.md. Root opened interrupted-mixed/project.arcadia, inspected the AFTER included bytes alongside the BEFORE entry, then recovered through the actual UI. The result was Recovery baseline, two objects, one allocation and zero checks; BEFORE comment/name bytes returned and the journal retired. The original UI put recovery below the file list and displayed an internal journal path without explaining rollback. Root replaced that with a prominent contextual banner.

Banner build SHA256 91C03202C495DD074DFF328419E215A5C3BE935B15EE88BBA90467565D874C22 passed native inspection at 1440×900 and exactly 1100×700 client. Save As to the missing-entry fixture stopped before overwriting its pending transaction and opened the banner. Root typed a new source comment; the banner remained. Recover removed the partial first-save files/journal while preserving the current 41-object model and the newly typed draft. Save All then succeeded at the same path without another picker, writing all 21 files. The app reported all changes saved, 41 objects, 25 links and zero checks. Disk inspection confirmed the new entry comment exactly. The build succeeded after adding the omitted tooltip import. A later Clippy run encountered the concurrent parser worker's incomplete Declaration initializer; rerun once parser/writeback interfaces settle rather than treating it as a verified lint pass.

At 1100×700, root opened included-syntax-error/project.arcadia. The included model.arcadia:4:17 diagnostic remained visible from the entry file, while the previous 41-object graphical model remained accepted. Clicking the diagnostic opened and selected the invalid line. Root inserted the missing description value, retained the inline comment, and applied all files. The repaired model adopted atomically as Recovery baseline, two objects, one allocation and zero checks. Save All succeeded. Native Open reopened the same project with all changes saved; the selected activity's property showed Saved activity rationale, and Source navigation showed the original UUID, alias and retained comments. The banner build then closed normally (session 29002, exit 0); its runtime log was empty. This completes these Windows source recovery/minimum-size trials; new nesting and REST implementations still need their own integrated evidence.

Production nested ownership and REST integration (2026-09-06)

The existing source3 parser now accepts nested element declarations with real lexical-owner/body spans. The structural writer relocates declaration pieces after scalar edits, carrying attached comments and original record provenance. The six-file examples/source-project/project.arcadia fixture compiles with stable IDs, zero semantic diagnostics and three warning-free initial views.

Initial Windows integration passed 416 workspace tests. Root's final pass after comment, provenance and example follow-ups passed 422; Linux passed 425 (including platform-specific filesystem checks). Strict Windows/Linux workspace Clippy, formatting, and optimized Linux desktop/server builds passed. Linux desktop SHA256 is C794426BD91980B2384B92884176542B2B081FA7B4ADE8ADA5D7EECEB72D6351; server SHA256 is F7EFAAF3FA42D33D7CA77633BFDC9B9C87B57C558C3D4B1C71D508BF80293992. Both normative reST documents rendered through strict Docutils after fixing an API section underline. Independent parser probes passed 14 malformed cases, 308 token mutations, budgets/spans/stabilization and 600 levels on a 128 KiB stack. Independent writer probes passed all 125 valid parent forests on four elements, whitespace/CRLF/compact variants, deletion/UUID reuse, unchanged-file fidelity and budget atomicity. No material implementation finding remains from those reviews. Source-preservation wording was qualified for explicit nested reparenting. A fresh agent spawn hit the thread limit, so reviewers examined artifacts implemented by another worker.

Review/reproduction fixed two material defects: equal-model delete/recreate commands skipped provenance-aware source updates in desktop/server, and a comment after an owner's opening brace followed a moved child. The latter failed its focused reproducer before the fix and all 12 nested updater tests passed afterward. All 12 Document tests passed, including exact nested source Undo/Redo, rejected Apply and reopen.

Root operated the actual server at loopback port 18788: imported a nested three-file project (201), applied a comment-only revision 2 with an unchanged model, reparented a child across files with its rationale at revision 3, rejected the original ETag (412), stopped and restarted the service, and verified exact model/source/ETag preservation plus conditional source GET (304). Trial records are in D:/CodexBuilds/Capella2-release1-01a06e2e/source-project-service-trial. Runtime logs contain only the expected listening messages. The restarted server was stopped; fresh process inventory confirmed it is no longer running.

Windows native build SHA256 51002E90C7C75880FF054931343ADD48DE9970A29D9DDD294EB749E3C665F49C opened a separate copy of the six-file example. Root inspected OA allocation, SA boundary/ports and functional containment views, changed Assess report's semantic parent to Assure response quality in Properties, and applied it. The outline and diagram adopted the new hierarchy; the performer remained Rescue coordination system and Report input stayed owned by Assess report. Existing coordinates remain authored: Arrange plus Fit produced a readable updated view. Source navigated directly to sa/assurance.arcadia and displayed the relocated function, port, UUIDs and leading/trailing rationale. Undo arrangement then Undo reparent restored the original files and all-changes-saved state. Both Redo operations succeeded. Save All wrote six files; disk inspection confirmed the new declaration ownership and unchanged allocation. Native Open reopened the saved project with 22 objects, 13 links, zero checks and all changes saved; the SA allocation view retained the same participant/function ports and exchanges. Minimum-size follow-up, independent native review and actual rebuilt Linux nesting trial are still pending.

Linux nested source acceptance — 2026-09-06

Root operated the optimized Linux desktop SHA256 C794426BD91980B2384B92884176542B2B081FA7B4ADE8ADA5D7EECEB72D6351 through Computer Use. The isolated six-file fixture was on the native Linux filesystem at /home/sorokinoleg/.cache/arcadia/nested-source-native-20260906/project.arcadia. Native Open loaded 22 objects, 13 links and zero checks. OA allocated hierarchy and SA system/actor nesting, real function-port triangles and component-port squares were visually inspected. Textual model opened the nested sa/reporting.arcadia declarations with highlighting, stable UUIDs and comments. A comment edit was applied, model Undo restored the original accepted file and clean state, Redo restored the edit, Save all completed, and native Open reopened the entry. The comment and nested source were personally verified after reopening, with all changes saved and zero checks. The Linux validation window closed normally; the original Windows window containing user edits was untouched. This is source/comment and rendered hierarchy evidence; the cross-file graphical reparent workflow was exercised on Windows, not repeated on Linux. Save latency was not benchmarked.