feat: preserve nested source projects across desktop and REST

docs/api.html

@@ -631,14 +631,17 @@ desktop.</p>
 <h2>Identity, revision and writes</h2>
 <p>All model routes begin with <span class="docutils literal">/api/v1</span>. A hosted project's immutable service
 <span class="docutils literal">resource_id</span> is different from its editable <span class="docutils literal">model.id</span>. Each stored file
-contains <span class="docutils literal">{resource_id, revision, model}</span>. The revision starts at 1, increments
-once per changed accepted transaction, and does not change for a no-op.</p>
+contains <span class="docutils literal">{resource_id, revision, model, source_project}</span>. <span class="docutils literal">source_project</span>
+is the accepted entry filename and map of authored source files; compiling it
+must reproduce <span class="docutils literal">model</span>. The revision starts at 1, increments once per changed
+accepted model/source pair, and does not change for an exact no-op. Comment-only
+edits and file/entry changes count as changes even when the model is unchanged.</p>
 <p>Existing-project writes require the exact quoted ETag returned by a project
 read, for example:</p>
 <pre class="literal-block">If-Match: &quot;p-6f7162cc-094c-4f30-9b70-3f6f355f0c7f-r1&quot;</pre>
 <p>Missing preconditions return 428; stale preconditions return 412. Weak ETags,
 wildcards and lists are not accepted on writes. GET of the envelope, model and
-source supports <span class="docutils literal"><span class="pre">If-None-Match</span></span> and returns an empty 304 when unchanged.
+source/source-project supports <span class="docutils literal"><span class="pre">If-None-Match</span></span> and returns an empty 304 when unchanged.
 Use ETags as opaque strings; revisions are unsigned integer counters in JSON.</p>
 <p>Every accepted changed write is persisted before the response. A worker captures
 a revision-qualified snapshot, then parses and validates its candidate outside
@@ -649,7 +652,8 @@ Malformed or rejected inputs preserve the accepted model. Incomplete-model
 warnings permit work and are returned in <span class="docutils literal">issues</span>.</p>
 <p>There is no retained revision history, idempotency receipt, tombstone or
 server undo. DELETE permanently removes the hosted envelope. Source/JSON
-exports contain the current model only. They preserve the model's persistent
+interchange exports contain the current model only; source-project export also
+preserves the authored file layout and comments. They preserve persistent
 aliases, diagrams, artifact references and OA/SA trace data. Artifact revision
 strings refer to external artifacts, not past server snapshots.</p>
 <p>A lost connection or 504 can occur after an admitted write has begun. That
@@ -707,6 +711,10 @@ bearer token, if any. Non-loopback binding requires a configured token.</p>
 <td><p><span class="docutils literal">/api/v1/projects/import</span></p></td>
 <td><p>Raw model JSON or Arcadia source; new service ID.</p></td>
 </tr>
+<tr><td><p>POST</p></td>
+<td><p><span class="docutils literal"><span class="pre">/api/v1/projects/import-source-project</span></span></p></td>
+<td><p>Complete source bundle as JSON; stabilize missing IDs and create a resource.</p></td>
+</tr>
 <tr><td><p>GET / PUT</p></td>
 <td><p><span class="docutils literal"><span class="pre">/api/v1/projects/{id}/model</span></span></p></td>
 <td><p>Export or conditionally replace native model JSON.</p></td>
@@ -715,6 +723,10 @@ bearer token, if any. Non-loopback binding requires a configured token.</p>
 <td><p><span class="docutils literal"><span class="pre">/api/v1/projects/{id}/source</span></span></p></td>
 <td><p>Export or conditionally replace canonical UTF-8 source.</p></td>
 </tr>
+<tr><td><p>GET / PUT</p></td>
+<td><p><span class="docutils literal"><span class="pre">/api/v1/projects/{id}/source-project</span></span></p></td>
+<td><p>Export exact authored files or conditionally replace the complete bundle.</p></td>
+</tr>
 <tr><td><p>GET</p></td>
 <td><p><span class="docutils literal"><span class="pre">/api/v1/projects/{id}/elements</span></span></p></td>
 <td><p>Filter by <span class="docutils literal">kind</span>, <span class="docutils literal">layer</span>, <span class="docutils literal">alias</span>, name <span class="docutils literal">q</span>.</p></td>
@@ -757,7 +769,10 @@ bearer token, if any. Non-loopback binding requires a configured token.</p>
 <span class="docutils literal">text/plain; <span class="pre">charset=utf-8</span></span> is used for Arcadia source. The service does not
 guess formats from a filename. Unknown DTO fields, invalid UUIDs/enums, duplicate
 model map keys and unsupported schemas are rejected. Maximum body size is
-8 MiB. Compressed or encoded request bodies and browser Origin requests are
+8 MiB. Accepted serialized model JSON and serialized source-bundle JSON each
+have an 8 MiB cap, including IDs inserted during compilation. The stored
+envelope allows both bounded representations plus small framing overhead.
+Compressed or encoded request bodies and browser Origin requests are
 not supported.</p>
 <p>Model admission permits at most 5,000 elements plus relations, 20,000 total
 persistent records (including the project, views and placements), and 50,000
@@ -770,10 +785,54 @@ Parsing, validation and query processing use snapshot workers, so this work
 does not hold the catalog lock or serialize unrelated model preparation.</p>
 <p>Import accepts schema-1 JSON through the shared explicit migration adapter,
 as well as schema-2 native JSON. Source import uses the shared language parser
-and its supported migrations. Export and native JSON replacement use schema 2.
+and its supported migrations. Export uses schema 3 / arcadia 3. Native JSON
+replacement accepts supported older schemas through the same explicit migration.
 Only documented query parameters are accepted. In particular, appending
 <span class="docutils literal"><span class="pre">?revision=...</span></span> to model/source export is an error, not historical retrieval.</p>
 </section>
+<section id="authored-source-projects">
+<h2>Authored source projects</h2>
+<p>A source bundle contains <span class="docutils literal">entry</span> and a map of <span class="docutils literal">files</span>. For example:</p>
+<pre class="literal-block">{
+  &quot;entry&quot;: &quot;rescue.arcadia&quot;,
+  &quot;files&quot;: {
+    &quot;rescue.arcadia&quot;: &quot;arcadia 3; project rescue \&quot;Rescue\&quot; { include \&quot;oa/work.arcadia\&quot;; }\n&quot;,
+    &quot;oa/work.arcadia&quot;: &quot;// Engineering rationale\noa_activity assess \&quot;Assess assistance\&quot; {}\n&quot;
+  }
+}</pre>
+<p>The entry is a root filename. All keys are canonical, portable, slash-separated
+relative paths ending in <span class="docutils literal">.arcadia</span>. Includes resolve only against these
+submitted strings. They never read the service filesystem, URLs, artifact
+targets or another hosted resource. All submitted files must be reachable;
+missing files, cycles, duplicate keys, case collisions and escaping paths are
+rejected. The bundle permits at most 256 files, 32 include edges along a path,
+and 1,024 include edges in total. Model and source-byte budgets apply across
+the complete bundle, not separately per file. Bundle syntax is source3;
+single-file legacy migration remains available through interchange import.</p>
+<p>Compilation inserts UUIDs into headers that omit them. Import returns 201 with
+Location/ETag; GET source-project retrieves the stabilized accepted bundle.
+PUT returns <span class="docutils literal">revision</span>, <span class="docutils literal">changed</span>, located <span class="docutils literal">issues</span> and that accepted
+<span class="docutils literal">source_project</span>. Preserve the returned IDs before subsequent editing.
+With explicit IDs, untouched text, comments, line endings, entry name and module
+ownership are preserved exactly. Re-submitting the same accepted bundle is a
+no-op. Removing a file means submitting a complete valid bundle without that
+file and its includes; no files are created or deleted on the service host.</p>
+<p>All typed command, transition, rename and model-interchange writes update this
+same accepted source project. Existing declarations and reference comments
+follow command identity correspondence, including UUID swaps. Full-model
+replacement uses common UUIDs as its correspondence boundary. It preserves
+the accepted source layout; canonical interchange formatting/comments are not
+adopted as a replacement authored workspace. Use source-project PUT for
+authoring file layout and comments. A failed source writeback rejects the whole
+write and never falls back to discarding authored files.</p>
+<p>Old model-only envelopes are migrated to a deterministic source bundle in
+memory when opened. Reading or an exact no-op does not rewrite their files or
+advance revisions. Their next changed write persists both representations.
+Startup rejects already-coupled envelopes whose source/model disagree or whose
+source lacks stable IDs. The 256 MiB store budget charges complete accepted
+envelopes, including the generated source of old envelopes before their next
+save; an over-capacity old store is reported rather than silently pruned.</p>
+</section>
 <section id="queries-and-pagination">
 <h2>Queries and pagination</h2>
 <p>List filters combine using AND. Enum spelling is case-sensitive and comes
@@ -841,11 +900,14 @@ creation identities allow later commands in the batch to reference earlier
 creations. The shared <span class="docutils literal">execute_batch</span> applies all operations to one candidate
 and validates the final state. A rejected operation or final structural error
 rejects the whole request. Reference rewrites/deletion cleanup follow the same
-core logic as desktop editing.</p>
+core logic as desktop editing. Command preparation retains original-to-candidate
+identity correspondence through UUID edits, swaps, deletion and reuse. Source
+writeback uses that correspondence to preserve declaration ownership and comments.</p>
 <p>Dry run still requires <span class="docutils literal"><span class="pre">If-Match</span></span>. Its response includes <span class="docutils literal">revision</span>,
-<span class="docutils literal">dry_run</span>, <span class="docutils literal">changed</span>, <span class="docutils literal">issues</span> and the proposed <span class="docutils literal">model</span>. It neither writes
-nor reserves IDs. A committed response has the same fields except the model is
-omitted. Commit must check the revision again. A no-op is successful with
+<span class="docutils literal">dry_run</span>, <span class="docutils literal">changed</span>, <span class="docutils literal">issues</span> and the proposed <span class="docutils literal">model</span> and
+<span class="docutils literal">source_project</span>. It validates source writeback as well as semantics and neither writes
+nor reserves IDs. A committed response omits the proposed model and source
+project. Commit must check the revision again. A no-op is successful with
 <span class="docutils literal">changed: false</span> and the original ETag.</p>
 </section>
 <section id="contextual-process-and-chain-topology">
@@ -898,10 +960,15 @@ reinterpret old mixed lifelines as a strict new viewpoint.</p>
 <pre class="literal-block">{&quot;format&quot;:&quot;json&quot;, &quot;model&quot;: {&quot;...&quot;:&quot;complete native Project record&quot;}}</pre>
 <p>or:</p>
 <pre class="literal-block">{&quot;format&quot;:&quot;arcadia&quot;, &quot;source&quot;:&quot;complete source document&quot;}</pre>
+<p>or:</p>
+<pre class="literal-block">{&quot;format&quot;:&quot;source-project&quot;, &quot;source_project&quot;:{&quot;entry&quot;:&quot;rescue.arcadia&quot;, &quot;files&quot;:{&quot;rescue.arcadia&quot;:&quot;arcadia 3; project rescue \&quot;Rescue\&quot; {}&quot;}}}</pre>
 <p>A well-formed validation request returns 200 with <span class="docutils literal">valid</span> and <span class="docutils literal">issues</span> even
 if its candidate is invalid. A malformed request envelope or JSON model record
 is a 400. Source diagnostics include one-based line and column according to the
-shared language parser. Candidate validation does not replace current state.</p>
+shared language parser. Candidate validation does not replace current state.
+Source-project validation locations refer to the submitted bytes before UUID
+insertion. Successful source-project PUT diagnostics refer to the returned
+accepted bundle. Missing IDs are not saved or adopted by validation.</p>
 </section>
 <section id="typed-state-machines-and-events">
 <h2>Typed state machines and events</h2>
@@ -934,7 +1001,8 @@ commit and disk persistence contract as other semantic records.</p>
 conflict resolutions and optional destination diagram. All fields are explicit
 in OpenAPI. At most 100 sources, 200 mappings and 400 field resolutions are
 accepted. A conflict-free plan must fit the same 200-command transaction limit
-and model-capacity limits.</p>
+and model/source-capacity limits. A conflict-free preview also checks source
+writeback, so its commands can update the same accepted authored workspace.</p>
 <p>The preview response is <span class="docutils literal"><span class="pre">{&quot;base_revision&quot;:1,&quot;plan&quot;:{...}}</span></span>. It contains
 planned changes, conflicts, warnings and exact commands/IDs. It does not write
 the model or persist a server-side plan. Apply accepts that entire response and
@@ -959,7 +1027,9 @@ not a security digest. There is no plan token, expiration store or receipt.</p>
   &quot;current_revision&quot;:3
 }</pre>
 <p><span class="docutils literal">issues</span> contain severity, code and message, and may include field,
-object_id, line and column. Use
+object_id, line, column, source <span class="docutils literal">path</span>, half-open UTF-8 byte <span class="docutils literal">span</span>
+(<span class="docutils literal">start</span>, <span class="docutils literal">end</span>), and <span class="docutils literal">related</span> source locations (path/span). Lines and
+columns are one-based; columns count Unicode scalars. Use
 <span class="docutils literal">code</span> for client decisions, not the English message. <span class="docutils literal"><span class="pre">X-Request-ID</span></span> is a
 response correlation UUID. This is not an idempotency key or receipt address.</p>
 <ul class="simple">
@@ -970,7 +1040,7 @@ response correlation UUID. This is not an idempotency key or receipt address.</p
 <li><p>405: unsupported HTTP method.</p></li>
 <li><p>409: requested query revision changed or revision counter exhausted.</p></li>
 <li><p>412: stale write precondition.</p></li>
-<li><p>413: body or serialized model exceeds size limit.</p></li>
+<li><p>413: body, serialized model, stabilized source bundle or envelope exceeds its size limit.</p></li>
 <li><p>415: unsupported Content-Type or Content-Encoding.</p></li>
 <li><p>422: rejected source/model/command, model count limit, or missing graph root.</p></li>
 <li><p>428: missing If-Match.</p></li>