feat: preserve nested source projects across desktop and REST

docs/api.rst

@@ -12,8 +12,11 @@ Identity, revision and writes
 
 All model routes begin with ``/api/v1``. A hosted project's immutable service
 ``resource_id`` is different from its editable ``model.id``. Each stored file
-contains ``{resource_id, revision, model}``. The revision starts at 1, increments
-once per changed accepted transaction, and does not change for a no-op.
+contains ``{resource_id, revision, model, source_project}``. ``source_project``
+is the accepted entry filename and map of authored source files; compiling it
+must reproduce ``model``. The revision starts at 1, increments once per changed
+accepted model/source pair, and does not change for an exact no-op. Comment-only
+edits and file/entry changes count as changes even when the model is unchanged.
 
 Existing-project writes require the exact quoted ETag returned by a project
 read, for example::
@@ -22,7 +25,7 @@ read, for example::
 
 Missing preconditions return 428; stale preconditions return 412. Weak ETags,
 wildcards and lists are not accepted on writes. GET of the envelope, model and
-source supports ``If-None-Match`` and returns an empty 304 when unchanged.
+source/source-project supports ``If-None-Match`` and returns an empty 304 when unchanged.
 Use ETags as opaque strings; revisions are unsigned integer counters in JSON.
 
 Every accepted changed write is persisted before the response. A worker captures
@@ -35,7 +38,8 @@ warnings permit work and are returned in ``issues``.
 
 There is no retained revision history, idempotency receipt, tombstone or
 server undo. DELETE permanently removes the hosted envelope. Source/JSON
-exports contain the current model only. They preserve the model's persistent
+interchange exports contain the current model only; source-project export also
+preserves the authored file layout and comments. They preserve persistent
 aliases, diagrams, artifact references and OA/SA trace data. Artifact revision
 strings refer to external artifacts, not past server snapshots.
 
@@ -81,12 +85,18 @@ bearer token, if any. Non-loopback binding requires a configured token.
    * - POST
      - ``/api/v1/projects/import``
      - Raw model JSON or Arcadia source; new service ID.
+   * - POST
+     - ``/api/v1/projects/import-source-project``
+     - Complete source bundle as JSON; stabilize missing IDs and create a resource.
    * - GET / PUT
      - ``/api/v1/projects/{id}/model``
      - Export or conditionally replace native model JSON.
    * - GET / PUT
      - ``/api/v1/projects/{id}/source``
      - Export or conditionally replace canonical UTF-8 source.
+   * - GET / PUT
+     - ``/api/v1/projects/{id}/source-project``
+     - Export exact authored files or conditionally replace the complete bundle.
    * - GET
      - ``/api/v1/projects/{id}/elements``
      - Filter by ``kind``, ``layer``, ``alias``, name ``q``.
@@ -119,7 +129,10 @@ bearer token, if any. Non-loopback binding requires a configured token.
 ``text/plain; charset=utf-8`` is used for Arcadia source. The service does not
 guess formats from a filename. Unknown DTO fields, invalid UUIDs/enums, duplicate
 model map keys and unsupported schemas are rejected. Maximum body size is
-8 MiB. Compressed or encoded request bodies and browser Origin requests are
+8 MiB. Accepted serialized model JSON and serialized source-bundle JSON each
+have an 8 MiB cap, including IDs inserted during compilation. The stored
+envelope allows both bounded representations plus small framing overhead.
+Compressed or encoded request bodies and browser Origin requests are
 not supported.
 
 Model admission permits at most 5,000 elements plus relations, 20,000 total
@@ -134,10 +147,60 @@ does not hold the catalog lock or serialize unrelated model preparation.
 
 Import accepts schema-1 JSON through the shared explicit migration adapter,
 as well as schema-2 native JSON. Source import uses the shared language parser
-and its supported migrations. Export and native JSON replacement use schema 2.
+and its supported migrations. Export uses schema 3 / arcadia 3. Native JSON
+replacement accepts supported older schemas through the same explicit migration.
 Only documented query parameters are accepted. In particular, appending
 ``?revision=...`` to model/source export is an error, not historical retrieval.
 
+Authored source projects
+------------------------
+
+A source bundle contains ``entry`` and a map of ``files``. For example::
+
+    {
+      "entry": "rescue.arcadia",
+      "files": {
+        "rescue.arcadia": "arcadia 3; project rescue \"Rescue\" { include \"oa/work.arcadia\"; }\n",
+        "oa/work.arcadia": "// Engineering rationale\noa_activity assess \"Assess assistance\" {}\n"
+      }
+    }
+
+The entry is a root filename. All keys are canonical, portable, slash-separated
+relative paths ending in ``.arcadia``. Includes resolve only against these
+submitted strings. They never read the service filesystem, URLs, artifact
+targets or another hosted resource. All submitted files must be reachable;
+missing files, cycles, duplicate keys, case collisions and escaping paths are
+rejected. The bundle permits at most 256 files, 32 include edges along a path,
+and 1,024 include edges in total. Model and source-byte budgets apply across
+the complete bundle, not separately per file. Bundle syntax is source3;
+single-file legacy migration remains available through interchange import.
+
+Compilation inserts UUIDs into headers that omit them. Import returns 201 with
+Location/ETag; GET source-project retrieves the stabilized accepted bundle.
+PUT returns ``revision``, ``changed``, located ``issues`` and that accepted
+``source_project``. Preserve the returned IDs before subsequent editing.
+With explicit IDs, untouched text, comments, line endings, entry name and module
+ownership are preserved exactly. Re-submitting the same accepted bundle is a
+no-op. Removing a file means submitting a complete valid bundle without that
+file and its includes; no files are created or deleted on the service host.
+
+All typed command, transition, rename and model-interchange writes update this
+same accepted source project. Existing declarations and reference comments
+follow command identity correspondence, including UUID swaps. Full-model
+replacement uses common UUIDs as its correspondence boundary. It preserves
+the accepted source layout; canonical interchange formatting/comments are not
+adopted as a replacement authored workspace. Use source-project PUT for
+authoring file layout and comments. A failed source writeback rejects the whole
+write and never falls back to discarding authored files.
+
+Old model-only envelopes are migrated to a deterministic source bundle in
+memory when opened. Reading or an exact no-op does not rewrite their files or
+advance revisions. Their next changed write persists both representations.
+Startup rejects already-coupled envelopes whose source/model disagree or whose
+source lacks stable IDs. The 256 MiB store budget charges complete accepted
+envelopes, including the generated source of old envelopes before their next
+save; an over-capacity old store is reported rather than silently pruned.
+
 Queries and pagination
 ----------------------
 
@@ -217,12 +280,15 @@ creation identities allow later commands in the batch to reference earlier
 creations. The shared ``execute_batch`` applies all operations to one candidate
 and validates the final state. A rejected operation or final structural error
 rejects the whole request. Reference rewrites/deletion cleanup follow the same
-core logic as desktop editing.
+core logic as desktop editing. Command preparation retains original-to-candidate
+identity correspondence through UUID edits, swaps, deletion and reuse. Source
+writeback uses that correspondence to preserve declaration ownership and comments.
 
 Dry run still requires ``If-Match``. Its response includes ``revision``,
-``dry_run``, ``changed``, ``issues`` and the proposed ``model``. It neither writes
-nor reserves IDs. A committed response has the same fields except the model is
-omitted. Commit must check the revision again. A no-op is successful with
+``dry_run``, ``changed``, ``issues`` and the proposed ``model`` and
+``source_project``. It validates source writeback as well as semantics and neither writes
+nor reserves IDs. A committed response omits the proposed model and source
+project. Commit must check the revision again. A no-op is successful with
 ``changed: false`` and the original ETag.
 
 Contextual process and chain topology
@@ -287,10 +353,17 @@ or::
 
     {"format":"arcadia", "source":"complete source document"}
 
+or::
+
+    {"format":"source-project", "source_project":{"entry":"rescue.arcadia", "files":{"rescue.arcadia":"arcadia 3; project rescue \"Rescue\" {}"}}}
+
 A well-formed validation request returns 200 with ``valid`` and ``issues`` even
 if its candidate is invalid. A malformed request envelope or JSON model record
 is a 400. Source diagnostics include one-based line and column according to the
 shared language parser. Candidate validation does not replace current state.
+Source-project validation locations refer to the submitted bytes before UUID
+insertion. Successful source-project PUT diagnostics refer to the returned
+accepted bundle. Missing IDs are not saved or adopted by validation.
 
 Typed state machines and events
 -------------------------------
@@ -327,7 +400,8 @@ Preview accepts the shared ``TransitionRequest`` record, including direction
 conflict resolutions and optional destination diagram. All fields are explicit
 in OpenAPI. At most 100 sources, 200 mappings and 400 field resolutions are
 accepted. A conflict-free plan must fit the same 200-command transaction limit
-and model-capacity limits.
+and model/source-capacity limits. A conflict-free preview also checks source
+writeback, so its commands can update the same accepted authored workspace.
 
 The preview response is ``{"base_revision":1,"plan":{...}}``. It contains
 planned changes, conflicts, warnings and exact commands/IDs. It does not write
@@ -357,7 +431,9 @@ All errors have ``Content-Type: application/problem+json``. Example::
     }
 
 ``issues`` contain severity, code and message, and may include field,
-object_id, line and column. Use
+object_id, line, column, source ``path``, half-open UTF-8 byte ``span``
+(``start``, ``end``), and ``related`` source locations (path/span). Lines and
+columns are one-based; columns count Unicode scalars. Use
 ``code`` for client decisions, not the English message. ``X-Request-ID`` is a
 response correlation UUID. This is not an idempotency key or receipt address.
 
@@ -368,7 +444,7 @@ response correlation UUID. This is not an idempotency key or receipt address.
 * 405: unsupported HTTP method.
 * 409: requested query revision changed or revision counter exhausted.
 * 412: stale write precondition.
-* 413: body or serialized model exceeds size limit.
+* 413: body, serialized model, stabilized source bundle or envelope exceeds its size limit.
 * 415: unsupported Content-Type or Content-Encoding.
 * 422: rejected source/model/command, model count limit, or missing graph root.
 * 428: missing If-Match.