Specify R6 data contracts and verify coupled REST migration

docs/language.rst

@@ -1,7 +1,7 @@
 Arcadia Studio OA / SA Language and Editing Contract
 ====================================================
 
-:Version: 3
+:Version: 4
 :Status: Normative current implementation contract; first-release integration
 :Core implementation: ``crates/oa-core``
 :Text suffix: ``.arcadia``
@@ -11,15 +11,14 @@ Arcadia Studio OA / SA Language and Editing Contract
 
 This specification defines the shared Operational Analysis (OA) and System
 Analysis (SA) model, its textual language, graphical editing obligations,
-validation, persistence, and explicit migration from versions 1 and 2. MUST, MUST NOT,
+validation, persistence, and explicit migration from versions 1, 2 and 3. MUST, MUST NOT,
 and MAY indicate requirements, prohibitions, and permitted choices.
 
-Schema 3 introduces typed function and component ports, required functional
-exchange endpoints, independently oriented boundary exchanges, and explicit
-PortAllocation relations. First-release scenario/state additions remain in
-scope; this schema change does not declare native acceptance or the wider
-release complete. The preserved schema-2 reader validates the original
-contract before migration.
+Schema 4 adds bounded reusable data types, typed scalar defaults, exchange
+mechanisms, qualified message values, specialization, identified interface
+allocations and reference associations. It retains schema 3 typed ports and
+state views. Explicit older readers preserve existing meanings and identities.
+This contract does not by itself establish native acceptance or release completion.
 
 OA describes stakeholders, operational work and needs. SA describes the
 selected system boundary, external actors, functions, services and expected
@@ -40,7 +39,7 @@ Actual verification evidence is separate from this contract.
 2. One authoritative model
 --------------------------
 
-A Project stores ``schema_version`` (3), ``id``, ``alias``, ``name``,
+A Project stores ``schema_version`` (4), ``id``, ``alias``, ``name``,
 ``description``, and maps named ``elements``, ``relations``, ``diagrams``,
 ``artifact_links``, and ``traces``. Each map is keyed by the UUID contained in
 its record. Duplicate map keys and mismatching keys/identities are errors.
@@ -96,17 +95,17 @@ lists all additional fields. ``ref`` means ``@alias``; ``ref?`` also accepts
      - ``steps``: occurrence refs
      - ``[]``
    * - ``oa_exchange_item``, ``sa_exchange_item``
-     - ``kind``: Information, Material, Energy, Event
-     - Information
+     - ``kind``: Information, Material, Energy; ``mechanism``: ExchangeMechanism; ``supertype``: item ref?
+     - Information; Unspecified; none
    * - ``oa_data_type``, ``sa_data_type``
-     - ``definition``: Boolean, Integer, Real, String, Record, or Enumeration constructor
-     - String
+     - ``definition``: DataTypeDefinition; ``default``: DataValue?; ``unit``: string
+     - String with minimum_length 0 / maximum_length none; none; ``""``
    * - ``oa_data_field``, ``sa_data_field``
-     - ``type_ref``; ``minimum``: u32; ``maximum``: optional u32; ``unit``: string
-     - String; 1; 1; ``""``
+     - ``type_ref``; ``minimum``: u32; ``maximum``: optional u32; ``unit``: string; ``direction``: ParameterDirection; ``storage``: FieldStorage; ``default``: DataValue list?
+     - String; 1; 1; ``""``; Unspecified; Value; none
    * - ``oa_interface``, ``sa_interface``
-     - ``items``: exchange-item refs
-     - ``[]``
+     - ``supertype``: interface ref?
+     - none; membership is an identified interface_item_allocation relation
    * - ``oa_port``, ``sa_port``
      - ``contract``: required PortContract; ``legacy``: optional LegacyPortContract
      - No implicit contract; ``legacy`` defaults none. OA admits Legacy only.
@@ -230,6 +229,12 @@ references. Their kind is the declaration keyword:
    * - ``allocation``
      - Same-layer participant → leaf activity/function
      - None
+   * - ``interface_item_allocation``
+     - Same-layer Interface → ExchangeItem
+     - ``order`` u32 / 0; ``transmission`` TransmissionProtocol / Unspecified; ``acquisition`` AcquisitionProtocol / Unspecified
+   * - ``data_association``
+     - Same-layer Record DataType → Record DataType
+     - ``source_role``, ``target_role`` / ""; ``source_minimum``, ``target_minimum`` / 0; ``source_maximum``, ``target_maximum`` / none; ``source_navigable``, ``target_navigable`` / false
    * - ``interaction``
      - OA activity → OA activity
      - ``items`` refs / []
@@ -290,7 +295,8 @@ references. Their kind is the declaration keyword:
 
 Relation endpoints and carried ExchangeItems stay in the same layer. Parallel
 exchanges with distinct identities are permitted. Self-relations are invalid
-except for a state transition that returns to its source state.
+except for a state transition that returns to its source state or an explicit
+DataAssociation between two ends of the same Record type.
 
 Direct Allocation has at most one performer per leaf. Role sharing is a
 different relationship: several OA participants can realize a Role. A direct
@@ -347,12 +353,13 @@ Port deletion cascades to incident FEs; FE deletion retains independent ports.
 
 For example::
 
-    arcadia 3;
+    arcadia 4;
     project ports_example "Typed function contract" {
         sa_function send "Send" {}
         sa_function receive "Receive" {}
         sa_exchange_item payload "Payload" {}
-        sa_interface old_service "Preserved service" { items [@payload]; }
+        sa_interface old_service "Preserved service" {}
+        interface_item_allocation service_payload "Service payload" { source @old_service; target @payload; }
         sa_port output "Output" {
             parent @send;
             contract { FunctionOutput { outgoing_items [@payload]; }; };
@@ -474,11 +481,19 @@ no receiver or the same receiver as sender. Its nonempty expression describes
 the delay/deadline; an empty expression warns. Conditions and expressions are
 stored engineering annotations, not an executable simulation language.
 
-Each MessageValue is ``{ field @data_field; expression "value expression"; }``.
-The field MUST be a DataField carried by the message exchange, including fields
-inside named nested records. At most one annotation for a given field is
-allowed in one message. Values do not overwrite the referenced type or global
-exchange item. Expression text is preserved exactly; an empty expression warns.
+Each MessageValue is ``{ path [@parameter]; field @data_field; expression "value expression"; }``.
+Its identity is the pair (path, field), so two uses of one Record can have
+different values for the same terminal field. Path contains enclosing DataField
+references, excluding the terminal field; direct parameters use ``[]``. Every
+hop MUST follow Value storage through a named Record's effective fields. Reference
+storage stops traversal. Operation Call accepts In/InOut and Reply accepts
+Out/InOut/Return, interpreted from the provider's perspective. Nested field paths
+inherit their direct parameter's direction. Unspecified direction warns.
+At most one annotation for a pair is allowed. Omitted path defaults to ``[]``;
+an ambiguous preserved legacy nested annotation warns ``message_value_ambiguous_path``
+until an explicit path is selected. New graphical choices always identify a path.
+Values never overwrite types or items. Expression text is descriptive, preserved
+exactly and not evaluated; an empty expression warns.
 
 ScenarioUse references an existing same-layer scenario and maps its lifelines
 with ``bindings [ { referenced @callee_line; local @caller_line; } ];``.
@@ -620,6 +635,141 @@ DeepHistory remembers nested configuration. Fallback applies when no resumable
 history exists. History is per Region, including top-level Regions in this
 Studio profile; no running history state is persisted or synthesized.
 
+4.5. Data contracts, specialization and interface allocation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ExchangeItem ``kind`` describes nature: Information, Material or Energy.
+``mechanism`` is Unspecified, Flow, Event, Operation or SharedData. Unknown
+mechanisms remain explicit Unspecified drafts with a warning. Material/Energy
+with Event/Operation/SharedData warns for engineering review; it is not silently
+reclassified. Empty Event notifications and void Operations are valid. An empty
+carried/allocated Flow, SharedData or Unspecified item warns.
+
+An Operation's directly owned DataFields are parameters. ``direction`` is
+Unspecified, In, Out, InOut or Return, from the provider's perspective. At most
+one effective parameter is Return. Unspecified parameters warn; non-operation
+fields MUST use Unspecified. Nested record fields inherit direction from their
+direct parameter when used in message paths, rather than storing context on a
+shared type. ``storage`` is Value or Reference. Reference requires a named Record,
+has no explicit default, and does not convey ownership or deletion of the type.
+
+DataTypeDefinition has exactly the following constructors:
+
+.. list-table:: Reusable definitions
+   :header-rows: 1
+
+   * - Constructor
+     - Required fields
+   * - Boolean
+     - None
+   * - Integer
+     - ``minimum``, ``maximum``: optional signed 64-bit integers
+   * - Real
+     - ``minimum``, ``maximum``: optional finite IEEE-754 binary64 values
+   * - String
+     - ``minimum_length``: u32; ``maximum_length``: optional u32
+   * - Enumeration
+     - ``literals``: unique nonempty, case-sensitive strings
+   * - Record
+     - ``supertype``: same-layer Record ref?
+
+Bounds are inclusive and ordered; an absent bound means unbounded within the
+represented numeric domain. Integer comparisons MUST NOT convert to binary64.
+NaN, infinities, overflow and reversed bounds reject. String length counts Unicode
+scalar values, not UTF-8 bytes or grapheme clusters. Enum defaults match exact
+literals. Empty enum definitions remain incomplete drafts with a warning.
+
+DataValue is a tagged scalar: ``{ Boolean true; }``, ``{ Integer 42; }``,
+``{ Real 1.5; }``, ``{ String "value"; }`` or ``{ Enumeration "Ready"; }``.
+There is no implicit numeric or string coercion. A type default is one compatible
+scalar within its bounds. Record type defaults and record/object instance literals
+are unsupported. A field default is absent (``none``) or a list of compatible
+scalars with length inside its multiplicity. ``[]`` explicitly supplies an empty
+collection and requires minimum 0, including Value Record fields; a nonempty
+Record literal rejects. Reference fields MUST use ``none``. A one-valued scalar
+field uses a one-value list. A named type default supplies an individual element's
+default, not a collection or an implicit instance of a mandatory field.
+
+Numeric types may carry unit text. Empty field units inherit the named type unit;
+an explicit different unit warns when that type has a unit. Units on nonnumeric
+fields/types warn and remain preserved. Unit strings never convert values or
+establish dimensional compatibility. Expressions are not executed.
+
+Record, ExchangeItem and Interface each permit one same-family, same-layer
+supertype. Specialization is acyclic and separate from containment. Effective
+fields are base fields followed by local fields, preserving original field UUIDs
+and owners. Local names MUST NOT shadow inherited names. Item specialization
+preserves base nature and any specified mechanism; an Unspecified base mechanism
+may be refined. Deleting a derived record never deletes its base. Mandatory
+Value-field cycles warn ``record_no_finite_value``; optional or Reference cycles
+are valid without that warning. Traversal is cycle guarded and bounded; a payload
+path budget failure is an explicit diagnostic, not silent truncation.
+
+InterfaceItemAllocation is an ordinary identified relation from Interface to
+ExchangeItem. It stores ``order`` and independent transmission/acquisition
+protocols. Transmission is Unspecified, Synchronous, Asynchronous, Unicast,
+Multicast or Broadcast; acquisition is Unspecified, Read or Accept. No protocol
+combination is inferred from mechanism. Effective allocations are base-first,
+then local ordered by (order, alias, UUID). A local item already allocated locally
+or inherited is an error. Removing an allocation keeps its item and other
+allocations; if live boundary compatibility would break, the transaction MUST
+repair the affected contract explicitly or reject atomically. Artifacts can
+reference the allocation identity independently of the global item.
+
+An exchange carrying a specialized item satisfies a declaration of its base,
+never the reverse. Standard provided/required compatibility uses explicit
+interface ancestry. Identity/ancestry, not matching names, defines conformance.
+The current typed function/component port contracts remain unchanged.
+
+DataAssociation connects Record types without adding fields or transitively
+carrying payload. Each named end has role, multiplicity and navigability. Source
+multiplicity counts source instances per target instance and vice versa; it is
+not a count of model elements. An end's navigable flag permits navigation toward
+that end. Self-associations are valid; if both ends are navigable they require
+distinct nonempty roles. Empty roles and duplicate role contracts warn, whereas
+invalid ranges reject. Composition/shared aggregation, multiple inheritance,
+redefinition, instance literals and arbitrary-precision decimal arithmetic are
+outside this bounded contract.
+
+This complete data contract example is also exercised through the REST API:
+
+.. code-block:: text
+
+    arcadia 4;
+    project engineering_data "Engineering data" {
+        sa_data_type voltage "Voltage" {
+            definition { Real { minimum 0.0; maximum 48.0; }; };
+            default { Real 24.0; }; unit "V";
+        }
+        sa_data_type coordinate "Coordinate" {
+            definition { Record { supertype none; }; };
+            sa_data_field latitude "Latitude" { type_ref Real; }
+        }
+        sa_data_type located "Located asset" {
+            definition { Record { supertype @coordinate; }; };
+            sa_data_field supply "Supply" { type_ref { Named @voltage; }; }
+        }
+        sa_exchange_item navigate "Navigate" {
+            mechanism Operation;
+            sa_data_field origin "Origin" { type_ref { Named @coordinate; }; direction In; }
+            sa_data_field destination "Destination" { type_ref { Named @coordinate; }; direction In; }
+            sa_data_field accepted "Accepted" { type_ref Boolean; direction Return; }
+        }
+        sa_interface navigation "Navigation" {}
+        sa_interface monitoring "Monitoring" {}
+        interface_item_allocation nav_operation "Navigation operation" {
+            source @navigation; target @navigate; transmission Synchronous; acquisition Accept;
+        }
+        interface_item_allocation monitor_operation "Monitoring operation" {
+            source @monitoring; target @navigate; transmission Asynchronous; acquisition Read;
+        }
+        data_association location_link "Location relationship" {
+            source @coordinate; target @located;
+            source_role "location"; target_role "assets";
+            source_maximum 1;
+        }
+    }
+
 5. Diagrams, artifacts, and cross-layer traceability
 ----------------------------------------------------
 
@@ -654,7 +804,7 @@ This complete source example places one state from an existing machine:
 
 .. code-block:: text
 
-    arcadia 3;
+    arcadia 4;
     project operating "Operating modes" {
         oa_entity crew "Crew" {}
         oa_state_machine modes "Modes" { parent @crew; }
@@ -875,7 +1025,7 @@ sets and enum constructors are closed by sections 3–5.
 
 .. code-block:: text
 
-    document    = "arcadia" "3" ";" project
+    document    = "arcadia" "4" ";" project
     project     = "project" alias string identity? "{" project-entry* "}"
     project-entry = "description" string ";" | declaration
     declaration = element
@@ -917,18 +1067,18 @@ Ordinary string fields require quoted strings and cannot contain an unquoted
 constructor records. Typed values MUST NOT nest more than 64 levels; malformed
 deeply nested drafts produce a diagnostic instead of exhausting the parser stack.
 
-6.1. Modular source3 projects
+6.1. Modular source projects
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 The pure ``source_project`` core API compiles a virtual source bundle, containing
 ``entry`` and a ``files`` map from logical paths to UTF-8 strings. Its entry is a
-root-level ``.arcadia`` file with the source3 project header above. A project entry
+root-level ``.arcadia`` file with the source4 project header above. A project entry
 MAY also be ``include string ";"``. An included fragment consists only of ordinary
 declarations and include statements; it MUST NOT contain another version header,
 project wrapper, or project description. Includes MUST occur at document level,
 never inside an element body or typed field value. For example::
 
-    arcadia 3;
+    arcadia 4;
     project rescue "Rescue" id "00000000-0000-4000-8000-000000000001" {
         description "Operational model";
         include "oa/participants.arcadia";
@@ -994,9 +1144,12 @@ spans never refer to concatenated or synthetic source.
 
 Single-string parse/lint APIs remain filesystem-free and reject include
 statements with an explicit source-bundle requirement. V1/V2 single-file imports
-continue through their existing migration readers and reports. Modular projects
-require source3; migration is performed before modular export, not independently
-on included fragments. Modular source does not change the semantic JSON schema.
+continue through explicit migration. Modular projects accept source3 and source4;
+source3 is upgraded as one project, preserving file paths, includes, nested
+ownership and comments through targeted token edits. Generated allocation
+declarations stay in their interface's existing file. Individual include fragments
+are not independently migrated. Missing UUIDs stabilize before migration so new
+allocation identities are deterministic on subsequent compilation.
 
 6.2. Nested element ownership
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -1228,10 +1381,35 @@ explicitly before Apply succeeds.
 
 Loading detects JSON from its first non-whitespace ``{``; other input is parsed
 as Arcadia source regardless of filename. Saving an ``.arcadia`` suffix uses
-the accepted V3 source bundle (``arcadia 3``); other suffixes use V3 JSON. Both forms preserve the same
+the accepted V4 source bundle (``arcadia 4``); other suffixes use V4 JSON. Both forms preserve the same
 complete model. JSON rejects unknown fields throughout typed payloads,
 duplicate maps, unsupported versions, and structural errors.
 
+The schema-3 reader freezes the changed data payloads and old constructor catalog;
+schema-4 fields/constructors MUST NOT be silently accepted as schema 3. Existing
+typed PortContract records remain unchanged. Information/Material/Energy retain
+their nature with Unspecified mechanism; old Event becomes Information with Event
+mechanism. Old bounds/defaults/supertypes are absent, String length is 0..unbounded,
+and fields retain type/multiplicity/unit with Unspecified direction, Value storage
+and no explicit default. No physical mechanism or direction is guessed.
+
+Each old Interface.items entry generates one InterfaceItemAllocation with its
+original zero-based order and Unspecified protocols. UUID v5 uses the project UUID
+namespace and name ``v2-interface-item:<interface-uuid>:<item-uuid>`` consistently
+for schema-2 and schema-3 inputs. Aliases are deterministic and collision-free;
+collision with any existing identity is an error rather than an overwrite. Existing
+interface/item identities remain intact. Old nested message values acquire a path
+only when exactly one valid path exists. Ambiguous paths, including recursive
+reuse, retain the old annotation with a located warning rather than guessing.
+
+Migration reports additionally contain ``semantic_defaults`` (object, field,
+description), ``generated_interface_allocations`` (id, source, target, alias,
+order) and ``unresolved_message_values`` (occurrence, field, reason). These new
+report arrays default empty when reading older responses. Version 1 normalizes
+through V2, then the typed-port V3 adapter and this shared V4 data upgrade. Original
+V2 and real V3 fixtures remain unchanged compatibility inputs. Current exports
+always use version 4; reading/migrating alone never saves or changes revisions.
+
 Native source Save All checks the original contents of observed dependencies
 before staging writes. Existing target files are replaced individually and the
 entry is installed last. A transaction journal records originals and intended
@@ -1256,7 +1434,7 @@ defaults without changing any prior identity, name, reference, order, geometry,
 artifact or trace field. The original V2 golden source/JSON in
 ``crates/oa-core/tests/fixtures`` MUST NOT be regenerated from the newer demo;
 they verify that every old persistent field survives. Canonical output writes
-the added defaults through the explicit schema-3 migration.
+the added defaults through the explicit schema-3 and schema-4 migrations.
 
 The V2 adapter preserves every original identity, alias, prose, parent,
 placement, trace and artifact. Generic ports become Legacy with their original
@@ -1275,14 +1453,14 @@ IDs, required resolutions, generated endpoint IDs/aliases/owners, and boundary
 orientation defaults. REST import, replacement, candidate validation and
 project snapshots surface the same report. Opening an old server envelope
 preserves its resource ID/revision and bytes; the next explicit edit writes
-schema 3. Model/source exports contain only the canonical model. Migration
+schema 4. Model/source exports contain only the canonical model. Migration
 reports are transient response metadata, not persisted model facts.
 
 Admission checks account for generated objects and their references before
-construction. Both the old V2 contract and the resulting V3 graph are
+construction. Both the old V2 contract and the resulting V4 graph are
 validated. Migration itself does not save or increase a stored revision.
 Canonical second saves are stable. V1 normalization produces the frozen V2
-model and then uses this same V2-to-3 stage.
+model and then uses these same V2-to-3-to-4 stages.
 
 Save validates a snapshot, writes a sibling temporary file, flushes it, and
 atomically replaces the destination. Failed serialization or validation must
@@ -1301,7 +1479,7 @@ step becomes a distinct occurrence element owned by its process. Generated
 diagram, node, and occurrence identities use UUID v5 with the old Project ID
 as namespace and deterministic role/index names. If such an identity collides,
 a deterministic suffix is tried. Reimporting the same fully identified V1
-input therefore yields the same V3 result, including repeated occurrences.
+input therefore yields the same V4 result, including repeated occurrences.
 No System or SA content is invented during an OA-only migration.
 
 V1 source aliases are retained when valid in V2. JSON imports generate the old
@@ -1309,7 +1487,7 @@ canonical aliases deterministically. Any alias newly reserved in V2 is changed
 by a deterministic suffix; aliases were not persistent V1 semantic data.
 A project alias and aliases for new records are generated without colliding
 with retained aliases. The existing bundled V1 text and JSON migrate to equal
-V3 projects. Future versions remain errors until a separate migration exists.
+V4 projects. Future versions remain errors until a separate migration exists.
 
 10. Complete authoring example
 ------------------------------
@@ -1319,7 +1497,7 @@ the assigned UUIDs explicitly.
 
 .. code-block:: text
 
-    arcadia 3;
+    arcadia 4;
     project harbor "Harbor assistance" {
         oa_entity control "Harbor control" {}
         oa_activity assess "Assess assistance" {}