ARCADIA-Studio/harness/release-semantic-batch-foundation-design.md

Multi-source semantic review foundation: concrete design

Read-only design against the current DATA working tree, after keyed/ordered review and workflow recovery. This adds implementation detail beyond release-semantic-remaining-plan.md; no production edits, Cargo run or acceptance claim accompanies it.

Current seams and two constraints

Workflow::Semantic(Box<semantic_review::State>) delegates view/update/command and recovery to State. App's dedicated semantic handler applies its single prepared Command batch through Document. Keep that external interface and atomic apply path. Current State contains one anchor/trace/direction, decisions/resets, references, scalar/group/keyed/order drafts, and one computed plan. load_trace clears drafts; request creates one source/mapping/review and at most one complete target candidate. Family adapters compose unvalidated typed records before the core validates them. They need not produce an individually valid Project to retain or edit a row.

The core already accepts multiple review requests derived from one frozen initial Project, chosen trace per source, a shared canonical OA→SA reference map and one complete SemanticRecord candidate per target ID. It rejects duplicate sources, duplicate candidate records, actual-slot disagreement and unselected candidate changes. Two constraints matter to the GUI design:

1. The core cannot see two independently authored Candidate values once the GUI has collapsed them into one record. If row A authors minimum 5 and row B authors minimum 7, both Candidate resolutions read whichever minimum the single candidate contains. The GUI must compare the two authored intentions before generating that record. Last-row-wins deduplication would silently erase user intent.
2. Convergence is directionally constrained today. Two OA sources can map to one SA target. Two SA sources cannot map to one OA target under the canonical BTreeMap<OA,SA> contract: review.rs::build_review inserts (targetOA,sourceSA) for each reverse transition, and rejects the second differing value as “Transition mapping disagrees with frozen correspondence.” Reverse inversion also rejects ambiguous maps. Support reverse batches with representable mappings; retain/display the core conflict for reverse convergence. Do not silently drop mappings or promise symmetric many-to-one support. A future relaxation requires a separate demonstrated core contract change, outside this foundation.

Recommended structure

Keep public State as the session facade. Extract its current row-specific state into ReviewRow, with an explicit RowAddress { source, target, trace } and stable draft identity. Store a row bank plus included flags, active row and display order. Switching the active row changes focus only. Excluding a row removes its request contribution while retaining all raw drafts. Changing its mapping/selected trace opens or restores the draft for that exact address; it must not repurpose existing raw values under a new identity. Only one included row per source is admissible; multiple historical traces for the same pair are deliberate alternatives, not two review requests for the same source.

Session owns direction, one frozen accepted Project/fingerprint, canonical reference pairs, one merged preview, accepted impacts and errors. An optional draft bank per direction retains work when switching direction; require explicit address selection where the reversed mapping is ambiguous. No direction switch commits model changes. Check the supplied current Project against the session's accepted fingerprint before planning and Apply; a changed accepted model leaves drafts intact and reports staleness. All proposals and observed history read the same frozen base.

Split Event into session actions and Row { address, event: RowEvent }. Tag row events with their originating draft identity so delayed text-editor messages cannot edit whichever row happens to be selected after a switch. Existing constructor, keyed, order and numeric form code remains the row editor. Project contexts can show the provisional combined destination, but never overwrite authored raw values. Raw incomplete numbers or conflicting constructor alternatives block inclusion; an excluded invalid row remains recoverable without blocking the included batch.

Candidate composition without sequential Apply

Add a closed compose.rs adapter, extracting reusable composition from today's combined_value. Its product is target contributions with row provenance, not Commands. For every included row:

1. Materialize only selected raw Candidate scopes and existing typed Take payloads against the frozen base and shared explicit references. Do not require its own final Project to validate: e.g. one row proposes a minimum above the old maximum, while another proposes a sufficient new maximum. Validate their joint result.
2. Produce typed desired observations indexed by actual destination slot: (target, scalar selector); (target, collection, destination member ID); exact literal; full Annotation (path,field); Binding referenced ID; and Order collection. Use the destination identity, never the full source/target pair as an agreement key. A row with no source identity and Take contributes frozen target preservation, distinct from explicit absent-source removal. Keep is also an explicit frozen-value constraint and cannot be dropped merely because its JSON diff is empty.
3. Compare all authored observations for the same actual slot. Equal intentions can share the final slot; differing ones retain both drafts and block preview with source names, facet/key and each proposed value. Offer explicit actions to edit a row, leave its decision unselected, or exclude it. Do not automatically synchronize raw Candidate drafts to whichever row was edited last.
4. Constructor proposals have coverage over their descendant slots, including slots introduced by another row and retired NotApplicable branches. Compare that coverage against every other proposal. Preserve the core's first-equal-kind-only acknowledgment exception; a replacing group remains a complete scoped proposal. Never apply the same-row “untouched candidate field” convenience across sources to silently accept another source's changed value. Existing constructors::roots, contains, typed capture_facet and semantic_facets are the seams for this closed adapter. Family-specific equality/reference semantics remain explicit.
5. Assemble one target record from the agreed contributions: constructor groups first, independent scalar/keyed changes next, complete order last. Preserve surviving unordered row positions and use canonical destination-key order for genuinely new rows. Include a complete candidate only where a Candidate decision requires one; its record must also contain the agreed Keep/Take effects on that target. Finally call plan_semantic_transition once with all source mappings, trace selections, row resolutions/resets, shared references and unique candidates. The core still controls validation, exact scope/retirement, induced impacts, preparation fingerprint and the only executable Command batch.

The session's disagreement checks protect independent raw intent; they do not replace the core's agreement check. Compare GUI requests/results to direct core calls and retain any core conflict that the GUI did not anticipate. Do not merge whole records, concatenate per-row candidate vectors, execute previews sequentially, or use successful per-row plans as a prerequisite for joint validity.

Ordered collections require particular care. Core build_review validates each row's explicit Order against that row's expected surviving membership before its final cross-source merge. An Order containing a member introduced only by another row can therefore require that membership to be explicitly reviewed in the order's row as well (a destination-only Candidate member address can express agreement). Expose the involved membership/Order decisions for explicit repair; do not secretly add acknowledgments. Likewise retain core “Contradictory anchored orders” and “Combined ordered membership needs an explicit final target order” conflicts. Prove any convenience to share a final order with a direct core request before offering it. This is a contract edge to test, not permission to invent a merge order.

Alternatives considered

Approach	Assessment
Wrap several complete current States and concatenate requests/candidates	Fast for distinct targets, but duplicates candidates for convergent targets and cannot preserve conflicting raw Candidate values when deduplicated. Reject as the final architecture.
Give all rows one mutable target Draft	Avoids duplicate records but makes one source edit silently change another row's authored proposal. Useful only as an explicitly accepted resolution surface, not draft storage.
Independent row drafts plus provenance-bearing typed contributions	Recommended. Reuses existing closed adapters, protects Keep and raw conflicts, admits jointly valid values, and leaves the core as the final planner.

Files, dependencies and bounded implementation order

One owner should edit semantic_review.rs, semantic_review/** for this foundation: new batch.rs for row selection/session orchestration, compose.rs for closed contributions, batch_tests.rs for core-oracle cases, and focused changes in view.rs, recovery.rs and the family adapters. Keep Workflow::Semantic(State) and its public methods intact so App/Workflow recovery wrappers need no changes. Coordinate any unavoidable wrapper adjustment with root; do not touch App source writeback, Save handling or native input. Core/source/API production edits are not part of this design.

Implement incrementally inside this structure: row bank and recovery with distinct targets; convergent scalar/keyed agreement and single candidate assembly; then constructor/order overlap and full same-target edge tests. The first checkpoint must visibly identify unsupported convergence instead of silently serializing it. Do not call the foundation complete until same-target disagreement and agreement cases pass. Attachments/artifact creation remain subsequent adapters.

Recovery v2 stores one base identity/fingerprint, direction and shared references, row addresses/inclusion/order, all row scalar/group/keyed/order raw drafts, row resets, active identity and exact accepted impacts. No proposed projects/commands are persisted. Restore checks the base once, rejects duplicate included source addresses and duplicate draft keys, restores excluded invalid rows, then recomposes and replans. Decode existing v1 snapshots as a one-row session; never silently drop a supported old workflow. Apply limits across all rows rather than independently allowing each row the full model budget.

Engineering workflow and decisive oracles

Two OA operating modes review their constraints against one SA Integer data type. Base range is 0..100. Normal operation row proposes minimum 120; emergency operation row proposes maximum 150. Retain both source rows and allow the jointly valid 120..150 result although the first row alone is invalid. If emergency also proposes minimum 130, show both authored minima and block. Leaving that conflicting minimum unselected restores the earlier 120 draft and the valid joint result. Exclude and restore the emergency row, recover the workflow, then apply once and undo once. Permuting display/review order must never change the outcome or erase a draft.

Add a second same-target fixture where one row Takes an Integer constructor while another Keeps a Real bound; both source orders must reject retirement of the kept fact. Ordered startup fixtures must preserve the local self-test and expose explicit membership/order repair when source anchors disagree. Reverse distinct-target batches must succeed; reverse convergence must retain the current core refusal.

Exact existing oracles in crates/oa-core/tests/r3_semantic.rs:

• independent_bounds_keep_take_and_stale_prepare for scoped bounds/stale plans; extend its pattern to the jointly valid two-source min/max case above.
• constructor_retirement_cannot_override_other_sources_kept_bound_in_either_order.
• constructor_union_conflicts_with_another_sources_new_member_in_either_order.
• combined_ordered_membership_commutes_or_requires_agreed_anchors.
• semantic_batch_is_one_undo_and_replanning_acknowledged_values_is_noop.
• explicit_trace_selection_keeps_independent_trace_baselines.
• same_target_member_keys_cannot_last_write_win validates historical slot uniqueness; add live GUI candidate disagreement tests rather than treating that baseline-only test as sufficient proof of batch candidate composition.

Add source/comment/history equality through App Apply, Undo/Redo and reopen; stale receipt/model rejection; exclusion and recovery of invalid drafts; explicit per-row reset with two histories; equal versus conflicting exact Annotation/Binding keys; kept slots versus mutations; first-equal constructor exception; and duplicate source/trace choices. Native acceptance must inspect row switching and retained text, conflicting values, the one combined preview and one transaction at normal and minimum window sizes.