ARCADIA-Studio/harness/release-semantic-batch-foundation-design.md

# Multi-source semantic review foundation: concrete design

Read-only design against the current DATA working tree, after keyed/ordered review
and workflow recovery. This adds implementation detail beyond
`release-semantic-remaining-plan.md`; no production edits, Cargo run or acceptance
claim accompanies it.

## Current seams and two constraints

`Workflow::Semantic(Box<semantic_review::State>)` delegates view/update/command and
recovery to State. App's dedicated semantic handler applies its single prepared
Command batch through Document. Keep that external interface and atomic apply path.
Current State contains one anchor/trace/direction, decisions/resets, references,
scalar/group/keyed/order drafts, and one computed plan. `load_trace` clears drafts;
`request` creates one source/mapping/review and at most one complete target candidate.
Family adapters compose unvalidated typed records before the core validates them.
They need not produce an individually valid Project to retain or edit a row.

The core already accepts multiple review requests derived from one frozen initial
Project, chosen trace per source, a shared canonical OA→SA reference map and one
complete `SemanticRecord` candidate per target ID. It rejects duplicate sources,
duplicate candidate records, actual-slot disagreement and unselected candidate
changes. Two constraints matter to the GUI design:

1. **The core cannot see two independently authored Candidate values once the GUI
   has collapsed them into one record.** If row A authors minimum 5 and row B authors
   minimum 7, both Candidate resolutions read whichever minimum the single candidate
   contains. The GUI must compare the two authored intentions before generating that
   record. Last-row-wins deduplication would silently erase user intent.
2. **Convergence is directionally constrained today.** Two OA sources can map to one
   SA target. Two SA sources cannot map to one OA target under the canonical
   `BTreeMap<OA,SA>` contract: `review.rs::build_review` inserts `(targetOA,sourceSA)`
   for each reverse transition, and rejects the second differing value as
   “Transition mapping disagrees with frozen correspondence.” Reverse inversion also
   rejects ambiguous maps. Support reverse batches with representable mappings;
   retain/display the core conflict for reverse convergence. Do not silently drop
   mappings or promise symmetric many-to-one support. A future relaxation requires
   a separate demonstrated core contract change, outside this foundation.

## Recommended structure

Keep public `State` as the session facade. Extract its current row-specific state
into `ReviewRow`, with an explicit `RowAddress { source, target, trace }` and stable
draft identity. Store a row bank plus included flags, active row and display order.
Switching the active row changes focus only. Excluding a row removes its request
contribution while retaining all raw drafts. Changing its mapping/selected trace
opens or restores the draft for that exact address; it must not repurpose existing
raw values under a new identity. Only one included row per source is admissible;
multiple historical traces for the same pair are deliberate alternatives, not two
review requests for the same source.

Session owns direction, one frozen accepted Project/fingerprint, canonical reference
pairs, one merged preview, accepted impacts and errors. An optional draft bank per
direction retains work when switching direction; require explicit address selection
where the reversed mapping is ambiguous. No direction switch commits model changes.
Check the supplied current Project against the session's accepted fingerprint
before planning and Apply; a changed accepted model leaves drafts intact and reports
staleness. All proposals and observed history read the same frozen base.

Split Event into session actions and `Row { address, event: RowEvent }`. Tag row
events with their originating draft identity so delayed text-editor messages cannot
edit whichever row happens to be selected after a switch. Existing constructor,
keyed, order and numeric form code remains the row editor. Project contexts can
show the provisional combined destination, but never overwrite authored raw values.
Raw incomplete numbers or conflicting constructor alternatives block inclusion;
an excluded invalid row remains recoverable without blocking the included batch.

## Candidate composition without sequential Apply

Add a closed `compose.rs` adapter, extracting reusable composition from today's
`combined_value`. Its product is target contributions with row provenance, not
Commands. For every included row:

1. Materialize only selected raw Candidate scopes and existing typed Take payloads
   against the frozen base and shared explicit references. Do not require its own
   final Project to validate: e.g. one row proposes a minimum above the old maximum,
   while another proposes a sufficient new maximum. Validate their joint result.
2. Produce typed desired observations indexed by actual destination slot:
   `(target, scalar selector)`; `(target, collection, destination member ID)`;
   exact literal; full Annotation `(path,field)`; Binding referenced ID; and Order
   collection. Use the destination identity, never the full source/target pair as
   an agreement key. A row with no source identity and Take contributes frozen
   target preservation, distinct from explicit absent-source removal. Keep is also
   an explicit frozen-value constraint and cannot be dropped merely because its
   JSON diff is empty.
3. Compare all authored observations for the same actual slot. Equal intentions
   can share the final slot; differing ones retain both drafts and block preview
   with source names, facet/key and each proposed value. Offer explicit actions to
   edit a row, leave its decision unselected, or exclude it. Do not automatically
   synchronize raw Candidate drafts to whichever row was edited last.
4. Constructor proposals have coverage over their descendant slots, including
   slots introduced by another row and retired NotApplicable branches. Compare that
   coverage against every other proposal. Preserve the core's first-equal-kind-only
   acknowledgment exception; a replacing group remains a complete scoped proposal.
   Never apply the same-row “untouched candidate field” convenience across sources
   to silently accept another source's changed value. Existing `constructors::roots`,
   `contains`, typed `capture_facet` and `semantic_facets` are the seams for this closed
   adapter. Family-specific equality/reference semantics remain explicit.
5. Assemble one target record from the agreed contributions: constructor groups
   first, independent scalar/keyed changes next, complete order last. Preserve
   surviving unordered row positions and use canonical destination-key order for
   genuinely new rows. Include a complete candidate only where a Candidate decision
   requires one; its record must also contain the agreed Keep/Take effects on that
   target. Finally call `plan_semantic_transition` once with all source mappings,
   trace selections, row resolutions/resets, shared references and unique candidates.
   The core still controls validation, exact scope/retirement, induced impacts,
   preparation fingerprint and the only executable Command batch.

The session's disagreement checks protect independent raw intent; they do not
replace the core's agreement check. Compare GUI requests/results to direct core
calls and retain any core conflict that the GUI did not anticipate. Do not merge
whole records, concatenate per-row candidate vectors, execute previews sequentially,
or use successful per-row plans as a prerequisite for joint validity.

Ordered collections require particular care. Core `build_review` validates each
row's explicit Order against that row's expected surviving membership before its
final cross-source merge. An Order containing a member introduced only by another
row can therefore require that membership to be explicitly reviewed in the order's
row as well (a destination-only Candidate member address can express agreement).
Expose the involved membership/Order decisions for explicit repair; do not secretly
add acknowledgments. Likewise retain core “Contradictory anchored orders” and
“Combined ordered membership needs an explicit final target order” conflicts.
Prove any convenience to share a final order with a direct core request before
offering it. This is a contract edge to test, not permission to invent a merge order.

## Alternatives considered

| Approach | Assessment |
| --- | --- |
| Wrap several complete current States and concatenate requests/candidates | Fast for distinct targets, but duplicates candidates for convergent targets and cannot preserve conflicting raw Candidate values when deduplicated. Reject as the final architecture. |
| Give all rows one mutable target Draft | Avoids duplicate records but makes one source edit silently change another row's authored proposal. Useful only as an explicitly accepted resolution surface, not draft storage. |
| Independent row drafts plus provenance-bearing typed contributions | Recommended. Reuses existing closed adapters, protects Keep and raw conflicts, admits jointly valid values, and leaves the core as the final planner. |

## Files, dependencies and bounded implementation order

One owner should edit `semantic_review.rs`, `semantic_review/**` for this foundation:
new `batch.rs` for row selection/session orchestration, `compose.rs` for closed
contributions, `batch_tests.rs` for core-oracle cases, and focused changes in
`view.rs`, `recovery.rs` and the family adapters. Keep `Workflow::Semantic(State)`
and its public methods intact so App/Workflow recovery wrappers need no changes.
Coordinate any unavoidable wrapper adjustment with root; do not touch App source
writeback, Save handling or native input. Core/source/API production edits are not
part of this design.

Implement incrementally inside this structure: row bank and recovery with distinct
targets; convergent scalar/keyed agreement and single candidate assembly; then
constructor/order overlap and full same-target edge tests. The first checkpoint
must visibly identify unsupported convergence instead of silently serializing it.
Do not call the foundation complete until same-target disagreement and agreement
cases pass. Attachments/artifact creation remain subsequent adapters.

Recovery v2 stores one base identity/fingerprint, direction and shared references,
row addresses/inclusion/order, all row scalar/group/keyed/order raw drafts, row
resets, active identity and exact accepted impacts. No proposed projects/commands
are persisted. Restore checks the base once, rejects duplicate included source
addresses and duplicate draft keys, restores excluded invalid rows, then recomposes
and replans. Decode existing v1 snapshots as a one-row session; never silently drop
a supported old workflow. Apply limits across all rows rather than independently
allowing each row the full model budget.

## Engineering workflow and decisive oracles

Two OA operating modes review their constraints against one SA Integer data type.
Base range is 0..100. Normal operation row proposes minimum 120; emergency operation
row proposes maximum 150. Retain both source rows and allow the jointly valid
120..150 result although the first row alone is invalid. If emergency also proposes
minimum 130, show both authored minima and block. Leaving that conflicting minimum
unselected restores the earlier 120 draft and the valid joint result. Exclude and
restore the emergency row, recover the workflow, then apply once and undo once.
Permuting display/review order must never change the outcome or erase a draft.

Add a second same-target fixture where one row Takes an Integer constructor while
another Keeps a Real bound; both source orders must reject retirement of the kept
fact. Ordered startup fixtures must preserve the local self-test and expose explicit
membership/order repair when source anchors disagree. Reverse distinct-target
batches must succeed; reverse convergence must retain the current core refusal.

Exact existing oracles in `crates/oa-core/tests/r3_semantic.rs`:

- `independent_bounds_keep_take_and_stale_prepare` for scoped bounds/stale plans;
  extend its pattern to the jointly valid two-source min/max case above.
- `constructor_retirement_cannot_override_other_sources_kept_bound_in_either_order`.
- `constructor_union_conflicts_with_another_sources_new_member_in_either_order`.
- `combined_ordered_membership_commutes_or_requires_agreed_anchors`.
- `semantic_batch_is_one_undo_and_replanning_acknowledged_values_is_noop`.
- `explicit_trace_selection_keeps_independent_trace_baselines`.
- `same_target_member_keys_cannot_last_write_win` validates historical slot uniqueness;
  add live GUI candidate disagreement tests rather than treating that baseline-only
  test as sufficient proof of batch candidate composition.

Add source/comment/history equality through App Apply, Undo/Redo and reopen;
stale receipt/model rejection; exclusion and recovery of invalid drafts; explicit
per-row reset with two histories; equal versus conflicting exact Annotation/Binding
keys; kept slots versus mutations; first-equal constructor exception; and duplicate
source/trace choices. Native acceptance must inspect row switching and retained
text, conflicting values, the one combined preview and one transaction at normal
and minimum window sizes.