Files
ARCADIA-Studio/harness/release-analysis-production-integration-plan-20260911.md

127 lines
27 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Production analysis integration: next vertical slice
Codex-956atf /root/requirements_delta, 2026-09-11. Read-only investigation after numeric freeze #201 and review #203; ownership coordinated in Gitea #210. This report is the only new file. Root has declared the numeric module and owns its first Cargo gate; this worker has not compiled, executed, or changed production code. The B/property-base, shared-check, R6/R7 and complete-capture architecture is ratified. This plan stages its implementation in the existing application, not an alternate Project or interpreter.
**Recommended next gate: author the complete response-budget model in real source, admit/persist it through the existing command/source boundary, and evaluate the same property and shared check through core, CLI and REST under explicit ModelBaseline and Case contexts.** Include the reusable two-parameter calculation: otherwise the first example misses the captured-definition and scoped-reference boundaries that determine the architecture. The existing complete UUID fixture is [active.arcadia.proposed](experiments/analysis-baseline-20260911/active.arcadia.proposed); its current suffix and explanatory text correctly say that it is not implemented source. Promote a newly implemented version into a production example/test fixture, leaving that frozen evidence unchanged.
## Existing boundaries and required changes
| Actual code | Consequence for the implementation |
| --- | --- |
| [model.rs:9](../crates/oa-core/src/model.rs#L9), [model.rs:229](../crates/oa-core/src/model.rs#L229), [model.rs:593](../crates/oa-core/src/model.rs#L593), [model.rs:1621](../crates/oa-core/src/model.rs#L1621) | Schema 7, closed Element/SharedBody and five Project UUID maps are authoritative. Add three analysis element kinds and one neutral definition map. ModelBaseline is a selector, not a sixth kind or record. |
| [session.rs:170](../crates/oa-core/src/session.rs#L170) and [store.rs:146](../crates/oa-server/src/store.rs#L146) | Clone/apply/finish deletions/validate and identity correspondence already exist; PreparedProject also updates and recompiles source. Extend these operations instead of making evaluator mutations or a separate save path. |
| [references.rs:17](../crates/oa-core/src/references.rs#L17), [references.rs:206](../crates/oa-core/src/references.rs#L206), [references.rs:392](../crates/oa-core/src/references.rs#L392) | Live reference visitors and cleanup have explicit policies. Expr dependencies, override keys and definitions need typed visitors; cleanup must not prune an operand or override and change its meaning. |
| [language.rs:273](../crates/oa-core/src/language.rs#L273), [language.rs:761](../crates/oa-core/src/language.rs#L761), [language.rs:811](../crates/oa-core/src/language.rs#L811), [language.rs:1297](../crates/oa-core/src/language.rs#L1297) | Declarations and source values are finite. Current references/enums are recognized by field-name lists; number tokens become serde_json::Value; typed values stop at depth 64. These mechanisms alone cannot represent exact R7 literals, scoped parameters, call argument keys or override references. |
| [source_project.rs:92](../crates/oa-core/src/source_project.rs#L92), [source_project.rs:286](../crates/oa-core/src/source_project.rs#L286), [source_project.rs:780](../crates/oa-core/src/source_project.rs#L780), [source_project.rs:1662](../crates/oa-core/src/source_project.rs#L1662) | Compilation stabilizes identities and indexes returned bytes. Updating preserves token trivia and file ownership. Add definition record classification, scoped parameter identity/token locations and typed expression locations; add definitions to canonical module ownership/order. |
| [persistence.rs:130](../crates/oa-core/src/persistence.rs#L130) and [source_migration.rs:4](../crates/oa-core/src/source_project/source_migration.rs#L4) | Existing readers migrate versions explicitly. A new required Project field needs a frozen V7 envelope reader/adapter, not a serde default that makes omission valid in the new schema. Source migration must retain prior comments/modules/history and change only required tokens. |
| [semantic/types.rs:81](../crates/oa-core/src/semantic/types.rs#L81), [semantic/historical.rs:6](../crates/oa-core/src/semantic/historical.rs#L6), [history_fields.rs:29](../crates/oa-desktop/src/forms/history_fields.rs#L29) | OA/SA review is a closed typed facet vocabulary; historical references have a separate walker. The desktop history editor reads docs/openapi.json shapes. Snapshot undo already retains the actual Project/source; reviewed semantic history needs deliberate analysis facets before claiming parity. |
| [transitions.rs:203](../crates/oa-core/src/transitions.rs#L203), [transitions.rs:889](../crates/oa-core/src/transitions.rs#L889), [validation.rs:2624](../crates/oa-core/src/validation.rs#L2624) | Merely adding a same-kind mapping can clone an analysis body and remap the wrong reference domain. Manual traces already admit arbitrary semantic endpoints. Explicit analysis selection/remapping and review guards are needed. |
| [routes.rs:29](../crates/oa-server/src/routes.rs#L29), [store.rs:20](../crates/oa-server/src/store.rs#L20), [lib.rs:192](../crates/oa-server/src/lib.rs#L192), [oa-cli/lib.rs:25](../crates/oa-cli/src/lib.rs#L25) | REST has worker slots, bounded bodies, accepted snapshots and ETags; CLI has actual Check/Stabilize/RenameAlias. Add evaluation adapters there. No GUI initialization, ad hoc local web service or second executable is needed. |
## Persistent types and trusted runtime boundary
Place the common persistent types under new oa-core analysis/types.rs and use them directly from model.rs, source and API schemas. All records/enums remain closed, with duplicate UUID-map keys rejected by an equivalent of unique_id_map.
~~~rust
// Existing Project, not a wrapper model:
definitions: BTreeMap<Id, Definition>
struct Definition { id: Id, alias: String, name: String,
description: String, body: DefinitionBody }
enum DefinitionBody { QuantityKind(QuantityKindDefinition),
Unit(UnitDefinition), Calculation(CalculationDefinition) }
enum Binding { Unbound, Expression(Expr) }
enum AnalysisSubject { Element(Id), Relation(Id) }
enum BaselineBase { ModelBaseline }
enum EvaluationContext { ModelBaseline, Case(Id) }
// SharedBody variants; existing Element header supplies identity and layer:
EngineeringProperty { subject: AnalysisSubject, value_type: AnalysisType,
display_unit: Option<DefinitionRef>, base: Binding }
AnalysisCase { subject: Option<AnalysisSubject>, base: BaselineBase,
overrides: BTreeMap<Id, Binding> }
AnalysisCheck { subject: Option<AnalysisSubject>,
documented_constraint: Option<Id>, predicate: Binding }
~~~
Use the ratified Expr tree directly: Boolean, Number(Exact), Quantity(magnitude/unit), Property, Parameter, List, Unary, Binary, If, Call, Sum with explicit initial, Power with integer exponent, and AsKind. Do not accept an uninterpreted formula string or serialize an experimental evaluator's AST. AnalysisType remains scalar Boolean/Number/Quantity or a finite flat list of one scalar type. Calculation parameters have stable scoped ParameterIds, names and AnalysisTypes; arguments map those IDs, and the body is a Binding. Calls are pure and cannot capture architectural properties. UnitDefinition retains exact scale/offset, kind reference and display symbol; quantity kinds retain checked dimensions, linear/affine algebra and the explicit difference-kind reference.
Existing DataType/DataField unit strings, DataValue defaults, Constraint.expression, message annotation expressions, scenario conditions and state/event text retain their current meaning and wire/source bytes. Migration does not resolve their words into new units, parse them as Expr, or use them as property defaults. A formal check may explicitly link a descriptive Constraint, but only its own typed predicate is evaluated. Any future adapter between exact analysis values and existing i64/f64 payload values is a separate explicit checked operation.
All three element kinds are layer roots. Required base/type/subject fields are genuinely required: current source factory-default merging must not invent a property subject, Number type or base. Use CreateElement for complete typed records and a dedicated source branch. Generic Add/AddElement should reject kinds requiring an explicit contract until their authoring UI supplies one; audit ElementBody::new/make_element consumers so no new Kind option reaches an unwrap with no valid factory value. Case/check can explicitly author a subjectless record, and Unbound remains a valid incomplete binding.
Extend Project.identities/contains/named/fresh_alias with local definitions; layer_of(definition) stays None. Parameters are unique only inside their calculation and never enter that global identity set. Local definition reidentification changes actual Local references; parameter rename preserves ParameterId and edits parameter declaration/use and named-call tokens. Never apply a generic UUID visitor to parameter IDs, imported members, quantity coordinates or frozen historical capsules.
The first example needs only local definitions. Recommended staging decision for root: admit a closed local definition environment first, using DefinitionRef::Local(Id); add the actual Imported form with the coordinated production package environment, not a placeholder resolver that accepts missing pins. This is a local first gate, not fulfillment of reusable-library or nonempty pinned-freeze requirements. If package production types land first, share their exact DefinitionRef and reject absent/corrupt imported environments at admission. Do not create a competing package format here. Root must coordinate the next schema number with that integration; 8 is available in inspected source but remains a proposal until allocated.
The resolver constructs a private ResolvedAnalysis view over the accepted Project and a bounded table of verified kinds/units/calculations. Numeric K should be a cheap private KindHandle index, not a cloned definition graph. Store exact declared refs separately for public output/provenance. No user can construct a trusted resolved environment merely by filling public CompiledSourceProject fields; its constructor performs shared admission, or consumes an owning component's validated capture. Typed models and source span indexes remain separate: Expr nodes use stable owner/field/child paths for diagnostics, with source locations derived from the compiled bytes.
## Mandatory ordering inside the core
1. Extract pure numeric result-type functions from NumericValue into NumericType: additive, multiplicative, unary, power, comparison compatibility and AsKind. Runtime NumericValue delegates to those same functions. Boolean/list rules wrap this algebra in analysis/typecheck.rs. A valid Number/Number type must not require evaluating a dummy zero divisor. Anonymous zero dimensions normalize to Number; declared result boundaries still require explicit nominal kinds.
2. Resolve definitions and check their kinds, dimensions, coordinate basis and pure call graph. Enforce exact named parameter sets, result types and all expression branches, including unused If branches and short-circuit operands. Check every property/check/case reference against its actual layer and record kind. Same-dimensional named kinds remain distinct without verified nominal equivalence.
3. Build the complete baseline dependency graph and every stored case's effective graph. An absent override uses the property's base; present Unbound masks it. An inherited formula resolves all subsequent Property reads in the originally requested case. Check baseline-induced case cycles before any candidate is accepted. Do not calculate values during model admission merely to find types or cycles.
4. Implement the sole evaluator over those admitted Expr nodes. Pass one WorkMeter through every requested target, expression occurrence, call entry, property binding and list element. Use fixed target and argument order, the ratified short-circuit rules, and classified Value/Missing/Failed outcomes; checks map Boolean true/false to Satisfied/Violated and other outcomes to Unresolved. Terminal work exhaustion ends traversal and marks causes incomplete. Initially perform the same logical work without a node-value cache; no optimization is required for first usefulness.
5. Return exact canonical coordinates, declared type anchoring, requested context/targets/profile/effective limits, capture identity, work usage and located Base/CaseOverride provenance for every consulted binding. No result is written back into a base, case, DataValue or history as an evaluator side effect.
Add CreateDefinition/EditDefinition and the three ratified narrow commands SetAnalysisBase, SetAnalysisOverride and ClearAnalysisOverride. Existing Rename/Describe/SetAlias/Reidentify/Delete must recognize definitions. Clear removes one override key and is an exact no-op if absent; it is not a persisted Binding variant. Existing CreateElement/EditElement/Batch already cover shared checks and complete property/case records. Deleting a referenced property/definition/subject must leave the candidate invalid unless that same batch explicitly repairs or removes its analysis users. Reuse final-candidate validation instead of silently applying the existing optional-reference/list cleanup to these fields.
## Source authoring, persistence and public calls
Add one typed adapter, language/analysis_source.rs, invoked only for the new declarations and typed analysis fields. It lowers directly into the real Expr/Definition/SharedBody types and writes those same types. Retain existing declaration headers, explicit IDs, include handling, stable file ownership and ordinary comments. Use the complete fixture's readable forms: quantity_kind/unit/calculation; oa_property/oa_analysis/oa_check (and SA counterparts); base/predicate infix expressions, exact ratio atoms, explicit unit suffixes, scoped $parameters and named call arguments. These are proposed production grammar additions, not descriptions of implemented source7.
The adapter must preserve raw numeric lexemes for Exact::parse_literal; never route them through serde_json::Number. Negative literal nodes print signed ratio, while unary minus stays a distinct Expr node. Scoped parameter declarations/argument keys and override keys need typed source syntax/identity handling; adding their field names to reference_field is insufficient. Extend tolerant highlighting and actual edit_declarations/SyntaxUpdate parsing too, not just compile/format. On alias-only edits, preserve expression spacing/comments and decimal spelling while changing only affected tokens. Add nested expression source locations so a fuel failure can identify the actual call or operator rather than only the enclosing base field.
Migration: freeze a V7 top-level DTO with the old required maps, adapt it to an empty definitions catalog and current schema, and reject new analysis constructors/fields in all earlier versions, including explicit empty/null attempts. Do not default away missing definitions or Binding fields in native new-schema input. The V5/V6 decode branches also currently deserialize through Project; route them through an appropriate frozen top-level DTO before their existing migration logic, and update all older-adapter Project constructors. The existing source upgrade should preserve old semantic baseline bytes and authored modules; ordinary schema7 models acquire no analysis assumptions.
Use new pure read endpoints POST /api/v1/projects/{id}/analysis/evaluate and /analysis/freeze, plus a private-capture replay endpoint when capture support is complete. The request must carry context, targets, numeric_profile and work_limits; the hosted envelope also requires an exact requested revision. Reuse the existing revision-check and captured ETag response convention. A valid false/Missing/Failed evaluation returns HTTP 200 with its typed result; malformed request is 400, unavailable revision follows the existing conflict policy, and invalid semantic target/type is 422. Evaluation does not increment revision or keep the store mutex while doing arithmetic. Existing store.get already clones an accepted StoredProject for worker work.
Add a CLI evaluate operation consuming a request JSON file with the identical core request, rather than a second formula language or implicit current context. Existing source Check/Stabilize/RenameAlias remain the authoring workflow. If disk source still needs identity stabilization/migration, first evaluation should require the explicit existing stabilization/save step so repeated evaluations do not invent new local identities. A CLI capture records its actual source/document digest; it must not invent a hosted ETag or a persisted local revision counter. The native Document already has accepted snapshot/revision/history and can later call the same evaluator without applying a pending editor.
## Admission, capture, history and transition boundaries
Retain current transport caps: REST 8 MiB model/body, 8 MiB serialized source bundle, 5,000 graph objects, 20,000 records and 50,000 ordered references; CLI currently allows 64 MiB source with its separate graph/record caps. Count definitions and parameters in record budgets, and count Expr nodes, argument/override entries and list members across the entire admitted document, not separately per included file or target. The ratified expression limits are depth 128, 65,536 stored nodes per closure, list length 1,024 and call depth 32. Also impose a deterministic aggregate admission-work budget for traversing all case-effective graphs, so many small cases cannot multiply validation work without charge. Its exact ceiling requires root's bounded measurements; keep that admission policy separate from per-evaluation WorkLimits.
Raw byte bounds must precede JSON/String allocation; decoded exact components are capped before BigInt allocation. Native new-schema JSON, command bodies and source need the same aggregate literal-admission meter; derived per-Exact Serde alone does not supply it. The existing source value-depth 64 and JSON decoding recursion behavior must be reconciled with a logical Expr depth of 128 through bounded typed readers and explicit positive boundary tests. Do not silently reduce the ratified profile or disable decoder limits without a replacement bound. Effective evaluation limits/profile are recorded in request/result/capture identity; the numeric module's provisional ceilings are not calibrated timing guarantees.
There is no production PortableDocument type in the inspected core. Add a shared capture type containing the actual Project plus exact SourceBundle, and later the common package environment's pins and exact archive bytes. Reopening verifies captured source compilation equals the captured Project, exact capsule digest, full local-definition inventory and redundant effective-binding/check manifests. This is a private immutable namespace and never goes through live Reidentify. Whole-document local capture must include unused definitions and other cases; a result or reachable-value subset is insufficient. Bound encoded envelope, decoded bytes, records and manifest entries before allocation, with any route-specific larger freeze ceiling explicit. Nonempty pinned offline replay/tamper rejection is a separate mandatory gate once the real library environment exists; an empty local example cannot prove it.
Snapshot undo/redo in Session and desktop Document automatically includes new typed fields once source updates are correct. OA/SA reviewed history is different: add explicit typed Property/Case/Check facets and historical Expr/reference walkers together with their schema descriptors when analysis transitions are implemented. Do not reuse live resolution for deleted historical refs, or reinterpret existing ConstraintFacet::Expression text. Until that support lands, reject analysis requests in semantic review/transition entry points with an explicit capability error; adding Kind mappings alone is unsafe, and manual traces must not produce a misleading header-only semantic-success result. Ordinary descriptive manual trace storage can remain descriptive. The subsequent transition slice needs explicit AnalysisSelection, whole selected cases, remapped subject/property/check IDs, unchanged neutral definition identities and explicit inclusion/omission; it must not inverse-pull every unrelated case mentioning a property.
## Proposed file ownership and sequence
These are proposed implementation leases, not permission for concurrent edits yet. Root retains manifests, lib.rs declarations, shared integration hooks and Cargo; each existing file has one writer at a time.
| Order / owner role | Concrete files | Exit evidence |
| --- | --- | --- |
| 0. Root + numeric owner, renewed lease | analysis_numeric/quantity.rs and tests.rs only after current freeze gate | Pure type helpers agree with runtime R6; terminal-wrapper controls; existing numeric tests stay passing. |
| 1. Core contract owner | new analysis.rs, analysis/types.rs, analysis/references.rs, analysis/admission.rs; model.rs, session.rs, references.rs, validation.rs, persistence.rs; new migration_v7.rs | Actual Project JSON round-trip, unique IDs/keys, explicit bases, transactions/deletion, old-schema rejection/migration. Root applies lib.rs hook. |
| 2. Source owner | language.rs, new language/analysis_source.rs, source_project.rs, source_project/source_structure.rs, source_project/source_migration.rs; new crates/oa-core/tests/analysis_source.rs; new examples/analysis/response-budget/ | Entire real fixture, fresh-ID stabilization, parameter rename, aliases/includes/comments, R7 literal AST equality, exact command-source correspondence. |
| 3. Evaluator owner after types fixed | new analysis/resolve.rs, analysis/typecheck.rs, analysis/evaluate.rs, analysis/tests.rs; shared analysis.rs glue stays with contract owner | Actual shared Expr traversal, B effective graphs, R6/R7 outcomes, actual C1 exponential-call witness bounded across all targets, origins and incomplete causes. |
| 4. API/CLI owner after core callable | routes.rs, new routes/analysis.rs, minimal store.rs snapshot hook if needed; oa-cli/src/lib.rs/main.rs and CLI tests; new oa-server/tests/http/analysis.rs plus test registration | Same accepted source/JSON report via CLI and HTTP; revision races, no save on Evaluate, body/admission limits. |
| 4. Same API/schema owner | docs/openapi.json, oa-server/tests/semantic_schema.py, docs/api.rst, docs/language.rst, docs/source-cli.md and generated HTML | Closed schemas/required fields/commands/request/outcome match actual Serde; schema fixtures and documented grammar compile. Coordinate peer language-doc ownership before touching them. |
| 5. Capture owner | new analysis/capture.rs and capture tests; API/CLI adapters under their current owner | Whole local snapshot replay after live source/definition changes; tampered capsule/manifests reject. Then integrate real package pins/bytes and nonempty offline controls. |
| 6. History/transition + native authoring owners | semantic/types.rs/catalog.rs/historical.rs/mutate.rs/review.rs, transitions.rs and new transitions/analysis.rs; descriptor/API owner updates schema; desktop analysis/forms/document hooks | Explicit OA/SA analysis selection and historical parity; then native base/override/check/context authoring, source equality, undo/reopen and stale result handling. |
Steps 1–4 form the first useful source-authored persisted evaluation gate; step 5 supplies reproducible export, and step 6 closes existing transition/native workflows. They are implementation sequencing, not a narrowed release objective. Approximate engineering functions/irrational unit scales, real reusable libraries and dense full-model textual usability remain required.
## First acceptance fixture and falsifiers
Use the frozen fixture's actual identities and independent expected model, never an evaluator-generated expected Project. Model 4210b9f2-7046-47f0-aa3b-153f626a579d has local Duration, second/millisecond and combine definitions; four properties; degraded and unmeasured cases; one shared check. Requested total is bd5b68e8-65fc-4e35-9071-a73061316a4b and check is bac0de99-9d2f-4e53-9158-d424ecac61bf. The degraded case is 38036c65-16d4-449c-837b-91eb141c1d93; unmeasured is 99d3e09a-0ec3-4194-9dd4-9f3de6625cde.
| Transaction/request | Required real observation |
| --- | --- |
| Source import/save/reopen; evaluate ModelBaseline | 750 ms + 1 s = 7/4 seconds; shared check Satisfied against 2 s. Equivalent JSON produces the same persistent AST. |
| Evaluate degraded | 1250 ms + 1 s = 9/4 seconds; the same check ID is Violated. Total formula origin is Base, receive-time origin is this case's override. |
| Evaluate unmeasured; then ClearAnalysisOverride | Missing receive-time/Unresolved check; clearing restores 7/4 and Satisfied. Setting Unbound and clearing must remain distinct commands/source edits. |
| Edit threshold property once | Both contexts use the same check and new threshold unless that property is explicitly overridden. No predicate copies are rewritten. |
| Baseline edit causing only a case-effective cycle; invalid unused branch | Entire transaction/source admission rejects; prior model, comments, revision and undo remain unchanged. |
| Rename receive_time alias and calculation parameter a; Reidentify local definition | Persistent IDs stay stable for renames; exact typed refs/argument tokens change appropriately; unrelated comments, historical strings and ParameterIds are untouched by ordinary global reidentification. |
| Delete a used property/definition/subject without repair; repair in same batch | First rejects; second admits once. No automatic dropping of an override or conversion to Unbound. |
| Full R7 atoms, affine and anonymous energy expressions | Source/JSON retain exact literal AST and runtime R6 types; no f64 conversion, dimension-only nominal matching or dummy-value type checking. |
| Real C1 call tree, multiple targets, identical repeat | Actual Expr evaluator stops at the same first location/work usage with incomplete causes and no later visits. Cold/warm policy cannot change effective limits or outcome. |
| HTTP expected revision changes before capture | Existing revision conflict; evaluation never acts on a different accepted snapshot than the reported revision/ETag. |
| Whole local freeze then edit live dispatch and combine | Frozen degraded stays 9/4 with old exact sources/definitions while the live result changes. Missing captured content or a tampered redundant manifest rejects. |
Root's remaining staging ratifications are narrow: allocate the schema number; choose whether the real package environment precedes this local fixture or follows it; approve concrete decoder/admission limits after bounded tests. The selected B/R6/R7 semantics do not need another architecture comparison before production work.