Compare commits

..
23 changed files with 4766 additions and 787 deletions
+10 -10
View File
@@ -35,26 +35,26 @@ Use Gitea for live coordination and the relevant code version for implementation
## Coordination and integration
All independent agents coordinating through Gitea run on completely separate machines, each with its own clone, filesystem, tools, processes, and runtime. Gitea and pushed Git history are the shared communication and delivery channels. Local files (including `harness`), unpushed commits, worktrees, test results, and localhost services are not accessible to peers. Matching branch names or paths do not imply synchronized contents.
1. On starting or resuming, including after compaction, inspect the working branch, local changes, applicable instructions, relevant shared records, and existing implementation. Recover context from authoritative records. Check for newer shared instructions on the actual remote default branch without overwriting local work. Read what is relevant; avoid repeated full-history scans and constant polling.
2. Search before creating tasks. Claim a meaningful outcome in its Gitea issue with a stable session identity, scope, branch/worktree, and acceptance criteria. Keep small steps within that task. Recheck after claiming; comments are not atomic locks. Respect existing ownership and resolve overlapping claims before overlapping work.
3. Independent sessions are peers. Each owns its contribution and delegated work. Record one integration/release owner when a milestone needs shared coordination. Use separate worktrees or checkouts, partition files/interfaces, and coordinate changes to shared specifications and documents.
4. At meaningful checkpoints or handoffs, record changed results, commit/PR links, verification commands and environment, blockers, and next actions. Identify the tested code version and uncommitted or unpublished work; make required artifacts accessible to other sessions. Do not assume another agent is running or has read a message; silence alone does not establish abandonment. Record evidence and preserve existing work when reassigning a task.
5. Follow project review and merge conventions. Inspect upstream changes, resolve conflicts while preserving others' edits, and verify the combined result. Do not overwrite others' work or force-push shared history to resolve coordination problems.
2. Search before creating tasks. Claim a meaningful outcome in its Gitea issue with a stable session identity, scope, remote repository/branch, and acceptance criteria. Keep small steps within that task. Recheck after claiming; comments are not atomic locks. Respect existing ownership and resolve overlapping claims before overlapping work.
3. Independent sessions are peers. Each owns its contribution and delegated work on its own machine. Record one integration/release owner when a milestone needs shared coordination. Use distinct remote work branches, partition files/interfaces, and coordinate changes to shared specifications and documents. Worktrees provide local isolation only; they do not share work between machines.
4. At meaningful checkpoints or handoffs, record changed results, commit/PR links, verification commands and environment, blockers, and next actions. Identify the tested code version and label uncommitted or unpublished work as local-only. Before asking peers to consume results, push the required code and versioned documentation, including relevant `harness` updates; publish other required artifacts in Gitea and link them. Use repository-relative paths with commit references, reproducible commands, and necessary environment details; local paths and localhost URLs alone are not handoffs. Do not assume another agent is running or has read a message; silence alone does not establish abandonment. Record evidence and preserve existing work when reassigning a task.
5. Follow project review and merge conventions. Fetch the relevant remote branches and inspect the identified commits before reviewing or integrating another machine's work. Resolve conflicts while preserving others' edits, and verify the combined result locally; report material environment differences. Local instruction-file edits reach peers only after publication and retrieval. Do not overwrite others' work or force-push shared history to resolve coordination problems.
6. Close work when its acceptance criteria are met and the result is integrated or delivered as appropriate. Mark duplicates and superseded tasks explicitly with replacement links. Then choose further useful unowned work while the requested objective remains incomplete.
Use Gitea MCP for shared coordination. If unavailable, report the failure, buffer necessary handoff information locally, and continue already-owned or clearly non-overlapping work. Reconcile with current Gitea state when access returns; do not claim an ownership update, review, or publication succeeded without confirmation.
## Delegation and reasoning
Use GPT-6 Astra exclusively. Delegate bounded independent research, design, implementation, or review when it improves progress, judgment, or preserves the parent's context. Give subagents the relevant context, ownership boundaries, result format, and evidence requirements. Use fresh critics when practical. The parent owns synthesis and integration and normally consolidates results into Gitea. Choose concurrency according to useful independent work and available resources, including other sessions; there is no agent-count quota.
Use GPT-6 Astra exclusively. Gitea peers are independent remote sessions, not subagents controlled by this session. Spawned subagents use the execution environment actually provided by the agent tools; do not assume they share files, and publish their results before Gitea peers need them. Delegate bounded independent research, design, implementation, or review when it improves progress, judgment, or preserves the parent's context. Give subagents the relevant context, ownership boundaries, result format, and evidence requirements. Use fresh critics when practical. The parent owns synthesis and integration and normally consolidates results into Gitea. Choose concurrency according to useful independent work and available resources, including other sessions; there is no agent-count quota.
Optimize reasoning effort for verified outcomes per cost, including review, retries, and repair:
Main agent: default to high for planning, synthesis, integration, and acceptance decisions; medium for dispatching a settled plan and routine coordination.
Subagents: default to medium for bounded implementation and targeted research; low for straightforward retrieval, mechanical edits, and running established checks; high for complex debugging, correctness review, adversarial analysis, or ambiguous design.
Use extra-high selectively for difficult foundational semantic, methodological, formal, or cross-domain decisions. Reserve max for exceptional unresolved problems where additional reasoning is likely to justify its cost. Retain higher settings when evidence supports their value.
- Main agent: default to high for planning, synthesis, integration, and acceptance decisions; medium for dispatching a settled plan and routine coordination.
- Subagents: default to medium for bounded implementation and targeted research; low for straightforward retrieval, mechanical edits, and running established checks; high for complex debugging, correctness review, adversarial analysis, or ambiguous design.
- Use extra-high selectively for difficult foundational semantic, methodological, formal, or cross-domain decisions. Reserve max for exceptional unresolved problems where additional reasoning is likely to justify its cost. Retain higher settings when evidence supports their value.
Allocate by task difficulty and consequences, not role or length alone. Start higher when a clearly difficult decision has expensive downstream consequences. Escalate for concrete reasoning difficulties; resolve missing evidence or unclear requirements at their source. Set subagent effort explicitly through supported controls to avoid unintended inheritance. Do not claim an unapplied change or alter global configuration merely to enforce these preferences.
+201 -33
View File
@@ -1,8 +1,8 @@
//! Typed OA/SA language, tolerant source analysis, and explicit V1/V2/V3/V4 migration.
mod raw_tokens;
#[path = "language/semantic_source.rs"]
mod semantic_source;
mod source3_bridge;
mod raw_tokens;
// Staged syntax foundation only. Public source8 admission waits for complete
// declaration lowering, source updates and the root-owned resource gate.
#[cfg(test)]
@@ -480,6 +480,8 @@ fn parse_document(source: &str, limits: ParseLimits) -> Result<ParsedDocument, V
cursor: 0,
value_depth: 0,
limits,
#[cfg(test)]
mixed: None,
}
.document()
.map_err(|e| vec![diagnostic(source, e)])
@@ -914,6 +916,8 @@ pub(crate) fn parse_unit(
cursor: 0,
value_depth: 0,
limits,
#[cfg(test)]
mixed: None,
}
.unit(entry, true)
.map_err(|e| Box::new(diagnostic(source, e)))
@@ -1005,31 +1009,52 @@ fn edit_positions(
unit: SourceUnit,
) -> Result<Vec<(Id, EditDeclaration)>, Box<Diagnostic>> {
let tokens = strict_tokens(source).map_err(|e| Box::new(diagnostic(source, e)))?;
let mut ends: Vec<_> = tokens.iter().map(|token| token.span.end).collect();
edit_positions_from_tokens(source, &unit, |at| tokens[at], tokens.len(), None)
}
fn edit_positions_from_tokens(
source: &str,
unit: &SourceUnit,
token: impl Fn(usize) -> Lexeme,
count: usize,
value_ends: Option<&BTreeMap<usize, usize>>,
) -> Result<Vec<(Id, EditDeclaration)>, Box<Diagnostic>> {
let mut ends: Vec<_> = (0..count).map(|at| token(at).span.end).collect();
let mut stack = Vec::new();
for (index, token) in tokens.iter().enumerate() {
for index in 0..count {
let token = token(index);
match token.text(source) {
"{" | "[" => stack.push(index),
"}" | "]" => ends[stack.pop().expect("parser checked delimiters")] = token.span.end,
_ => {}
}
}
struct Positions<'a> {
tokens: &'a [Lexeme],
struct Positions<'a, F> {
token_at: F,
ends: Vec<usize>,
value_ends: Option<&'a BTreeMap<usize, usize>>,
}
impl Positions<'_> {
impl<F: Fn(usize) -> Lexeme> Positions<'_, F> {
fn token(&self, start: usize) -> usize {
self.tokens
.partition_point(|token| token.span.start < start)
let mut low = 0;
let mut high = self.ends.len();
while low < high {
let middle = low + (high - low) / 2;
if (self.token_at)(middle).span.start < start {
low = middle + 1;
} else {
high = middle;
}
}
low
}
fn fields(&self, fields: &BTreeMap<String, Ast>) -> BTreeMap<String, EditField> {
fields
.iter()
.map(|(key, ast)| {
let value = self.value(ast);
let start = self.tokens[self.token(value.span.start) - 1].span.start;
let end = self.tokens[self.token(value.span.end)].span.end;
let start = (self.token_at)(self.token(value.span.start) - 1).span.start;
let end = (self.token_at)(self.token(value.span.end)).span.end;
(
key.clone(),
EditField {
@@ -1044,10 +1069,15 @@ fn edit_positions(
let index = self.token(ast.offset());
let span = Span {
start: ast.offset(),
end: match ast {
Ast::Reference(..) | Ast::HistoricalUuid(..) => self.tokens[index + 1].span.end,
_ => self.ends[index],
},
end: self
.value_ends
.and_then(|ends| ends.get(&ast.offset()).copied())
.unwrap_or_else(|| match ast {
Ast::Reference(..) | Ast::HistoricalUuid(..) => {
(self.token_at)(index + 1).span.end
}
_ => self.ends[index],
}),
};
let kind = match ast {
Ast::Object(fields, _) => EditKind::Record(self.fields(fields)),
@@ -1062,11 +1092,12 @@ fn edit_positions(
}
}
let positions = Positions {
tokens: &tokens,
token_at: &token,
ends,
value_ends,
};
unit.declarations
.into_iter()
.iter()
.map(|declaration| {
if declaration.id_insertion.is_some() {
return Err(Box::new(diagnostic(
@@ -1086,8 +1117,8 @@ fn edit_positions(
let start = positions.token(declaration.span.start);
let header = roles
.iter()
.zip(&tokens[start..])
.map(|(role, token)| (*role, token.span))
.zip(start..count)
.map(|(role, at)| (*role, token(at).span))
.collect();
Ok((
declaration.id,
@@ -1114,10 +1145,35 @@ struct Parser<'a> {
cursor: usize,
value_depth: usize,
limits: ParseLimits,
#[cfg(test)]
mixed: Option<analysis_source::mixed::State<'a>>,
}
impl Parser<'_> {
fn token(&self, index: usize) -> Option<Lexeme> {
#[cfg(test)]
if let Some(mixed) = &self.mixed {
return mixed.token(index);
}
self.tokens.get(index).copied()
}
fn current(&self) -> Lexeme {
self.tokens[self.cursor.min(self.tokens.len() - 1)]
self.token(self.cursor).unwrap_or(Lexeme {
class: TokenClass::Punctuation,
span: Span {
start: self.source.len(),
end: self.source.len(),
},
})
}
fn syntax_entry(&mut self) -> Result<(), Error> {
#[cfg(test)]
{
let span = self.current().span;
if let Some(mixed) = &mut self.mixed {
mixed.entry(span)?;
}
}
Ok(())
}
fn text(&self) -> &str {
self.current().text(self.source)
@@ -1154,6 +1210,10 @@ impl Parser<'_> {
}
fn optional_id(&mut self) -> Result<Id, Error> {
if self.text() != "id" {
#[cfg(test)]
if self.mixed.is_some() {
return Err(self.error("Private mixed syntax requires explicit IDs."));
}
return Ok(Id::new_v4());
}
self.cursor += 1;
@@ -1194,6 +1254,7 @@ impl Parser<'_> {
Ok(fields)
}
fn field(&mut self) -> Result<(String, Ast, Span), Error> {
self.syntax_entry()?;
let key = self.text().to_owned();
if !matches!(
self.current().class,
@@ -1204,15 +1265,19 @@ impl Parser<'_> {
self.cursor += 1;
let start = self.current().span.start;
let value = self.value()?;
let end = self.tokens[self.cursor - 1].span.end;
let end = self
.token(self.cursor - 1)
.expect("consumed token")
.span
.end;
self.take(";")?;
Ok((key, value, Span { start, end }))
}
fn nested_header_kind(&self) -> Option<DeclKind> {
let kind = declaration_kind(self.text())?;
let alias = self.tokens.get(self.cursor + 1)?;
let following = self.tokens.get(self.cursor + 2)?;
let alias = self.token(self.cursor + 1)?;
let following = self.token(self.cursor + 2)?;
// A node's ordinary `diagram @view;` field is not a declaration.
if !crate::valid_alias(alias.text(self.source)) {
return None;
@@ -1227,14 +1292,33 @@ impl Parser<'_> {
if self.value_depth >= 64 {
return Err(self.error("Typed values may not nest more than 64 levels."));
}
self.syntax_entry()?;
self.value_depth += 1;
let result = self.value_inner();
self.value_depth -= 1;
#[cfg(test)]
if let Ok(ast) = &result {
let end = self
.token(self.cursor - 1)
.expect("consumed value")
.span
.end;
if let Some(mixed) = &mut self.mixed {
mixed.value_ends.insert(ast.offset(), end);
}
}
result
}
fn value_inner(&mut self) -> Result<Ast, Error> {
let token = self.current();
let offset = token.span.start;
#[cfg(test)]
if let Some(mixed) = &self.mixed
&& (token.class == TokenClass::Number || matches!(self.text(), "-" | "+"))
{
let value = mixed.ordinary_number(&mut self.cursor)?;
return Ok(Ast::Literal(value, offset));
}
match self.text() {
"uuid" => {
self.cursor += 1;
@@ -1332,16 +1416,31 @@ impl Parser<'_> {
fn unit(&mut self, entry: bool, includes_allowed: bool) -> Result<SourceUnit, Error> {
let mut header = if entry {
self.take("arcadia")?;
if !matches!(self.text(), "3" | "4" | "5" | "6" | "7") {
return Err(
self.error("Unsupported schema version: expected arcadia 3, 4, 5, 6 or 7.")
);
#[cfg(test)]
let mixed_mode = self.mixed.is_some();
#[cfg(not(test))]
let mixed_mode = false;
if if mixed_mode {
self.text() != "8"
} else {
!matches!(self.text(), "3" | "4" | "5" | "6" | "7")
} {
return Err(self.error(if mixed_mode {
"Private mixed syntax requires arcadia 8."
} else {
"Unsupported schema version: expected arcadia 3, 4, 5, 6 or 7."
}));
}
let schema_version = self.text().parse().expect("checked source version");
self.cursor += 1;
self.take(";")?;
let start = self.current().span.start;
self.take("project")?;
#[cfg(test)]
if let Some(mixed) = &mut self.mixed {
mixed.record(false, Span { start, end: start })?;
}
self.syntax_entry()?;
let alias = self.identifier()?;
let name = self.quoted()?;
let id_insertion = (self.text() != "id").then_some(self.current().span.start);
@@ -1382,6 +1481,14 @@ impl Parser<'_> {
let mut open: Vec<usize> = Vec::new();
loop {
let offset = self.current().span.start;
#[cfg(test)]
if let Some(mixed) = &mut self.mixed
&& mixed.typed_header(self.cursor, open.is_empty())
{
let owner = open.last().map(|&at| declarations[at].id);
mixed.parse_typed(&mut self.cursor, owner)?;
continue;
}
if let Some(&owner) = open.last() {
if self.text() == "}" {
declarations[owner].body.end = offset;
@@ -1425,6 +1532,14 @@ impl Parser<'_> {
if !includes_allowed {
return Err(self.error("An include requires a source bundle; single-string parsing never reads files."));
}
#[cfg(test)]
if let Some(mixed) = &mut self.mixed {
mixed.include(Span {
start: offset,
end: offset,
})?;
}
self.syntax_entry()?;
self.cursor += 1;
let path = self.quoted()?;
self.take(";")?;
@@ -1432,7 +1547,11 @@ impl Parser<'_> {
path,
Span {
start: offset,
end: self.tokens[self.cursor - 1].span.end,
end: self
.token(self.cursor - 1)
.expect("consumed token")
.span
.end,
},
));
continue;
@@ -1444,20 +1563,40 @@ impl Parser<'_> {
if entry_header.description.is_some() {
return Err(self.error("Duplicate project description."));
}
self.syntax_entry()?;
self.cursor += 1;
let description = self.quoted()?;
self.take(";")?;
entry_header.project.description = description;
entry_header.description = Some(Span {
start: offset,
end: self.tokens[self.cursor - 1].span.end,
end: self
.token(self.cursor - 1)
.expect("consumed token")
.span
.end,
});
header = Some(entry_header);
continue;
}
#[cfg(test)]
if self.mixed.is_some() && analysis_source::mixed::long_analysis_word(self.text()) {
return Err(self.error("Use the short source8 analysis declaration word."));
}
self.syntax_entry()?;
let word = self.text().to_owned();
let kind = declaration_kind(&word)
.ok_or_else(|| self.error(format!("Unknown declaration '{word}'.")))?;
#[cfg(test)]
if let Some(mixed) = &mut self.mixed {
mixed.record(
matches!(kind, DeclKind::Element(..) | DeclKind::Relation(..)),
Span {
start: offset,
end: offset,
},
)?;
}
graph_count += usize::from(matches!(
kind,
DeclKind::Element(..) | DeclKind::Relation(..)
@@ -1485,6 +1624,10 @@ impl Parser<'_> {
let body_start = self.current().span.end;
self.take("{")?;
let lexical_owner = open.last().map(|&owner| declarations[owner].id);
#[cfg(test)]
if let Some(mixed) = &mut self.mixed {
mixed.lexical_owners.push(open.last().copied());
}
declarations.push(Declaration {
kind,
alias,
@@ -1521,17 +1664,25 @@ impl Parser<'_> {
}
}
pub(crate) fn resolve_declarations(
// The construction phase is shared with private mixed resolution. Migration and
// complete-model admission remain exclusively in each caller's finishing tail.
struct ConstructedDeclarations {
project: Project,
legacy_elements: serde_json::Map<String, Value>,
}
fn construct_declarations<'a>(
mut project: Project,
declarations: &[Declaration],
declarations: impl IntoIterator<Item = &'a Declaration>,
aliases: &BTreeMap<String, Id>,
limits: ParseLimits,
) -> Result<crate::persistence::DecodeResult, (usize, Error)> {
mut before_fields: impl FnMut(&Declaration) -> Result<(), Error>,
) -> Result<ConstructedDeclarations, (usize, Error)> {
let legacy = project.schema_version == 3;
let mut legacy_elements = serde_json::Map::new();
let mut nodes = Vec::new();
for (index, declaration) in declarations.iter().enumerate() {
for (index, declaration) in declarations.into_iter().enumerate() {
(|| -> Result<(), Error> {
before_fields(declaration)?;
let mut fields = declaration
.fields
.iter()
@@ -1756,6 +1907,23 @@ pub(crate) fn resolve_declarations(
.map_err(|error| (index, error))?;
target.nodes.insert(node.id, node);
}
Ok(ConstructedDeclarations {
project,
legacy_elements,
})
}
pub(crate) fn resolve_declarations(
project: Project,
declarations: &[Declaration],
aliases: &BTreeMap<String, Id>,
limits: ParseLimits,
) -> Result<crate::persistence::DecodeResult, (usize, Error)> {
let legacy = project.schema_version == 3;
let ConstructedDeclarations {
project,
legacy_elements,
} = construct_declarations(project, declarations, aliases, |_| Ok(()))?;
if legacy {
let migrated = source3_bridge::from_scratch(project, legacy_elements)
.and_then(|old| crate::persistence::v3::migrate(old, limits))
@@ -6,6 +6,7 @@ mod declarations;
mod index;
mod lex;
mod lower;
pub(super) mod mixed;
mod syntax;
mod update;
mod write;
@@ -124,7 +124,7 @@ struct Symbol {
}
#[derive(Default)]
struct Tables {
pub(super) struct Tables {
aliases: BTreeMap<String, Symbol>,
ids: BTreeSet<Id>,
parameters: BTreeMap<Id, BTreeMap<String, a::ParameterId>>,
@@ -157,7 +157,7 @@ impl Tables {
self.entries += 1;
Ok(())
}
fn insert(
pub(super) fn insert(
&mut self,
id: Id,
alias: &str,
@@ -233,7 +233,11 @@ impl Tables {
Ok(())
}
fn typed(&mut self, declaration: &Declaration, budget: &Budget) -> Result<(), Error> {
pub(super) fn typed(
&mut self,
declaration: &Declaration,
budget: &Budget,
) -> Result<(), Error> {
let header = &declaration.header;
let id = identity(&header.identity, header.alias.span)?;
self.insert(
@@ -299,31 +303,49 @@ pub(super) fn lower(
})?;
}
}
lower_bodies(
&tables,
units
.iter()
.flat_map(|unit| unit.declarations.iter().map(move |d| (unit.file, d))),
budget,
)
}
/// Catalog and borrowed declarations must be from the same private witness.
/// The mixed caller reserves aggregate live storage before entering this body;
/// the legacy wrapper retains its original individual phase checks.
pub(super) fn lower_bodies<'a>(
tables: &Tables,
declarations: impl IntoIterator<Item = (usize, &'a Declaration)>,
budget: &Budget,
) -> Result<Lowered, Failure> {
budget
.active()
.map_err(|error| Failure { file: None, error })?;
let mut output = Lowered {
definitions: BTreeMap::new(),
elements: BTreeMap::new(),
occurrences: Vec::new(),
expressions: Vec::new(),
};
for unit in units {
for declaration in &unit.declarations {
let owner = declaration
.header
.identity
.value
.expect("collected stable identity");
let mut resolver = Resolver {
tables: &tables,
output: &mut output,
budget,
file: unit.file,
owner,
};
resolver.declaration(declaration).map_err(|error| Failure {
file: Some(unit.file),
error,
})?;
}
for (file, declaration) in declarations {
let owner = declaration
.header
.identity
.value
.expect("collected stable identity");
let mut resolver = Resolver {
tables,
output: &mut output,
budget,
file,
owner,
};
resolver.declaration(declaration).map_err(|error| Failure {
file: Some(file),
error,
})?;
}
Ok(output)
}
@@ -0,0 +1,406 @@
//! Private syntax-only complete-bundle witness. No model admission or edit authority.
//!
//! One scan charges original bytes and raw tokens (including trivia). Typed
//! syntax keeps its existing entry/reference/numeric charges. Ordinary syntax
//! charges one entry per project/declaration/include/description, field, and
//! value before allocation; array values already account for their list slots.
//! These private storage ceilings are independent of caller model record counts.
//! Ordinary references consume value storage, not ordered-model-step allowance.
//! Typed reference occurrences retain the caller max_step_references ceiling.
//! Only declaration graph/record counts are known here; semantic nested-model
//! records and ordinary ordered steps still require the later complete admission.
//! Metadata phase slots aggregate across all files: retained token-end indices,
//! value-end entries, projected value/field/header/declaration positions, lexical
//! owner slots and the file table. Checks do not recharge syntax or latch errors;
//! every failure must return immediately. Tokens themselves use the token bound.
//! This deliberately bounded staging policy is not calibrated public source8.
use super::{
budget::{Budget, Limits},
cursor::Cursor,
declarations, lex,
};
use crate::Id;
use crate::language::{
self, Ast, EditDeclaration, Error, Lexeme, ParseLimits, Parser, SourceUnit, Span, TokenClass,
};
use crate::source_project::{ProjectLimits, SourceBundle, SourceDiagnostic, traversal::Traversal};
use std::collections::BTreeMap;
struct ModelCounts {
limits: ProjectLimits,
graph: usize,
records: usize,
includes: usize,
}
pub(super) struct ParsedMixedSyntax<'s> {
bundle: &'s SourceBundle,
// Preserve caller policy for the later semantic pass. Syntax storage uses
// separate ceilings and cannot stand in for these graph/record/step limits.
limits: ProjectLimits,
files: Vec<FileSyntax<'s>>,
budget: Budget,
// Includes the conservative dropped value-end subtotal. The semantic
// phase adds syntax entries to this once, not tokens/positions again.
metadata: usize,
}
struct FileSyntax<'s> {
path: &'s str,
source: &'s str,
tokens: Vec<lex::Token>,
ordinary: SourceUnit,
typed: Vec<declarations::Declaration>,
positions: Vec<(Id, EditDeclaration)>,
// A lexical owner is an occurrence index, not only a possibly repeated UUID.
lexical_owners: Vec<Option<usize>>,
}
// Only Parser borrows this state; the stored witness has no self-reference.
pub(in crate::language) struct State<'a> {
source: &'a str,
tokens: &'a [lex::Token],
budget: &'a mut Budget,
model: &'a mut ModelCounts,
typed: Vec<declarations::Declaration>,
pub(in crate::language) value_ends: BTreeMap<usize, usize>,
pub(in crate::language) lexical_owners: Vec<Option<usize>>,
}
fn ordinary_error(error: super::Error) -> Error {
Error {
code: "invalid_source",
offset: error.span.start,
message: error.message,
}
}
fn lexeme(token: lex::Token) -> Lexeme {
Lexeme {
class: match token.class {
lex::Class::Word => TokenClass::Identifier,
lex::Class::Number => TokenClass::Number,
lex::Class::String => TokenClass::String,
lex::Class::Punctuation => TokenClass::Punctuation,
},
span: token.span,
}
}
impl State<'_> {
pub(in crate::language) fn token(&self, at: usize) -> Option<Lexeme> {
self.tokens.get(at).copied().map(lexeme)
}
pub(in crate::language) fn entry(&mut self, at: Span) -> Result<(), Error> {
self.budget.entry(at).map_err(ordinary_error)
}
pub(in crate::language) fn record(&mut self, graph: bool, at: Span) -> Result<(), Error> {
let records = self.model.records.checked_add(1);
let graph = self.model.graph.checked_add(usize::from(graph));
let (Some(records), Some(graph)) = (records, graph) else {
return Err(Error {
code: "model_limit",
offset: at.start,
message: "Declaration count overflow.".into(),
});
};
self.model
.limits
.model
.check_counts(graph, records, 0)
.map_err(|message| Error {
code: "model_limit",
offset: at.start,
message,
})?;
self.model.records = records;
self.model.graph = graph;
Ok(())
}
pub(in crate::language) fn include(&mut self, at: Span) -> Result<(), Error> {
if self.model.includes >= self.model.limits.max_include_edges {
return Err(Error {
code: "project_limit",
offset: at.start,
message: format!(
"Include graph exceeds {} edges.",
self.model.limits.max_include_edges
),
});
}
self.model.includes += 1;
Ok(())
}
pub(in crate::language) fn ordinary_number(
&self,
at: &mut usize,
) -> Result<serde_json::Value, Error> {
let first = self.tokens[*at];
let mut end = first.span.end;
let mut after = *at + 1;
// Match the old greedy numeric alphabet only across byte-adjacent raw
// tokens. This is consumption of the shared stream, not another scan.
while let Some(next) = self.tokens.get(after) {
if next.span.start != end
|| !self.source[next.span.start..next.span.end]
.bytes()
.all(|byte| byte.is_ascii_alphanumeric() || b"_.+-".contains(&byte))
{
break;
}
end = next.span.end;
after += 1;
}
let value: serde_json::Number = serde_json::from_str(&self.source[first.span.start..end])
.map_err(|_| Error {
code: "invalid_source",
offset: first.span.start,
message: "Expected a byte-adjacent finite JSON decimal number.".into(),
})?;
*at = after;
Ok(serde_json::Value::Number(value))
}
pub(in crate::language) fn typed_header(&self, at: usize, root: bool) -> bool {
let Some(token) = self.tokens.get(at) else {
return false;
};
if declarations::declaration_kind(&self.source[token.span.start..token.span.end]).is_none()
{
return false;
}
// A generic field named `unit` remains an ordinary field. A declaration
// header has an alias and display-name token; only that shape is nested.
root || (self
.tokens
.get(at + 1)
.is_some_and(|t| t.class == lex::Class::Word)
&& self
.tokens
.get(at + 2)
.is_some_and(|t| t.class == lex::Class::String))
}
pub(in crate::language) fn parse_typed(
&mut self,
at: &mut usize,
owner: Option<Id>,
) -> Result<(), Error> {
// Transfer the one active position into the typed consumer and back;
// neither consumer scans, clones a stream, nor runs concurrently.
self.record(true, self.tokens[*at].span)?;
let mut cursor = Cursor::new(self.source, self.tokens, self.budget);
cursor.cursor = *at;
let result = declarations::parse(&mut cursor, 8, owner);
*at = cursor.cursor;
let declaration = result.map_err(ordinary_error)?;
let missing = declaration.header.identity.value.is_none()
|| matches!(&declaration.body, declarations::Body::Calculation { parameters, .. }
if parameters.iter().any(|p| p.identity.value.is_none()));
if missing {
return Err(Error {
code: "invalid_source",
offset: declaration.span.start,
message: "Private mixed syntax requires explicit IDs.".into(),
});
}
self.typed.push(declaration);
Ok(())
}
}
pub(in crate::language) fn long_analysis_word(word: &str) -> bool {
matches!(
word,
"oa_engineering_property"
| "sa_engineering_property"
| "oa_analysis_check"
| "sa_analysis_check"
| "oa_analysis_case"
| "sa_analysis_case"
)
}
pub(super) fn parse(
bundle: &SourceBundle,
limits: ProjectLimits,
) -> Result<ParsedMixedSyntax<'_>, Vec<SourceDiagnostic>> {
let profile = Limits::source8(ParseLimits {
// Storage is not the caller's final model record/ordered-step accounting.
max_source_bytes: limits.model.max_source_bytes,
max_step_references: limits.model.max_step_references,
..ParseLimits::UNLIMITED
});
parse_with_profile(bundle, limits, profile)
}
fn located(path: &str, source: &str, error: Error) -> Vec<SourceDiagnostic> {
vec![SourceDiagnostic {
path: path.into(),
diagnostic: language::diagnostic(source, error),
related: Vec::new(),
}]
}
fn parse_with_profile(
bundle: &SourceBundle,
limits: ProjectLimits,
profile: Limits,
) -> Result<ParsedMixedSyntax<'_>, Vec<SourceDiagnostic>> {
let entry_source = bundle.files.get(&bundle.entry).map_or("", String::as_str);
let at_entry = |error| located(&bundle.entry, entry_source, ordinary_error(error));
let maximum = Limits::source8(ParseLimits {
max_source_bytes: limits.model.max_source_bytes,
max_step_references: limits.model.max_step_references,
..ParseLimits::UNLIMITED
});
if profile.bytes > maximum.bytes
|| profile.tokens > maximum.tokens
|| profile.entries > maximum.entries
|| profile.references > maximum.references
|| profile.frames > maximum.frames
|| profile.numeric_units > maximum.numeric_units
{
return Err(at_entry(super::error(
Span::default(),
"Mixed test profile may only tighten inherited limits.",
)));
}
let mut budget = Budget::new(profile).map_err(at_entry)?;
let bytes = bundle
.files
.values()
.try_fold(0usize, |n, s| n.checked_add(s.len()))
.ok_or_else(|| at_entry(super::error(Span::default(), "Source size overflow.")))?;
budget
.index_bytes(bytes, Span::default())
.map_err(at_entry)?;
let mut metadata = bundle.files.len();
budget
.lowering_entries(metadata, Span::default())
.map_err(at_entry)?;
// Tighter private byte/file-slot limits precede folded-path table and
// traversal allocations. Public Traversal's own validation order is intact.
let mut traversal = Traversal::new(bundle, limits)?;
let mut parsed = BTreeMap::new();
let mut model = ModelCounts {
limits,
graph: 0,
records: 0,
includes: 0,
};
while let Some(path) = traversal.next()? {
let (path, source) = bundle
.files
.get_key_value(&path)
.expect("traversal checked source");
let failure = |error| located(path, source, ordinary_error(error));
let tokens = lex::tokens(source, &mut budget).map_err(failure)?;
let mut parser = Parser {
source,
tokens: Vec::new(),
cursor: 0,
value_depth: 0,
limits: limits.model,
mixed: Some(State {
source,
tokens: &tokens,
budget: &mut budget,
model: &mut model,
typed: Vec::new(),
value_ends: BTreeMap::new(),
lexical_owners: Vec::new(),
}),
};
let ordinary = parser
.unit(path == &bundle.entry, true)
.map_err(|e| located(path, source, e))?;
let state = parser.mixed.take().expect("private parser state");
drop(parser);
// Projected ordinary AST nodes/fields are bounded before constructing a
// second position representation. Count references as slots, not syntax.
let projection_slots = ordinary_position_slots(&ordinary);
metadata = metadata
.checked_add(tokens.len())
.and_then(|n| n.checked_add(state.value_ends.len()))
.and_then(|n| n.checked_add(projection_slots))
.and_then(|n| n.checked_add(state.lexical_owners.len()))
.ok_or_else(|| failure(super::error(Span::default(), "Metadata size overflow.")))?;
state
.budget
.lowering_entries(metadata, Span::default())
.map_err(failure)?;
let positions = language::edit_positions_from_tokens(
source,
&ordinary,
|at| lexeme(tokens[at]),
tokens.len(),
Some(&state.value_ends),
)
.map_err(|e| {
vec![SourceDiagnostic {
path: path.clone(),
diagnostic: *e,
related: Vec::new(),
}]
})?;
for (raw, span) in &ordinary.includes {
traversal.include(path, raw, *span)?;
}
traversal.finish_file(path.clone());
let State {
typed,
lexical_owners,
..
} = state;
parsed.insert(
path.as_str(),
FileSyntax {
path,
source,
tokens,
ordinary,
typed,
positions,
lexical_owners,
},
);
}
traversal.finish()?;
budget.active().map_err(at_entry)?;
Ok(ParsedMixedSyntax {
bundle,
limits,
files: parsed.into_values().collect(),
budget,
metadata,
})
}
fn ordinary_position_slots(unit: &SourceUnit) -> usize {
// Iterative accounting avoids adding a second recursive walk before the
// already depth-bounded position projection. Each reference is one value.
let mut slots = 0usize;
let mut stack = Vec::new();
for declaration in &unit.declarations {
slots = slots.saturating_add(7); // declaration/body + at most five header roles
slots = slots.saturating_add(declaration.fields.len());
stack.extend(declaration.fields.values());
}
while let Some(value) = stack.pop() {
slots = slots.saturating_add(1);
match value {
Ast::Array(items, _) => stack.extend(items),
Ast::Object(fields, _) => {
slots = slots.saturating_add(fields.len());
stack.extend(fields.values());
}
_ => {}
}
}
slots
}
#[cfg(test)]
mod tests;
pub(super) mod project;
@@ -0,0 +1,639 @@
//! Private complete-Project admission from the actual complete syntax witness.
//! No Index, update authority, public source8 dispatch or replacement policy.
use super::super::{
Binding, Budget, Error as TypedError, Expression, ExpressionKind, Reference, Span,
declarations as d, error, lower,
};
use super::{FileSyntax, ParsedMixedSyntax};
use crate::language::{Ast, DeclKind};
use crate::source_project::{SourceDiagnostic, SourceLocation};
use crate::{Id, Kind, Project, RelationKind, analysis, language, validation};
use std::collections::BTreeMap;
mod accounting;
use accounting::{Ledger, Slots, add};
/// Read-only authority: every component originates in the consumed syntax.
pub(in crate::language::analysis_source) struct ResolvedMixedProject<'s> {
syntax: ParsedMixedSyntax<'s>,
project: Project,
warnings: Vec<SourceDiagnostic>,
static_limits: analysis::AdmissionLimits,
static_usage: analysis::AdmissionUsage,
expressions: Vec<lower::ExpressionSite>,
occurrences: Vec<lower::Occurrence>,
storage: Ledger,
}
impl ResolvedMixedProject<'_> {
pub fn project(&self) -> &Project {
&self.project
}
pub fn warnings(&self) -> &[SourceDiagnostic] {
&self.warnings
}
}
#[derive(Debug)]
pub(in crate::language::analysis_source) struct Failure {
pub diagnostics: Vec<SourceDiagnostic>,
}
impl Failure {
// Rejection transport is not retained in a successful witness. In particular
// a storage refusal needs a small diagnostic even when its allowance is zero.
fn typed(syntax: &ParsedMixedSyntax<'_>, file: Option<usize>, e: TypedError) -> Self {
let mut failure = Self::ordinary(
syntax,
file,
language::Error {
code: "invalid_source",
offset: e.span.start,
message: e.message,
},
);
failure.diagnostics[0].diagnostic.span = e.span;
failure
}
fn ordinary(syntax: &ParsedMixedSyntax<'_>, file: Option<usize>, e: language::Error) -> Self {
let file = file.unwrap_or_else(|| entry(syntax));
let unit = &syntax.files[file];
Self {
diagnostics: super::located(unit.path, unit.source, e),
}
}
}
#[derive(Clone, Copy)]
enum OwnerKind {
Project,
Ordinary(usize),
Typed(usize),
}
#[derive(Clone, Copy)]
struct Owner {
file: usize,
kind: OwnerKind,
span: Span,
}
struct Catalog {
tables: lower::Tables,
aliases: BTreeMap<String, Id>,
owners: BTreeMap<Id, Owner>,
tables_storage: Slots,
aliases_storage: Slots,
owners_storage: Slots,
}
fn entry(syntax: &ParsedMixedSyntax<'_>) -> usize {
syntax
.files
.iter()
.position(|f| f.path == syntax.bundle.entry)
.expect("complete entry")
}
fn category(kind: DeclKind) -> lower::Category {
match kind {
DeclKind::Element(kind, layer) => lower::Category::Element(kind, layer),
DeclKind::Relation(_) => lower::Category::Relation,
DeclKind::Diagram => lower::Category::Diagram,
DeclKind::Node => lower::Category::Node,
DeclKind::Artifact => lower::Category::Artifact,
DeclKind::Trace => lower::Category::Trace,
}
}
impl Catalog {
fn insert(
&mut self,
syntax: &ParsedMixedSyntax<'_>,
ledger: &mut Ledger,
id: Id,
alias: &str,
owner: Owner,
) -> Result<(), Failure> {
// Alias first, even when both aliases and IDs collide. Owner is an
// occurrence position, never a lookup through the rejected UUID.
let prior = self
.aliases
.get(alias)
.map(|id| ("alias", self.owners[id]))
.or_else(|| self.owners.get(&id).map(|o| ("identity", *o)));
if let Some((domain, prior)) = prior {
let mut failure = Failure::typed(
syntax,
Some(owner.file),
error(owner.span, format!("Duplicate global {domain}.")),
);
failure.diagnostics[0].related.push(SourceLocation {
path: syntax.files[prior.file].path.into(),
span: prior.span,
});
return Err(failure);
}
let alias_slots = Slots {
entries: 1,
text: alias.len(),
..Slots::default()
};
ledger
.reserve(
&syntax.budget,
alias_slots
.add(Slots::entries(1), owner.span)
.map_err(|e| Failure::typed(syntax, Some(owner.file), e))?,
owner.span,
)
.map_err(|e| Failure::typed(syntax, Some(owner.file), e))?;
self.aliases_storage = self
.aliases_storage
.add(alias_slots, owner.span)
.map_err(|e| Failure::typed(syntax, Some(owner.file), e))?;
self.owners_storage = self
.owners_storage
.add(Slots::entries(1), owner.span)
.map_err(|e| Failure::typed(syntax, Some(owner.file), e))?;
self.aliases.insert(alias.into(), id);
self.owners.insert(id, owner);
Ok(())
}
fn reserve_tables(
&mut self,
syntax: &ParsedMixedSyntax<'_>,
ledger: &mut Ledger,
slots: Slots,
owner: Owner,
) -> Result<(), Failure> {
ledger
.reserve(&syntax.budget, slots, owner.span)
.map_err(|e| Failure::typed(syntax, Some(owner.file), e))?;
self.tables_storage = self
.tables_storage
.add(slots, owner.span)
.map_err(|e| Failure::typed(syntax, Some(owner.file), e))?;
Ok(())
}
}
fn catalog(syntax: &ParsedMixedSyntax<'_>, ledger: &mut Ledger) -> Result<Catalog, Failure> {
let mut catalog = Catalog {
tables: lower::Tables::default(),
aliases: BTreeMap::new(),
owners: BTreeMap::new(),
tables_storage: Slots::default(),
aliases_storage: Slots::default(),
owners_storage: Slots::default(),
};
let file = entry(syntax);
let header = syntax.files[file]
.ordinary
.header
.as_ref()
.expect("entry header");
let owner = Owner {
file,
kind: OwnerKind::Project,
span: header.span,
};
catalog.insert(
syntax,
ledger,
header.project.id,
&header.project.alias,
owner,
)?;
catalog.reserve_tables(
syntax,
ledger,
Slots {
entries: 2,
text: header.project.alias.len(),
..Slots::default()
},
owner,
)?;
catalog
.tables
.insert(
header.project.id,
&header.project.alias,
lower::Category::Project,
header.span,
&syntax.budget,
)
.map_err(|e| Failure::typed(syntax, Some(file), e))?;
for (file, unit) in syntax.files.iter().enumerate() {
let mut ordinary = unit.ordinary.declarations.iter().enumerate().peekable();
let mut typed = unit.typed.iter().enumerate().peekable();
while ordinary.peek().is_some() || typed.peek().is_some() {
let use_ordinary = match (ordinary.peek(), typed.peek()) {
(Some((_, a)), Some((_, b))) => a.span.start < b.span.start,
(Some(_), None) => true,
_ => false,
};
if use_ordinary {
let (index, d) = ordinary.next().expect("ordinary head");
let owner = Owner {
file,
kind: OwnerKind::Ordinary(index),
span: d.span,
};
catalog.insert(syntax, ledger, d.id, &d.alias, owner)?;
catalog.reserve_tables(
syntax,
ledger,
Slots {
entries: 2,
text: d.alias.len(),
..Slots::default()
},
owner,
)?;
catalog
.tables
.insert(d.id, &d.alias, category(d.kind), d.span, &syntax.budget)
.map_err(|e| Failure::typed(syntax, Some(file), e))?;
} else {
let (index, d) = typed.next().expect("typed head");
let owner = Owner {
file,
kind: OwnerKind::Typed(index),
span: d.span,
};
catalog.insert(
syntax,
ledger,
d.header.identity.value.expect("syntax explicit ID"),
&d.header.alias.name,
owner,
)?;
let mut slots = Slots {
entries: 2,
text: d.header.alias.name.len(),
..Slots::default()
};
let mut scratch = 0;
if let d::Body::Calculation { parameters, .. } = &d.body {
slots.entries = add(
slots.entries,
add(1, parameters.len(), d.span)
.map_err(|e| Failure::typed(syntax, Some(file), e))?,
d.span,
)
.map_err(|e| Failure::typed(syntax, Some(file), e))?;
scratch = parameters.len(); // per-signature duplicate-ID set
for p in parameters {
slots.text = add(slots.text, p.name.name.len(), p.span)
.map_err(|e| Failure::typed(syntax, Some(file), e))?;
}
}
catalog.reserve_tables(syntax, ledger, slots, owner)?;
ledger
.reserve(&syntax.budget, Slots::entries(scratch), d.span)
.map_err(|e| Failure::typed(syntax, Some(file), e))?;
catalog
.tables
.typed(d, &syntax.budget)
.map_err(|e| Failure::typed(syntax, Some(file), e))?;
ledger.release(Slots::entries(scratch));
}
}
}
Ok(catalog)
}
fn object<'a>(value: Option<&'a Ast>, key: &str) -> Option<&'a Ast> {
match value {
Some(Ast::Object(fields, _)) => fields.get(key),
_ => None,
}
}
/// Selector only, not a decoder. This executes before ordinary fields clone.
fn active_references(
declaration: &language::Declaration,
budget: &mut Budget,
) -> Result<(), language::Error> {
let field = |key: &str| declaration.fields.get(key);
let arrays = match declaration.kind {
DeclKind::Element(Kind::Process | Kind::FunctionalChain | Kind::Scenario, _) => {
[field("steps"), None]
}
DeclKind::Element(Kind::Occurrence, _) => {
[object(object(field("body"), "Branch"), "occurrences"), None]
}
DeclKind::Relation(RelationKind::ContextualLink) => [
object(field("source"), "references"),
object(field("target"), "references"),
],
_ => [None, None],
};
for array in arrays.into_iter().flatten() {
if let Ast::Array(items, _) = array {
for item in items {
let start = item.offset();
budget
.reference(Span { start, end: start })
.map_err(super::ordinary_error)?;
}
}
}
Ok(())
}
pub(in crate::language::analysis_source) fn resolve(
mut syntax: ParsedMixedSyntax<'_>,
) -> Result<ResolvedMixedProject<'_>, Failure> {
let mut storage = Ledger::default();
let lower_storage = accounting::syntax(&syntax, &mut storage)?;
let catalog = catalog(&syntax, &mut storage)?;
let entry = entry(&syntax);
// Only the empty model header is copied: retain its original syntax/header
// ownership and construct the final model through the shared decoder once.
let project = syntax.files[entry]
.ordinary
.header
.as_ref()
.expect("entry header")
.project
.clone();
let count = syntax
.files
.iter()
.try_fold(0usize, |n, f| {
add(n, f.ordinary.declarations.len(), Span::default())
})
.map_err(|e| Failure::typed(&syntax, None, e))?;
storage
.reserve(&syntax.budget, Slots::entries(count), Span::default())
.map_err(|e| Failure::typed(&syntax, None, e))?;
let declarations: Vec<_> = syntax
.files
.iter()
.enumerate()
.flat_map(|(file, unit)| unit.ordinary.declarations.iter().map(move |d| (file, d)))
.collect();
let mut nodes = 0;
let constructed = language::construct_declarations(
project,
declarations.iter().map(|(_, d)| *d),
&catalog.aliases,
|d| {
active_references(d, &mut syntax.budget)?;
if matches!(d.kind, DeclKind::Node) {
storage
.reserve(&syntax.budget, Slots::entries(1), d.span)
.map_err(super::ordinary_error)?;
nodes += 1;
}
Ok(())
},
)
.map_err(|(index, error)| {
Failure::ordinary(
&syntax,
declarations.get(index).map(|(file, _)| *file),
error,
)
})?;
let language::ConstructedDeclarations {
mut project,
legacy_elements,
} = constructed;
debug_assert!(legacy_elements.is_empty()); // mixed syntax only admits source8
drop(legacy_elements);
drop(declarations);
storage.release(Slots::entries(count));
storage.release(Slots::entries(nodes));
drop(catalog.aliases);
storage.release(catalog.aliases_storage);
let all_lower = lower_storage
.all()
.map_err(|e| Failure::typed(&syntax, None, e))?;
storage
.reserve(&syntax.budget, all_lower, Span::default())
.map_err(|e| Failure::typed(&syntax, None, e))?;
let lower::Lowered {
definitions,
elements,
occurrences,
expressions,
} = lower::lower_bodies(
&catalog.tables,
syntax
.files
.iter()
.enumerate()
.flat_map(|(file, unit)| unit.typed.iter().map(move |d| (file, d))),
&syntax.budget,
)
.map_err(|e| Failure::typed(&syntax, e.file, e.error))?;
// One global arena transfers verbatim; only now are all Resolver temporaries
// dead. Output counts must fit the independently counted reservation.
storage.release(Slots {
entries: all_lower
.entries
.checked_sub(expressions.len())
.expect("expression bound"),
occurrences: all_lower
.occurrences
.checked_sub(occurrences.len())
.expect("occurrence bound"),
text: 0,
});
drop(catalog.tables);
storage.release(catalog.tables_storage);
for (id, value) in definitions {
if project.definitions.contains_key(&id) {
return Err(Failure::typed(
&syntax,
None,
error(Span::default(), "Non-disjoint definition merge."),
));
}
project.definitions.insert(id, value);
}
for (id, value) in elements {
if project.elements.contains_key(&id) {
return Err(Failure::typed(
&syntax,
None,
error(Span::default(), "Non-disjoint element merge."),
));
}
project.elements.insert(id, value);
}
syntax
.limits
.model
.check_model(&project)
.map_err(|message| {
Failure::ordinary(
&syntax,
None,
language::Error {
code: "model_limit",
offset: 0,
message,
},
)
})?;
let (issues, admission) = validation::validate_with_usage(&project);
// Raw architecture Issues are excluded. Each transferred string, copied path
// and destination slot is admitted before mapping/retention; never truncate.
let mut warnings = Vec::new();
let mut rejected = false;
for issue in issues {
let owner = issue
.element
.and_then(|id| catalog.owners.get(&id))
.copied()
.unwrap_or(catalog.owners[&project.id]);
let span = field_span(&syntax.files[owner.file], owner, issue.field.as_deref());
let unit = &syntax.files[owner.file];
let text = [
unit.path.len(),
issue.code.len(),
issue.field.as_ref().map_or(0, String::len),
issue.message.len(),
]
.into_iter()
.try_fold(0, |n, k| add(n, k, span))
.map_err(|e| Failure::typed(&syntax, Some(owner.file), e))?;
storage
.reserve(
&syntax.budget,
Slots {
entries: 1,
occurrences: 0,
text,
},
span,
)
.map_err(|e| Failure::typed(&syntax, Some(owner.file), e))?;
rejected |= issue.severity == validation::Severity::Error;
let prefix = &unit.source[..span.start];
let line = prefix.bytes().filter(|b| *b == b'\n').count() + 1;
let column = prefix.rsplit('\n').next().expect("prefix").chars().count() + 1;
warnings.push(SourceDiagnostic {
path: unit.path.into(),
related: Vec::new(),
diagnostic: language::Diagnostic {
severity: issue.severity,
code: issue.code,
object_id: issue.element,
field: issue.field,
message: issue.message,
span,
line,
column,
},
});
}
if rejected {
return Err(Failure {
diagnostics: warnings,
});
}
let (static_limits, static_usage) = admission.expect("successful validation retains admission");
drop(catalog.owners);
storage.release(catalog.owners_storage);
Ok(ResolvedMixedProject {
syntax,
project,
warnings,
static_limits,
static_usage,
expressions,
occurrences,
storage,
})
}
/// Exact known field or explicit whole-owner fallback. Never guess by offsets.
fn field_span(file: &FileSyntax<'_>, owner: Owner, field: Option<&str>) -> Span {
let Some(field) = field else {
return owner.span;
};
match owner.kind {
OwnerKind::Project => file
.ordinary
.header
.as_ref()
.and_then(|h| (field == "description").then_some(h.description).flatten())
.unwrap_or(owner.span),
OwnerKind::Ordinary(index) => file.ordinary.declarations[index]
.field_spans
.get(field)
.copied()
.unwrap_or(owner.span),
OwnerKind::Typed(index) => typed_field(&file.typed[index], field).unwrap_or(owner.span),
}
}
fn typed_field(d: &d::Declaration, field: &str) -> Option<Span> {
if field == "description" {
return d.description.as_ref().map(|f| f.value_span);
}
if field == "alias" {
return Some(d.header.alias.span);
}
if field == "name" {
return Some(d.header.name_span);
}
match &d.body {
d::Body::QuantityKind { dimension, algebra } => match field {
"dimension" => Some(dimension.value_span),
"algebra" => Some(algebra.value_span),
_ => None,
},
d::Body::Unit {
kind,
scale,
offset,
symbol,
} => match field {
"kind" => Some(kind.value_span),
"scale" => Some(scale.value_span),
"offset" => Some(offset.value_span),
"symbol" => Some(symbol.value_span),
_ => None,
},
d::Body::Calculation { result, body, .. } => match field {
"result" => Some(result.value_span),
"body" => Some(body.value_span),
// Current semantic admission reports aggregate "parameters", not a
// qualified parameter/type path. Do not manufacture precision from
// a UUID substring; lowerer failures already carry exact file/span.
_ => None,
},
d::Body::Property {
subject,
value_type,
base,
display_unit,
} => match field {
"subject" => Some(subject.value_span),
"type" | "value_type" => Some(value_type.value_span),
"base" => Some(base.value_span),
"display_unit" => display_unit.as_ref().map(|f| f.value_span),
_ => None,
},
d::Body::Check {
subject,
documented_constraint,
predicate,
} => match field {
"subject" => subject.as_ref().map(|f| f.value_span),
"documented_constraint" => documented_constraint.as_ref().map(|f| f.value_span),
"predicate" => Some(predicate.value_span),
_ => None,
},
d::Body::Case {
subject,
base,
overrides,
} => match field {
"subject" => subject.as_ref().map(|f| f.value_span),
"base" => Some(base.value_span),
"overrides" => Some(overrides.value_span),
_ => None,
},
}
}
#[cfg(test)]
mod tests;
@@ -0,0 +1,352 @@
//! Logical auxiliary storage, separate from model/serde and raw validator heap.
use super::*;
use crate::language::{EditKind, EditValue};
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub(super) struct Slots {
pub entries: usize,
pub occurrences: usize,
pub text: usize,
}
impl Slots {
pub fn entries(entries: usize) -> Self {
Self {
entries,
..Self::default()
}
}
pub fn text(text: usize) -> Self {
Self {
text,
..Self::default()
}
}
pub fn add(self, other: Self, at: Span) -> Result<Self, super::super::super::Error> {
Ok(Self {
entries: add(self.entries, other.entries, at)?,
occurrences: add(self.occurrences, other.occurrences, at)?,
text: add(self.text, other.text, at)?,
})
}
}
pub(super) fn add(a: usize, b: usize, at: Span) -> Result<usize, super::super::super::Error> {
a.checked_add(b)
.ok_or_else(|| error(at, "Mixed auxiliary storage size overflow."))
}
pub(super) fn mul(a: usize, b: usize, at: Span) -> Result<usize, super::super::super::Error> {
a.checked_mul(b)
.ok_or_else(|| error(at, "Mixed auxiliary storage size overflow."))
}
#[derive(Debug, Default)]
pub(super) struct Ledger {
pub live: Slots,
}
impl Ledger {
pub fn reserve(&mut self, budget: &Budget, slots: Slots, at: Span) -> Result<(), TypedError> {
let next = self.live.add(slots, at)?;
budget.lowering_entries(next.entries, at)?;
budget.lowering_occurrences(next.occurrences, at)?;
budget.index_bytes(next.text, at)?;
self.live = next;
Ok(())
}
pub fn release(&mut self, slots: Slots) {
self.live.entries = self
.live
.entries
.checked_sub(slots.entries)
.expect("entry reservation");
self.live.occurrences = self
.live
.occurrences
.checked_sub(slots.occurrences)
.expect("occurrence reservation");
self.live.text = self
.live
.text
.checked_sub(slots.text)
.expect("text reservation");
}
}
/// Exact output-site counts and conservative whole-lowering temporary bound.
/// N expression nodes: at most 2N Visit/Build slots, N borrowed child slots,
/// N value/split slots. A arguments: A signature slots plus A seen-set slots.
/// `Resolver::call` borrows the callee signature. Its parameter Vec/seen set
/// contain only supplied arguments, including on incomplete/invalid calls;
/// no allocation is proportional to the full signature during body lowering.
/// The model expressions/maps themselves are outside this auxiliary ledger.
#[derive(Default)]
pub(super) struct LowerStorage {
pub nodes: usize,
pub occurrences: usize,
pub arguments: usize,
}
impl LowerStorage {
pub fn all(&self) -> Result<Slots, TypedError> {
let at = Span::default();
Ok(Slots {
entries: add(mul(self.nodes, 5, at)?, mul(self.arguments, 2, at)?, at)?,
occurrences: self.occurrences,
text: 0,
})
}
}
enum Item<'a> {
Ast(&'a Ast),
Edit(&'a EditValue),
Expr(&'a Expression),
}
struct Walk<'a, 'b> {
ledger: &'b mut Ledger,
budget: &'b Budget,
work: Vec<Item<'a>>,
lower: LowerStorage,
}
impl<'a> Walk<'a, '_> {
fn text(&mut self, text: &str, at: Span) -> Result<(), TypedError> {
self.ledger
.reserve(self.budget, Slots::text(text.len()), at)
}
fn push(&mut self, item: Item<'a>) -> Result<(), TypedError> {
self.ledger
.reserve(self.budget, Slots::entries(1), Span::default())?;
self.work.push(item);
Ok(())
}
fn reference(&mut self, r: &Reference) -> Result<(), TypedError> {
self.text(&r.name, r.span)?;
self.lower.occurrences = add(self.lower.occurrences, 1, r.span)?;
Ok(())
}
fn optional(&mut self, field: &Option<d::Field<Option<Reference>>>) -> Result<(), TypedError> {
if let Some(r) = field.as_ref().and_then(|f| f.value.as_ref()) {
self.reference(r)?;
}
Ok(())
}
fn ty(&mut self, ty: &d::TypeSyntax) -> Result<(), TypedError> {
if let d::ScalarKind::Quantity(r) = &ty.scalar.kind {
self.reference(r)?;
}
Ok(())
}
fn binding(&mut self, binding: &'a Binding) -> Result<(), TypedError> {
if let Binding::Expression(e) = binding {
self.push(Item::Expr(e))?;
}
self.drain()
}
fn drain(&mut self) -> Result<(), TypedError> {
while let Some(item) = self.work.pop() {
self.ledger.release(Slots::entries(1));
match item {
Item::Ast(ast) => match ast {
Ast::Literal(serde_json::Value::String(s), _)
| Ast::Word(s, _)
| Ast::Reference(s, _) => self.text(s, Span::default())?,
Ast::Array(items, _) => {
for child in items {
self.push(Item::Ast(child))?;
}
}
Ast::Object(fields, _) => {
for (key, child) in fields {
self.text(key, Span::default())?;
self.push(Item::Ast(child))?;
}
}
_ => {}
},
Item::Edit(value) => match &value.kind {
EditKind::Reference(s) => self.text(s, value.span)?,
EditKind::Record(fields) => {
for (key, field) in fields {
self.text(key, field.span)?;
self.push(Item::Edit(&field.value))?;
}
}
EditKind::List(items) => {
for child in items {
self.push(Item::Edit(child))?;
}
}
_ => {}
},
Item::Expr(e) => {
self.lower.nodes = add(self.lower.nodes, 1, e.span)?;
use ExpressionKind as E;
match &e.kind {
E::Quantity { unit, .. } | E::Property(unit) | E::Parameter(unit) => {
self.reference(unit)?
}
E::List(items) => {
for child in items {
self.push(Item::Expr(child))?;
}
}
E::Unary { operand, .. } => self.push(Item::Expr(operand))?,
E::Binary { left, right, .. } => {
self.push(Item::Expr(left))?;
self.push(Item::Expr(right))?;
}
E::If {
condition,
when_true,
when_false,
} => {
self.push(Item::Expr(condition))?;
self.push(Item::Expr(when_true))?;
self.push(Item::Expr(when_false))?;
}
E::Call {
definition,
arguments,
} => {
self.reference(definition)?;
self.lower.arguments =
add(self.lower.arguments, arguments.len(), e.span)?;
for argument in arguments {
self.reference(&argument.parameter)?;
self.push(Item::Expr(&argument.value))?;
}
}
E::Sum { values, initial } => {
self.push(Item::Expr(values))?;
self.push(Item::Expr(initial))?;
}
E::Power { base, .. } => self.push(Item::Expr(base))?,
E::AsKind { kind, value } => {
self.reference(kind)?;
self.push(Item::Expr(value))?;
}
E::Boolean(_) | E::Number(_) => {}
}
}
}
}
Ok(())
}
}
pub(super) fn syntax(
syntax: &ParsedMixedSyntax<'_>,
ledger: &mut Ledger,
) -> Result<LowerStorage, Failure> {
let at = Span::default();
ledger
.reserve(
&syntax.budget,
Slots::entries(
add(syntax.metadata, syntax.budget.usage.entries, at)
.map_err(|e| Failure::typed(syntax, None, e))?,
),
at,
)
.map_err(|e| Failure::typed(syntax, None, e))?;
let mut walk = Walk {
ledger,
budget: &syntax.budget,
work: Vec::new(),
lower: LowerStorage::default(),
};
for (file, unit) in syntax.files.iter().enumerate() {
let mut count = || -> Result<(), TypedError> {
if let Some(header) = &unit.ordinary.header {
for s in [
&header.project.alias,
&header.project.name,
&header.project.description,
] {
walk.text(s, header.span)?;
}
}
for (path, span) in &unit.ordinary.includes {
walk.text(path, *span)?;
}
for declaration in &unit.ordinary.declarations {
walk.text(&declaration.alias, declaration.span)?;
walk.text(&declaration.name, declaration.span)?;
for key in declaration.field_spans.keys() {
walk.text(key, declaration.span)?;
}
for (key, value) in &declaration.fields {
walk.text(key, declaration.span)?;
walk.push(Item::Ast(value))?;
walk.drain()?;
}
}
for (_, position) in &unit.positions {
walk.push(Item::Edit(&position.fields))?;
walk.drain()?;
}
for declaration in &unit.typed {
walk.text(
&declaration.header.alias.name,
declaration.header.alias.span,
)?;
walk.text(&declaration.header.name, declaration.header.name_span)?;
if let Some(f) = &declaration.description {
walk.text(&f.value, f.span)?;
}
match &declaration.body {
d::Body::QuantityKind { algebra, .. } => {
if let d::AlgebraSyntax::Affine(r) = &algebra.value {
walk.reference(r)?;
}
}
d::Body::Unit { kind, symbol, .. } => {
walk.reference(&kind.value)?;
walk.text(&symbol.value, symbol.span)?;
}
d::Body::Calculation {
parameters,
result,
body,
} => {
for p in parameters {
walk.reference(&p.name)?;
walk.ty(&p.value_type)?;
}
walk.ty(&result.value)?;
walk.binding(&body.value)?;
}
d::Body::Property {
subject,
value_type,
base,
display_unit,
} => {
walk.reference(&subject.value)?;
walk.ty(&value_type.value)?;
walk.binding(&base.value)?;
walk.optional(display_unit)?;
}
d::Body::Check {
subject,
documented_constraint,
predicate,
} => {
walk.optional(subject)?;
walk.optional(documented_constraint)?;
walk.binding(&predicate.value)?;
}
d::Body::Case {
subject, overrides, ..
} => {
walk.optional(subject)?;
for entry in &overrides.value {
walk.reference(&entry.target)?;
walk.binding(&entry.binding)?;
}
}
}
}
Ok(())
};
count().map_err(|e| Failure::typed(syntax, Some(file), e))?;
}
Ok(walk.lower)
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -25,6 +25,17 @@ impl<'i, 's> Plan<'i, 's> {
renames: &BTreeMap<Target, String>,
budget: &Budget,
) -> Result<Self, Error> {
let plan = Self::prepare(index, renames, budget)?;
plan.validate(index.sources(), budget)?;
plan.output_lengths(index.sources(), budget)?;
Ok(plan)
}
fn validate_names(
index: &Index<'_>,
renames: &BTreeMap<Target, String>,
budget: &Budget,
) -> Result<(), Error> {
budget.active()?;
budget.lowering_entries(renames.len(), Span::default())?;
let mut bytes = 0usize;
@@ -61,6 +72,16 @@ impl<'i, 's> Plan<'i, 's> {
));
}
}
Ok(())
}
fn prepare(
index: &'i Index<'s>,
renames: &BTreeMap<Target, String>,
budget: &Budget,
) -> Result<Self, Error> {
// Validate record/name storage before copying any owned final names.
Self::validate_names(index, renames, budget)?;
let mut edits = Vec::new();
for (key, location) in index.occurrences() {
if renames
@@ -81,26 +102,20 @@ impl<'i, 's> Plan<'i, 's> {
edit.location.span.end,
)
});
let plan = Self {
Ok(Self {
index,
names: renames.clone(),
edits,
};
plan.validate(index.sources(), budget)?;
Ok(plan)
})
}
pub fn edit_count(&self) -> usize {
self.edits.len()
}
/// Recheck all files, all edit ownership and all final sizes before any
/// output String is allocated. A changed unrelated file is stale too.
fn validate(
&self,
current: &BTreeMap<usize, String>,
budget: &Budget,
) -> Result<BTreeMap<usize, usize>, Error> {
/// Recheck the complete original witness and complete rename intent.
/// A changed unrelated file is stale too. This does not construct output.
fn validate(&self, current: &BTreeMap<usize, String>, budget: &Budget) -> Result<(), Error> {
budget.active()?;
if current != self.index.sources() {
return Err(error(
@@ -110,10 +125,6 @@ impl<'i, 's> Plan<'i, 's> {
}
budget.lowering_entries(current.len(), Span::default())?;
budget.lowering_occurrences(self.edits.len(), Span::default())?;
let mut lengths: BTreeMap<_, _> = current
.iter()
.map(|(&file, source)| (file, source.len()))
.collect();
let mut previous: Option<Location> = None;
let mut seen = BTreeSet::new();
for edit in &self.edits {
@@ -130,8 +141,7 @@ impl<'i, 's> Plan<'i, 's> {
{
return Err(error(at.span, "Stale original token spelling."));
}
let replacement = self
.names
self.names
.get(&edit.key.target)
.ok_or_else(|| error(at.span, "Edit has no final target name."))?;
if previous.is_some_and(|prior| {
@@ -140,13 +150,6 @@ impl<'i, 's> Plan<'i, 's> {
return Err(error(at.span, "Overlapping or unordered source edits."));
}
previous = Some(at);
let length = lengths
.get_mut(&at.file)
.ok_or_else(|| error(at.span, "Unknown edit file."))?;
*length = length
.checked_sub(old.len())
.and_then(|size| size.checked_add(replacement.len()))
.ok_or_else(|| error(at.span, "Updated source size overflow."))?;
}
for (key, at) in self.index.occurrences() {
let required = self
@@ -160,6 +163,28 @@ impl<'i, 's> Plan<'i, 's> {
));
}
}
Ok(())
}
/// Standalone output preflight, after ownership validation. Composition
/// defers this hypothetical typed-only size to its future combined emitter.
fn output_lengths(
&self,
current: &BTreeMap<usize, String>,
budget: &Budget,
) -> Result<BTreeMap<usize, usize>, Error> {
let mut lengths: BTreeMap<_, _> = current
.iter()
.map(|(&file, source)| (file, source.len()))
.collect();
for edit in &self.edits {
let at = edit.location;
let length = lengths.get_mut(&at.file).expect("validated edit file");
*length = length
.checked_sub(at.span.end - at.span.start)
.and_then(|size| size.checked_add(self.names[&edit.key.target].len()))
.ok_or_else(|| error(at.span, "Updated source size overflow."))?;
}
let mut total = 0usize;
for length in lengths.values() {
total = total
@@ -175,7 +200,8 @@ impl<'i, 's> Plan<'i, 's> {
current: &BTreeMap<usize, String>,
budget: &Budget,
) -> Result<BTreeMap<usize, String>, Error> {
let lengths = self.validate(current, budget)?;
self.validate(current, budget)?;
let lengths = self.output_lengths(current, budget)?;
let mut result = BTreeMap::new();
let mut edits = self.edits.iter().peekable();
for (&file, source) in current {
@@ -194,5 +220,97 @@ impl<'i, 's> Plan<'i, 's> {
}
}
/// Private typed-only composition planning. The caller must supply its inherited
/// parse Budget; this type keeps that same immutable budget through export.
/// It does not establish the future complete bundle/file-table/parse-session
/// witness. In particular, an Index currently does not store its parse profile.
pub(super) struct Composition<'b, 'i, 's> {
plan: Plan<'i, 's>,
budget: &'b Budget,
}
impl std::fmt::Debug for Composition<'_, '_, '_> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("Composition")
.field("plan", &self.plan)
.finish_non_exhaustive()
}
}
impl<'b, 'i, 's> Composition<'b, 'i, 's> {
pub fn renames(
index: &'i Index<'s>,
renames: &BTreeMap<Target, String>,
inherited: &'b Budget,
) -> Result<Self, Error> {
let plan = Plan::prepare(index, renames, inherited)?;
plan.validate(index.sources(), inherited)?;
Ok(Self {
plan,
budget: inherited,
})
}
/// Export the entire validated plan against exact original numeric file keys
/// and bytes. There is no output allocation, application or subset-accept API.
/// A future collector must check this actual index witness and the complete
/// combined output bounds before applying either producer's edits once.
pub fn original_edits(
&self,
current: &BTreeMap<usize, String>,
) -> Result<OriginalEdits<'_, 'i, 's>, Error> {
Plan::validate_names(self.plan.index, &self.plan.names, self.budget)?;
self.plan.validate(current, self.budget)?;
Ok(OriginalEdits { plan: &self.plan })
}
}
#[derive(Debug)]
pub(super) struct OriginalEdits<'p, 'i, 's> {
plan: &'p Plan<'i, 's>,
}
impl<'s> OriginalEdits<'_, '_, 's> {
pub fn index(&self) -> &Index<'s> {
self.plan.index
}
pub fn len(&self) -> usize {
self.plan.edits.len()
}
pub fn is_empty(&self) -> bool {
self.plan.edits.is_empty()
}
pub fn iter(&self) -> impl ExactSizeIterator<Item = OriginalEdit<'_>> {
self.plan.edits.iter().map(|edit| OriginalEdit {
key: &edit.key,
location: edit.location,
original: &self.plan.index.sources()[&edit.location.file]
[edit.location.span.start..edit.location.span.end],
replacement: &self.plan.names[&edit.key.target],
})
}
}
/// A borrowed inspection record, never independent edit authority. Only an
/// intact OriginalEdits batch can carry this producer's complete rename intent.
#[derive(Debug)]
pub(super) struct OriginalEdit<'p> {
key: &'p OccurrenceKey,
location: Location,
original: &'p str,
replacement: &'p str,
}
impl OriginalEdit<'_> {
pub fn key(&self) -> &OccurrenceKey {
self.key
}
pub fn location(&self) -> Location {
self.location
}
pub fn original(&self) -> &str {
self.original
}
pub fn replacement(&self) -> &str {
self.replacement
}
}
#[cfg(test)]
mod tests;
@@ -322,3 +322,423 @@ fn output_and_plan_limits_are_aggregate_and_do_not_reset_parse_work() {
limits.entries = 0;
assert!(Plan::renames(&index, &rename_map(), &Budget::new(limits).unwrap()).is_err());
}
#[test]
fn composition_exports_complete_borrowed_swap_witnesses_and_empty_noops() {
use super::super::index::TokenRole;
use super::super::lower::{Child, Field, Role};
type Key = (Id, Option<Field>, Option<Vec<Child>>, TokenRole, Target);
type Row = (Location, String, String, Key);
let sources = sources();
let mut work = budget();
let (index, _) = Index::parse(&skeleton(), &sources, &mut work).unwrap();
let usage = work.usage;
let numeric = work.numeric_units();
let composition = Composition::renames(&index, &rename_map(), &work).unwrap();
assert!(std::ptr::eq(composition.budget, &work));
// A byte-identical snapshot is valid; records still borrow the actual index.
let batch = composition.original_edits(&sources.clone()).unwrap();
assert!(std::ptr::eq(batch.index(), &index));
assert_eq!(batch.len(), 13);
assert!(!batch.is_empty());
assert_eq!(batch.iter().len(), 13);
// Independently authored occurrence identities and full source contexts.
// No index offset/path or generated output defines this oracle.
let mut expected: Vec<Row> = Vec::new();
let mut add = |file, context: &str, prefix: &str, old: &str, new: &str, key: Key| {
let start = sources[&file].find(context).unwrap() + prefix.len();
assert_eq!(&sources[&file][start..start + old.len()], old);
expected.push((
Location {
file,
span: Span {
start,
end: start + old.len(),
},
},
old.into(),
new.into(),
key,
));
};
for (id, old, new, context) in [
(0x100, "f", "g", "calculation f "),
(0x200, "g", "f", "calculation g "),
] {
add(
11,
context,
"calculation ",
old,
new,
(
Id::from_u128(id),
None,
None,
TokenRole::Declaration,
global(id),
),
);
}
for (id, old, new, context, edge, prefix) in [
(0x200, "a", "b", "input a id", Child::Left, "body $"),
(0x201, "b", "a", "input b id", Child::Right, "body $a + $"),
] {
let target = parameter(0x100, id);
add(
11,
context,
"input ",
old,
new,
(
Id::from_u128(0x100),
Some(Field::Input(a::ParameterId(Id::from_u128(id)))),
None,
TokenRole::Use(Role::ParameterDeclaration),
target,
),
);
add(
11,
"body $a + $b;",
prefix,
old,
new,
(
Id::from_u128(0x100),
Some(Field::Body),
Some(vec![edge]),
TokenRole::Use(Role::ParameterUse),
target,
),
);
}
for (context, path) in [
("@f(b: 2, a: 1)", vec![Child::Left, Child::Left]),
("@f(a: 3, b: 4)", vec![Child::Left, Child::Right]),
] {
add(
23,
context,
"@",
"f",
"g",
(
Id::from_u128(0x300),
Some(Field::Body),
Some(path.clone()),
TokenRole::Use(Role::CallTarget),
global(0x100),
),
);
for (id, old, new) in [(0x200, "a", "b"), (0x201, "b", "a")] {
let offset = context.find(&format!("{old}:")).unwrap();
add(
23,
context,
&context[..offset],
old,
new,
(
Id::from_u128(0x300),
Some(Field::Body),
Some(path.clone()),
TokenRole::Use(Role::ArgumentLabel),
parameter(0x100, id),
),
);
}
}
add(
23,
"@g(a: 5, b: 6)",
"@",
"g",
"f",
(
Id::from_u128(0x300),
Some(Field::Body),
Some(vec![Child::Right]),
TokenRole::Use(Role::CallTarget),
global(0x200),
),
);
expected.sort_by_key(|row| (row.0.file, row.0.span.start, row.0.span.end));
let actual: Vec<Row> = batch
.iter()
.map(|edit| {
let key = edit.key();
(
edit.location(),
edit.original().into(),
edit.replacement().into(),
(
key.owner,
key.field,
key.path.as_ref().map(|path| path.edges()),
key.role,
key.target,
),
)
})
.collect();
assert_eq!(actual, expected);
let noop = Composition::renames(
&index,
&BTreeMap::from([
(global(0x100), "f".into()),
(parameter(0x200, 0x200), "a".into()),
]),
&work,
)
.unwrap();
let empty = noop.original_edits(&sources).unwrap();
assert!(empty.is_empty());
assert_eq!(empty.len(), 0);
assert_eq!(empty.iter().count(), 0);
assert_eq!(work.usage, usage);
assert_eq!(work.numeric_units(), numeric);
}
#[test]
fn composition_rejects_stale_or_damaged_complete_batches_before_export() {
let mut sources = sources();
sources.insert(99, "// unrelated 雪\r\n".into());
let original = sources.clone();
let mut work = budget();
let (index, _) = Index::parse(&skeleton(), &sources, &mut work).unwrap();
for damage in 0..10 {
let mut current = sources.clone();
let mut composition = Composition::renames(&index, &rename_map(), &work).unwrap();
match damage {
0 => current.get_mut(&99).unwrap().push_str("changed"),
1 => {
current.remove(&23);
}
2 => {
current.insert(101, String::new());
}
3 => composition.plan.edits[0].location.span.start += 1,
4 => composition
.plan
.edits
.insert(1, composition.plan.edits[0].clone()),
5 => {
composition.plan.edits.pop();
}
6 => composition.plan.edits.swap(0, 1),
7 => {
composition
.plan
.names
.insert(global(0x100), "bad name".into());
}
8 => {
composition.plan.names.insert(global(0x100), "f".into());
}
_ => {
composition.plan.names.insert(global(999), "missing".into());
}
}
let before = current.clone();
assert!(
composition.original_edits(&current).is_err(),
"damage {damage}"
);
assert_eq!(current, before);
}
assert_eq!(sources, original);
let failure = work.frame(257, Span::default()).unwrap_err();
assert_eq!(
Composition::renames(&index, &rename_map(), &work).unwrap_err(),
failure
);
}
#[test]
fn composition_defers_only_typed_output_size_with_independent_shrink_arithmetic() {
let sources = sources();
let cap = sources.values().map(String::len).sum();
let mut limits = Limits::source8(ParseLimits::UNLIMITED);
limits.bytes = cap;
let mut work = Budget::new(limits).unwrap();
let (index, _) = Index::parse(&skeleton(), &sources, &mut work).unwrap();
assert_eq!(work.usage.bytes, cap);
let usage = work.usage;
let numeric = work.numeric_units();
let names = BTreeMap::from([(global(0x100), "ff".into())]);
assert_eq!(
Plan::renames(&index, &names, &work).unwrap_err().message,
"Source8 index/output byte limit exceeded."
);
let composition = Composition::renames(&index, &names, &work).unwrap();
let batch = composition.original_edits(&sources).unwrap();
assert_eq!(batch.len(), 3); // declaration and two distinct call targets
let growth: usize = batch
.iter()
.map(|edit| edit.replacement().len() - edit.original().len())
.sum();
assert_eq!(growth, 3);
assert_eq!(
composition.plan.apply(&sources, &work).unwrap_err().message,
"Source8 index/output byte limit exceeded."
);
// Independent future ordinary/scalar producer witness on untouched display
// name bytes. This is size/offset arithmetic only: this typed-only harness
// has no ordinary collector, mixed ownership proof or accepted output.
let start = DEFINITIONS.find("\"f a b\"").unwrap() + 1;
let shrink = Location {
file: 11,
span: Span {
start,
end: start + "f a b".len(),
},
};
assert_eq!(
&sources[&shrink.file][shrink.span.start..shrink.span.end],
"f a b"
);
for edit in batch.iter() {
let at = edit.location();
assert!(
at.file != shrink.file
|| at.span.end <= shrink.span.start
|| shrink.span.end <= at.span.start
);
}
let original_len = shrink.span.end - shrink.span.start;
assert_eq!(cap + growth - original_len + "fa".len(), cap);
assert_eq!(cap + growth - original_len + "fab".len(), cap + 1);
assert_eq!(work.usage, usage);
assert_eq!(work.numeric_units(), numeric);
}
#[test]
fn composition_keeps_name_storage_and_rename_record_limits_before_allocation() {
let sources = sources();
let cap = sources.values().map(String::len).sum();
let mut limits = Limits::source8(ParseLimits::UNLIMITED);
limits.bytes = cap;
let mut work = Budget::new(limits).unwrap();
let (index, _) = Index::parse(&skeleton(), &sources, &mut work).unwrap();
let exact = BTreeMap::from([(global(0x100), "x".repeat(cap))]);
assert_eq!(
Composition::renames(&index, &exact, &work)
.unwrap()
.original_edits(&sources)
.unwrap()
.len(),
3
);
let over = BTreeMap::from([(global(0x100), "x".repeat(cap + 1))]);
assert_eq!(
Composition::renames(&index, &over, &work)
.unwrap_err()
.message,
"Source8 index/output byte limit exceeded."
);
for invalid in [
BTreeMap::from([(global(999), "unknown".into())]),
BTreeMap::from([(global(0x100), "bad name".into())]),
BTreeMap::from([(global(0x100), "g".into())]),
BTreeMap::from([(parameter(0x100, 0x200), "b".into())]),
] {
assert!(Composition::renames(&index, &invalid, &work).is_err());
}
// A typed-empty fixture isolates the rename-map record bound without
// syntax entry usage masking it. No ordinary edits are authorized here.
let ordinary = skeleton();
let count = ordinary.identities().len();
let files = BTreeMap::from([(1, "// empty typed unit\n".into())]);
let mut limits = Limits::source8(ParseLimits::UNLIMITED);
limits.entries = count;
limits.references = 0;
let mut inherited = Budget::new(limits).unwrap();
let (index, _) = Index::parse(&ordinary, &files, &mut inherited).unwrap();
let mut names: BTreeMap<_, _> = index
.names()
.keys()
.enumerate()
.map(|(i, target)| (*target, format!("renamed_{i}")))
.collect();
assert_eq!(names.len(), count);
assert!(
Composition::renames(&index, &names, &inherited)
.unwrap()
.original_edits(&files)
.unwrap()
.is_empty()
);
names.insert(global(999), "additional".into());
assert_eq!(
Composition::renames(&index, &names, &inherited)
.unwrap_err()
.message,
"Source8 lowering entry limit exceeded."
);
}
#[test]
fn composition_checks_edit_record_ceiling_before_duplicate_membership() {
let sources = sources();
let mut limits = Limits::source8(ParseLimits::UNLIMITED);
limits.entries = 128;
limits.references = 32;
let ceiling = limits.entries + limits.references;
let mut work = Budget::new(limits).unwrap();
let (index, _) = Index::parse(&skeleton(), &sources, &mut work).unwrap();
let mut composition = Composition::renames(&index, &rename_map(), &work).unwrap();
assert_eq!(composition.original_edits(&sources).unwrap().len(), 13);
let duplicate = composition.plan.edits[0].clone();
composition.plan.edits = vec![duplicate.clone(); ceiling];
// At the exact count bound validation proceeds to the independent ownership
// error. The damaged plan never becomes a successful/authoritative batch.
assert_eq!(
composition.original_edits(&sources).unwrap_err().message,
"Forged or duplicate edit occurrence."
);
composition.plan.edits.push(duplicate);
assert_eq!(
composition.original_edits(&sources).unwrap_err().message,
"Source8 lowering occurrence limit exceeded."
);
}
#[test]
fn standalone_noop_apply_keeps_tighter_file_only_budget_and_output_boundary() {
let sources = sources();
let mut inherited = budget();
let (index, _) = Index::parse(&skeleton(), &sources, &mut inherited).unwrap();
let plan = Plan::renames(
&index,
&BTreeMap::from([(global(0x100), "f".into())]),
&inherited,
)
.unwrap();
assert_eq!(plan.edit_count(), 0);
assert!(index.names().len() > sources.len());
let mut limits = Limits::source8(ParseLimits::UNLIMITED);
limits.entries = sources.len();
limits.references = 0;
limits.bytes = sources.values().map(String::len).sum();
// Standalone apply checks file/edit records, not the already validated
// catalog/name-storage allocation from construction. Rechecking the catalog
// under this intentionally tighter probe budget would reject this no-op.
assert_eq!(
plan.apply(&sources, &Budget::new(limits).unwrap()).unwrap(),
sources
);
limits.bytes -= 1;
assert_eq!(
plan.apply(&sources, &Budget::new(limits).unwrap())
.unwrap_err()
.message,
"Source8 index/output byte limit exceeded."
);
}
@@ -194,6 +194,8 @@ pub(super) fn row_key(source: &str, origins: &BTreeMap<Id, Id>) -> Option<String
cursor: 0,
value_depth: 0,
limits: ParseLimits::UNLIMITED,
#[cfg(test)]
mixed: None,
};
let ast = parser.value().ok()?;
let facet = serde_json::from_value(raw(&ast).ok()?).ok()?;
+7 -147
View File
@@ -8,6 +8,7 @@ use std::collections::{BTreeMap, BTreeSet};
mod source_migration;
#[path = "source_project/source_structure.rs"]
mod source_structure;
pub(crate) mod traversal;
/// A portable slash-separated path, checked at every bundle boundary.
pub type LogicalPath = String;
@@ -304,71 +305,7 @@ fn compile_once(
bundle: &SourceBundle,
limits: ProjectLimits,
) -> Result<(CompiledSourceProject, Edits), Vec<SourceDiagnostic>> {
let fail = |code, message| {
vec![problem(
bundle,
&bundle.entry,
Span::default(),
code,
message,
)]
};
if bundle.entry.contains('/') {
return Err(fail(
"invalid_path",
"The entry file must be at the source project root.".into(),
));
}
if bundle.files.len() > limits.max_files {
return Err(fail(
"project_limit",
format!("Source project exceeds {} files.", limits.max_files),
));
}
let bytes = bundle
.files
.values()
.fold(0usize, |sum, text| sum.saturating_add(text.len()));
if bytes > limits.model.max_source_bytes {
return Err(fail(
"source_limit",
format!(
"Combined source exceeds {} bytes.",
limits.model.max_source_bytes
),
));
}
let mut folded = BTreeMap::new();
for path in std::iter::once(&bundle.entry).chain(bundle.files.keys()) {
if resolve_include("", path).as_ref() != Ok(path) {
return Err(fail(
"invalid_path",
format!("Non-canonical source path '{path}'."),
));
}
if let Some(previous) = folded.insert(path.to_lowercase(), path)
&& previous != path
{
return Err(fail(
"duplicate_path",
format!("Case-colliding source paths '{previous}' and '{path}'."),
));
}
}
if !bundle.files.contains_key(&bundle.entry) {
return Err(fail(
"missing_include",
"The entry file is missing from the source bundle.".into(),
));
}
enum Visit {
Enter(LogicalPath, usize, Option<SourceLocation>),
Leave,
}
let mut stack = vec![Visit::Enter(bundle.entry.clone(), 0, None)];
let mut active: Vec<LogicalPath> = Vec::new();
let mut visited = BTreeSet::new();
let mut traversal = traversal::Traversal::new(bundle, limits)?;
let mut declarations = Vec::new();
let mut declaration_paths = Vec::new();
let mut aliases = BTreeMap::new();
@@ -381,51 +318,8 @@ fn compile_once(
let mut includes = Vec::new();
let mut insertions: Edits = BTreeMap::new();
let mut graph_count = 0usize;
while let Some(visit) = stack.pop() {
let Visit::Enter(path, depth, origin) = visit else {
active.pop();
continue;
};
let at = origin.unwrap_or(SourceLocation {
path: path.clone(),
span: Span::default(),
});
if let Some(start) = active.iter().position(|file| file == &path) {
let chain = active[start..]
.iter()
.chain(std::iter::once(&path))
.cloned()
.collect::<Vec<_>>()
.join(" -> ");
return Err(vec![problem(
bundle,
&at.path,
at.span,
"include_cycle",
format!("Include cycle: {chain}."),
)]);
}
if depth > limits.max_include_depth {
return Err(vec![problem(
bundle,
&at.path,
at.span,
"project_limit",
format!("Include depth exceeds {}.", limits.max_include_depth),
)]);
}
if !visited.insert(path.clone()) {
continue;
}
let Some(source) = bundle.files.get(&path) else {
return Err(vec![problem(
bundle,
&at.path,
at.span,
"missing_include",
format!("Included file '{path}' is missing."),
)]);
};
while let Some(path) = traversal.next()? {
let source = &bundle.files[&path];
let mut unit = language::parse_unit(source, path == bundle.entry, limits.model).map_err(
|diagnostic| {
vec![SourceDiagnostic {
@@ -513,46 +407,12 @@ fn compile_once(
declarations.push(declaration);
declaration_paths.push(path.clone());
}
let mut children = Vec::new();
for (raw, span) in unit.includes {
let target = resolve_include(&path, &raw)
.map_err(|message| vec![problem(bundle, &path, span, "invalid_path", message)])?;
let location = SourceLocation {
path: path.clone(),
span,
};
includes.push(IncludeLocation {
location: location.clone(),
target: target.clone(),
});
if includes.len() > limits.max_include_edges {
return Err(vec![problem(
bundle,
&path,
span,
"project_limit",
format!("Include graph exceeds {} edges.", limits.max_include_edges),
)]);
}
children.push(Visit::Enter(
target,
depth.saturating_add(1),
Some(location),
));
includes.push(traversal.include(&path, &raw, span)?);
}
active.push(path);
stack.push(Visit::Leave);
stack.extend(children.into_iter().rev());
}
if let Some(path) = bundle.files.keys().find(|path| !visited.contains(*path)) {
return Err(vec![problem(
bundle,
path,
Span::default(),
"unreachable_file",
"Supplied source file is not reachable from the entry file.",
)]);
traversal.finish_file(path);
}
traversal.finish()?;
let header = project_header.expect("validated entry header");
let index = SourceIndex {
declarations: declaration_locations,
@@ -0,0 +1,258 @@
//! Shared include traversal policy for ordinary compilation and private syntax.
use super::*;
enum Visit {
Enter(LogicalPath, usize, Option<SourceLocation>),
Leave,
}
pub(crate) struct Traversal<'a> {
bundle: &'a SourceBundle,
limits: ProjectLimits,
stack: Vec<Visit>,
active: Vec<LogicalPath>,
visited: BTreeSet<LogicalPath>,
children: Vec<Visit>,
depth: usize,
edges: usize,
}
impl<'a> Traversal<'a> {
pub(crate) fn new(
bundle: &'a SourceBundle,
limits: ProjectLimits,
) -> Result<Self, Vec<SourceDiagnostic>> {
let fail = |code, message| {
vec![problem(
bundle,
&bundle.entry,
Span::default(),
code,
message,
)]
};
if bundle.entry.contains('/') {
return Err(fail(
"invalid_path",
"The entry file must be at the source project root.".into(),
));
}
if bundle.files.len() > limits.max_files {
return Err(fail(
"project_limit",
format!("Source project exceeds {} files.", limits.max_files),
));
}
let bytes = bundle
.files
.values()
.fold(0usize, |sum, text| sum.saturating_add(text.len()));
if bytes > limits.model.max_source_bytes {
return Err(fail(
"source_limit",
format!(
"Combined source exceeds {} bytes.",
limits.model.max_source_bytes
),
));
}
let mut folded = BTreeMap::new();
for path in std::iter::once(&bundle.entry).chain(bundle.files.keys()) {
if resolve_include("", path).as_ref() != Ok(path) {
return Err(fail(
"invalid_path",
format!("Non-canonical source path '{path}'."),
));
}
if let Some(previous) = folded.insert(path.to_lowercase(), path)
&& previous != path
{
return Err(fail(
"duplicate_path",
format!("Case-colliding source paths '{previous}' and '{path}'."),
));
}
}
if !bundle.files.contains_key(&bundle.entry) {
return Err(fail(
"missing_include",
"The entry file is missing from the source bundle.".into(),
));
}
Ok(Self {
bundle,
limits,
stack: vec![Visit::Enter(bundle.entry.clone(), 0, None)],
active: Vec::new(),
visited: BTreeSet::new(),
children: Vec::new(),
depth: 0,
edges: 0,
})
}
pub(crate) fn next(&mut self) -> Result<Option<LogicalPath>, Vec<SourceDiagnostic>> {
while let Some(visit) = self.stack.pop() {
let Visit::Enter(path, depth, origin) = visit else {
self.active.pop();
continue;
};
let at = origin.unwrap_or(SourceLocation {
path: path.clone(),
span: Span::default(),
});
if let Some(start) = self.active.iter().position(|file| file == &path) {
let chain = self.active[start..]
.iter()
.chain(std::iter::once(&path))
.cloned()
.collect::<Vec<_>>()
.join(" -> ");
return Err(vec![problem(
self.bundle,
&at.path,
at.span,
"include_cycle",
format!("Include cycle: {chain}."),
)]);
}
if depth > self.limits.max_include_depth {
return Err(vec![problem(
self.bundle,
&at.path,
at.span,
"project_limit",
format!("Include depth exceeds {}.", self.limits.max_include_depth),
)]);
}
if !self.visited.insert(path.clone()) {
continue;
}
if !self.bundle.files.contains_key(&path) {
return Err(vec![problem(
self.bundle,
&at.path,
at.span,
"missing_include",
format!("Included file '{path}' is missing."),
)]);
}
self.depth = depth;
return Ok(Some(path));
}
Ok(None)
}
pub(crate) fn include(
&mut self,
path: &str,
raw: &str,
span: Span,
) -> Result<IncludeLocation, Vec<SourceDiagnostic>> {
let target = resolve_include(path, raw)
.map_err(|message| vec![problem(self.bundle, path, span, "invalid_path", message)])?;
let location = SourceLocation {
path: path.into(),
span,
};
if self.edges >= self.limits.max_include_edges {
return Err(vec![problem(
self.bundle,
path,
span,
"project_limit",
format!(
"Include graph exceeds {} edges.",
self.limits.max_include_edges
),
)]);
}
self.edges += 1;
self.children.push(Visit::Enter(
target.clone(),
self.depth.saturating_add(1),
Some(location.clone()),
));
Ok(IncludeLocation { location, target })
}
pub(crate) fn finish_file(&mut self, path: LogicalPath) {
self.active.push(path);
self.stack.push(Visit::Leave);
self.stack.extend(self.children.drain(..).rev());
}
pub(crate) fn finish(self) -> Result<(), Vec<SourceDiagnostic>> {
if let Some(path) = self
.bundle
.files
.keys()
.find(|path| !self.visited.contains(*path))
{
return Err(vec![problem(
self.bundle,
path,
Span::default(),
"unreachable_file",
"Supplied source file is not reachable from the entry file.",
)]);
}
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn ordinary_compile_once_keeps_include_order_and_cycle_source_location() {
let mut bundle = SourceBundle {
entry: "main.arcadia".into(),
files: BTreeMap::from([
("main.arcadia".into(), "arcadia 7; project p \"P\" id \"00000000-0000-0000-0000-000000000001\" { include \"dir/left.arcadia\"; include \"dir/right.arcadia\"; }".into()),
("dir/left.arcadia".into(), "include \"../shared.arcadia\";".into()),
("dir/right.arcadia".into(), "include \"../shared.arcadia\";".into()),
("shared.arcadia".into(), "// one shared ordinary fragment\n".into()),
]),
};
let original = bundle.clone();
// This is the existing initial ordinary pass, before schema migration's
// separate output/reparse. It makes no public source8 acceptance claim.
let (compiled, insertions) =
super::super::compile_once(&bundle, ProjectLimits::default()).unwrap();
assert!(insertions.is_empty());
assert_eq!(compiled.bundle, original);
assert_eq!(compiled.index.declarations.len(), 1);
assert_eq!(
compiled
.index
.includes
.iter()
.map(|i| (i.location.path.as_str(), i.target.as_str()))
.collect::<Vec<_>>(),
[
("main.arcadia", "dir/left.arcadia"),
("main.arcadia", "dir/right.arcadia"),
("dir/left.arcadia", "shared.arcadia"),
("dir/right.arcadia", "shared.arcadia"),
]
);
let cycle = "// 雪\r\ninclude \"main.arcadia\";";
bundle.files.insert("shared.arcadia".into(), cycle.into());
let before = bundle.clone();
let errors = super::super::compile_once(&bundle, ProjectLimits::default()).unwrap_err();
assert_eq!(errors[0].diagnostic.code, "include_cycle");
assert_eq!(errors[0].path, "shared.arcadia");
assert_eq!(errors[0].diagnostic.line, 2);
assert_eq!(
&cycle[errors[0].diagnostic.span.start..errors[0].diagnostic.span.end],
"include \"main.arcadia\";"
);
assert_eq!(
errors[0].diagnostic.message,
"Include cycle: main.arcadia -> dir/left.arcadia -> shared.arcadia -> main.arcadia."
);
assert_eq!(bundle, before);
}
}
+87 -5
View File
@@ -180,14 +180,96 @@ fn check_ids(out: &mut Vec<Issue>, owner: Id, field: &str, ids: &[Id]) {
}
pub fn validate(p: &Project) -> Vec<Issue> {
validate_with_usage(p).0
}
/// Runs the shared static/architectural route once and retains successful static
/// metadata without retaining a Project-borrowing environment. Architectural
/// issue production is outside the static meter and any caller auxiliary ledger;
/// callers must reserve separately before mapping or retaining these issues.
pub(crate) fn validate_with_usage(
p: &Project,
) -> (
Vec<Issue>,
Option<(
crate::analysis::AdmissionLimits,
crate::analysis::AdmissionUsage,
)>,
) {
// Bound analysis trees before architectural visitors can clone an element.
// Admission itself never enters this public validator recursively.
if let Err(failure) = crate::analysis::admit_structure_checked(p, crate::analysis::AdmissionLimits::REFERENCE) {
let mut out = Vec::new();
error(&mut out, failure.object, &failure.code, &failure.field, failure.message);
return out;
let environment = match crate::analysis::admit_structure_checked(
p,
crate::analysis::AdmissionLimits::REFERENCE,
) {
Ok(environment) => environment,
Err(failure) => {
let mut out = Vec::new();
error(
&mut out,
failure.object,
&failure.code,
&failure.field,
failure.message,
);
return (out, None);
}
};
let metadata = (
environment.admission_limits(),
environment.admission_usage(),
);
drop(environment);
(validate_structure(p), Some(metadata))
}
#[cfg(test)]
mod shared_admission_tests {
use super::*;
#[test]
fn successful_static_metadata_survives_architectural_warnings() {
let mut project = Project::empty();
project.name.clear();
let (issues, metadata) = validate_with_usage(&project);
let (limits, usage) = metadata.expect("static admission succeeds");
assert_eq!(limits, crate::analysis::AdmissionLimits::REFERENCE);
assert!(usage.static_events > 0);
assert_eq!(usage.stored_expression_nodes, 0);
assert!(issues.iter().any(|issue| issue.code == "unnamed"
&& issue.element == Some(project.id)
&& issue.severity == Severity::Warning));
}
#[test]
fn static_failure_returns_no_metadata_and_skips_architectural_warnings() {
use crate::analysis::{
AnalysisType, Binding, CalculationDefinition, Definition, DefinitionBody, ScalarType,
};
let mut project = Project::empty();
project.name.clear();
let key = Id::from_u128(10);
project.definitions.insert(
key,
Definition {
id: Id::from_u128(11),
alias: "f".into(),
name: "F".into(),
description: String::new(),
body: DefinitionBody::Calculation(CalculationDefinition {
parameters: Vec::new(),
result: AnalysisType::Scalar(ScalarType::Number),
body: Binding::Unbound,
}),
},
);
let (issues, metadata) = validate_with_usage(&project);
assert!(metadata.is_none());
assert_eq!(issues.len(), 1);
assert_eq!(issues[0].severity, Severity::Error);
assert_eq!(issues[0].element, Some(key));
assert_eq!(issues[0].code, "identity_mismatch");
}
validate_structure(p)
}
/// Architecture/header validation without recursively re-entering analysis admission.
@@ -0,0 +1,36 @@
# Private mixed Project implementation
Source-only checkpoint; compilation and runtime acceptance are pending. Base is `a578ac98a06772a60bec00ae287e65650680d119`. The accepted contract is `aecee2aff3f4410648a501a70cb2e6160c81adc5:harness/analysis-mixed-project-contract-20260923.md`; this note records implementation choices, not a replacement specification.
`mixed::project::resolve` consumes the actual complete syntax witness. The sealed result retains Project, warnings, static admission metadata, original syntax/Budget/ProjectLimits/bundle, unchanged expression/occurrence vectors and their canonical file keys. Only Project/warnings are exposed read-only. The empty Project header is copied into model ownership; the original header remains in syntax. No declarations, typed models, tokens or Exact values are reparsed to establish authority.
The catalog seeds the entry first and merges ordinary/typed headers by original declaration start within canonical file order. It checks alias before global ID, retaining both occurrence owners on collision. Existing typed-only catalog precedence remains unchanged. Parameter names/IDs remain calculation scoped. Ordinary construction is the shared existing loop with a pre-field hook; its legacy wrapper preserves source3 scratch and all migration/admission tails. Typed lowering shares the existing body resolver over borrowed declarations and one complete catalog. Moved model records must be disjoint before insertion.
## Live auxiliary storage
Every reservation uses checked arithmetic and tests the aggregate through the inherited Budget's noncharging entry, occurrence and owned-byte checks. No new Budget or reset occurs. Failure aborts the phase. Logical slots are not allocator capacity/RSS.
| Representation | Reservation and end of lifetime |
| --- | --- |
| Original syntax | Retained mixed metadata subtotal plus parsed syntax entries. Includes the existing conservative dropped value-end subtotal. Tokens/positions are not counted a second time. An iterative walk counts separately owned strings, with each counting-stack push reserved first. Source bytes stay separately charged once. Baseline remains live. |
| Complete catalog | Lower aliases+global ID set: two entries/global and owned alias bytes. Scoped parameter map: one entry/calculation plus one entry/name and copied name bytes. Per-signature temporary ID set reserves one entry/parameter and releases after `Tables::typed` returns. |
| Alias projection and diagnostic owners | One entry+copied alias bytes/global for projection; one entry/global for owner registry. Projection drops after construction, owner registry after diagnostic mapping. Both overlap the lower catalog initially. |
| Ordinary coordination | One borrowed `(file,declaration)` entry/ordinary occurrence. One deferred-node entry before that declaration's field cloning. Release after construction/attachment returns and the views drop. |
| Typed lowering | Bounded syntax walk counts N expressions, A supplied call arguments, O emitted occurrences. Reserve `5N+2A` entries and O occurrences before lowering. N entries are retained expression sites. Temporaries are bounded by 2N Visit/Build slots, N borrowed child slots, N value/split slots, A call-parameter slots and A seen-set slots. Hold this conservative whole-phase allowance until all Resolver temporaries die; then release only the difference from actual retained global sites. |
| Mapped diagnostics | Before retention, reserve one output slot, copied canonical path bytes and transferred Issue code/field/message String bytes. No line cache is allocated. Owner registry and syntax/sites remain live during mapping. Failure cannot produce truncated-warning success. |
The call signature is borrowed from the already accounted catalog; no lowerer allocation grows with unprovided callee parameters. Unknown/incomplete calls cannot exceed the supplied-argument bound. O includes parameter declarations, scalar-type references, non-null optional references, call targets/labels and expression reference forms; headers and display text count only as text. Failed lowering drops all partial output with its reservation.
Model/serde allocations and architectural validation's raw Issue vector/strings/indexes remain explicitly outside this preallocation ledger. Model limits are checked on the complete Project; static analysis retains its existing independent meter. Raw Issues are never retained uncharged in an accepted result. Rejection diagnostics are small error transport outside accepted-witness storage, so exhaustion itself remains reportable.
The ordinary hook selects exactly Process/FunctionalChain/Scenario steps, Occurrence Branch occurrences, and ContextualLink source then target reference arrays before field resolution/cloning. Repeated items count separately; scalar parent/owner, endpoint occurrence, unrelated fields and every Trace baseline snapshot are excluded. Recognizable malformed arrays are charged before the unchanged decoder reports their shape errors.
## Admission, mapping and verification
`validation::validate_with_usage` performs one static admission, copies owned usage/limits, drops the borrowing environment, then performs one architecture validation. `validate` uses the same route and returns its Issues. Static failure preserves the original short circuit. The mixed path maps all Issues and refuses any error. Known typed fields map precisely; unsupported semantic field paths, including the current aggregate `parameters`, explicitly fall back to their owner. Exact scoped-parameter lowerer failures retain their original file and full span; UUID substrings never manufacture precision.
Authored tests cover whole OA/SA models and exact warnings; a complete 15-expression/18-occurrence arena oracle with repeated/reordered calls, scoped UUID reuse and coincident file offsets; catalog collisions; ordinary reference selectors/cumulative limits; simultaneous storage and diagnostic-path copying; inherited nested records; legacy scratch/default/errors; static/architecture refusal; parameter diagnostic fallback; large-signature incomplete calls; and 128 KiB success/rejection cleanup. The full model/site oracle was independently designed before implementation. Tests are unexecuted at this checkpoint.
Required later gates: Windows listing, focused new Project tests, complete private typed tests (including lower/index/update and small-stack controls), existing public parser/traversal/migration regressions with preserved baseline disposition, and strict all-target core Clippy. The known inherited formatting/full-suite failures must remain explicit. Source formatting is a parsing check, not typechecking or runtime evidence.
Index construction/verification, ordinary occurrence indexing, old-updater access, public source8 activation, stabilization, mixed edit composition, output recompile/admission, intended-Project equality and root Scenario/editor reconciliation remain deferred. `index.rs` and `update.rs` are unchanged.
@@ -1 +0,0 @@
* -text
@@ -1,46 +0,0 @@
# Private S3 Windows evidence
Source `1b1c3e3abb6fc3d0934c9dbfbd21fddc8ae4f594`, following `90fae14818f684bf778640be2e613f34e3c7e999`. This checkpoint verifies the private typed index/rename slice and the rejected-depth syntax cleanup. **It does not establish current full-core runtime, public source8, integration, native UI or release acceptance.** Parent owns review/publication and integration.
| Recorded scope | Result |
| --- | --- |
| Full library listing | 167 test names; original 166 retained plus one cleanup test |
| Typed source runtime | 59 passed, including all 9 index and 5 update tests |
| Existing raw-token control | 1 passed |
| Three focused 128 KiB controls | Original whole-flow, eight-parent cleanup and synthetic 256-frame Path tests passed |
| Strict core, all targets | Clippy with `-D warnings`, exit0 |
| Original90fae typed gate | Stack overflow retained; aborted run is not a passing full gate |
| Temporary diagnostics/probes | Two dirty overflow reproductions; four dirty direct-parser malformed-input probes passed |
Actual platform was Windows x86_64-pc-windows-gnullvm, Rust/Cargo 1.98.1, one compiler job. Every clean gate binds 538 committed inputs before/after, exact commands and source. Raw source manifests, logs, receipts, helper scripts and diagnostic diffs remain byte-for-byte unchanged in [evidence.zip](evidence.zip). Its SHA256 is `4486f4c42b140ae4b9b3caf0aacdf151a2acb1fd85eda09a57764fd69e980574`; 658251 bytes, 61 members. No executables, caches or profiles are included.
The failure occurred when `Cursor::node` recursively dropped a rejected depth129 syntax tree below parser frames. Complete depth128 parsing/indexing/renaming/output/model equality/admission and earlier cleanup had finished. The source fix uses iterative rejected-kind disposal and preserves the existing stack request, depth limit and assertions. The 14 original S3 tests and the added mixed-shape cleanup test now have actual runtime evidence. Ordinary AST destruction and all possible parser-error paths are not claimed iterative or universally stack-safe.
## Reproduce evidence verification
With Python 3 standard library and Git, from any working directory:
```text
python -B PATH/verify.py --repo PATH/TO/GIT/REPOSITORY
python -B PATH/negative_controls.py --repo PATH/TO/GIT/REPOSITORY
```
The repository needs immutable commits `1b1c3e3`, `90fae148` and `1f828323` (the publication commit contains that ancestry). Verification reads commit blobs; it neither depends on nor changes the current checkout. `--evidence-dir PATH` optionally selects another complete extracted payload directory. No archived helper, compiler or native artifact is executed. Optimized Python is refused.
The verifier checks both manifests/ZIP byte integrity, actual listing and exact selected runtime membership, 538 committed inputs, recorded commands/times/environment, strict completion, historical source-artifact manifests and the original aborted gate. It reconstructs the dirty source patches in memory and verifies the corresponding 538 test-input hashes. Broader tracked-file hashes also remain in the original captures; the verifier does not claim to audit unrelated tracked files outside that test-input scope. Binary identities are checked against raw executable-path receipts, not rehashed executable bytes, because binaries are deliberately absent. Initial focused tests and final restored typed/token/listing tests were separate builds of identical source and have their own binary receipts.
[checks.json](checks.json) records the positive working evidence audit and 13 discriminating negative controls. Those controls corrupt evidence only in memory; raw archives stay intact. Semantic cases reach their specific checks instead of failing only at the outer ZIP checksum. Package verification also runs against the saved commit and a fresh extraction; exact publication-commit readback is reported in the handoff to the parent, since embedding a commit's own hash in its contents is not possible.
## Review and historical evidence navigation
Extract the ZIP only if desired; the verifier itself inspects it in memory. Current authoritative raw closure is `closure.json` plus `raw/`. Relevant retained notes inside the ZIP:
- `notes/analysis-source8-s3-index-independent-review-20260913.md`: original S3 source review and required corrections.
- `notes/analysis-source8-s3-correction-independent-review-20260922.md`: corrected Path/index source review before runtime.
- `notes/analysis-source3-stack-cleanup-review-20260922.md`: independent review of iterative rejected-kind cleanup.
- `notes/analysis-source3-stack-cleanup-handoff-20260922.md`: exact failure isolation, fix and completed Windows gates.
- Original index/correction source manifests bind their respective immutable Git versions; their historical `unrun` labels are preserved and superseded only by the later actual receipts.
The ZIP's initial `README.md` contains the earlier portable-verifier proposal. It is retained as historical package text; this outer README and actual `verify.py` describe the completed verifier. No peer should execute the archived local orchestration helpers to verify evidence.
Source-bound original-span export, mixed ordinary/typed composition, complete output recompile/admission/intended-Project equality and public dispatch/migration/save integration remain separate work. This evidence does not close those requirements.
@@ -1,107 +0,0 @@
{
"source": "1b1c3e3abb6fc3d0934c9dbfbd21fddc8ae4f594",
"positive_audit": {
"source": "1b1c3e3abb6fc3d0934c9dbfbd21fddc8ae4f594",
"verified_git_inputs": 538,
"listing": 167,
"typed_pass": 59,
"index_pass": 9,
"update_pass": 5,
"token_pass": 1,
"strict_exit": 0,
"historical_overflow_retained": true,
"dirty_diagnostics": 2,
"dirty_direct_parser_probes_pass": 4,
"recorded_test_artifacts": {
"stack-fixed-original-1b1c3e3": {
"path": "C:\\Users\\User\\Documents\\Git\\ARCADIA-Studio\\.local\\analysis-checked-save-target-20260913\\debug\\deps\\oa_core-73987e0d7089bf1e.exe",
"bytes": 39014400,
"sha256": "56c0e5cd44ac5cb96b814d25ea403356195271a1aa9b6c6d1165fed4ffd09ddf"
},
"stack-fixed-shapes-1b1c3e3": {
"path": "C:\\Users\\User\\Documents\\Git\\ARCADIA-Studio\\.local\\analysis-checked-save-target-20260913\\debug\\deps\\oa_core-73987e0d7089bf1e.exe",
"bytes": 39014400,
"sha256": "56c0e5cd44ac5cb96b814d25ea403356195271a1aa9b6c6d1165fed4ffd09ddf"
},
"stack-fixed-paths-1b1c3e3": {
"path": "C:\\Users\\User\\Documents\\Git\\ARCADIA-Studio\\.local\\analysis-checked-save-target-20260913\\debug\\deps\\oa_core-73987e0d7089bf1e.exe",
"bytes": 39014400,
"sha256": "56c0e5cd44ac5cb96b814d25ea403356195271a1aa9b6c6d1165fed4ffd09ddf"
},
"typed-fixed-1b1c3e3": {
"path": "C:\\Users\\User\\Documents\\Git\\ARCADIA-Studio\\.local\\analysis-checked-save-target-20260913\\debug\\deps\\oa_core-73987e0d7089bf1e.exe",
"bytes": 39014400,
"sha256": "7b38b54ad94678171bd8b5964b1e00dd7da8346d96276c1ad790d1a6eb335208"
},
"token-fixed-1b1c3e3": {
"path": "C:\\Users\\User\\Documents\\Git\\ARCADIA-Studio\\.local\\analysis-checked-save-target-20260913\\debug\\deps\\oa_core-73987e0d7089bf1e.exe",
"bytes": 39014400,
"sha256": "7b38b54ad94678171bd8b5964b1e00dd7da8346d96276c1ad790d1a6eb335208"
},
"list-fixed-1b1c3e3": {
"path": "C:\\Users\\User\\Documents\\Git\\ARCADIA-Studio\\.local\\analysis-checked-save-target-20260913\\debug\\deps\\oa_core-73987e0d7089bf1e.exe",
"bytes": 39014400,
"sha256": "7b38b54ad94678171bd8b5964b1e00dd7da8346d96276c1ad790d1a6eb335208"
}
},
"scope": "Private S3 only; no full-core runtime/public8/integration/native/release acceptance. Recorded binary identities verified against receipts, not rehashed executable bytes; binaries intentionally absent."
},
"negative_control_count": 13,
"negative_controls": [
{
"name": "changed_zip_bytes",
"refusal": "Evidence ZIP SHA256 differs"
},
{
"name": "duplicate_json_key",
"refusal": "Duplicate JSON key: source"
},
{
"name": "duplicate_zip_member_with_test_only_pin",
"refusal": "Exact unique ZIP membership"
},
{
"name": "unsafe_zip_member_with_test_only_pin",
"refusal": "Unsafe ZIP path"
},
{
"name": "truncated_strict_completion",
"refusal": "Closed clean strict log"
},
{
"name": "missing_typed_runtime_member",
"refusal": "Exact ordered runtime results: typed-fixed-1b1c3e3"
},
{
"name": "altered_gate_command",
"refusal": "Exact command: token-fixed-1b1c3e3"
},
{
"name": "altered_committed_input_hash",
"refusal": "Exact538 source bindings: typed-fixed-1b1c3e3"
},
{
"name": "dirty_run_mislabeled_as_fixed_source",
"refusal": "Dirty capture identity/stability"
},
{
"name": "historical_abort_mislabeled_success",
"refusal": "Historical aborted gate"
},
{
"name": "diagnostic_patch_disagrees_with_captured_hash",
"refusal": "Reconstructed diagnostic input: crates/oa-core/src/language/analysis_source/index/tests.rs"
},
{
"name": "probe_case_mislabeled",
"refusal": "Dirty probe result/case"
},
{
"name": "optimized_python_refused",
"refusal": "Optimized Python refused"
}
],
"verifier_sha256": "4da79ba0c6fe9dc0ae9b81da6589756d3a215620f2e2fd6cbefaa621f646ebc9",
"controls_sha256": "a969264c98ab2fb95a0585b81c9d27a277ebe629488f20f45216d4483c26b396",
"method": "Raw archive untouched. Semantic corruptions pass directly to audit() after authentic archive validation; ZIP structural negatives temporarily substitute only the test module pin. No compiler, artifact execution or worktree write."
}
@@ -1,36 +0,0 @@
{
"source": "1b1c3e3abb6fc3d0934c9dbfbd21fddc8ae4f594",
"archive_sha256": "4486f4c42b140ae4b9b3caf0aacdf151a2acb1fd85eda09a57764fd69e980574",
"files": [
{
"path": ".gitattributes",
"bytes": 8,
"sha256": "705fd4d6451a31d36b3df7de96f83f30ac976c9b4a6d1e51671d8e2f33e2d0da"
},
{
"path": "README.md",
"bytes": 5456,
"sha256": "8059aa1794544272e3cdaa622ff48a9d0fdf4c257a4a1ca0daf9cc1adcc8b061"
},
{
"path": "verify.py",
"bytes": 17589,
"sha256": "4da79ba0c6fe9dc0ae9b81da6589756d3a215620f2e2fd6cbefaa621f646ebc9"
},
{
"path": "negative_controls.py",
"bytes": 5767,
"sha256": "a969264c98ab2fb95a0585b81c9d27a277ebe629488f20f45216d4483c26b396"
},
{
"path": "checks.json",
"bytes": 4506,
"sha256": "295acaadd2b1fe0f55c08c71c66c193635f70c2f7c63311ea42b039035065139"
},
{
"path": "evidence.zip",
"bytes": 658251,
"sha256": "4486f4c42b140ae4b9b3caf0aacdf151a2acb1fd85eda09a57764fd69e980574"
}
]
}
@@ -1,98 +0,0 @@
"""Read-only in-memory falsifiers for verify.py; does not modify archived evidence."""
import argparse
import importlib.util
import io
import json
from pathlib import Path
import subprocess
import sys
import warnings
import zipfile
def run(repo, directory):
spec = importlib.util.spec_from_file_location('s3_evidence_verifier', directory / 'verify.py')
verifier = importlib.util.module_from_spec(spec)
spec.loader.exec_module(verifier)
verifier.require(__debug__, 'Optimized Python refused')
raw = (directory / 'evidence.zip').read_bytes()
payload = verifier.archive_payload(raw)
positive = verifier.audit(repo, payload)
cases = []
def refused(name, expected, action):
try:
action()
except RuntimeError as error:
verifier.require(expected in str(error), f'{name}: wrong refusal: {error}')
cases.append({'name': name, 'refusal': str(error)})
else:
raise RuntimeError(name + ': invalid evidence was accepted')
def changed_json(name, change):
candidate = dict(payload)
data = verifier.document(candidate[name])
change(data)
candidate[name] = json.dumps(data).encode()
return candidate
refused('changed_zip_bytes', 'Evidence ZIP SHA256 differs', lambda: verifier.archive_payload(raw + b'tampered'))
refused('duplicate_json_key', 'Duplicate JSON key', lambda: verifier.document(b'{"source":1,"source":2}'))
def changed_zip(mode):
buffer = io.BytesIO()
with zipfile.ZipFile(buffer, 'w') as archive:
for name, content in payload.items():
archive.writestr('../README.md' if mode == 'unsafe' and name == 'README.md' else name, content)
if mode == 'duplicate':
with warnings.catch_warnings():
warnings.simplefilter('ignore', UserWarning)
archive.writestr('README.md', payload['README.md'])
malformed = buffer.getvalue()
# Test-only pin substitution reaches ZIP structure checks. Production
# archive_payload always requires its checked-in immutable ZIP digest.
original_pin = verifier.ZIP_SHA
try:
verifier.ZIP_SHA = verifier.sha(malformed)
verifier.archive_payload(malformed)
finally:
verifier.ZIP_SHA = original_pin
refused('duplicate_zip_member_with_test_only_pin', 'Exact unique ZIP membership', lambda: changed_zip('duplicate'))
refused('unsafe_zip_member_with_test_only_pin', 'Unsafe ZIP path', lambda: changed_zip('unsafe'))
changed = dict(payload)
changed['raw/strict-fixed-1b1c3e3.log'] = b' Compiling oa-core\n'
refused('truncated_strict_completion', 'Closed clean strict log', lambda: verifier.audit(repo, changed))
changed = dict(payload)
name = 'raw/typed-fixed-1b1c3e3.log'
changed[name] = b'\n'.join(line for line in changed[name].split(b'\n') if verifier.NEW.encode() not in line)
refused('missing_typed_runtime_member', 'Exact ordered runtime results', lambda: verifier.audit(repo, changed))
changed = changed_json('raw/token-fixed-1b1c3e3.result.json', lambda data: data['command'].remove('--offline'))
refused('altered_gate_command', 'Exact command', lambda: verifier.audit(repo, changed))
changed = changed_json('raw/typed-fixed-1b1c3e3.inputs.json', lambda data: data['inputs'][0].update(sha256='0' * 64))
refused('altered_committed_input_hash', 'Exact538 source bindings', lambda: verifier.audit(repo, changed))
changed = changed_json('raw/stack-diag-02-dirty.inputs.json', lambda data: data.update(head=verifier.SOURCE))
refused('dirty_run_mislabeled_as_fixed_source', 'Dirty capture identity/stability', lambda: verifier.audit(repo, changed))
changed = changed_json('raw/typed-90fae148.result.json', lambda data: data.update(exit=0))
refused('historical_abort_mislabeled_success', 'Historical aborted gate', lambda: verifier.audit(repo, changed))
changed = changed_json('raw/stack-diag-01-dirty.inputs.json', lambda data: data.update(dirty=data['dirty'].replace('STAGE entered', 'STAGE changed')))
refused('diagnostic_patch_disagrees_with_captured_hash', 'Reconstructed diagnostic input', lambda: verifier.audit(repo, changed))
changed = dict(payload)
changed['raw/stack-probe-if-01-dirty.log'] = changed['raw/stack-probe-if-01-dirty.log'].replace(b'PROBE after malformed case 1:', b'PROBE after malformed case 0:')
refused('probe_case_mislabeled', 'Dirty probe result/case', lambda: verifier.audit(repo, changed))
result = subprocess.run([sys.executable, '-B', '-O', str(directory / 'verify.py'), '--repo', str(repo)], capture_output=True, text=True)
verifier.require(result.returncode != 0 and 'Optimized Python refused' in result.stderr, 'Optimized verifier must refuse before audit')
cases.append({'name': 'optimized_python_refused', 'refusal': 'Optimized Python refused'})
return {'source': verifier.SOURCE, 'positive_audit': positive, 'negative_control_count': len(cases), 'negative_controls': cases, 'verifier_sha256': verifier.sha((directory / 'verify.py').read_bytes()), 'controls_sha256': verifier.sha(Path(__file__).read_bytes()), 'method': 'Raw archive untouched. Semantic corruptions pass directly to audit() after authentic archive validation; ZIP structural negatives temporarily substitute only the test module pin. No compiler, artifact execution or worktree write.'}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--repo', type=Path, required=True)
args = parser.parse_args()
print(json.dumps(run(args.repo.resolve(), Path(__file__).resolve().parent), indent=2))
if __name__ == '__main__':
main()
@@ -1,257 +0,0 @@
"""Read-only portable S3 evidence verification; never compiles or runs artifacts."""
import argparse
from datetime import datetime
import hashlib
import io
import json
from pathlib import Path, PurePosixPath, PureWindowsPath
import re
import subprocess
import zipfile
SOURCE = '1b1c3e3abb6fc3d0934c9dbfbd21fddc8ae4f594'
PRIOR = '90fae14818f684bf778640be2e613f34e3c7e999'
ZIP_SHA = '4486f4c42b140ae4b9b3caf0aacdf151a2acb1fd85eda09a57764fd69e980574'
RECORDER = '328970b226d9f4c3b8d3feb7d64478fb9756310ae5dfd3caa43b23c8b9d6f245'
PREFIX = ['cargo', 'test', '--locked', '--offline', '-p', 'oa-core', '--lib']
SCOPES = ['Cargo.toml', 'Cargo.lock', 'rust-toolchain.toml', '.cargo', 'build', 'crates', 'assets', 'docs', 'examples', 'harness/fixtures', 'vendor']
STACK = 'deep_parse_index_plan_and_failed_build_cleanup_on_128k_stack'
NEW = 'every_parent_shape_rejects_depth129_and_cleans_children_on_128k_stack'
PATHS = 'maximum_index_paths_compare_lookup_debug_and_drop_on_128k_stack'
OVERFLOW = 3221225725
def require(ok, message):
if not ok:
raise RuntimeError(message)
def sha(raw):
return hashlib.sha256(raw).hexdigest()
def unique(pairs):
value = {}
for key, item in pairs:
require(key not in value, 'Duplicate JSON key: ' + key)
value[key] = item
return value
def document(raw):
return json.loads(raw, object_pairs_hook=unique)
def git(repo, *args, data=None):
return subprocess.check_output(['git', *args], cwd=repo, input=data)
def source_inputs(repo, commit):
paths = git(repo, 'ls-tree', '-r', '--name-only', commit, '--', *SCOPES).decode().splitlines()
require(len(paths) == len(set(paths)) == 538, 'Exactly538 committed test inputs')
batch = git(repo, 'cat-file', '--batch', data=''.join(f'{commit}:{p}\n' for p in paths).encode())
offset, inputs, blobs = 0, [], {}
for path in paths:
end = batch.index(b'\n', offset)
_, kind, size = batch[offset:end].split()
size = int(size)
raw = batch[end + 1:end + 1 + size]
require(kind == b'blob' and len(raw) == size and batch[end + 1 + size:end + 2 + size] == b'\n', 'Git blob framing: ' + path)
inputs.append({'path': path, 'bytes': size, 'sha256': sha(raw)})
blobs[path] = raw
offset = end + size + 2
require(offset == len(batch), 'Complete Git blob stream')
return inputs, blobs
def archive_payload(raw):
require(sha(raw) == ZIP_SHA, 'Evidence ZIP SHA256 differs')
with zipfile.ZipFile(io.BytesIO(raw)) as archive:
names = archive.namelist()
require(len(names) == len(set(names)) == 61, 'Exact unique ZIP membership')
for name in names:
path = PurePosixPath(name)
require(not path.is_absolute() and '..' not in path.parts and str(path) == name and '\\' not in name and ':' not in name, 'Unsafe ZIP path')
payload = {name: archive.read(name) for name in names}
manifest = document(payload['manifest.json'])
require(manifest['source'] == SOURCE, 'Archive manifest source')
listed = [item['path'] for item in manifest['files']]
require(len(listed) == len(set(listed)) and set(listed) | {'manifest.json'} == set(payload), 'Archive manifest membership')
for item in manifest['files']:
require(len(payload[item['path']]) == item['bytes'] and sha(payload[item['path']]) == item['sha256'], 'Archive member binding: ' + item['path'])
return payload
def apply_recorded_diff(blobs, diff):
"""Reconstruct diagnostic source without touching an index or worktree."""
changed = {}
sections = re.split(r'(?m)^diff --git ', diff)[1:]
require(sections, 'Missing dirty diff')
for section in sections:
lines = section.splitlines(keepends=True)
names = re.fullmatch(r'a/(\S+) b/(\S+)\n', lines[0])
require(names is not None and names[1] == names[2] and names[1] in blobs, 'Unexpected diagnostic diff path')
path = names[1]
old = blobs[path].decode().splitlines(keepends=True)
output, cursor, index = [], 0, 1
while index < len(lines) and not lines[index].startswith('@@ '):
index += 1
require(index < len(lines), 'Missing diagnostic hunk')
while index < len(lines):
match = re.match(r'@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@', lines[index])
require(match is not None, 'Invalid diagnostic hunk header')
old_start, old_count = int(match[1]), int(match[2] or 1)
new_start, new_count = int(match[3]), int(match[4] or 1)
start = old_start - 1 if old_count else old_start
require(cursor <= start <= len(old), 'Diagnostic hunk position')
output.extend(old[cursor:start])
cursor = start
require(len(output) == (new_start - 1 if new_count else new_start), 'Diagnostic new hunk position')
consumed = emitted = 0
index += 1
while index < len(lines) and not lines[index].startswith('@@ '):
line = lines[index]
require(line[:1] in [' ', '+', '-'], 'Unsupported diagnostic patch line')
if line[0] in ' -':
require(cursor < len(old) and old[cursor] == line[1:], 'Diagnostic old source mismatch')
cursor += 1
consumed += 1
if line[0] in ' +':
output.append(line[1:])
emitted += 1
index += 1
require((consumed, emitted) == (old_count, new_count), 'Diagnostic hunk counts')
output.extend(old[cursor:])
require(path not in changed, 'Repeated diagnostic path')
changed[path] = ''.join(output).encode()
return changed
def audit(repo, payload):
read = lambda name: payload['raw/' + name]
doc = lambda name: document(read(name))
closure = document(payload['closure.json'])
expected, blobs = source_inputs(repo, SOURCE)
prior_inputs, prior_blobs = source_inputs(repo, PRIOR)
require(git(repo, 'rev-parse', SOURCE + '^').decode().strip() == PRIOR, 'Correction parent')
require(sha(payload['helpers/run-analysis-source3-gate-20260922.py']) == RECORDER, 'Recorder bytes')
for manifest_name, commit, count in [
('analysis-source8-s3-index-source-manifest-20260913.json', '1f8283236f8ccdefca13814003e7f046860b4f22', 7),
('analysis-source8-s3-index-correction-source-manifest-20260913.json', PRIOR, 2),
]:
manifest = document(payload['notes/' + manifest_name])
require(manifest['source'] == commit and len(manifest['files']) == count, 'Historical source artifact manifest')
for item in manifest['files']:
raw = git(repo, 'show', commit + ':' + item['path'])
require(len(raw) == item['bytes'] and sha(raw) == item['sha256'] and git(repo, 'rev-parse', commit + ':' + item['path']).decode().strip() == item['git_blob'], 'Historical source artifact: ' + item['path'])
names = re.findall(r'^([^\r\n]+): test\r?$', read('list-fixed-1b1c3e3.log').decode(), re.M)
old_names = re.findall(r'^([^\r\n]+): test\r?$', read('list-90fae148.log').decode(), re.M)
require(len(names) == len(set(names)) == 167 and len(old_names) == len(set(old_names)) == 166, 'Actual listing sizes')
qualified_new = 'language::analysis_source::tests::' + NEW
require(set(names) == set(old_names) | {qualified_new} and qualified_new not in old_names, 'Retained166 plus one new case')
require(len([n for n in names if 'language::analysis_source' in n]) == 59 and len([n for n in names if 'language::raw_tokens' in n]) == 1, 'Typed/token listing counts')
for module, count in [('index', 9), ('update', 5)]:
path = f'crates/oa-core/src/language/analysis_source/{module}/tests.rs'
authored = [f'language::analysis_source::{module}::tests::{name}' for name in re.findall(r'#\[test\]\s*fn (\w+)\(', blobs[path].decode())]
require(len(authored) == count and set(authored) == {n for n in names if n.startswith(f'language::analysis_source::{module}::tests::')}, 'Source-derived membership: ' + module)
selectors = {'stack-fixed-original-1b1c3e3': STACK, 'stack-fixed-shapes-1b1c3e3': NEW, 'stack-fixed-paths-1b1c3e3': PATHS, 'typed-fixed-1b1c3e3': 'language::analysis_source', 'token-fixed-1b1c3e3': 'language::raw_tokens', 'list-fixed-1b1c3e3': None, 'strict-fixed-1b1c3e3': 'clippy'}
environment, artifacts, last_end = None, {}, None
for stem, selector in selectors.items():
record, log = doc(stem + '.result.json'), read(stem + '.log').decode()
require(record['source'] == record['head_after'] == SOURCE and type(record['exit']) is int and record['exit'] == 0 and record['changed_inputs'] == [], 'Clean source/result: ' + stem)
require(record['inputs'] == stem + '.inputs.json' and doc(record['inputs']) == {'source': SOURCE, 'inputs': expected}, 'Exact538 source bindings: ' + stem)
require(record['launcher_sha256'] == RECORDER, 'Recorded launcher')
start, end = (datetime.fromisoformat(record[k]) for k in ['start', 'end'])
require(start.tzinfo is not None and end.tzinfo is not None and start <= end and (last_end is None or last_end <= start), 'Closed ordered gate times')
last_end = end
if environment is None:
environment = record['environment']
require(record['environment'] == environment and environment['CARGO_BUILD_JOBS'] == '1' and PureWindowsPath(environment['CARGO_TARGET_DIR']).name == 'analysis-checked-save-target-20260913', 'Recorded environment')
if selector == 'clippy':
command = ['cargo', 'clippy', '--locked', '--offline', '-p', 'oa-core', '--all-targets', '--', '-D', 'warnings']
require(record['artifacts'] == [] and 'Finished `dev` profile [unoptimized]' in log and not re.search(r'^\s*(warning|error)(\[|:)', log, re.M), 'Closed clean strict log')
selected = []
else:
require(len(record['artifacts']) == 1, 'One recorded test artifact')
artifact = record['artifacts'][0]
emitted = re.findall(r'Running unittests [^\r\n]+ \(([^\r\n]+\.exe)\)', log)
require(len(emitted) == 1 and PureWindowsPath(emitted[0]) == PureWindowsPath(artifact['path']) and artifact['bytes'] > 0 and re.fullmatch('[0-9a-f]{64}', artifact['sha256']), 'Recorded executable/log binding')
artifacts[stem] = artifact
if selector is None:
command = PREFIX + ['--', '--list']
selected = names
require(record['listing_members'] == names and record['listing_count'] == 167 and re.findall(r'^(\d+) tests, (\d+) benchmarks\s*$', log, re.M) == [('167', '0')], 'Listing receipt and summary')
else:
command = PREFIX + [selector, '--', '--test-threads=1']
selected = [name for name in names if selector in name]
rows = re.findall(r'^test ([^\r\n]+) \.\.\. (ok|FAILED|ignored)\r?$', log, re.M)
require(rows == [(name, 'ok') for name in selected], 'Exact ordered runtime results: ' + stem)
require(re.findall(r'^test result: ok\. (\d+) passed; (\d+) failed; (\d+) ignored; (\d+) measured; (\d+) filtered out;', log, re.M) == [(str(len(selected)), '0', '0', '0', str(167-len(selected)))], 'Runtime summary')
require(record['command'] == command, 'Exact command: ' + stem)
if selector is not None:
require(record['listing_count'] == 0 and record['listing_members'] == [], 'Non-list metadata')
saved = closure['gates'][stem]
require([saved[k] for k in ['command', 'exit', 'start', 'end', 'names']] == [command, 0, record['start'], record['end'], selected], 'Closure gate facts')
require(saved['pins'] == {stem+s: sha(read(stem+s)) for s in ['.inputs.json', '.log', '.result.json']}, 'Closure raw pins')
require(artifacts['stack-fixed-original-1b1c3e3'] == artifacts['stack-fixed-shapes-1b1c3e3'] == artifacts['stack-fixed-paths-1b1c3e3'], 'First focused build receipt')
require(artifacts['typed-fixed-1b1c3e3'] == artifacts['token-fixed-1b1c3e3'] == artifacts['list-fixed-1b1c3e3'], 'Final restored clean build receipt')
require([closure[k] for k in ['source', 'parent', 'verified_inputs', 'listing', 'typed_pass', 'index_pass', 'update_pass', 'token_pass', 'strict_exit']] == [SOURCE, PRIOR, 538, 167, 59, 9, 5, 1, 0], 'Closure counts')
for stem, mode in [('list-90fae148', 'list'), ('typed-90fae148', 'failed')]:
record, log = doc(stem + '.result.json'), read(stem + '.log').decode()
require(record['source'] == record['head_after'] == PRIOR and record['changed_inputs'] == [] and record['launcher_sha256'] == RECORDER, 'Historical frozen source')
require(doc(stem + '.inputs.json') == {'source': PRIOR, 'inputs': prior_inputs}, 'Historical538 inputs')
if mode == 'list':
require(record['exit'] == 0 and record['listing_members'] == old_names and record['listing_count'] == 166 and record['command'] == PREFIX + ['--', '--list'], 'Historical listing')
else:
require(record['exit'] == OVERFLOW and record['command'] == PREFIX + ['language::analysis_source', '--', '--test-threads=1'] and 'STATUS_STACK_OVERFLOW' in log and STACK + ' ...' in log and 'test result:' not in log, 'Historical aborted gate')
selected = [n for n in old_names if 'language::analysis_source' in n]
failing = 'language::analysis_source::index::tests::' + STACK
rows = re.findall(r'^test ([^\r\n]+) \.\.\. (ok|FAILED|ignored)\r?$', log, re.M)
require(rows == [(n, 'ok') for n in selected[:selected.index(failing)]] and 'running 58 tests' in log, 'Historical completed prefix before abort')
dirty = ['stack-diag-01-dirty', 'stack-diag-02-dirty', 'stack-probe-sum-01-dirty', 'stack-probe-if-01-dirty', 'stack-probe-power-01-dirty', 'stack-probe-trailing-01-dirty']
for index, stem in enumerate(dirty):
capture, result, log = doc(stem + '.inputs.json'), doc(stem + '.result.json'), read(stem + '.log').decode()
diagnostic = index < 2
commit, original = (PRIOR, prior_blobs) if diagnostic else (SOURCE, blobs)
require(capture['head'] == commit and result['changed'] == [], 'Dirty capture identity/stability')
patched = apply_recorded_diff(original, capture['dirty'])
expected_paths = {'crates/oa-core/src/language/analysis_source/index/tests.rs'} if diagnostic else {'crates/oa-core/src/language/analysis_source/tests.rs'}
if index == 1:
expected_paths.add('crates/oa-core/src/language/analysis_source/syntax.rs')
require(set(patched) == expected_paths, 'Bounded diagnostic paths')
for path, raw in original.items():
require(capture['inputs'][path] == sha(patched.get(path, raw)), 'Reconstructed diagnostic input: ' + path)
expected_filter = STACK if diagnostic else 'diagnostic_malformed_cleanup_on_128k_stack'
require(capture['command'] == PREFIX + [expected_filter, '--', '--test-threads=1', '--nocapture'], 'Diagnostic command')
if diagnostic:
require(result['exit'] == OVERFLOW and 'STATUS_STACK_OVERFLOW' in log and 'STAGE failed builds dropped' in log and 'STAGE finished' not in log, 'Diagnostic abort')
if index == 1:
require('STAGE node rejecting depth 129 before kind drop' in log and 'STAGE node rejected kind dropped' not in log, 'Isolated rejected-kind cleanup')
else:
case = index - 2
require(result['exit'] == 0 and f'PROBE before malformed case {case}' in log and f'PROBE after malformed case {case}:' in log and '1 passed; 0 failed; 0 ignored' in log, 'Dirty probe result/case')
return {'source': SOURCE, 'verified_git_inputs': 538, 'listing': 167, 'typed_pass': 59, 'index_pass': 9, 'update_pass': 5, 'token_pass': 1, 'strict_exit': 0, 'historical_overflow_retained': True, 'dirty_diagnostics': 2, 'dirty_direct_parser_probes_pass': 4, 'recorded_test_artifacts': artifacts, 'scope': 'Private S3 only; no full-core runtime/public8/integration/native/release acceptance. Recorded binary identities verified against receipts, not rehashed executable bytes; binaries intentionally absent.'}
def verify_package(directory):
manifest = document((directory / 'manifest.json').read_bytes())
require(manifest['source'] == SOURCE and manifest['archive_sha256'] == ZIP_SHA, 'Outer manifest identity')
names = [item['path'] for item in manifest['files']]
require(set(names) == {'.gitattributes', 'README.md', 'verify.py', 'negative_controls.py', 'checks.json', 'evidence.zip'} and len(names) == len(set(names)), 'Outer manifest membership')
for item in manifest['files']:
raw = (directory / item['path']).read_bytes()
require(len(raw) == item['bytes'] and sha(raw) == item['sha256'], 'Outer manifest bytes: ' + item['path'])
return archive_payload((directory / 'evidence.zip').read_bytes())
def main():
require(__debug__, 'Optimized Python refused')
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--repo', type=Path, required=True, help='Git repo containing immutable source commits; current checkout is not used')
parser.add_argument('--evidence-dir', type=Path, default=Path(__file__).resolve().parent)
args = parser.parse_args()
print(json.dumps(audit(args.repo, verify_package(args.evidence_dir)), indent=2))
if __name__ == '__main__':
main()