docs: publish saved-view group selection verification and native evidence
This commit is contained in:
@@ -0,0 +1,91 @@
|
||||
# Native multiselect evidence package
|
||||
|
||||
This package preserves native run-1 and run-2, both closed with exit 0, empty app
|
||||
stdout/stderr and all three GUI/checker/fixture unchanged flags true. Each run
|
||||
contains nine positive complete-Project checker snapshots, nine source files per
|
||||
snapshot and three exact source-history equality groups. Only
|
||||
`views/diagram_oa_view.arcadia` differs from baseline in its move and hidden
|
||||
states. These artifacts do not establish native acceptance on their own; consult
|
||||
the separate [saved-view report](../../../multiselect-saved-view-20260923.md) for
|
||||
action observations, usability findings and the implemented subset of issue 29.
|
||||
|
||||
| Run | Source commit | Captures | Layout scope |
|
||||
| --- | --- | ---: | --- |
|
||||
| run-1 | `12a7352ddef5f69a144ead8219a50916233d63a9` | 27 | Before selection/status toolbar correction |
|
||||
| run-2 | `f6a6010baed9e6164a1fcc7ecd58256a562827d7` | 24 | Corrected toolbar layout build |
|
||||
|
||||
Run-1 GUI/checker build receipts refer to the 729-file input map
|
||||
`55a5f354bb5049a8c77ba9bafa833438ebf754649b0cf5d175d0c8dcfcedd7bb`;
|
||||
run-2 refers to the distinct 729-file map
|
||||
`99dc46645236e674244e7c4867595a6000d3d85b54bcc9ab2b0c601923ecf66c`.
|
||||
The [source identity evidence](../source/README.md) ties those maps to the commits
|
||||
above. The GUI binary pins differ; the recorded checker binary pins match.
|
||||
Run-1 is retained as evidence of the earlier layout, not as verification of the
|
||||
corrected build.
|
||||
|
||||
The first run-2 baseline checker attempt failed before the successful retry after
|
||||
the toolchain environment was initialized. Its unchanged raw snapshot and retry
|
||||
comparison are preserved separately in the [environment incident](../incidents/README.md).
|
||||
The successful retry is the baseline snapshot included in this native package;
|
||||
the failed invocation is not silently reclassified or presented as a GUI failure.
|
||||
|
||||
These Python 3.10+ standard-library tools never operate the GUI or rerun the model
|
||||
checker. From the checkout root, validate both raw runs without writing a package:
|
||||
|
||||
```text
|
||||
python -B harness/experiments/multiselect-20260923/native/prepare.py --root . --runs run-1 run-2 --check-only
|
||||
```
|
||||
|
||||
Omit `--check-only` to package; optionally choose `--output DIRECTORY`. The default
|
||||
is this native directory. Existing package outputs are not overwritten. A missing
|
||||
or unfinished run, changed recorded inputs, missing/extra fixture or snapshot
|
||||
files, unmatched helper bytes, or inconsistent receipts fails preparation.
|
||||
|
||||
Verify a completed package independently of the original machine:
|
||||
|
||||
```text
|
||||
python -B PATH_TO_PACKAGE/verify.py
|
||||
```
|
||||
|
||||
Each run includes all original capture JSON/images, process/start/finish records,
|
||||
generation and app streams, all nine source snapshots and checker streams/receipts,
|
||||
and the history-bytes record. Its fixture includes baseline JSON, the manifest,
|
||||
and canonical source files. Exact helpers are found by their recorded hashes among
|
||||
the native directory's `.py`/`.js` files, so original and suffixed run-2 helpers can
|
||||
coexist. Helpers are archived as data and never executed by these tools.
|
||||
|
||||
Referenced GUI/checker build results, exact gate helpers and input maps are
|
||||
included. The checker example is read from its pinned historical input ZIP, not
|
||||
from current source.
|
||||
Full build logs and large source ZIPs belong in the sibling `checks` package;
|
||||
verify that package separately. This package validates build receipt, input map,
|
||||
example-source and executable-pin relationships without duplicating those ZIPs.
|
||||
It does not contain or execute the GUI/checker binaries.
|
||||
|
||||
The verifier checks every archived byte pin and exact safe membership, clean
|
||||
recorded exit and unchanged flags, process and capture lifetimes, each capture's
|
||||
normalized executable path against the launched GUI, one consistent positive
|
||||
window ID per run, fixture/source inventories, nine positive complete-Project
|
||||
checker reports, baseline/binary/helper
|
||||
relationships, and the actual archived bytes in three history equality groups.
|
||||
It verifies that only `views/diagram_oa_view.arcadia` differs from baseline in the
|
||||
move and hidden source states, and that baseline source bytes match the fixture.
|
||||
Window identity checks join recorded metadata; the receipts do not independently
|
||||
map that window handle to the recorded PID. Source models are not recompiled by
|
||||
this offline Python verifier. Screenshot dimensions are recorded metadata, not a
|
||||
minimum-window usability evaluation.
|
||||
|
||||
`manifest.json` maps each run's original historical paths to archived run,
|
||||
fixture, helper, build and checker-source members and pins their bytes.
|
||||
`files.json` pins the packager, verifier, README, manifest and archive. Raw bytes
|
||||
are preserved; ZIP ordering and timestamps are fixed. Historic absolute paths are
|
||||
labels and are not accessed by offline verification.
|
||||
|
||||
GUI actions remain separately documented parent observations. Screenshots do
|
||||
not prove which inputs occurred, and the checker infers a bounded common delta
|
||||
from saved geometry; it cannot validate the exact pointer destination. Source
|
||||
history byte equality supports the stated saved-state relationships but does not
|
||||
independently authenticate button presses, focus behavior or Undo stack structure.
|
||||
Run-1 evidence does not establish the corrected toolbar's usability or any later
|
||||
binary's behavior. Consult the parent report for the evaluated version, observed
|
||||
limitations, run differences and the remaining full-issue acceptance work.
|
||||
Binary file not shown.
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"prepare.py": {
|
||||
"bytes": 7237,
|
||||
"sha256": "d9d49d3082e485e670b43010822ebd3ab7a79762b36a247dcd3bdfc6b5d7f41d"
|
||||
},
|
||||
"verify.py": {
|
||||
"bytes": 14703,
|
||||
"sha256": "7ef5f28b6c9a43bf0361eec2d8be33a1e9f284d0ffc21b50fd09a2d082642ce3"
|
||||
},
|
||||
"README.md": {
|
||||
"bytes": 5418,
|
||||
"sha256": "8d6fa25d8bd97c654e78b89194bd499637427ce027157d568215e13421b22c03"
|
||||
},
|
||||
"manifest.json": {
|
||||
"bytes": 72794,
|
||||
"sha256": "de7642ad43aa881f7d984629db52ea41b62d0ebacc09260edd80a5c1fee7dd7b"
|
||||
},
|
||||
"evidence.zip": {
|
||||
"bytes": 4399281,
|
||||
"sha256": "7dc3e0571179e0f67fefe3c73462308b79e4c7f7a09bbd805e0acf0e4faeb1cf"
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,129 @@
|
||||
"""Package explicit completed native runs; helpers are pinned data, never executed."""
|
||||
from pathlib import Path
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import zipfile
|
||||
from verify import pin, parsed, need, norm, relative, safe, verify_data, verify_package
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
|
||||
|
||||
def collect(root, runs):
|
||||
native = root / ".local/multiselect-native"
|
||||
candidates = [path for path in native.iterdir() if path.is_file() and path.suffix in (".py", ".js")]
|
||||
candidates.extend(sorted((root / ".local/group-movement-gates").glob("*.py")))
|
||||
helpers = [(path.suffix, path.read_bytes()) for path in candidates]
|
||||
payload, specs = {}, {}
|
||||
def add(name, data):
|
||||
need(safe(name) and (name not in payload or payload[name] == data), "member collision: " + name)
|
||||
payload[name] = data
|
||||
def tree(folder, prefix):
|
||||
need(folder.is_dir(), "missing directory: " + str(folder))
|
||||
for path in sorted(folder.rglob("*")):
|
||||
need(not path.is_symlink(), "symlink evidence input")
|
||||
if path.is_file():
|
||||
add(prefix + path.relative_to(folder).as_posix(), path.read_bytes())
|
||||
for run in runs:
|
||||
folder = native / run
|
||||
start = parsed((folder / "started.json").read_bytes())
|
||||
end = parsed((folder / "finished.json").read_bytes())
|
||||
need(end["exitCode"] == 0 and all(end[k] is True for k in
|
||||
("guiUnchanged", "checkerUnchanged", "fixtureUnchanged")), "unfinished/changed native run: " + run)
|
||||
generation = parsed((folder / "generation.json").read_bytes())
|
||||
history = parsed((folder / "history-bytes.json").read_bytes())
|
||||
tree(folder, "runs/" + run + "/raw/")
|
||||
fixture_relative = relative(generation["command"][2], start["cwd"])
|
||||
need(fixture_relative.startswith(".local/multiselect-native/"), "fixture outside native root")
|
||||
tree(root / fixture_relative, "runs/" + run + "/fixture/")
|
||||
spec = {"originalRun": norm(start["cwd"]) + "/.local/multiselect-native/" + run,
|
||||
"originalFixture": norm(generation["command"][2]), "helpers": {}, "builds": {}}
|
||||
for key, expected in [("helper", start["helper"]), ("capture", start["capture"]),
|
||||
("snapshotHelper", start["snapshotHelper"]), ("history", history["helper"])]:
|
||||
matches = [(suffix, data) for suffix, data in helpers if pin(data) == expected]
|
||||
need(bool(matches), "exact historical helper unavailable: " + key)
|
||||
suffix, data = matches[0]
|
||||
member = "helpers/" + expected["sha256"] + suffix
|
||||
add(member, data)
|
||||
spec["helpers"][key] = member
|
||||
for kind in ("gui", "checker"):
|
||||
recorded = start[kind]
|
||||
rel = relative(recorded["receipt"], start["cwd"])
|
||||
need(rel.startswith(".local/group-movement-gates/") and rel.endswith("/result.json"), "unexpected build receipt location")
|
||||
receipt = root / rel
|
||||
data = receipt.read_bytes()
|
||||
need(pin(data) == recorded["receiptPin"], "historical build receipt changed")
|
||||
prefix = "builds/" + pin(data)["sha256"] + "/"
|
||||
add(prefix + "result.json", data)
|
||||
inputs_data = (receipt.parent / "inputs.json").read_bytes()
|
||||
need(pin(inputs_data) == recorded["inputs"], "historical build input map changed")
|
||||
add(prefix + "inputs.json", inputs_data)
|
||||
spec["builds"][kind] = {"receipt": prefix + "result.json", "inputs": prefix + "inputs.json", "originalRelative": rel}
|
||||
build_helper = parsed(data)["helper"]
|
||||
matching_helpers = [(suffix, value) for suffix, value in helpers if pin(value) == build_helper]
|
||||
need(bool(matching_helpers), "exact historical build helper unavailable")
|
||||
suffix, helper_data = matching_helpers[0]
|
||||
helper_member = "helpers/" + build_helper["sha256"] + suffix
|
||||
add(helper_member, helper_data)
|
||||
spec["builds"][kind]["helper"] = helper_member
|
||||
if kind == "checker":
|
||||
# Read only the pinned historical example, not today's checkout.
|
||||
archive_path = receipt.parent / "inputs.zip"
|
||||
need(pin(archive_path.read_bytes()) == parsed(data)["archive"], "checker input archive pin")
|
||||
source_path = "crates/oa-core/examples/multiselect_native.rs"
|
||||
with zipfile.ZipFile(archive_path) as archive:
|
||||
source = archive.read(source_path)
|
||||
need(pin(source) == parsed(inputs_data)[source_path], "historical checker source pin")
|
||||
member = "checker-source/" + pin(source)["sha256"] + ".rs"
|
||||
add(member, source)
|
||||
spec["checkerSource"] = {"inputPath": source_path, "member": member}
|
||||
specs[run] = spec
|
||||
manifest = {"schema": 1, "kind": "multiselect-native", "runs": specs,
|
||||
"members": {name: pin(data) for name, data in sorted(payload.items())}}
|
||||
summary = verify_data(manifest, payload.__getitem__)
|
||||
return payload, manifest, summary
|
||||
|
||||
|
||||
def write_json(path, value):
|
||||
with path.open("xb") as stream:
|
||||
stream.write((json.dumps(value, indent=2) + "\n").encode())
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--root", type=Path, default=HERE.parents[3])
|
||||
parser.add_argument("--runs", nargs="+", required=True)
|
||||
parser.add_argument("--output", type=Path, default=HERE)
|
||||
parser.add_argument("--check-only", action="store_true")
|
||||
args = parser.parse_args()
|
||||
need(len(args.runs) == len(set(args.runs)) and all(re.fullmatch(r"run-[0-9]+", name) for name in args.runs), "explicit unique run-N names required")
|
||||
payload, manifest, summary = collect(args.root.resolve(), args.runs)
|
||||
if args.check_only:
|
||||
print("PASS raw consistency; no archive written:", summary)
|
||||
return
|
||||
out = args.output.resolve()
|
||||
out.mkdir(parents=True, exist_ok=True)
|
||||
for name in ("evidence.zip", "manifest.json", "files.json"):
|
||||
need(not (out / name).exists(), "existing output refused: " + name)
|
||||
for name in ("prepare.py", "verify.py", "README.md"):
|
||||
data = (HERE / name).read_bytes()
|
||||
if (out / name).exists():
|
||||
need((out / name).read_bytes() == data, "different existing package tool")
|
||||
else:
|
||||
with (out / name).open("xb") as stream:
|
||||
stream.write(data)
|
||||
with zipfile.ZipFile(out / "evidence.zip", "x", compression=zipfile.ZIP_DEFLATED, compresslevel=9) as archive:
|
||||
for name, data in sorted(payload.items()):
|
||||
info = zipfile.ZipInfo(name, date_time=(2026, 9, 23, 0, 0, 0))
|
||||
info.compress_type = zipfile.ZIP_DEFLATED
|
||||
info.external_attr = 0o100644 << 16
|
||||
archive.writestr(info, data)
|
||||
write_json(out / "manifest.json", manifest)
|
||||
names = ("prepare.py", "verify.py", "README.md", "manifest.json", "evidence.zip")
|
||||
write_json(out / "files.json", {name: pin((out / name).read_bytes()) for name in names})
|
||||
need(verify_package(out) == summary, "written package differs from validated evidence")
|
||||
print("PASS package readback:", out, summary)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,235 @@
|
||||
"""Verify archived native receipts and bytes; do not rerun the GUI or checker."""
|
||||
from pathlib import Path, PurePosixPath
|
||||
from datetime import datetime
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import stat
|
||||
import zipfile
|
||||
|
||||
BASE = {"started.json", "finished.json", "generation.json", "generation.stdout.log",
|
||||
"generation.stderr.log", "app.stdout.log", "app.stderr.log", "processes.json", "history-bytes.json"}
|
||||
GROUPS = [["baseline", "move-undo", "before-hide", "hide-undo"], ["move", "move-redo"],
|
||||
["hidden", "hide-redo", "reopened-hidden"]]
|
||||
MODES = {name: mode for group, mode in zip(GROUPS, ["unchanged", "moved", "hide"]) for name in group}
|
||||
OUTER = {"prepare.py", "verify.py", "README.md", "manifest.json", "evidence.zip"}
|
||||
|
||||
|
||||
def need(ok, message):
|
||||
if not ok:
|
||||
raise ValueError(message)
|
||||
|
||||
|
||||
def pin(data):
|
||||
return {"bytes": len(data), "sha256": hashlib.sha256(data).hexdigest()}
|
||||
|
||||
|
||||
def safe(name):
|
||||
return (isinstance(name, str) and bool(name) and "\\" not in name and ":" not in name
|
||||
and not PurePosixPath(name).is_absolute() and all(ord(c) >= 32 for c in name)
|
||||
and all(part not in ("", ".", "..") for part in name.split("/")))
|
||||
|
||||
|
||||
def parsed(data):
|
||||
def unique(items):
|
||||
result = {}
|
||||
for key, value in items:
|
||||
need(key not in result, "duplicate JSON key: " + key)
|
||||
result[key] = value
|
||||
return result
|
||||
return json.loads(data, object_pairs_hook=unique)
|
||||
|
||||
|
||||
def norm(path):
|
||||
return path.replace("\\", "/").rstrip("/")
|
||||
|
||||
|
||||
def relative(path, root):
|
||||
prefix = norm(root) + "/"
|
||||
need(norm(path).startswith(prefix), "recorded path outside owner: " + path)
|
||||
value = norm(path)[len(prefix):]
|
||||
need(safe(value), "unsafe recorded relative path")
|
||||
return value
|
||||
|
||||
|
||||
def ordered(*values):
|
||||
times = [datetime.fromisoformat(value) for value in values]
|
||||
return all(t.utcoffset() is not None for t in times) and times == sorted(times)
|
||||
|
||||
|
||||
def zip_inventory(archive, members):
|
||||
names = archive.namelist()
|
||||
need(len(names) == len(set(names)) and set(names) == set(members), "exact archive members")
|
||||
for info in archive.infolist():
|
||||
need(safe(info.filename) and not info.is_dir() and not stat.S_ISLNK(info.external_attr >> 16), "unsafe archive member")
|
||||
need(info.file_size == members[info.filename]["bytes"], "archive member size")
|
||||
|
||||
|
||||
def verify_data(manifest, read):
|
||||
need(manifest["schema"] == 1 and manifest["kind"] == "multiselect-native", "manifest kind")
|
||||
members, runs = manifest["members"], manifest["runs"]
|
||||
need(bool(runs) and all(safe(run) and "/" not in run for run in runs), "run names")
|
||||
need(all(safe(name) for name in members), "safe member names")
|
||||
for name, expected in members.items():
|
||||
need(pin(read(name)) == expected, "member pin: " + name)
|
||||
used = set()
|
||||
def raw(name):
|
||||
need(name in members, "missing member: " + name)
|
||||
used.add(name)
|
||||
return read(name)
|
||||
def obj(name):
|
||||
return parsed(raw(name))
|
||||
def tree(prefix):
|
||||
return {name[len(prefix):]: raw(name) for name in members if name.startswith(prefix)}
|
||||
def pinned_tree(prefix, expected):
|
||||
actual = tree(prefix)
|
||||
need({name: pin(data) for name, data in actual.items()} == expected, "exact pinned tree: " + prefix)
|
||||
return actual
|
||||
for run, spec in runs.items():
|
||||
prefix = "runs/" + run + "/raw/"
|
||||
start, end = obj(prefix + "started.json"), obj(prefix + "finished.json")
|
||||
generation, process = obj(prefix + "generation.json"), obj(prefix + "processes.json")
|
||||
history = obj(prefix + "history-bytes.json")
|
||||
need(end["exitCode"] == 0 and all(end[key] is True for key in
|
||||
("guiUnchanged", "checkerUnchanged", "fixtureUnchanged")), run + ": clean unchanged closure")
|
||||
need(ordered(generation["start"], generation["finish"], process["CreationDate"], start["start"], end["finish"], history["verified"]), run + ": process/generation times")
|
||||
need(process["ProcessId"] == start["pid"] and type(start["pid"]) is int and start["pid"] > 0, run + ": process identity")
|
||||
need(norm(process["ExecutablePath"]) == norm(start["gui"]["binary"])
|
||||
and start["command"] == [start["gui"]["binary"]], run + ": GUI command")
|
||||
need(spec["originalRun"] == norm(start["cwd"]) + "/.local/multiselect-native/" + run, run + ": original run mapping")
|
||||
need(generation["exitCode"] == 0 and generation["checker"] == start["checker"], run + ": generation checker")
|
||||
need(generation["command"][:2] == [start["checker"]["binary"], "generate"]
|
||||
and len(generation["command"]) == 3 and norm(generation["command"][2]) == spec["originalFixture"], run + ": generation command")
|
||||
for stream in ("stdout", "stderr"):
|
||||
need(end[stream] == pin(raw(prefix + "app." + stream + ".log")), run + ": app stream")
|
||||
need(generation[stream] == pin(raw(prefix + "generation." + stream + ".log")), run + ": generation stream")
|
||||
fixture = pinned_tree("runs/" + run + "/fixture/", start["fixture"])
|
||||
need(set(fixture) == {"baseline.json", "manifest.json"} | {"source/" + name for name in
|
||||
obj(prefix + "snapshots/baseline/result.json")["source"]}, run + ": fixture membership")
|
||||
fixture_manifest = parsed(fixture["manifest.json"])
|
||||
need(fixture_manifest["fixture"] == "multiselect-native-20260923"
|
||||
and fixture_manifest["source_entry"] == "project.arcadia", run + ": fixture identity")
|
||||
need(parsed(fixture["baseline.json"])["id"] == fixture_manifest["ids"]["project"], run + ": baseline identity")
|
||||
for key, expected in [("helper", start["helper"]), ("capture", start["capture"]),
|
||||
("snapshotHelper", start["snapshotHelper"]), ("history", history["helper"])]:
|
||||
need(pin(raw(spec["helpers"][key])) == expected, run + ": helper " + key)
|
||||
builds = {}
|
||||
for kind, target in [("gui", "arcadia-studio"), ("checker", "multiselect_native")]:
|
||||
recorded = start[kind]
|
||||
build = obj(spec["builds"][kind]["receipt"])
|
||||
need(pin(raw(spec["builds"][kind]["receipt"])) == recorded["receiptPin"], run + ": build receipt pin")
|
||||
need(relative(recorded["receipt"], start["cwd"]) == spec["builds"][kind]["originalRelative"], run + ": build path mapping")
|
||||
need(build["exitCode"] == 0 and build["inputsUnchanged"] is True
|
||||
and ordered(build["start"], build["finish"], generation["start"]), run + ": accepted preceding build")
|
||||
need(build["inputs"] == recorded["inputs"] == pin(raw(spec["builds"][kind]["inputs"])), run + ": build input map")
|
||||
need(pin(raw(spec["builds"][kind]["helper"])) == build["helper"], run + ": build helper pin")
|
||||
inputs = obj(spec["builds"][kind]["inputs"])
|
||||
need(len(inputs) == build["inputCount"] and sum(name.startswith("crates/vendor/iced_widget/") for name in inputs) == 71, run + ": build input count")
|
||||
artifacts = [item for item in build["artifacts"] if item["cargo"]["target"]["name"] == target]
|
||||
need(len(artifacts) == 1 and artifacts[0]["readback"] == recorded["pin"]
|
||||
and artifacts[0]["cargo"]["executable"] == recorded["binary"], run + ": executable relationship")
|
||||
builds[kind] = inputs
|
||||
need(start["gui"]["inputs"] == start["checker"]["inputs"], run + ": matching GUI/checker inputs")
|
||||
source = spec["checkerSource"]
|
||||
need(source["inputPath"] == "crates/oa-core/examples/multiselect_native.rs"
|
||||
and pin(raw(source["member"])) == builds["checker"][source["inputPath"]], run + ": archived exact checker source")
|
||||
|
||||
top = {name[len(prefix):] for name in members if name.startswith(prefix) and "/" not in name[len(prefix):]}
|
||||
captures = sorted(name for name in top - BASE if name.endswith(".json"))
|
||||
need(bool(captures), run + ": missing captures")
|
||||
expected_top = set(BASE) | set(captures)
|
||||
captured_window = None
|
||||
for name in captures:
|
||||
capture = obj(prefix + name)
|
||||
need(ordered(start["start"], capture["time"], end["finish"]), run + ": capture lifetime")
|
||||
window = capture["window"]
|
||||
app = window["app"]
|
||||
need(isinstance(app, str) and app.startswith("process:")
|
||||
and norm(app[len("process:"):]) == norm(start["gui"]["binary"]), run + ": captured executable relationship")
|
||||
need(type(window["id"]) is int and window["id"] > 0, run + ": captured window identity")
|
||||
if captured_window is None:
|
||||
captured_window = window["id"]
|
||||
need(window["id"] == captured_window, run + ": inconsistent captured window identity")
|
||||
need(isinstance(capture["images"], list), run + ": capture images")
|
||||
for image in capture["images"]:
|
||||
filename = image["file"]
|
||||
need(safe(filename) and "/" not in filename and filename.startswith(name[:-5] + "-")
|
||||
and filename.endswith((".jpg", ".png")), run + ": capture image mapping")
|
||||
need(type(image["width"]) is int and image["width"] > 0
|
||||
and type(image["height"]) is int and image["height"] > 0, run + ": image dimensions")
|
||||
need(filename not in expected_top, run + ": duplicate image reference")
|
||||
expected_top.add(filename)
|
||||
raw(prefix + filename)
|
||||
need(top == expected_top, run + ": extra/missing top-level run files")
|
||||
|
||||
snapshots = {name[len(prefix + "snapshots/"):].split("/")[0] for name in members if name.startswith(prefix + "snapshots/")}
|
||||
need(snapshots == set(MODES), run + ": exact nine snapshot labels")
|
||||
states = {}
|
||||
for label, mode in MODES.items():
|
||||
folder = prefix + "snapshots/" + label + "/"
|
||||
began, result = obj(folder + "started.json"), obj(folder + "result.json")
|
||||
need(all(result.get(key) == value for key, value in began.items()), label + ": start/result")
|
||||
need(result["exitCode"] == 0 and all(result[key] is True for key in
|
||||
("binaryUnchanged", "workingSourceUnchanged", "baselineUnchanged")), label + ": unchanged successful checker")
|
||||
need(ordered(start["start"], result["start"], result["finish"], end["finish"]), label + ": checker lifetime")
|
||||
need(result["buildReceipt"] == start["checker"]["receiptPin"] and result["binary"] == start["checker"]["pin"]
|
||||
and result["helper"] == start["snapshotHelper"], label + ": checker provenance")
|
||||
need(result["baseline"] == pin(fixture["baseline.json"]), label + ": baseline pin")
|
||||
command = result["command"]
|
||||
need(len(command) == 5 and command[:3] == [start["checker"]["binary"], "check", mode]
|
||||
and norm(command[3]) == spec["originalFixture"] + "/baseline.json"
|
||||
and norm(command[4]) == spec["originalRun"] + "/snapshots/" + label + "/source/project.arcadia", label + ": exact checker command")
|
||||
for stream in ("stdout", "stderr"):
|
||||
need(result[stream] == pin(raw(folder + stream + ".log")), label + ": stream pin")
|
||||
need(not raw(folder + "stderr.log"), label + ": checker stderr")
|
||||
report = obj(folder + "stdout.log")
|
||||
need(report["result"] == "PASS" and report["comparison"] == "complete Project equality"
|
||||
and report["mode"] == mode and report["baseline"] == command[3] and report["saved"] == command[4], label + ": positive checker report")
|
||||
delta = report["inferred_common_delta"]
|
||||
if mode == "moved":
|
||||
need(isinstance(delta, dict) and set(delta) == {"x", "y"}
|
||||
and all(type(v) in (int, float) and math.isfinite(v) and abs(v) <= 500 for v in delta.values())
|
||||
and math.hypot(delta["x"], delta["y"]) >= 10, label + ": inferred movement bounds")
|
||||
else:
|
||||
need(delta is None, label + ": unexpected inferred delta")
|
||||
states[label] = pinned_tree(folder + "source/", result["source"])
|
||||
need(len(states[label]) == history["sourceFilesPerSnapshot"] == 9, label + ": source count")
|
||||
expected = {folder + name for name in ("started.json", "result.json", "stdout.log", "stderr.log")}
|
||||
expected.update(folder + "source/" + name for name in states[label])
|
||||
need({name for name in members if name.startswith(folder)} == expected, label + ": extra snapshot files")
|
||||
need(history["status"] == "PASS" and history["equalByteGroups"] == GROUPS, run + ": history groups")
|
||||
for group in GROUPS:
|
||||
need(all(states[label] == states[group[0]] for label in group), run + ": actual history bytes " + str(group))
|
||||
need(all(states[a[0]] != states[b[0]] for i, a in enumerate(GROUPS) for b in GROUPS[i + 1:]), run + ": distinct history states")
|
||||
need(states["baseline"] == {name[7:]: data for name, data in fixture.items() if name.startswith("source/")}, run + ": baseline source exact")
|
||||
changed = {label: sorted(name for name in set(states["baseline"]) | set(states[label])
|
||||
if states["baseline"].get(name) != states[label].get(name)) for label in ("move", "hidden")}
|
||||
need(changed == history["changedFilesFromBaseline"]
|
||||
== {label: ["views/diagram_oa_view.arcadia"] for label in ("move", "hidden")}, run + ": changed source files")
|
||||
print(run, "closed exit 0;", len(captures), "captures; nine positive snapshots; exact three-group source history")
|
||||
need(used == set(members), "unreferenced/extra package members")
|
||||
return {"runs": len(runs), "members": len(members)}
|
||||
|
||||
|
||||
def verify_package(root):
|
||||
files = parsed((root / "files.json").read_bytes())
|
||||
need(set(files) == OUTER, "outer files")
|
||||
for name, expected in files.items():
|
||||
need(pin((root / name).read_bytes()) == expected, "outer pin: " + name)
|
||||
manifest = parsed((root / "manifest.json").read_bytes())
|
||||
with zipfile.ZipFile(root / "evidence.zip") as archive:
|
||||
zip_inventory(archive, manifest["members"])
|
||||
return verify_data(manifest, archive.read)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("package", nargs="?", type=Path, default=Path(__file__).resolve().parent)
|
||||
args = parser.parse_args()
|
||||
print("PASS native evidence byte/receipt consistency:", verify_package(args.package))
|
||||
print("No GUI/checker execution; actions and native acceptance require the separate parent review.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user