Evidence levels, receipts, and gate verdicts.
Every run is graded to an explicit evidence level, sealed in a signed receipt, and judged by a gate whose verdict lists exact blocking reasons — never inferred from a green process exit.
Evidence levels#
Probierz grades each run on an ordered E-scale. The run-level computation is deterministic:
- E0 — the run did not pass. A failed run never contributes evidence.
- E2 — a passed run with its report and analysis, without forced capture.
- E3 — a passed run executed with recording on, whose report, analysis, and captured media are all present.
- E4 — a full condition-matrix run in which every cell passed and met the required cell-level evidence.
- Gate policies declare their own minimum level per mode. A missing policy or minimum is refused; no E3 floor is inferred.
Signed evidence receipts#
A receipt binds runs to their exact identities: run ID, target, journey identities, status, evidence level, harness source, application source (git SHA plus worktree hash per repository), build hash, timestamps, artifact hashes, and media. Receipts are signed with Ed25519 by `probierz receipt create` and verified with `probierz receipt verify` against a trusted public key.
Around receipts sit retention policies, encrypted-at-rest protected bundles, pre-upload secret scanning, and audit history. An unavailable evidence store is an error, not an empty history.
Run-record updates never replace an unreadable or malformed existing manifest with a partial patch. A read error, invalid JSON, non-object manifest or non-object patch refuses at `run.record.update` with the manifest path and cause before writing. Only a missing manifest starts an empty object; existing run identities and evidence remain untouched after a refused update.
Receipt verification reports `signatureValid`, `trusted` and `payloadSha256Matches`. An invalid receipt refuses with the actual file and failed conditions; the in-process `probierz_verify_receipt` MCP tool returns a request-local error and keeps serving later requests.
Real verification and recovery qualification: `node tests/mcp/receipt-verification.mjs` requires `PROBIERZ`, `PROBIERZ_TEST_SOURCE_REVISION`, `PROBIERZ_TEST_RELEASE_EVIDENCE`, `PROBIERZ_TEST_SIGNED_RECEIPT` from a real run, and `PROBIERZ_TEST_RECEIPT_PUBLIC_KEY`. It verifies the original receipt, refuses a modified test-owned payload with signature and hash causes, then verifies the original again on the same MCP server. Original receipt and key hashes remain unchanged; source-bound commands, replies and process results are retained under `probierz-rs/target/real-tests/mcp-receipt/`. Receipt creation and publication require their own real journeys.
Gate verdicts#
A gate evaluates in `pull-request` or `release` mode against the application manifest's policy for that mode. The verdict is explicit: passed, plus the complete list of blocking errors. Probierz never infers approval from screenshots, prose, figure verdicts, or an unverified process exit.
The selected pullRequestPolicy or releasePolicy must be a mapping with minimumEvidence: E<n>. Optional requiredTargets and requiredJourneys are lists of non-empty text, requireProtectedArtifacts and requireSecretScan are booleans, and requiredMatrixProfile is non-empty text. Omitted optional requirements require none; malformed values are refused with the manifest path, field and observed value.
Blocking reasons include: a run that did not pass; missing or mismatched harness and application source identity; a missing exact build hash; an evidence level below the policy minimum; missing or hash-mismatched artifacts; a run kind that does not match the gate mode; runs that do not identify one exact application source; and, when the policy requires them, a missing encrypted artifact bundle or a missing passing secret scan.
Evidence collection refuses an unreadable or malformed run manifest with its path and actual cause. A directory, directory-entry or file-type read failure also names the affected evidence path; it is not treated as an empty inventory. Only a missing evidence directory or manifest counts as absent, so unavailable newer evidence cannot silently disappear behind an older passing run.
Gate run artifacts and signed-receipt runs/runIds must be arrays. Mistyped arrays and non-text run IDs are refused rather than silently removed from the evidence.
Evaluate-only inspection is separate from activating or enforcing a repository gate. A gate that has not yet been activated green denies with "gate is pending green activation", and activation itself is refused while the gate is not green.
Status projection#
`probierz status` composes the manifest, run history, affected-mappings, and git into one view: whether each required journey has evidence at all, how fresh that evidence is against the current HEAD, its evidence level, which journeys the current diff affects, and whether the pull-request policy would let HEAD merge right now — with the exact reasons when it would not.
Status reads the selected application's manifest, not every other registered application. Required journeys remain part of the decision even when the comparison contains no changed files; missing required evidence cannot become a green result merely because HEAD already matches the comparison ref.
Probierz Desktop retains unreadable history paths with their read or decoding causes alongside readable runs. Workspace screens using the shared snapshot show “Some runs could not be read and are not listed”; an unreadable manifest is not evidence that the run never existed. A missing test-results root means no recorded runs, but a regular file or symbolic link at that root is a refusal with its observed type.
Desktop's Verdicts table and inspector show Unknown for an unavailable policy minimum and block that journey with the manifest's cause. The shared manifest warning lists missing or malformed policies, invalid evidence minima, mistyped lists and flags, and read failures. The YAML parser reads mappings, flow syntax and quoted scalars structurally, not by fixed indentation; a successful run cannot supply a missing policy.
The real gate policy journey is tests/gate/policy.rs in the gate_group target. Supply PROBIERZ_TEST_SOURCE_REVISION, PROBIERZ_TEST_RELEASE_EVIDENCE, PROBIERZ_TEST_GATE_HARNESS, PROBIERZ_TEST_GATE_APP, PROBIERZ_TEST_GATE_HARNESS_SHA, PROBIERZ_TEST_GATE_SOURCE_SHA and PROBIERZ_TEST_GATE_RUNS from a genuine qualifying pull-request run. It tests the real CLI in isolated app and run-evidence directories, refuses missing policies in both modes, preserves activation through malformed-policy refusals, restores the original declaration and requires qualification again. Unreadable activation is a test failure, not unchanged state. Commands, exit statuses and binary/source identities are retained under Cargo's test directory, in gate-policy/.