WisentProbierz
Probierz/Docs

Proof that your software works as intended.

Probierz is the Wisent cross-platform test toolkit. It runs your users' journeys directly where your product lives — browser (through Weles), terminal, mobile, or native desktop — and turns every run into source-bound evidence, signed receipts, and one explainable release decision.

What Probierz is#

A release decision usually depends on test definitions, target-specific tooling, screenshots, traces, video, source identity, run history, and policy. When those pieces live in unrelated scripts and CI logs, teams cannot tell which user journeys were exercised, whether evidence belongs to the current source, or why a release was allowed. Probierz binds that chain together: from intended journey, through execution and artifacts, to one explainable gate verdict.

Every run produces the evidence you need — reports, screenshots, and videos where the selected driver supports them — so you can see exactly what is broken in the pipeline.

Current status: the documented binary reports `probierz 0.1.5` (source revision `f65b755`); it provides the Rust CLI with its MCP server, local execution, evidence, receipts, and gate evaluation. No hosted service or prebuilt binary release is currently promised.

How a run becomes a release decision#

Application manifests define intended journeys and target coordinates. Runner modules own execution, analyzers own report and media interpretation, and the evidence store owns durable run records and receipts. The dashboard and MCP surface are projections over those contracts — they are not alternate sources of truth.

Evidence pipeline
application manifest + required journeys + exact source identity
              │
              ▼
 deterministic discovery / affected
              │
              ▼
  target-specific preflight check
              │
    ┌─────────┴─────────┐
    │                   │
local runner        Stado runner
    │                   │
    └─────────┬─────────┘
              ▼
report + screenshots + traces + video metadata
              │
              ▼
analysis + history + comparison + signed evidence receipt
              │
              ▼
status projection and explicit gate verdict

Supported surfaces#

  • Web — journeys registered in Probierz drive pages through a Weles browser (`weles mcp`) at desktop, tablet and mobile viewports; Probierz links no browser library, and `probierz run web` runs only inside a Stado job (`probierz remote run web --app APP_ID --host …`).
  • Terminal — journeys registered in Probierz drive a PTY or tmux; jeden's journeys also compare it with omp side by side and have a model routed through Stado read what each view shows.
  • Mobile iOS — WebdriverIO + Appium (XCUITest); needs macOS, Xcode, and a simulator or authorized device.
  • Mobile Android — WebdriverIO + Appium (UiAutomator2); needs the Android SDK and an emulator or authorized device.
  • Native macOS — Appium Mac2 with full Xcode, or cua-driver when Accessibility-based automation and screenshot evidence are sufficient. A launched app's first window is awaited on its own window-created notification or its exit, so the program that runs Probierz needs Accessibility as well as CuaDriver; the desktop:cua preflight reports both.
  • Native Windows — WebdriverIO + WinAppDriver with Developer Mode.
  • Remote execution — the Stado bridge runs the same target contract on admitted remote capacity and returns evidence through the configured object store.
  • Scientific figures and SEO — rubric-scored figure comparison and a release SEO evaluator, both routed through the authenticated Stado model router.

Interfaces#

The Glina worker entrypoint, apps/game-asset-creator/remote/sculpt-job.sh, calls Glina's check-config, setup, doctor, sculpt and verify commands. Its Stado-selected worker must supply the live Blender addon. Probierz never patches vendor code, starts an unmanaged Blender process, or polls a log for readiness. A failed command ends the job with its actual status, including verification and artifact-copy failures.

GAC_ROOT selects the Glina checkout; otherwise the runner uses Stado's PROBIERZ_APP_SOURCE, then the sibling glina directory. RESOLVED_CONFIG selects its read-only configuration input and defaults to pipeline.config.json in that checkout. RESULTS_DIR defaults to test-results/sculpt-job for Stado's artifact collector, with SCULPT_OUT defaulting to its models directory. Stage reports stay in RESULTS_DIR; use a distinct directory per concurrent job. SCULPT_ROUNDS only overrides Glina's configured policy when explicitly supplied.

  • Human CLI: `probierz` is canonical for discovery, setup, execution, analysis, figure and SEO evaluation, authoring, evidence, gate, retention, security, and Stado workflows.
  • Machine output: status, overview, run, analysis, figure, SEO, and gate commands expose structured data; automation must not infer state from prose.
  • Overview scans: `probierz overview <app> --json` runs `tama violations find --repo <root> --json` with an announced process wait. Counts require exactly the requested repository, matched by resolved path, and complete violations, skippedFiles and errors arrays. Missing or mismatched repository paths, scan problems, decoding failures and launch failures remain named refusals; returned scanner failures retain the exit or signal and complete stdout and stderr instead of appearing as a clean scan.
  • Every child process and kernel event Probierz waits for — dependency command capture, git reads, Stado, Jeden, Skarbiec and curl calls, CuaDriver calls, its socket and a launched app's first window, the retired intake's exit, benchmark contenders, figure and GIF renders, the MCP job runner and first-use steps — waits through the shared Stado process wait, which writes on stderr what it waits for, where and since when before it waits, and how it ended (`czekam: …`, `koniec czekania: …`, `blad czekania: …`), so a session standing on a child or an app that never opens its window is named. A launch, output-read or process-wait failure remains a named error with its operating-system cause, not successful empty output. Completed commands retain their actual exit status and both output streams.
  • MCP: `probierz mcp` exposes the same discovery and explicitly named side-effecting operations over stdio JSON-RPC, running each tool in the product's own process and keeping read-only and mutating operations distinct.
  • Repository gate: `probierz gate install` installs the pre-push integration; `probierz gate evaluate` and `probierz gate enforce` remain distinct leaves of the same gate group.
  • Stado bridge: `probierz remote run`, `probierz remote author`, and `probierz remote seo` submit exact remote contracts; `probierz remote collect` and `probierz remote resume` recover existing jobs without submitting them again.

Credentials and remote capacity#

Local discovery requires no credentials. Model authoring and figure evaluation authenticate through a distinct Stado model-router URL and a router-scoped token — Probierz never receives provider credentials. Remote Stado jobs materialize secrets from scoped references instead of embedding them in the job payload, and selecting a host grants no broader machine or cloud authority: an unavailable fleet is reported as unavailable, not as empty or successful.

Remote authoring reads STADO_MODEL_ROUTER_URL from the selected surface's manifest conditions, matching the verified run. When that condition is absent, it uses the submitting process environment. A missing or invalid URL is refused before upload or job submission.

PROBIERZ_MODEL selects the Brama model or selector used for authoring. Spec authoring first reads the selected surface's condition, then the process environment, and otherwise uses any. A model-scoped bearer needs its exact allowed model, such as codex/gpt-5.6-sol; an explicitly empty value is refused.

A repair that cannot search the recorded specification directory retains the affected directory or entry and its actual listing or metadata error. The repair brief does not report the specification as absent, and specification repair refuses until that search succeeds. An absent directory and an unreadable existing directory are different outcomes.

A failed lookup of earlier successful runs is retained in the repair brief with its cause. It is not presented to the model as if no successful run had ever been recorded.

A Stado host selector constrains worker placement, not the queue endpoint. Submission, observation and evidence collection all use Stado's configured endpoint; selecting stado:macbook never replaces it with a hard-coded local port.

Quick start (discovery only)#

Prerequisites: macOS or Linux, Git, and a Rust toolchain of version 1.85 or newer (`rustup` from https://rustup.rs installs one; `cargo --version` shows yours). Discovery needs no account, credential, browser, or driver.

Build the product binary from source. The discovery commands below do not start a browser, install Appium drivers, drive an application, or create run evidence.

Shell
git clone https://github.com/wisent-ai/probierz.git
cd probierz/probierz-rs
cargo build --release
export PATH="$PWD/target/release:$PATH"
probierz --version
probierz list
probierz onboarding

Next step#

`probierz list` prints every test surface with its tool, target coordinates, and the environment variables it reads. `probierz onboarding` prints the five-step first-run walkthrough, from one declared journey to your first evidence record, and keeps track of which step is still open. Every command's exact invocation is in the CLI reference.

Status and support#

  • Maturity: public development source, version 0.1.5 — local execution, evidence contracts, receipts, and gate evaluation are available from source; host and remote target availability remains environment-specific.
  • Source and defects: github.com/wisent-ai/probierz; security reports via a private GitHub Security Advisory.
  • License: Apache-2.0.