WisentProbierz
Probierz/Docs

MCP

Probierz MCP tool reference.

`probierz mcp` exposes Probierz discovery, execution, evidence, and control operations over newline-delimited stdio JSON-RPC, running each tool in the product's own process.

Start the server#

Build the repository with `cargo build --release` in `probierz-rs`, put `target/release` on `PATH`, and configure your MCP client to launch `probierz mcp` (with `--harness <dir>` when the harness is not the repository the binary was built in). The server reads JSON-RPC from standard input and writes one JSON response per line to standard output. Every tool is the CLI command of the same name run inside this process; only an asynchronous run is a child process, so that its whole process tree can be cancelled.

The tool names, descriptions, and schemas below are generated by starting that server and calling `tools/list`; the documentation check refuses missing, extra, or changed tools.

MCP server command
probierz mcp

Asynchronous run queue#

Initialize the stdio session, send `notifications/initialized`, start a run, and poll its durable run ID until it settles. Artifacts are listed and read separately; `probierz_get_artifact` returns base64 content. Cancellation is idempotent: asking to cancel a settled run returns `cancelRequested: false`.

JavaScript
import { spawn } from "node:child_process";
import { createInterface } from "node:readline";

const server = spawn("probierz", ["mcp"], { stdio: ["pipe", "pipe", "inherit"] });
const lines = createInterface({ input: server.stdout });
const pending = new Map();
let id = 0;
lines.on("line", (line) => {
  const message = JSON.parse(line);
  pending.get(message.id)?.(message);
  pending.delete(message.id);
});
function rpc(method, params) {
  id += 1;
  server.stdin.write(JSON.stringify({ jsonrpc: "2.0", id, method, params }) + "\n");
  return new Promise((resolve) => pending.set(id, resolve));
}
async function tool(name, args) {
  const response = await rpc("tools/call", { name, arguments: args });
  if (response.error) throw new Error(response.error.message);
  return JSON.parse(response.result.content[0].text);
}
await rpc("initialize", {
  protocolVersion: "2024-11-05",
  capabilities: {},
  clientInfo: { name: "probierz-docs", version: "1" },
});
server.stdin.write(JSON.stringify({ jsonrpc: "2.0", method: "notifications/initialized" }) + "\n");
const started = await tool("probierz_start_run", { target: "tui", appId: "docs-demo" });
const STATUS_POLL_MS = 500;
let status = started;
while (["queued", "running"].includes(status.status)) {
  await new Promise((resolve) => setTimeout(resolve, STATUS_POLL_MS));
  status = await tool("probierz_run_status", { runId: started.runId });
}
const artifacts = await tool("probierz_list_artifacts", { runId: started.runId });
const report = await tool("probierz_get_artifact", { runId: started.runId, file: "report.json" });
const decoded = JSON.parse(Buffer.from(report.content, "base64").toString("utf8"));
if (decoded.probierz.runId !== started.runId) throw new Error("artifact runId mismatch");
const cancelled = await tool("probierz_cancel_run", { runId: started.runId });
if (cancelled.cancelRequested !== false) throw new Error("settled run requested cancellation");
console.log({ status: status.status, artifacts: artifacts.artifacts });
server.stdin.end();

Tools#

  • probierz_list_surfaces — List the test surfaces (web through Weles, mobile, desktop-native, desktop-cua, tui): tool, run command, targets, and relevant env vars.
  • probierz_list_specs — Discover journeys: spec files on disk, or the journeys registered in probierz for web, tui and desktop:cua; optional surface narrows to one (web|mobile|desktop-native|desktop:cua|tui).
  • probierz_describe_spec — Static outline of a spec (describe/it/test titles in file order) by its path under the probierz root. Does not execute anything.
  • probierz_run_command — Return the exact shell command to run a target yourself (web|mobile:ios|mobile:android|desktop:mac|desktop:win|desktop:cua|tui). Read-only: probierz never runs it.
  • probierz_check — Preflight a target's toolchain WITHOUT running anything: reports whether it is ready and, for each missing piece, exactly how to fix it -- `probierz setup <target>` for parts probierz owns (npm dependencies, Appium drivers) or a host install command for the rest (weles, Xcode, Android SDK, simulators, WinAppDriver). Read-only.
  • probierz_setup — Install the toolchain parts probierz owns for a target (npm deps, plus the Appium driver for mobile and desktop-native; web needs only the weles binary). Does NOT install host-level dependencies (weles, Xcode, Android SDK, simulators, WinAppDriver) -- probierz_check reports those. Side-effecting: runs npm / appium driver install; each step runs to its own exit.
  • probierz_run — EXECUTE a target end-to-end, capture evidence, analyze it, and dispatch a bounded Brama repair worker when it fails. Heavy + side-effecting: needs the target toolchain; noRepair=true records without repair.
  • probierz_analyze — Parse a finished run's report (the canonical Probierz report or the WDIO probierz-<kind>-results.json) and inventory its media: totals, per-test status, failure reasons, and recording metadata (duration/dimensions via ffprobe, optional frame montage via ffmpeg).
  • probierz_evaluate_figure — SIDE-EFFECTING: render a scientific reference/candidate pair, run deterministic geometry checks, score the declared visual rubric through the authenticated model router, and write immutable PNG evidence plus a JSON verdict.
  • probierz_evaluate_seo — SIDE-EFFECTING: crawl a declared site as ordinary Chrome and Googlebot Smartphone, enforce indexability and structured-data contracts, collect mobile performance evidence, run two independent Brama content graders with conditional adjudication, ingest optional Search Console/CrUX evidence, and write an immutable signed SEO verdict.
  • probierz_evaluate_talk — SIDE-EFFECTING: judge a rendered talk's slide pictures and speaker script against the documents it presents through the authenticated model router, criterion by criterion (source fidelity, argument, slide design, human authorship, delivery), and write an immutable JSON verdict that passes only when every criterion is met.
  • probierz_evaluate_security — SIDE-EFFECTING: run GET security checks only against the exact owned origin and approved addresses in an application manifest. Select either an explicit model or deterministic mode without model calls. Refuses redirects, undeclared checks and out-of-scope DNS; writes immutable evidence including failures. Model credentials come from the server environment, not tool arguments.
  • probierz_security_journal — Read a retained security operation journal without contacting a target or model. Returns complete events and explicitly marks an incomplete final line; journal progress is never a passing security verdict.
  • probierz_create_readme_gif — SIDE-EFFECTING: convert one recorded journey video into a silent, looping README GIF and write a provenance sidecar with source/output SHA-256 and mandatory publication checks. Requires ffmpeg.
  • probierz_affected — Given a change, report which run targets it could affect, so you re-run only what is relevant. Deterministic + structural (maps files to targets by package containment; agent/ or repo-root files are cross-cutting -> all targets). Provide `files` explicitly, or omit to diff the working tree against `ref` (default HEAD) via git. Read-only.
  • probierz_ci — Change-driven pass: select affected targets, run and analyze them, then dispatch a bounded Brama repair worker for each failure unless noRepair=true. Selection and blockers stay deterministic; only the explicit repair step asks a model what to change.
  • probierz_history — Read deterministic E5 stability history: pass rate, infrastructure failures, duration trend, flaky tests, journeys, latest run, and the newest passing run whole (summary.lastGreenRun), optionally for one target and one journey.
  • probierz_dashboard — Project evidence for product → version → journey → surface → device → result → artifact dashboard navigation.
  • probierz_matrix_plan — Read the deterministic nightly or release matrix without executing it.
  • probierz_run_matrix — HEAVY + SIDE-EFFECTING: execute every cell of a declared nightly or release matrix and return an E4 verdict.
  • probierz_protect_run — SIDE-EFFECTING: encrypt a complete run into an authenticated AES-256-GCM evidence bundle; optionally remove plaintext artifacts.
  • probierz_restore_bundle — SIDE-EFFECTING: authenticate and restore an encrypted evidence bundle into an empty directory.
  • probierz_retention — Plan retention expiry; with apply=true, delete expired plaintext runs and encrypted bundles.
  • probierz_secret_scan — Scan a plaintext artifact directory for high-confidence secrets without returning secret values.
  • probierz_audit — Read and integrity-check access audit records, optionally filtered by app, run, or action.
  • probierz_source_identity — Compute exact path-independent harness and app source SHA-256 identities.
  • probierz_gate_status — Read pull-request and release gate activation state.
  • probierz_status — Journey coverage, evidence freshness against a base ref, untested surfaces, and pull-request merge eligibility for an app.
  • probierz_gate_prepush — Pre-push merge gate: select affected journeys from the push diff and evaluate the newest passing runs against the exact current HEAD identity (pull-request policy).
  • probierz_author_spec — SIDE-EFFECTING: use the authenticated Stado model router to draft one journey spec from a probe of the real app, verify it with an actual run, and keep it on green (registers the journey in the app manifest).
  • probierz_repair — SIDE-EFFECTING: dispatch one Brama worker at a recorded failed run for the stated number of rounds. Product fixes land on a fresh published branch; spec fixes must pass the real journey before publication.
  • probierz_author_manifest — SIDE-EFFECTING: use the authenticated Stado model router to draft the whole app journey manifest from a probe and repository layout, validate it, and optionally cover every journey with author spec.
  • probierz_stado_run — SIDE-EFFECTING: run a target on a chosen stado host (provider/pin/spot/GPU); evidence lands back in test-results.
  • probierz_stado_evaluate_seo — SIDE-EFFECTING: submit the complete SEO evaluator to a Stado-selected dedicated host, materialize only the declared Brama and signing secrets, and fetch the immutable evidence bundle.
  • probierz_gate_evaluate — Evaluate exact build and evidence against the application's explicitly declared mode policy, including coverage, matrix, encryption, secret scan and signed receipt requirements; refuses missing or malformed policy declarations and appends an audit record for evaluated evidence.
  • probierz_gate_enforce — Enforce an activated gate against current evidence; pending-green gates fail closed.
  • probierz_gate_activate — SIDE-EFFECTING: atomically activate a gate only after all green evidence requirements pass.
  • probierz_compare_runs — Deterministically compare status, duration, tests, evidence, build identity, and artifact hashes between two run IDs.
  • probierz_create_receipt — SIDE-EFFECTING: secret-scan evidence, verify exact source/build/artifact provenance, and sign a release receipt with immutable journey identities and report-typed publication media. The signing key is the file PROBIERZ_RECEIPT_PRIVATE_KEY_FILE names in the MCP server's environment.
  • probierz_verify_receipt — Verify receipt payload hash and Ed25519 signature against an explicit trusted public key or fingerprint.
  • probierz_create_publication_manifest — SIDE-EFFECTING: verify a signed receipt, current source, secret scan, evidence hashes, driver capability, redaction review, and immutable storage registrations before emitting a deterministic first-use publication manifest.
  • probierz_start_run — HEAVY + SIDE-EFFECTING: start a real run asynchronously and return its runId immediately. Poll with probierz_run_status; cancel with probierz_cancel_run.
  • probierz_run_status — Return queued/running/blocked/passed/failed/canceled state for an asynchronous run.
  • probierz_cancel_run — Cancel an asynchronous run and terminate its complete spawned process tree.
  • probierz_get_result — Return the completed normalized result and evidence for an asynchronous run.
  • probierz_list_artifacts — List run-scoped evidence artifacts for a completed asynchronous run.
  • probierz_get_artifact — Read one run-scoped artifact whole as base64; path traversal is rejected.
  • probierz_stado_collect — Read an existing Stado job once and, when it is terminal, import its retained evidence without submitting or running it again.
  • probierz_stado_resume — Wait for an existing Stado job with its original run budget and import its retained evidence; never submits a replacement.

Source snapshot#

Generated from https://github.com/wisent-ai/probierz.git revision `f65b755da397f2ec7435eb80ed722aa6017437c1` by calling the Rust server's `tools/list` operation.

The captured server exposes 49 tools. Binary SHA-256: `d5398832d3ab52c541b1efe979730dea62941ac6197898a20fccfc5d3fca44df`; tools/list SHA-256: `e0ac65d9045c1fdeb35ed9b386f1532ec72ee0dca4586e624c817fec41bbaf86`.