WisentProbierz
Probierz/Docs

CLI command

probierz benchmark compare

Compare two runs of the same suite contender by contender.

Invocation#

Shell
probierz benchmark compare --baseline <BASELINE> --candidate <CANDIDATE> <APP_ID>

Required inputs and options#

  • Both runs must carry the same suite hash.

Output and state effects#

Read-only. For every contender in both runs: the pass-rate change in percentage points and the p50 and p95 ratios (candidate divided by baseline, lower is faster). Contenders present in only one run are listed.

Exact refusals#

  • `<baseline> ran suite <id> version <v> (<hash>) and <candidate> ran suite <id> version <v> (<hash>); only runs of the same suite are compared`.

Source snapshot#

Generated from the Rust binaries at https://github.com/wisent-ai/probierz.git revision `f65b755da397f2ec7435eb80ed722aa6017437c1`.

Combined help SHA-256: `4a12655d233812abc29eeebbf3db3065efe3e00288024383f3eef3970d092e94`; binary SHA-256: `d5398832d3ab52c541b1efe979730dea62941ac6197898a20fccfc5d3fca44df`.