CLI command
probierz benchmark board
Write the public leaderboard of every market across products' newest runs.
Invocation#
probierz benchmark board --out <OUT>Required inputs and options#
--appnames a product; repeat it for several. Without it every product with a directory undertest-results/.benchmark/is on the board.--outis the directory written.
Output and state effects#
For each product, the newest run of each suite is scored on the axes of its catalog product, and every market it measures is one entry of that market: board.json (ai.wisent.probierz.benchmark.board.v1: {schema, generatedAt, apps, markets: {<market>: [{app, product, suite, runId, startedAt, ranking, axes, ours, definitions, unmeasured, scored}]}}) and index.html, a static page with no script and its style inline: per market, contenders by composite with their success rate and each axis's median and score, each axis linking the page its scale comes from. Answers the two paths, the apps and the markets.
Exact refusals#
- Everything the manifest's declaration and
benchmark standing's markets refuse for a product named. probierz.benchmark.board:apps-unlistablewhen the runs directory cannot be listed, andboard-unwrittenwhen--outor a file in it cannot be written.
Source snapshot#
Generated from the Rust binaries at revision 99a74945a3598a8d73b6c6cbe5aae4f283becea3 of Probierz's private source repository.
Combined help SHA-256: 77529678a61237f774b59e76df0d1907cb749ef5da56d2ed913b040793b7d30b; binary SHA-256: be9e0eaff0e6ea50403a0c6f6d467697a4eac676f9523b8dc13a596d2b5526c5.