WisentProbierz
Probierz/Docs

CLI command

probierz benchmark scout

Scout a product that does not exist yet from a Trends topic: the products the topic's observations show, the rivals a new product must beat among them, and the suite that measures it.

Invocation#

Shell
probierz benchmark scout --owner <OWNER> --observations <OBSERVATIONS> --rounds <ROUNDS> <TOPIC>

Required inputs and options#

  • `<topic>` is a topic Trends watches, and `--owner` the GitHub owner the new repository is created under; both must be lowercase letters, digits and inner hyphens starting with a letter, because both become Stado identities. `--observations` (how many of the topic's newest observations the model reads) and `--rounds` (how many drafts each question is allowed) are required.
  • `trends` on PATH (with `TRENDS_STATE_FILE` when its state is not the default), the `competitors` CLI of competitors-cli on PATH, Stado for the product catalog, and the same model router settings as `benchmark author suite`.

Output and state effects#

Reads Trends' verdict for the topic (`trends detect`) and goes on only when Trends holds at least its evidence floor for the topic and the recent window is not below the one before. Reads the newest observations (`trends observations`), asks the model router which products those observations themselves show — each with its domain, https URL, one-sentence description and the observation ids that show it — and passes them to `competitors discover`, which merges and deduplicates them by its own identity rules. Then asks for the one new product to build: its id (none of the catalog's), name and description (at most 350 bytes), its first surface, the gap none of the rivals fills, the candidate ids it must beat and the suite id that measures them, with the observation ids the gap rests on. A draft that cites an id it was not given, names an unknown candidate or reuses a catalog id is sent back with the reason. Writes the brief (`ai.wisent.probierz.benchmark.opportunity.v1`) once to `test-results/.scout/<topic>/scout-<topic>-<timestamp>.json`, beside the discovery records competitors-cli read: the trend, the observations, the candidates, the gap, the catalog rivals `{id, name, url, evidence}`, the benchmark `{app, suites}`, the Stado creation request, and each round's model and usage. The answer names the brief and the `adopt` command that acts on it.

Exact refusals#

  • `topic … and --owner … must be lowercase letters, digits and inner hyphens, starting with a letter: both become Stado identities`, and `--observations and --rounds must be at least 1`.
  • Trends' own refusal, such as `no topic '<topic>'`, quoted with its exit status.
  • `Trends reads topic <topic> as <verdict>: <n> observations against a floor of <m>, <r> recent against <b> before; …` (exit 1, refused), after printing the trend: no brief is written.
  • `trends`, `competitors` or `stado` that cannot start is named with what must be on PATH (exit 1, prerequisite); a model router that is unset or refuses is named with its own sentence (exit 75, unavailable).
  • `after <n> draft(s) no <tool> answer cited only what it was given: <reason>` (exit 1, refused).

Source snapshot#

Generated from the Rust binaries at https://github.com/wisent-ai/probierz.git revision `f65b755da397f2ec7435eb80ed722aa6017437c1`.

Combined help SHA-256: `4a12655d233812abc29eeebbf3db3065efe3e00288024383f3eef3970d092e94`; binary SHA-256: `d5398832d3ab52c541b1efe979730dea62941ac6197898a20fccfc5d3fca44df`.