CLI command

probierz durations measure

Run one operation several times, measure each run, and keep the median with the exact code it ran.

Invocation#

Shell
probierz durations measure <name> --runs <N> [--cwd <directory>] -- <program> [<argument>...]

Required inputs and options#

  • <name> names the operation across measurements: lowercase letters, digits, ., - and _, starting with a letter or digit. --runs is how many times it runs, a whole number above zero. --cwd is the absolute directory it runs in, the current one unless named.
  • <program> is found on PATH unless it names a path. A name measured before keeps its command and directory: an optimization changes the code a command runs, not the command.

Output and state effects#

Runs the command once per run, one after the other, standard input empty, its output captured and not printed, each run waiting through Stado's process wait, and times each run on the wall clock. Keeps {id, name, command, cwd, program: {path, sha256}, checkout: {root, revision, dirty} | null, runs: [{seconds, exit}], medianSeconds, minSeconds, maxSeconds, threshold, slow, measuredAt} in test-results/.durations/<name>/<id>.json under the harness and prints it. program.sha256 is the digest of the executable that ran and checkout the git checkout holding the directory, if any, with its HEAD and whether its tree held uncommitted changes or a file no commit holds (an untracked file is code the command may have run beside the revision), so a measurement names the code it measured. slow is whether the median exceeds durations.slow-seconds; with no threshold set, threshold and slow are null.

Exact refusals#

  • name-invalid, runs-invalid, cwd-invalid (exit 2): the name, the number of runs or the directory is not as above; nothing runs.
  • command-changed (exit 2): the name was measured before with another command or directory; both are named, and nothing runs.
  • program-missing (exit 1): no executable answers to the program on PATH or at the path named; nothing runs.
  • run-failed (exit 1): a run could not be started, exited other than 0, or was ended by a signal; the run's number, its status and the end of its standard error are named, and nothing is kept, since a failing command measures nothing. Real test: tests/durations/durations.rs.

Declared refusals#

What probierz refusals list declares for this command at revision d2cd6156e8b8dc6204dab7b46ec0b34b2bb03cea (probierz 0.1.7 (revision unknown, not built from a git checkout)): each cause's id, the operation declaring it, its exit status and code, and its remedy.

  • threshold-unset (probierz.durations, exit 1, config): {doing}: no threshold names how long an operation may take before it counts as slow: neither --over nor this instance's durations.slow-seconds is set. set the threshold with probierz config set durations.slow-seconds <seconds>, or name it with --over.
  • register-unread (probierz.durations, exit 69, infra_down): reading the waits Stado measured on this machine ({command}): {why}. install Stado where PATH finds it (stado --version), then repair what its answer names; stado waits measured --json is the reading.
  • name-invalid (probierz.durations, exit 2, refused): naming the operation {name}: an operation's name is lowercase letters, digits, ., - and _, starting with a letter or digit, so it is one directory of the kept measurements. name it like oko-defects-list.
  • runs-invalid (probierz.durations, exit 2, refused): measuring {name} {runs} time(s): a measurement runs the operation at least once. pass --runs with a whole number above zero.
  • cwd-invalid (probierz.durations, exit 2, refused): running {name} in {cwd}: the directory an operation runs in is absolute and exists, so every measurement of it runs in the same place. pass --cwd with an absolute directory that exists.
  • command-changed (probierz.durations, exit 2, refused): measuring {name} as {command} in {cwd}: {name} was first measured as {first} in {first_cwd}; an optimization changes the code a command runs, not the command, so its measurements compare only while the command stays. measure it with its first command, or name the new command another operation.
  • program-missing (probierz.durations, exit 1, config): finding the program {program} of {name}: {why}. name the program by a path that exists, or install it where PATH finds it.
  • run-failed (probierz.durations, exit 1, config): run {run} of {runs} of {name} ({command}): {status}, and a command that fails measures nothing. make the command succeed in {cwd}, then measure it again.
  • measurement-unreadable (probierz.durations, exit 1, config): reading the kept measurement {file}: {why}. remove the file, or put back the measurement probierz durations measure wrote.
  • measurement-unknown (probierz.durations, exit 2, refused): finding {what}: no measurement is kept under that name or id. name one probierz durations list (or show <name>) shows.
  • reduction-unset (probierz.durations, exit 1, config): deciding whether {name} was cut by the share this instance requires: this instance's durations.reduction is unset, so no share decides it. set the share with probierz config set durations.reduction <share>.
  • baseline-missing (probierz.durations, exit 1, config): comparing {name} with its first measurement: {name} was measured once, so nothing is there to compare it with. measure it again after the optimization: probierz durations measure {name} … with its first command.
  • not-reduced (probierz.durations, exit 1, config): cutting the time of {count} operation(s) by {reduction}: their median is above {required} of their first measurement's. optimize what each named operation spends its time on, then measure it again.

Source snapshot#

Generated from the Rust binaries at revision 99a74945a3598a8d73b6c6cbe5aae4f283becea3 of Probierz's private source repository.

Combined help SHA-256: 77529678a61237f774b59e76df0d1907cb749ef5da56d2ed913b040793b7d30b; binary SHA-256: be9e0eaff0e6ea50403a0c6f6d467697a4eac676f9523b8dc13a596d2b5526c5.