CLI command

probierz load run

Put a journey's load profile on the product's test environment.

Invocation#

Shell
probierz load run <APP_ID> <JOURNEY>

Required inputs and options#

  • <APP_ID> is the product, as probierz apps list names it.
  • <JOURNEY> declares load: { users, durationSeconds, rampSeconds, windowSeconds } and its api steps in .probierz/<app>/api/<spec>.json. windowSeconds is optional: for a long run (a soak), the length of each window of time it is also measured in; it must leave at least two windows in durationSeconds.
  • The manifest's optional data.metrics is the product's own command (the same command, args and cwd as data.seed) that prints one JSON object of numbers about its servers now (memory, open connections, whatever it counts); it is run with PROBIERZ_LOAD_ID and PROBIERZ_ENVIRONMENT_URL.

Output and state effects#

Adds users evenly over rampSeconds up to users and holds them until durationSeconds; every request is timed and counted per level of concurrent users, and the first level with errors is the breaking one. Each user sends the journey's steps in order, again and again, keeping what its own steps capture (a login's token) for its next steps as a run does, so a journey that signs in is loaded with each user's own session; a step needing a value its user never captured fails, naming the value. Each level counts every request it sent once: an answer off the step's expected status is timed and is one error named by both statuses, and a request that got no answer is one error named by why, so errorRate is errors over requests (1 when every answer is off its status). With windowSeconds, the same requests are also counted per window since the start (windows, each with fromSeconds, requests, errors and percentiles), and drift is the last window's 95th percentile over that of the first window that starts once every user arrived, so a product that slows as a soak goes on (a leak, a growing table) says by how much; without two such windows that timed an answer it is null. With windowSeconds and data.metrics, the command runs at the end of each window and what it printed is that window's metrics; metricsGrowth is each number's last value minus its value in the first window after the ramp, so memory or connections the product never gives back show as they grow. A sample that cannot be taken (the command fails, or prints anything but a JSON object of numbers) is that window's metricsError naming why, and the load still completes. Users that ended abnormally are counted as usersLost. The run is kept under test-results/.load/<app>/ with its host time as cost. Real tests: tests/load/run.rs (product servers on the machine, one answering 500 to every request, one answering 401 to a request without a token it issued) and tests/load/product.rs (the test product, its pool fault, a soak in windows, and a soak sampling the product's cli.mjs metrics).

Exact refusals#

  • A journey without a load profile, a manifest without environments.test.baseUrl (load never goes to production), and a profile that needs more host seconds than today's budget.host-seconds-per-day leaves are refused naming the fix.
  • manifest-invalid (exit 1): a windowSeconds that is not whole seconds above 0, or that leaves fewer than two windows in durationSeconds, naming the journey. Real test: tests/load/product.rs.

Source snapshot#

Generated from the Rust binaries at revision 99a74945a3598a8d73b6c6cbe5aae4f283becea3 of Probierz's private source repository.

Combined help SHA-256: 77529678a61237f774b59e76df0d1907cb749ef5da56d2ed913b040793b7d30b; binary SHA-256: be9e0eaff0e6ea50403a0c6f6d467697a4eac676f9523b8dc13a596d2b5526c5.