Skip to main content

Artillery CLI

What you'll learn
  • How to run Artillery tests from the command line
  • How to run tests from a single machine with artillery run
  • How to accomplish other tasks with the Artillery CLI, such as generating HTML reports

Overview

Requirements

This guide assumes you've already read our Installing Artillery guide, and installed Artillery on your machine.

For a reference of commands added by Artillery Pro please refer to Artillery Pro CLI.

Using the CLI

Listing all commands

Run artillery --help to see all of the available commands.

To see more information about a specific command, such as the full list of flags it supports, use the --help flag with that command. For instance, to list all flags supported by the run command:

artillery run --help

Getting version info

Run artillery -V to print the version of Artillery (and Artillery Pro if installed), as well as information about the environment, such as Node.js version and the OS.

$ artillery -V

___ __ _ ____ _
_____/ | _____/ /_(_) / /__ _______ __ (_)___ _____
/____/ /| | / ___/ __/ / / / _ \/ ___/ / / / / / __ \/____/
/____/ ___ |/ / / /_/ / / / __/ / / /_/ / / / /_/ /____/
/_/ |_/_/ \__/_/_/_/\___/_/ \__, (_)_/\____/
/____/

------------ Version Info ------------
Artillery: 1.6.0
Artillery Pro: 2.2.0
Node.js: v12.16.0
OS: darwin/x64
--------------------------------------

run

artillery run [options] <script>

The run command executes a test script through to completion. The basic way to run a test is with:

artillery run my-script.yaml

Options

OptionDescription
--output, -oWrite a JSON report to a file
--environment, -eRun the test using the specified environment
--config, -cLoad config section from another file
--overridesOverride values in the test script dynamically
--target, -tSet or override target URL for the test
--variables, -vSet scenario variables dynamically
--insecure, -kTurn off TLS verification. Should not be used in production
--quiet, -qRun in "quiet" mode

Create a JSON report

You can tell Artillery to write a JSON report of a test run into a file. The JSON report can be used to generate an HTML report with the report command, or to run queries on with a tool like jq.

artillery run --output report.json my-script.yaml

Test different deployment environments

A service is usually deployed in more than one place. For example, your team may use pull request or preview environments, which create a separate deployment of a service for each pull request or feature branch. Most teams often have one or more staging deployment and at least one production deployment. Artillery lets you describe those environments in your test script and switch between to run tests.

Environment-specific configuration is set in the config.environments settings in the test script:

config:
target: "https://service-foo.acmecorp.digital" # default target
phases:
- arrivalRate: 50
duration: 600
environments:
local-dev:
target: "http://localhost:8080"
phases:
- arrivalRate: 10
duration: 60
preprod:
target: "https://service-foo.preprod.acmecorp.digital"
scenarios:
# Scenario definitions would go here.

The above example script defines two environments: local-dev and preprod. Environment-specific definitions such as target or phases will override the top-level ones defined in config.

You may then run the same test against the service running locally on http://localhost:8080 with:

artillery run --environment local-dev my-script.yaml

The same script could be used in a post-deployment CI job to run tests against the preprod version of the service on https://service-foo.preprod.acmecorp.digital with:

artillery run --environment preprod my-script.yaml

Extract and reuse configuration

While Artillery test scripts allow for any number of scenarios to be defined in the same test script, it's useful to keep individual scenarios in their own files for reusability and to prevent duplicating the configuration settings.

Consider the following example layout of a repository containing two Artillery test scripts, edit-records.yaml and search-records.yaml:

acmecorp-backend-tests/
└── services/
└── service-foo/
├── config.yaml
└── scenarios/
├── edit-records.yaml
└── search-records.yaml

For both tests, you can define the same config settings to run each test script individually. However, if you need to modify the test configuration (like changing the target URL, for instance), you would need to make the same change on all scripts, which has the risk of forgetting to update all test scripts where needed.

To keep your test script settings identical, you can reuse the same config settings by defining a separate YAML file. In this example, the config.yaml contains common configuration used for both test scripts:

config:
target: "https://service-foo.acmecorp.digital"
phases:
- arrivalRate: 50
duration: 600

With a shared configuration, each test script can only contain a scenarios section to define the scenarios for executing the test. You can use the shared configuration with both test scripts using the --config flag:

# Run edit-records scenarios with the shared configuration:
artillery run --config config.yaml scenarios/edit-records.yaml

# Run search-records scenarios with the shared configuration:
artillery run --config config.yaml scenarios/search-records.yaml

When using --config, you can merge different configuration settings between the shared configuration file and the test scripts. However, the settings from the shared configuration will take precedence over the same settings inside the test scenario.

Override parts of the test script on the fly

If you want to override parts of a test script from the command line, you can use the --overrides flag. The flag accepts a JSON string with parts of the test script you wish to override.

For example, you have the following test script, containing a phase generating 50 virtual users per second for 10 minutes:

config:
target: "https://service-foo.acmecorp.digital"
phases:
- arrivalRate: 50
duration: 600
scenarios:
# Scenario definitions would go here.

You can use the --overrides flag to override the configuration and run a phase generating 1 virtual user per second for 10 seconds:

artillery run \
--overrides '{"config": { "phases": [{ "duration": 10, "arrivalRate": 1 }] } }' \
my-script.yaml

If you're running Artillery on Windows using the Command Prompt, you'll need to wrap the --overrides flag values using double-quotes and escape the inner quotes of the JSON string using backslashes:

artillery run \
--overrides "{\"config\": { \"phases\": [{ \"duration\": 10, \"arrivalRate\": 1 }] } }" \
my-script.yaml

If you're running Artillery on Windows using PowerShell, you'll need to wrap the --overrides flag values using single-quotes and escape the inner quotes of the JSON string using backslashes:

artillery run \
--overrides '{\"config\": { \"phases\": [{ \"duration\": 10, \"arrivalRate\": 1 }] } }' \
my-script.yaml

Set scenario variables on the fly

To define variables on the fly to use in your test scenarios, the --variables flag allows you to set a JSON string containing one or more variable names with multiple values.

The following example makes two variables - color and size - available in scenario definitions:

artillery run \
--variables '{ "color": ["red", "yellow", "blue"], "size": [120, 150, 200] }' \
my-script.yaml

The variables may be used as normal in scenario definitions:

scenario:
- name: Create record
flow:
- post:
url: "/items"
json:
itemColor: "{{ color }}"
itemSize: "{{ size }}"

As with setting variables in the configuration, each virtual user will get one of the available values for each variable.

If you're running Artillery on Windows using the Command Prompt, you'll need to wrap the --variables flag values using double-quotes and escape the inner quotes of the JSON string using backslashes:

artillery run \
--variables "{ \"color\": [\"red\", \"yellow\", \"blue\"], \"size\": [120, 150, 200] }" \
my-script.yaml

If you're running Artillery on Windows using PowerShell, you'll need to wrap the --variables flag values using single-quotes and escape the inner quotes of the JSON string using backslashes:

artillery run \
--variables '{ \"color\": [\"red\", \"yellow\", \"blue\"], \"size\": [120, 150, 200] }' \
my-script.yaml

Override the target URL

The --target flag lets you override the specified target URL of a test script:

artillery run --target https://service-bar.acmecorp.digital my-script.yaml

Turn off TLS verification

By default, Artillery will reject SSL certificates that it's unable to validate. You can disable certificate validation with the --insecure flag:

artillery run --insecure my-script.yaml

This flag is useful for testing a service in a development or staging environment that doesn't have a SSL certificate that can be validated, like a self-signed certificate.

caution

You should never use the --insecure flag on a production environment. Ignoring certificate errors in a production system can lead to potential security vulnerabilities, like man-in-the-middle attacks.

Suppress output when running tests

When running a test, Artillery prints a report on the console every 10 seconds for the number of scenarios executed during that period. At the end of the performance test, it prints a complete summary.

You can suppress this output if you don't need it (like when running tests on a continuous integration environment) using the --quiet flag:

artillery run --quiet my-script.yaml

quick v1 only

v1 only

artillery quick is not available in Artillery v2

artillery quick [options] <target>

Use this command to run a quick HTTP test without needing to write any scenarios or need a test script.

The quick command only works to test HTTP services. You cannot run quick tests for other services, such as Socket.IO or WebSockets.

Options

OptionDescription
--rate, -rSpecify the number of new arrivals per second
--count, -cSpecify a fixed number of arrivals
--duration, -dSpecify the duration of the arrival phase
--num, -nSpecify the number of GET requests each new arrival will send
--content-type, -tSet the Content-Type for the request (defaults to application/json)
--output, -oWrite a JSON report to a file
--insecure, -kAllow insecure TLS connections
--quiet, -qRun in "quiet" mode

Examples

Run a quick test which generates 20 virtual users, each sending 100 GET requests to the specified target:

artillery quick \
--count 20 \
--num 100 \
https://service-foo.preprod.acmecorp.digital/items

Run a quick test which generates 10 virtual users per second, each sending 50 GET requests to the specified target:

artillery quick \
--rate 10 \
--num 50 \
https://service-foo.preprod.acmecorp.digital/items

report

artillery report [options] <file>

The report command converts a JSON report generated by Artillery into a self-contained HTML report.

Options

OptionDescription
--output, -oSet the filename to write the HTML report

Generating an HTML report

First, run a test and create a JSON report with the --output flag.

artillery run --output report.json my-script.yaml

You can then use the generated JSON report to create an HTML report:

artillery report --output report report.json

If you don't need to save the HTML report into a file, you can omit the --output flag to open the report in your default browser.