Artillery Pro
CLI Reference

Artillery Pro CLI

Artillery Pro has entered maintenance mode and will be officially sunset in June 2024. New releases will be made only to address critical bugs or security vulnerabilities.

The distributed load testing functionality of Artillery Pro is now available in the main open source distribution of Artillery.

Requirements

Artillery Pro needs to be installed. Once Artillery Pro is installed, several new commands become available in the artillery CLI. This page documents those commands.

Artillery Pro is on a different release cycle from Artillery and has its own versioning.

All Artillery Pro versions are compatible with both Artillery v1 and Artillery v2. If an Artillery Pro release requires a specific version of Artillery, it will be indicated in the Changelog and checked by Artillery Pro itself at runtime.

run-test - run a test from your cloud environment

artillery run-test [options] <test>

This command is similar to the Core run command, but instead of running a test from a single machine, the run-test command runs it from an Amazon Elastic Container Service (opens in a new tab) or Amazon Fargate (opens in a new tab) cluster. This command lets you scale up and run your tests in distributed mode across multiple workers, and from a number of geographical regions.

For example, if you have a test script called my-script.yaml, you can run it in a single, local instance with Artillery Core using the run command:

artillery run my-script.yaml

The run-test command makes it easy to scale up and run the same script from the cloud. For example, you can scale up to 20 workers (instead of single worker) running in the us-east-1 region on Fargate with the following Artillery Pro command:

artillery run-test \
    --region us-east-1 \
    --cluster mycluster \
    --count 20 \
    --launch-type ecs:fargate \
    my-script.yaml

See Running tests from AWS for a detailed guide on running Artillery tests from AWS.

Options

The run-test command shares several flags with run, which allows for a seamless transition from running tests locally to running them in the cloud.

OptionDescription
--output, -oWrite a JSON report to a file
--environment, -eRun the test using the specified environment
--config, -cLoad config section from another file
--overridesOverride values in the test script dynamically
--target, -tSet or override target URL for the test
--insecure, -kTurn off TLS verification. Should not be used in production

These flags work in the exact same way in run-test as in run.

The run-test command has several other flags specifically for configuring how a test should be executed from a cloud environment.

OptionDescription
--clusterThe name of the cluster to run the test from
--launch-typeSet the platform to run tests from. Possible values: ecs:fargate and ecs:ec2 (default)
--countSet the number of workers for running the test
--secretMake a secret created with set-secret command available in the test as an environment variable  
--region, -rThe region where the test will run from. Artillery Pro supports 13 different regions
--launch-configA JSON object describing launch configuration for ECS tasks
--subnet-ids[Fargate-only] Comma-separated list of subnet IDs to place tasks into
--security-group-ids[Fargate-only] Comma-separated list of security group IDs to attach to tasks
--max-duration <time>Set maximum duration of a test run. Duration is set with a string like 10m for 10 minutes or 1h for one hour. Only integer values may be specified. Supported units are ms, s, m and h. The test run will be stopped if the running time of a test exceeds the limit
--dotenv <path>Set environment variables in workers with a .env file
--tags <tags>Comma-separated list of tags in key:value format to tag the test run, for example: --tags "team:sre,service:foo"
--note <msg>Add a note to the test run

Examples

Run a test by launching 10 workers on Amazon Fargate in the US West (Oregon) region:

artillery run-test \
    --region us-west-2 \
    --cluster mycluster \
    --count 10 \
    --launch-type ecs:fargate \
    my-script.yaml

Run a test by launching 20 workers on Amazon ECS in the Europe (Ireland) region:

artillery run-test \
    --region eu-west-1 \
    --cluster mycluster \
    --count 20 \
    --launch-type ecs:ec2 \
    my-script.yaml

stop-test - stop a running test

The stop-test command stops a running test gracefully, by stopping worker tasks and performing other clean up tasks as necessary.

artillery stop-test <id>

The run-test command prints test IDs at the beginning of each test run. IDs of currently running tests may also be seen by running artillery get-tests.

Example

Stop a running test with id = e84a482c-cf34-404c-a95d-22446b92a2c1.

artillery stop-test e84a482c-cf34-404c-a95d-22446b92a2c1

describe-test-run v3

Description

Return the details of a test that was run or is still running, as JSON.

Synopsis

artillery describe-test-run <id>

id is the test run id, which is printed by test-run or returned by list-test-runs command.

Examples

# Get the details for a test run with id = test-e7107e1
artillery describe-test-run test-e7107e1

list-test-runs v3

Description

List ids of past test runs, as JSON.

Synopsis

artillery list-test-runs

delete-test-run v3

Description

Delete a test run by its id.

Examples

artillery delete-test-run aad11369-9c3e-4f18-b517-62aa2a376705

get-tests v2list-tests v3

List currently running tests.

# Artillery Pro v2
artillery get-tests
 
# Artillery Pro v3
artillery list-tests

Options

OptionDescription
--jsonFormat output as JSON

create-test v2create-test-bundle v3

Description

The create-test-bundle command resolves a test script's dependencies (such as external CSV files, custom JS code and its dependencies), and uploads everything needed to run the script to a location in Amazon S3.

The name given to the test may then be used with the run-test command to reduce the amount of time it takes to set up a test environment before the running the test script.

You don't need to create a test bundle before running a test script with run-test. This command provides a way to optimize the running time of a script with many dependencies, but its usage is optional.

Synopsis

artillery create-test-bundle [options] <script>

Options

OptionDescription
--name, -nThe name for the test (to be used in run-test)
--configUse common test configuration for this named test

Examples

Create a named test from a test script in my-script.yaml called my-named-test:

artillery create-test-bundle --name my-named-test my-script.yaml

If the script makes use of CSV files for variable data, plugins, or custom code (which may make use of external npm dependencies), all of those references will be resolved automatically.

After creating the test, you can run the test using run-test with the --bundle option:

artillery run-test \
    --region ap-northeast-1 \
    --count 25 \
    --bundle my-named-test

delete-test v2delete-test-bundle v3

Description

Delete a named test bundle created with create-test-bundle.

Synopsis

artillery delete-test-bundle <name>

Examples

Delete an existing named test called my-named-test:

artillery delete-test-bundle my-named-test

list-tests v2list-test-bundles v3

Description

List all named tests created with create-test-bundle.

Synopsis

artillery list-test-bundles [options]

Options

OptionDescription
--jsonFormat output as JSON

deploy - deploy Artillery Pro backend

Description

Set up the Artillery Pro backend components in your AWS account. A CloudFormation stack is created (or updated) with the resources needed to run Artillery Pro tests. See Installing Artillery Pro for more information on deploying the backend.

Synopsis

artillery deploy [options]

Options

OptionDescription
--region, -rThe AWS region to deploy the backend into
--upgradeUpgrade an existing Artillery Pro installation in an AWS account
--license-key-fileSpecify the path to file containing an Artillery Pro license key
--license-key-textSpecify the contents of an Artillery Pro license key
--set <key=value>Customize names of resources in the CloudFormation stack

Upgrading an existing installation

To upgrade an existing installation of Artillery Pro, install the latest version of the CLI, and run the deploy command in the region in which the backend is already deployed, e.g.:

artillery deploy --upgrade --region us-east-1

Customizing CloudFormation resource names

You can customize the names of some resources created by the Artillery Pro CloudFormation stack (to follow naming standards in your organization for example). The --set flag currently supports the following options:

KeyDescription
cli-user-role-nameThe name of IAM role for CLI users
cli-user-policy-nameThe name of IAM managed policy assumed by CLI user role

Example: set a custom name for the IAM role for CLI users who run tests

This examples presumes an existing installation in us-east-1 that we're wishing to upgrade, and set the name of the IAM role generated for Artillery Pro CLI users to tools-sre-artillery.

artillery deploy --set cli-user-role-name=tools-sre-artillery --region us-east-1

set-secret

artillery set-secret [options]

The set-secret command creates an encrypted secret value which can be used in test runs. You can then access these values in your test scenarios via the $env variable.

How are the secrets stored?

The secrets are encrypted and stored securely in AWS Parameter Store (opens in a new tab) in your AWS account (under /artilleryio/ scope).

Options

OptionDescription
--name, -nName of the secret
--value, -vValue of the secret
--backendSpecify the secret storage backend (currently only ssm)
--region, -rAWS region where the secret should be created

Examples

Create a secret called API_KEY with a value of 012345 in the US East (N. Virginia) region:

artillery create-secret --name API_KEY --value 012345 --region us-east-1

You can then make this secret available in a test run using the --secret flag in the run-test command:

artillery run-test --secret API_KEY --launch-type ecs:fargate my-test.yml

The value for the secret will be available as an API_KEY environment variable accessible via the special $env variable:

- post:
    url: "/pets"
    headers:
      x-api-key: "{{ $env.API_KEY }}"
    json:
      name: Tiki
      species: pony

list-secrets v3

Description

List secrets created with set-secret.

Synopsis

artillery list-secrets

list-tags v3

Description

List all tags that have been used to tag test runs, as JSON.

Synopsis

artillery list-tags

post-note v3

Description

Add a note to a test run. The notes are available in the output from describe-test-run.

A common use-case is using post-note to annotate test runs with links to descriptions of services tested, monitoring dashboards, project management boards, or runbooks.

Synopsis

artillery post-note --test-run-id <id> --text <text>

Examples

artillery post-note \
  --test-run-id test-e7107e1 \
  --text "See monitoring dashboard on https://app.datadoghq.com/dashboards/foo-service "

set-config-value

Description

Set runtime configuration values which will be available to workers running on ECS/Fargate in a secure way. These configuration values can be used for installing private packages from npm (opens in a new tab) and Artifactory (opens in a new tab), and for customizing your npm configuration.

See Private npm packages for more information on using private packages in your tests.

Synopsis

artillery set-config-value [options]

Options

OptionDescription
--name, -nName of the configuration parameter (see available parameters below)
--value, -vValue of the configuration parameter
--region, -rAWS region where Artillery Pro is deployed

Configuration parameters

You can set values for the following configuration parameters:

ParameterDescription
NPM_TOKENAuth token to use with npm
NPM_REGISTRYURL of private npm registry to use
ARTIFACTORY_AUTHAuth token to use with Artifactory
ARTIFACTORY_EMAILEmail to use with Artifactory
NPMRCCustomize .npmrc with arbitrary config
GITHUB_CLIENT_ID, GITHUB_CLIENT_SECRETClient ID and secret. Set these to enable GitHub-based login for Artillery dashboard
GITHUB_ALLOWED_USERS, GITHUB_ALLOWED_ORGSA JSON-encoded list of usernames or organization names which are allowed to access the dashboard, for example: ["artilleryio"]

Examples

Set the npm authentication token for an Artillery Pro deployment in the US East (N. Virginia) region:

artillery set-config-value \
    --name NPM_TOKEN \
    --value 01234567 \
    --region us-east-1

Set the loglevel setting for the .npmrc configuration to debug for the workers in an Artillery Pro deployment in the Europe (Ireland) region:

artillery set-config-value \
    --name NPMRC \
    --value "loglevel=debug" \
    --region eu-west-1

subscription

artillery subscription [options]

The subscription command shows the status of your team's Artillery Pro subscription. You can also use this command to request an Artillery Pro license key to be sent to an email address associated with the subscription.

Options

OptionDescription
--emailRequest license key to be sent to an email address