Artillery Pro CLI

Requirements

Artillery Pro needs to be installed. Once Artillery Pro is installed, several new commands become available in the artillery CLI. This page documents those commands.

run-test - run a test from your cloud environment

artillery run-test [options] <test>

This command is similar to the Core run command, but instead of running a test from a single machine, the run-test command runs it from an Amazon Elastic Container Service (opens in a new tab) or Amazon Fargate (opens in a new tab) cluster. This command lets you scale up and run your tests in distributed mode across multiple workers, and from a number of geographical regions.

For example, if you have a test script called my-script.yaml, you can run it in a single, local instance with Artillery Core using the run command:

artillery run my-script.yaml

The run-test command makes it easy to scale up and run the same script from the cloud. For example, you can scale up to 20 workers (instead of single worker) running in the us-east-1 region on Fargate with the following Artillery Pro command:

artillery run-test \
    --region us-east-1 \
    --cluster mycluster \
    --count 20 \
    --launch-type ecs:fargate \
    my-script.yaml

See Running tests from AWS for a detailed guide on running Artillery tests from AWS.

Options

The run-test command shares several flags with run, which allows for a seamless transition from running tests locally to running them in the cloud.

These flags work in the exact same way in run-test as in run.

The run-test command has several other flags specifically for configuring how a test should be executed from a cloud environment.

Examples

Run a test by launching 10 workers on Amazon Fargate in the US West (Oregon) region:

artillery run-test \
    --region us-west-2 \
    --cluster mycluster \
    --count 10 \
    --launch-type ecs:fargate \
    my-script.yaml

Run a test by launching 20 workers on Amazon ECS in the Europe (Ireland) region:

artillery run-test \
    --region eu-west-1 \
    --cluster mycluster \
    --count 20 \
    --launch-type ecs:ec2 \
    my-script.yaml

stop-test - stop a running test

The stop-test command stops a running test gracefully, by stopping worker tasks and performing other clean up tasks as necessary.

artillery stop-test <id>

The run-test command prints test IDs at the beginning of each test run. IDs of currently running tests may also be seen by running artillery get-tests.

Example

Stop a running test with id = e84a482c-cf34-404c-a95d-22446b92a2c1.

artillery stop-test e84a482c-cf34-404c-a95d-22446b92a2c1

describe-test-run v3

Description

Return the details of a test that was run or is still running, as JSON.

Synopsis

artillery describe-test-run <id>

id is the test run id, which is printed by test-run or returned by list-test-runs command.

Examples

# Get the details for a test run with id = test-e7107e1
artillery describe-test-run test-e7107e1

list-test-runs v3

Description

List ids of past test runs, as JSON.

Synopsis

artillery list-test-runs

delete-test-run v3

Description

Delete a test run by its id.

Examples

artillery delete-test-run aad11369-9c3e-4f18-b517-62aa2a376705

get-tests v2 • list-tests v3

List currently running tests.

# Artillery Pro v2
artillery get-tests
 
# Artillery Pro v3
artillery list-tests

Options

create-test v2 • create-test-bundle v3

Description

The create-test-bundle command resolves a test script's dependencies (such as external CSV files, custom JS code and its dependencies), and uploads everything needed to run the script to a location in Amazon S3.

The name given to the test may then be used with the run-test command to reduce the amount of time it takes to set up a test environment before the running the test script.

Synopsis

artillery create-test-bundle [options] <script>

Options

Examples

Create a named test from a test script in my-script.yaml called my-named-test:

artillery create-test-bundle --name my-named-test my-script.yaml

If the script makes use of CSV files for variable data, plugins, or custom code (which may make use of external npm dependencies), all of those references will be resolved automatically.

After creating the test, you can run the test using run-test with the --bundle option:

artillery run-test \
    --region ap-northeast-1 \
    --count 25 \
    --bundle my-named-test

delete-test v2 • delete-test-bundle v3

Description

Delete a named test bundle created with create-test-bundle.

Synopsis

artillery delete-test-bundle <name>

Examples

Delete an existing named test called my-named-test:

artillery delete-test-bundle my-named-test

list-tests v2 • list-test-bundles v3

Description

List all named tests created with create-test-bundle.

Synopsis

artillery list-test-bundles [options]

Options

deploy - deploy Artillery Pro backend

Description

Set up the Artillery Pro backend components in your AWS account. A CloudFormation stack is created (or updated) with the resources needed to run Artillery Pro tests. See Installing Artillery Pro for more information on deploying the backend.

Synopsis

artillery deploy [options]

Options

Upgrading an existing installation

To upgrade an existing installation of Artillery Pro, install the latest version of the CLI, and run the deploy command in the region in which the backend is already deployed, e.g.:

artillery deploy --upgrade --region us-east-1

Customizing CloudFormation resource names

You can customize the names of some resources created by the Artillery Pro CloudFormation stack (to follow naming standards in your organization for example). The --set flag currently supports the following options:

Example: set a custom name for the IAM role for CLI users who run tests

This examples presumes an existing installation in us-east-1 that we're wishing to upgrade, and set the name of the IAM role generated for Artillery Pro CLI users to tools-sre-artillery.

artillery deploy --set cli-user-role-name=tools-sre-artillery --region us-east-1

set-secret

artillery set-secret [options]

The set-secret command creates an encrypted secret value which can be used in test runs. You can then access these values in your test scenarios via the $env variable.

Options

Examples

Create a secret called API_KEY with a value of 012345 in the US East (N. Virginia) region:

artillery create-secret --name API_KEY --value 012345 --region us-east-1

You can then make this secret available in a test run using the --secret flag in the run-test command:

artillery run-test --secret API_KEY --launch-type ecs:fargate my-test.yml

The value for the secret will be available as an API_KEY environment variable accessible via the special $env variable:

- post:
    url: "/pets"
    headers:
      x-api-key: "{{ $env.API_KEY }}"
    json:
      name: Tiki
      species: pony

list-secrets v3

Description

List secrets created with set-secret.

Synopsis

artillery list-secrets

list-tags v3

Description

List all tags that have been used to tag test runs, as JSON.

Synopsis

artillery list-tags

post-note v3

Description

Add a note to a test run. The notes are available in the output from describe-test-run.

A common use-case is using post-note to annotate test runs with links to descriptions of services tested, monitoring dashboards, project management boards, or runbooks.

Synopsis

artillery post-note --test-run-id <id> --text <text>

Examples

artillery post-note \
  --test-run-id test-e7107e1 \
  --text "See monitoring dashboard on https://app.datadoghq.com/dashboards/foo-service "

set-config-value

Description

Set runtime configuration values which will be available to workers running on ECS/Fargate in a secure way. These configuration values can be used for installing private packages from npm (opens in a new tab) and Artifactory (opens in a new tab), and for customizing your npm configuration.

See Private npm packages for more information on using private packages in your tests.

Synopsis

artillery set-config-value [options]

Options

Configuration parameters

You can set values for the following configuration parameters:

Examples

Set the npm authentication token for an Artillery Pro deployment in the US East (N. Virginia) region:

artillery set-config-value \
    --name NPM_TOKEN \
    --value 01234567 \
    --region us-east-1

Set the loglevel setting for the .npmrc configuration to debug for the workers in an Artillery Pro deployment in the Europe (Ireland) region:

artillery set-config-value \
    --name NPMRC \
    --value "loglevel=debug" \
    --region eu-west-1

subscription

artillery subscription [options]

The subscription command shows the status of your team's Artillery Pro subscription. You can also use this command to request an Artillery Pro license key to be sent to an email address associated with the subscription.

Options