Set up Artillery CLI
Install Artillery CLI
Artillery is available on npm
, and should be installed globally.
npm install -g artillery@latest
You can also run the Artillery CLI with npx
:
npx artillery@latest
System requirements
Artillery runs on Windows, MacOS, and Linux systems and requires the most recent LTS release of Node.js .
Common issues
Windows PowerShell
By default, PowerShell sets an execution policy on Windows Desktop clients that prevents the execution of scripts. If you use Windows PowerShell, you may see the following error message when attempting to run Artillery:
artillery: File C:\Users\Artillery\AppData\Roaming\npm\artillery.ps1
cannot be loaded because running scripts is disabled on this system.
To allow your Windows system to use Artillery, you’ll need to change the PowerShell execution policy to RemoteSigned
. Open PowerShell as an administrator and use the Set-ExecutionPolicy
to change the execution policy to RemoteSigned
:
PS C:\Windows\system32> Set-ExecutionPolicy RemoteSigned
Execution Policy Change
The execution policy helps protect you from scripts that you do not trust. Changing the execution policy might expose
you to the security risks described in the about_Execution_Policies help topic at
https:/go.microsoft.com/fwlink/?LinkID=135170. Do you want to change the execution policy?
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "N"): Y
Installing as root in Docker
If you’re installing Artillery in a Docker container (or a Docker image, e.g. via RUN npm install -g artillery
), make sure the installation is not run as the root
user.
Installing as a devDependency
in Node.js projects
Generally we do not recommend installing Artillery as a dev dependency inside application codebases (i.e. installing it with npm install -D
). The recommended best practice is to maintain a separate repository/project with performance tests and install Artillery as a dependency in that project.
Canary releases
A canary release is published on every new commit to Artillery’s GitHub repository. You can install the latest canary release with:
npm install -g artillery@canary
Canary releases are made available to users who want to try the latest changes that haven’t been officially released yet, and for users who want help test and report bugs. Be warned, things may break! If they do, please let us know via Github Issues .
Checking your installation
Check if Artillery is installed by running:
npx artillery dino
You can also check the version of Artillery you have installed by running:
artillery version
Set up cloud reporting
To use advanced reporting & visualizations for your load tests, you will need to create an Artillery Cloud account, and enable reporting when you run your tests.
All load tests still run completely within your own environment (i.e. locally, on AWS Lambda, AWS Fargate or Azure Container Instances). The use of the cloud reporting is opt-in per test run. The CLI will send test reports to the cloud dashboard only if the --record
flag is used.
Create an Artillery Cloud account
- Create an Artillery Cloud account by signing in with your GitHub account on https://app.artillery.io
- Grab an API key via Settings → API Keys
Configure the CLI to send data to Artillery Cloud
The use of Artillery Cloud is opt-in. You have to tell the CLI when you want test results to be sent to the cloud dashboard.
To record a test run to the dashboard, use the --record
and --key
flags with the artillery run
command. For example, if you’re currently running load tests with:
artillery run my-service-test.yml
To enable cloud dashboard reporting, add --record
and set the API key with --key
:
artillery run my-service-test.yml --record --key a9_myapikey
When cloud reporting is enabled, the Artillery CLI will print the URL for the test report when the load test begins.
The API key may also be set via the ARTILLERY_CLOUD_API_KEY
environment
variable. When that variable is set, the --key
flag may be omitted.
Best practices
Naming tests
By default, Artillery Cloud will use the name of the test script file as the name of the test.
You can override this by using the --name
flag.
artillery run service-foo.yml --name "service-foo-spike-test" --record --key $MY_ARTILLERY_CLOUD_KEY
The name of the test is used in several places in the dashboard:
- The name shown in the list of test runs
- The name of the group in group view
- When showing other runs of the same test in Comparison View
- As the basis for calculating and showing performance trends across multiple runs
Using tags
Having a consistent strategy for tagging your tests will allow you to filter and search for tests more easily, and to aggregate and analyze test results across multiple runs.
Use the --tags
flag to pass a comma-separated list of tags to your test script, for example:
artillery run my-service-test.yml --tags "env:staging,service:auth,version:1.2.3" --record --key $MY_ARTILLERY_CLOUD_KEY
Here are some recommendations we have seen work well:
env
— different environments where the application is deployed
env
— different environments where the application is deployedFor example, you may use your staging environment to gather baselines under high load, and your production environment to run synthetic tests at lower loads.
Examples: env:staging
, env:prod
, env:qa
service
— name of the application or service
service
— name of the application or serviceExamples: service:auth
, service:billing
version
— version ID of the application or service
version
— version ID of the application or serviceFor example, you could use git rev-parse --short HEAD
to get a short Git SHA to tag your test with.
Examples: version:a231b3e
, version:1.2.3
type
— type of load test
type
— type of load testFor example, you may run nightly tests to gather baselines on the environment, but also run a pre-release soak test in the same environment every two weeks.
Examples: type:spike
, type:baseline
, type:soak
, type:acceptance
ci
— whether a run was triggered from CI or not
ci
— whether a run was triggered from CI or notExamples: ci:true
, ci:jenkins
team
— name of the team running the test
team
— name of the team running the testExamples: team:billing
, team:capybara
actor
— ID of the person or workflow that triggered the test
actor
— ID of the person or workflow that triggered the testFor example, in GitHub Actions, you can use the github.actor
or github.job
variables.
Examples: actor:github-post-deployment-check
, actor:deepak@example.com
Troubleshooting
Error: unable to reach Artillery Cloud API
If you see this error message, it means that the CLI is unable to reach the Artillery Cloud API.
The most common cause of this error is a corporate proxy or firewall blocking POST requests to the Artillery Cloud API (https://app.artillery.io ).
To check if requests are blocked run the following command:
MacOS or Linux
curl -X POST -H "user-agent: Artillery CLI v2.0.23" https://app.artillery.io/api/ping
You should see:
{"message":"pong"}
If you see an error message, such as a network timeout, or a message indicating that the request was blocked, you will need to work with your network administrators to allow requests to the Artillery Cloud API.
In the meantime, you can try running Artillery from a machine on a different network (e.g. from your home network, or from a cloud instance).