Distributed tests on AWS Lambda
What you'll learn
- How to scale out your Artillery tests using built-in AWS Lambda support
- What AWS resources Artillery creates on your behalf to run your tests
- Current limitations in AWS Lambda support in Artillery
This guide describes Artillery's support for running highly-distributed serverless load tests on AWS Lambda.
AWS Lambda support is available in Artillery v2.0.0-21 or later.
You can the version of Artillery installed on your system with:
Folllow the Artillery installation guide for details on how to install the latest version of Artillery.
To execute tests in AWS Lambda the Artillery CLI makes use of the official AWS SDK to create the resources needed to run your tests (see the AWS Resources seciton for details on what Artillery creates).
The SDK requires AWS credentials to be present to work. Please refer to the official AWS documentation if you don't have one set up already: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html
Please see the IAM Permissions section for details on permissions required to run tests from AWS Lambda.
Running tests from AWS Lambda
To run an existing test script from AWS Lambda, add
--platform-opt flags to the
For example, if you have a test script saved in
blitz.yml and you want to run it from
eu-west-1 region, run the following command:
artillery run \
--platform aws:lambda \
--platform-opt region=eu-west-1 \
AWS resources created
Artillery will create a number of AWS resources behind the scenes to be able to execute your tests. All resources created by Artillery are serverless and created on-demand. There are no long-running infrastructure components involved.
- An S3 bucket to store an AWS Lambda deployment package generated from your test script
- An AWS Lambda function. This function is deleted once the test run completes.
- An SQS queue for communication between Lambda workers executing your test and the Artillery CLI. This queue is deleted once the test run completes.
- (Optional) An IAM role named
artilleryio-default-lambda-rolefor Lambda functions running your test. This role uses the official AWS-managed
AWSLambdaVPCAccessExecutionRolepolicies, and an additional policy which grants access to the SQS queues created for these tests. If a role with that name already exists, Artillery will use it instead of creating it.
The AWS profile that the Artillery CLI runs under when you run
artillery run needs to have sufficient permissions to be able to create the resources listed above.
An example IAM policy definition is below. The AWS IAM user you're using will need permissions to assume a role which makes use of the policy created with this template.
123456789000 will need to be replaced with the id of the AWS account you'll be using.
// ListQueues does cannot be scoped to individual resources
AWS Lambda support is in preview. There are some limitations to what's possible, and you may run into bugs. Please report any issues via GitHub issues on https://github.com/artilleryio/artillery/issues
Each AWS Lambda is limited to 15 minutes of running time, which means that the entire load test cannot run for longer than 15 minutes at the moment.
A running test cannot be stopped
Once an AWS Lambda starts running, there is no way to stop it. Neither the AWS SDK, nor the AWS Console provide that ability.
This means that once a load test starts, it will run to completion. Be mindful of this, and ramp up load on your applications gradually.
Unavailable Artillery features
The following Artillery features are not supported yet:
- Only built-in engines (HTTP, WebSocket and Socket.io) may be used
- Only built-in
metrics-by-endpointplugins may be used
- Custom JS code provided with
config.processorcannot yet use any third-party npm packages
- Using any third-party plugins or engines
--insecureflags for the
afterhooks run once in each Lambda worker rather than once per test run
- Post your questions on the community forum on GitHub at https://github.com/artilleryio/artillery/discussions
- Report a bug or raise an issue via https://github.com/artilleryio/artillery/issues