Distributed load testing at scale with Artillery

Artillery comes with built-in support for distributed load testing (unlike many other load testing tools). Even better: it's cloud-native and serverless, which means there's zero infrastructure to set up or manage, and no DevOps knowledge required in order to run distributed high-scale load tests. Artillery runs from your team's own AWS account to make continuous load testing cost efficient, and to enable you to test internal services as easily as public ones.

Features

  • Supported Runtime Platforms:
    • Run tests on AWS Lambda or AWS Fargate
    • We are planning to add support for Azure, Google Cloud and Kubernetes in the future.
  • Serverless: there is no infrastructure to set up or manage.
  • Automatic provisioning: Artillery creates and removes necessary AWS resources on the fly
  • Automatic bundling: Artillery bundles all dependencies needed to run a load test automatically (e.g. plugins and extensions, CSV data files, custom JS code and its dependencies and so on)
  • Metric aggregation: metrics are aggregated and merged from multiple load generator nodes automatically and in a statistically-sound way

How it works

When you run a load test with AWS Fargate or AWS Lambda, Artillery tries to make it as seamless as possible.

Bundle load test

Artillery analyzes the test script and packages up all of its dependencies into a bundle to make it available to Fargate or Lambda workers (containers) executing the test. Those dependencies include:

  • The test script itself, which may be split across more than one YAML file
  • CSV data files used by the test
  • Third-party plugins and engines
  • Custom JavaScript code and all of its dependencies

Create AWS resources

If required, Artillery creates an IAM role for Fargate and Lambda workers. Any resources like ECS Fargate clusters, S3 buckets, SQS queues, and Lambda functions are created on the fly.

Create load generators

Artillery spins up a number of Fargate/Lambda workers (specified with the --count flag by the user), distributes the test bundle to all of them, and waits for all of them to sync up and become available.

Aggregate results from load generators

Once load generators are running, Artillery aggregates reports from individual containers and aggregates them in a statistically correct way to present to the user.

Clean up AWS resources

When the test finishes, Artillery removes AWS resources that are no longer needed.

Custom Extension APIsAWS Lambda