CloudWatch

This reporter supports sending metrics and traces to CloudWatch.

In order to enable the Cloudwatch reporter set type to cloudwatch, and region to the AWS region that you want metrics to be sent to:


config:
  plugins:
    publish-metrics:
      - type: cloudwatch
        region: eu-west-1

Important: AWS credentials have to be present in the environment and have sufficient IAM permissions to send metrics to Cloudwatch and/or traces to AWS X-Ray.

Metrics

By default, all Artillery metrics will be sent to CloudWatch. Each Artillery metric will create a custom CloudWatch metric , which incur extra AWS fee for each metric. You can configure a specific list of metrics to send with the includeOnly setting (see Configuration section below).

You can learn more about AWS CloudWatch custom metric pricing in this blog post from Vantage.sh: https://www.vantage.sh/blog/cloudwatch-metrics-pricing-explained-in-plain-english

Configuration options

Example configuration

The following configuration enables CloudWatch reporting. All metrics generated by the test will be sent to us-east-1 region, and tagged with extra dimensions to specify the team running the tests (SQA), and the name of the service under test (checkout-svc).


config:
  plugins:
    publish-metrics:
      - type: cloudwatch
        region: us-east-1
        namespace: continous-testing
        dimensions:
          - name: Team
            value: SQA
          - name: Service
            value: checkout-svc

Name	Description
`region`	CloudWatch region where metrics will be sent.
`namespace`	Metric namespace. Defaults to “artillery”.
`name`	The value of the default `Name` dimension attached to every metric. Defaults to “loadtest”.
`excluded`	A list of metric names which should not be sent to CloudWatch. Defaults to an empty list, i.e. all metrics are sent to CloudWatch.
`includeOnly`	A list of specific metrics to send to CloudWatch. No other metrics will be sent. Defaults to an empty list, i.e. all metrics are sent to CloudWatch.
`dimensions`	A list of extra dimensions to attache to every metric as `name`/`value` pairs. Defaults to an empty list.
`traces` Added inv2.0.9	Set to send traces to AWS X-Ray. See traces specific configuration

Traces

Added inv2.0.9

Powered by: OpenTelemetry reporter

Experimental This feature is under active development. We aim to keep it stable, but it may need to introduce breaking changes. Please open an issue or discussion for any feedback.

With the CloudWatch reporter you can also send traces to AWS X-Ray . Tracing is available for both HTTP and Playwright engines, and the tracing data available is different depending on the engine used.

An additional plugins.publish-metrics.spans.exported metric will be recorded and will appear in your reports when tracing is enabled. It represents the number of spans exported to AWS X-Ray.

Configuration options

CloudWatch requires a running instance of the CloudWatch Agent (or the AWS Distro for OpenTelemetry ) in order to ingest traces. When you run a test on AWS Fargate, Artillery will take care of this for you automatically, but for testing on your local machine you will need to run the agent yourself.
Note: AWS Lambda is not currently supported.

Example configuration


plugins:
  publish-metrics:
    - type: 'cloudwatch'
      traces:
        sampleRate: 0.5
        replaceSpanNameRegex:
          - pattern: get_user_[a-zA-Z0-9]+
            as: get_user_id
        annotations:
          testType: LoadTest
          tool: Artillery

Name	Description
`serviceName`	Name of your service. Defaults to `Artillery-test`
`sampleRate`	Sample rate. Percentage of traces to send represented by a value between 0 and 1. (defaults to 1 - all traces are sent)
`useRequestNames`	If set to `true` the request `name`s provided in test script will be used as span names
`annotations`	Custom annotations to be added to each span in `key: value` pairs.
`replaceSpanNameRegex`	A list of replacement maps that consist of: `pattern` - regex pattern to match against the span names `as` - string to replace the matched pattern in the span names

AWS Fargate - additional setup

Artillery sets up the AWS Distro for OpenTelemetry configured with the AWS X-Ray exporter as a sidecar container.

Important: If you ran Artillery tests on AWS Fargate in the past on the account you are using, you will need to delete the existing artillery-worker-role IAM role and artilleryio-ecs-worker-policy IAM policy from your account prior to using this feature.

Artillery will automatically recreate the role and policy with the correct permissions to send traces to AWS X-Ray.

To run the test run:


artillery run-fargate scenario.yaml

Local runs - additional setup

You’ll need to install and configure a CloudWatch Agent or AWS Distro for OpenTelemetry to ingest OTLP traces:

CloudWatch Agent

You can find the installation instructions for the CloudWatch Agent here , and for the configuration here .
The agent’s IAM role or IAM user must have the AWSXrayWriteOnlyAccess policy to send trace data to AWS X-Ray. For more information, see Create IAM roles and users for use with CloudWatch agent ].
To configure the agent to receive traces from Artillery set up the otlp receiver in the CloudWatch Agent configuration file as follows:


"traces": {
  "traces_collected": {
    "otlp": {}
  }
}

AWS Distro for OpenTelemetry

You can find the installation and configuration instructions here .

Debugging

Set DEBUG=plugin:publish-metrics:cloudwatch when running your tests to print out helpful debugging messages when sending metrics to Datadog.


DEBUG=plugin:publish-metrics:cloudwatch artillery run my-script.yaml

Examples


config:
  plugins:
    publish-metrics:
      - type: cloudwatch
        region: us-east-1
        namespace: continous-testing
        dimensions:
          - name: Team
            value: SQA
        traces:
          serviceName: my-service
          useRequestNames: true
          annotations:
            testType: LoadTest
            tool: Artillery