# Load testing at scale > Aws fargate # Distributed Load Testing on AWS Fargate This guide describes how to run high-scale distributed load tests with Artillery on AWS Fargate. You’ll learn: * How to scale out your Artillery tests using built-in AWS Fargate support * What AWS resources Artillery creates on your behalf to run your tests ## AWS credentials To execute tests on AWS Fargate the Artillery CLI makes use of the official [AWS SDK](https://aws.amazon.com/tools/) to create the resources needed to run your tests (see the [AWS Resources](#aws-resources-created) seciton for details on what Artillery creates). The SDK requires [AWS credentials](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html) to be present to work. Please refer to the official AWS documentation if you don't have one set up already: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html Please see the [IAM Permissions](#iam-permissions) section for details on permissions required to run tests from AWS Fargate. ## Running tests from AWS Fargate To run an existing test script from AWS Fargate, use `run-fargate` command instead of the `run` command. For example, if you have a test script saved in `blitz.yml` and you want to run it from `eu-west-1` region, run the following command: ```sh artillery run-fargate \ --region eu-west-1 \ blitz.yml ``` ## AWS resources created Artillery will create a number of AWS resources behind the scenes to be able to execute your tests. All resources created by Artillery are **serverless** and created **on-demand**. There are no long-running infrastructure components involved. * An **S3 bucket** to store test bundles * An **SQS queue** for communication between Fargate containers executing your test and the Artillery CLI. This queue is deleted once the test run completes. * (Optional) An **IAM role** named `artilleryio-ecs-worker-role` for Fargate tasks that execute the test. If a role with that name already exists, Artillery will use it instead of creating it. ### IAM permissions The AWS profile that the Artillery CLI runs under needs to have sufficient permissions to be able to create the resources listed above. If running in a sandbox/developer account, the easiest way is to run with **admin** privileges using the default AWS [**AdministratorAccess**](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_job-functions.html#jf_administrator) managed policy. An example IAM policy definition to use instead of \*\*AdministratorAccess\*\* Notes: Create an IAM Role with the following policy and attach it to the IAM user you're using to run Artillery tests. \`123456789000\` will need to be replaced with the id of the AWS account you'll be using. \`\{ "Version": "2012-10-17", "Statement": \[ \{ "Sid": "CreateOrGetECSRole", "Effect": "Allow", "Action": \["iam:CreateRole", "iam:GetRole", "iam:AttachRolePolicy"], "Resource": "arn:aws:iam::123456789000:role/artilleryio-ecs-worker-role" }, \{ "Sid": "CreateECSPolicy", "Effect": "Allow", "Action": \["iam:CreatePolicy"], "Resource": "arn:aws:iam::123456789000:policy/artilleryio-ecs-worker-policy" }, // Allow Artillery CLI to create AWS service role for ECS when creating a Fargate cluster // https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using-service-linked-roles.html#create-service-linked-role \{ "Effect": "Allow", "Action": \["iam:CreateServiceLinkedRole"], "Resource": \[ "arn:aws:iam::\*:role/aws-service-role/ecs.amazonaws.com/AWSServiceRoleForECS\*" ], "Condition": \{ "StringLike": \{ "iam:AWSServiceName": "ecs.amazonaws.com" } } }, \{ "Effect": "Allow", "Action": \["iam:PassRole"], "Resource": \["arn:aws:iam::123456789000:role/artilleryio-ecs-worker-role"] }, \{ "Sid": "SQSPermissions", "Effect": "Allow", "Action": \["sqs:\*"], "Resource": "arn:aws:sqs:\*:123456789000:artilleryio\*" }, \{ // ListQueues cannot be scoped to individual resources // https://docs.aws.amazon.com/service-authorization/latest/reference/list\_amazonsqs.html#amazonsqs-queue "Sid": "SQSListQueues", "Effect": "Allow", "Action": \["sqs:ListQueues"], "Resource": "\*" }, \{ "Sid": "ECSPermissionsGeneral", "Effect": "Allow", "Action": \[ "ecs:ListClusters", "ecs:CreateCluster", "ecs:RegisterTaskDefinition", "ecs:DeregisterTaskDefinition" ], "Resource": "\*" }, \{ "Sid": "ECSPermissionsScopedToCluster", "Effect": "Allow", "Action": \["ecs:DescribeClusters", "ecs:ListContainerInstances"], "Resource": "arn:aws:ecs:\*:123456789000:cluster/\*" }, \{ "Sid": "ECSPermissionsScopedWithCondition", "Effect": "Allow", "Action": \[ "ecs:SubmitTaskStateChange", "ecs:DescribeTasks", "ecs:ListTasks", "ecs:ListTaskDefinitions", "ecs:DescribeTaskDefinition", "ecs:StartTask", "ecs:StopTask", "ecs:RunTask" ], "Condition": \{ "ArnEquals": \{ "ecs:cluster": "arn:aws:ecs:\*:123456789000:cluster/\*" } }, "Resource": "\*" }, \{ "Sid": "S3Permissions", "Effect": "Allow", "Action": \[ "s3:CreateBucket", "s3:DeleteObject", "s3:GetObject", "s3:GetObjectAcl", "s3:GetObjectTagging", "s3:GetObjectVersion", "s3:PutObject", "s3:PutObjectAcl", "s3:ListBucket", "s3:GetBucketLocation", "s3:GetBucketLogging", "s3:GetBucketPolicy", "s3:GetBucketTagging", "s3:PutBucketPolicy", "s3:PutBucketTagging", "s3:PutMetricsConfiguration", "s3:GetLifecycleConfiguration", "s3:PutLifecycleConfiguration" ], "Resource": \[ "arn:aws:s3:::artilleryio-test-data-\*", "arn:aws:s3:::artilleryio-test-data-\*/\*" ] }, \{ "Sid": "LogsPermissions", "Effect": "Allow", "Action": \["logs:PutRetentionPolicy"], "Resource": \[ "arn:aws:logs:\*:123456789000:log-group:artilleryio-log-group/\*" ] }, \{ "Effect": "Allow", "Action": \["secretsmanager:GetSecretValue"], "Resource": \["arn:aws:secretsmanager:\*:123456789000:secret:artilleryio/\*"] }, \{ "Effect": "Allow", "Action": \[ "ssm:PutParameter", "ssm:GetParameter", "ssm:GetParameters", "ssm:DeleteParameter", "ssm:DescribeParameters", "ssm:GetParametersByPath" ], "Resource": \[ "arn:aws:ssm:us-east-1:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:us-east-2:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:us-west-1:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:us-west-2:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:ca-central-1:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:eu-west-1:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:eu-west-2:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:eu-west-3:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:eu-central-1:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:eu-north-1:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:ap-south-1:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:ap-east-1:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:ap-northeast-1:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:ap-northeast-2:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:ap-southeast-1:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:ap-southeast-2:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:me-south-1:123456789000:parameter/artilleryio/\*", "arn:aws:ssm:sa-east-1:123456789000:parameter/artilleryio/\*" ] }, \{ "Effect": "Allow", "Action": \[ "ec2:DescribeRouteTables", "ec2:DescribeVpcs", "ec2:DescribeSubnets" ], "Resource": \["\*"] } ] }\` #### Setting up the IAM Policy and Role in AWS > **Info:** You do not need to set these up if you're running as **admin** using the default **AdministratorAccess** policy. To help you get started, we've set up a [CloudFormation stack](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/stacks.html) you can use to automatically provision the IAM role with required permissions. The stack will create: * An IAM role named **ArtilleryDistributedTestingFargateRole** * An **ArtilleryDistributedTestingFargatePolicy** (the policy definition outlined above) that will be attached to the role. ##### Create the IAM Permissions using the CloudFormation stack [![Launch Stack](/img/launch-stack.svg)](https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?templateURL=https://artillery-cf-templates.s3.amazonaws.com/aws-iam-fargate-cf-template.yml\&stackName=ArtilleryDistributedTestingFargateIAMStack) 1. Click the **Launch Stack** button above to open the AWS CloudFormation **Quick create stack** page. 2. Adjust the **trust relationship** if needed: * By default, the role will trust your **AWS account**, meaning any [principal](https://docs.aws.amazon.com/IAM/latest/UserGuide/intro-structure.html#intro-structure-principal) from your account (e.g. IAM User, IAM Role) will be trusted to assume the role. If you would prefer to configure the role to trust only a **specific IAM user or role**, enter their name in the appropriate **User**/**Role** parameter field before creating the stack. 3. Click **Create stack**. 4. Once the stack is created, the **ArtilleryDistributedTestingFargateRole** will be ready to use. *Note: Ensure the AWS entity (e.g. IAM user) you are using has permission to assume the role you just created. This is necessary to run Artillery tests.* (Alternative) Create the IAM Permissions manually You should first create the policy. You can do that from the AWS UI or CLI. To do it from the UI: Go to \*\*IAM\*\* -> \*\*Policies\*\* and select \*\*Create Policy\*\* . Select \*\*JSON\*\* . Copy the example JSON policy definition from the section above. You’ll need to change the following: Change the AWS account from \`123456789000\` to your account ID; Remove any comments from the JSON policy, as that is not allowed; Press \*\*Next\*\* and give the policy a meaningful name. After the policy created, you'll need to use that policy in a role: Go to \*\*IAM\*\* -> \*\*Roles\*\* and select \*\*Create Role\*\* ; Select \*\*Custom Trust Policy\*\* : The trust policy of a role is used to define principals that you trust to assume the role (e.g. IAM user, IAM role, service). If you're using a specific IAM user directly when running Artillery tests, you can use the following trust policy: \`\{ "Version": "2012-10-17", "Statement": \[ \{ "Effect": "Allow", "Principal": \{ "AWS": "arn:aws:iam::123456789000:user/iam-user-name" // Replace the value with the ARN of your user }, "Action": "sts:AssumeRole" } ] }\` If you are using a specific IAM role when running Artillery tests set the \*\*AWS\*\* in \*\*Principal\*\* to your \*\*IAM role ARN\*\* instead of the \*\*user ARN\*\* . For more info about how to configure the trust policy, check out this blog post and the AWS documentation . After, on the \*\*Add Permissions\*\* UI, select the policy created in the previous step; Finally, press \*\*Next\*\* and give the role a meaningful name. \*\*That's it!\*\* You should now be able to use this role to run Artillery tests in Lambda. If you're setting up OIDC, for example to interact with GitHub Actions, then you might need to set a different Custom Trust Policy for the role created above (added in step 3). Follow the GH Actions guide to get the appropriate Trust Policy. ## Accessing worker logs in Cloudwatch Artillery automatically outputs the test run metrics to your console while the test runs. However, if you run into unexpected errors in the worker containers where Artillery runs, you may need to access the logs in Cloudwatch to debug this. ### Via Artillery Cloud If your test ran using [Artillery Cloud](../cloud/get-started.mdx), direct links to worker logs will be available inside the test report by clicking `Logs` -> `Worker logs`. ### Via ECS (Fargate) UI This is the simplest way to access the logs if not using Artillery Cloud. However, since Stopped tasks only show in ECS for 1 hour, you'll only be able to see the logs from this UI if the test has run in the last hour. Otherwise, you'll need to use the Cloudwatch UI. 1. Go to AWS ECS (Elastic Container Service) in the region you used (default is **us-east-1**), and select the cluster that was created for the test (**artilleryio-cluster** by default); 2. Select the **Tasks** tab and change **Filter desired status** to **Stopped** if the task has already stopped/failed. Otherwise, leave it as **Running**; 3. Select the task that you want to see the logs for; 4. Click the **Logs** tab and then click the **View in Cloudwatch** dropdown, selecting the first one. The first one should contain logs for the **artillery** container, and the second one logs for the **datadog-agent** container that runs alongside it. 5. That should take you straight to the relevant Cloudwatch log stream for that task. *Note: Logs may not be immediately available when you start the test. The workers need to be running before you can access logs.* ### Via Cloudwatch UI 1. Go to Cloudwatch in AWS, in the region you're using. Then click **Log groups**. 2. Search for the Artillery log group (artilleryio-log-group/artilleryio-cluster) and click it. 3. You'll now be in the **Log Streams** tab, which will be ordered by Last event time. 4. The streams you need will be in the format `artilleryio//artillery/`. If you search by the testID (it's outputted when you ran the test), it should show you 2 streams per worker (so you would see 2 with the default `--count` of 1). 5. You'll need to select the one that has **artillery** in the name (not datadog). *Note: Logs may not be immediately available when you start the test. The workers need to be running before you can access logs.* ## Retention Settings ### **S3** Lifecycle Configuration Objects created in Artillery's s3 bucket in your AWS account have the following lifecycle rules by default: * Test Run artifacts (e.g. custom npm dependencies): 2 days * Test Run metadata: 7 days ### **Cloudwatch** Retention Policy Cloudwatch logs (under the `artilleryio-log-group`) are retained within your AWS account for 180 days. You can configure this value with the `ARTILLERY_LOGGROUP_RETENTION_DAYS` environment variable, making sure to use a [valid number](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutRetentionPolicy.html#API_PutRetentionPolicy_RequestSyntax). ## Supported regions * `us-east-1` * `us-east-2` * `us-west-1` * `us-west-2` * `us-gov-east-1` * `us-gov-west-1` * `ca-central-1` * `eu-west-1` * `eu-west-2` * `eu-west-3` * `eu-central-1` * `eu-north-1` * `ap-south-1` * `ap-east-1` * `ap-northeast-2` * `ap-southeast-1` * `ap-southeast-2` * `ap-northeast-1` * `me-south-1` * `il-central-1` * `sa-east-1` * `cn-north-1` * `cn-northwest-1` ## Troubleshooting ### Error: Timed out waiting for workers to sync You may see this error when running tests with a large number of workers (200+), especially if your test has a large dependency tree. #### Why you see this message By default, the Artillery CLI will wait for up to 10 minutes for all worker nodes to start. If all worker nodes are not ready after 10m, the Artillery CLI will abort the test. Factors that affect worker start up time include: 1. The amount of time it takes for the Artillery worker Docker image to be pulled 2. The amount of time it takes to install all dependencies for your test, such as custom npm modules. This only runs once per test run, but can add a significant amount of time for large dependency trees 3. The amount of time it takes for each worker to sync those dependencies from S3 #### How to fix it You can increase the wait timeout by setting the `WORKER_WAIT_TIMEOUT_SEC` environment variable, e.g. to increase the timeout to 20 minutes: ```sh # Set timeout to 20 minutes (in seconds): export WORKER_WAIT_TIMEOUT_SEC=1200 # Run test as normal: artillery run-fargate my-test.ts ``` ### Unable to assume the service linked role ``` `InvalidParameterException`: Unable to assume the service linked role. Please verify that the ECS service linked role exists ``` You may get the error above if, in some situations, your **AdministratorAccess** role may be misconfigured from AWS. This seems more likely to happen when no ECS cluster has been created before in that AWS account. To resolve this, try to run the ECS wizard to create a Fargate cluster manually from the AWS console (by going to ECS in the console). Doing so seems to do something internally in AWS to resolve the misconfiguration. ### Could not find public subnets in default VPC If you're getting this error from the CLI, then it's likely that the region where you're trying to run the test has no default VPC, or that the default VPC does not have any public subnets. You should use the `--subnet-ids` flag to explicitly specify subnet IDs in an existing non-default VPC for Artillery to use. You may also try [creating a default VPC](https://docs.aws.amazon.com/vpc/latest/userguide/default-vpc.html#create-default-vpc), however it may have been removed by the team in your organization that manages that AWS account for security reasons. ## Questions? * Post your questions on the community forum on GitHub at https://github.com/artilleryio/artillery/discussions * Report a bug or raise an issue via https://github.com/artilleryio/artillery/issues