Configuring Aurora Serverless v1

What you'll learn

• How to create and configure an Aurora Serverless v1 cluster with DataAPI to use with Artillery Pro
• How to configure Artillery Pro to use an Aurora Serverless cluster
• How to seed the initial database and run data migrations

Overview

Artillery Pro can be configured to use an Aurora Serverless v1 cluster (via Data API) as the backing store.

The Aurora Serverless database will be used to store information such as test run history, and associated test run data such as metrics, logs, notes and annotations and so on. The data can be queried via the CLI or through the web dashboard.

Artillery Pro does not provision a database cluster as part of its deployment. An Aurora Serverless v1 cluster has to be created separately, and its configuration needs to be provided to Artillery Pro. The rest of this document provides a walk-through of creating and configuring an Aurora Serverless v1 cluster for Artillery Pro.

info

Aurora Serverless v1 support is in beta. We do not recommend running production workloads with an Aurora Serverless cluster yet.

Key Facts

• Only Aurora Serverless v1 is supported, as Aurora Serverless v2 does not provide a Data API yet
• Only Postgres is supported as the underlying database engine
• If you're relying on current DynamoDB support note that Artillery Pro will not migrate existing data automatically from Dynamo to Aurora. Please get in touch via support@artillery.io if you need assistance migrating existing data.

Create a new Aurora Serverless v1 cluster

Run the following command to create a new database cluster:

export DB_PASS=`pwgen 32 -1` # generate a random 32-character string; substitute if needed
export REGION=us-east-1 # change this to the region you deployed Artillery Pro into

aws rds create-db-cluster \
  --region "$REGION" \
  --db-cluster-identifier artilleryio-cluster \
  --engine aurora-postgresql --engine-version 10.14 \
  --engine-mode serverless \
  --scaling-configuration MinCapacity=2,MaxCapacity=4,SecondsUntilAutoPause=1000,AutoPause=true \
  --master-username postgres \
  --master-user-password "$DB_PASS" \
  --enable-http-endpoint \
  --output json

This command will return a JSON response. Take note of the following fields, which will be needed for further steps:

1. Endpoint e.g. artilleryio-cluster.cluster-ro-cinrx1234567.us-east-1.rds.amazonaws.com
2. DBClusterArn, e.g. arn:aws:rds:us-east-1:01234567890:cluster:artilleryio-cluster

Create a Secrets Manager secret

Using an Aurora Serverless cluster with Data API requires that an AWS Secrets Manager secret is created which needs to contain access credentials for the cluster.

Create a JSON file with credentials

Create a JSON file in artillery-db-cluster-creds.json containing the following information:

{
  "host": "artilleryio-cluster.cluster-ro-cinrx1234567.us-east-1.rds.amazonaws.com",
  "password": "<password we generated in $DB_PASS>",
  "engine": "postgres",
  "username": "postgres",
  "port": 5432,
  "dbClusterIdentifier": "artilleryio-cluster"
}

Update host and password values with the value of Endpoint field from the previous step, and the value of $DB_PASS respectively.

Create a secret

Now we can use the JSON file to create a Secrets Manager secret:

aws secretsmanager create-secret --secret-string file://artillery-db-cluster-creds.json \
  --region "$REGION" \
  --name "artilleryio/db-credentials" \
  --output json

note

Do not change the name of the secret as Artillery Pro expects it to be artilleryio/db-credentials, and won't have IAM permissions to read any other secrets.

The command will return a JSON response. Take note of the ARN field as we will need it in the next step.

Configure Artillery Pro to use the database cluster

Artillery Pro will look for database configuration in BACKEND_DATABASE_CONFIG config value, and use that if it exists. The value of the BACKEND_DATABASE_CONFIG is expected to be a string containing a JSON object with the following fields:

1. type - set to aws:aurora-v1
2. secretArn - the ARN of Secrets Manager secret we created in the previous step
3. resourceArn - the ARN of the database cluster we created in the first step
4. region - the region in which the database cluster was created
5. database - set to postgres

Run set-config-value command to create database configuration for Artillery Pro:

artillery set-config-value \
    --region "$REGION" \
    --name BACKEND_DATABASE_CONFIG \
    --value '{"type":"aws:aurora-v1","secretArn":"arn:aws:secretsmanager:us-east-1:01234567890:secret:artilleryio/db-credentials-v7LmhK","resourceArn":"arn:aws:rds:us-east-1:01234567890:cluster:artilleryio-cluster","region":"us-east-1","database":"postgres"}'

Run database migrations

The final step is to initialize the new database with a schema.

Run artillery admin:run-db-migrations to initialize the database. The output should look similar to the following:

Checking status...
Running migrations...
migrate01InitialDb1652173578530,migrate02AddTestrunFields1652207384033,migrate03AddNotesField1652212410674,migrate04AddChecksField1653575679368

query: SELECT * FROM current_schema()
query: SELECT * FROM "information_schema"."tables" WHERE "table_schema" = 'public' AND "table_name" = 'migrations'
query: CREATE TABLE "migrations" ("id" SERIAL NOT NULL, "timestamp" bigint NOT NULL, "name" character varying NOT NULL, CONSTRAINT "PK_8c82d7f526340ab734260ea46be" PRIMARY KEY ("id"))
query: SELECT * FROM "migrations" "migrations" ORDER BY "id" DESC
query: CREATE TABLE "test_run" ("id" SERIAL NOT NULL, "testRunId" character varying NOT NULL, "status" character varying, "createdTime" TIMESTAMP, "startedAt" TIMESTAMP, "endedAt" TIMESTAMP, "metadata" jsonb, "launchConfig" jsonb, "tasks" text, "tags" jsonb, CONSTRAINT "PK_011c050f566e9db509a0fadb9b9" PRIMARY KEY ("id"))
query: CREATE TABLE "tag" ("id" SERIAL NOT NULL, "tagString" character varying NOT NULL, "name" character varying NOT NULL, "value" character varying NOT NULL, CONSTRAINT "UQ_89afdbf8c60805ec429a25c5aa5" UNIQUE ("tagString"), CONSTRAINT "PK_8e4052373c579afc1471f526760" PRIMARY KEY ("id"))

tip

If you're testing Aurora Serverless support with a secondary Artillery Pro deployment, remember to set ARTILLERY_BACKEND environment variable accordingly before running migrations.

Next steps

Once Aurora Serverless database cluster is configured, Artillery Pro will use it to store all test related data. No changes are required to how tests are invoked. The upgrade is designed to be completely transparent.