Skip to main content

Extension APIs

What you'll learn
  • Different ways of extending and customizing Artillery
  • How to track custom metrics from your tests
  • How to customize VU behavior with custom JS code
  • How Artillery plugins work

Artillery is designed to be extensible and hackable to let users customize it if needed. Artillery is powered by Node.js, allowing you to leverage thousands of high-quality npm modules to do pretty much anything you may need to do.

Extending VU behavior​

Virtual user behavior can be customized via custom functions written in JS and attached to scenarios via extension points also known as "hooks". Different engines provide different extension points. Please refer to documentation of individual engines for details on what hooks they support, and what the APIs look like:

Tracking custom metrics​

You can track custom metrics through JS code. The following metric types are supported:

  1. counters - a counter to keep track of certain events, like the number of times a VU completed a specific transaction.
  2. histograms - a distribution (max, min, median, p95, p99) of some measurements, like API response times or content length.
  3. rates - the average number of times something happened in a time period, like the number of HTTP requests sent.

Custom metrics will get included in Artillery's output alongside standard metrics.

Custom metrics API​

Custom metrics are sent through an EventEmitter object, available as an argument in hook functions and as a constructor parameter for plugins and engines.

  • events.emit('counter', 'my_counter', 1) - Increments the value of a counter called my_counter by 1.
  • events.emit('histogram, 'my_histogram', 42) - Records a measurement of 42 in a histogram called my_histogram.
  • events.emit('rate', 'my_rate') - Record an instance of something happening, tracked as a rate metric named my_rate.
tip

While you can use any string in the custom metrics API, a standard convention is to use snake_case for naming your metrics. When sending metrics to external monitoring systems (such as when using the artillery-publish-metrics plugin), some of these services will automatically convert spaces in metric names to underscores. If your metric name uses spaces, it may make your metrics difficult to search and categorize on these services.

Example​

Let's say we have an HTTP API with a route that creates a new pet object. We want to count how many pets we've created during the load test. Pets are also computationally expensive to create on the server, so we'd like to track the response time for the requests through the Server-Timing response header.

The test script contains the following:

config:
  target: "https://pet-generator.test"
  processor: "./metrics.js"
  phases:
    - arrivalRate: 25
      duration: 300

scenarios:
  - afterResponse: "trackPets"
    flow:
      - post:
          url: "/pets"
          json:
            species: "pony"
            name: "Tiki"

The test script contains one scenario with an afterResponse hook. The hook executes the trackPets function located inside the metrics.js file:

module.exports = { trackPets };

function trackPets(req, res, context, events, done) {
  // After every response, increment the 'Pets created' counter by 1.
  events.emit("counter", "pets_created", 1);

  // Parse the 'server-timing' header and look for the 'pets' metric,
  // and add it to 'Pet creation latency' histogram.
  const latency = parseServerTimingLatency(res.headers["server-timing"], "pets");
  events.emit("histogram", "pet_created_latency", latency);

  return done();
}

function parseServerTimingLatency(header, timingMetricName) {
  const serverTimings = header.split(",");

  for (let timing of serverTimings) {
    const timingDetails = timing.split(";");
    if (timingDetails[0] === timingMetricName) {
      return parseFloat(timingDetails[1].split("=")[1]);
    }
  }
}

The trackPets function collects two metrics after every response returned to a virtual user:

  • A counter called pets_created, incremented by 1.
  • A histogram called pet_created_latency, taken from the server-timing header.

When running the test script, Artillery will collect the custom metrics and display the metrics as part of the test results:

All virtual users finished
Summary report @ 17:04:07(+0900) 2021-05-19
  Scenarios launched:  7500
  Scenarios completed: 7500
  Requests completed:  7500
  Mean response/sec: 24.96
  Response time (msec):
    min: 117
    max: 532
    median: 264
    p95: 489
    p99: 501
  Scenario counts:
    0: 7500 (100%)
  Codes:
    200: 7500
  pet_created_latency:
    min: 100.1
    max: 500.6
    median: 252.1
    p95: 477.2
    p99: 497
  pets_created: 7500

Plugins​

Plugins provide a way to package up some functionality in an easy-to-use interface. Plugins in Artillery have full access to the test script and Artillery's internals, and it's possible to get very creative with what they can do.

Naming conventions​

Artillery plugins are distributed as npm packages and follow the artillery-plugin-$NAME convention, e.g. artillery-plugin-publish-metrics.

Plugin lifecycle​

  1. Plugins are enabled with the config.plugins section of a test script, which makes Artillery try to load the plugin:
config:
  target: "http://svc-foo.prod.acme-corp.internal"
  plugins:
    my-custom-plugin:
      option1: some value
      option2: some other value

This example will make Artillery try to find and load the artillery-plugin-my-custom-plugin package.

  1. The plugin package is expected to export a Plugin class. The constructor receives two arguments:

    1. script - The full test script definition, which your plugin code can modify. For example, a plugin may attach additional hook functions to scenario definitions, like the metrics-by-endpoint plugin.
    2. events - An EventEmitter which receives events from Artillery and can also send custom metrics, like the publish-metrics plugin.

  2. A plugin may define a cleanup function, which takes a done callback. Artillery will call this function before exiting when a test finishes, providing an opportunity for any cleanup that a plugin needs to do. For example, you can use the function to flush buffered metrics to an external monitoring system.

Plugin events​

Plugins can subscribe to the following events:

  1. phaseStarted - A new arrival phase has started.
  2. phaseCompleted - An arrival phase has finished.
  3. stats - Provides metrics for the previous reporting period.
  4. done - All VUs finished running their scenarios.

Example​

The artillery-plugin-hello-world package contains an example plugin to learn about Artillery's plugin interface and can serve as a starting point to create your custom plugins.

Engines​

Engines provide support for different actions inside virtual user scenarios. For example, the built-in HTTP engine provides support for get, post, put, and other HTTP actions.

An engine may package up a way to communicate with a particular type of server in Artillery, like HTTP, Socket.IO, or WebSockets. Such an engines may work on levels 4 or 7 of the OSI model.

Engines may also provide actions for a higher-level protocol level like an SQL engine or a AWS Kinesis engine.

They may also be used to package up application-specific actions to make a particular application easier to test. For example an engine for a chat and collaboration platform like Slack or Discord may provide actions such as registerUser, loginAsUser, joinChannel, postMessage and so on.

Reporters​

Custom reporters to send metrics and reports to external systems can get implemented as a plugin.

Please see the official publish-metrics plugin for a complete example of a custom reporter.

Table of contentsOn this page