Artillery v2 Migration Guide
Upgrading from Artillery v1
Artillery v1 has been deprecated and will receive no further updates other than dependency updates for critical security vulnerabilities. We recommend that you upgrade as soon as possible.
What's new in Artillery v2
Artillery v2 is a major upgrade of Artillery, specifically in the following areas:
- Performance - Artillery v2 is faster and more efficient across the board
- Extensibility and hackability - to make extension APIs even more powerful so that the community can conitnue building great plugins and extensions on top of Artillery
- Removing cruft - Artillery has been around for about six years now, and accumulation of some cruft and less-than-ideal design decisions is inevitable. Artillery v2 removes some of that cruft and revisit those design decisions.
- Modernization - Node.js, Javascript itself and the ecosystem have evolved since Artillery's core code was written a couple of years back. Artillery v2 makes use of features such as
async/await, new Node.js APIs such asworker_threads, and new libraries such as Got.js to modernize both Artillery's internals and its external APIs. - Community - most important of all, we want to make Artillery even easier to contribute to. More than 90 people have contributed to Artillery over the years, and that number is likely to double or triple in the years to come. We want to help the community by refactoring the test suite, refactoring the code to make it easier to understand, documenting the internals, and providing more details contribution guides and templates.
Major changes and new features
This section documents major new features and changes that are already available in a dev release.
Metrics engine
The underlying metrics engine has been rewritten to offer better performance (less memory and CPU usage when tracking large sets of metrics), and more flexbility. All code - built-in engines and plugins, user JS code, and third-party plugins and engines - will now use the same APIs for tracking metrics.
We replaced HDRHistogram with DDSketch for computing high-precision percentiles.
Metrics are now tracked and aggregated at pre-determined fixed window size.
The format of the metric summary object provided to plugins and user JS code has changed. Setting ARTILLERY_USE_LEGACY_REPORT_FORMAT=1 should make most existing plugins work. If not, please open an issue. We will work with plugin authors to update their code to work with v2.
Multithreading
Artillery will now take advantage of multiple CPUs if available and run in multithreaded mode. This is transparent to the user - there is nothing to configure.
Console report format
Console reports print metric names as provided by engines, plugins or user code.
Metrics are output on the zeroth second boundary, i.e. at:
- 00 seconds (e.g. at 08:17:00)
- 10 seconds past the minute (i.e. at 08:17:10)
- 20 seconds past the minute (i.e. at 08:17:20)
- 30 seconds past the minute, and so on
HTTP engine
New underlying HTTP library (Got)
The HTTP engine has been rewritten and now uses Got instead of Request.js under-the-hood.
Our goal is to maintain full backwards-compatibility with the official HTTP engine API, and existing test scripts should be expected to work without changes.
However, if your scripts relied on Request.js-specific behavior, for example via code in custom JS hooks, some modifications may be required.
Extended performance metrics
Additional performance metrics are reported when config.http.extendedMetrics is set to true:
http.dns- amount of time taken by DNS lookupshttp.tcp- amount of time taken to establish TCP connectionshttp.tls- amount of time taken by TLS handshakehttp.total- amount of time for full response to be downloaded (vs TTFB)