Artillery v2 is under active development
Artillery v2 is under active development. Expect things to break. We will do our best to document breaking changes on this page.
Please report any issues you run into via Github Issues.
What we're working on
Artillery v2 is an ongoing major upgrade of Artillery, specifically focused on the following areas:
- Performance - we're making Artillery faster and more efficient across the board
- Extensibility and hackability - to make extension APIs even more powerful so that the community can conitnue building great plugins and extensions on top of Artillery
- Removing cruft - Artillery has been around for about six years now, and accumulation of some cruft and less-than-ideal design decisions is inevitable. We will take the opportunity that comes with a major semver upgrade to remove some of that cruft and revisit those design decisions.
async/await, new Node.js APIs such as
worker_threads, and new libraries such as Got.js to modernize both Artillery's internals and its external APIs.
- Community - most important of all, we want to make Artillery even easier to contribute to. More than 80 people have contributed to Artillery over the years, and that number is likely to double or triple in the years to come. We want to help the community by refactoring the test suite, refactoring the code to make it easier to understand, documenting the internals, and providing more details contribution guides and templates.
The overarching goal behind the above is to make Artillery v2 even better for our users, and to provide a great foundation for the next few years of Artillery, and all of the new features we have planned.
Some of the things you can look forward to are: New & improved HTTP, Socket.io and WebSocket engines. More flexibility for expressing load profiles. Improvements in how complex test suites can be organized. Ability to mix and match actions from multiple engines in the same scenario. New extension APIs. New high-precision metrics engine. More engines and plugins out-of-the-box in line with our "batteries-included" philosophy, and more!
Major changes and new features
This section documents major new features and changes that are already available in a
The underlying metrics engine has been rewritten to offer better performance (less memory and CPU usage when tracking large sets of metrics), and more flexbility. All code - built-in engines and plugins, user JS code, and third-party plugins and engines - will now use the same APIs for tracking metrics.
We replaced HDRHistogram with DDSketch for computing high-precision percentiles.
Metrics are now tracked and aggregated at pre-determined fixed window size.
The format of the metric summary object provided to plugins and user JS code has changed. Setting
ARTILLERY_USE_LEGACY_REPORT_FORMAT=1 should make most existing plugins work. If not, please open an issue. We will work with plugin authors to update their code to work with v2.
Artillery will now take advantage of multiple CPUs if available and run in multithreaded mode. This is transparent to the user - there is nothing to configure.
Console report format
Console reports print metric names as provided by engines, plugins or user code.
Metrics are output on the zeroth second boundary, i.e. at:
- 00 seconds (e.g. at 08:17:00)
- 10 seconds past the minute (i.e. at 08:17:10)
- 20 seconds past the minute (i.e. at 08:17:20)
- 30 seconds past the minute, and so on
New underlying HTTP library (Got)
Our goal is to maintain full backwards-compatibility with the official HTTP engine API, and existing test scripts should be expected to work without changes.
However, if your scripts relied on Request.js-specific behavior, for example via code in custom JS hooks, some modifications may be required.
Extended performance metrics
Additional performance metrics are reported when
config.http.extendedMetrics is set to
http.dns- amount of time taken by DNS lookups
http.tcp- amount of time taken to establish TCP connections
http.tls- amount of time taken by TLS handshake
http.total- amount of time for full response to be downloaded (vs TTFB)