Why Artillery?
What you'll learn
- What Artillery is and why you'd want use it
- Artillery's philosophy
- Key Artillery features
Artillery in a nutshell
Artillery is a modern, powerful & easy-to-use performance testing toolkit. Use it to ship scalable applications that stay performant & resilient under high load.
Artillery prioritizes developer productivity and happiness, and follows the "batteries-included" philosophy.
What do we mean by "performance testing"?
You can use Artillery to run two types of performance tests:
- Tests that put load on a system, i.e. load tests, stress tests, and soak tests
- Tests that verify that a system is working as expected, i.e. continuous functional tests, also known by a number of other names such as: synthetic monitoring, semantic monitoring, production scripted testing, and continuous verification. Think
pingon steroids - automated probes running continuously against services & APIs to test key user journeys.
What kinds of services can I test with Artillery?
Artillery is designed for testing backend systems, such as API services, ecommerce backends, chat systems, game backends, databases, message brokers and queues, and anything else that can be communicated with over a network.
It can be used to test any backend regardless of what protocol it speaks or what language it's written in. Artillery supports HTTP, WebSocket, and Socket.io out of the box, and many additional protocols such as HLS, Kinesis, and Kafka via plugins. Support for additional protocols can be added via Artillery's plugin interface.
Can I test web UIs with Artillery?
Yes, Artillery can be used to run Playwright-based scripts with artillery-engine-playwright. This is an early experimental feature, but it can be used for load testing or synthetic testing against deployed applications (in production or staging).
For E2E testing of web apps in development, we would recommend using Cypress.io.
Who uses Artillery?
Artillery is typically used across teams responsible for delivery, testing, and operating production backend systems: from application developers, to test & QA engineers, and ops/SREs.
Typical usage scenarios
Artillery is used in a large variety of scenarios, which typically include:
- Running ad-hoc load tests against individual APIs or microservices as part of a development process to explore their performance characteristics, and make performance optimizations if needed (e.g. preventing memory leaks or optimizing CPU-heavy code)
- Running tests against staging/feature environments as part of a CI/CD pipeline to catch performance regressions early, and to verify SLOs
- Large-scale load tests before production releases of new services, or to prepare applications and infrastructure for heavy traffic, such as Black Friday/Cyber Monday traffic
- Adding synthetic traffic in production to maintain a margin of safety against traffic spikes
- Running synthetic monitoring against key APIs from multiple geographical locations to verify that key transactions & flows work as expected, and alert if anything breaks
Features
Scenarios
Emulate complex user behavior with scenarios - Define complex user behavior with multiple steps and transactions, loops, conditionals and custom Javascript code. Artillery is designed to support testing the most complex of systems such as e-commerce backends, transactional APIs, IoT backends, chat services, game backends and all kinds of stateful services.
Load testing and smoke testing
Load and smoke testing in one package - Re-use the same scenarios to run either load or functional tests. Use assertions and expectations on the responses, and easily extract and store data from responses to re-use in subsequent requests.
Batteries included
Batteries-included -you can test any system regardless of the protocol it speaks or tech stack it's built upon. HTTP, Socket.io, and WebSockets are supported out of the box, with support for many other protocols available via plugins (gRPC, Kafka, Kinesis, SQL, Lambda). A number of monitoring systems are supported out of the box (Datadog, InfluxDB, Statsd), with third-party plugins providing support for other systems.
Extensible & hackable
Artillery has a number of extension APIs to make it easy to extend in Javascript with custom engines (for additional protocols), plugins (e.g. to generate data for tests or integrate with external systems), or custom reporters (to send metrics and test results to another location).
Because Artillery is built on top of Node.js, you can use any of the thousands of modules available via npm in your plugins to save time.
Integrations and add-ons
Artillery ships with several plugins to make common tasks easier, and there's a number of ready-to-use community plugins available.
Please see Integrations on our website for a full list of integrations and add-ons.
Designed for cross-team collaboration
Artillery's readable YAML-based test scripts encourage collaboration across teams and functions. No knowledge of any specific programming language is needed to read and understand Artillery scripts. We believe that performance is everybody's job and YAML makes it easier for everyone to contribute.
Planet-scale testing
Run your tests at scale - Artillery Pro makes it easy to go from running a test on a developer's machine, to running geographically distributed massively parallel load tests from your team's own AWS account, including a fully-serverless option so that there's no infra to manage.
Ecosystem
Artillery consists of Artillery Core - a free, open source tool, which can be used to write and run load and functional tests, and Artillery Pro - a commercial subscription-based package which adds features for enterprise teams, such as the ability to run large-scale tests from your organization's own AWS account.
Differences between Artillery Core and Artillery Pro
Artillery Pro builds on top of Artillery Core to provide a self-hosted (AWS) performance testing solution. It provides development teams with an internal self-service performance testing platform, with the ability to run large-scale tests with no resource limits, and additional enterprise-grade features, such as the ability to run tests from multiple geographical regions.
Every feature of Artillery Core is available in Artillery Pro, which provides a seamless upgrade path from tests running on a developer's machine, to scaling up & running the same test scripts from your organization's AWS account.
Our philosophy
Our mission is to help development teams ship performant, resilient, and scalable software systems.
Software is eating the world
We believe that software is eating the world. Complex software systems underpin all aspects of our daily lives. There are no industries that don't rely on software anymore - hundreds of software systems were involved in getting your breakfast cereal to your cupboard for example.
At the same time, software systems are only getting more complex, and complexity is the enemy of performance and reliability. As our dependence on software system grows, the importance of shipping performant and reliable software grows along with it.
Performance is a feature
We believe that performance is a feature. Software which is fast and reliable makes for happier users, and happier users make for more revenue.
We believe that performance is everyone's job and requires cross-functional collaboration between product owners, developers, QA engineers, ops & SREs.
Developer productivity above all
Artillery prioritizes developer productivity above everything else. We believe that performance tests should be as easy to write and run as possible. Every product decision in Artillery is viewed through the lens of developer experience and a batteries-included philosophy.