# Reference > Extensions > Expect # `expect` - Expectations & Assertions ## Overview The `expect` plugin adds support for checks and assertions on HTTP requests to enable functional testing in Artillery tests. ### Features * Use expectations and assertions in your HTTP scenarios * Use the same `artillery` command to run functional / acceptance tests on your APIs * See details of any failed assertions (request headers & body, response etc) * Run post-deployment smoke tests from CI/CD pipelines > **Info:** This plugin is only compatible with the `http` engine. > This plugin is not compatible with `before`/`after` hooks. ## Usage ### Enable the plugin To enable the plugin, add it to the `config` section of a test script. ```yaml config: target: 'http://lab.artillery.io' plugins: expect: {} ``` ### Add expectations to HTTP requests ```yaml scenarios: - name: Get a movie flow: - get: url: '/movies' capture: - json: '$[5].title' as: title expect: - statusCode: 200 - contentType: json - equals: - 'From Dusk Till Dawn' - '{{ title }}' ``` [Try it live!](https://play.artillery.io/?s=a16ede88-874d-482d-9175-8657afdd0109\&hR=true) ### Run the test Run your script that uses expectations with: ``` artillery run --quiet my-script.yaml ``` (The `--quiet` option is to stop Artillery from printing its default reports to the console.) ### Run load + functional tests This plugin allows for the same scenario to be re-used for either load testing or functional testing of an API, with the help of `config.environments` definition in your test script. (The only real difference between the two, of course, is how many virtual users you run -- only one for functional tests, and lots of virtual users for a load test.) ```yaml config: target: 'https://my.api.internal' environments: # # This is our load testing profile, where we create a lot of virtual users. # Note that we don't load the plugin here, so that we don't see the output # from the plugin. # load: phases: - duration: 600 arrivalRate: 10 # # This is our functional testing profile, with a single virtual user, and # the plugin enabled. # functional: phases: - duration: 1 arrivalCount: 1 plugins: expect: {} scenarios: # Your scenario definitions go here. ``` You could run the script above in load testing mode with: ```sh artillery run --environment load script.yaml ``` and in functional testing mode with: ```sh artillery run --environment functional script.yaml ``` ## Configuration You can set your plugin configuration in your config. For example: ``` config: target: "http://lab.artillery.io" plugins: expect: reportFailuresAsErrors: true outputFormat: prettyError ``` Here are the configuration options available to you: | Name | Valid Options | Description | | ------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | | \`outputFormat\` \`formatter\` (deprecated) | \`pretty\` (default): prints HTTP method, URL for each request, along with each expectation; \`json\` : provides newline-delimited JSON output; \`prettyError\` : only prints details of failed expectations; \`silent\` : suppresses all output from the plugin. | Choose from a different number of output formats supported by the plugin. | | `reportFailuresAsErrors` | \`false\` (default) \`true\` | Reports failures from expect plugin as errors in the report ("Failed expectation for request ...") | | `useOnlyRequestNames` | \`false\` (default) \`true\` | When used along with `reportFailuresAsErrors` setting, it reports failures by request `name` instead of URL, when `name` is set. | | `expectDefault200` | \`false\` (default) \`true\` | Sets `{ statusCode: 200 }` expectation by default for all requests. | ## Metrics The plugin tracks successful and failed expectations with [custom metrics](/docs/guides/guides/extending#Tracking-custom-metrics). * `expect.ok` - count of all expectations that have been successful * `expect.failed` - count of all expectations that failed * `expect.ok.{type}` - count of successful expectations of `{type}`, for example: `expect.ok.statusCode` for `statusCode` expectations * `expect.failed.{type}` - count of failed expectations of `{type}`, for example: `expect.failed.statusCode` for `statusCode` expectations ## Expectations ### `statusCode` Check that the status code of the response. ```yaml expect: - statusCode: 201 ``` The value may also be a list to indicate that any of the codes in the list is OK: ```yaml expect: - statusCode: - 200 - 301 ``` ### `notStatusCode` This check succeeds if the status code is anything but the list of codes specified. For example, this expectation succeeds when the response status code is anything but `403` or `500`: ```yaml expect: - notStatusCode: - 403 - 500 ``` ### `contentType` Check the value of [`Content-Type`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type) header. ### `hasProperty` and `notHasProperty` When the response is JSON, check that the response object has a property (or does not in case of `notHasProperty`). Same behavior as [`lodash#has`](https://lodash.com/docs/#has). ```yaml expect: - hasProperty: 'data[0].id' ``` ### `equals` Check that two or more values are the same. **NOTE** only primitive values (e.g. booleans, strings and numbers) are currently supported. ```yaml - get: url: '/pets/f037ed9a' capture: - json: '$.species' as: species expect: - equals: - '{{ species }}' - 'dog' ``` ### `hasHeader` Check that the response contains a header. ```yaml - get: url: '/pets/f037ed9a' expect: - hasHeader: 'object-version' ``` ### `headerEquals` Check that the response contains a header and its value matches some string. ```yaml - get: url: '/pets/f037ed9a' expect: - headerEquals: - 'object-version' - 'v3' ``` Multiple headers with the same name can also be matched. For example, if we expect a request to return two `Set-Cookie` headers with some known values we can check that they match like this: ``` - get: url: "/protected/url" expect: - headerEquals: # Expect two Set-Cookie headers returned by this request, # in a particular order: - set-cookie - - platform=astmh;version=1 - status=draft;version=1 ``` ### `matchesRegexp` Check that response body matches a regular expression. The regular expression provided must be a string which is a valid argument to the [RegExp constructor](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp). ```yaml - get: url: '/pets/f037ed9a' expect: - matchesRegexp: .+ # body is not empty ``` ### `cdnHit` This expectation checks for presence of a cache hit/miss header from a CDN. The following CDNs are supported: * Cloudflare * CloudFront * Fastly * Vercel The check will succeed if: * A CDN cache hit/miss header is present in the response (one of `cf-cache-status`, `x-cache`, `x-vercel-cache`) * The value of the header is either `hit` or `stale` ```yaml - get: url: '/pets/f037ed9a' expect: cdnHit: true ``` ### `jmespath` Evaluate JMESPath expressions as an expectation. If the expression evaluates to a truthy value, the expectation will pass. For exaample: ```yaml - get: url: '/foo/1234' expect: jmespath: 'title != null' ``` ## Metrics reported by the plugin In addition to the [default metrics](../reported-metrics#metrics-reported-by-artillery) reported by Artillery and your chosen engine(s), this plugin reports the following additional metrics: | Metric | Type | Description | | ------------------------------------------ | ------------------------------------------ | ------------------------------------------------------------------------------ | | `plugins.expect.ok` | Counter (count) | Total number of passed (ok) expectations. | | `plugins.expect.ok.` | Counter (count) | Total number of passed (ok) expectations for each specific `expectation_type`. | | `plugins.expect.failed` | Counter (count) | Total number of failed expectations. | | `plugins.expect.failed.` | Counter (count) | Total number of passed expectations for each specific `expectation_type`. | ## Debugging If you're having issues with the plugin, you can print out helpful debugging messages using the `DEBUG` environment variable. ### Print expectations and values Set `DEBUG=plugin:expect` when running your tests to view when using an expectation and any values to check. ```sh DEBUG=plugin:expect artillery run my-script.yaml ```