WebdriverIO reporter

@qualflare/webdriverio is the Qualflare reporter for WebdriverIO 8 and 9: each worker writes its results, with per-attempt retries, screenshots and the browser or device it ran on, to ./qualflare-results, and qf <identifier> collect merges them into one launch.

Before you begin

• The Qualflare CLI, qf: Install the CLI.
• A saved login: qf login <identifier> with a project access token whose creator still holds the project role that uploads need Editor+ · project (requires the Editor role or higher in the project) (Authenticate with qf login).
• The Identifier of the environment the results belong to (Environments).

Install

npm install --save-dev @qualflare/webdriverio

Configure

// wdio.conf.js
export const config = {
  // ...
  reporters: ['spec', ['@qualflare/webdriverio', { environment: 'staging' }]],
};

Running Appium through WebdriverIO? Use @qualflare/appium, the same reporter with the Appium label: see Mobile reporters.

Every reporter shares the other settings: see Reporter configuration.

Set environment to the environment's Identifier: a name such as Staging fails the upload with HTTP 400 (Environment).

Run and upload

npx wdio run wdio.conf.js
qf <identifier> collect ./qualflare-results

Run in CI

- name: Upload results to Qualflare
  if: ${{ !cancelled() }}
  env:
    QF_TOKEN: ${{ secrets.QF_TOKEN }}
    QF_BRANCH: ${{ github.head_ref || github.ref_name }}
    QF_COMMIT: ${{ github.event.pull_request.head.sha || github.sha }}
  run: |
    npm install -g @qualflare/cli
    qf login ci --force
    qf ci collect ./qualflare-results

if: ${{ !cancelled() }} uploads the results when tests fail, and QF_BRANCH and QF_COMMIT record a pull request's own branch and commit (Pull request branches). For GitLab CI/CD, Jenkins and Azure Pipelines, see Run the upload when tests fail.

Sharding and retries

• Every worker writes its own report, and qf merges the reports that share a run ID. Workers take it from QUALFLARE_RUN_ID, then the CI run ID, then the wdio run launcher process, so one run lands in one launch.
• Add the optional service, services: ['@qualflare/webdriverio/service'], when you start WebdriverIO more than once in one process, or to delete this package's reports from earlier runs before each run.
• --shard gives each test its shard index. See Merge sharded CI jobs into one launch.
• Mocha this.retries(n) gives one test with every attempt; it is flaky when a retry passed.

What gets uploaded

• Status and retries.
• Screenshots.
• The platform: web, or ios and android for an Appium session.
• Device capabilities.
• A test ID that includes the capability, so Chrome and Firefox keep separate histories.

A test sets its own metadata with the qualflare calls; a masked parameter's value is dropped before the report is written:

import { qualflare } from '@qualflare/webdriverio/runtime';

it('checks out', async () => {
  qualflare.label('owner', 'payments');
  qualflare.link('https://jira.example.com/PAY-12', { type: 'issue' });
  qualflare.parameter('card', '4242…', { masked: true });

  await qualflare.step('fill the form', async () => {
    await $('#card').setValue('4242 4242 4242 4242');
  });
});

qf doesn't read the branch, commit, language or milestone from the report: set them on the upload (Branch, commit, language and milestone).

Troubleshooting

• WebdriverIO's Cucumber runner isn't supported yet.
• specFileRetries reports a re-run spec file as duplicate tests.
• On WebdriverIO 8 with Jasmine, afterTest hooks are told a failed spec passed.
• environment "Staging" does not exist in this project: use the environment's Identifier, or pass --environment on the upload (Environment).
• Screenshots can't be downloaded: qf is older than 0.1.24. Upgrade qf.

Related

• Guides: Reporters · Upload results with qf collect · Sharded runs and result directories · Upload screenshots, videos and traces
• Reference: Reporter configuration · WebdriverIO format
• More: WebdriverIO test reporting in Qualflare