Appium reporter

@qualflare/appium is the Qualflare reporter for Appium suites run through WebdriverIO 8 or 9: it records iOS and Android results with the device each test ran on, retries and screenshots in ./qualflare-results, and qf <identifier> collect uploads them as one launch.

Before you begin

• The Qualflare CLI, qf: Install the CLI.
• A saved login: qf login <identifier> with a project access token whose creator still holds the project role that uploads need Editor+ · project (requires the Editor role or higher in the project) (Authenticate with qf login).
• The Identifier of the environment the results belong to (Environments).

Driving Appium from Java or Python? Use your test runner's reporter instead: TestNG, JUnit 5 or pytest.

Install

npm install --save-dev @qualflare/appium

Configure

Add the reporter to wdio.conf.js, next to the Appium service that starts the server:

export const config = {
  // ...
  capabilities: [{
    platformName: 'iOS',
    'appium:automationName': 'XCUITest',
    'appium:deviceName': 'iPhone 17 Pro',
    'appium:app': './build/MyApp.app',
  }],
  services: ['appium'],
  reporters: ['spec', ['@qualflare/appium', { environment: 'staging' }]],
};

The options, the optional service (@qualflare/appium/service) and the limits are those of the WebdriverIO reporter, except that the framework label is always Appium.

Every reporter shares the other settings: see Reporter configuration.

Set environment to the environment's Identifier: a name such as Staging fails the upload with HTTP 400 (Environment).

Run and upload

npx wdio run wdio.conf.js
qf <identifier> collect ./qualflare-results

Run in CI

- name: Upload results to Qualflare
  if: ${{ !cancelled() }}
  env:
    QF_TOKEN: ${{ secrets.QF_TOKEN }}
    QF_BRANCH: ${{ github.head_ref || github.ref_name }}
    QF_COMMIT: ${{ github.event.pull_request.head.sha || github.sha }}
  run: |
    npm install -g @qualflare/cli
    qf login ci --force
    qf ci collect ./qualflare-results

if: ${{ !cancelled() }} uploads the results when tests fail, and QF_BRANCH and QF_COMMIT record a pull request's own branch and commit (Pull request branches). For GitLab CI/CD, Jenkins and Azure Pipelines, see Run the upload when tests fail.

Sharding and retries

• Every spec file runs in its own worker and writes its own report; qf merges the reports that share a run ID, which every worker takes from the wdio run launcher or the CI run. See Merge sharded CI jobs into one launch.
• Mocha this.retries(n) gives one test with every attempt's error and timing; it is flaky when a retry passed.

What gets uploaded

• The platform: ios or android from platformName. Mobile Safari is ios.
• platformName, platformVersion, deviceName and automationName on every test.
• A test ID that includes the platform, so iOS and Android keep separate histories.
• Every driver.takeScreenshot() and saveScreenshot(), attached to its test.

Metadata calls come from @qualflare/appium/runtime:

import { qualflare } from '@qualflare/appium/runtime';

qf doesn't read the branch, commit, language or milestone from the report: set them on the upload (Branch, commit, language and milestone).

Troubleshooting

• The launch shows the category generic: qf is older than 0.1.37.
• environment "Staging" does not exist in this project: use the environment's Identifier, or pass --environment on the upload (Environment).
• Screenshots can't be downloaded: qf is older than 0.1.24. Upgrade qf.

Related

• Guides: Reporters · Upload results with qf collect · Sharded runs and result directories · Upload screenshots, videos and traces
• Reference: Reporter configuration · Appium format
• More: Appium test reporting in Qualflare