Playwright reporter

@qualflare/playwright is the Qualflare reporter for Playwright Test 1.40 or later: it writes each run's results, with nested test.step() trees, per-attempt retries, screenshots, videos and traces, to ./qualflare-results, and qf <identifier> collect uploads the folder as one launch.

Before you begin

• The Qualflare CLI, qf: Install the CLI.
• A saved login: qf login <identifier> with a project access token whose creator still holds the project role that uploads need Editor+ · project (requires the Editor role or higher in the project) (Authenticate with qf login).
• The Identifier of the environment the results belong to (Environments).

Install

npm install --save-dev @qualflare/playwright

Configure

// playwright.config.ts
import { defineConfig } from '@playwright/test';
import { qualflareReporter } from '@qualflare/playwright';

export default defineConfig({
  reporter: [['list'], qualflareReporter({ environment: 'staging' })],
});

The tuple form works too: ['@qualflare/playwright/reporter', { environment: 'staging' }].

A relative outputDir resolves against the folder of your Playwright config, not the shell's working directory.

Every reporter shares the other settings: see Reporter configuration.

Set environment to the environment's Identifier: a name such as Staging fails the upload with HTTP 400 (Environment).

Run and upload

npx playwright test
qf <identifier> collect ./qualflare-results

Screenshots upload by default. Add --upload-artifacts video,trace to upload videos and traces; qf prints how many it skipped otherwise.

Run in CI

- name: Upload results to Qualflare
  if: ${{ !cancelled() }}
  env:
    QF_TOKEN: ${{ secrets.QF_TOKEN }}
    QF_BRANCH: ${{ github.head_ref || github.ref_name }}
    QF_COMMIT: ${{ github.event.pull_request.head.sha || github.sha }}
  run: |
    npm install -g @qualflare/cli
    qf login ci --force
    qf ci collect ./qualflare-results

if: ${{ !cancelled() }} uploads the results when tests fail, and QF_BRANCH and QF_COMMIT record a pull request's own branch and commit (Pull request branches). For GitLab CI/CD, Jenkins and Azure Pipelines, see Run the upload when tests fail.

Sharding and retries

Point every shard at the same folder. Playwright passes its own --shard value to the reporter, which stamps each test with a 0-based shard index, so nothing needs configuring. Playwright's merge-reports mode and blob reporter aren't supported: use the folder. Merge sharded CI jobs into one launch has a GitHub Actions example with this reporter.

Each attempt's status, duration and error is uploaded; steps, labels, tags and attachments come from the final attempt. A test that ran once sends no attempts.

What gets uploaded

• Status, with the history of each retried attempt.
• Nested steps. qualflare.step() runs Playwright's own test.step(), so the steps also show in Playwright's HTML report and trace viewer.
• Screenshots, videos and traces.
• Labels, links, tags, priority and parameters.

A test sets its own metadata with the qualflare calls:

import { test } from '@playwright/test';
import { qualflare } from '@qualflare/playwright';

test('a user can check out @smoke', async ({ page }) => {
  qualflare.label('epic', 'Billing');
  qualflare.link('https://tracker.example/QF-42', { type: 'issue', name: 'QF-42' });
  qualflare.priority('high');

  await qualflare.step('add an item to the cart', async () => {
    qualflare.parameter('sku', 'BOOK-1');
    await page.getByRole('button', { name: 'Add' }).click();
  });
});

qf doesn't read the branch, commit, language or milestone from the report: set them on the upload (Branch, commit, language and milestone).

Troubleshooting

• Traces are missing: pass --upload-artifacts trace to the upload, and use qf 0.1.20 or later.
• Hundreds of steps per test: includeApiSteps is on; turn it off.
• environment "Staging" does not exist in this project: use the environment's Identifier, or pass --environment on the upload (Environment).
• Screenshots can't be downloaded: qf is older than 0.1.24. Upgrade qf.

Related

• Guides: Reporters · Upload results with qf collect · Sharded runs and result directories · Upload screenshots, videos and traces
• Reference: Reporter configuration · Playwright format
• More: Playwright test reporting in Qualflare