Flutter reporter

Qualflare reads Flutter's own JSON test results with qf <identifier> collect from qf 0.2.0, and the qualflare_flutter package, read by qf 0.3.0 or later, adds labels, links, tags, priority, named steps and screenshots to the same file.

Before you begin

• The Qualflare CLI, qf: Install the CLI.
• A saved login: qf login <identifier> with a project access token whose creator still holds the project role that uploads need Editor+ · project (requires the Editor role or higher in the project) (Authenticate with qf login).
• The Identifier of the environment the results belong to (Environments).

Install

flutter pub add --dev qualflare_flutter

The results upload without it; add it for the metadata below.

Add labels, steps and screenshots

The package has no options. Set the environment on the upload, with --environment or QF_ENVIRONMENT; without either, the launch goes to development. Use the environment's Identifier (Environment).

testWidgets('pays with a card', (tester) async {
  qualflare.label('owner', 'mobile-team');            // any name/value
  qualflare.link('https://tracker/QF-1', type: 'issue', name: 'QF-1'); // issue | tms | custom
  qualflare.tags(['checkout', 'smoke']);
  qualflare.priority('high');                          // low | medium | high | critical
  await qualflare.step('fill in the card', () async {
    // … steps nest, and a failing step fails the test as usual
  });
  qualflare.attachment('response.json', bytes, mimeType: 'application/json');
  await qualflare.screenshot(tester, 'checkout');      // checkout.png on this test
});

The calls match the JavaScript reporters' qualflare.* API. Calls in setUp and tearDown apply to each test; outside a test, and in setUpAll and tearDownAll, they do nothing.

Run and upload

# Widget tests
flutter test --file-reporter json:flutter-results.json
qf <identifier> collect flutter-results.json

# integration_test on a device or emulator
flutter test integration_test -d <device> --file-reporter json:flutter-results.json
qf <identifier> collect flutter-results.json --platform android

Flutter's output never names the device, so pass --platform android or --platform ios for device runs.

Run in CI

- name: Upload results to Qualflare
  if: ${{ !cancelled() }}
  env:
    QF_TOKEN: ${{ secrets.QF_TOKEN }}
    QF_BRANCH: ${{ github.head_ref || github.ref_name }}
    QF_COMMIT: ${{ github.event.pull_request.head.sha || github.sha }}
  run: |
    npm install -g @qualflare/cli
    qf login ci --force
    qf ci collect flutter-results.json

if: ${{ !cancelled() }} uploads the results when tests fail, and QF_BRANCH and QF_COMMIT record a pull request's own branch and commit (Pull request branches). For GitLab CI/CD, Jenkins and Azure Pipelines, see Run the upload when tests fail.

Sharding and retries

qf reads per-attempt retry history from Flutter's output and marks flaky tests. To upload several results files as one launch, pass them to one qf <identifier> collect call: see Number shards with --shard.

What gets uploaded

Without the package:

• Statuses and durations.
• Each test's file and line, and its print output.
• Widget-test failures with the assertion message and stack.
• Retry history and flaky detection.

With it, also:

• Labels, links (issue, tms or custom), tags and priority (low to critical).
• Nested steps.
• Attachments and screenshots.

qualflare.screenshot(tester, name) attaches a PNG of the screen; qualflareTestWidgets attaches failure.png when the body throws; in an integration_test run on a device, native: true uses the platform's capture. Host widget tests draw text as boxes unless you call qualflare.loadFonts() from setUpAll.

Attachments are capped at 5 MiB each and 20 MiB per test. qf uploads PNG, JPEG and GIF files separately; any other attachment travels inside the results file, and qf drops one over 1 MiB, and further ones once a file's inline attachments pass 8 MiB.

Troubleshooting

• flutter test --platform chrome isn't supported: the package uses dart:io.
• Marker and screenshot lines also appear in flutter test's console and CI log.
• native: true warns if your test already calls convertFlutterSurfaceToImage().
• screenshot inside your own tester.runAsync warns: call it outside.
• environment "Staging" does not exist in this project: use the environment's Identifier, or pass --environment on the upload (Environment).

Related

• Guides: Reporters · Upload results with qf collect · Sharded runs and result directories · Upload screenshots, videos and traces
• Reference: Reporter configuration · Flutter format
• More: qualflare_flutter on pub.dev · Flutter test reporting in Qualflare