Merge sharded CI test runs with qf collect

Upload sharded or parallel CI test runs to Qualflare as one launch with qf collect: result directories, run selection, --allow-mixed-runs and --shard.

To upload a sharded or parallel CI run to Qualflare as one launch, run qf <identifier> collect once over every shard's results, as one directory or a list of files. qf merges them and records which shard ran each test.

Before you begin

A saved identifier: Authenticate with qf login.
A reporter that writes a results directory, such as a native reporter, or one result file per shard.

Collect a directory

Pass a directory instead of files to upload every .json report directly inside it. Subdirectories are not read. Native reporters such as @qualflare/cypress write their reports into a directory like this. A directory with a *-result.json file directly inside it is read as Allure results instead, and its other .json files are not uploaded.

qf myapp collect ./qualflare-results

In a glob that qf expands, ** matches one directory level, like *. A directory with no .json report directly inside it fails with directory "<dir>" contains no .json report files.

If the reports come from different runs, collect uploads only the newest run and prints ignored N file(s) from M earlier run(s). Reports without a run ID are always included. To upload every run as one launch, add --allow-mixed-runs. qf groups the reports by the run ID their reporter wrote (metadata.runId) and keeps the run with the newest report:

ignored 3 file(s) from 1 earlier run(s) (--allow-mixed-runs to include them)
merging 2 runs into one launch (--allow-mixed-runs)

Merge sharded CI jobs into one launch

Save each shard's report

Each shard writes its report to a directory and saves it as a CI artifact.

Gather the reports in one job

A final job downloads every shard's artifact into one directory.

Upload once

That job runs qf login ci --force and qf ci collect <dir> once.

On GitHub Actions, with @qualflare/playwright as the reporter, which writes ./qualflare-results:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3]
    steps:
      - uses: actions/checkout@v4
      # Install your dependencies and browsers here
      - run: npx playwright test --shard=${{ matrix.shard }}/3
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: qualflare-results-${{ matrix.shard }}
          path: qualflare-results

  upload:
    needs: test
    if: ${{ !cancelled() }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/download-artifact@v4
        with:
          pattern: qualflare-results-*
          merge-multiple: true
          path: qualflare-results
      - run: npm install -g @qualflare/cli
      - name: Upload to Qualflare
        env:
          QF_TOKEN: ${{ secrets.QF_TOKEN }}
        run: |
          qf login ci --force
          qf ci collect ./qualflare-results

Other CI systems follow the same pattern: see CI/CD Integration.

Number shards with --shard

If your tests run in parallel across multiple workers, pass --shard and give collect one result file per worker. Each file is treated as one shard, numbered by its position in the argument list, starting at 0:

qf myapp collect worker-0.xml worker-1.xml worker-2.xml --shard

This uploads all three files as a single launch, with each case tagged with its shard index. Launch Detail shows a Shards tab with a timeline view — one lane per shard — so you can see how work was distributed and spot the slowest shard. See Launches: Shards for what that view looks like.

--shard requires at least two files. A single file is a no-op — the CLI prints a warning and uploads it as an unsharded launch. If you pass a glob that expands to one file after deduplication, the same no-op applies.

Glob expansion orders files lexically by filename (t1, t10, t100, t2, ...), not numerically and not by shard identity. If shard index needs to be stable across runs, list files explicitly in the order you want them numbered rather than relying on a glob.

The warning for a single file:

warning: --shard requires 2+ input files to have any effect; 1 file(s) left after glob expansion and de-duplication, shard tagging skipped

--shard overrides any shard index the reports carry.

Shard indexes qf reads on its own

You don't need --shard if your test framework already reports which worker ran each test:

JUnit-family reports (JUnit, TestNG, XCTest, Espresso, Maestro) and pytest: add a <property name="shard" value="N"/> to the report (0-based).
Playwright: the native workerIndex field is read automatically.
CTRF: environment.shardId.
Qualflare JSON, which the native reporters write: shardIndex.

A native signal like Playwright's workerIndex always takes priority over the <property name="shard"> fallback if both are present.

Check the launch

Open the launch: its Shards tab shows one lane per shard. See Launches.

Troubleshoot

ignored N file(s) from M earlier run(s): the directory holds stale reports from an earlier run. Clear it between runs, and add --allow-mixed-runs only when the files belong to this run.
no files match pattern "<glob>": the glob matched no file. Check the path from the directory qf runs in.
directory "<dir>" contains no .json report files: the reports sit in a subdirectory, or the reporter wrote none.
warning: --shard requires 2+ input files …: pass two or more files, or leave out --shard.

Other upload errors: Troubleshoot qf and CI uploads.