CLI Tool

Collect Test Results - JUnit, Pytest, Jest, Playwright

Collect and upload test results to Qualflare with the qf collect command. Supports 23 frameworks including JUnit, pytest, Jest, Playwright, Cypress, and more.

Collect Command

The qf collect command parses test result files and uploads them to Qualflare. It supports 23 different test frameworks across six categories: Generic (JUnit-compatible), Unit Testing, BDD, UI/E2E/Mobile Testing, API Testing, and Security Testing.

Basic Syntax

qf <identifier> collect [files...] [flags]

<identifier> is a project alias you saved with qf login. See Configuration for setup.

Arguments

ArgumentDescription
files...One or more test result files to collect. Supports glob patterns like *.xml or **/*.json

Examples

# Collect a single JUnit XML file
qf myapp collect results.xml --format junit

# Auto-detect format from file
qf myapp collect playwright-results.json

# Collect multiple files with glob pattern
qf myapp collect test-results/*.xml --format junit

# Specify environment and git metadata
qf myapp collect results.xml --environment production --branch main --commit abc123

# Dry run - parse without uploading
qf myapp collect results.xml --dry-run

# Output parsed results as JSON (use with dry-run)
qf myapp collect results.xml --dry-run --output json

Flags

All flags can be provided as command-line options or set via environment variables (see CI/CD Integration).

FlagShortTypeDefaultDescription
--format-fstring"" (auto-detect)Test framework format. Use qf list-formats to see all supported formats.
--environment-estringdevelopmentEnvironment name (e.g., development, staging, production)
--langstringen-USLanguage/culture code for test result labels. BCP 47 format (e.g., en-US, de-DE, ja-JP).
--platformstringapiPlatform the tests ran on. One of android, ios, desktop, web, api.
--branchstring""Git branch name (e.g., main, develop, feature/login)
--commitstring""Git commit hash for tracking which code version was tested
--milestoneint640 (none)Milestone sequence number to link this launch to.
--shardboolfalseTreat each input file as one shard of a parallel test run. See Sharding Parallel Test Runs below.
--no-capture-outputboolfalseStrip captured stdout/stderr and custom test properties before uploading.
--debugboolfalseLog full HTTP request/response detail to stderr (token redacted).
--timeoutduration30s (effective: 120s, see note)Request timeout. Valid units: s (seconds), m (minutes). Example: --timeout 2m
--dry-runboolfalseParse files without uploading to Qualflare. Useful for validation and debugging.
--output-ostring""Output format for dry-run mode. Currently supports json.

Flag Descriptions

--format / -f

Specifies the test framework format. If not provided, the CLI attempts to auto-detect the format based on file content and extension.

qf myapp collect results.xml --format junit
qf myapp collect results.json --format playwright

--environment / -e

Labels the test run with an environment name. Useful for filtering results by deployment environment. Defaults to development.

qf myapp collect results.xml --environment production
qf myapp collect results.xml --environment dev

--lang

Sets the language/culture code used for test result labels. Accepts any BCP 47 language tag.

qf myapp collect results.xml --lang de-DE
qf myapp collect results.xml --lang ja-JP

--platform

Labels the test run with the platform it ran on. Useful for separating mobile, desktop, and API results in the same project.

qf myapp collect espresso-results.xml --platform android
qf myapp collect xctest-results.xml --platform ios

--branch and --commit

Git metadata for tracking which code version was tested. Enables integration with Git workflows.

qf myapp collect results.xml --branch main --commit a1b2c3d

In CI/CD, these are often auto-populated from common environment variables (see CI/CD Integration):

# GitHub Actions
qf myapp collect results.xml --branch ${GITHUB_REF_NAME} --commit ${GITHUB_SHA}

# GitLab CI
qf myapp collect results.xml --branch ${CI_COMMIT_REF_NAME} --commit ${CI_COMMIT_SHA}

--milestone

Links this launch to a milestone by its sequence number. Useful for tracking release-health across multiple launches tied to the same milestone.

qf myapp collect results.xml --milestone 12

--timeout

Sets the HTTP request timeout for uploading to Qualflare. Default is 120 seconds.

qf myapp collect results.xml --timeout 2m  # 2 minutes

The CLI's HTTP client is configured with this timeout before the command's flags are applied, so in practice the effective request timeout is fixed at 120 seconds regardless of the value passed to --timeout. This is a known CLI behavior, not a sign of misconfiguration on your end.

--dry-run

Parse and validate test files without uploading. Useful for:

  • Validating file format
  • Debugging parsing issues
  • Previewing what will be collected
qf myapp collect results.xml --dry-run
# Output: OK Test results parsed successfully (dry run)

--output / -o

Used with --dry-run to output parsed results as JSON. Useful for debugging and integration.

qf myapp collect results.xml --dry-run --output json

--no-capture-output

Strips captured system-out/system-err output, and any non-structural custom test property (e.g. a JUnit or pytest <property>, or TestCafe fixture.*/test.* metadata), before uploading. Properties a parser generates itself to describe the test (file path, duration, etc.) are still uploaded — only custom, test-authored properties and captured console output are removed.

qf myapp collect results.xml --no-capture-output

Use this when your test output may contain sensitive data (secrets, tokens, PII) that shouldn't leave your CI environment.

--debug

Logs the full HTTP request and response for the upload to stderr, with the auth token redacted. Useful when diagnosing upload failures.

qf myapp collect results.xml --debug

Global Flags

These flags are available on all commands:

FlagShortDescription
--verbose-vEnable detailed debug output
--quiet-qSuppress all non-error output

Exit Codes

CodeMeaning
0Success - test results collected or validated successfully
1Generic error - file not found, parse error, invalid configuration, or invalid arguments/flags
3Auth error - 401, invalid or expired token
4Forbidden - 402/403, quota exceeded or plan/access denied
5Not found - 404
7Transient error - 429 or 5xx from the server; safe to retry

Error Handling

The CLI provides clear error messages for common issues:

File Not Found

$ qf myapp collect missing.xml
Error: file does not exist: missing.xml

Invalid Format

$ qf myapp collect results.xml --format unsupported
Error: unsupported format: unsupported. Use 'qf list-formats' to see supported formats

Parse Error

$ qf myapp collect invalid.xml
Error: failed to parse test results: invalid XML format

Not Logged In

$ qf myapp collect results.xml
Error: no identifier "myapp" configured. Run 'qf login myapp <token>' to add it. (See 'qf projects' for the list.)

Network Error

$ qf myapp collect results.xml
Error: failed to process test results: connection refused

Sharding Parallel Test Runs

If your tests run in parallel across multiple workers, pass --shard and give collect one result file per worker. Each file is treated as one shard, numbered by its position in the argument list, starting at 0:

qf myapp collect worker-0.xml worker-1.xml worker-2.xml --shard

This uploads all three files as a single launch, with each case tagged with its shard index. Launch Detail shows a Shards tab with a timeline view — one lane per shard — so you can see how work was distributed and spot the slowest shard. See Launches: Shards for what that view looks like.

--shard requires at least two files. A single file is a no-op — the CLI prints a warning and uploads it as an unsharded launch. If you pass a glob that expands to one file after deduplication, the same no-op applies.

Glob expansion orders files lexically by filename (t1, t10, t100, t2, ...), not numerically and not by shard identity. If shard index needs to be stable across runs, list files explicitly in the order you want them numbered rather than relying on a glob.

Auto-detected shard index

You don't need --shard if your test framework already reports which worker ran each test:

  • JUnit-family reports (JUnit, TestNG, XCTest, Espresso, Maestro) and pytest: add a <property name="shard" value="N"/> to the report (0-based).
  • Playwright: the native workerIndex field is read automatically.

A native signal like Playwright's workerIndex always takes priority over the <property name="shard"> fallback if both are present.

Supported Frameworks

The qf collect command supports 23 test frameworks across six categories. Use the --format flag to specify your framework, or let the CLI auto-detect it.

JUnit
--format junitXML
pytest
--format pythonXML
Go
--format golangJSON
Jest
--format jestJSON
Mocha
--format mochaJSON
RSpec
--format rspecJSON
PHPUnit
--format phpunitXML
TestNG
--format testngXML
Cucumber
--format cucumberJSON
Karate
--format karateJSON
Playwright
--format playwrightJSON
Cypress
--format cypressJSON
Selenium
--format seleniumJSON
TestCafe
--format testcafeJSON
Maestro
--format maestroXML
XCTest
--format xctestXML
Espresso
--format espressoXML
Newman
--format newmanJSON
k6
--format k6JSON
OWASP ZAP
--format zapJSON
Trivy
--format trivyJSON
Snyk
--format snykJSON
SonarQube
--format sonarqubeJSON

JUnit-compatible Formats

JUnit, TestNG, Maestro, XCTest, and Espresso all emit standard JUnit XML, as do NUnit, MSTest, xUnit.net, and Robot Framework. Pass --format <name> or use a recognizable filename to tell qf which framework generated the file:

qf myapp collect results.xml --format testng
qf myapp collect maestro-results.xml   # filename hint is enough

Retry and Flaky Test Tracking

The Playwright, Cypress, Mocha, and all JUnit-XML parsers (JUnit, TestNG, Maestro, XCTest, Espresso) automatically capture retry counts and flaky test status. No additional flags are required — this data appears in Qualflare alongside your test results.

Generic (JUnit-compatible) Examples

# JUnit (Java, Android, and any JUnit-XML emitter)
qf myapp collect build/test-results/test/*.xml --format junit

Unit Test Examples

# pytest
qf myapp collect pytest-report.xml --format python

# Go tests
qf myapp collect go-test.json --format golang

# Jest
qf myapp collect jest-results.json --format jest

# Mocha
qf myapp collect mocha-report.json --format mocha

# RSpec
qf myapp collect rspec-report.json --format rspec

# PHPUnit
qf myapp collect phpunit-report.xml --format phpunit

# TestNG
qf myapp collect testng-results.xml --format testng

BDD Examples

# Cucumber
qf myapp collect cucumber-report.json --format cucumber

# Karate
qf myapp collect karate-report.json --format karate

UI / E2E / Mobile Examples

# Playwright
qf myapp collect playwright-results.json --format playwright

# Cypress
qf myapp collect cypress-results.json --format cypress

# Selenium
qf myapp collect selenium-results.json --format selenium

# TestCafe
qf myapp collect testcafe-report.json --format testcafe

# Maestro
qf myapp collect maestro-results.xml --format maestro

# XCTest
qf myapp collect xctest-results.xml --format xctest

# Espresso
qf myapp collect espresso-results.xml --format espresso

API Testing Examples

# Newman (Postman)
qf myapp collect newman-report.json --format newman

# k6
qf myapp collect k6-summary.json --format k6

Security Testing Examples

# OWASP ZAP
qf myapp collect zap-report.json --format zap

# Trivy
qf myapp collect trivy-results.json --format trivy

# Snyk
qf myapp collect snyk-test.json --format snyk

# SonarQube
qf myapp collect sonarqube-report.json --format sonarqube

Auto-Detection

If you don't specify --format, the CLI will attempt to auto-detect the format based on:

  • File extension (.xml, .json)
  • File content patterns
  • Common framework-specific markers

For reliable results, explicitly specify the format when dealing with ambiguous files.

# Auto-detection
qf myapp collect results.json

# Explicit format (recommended)
qf myapp collect results.json --format playwright

Usage Examples

CI/CD Integration

Collect test results as part of your CI/CD pipeline (see CI/CD Integration for full platform examples):

# GitHub Actions
- name: Collect test results to Qualflare
  run: |
    qf login ci "$QF_TOKEN" --force
    qf ci collect test-results/*.xml
  env:
    QF_TOKEN: ${{ secrets.QF_TOKEN }}
    QF_BRANCH: ${{ github.ref_name }}
    QF_COMMIT: ${{ github.sha }}

Multiple Test Suites

Collect results from multiple test types:

# Unit tests
qf myapp collect unit-results.xml --format junit --environment testing

# E2E tests
qf myapp collect e2e-results.json --format playwright --environment testing

# Security scan
qf myapp collect security-report.json --format trivy --environment testing

Dry Run Validation

Validate your test results before collecting:

# Validate file can be parsed
qf myapp collect results.xml --format junit --dry-run

# Preview what will be sent
qf myapp collect results.xml --format junit --dry-run --output json

Verbose Debugging

Enable verbose output for debugging:

qf myapp collect results.xml --verbose
# Output includes:
# - Processing 1 test result file(s)...
# - Detected format: junit
# - Parsing file...
# - Uploading to Qualflare...
# - OK Test results collected successfully

See Also