Supported formats

The Qualflare CLI (qf) reads 27 frameworks and 4 generic formats. This page lists every --format value that qf list-formats prints in qf 0.4.0, with the file each one reads and an example upload.

How qf picks a format

Without --format, qf picks a parser for each file on its own:

1. It reads the file's contents first: a .json or .jsonl file by its top-level keys, and an .xml file by its <testsuite> or <testsuites> root, which reads as junit, or as python when the file contains pytest.
2. If the contents match no format, it goes by the file name: a name that contains a tool's name, such as playwright or trivy, picks that tool's parser, and any other .xml file reads as junit.
3. A name that points at a parser for another file type is refused (see Errors).

Pass --format for TestNG, Maestro, XCTest, Espresso and PHPUnit XML: these formats have the same <testsuite> root, so an unflagged file is labelled plain junit however it is named. The value is case-insensitive.

qf list-formats prints every value; its aliases are qf formats and qf lf, and -c generic, unit, bdd, e2e, api or security prints one category. To check a file without uploading it, run qf validate <files> [--format …], which needs no saved login: see qf validate.

# Let qf detect the format
qf myapp collect results.json

# Name it, for JUnit-style XML from another tool
qf myapp collect results.xml --format testng

Generic formats

JUnit XML

--format junit reads JUnit XML, which most test tools can write, for example NUnit, MSTest, xUnit.net and Robot Framework. qf detects it from its <testsuite> root. JUnit 5 has a native reporter, com.qualflare:qualflare-junit5.

qf myapp collect build/test-results/test/*.xml --format junit

CTRF

--format ctrf reads a CTRF JSON report. qf detects it from its root or from a file name that contains ctrf, labels the launch with the tool in results.tool.name, and takes a shard index from environment.shardId.

qf myapp collect ctrf-report.json --format ctrf

Allure results

To upload Allure 2 results from allure-pytest, allure-junit5, allure-testng, allure-playwright or another Allure adapter, pass the results directory: qf <identifier> collect allure-results/. qf reads a directory as Allure results when a *-result.json file sits directly inside it, whatever the directory is called. If qf list-formats doesn't list allure, upgrade qf.

Pass the directory, not its files. qf groups the results that share a historyId into one test: the attempt that finished last is the result, the earlier ones are its retries, and a test that failed and then passed is marked flaky. A single result file uploads without its attachments, and its retries are not grouped.

From each *-result.json, qf reads the status (broken and unknown become Error), the name, the parentSuite, suite and subSuite path, labels and tags, severity as the priority, links, nested steps, parameters (without masked values or hidden parameters), the failure message and stack trace, and attachments. Images and videos go through the artifact upload, and so does a zip file whose file name contains trace. Other attachments up to 1 MiB are sent inside the report, and larger ones are left out. qf does not read test descriptions, *-container.json fixtures (a failing fixture still marks its tests broken), executor.json, categories.json, environment.properties, subdirectories or other .json files in the directory.

The launch is labelled with the framework in the results' framework label, as qf list-formats names it (pytest becomes python, junit5 becomes junit), when the results name exactly one framework qf recognises. Otherwise it is labelled allure.

# The whole results directory, from any Allure 2 adapter
qf myapp collect allure-results/

Qualflare JSON

--format qualflare-json reads the JSON that Qualflare's native reporters write, one report per file in a results directory. qf detects it from its framework, metadata and suites keys and labels the launch with the framework that wrote it; from a directory with reports of several runs, it uploads the newest run (Collect a directory).

# The whole results directory: qf detects the format
qf myapp collect ./qualflare-results

# The same, with the format named
qf myapp collect ./qualflare-results --format qualflare-json

Unit testing

pytest

--format python reads the JUnit XML that pytest writes with --junitxml. qf detects it when the XML contains pytest, which pytest's own report does, so auto-detection usually works and --format python makes it explicit; a <property name="shard"> sets the shard index. pytest has a native reporter, qualflare-pytest, which adds retry history, steps and attachments.

pytest --junitxml=pytest-report.xml
qf myapp collect pytest-report.xml --format python

Go

--format golang reads the JSON events of go test -json. Go has a native reporter, qualflare-go, which also catches build failures.

go test -json ./... > go-test.json
qf myapp collect go-test.json --format golang

Jest

--format jest reads Jest's JSON results. Jest has a native reporter, @qualflare/jest.

qf myapp collect jest-results.json --format jest

Vitest

--format vitest reads Vitest's Jest-compatible JSON results, with the same parser as jest. Vitest has a native reporter, @qualflare/vitest.

qf myapp collect vitest-results.json --format vitest

Mocha

--format mocha reads the output of Mocha's JSON reporter, retries included. Mocha has a native reporter, @qualflare/mocha.

qf myapp collect mocha-report.json --format mocha

RSpec

--format rspec reads the output of RSpec's JSON formatter.

qf myapp collect rspec-report.json --format rspec

PHPUnit

--format phpunit reads PHPUnit's XML report. Its root is the JUnit one, so pass --format phpunit: without it the file reads as junit.

qf myapp collect phpunit-report.xml --format phpunit

TestNG

--format testng reads TestNG results written as JUnit XML; pass it, because the file reads as junit without it. TestNG's own testng-results.xml is not JUnit XML, and qf doesn't read it. TestNG has a native reporter, com.qualflare:qualflare-testng.

qf myapp collect "build/test-results/test/*.xml" --format testng

BDD

Cucumber

--format cucumber reads Cucumber's JSON report. Cucumber.js has a native reporter, @qualflare/cucumberjs.

qf myapp collect cucumber-report.json --format cucumber

Karate

--format karate reads Karate's JSON report.

qf myapp collect karate-report.json --format karate

UI, E2E and mobile

Detox

--format detox reads the Jest JSON that Detox's Jest runner writes, with the same parser as jest. Detox has a native reporter, @qualflare/detox, which writes ./qualflare-results instead; both take --artifacts-dir for the files in Detox's artifacts folder (Attach Detox artifacts).

# The @qualflare/detox report directory, plus Detox's artifacts folder
qf myapp collect ./qualflare-results --artifacts-dir ./artifacts

# Detox's own Jest JSON
qf myapp collect jest-results.json --format detox --artifacts-dir ./artifacts

Playwright

--format playwright reads the report of Playwright's JSON reporter (--reporter=json), retries included, and takes each test's shard index from its workerIndex. Playwright has a native reporter, @qualflare/playwright.

qf myapp collect playwright-results.json --format playwright

Cypress

--format cypress reads Cypress results as Mochawesome JSON, retries included. Cypress has a native reporter, @qualflare/cypress, which writes a results directory instead.

qf myapp collect cypress-results.json --format cypress

Selenium

--format selenium reads Selenium/WebDriver JSON results.

qf myapp collect selenium-results.json --format selenium

TestCafe

--format testcafe reads the output of TestCafe's JSON reporter. --no-capture-output leaves out its fixture.* and test.* metadata.

qf myapp collect testcafe-report.json --format testcafe

Maestro

--format maestro reads Maestro's JUnit XML; pass it, because the file reads as junit without it.

Maestro has no reporter or listener API, so qualflare-maestro wraps the command instead — qualflare-maestro -- maestro test .maestro/ runs Maestro, reads what it leaves behind, and returns its exit code unchanged.

qf myapp collect maestro-results.xml --format maestro

XCTest

--format xctest reads XCTest results converted to JUnit XML, for example with xcbeautify --report junit; pass it, because the file reads as junit without it. collect reads a directory as a folder of .json reports, so it refuses an .xcresult bundle: convert the bundle first.

qf myapp collect xctest-results.xml --format xctest

Espresso

--format espresso reads the JUnit XML that Gradle writes for Espresso tests; pass it, because the file reads as junit without it. Espresso has a native reporter, com.qualflare:qualflare-espresso.

The reporter runs inside the instrumented app, registered by naming its listener in testInstrumentationRunnerArguments["listener"] (JUnit 4 has no ServiceLoader hook). It writes through androidx.test storage into the directory Gradle already pulls off the device, so on API 29 and up the upload needs no adb command. Gradle nests the reports under a variant folder and a device folder, and collect reads only the .json files directly inside a directory, so pass the report files with a quoted glob:

qf myapp collect "app/build/outputs/connected_android_test_additional_output/*/connected/*/qualflare-espresso-*.json"

# Gradle's own JUnit XML, without the reporter
qf myapp collect espresso-results.xml --format espresso

Below API 29 Gradle passes no output directory, so the reporter writes into the app's files directory and prints the commands to pull it — including the flag that keeps the APKs installed, since connectedAndroidTest otherwise uninstalls the app and deletes the report with it.

WebdriverIO

--format webdriverio reads the reports of the native reporter @qualflare/webdriverio, with the Qualflare JSON parser. For WebdriverIO's own JUnit XML, pass --format junit.

# The whole directory: one report per worker, merged by run id
# (every worker derives it from the wdio launcher)
qf myapp collect ./qualflare-results

Appium

--format appium reads the reports of the native reporter @qualflare/appium, with the Qualflare JSON parser.

qf myapp collect ./qualflare-results

Flutter

--format flutter reads the JSON results file that flutter test --file-reporter json:<path> writes. flutter test --machine output also works (qf 0.2.0+). For runs on a device, pass --platform android or --platform ios. Statuses, durations, the test's own file and line, print output, widget-test failures with the real assertion message and stack, per-attempt retry history and flaky detection all arrive without adding anything to your project. The optional qualflare_flutter package (flutter pub add --dev qualflare_flutter, read by qf 0.3.0+) adds labels, links, tags, priority, nested steps, and screenshots to the same results file. See the Flutter test reporting page for more.

# Write Flutter's JSON results file, then collect it
flutter test --file-reporter json:flutter-results.json
qf myapp collect flutter-results.json

# Flutter on a device or emulator — Flutter's output never names the device
qf myapp collect flutter-results.json --platform android

API testing

Newman

--format newman reads Newman's JSON report of a Postman collection run.

qf myapp collect newman-report.json --format newman

k6

--format k6 reads the k6 JSON summary.

qf myapp collect k6-summary.json --format k6

Security testing

OWASP ZAP

--format zap reads OWASP ZAP's JSON report; qf reads no other ZAP report format.

qf myapp collect zap-report.json --format zap

Trivy

--format trivy reads Trivy's JSON results.

qf myapp collect trivy-results.json --format trivy

Snyk

--format snyk reads Snyk's JSON test results.

qf myapp collect snyk-test.json --format snyk

SonarQube

--format sonarqube reads a SonarQube issues export in JSON.

qf myapp collect sonarqube-report.json --format sonarqube

Retries and flaky tests

The Playwright, Cypress, Mocha and Allure parsers, and all JUnit-XML parsers (JUnit, TestNG, Maestro, XCTest, Espresso), automatically capture retry counts and flaky test status. No additional flags are required — this data appears in Qualflare alongside your test results.

Errors

collect puts failed to process test results: failed to detect framework for file <file>: before the detection errors. Fixes: unsupported format, or qf reads the wrong format.

Limits

Each input file can be up to 100 MB; see Size limits.

Related

• Guides: Upload results with qf collect · Sharded runs and result directories · Upload screenshots, videos and traces
• Reference: Native reporters · qf command reference · Troubleshoot qf and CI uploads · Glossary: CTRF