Upload Allure results
Upload an Allure 2 results directory to Qualflare with qf 0.4.0 or later: no --format needed, retries grouped into one test, and the fields qf reads.
Last updated
To upload Allure 2 results to Qualflare, run qf <identifier> collect allure-results/ on the directory your Allure adapter writes; you need qf 0.4.0 or later and a saved identifier for the project. qf reads the directory without --format, and groups each test's retries into one result.
Before you begin
- qf 0.4.0 or later:
qf list-formatslistsallureunder Generic. If it doesn't, upgrade qf. - A saved identifier for the project:
qf login <identifier>(Authenticate with qf login). - Tests that write Allure 2 results, through an adapter such as allure-pytest, allure-junit5, allure-testng or allure-playwright.
- A clean results directory for each run. qf reads every
*-result.jsonin it, so results left from an earlier run upload too, as retries of tests that ran again and as tests of their own otherwise.
Upload the results
Run your tests
Run your tests with the Allure adapter on. It writes one -result.json file per test execution, and the attachment files, into its results directory, such as allure-results/.
Check the results directory
Optionally, check the directory before you upload; this needs no saved identifier:
$ qf validate allure-results/
Validating 1 test result file(s)...
OK allure-results/: valid (allure, 5 tests)The count is of tests, with each test's retries counted once.
Upload the directory
Run collect on the directory, with your identifier in place of myapp:
qf myapp collect allure-results/Pass the directory, not its files. A single result file uploads without its attachments, and its retries are not grouped. The upload becomes one launch.
Upload from CI
In a pipeline, run the same command in a step after your tests, with qf login <identifier> --force and the token in QF_TOKEN first, and let the step run when the tests fail. See CI/CD integration.
Check the launch
Open Launches in the project; the newest launch is your upload.
- Suite: on the Test Cases tab, every test lands in one suite, named after the results' top-level suite label when only one appears, or
Allure resultsotherwise. - Retries: a test that ran more than once shows its Retries count, and one that failed and then passed is marked flaky.
- Framework: the launch's
frameworkfield, whichqf <identifier> launch getprints, is the framework your results name in theirframeworklabel, asqf list-formatsnames it, when they name exactly one framework qf recognises; otherwiseallure.
What qf reads
From each -result.json, qf reads the test's status, name, suite labels, steps, parameters, links, failure message and attachments:
brokenandunknownbecome Error.- Hidden parameters are left out; masked ones keep their name but not their value.
- Images, videos and trace zips go through the artifact upload: screenshots by default, videos and traces with
--upload-artifacts video,trace. Other attachments up to 1 MiB travel inside the report. - Not read:
*-container.jsonfixtures,executor.json,categories.jsonandenvironment.properties. - Pass a directory to upload every
.jsonreport directly inside it; a directory with a*-result.jsonfile directly inside it is read as Allure results instead, and its other.jsonfiles are not uploaded.
Every field, one by one: Allure results.
Troubleshoot
failed to detect frameworkwhen you pass the directory: your qf is older than 0.4.0. Upgrade qf.directory "allure-results/" contains no .json report files: no.jsonfile sits directly inside the directory. Check that the tests ran with the adapter on and wrote there.file does not exist: allure-results/: the path is wrong.- The launch has no attachments: you passed
-result.jsonfiles instead of the directory. - Videos or traces are missing: add
--upload-artifacts video,trace(Upload videos and traces). - Other errors: Troubleshoot qf and CI uploads.
Next steps
- Allure results: every field qf reads
- CI/CD integration: upload from your pipeline
- Upload screenshots, videos and traces
- Launches
- Migrate from Allure TestOps: import your test cases as well
Supported formats
Every test result format the Qualflare CLI reads, 27 frameworks and 4 generic formats: the --format value, file type, how qf detects it and an example for each.
Upload screenshots, videos and traces
Attach screenshots, videos and Playwright traces to results you upload with qf collect: what uploads by default, --upload-artifacts, Detox files and limits.