CI/CD integration
Upload test results to Qualflare from GitHub Actions, GitLab CI/CD, Jenkins, Azure Pipelines or any CI with the qf CLI: tokens, variables and branch detection.
Last updated
To upload test results to Qualflare from any CI/CD pipeline, run qf login <identifier> --force with the access token in QF_TOKEN, then qf <identifier> collect <report> after your tests; qf reads the branch and commit that GitHub Actions, GitLab CI/CD, Jenkins and Bitbucket Pipelines set.
After a later CI execution uploads successfully, compare it with an earlier run to review matching cases and choose the next investigation.
Choose your CI provider
Each CI provider has its own page with a complete pipeline, the secret setup and what to set for branch detection.
GitHub Actions
qualflare-action or the qf CLI in a workflow, with a QF_TOKEN repository secret.
GitLab CI/CD
A report job in .gitlab-ci.yml that runs when the test job fails.
Jenkins
A Jenkinsfile stage with a Secret text credential named QF_TOKEN.
Azure Pipelines
An azure-pipelines.yml step with a secret variable, and QF_BRANCH set from the build.
CircleCI
A config.yml step with when: always, and QF_BRANCH set from CIRCLE_BRANCH.
Bitbucket Pipelines
An after-script in bitbucket-pipelines.yml with a secured QF_TOKEN variable.
Other CI systems
Travis CI, the Docker image, npx and self-hosted runners.
On Travis CI or any other CI system, see Other CI systems, Docker and npx: the same two commands, with the values Branch and commit detection lists.
How a CI upload works
Every CI integration follows the same two commands:
qf login <identifier> --force
qf <identifier> collect test-results/*.xmlqf login --forcereads the token from theQF_TOKENenvironment variable and saves it without prompting, which a non-interactive CI job needs.qf <identifier> collectuploads the results.
Store the token as a CI secret named QF_TOKEN and expose it to the job as an environment variable. Don't pass the token as an argument: the process list shows it.
Each provider page wraps this flow in its own YAML or Groovy:
Use ci as the identifier in CI pipelines — it's short and descriptive.
Before you begin
- A project access token with Allow writes off, which you create in Project Settings > Access Tokens Editor+ · project (requires the Editor role or higher in the project). See Access Tokens.
- An environment that exists in the project.
QF_ENVIRONMENTdefaults todevelopment, and an upload that names an environment the project doesn't have fails. See Environments. - A report file your test runner writes, in one of the supported formats.
The manual install steps on the provider pages pin qf 0.7.0. qualflare-action installs the latest qf unless you set its version input.
Run the upload when tests fail
qf <identifier> collect exits 0 when the upload succeeds, whether or not tests failed, so the test step decides the job's result. Run the upload step even when the test step fails:
| Provider | What keeps the upload running |
|---|---|
| GitHub Actions | if: ${{ !cancelled() }} on the upload step |
| GitLab CI/CD | when: always on the upload job, and artifacts: when: always on the test job |
| Jenkins | catchError(buildResult: 'FAILURE', stageResult: 'FAILURE') around the test command |
| Azure Pipelines | condition: always() on the upload step |
| CircleCI | when: always on the upload step |
| Bitbucket Pipelines | the upload in the step's after-script |
| Travis CI | the upload in after_script |
Don't let the test step itself pass on failure: GitHub's continue-on-error: true turns the job and the pull request check green. A failed upload exits with 1, or with 3, 4, 5 or 7 for an API error: see exit codes.
Environment variables
In CI, set qf's options as QF_ environment variables on the job or step; a command-line flag still wins over them.
Configuration priority
Values are applied in the following order (highest to lowest priority):
- CLI flags (
--environment,--branch, etc.) - Environment variables (
QF_*) - The report file: the environment and platform a native reporter wrote into its report
- Auto-detection: CI platform variables, then the local git checkout (branch and commit only)
- Built-in defaults
QF environment variables
| Variable | Purpose | Default | Example |
|---|---|---|---|
QF_TOKEN | Access token, read only by qf login when no token argument is given | (none) | Your CI secret |
QF_ENVIRONMENT | Identifier of an environment that exists in the project | development | production, staging, qa |
QF_BRANCH | Git branch name | Auto-detected | main, feature/login |
QF_COMMIT | Git commit SHA | Auto-detected | a1b2c3d4e5f6... |
QF_MILESTONE | Milestone sequence number to link the launch to. Values below 1 are ignored, and the milestone must exist in the project. | (none) | 3 |
QF_RETRY_MAX | Maximum number of retry attempts | 3 | 5 |
QF_RETRY_DELAY | Base delay between retries (Go duration format) | 1s | 2s, 500ms |
QF_RETRY_MAX_DELAY | Maximum delay between retries (Go duration format) | 30s | 60s |
QF_TIMEOUT | Timeout for each HTTP request attempt (Go duration format). The overall collect deadline is the --timeout flag (default 30s). | 120s | 60s, 2m |
QF_VERBOSE | Enable verbose output | false | true, 1 |
QF_QUIET | Suppress non-error output | false | true, 1 |
See Configuration and environment variables for the full reference.
Duration format
Duration-related variables (QF_RETRY_DELAY, QF_RETRY_MAX_DELAY, QF_TIMEOUT) use Go duration format:
| Unit | Suffix | Examples |
|---|---|---|
| Seconds | s | 30s, 90s |
| Milliseconds | ms | 500ms, 1500ms |
| Minutes | m | 1m, 5m |
Branch and commit detection
qf records each launch's branch and commit from the first of these variables that has a value:
| Value | Read from, in order | Then |
|---|---|---|
| Branch | QF_BRANCH, GIT_BRANCH, GITHUB_REF_NAME, CI_COMMIT_REF_NAME, BITBUCKET_BRANCH | The local git checkout |
| Commit | QF_COMMIT, GIT_COMMIT, GITHUB_SHA, CI_COMMIT_SHA, BITBUCKET_COMMIT | The local git checkout |
If none of these is set, collect reads the branch and commit from the local git checkout. A detached HEAD, which many CI checkouts use, gives no branch, so set QF_BRANCH or pass --branch there.
| CI system | Branch | Commit | What to set |
|---|---|---|---|
| GitHub Actions | GITHUB_REF_NAME | GITHUB_SHA | On pull_request runs, see Pull request branches |
| GitLab CI/CD | CI_COMMIT_REF_NAME | CI_COMMIT_SHA | Nothing |
| Jenkins (Git plugin) | GIT_BRANCH | GIT_COMMIT | QF_BRANCH when GIT_BRANCH holds a remote prefix such as origin/main |
| Bitbucket Pipelines | BITBUCKET_BRANCH | BITBUCKET_COMMIT | Nothing |
| Azure Pipelines | None: no Azure variable is read, and the checkout is a detached HEAD | The local checkout | QF_BRANCH from Build.SourceBranch |
| CircleCI | The local checkout (no CircleCI variable is read) | The local checkout | QF_BRANCH and QF_COMMIT from CIRCLE_BRANCH and CIRCLE_SHA1 |
| Travis CI | The local checkout (a detached HEAD: no branch) | The local checkout | QF_BRANCH and QF_COMMIT from TRAVIS_* |
| Any other | The local checkout | The local checkout | QF_BRANCH on a detached HEAD |
Pull request branches on GitHub Actions
On pull_request events, GITHUB_REF_NAME is <number>/merge and GITHUB_SHA is the merge commit GitHub creates, so the launch would record that branch and commit instead of the pull request's own. Set both from the pull request:
env:
QF_BRANCH: ${{ github.head_ref || github.ref_name }}
QF_COMMIT: ${{ github.event.pull_request.head.sha || github.sha }}On other events head_ref is empty, so both fall back to the pushed branch and commit. Where this goes in a workflow: GitHub Actions.
Security best practices
Token management
- Never hardcode tokens in your repository
- Always use your CI/CD platform's secret/variable management
- Use one token per pipeline. Each token belongs to one project, so you can revoke a leaked token without breaking other pipelines.
- Leave Allow writes off for CI tokens. Uploading results does not need it.
- Tokens expire after at most 90 days. Rotate the CI token before it stops working (Access Tokens).
Check for leaked tokens
Add a pre-commit hook to catch accidentally committed tokens:
#!/bin/bash
# .git/hooks/pre-commit
# Qualflare access tokens start with qfp_ followed by 49 more characters
if git diff --cached --name-only | xargs grep -lE 'qfp_[A-Za-z0-9_-]{49}' 2>/dev/null; then
echo "ERROR: Possible Qualflare access token in staged files!"
echo "Remove the token before committing."
exit 1
fiTokens created before the qfp_ prefix existed have no prefix, so this check cannot catch them.
Troubleshooting
Login fails in CI
Ensure the token value is correct and not empty:
- run: |
[ -n "$QF_TOKEN" ] || { echo "QF_TOKEN is not set"; exit 1; }
qf login ci --forceWith no token argument and an empty QF_TOKEN, qf login reads standard input instead, and an empty token fails with Error: token cannot be empty. The guard line stops the job first, with a clear message.
Upload fails with timeout
Raise the overall upload deadline with --timeout (default 30s):
qf ci collect results.xml --timeout 2mQF_TIMEOUT (default 120s) only caps each single HTTP request. Raise it as well only if one request takes longer than two minutes.
Retry failures
Adjust retry settings for unreliable networks:
env:
QF_RETRY_MAX: 5
QF_RETRY_DELAY: 2s
QF_RETRY_MAX_DELAY: 60sRetries run inside the --timeout deadline (default 30s), so raise it with them, for example qf ci collect results.xml --timeout 5m.
Branch or commit not detected
Manually override if auto-detection fails. The Branch and commit detection table names the variable to set on each CI, such as QF_BRANCH on Azure Pipelines or any detached HEAD.
Identifier already exists on a reused runner
On a self-hosted runner that keeps its home directory between jobs, a second qf login of the same identifier without --force fails with:
Error: identifier "ci" already exists; cannot prompt because stdin is not a terminal — use --force to overwriteqf login <identifier> --force overwrites the saved token, as every qf login command in these pages' examples does. qualflare-action does the same from v1.0.3, which @v1 runs; a workflow pinned to an earlier tag fails here (Version pinning).
For every exit code and API error, see Troubleshoot qf and CI uploads.
Advanced usage
Conditional upload
Only upload on specific branches:
- name: Upload to Qualflare
if: github.ref == 'refs/heads/main'
env:
QF_TOKEN: ${{ secrets.QF_TOKEN }}
run: |
qf login ci --force
qf ci collect results.xmlMultiple test files
Upload multiple test result files:
# All XML files
qf ci collect test-results/*.xml
# Specific files
qf ci collect unit-tests.xml integration-tests.xmlOne collect call is one launch, and the files in it merge into that launch. For parallel jobs, see Sharded runs.
Dry run in CI
Test your CI configuration without actually uploading:
- name: Validate test results (dry run)
env:
QF_TOKEN: ${{ secrets.QF_TOKEN }}
run: |
qf login ci --force
qf ci collect results.xml --dry-runNote: --dry-run does not upload to Qualflare, so the token does not need to be valid — but the identifier must still be registered locally with qf login.
qf validate checks report files with no token at all: see the qf command reference.
Related
- Guides: GitHub Actions · GitLab CI/CD · Jenkins · Azure Pipelines · CircleCI · Bitbucket Pipelines · Other CI systems, Docker and npx · Setting up CI for an open-source project · Rotate the CI access token
- Reference: Upload results with qf collect · Configuration · qf command reference
- CLI/API: Access Tokens
Troubleshoot qf and CI uploads
Fix Qualflare CLI and CI upload errors: what each qf exit code means, the exact messages for tokens, environments, formats and timeouts, and how to fix them.
GitHub Actions
Upload test results from GitHub Actions to Qualflare with qualflare-action or the qf CLI: workflow YAML, the QF_TOKEN secret, inputs and pull request branches.