Skip to content
The docs are now organized by product area. Old links still work. Report a broken link

CI/CD integration

Upload test results to Qualflare from GitHub Actions, GitLab CI/CD, Jenkins, Azure Pipelines or any CI with the qf CLI: tokens, variables and branch detection.

Last updated

To upload test results to Qualflare from any CI/CD pipeline, run qf login <identifier> --force with the access token in QF_TOKEN, then qf <identifier> collect <report> after your tests; qf reads the branch and commit that GitHub Actions, GitLab CI/CD, Jenkins and Bitbucket Pipelines set.

After a later CI execution uploads successfully, compare it with an earlier run to review matching cases and choose the next investigation.

Choose your CI provider

Each CI provider has its own page with a complete pipeline, the secret setup and what to set for branch detection.

On Travis CI or any other CI system, see Other CI systems, Docker and npx: the same two commands, with the values Branch and commit detection lists.

How a CI upload works

Every CI integration follows the same two commands:

qf login <identifier> --force
qf <identifier> collect test-results/*.xml
  1. qf login --force reads the token from the QF_TOKEN environment variable and saves it without prompting, which a non-interactive CI job needs.
  2. qf <identifier> collect uploads the results.

Store the token as a CI secret named QF_TOKEN and expose it to the job as an environment variable. Don't pass the token as an argument: the process list shows it.

Each provider page wraps this flow in its own YAML or Groovy:

Use ci as the identifier in CI pipelines — it's short and descriptive.

Before you begin

  • A project access token with Allow writes off, which you create in Project Settings > Access Tokens Editor+ · project (requires the Editor role or higher in the project). See Access Tokens.
  • An environment that exists in the project. QF_ENVIRONMENT defaults to development, and an upload that names an environment the project doesn't have fails. See Environments.
  • A report file your test runner writes, in one of the supported formats.

The manual install steps on the provider pages pin qf 0.7.0. qualflare-action installs the latest qf unless you set its version input.

Run the upload when tests fail

qf <identifier> collect exits 0 when the upload succeeds, whether or not tests failed, so the test step decides the job's result. Run the upload step even when the test step fails:

ProviderWhat keeps the upload running
GitHub Actionsif: ${{ !cancelled() }} on the upload step
GitLab CI/CDwhen: always on the upload job, and artifacts: when: always on the test job
JenkinscatchError(buildResult: 'FAILURE', stageResult: 'FAILURE') around the test command
Azure Pipelinescondition: always() on the upload step
CircleCIwhen: always on the upload step
Bitbucket Pipelinesthe upload in the step's after-script
Travis CIthe upload in after_script

Don't let the test step itself pass on failure: GitHub's continue-on-error: true turns the job and the pull request check green. A failed upload exits with 1, or with 3, 4, 5 or 7 for an API error: see exit codes.

Environment variables

In CI, set qf's options as QF_ environment variables on the job or step; a command-line flag still wins over them.

Configuration priority

Values are applied in the following order (highest to lowest priority):

  1. CLI flags (--environment, --branch, etc.)
  2. Environment variables (QF_*)
  3. The report file: the environment and platform a native reporter wrote into its report
  4. Auto-detection: CI platform variables, then the local git checkout (branch and commit only)
  5. Built-in defaults

QF environment variables

VariablePurposeDefaultExample
QF_TOKENAccess token, read only by qf login when no token argument is given(none)Your CI secret
QF_ENVIRONMENTIdentifier of an environment that exists in the projectdevelopmentproduction, staging, qa
QF_BRANCHGit branch nameAuto-detectedmain, feature/login
QF_COMMITGit commit SHAAuto-detecteda1b2c3d4e5f6...
QF_MILESTONEMilestone sequence number to link the launch to. Values below 1 are ignored, and the milestone must exist in the project.(none)3
QF_RETRY_MAXMaximum number of retry attempts35
QF_RETRY_DELAYBase delay between retries (Go duration format)1s2s, 500ms
QF_RETRY_MAX_DELAYMaximum delay between retries (Go duration format)30s60s
QF_TIMEOUTTimeout for each HTTP request attempt (Go duration format). The overall collect deadline is the --timeout flag (default 30s).120s60s, 2m
QF_VERBOSEEnable verbose outputfalsetrue, 1
QF_QUIETSuppress non-error outputfalsetrue, 1

See Configuration and environment variables for the full reference.

Duration format

Duration-related variables (QF_RETRY_DELAY, QF_RETRY_MAX_DELAY, QF_TIMEOUT) use Go duration format:

UnitSuffixExamples
Secondss30s, 90s
Millisecondsms500ms, 1500ms
Minutesm1m, 5m

Branch and commit detection

qf records each launch's branch and commit from the first of these variables that has a value:

ValueRead from, in orderThen
BranchQF_BRANCH, GIT_BRANCH, GITHUB_REF_NAME, CI_COMMIT_REF_NAME, BITBUCKET_BRANCHThe local git checkout
CommitQF_COMMIT, GIT_COMMIT, GITHUB_SHA, CI_COMMIT_SHA, BITBUCKET_COMMITThe local git checkout

If none of these is set, collect reads the branch and commit from the local git checkout. A detached HEAD, which many CI checkouts use, gives no branch, so set QF_BRANCH or pass --branch there.

CI systemBranchCommitWhat to set
GitHub ActionsGITHUB_REF_NAMEGITHUB_SHAOn pull_request runs, see Pull request branches
GitLab CI/CDCI_COMMIT_REF_NAMECI_COMMIT_SHANothing
Jenkins (Git plugin)GIT_BRANCHGIT_COMMITQF_BRANCH when GIT_BRANCH holds a remote prefix such as origin/main
Bitbucket PipelinesBITBUCKET_BRANCHBITBUCKET_COMMITNothing
Azure PipelinesNone: no Azure variable is read, and the checkout is a detached HEADThe local checkoutQF_BRANCH from Build.SourceBranch
CircleCIThe local checkout (no CircleCI variable is read)The local checkoutQF_BRANCH and QF_COMMIT from CIRCLE_BRANCH and CIRCLE_SHA1
Travis CIThe local checkout (a detached HEAD: no branch)The local checkoutQF_BRANCH and QF_COMMIT from TRAVIS_*
Any otherThe local checkoutThe local checkoutQF_BRANCH on a detached HEAD

Pull request branches on GitHub Actions

On pull_request events, GITHUB_REF_NAME is <number>/merge and GITHUB_SHA is the merge commit GitHub creates, so the launch would record that branch and commit instead of the pull request's own. Set both from the pull request:

env:
  QF_BRANCH: ${{ github.head_ref || github.ref_name }}
  QF_COMMIT: ${{ github.event.pull_request.head.sha || github.sha }}

On other events head_ref is empty, so both fall back to the pushed branch and commit. Where this goes in a workflow: GitHub Actions.

Security best practices

Token management

  • Never hardcode tokens in your repository
  • Always use your CI/CD platform's secret/variable management
  • Use one token per pipeline. Each token belongs to one project, so you can revoke a leaked token without breaking other pipelines.
  • Leave Allow writes off for CI tokens. Uploading results does not need it.
  • Tokens expire after at most 90 days. Rotate the CI token before it stops working (Access Tokens).

Check for leaked tokens

Add a pre-commit hook to catch accidentally committed tokens:

#!/bin/bash
# .git/hooks/pre-commit

# Qualflare access tokens start with qfp_ followed by 49 more characters
if git diff --cached --name-only | xargs grep -lE 'qfp_[A-Za-z0-9_-]{49}' 2>/dev/null; then
    echo "ERROR: Possible Qualflare access token in staged files!"
    echo "Remove the token before committing."
    exit 1
fi

Tokens created before the qfp_ prefix existed have no prefix, so this check cannot catch them.

Troubleshooting

Login fails in CI

Ensure the token value is correct and not empty:

- run: |
    [ -n "$QF_TOKEN" ] || { echo "QF_TOKEN is not set"; exit 1; }
    qf login ci --force

With no token argument and an empty QF_TOKEN, qf login reads standard input instead, and an empty token fails with Error: token cannot be empty. The guard line stops the job first, with a clear message.

Upload fails with timeout

Raise the overall upload deadline with --timeout (default 30s):

qf ci collect results.xml --timeout 2m

QF_TIMEOUT (default 120s) only caps each single HTTP request. Raise it as well only if one request takes longer than two minutes.

Retry failures

Adjust retry settings for unreliable networks:

env:
  QF_RETRY_MAX: 5
  QF_RETRY_DELAY: 2s
  QF_RETRY_MAX_DELAY: 60s

Retries run inside the --timeout deadline (default 30s), so raise it with them, for example qf ci collect results.xml --timeout 5m.

Branch or commit not detected

Manually override if auto-detection fails. The Branch and commit detection table names the variable to set on each CI, such as QF_BRANCH on Azure Pipelines or any detached HEAD.

Identifier already exists on a reused runner

On a self-hosted runner that keeps its home directory between jobs, a second qf login of the same identifier without --force fails with:

Error: identifier "ci" already exists; cannot prompt because stdin is not a terminal — use --force to overwrite

qf login <identifier> --force overwrites the saved token, as every qf login command in these pages' examples does. qualflare-action does the same from v1.0.3, which @v1 runs; a workflow pinned to an earlier tag fails here (Version pinning).

For every exit code and API error, see Troubleshoot qf and CI uploads.

Advanced usage

Conditional upload

Only upload on specific branches:

- name: Upload to Qualflare
  if: github.ref == 'refs/heads/main'
  env:
    QF_TOKEN: ${{ secrets.QF_TOKEN }}
  run: |
    qf login ci --force
    qf ci collect results.xml

Multiple test files

Upload multiple test result files:

# All XML files
qf ci collect test-results/*.xml

# Specific files
qf ci collect unit-tests.xml integration-tests.xml

One collect call is one launch, and the files in it merge into that launch. For parallel jobs, see Sharded runs.

Dry run in CI

Test your CI configuration without actually uploading:

- name: Validate test results (dry run)
  env:
    QF_TOKEN: ${{ secrets.QF_TOKEN }}
  run: |
    qf login ci --force
    qf ci collect results.xml --dry-run

Note: --dry-run does not upload to Qualflare, so the token does not need to be valid — but the identifier must still be registered locally with qf login.

qf validate checks report files with no token at all: see the qf command reference.

Related