Azure Pipelines

To upload test results from Azure Pipelines to Qualflare, add a step that runs qf <identifier> collect <report> after your tests, with the access token in a secret variable named QF_TOKEN; qf reads no Azure variables, so set QF_BRANCH yourself.

Before you begin

• A project access token with Allow writes off, which you create in Project Settings > Access Tokens Editor+ · project (requires the Editor role or higher in the project). See Access Tokens.
• A report file your test step writes, such as JUnit XML. The examples use ci as the identifier.

Store the token as a secret variable

1. Go to your project Pipelines > Library
2. Go to Variable groups, create or select a group, for example qualflare
3. Add variable: QF_TOKEN, value: Your Qualflare access token
4. Mark as Secret: Checked
5. Click Save

Then reference the group in the YAML, as - group: qualflare does in the pipeline below.

Add the upload step

Add to your azure-pipelines.yml:

trigger:
  branches:
    include:
      - main
      - develop

variables:
- group: qualflare  # the variable group that holds QF_TOKEN
- name: QF_ENVIRONMENT
  value: 'staging'  # must be an environment that exists in the project

stages:
- stage: Test
  jobs:
  - job: TestAndReport
    pool:
      vmImage: 'ubuntu-latest'

    steps:
    - checkout: self
      displayName: 'Checkout code'

    - script: |
        VERSION=0.4.0
        curl -L "https://github.com/qualflare/qualflare-cli/releases/download/v${VERSION}/qf_${VERSION}_linux_amd64.tar.gz" -o qf.tar.gz
        tar -xzf qf.tar.gz && sudo mv qf /usr/local/bin/
      displayName: 'Install qf CLI'

    - script: pytest --junitxml=results.xml
      displayName: 'Run tests'

    - script: |
        BRANCH="${SYSTEM_PULLREQUEST_SOURCEBRANCH:-$BUILD_SOURCEBRANCH}"
        export QF_BRANCH="${BRANCH#refs/heads/}"
        qf login ci --force
        qf ci collect results.xml
      env:
        QF_TOKEN: $(QF_TOKEN)  # secret variables reach a script only through env
      displayName: 'Upload results to Qualflare'
      condition: always()

See the official Azure Pipelines documentation for azure-pipelines.yml syntax and pipeline concepts beyond what's shown here.

Branch and commit

qf reads no Azure variable, and the Azure checkout is a detached HEAD, so qf finds the commit through local git and no branch. The first two lines of the upload script set QF_BRANCH from Build.SourceBranch, or from System.PullRequest.SourceBranch on a pull request build, without the refs/heads/ prefix. Setting QF_COMMIT: $(Build.SourceVersion) in env: is optional: it is the same commit.

Run the upload when tests fail

condition: always() on the upload step runs it when the test step fails.

Parallel jobs

One qf <identifier> collect call is one launch, and several files in one call merge into it. With the Azure parallel strategy, gather every job's report and upload them all from one later job with --shard: see Sharded runs.

Pin the qf version

The example installs qf 0.4.0 through VERSION=0.4.0. To pin another release, set VERSION to a version from the qf releases page.

Troubleshooting

Error: token cannot be empty means QF_TOKEN didn't reach the script: map it in the step's env:. If qf login succeeds but the upload then fails with a 401 (exit code 3), check that the variable group is linked in variables:: Azure passes an unresolved $(QF_TOKEN) on as that literal text, and qf saves it as the token. Launches with no branch: keep the two QF_BRANCH lines at the top of the upload script. More fixes: Troubleshooting on CI/CD integration.

Related

• Guides: CI/CD integration
• Reference: Upload results with qf collect · Configuration
• CLI/API: Access Tokens