CircleCI

To upload test results from CircleCI to Qualflare, add a step with when: always that runs qf <identifier> collect <report>, with the access token in a project environment variable named QF_TOKEN; qf reads no CircleCI variables, so set QF_BRANCH and QF_COMMIT yourself.

Before you begin

• A project access token with Allow writes off, which you create in Project Settings > Access Tokens Editor+ · project (requires the Editor role or higher in the project). See Access Tokens.
• A report file your test step writes, such as JUnit XML. The examples use ci as the identifier.

Store the token as an environment variable

1. In CircleCI, open your project's Project Settings > Environment Variables
2. Click Add Variable
3. Name: QF_TOKEN
4. Value: Your Qualflare access token
5. Save the variable

A context that holds QF_TOKEN, attached to the job with context: in the workflow, works the same way.

Add the upload step

Add to your .circleci/config.yml:

version: 2.1

jobs:
  test:
    docker:
      - image: cimg/python:3.12
    environment:
      QF_ENVIRONMENT: staging  # must be an environment that exists in the project
    steps:
      - checkout
      - run:
          name: Run tests
          command: pytest --junitxml=results.xml
      - run:
          name: Upload results to Qualflare
          when: always  # upload even when the tests fail
          command: |
            VERSION=0.4.0
            curl -L "https://github.com/qualflare/qualflare-cli/releases/download/v${VERSION}/qf_${VERSION}_linux_amd64.tar.gz" -o qf.tar.gz
            tar -xzf qf.tar.gz && sudo mv qf /usr/local/bin/
            export QF_BRANCH="$CIRCLE_BRANCH" QF_COMMIT="$CIRCLE_SHA1"
            qf login ci --force
            qf ci collect results.xml

workflows:
  test:
    jobs:
      - test

The image needs curl, tar and sudo, which the cimg images have.

Branch and commit

qf reads no CircleCI variable, so the export line in the upload step sets QF_BRANCH and QF_COMMIT from CIRCLE_BRANCH and CIRCLE_SHA1. It runs in the shell because CircleCI doesn't expand variables under environment:. On a tag build CIRCLE_BRANCH is empty: qf skips an empty variable, and the local checkout decides. The Branch and commit detection table lists what qf reads on each CI.

Run the upload when tests fail

when: always on the upload step runs it after a failed test step. qf <identifier> collect exits 0 when the upload succeeds, so the failed test step still fails the job.

Parallel jobs

One qf <identifier> collect call is one launch, and several files in one call merge into it. With CircleCI parallelism:, save each container's report to a workspace and upload them all from one later job with --shard: see Sharded runs.

Pin the qf version

The example installs qf 0.4.0 through VERSION=0.4.0. To pin another release, set VERSION to a version from the qf releases page.

Troubleshooting

Error: token cannot be empty means QF_TOKEN didn't reach the job: attach the context that holds it, and note that a build of a pull request from a fork gets no project variables unless the project allows it. Launches with no branch: keep the export line in the upload step. More fixes: Troubleshooting on CI/CD integration.

Related

• Guides: CI/CD integration
• Reference: Upload results with qf collect · Configuration
• CLI/API: Access Tokens