Skip to content
The docs are now organized by product area. Old links still work. Report a broken link

Bitbucket Pipelines

Upload test results from Bitbucket Pipelines to Qualflare with the qf CLI: an after-script that runs when tests fail and a secured QF_TOKEN repository variable.

Last updated

To upload test results from Bitbucket Pipelines to Qualflare, run qf <identifier> collect <report> in the step's after-script, with the access token in a secured repository variable named QF_TOKEN; qf reads BITBUCKET_BRANCH and BITBUCKET_COMMIT itself.

Before you begin

  • A project access token with Allow writes off, which you create in Project Settings > Access Tokens Editor+ · project (requires the Editor role or higher in the project). See Access Tokens.
  • A report file your test script writes, such as JUnit XML. The examples use ci as the identifier.

Store the token as a repository variable

  1. Go to your repository's Repository settings > Pipelines > Repository variables
  2. Click Add variable
  3. Name: QF_TOKEN
  4. Value: Your Qualflare access token
  5. Secured: Checked
  6. Click Add

Add the upload

Add to your bitbucket-pipelines.yml:

image: python:3.12

pipelines:
  default:
    - step:
        name: Test and upload
        script:
          - pytest --junitxml=results.xml
        after-script:  # runs even when the script fails
          - VERSION=0.4.0
          - curl -L "https://github.com/qualflare/qualflare-cli/releases/download/v${VERSION}/qf_${VERSION}_linux_amd64.tar.gz" -o qf.tar.gz
          - tar -xzf qf.tar.gz && mv qf /usr/local/bin/
          - export QF_ENVIRONMENT=staging  # must be an environment that exists in the project
          - qf login ci --force
          - qf ci collect results.xml

The image needs curl and tar. Bitbucket runs it as root by default, so mv can write /usr/local/bin.

Branch and commit

qf reads the branch from BITBUCKET_BRANCH and the commit from BITBUCKET_COMMIT, which Bitbucket sets, so there is nothing to set. A tag pipeline has no BITBUCKET_BRANCH, and the local checkout decides the branch. The Branch and commit detection table lists what qf reads on each CI.

Run the upload when tests fail

after-script runs whether script passed or failed, and the step keeps the result of script, so failed tests still fail the step. A failed after-script command doesn't change the step's result, so a failed upload leaves the build as the tests left it: read the upload's log and its exit code.

The after-script runs whether the tests passed or failed, so results reach Qualflare either way.

Parallel jobs

One qf <identifier> collect call is one launch, and several files in one call merge into it, so upload once for all parallel: steps, not from each step's after-script. A failed step stops the pipeline, so an ordinary step after the parallel steps doesn't run when tests fail. A final step runs whatever the earlier steps did: save each step's report as an artifact with capture-on: always, because by default a step saves its artifacts only when it succeeds, and upload them all from the final step with --shard.

image: python:3.12

pipelines:
  default:
    - parallel:
        steps:
          - step:
              name: Unit tests
              script:
                - pytest tests/unit --junitxml=test-results/unit.xml
              artifacts:
                upload:
                  - name: unit-results
                    paths:
                      - test-results/unit.xml
                    capture-on: always  # keep the report when the tests fail
          - step:
              name: Integration tests
              script:
                - pytest tests/integration --junitxml=test-results/integration.xml
              artifacts:
                upload:
                  - name: integration-results
                    paths:
                      - test-results/integration.xml
                    capture-on: always
    - final:  # runs after the parallel steps, even when they fail
        name: Upload to Qualflare
        script:
          - VERSION=0.4.0
          - curl -L "https://github.com/qualflare/qualflare-cli/releases/download/v${VERSION}/qf_${VERSION}_linux_amd64.tar.gz" -o qf.tar.gz
          - tar -xzf qf.tar.gz && mv qf /usr/local/bin/
          - export QF_ENVIRONMENT=staging  # must be an environment that exists in the project
          - qf login ci --force
          - qf ci collect "test-results/*.xml" --shard

Give each parallel step's report its own path: when two parallel steps save a file at the same path, only the file from the step listed last is kept. Keep fail-fast off, its default, so a failed step doesn't stop the others. If the upload fails, read the final step's log and the upload's exit code. See Sharded runs.

Pin the qf version

The example installs qf 0.4.0 through VERSION=0.4.0. To pin another release, set VERSION to a version from the qf releases page.

Troubleshooting

Error: token cannot be empty means QF_TOKEN didn't reach the step: the variable's name isn't QF_TOKEN, or the pipeline runs in a fork, which doesn't have your repository's variables. A failed upload but a green build: after-script failures don't fail the step, so check its log. More fixes: Troubleshooting on CI/CD integration.