Bitbucket Pipelines
Upload test results from Bitbucket Pipelines to Qualflare with the qf CLI: an after-script that runs when tests fail and a secured QF_TOKEN repository variable.
Last updated
To upload test results from Bitbucket Pipelines to Qualflare, run qf <identifier> collect <report> in the step's after-script, with the access token in a secured repository variable named QF_TOKEN; qf reads BITBUCKET_BRANCH and BITBUCKET_COMMIT itself.
Before you begin
- A project access token with Allow writes off, which you create in Project Settings > Access Tokens Editor+ · project (requires the Editor role or higher in the project). See Access Tokens.
- A report file your test script writes, such as JUnit XML. The examples use
cias the identifier.
Store the token as a repository variable
- Go to your repository's Repository settings > Pipelines > Repository variables
- Click Add variable
- Name:
QF_TOKEN - Value: Your Qualflare access token
- Secured: Checked
- Click Add
Add the upload
Add to your bitbucket-pipelines.yml:
image: python:3.12
pipelines:
default:
- step:
name: Test and upload
script:
- pytest --junitxml=results.xml
after-script: # runs even when the script fails
- VERSION=0.4.0
- curl -L "https://github.com/qualflare/qualflare-cli/releases/download/v${VERSION}/qf_${VERSION}_linux_amd64.tar.gz" -o qf.tar.gz
- tar -xzf qf.tar.gz && mv qf /usr/local/bin/
- export QF_ENVIRONMENT=staging # must be an environment that exists in the project
- qf login ci --force
- qf ci collect results.xmlThe image needs curl and tar. Bitbucket runs it as root by default, so mv can write /usr/local/bin.
Branch and commit
qf reads the branch from BITBUCKET_BRANCH and the commit from BITBUCKET_COMMIT, which Bitbucket sets, so there is nothing to set. A tag pipeline has no BITBUCKET_BRANCH, and the local checkout decides the branch. The Branch and commit detection table lists what qf reads on each CI.
Run the upload when tests fail
after-script runs whether script passed or failed, and the step keeps the result of script, so failed tests still fail the step. A failed after-script command doesn't change the step's result, so a failed upload leaves the build as the tests left it: read the upload's log and its exit code.
Parallel jobs
One qf <identifier> collect call is one launch, and several files in one call merge into it, so upload once for all parallel: steps, not from each step's after-script. A failed step stops the pipeline, so an ordinary step after the parallel steps doesn't run when tests fail. A final step runs whatever the earlier steps did: save each step's report as an artifact with capture-on: always, because by default a step saves its artifacts only when it succeeds, and upload them all from the final step with --shard.
image: python:3.12
pipelines:
default:
- parallel:
steps:
- step:
name: Unit tests
script:
- pytest tests/unit --junitxml=test-results/unit.xml
artifacts:
upload:
- name: unit-results
paths:
- test-results/unit.xml
capture-on: always # keep the report when the tests fail
- step:
name: Integration tests
script:
- pytest tests/integration --junitxml=test-results/integration.xml
artifacts:
upload:
- name: integration-results
paths:
- test-results/integration.xml
capture-on: always
- final: # runs after the parallel steps, even when they fail
name: Upload to Qualflare
script:
- VERSION=0.4.0
- curl -L "https://github.com/qualflare/qualflare-cli/releases/download/v${VERSION}/qf_${VERSION}_linux_amd64.tar.gz" -o qf.tar.gz
- tar -xzf qf.tar.gz && mv qf /usr/local/bin/
- export QF_ENVIRONMENT=staging # must be an environment that exists in the project
- qf login ci --force
- qf ci collect "test-results/*.xml" --shardGive each parallel step's report its own path: when two parallel steps save a file at the same path, only the file from the step listed last is kept. Keep fail-fast off, its default, so a failed step doesn't stop the others. If the upload fails, read the final step's log and the upload's exit code. See Sharded runs.
Pin the qf version
The example installs qf 0.4.0 through VERSION=0.4.0. To pin another release, set VERSION to a version from the qf releases page.
Troubleshooting
Error: token cannot be empty means QF_TOKEN didn't reach the step: the variable's name isn't QF_TOKEN, or the pipeline runs in a fork, which doesn't have your repository's variables. A failed upload but a green build: after-script failures don't fail the step, so check its log. More fixes: Troubleshooting on CI/CD integration.
Related
- Guides: CI/CD integration
- Reference: Upload results with qf collect · Configuration
- CLI/API: Access Tokens · Rotate the CI access token
CircleCI
Upload test results from CircleCI to Qualflare with the qf CLI: a config.yml step that runs when tests fail, a QF_TOKEN variable, and QF_BRANCH from CircleCI.
Other CI systems, Docker and npx
Upload test results to Qualflare from Travis CI or any other CI with the qf CLI, its Docker image or npx: self-hosted runners and branch detection included.