# Setting Up CI

Source: https://docs.qualflare.com/docs/open-source/ci-setup · Last updated: 2026-11-06

Get real test results flowing into your published report, using the qualflare-action GitHub Action or the qf CLI directly.

A published report with no launches just shows "No runs yet." This page is
the fastest path from there to a report that actually reflects your test
history — full detail lives in the [CLI Tool](https://docs.qualflare.com/docs/cli) docs; this is the
short version for an open-source project on GitHub Actions.

> **Note:** Some frameworks have a native Qualflare reporter. See
> [Native reporters](https://docs.qualflare.com/docs/reporters) for the list. A native
> reporter keeps steps, attachments and retry history that JUnit XML drops, so
> use one where it exists. For any other framework, have your test runner write a
> results file it already supports: `qf` supports 27 frameworks and 4 generic formats. Run
> `qf list-formats` to see them all. Either way, an upload step runs after your
> tests and sends the file or report directory to Qualflare.

## Option 1: `qualflare-action` (recommended for GitHub Actions)

A GitHub Action that wraps the CLI — one `uses:` step instead of a manual
binary install:

```yaml
- name: Run tests
  run: pytest --junitxml=results.xml

- name: Upload results to Qualflare
  if: ${{ !cancelled() }}   # upload even when tests fail
  uses: Qualflare/qualflare-action@v1
  with:
    token: ${{ secrets.QF_TOKEN }}
    project: myapp
    results: results.xml
```

> **Warning:** Put `if: ${{ !cancelled() }}` on the upload step so it runs when your tests
> fail. Do not add `continue-on-error: true` to the test step: it marks the failed
> step as passed, so the job and the pull request check turn green even when
> tests fail.

Full reference — every input, the runner support matrix, version pinning — is
on [the GitHub Actions page](https://docs.qualflare.com/docs/cli/ci-cd/github-actions).

## Option 2: the CLI directly

Works on any CI platform, not just GitHub Actions, and takes every
`qf <identifier> collect` flag, including the ones the Action has no input
or variable for, such as `--format`, `--shard` and `--dry-run`:

```bash
# qf login reads the token from the QF_TOKEN environment variable
qf login ci --force
qf ci collect results.xml
```

You don't need the CLI for `--environment`, `--milestone` or `--branch`:
with the Action, set `QF_ENVIRONMENT`, `QF_MILESTONE` or `QF_BRANCH` in the
job's `env:`, and qf reads them inside the Action too
([What the action doesn't expose](https://docs.qualflare.com/docs/cli/ci-cd/github-actions#what-it-doesnt-expose)).

See [CI/CD Integration](https://docs.qualflare.com/docs/cli/ci-cd) for the full multi-platform guide
(GitHub Actions, GitLab CI/CD, Jenkins, Azure Pipelines, CircleCI,
Bitbucket Pipelines and any other CI) and
[Upload results with qf collect](https://docs.qualflare.com/docs/cli/collect) for the full flag reference.

## Getting a token

Both options need a Qualflare access token. Mint one at **Project Settings →
Access Tokens**, then store it as a CI secret. Option 1 passes the secret to
the Action's `token` input, so any secret name works. Option 2 needs the job to
expose it as the `QF_TOKEN` environment variable, which `qf login` reads.

## Keep the CI token valid

Every access token expires after 7, 30, 60 or 90 days, as chosen when it is
created (30 by default); 90 days is the longest Qualflare allows. Seven days
before a token expires, Qualflare emails the person who created it (not for
a 7-day token), and **Access Tokens** marks it **Expires soon**. After it
expires, every upload fails with exit code 3. Note the expiry date where
every maintainer can see it, such as a shared calendar, and replace the
token before then with
[Rotate the CI access token](https://docs.qualflare.com/docs/cli/ci-cd/rotate-tokens).

## Pull requests from forks

GitHub passes no secrets to a workflow run for a pull request from a fork,
and treats runs that Dependabot starts the same way. The upload step then
gets an empty token: qualflare-action fails with
`Error: token cannot be empty`, and the job fails. To skip the upload for
those runs, put this `if:` on the upload step in place of
`if: ${{ !cancelled() }}`, whether it uses the Action or the CLI:

```yaml
if: ${{ !cancelled() && github.actor != 'dependabot[bot]' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository) }}
```

With it, the upload is skipped only for pull requests from forks, for
Dependabot's runs and for cancelled runs. Pushes, pull requests from your
repository's own branches, and scheduled and manual runs upload, also when
tests fail.
Don't switch the workflow to `pull_request_target` to get the secret:
GitHub says to avoid that event if you build or run code from the pull
request.

## Persistent self-hosted runners

On a self-hosted runner that keeps its home directory between jobs, the
saved login stays too. qualflare-action v1.0.3 and later (what `@v1` runs)
reads the token from `QF_TOKEN` and logs in with `--force`, replacing the
saved token on every run, and so does `qf login ci --force` with
[the CLI directly](https://docs.qualflare.com/docs/open-source/ci-setup#option-2-the-cli-directly). A workflow pinned to an
earlier Action tag fails from its second run with
`identifier "myapp" already exists`
([Identifier already exists](https://docs.qualflare.com/docs/cli/ci-cd#identifier-already-exists)).
The token stays in qf's credentials file after the job; to keep that file in the job's directory and remove it, see
[Self-hosted runners](https://docs.qualflare.com/docs/cli/ci-cd/other-ci#self-hosted-runners).

GitHub says self-hosted runners should almost never be used for public
repositories, because anyone can open a pull request against the
repository and compromise the runner.

## What the report makes public

While the report is published, every launch your CI uploads to the project
appears on it, with each test's name, status and failure message
([what a public report shows](https://docs.qualflare.com/docs/open-source/faq#does-publishing-cost-anything-or-expose-private-data)).

* Search engines can index the report's launches page, and
  [the reports sitemap](https://reports.qualflare.com/sitemap.xml) lists it.
* Each run's page is `noindex`, so search engines don't list it, but anyone
  with its link can open it.
* The badge and banner are cached for a minute
  ([Caching](https://docs.qualflare.com/docs/api/public-reports#caching)).
* When you unpublish, Qualflare stops serving the report at once, but
  cached pages can stay visible for up to about an hour. Then the launches
  page answers `404` with `noindex`, and the sitemap no longer lists it.

## Related

* [GitHub Actions](https://docs.qualflare.com/docs/cli/ci-cd/github-actions) — qualflare-action's inputs, outputs and caveats
* [CI/CD Integration](https://docs.qualflare.com/docs/cli/ci-cd) — the CLI-based guide for every CI platform
* [Publishing a Report](https://docs.qualflare.com/docs/open-source/publishing) — where these results end up
