Troubleshoot qf and CI uploads

When the Qualflare CLI (qf) fails, it prints one Error: line and exits with code 1, 3, 4, 5 or 7; code 0 means the upload succeeded, even when tests in the report failed. Find the exact message below for its cause and fix.

Read an error message

An error from the API names the operation, the server's message, the HTTP status, a hint for some statuses and a request ID:

Error: failed to process test results: send: environment "dev" does not exist in this project (status: 400) [request_id: …]

• failed to process test results: means collect failed while it parsed or uploaded the files.
• send: is the upload request, and the text after it is the server's message.
• (status: 400) is the HTTP status, which decides the exit code.
• After a 401, 402 or 403, a hint follows, such as — the token lacks access to this project.
• [request_id: …] identifies the request. Include it when you contact support (Getting Help).

Exit codes

Install and log in

These errors come from installing qf and from the token it uploads with.

command not found: qf after you install it

Symptoms:

• "command not found: qf"
• "permission denied" when you move the binary into place
• The install command fails or hangs

Possible causes:

• The directory that holds qf is not on your PATH
• Your shell has not picked up a PATH change yet
• No write access to the target directory, such as /usr/local/bin
• Your OS or CPU has no release build. Builds exist for macOS, Linux and Windows on amd64 and arm64.
• A network or proxy problem during the download

Resolution steps:

1. Install with a supported method. There is no install script. See Installation for Docker and release binaries.

   # macOS or Linux (Homebrew)
   brew install qualflare/tap/qf
   
   # macOS, Linux or Windows (npm)
   npm install -g @qualflare/cli

2. If you downloaded a release binary, move it into a directory on your PATH:

   sudo mv qf /usr/local/bin/

   On Windows, move qf.exe into a folder on your PATH.

3. Open a new terminal, then check that your shell finds qf:

   # macOS and Linux
   which qf
   
   # Windows
   where.exe qf

   If it is still not found, add the directory that holds qf to your PATH: in your shell profile on macOS and Linux, or in Environment Variables on Windows.

4. Verify the installation:

   qf version

invalid or expired access token (exit code 3)

Qualflare calls API keys access tokens.

Symptoms:

• collect or a read command fails with invalid or expired access token (status: 401) and exits with code 3:

  Error: failed to process test results: send: invalid or expired access token (status: 401) — run `qf login <identifier> <token>` to re-authenticate [request_id: …]

Possible causes:

• The token expired. A token stops working after the expiration you chose when you created it.
• Someone revoked the token.
• The token's creator signed out everywhere, reset their password or locked their account, or the workspace turned on a 2FA or SSO requirement.
• The account of the person who created the token was deactivated or deleted.
• The token was copied incompletely. New tokens start with qfp_.
• The job still sets QF_API_KEY, which qf no longer reads.

Resolution steps:

1. Create a new token:

   • In your project, open Settings → Access Tokens
   • Click Create Access Token
   • Copy the token. It is shown only once.

2. Save the new token under your identifier. --force replaces the old one:

   qf login myapp --force

   In CI, update the QF_TOKEN secret instead. qf login ci --force reads it.

3. Check the token with a real request. --dry-run never contacts Qualflare, so it cannot test a token:

   qf myapp launches list

the token lacks access to this project (exit code 4)

A 403 means the token is valid, but the person who created it no longer has the role the command needs on the project: Editor or higher for collect, Viewer or higher for the read commands. The token's own access level is not the cause: Read & upload is enough for collect.

Error: failed to process test results: send: no project access (status: 403) — the token lacks access to this project [request_id: …]

Ask a project Maintainer or Owner to restore the role, or create a new token as someone who has it. Roles and permissions lists what each role can do.

identifier "ci" already exists when you log in

qf login asks before it replaces a saved identifier, and with no terminal it can't ask, so it stops:

Error: identifier "ci" already exists; cannot prompt because stdin is not a terminal — use --force to overwrite

Add --force: qf login ci --force. It happens on a CI runner that keeps its home directory between jobs.

Upload results

file does not exist, no files match pattern or failed to parse file

Symptoms:

• "file does not exist" error
• "no files match pattern" error
• "failed to parse file" error
• Upload hangs or times out

Possible causes:

• Incorrect file path
• A glob or directory that matches no report
• Invalid file format
• Network connectivity issues
• API endpoint unreachable

Resolution steps:

1. Verify file exists:

   ls -la test-results.xml
   pwd  # Check current directory

2. Use absolute path:

   qf myapp collect /full/path/to/test-results.xml

   A glob that matches nothing fails with no files match pattern, and a directory with no .json report directly inside it fails with contains no .json report files: see Collect a directory.

3. Validate file format:

   # Dry run to check parsing
   qf myapp collect test-results.xml --dry-run --verbose
   
   # Check file format
   qf validate test-results.xml

4. Raise the overall deadline for large files (default 30s):

   qf myapp collect test-results.xml --timeout 2m

5. Check network connectivity. The API answers {"status":"ok"}:

   # Test API endpoint
   curl -fsS https://api.qualflare.com/health

6. If collect fails with the results are …, over /collect's 10 MB limit even without attachments; split the upload, e.g. collect fewer files per run, upload fewer files per collect: see Size limits.

unsupported format, or qf reads the wrong format

Symptoms:

• Error: "unsupported format"
• CLI cannot identify test framework
• Auto-detection picks wrong format

Possible causes:

• Unsupported test framework
• Non-standard file format
• Missing format specification
• Ambiguous file content

Resolution steps:

1. List supported formats:

   qf list-formats

2. Specify format explicitly:

   qf myapp collect results.xml --format junit
   qf myapp collect results.json --format playwright

   JUnit-style XML from TestNG, Maestro, XCTest, Espresso or PHPUnit always needs its --format value: see How qf picks a format.

3. Validate file before uploading:

   qf validate results.json --format your-framework --verbose

4. Check file content:

   # View first few lines
   head -20 results.xml
   
   # Verify format structure
   cat results.xml | grep -E '<testsuite|<testcase'

5. Convert to supported format:

   • If using unsupported framework, convert to JUnit XML
   • Many test frameworks have JUnit XML reporters
   • If your framework has an Allure adapter, upload its results directory instead (see Allure results)

environment "staging" does not exist in this project

Symptoms:

• Results show under the wrong environment
• The upload fails with environment "staging" does not exist in this project (status: 400)

Possible causes:

• No environment was passed, so qf used development
• The value doesn't match an environment Identifier exactly
• The environment doesn't exist in this project yet

Resolution steps:

1. Find the Identifier:

   • In the project, open Settings → Environments
   • The Identifier is the smaller text under each environment's name
   • Every new project starts with development, staging, production and qa

2. Create a missing environment:

   • Click Create Environment
   • Fill in Identifier, Name and Description (at least 3 characters each), then click Create
   • You need the Editor role or higher on the project

3. Pass the Identifier when you upload:

   qf myapp collect results.xml --environment staging

   You can set QF_ENVIRONMENT instead. --environment wins when both are set. If you set neither, qf uses the environment recorded by a native reporter, or development.

4. Match the case exactly:

   • Identifiers are case-sensitive: Production and production are different
   • An upload with an unknown environment fails with HTTP 400 and exits with code 1

A milestone or language the project doesn't have fails the same way, with 400 and exit code 1: milestone #12 does not exist in this project or language "xx-XX" is not supported. See Launch metadata for each field's flag, variable and default.

a plan limit was reached (exit code 4)

A 402 means the workspace reached a limit of its plan:

Error: failed to process test results: send: … (status: 402) — a plan limit was reached; check your Qualflare subscription [request_id: …]

The workspace Owner can check usage under Plan & Billing. A retried upload of a launch that was already created is not refused. See Plans and limits.

context deadline exceeded during an upload

Symptoms:

• Upload takes too long
• "timeout" error after 30 seconds
• Large files fail consistently

Possible causes:

• File too large
• Slow network connection
• Server overloaded
• Firewall blocking upload

Resolution steps:

1. Raise the overall deadline. --timeout covers parsing and uploading, retries included, and defaults to 30s:

   qf myapp collect results.xml --timeout 2m

   QF_TIMEOUT (default 120s) only caps each single HTTP request. Raise it as well only if one request takes longer than two minutes.

2. Configure retry settings. Retries run inside the --timeout deadline, so raise that too:

   export QF_RETRY_MAX=5
   export QF_RETRY_DELAY=2s
   export QF_RETRY_MAX_DELAY=60s
   qf myapp collect results.xml --timeout 5m

3. Test network speed:

   # Check connection to API
   ping api.qualflare.com
   
   # Test upload speed
   curl -o /dev/null -s -w "%{time_total}\n" https://api.qualflare.com

The upload still fails after retries (exit code 7)

Symptoms:

• Upload fails after multiple retries and shows the last attempt's error. The exit code is 7 when the server answered 429 or a 5xx status, and 1 when the request got no answer, for example after a network error or a timeout
• Intermittent upload failures

Possible causes:

• Unstable network connection
• Server temporarily unavailable
• Rate limiting
• Corrupted test result file

qf retries a 429, 500, 502, 503 or 504 answer and a network error, 3 times by default and at most 10. Every attempt sends the same Idempotency-Key, so a retry never creates a second launch.

Resolution steps:

1. Check the exit code. 7 means the server answered 429 or a 5xx status on every attempt:

   qf myapp collect results.xml; echo "exit code: $?"

2. Adjust retry configuration:

   # Increase max retries
   export QF_RETRY_MAX=10
   
   # Increase delay between retries
   export QF_RETRY_DELAY=5s
   export QF_RETRY_MAX_DELAY=120s
   
   # Retries run inside --timeout, so raise it too
   qf myapp collect results.xml --timeout 10m

3. Test file integrity:

   # Validate file format
   qf validate results.xml
   
   # Check file size
   ls -lh results.xml

4. Exit code 7 marks a temporary failure. Re-run the job.

CI pipelines

token cannot be empty or no identifier "ci" configured in CI

Symptoms:

• qf login fails with token cannot be empty
• collect fails with no identifier "ci" configured

Possible causes:

• The QF_TOKEN secret is not set, or the job cannot see it
• The secret has a different name. qf login reads only QF_TOKEN.
• qf login did not run in this job. Each CI job starts with no saved login.

Resolution steps:

1. Verify the secret is set:

   • GitHub Actions: Settings > Secrets and variables > Actions
   • GitLab CI: Settings > CI/CD > Variables
   • Jenkins: Manage Credentials > Global credentials

2. Expose it to the job as QF_TOKEN, then log in and upload in the same job:

   - name: Upload to Qualflare
     env:
       QF_TOKEN: ${{ secrets.QF_TOKEN }}
     run: |
       qf login ci --force
       qf ci collect results.xml

3. Check that the secret reached the job, without printing it:

   [ -n "$QF_TOKEN" ] || { echo "QF_TOKEN is not set"; exit 1; }

4. Check where the secret is available. Some CI platforms limit secrets to protected branches or environments, so other jobs do not get them.

More on CI logins: Log in in CI.

The upload times out in CI

Symptoms:

• CI/CD pipeline times out during upload
• "connection timeout" or "request timeout" error

Possible causes:

• CI environment has strict timeout
• Network restrictions in CI environment
• Large test result files
• API endpoint not reachable from CI

Resolution steps:

1. Raise the overall upload deadline with --timeout (default 30s):

   - qf ci collect results.xml --timeout 2m

   QF_TIMEOUT (default 120s) only caps each single HTTP request. Set it as well only if one request takes longer than two minutes.

2. Configure retry settings:

   env:
     QF_RETRY_MAX: 5
     QF_RETRY_DELAY: 2s
     QF_RETRY_MAX_DELAY: 60s

3. Test API connectivity from CI. The API answers {"status":"ok"}:

   - name: Test API connectivity
     run: curl -fsS https://api.qualflare.com/health

4. Check CI firewall rules:

   • Some CI environments block outbound HTTPS
   • Contact CI platform support for firewall issues

The launch has no branch or commit

Symptoms:

• Git branch and commit show as blank
• Launch metadata missing git information

Possible causes:

• Auto-detection not supported by CI platform
• A detached HEAD checkout, which has no branch name
• Environment variable names different

Resolution steps:

1. Check current environment variables:

   - name: Debug CI variables
     run: |
       echo "GIT_BRANCH=$GIT_BRANCH"
       echo "GIT_COMMIT=$GIT_COMMIT"
       echo "GITHUB_REF_NAME=$GITHUB_REF_NAME"
       echo "GITHUB_SHA=$GITHUB_SHA"

2. Pass the branch and commit on the command line:

   qf ci collect results.xml --branch "$BRANCH_NAME" --commit "$COMMIT_SHA"

3. Know what qf reads on its own. For the branch it uses the first of QF_BRANCH, GIT_BRANCH, GITHUB_REF_NAME, CI_COMMIT_REF_NAME and BITBUCKET_BRANCH that is set. For the commit it uses QF_COMMIT, GIT_COMMIT, GITHUB_SHA, CI_COMMIT_SHA and BITBUCKET_COMMIT. If none is set, it asks the local git checkout. A detached HEAD gives no branch, so set QF_BRANCH or --branch there.

The results file isn't found in CI

Symptoms:

• "file does not exist" error in CI
• Upload command cannot find test results

Possible causes:

• Tests not generating output file
• File path incorrect
• Working directory different
• Artifact not created

Resolution steps:

1. Verify tests generate output:

   - name: Run tests
     run: |
       pytest --junitxml=results.xml
       ls -la results.xml  # Verify file exists

2. Check working directory:

   - name: Upload results
     run: |
       pwd  # Check current directory
       ls -la  # List files
       qf ci collect test-results/results.xml  # Use correct path

3. Use absolute path:

   - run: qf ci collect "$GITHUB_WORKSPACE/test-results/results.xml"

4. Persist the results file when the upload runs in a later job:

   # In the test job
   - name: Run tests
     run: pytest --junitxml=results.xml
   - uses: actions/upload-artifact@v4
     with:
       name: test-results
       path: results.xml
   
   # In the upload job
   - uses: actions/download-artifact@v4
     with:
       name: test-results
   - name: Upload to Qualflare
     run: qf ci collect results.xml

Debug an upload

Enable verbose logging:

qf myapp collect results.xml --verbose

Log the full HTTP request and response (the token is redacted):

qf myapp collect results.xml --debug

Use dry run to test:

qf myapp collect results.xml --dry-run --output json

Check a file without a login: qf validate results.xml parses it without uploading and needs no saved identifier (qf validate).

Related

• Guides: Authenticate with qf login · CI/CD Integration
• Reference: Upload results with qf collect · Supported formats · Configuration and environment variables
• Help: Troubleshooting Guide (app issues) · Getting Help