# TestNG reporter

Source: https://docs.qualflare.com/docs/reporters/testng · Last updated: 2026-10-29

Add com.qualflare:qualflare-testng as a test dependency, run mvn test or gradle test, and upload the qualflare-results folder with the Qualflare CLI.

`com.qualflare:qualflare-testng` is the Qualflare reporter for TestNG 7.4.0 or later on Java 11 or later: it registers itself, writes each run's results, with retries, steps and attachments, to `qualflare-results`, and `qf <identifier> collect` uploads the folder as one launch.

| Fact                 | Detail                                                                                                                                     |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| Package              | [`com.qualflare:qualflare-testng`](https://central.sonatype.com/artifact/com.qualflare/qualflare-testng) on Maven Central                  |
| Works with           | TestNG 7.4.0 or later, Java 11 or later                                                                                                    |
| qf                   | (requires qf 0.1.24 or later) for image attachments                                                                                        |
| License              | Apache-2.0                                                                                                                                 |
| Source               | [Qualflare/qualflare-testng](https://github.com/Qualflare/qualflare-testng) on GitHub                                                      |
| Without the reporter | `--format testng` with [TestNG results written as JUnit XML](https://docs.qualflare.com/docs/cli/formats#testng), not `testng-results.xml` |

## Before you begin

* The Qualflare CLI, `qf`: [Install the CLI](https://docs.qualflare.com/docs/cli/installation).
* A saved login: `qf login <identifier>` with a project access token whose creator still holds the project role that uploads need (requires the Editor role or higher in the project) ([Authenticate with qf login](https://docs.qualflare.com/docs/cli/authentication)).
* The **Identifier** of the environment the results belong to ([Environments](https://docs.qualflare.com/docs/runs/environments#identifier)).

## Install

Add the test dependency. The listener registers itself through TestNG's ServiceLoader: no `@Listeners`, no `-listener` flag, no `testng.xml` edit.

```xml
<dependency>
  <groupId>com.qualflare</groupId>
  <artifactId>qualflare-testng</artifactId>
  <version>0.1.0</version>
  <scope>test</scope>
</dependency>
```

With Gradle (`build.gradle.kts`):

```kotlin
testImplementation("com.qualflare:qualflare-testng:0.1.0")
```

## Configure

Set options as `qualflare.*` system properties, or `QUALFLARE_*` variables. In Maven Surefire:

```xml
<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <systemPropertyVariables>
      <qualflare.environment>staging</qualflare.environment>
    </systemPropertyVariables>
  </configuration>
</plugin>
```

In Gradle:

```kotlin
tasks.test {
    systemProperty("qualflare.environment", "staging")
}
```

| Option | Type | Default | Required | Description |
| --- | --- | --- | --- | --- |
| `qualflare.environment` | `string` | `development` | No | The environment's Identifier. Variable `QUALFLARE_ENVIRONMENT`. |
| `qualflare.outputDir` | `string` | `qualflare-results` | No | Variable `QUALFLARE_OUTPUT_DIR`. |
| `qualflare.platform` | `string` | `api` | No | Variable `QUALFLARE_PLATFORM`. |
| `qualflare.enabled` | `string` | `true` | No | `0`, `false`, `no` and `off` turn it off. Variable `QUALFLARE_ENABLED`. |

`qualflare.enabled` is read when the report is written, so setting it in a `@BeforeSuite` works.

Every reporter shares the other settings: see [Reporter configuration](https://docs.qualflare.com/docs/reporters/configuration).

Set `qualflare.environment` to the environment's Identifier: a name such as `Staging` fails the upload with HTTP 400 ([Environment](https://docs.qualflare.com/docs/reporters/configuration#environment)).

## Run and upload

```bash
mvn test
qf <identifier> collect ./qualflare-results
```

Each JVM writes its own file, and `qf` merges the folder into one launch.

## Run in CI

```yaml
- name: Upload results to Qualflare
  if: ${{ !cancelled() }}
  env:
    QF_TOKEN: ${{ secrets.QF_TOKEN }}
    QF_BRANCH: ${{ github.head_ref || github.ref_name }}
    QF_COMMIT: ${{ github.event.pull_request.head.sha || github.sha }}
  run: |
    npm install -g @qualflare/cli
    qf login ci --force
    qf ci collect ./qualflare-results
```

`if: ${{ !cancelled() }}` uploads the results when tests fail, and `QF_BRANCH` and `QF_COMMIT` record a pull request's own branch and commit ([Pull request branches](https://docs.qualflare.com/docs/cli/ci-cd#pull-request-branches)). For GitLab CI/CD, Jenkins and Azure Pipelines, see [Run the upload when tests fail](https://docs.qualflare.com/docs/cli/ci-cd#run-when-tests-fail).

## Sharding and retries

Retries come from `IRetryAnalyzer`, in the same JVM: one test with every attempt, flaky when a retry passed. TestNG reports a retried failure as a skip with `wasRetried()` set; the reporter reads that flag, so the attempts aren't skips.

Maven Surefire's `rerunFailingTestsCount` doesn't apply to TestNG.

For shards that run as separate CI jobs, see [Merge sharded CI jobs into one launch](https://docs.qualflare.com/docs/cli/sharding#merge-sharded-ci-jobs).

## What gets uploaded

* Each test's status, with a timeout distinct from a failure, and its per-attempt history
* Nested steps and attachments
* Labels, tags, priority and parameters

```java
import com.qualflare.testng.Qualflare;
import org.testng.annotations.Test;

public class CheckoutTest {
    @Test
    public void checksOut() {
        Qualflare.label("feature", "checkout");
        Qualflare.step("add to cart", () -> {
            Qualflare.maskedParameter("token");
        });
    }
}
```

The API finds the running test itself: no annotation. A call with no test in scope, a `@BeforeMethod` included, is dropped with a warning.

## Troubleshooting

* `testng-results.xml` isn't a format `qf` reads; use the reporter, or JUnit XML with `--format testng`.
* A launch has results from an earlier run: the reporter writes no run ID, so `qf` uploads every report in `qualflare-results`. Empty the folder before each run ([Run ID](https://docs.qualflare.com/docs/reporters/configuration#run-id)).
* `environment "Staging" does not exist in this project`: use the environment's Identifier, or pass `--environment` on the upload ([Environment](https://docs.qualflare.com/docs/reporters/configuration#environment)).
* Screenshots can't be downloaded: `qf` is older than 0.1.24. [Upgrade qf](https://docs.qualflare.com/docs/cli/installation#upgrading).

## Related

* **Guides:** [Reporters](https://docs.qualflare.com/docs/reporters) · [Upload results with qf collect](https://docs.qualflare.com/docs/cli/collect) · [Sharded runs and result directories](https://docs.qualflare.com/docs/cli/sharding) · [Upload screenshots, videos and traces](https://docs.qualflare.com/docs/cli/artifacts)
* **Reference:** [Reporter configuration](https://docs.qualflare.com/docs/reporters/configuration) · [TestNG format](https://docs.qualflare.com/docs/cli/formats#testng)
* **More:** [TestNG test reporting in Qualflare](https://qualflare.com/testng-test-reporting/)