JUnit 5 reporter

com.qualflare:qualflare-junit5 is the Qualflare reporter for JUnit Platform 1.12 or later on Java 11 or later: it registers itself, writes each run's results, with retries, steps and attachments, to qualflare-results, and qf <identifier> collect uploads the folder as one launch.

Before you begin

• The Qualflare CLI, qf: Install the CLI.
• A saved login: qf login <identifier> with a project access token whose creator still holds the project role that uploads need Editor+ · project (requires the Editor role or higher in the project) (Authenticate with qf login).
• The Identifier of the environment the results belong to (Environments).

Install

Add the test dependency. The reporter registers itself through the JUnit Platform's ServiceLoader: no build plugin, no runner change.

<dependency>
  <groupId>com.qualflare</groupId>
  <artifactId>qualflare-junit5</artifactId>
  <version>0.2.0</version>
  <scope>test</scope>
</dependency>

With Gradle (build.gradle.kts):

testImplementation("com.qualflare:qualflare-junit5:0.2.0")

JUnit 4 tests that run on the platform through junit-vintage are reported too.

Configure

Set options as qualflare.* system properties, or QUALFLARE_* variables. In Maven Surefire:

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <systemPropertyVariables>
      <qualflare.environment>staging</qualflare.environment>
    </systemPropertyVariables>
  </configuration>
</plugin>

In Gradle:

tasks.test {
    systemProperty("qualflare.environment", "staging")
}

The metadata calls need @ExtendWith(QualflareExtension.class) on the test class, or this line in junit-platform.properties:

junit.jupiter.extensions.autodetection.enabled=true

Every reporter shares the other settings: see Reporter configuration.

Set qualflare.environment to the environment's Identifier: a name such as Staging fails the upload with HTTP 400 (Environment).

Run and upload

mvn test  # or: gradle test
qf <identifier> collect ./qualflare-results

Run in CI

- name: Upload results to Qualflare
  if: ${{ !cancelled() }}
  env:
    QF_TOKEN: ${{ secrets.QF_TOKEN }}
    QF_BRANCH: ${{ github.head_ref || github.ref_name }}
    QF_COMMIT: ${{ github.event.pull_request.head.sha || github.sha }}
  run: |
    npm install -g @qualflare/cli
    qf login ci --force
    qf ci collect ./qualflare-results

if: ${{ !cancelled() }} uploads the results when tests fail, and QF_BRANCH and QF_COMMIT record a pull request's own branch and commit (Pull request branches). For GitLab CI/CD, Jenkins and Azure Pipelines, see Run the upload when tests fail.

Upload JUnit XML without the reporter

Maven Surefire and Gradle already write JUnit XML. Upload it with --format junit, and quote the glob so qf expands it:

# Maven Surefire
qf <identifier> collect "target/surefire-reports/*.xml" --format junit
# Gradle
qf <identifier> collect "build/test-results/test/*.xml" --format junit

JUnit XML carries a status, a duration and a class name: no per-attempt history, attachments or metadata, and no way to tell a failed assumption from a skip. Other tools that write JUnit XML, such as NUnit and xUnit.net, upload the same way: see JUnit XML.

Sharding and retries

With forkCount above 1, reuseForks=false or parallel execution, every JVM writes its own file to the folder and qf merges them; nothing to configure. For shards that run as separate CI jobs, see Merge sharded CI jobs into one launch.

Surefire's rerunFailingTestsCount gives one test with every attempt, flaky when a rerun passed; a test that fails every attempt is failed.

Gradle's test-retry plugin starts a new JVM per retry, so each attempt is written to its own file as a single-attempt test and no per-attempt history survives. Flaky test detection still works, because Qualflare scores it across launches.

What gets uploaded

• Each test's status, aborted included, with its per-attempt history
• Nested steps
• Attachments from TestReporter.publishFile: PNG, JPEG and GIF are copied as files, others inline, within an 8 MiB budget
• Labels, tags, links, priority and parameters

import com.qualflare.junit5.Qualflare;
import com.qualflare.junit5.QualflareExtension;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;

@ExtendWith(QualflareExtension.class)
class CheckoutTest {
    @Test
    void checksOut() {
        Qualflare.step("add to cart", () -> {
            Qualflare.maskedParameter("token");
        });
    }
}

Qualflare.maskedParameter takes no value.

Troubleshooting

• A @BeforeAll that throws uploads one [container failure] test instead of the tests it stopped.
• Each @RepeatedTest and @ParameterizedTest row is its own test, not an attempt.
• Steps appear only where you call Qualflare.step.
• Import org.junit.jupiter.api.extension.MediaType, not org.junit.jupiter.api.MediaType: JUnit 5.12 ships only the first.
• A launch has results from an earlier run: the reporter writes no run ID, so qf uploads every report in qualflare-results. Empty the folder before each run (Run ID).
• environment "Staging" does not exist in this project: use the environment's Identifier, or pass --environment on the upload (Environment).
• Screenshots can't be downloaded: qf is older than 0.1.24. Upgrade qf.

Related

• Guides: Reporters · Upload results with qf collect · Sharded runs and result directories · Upload screenshots, videos and traces
• Reference: Reporter configuration · JUnit XML format
• More: JUnit test reporting in Qualflare