documentation/src/docs/asciidoc/user-guide/running-tests.adoc

[[running-tests]]
== Running Tests

[[running-tests-ide]]
=== IDE Support

[[running-tests-ide-intellij-idea]]
==== IntelliJ IDEA

IntelliJ IDEA supports running tests on the JUnit Platform since version 2016.2. For
details please see the
https://blog.jetbrains.com/idea/2016/08/using-junit-5-in-intellij-idea/[post on the
IntelliJ IDEA blog]. Note, however, that it is recommended to use IDEA 2017.3 or newer
since these newer versions of IDEA will download the following JARs automatically based
on the API version used in the project: `junit-platform-launcher`,
`junit-jupiter-engine`, and `junit-vintage-engine`.

WARNING: IntelliJ IDEA releases prior to IDEA 2017.3 bundle specific versions of JUnit 5.
Thus, if you want to use a newer version of JUnit Jupiter, execution of tests within the
IDE might fail due to version conflicts. In such cases, please follow the instructions
below to use a newer version of JUnit 5 than the one bundled with IntelliJ IDEA.

In order to use a different JUnit 5 version (e.g., {jupiter-version}), you may need to
include the corresponding versions of the `junit-platform-launcher`,
`junit-jupiter-engine`, and `junit-vintage-engine` JARs in the classpath.

.Additional Gradle Dependencies
[source,groovy]
[subs=attributes+]
----
testImplementation(platform("org.junit:junit-bom:{bom-version}"))
testRuntimeOnly("org.junit.platform:junit-platform-launcher") {
  because("Only needed to run tests in a version of IntelliJ IDEA that bundles older versions")
}
testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine")
testRuntimeOnly("org.junit.vintage:junit-vintage-engine")
----

.Additional Maven Dependencies
[source,xml]
[subs=attributes+]
----
<!-- ... -->
<dependencies>
	<!-- Only needed to run tests in a version of IntelliJ IDEA that bundles older versions -->
	<dependency>
		<groupId>org.junit.platform</groupId>
		<artifactId>junit-platform-launcher</artifactId>
		<scope>test</scope>
	</dependency>
	<dependency>
		<groupId>org.junit.jupiter</groupId>
		<artifactId>junit-jupiter-engine</artifactId>
		<scope>test</scope>
	</dependency>
	<dependency>
		<groupId>org.junit.vintage</groupId>
		<artifactId>junit-vintage-engine</artifactId>
		<scope>test</scope>
	</dependency>
</dependencies>
<dependencyManagement>
	<dependencies>
		<dependency>
			<groupId>org.junit</groupId>
			<artifactId>junit-bom</artifactId>
			<version>{bom-version}</version>
			<type>pom</type>
			<scope>import</scope>
		</dependency>
	</dependencies>
</dependencyManagement>
----

[[running-tests-ide-eclipse]]
==== Eclipse

Eclipse IDE offers support for the JUnit Platform since the Eclipse Oxygen.1a (4.7.1a)
release.

For more information on using JUnit 5 in Eclipse consult the official _Eclipse support
for JUnit 5_ section of the
https://www.eclipse.org/eclipse/news/4.7.1a/#junit-5-support[Eclipse Project Oxygen.1a
(4.7.1a) - New and Noteworthy] documentation.

[[running-tests-ide-netbeans]]
==== NetBeans

NetBeans offers support for JUnit Jupiter and the JUnit Platform since the
https://netbeans.apache.org/download/nb100/nb100.html[Apache NetBeans 10.0 release].

For more information consult the JUnit 5 section of the
https://netbeans.apache.org/download/nb100/index.html#_junit_5[Apache NetBeans 10.0
release notes].

[[running-tests-ide-vscode]]
==== Visual Studio Code

https://code.visualstudio.com/[Visual Studio Code] supports JUnit Jupiter and the JUnit
Platform via the
https://marketplace.visualstudio.com/items?itemName=vscjava.vscode-java-test[Java Test
Runner] extension which is installed by default as part of the
https://marketplace.visualstudio.com/items?itemName=vscjava.vscode-java-pack[Java
Extension Pack].

For more information consult the _Testing_ section of the
https://code.visualstudio.com/docs/languages/java#_testing[Java in Visual Studio Code]
documentation.

[[running-tests-ide-other]]
==== Other IDEs

If you are using an editor or IDE other than one of those listed in the previous sections,
the JUnit team provides two alternative solutions to assist you in using JUnit 5. You can
use the <<running-tests-console-launcher>> manually -- for example, from the command line
-- or execute tests with a <<running-tests-junit-platform-runner,JUnit 4 based Runner>> if
your IDE has built-in support for JUnit 4.

[[running-tests-build]]
=== Build Support

[[running-tests-build-gradle]]
==== Gradle

[WARNING]
.The JUnit Platform Gradle Plugin has been discontinued
====
The `junit-platform-gradle-plugin` developed by the JUnit team was deprecated in JUnit
Platform 1.2 and discontinued in 1.3. Please switch to Gradle's standard `test` task.
====

Starting with https://docs.gradle.org/4.6/release-notes.html[version 4.6], Gradle provides
https://docs.gradle.org/current/userguide/java_testing.html#using_junit5[native support]
for executing tests on the JUnit Platform. To enable it, you just need to specify
`useJUnitPlatform()` within a `test` task declaration in `build.gradle`:

[source,groovy,indent=0]
[subs=attributes+]
----
test {
	useJUnitPlatform()
}
----

Filtering by <<running-tests-tags, tags>>,
<<running-tests-tag-expressions, tag expressions>>, or engines is also supported:

[source,groovy,indent=0]
[subs=attributes+]
----
test {
	useJUnitPlatform {
		includeTags("fast", "smoke & feature-a")
		// excludeTags("slow", "ci")
		includeEngines("junit-jupiter")
		// excludeEngines("junit-vintage")
	}
}
----

Please refer to the
https://docs.gradle.org/current/userguide/java_plugin.html#sec:java_test[official Gradle documentation]
for a comprehensive list of options.

[[running-tests-build-gradle-config-params]]
===== Configuration Parameters

The standard Gradle `test` task currently does not provide a dedicated DSL to set JUnit
Platform <<running-tests-config-params, configuration parameters>> to influence test
discovery and execution. However, you can provide configuration parameters within the
build script via system properties (as shown below) or via the
`junit-platform.properties` file.

[source,groovy,indent=0]
----
test {
	// ...
	systemProperty("junit.jupiter.conditions.deactivate", "*")
	systemProperty("junit.jupiter.extensions.autodetection.enabled", true)
	systemProperty("junit.jupiter.testinstance.lifecycle.default", "per_class")
	// ...
}
----

[[running-tests-build-gradle-engines-configure]]
===== Configuring Test Engines

In order to run any tests at all, a `TestEngine` implementation must be on the classpath.

To configure support for JUnit Jupiter based tests, configure a `testImplementation` dependency
on the dependency-aggregating JUnit Jupiter artifact similar to the following.

[source,groovy,indent=0]
[subs=attributes+]
----
dependencies {
	testImplementation("org.junit.jupiter:junit-jupiter:{jupiter-version}")
}
----

The JUnit Platform can run JUnit 4 based tests as long as you configure a `testImplementation`
dependency on JUnit 4 and a `testRuntimeOnly` dependency on the JUnit Vintage `TestEngine`
implementation similar to the following.

[source,groovy,indent=0]
[subs=attributes+]
----
dependencies {
	testImplementation("junit:junit:{junit4-version}")
	testRuntimeOnly("org.junit.vintage:junit-vintage-engine:{vintage-version}")
}
----

[[running-tests-build-gradle-logging]]
===== Configuring Logging (optional)

JUnit uses the Java Logging APIs in the `java.util.logging` package (a.k.a. _JUL_) to
emit warnings and debug information. Please refer to the official documentation of
`{LogManager}` for configuration options.

Alternatively, it's possible to redirect log messages to other logging frameworks such as
{Log4j} or {Logback}. To use a logging framework that provides a custom implementation of
`{LogManager}`, set the `java.util.logging.manager` system property to the _fully
qualified class name_ of the `{LogManager}` implementation to use. The example below
demonstrates how to configure Log4j{nbsp}2.x (see {Log4j_JDK_Logging_Adapter} for
details).

[source,groovy,indent=0]
[subs=attributes+]
----
test {
	systemProperty("java.util.logging.manager", "org.apache.logging.log4j.jul.LogManager")
}
----

Other logging frameworks provide different means to redirect messages logged using
`java.util.logging`. For example, for {Logback} you can use the
https://www.slf4j.org/legacy.html#jul-to-slf4j[JUL to SLF4J Bridge] by adding an
additional dependency to the runtime classpath.

[[running-tests-build-maven]]
==== Maven

[WARNING]
.The JUnit Platform Maven Surefire Provider has been discontinued
====
The `junit-platform-surefire-provider`, which was originally developed by the JUnit team,
was deprecated in JUnit Platform 1.3 and discontinued in 1.4. Please use Maven Surefire's
native support instead.
====

Starting with https://issues.apache.org/jira/browse/SUREFIRE-1330[version 2.22.0], Maven
Surefire and Maven Failsafe provide
https://maven.apache.org/surefire/maven-surefire-plugin/examples/junit-platform.html[native support]
for executing tests on the JUnit Platform. The `pom.xml` file in the
`{junit5-jupiter-starter-maven}` project demonstrates how to use the Maven Surefire plugin
and can serve as a starting point for configuring your Maven build.

[[running-tests-build-maven-engines-configure]]
===== Configuring Test Engines

In order to have Maven Surefire or Maven Failsafe run any tests at all, at least one
`TestEngine` implementation must be added to the test classpath.

To configure support for JUnit Jupiter based tests, configure `test` scoped dependencies
on the JUnit Jupiter API and the JUnit Jupiter `TestEngine` implementation similar to the
following.

[source,xml,indent=0]
[subs=attributes+]
----
	<!-- ... -->
	<dependencies>
		<!-- ... -->
		<dependency>
			<groupId>org.junit.jupiter</groupId>
			<artifactId>junit-jupiter</artifactId>
			<version>{jupiter-version}</version>
			<scope>test</scope>
		</dependency>
		<!-- ... -->
	</dependencies>
	<build>
		<plugins>
			<plugin>
				<artifactId>maven-surefire-plugin</artifactId>
				<version>{surefire-version}</version>
			</plugin>
			<plugin>
				<artifactId>maven-failsafe-plugin</artifactId>
				<version>{surefire-version}</version>
			</plugin>
		</plugins>
	</build>
	<!-- ... -->
----

Maven Surefire and Maven Failsafe can run JUnit 4 based tests alongside Jupiter tests as
long as you configure `test` scoped dependencies on JUnit 4 and the JUnit Vintage
`TestEngine` implementation similar to the following.

[source,xml,indent=0]
[subs=attributes+]
----
	<!-- ... -->
	<dependencies>
		<!-- ... -->
		<dependency>
			<groupId>junit</groupId>
			<artifactId>junit</artifactId>
			<version>{junit4-version}</version>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>org.junit.vintage</groupId>
			<artifactId>junit-vintage-engine</artifactId>
			<version>{vintage-version}</version>
			<scope>test</scope>
		</dependency>
		<!-- ... -->
	</dependencies>
	<!-- ... -->
	<build>
		<plugins>
			<plugin>
				<artifactId>maven-surefire-plugin</artifactId>
				<version>{surefire-version}</version>
			</plugin>
			<plugin>
				<artifactId>maven-failsafe-plugin</artifactId>
				<version>{surefire-version}</version>
			</plugin>
		</plugins>
	</build>
	<!-- ... -->
----

[[running-tests-build-maven-filter-test-class-names]]
===== Filtering by Test Class Names

The Maven Surefire Plugin will scan for test classes whose fully qualified names match
the following patterns.

- `+**/Test*.java+`
- `+**/*Test.java+`
- `+**/*Tests.java+`
- `+**/*TestCase.java+`

Moreover, it will exclude all nested classes (including static member classes) by default.

Note, however, that you can override this default behavior by configuring explicit
`include` and `exclude` rules in your `pom.xml` file. For example, to keep Maven Surefire
from excluding static member classes, you can override its exclude rules as follows.

[source,xml,indent=0]
[subs=attributes+]
.Overriding exclude rules of Maven Surefire
----
	<!-- ... -->
	<build>
		<plugins>
			<plugin>
				<artifactId>maven-surefire-plugin</artifactId>
				<version>{surefire-version}</version>
				<configuration>
					<excludes>
						<exclude/>
					</excludes>
				</configuration>
			</plugin>
		</plugins>
	</build>
	<!-- ... -->
----

Please see the
https://maven.apache.org/surefire/maven-surefire-plugin/examples/inclusion-exclusion.html[Inclusions and Exclusions of Tests]
documentation for Maven Surefire for details.

[[running-tests-build-maven-filter-tags]]
===== Filtering by Tags

You can filter tests by <<running-tests-tags, tags>> or
<<running-tests-tag-expressions, tag expressions>> using the following configuration
properties.

- to include _tags_ or _tag expressions_, use `groups`.
- to exclude _tags_ or _tag expressions_, use `excludedGroups`.

[source,xml,indent=0]
[subs=attributes+]
----
	<!-- ... -->
	<build>
		<plugins>
			<plugin>
				<artifactId>maven-surefire-plugin</artifactId>
				<version>{surefire-version}</version>
				<configuration>
					<groups>acceptance | !feature-a</groups>
					<excludedGroups>integration, regression</excludedGroups>
				</configuration>
			</plugin>
		</plugins>
	</build>
	<!-- ... -->
----

[[running-tests-build-maven-config-params]]
===== Configuration Parameters

You can set JUnit Platform <<running-tests-config-params, configuration parameters>> to
influence test discovery and execution by declaring the `configurationParameters`
property and providing key-value pairs using the Java `Properties` file syntax (as shown
below) or via the `junit-platform.properties` file.

[source,xml,indent=0]
[subs=attributes+]
----
	<!-- ... -->
	<build>
		<plugins>
			<plugin>
				<artifactId>maven-surefire-plugin</artifactId>
				<version>{surefire-version}</version>
				<configuration>
					<properties>
						<configurationParameters>
							junit.jupiter.conditions.deactivate = *
							junit.jupiter.extensions.autodetection.enabled = true
							junit.jupiter.testinstance.lifecycle.default = per_class
						</configurationParameters>
					</properties>
				</configuration>
			</plugin>
		</plugins>
	</build>
	<!-- ... -->
----

[[running-tests-build-ant]]
==== Ant

Starting with version `1.10.3` of link:https://ant.apache.org/[Ant], a new
link:https://ant.apache.org/manual/Tasks/junitlauncher.html[`junitlauncher`] task has
been introduced to provide native support for launching tests on the JUnit Platform. The
`junitlauncher` task is solely responsible for launching the JUnit Platform and passing
it the selected collection of tests. The JUnit Platform then delegates to registered test
engines to discover and execute the tests.

The `junitlauncher` task attempts to align as close as possible with native Ant
constructs such as
link:https://ant.apache.org/manual/Types/resources.html#collection[resource collections]
for allowing users to select the tests that they want executed by test engines. This
gives the task a consistent and natural feel when compared to many other core Ant tasks.

Starting with version `1.10.6` of Ant, the `junitlauncher` task supports
link:https://ant.apache.org/manual/Tasks/junitlauncher.html#fork[forking the tests in a separate JVM].

The `build.xml` file in the `{junit5-jupiter-starter-ant}` project demonstrates how to use
the task and can serve as a starting point.

===== Basic Usage

The following example demonstrates how to configure the `junitlauncher` task to select a
single test class (i.e., `org.myapp.test.MyFirstJUnit5Test`).

[source,xml,indent=0]
----
	<path id="test.classpath">
		<!-- The location where you have your compiled classes -->
		<pathelement location="${build.classes.dir}" />
	</path>

	<!-- ... -->

	<junitlauncher>
		<classpath refid="test.classpath" />
		<test name="org.myapp.test.MyFirstJUnit5Test" />
	</junitlauncher>
----

The `test` element allows you to specify a single test class that you want to be selected
and executed. The `classpath` element allows you to specify the classpath to be used to
launch the JUnit Platform. This classpath will also be used to locate test classes that
are part of the execution.

The following example demonstrates how to configure the `junitlauncher` task to select
test classes from multiple locations.

[source,xml,indent=0]
----
	<path id="test.classpath">
		<!-- The location where you have your compiled classes -->
		<pathelement location="${build.classes.dir}" />
	</path>
	<!-- ... -->
	<junitlauncher>
		<classpath refid="test.classpath" />
		<testclasses outputdir="${output.dir}">
			<fileset dir="${build.classes.dir}">
				<include name="org/example/**/demo/**/" />
			</fileset>
			<fileset dir="${some.other.dir}">
				<include name="org/myapp/**/" />
			</fileset>
		</testclasses>
	</junitlauncher>
----

In the above example, the `testclasses` element allows you to select multiple test
classes that reside in different locations.

For further details on usage and configuration options please refer to the official Ant
documentation for the
link:https://ant.apache.org/manual/Tasks/junitlauncher.html[`junitlauncher` task].

[[running-tests-console-launcher]]
=== Console Launcher

The `{ConsoleLauncher}` is a command-line Java application that lets you launch the JUnit
Platform from the console. For example, it can be used to run JUnit Vintage and JUnit
Jupiter tests and print test execution results to the console.

An executable `junit-platform-console-standalone-{platform-version}.jar` with all
dependencies included is published in the {Maven_Central} repository under the
https://repo1.maven.org/maven2/org/junit/platform/junit-platform-console-standalone[junit-platform-console-standalone]
directory. It includes the following dependencies:

include::{standaloneConsoleLauncherShadowedArtifactsFile}[]

You can https://docs.oracle.com/javase/tutorial/deployment/jar/run.html[run] the
standalone `ConsoleLauncher` as shown below.

[source,console,subs=attributes+]
----
$ java -jar junit-platform-console-standalone-{platform-version}.jar <OPTIONS>

├─ JUnit Vintage
│  └─ example.JUnit4Tests
│     └─ standardJUnit4Test ✔
└─ JUnit Jupiter
   ├─ StandardTests
   │  ├─ succeedingTest() ✔
   │  └─ skippedTest() ↷ for demonstration purposes
   └─ A special test case
      ├─ Custom test name containing spaces ✔
      ├─ ╯°□°)╯ ✔
      └─ 😱 ✔

Test run finished after 64 ms
[         5 containers found      ]
[         0 containers skipped    ]
[         5 containers started    ]
[         0 containers aborted    ]
[         5 containers successful ]
[         0 containers failed     ]
[         6 tests found           ]
[         1 tests skipped         ]
[         5 tests started         ]
[         0 tests aborted         ]
[         5 tests successful      ]
[         0 tests failed          ]
----

You can also run the standalone `ConsoleLauncher` as shown below (for example, to include
all jars in a directory):

[source,console,subs=attributes+]
----
$ java -cp classes:testlib/* org.junit.platform.console.ConsoleLauncher <OPTIONS>
----

.Exit Code
NOTE: The `{ConsoleLauncher}` exits with a status code of `1` if any containers or tests
failed. If no tests are discovered and the `--fail-if-no-tests` command-line option is
supplied, the `ConsoleLauncher` exits with a status code of `2`. Otherwise the exit code
is `0`.

[[running-tests-console-launcher-options]]
==== Options

----
include::{consoleLauncherOptionsFile}[]
----

[[running-tests-console-launcher-argument-files]]
==== Argument Files (@-files)

On some platforms you may run into system limitations on the length of a command line
when creating a command line with lots of options or with long arguments.

Since version 1.3, the `ConsoleLauncher` supports _argument files_, also known as
_@-files_. Argument files are files that themselves contain arguments to be passed to the
command. When the underlying https://github.com/remkop/picocli[picocli] command line
parser encounters an argument beginning with the character `@`, it expands the contents
of that file into the argument list.

The arguments within a file can be separated by spaces or newlines. If an argument
contains embedded whitespace, the whole argument should be wrapped in double or single
quotes -- for example, `"-f=My Files/Stuff.java"`.

If the argument file does not exist or cannot be read, the argument will be treated
literally and will not be removed. This will likely result in an "unmatched argument"
error message. You can troubleshoot such errors by executing the command with the
`picocli.trace` system property set to `DEBUG`.

Multiple _@-files_ may be specified on the command line. The specified path may be
relative to the current directory or absolute.

You can pass a real parameter with an initial `@` character by escaping it with an
additional `@` symbol. For example, `@@somearg` will become `@somearg` and will not be
subject to expansion.


[[running-tests-junit-platform-runner]]
=== Using JUnit 4 to run the JUnit Platform

[WARNING]
.The `JUnitPlatform` runner has been deprecated
====
The `JUnitPlatform` runner was developed by the JUnit team as an interim solution for
running test suites and tests on the JUnit Platform in a JUnit 4 environment.

In recent years, all mainstream build tools and IDEs provide built-in support for running
tests directly on the JUnit Platform.

In addition, the introduction of `@Suite` support provided by the
`junit-platform-suite-engine` module makes the `JUnitPlatform` runner obsolete. See
<<junit-platform-suite-engine>> for details.

The `JUnitPlatform` runner and `@UseTechnicalNames` annotation have therefore been
deprecated in JUnit Platform 1.8 and will be removed in JUnit Platform 2.0.

If you are using the `JUnitPlatform` runner, please migrate to the `@Suite` support.
====

The `JUnitPlatform` runner is a JUnit 4 based `Runner` which enables you to run any test
whose programming model is supported on the JUnit Platform in a JUnit 4 environment --
for example, a JUnit Jupiter test class.

Annotating a class with `@RunWith(JUnitPlatform.class)` allows it to be run with IDEs and
build systems that support JUnit 4 but do not yet support the JUnit Platform directly.

NOTE: Since the JUnit Platform has features that JUnit 4 does not have, the runner is
only able to support a subset of the JUnit Platform functionality, especially with regard
to reporting (see <<running-tests-junit-platform-runner-technical-names>>).

[[running-tests-junit-platform-runner-setup]]
==== Setup

You need the following artifacts and their dependencies on the classpath. See
<<dependency-metadata>> for details regarding group IDs, artifact IDs, and versions.

[[running-tests-junit-platform-runner-setup-explicit-dependencies]]
===== Explicit Dependencies

* `junit-platform-runner` in _test_ scope: location of the `JUnitPlatform` runner
* `junit-{junit4-version}.jar` in _test_ scope: to run tests using JUnit 4
* `junit-jupiter-api` in _test_ scope: API for writing tests using JUnit Jupiter,
  including `@Test`, etc.
* `junit-jupiter-engine` in _test runtime_ scope: implementation of the `TestEngine` API
  for JUnit Jupiter

[[running-tests-junit-platform-runner-setup-transitive-dependencies]]
===== Transitive Dependencies

* `junit-platform-suite-api` in _test_ scope
* `junit-platform-suite-commons` in _test_ scope
* `junit-platform-launcher` in _test_ scope
* `junit-platform-engine` in _test_ scope
* `junit-platform-commons` in _test_ scope
* `opentest4j` in _test_ scope

[[running-tests-junit-platform-runner-technical-names]]
==== Display Names vs. Technical Names

To define a custom _display name_ for the class run via `@RunWith(JUnitPlatform.class)`
annotate the class with `@SuiteDisplayName` and provide a custom value.

By default, _display names_ will be used for test artifacts; however, when the
`JUnitPlatform` runner is used to execute tests with a build tool such as Gradle or
Maven, the generated test report often needs to include the _technical names_ of test
artifacts — for example, fully qualified class names — instead of shorter display names
like the simple name of a test class or a custom display name containing special
characters. To enable technical names for reporting purposes, declare the
`@UseTechnicalNames` annotation alongside `@RunWith(JUnitPlatform.class)`.

Note that the presence of `@UseTechnicalNames` overrides any custom display name
configured via `@SuiteDisplayName`.

[[running-tests-junit-platform-runner-single-test]]
==== Single Test Class

One way to use the `JUnitPlatform` runner is to annotate a test class with
`@RunWith(JUnitPlatform.class)` directly. Please note that the test methods in the
following example are annotated with `org.junit.jupiter.api.Test` (JUnit Jupiter), not
`org.junit.Test` (JUnit 4). Moreover, in this case the test class must be `public`;
otherwise, some IDEs and build tools might not recognize it as a JUnit 4 test class.

[source,java,indent=0]
----
include::{testDir}/example/JUnitPlatformClassDemo.java[tags=user_guide]
----

[[running-tests-junit-platform-runner-test-suite]]
==== Test Suite

If you have multiple test classes you can create a test suite as can be seen in the
following example.

[source,java,indent=0]
----
include::{testDir}/example/JUnitPlatformSuiteDemo.java[tags=user_guide]
----

The `JUnitPlatformSuiteDemo` will discover and run all tests in the `example` package and
its subpackages. By default, it will only include test classes whose names either begin
with `Test` or end with `Test` or `Tests`.

.Additional Configuration Options
NOTE: There are more configuration options for discovering and filtering tests than just
`@SelectPackages`. Please consult the Javadoc of the `{suite-api-package}` package for
further details.

WARNING: Test classes and suites annotated with `@RunWith(JUnitPlatform.class)`
**cannot** be executed directly on the JUnit Platform (or as a "JUnit 5" test as
documented in some IDEs). Such classes and suites can only be executed using JUnit 4
infrastructure.

[[running-tests-config-params]]
=== Configuration Parameters

In addition to instructing the platform which test classes and test engines to include,
which packages to scan, etc., it is sometimes necessary to provide additional custom
configuration parameters that are specific to a particular test engine, listener, or
registered extension. For example, the JUnit Jupiter `TestEngine` supports _configuration
parameters_ for the following use cases.

- <<writing-tests-test-instance-lifecycle-changing-default>>
- <<extensions-registration-automatic-enabling>>
- <<extensions-conditions-deactivation>>
- <<writing-tests-display-name-generator-default>>

_Configuration Parameters_ are text-based key-value pairs that can be supplied to test
engines running on the JUnit Platform via one of the following mechanisms.

1. The `configurationParameter()` and `configurationParameters()` methods in the
  `LauncherDiscoveryRequestBuilder` which is used to build a request supplied to the
  <<launcher-api, `Launcher` API>>. When running tests via one of the tools provided
  by the JUnit Platform you can specify configuration parameters as follows:
  * <<running-tests-console-launcher,Console Launcher>>: use the `--config`
    command-line option.
  * <<running-tests-build-gradle-config-params,Gradle>>: use the
    `systemProperty` or `systemProperties` DSL.
  * <<running-tests-build-maven-config-params,Maven Surefire provider>>: use the
    `configurationParameters` property.
2. JVM system properties.
3. The JUnit Platform configuration file: a file named `junit-platform.properties` in the
  root of the class path that follows the syntax rules for a Java `Properties` file.

NOTE: Configuration parameters are looked up in the exact order defined above.
Consequently, configuration parameters supplied directly to the `Launcher` take
precedence over those supplied via system properties and the configuration file.
Similarly, configuration parameters supplied via system properties take precedence over
those supplied via the configuration file.

[[running-tests-config-params-deactivation-pattern]]
==== Pattern Matching Syntax

This section describes the pattern matching syntax that is applied to the _configuration
parameters_ used for the following features.

- <<extensions-conditions-deactivation>>
- <<launcher-api-listeners-custom-deactivation>>

If the value for the given _configuration parameter_ consists solely of an asterisk
(`+*+`), the pattern will match against all candidate classes. Otherwise, the value will
be treated as a comma-separated list of patterns where each pattern will be matched
against the fully qualified class name (_FQCN_) of each candidate class. Any dot (`.`) in
a pattern will match against a dot (`.`) or a dollar sign (`$`) in a FQCN. Any asterisk
(`+*+`) will match against one or more characters in a FQCN. All other characters in a
pattern will be matched one-to-one against a FQCN.

Examples:

- `+*+`: matches all candidate classes.
- `+org.junit.*+`: matches all candidate classes under the `org.junit` base package and
  any of its subpackages.
- `+*.MyCustomImpl+`: matches every candidate class whose simple class name is exactly
  `MyCustomImpl`.
- `+*System*+`: matches every candidate class whose FQCN contains `System`.
- `+*System*+, +*Unit*+`: matches every candidate class whose FQCN contains `System` or
  `Unit`.
- `org.example.MyCustomImpl`: matches the candidate class whose FQCN is exactly
  `org.example.MyCustomImpl`.
- `org.example.MyCustomImpl, org.example.TheirCustomImpl`: matches candidate classes whose
  FQCN is exactly `org.example.MyCustomImpl` or `org.example.TheirCustomImpl`.

[[running-tests-tags]]
=== Tags

Tags are a JUnit Platform concept for marking and filtering tests. The programming model
for adding tags to containers and tests is defined by the testing framework. For example,
in JUnit Jupiter based tests, the `@Tag` annotation (see
<<writing-tests-tagging-and-filtering>>) should be used. For JUnit 4 based tests, the
Vintage engine maps `@Category` annotations to tags (see
<<migrating-from-junit4-categories-support>>). Other testing frameworks may define their
own annotation or other means for users to specify tags.

[[running-tests-tag-syntax-rules]]
==== Syntax Rules for Tags

Regardless how a tag is specified, the JUnit Platform enforces the following rules:

* A tag must not be `null` or _blank_.
* A _trimmed_ tag must not contain whitespace.
* A _trimmed_ tag must not contain ISO control characters.
* A _trimmed_ tag must not contain any of the following _reserved characters_.
- `,`: _comma_
- `(`: _left parenthesis_
- `)`: _right parenthesis_
- `&`: _ampersand_
- `|`: _vertical bar_
- `!`: _exclamation point_

NOTE: In the above context, "trimmed" means that leading and trailing whitespace
characters have been removed.

[[running-tests-tag-expressions]]
==== Tag Expressions

Tag expressions are boolean expressions with the operators `!`, `&` and `|`. In addition,
`(` and `)` can be used to adjust for operator precedence.

Two special expressions are supported, `any()` and `none()`, which select all tests _with_
any tags at all, and all tests _without_ any tags, respectively.
These special expressions may be combined with other expressions just like normal tags.

.Operators (in descending order of precedence)
|===
| Operator | Meaning | Associativity

| `!`      | not     | right
| `&`      | and     | left
| `\|`     | or      | left
|===

If you are tagging your tests across multiple dimensions, tag expressions help you to
select which tests to execute. When tagging by test type (e.g., _micro_, _integration_,
_end-to-end_) and feature (e.g., *product*, *catalog*, *shipping*), the following tag
expressions can be useful.

[%header,cols="40,60"]
|===
| Tag Expression
| Selection

| +product+
| all tests for *product*

| +catalog \| shipping+
| all tests for *catalog* plus all tests for *shipping*

| +catalog & shipping+
| all tests for the intersection between *catalog* and *shipping*

| +product & !end-to-end+
| all tests for *product*, but not the _end-to-end_ tests

| +(micro \| integration) & (product \| shipping)+
| all _micro_ or _integration_ tests for *product* or *shipping*
|===

[[running-tests-capturing-output]]
=== Capturing Standard Output/Error

Since version 1.3, the JUnit Platform provides opt-in support for capturing output
printed to `System.out` and `System.err`. To enable it, set the
`junit.platform.output.capture.stdout` and/or `junit.platform.output.capture.stderr`
<<running-tests-config-params, configuration parameter>> to `true`. In addition, you may
configure the maximum number of buffered bytes to be used per executed test or container
using `junit.platform.output.capture.maxBuffer`.

If enabled, the JUnit Platform captures the corresponding output and publishes it as a
report entry using the `stdout` or `stderr` keys to all registered
`{TestExecutionListener}` instances immediately before reporting the test or container as
finished.

Please note that the captured output will only contain output emitted by the thread that
was used to execute a container or test. Any output by other threads will be omitted
because particularly when
<<writing-tests-parallel-execution, executing tests in parallel>> it would be impossible
to attribute it to a specific test or container.

[[running-tests-listeners]]
=== Using Listeners

The JUnit Platform provides the following listener APIs that allow JUnit, third parties,
and custom user code to react to events fired at various points during the discovery and
execution of a `TestPlan`.

* `{LauncherSessionListener}`: receives events when a `{LauncherSession}` is opened and
  closed.
* `{LauncherDiscoveryListener}`: receives events that occur during test discovery.
* `{TestExecutionListener}`: receives events that occur during test execution.

The `LauncherSessionListener` API is typically implemented by build tools or IDEs and
registered automatically for you in order to support some feature of the build tool or IDE.

The `LauncherDiscoveryListener` and `TestExecutionListener` APIs are often implemented in
order to produce some form of report or to display a graphical representation of the test
plan in an IDE. Such listeners may be implemented and automatically registered by a build
tool or IDE, or they may be included in a third-party library – potentially registered
for you automatically. You can also implement and register your own listeners.

For details on registering and configuring listeners, see the following sections of this
guide.

* <<launcher-api-launcher-session-listeners-custom>>
* <<launcher-api-launcher-discovery-listeners-custom>>
* <<launcher-api-listeners-custom>>
* <<launcher-api-listeners-config>>
* <<launcher-api-listeners-custom-deactivation>>

The JUnit Platform provides the following listeners which you may wish to use with your
test suite.

<<running-tests-listeners-flight-recorder>> ::
  `FlightRecordingExecutionListener` and `FlightRecordingDiscoveryListener` that generate
  Java Flight Recorder events during test discovery and execution.

`{LegacyXmlReportGeneratingListener}` ::
  `TestExecutionListener` that generates XML reports compatible with the de facto
  standard for JUnit 4 based test reports. See <<junit-platform-reporting>> for
  details.

`{LoggingListener}` ::
  `TestExecutionListener` for logging informational messages for all events via a
  `BiConsumer` that consumes `Throwable` and `Supplier<String>`.

`{SummaryGeneratingListener}` ::
  `TestExecutionListener` that generates a summary of the test execution which can be
  printed via a `PrintWriter`.

`{UniqueIdTrackingListener}` ::
  `TestExecutionListener` that that tracks the unique IDs of all tests that were skipped
  or executed during the execution of the `TestPlan` and generates a file containing the
  unique IDs once execution of the `TestPlan` has finished.

[[running-tests-listeners-flight-recorder]]
==== Flight Recorder Support

Since version 1.7, the JUnit Platform provides opt-in support for generating Flight
Recorder events. https://openjdk.java.net/jeps/328[JEP 328] describes the Java Flight
Recorder (JFR) as:

NOTE: Flight Recorder records events originating from applications, the JVM and the OS.
Events are stored in a single file that can be attached to bug reports and examined by
support engineers, allowing after-the-fact analysis of issues in the period leading up
to a problem.

In order to record Flight Recorder events generated while running tests, you need to:

1. Ensure that you are using either Java 8 Update 262 or higher or Java 11 or later.
2. Provide the `org.junit.platform.jfr` module (`junit-platform-jfr-{platform-version}.jar`)
   on the class-path or module-path at test runtime.
3. Start flight recording when launching a test run. Flight Recorder can be started via
   java command line option:

   -XX:StartFlightRecording:filename=...

Please consult the manual of your build tool for the appropriate commands.

To analyze the recorded events, use the
https://docs.oracle.com/en/java/javase/14/docs/specs/man/jfr.html[jfr]
command line tool shipped with recent JDKs or open the recording file with
https://jdk.java.net/jmc/[JDK Mission Control].

WARNING: Flight Recorder support is currently an _experimental_ feature. You're invited to
give it a try and provide feedback to the JUnit team so they can improve and eventually
<<api-evolution, promote>> this feature.