Google Git
Sign in
third-party-mirror / junit5 / refs/heads/main / . / documentation / src / docs / asciidoc / user-guide / writing-tests.adoc
blob: 88028d9b8306fa17b2ee12ce11e6bb6a67e6ecb3 [file] [log] [blame] [edit]
[[writing-tests]]
== Writing Tests
The following example provides a glimpse at the minimum requirements for writing a test in
JUnit Jupiter. Subsequent sections of this chapter will provide further details on all
available features.
[source,java,indent=0]
.A first test case
----
include::{testDir}/example/MyFirstJUnitJupiterTests.java[tags=user_guide]
----
[[writing-tests-annotations]]
=== Annotations
JUnit Jupiter supports the following annotations for configuring tests and extending the
framework.
Unless otherwise stated, all core annotations are located in the `{api-package}` package
in the `junit-jupiter-api` module.
[cols="20,80"]
|===
| Annotation | Description
| `@Test` | Denotes that a method is a test method. Unlike JUnit 4's `@Test` annotation, this annotation does not declare any attributes, since test extensions in JUnit Jupiter operate based on their own dedicated annotations. Such methods are _inherited_ unless they are _overridden_.
| `@ParameterizedTest` | Denotes that a method is a <<writing-tests-parameterized-tests, parameterized test>>. Such methods are _inherited_ unless they are _overridden_.
| `@RepeatedTest` | Denotes that a method is a test template for a <<writing-tests-repeated-tests, repeated test>>. Such methods are _inherited_ unless they are _overridden_.
| `@TestFactory` | Denotes that a method is a test factory for <<writing-tests-dynamic-tests, dynamic tests>>. Such methods are _inherited_ unless they are _overridden_.
| `@TestTemplate` | Denotes that a method is a <<writing-tests-test-templates, template for test cases>> designed to be invoked multiple times depending on the number of invocation contexts returned by the registered <<extensions-test-templates, providers>>. Such methods are _inherited_ unless they are _overridden_.
| `@TestClassOrder` | Used to configure the <<writing-tests-test-execution-order-classes, test class execution order>> for `@Nested` test classes in the annotated test class. Such annotations are _inherited_.
| `@TestMethodOrder` | Used to configure the <<writing-tests-test-execution-order-methods, test method execution order>> for the annotated test class; similar to JUnit 4's `@FixMethodOrder`. Such annotations are _inherited_.
| `@TestInstance` | Used to configure the <<writing-tests-test-instance-lifecycle, test instance lifecycle>> for the annotated test class. Such annotations are _inherited_.
| `@DisplayName` | Declares a custom <<writing-tests-display-names,display name>> for the test class or test method. Such annotations are not _inherited_.
| `@DisplayNameGeneration` | Declares a custom <<writing-tests-display-name-generator,display name generator>> for the test class. Such annotations are _inherited_.
| `@BeforeEach` | Denotes that the annotated method should be executed _before_ *each* `@Test`, `@RepeatedTest`, `@ParameterizedTest`, or `@TestFactory` method in the current class; analogous to JUnit 4's `@Before`. Such methods are _inherited_ unless they are _overridden_.
| `@AfterEach` | Denotes that the annotated method should be executed _after_ *each* `@Test`, `@RepeatedTest`, `@ParameterizedTest`, or `@TestFactory` method in the current class; analogous to JUnit 4's `@After`. Such methods are _inherited_ unless they are _overridden_.
| `@BeforeAll` | Denotes that the annotated method should be executed _before_ *all* `@Test`, `@RepeatedTest`, `@ParameterizedTest`, and `@TestFactory` methods in the current class; analogous to JUnit 4's `@BeforeClass`. Such methods are _inherited_ (unless they are _hidden_ or _overridden_) and must be `static` (unless the "per-class" <<writing-tests-test-instance-lifecycle, test instance lifecycle>> is used).
| `@AfterAll` | Denotes that the annotated method should be executed _after_ *all* `@Test`, `@RepeatedTest`, `@ParameterizedTest`, and `@TestFactory` methods in the current class; analogous to JUnit 4's `@AfterClass`. Such methods are _inherited_ (unless they are _hidden_ or _overridden_) and must be `static` (unless the "per-class" <<writing-tests-test-instance-lifecycle, test instance lifecycle>> is used).
| `@Nested` | Denotes that the annotated class is a non-static <<writing-tests-nested,nested test class>>. `@BeforeAll` and `@AfterAll` methods cannot be used directly in a `@Nested` test class unless the "per-class" <<writing-tests-test-instance-lifecycle, test instance lifecycle>> is used. Such annotations are not _inherited_.
| `@Tag` | Used to declare <<writing-tests-tagging-and-filtering,tags for filtering tests>>, either at the class or method level; analogous to test groups in TestNG or Categories in JUnit 4. Such annotations are _inherited_ at the class level but not at the method level.
| `@Disabled` | Used to <<writing-tests-disabling,disable>> a test class or test method; analogous to JUnit 4's `@Ignore`. Such annotations are not _inherited_.
| `@Timeout` | Used to fail a test, test factory, test template, or lifecycle method if its execution exceeds a given duration. Such annotations are _inherited_.
| `@ExtendWith` | Used to <<extensions-registration-declarative,register extensions declaratively>>. Such annotations are _inherited_.
| `@RegisterExtension` | Used to <<extensions-registration-programmatic,register extensions programmatically>> via fields. Such fields are _inherited_ unless they are _shadowed_.
| `@TempDir` | Used to supply a <<writing-tests-built-in-extensions-TempDirectory,temporary directory>> via field injection or parameter injection in a lifecycle method or test method; located in the `org.junit.jupiter.api.io` package.
|===
WARNING: Some annotations may currently be _experimental_. Consult the table in
<<api-evolution-experimental-apis>> for details.
[[writing-tests-meta-annotations]]
==== Meta-Annotations and Composed Annotations
JUnit Jupiter annotations can be used as _meta-annotations_. That means that you can
define your own _composed annotation_ that will automatically _inherit_ the semantics of
its meta-annotations.
For example, instead of copying and pasting `@Tag("fast")` throughout your code base (see
<<writing-tests-tagging-and-filtering>>), you can create a custom _composed annotation_
named `@Fast` as follows. `@Fast` can then be used as a drop-in replacement for
`@Tag("fast")`.
[source,java,indent=0]
----
include::{testDir}/example/Fast.java[tags=user_guide]
----
The following `@Test` method demonstrates usage of the `@Fast` annotation.
[source,java,indent=0]
----
@Fast
@Test
void myFastTest() {
// ...
}
----
You can even take that one step further by introducing a custom `@FastTest` annotation
that can be used as a drop-in replacement for `@Tag("fast")` _and_ `@Test`.
[source,java,indent=0]
----
include::{testDir}/example/FastTest.java[tags=user_guide]
----
JUnit automatically recognizes the following as a `@Test` method that is tagged with
"fast".
[source,java,indent=0]
----
@FastTest
void myFastTest() {
// ...
}
----
[[writing-tests-classes-and-methods]]
=== Test Classes and Methods
**Test Class**: any top-level class, `static` member class, or <<writing-tests-nested,
`@Nested` class>> that contains at least one _test method_.
Test classes must not be `abstract` and must have a single constructor.
**Test Method**: any instance method that is directly annotated or meta-annotated with
`@Test`, `@RepeatedTest`, `@ParameterizedTest`, `@TestFactory`, or `@TestTemplate`.
**Lifecycle Method**: any method that is directly annotated or meta-annotated with
`@BeforeAll`, `@AfterAll`, `@BeforeEach`, or `@AfterEach`.
Test methods and lifecycle methods may be declared locally within the current test class,
inherited from superclasses, or inherited from interfaces (see
<<writing-tests-test-interfaces-and-default-methods>>). In addition, test methods and
lifecycle methods must not be `abstract` and must not return a value (except `@TestFactory`
methods which are required to return a value).
[NOTE]
.Class and method visibility
====
Test classes, test methods, and lifecycle methods are not required to be `public`, but
they must _not_ be `private`.
It is generally recommended to omit the `public` modifier for test classes, test methods,
and lifecycle methods unless there is a technical reason for doing so – for example, when
a test class is extended by a test class in another package. Another technical reason for
making classes and methods `public` is to simplify testing on the module path when using
the Java Module System.
====
The following test class demonstrates the use of `@Test` methods and all supported
lifecycle methods. For further information on runtime semantics, see
<<writing-tests-test-execution-order>> and
<<extensions-execution-order-wrapping-behavior>>.
[source,java,indent=0]
.A standard test class
----
include::{testDir}/example/StandardTests.java[tags=user_guide]
----
[[writing-tests-display-names]]
=== Display Names
Test classes and test methods can declare custom display names via `@DisplayName` -- with
spaces, special characters, and even emojis -- that will be displayed in test reports and
by test runners and IDEs.
[source,java,indent=0]
----
include::{testDir}/example/DisplayNameDemo.java[tags=user_guide]
----
[[writing-tests-display-name-generator]]
==== Display Name Generators
JUnit Jupiter supports custom display name generators that can be configured via the
`@DisplayNameGeneration` annotation. Values provided via `@DisplayName` annotations
always take precedence over display names generated by a `DisplayNameGenerator`.
Generators can be created by implementing `DisplayNameGenerator`. Here are some default
ones available in Jupiter:
[cols="20,80"]
|===
| DisplayNameGenerator | Behavior
| `Standard` | Matches the standard display name generation behavior in place since JUnit Jupiter 5.0 was released.
| `Simple` | Removes trailing parentheses for methods with no parameters.
| `ReplaceUnderscores` | Replaces underscores with spaces.
| `IndicativeSentences` | Generates complete sentences by concatenating the names of the test and the enclosing classes.
|===
Note that for `IndicativeSentences`, you can customize the separator and the
underlying generator by using `@IndicativeSentencesGeneration` as shown in the
following example.
[source,java,indent=0]
----
include::{testDir}/example/DisplayNameGeneratorDemo.java[tags=user_guide]
----
```
+-- DisplayNameGeneratorDemo [OK]
+-- A year is not supported [OK]
| +-- A negative value for year is not supported by the leap year computation. [OK]
| | +-- For example, year -1 is not supported. [OK]
| | '-- For example, year -4 is not supported. [OK]
| '-- if it is zero() [OK]
'-- A year is a leap year [OK]
+-- A year is a leap year -> if it is divisible by 4 but not by 100. [OK]
'-- A year is a leap year -> if it is one of the following years. [OK]
+-- Year 2016 is a leap year. [OK]
+-- Year 2020 is a leap year. [OK]
'-- Year 2048 is a leap year. [OK]
```
[[writing-tests-display-name-generator-default]]
==== Setting the Default Display Name Generator
You can use the `junit.jupiter.displayname.generator.default`
<<running-tests-config-params, configuration parameter>> to specify the fully qualified
class name of the `DisplayNameGenerator` you would like to use by default. Just like for
display name generators configured via the `@DisplayNameGeneration` annotation, the
supplied class has to implement the `DisplayNameGenerator` interface. The default display
name generator will be used for all tests unless the `@DisplayNameGeneration` annotation
is present on an enclosing test class or test interface. Values provided via
`@DisplayName` annotations always take precedence over display names generated by a
`DisplayNameGenerator`.
For example, to use the `ReplaceUnderscores` display name generator by default, you should
set the configuration parameter to the corresponding fully qualified class name (e.g., in
`src/test/resources/junit-platform.properties`):
[source,properties,indent=0]
----
junit.jupiter.displayname.generator.default = \
org.junit.jupiter.api.DisplayNameGenerator$ReplaceUnderscores
----
Similarly, you can specify the fully qualified name of any custom class that implements
`DisplayNameGenerator`.
[[writing-tests-display-name-generator-precedence-rules]]
In summary, the display name for a test class or method is determined according to the
following precedence rules:
1. value of the `@DisplayName` annotation, if present
2. by calling the `DisplayNameGenerator` specified in the `@DisplayNameGeneration`
annotation, if present
3. by calling the default `DisplayNameGenerator` configured via the configuration
parameter, if present
4. by calling `org.junit.jupiter.api.DisplayNameGenerator.Standard`
[[writing-tests-assertions]]
=== Assertions
JUnit Jupiter comes with many of the assertion methods that JUnit 4 has and adds a few
that lend themselves well to being used with Java 8 lambdas. All JUnit Jupiter assertions
are `static` methods in the `{Assertions}` class.
[source,java,indent=0]
----
include::{testDir}/example/AssertionsDemo.java[tags=user_guide]
----
[[writing-tests-assertions-preemptive-timeouts]]
[WARNING]
.Preemptive Timeouts with `assertTimeoutPreemptively()`
====
Contrary to <<writing-tests-declarative-timeouts, declarative timeouts>>, the various
`assertTimeoutPreemptively()` methods in the `Assertions` class execute the provided
`executable` or `supplier` in a different thread than that of the calling code. This
behavior can lead to undesirable side effects if the code that is executed within the
`executable` or `supplier` relies on `java.lang.ThreadLocal` storage.
One common example of this is the transactional testing support in the Spring Framework.
Specifically, Spring's testing support binds transaction state to the current thread (via
a `ThreadLocal`) before a test method is invoked. Consequently, if an `executable` or
`supplier` provided to `assertTimeoutPreemptively()` invokes Spring-managed components
that participate in transactions, any actions taken by those components will not be rolled
back with the test-managed transaction. On the contrary, such actions will be committed to
the persistent store (e.g., relational database) even though the test-managed transaction
is rolled back.
Similar side effects may be encountered with other frameworks that rely on
`ThreadLocal` storage.
====
[[writing-tests-assertions-kotlin]]
==== Kotlin Assertion Support
JUnit Jupiter also comes with a few assertion methods that lend themselves well to being
used in https://kotlinlang.org/[Kotlin]. All JUnit Jupiter Kotlin assertions are top-level
functions in the `org.junit.jupiter.api` package.
[source,kotlin,indent=0]
----
include::{kotlinTestDir}/example/KotlinAssertionsDemo.kt[tags=user_guide]
----
[[writing-tests-assertions-third-party]]
==== Third-party Assertion Libraries
Even though the assertion facilities provided by JUnit Jupiter are sufficient for many
testing scenarios, there are times when more power and additional functionality such as
_matchers_ are desired or required. In such cases, the JUnit team recommends the use of
third-party assertion libraries such as {AssertJ}, {Hamcrest}, {Truth}, etc. Developers
are therefore free to use the assertion library of their choice.
For example, the combination of _matchers_ and a fluent API can be used to make
assertions more descriptive and readable. However, JUnit Jupiter's `{Assertions}` class
does not provide an
https://junit.org/junit4/javadoc/latest/org/junit/Assert.html#assertThat[`assertThat()`]
method like the one found in JUnit 4's `org.junit.Assert` class which accepts a Hamcrest
https://junit.org/junit4/javadoc/latest/org/hamcrest/Matcher.html[`Matcher`]. Instead,
developers are encouraged to use the built-in support for matchers provided by third-party
assertion libraries.
The following example demonstrates how to use the `assertThat()` support from Hamcrest in
a JUnit Jupiter test. As long as the Hamcrest library has been added to the classpath,
you can statically import methods such as `assertThat()`, `is()`, and `equalTo()` and
then use them in tests like in the `assertWithHamcrestMatcher()` method below.
[source,java,indent=0]
----
include::{testDir}/example/HamcrestAssertionsDemo.java[tags=user_guide]
----
Naturally, legacy tests based on the JUnit 4 programming model can continue using
`org.junit.Assert#assertThat`.
[[writing-tests-assumptions]]
=== Assumptions
JUnit Jupiter comes with a subset of the assumption methods that JUnit 4 provides and
adds a few that lend themselves well to being used with Java 8 lambda expressions and
method references. All JUnit Jupiter assumptions are static methods in the
`{Assumptions}` class.
[source,java,indent=0]
----
include::{testDir}/example/AssumptionsDemo.java[tags=user_guide]
----
NOTE: As of JUnit Jupiter 5.4, it is also possible to use methods from JUnit 4's
`org.junit.Assume` class for assumptions. Specifically, JUnit Jupiter supports JUnit 4's
`AssumptionViolatedException` to signal that a test should be aborted instead of marked
as a failure.
[[writing-tests-disabling]]
=== Disabling Tests
Entire test classes or individual test methods may be _disabled_ via the `{Disabled}`
annotation, via one of the annotations discussed in
<<writing-tests-conditional-execution>>, or via a custom <<extensions-conditions,
`ExecutionCondition`>>.
Here's a `@Disabled` test class.
[source,java,indent=0]
----
include::{testDir}/example/DisabledClassDemo.java[tags=user_guide]
----
And here's a test class that contains a `@Disabled` test method.
[source,java,indent=0]
----
include::{testDir}/example/DisabledTestsDemo.java[tags=user_guide]
----
NOTE: `@Disabled` may be declared without providing a _reason_; however, the JUnit team
recommends that developers provide a short explanation for why a test class or test
method has been disabled. Consequently, the above examples both show the use of a reason
-- for example, `@Disabled("Disabled until bug #42 has been resolved")`. Some development
teams even require the presence of issue tracking numbers in the _reason_ for automated
traceability, etc.
[[writing-tests-conditional-execution]]
=== Conditional Test Execution
The <<extensions-conditions, `ExecutionCondition`>> extension API in JUnit Jupiter allows
developers to either _enable_ or _disable_ a container or test based on certain
conditions _programmatically_. The simplest example of such a condition is the built-in
`{DisabledCondition}` which supports the `{Disabled}` annotation (see
<<writing-tests-disabling>>). In addition to `@Disabled`, JUnit Jupiter also supports
several other annotation-based conditions in the `org.junit.jupiter.api.condition`
package that allow developers to enable or disable containers and tests _declaratively_.
When multiple `ExecutionCondition` extensions are registered, a container or test is
disabled as soon as one of the conditions returns _disabled_. If you wish to provide
details about why they might be disabled, every annotation associated with these built-in
conditions has a `disabledReason` attribute available for that purpose.
See <<extensions-conditions, `ExecutionCondition`>> and the following sections for
details.
[TIP]
.Composed Annotations
====
Note that any of the _conditional_ annotations listed in the following sections may also
be used as a meta-annotation in order to create a custom _composed annotation_. For
example, the `@TestOnMac` annotation in the
<<writing-tests-conditional-execution-os-demo, @EnabledOnOs demo>> shows how you can
combine `@Test` and `@EnabledOnOs` in a single, reusable annotation.
====
[WARNING]
====
Unless otherwise stated, each of the _conditional_ annotations listed in the following
sections can only be declared once on a given test interface, test class, or test method.
If a conditional annotation is directly present, indirectly present, or meta-present
multiple times on a given element, only the first such annotation discovered by JUnit will
be used; any additional declarations will be silently ignored. Note, however, that each
conditional annotation may be used in conjunction with other conditional annotations in
the `org.junit.jupiter.api.condition` package.
====
[[writing-tests-conditional-execution-os]]
==== Operating System Conditions
A container or test may be enabled or disabled on a particular operating system via the
`{EnabledOnOs}` and `{DisabledOnOs}` annotations.
[[writing-tests-conditional-execution-os-demo]]
[source,java,indent=0]
----
include::{testDir}/example/ConditionalTestExecutionDemo.java[tags=user_guide_os]
----
[[writing-tests-conditional-execution-jre]]
==== Java Runtime Environment Conditions
A container or test may be enabled or disabled on particular versions of the Java
Runtime Environment (JRE) via the `{EnabledOnJre}` and `{DisabledOnJre}` annotations
or on a particular range of versions of the JRE via the `{EnabledForJreRange}` and
`{DisabledForJreRange}` annotations. The range defaults to `{JRE}.JAVA_8` as the lower
border (`min`) and `{JRE}.OTHER` as the higher border (`max`), which allows usage of
half open ranges.
[source,java,indent=0]
----
include::{testDir}/example/ConditionalTestExecutionDemo.java[tags=user_guide_jre]
----
[[writing-tests-conditional-execution-system-properties]]
==== System Property Conditions
A container or test may be enabled or disabled based on the value of the `named` JVM
system property via the `{EnabledIfSystemProperty}` and `{DisabledIfSystemProperty}`
annotations. The value supplied via the `matches` attribute will be interpreted as a
regular expression.
[source,java,indent=0]
----
include::{testDir}/example/ConditionalTestExecutionDemo.java[tags=user_guide_system_property]
----
[TIP]
====
As of JUnit Jupiter 5.6, `{EnabledIfSystemProperty}` and `{DisabledIfSystemProperty}` are
_repeatable annotations_. Consequently, these annotations may be declared multiple times
on a test interface, test class, or test method. Specifically, these annotations will be
found if they are directly present, indirectly present, or meta-present on a given element.
====
[[writing-tests-conditional-execution-environment-variables]]
==== Environment Variable Conditions
A container or test may be enabled or disabled based on the value of the `named`
environment variable from the underlying operating system via the
`{EnabledIfEnvironmentVariable}` and `{DisabledIfEnvironmentVariable}` annotations. The
value supplied via the `matches` attribute will be interpreted as a regular expression.
[source,java,indent=0]
----
include::{testDir}/example/ConditionalTestExecutionDemo.java[tags=user_guide_environment_variable]
----
[TIP]
====
As of JUnit Jupiter 5.6, `{EnabledIfEnvironmentVariable}` and
`{DisabledIfEnvironmentVariable}` are _repeatable annotations_. Consequently, these
annotations may be declared multiple times on a test interface, test class, or test
method. Specifically, these annotations will be found if they are directly present,
indirectly present, or meta-present on a given element.
====
[[writing-tests-conditional-execution-custom]]
==== Custom Conditions
A container or test may be enabled or disabled based on the boolean return of a
method via the `{EnabledIf}` and `{DisabledIf}` annotations. The method is provided to
the annotation via its name.
If needed, the condition method can take a single parameter of type `ExtensionContext`.
[source,java,indent=0]
----
include::{testDir}/example/ConditionalTestExecutionDemo.java[tags=user_guide_custom]
----
Alternatively, the condition method can be located outside the test class. In this case,
it has to be referenced by its _fully qualified name_ as demonstrated in the following
example.
[source,java,indent=0]
----
package example;
include::{testDir}/example/ExternalCustomConditionDemo.java[tags=user_guide_external_custom_condition]
----
NOTE: When `{EnabledIf}` or `{DisabledIf}` is used at class level, the condition method
must always be `static`. Condition methods located in external classes must also be
`static`. In any other case, you can use both static or instance methods.
[[writing-tests-tagging-and-filtering]]
=== Tagging and Filtering
Test classes and methods can be tagged via the `@Tag` annotation. Those tags can later be
used to filter <<running-tests, test discovery and execution>>. Please refer to the
<<running-tests-tags>> section for more information about tag support in the JUnit
Platform.
[source,java,indent=0]
----
include::{testDir}/example/TaggingDemo.java[tags=user_guide]
----
TIP: See <<writing-tests-meta-annotations>> for examples demonstrating how to create
custom annotations for tags.
[[writing-tests-test-execution-order]]
=== Test Execution Order
By default, test classes and methods will be ordered using an algorithm that is
deterministic but intentionally nonobvious. This ensures that subsequent runs of a test
suite execute test classes and test methods in the same order, thereby allowing for
repeatable builds.
NOTE: See <<writing-tests-classes-and-methods>> for a definition of _test method_ and
_test class_.
[[writing-tests-test-execution-order-methods]]
==== Method Order
Although true _unit tests_ typically should not rely on the order in which they are
executed, there are times when it is necessary to enforce a specific test method execution
order -- for example, when writing _integration tests_ or _functional tests_ where the
sequence of the tests is important, especially in conjunction with
`@TestInstance(Lifecycle.PER_CLASS)`.
To control the order in which test methods are executed, annotate your test class or test
interface with `{TestMethodOrder}` and specify the desired `{MethodOrderer}`
implementation. You can implement your own custom `MethodOrderer` or use one of the
following built-in `MethodOrderer` implementations.
* `{MethodOrderer_DisplayName}`: sorts test methods _alphanumerically_ based on their
display names (see <<writing-tests-display-name-generator-precedence-rules, display name
generation precedence rules>>)
* `{MethodOrderer_MethodName}`: sorts test methods _alphanumerically_ based on their names
and formal parameter lists
* `{MethodOrderer_OrderAnnotation}`: sorts test methods _numerically_ based on values
specified via the `{Order}` annotation
* `{MethodOrderer_Random}`: orders test methods _pseudo-randomly_ and supports
configuration of a custom _seed_
* `{MethodOrderer_Alphanumeric}`: sorts test methods _alphanumerically_ based on their
names and formal parameter lists; **deprecated in favor of `{MethodOrderer_MethodName}`,
to be removed in 6.0**
NOTE: See also: <<extensions-execution-order-wrapping-behavior>>
The following example demonstrates how to guarantee that test methods are executed in the
order specified via the `@Order` annotation.
[source,java,indent=0]
----
include::{testDir}/example/OrderedTestsDemo.java[tags=user_guide]
----
[[writing-tests-test-execution-order-methods-default]]
===== Setting the Default Method Orderer
You can use the `junit.jupiter.testmethod.order.default` <<running-tests-config-params,
configuration parameter>> to specify the fully qualified class name of the
`{MethodOrderer}` you would like to use by default. Just like for the orderer configured
via the `{TestMethodOrder}` annotation, the supplied class has to implement the
`MethodOrderer` interface. The default orderer will be used for all tests unless the
`@TestMethodOrder` annotation is present on an enclosing test class or test interface.
For example, to use the `{MethodOrderer_OrderAnnotation}` method orderer by default, you
should set the configuration parameter to the corresponding fully qualified class name
(e.g., in `src/test/resources/junit-platform.properties`):
[source,properties,indent=0]
----
junit.jupiter.testmethod.order.default = \
org.junit.jupiter.api.MethodOrderer$OrderAnnotation
----
Similarly, you can specify the fully qualified name of any custom class that implements
`MethodOrderer`.
[[writing-tests-test-execution-order-classes]]
==== Class Order
Although test classes typically should not rely on the order in which they are executed,
there are times when it is desirable to enforce a specific test class execution order. You
may wish to execute test classes in a random order to ensure there are no accidental
dependencies between test classes, or you may wish to order test classes to optimize build
time as outlined in the following scenarios.
* Run previously failing tests and faster tests first: "fail fast" mode
* With parallel execution enabled, run longer tests first: "shortest test plan execution
duration" mode
* Various other use cases
To configure test class execution order _globally_ for the entire test suite, use the
`junit.jupiter.testclass.order.default` <<running-tests-config-params, configuration
parameter>> to specify the fully qualified class name of the `{ClassOrderer}` you would
like to use. The supplied class must implement the `ClassOrderer` interface.
You can implement your own custom `ClassOrderer` or use one of the following built-in
`ClassOrderer` implementations.
* `{ClassOrderer_ClassName}`: sorts test classes _alphanumerically_ based on their fully
qualified class names
* `{ClassOrderer_DisplayName}`: sorts test classes _alphanumerically_ based on their
display names (see <<writing-tests-display-name-generator-precedence-rules, display name
generation precedence rules>>)
* `{ClassOrderer_OrderAnnotation}`: sorts test classes _numerically_ based on values
specified via the `{Order}` annotation
* `{ClassOrderer_Random}`: orders test classes _pseudo-randomly_ and supports
configuration of a custom _seed_
For example, for the `@Order` annotation to be honored on _test classes_, you should
configure the `{ClassOrderer_OrderAnnotation}` class orderer using the configuration
parameter with the corresponding fully qualified class name (e.g., in
`src/test/resources/junit-platform.properties`):
[source,properties,indent=0]
----
junit.jupiter.testclass.order.default = \
org.junit.jupiter.api.ClassOrderer$OrderAnnotation
----
The configured `ClassOrderer` will be applied to all top-level test classes (including
`static` nested test classes) and `@Nested` test classes.
NOTE: Top-level test classes will be ordered relative to each other; whereas, `@Nested`
test classes will be ordered relative to other `@Nested` test classes sharing the same
_enclosing class_.
To configure test class execution order _locally_ for `@Nested` test classes, declare the
`{TestClassOrder}` annotation on the enclosing class for the `@Nested` test classes you
want to order, and supply a class reference to the `ClassOrderer` implementation you would
like to use directly in the `@TestClassOrder` annotation. The configured `ClassOrderer`
will be applied recursively to `@Nested` test classes and their `@Nested` test classes.
Note that a local `@TestClassOrder` declaration always overrides an inherited
`@TestClassOrder` declaration or a `ClassOrderer` configured globally via the
`junit.jupiter.testclass.order.default` configuration parameter.
The following example demonstrates how to guarantee that `@Nested` test classes are
executed in the order specified via the `@Order` annotation.
[source,java,indent=0]
----
include::{testDir}/example/OrderedNestedTestClassesDemo.java[tags=user_guide]
----
[[writing-tests-test-instance-lifecycle]]
=== Test Instance Lifecycle
In order to allow individual test methods to be executed in isolation and to avoid
unexpected side effects due to mutable test instance state, JUnit creates a new instance
of each test class before executing each _test method_ (see
<<writing-tests-classes-and-methods>>). This "per-method" test instance lifecycle is the
default behavior in JUnit Jupiter and is analogous to all previous versions of JUnit.
NOTE: Please note that the test class will still be instantiated if a given _test method_
is _disabled_ via a <<writing-tests-conditional-execution,condition>> (e.g., `@Disabled`,
`@DisabledOnOs`, etc.) even when the "per-method" test instance lifecycle mode is active.
If you would prefer that JUnit Jupiter execute all test methods on the same test
instance, annotate your test class with `@TestInstance(Lifecycle.PER_CLASS)`. When using
this mode, a new test instance will be created once per test class. Thus, if your test
methods rely on state stored in instance variables, you may need to reset that state in
`@BeforeEach` or `@AfterEach` methods.
The "per-class" mode has some additional benefits over the default "per-method" mode.
Specifically, with the "per-class" mode it becomes possible to declare `@BeforeAll` and
`@AfterAll` on non-static methods as well as on interface `default` methods. The
"per-class" mode therefore also makes it possible to use `@BeforeAll` and `@AfterAll`
methods in `@Nested` test classes.
If you are authoring tests using the Kotlin programming language, you may also find it
easier to implement `@BeforeAll` and `@AfterAll` methods by switching to the "per-class"
test instance lifecycle mode.
[[writing-tests-test-instance-lifecycle-changing-default]]
==== Changing the Default Test Instance Lifecycle
If a test class or test interface is not annotated with `@TestInstance`, JUnit Jupiter
will use a _default_ lifecycle mode. The standard _default_ mode is `PER_METHOD`;
however, it is possible to change the _default_ for the execution of an entire test plan.
To change the default test instance lifecycle mode, set the
`junit.jupiter.testinstance.lifecycle.default` _configuration parameter_ to the name of
an enum constant defined in `TestInstance.Lifecycle`, ignoring case. This can be supplied
as a JVM system property, as a _configuration parameter_ in the
`LauncherDiscoveryRequest` that is passed to the `Launcher`, or via the JUnit Platform
configuration file (see <<running-tests-config-params>> for details).
For example, to set the default test instance lifecycle mode to `Lifecycle.PER_CLASS`,
you can start your JVM with the following system property.
`-Djunit.jupiter.testinstance.lifecycle.default=per_class`
Note, however, that setting the default test instance lifecycle mode via the JUnit
Platform configuration file is a more robust solution since the configuration file can be
checked into a version control system along with your project and can therefore be used
within IDEs and your build software.
To set the default test instance lifecycle mode to `Lifecycle.PER_CLASS` via the JUnit
Platform configuration file, create a file named `junit-platform.properties` in the root
of the class path (e.g., `src/test/resources`) with the following content.
`junit.jupiter.testinstance.lifecycle.default = per_class`
WARNING: Changing the _default_ test instance lifecycle mode can lead to unpredictable
results and fragile builds if not applied consistently. For example, if the build
configures "per-class" semantics as the default but tests in the IDE are executed using
"per-method" semantics, that can make it difficult to debug errors that occur on the
build server. It is therefore recommended to change the default in the JUnit Platform
configuration file instead of via a JVM system property.
[[writing-tests-nested]]
=== Nested Tests
`@Nested` tests give the test writer more capabilities to express the relationship among
several groups of tests. Such nested tests make use of Java's nested classes and
facilitate hierarchical thinking about the test structure. Here's an elaborate example,
both as source code and as a screenshot of the execution within an IDE.
[source,java,indent=0]
.Nested test suite for testing a stack
----
include::{testDir}/example/TestingAStackDemo.java[tags=user_guide]
----
When executing this example in an IDE, the test execution tree in the GUI will look
similar to the following image.
image::writing-tests_nested_test_ide.png[caption='',title='Executing a nested test in an IDE']
In this example, preconditions from outer tests are used in inner tests by defining
hierarchical lifecycle methods for the setup code. For example, `createNewStack()` is a
`@BeforeEach` lifecycle method that is used in the test class in which it is defined and
in all levels in the nesting tree below the class in which it is defined.
The fact that setup code from outer tests is run before inner tests are executed gives you
the ability to run all tests independently. You can even run inner tests alone without
running the outer tests, because the setup code from the outer tests is always executed.
NOTE: _Only non-static nested classes_ (i.e. _inner classes_) can serve as `@Nested` test
classes. Nesting can be arbitrarily deep, and those inner classes are subject to full
lifecycle support with one exception: `@BeforeAll` and `@AfterAll` methods do not work
_by default_. The reason is that Java does not allow `static` members in inner classes.
However, this restriction can be circumvented by annotating a `@Nested` test class with
`@TestInstance(Lifecycle.PER_CLASS)` (see <<writing-tests-test-instance-lifecycle>>).
[[writing-tests-dependency-injection]]
=== Dependency Injection for Constructors and Methods
In all prior JUnit versions, test constructors or methods were not allowed to have
parameters (at least not with the standard `Runner` implementations). As one of the major
changes in JUnit Jupiter, both test constructors and methods are now permitted to have
parameters. This allows for greater flexibility and enables _Dependency Injection_ for
constructors and methods.
`{ParameterResolver}` defines the API for test extensions that wish to _dynamically_
resolve parameters at runtime. If a _test class_ constructor, a _test method_, or a
_lifecycle method_ (see <<writing-tests-classes-and-methods>>) accepts a parameter, the
parameter must be resolved at runtime by a registered `ParameterResolver`.
There are currently three built-in resolvers that are registered automatically.
* `{TestInfoParameterResolver}`: if a constructor or method parameter is of type
`{TestInfo}`, the `TestInfoParameterResolver` will supply an instance of `TestInfo`
corresponding to the current container or test as the value for the parameter. The
`TestInfo` can then be used to retrieve information about the current container or test
such as the display name, the test class, the test method, and associated tags. The
display name is either a technical name, such as the name of the test class or test
method, or a custom name configured via `@DisplayName`.
+
`{TestInfo}` acts as a drop-in replacement for the `TestName` rule from JUnit 4. The
following demonstrates how to have `TestInfo` injected into a test constructor,
`@BeforeEach` method, and `@Test` method.
[source,java,indent=0]
----
include::{testDir}/example/TestInfoDemo.java[tags=user_guide]
----
* `{RepetitionInfoParameterResolver}`: if a method parameter in a `@RepeatedTest`,
`@BeforeEach`, or `@AfterEach` method is of type `{RepetitionInfo}`, the
`RepetitionInfoParameterResolver` will supply an instance of `RepetitionInfo`.
`RepetitionInfo` can then be used to retrieve information about the current repetition
and the total number of repetitions for the corresponding `@RepeatedTest`. Note,
however, that `RepetitionInfoParameterResolver` is not registered outside the context
of a `@RepeatedTest`. See <<writing-tests-repeated-tests-examples>>.
* `{TestReporterParameterResolver}`: if a constructor or method parameter is of type
`{TestReporter}`, the `TestReporterParameterResolver` will supply an instance of
`TestReporter`. The `TestReporter` can be used to publish additional data about the
current test run. The data can be consumed via the `reportingEntryPublished()` method in
a `{TestExecutionListener}`, allowing it to be viewed in IDEs or included in reports.
+
In JUnit Jupiter you should use `TestReporter` where you used to print information to
`stdout` or `stderr` in JUnit 4. Using `@RunWith(JUnitPlatform.class)` will output all
reported entries to `stdout`. In addition, some IDEs print report entries to `stdout` or
display them in the user interface for test results.
[source,java,indent=0]
----
include::{testDir}/example/TestReporterDemo.java[tags=user_guide]
----
NOTE: Other parameter resolvers must be explicitly enabled by registering appropriate
<<extensions,extensions>> via `@ExtendWith`.
Check out the `{RandomParametersExtension}` for an example of a custom
`{ParameterResolver}`. While not intended to be production-ready, it demonstrates the
simplicity and expressiveness of both the extension model and the parameter resolution
process. `MyRandomParametersTest` demonstrates how to inject random values into `@Test`
methods.
[source,java,indent=0]
----
@ExtendWith(RandomParametersExtension.class)
class MyRandomParametersTest {
@Test
void injectsInteger(@Random int i, @Random int j) {
assertNotEquals(i, j);
}
@Test
void injectsDouble(@Random double d) {
assertEquals(0.0, d, 1.0);
}
}
----
For real-world use cases, check out the source code for the `{MockitoExtension}` and the
`{SpringExtension}`.
When the type of the parameter to inject is the only condition for your
`{ParameterResolver}`, you can use the generic `{TypeBasedParameterResolver}` base class.
The `supportsParameters` method is implemented behind the scenes and supports
parameterized types.
[[writing-tests-test-interfaces-and-default-methods]]
=== Test Interfaces and Default Methods
JUnit Jupiter allows `@Test`, `@RepeatedTest`, `@ParameterizedTest`, `@TestFactory`,
`@TestTemplate`, `@BeforeEach`, and `@AfterEach` to be declared on interface `default`
methods. `@BeforeAll` and `@AfterAll` can either be declared on `static` methods in a
test interface or on interface `default` methods _if_ the test interface or test class is
annotated with `@TestInstance(Lifecycle.PER_CLASS)` (see
<<writing-tests-test-instance-lifecycle>>). Here are some examples.
[source,java]
----
include::{testDir}/example/testinterface/TestLifecycleLogger.java[tags=user_guide]
----
[source,java]
----
include::{testDir}/example/testinterface/TestInterfaceDynamicTestsDemo.java[tags=user_guide]
----
`@ExtendWith` and `@Tag` can be declared on a test interface so that classes that
implement the interface automatically inherit its tags and extensions. See
<<extensions-lifecycle-callbacks-before-after-execution>> for the source code of the
<<extensions-lifecycle-callbacks-timing-extension, TimingExtension>>.
[source,java]
----
include::{testDir}/example/testinterface/TimeExecutionLogger.java[tags=user_guide]
----
In your test class you can then implement these test interfaces to have them applied.
[source,java]
----
include::{testDir}/example/testinterface/TestInterfaceDemo.java[tags=user_guide]
----
Running the `TestInterfaceDemo` results in output similar to the following:
....
INFO example.TestLifecycleLogger - Before all tests
INFO example.TestLifecycleLogger - About to execute [dynamicTestsForPalindromes()]
INFO example.TimingExtension - Method [dynamicTestsForPalindromes] took 19 ms.
INFO example.TestLifecycleLogger - Finished executing [dynamicTestsForPalindromes()]
INFO example.TestLifecycleLogger - About to execute [isEqualValue()]
INFO example.TimingExtension - Method [isEqualValue] took 1 ms.
INFO example.TestLifecycleLogger - Finished executing [isEqualValue()]
INFO example.TestLifecycleLogger - After all tests
....
Another possible application of this feature is to write tests for interface contracts.
For example, you can write tests for how implementations of `Object.equals` or
`Comparable.compareTo` should behave as follows.
[source,java]
----
include::{testDir}/example/defaultmethods/Testable.java[tags=user_guide]
----
[source,java]
----
include::{testDir}/example/defaultmethods/EqualsContract.java[tags=user_guide]
----
[source,java]
----
include::{testDir}/example/defaultmethods/ComparableContract.java[tags=user_guide]
----
In your test class you can then implement both contract interfaces thereby inheriting the
corresponding tests. Of course you'll have to implement the abstract methods.
[source,java]
----
include::{testDir}/example/defaultmethods/StringTests.java[tags=user_guide]
----
NOTE: The above tests are merely meant as examples and therefore not complete.
[[writing-tests-repeated-tests]]
=== Repeated Tests
JUnit Jupiter provides the ability to repeat a test a specified number of times by
annotating a method with `@RepeatedTest` and specifying the total number of repetitions
desired. Each invocation of a repeated test behaves like the execution of a regular
`@Test` method with full support for the same lifecycle callbacks and extensions.
The following example demonstrates how to declare a test named `repeatedTest()` that
will be automatically repeated 10 times.
[source,java]
----
@RepeatedTest(10)
void repeatedTest() {
// ...
}
----
In addition to specifying the number of repetitions, a custom display name can be
configured for each repetition via the `name` attribute of the `@RepeatedTest`
annotation. Furthermore, the display name can be a pattern composed of a combination of
static text and dynamic placeholders. The following placeholders are currently supported.
- `{displayName}`: display name of the `@RepeatedTest` method
- `{currentRepetition}`: the current repetition count
- `{totalRepetitions}`: the total number of repetitions
The default display name for a given repetition is generated based on the following
pattern: `"repetition {currentRepetition} of {totalRepetitions}"`. Thus, the display
names for individual repetitions of the previous `repeatedTest()` example would be:
`repetition 1 of 10`, `repetition 2 of 10`, etc. If you would like the display name of
the `@RepeatedTest` method included in the name of each repetition, you can define your
own custom pattern or use the predefined `RepeatedTest.LONG_DISPLAY_NAME` pattern. The
latter is equal to `"{displayName} :: repetition {currentRepetition} of
{totalRepetitions}"` which results in display names for individual repetitions like
`repeatedTest() :: repetition 1 of 10`, `repeatedTest() :: repetition 2 of 10`, etc.
In order to retrieve information about the current repetition and the total number of
repetitions programmatically, a developer can choose to have an instance of
`RepetitionInfo` injected into a `@RepeatedTest`, `@BeforeEach`, or `@AfterEach` method.
[[writing-tests-repeated-tests-examples]]
==== Repeated Test Examples
The `RepeatedTestsDemo` class at the end of this section demonstrates several examples of
repeated tests.
The `repeatedTest()` method is identical to example from the previous section; whereas,
`repeatedTestWithRepetitionInfo()` demonstrates how to have an instance of
`RepetitionInfo` injected into a test to access the total number of repetitions for the
current repeated test.
The next two methods demonstrate how to include a custom `@DisplayName` for the
`@RepeatedTest` method in the display name of each repetition. `customDisplayName()`
combines a custom display name with a custom pattern and then uses `TestInfo` to verify
the format of the generated display name. `Repeat!` is the `{displayName}` which comes
from the `@DisplayName` declaration, and `1/1` comes from
`{currentRepetition}/{totalRepetitions}`. In contrast,
`customDisplayNameWithLongPattern()` uses the aforementioned predefined
`RepeatedTest.LONG_DISPLAY_NAME` pattern.
`repeatedTestInGerman()` demonstrates the ability to translate display names of repeated
tests into foreign languages -- in this case German, resulting in names for individual
repetitions such as: `Wiederholung 1 von 5`, `Wiederholung 2 von 5`, etc.
Since the `beforeEach()` method is annotated with `@BeforeEach` it will get executed
before each repetition of each repeated test. By having the `TestInfo` and
`RepetitionInfo` injected into the method, we see that it's possible to obtain
information about the currently executing repeated test. Executing `RepeatedTestsDemo`
with the `INFO` log level enabled results in the following output.
....
INFO: About to execute repetition 1 of 10 for repeatedTest
INFO: About to execute repetition 2 of 10 for repeatedTest
INFO: About to execute repetition 3 of 10 for repeatedTest
INFO: About to execute repetition 4 of 10 for repeatedTest
INFO: About to execute repetition 5 of 10 for repeatedTest
INFO: About to execute repetition 6 of 10 for repeatedTest
INFO: About to execute repetition 7 of 10 for repeatedTest
INFO: About to execute repetition 8 of 10 for repeatedTest
INFO: About to execute repetition 9 of 10 for repeatedTest
INFO: About to execute repetition 10 of 10 for repeatedTest
INFO: About to execute repetition 1 of 5 for repeatedTestWithRepetitionInfo
INFO: About to execute repetition 2 of 5 for repeatedTestWithRepetitionInfo
INFO: About to execute repetition 3 of 5 for repeatedTestWithRepetitionInfo
INFO: About to execute repetition 4 of 5 for repeatedTestWithRepetitionInfo
INFO: About to execute repetition 5 of 5 for repeatedTestWithRepetitionInfo
INFO: About to execute repetition 1 of 1 for customDisplayName
INFO: About to execute repetition 1 of 1 for customDisplayNameWithLongPattern
INFO: About to execute repetition 1 of 5 for repeatedTestInGerman
INFO: About to execute repetition 2 of 5 for repeatedTestInGerman
INFO: About to execute repetition 3 of 5 for repeatedTestInGerman
INFO: About to execute repetition 4 of 5 for repeatedTestInGerman
INFO: About to execute repetition 5 of 5 for repeatedTestInGerman
....
[source,java]
----
include::{testDir}/example/RepeatedTestsDemo.java[tags=user_guide]
----
When using the `ConsoleLauncher` with the unicode theme enabled, execution of
`RepeatedTestsDemo` results in the following output to the console.
....
├─ RepeatedTestsDemo ✔
│ ├─ repeatedTest() ✔
│ │ ├─ repetition 1 of 10 ✔
│ │ ├─ repetition 2 of 10 ✔
│ │ ├─ repetition 3 of 10 ✔
│ │ ├─ repetition 4 of 10 ✔
│ │ ├─ repetition 5 of 10 ✔
│ │ ├─ repetition 6 of 10 ✔
│ │ ├─ repetition 7 of 10 ✔
│ │ ├─ repetition 8 of 10 ✔
│ │ ├─ repetition 9 of 10 ✔
│ │ └─ repetition 10 of 10 ✔
│ ├─ repeatedTestWithRepetitionInfo(RepetitionInfo) ✔
│ │ ├─ repetition 1 of 5 ✔
│ │ ├─ repetition 2 of 5 ✔
│ │ ├─ repetition 3 of 5 ✔
│ │ ├─ repetition 4 of 5 ✔
│ │ └─ repetition 5 of 5 ✔
│ ├─ Repeat! ✔
│ │ └─ Repeat! 1/1 ✔
│ ├─ Details... ✔
│ │ └─ Details... :: repetition 1 of 1 ✔
│ └─ repeatedTestInGerman() ✔
│ ├─ Wiederholung 1 von 5 ✔
│ ├─ Wiederholung 2 von 5 ✔
│ ├─ Wiederholung 3 von 5 ✔
│ ├─ Wiederholung 4 von 5 ✔
│ └─ Wiederholung 5 von 5 ✔
....
[[writing-tests-parameterized-tests]]
=== Parameterized Tests
Parameterized tests make it possible to run a test multiple times with different
arguments. They are declared just like regular `@Test` methods but use the
`{ParameterizedTest}` annotation instead. In addition, you must declare at least one
_source_ that will provide the arguments for each invocation and then _consume_ the
arguments in the test method.
The following example demonstrates a parameterized test that uses the `@ValueSource`
annotation to specify a `String` array as the source of arguments.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=first_example]
----
When executing the above parameterized test method, each invocation will be reported
separately. For instance, the `ConsoleLauncher` will print output similar to the
following.
....
palindromes(String) ✔
├─ [1] candidate=racecar ✔
├─ [2] candidate=radar ✔
└─ [3] candidate=able was I ere I saw elba ✔
....
[[writing-tests-parameterized-tests-setup]]
==== Required Setup
In order to use parameterized tests you need to add a dependency on the
`junit-jupiter-params` artifact. Please refer to <<dependency-metadata>> for details.
[[writing-tests-parameterized-tests-consuming-arguments]]
==== Consuming Arguments
Parameterized test methods typically _consume_ arguments directly from the configured
source (see <<writing-tests-parameterized-tests-sources>>) following a one-to-one
correlation between argument source index and method parameter index (see examples in
<<writing-tests-parameterized-tests-sources-CsvSource>>). However, a parameterized test
method may also choose to _aggregate_ arguments from the source into a single object
passed to the method (see <<writing-tests-parameterized-tests-argument-aggregation>>).
Additional arguments may also be provided by a `ParameterResolver` (e.g., to obtain an
instance of `TestInfo`, `TestReporter`, etc.). Specifically, a parameterized test method
must declare formal parameters according to the following rules.
* Zero or more _indexed arguments_ must be declared first.
* Zero or more _aggregators_ must be declared next.
* Zero or more arguments supplied by a `ParameterResolver` must be declared last.
In this context, an _indexed argument_ is an argument for a given index in the
`Arguments` provided by an `ArgumentsProvider` that is passed as an argument to the
parameterized method at the same index in the method's formal parameter list. An
_aggregator_ is any parameter of type `ArgumentsAccessor` or any parameter annotated with
`@AggregateWith`.
[NOTE]
.AutoCloseable arguments
====
Arguments that implement `java.lang.AutoCloseable` (or `java.io.Closeable` which extends
`java.lang.AutoCloseable`) will be automatically closed after `@AfterEach` methods and
`AfterEachCallback` extensions have been called for the current parameterized test
invocation.
To prevent this from happening, set the `autoCloseArguments` attribute in
`@ParameterizedTest` to `false`. Specifically, if an argument that implements
`AutoCloseable` is reused for multiple invocations of the same parameterized test method,
you must annotate the method with `@ParameterizedTest(autoCloseArguments = false)` to
ensure that the argument is not closed between invocations.
====
[[writing-tests-parameterized-tests-sources]]
==== Sources of Arguments
Out of the box, JUnit Jupiter provides quite a few _source_ annotations. Each of the
following subsections provides a brief overview and an example for each of them. Please
refer to the Javadoc in the `{params-provider-package}` package for additional
information.
[[writing-tests-parameterized-tests-sources-ValueSource]]
===== @ValueSource
`@ValueSource` is one of the simplest possible sources. It lets you specify a single
array of literal values and can only be used for providing a single argument per
parameterized test invocation.
The following types of literal values are supported by `@ValueSource`.
- `short`
- `byte`
- `int`
- `long`
- `float`
- `double`
- `char`
- `boolean`
- `java.lang.String`
- `java.lang.Class`
For example, the following `@ParameterizedTest` method will be invoked three times, with
the values `1`, `2`, and `3` respectively.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=ValueSource_example]
----
[[writing-tests-parameterized-tests-sources-null-and-empty]]
===== Null and Empty Sources
In order to check corner cases and verify proper behavior of our software when it is
supplied _bad input_, it can be useful to have `null` and _empty_ values supplied to our
parameterized tests. The following annotations serve as sources of `null` and empty values
for parameterized tests that accept a single argument.
* `{NullSource}`: provides a single `null` argument to the annotated `@ParameterizedTest`
method.
- `@NullSource` cannot be used for a parameter that has a primitive type.
* `{EmptySource}`: provides a single _empty_ argument to the annotated `@ParameterizedTest`
method for parameters of the following types: `java.lang.String`, `java.util.List`,
`java.util.Set`, `java.util.Map`, primitive arrays (e.g., `int[]`, `char[][]`, etc.),
object arrays (e.g.,`String[]`, `Integer[][]`, etc.).
- Subtypes of the supported types are not supported.
* `{NullAndEmptySource}`: a _composed annotation_ that combines the functionality of
`@NullSource` and `@EmptySource`.
If you need to supply multiple varying types of _blank_ strings to a parameterized test,
you can achieve that using <<writing-tests-parameterized-tests-sources-ValueSource>> --
for example, `@ValueSource(strings = {"{nbsp}", "{nbsp}{nbsp}{nbsp}", "\t", "\n"})`.
You can also combine `@NullSource`, `@EmptySource`, and `@ValueSource` to test a wider
range of `null`, _empty_, and _blank_ input. The following example demonstrates how to
achieve this for strings.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=NullAndEmptySource_example1]
----
Making use of the composed `@NullAndEmptySource` annotation simplifies the above as
follows.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=NullAndEmptySource_example2]
----
NOTE: Both variants of the `nullEmptyAndBlankStrings(String)` parameterized test method
result in six invocations: 1 for `null`, 1 for the empty string, and 4 for the explicit
blank strings supplied via `@ValueSource`.
[[writing-tests-parameterized-tests-sources-EnumSource]]
===== @EnumSource
`@EnumSource` provides a convenient way to use `Enum` constants.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=EnumSource_example]
----
The annotation's `value` attribute is optional. When omitted, the declared type of the
first method parameter is used. The test will fail if it does not reference an enum type.
Thus, the `value` attribute is required in the above example because the method parameter
is declared as `TemporalUnit`, i.e. the interface implemented by `ChronoUnit`, which isn't
an enum type. Changing the method parameter type to `ChronoUnit` allows you to omit the
explicit enum type from the annotation as follows.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=EnumSource_example_autodetection]
----
The annotation provides an optional `names` attribute that lets you specify which
constants shall be used, like in the following example. If omitted, all constants will be
used.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=EnumSource_include_example]
----
The `@EnumSource` annotation also provides an optional `mode` attribute that enables
fine-grained control over which constants are passed to the test method. For example, you
can exclude names from the enum constant pool or specify regular expressions as in the
following examples.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=EnumSource_exclude_example]
----
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=EnumSource_regex_example]
----
[[writing-tests-parameterized-tests-sources-MethodSource]]
===== @MethodSource
`{MethodSource}` allows you to refer to one or more _factory_ methods of the test class
or external classes.
Factory methods within the test class must be `static` unless the test class is annotated
with `@TestInstance(Lifecycle.PER_CLASS)`; whereas, factory methods in external classes
must always be `static`. In addition, such factory methods must not accept any arguments.
Each factory method must generate a _stream_ of _arguments_, and each set of arguments
within the stream will be provided as the physical arguments for individual invocations
of the annotated `@ParameterizedTest` method. Generally speaking this translates to a
`Stream` of `Arguments` (i.e., `Stream<Arguments>`); however, the actual concrete return
type can take on many forms. In this context, a "stream" is anything that JUnit can
reliably convert into a `Stream`, such as `Stream`, `DoubleStream`, `LongStream`,
`IntStream`, `Collection`, `Iterator`, `Iterable`, an array of objects, or an array of
primitives. The "arguments" within the stream can be supplied as an instance of
`Arguments`, an array of objects (e.g., `Object[]`), or a single value if the
parameterized test method accepts a single argument.
If you only need a single parameter, you can return a `Stream` of instances of the
parameter type as demonstrated in the following example.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=simple_MethodSource_example]
----
If you do not explicitly provide a factory method name via `@MethodSource`, JUnit Jupiter
will search for a _factory_ method that has the same name as the current
`@ParameterizedTest` method by convention. This is demonstrated in the following example.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=simple_MethodSource_without_value_example]
----
Streams for primitive types (`DoubleStream`, `IntStream`, and `LongStream`) are also
supported as demonstrated by the following example.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=primitive_MethodSource_example]
----
If a parameterized test method declares multiple parameters, you need to return a
collection, stream, or array of `Arguments` instances or object arrays as shown below
(see the Javadoc for `{MethodSource}` for further details on supported return types).
Note that `arguments(Object...)` is a static factory method defined in the `Arguments`
interface. In addition, `Arguments.of(Object...)` may be used as an alternative to
`arguments(Object...)`.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=multi_arg_MethodSource_example]
----
An external, `static` _factory_ method can be referenced by providing its _fully
qualified method name_ as demonstrated in the following example.
[source,java,indent=0]
----
package example;
include::{testDir}/example/ExternalMethodSourceDemo.java[tags=external_MethodSource_example]
----
[[writing-tests-parameterized-tests-sources-CsvSource]]
===== @CsvSource
`@CsvSource` allows you to express argument lists as comma-separated values (i.e., CSV
`String` literals). Each string provided via the `value` attribute in `@CsvSource`
represents a CSV record and results in one invocation of the parameterized test. The first
record may optionally be used to supply CSV headers (see the Javadoc for the
`useHeadersInDisplayName` attribute for details and an example).
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=CsvSource_example]
----
The default delimiter is a comma (`,`), but you can use another character by setting the
`delimiter` attribute. Alternatively, the `delimiterString` attribute allows you to use a
`String` delimiter instead of a single character. However, both delimiter attributes
cannot be set simultaneously.
By default, `@CsvSource` uses a single quote (`'`) as its quote character, but this can be
changed via the `quoteCharacter` attribute. See the `'lemon, lime'` value in the example
above and in the table below. An empty, quoted value (`''`) results in an empty `String`
unless the `emptyValue` attribute is set; whereas, an entirely _empty_ value is
interpreted as a `null` reference. By specifying one or more `nullValues`, a custom value
can be interpreted as a `null` reference (see the `NIL` example in the table below). An
`ArgumentConversionException` is thrown if the target type of a `null` reference is a
primitive type.
NOTE: An _unquoted_ empty value will always be converted to a `null` reference regardless
of any custom values configured via the `nullValues` attribute.
Except within a quoted string, leading and trailing whitespace in a CSV column is trimmed
by default. This behavior can be changed by setting the
`ignoreLeadingAndTrailingWhitespace` attribute to `true`.
[cols="50,50"]
|===
| Example Input | Resulting Argument List
| `@CsvSource({ "apple, banana" })` | `"apple"`, `"banana"`
| `@CsvSource({ "apple, 'lemon, lime'" })` | `"apple"`, `"lemon, lime"`
| `@CsvSource({ "apple, ''" })` | `"apple"`, `""`
| `@CsvSource({ "apple, " })` | `"apple"`, `null`
| `@CsvSource(value = { "apple, banana, NIL" }, nullValues = "NIL")` | `"apple"`, `"banana"`, `null`
| `@CsvSource(value = { " apple , banana" }, ignoreLeadingAndTrailingWhitespace = false)` | `" apple "`, `" banana"`
|===
If the programming language you are using supports _text blocks_ -- for example, Java SE
15 or higher -- you can alternatively use the `textBlock` attribute of `@CsvSource`. Each
record within a text block represents a CSV record and results in one invocation of the
parameterized test. The first record may optionally be used to supply CSV headers by
setting the `useHeadersInDisplayName` attribute to `true` as in the example below.
Using a text block, the previous example can be implemented as follows.
[source,java,indent=0]
----
@ParameterizedTest(name = "[{index}] {arguments}")
@CsvSource(useHeadersInDisplayName = true, textBlock = """
FRUIT, RANK
apple, 1
banana, 2
'lemon, lime', 0xF1
strawberry, 700_000
""")
void testWithCsvSource(String fruit, int rank) {
// ...
}
----
The generated display names for the previous example include the CSV header names.
----
[1] FRUIT = apple, RANK = 1
[2] FRUIT = banana, RANK = 2
[3] FRUIT = lemon, lime, RANK = 0xF1
[4] FRUIT = strawberry, RANK = 700_000
----
In contrast to CSV records supplied via the `value` attribute, a text block can contain
comments. Any line beginning with a `+++#+++` symbol will be treated as a comment and
ignored. Note, however, that the `+++#+++` symbol must be the first character on the line
without any leading whitespace. It is therefore recommended that the closing text block
delimiter (`"""`) be placed either at the end of the last line of input or on the
following line, left aligned with the rest of the input (as can be seen in the example
below which demonstrates formatting similar to a table).
[source,java,indent=0]
----
@ParameterizedTest
@CsvSource(delimiter = '|', quoteCharacter = '"', textBlock = """
#-----------------------------
# FRUIT | RANK
#-----------------------------
apple | 1
#-----------------------------
banana | 2
#-----------------------------
"lemon lime" | 0xF1
#-----------------------------
strawberry | 700_000
#-----------------------------
""")
void testWithCsvSource(String fruit, int rank) {
// ...
}
----
[NOTE]
====
Java's https://docs.oracle.com/en/java/javase/15/text-blocks/index.html[text block]
feature automatically removes _incidental whitespace_ when the code is compiled.
However other JVM languages such as Groovy and Kotlin do not. Thus, if you are using a
programming language other than Java and your text block contains comments or new lines
within quoted strings, you will need to ensure that there is no leading whitespace within
your text block.
====
[[writing-tests-parameterized-tests-sources-CsvFileSource]]
===== @CsvFileSource
`@CsvFileSource` lets you use comma-separated value (CSV) files from the classpath or the
local file system. Each record from a CSV file results in one invocation of the
parameterized test. The first record may optionally be used to supply CSV headers. You can
instruct JUnit to ignore the headers via the `numLinesToSkip` attribute. If you would like
for the headers to be used in the display names, you can set the `useHeadersInDisplayName`
attribute to `true`. The examples below demonstrate the use of `numLinesToSkip` and
`useHeadersInDisplayName`.
The default delimiter is a comma (`,`), but you can use another character by setting the
`delimiter` attribute. Alternatively, the `delimiterString` attribute allows you to use a
`String` delimiter instead of a single character. However, both delimiter attributes
cannot be set simultaneously.
.Comments in CSV files
NOTE: Any line beginning with a `+++#+++` symbol will be interpreted as a comment and will
be ignored.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=CsvFileSource_example]
----
[source,csv,indent=0]
.two-column.csv
----
include::{testResourcesDir}/two-column.csv[]
----
The following listing shows the generated display names for the first two parameterized
test methods above.
----
[1] country=Sweden, reference=1
[2] country=Poland, reference=2
[3] country=United States of America, reference=3
[4] country=France, reference=700_000
----
The following listing shows the generated display names for the last parameterized test
method above that uses CSV header names.
----
[1] COUNTRY = Sweden, REFERENCE = 1
[2] COUNTRY = Poland, REFERENCE = 2
[3] COUNTRY = United States of America, REFERENCE = 3
[4] COUNTRY = France, REFERENCE = 700_000
----
In contrast to the default syntax used in `@CsvSource`, `@CsvFileSource` uses a double
quote (`+++"+++`) as the quote character by default, but this can be changed via the
`quoteCharacter` attribute. See the `"United States of America"` value in the example
above. An empty, quoted value (`+++""+++`) results in an empty `String` unless the
`emptyValue` attribute is set; whereas, an entirely _empty_ value is interpreted as a
`null` reference. By specifying one or more `nullValues`, a custom value can be
interpreted as a `null` reference. An `ArgumentConversionException` is thrown if the
target type of a `null` reference is a primitive type.
NOTE: An _unquoted_ empty value will always be converted to a `null` reference regardless
of any custom values configured via the `nullValues` attribute.
Except within a quoted string, leading and trailing whitespace in a CSV column is trimmed
by default. This behavior can be changed by setting the
`ignoreLeadingAndTrailingWhitespace` attribute to `true`.
[[writing-tests-parameterized-tests-sources-ArgumentsSource]]
===== @ArgumentsSource
`@ArgumentsSource` can be used to specify a custom, reusable `ArgumentsProvider`. Note
that an implementation of `ArgumentsProvider` must be declared as either a top-level
class or as a `static` nested class.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=ArgumentsSource_example]
----
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=ArgumentsProvider_example]
----
[[writing-tests-parameterized-tests-argument-conversion]]
==== Argument Conversion
[[writing-tests-parameterized-tests-argument-conversion-widening]]
===== Widening Conversion
JUnit Jupiter supports
https://docs.oracle.com/javase/specs/jls/se8/html/jls-5.html#jls-5.1.2[Widening Primitive
Conversion] for arguments supplied to a `@ParameterizedTest`. For example, a
parameterized test annotated with `@ValueSource(ints = { 1, 2, 3 })` can be declared to
accept not only an argument of type `int` but also an argument of type `long`, `float`,
or `double`.
[[writing-tests-parameterized-tests-argument-conversion-implicit]]
===== Implicit Conversion
To support use cases like `@CsvSource`, JUnit Jupiter provides a number of built-in
implicit type converters. The conversion process depends on the declared type of each
method parameter.
For example, if a `@ParameterizedTest` declares a parameter of type `TimeUnit` and the
actual type supplied by the declared source is a `String`, the string will be
automatically converted into the corresponding `TimeUnit` enum constant.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=implicit_conversion_example]
----
`String` instances are implicitly converted to the following target types.
NOTE: Decimal, hexadecimal, and octal `String` literals will be converted to their
integral types: `byte`, `short`, `int`, `long`, and their boxed counterparts.
[[writing-tests-parameterized-tests-argument-conversion-implicit-table]]
[cols="10,90"]
|===
| Target Type | Example
| `boolean`/`Boolean` | `"true"` -> `true`
| `byte`/`Byte` | `"15"`, `"0xF"`, or `"017"` -> `(byte) 15`
| `char`/`Character` | `"o"` -> `'o'`
| `short`/`Short` | `"15"`, `"0xF"`, or `"017"` -> `(short) 15`
| `int`/`Integer` | `"15"`, `"0xF"`, or `"017"` -> `15`
| `long`/`Long` | `"15"`, `"0xF"`, or `"017"` -> `15L`
| `float`/`Float` | `"1.0"` -> `1.0f`
| `double`/`Double` | `"1.0"` -> `1.0d`
| `Enum` subclass | `"SECONDS"` -> `TimeUnit.SECONDS`
| `java.io.File` | `"/path/to/file"` -> `new File("/path/to/file")`
| `java.lang.Class` | `"java.lang.Integer"` -> `java.lang.Integer.class` _(use `$` for nested classes, e.g. `"java.lang.Thread$State"`)_
| `java.lang.Class` | `"byte"` -> `byte.class` _(primitive types are supported)_
| `java.lang.Class` | `"char[]"` -> `char[].class` _(array types are supported)_
| `java.math.BigDecimal` | `"123.456e789"` -> `new BigDecimal("123.456e789")`
| `java.math.BigInteger` | `"1234567890123456789"` -> `new BigInteger("1234567890123456789")`
| `java.net.URI` | `"https://junit.org/"` -> `URI.create("https://junit.org/")`
| `java.net.URL` | `"https://junit.org/"` -> `new URL("https://junit.org/")`
| `java.nio.charset.Charset` | `"UTF-8"` -> `Charset.forName("UTF-8")`
| `java.nio.file.Path` | `"/path/to/file"` -> `Paths.get("/path/to/file")`
| `java.time.Duration` | `"PT3S"` -> `Duration.ofSeconds(3)`
| `java.time.Instant` | `"1970-01-01T00:00:00Z"` -> `Instant.ofEpochMilli(0)`
| `java.time.LocalDateTime` | `"2017-03-14T12:34:56.789"` -> `LocalDateTime.of(2017, 3, 14, 12, 34, 56, 789_000_000)`
| `java.time.LocalDate` | `"2017-03-14"` -> `LocalDate.of(2017, 3, 14)`
| `java.time.LocalTime` | `"12:34:56.789"` -> `LocalTime.of(12, 34, 56, 789_000_000)`
| `java.time.MonthDay` | `"--03-14"` -> `MonthDay.of(3, 14)`
| `java.time.OffsetDateTime` | `"2017-03-14T12:34:56.789Z"` -> `OffsetDateTime.of(2017, 3, 14, 12, 34, 56, 789_000_000, ZoneOffset.UTC)`
| `java.time.OffsetTime` | `"12:34:56.789Z"` -> `OffsetTime.of(12, 34, 56, 789_000_000, ZoneOffset.UTC)`
| `java.time.Period` | `"P2M6D"` -> `Period.of(0, 2, 6)`
| `java.time.YearMonth` | `"2017-03"` -> `YearMonth.of(2017, 3)`
| `java.time.Year` | `"2017"` -> `Year.of(2017)`
| `java.time.ZonedDateTime` | `"2017-03-14T12:34:56.789Z"` -> `ZonedDateTime.of(2017, 3, 14, 12, 34, 56, 789_000_000, ZoneOffset.UTC)`
| `java.time.ZoneId` | `"Europe/Berlin"` -> `ZoneId.of("Europe/Berlin")`
| `java.time.ZoneOffset` | `"+02:30"` -> `ZoneOffset.ofHoursMinutes(2, 30)`
| `java.util.Currency` | `"JPY"` -> `Currency.getInstance("JPY")`
| `java.util.Locale` | `"en"` -> `new Locale("en")`
| `java.util.UUID` | `"d043e930-7b3b-48e3-bdbe-5a3ccfb833db"` -> `UUID.fromString("d043e930-7b3b-48e3-bdbe-5a3ccfb833db")`
|===
[[writing-tests-parameterized-tests-argument-conversion-implicit-fallback]]
====== Fallback String-to-Object Conversion
In addition to implicit conversion from strings to the target types listed in the above
table, JUnit Jupiter also provides a fallback mechanism for automatic conversion from a
`String` to a given target type if the target type declares exactly one suitable _factory
method_ or a _factory constructor_ as defined below.
- __factory method__: a non-private, `static` method declared in the target type that
accepts a single `String` argument and returns an instance of the target type. The name
of the method can be arbitrary and need not follow any particular convention.
- __factory constructor__: a non-private constructor in the target type that accepts a
single `String` argument. Note that the target type must be declared as either a
top-level class or as a `static` nested class.
NOTE: If multiple _factory methods_ are discovered, they will be ignored. If a _factory
method_ and a _factory constructor_ are discovered, the factory method will be used
instead of the constructor.
For example, in the following `@ParameterizedTest` method, the `Book` argument will be
created by invoking the `Book.fromTitle(String)` factory method and passing `"42 Cats"`
as the title of the book.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=implicit_fallback_conversion_example]
----
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=implicit_fallback_conversion_example_Book]
----
[[writing-tests-parameterized-tests-argument-conversion-explicit]]
===== Explicit Conversion
Instead of relying on implicit argument conversion you may explicitly specify an
`ArgumentConverter` to use for a certain parameter using the `@ConvertWith` annotation
like in the following example. Note that an implementation of `ArgumentConverter` must be
declared as either a top-level class or as a `static` nested class.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=explicit_conversion_example]
----
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=explicit_conversion_example_ToStringArgumentConverter]
----
If the converter is only meant to convert one type to another, you can extend
`TypedArgumentConverter` to avoid boilerplate type checks.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=explicit_conversion_example_TypedArgumentConverter]
----
Explicit argument converters are meant to be implemented by test and extension authors.
Thus, `junit-jupiter-params` only provides a single explicit argument converter that may
also serve as a reference implementation: `JavaTimeArgumentConverter`. It is used via the
composed annotation `JavaTimeConversionPattern`.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=explicit_java_time_converter]
----
[[writing-tests-parameterized-tests-argument-aggregation]]
==== Argument Aggregation
By default, each _argument_ provided to a `@ParameterizedTest` method corresponds to a
single method parameter. Consequently, argument sources which are expected to supply a
large number of arguments can lead to large method signatures.
In such cases, an `{ArgumentsAccessor}` can be used instead of multiple parameters. Using
this API, you can access the provided arguments through a single argument passed to your
test method. In addition, type conversion is supported as discussed in
<<writing-tests-parameterized-tests-argument-conversion-implicit>>.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=ArgumentsAccessor_example]
----
_An instance of `ArgumentsAccessor` is automatically injected into any parameter of type
`ArgumentsAccessor`._
[[writing-tests-parameterized-tests-argument-aggregation-custom]]
===== Custom Aggregators
Apart from direct access to a `@ParameterizedTest` method's arguments using an
`ArgumentsAccessor`, JUnit Jupiter also supports the usage of custom, reusable
_aggregators_.
To use a custom aggregator, implement the `{ArgumentsAggregator}` interface and register
it via the `@AggregateWith` annotation on a compatible parameter in the
`@ParameterizedTest` method. The result of the aggregation will then be provided as an
argument for the corresponding parameter when the parameterized test is invoked. Note
that an implementation of `ArgumentsAggregator` must be declared as either a top-level
class or as a `static` nested class.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=ArgumentsAggregator_example]
----
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=ArgumentsAggregator_example_PersonAggregator]
----
If you find yourself repeatedly declaring `@AggregateWith(MyTypeAggregator.class)` for
multiple parameterized test methods across your codebase, you may wish to create a custom
_composed annotation_ such as `@CsvToMyType` that is meta-annotated with
`@AggregateWith(MyTypeAggregator.class)`. The following example demonstrates this in
action with a custom `@CsvToPerson` annotation.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=ArgumentsAggregator_with_custom_annotation_example]
----
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=ArgumentsAggregator_with_custom_annotation_example_CsvToPerson]
----
[[writing-tests-parameterized-tests-display-names]]
==== Customizing Display Names
By default, the display name of a parameterized test invocation contains the invocation
index and the `String` representation of all arguments for that specific invocation.
Each of them is preceded by the parameter name (unless the argument is only available via
an `ArgumentsAccessor` or `ArgumentAggregator`), if present in the bytecode (for Java,
test code must be compiled with the `-parameters` compiler flag).
However, you can customize invocation display names via the `name` attribute of the
`@ParameterizedTest` annotation like in the following example.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=custom_display_names]
----
When executing the above method using the `ConsoleLauncher` you will see output similar to
the following.
....
Display name of container ✔
├─ 1 ==> the rank of 'apple' is 1 ✔
├─ 2 ==> the rank of 'banana' is 2 ✔
└─ 3 ==> the rank of 'lemon, lime' is 3 ✔
....
Please note that `name` is a `MessageFormat` pattern. Thus, a single quote (`'`) needs to be represented as a doubled single quote (`''`) in order to be displayed.
The following placeholders are supported within custom display names.
[cols="20,80"]
|===
| Placeholder | Description
| `{displayName}` | the display name of the method
| `{index}` | the current invocation index (1-based)
| `{arguments}` | the complete, comma-separated arguments list
| `{argumentsWithNames}` | the complete, comma-separated arguments list with parameter names
| `{0}`, `{1}`, ... | an individual argument
|===
NOTE: When including arguments in display names, their string representations are truncated
if they exceed the configured maximum length. The limit is configurable via the
`junit.jupiter.params.displayname.argument.maxlength` configuration parameter and defaults
to 512 characters.
When using `@MethodSource` or `@ArgumentSource`, you can give names to arguments. This
name will be used if the argument is included in the invocation display name, like in
the example below.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=named_arguments]
----
....
A parameterized test with named arguments ✔
├─ 1: An important file ✔
└─ 2: Another file ✔
....
If you'd like to set default name pattern for all parameterized tests in your project, you can add the following
configuration to `junit-platform.properties`
[source,properties,indent=0]
----
junit.jupiter.params.displayname.default = {index}
----
the display name for a parameterized method is determined according to the following precedence rules:
1. `name` of `@ParameterizedTest`, if present
2. the value of `junit.jupiter.params.displayname.default` (from junit-platform.properties), if present
3. `DEFAULT_DISPLAY_NAME` constant defined in `@ParameterizedTest`
[[writing-tests-parameterized-tests-lifecycle-interop]]
==== Lifecycle and Interoperability
Each invocation of a parameterized test has the same lifecycle as a regular `@Test`
method. For example, `@BeforeEach` methods will be executed before each invocation.
Similar to <<writing-tests-dynamic-tests>>, invocations will appear one by one in the
test tree of an IDE. You may at will mix regular `@Test` methods and `@ParameterizedTest`
methods within the same test class.
You may use `ParameterResolver` extensions with `@ParameterizedTest` methods. However,
method parameters that are resolved by argument sources need to come first in the
argument list. Since a test class may contain regular tests as well as parameterized
tests with different parameter lists, values from argument sources are not resolved for
lifecycle methods (e.g. `@BeforeEach`) and test class constructors.
[source,java,indent=0]
----
include::{testDir}/example/ParameterizedTestDemo.java[tags=ParameterResolver_example]
----
[[writing-tests-test-templates]]
=== Test Templates
A `{TestTemplate}` method is not a regular test case but rather a template for test
cases. As such, it is designed to be invoked multiple times depending on the number of
invocation contexts returned by the registered providers. Thus, it must be used in
conjunction with a registered `{TestTemplateInvocationContextProvider}` extension. Each
invocation of a test template method behaves like the execution of a regular `@Test`
method with full support for the same lifecycle callbacks and extensions. Please refer to
<<extensions-test-templates>> for usage examples.
NOTE: <<writing-tests-repeated-tests>> and <<writing-tests-parameterized-tests>> are
built-in specializations of test templates.
[[writing-tests-dynamic-tests]]
=== Dynamic Tests
The standard `@Test` annotation in JUnit Jupiter described in
<<writing-tests-annotations>> is very similar to the `@Test` annotation in JUnit 4. Both
describe methods that implement test cases. These test cases are static in the sense that
they are fully specified at compile time, and their behavior cannot be changed by
anything happening at runtime. _Assumptions provide a basic form of dynamic behavior but
are intentionally rather limited in their expressiveness._
In addition to these standard tests a completely new kind of test programming model has
been introduced in JUnit Jupiter. This new kind of test is a _dynamic test_ which is
generated at runtime by a factory method that is annotated with `@TestFactory`.
In contrast to `@Test` methods, a `@TestFactory` method is not itself a test case but
rather a factory for test cases. Thus, a dynamic test is the product of a factory.
Technically speaking, a `@TestFactory` method must return a single `DynamicNode` or a
`Stream`, `Collection`, `Iterable`, `Iterator`, or array of `DynamicNode` instances.
Instantiable subclasses of `DynamicNode` are `DynamicContainer` and `DynamicTest`.
`DynamicContainer` instances are composed of a _display name_ and a list of dynamic child
nodes, enabling the creation of arbitrarily nested hierarchies of dynamic nodes.
`DynamicTest` instances will be executed lazily, enabling dynamic and even
non-deterministic generation of test cases.
Any `Stream` returned by a `@TestFactory` will be properly closed by calling
`stream.close()`, making it safe to use a resource such as `Files.lines()`.
As with `@Test` methods, `@TestFactory` methods must not be `private` or `static` and may
optionally declare parameters to be resolved by `ParameterResolvers`.
A `DynamicTest` is a test case generated at runtime. It is composed of a _display name_
and an `Executable`. `Executable` is a `@FunctionalInterface` which means that the
implementations of dynamic tests can be provided as _lambda expressions_ or _method
references_.
.Dynamic Test Lifecycle
WARNING: The execution lifecycle of a dynamic test is quite different than it is for a
standard `@Test` case. Specifically, there are no lifecycle callbacks for individual
dynamic tests. This means that `@BeforeEach` and `@AfterEach` methods and their
corresponding extension callbacks are executed for the `@TestFactory` method but not for
each _dynamic test_. In other words, if you access fields from the test instance within a
lambda expression for a dynamic test, those fields will not be reset by callback methods
or extensions between the execution of individual dynamic tests generated by the same
`@TestFactory` method.
As of JUnit Jupiter {jupiter-version}, dynamic tests must always be created by factory
methods; however, this might be complemented by a registration facility in a later
release.
[[writing-tests-dynamic-tests-examples]]
==== Dynamic Test Examples
The following `DynamicTestsDemo` class demonstrates several examples of test factories
and dynamic tests.
The first method returns an invalid return type. Since an invalid return type cannot be
detected at compile time, a `JUnitException` is thrown when it is detected at runtime.
The next six methods are very simple examples that demonstrate the generation of a
`Collection`, `Iterable`, `Iterator`, array, or `Stream` of `DynamicTest` instances. Most of
these examples do not really exhibit dynamic behavior but merely demonstrate the
supported return types in principle. However, `dynamicTestsFromStream()` and
`dynamicTestsFromIntStream()` demonstrate how easy it is to generate dynamic tests for a
given set of strings or a range of input numbers.
The next method is truly dynamic in nature. `generateRandomNumberOfTests()` implements an
`Iterator` that generates random numbers, a display name generator, and a test executor
and then provides all three to `DynamicTest.stream()`. Although the non-deterministic
behavior of `generateRandomNumberOfTests()` is of course in conflict with test
repeatability and should thus be used with care, it serves to demonstrate the
expressiveness and power of dynamic tests.
The next method is similar to `generateRandomNumberOfTests()` in terms of flexibility;
however, `dynamicTestsFromStreamFactoryMethod()` generates a stream of dynamic tests from
an existing `Stream` via the `DynamicTest.stream()` factory method.
For demonstration purposes, the `dynamicNodeSingleTest()` method generates a single
`DynamicTest` instead of a stream, and the `dynamicNodeSingleContainer()` method generates
a nested hierarchy of dynamic tests utilizing `DynamicContainer`.
[source,java]
----
include::{testDir}/example/DynamicTestsDemo.java[tags=user_guide]
----
[[writing-tests-dynamic-tests-uri-test-source]]
==== URI Test Sources for Dynamic Tests
The JUnit Platform provides `TestSource`, a representation of the source of a test or
container used to navigate to its location by IDEs and build tools.
The `TestSource` for a dynamic test or dynamic container can be constructed from a
`java.net.URI` which can be supplied via the `DynamicTest.dynamicTest(String, URI,
Executable)` or `DynamicContainer.dynamicContainer(String, URI, Stream)` factory method,
respectively. The `URI` will be converted to one of the following `TestSource`
implementations.
`ClasspathResourceSource` ::
If the `URI` contains the `classpath` scheme -- for example,
`classpath:/test/foo.xml?line=20,column=2`.
`DirectorySource` ::
If the `URI` represents a directory present in the file system.
`FileSource` ::
If the `URI` represents a file present in the file system.
`MethodSource` ::
If the `URI` contains the `method` scheme and the fully qualified method name (FQMN) --
for example, `method:org.junit.Foo#bar(java.lang.String, java.lang.String[])`. Please
refer to the Javadoc for `DiscoverySelectors.selectMethod(String)` for the supported
formats for a FQMN.
`ClassSource` ::
If the `URI` contains the `class` scheme and the fully qualified class name --
for example, `class:org.junit.Foo?line=42`.
`UriSource` ::
If none of the above `TestSource` implementations are applicable.
[[writing-tests-declarative-timeouts]]
=== Timeouts
The `@Timeout` annotation allows one to declare that a test, test factory, test template,
or lifecycle method should fail if its execution time exceeds a given duration. The time
unit for the duration defaults to seconds but is configurable.
The following example shows how `@Timeout` is applied to lifecycle and test methods.
[source,java]
----
include::{testDir}/example/TimeoutDemo.java[tags=user_guide]
----
Contrary to the `assertTimeoutPreemptively()` assertion, the execution of the annotated
method proceeds in the main thread of the test. If the timeout is exceeded, the main
thread is interrupted from another thread. This is done to ensure interoperability with
frameworks such as Spring that make use of mechanisms that are sensitive to the currently
running thread — for example, `ThreadLocal` transaction management.
To apply the same timeout to all test methods within a test class and all of its `@Nested`
classes, you can declare the `@Timeout` annotation at the class level. It will then be
applied to all test, test factory, and test template methods within that class and its
`@Nested` classes unless overridden by a `@Timeout` annotation on a specific method or
`@Nested` class. Please note that `@Timeout` annotations declared at the class level are
not applied to lifecycle methods.
Declaring `@Timeout` on a `@TestFactory` method checks that the factory method returns
within the specified duration but does not verify the execution time of each individual
`DynamicTest` generated by the factory. Please use
`assertTimeout()` or `assertTimeoutPreemptively()` for that purpose.
If `@Timeout` is present on a `@TestTemplate` method — for example, a `@RepeatedTest` or
`@ParameterizedTest` — each invocation will have the given timeout applied to it.
The following <<running-tests-config-params, configuration parameters>> can be used to
specify global timeouts for all methods of a certain category unless they or an enclosing
test class is annotated with `@Timeout`:
`junit.jupiter.execution.timeout.default`::
Default timeout for all testable and lifecycle methods
`junit.jupiter.execution.timeout.testable.method.default`::
Default timeout for all testable methods
`junit.jupiter.execution.timeout.test.method.default`::
Default timeout for `@Test` methods
`junit.jupiter.execution.timeout.testtemplate.method.default`::
Default timeout for `@TestTemplate` methods
`junit.jupiter.execution.timeout.testfactory.method.default`::
Default timeout for `@TestFactory` methods
`junit.jupiter.execution.timeout.lifecycle.method.default`::
Default timeout for all lifecycle methods
`junit.jupiter.execution.timeout.beforeall.method.default`::
Default timeout for `@BeforeAll` methods
`junit.jupiter.execution.timeout.beforeeach.method.default`::
Default timeout for `@BeforeEach` methods
`junit.jupiter.execution.timeout.aftereach.method.default`::
Default timeout for `@AfterEach` methods
`junit.jupiter.execution.timeout.afterall.method.default`::
Default timeout for `@AfterAll` methods
More specific configuration parameters override less specific ones. For example,
`junit.jupiter.execution.timeout.test.method.default` overrides
`junit.jupiter.execution.timeout.testable.method.default` which overrides
`junit.jupiter.execution.timeout.default`.
The values of such configuration parameters must be in the following, case-insensitive
format: `<number> [ns|μs|ms|s|m|h|d]`. The space between the number and the unit may be
omitted. Specifying no unit is equivalent to using seconds.
.Example timeout configuration parameter values
[cols="20,80"]
|===
| Parameter value | Equivalent annotation
| `42` | `@Timeout(42)`
| `42 ns` | `@Timeout(value = 42, unit = NANOSECONDS)`
| `42 μs` | `@Timeout(value = 42, unit = MICROSECONDS)`
| `42 ms` | `@Timeout(value = 42, unit = MILLISECONDS)`
| `42 s` | `@Timeout(value = 42, unit = SECONDS)`
| `42 m` | `@Timeout(value = 42, unit = MINUTES)`
| `42 h` | `@Timeout(value = 42, unit = HOURS)`
| `42 d` | `@Timeout(value = 42, unit = DAYS)`
|===
[[writing-tests-declarative-timeouts-polling]]
==== Using @Timeout for Polling Tests
When dealing with asynchronous code, it is common to write tests that poll while waiting
for something to happen before performing any assertions. In some cases you can rewrite
the logic to use a `CountDownLatch` or another synchronization mechanism, but sometimes
that is not possible — for example, if the subject under test sends a message to a channel
in an external message broker and assertions cannot be performed until the message has
been successfully sent through the channel. Asynchronous tests like these require some
form of timeout to ensure they don't hang the test suite by executing indefinitely, as
would be the case if an asynchronous message never gets successfully delivered.
By configuring a timeout for an asynchronous test that polls, you can ensure that the test
does not execute indefinitely. The following example demonstrates how to achieve this with
JUnit Jupiter's `@Timeout` annotation. This technique can be used to implement "poll
until" logic very easily.
[source,java]
----
include::{testDir}/example/PollingTimeoutDemo.java[tags=user_guide,indent=0]
----
NOTE: If you need more control over polling intervals and greater flexibility with
asynchronous tests, consider using a dedicated library such as
link:https://github.com/awaitility/awaitility[Awaitility].
[[writing-tests-declarative-timeouts-mode]]
==== Disable @Timeout Globally
When stepping through your code in a debug session, a fixed timeout limit may influence
the result of the test, e.g. mark the test as failed although all assertions were met.
JUnit Jupiter supports the `junit.jupiter.execution.timeout.mode` configuration parameter
to configure when timeouts are applied. There are three modes: `enabled`, `disabled`,
and `disabled_on_debug`. The default mode is `enabled`.
A VM runtime is considered to run in debug mode when one of its input parameters starts
with `-agentlib:jdwp`. This heuristic is queried by the `disabled_on_debug` mode.
[[writing-tests-parallel-execution]]
=== Parallel Execution
.Parallel test execution is an experimental feature
WARNING: You're invited to give it a try and provide feedback to the JUnit team so they
can improve and eventually <<api-evolution, promote>> this feature.
By default, JUnit Jupiter tests are run sequentially in a single thread. Running tests in
parallel -- for example, to speed up execution -- is available as an opt-in feature since
version 5.3. To enable parallel execution, set the
`junit.jupiter.execution.parallel.enabled` configuration parameter to `true` -- for
example, in `junit-platform.properties` (see <<running-tests-config-params>> for other
options).
Please note that enabling this property is only the first step required to execute tests
in parallel. If enabled, test classes and methods will still be executed sequentially by
default. Whether or not a node in the test tree is executed concurrently is controlled by
its execution mode. The following two modes are available.
`SAME_THREAD`::
Force execution in the same thread used by the parent. For example, when used on a test
method, the test method will be executed in the same thread as any `@BeforeAll` or
`@AfterAll` methods of the containing test class.
`CONCURRENT`::
Execute concurrently unless a resource lock forces execution in the same thread.
By default, nodes in the test tree use the `SAME_THREAD` execution mode. You can change
the default by setting the `junit.jupiter.execution.parallel.mode.default` configuration
parameter. Alternatively, you can use the `{Execution}` annotation to change the
execution mode for the annotated element and its subelements (if any) which allows you to
activate parallel execution for individual test classes, one by one.
[source,properties]
.Configuration parameters to execute all tests in parallel
----
junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = concurrent
----
The default execution mode is applied to all nodes of the test tree with a few notable
exceptions, namely test classes that use the `Lifecycle.PER_CLASS` mode or a
`{MethodOrderer}` (except for `{MethodOrderer_Random}`). In the former case, test authors
have to ensure that the test class is thread-safe; in the latter, concurrent execution
might conflict with the configured execution order. Thus, in both cases, test methods in
such test classes are only executed concurrently if the `@Execution(CONCURRENT)`
annotation is present on the test class or method.
All nodes of the test tree that are configured with the `CONCURRENT` execution mode will
be executed fully in parallel according to the provided
<<writing-tests-parallel-execution-config, configuration>> while observing the
declarative <<writing-tests-parallel-execution-synchronization, synchronization>>
mechanism. Please note that <<running-tests-capturing-output>> needs to be enabled
separately.
In addition, you can configure the default execution mode for top-level classes by setting
the `junit.jupiter.execution.parallel.mode.classes.default` configuration parameter. By
combining both configuration parameters, you can configure classes to run in parallel but
their methods in the same thread:
[source,properties]
.Configuration parameters to execute top-level classes in parallel but methods in same thread
----
junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = same_thread
junit.jupiter.execution.parallel.mode.classes.default = concurrent
----
The opposite combination will run all methods within one class in parallel, but top-level
classes will run sequentially:
[source,properties]
.Configuration parameters to execute top-level classes sequentially but their methods in parallel
----
junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = concurrent
junit.jupiter.execution.parallel.mode.classes.default = same_thread
----
The following diagram illustrates how the execution of two top-level test classes `A` and
`B` with two test methods per class behaves for all four combinations of
`junit.jupiter.execution.parallel.mode.default` and
`junit.jupiter.execution.parallel.mode.classes.default` (see labels in first column).
////
Source: https://mermaidjs.github.io/mermaid-live-editor/#/view/eyJjb2RlIjoiZ2FudHRcbiAgICBkYXRlRm9ybWF0ICBZWVlZLU1NLUREXG5cbiAgICBzZWN0aW9uIChzYW1lX3RocmVhZCwgc2FtZV90aHJlYWQpXG4gICAgQS50ZXN0MSgpIDphc3MxLCAyMDE5LTAxLTAxLCAxZFxuICAgIEEudGVzdDIoKSA6YXNzMiwgYWZ0ZXIgYXNzMSwgMWRcbiAgICBCLnRlc3QxKCkgOmJzczEsIGFmdGVyIGFzczIsIDFkXG4gICAgQi50ZXN0MigpIDpic3MyLCBhZnRlciBic3MxLCAxZFxuXG4gICAgc2VjdGlvbiAoc2FtZV90aHJlYWQsIGNvbmN1cnJlbnQpXG4gICAgQS50ZXN0MSgpIDphc2MxLCAyMDE5LTAxLTAxLCAxZFxuICAgIEEudGVzdDIoKSA6YXNjMiwgYWZ0ZXIgYXNjMSwgMWRcbiAgICBCLnRlc3QxKCkgOmJzYzEsIDIwMTktMDEtMDEsIDFkXG4gICAgQi50ZXN0MigpIDpic2MyLCBhZnRlciBic2MxLCAxZFxuXG4gICAgc2VjdGlvbiAoY29uY3VycmVudCwgc2FtZV90aHJlYWQpXG4gICAgQS50ZXN0MSgpIDphY3MxLCAyMDE5LTAxLTAxLCAxZFxuICAgIEEudGVzdDIoKSA6YWNzMiwgMjAxOS0wMS0wMSwgMWRcbiAgICBCLnRlc3QxKCkgOmJjczEsIGFmdGVyIGFjczIsIDFkXG4gICAgQi50ZXN0MigpIDpiY3MyLCBhZnRlciBhY3MyLCAxZFxuXG4gICAgc2VjdGlvbiAoY29uY3VycmVudCwgY29uY3VycmVudClcbiAgICBBLnRlc3QxKCkgOmFjYzEsIDIwMTktMDEtMDEsIDFkXG4gICAgQS50ZXN0MigpIDphY2MyLCAyMDE5LTAxLTAxLCAxZFxuICAgIEIudGVzdDEoKSA6YmNjMSwgMjAxOS0wMS0wMSwgMWRcbiAgICBCLnRlc3QyKCkgOmJjYzIsIDIwMTktMDEtMDEsIDFkXG4iLCJtZXJtYWlkIjp7InRoZW1lIjoibmV1dHJhbCIsImdhbnR0Ijp7ImxlZnRQYWRkaW5nIjoyMjUsImJhckdhcCI6NSwiZ3JpZExpbmVTdGFydFBhZGRpbmciOjEwLCJiYXJIZWlnaHQiOjMwLCJmb250U2l6ZSI6MTV9LCJ0aGVtZUNTUyI6Ii50YXNrVGV4dCwgLnNlY3Rpb25UaXRsZSB7IGZvbnQtZmFtaWx5OiAnT3BlbiBTYW5zJzsgZm9udC1zaXplOjE1cHggfSAuZ3JpZCAudGljayB0ZXh0IHsgZGlzcGxheTpub25lIH0gLmdyaWQgLnRpY2s6bnRoLWNoaWxkKDJuKzEpIHsgZGlzcGxheTpub25lIH0ifX0
gantt
dateFormat YYYY-MM-DD
section (same_thread, same_thread)
A.test1() :ass1, 2019-01-01, 1d
A.test2() :ass2, after ass1, 1d
B.test1() :bss1, after ass2, 1d
B.test2() :bss2, after bss1, 1d
section (same_thread, concurrent)
A.test1() :asc1, 2019-01-01, 1d
A.test2() :asc2, after asc1, 1d
B.test1() :bsc1, 2019-01-01, 1d
B.test2() :bsc2, after bsc1, 1d
section (concurrent, same_thread)
A.test1() :acs1, 2019-01-01, 1d
A.test2() :acs2, 2019-01-01, 1d
B.test1() :bcs1, after acs2, 1d
B.test2() :bcs2, after acs2, 1d
section (concurrent, concurrent)
A.test1() :acc1, 2019-01-01, 1d
A.test2() :acc2, 2019-01-01, 1d
B.test1() :bcc1, 2019-01-01, 1d
B.test2() :bcc2, 2019-01-01, 1d
////
image::writing-tests_execution_mode.svg[caption='',title='Default execution mode configuration combinations']
If the `junit.jupiter.execution.parallel.mode.classes.default` configuration parameter is
not explicitly set, the value for `junit.jupiter.execution.parallel.mode.default` will be
used instead.
[[writing-tests-parallel-execution-config]]
==== Configuration
Properties such as the desired parallelism and the maximum pool size can be configured
using a `{ParallelExecutionConfigurationStrategy}`. The JUnit Platform provides two
implementations out of the box: `dynamic` and `fixed`. Alternatively, you may implement a
`custom` strategy.
To select a strategy, set the `junit.jupiter.execution.parallel.config.strategy`
configuration parameter to one of the following options.
`dynamic`::
Computes the desired parallelism based on the number of available processors/cores
multiplied by the `junit.jupiter.execution.parallel.config.dynamic.factor`
configuration parameter (defaults to `1`).
`fixed`::
Uses the mandatory `junit.jupiter.execution.parallel.config.fixed.parallelism`
configuration parameter as the desired parallelism.
`custom`::
Allows you to specify a custom `{ParallelExecutionConfigurationStrategy}`
implementation via the mandatory `junit.jupiter.execution.parallel.config.custom.class`
configuration parameter to determine the desired configuration.
If no configuration strategy is set, JUnit Jupiter uses the `dynamic` configuration
strategy with a factor of `1`. Consequently, the desired parallelism will be equal to the
number of available processors/cores.
.Parallelism does not imply maximum number of concurrent threads
NOTE: JUnit Jupiter does not guarantee that the number of concurrently executing tests
will not exceed the configured parallelism. For example, when using one of the
synchronization mechanisms described in the next section, the `ForkJoinPool` that is used
behind the scenes may spawn additional threads to ensure execution continues with
sufficient parallelism. Thus, if you require such guarantees in a test class, please use
your own means of controlling concurrency.
[[writing-tests-parallel-execution-synchronization]]
==== Synchronization
In addition to controlling the execution mode using the `{Execution}` annotation, JUnit
Jupiter provides another annotation-based declarative synchronization mechanism. The
`{ResourceLock}` annotation allows you to declare that a test class or method uses a
specific shared resource that requires synchronized access to ensure reliable test
execution. The shared resource is identified by a unique name which is a `String`. The
name can be user-defined or one of the predefined constants in `{Resources}`:
`SYSTEM_PROPERTIES`, `SYSTEM_OUT`, `SYSTEM_ERR`, `LOCALE`, or `TIME_ZONE`.
If the tests in the following example were run in parallel _without_ the use of
{ResourceLock}, they would be _flaky_. Sometimes they would pass, and at other times they
would fail due to the inherent race condition of writing and then reading the same JVM
System Property.
When access to shared resources is declared using the `{ResourceLock}` annotation, the
JUnit Jupiter engine uses this information to ensure that no conflicting tests are run in
parallel.
[NOTE]
.Running tests in isolation
====
If most of your test classes can be run in parallel without any synchronization but you
have some test classes that need to run in isolation, you can mark the latter with the
`{Isolated}` annotation. Tests in such classes are executed sequentially without any other
tests running at the same time.
====
In addition to the `String` that uniquely identifies the shared resource, you may specify
an access mode. Two tests that require `READ` access to a shared resource may run in
parallel with each other but not while any other test that requires `READ_WRITE` access
to the same shared resource is running.
[source,java]
----
include::{testDir}/example/SharedResourcesDemo.java[tags=user_guide]
----
[[writing-tests-built-in-extensions]]
=== Built-in Extensions
While the JUnit team encourages reusable extensions to be packaged and maintained in
separate libraries, the JUnit Jupiter API artifact includes a few user-facing extension
implementations that are considered so generally useful that users shouldn't have to add
another dependency.
[[writing-tests-built-in-extensions-TempDirectory]]
==== The TempDirectory Extension
.`@TempDir` is an experimental feature
WARNING: You're invited to give it a try and provide feedback to the JUnit team so they
can improve and eventually <<api-evolution, promote>> this feature.
The built-in `{TempDirectory}` extension is used to create and clean up a temporary
directory for an individual test or all tests in a test class. It is registered by
default. To use it, annotate a non-final, unassigned field of type `java.nio.file.Path` or
`java.io.File` with `{TempDir}` or add a parameter of type `java.nio.file.Path` or
`java.io.File` annotated with `@TempDir` to a lifecycle method or test method.
For example, the following test declares a parameter annotated with `@TempDir` for a
single test method, creates and writes to a file in the temporary directory, and checks
its content.
[source,java,indent=0]
.A test method that requires a temporary directory
----
include::{testDir}/example/TempDirectoryDemo.java[tags=user_guide_parameter_injection]
----
You can inject multiple temporary directories by specifying multiple annotated parameters.
[source,java,indent=0]
.A test method that requires multiple temporary directories
----
include::{testDir}/example/TempDirectoryDemo.java[tags=user_guide_multiple_directories]
----
WARNING: To revert to the old behavior of using a single temporary directory for the
entire test class or method (depending on which level the annotation is used), you can set
the `junit.jupiter.tempdir.scope` configuration parameter to `per_context`. However,
please note that this option is deprecated and will be removed in a future release.
`@TempDir` is not supported on constructor parameters. If you wish to retain a single
reference to a temp directory across lifecycle methods and the current test method, please
use field injection by annotating an instance field with `@TempDir`.
The following example stores a _shared_ temporary directory in a `static` field. This
allows the same `sharedTempDir` to be used in all lifecycle methods and test methods of
the test class. For better isolation, you should use an instance field so that each test
method uses a separate directory.
[source,java,indent=0]
.A test class that shares a temporary directory across test methods
----
include::{testDir}/example/TempDirectoryDemo.java[tags=user_guide_field_injection]
----
The `@TempDir` annotation has an optional `cleanup` attribute that can be set to either
`NEVER`, `ON_SUCCESS`, or `ALWAYS`. If the cleanup mode is set to `NEVER`, temporary
directories are not deleted after a test completes. If it is set to `ON_SUCCESS`,
temporary directories are deleted only after a test completed successfully.
The default cleanup mode is `ALWAYS`. You can use the
`junit.jupiter.tempdir.cleanup.mode.default`
<<running-tests-config-params, configuration parameter>> to override this default.
[source,java,indent=0]
.A test class with a temporary directory that doesn't get cleaned up
----
include::{testDir}/example/TempDirCleanupModeDemo.java[tags=user_guide]
----