docs/src/main/docbook/test-framework.xml

<?xml version="1.0"?>
<!--

    Copyright (c) 2012, 2020 Oracle and/or its affiliates. All rights reserved.

    This program and the accompanying materials are made available under the
    terms of the Eclipse Public License v. 2.0, which is available at
    http://www.eclipse.org/legal/epl-2.0.

    This Source Code may also be made available under the following Secondary
    Licenses when the conditions for such availability set forth in the
    Eclipse Public License v. 2.0 are satisfied: GNU General Public License,
    version 2 with the GNU Classpath Exception, which is available at
    https://www.gnu.org/software/classpath/license.html.

    SPDX-License-Identifier: EPL-2.0 OR GPL-2.0 WITH Classpath-exception-2.0

-->

<!DOCTYPE chapter [<!ENTITY % ents SYSTEM "jersey.ent" > %ents; ]>
<chapter xmlns="http://docbook.org/ns/docbook"
         version="5.0"
         xml:lang="en"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xmlns:xi="http://www.w3.org/2001/XInclude"
         xmlns:xlink="http://www.w3.org/1999/xlink"
         xsi:schemaLocation="http://docbook.org/ns/docbook http://docbook.org/xml/5.0/xsd/docbook.xsd
                             http://www.w3.org/1999/xlink http://www.w3.org/1999/xlink.xsd"
         xml:id="test-framework">

    <title>Jersey Test Framework</title>

    <para>
        Jersey Test Framework originated as an internal tool used for verifying the correct implementation of
        server-side components. Testing RESTful applications became a more pressing issue with "modern" approaches like
        test-driven development and users started to look for a tool that could help with designing and running
        the tests as fast as possible but with many options related to test execution environment.
    </para>

    <para>
        Current implementation of Jersey Test Framework supports the following set of features:

        <itemizedlist>
            <listitem><para>pre-configured client to access deployed application</para></listitem>
            <listitem><para>support for multiple containers - grizzly, in-memory, jdk, simple, jetty</para></listitem>
            <listitem><para>able to run against any external container</para></listitem>
            <listitem><para>automated configurable traffic logging</para></listitem>
        </itemizedlist>

        Jersey Test Framework is primarily based on JUnit but you can run tests using TestNG as well. It works almost out-of-the
        box and it is easy to integrate it within your Maven-based project. While it is usable on all environments where you can
        run JUnit, we support primarily the Maven-based setups.
    </para>

    <section>
        <title>Basics</title>

        <programlisting language="java" linenumbering="numbered">public class SimpleTest extends JerseyTest {

    @Path("hello")
    public static class HelloResource {
        @GET
        public String getHello() {
            return "Hello World!";
        }
    }

    @Override
    protected Application configure() {
        return new ResourceConfig(HelloResource.class);
    }

    @Test
    public void test() {
        final String hello = target("hello").request().get(String.class);
        assertEquals("Hello World!", hello);
    }
}</programlisting>

        <para>
            If you want to develop a test using Jersey Test Framework, you need to subclass &jersey.test.JerseyTest; and
            configure the set of resources and/or providers that will be deployed as part of the test application. This short
            code snippet shows basic resource class <literal>HelloResource</literal> used in tests defined as part of the
            <literal>SimpleTest</literal> class. The overridden <literal>configure</literal> method returns
            a &jersey.server.ResourceConfig; of the test application,that contains only the <literal>HelloResource</literal>
            resource class. &lit.jersey.server.ResourceConfig; is a sub-class of JAX-RS &jaxrs.core.Application;. It is a Jersey
            convenience class for configuring JAX-RS applications. &lit.jersey.server.ResourceConfig; also implements JAX-RS
            &jaxrs.core.Configurable; interface to make the application configuration more flexible.
        </para>
    </section>

    <section>
        <title>Supported Containers</title>

        <para>
            &jersey.test.JerseyTest; supports deploying applications on various containers, all (except the external container
            wrapper) need to have some "glue" code to be supported. Currently Jersey Test Framework provides support for
            Grizzly, In-Memory, JDK (<literal>com.sun.net.httpserver.HttpServer</literal>), Simple HTTP container
            (<literal>org.simpleframework.http</literal>) and Jetty HTTP container (<literal>org.eclipse.jetty</literal>).
        </para>

        <para>
            A test container is selected based on various inputs.
            <link xlink:href="&jersey.javadoc.uri.prefix;/test/JerseyTest.html#getTestContainerFactory()">JerseyTest#getTestContainerFactory()</link>
            is always executed, so if you override it and provide your own version of
            &jersey.test.spi.TestContainerFactory;, nothing else will be considered.
            Setting a system variable
            <link xlink:href="&jersey.javadoc.uri.prefix;/test/TestProperties.html#CONTAINER_FACTORY">TestProperties#CONTAINER_FACTORY</link>
            has similar effect. This way you may defer the decision on which containers you want to run your tests
            from the compile time to the test execution time.
            Default implementation of &lit.jersey.test.spi.TestContainerFactory; looks for container factories on classpath.
            If more than one instance is found and there is a Grizzly test container factory among them, it will be used; if not,
            a warning will be logged and the first found factory will be instantiated.
        </para>

        <para>
            Following is a brief description of all container factories supported in Jersey Test Framework.
            <itemizedlist>
                <listitem>
                    <para>
                        Jersey provides 2 different test container factories based on Grizzly.
                        The <literal>GrizzlyTestContainerFactory</literal> creates a container that can run as a light-weight,
                        plain HTTP container. Almost all Jersey tests are using Grizzly HTTP test container factory.
                        Second factory is <literal>GrizzlyWebTestContainerFactory</literal> that is Servlet-based and supports
                        Servlet deployment context for tested applications. This factory can be useful when testing more complex
                        Servlet-based application deployments.

                        <programlisting language="xml">&lt;dependency&gt;
    &lt;groupId&gt;org.glassfish.jersey.test-framework.providers&lt;/groupId&gt;
    &lt;artifactId&gt;jersey-test-framework-provider-grizzly2&lt;/artifactId&gt;
    &lt;version&gt;&version;&lt;/version&gt;
&lt;/dependency&gt;</programlisting>
                    </para>
                </listitem>

                <listitem>
                    <para>
                        In-Memory container is not a real container. It starts Jersey application and directly calls internal
                        APIs to handle request created by client provided by test framework. There is no network communication
                        involved. This containers does not support servlet and other container dependent features, but it is
                        a perfect choice for simple unit tests.

                        <programlisting language="xml">&lt;dependency&gt;
    &lt;groupId&gt;org.glassfish.jersey.test-framework.providers&lt;/groupId&gt;
    &lt;artifactId&gt;jersey-test-framework-provider-inmemory&lt;/artifactId&gt;
    &lt;version&gt;&version;&lt;/version&gt;
&lt;/dependency&gt;</programlisting>
                    </para>
                </listitem>
                <listitem>
                    <para>
                        <literal>HttpServer</literal> from Oracle JDK is another supported test container.

                        <programlisting language="xml">&lt;dependency&gt;
    &lt;groupId&gt;org.glassfish.jersey.test-framework.providers&lt;/groupId&gt;
    &lt;artifactId&gt;jersey-test-framework-provider-jdk-http&lt;/artifactId&gt;
    &lt;version&gt;&version;&lt;/version&gt;
&lt;/dependency&gt;</programlisting>
                    </para>
                </listitem>
                <listitem>
                    <para>
                        Simple container (<literal>org.simpleframework.http</literal>) is another light-weight HTTP container
                        that integrates with Jersey and is supported by Jersey Test Framework.

                        <programlisting language="xml">&lt;dependency&gt;
    &lt;groupId&gt;org.glassfish.jersey.test-framework.providers&lt;/groupId&gt;
    &lt;artifactId&gt;jersey-test-framework-provider-simple&lt;/artifactId&gt;
    &lt;version&gt;&version;&lt;/version&gt;
&lt;/dependency&gt;</programlisting>
                    </para>
                </listitem>
                <listitem>
                    <para>
                        Jetty container (<literal>org.eclipse.jetty</literal>) is another high-performance, light-weight HTTP server
                        that integrates with Jersey and is supported by Jersey Test Framework.

                        <programlisting language="xml">&lt;dependency&gt;
    &lt;groupId&gt;org.glassfish.jersey.test-framework.providers&lt;/groupId&gt;
    &lt;artifactId&gt;jersey-test-framework-provider-jetty&lt;/artifactId&gt;
    &lt;version&gt;&version;&lt;/version&gt;
&lt;/dependency&gt;</programlisting>
                    </para>
                </listitem>
            </itemizedlist>
        </para>
    </section>

    <section xml:id="testng">
        <title>Running TestNG Tests</title>

        <para>
            It is possible to run not only JUnit tests but also tests based on TestNG. In order to do this you need to make sure
            the following 2 steps are fulfilled:

            <itemizedlist>
                <listitem>
                    <para>
                        Extend &jersey.test.JerseyTestNg;, or one of its inner classes &jersey.test.JerseyTestNg.ContainerPerClassTest;
                        / &jersey.test.JerseyTestNg.ContainerPerMethodTest;, instead of &jersey.test.JerseyTest;.
                    </para>
                </listitem>
                <listitem>
                    <para>
                        Add TestNG to your class-patch, i.e.:

                        <programlisting language="xml">&lt;dependency&gt;
    &lt;groupId&gt;org.glassfish.jersey.test-framework&lt;/groupId&gt;
    &lt;artifactId&gt;jersey-test-framework-core&lt;/artifactId&gt;
    &lt;version&gt;&version;&lt;/version&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
    &lt;groupId&gt;org.testng&lt;/groupId&gt;
    &lt;artifactId&gt;testng&lt;/artifactId&gt;
    &lt;version&gt;...&lt;/version&gt;
&lt;/dependency&gt;</programlisting>
                    </para>
                </listitem>
            </itemizedlist>
        </para>
        <para>
            To discuss the former requirement in more depth we need to take a look at the differences between JUnit and TestNG.
            JUnit creates a new instance of a test class for every test present in that class which, from the point of view of
            Jersey Test Framework, means that new test container and client is created for each test of a test class. However,
            TestNG creates only one instance of a test class and the initialization of the test container depends more on
            setup/teardown methods (driven by <literal>@BeforeXXX</literal> and <literal>@AfterXXX</literal> annotations) than in
            JUnit. This means that, basically, you can start one instance of test container for all tests present in a test class
            or separate test container for each and every test. For this reason a separate subclasses of &jersey.test.JerseyTestNg;
            have been created:

            <itemizedlist>
                <listitem>
                    <para>
                        &jersey.test.JerseyTestNg.ContainerPerClassTest; creates one container to run all the tests in. Setup method
                        is annotated with <literal>@BeforeClass</literal>, teardown method with <literal>@AfterClass</literal>.
                    </para>
                    <para>
                        For example take a look at <literal>ContainerPerClassTest</literal> test. It contains two test
                        methods (<literal>first</literal> and <literal>second</literal>), one singleton resource that returns an
                        increasing sequence of number. Since we spawn only one instance of a test container for the whole class the
                        value expected in the first test is <literal>1</literal> and in the second it's <literal>2</literal>.

                        <programlisting language="java" linenumbering="numbered">public class ContainerPerClassTest extends JerseyTestNg.ContainerPerClassTest {

    @Path("/")
    @Singleton
    @Produces("text/plain")
    public static class Resource {

        private int i = 1;

        @GET
        public int get() {
            return i++;
        }
    }

    @Override
    protected Application configure() {
        return new ResourceConfig(Resource.class);
    }

    @Test(priority = 1)
    public void first() throws Exception {
        test(1);
    }

    @Test(priority = 2)
    public void second() throws Exception {
        test(2);
    }

    private void test(final Integer expected) {
        final Response response = target().request().get();

        assertEquals(response.getStatus(), 200);
        assertEquals(response.readEntity(Integer.class), expected);
    }
}</programlisting>
                    </para>
                </listitem>
                <listitem>
                    <para>
                        &jersey.test.JerseyTestNg.ContainerPerMethodTest; creates separate container for each test. Setup method
                        is annotated with <literal>@BeforeMethod</literal>, teardown method with <literal>@AfterMethod</literal>.
                    </para>
                    <para>
                        We can create a similar test to the previous one. Take a look at <literal>ContainerPerMethodTest</literal>
                        test. It looks the same except the expected values and extending class: it contains two test
                        methods (<literal>first</literal> and <literal>second</literal>), one singleton resource that returns an
                        increasing sequence of number. In this case we create a separate test container for each test so
                        value expected in the first test is <literal>1</literal> and in the second it's also <literal>1</literal>.

                        <programlisting language="java" linenumbering="numbered">public class ContainerPerMethodTest extends JerseyTestNg.ContainerPerMethodTest {

    @Path("/")
    @Singleton
    @Produces("text/plain")
    public static class Resource {

        private int i = 1;

        @GET
        public int get() {
            return i++;
        }
    }

    @Override
    protected Application configure() {
        return new ResourceConfig(Resource.class);
    }

    @Test
    public void first() throws Exception {
        test(1);
    }

    @Test
    public void second() throws Exception {
        test(1);
    }

    private void test(final Integer expected) {
        final Response response = target().request().get();

        assertEquals(response.getStatus(), 200);
        assertEquals(response.readEntity(Integer.class), expected);
    }
}</programlisting>
                    </para>
                </listitem>
            </itemizedlist>
        </para>

        <para>
            If you need more complex setup of your test you can achieve this by directly extending the &jersey.test.JerseyTestNg;
            class create setup/teardown methods suited to your needs and provide a strategy for storing and handling a test
            container / client instance (see <literal>JerseyTestNg.configureStrategy(TestNgStrategy)</literal> method).
        </para>
    </section>

    <section>
        <title>Advanced features</title>

        <section>
            <title>&lit.jersey.test.JerseyTest; Features</title>

            <para>&lit.jersey.test.JerseyTest; provide
                <link xlink:href="&jersey.javadoc.uri.prefix;/test/JerseyTest.html#enable(java.lang.String)">enable(...)</link>,
                <link xlink:href="&jersey.javadoc.uri.prefix;/test/JerseyTest.html#forceEnable(java.lang.String)">forceEnable(...)</link>
                and <link xlink:href="&jersey.javadoc.uri.prefix;/test/JerseyTest.html#disable(java.lang.String)">disable(...)</link>
                methods, that give you control over configuring values of the properties defined and described in the
                &lit.jersey.test.TestProperties; class. A typical code that overrides the default property values is listed
                below:

                <programlisting language="java" linenumbering="numbered">public class SimpleTest extends JerseyTest {
    // ...

    @Override
    protected Application configure() {
        enable(TestProperties.LOG_TRAFFIC);
        enable(TestProperties.DUMP_ENTITY);

        // ...

    }
}</programlisting>

                The code in the example above enables test traffic logging (inbound and outbound headers) as well as
                dumping the HTTP message entity as part of the traffic logging.
            </para>
        </section>


        <section>
            <title>External container</title>

            <para>
                Complicated test scenarios may require fully started containers with complex setup configuration, that is not
                easily doable with current Jersey container support. To address these use cases, Jersey Test Framework providers
                general fallback mechanism - an External Test Container Factory. Support of this external container "wrapper" is
                provided as the following module:

                <programlisting language="xml">&lt;dependency&gt;
    &lt;groupId&gt;org.glassfish.jersey.test-framework.providers&lt;/groupId&gt;
    &lt;artifactId&gt;jersey-test-framework-provider-external&lt;/artifactId&gt;
    &lt;version&gt;&version;&lt;/version&gt;
&lt;/dependency&gt;</programlisting>

                As indicated, the "container" exposed by this module is just a wrapper or stub, that redirects all request to
                a configured host and port. Writing tests for this container is similar to any other but you have to provide
                the information about host and port during the test execution:

                <screen>mvn test -Djersey.test.host=myhost.org -Djersey.config.test.container.port=8080</screen>
            </para>
        </section>

        <section>
            <title>Test Client configuration</title>

            <para>
                Tests might require some advanced client configuration. This is possible by overriding
                <link xlink:href="&jersey.javadoc.uri.prefix;/test/JerseyTest.html#configureClient(org.glassfish.jersey.client.ClientConfig)">configureClient(ClientConfig clientConfig)</link>
                method. Typical use case for this is registering more providers, such as &jaxrs.ext.MessageBodyReader;s or
                &jaxrs.ext.MessageBodyWriter;s, or enabling additional features.
            </para>
        </section>

        <section>
            <title>Accessing the logged test records programmatically</title>

            <para>
                Sometimes you might need to check a logged message as part of your test assertions. For this purpose Jersey Test
                Framework provides convenient access to the logged records via
                <link xlink:href="&jersey.javadoc.uri.prefix;/test/JerseyTest.html#getLastLoggedRecord()">JerseyTest#getLastLoggedRecord()</link>
                and
                <link xlink:href="&jersey.javadoc.uri.prefix;/test/JerseyTest.html#getLoggedRecords()">JerseyTest#getLoggedRecords()</link>
                methods. Note that this feature is not enabled by default, see
                <link xlink:href="&jersey.javadoc.uri.prefix;/test/TestProperties.html#RECORD_LOG_LEVEL">TestProperties#RECORD_LOG_LEVEL</link>
                for more information.
            </para>
        </section>
    </section>

    <section xml:id="parallel">
        <title>Parallel Testing with Jersey Test Framework</title>

        <para>
            For a purpose of running multiple test containers in parallel you need to set the
            <link xlink:href='&jersey.javadoc.uri.prefix;/test/TestProperties.CONTAINER_PORT.html'>TestProperties.CONTAINER_PORT</link>
            to <literal>0</literal> value. This will tell Jersey Test Framework (and the underlying test container) to use the
            first available port.
        </para>

        <para>
            You can set the value as a system property (via command line option) or directly in the test (to not affect ports of
            other tests):

            <programlisting language="java" linenumbering="numbered">@Override
    protected Application configure() {
        // Find first available port.
        forceSet(TestProperties.CONTAINER_PORT, "0");

        return new ResourceConfig(Resource.class);
    }</programlisting>
        </para>

        <para>
            The easiest way to setup your JUnit or TestNG tests to run in parallel is to configure Maven Surefire plugin. You can
            do this via configuration options <literal>parallel</literal> and <literal>threadCount</literal>, i.e.:

                            <programlisting language="xml">...
&lt;configuration&gt;
    &lt;parallel&gt;methods&lt;/parallel&gt;
    &lt;threadCount&gt;5&lt;/threadCount&gt;
    ...
&lt;/configuration&gt;
...</programlisting>

            For more information about this topic consult the following Maven Surefire articles:

            <itemizedlist>
                <listitem>
                    <para><link xlink:href="http://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html">Fork Options and Parallel Test Execution</link></para>
                </listitem>
                <listitem>
                    <para><link xlink:href="http://maven.apache.org/surefire/maven-surefire-plugin/examples/testng.html#Running_tests_in_parallel">Using TestNG - Running tests in parallel</link></para>
                </listitem>
                <listitem>
                    <para><link xlink:href="http://maven.apache.org/surefire/maven-surefire-plugin/examples/junit.html#Running_tests_in_parallel">Using JUnit - Running tests in parallel</link></para>
                </listitem>
            </itemizedlist>
        </para>
    </section>
</chapter>