| This document should be used as a guideline when creating new |
| development unit tests based on TestNG. |
| |
| The directory structure overall looks like this: |
| |
| appserv-tests |
| | |
| ---- config.properties |
| | |
| ---- common/ |
| | | |
| | ---- common.xml |
| | | |
| | ---- properties.xml |
| | | |
| | ...(others like derby.properties etc.) |
| | |
| ---- devtestsNG/ |
| | |
| ---- build.xml |
| | |
| ---- common |
| | | |
| | ---- util/java/src/.... |
| | | |
| | ---- build.xml |
| | |
| ---- mytest-module-1 |
| | | |
| | ---- build.xml |
| | | |
| | ---- build.properties |
| | | |
| | ---- testng.xml |
| | | |
| | ---- tests |
| | | |
| | ---- com/sun/mytests/Test1.java |
| | ... |
| | |
| |---- mytest-module-2 |
| | | |
| | ---- build.xml |
| | | |
| | ---- build.properties |
| | | |
| | ---- my-submodule-2-1 |
| | | | |
| | | ---- (replicate dir structure of module-1 or 2) |
| | | |
| | ---- my-submodule-2-2 |
| | | |
| | ---- (replicate dir structure of module-1 or 2) |
| | |
| |----- ... other modules |
| |
| |
| Features: |
| |
| - Ability to run test across all modules based on group membership. |
| - Ability to only run failed tests from previous test runs |
| - Ability to generate a drill-down report of the test output |
| - Provide an Java API and implementation to programmatically |
| perform certain services like check if the server is |
| running or if a certain app is deployed successfully etc. |
| Essentially these features will provide the ability to check |
| whether certain pre-conditions that are needed for the tests |
| to be performed are satisfied are not. |
| The actual tests themselves can be configured to run only if |
| the preconditions are satisfied. This feature is work in progress. |
| |
| |
| Instructions: |
| |
| - Env variable like APS_HOME, S1AS_HOME need to be set |
| See https://glassfish.dev.java.net/public/GuidelinesandConventions.html |
| - The common framework facilities is extended by adding |
| common target in common-build.xml |
| - Any module/submodule build targets linked with any of the |
| targets at the top-level(appserv-tests/devtests*/) should |
| have their configuration defined in the top-level |
| configuration files(common/config.properties) |
| - Please see the README in the template directory for writing tests |
| for a new module |
| |
| The following directories are use by TestNG and the devtests framework |
| and should not used by the devtest authors directly |
| - test-output |
| - test-report |
| - test-tmp |
| |
| New (a bunch of common targets already exist - see appserv-tests/config/common.xml) |
| top-level targets defined in common-build.xml |
| common-compile-testng-tests |
| common-run |
| common-run-groups |
| common-run-failed |
| common-report |
| |
| TODO/Suggestions (in no particular order): |
| |
| - Provide a list model for running the tests. The list model, as opposed |
| to the current hierarchichal tree model, has a list of all the test |
| directory in a single file. The advantage of using list model is that |
| each of the test directories can be executed in a separate ant process |
| (via a shell/batch script) and the failure of one does affect |
| the execution of others - TBD |
| - Reduce number of environmental variables needed to get the tests going |
| - Configuration parameter - documentation/clarification - so that the user can |
| understand the meaning of the configuration parameters and set them correctly |
| - Ownership issues - designate owners for the tests for maintenance and quick resolution |
| - Provide default values commonly used property names |
| like web.xml, sun-web.xml |
| These would say default to ${basedir}/descriptors/web.xml |
| and ${basedir}/descriptors/sun-web.xml respectively |
| - Clean up properties.xml/common.xml - remove unused targets, standardize |
| target/property names |
| - Come up with a standardized set of group names for TestNG |
| group membership which will enable us to run tests tagged |
| to specific groups across modules. |
| - Support running devtests running on a different machine. |
| (I presume this will still require an installation of glassfish |
| on the local machine since a lot of the current tests |
| add ${env.S1AS_HOME}/lib/*.jar to the execution classpath) |
| - maven'ize the top-level devtest targets so that it |
| blends in seamlessly with the rest of the glassfish |
| - IDE support |
| - Able to run a single test |
| - Logs |
| Test Log <=> Server Log |
| Both logs captured, and marked for each test |
| - Remote Instances |
| PE-EE flexibility |
| - Persistence EE and Java SE containers |
| - Roadmap should include Tools, i.e. NetBeans integration |
| - Deployment, jars support |
| Java EE technology and Non-Java EE Extensions (i.e. MBean, etc) |
| - Able to run old framework |
| |
| For questions/comments email: quality@glassfish.dev.java.net |
| |