| .\" |
| .\" This file and its contents are supplied under the terms of the |
| .\" Common Development and Distribution License ("CDDL"), version 1.0. |
| .\" You may only use this file in accordance with the terms of version |
| .\" 1.0 of the CDDL. |
| .\" |
| .\" A full copy of the text of the CDDL should have accompanied this |
| .\" source. A copy of the CDDL is also available via the Internet at |
| .\" http://www.illumos.org/license/CDDL. |
| .\" |
| .\" |
| .\" Copyright (c) 2012 by Delphix. All rights reserved. |
| .\" |
| .TH run 1 "23 Sep 2012" |
| .SH NAME |
| run \- find, execute, and log the results of tests |
| .SH SYNOPSIS |
| .LP |
| .nf |
| \fBrun\fR [\fB-dgq] [\fB-o\fR \fIoutputdir\fR] [\fB-pP\fR \fIscript\fR] [\fB-t\fR \fIseconds\fR] [\fB-uxX\fR \fIusername\fR] |
| \fIpathname\fR ... |
| .fi |
| |
| .LP |
| .nf |
| \fBrun\fR \fB-w\fR \fIrunfile\fR [\fB-gq\fR] [\fB-o\fR \fIoutputdir\fR] [\fB-pP\fR \fIscript\fR] [\fB-t\fR \fIseconds\fR] |
| [\fB-uxX\fR \fIusername\fR] \fIpathname\fR ... |
| .fi |
| |
| .LP |
| .nf |
| \fBrun\fR \fB-c\fR \fIrunfile\fR [\fB-dq\fR] |
| .fi |
| |
| .LP |
| .nf |
| \fBrun\fR [\fB-h\fR] |
| .fi |
| |
| .SH DESCRIPTION |
| .sp |
| .LP |
| The \fBrun\fR command has three basic modes of operation. With neither the |
| \fB-c\fR nor the \fB-w\fR option, \fBrun\fR processes the arguments provided on |
| the command line, adding them to the list for this run. If a specified |
| \fIpathname\fR is an executable file, it is added as a test. If a specified |
| \fIpathname\fR is a directory, the behavior depends upon the \fB-g\fR option. |
| If \fB-g\fR is specified, the directory is treated as a test group. See the |
| section on "Test Groups" below. Without the \fB-g\fR option, \fBrun\fR simply |
| descends into the directory looking for executable files. The tests are then |
| executed, and the results are logged. |
| |
| With the \fB-w\fR option, \fBrun\fR finds tests in the manner described above. |
| Rather than executing the tests and logging the results, the test configuration |
| is stored in a \fIrunfile\fR which can be used in future invocations, or edited |
| to modify which tests are executed and which options are applied. Options |
| included on the command line with \fB-w\fR become defaults in the |
| \fIrunfile\fR. |
| |
| With the \fB-c\fR option, \fBrun\fR parses a \fIrunfile\fR, which can specify a |
| series of tests and test groups to be executed. The tests are then executed, |
| and the results are logged. |
| .sp |
| .SS "Test Groups" |
| .sp |
| .LP |
| A test group is comprised of a set of executable files, all of which exist in |
| one directory. The options specified on the command line or in a \fIrunfile\fR |
| apply to individual tests in the group. The exception is options pertaining to |
| pre and post scripts, which act on all tests as a group. Rather than running |
| before and after each test, these scripts are run only once each at the start |
| and end of the test group. |
| .SS "Test Execution" |
| .sp |
| .LP |
| The specified tests run serially, and are typically assigned results according |
| to exit values. Tests that exit zero and non-zero are marked "PASS" and "FAIL" |
| respectively. When a pre script fails for a test group, only the post script is |
| executed, and the remaining tests are marked "SKIPPED." Any test that exceeds |
| its \fItimeout\fR is terminated, and marked "KILLED." |
| |
| By default, tests are executed with the credentials of the \fBrun\fR script. |
| Executing tests with other credentials is done via \fBsudo\fR(1m), which must |
| be configured to allow execution without prompting for a password. Environment |
| variables from the calling shell are available to individual tests. During test |
| execution, the working directory is changed to \fIoutputdir\fR. |
| .SS "Output Logging" |
| .sp |
| .LP |
| By default, \fBrun\fR will print one line on standard output at the conclusion |
| of each test indicating the test name, result and elapsed time. Additionally, |
| for each invocation of \fBrun\fR, a directory is created using the ISO 8601 |
| date format. Within this directory is a file named \fIlog\fR containing all the |
| test output with timestamps, and a directory for each test. Within the test |
| directories, there is one file each for standard output, standard error and |
| merged output. The default location for the \fIoutputdir\fR is |
| \fI/var/tmp/test_results\fR. |
| .SS "Runfiles" |
| .sp |
| .LP |
| The \fIrunfile\fR is an ini style configuration file that describes a test run. |
| The file has one section named "DEFAULT," which contains configuration option |
| names and their values in "name = value" format. The values in this section |
| apply to all the subsequent sections, unless they are also specified there, in |
| which case the default is overridden. The remaining section names are the |
| absolute pathnames of files and directories, describing tests and test groups |
| respectively. The legal option names are: |
| .sp |
| .ne 2 |
| .na |
| \fBoutputdir\fR = \fIpathname\fR |
| .ad |
| .sp .6 |
| .RS 4n |
| The name of the directory that holds test logs. |
| .RE |
| .sp |
| .ne 2 |
| .na |
| \fBpre\fR = \fIscript\fR |
| .ad |
| .sp .6 |
| .RS 4n |
| Run \fIscript\fR prior to the test or test group. |
| .RE |
| .sp |
| .ne 2 |
| .na |
| \fBpre_user\fR = \fIusername\fR |
| .ad |
| .sp .6 |
| .RS 4n |
| Execute the pre script as \fIusername\fR. |
| .RE |
| .sp |
| .ne 2 |
| .na |
| \fBpost\fR = \fIscript\fR |
| .ad |
| .sp .6 |
| .RS 4n |
| Run \fIscript\fR after the test or test group. |
| .RE |
| .sp |
| .ne 2 |
| .na |
| \fBpost_user\fR = \fIusername\fR |
| .ad |
| .sp .6 |
| .RS 4n |
| Execute the post script as \fIusername\fR. |
| .RE |
| .sp |
| .ne 2 |
| .na |
| \fBquiet\fR = [\fITrue\fR|\fIFalse\fR] |
| .ad |
| .sp .6 |
| .RS 4n |
| If set to True, only the results summary is printed to standard out. |
| .RE |
| .sp |
| .ne 2 |
| .na |
| \fBtests\fR = [\fI'filename'\fR [,...]] |
| .ad |
| .sp .6 |
| .RS 4n |
| Specify a list of \fIfilenames\fR for this test group. Only the basename of the |
| absolute path is required. This option is only valid for test groups, and each |
| \fIfilename\fR must be single quoted. |
| .RE |
| .sp |
| .ne 2 |
| .na |
| \fBtimeout\fR = \fIn\fR |
| .ad |
| .sp .6 |
| .RS 4n |
| A timeout value of \fIn\fR seconds. |
| .RE |
| .sp |
| .ne 2 |
| .na |
| \fBuser\fR = \fIusername\fR |
| .ad |
| .sp .6 |
| .RS 4n |
| Execute the test or test group as \fIusername\fR. |
| .RE |
| |
| .SH OPTIONS |
| .sp |
| .LP |
| The following options are available for the \fBrun\fR command. |
| .sp |
| .ne 2 |
| .na |
| \fB-c\fR \fIrunfile\fR |
| .ad |
| .RS 6n |
| Specify a \fIrunfile\fR to be consumed by the run command. |
| .RE |
| |
| .ne 2 |
| .na |
| \fB-d\fR |
| .ad |
| .RS 6n |
| Dry run mode. Execute no tests, but print a description of each test that would |
| have been run. |
| .RE |
| |
| .ne 2 |
| .na |
| \fB-g\fR |
| .ad |
| .RS 6n |
| Create test groups from any directories found while searching for tests. |
| .RE |
| |
| .ne 2 |
| .na |
| \fB-o\fR \fIoutputdir\fR |
| .ad |
| .RS 6n |
| Specify the directory in which to write test results. |
| .RE |
| |
| .ne 2 |
| .na |
| \fB-p\fR \fIscript\fR |
| .ad |
| .RS 6n |
| Run \fIscript\fR prior to any test or test group. |
| .RE |
| |
| .ne 2 |
| .na |
| \fB-P\fR \fIscript\fR |
| .ad |
| .RS 6n |
| Run \fIscript\fR after any test or test group. |
| .RE |
| |
| .ne 2 |
| .na |
| \fB-q\fR |
| .ad |
| .RS 6n |
| Print only the results summary to the standard output. |
| .RE |
| |
| .ne 2 |
| .na |
| \fB-t\fR \fIn\fR |
| .ad |
| .RS 6n |
| Specify a timeout value of \fIn\fR seconds per test. |
| .RE |
| |
| .ne 2 |
| .na |
| \fB-u\fR \fIusername\fR |
| .ad |
| .RS 6n |
| Execute tests or test groups as \fIusername\fR. |
| .RE |
| |
| .ne 2 |
| .na |
| \fB-w\fR \fIrunfile\fR |
| .ad |
| .RS 6n |
| Specify the name of the \fIrunfile\fR to create. |
| .RE |
| |
| .ne 2 |
| .na |
| \fB-x\fR \fIusername\fR |
| .ad |
| .RS 6n |
| Execute the pre script as \fIusername\fR. |
| .RE |
| |
| .ne 2 |
| .na |
| \fB-X\fR \fIusername\fR |
| .ad |
| .RS 6n |
| Execute the post script as \fIusername\fR. |
| .RE |
| |
| .SH EXAMPLES |
| .LP |
| \fBExample 1\fR Running ad-hoc tests. |
| .sp |
| .LP |
| This example demonstrates the simplest invocation of \fBrun\fR. |
| |
| .sp |
| .in +2 |
| .nf |
| % \fBrun my-tests\fR |
| Test: /home/jkennedy/my-tests/test-01 [00:02] [PASS] |
| Test: /home/jkennedy/my-tests/test-02 [00:04] [PASS] |
| Test: /home/jkennedy/my-tests/test-03 [00:01] [PASS] |
| |
| Results Summary |
| PASS 3 |
| |
| Running Time: 00:00:07 |
| Percent passed: 100.0% |
| Log directory: /var/tmp/test_results/20120923T180654 |
| .fi |
| .in -2 |
| |
| .LP |
| \fBExample 2\fR Creating a \fIrunfile\fR for future use. |
| .sp |
| .LP |
| This example demonstrates creating a \fIrunfile\fR with non default options. |
| |
| .sp |
| .in +2 |
| .nf |
| % \fBrun -p setup -x root -g -w new-tests.run new-tests\fR |
| % \fBcat new-tests.run\fR |
| [DEFAULT] |
| pre = setup |
| post_user = |
| quiet = False |
| user = |
| timeout = 60 |
| post = |
| pre_user = root |
| outputdir = /var/tmp/test_results |
| |
| [/home/jkennedy/new-tests] |
| tests = ['test-01', 'test-02', 'test-03'] |
| .fi |
| .in -2 |
| |
| .SH EXIT STATUS |
| .sp |
| .LP |
| The following exit values are returned: |
| .sp |
| .ne 2 |
| .na |
| \fB\fB0\fR\fR |
| .ad |
| .sp .6 |
| .RS 4n |
| Successful completion. |
| .RE |
| .sp |
| .ne 2 |
| .na |
| \fB\fB1\fR\fR |
| .ad |
| .sp .6 |
| .RS 4n |
| An error occurred. |
| .RE |
| |
| .SH SEE ALSO |
| .sp |
| .LP |
| \fBsudo\fR(1m) |