NAME

nihtest — testing tool

SYNOPSIS

nihtest [-hqVv] [--all] [-C config] [--debug] [--keep-broken] [--no-cleanup] [--setup-only] [testcase ...]

DESCRIPTION

nihtest is a testing tool for command line utilities. It uses .test files as input. The format of these files is described in nihtest-case(5).

nihtest runs the test cases specified on the command line or, if the --all option is given, in the suite section of the config file. Shell globs can be used on the command line, test cases are expected in the test case directory.

nihtest searches the current directory and the source-directory specified in nihtest.conf(5) for test cases, input and output files.

nihtest supports the following options

--all: Run all tests specified in the suite section of the config file.
-C config, --config-file config: Use config as configuration file instead of ./nihtest.conf.
--debug: Set up the sandbox, but run the program inside a debugger. Results are not checked.
-h, --help: Display a short help message and exit.
--keep-broken: Do not delete the sandbox if the test fails.
--no-cleanup: Do not delete the sandbox after the test finishes (successfully or not).
-q, --quiet: Do not print test results.
--setup-only: Only populate the sandbox, but do not run the actual test.
-v, --verbose: Print detailed test results.
-V, --version: Print nihtest version number and exit.

A test run consists of the following steps:

check if test should be skipped
create sandbox directory
populate sandbox directory with input files
run program
compare program output, exit code, and output files
remove sandbox

FILES

nihtest.conf: nihtest configuration file, see nihtest.conf(5) for details

EXIT STATUS

nihtest uses the following exit codes:

0: Test passed
1: Test failed
2: Other error
77: Test was skipped.

DIAGNOSTICS

In general, differences to the test case expectation are marked up with “-” if something expected did not happen and “+” if something unexpected happened.

In case the return value does not match, nihtest will report “Exit code not as expected”. If the standard output or standard error output are not correct, nihtest will report “Output not as expected” or “Error output not as expected” respectively. And finally, if the file contents of the sandbox are not correct, nihtest will report “Files in sandbox not as expected”.