testing tool

nihtest [-hqVv] [-C config] [--keep-broken] [--no-cleanup] [--setup-only] testcase

nihtest is a testing tool for command line utilities. It uses .test files as input. The format of these files is described in nihtest-case(5).

nihtest runs the test case testcase expected in the test case directory (see nihtest.conf(5)) in the file testcase or testcase.test. nihtest searches the current directory and the source-directory specified in nihtest.conf(5) for test cases, input and output files.

nihtest supports the following options

config, --config-file config
Use config as configuration file instead of ./nihtest.conf.
, --help
Display a short help message and exit.
Do not delete the sandbox if the test fails.
Do not delete the sandbox after the test finishes (successfully or not).
, --quiet
Do not print test results.
Only populate the sandbox, but do not run the actual test.
, --verbose
Print detailed test results.
, --version
Print nihtest version number and exit.

A test run consists of the following steps:

  • check if test should be skipped
  • create sandbox directory
  • populate sandbox directory with input files
  • run program
  • compare program output, exit code, and output files
  • remove sandbox

nihtest configuration file, see nihtest.conf(5) for details

nihtest uses the following exit codes:
Test passed
Test failed
Other error
Test was skipped.

In general, differences to the test case expectation are marked up with “-” if something expected did not happen and “+” if something unexpected happened.

In case the return value does not match, nihtest will report “Exit code not as expected”. If the standard output or standard error output are not correct, nihtest will report “Output not as expected” or “Error output not as expected” respectively. And finally, if the file contents of the sandbox are not correct, nihtest will report “Files in sandbox not as expected”.

nihtest-case(5), nihtest.conf(5)