NAME

nihtest.conf — testing tool configuration file format

DESCRIPTION

nihtest(1) is a testing tool. It reads a configuration file to get its default settings. The default path for the file is nihtest.conf in the current directory; this can be overridden with nihtest(1)'s -C flag. This man page describes the format of this configuration file.

FILE FORMAT

The configuration file consists of multiple sections, which are started with [section-name]. The supported sections are settings, environment, and comparators.

Each section consists of variable assignments in the format name = value, one per line.

Multiple values for the same variable can be specified, one per line, where subsequent lines must be indented with spaces.

SETTINGS

In the settings section, the following variables are recognized:

default-program = program

Test program if no program directive is found in the test.

default-stderr-replace = pattern replacement ...

Each line consists of two values, pattern is a regular expression and replacement the corresponding replacement string. These are used for test cases without stderr-replace directives. See nihtest-case(5) for details on stderr-replace.

default-working-directory = directory

Use directory as the working directory if no working-directory directive is found in the test.

environment-clear = true

If environment-clear is set to true, only variables explicitly set or passed through are included in the environment passed to the tested program.

environment-passthrough = variable ...

The named environment variables are passed through from the environment nihtest(1) is run in to the environment passed to the tested program.

environment-unset = variable ...

The named environment variables are removed from the environment passed to the tested program.

features-files = file ...

Specifies the files to search for feature defines. This is used in the features directive in test cases.

keep-sandbox = when

Describe when to keep the sandbox (i.e., not delete it) after running the test. The following values are supported:

never: Never keep the sandbox.
failed: Keep the sandbox when the test failed.
always: Always keep the sandbox.

This can be overridden with the nihtest(1) flags --keep-broken -= when or --no-cleanup. The default is never.

print-results = when

Describe when to print the test results verbosely. The following values are supported:

never: Quiet mode.
failed: Print details when the test failed.
always: Verbose mode.

This can be overridden with the nihtest(1) flags --quiet or --verbose. The default is failed.

program-directories = directory ...

Specifies the directories programs are searched for. They are prepended to PATH.

sandbox-directories = directory ...

Create sandboxes in directory. By default, the sandboxes will be created in the current directory. A random directory of the pattern sandbox_* will be used.

test-input-directories = directory ...

nihtest(1) searches the current directory and directory for test cases, input and output files.

ENVIRONMENT

The environment section contains variable and values that will be added as environment variables when the test program is run.

COMPARATORS

The comparators section specifies programs to use to compare files of certain types. The variable names have the format got-extension. expected-extension, the value specifies the command line to use to compare these files. The command will be run with the two files as arguments, the file in the sandbox first and the expected last.

The command is expected to exit with code 0 if the two files are considered equal, any other exit code if not. The command output will be displayed if verbose test results are enabled (see print-results above).

For example, with comparators containing

zip.zip = zipcmp -v

and the test case containing

file got.zip
  expected.zip

the command

zipcmp -v got.zip
  expected.zip

will be run to compare the files.

COMPARATOR-PREPROCESSORS

The comparator-preprocessors section specifies programs to use to preprocess certain file types for comparison. The variable names have the format got-extension. expected-extension, the value specifies the command line to use to preprocess the file in the sandbox. The command will be run with the file in the sandbox first as argument. The command's output will be compared to the expected file.

If the command doesn't exit with code 0, the test will fail.

COPIERS

The copiers section specifies programs to use to copy certain file types into the sandbox. The variable names have the format sandbox-extension. source-extension, the value specifies the command line to use to copy the file into the sandbox. The command will be run with the two files as arguments, the source file first and the file in the sandbox last.

If the command doesn't exit with code 0, the test will fail.

SUITE

The optional suite can be used to define a suite of tests to be run with the --all command line option.

tests = tests: This specifies the tests to run. It can contain names or shell globs. .test will be appended if not present, and files will be searched for in test-input-directories.
expected-failing-tests = tests: This specifies the tests that are expected to fail. Tests are searched in the same way as for tests.

EXAMPLES

[settings]
default-program = program_to_test
program-directories = bin
    tests
[environment]
LC_ALL=en_US.UTF-8
[comparators]
zip.zip = zipcmp -v
[suite]
tests = *.test
expected_failing_tests = *-broken.test
    completely_broken_test.test