1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
|
README for cmdtest
==================
`cmdtest` black box tests Unix command line tools.
Roughly, it is given a command line and input files, and the expected output,
and it verifies that the command line produces the expected output.
If not, it reports a problem, and shows the differences.
The command line arguments, inputs, and outputs are given in files
that follow a naming convention. The test report is written to the
standard output.
Each test case consists of:
* a script to run the test (`foo.script`)
* the file fed to standard input (`foo.stdin`)
* the expected output to the standard output (`foo.stdout`)
* the expected output to the standard error (`foo.stderr`)
* the expected exit code (`foo.exit`)
* a shell script to run before the test (`foo.setup`)
* a shell script to run after the test (`foo.teardown`)
Usually, a single test is not enough. All tests can be put into the
same directory, and they may share some setup and teardown code:
* a shell script to run once, before all tests (`setup-once`)
* a shell script to run before each test (`setup`)
* a shell script to run after each test (`teardown`)
* a shell script to run after all tests (`teardown-once`)
`cmdtest` is given the name of the directory with all the tests,
or several such directories, and it does the following:
* execute `setup-once`
* for each test case (unique prefix `foo`):
- execute `setup`
- execute `foo.setup`
- execute the command, by running `foo.script`,
and redirecting standard input to come from `foo.stdin`
- capture standard output and error and exit codes
- execute `foo.teardown`
- execute `teardown`
- report result of test: does exit code match `foo.exit`, standard
output match `foo.stdout`, and standard error match `foo.stderr`?
* execute `teardown-once`
All of these files are optional. If a setup/teardown script is missing,
it is simply not executed. If one of the standard input, output, or
error files is missing, it is treated as if it were empty. If the
exit code file is missing, it is treated as if it specified an exit
code of zero.
The shell scripts may use the following environment variables:
* `DATADIR`: a temporary directory where files may be created by the test
|
