From 27c6d616138df629c424c79202967e42b59ef3d8 Mon Sep 17 00:00:00 2001
From: Lars Wirzenius <liw@liw.fi>
Date: Sun, 25 Dec 2011 18:32:13 +0000
Subject: make manpage muchly more detailed

---
 cmdtest.1.in | 132 ++++++++++++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 118 insertions(+), 14 deletions(-)

(limited to 'cmdtest.1.in')

diff --git a/cmdtest.1.in b/cmdtest.1.in
index 8e392bc..4791bb1 100644
--- a/cmdtest.1.in
+++ b/cmdtest.1.in
@@ -18,18 +18,109 @@
 cmdtest \- blackbox testing of Unix command line tools
 .SH SYNOPSIS
 .SH DESCRIPTION
+.B cmdtest 
+black box tests Unix command line tools.
+Given some test scripts, their inputs, and expected outputs,
+it verifies that the command line produces the expected output.
+If not, it reports problems, and shows the differences.
+.PP
+Each test case
+.I foo
+consists of the following files:
+.TP
+.I foo.script
+a script to run the test (this is required)
+.TP
+.I foo.stdin
+the file fed to standard input
+.TP
+.I foo.stdout
+the expected output to the standard output
+.TP
+.I foo.stderr
+the expected output to the standard error
+.TP
+.I foo.exit
+the expected exit code
+.TP
+.I foo.setup
+a shell script to run before the test
+.TP
+.I foo.teardown
+a shell script to run after test
+.PP
+Usually, a single test is not enough. All tests are put into the
+same directory, and they may share some setup and teardown code:
+.TP
+.I setup-once
+a shell script to run once, before any tests
+.TP
+.I setup
+a shell script to run before each test
+.TP
+.I teardown
+a shell script to run after each test
+.TP
+.I teardown-once
+a shell script to run once, after all tests
+.PP
 .B cmdtest
-tests Unix command line tools,
-preferably non-interactive ones.
-The user writes some tests,
-consisting of the command to run, the input files given to it,
-and the expected output.
-.B cmdtest
-runs the command,
-compares the actual output to the expected output,
-and reports any differences.
-If everything matches,
-then tests pass.
+is given the name of the directory with all the tests,
+or several such directories, and it does the following:
+.PP
+.na
+.nh
+\(bu execute
+.I setup-once
+.PP
+\(bu for each test case (unique prefix 
+.IR foo ):
+.RS
+.HP 2
+\(em execute
+.I setup
+.HP 2
+\(em execute 
+.I foo.setup
+.HP 2
+\(em execute the command, by running 
+.IR foo.script ,
+and redirecting standard input to come from 
+.IR foo.stdin ,
+and capturing standard output and error and exit codes
+.HP 2
+\(em execute
+.I foo.teardown
+.HP 2
+\(em execute 
+.I teardown
+.HP 2
+\(em report result of test: does exit code match
+.IR foo.exit , 
+standard output match 
+.IR foo.stdout , 
+and standard error match 
+.IR foo.stderr ?
+.RE
+.fi
+.hy
+.PP
+\(bu execute 
+.I teardown-once
+.PP
+Except for
+.IR foo.script , 
+all of these files are optional. 
+If a setup or teardown script is missing, it is simply not executed. 
+If one of the standard input, output, or error files is missing, 
+it is treated as if it were empty. 
+If the exit code file is missing, it is treated as if it specified an exit
+code of zero.
+.PP
+The shell scripts may use the following environment variable:
+.TP
+.B DATADIR
+a temporary directory where files may be created by the test
 .SH OPTIONS
 .SH EXAMPLE
 To test that the
@@ -41,7 +132,7 @@ containing the following content:
 .IP
 .nf
 #!/bin/sh
-$COMMAND hello, world
+echo hello, world
 .fi
 .PP
 Also create the file
@@ -53,7 +144,7 @@ hello, world
 Then you can run the tests:
 .IP
 .nf
-$ cmdtest echo-tests -c /bin/echo
+$ cmdtest echo-tests
 test 1/1
 1/1 tests OK, 0 failures
 .fi
@@ -63,7 +154,7 @@ If you change the stdout file to be something else,
 will report the differences:
 .IP
 .nf
-$ cmdtest echo-tests -c /bin/echo
+$ cmdtest echo-tests
 FAIL: hello: stdout diff:
 --- echo-tests/hello.stdout	2011-09-11 19:14:47 +0100
 +++ echo-tests/hello.stdout-actual	2011-09-11 19:14:49 +0100
@@ -74,5 +165,18 @@ FAIL: hello: stdout diff:
 test 1/1
 0/1 tests OK, 1 failures
 .fi
+.PP
+Furthermore, the 
+.I echo-tests
+directory will contain the actual output files,
+and diffs from the expected files.
+If one of the actual output files is actually correct,
+you can actualy rename it to be the expected file.
+Actually,
+that's a very convenient way of creating the expected output files:
+you run the test,
+fixing things,
+until you've manually checked the actual output is correct,
+then you rename the file.
 .SH "SEE ALSO"
 .BR cliapp (5).
-- 
cgit v1.2.1