From 11dc164ac731e630ba6cd54ac4136cb662387c1b Mon Sep 17 00:00:00 2001
From: Lars Wirzenius <liw@liw.fi>
Date: Mon, 15 Aug 2011 19:37:44 +0100
Subject: Add text specifying how the next version of cmdtest should work.

---
 README | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 62 insertions(+)

(limited to 'README')

diff --git a/README b/README
index 982791e..e206d21 100644
--- a/README
+++ b/README
@@ -75,3 +75,65 @@ and actual outputs.
     sorts-two-unsorted-lines.stdin # "def\nabc\n"
     sorts-two-unsorted-lines.stdout # "abc\ndef\n"
 
+---
+
+`cmdtest` black box tests Unix command line tools.
+Roughly, it is given a command line and input files, and the expected output,
+and it verifies that the command line produces the expected output.
+If not, it reports a problem, and shows the differences.
+
+The command line arguments, inputs, and outputs are given in files
+that follow a naming convention. The test report is written to the
+standard output.
+
+Each test case consists of:
+
+* a set of command line arguments, not including the command name (`foo.args`)
+* the file fed to standard input (`foo.stdin`)
+* the expected output to the standard output (`foo.stdout`)
+* the expected output to the standard error (`foo.stderr`)
+* the expected exit code (`foo.exit`)
+* a shell script to run before the test (`foo.setup`)
+* a shell script to run after the test (`foo.teardown`)
+
+Usually, a single test is not enough. All tests can be put into the
+same directory, and they may share some setup and teardown code:
+
+* a shell script to run once, before all tests (`setup-once`)
+* a shell script to run before each test (`setup`)
+* a shell script to run after each test (`teardown`)
+* a shell script to run after all tests (`teardown-once`)
+
+`cmdtest` is given the name of the directory with all the tests,
+or several such directories, and it does the following:
+
+* execute `setup-once`
+* for each test case (unique prefix `foo`):
+  - execute `setup`
+  - execute `foo.setup`
+  - execute the command, giving it command line arguments from
+    `foo.args`, and redirecting standard input to come from `foo.stdin`
+  - capture standard output and error and exit codes
+  - execute `foo.teardown`
+  - execute `teardown`
+  - report result of test: does exit code match `foo.exit`, standard
+    output match `foo.stdout`, and standard error match `foo.stderr`?
+* execute `teardown-once`
+
+All of these files are optional. If a setup/teardown script is missing,
+it is simply not executed. If one of the standard input, output, or
+error files is missing, it is treated as if it were empty. If the
+exit code file is missing, it is treated as if it specified an exit
+code of zero.
+
+The actual command is given to `cmdtest` separately. This allows
+testing various implementations of the same command, for example
+various checksum utilities, or different versions of the same command.
+If `foo.args` would hardcode the command name, this would be hard
+to achieve.
+
+The shell scripts may use the following environment variables:
+
+* `DATADIR`: a temporary directory where files may be created by the test
+
+
-- 
cgit v1.2.1