summaryrefslogtreecommitdiff
path: root/README.yarn
diff options
context:
space:
mode:
authorLars Wirzenius <liw@liw.fi>2013-06-09 11:10:12 +0100
committerLars Wirzenius <liw@liw.fi>2013-06-09 11:10:12 +0100
commitc4243235827bfd3e60c0ece54760f33f5feadf43 (patch)
tree845a761b0a04fd536c677e5797510ba9f523fd13 /README.yarn
parent164286c053967341736146abd69d54cf9465ccdb (diff)
downloadcmdtest-c4243235827bfd3e60c0ece54760f33f5feadf43.tar.gz
Add Daniel's README for yarn
Diffstat (limited to 'README.yarn')
-rw-r--r--README.yarn48
1 files changed, 36 insertions, 12 deletions
diff --git a/README.yarn b/README.yarn
index 4d59ec0..465711d 100644
--- a/README.yarn
+++ b/README.yarn
@@ -1,6 +1,12 @@
README for story testing
========================
+Document History
+----------------
+**TODO: Remove this when finalised**
+v1. liw wrote it
+v2. kinnison updated based on conversation with liw
+
Introduction
------------
@@ -85,8 +91,10 @@ The following keywords are defined.
the story. The name is used for documentation and reporting
purposes only and has no semantic meaning. STORY MUST be the
first keyword in a story, with the exception of IMPLEMENTING.
- A test document any number of stories, but must contain at least
- one. The IMPLEMENTING sections are shared between them.
+ The set of documents passed in a test run may define any number of
+ stories between them, but there must be at least one or it is a
+ test failure. The IMPLEMENTING sections are shared between the
+ documents and stories.
* **ASSUMING** defines a condition for the story. The rest of the
line is "matched text", which gets implemented by an
@@ -103,8 +111,9 @@ The following keywords are defined.
did the right thing. If the code fails, the story fails.
* **FINALLY** specifies how to clean up after a story. If the
- code fails, the story fails. The code gets run at the end of
- the story, regardless of whether the story is failing or not.
+ code fails, the story fails. All finally blocks get run either when
+ encountered in the story flow, or at the end of the story, regardless
+ of whether the story is failing or not.
* **AND** acts as ASSUMING, GIVEN, WHEN, THEN, or FINALLY: whichever
was used last. It must not be used unless the previous step was
@@ -115,8 +124,9 @@ The following keywords are defined.
the end of the block quoted code block. Markdown is unclear whether
an empty line (no characters, not even whitespace) between two
block quoted code blocks starts a new one or not, so we resolve the
- ambiguity by specifiying that the next code block is a continuation
- unless it starts with one of the story testing keywords.
+ ambiguity by specifiying that a code block directly following a code
+ block is a continuation unless it starts with one of the story
+ testing keywords.
The shell commands get parenthesised parts of the match of the
regular expression as positional arguments (`$1` etc). For
@@ -128,7 +138,10 @@ The following keywords are defined.
The shell commands get invoked with `/bin/sh -eu`, and need to
be written accordingly. Be careful about commands that return
- a non-zero exit code.
+ a non-zero exit code. There will be a library of shell functions
+ supplied which allow handling the testing of non-zero exit codes
+ cleanly. In addition functions for handling stdout and stderr will
+ be provided.
The code block of an IMPLEMENTING block fails if the shell
invocation exits with a non-zero exit code. Output to stderr is
@@ -138,12 +151,13 @@ The following keywords are defined.
Semantics:
* The name of each story (given with STORY) must be unique.
+* All names will be normalised before use (whitespace collapse, leading
+ and trailing whitespace
* Every ASSUMING, GIVEN, WHEN, THEN, FINALLY must be matched by
exactly one IMPLEMENTING. The test runner checks this before running
any code.
-* Every IMPLEMENTING must match at least one ASSUMING, GIVNE, WHEN,
- THEN, or FINALLY. The test runner checks this before running
- any code.
+* Every IMPLEMENTING may match any number of ASSUMING, GIVEN, WHEN,
+ THEN, or FINALLY. The test runner can warn if an IMPLEMENTING is unused.
* If ASSUMING fails, that story is skipped, and any FINALLY steps
are not run.
@@ -160,9 +174,19 @@ tools tries to do.
TODO
----
-* Add FINALLY to clean up at the end, regardless of test failures.
* Add DEFINING, PRODUCING, if they turn out to be useful.
* Need something like ASSUMING, except fail the story if the
pre-condition is not true. Useful for testing that you can ssh
to localhost when flinging, for example.
-
+ **DJAS**: We think this might be 'REQUIRING' and it still does
+ not run the FINALLY group.
+* Consider the ordering some more. IMPLEMENTING can come anywhere
+ but otherwise stories are defined as:
+ * STORY
+ * 0+: ASSUMING
+ * 1+:
+ * 1+: GIVEN
+ * 1+:
+ * 1+: WHEN
+ * 1+: THEN
+ * 0+: FINALLY