From c4243235827bfd3e60c0ece54760f33f5feadf43 Mon Sep 17 00:00:00 2001 From: Lars Wirzenius Date: Sun, 9 Jun 2013 11:10:12 +0100 Subject: Add Daniel's README for yarn --- README.yarn | 48 ++++++++++++++++++++++++++++++++++++------------ 1 file changed, 36 insertions(+), 12 deletions(-) (limited to 'README.yarn') diff --git a/README.yarn b/README.yarn index 4d59ec0..465711d 100644 --- a/README.yarn +++ b/README.yarn @@ -1,6 +1,12 @@ README for story testing ======================== +Document History +---------------- +**TODO: Remove this when finalised** +v1. liw wrote it +v2. kinnison updated based on conversation with liw + Introduction ------------ @@ -85,8 +91,10 @@ The following keywords are defined. the story. The name is used for documentation and reporting purposes only and has no semantic meaning. STORY MUST be the first keyword in a story, with the exception of IMPLEMENTING. - A test document any number of stories, but must contain at least - one. The IMPLEMENTING sections are shared between them. + The set of documents passed in a test run may define any number of + stories between them, but there must be at least one or it is a + test failure. The IMPLEMENTING sections are shared between the + documents and stories. * **ASSUMING** defines a condition for the story. The rest of the line is "matched text", which gets implemented by an @@ -103,8 +111,9 @@ The following keywords are defined. did the right thing. If the code fails, the story fails. * **FINALLY** specifies how to clean up after a story. If the - code fails, the story fails. The code gets run at the end of - the story, regardless of whether the story is failing or not. + code fails, the story fails. All finally blocks get run either when + encountered in the story flow, or at the end of the story, regardless + of whether the story is failing or not. * **AND** acts as ASSUMING, GIVEN, WHEN, THEN, or FINALLY: whichever was used last. It must not be used unless the previous step was @@ -115,8 +124,9 @@ The following keywords are defined. the end of the block quoted code block. Markdown is unclear whether an empty line (no characters, not even whitespace) between two block quoted code blocks starts a new one or not, so we resolve the - ambiguity by specifiying that the next code block is a continuation - unless it starts with one of the story testing keywords. + ambiguity by specifiying that a code block directly following a code + block is a continuation unless it starts with one of the story + testing keywords. The shell commands get parenthesised parts of the match of the regular expression as positional arguments (`$1` etc). For @@ -128,7 +138,10 @@ The following keywords are defined. The shell commands get invoked with `/bin/sh -eu`, and need to be written accordingly. Be careful about commands that return - a non-zero exit code. + a non-zero exit code. There will be a library of shell functions + supplied which allow handling the testing of non-zero exit codes + cleanly. In addition functions for handling stdout and stderr will + be provided. The code block of an IMPLEMENTING block fails if the shell invocation exits with a non-zero exit code. Output to stderr is @@ -138,12 +151,13 @@ The following keywords are defined. Semantics: * The name of each story (given with STORY) must be unique. +* All names will be normalised before use (whitespace collapse, leading + and trailing whitespace * Every ASSUMING, GIVEN, WHEN, THEN, FINALLY must be matched by exactly one IMPLEMENTING. The test runner checks this before running any code. -* Every IMPLEMENTING must match at least one ASSUMING, GIVNE, WHEN, - THEN, or FINALLY. The test runner checks this before running - any code. +* Every IMPLEMENTING may match any number of ASSUMING, GIVEN, WHEN, + THEN, or FINALLY. The test runner can warn if an IMPLEMENTING is unused. * If ASSUMING fails, that story is skipped, and any FINALLY steps are not run. @@ -160,9 +174,19 @@ tools tries to do. TODO ---- -* Add FINALLY to clean up at the end, regardless of test failures. * Add DEFINING, PRODUCING, if they turn out to be useful. * Need something like ASSUMING, except fail the story if the pre-condition is not true. Useful for testing that you can ssh to localhost when flinging, for example. - + **DJAS**: We think this might be 'REQUIRING' and it still does + not run the FINALLY group. +* Consider the ordering some more. IMPLEMENTING can come anywhere + but otherwise stories are defined as: + * STORY + * 0+: ASSUMING + * 1+: + * 1+: GIVEN + * 1+: + * 1+: WHEN + * 1+: THEN + * 0+: FINALLY -- cgit v1.2.1