Merge branch 'yarn'

17 files changed, 1060 insertions, 6 deletions

diff --git a/NEWS b/NEWS
index 2caeb36..f22a1a3 100644
--- a/NEWS
+++ b/NEWS
@@ -3,6 +3,11 @@ NEWS for cmdtest
 
 This file summarizes changes between releases of cmdtest.
 
+Version 0.7, released UNRELEASED
+--------------------------------
+
+* Added the new tool `yarn`, for doing story testing.
+
 Version 0.6, released 2013-03-14
 --------------------------------
 
diff --git a/README b/README
index 300770b..8c1a39c 100644
--- a/README
+++ b/README
@@ -1,6 +1,13 @@
 README for cmdtest
 ==================
 
+This project consists of two programs: the original `cmdtest`,
+and the newer `yarn`. Both are black box testing tools for Unix
+command line tools.
+
+cmdtest
+-------
+
 `cmdtest` black box tests Unix command line tools.
 Given some test scripts, their inputs, and expected outputs,
 it verifies that the command line produces the expected output.
@@ -9,10 +16,27 @@ If not, it reports problems, and shows the differences.
 See the manual page for details on how to use the program.
 
 
+yarn
+----
+
+`yarn` also black box tests Unix command line tools, but takes
+a different approach, where the emphasis is on verifying that the
+tools works correctly in a scenario, or sequence of operations, or
+what we call a "test story". `yarn` is inspired [BDD][BDD],
+behavior-driven development, and some of the implementations made
+by the Ruby community.
+
+See README.yarn for more details.
+
+`yarn` has been designed with Daniel Silverstone.
+
+[BDD]: https://en.wikipedia.org/wiki/Behavior-driven_development
+
+
 Legalese
 --------
 
-Copyright 2011  Lars Wirzenius
+Copyright 2011-2013 Lars Wirzenius
 
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
diff --git a/README.yarn b/README.yarn
new file mode 100644
index 0000000..a117866
--- /dev/null
+++ b/README.yarn
@@ -0,0 +1,186 @@
+README for story testing
+========================
+
+Introduction
+------------
+
+`yarn` is a story testing tool: you write a story describing how a
+user uses your software and what should happen, and express, using
+very lightweight syntax, the story in such a way that it can be tested
+automatically. The story has a simple, but strict structure:
+
+    GIVEN some setup for the test
+    WHEN thing that is to be tested happens
+    THEN the post-conditions must be true
+
+As an example, consider a very short test story for verifying that
+a backup program works, at least for one simple case.
+
+    GIVEN some live data in a directory
+    AND an empty backup repository
+    WHEN a backup is made
+    THEN the data case be restored
+
+(Note the addition of AND: you can have multiple GIVEN, WHEN, and
+THEN statements. The AND keyword makes the text be more readable.)
+
+Stories are meant to be written in somewhat human readable language.
+However, they are not free form text. In addition to the GIVEN/WHEN/THEN
+structure, the text for each of the steps needs a computer-executable
+implementation. This is done by using IMPLEMENTS. The backup story
+from above might be implemented as follows:
+
+    IMPLEMENTS GIVEN some live data in a directory
+    rm -rf "$TESTDIR/data"
+    mkdir "$TESTDIR/data"
+    echo foo > "$TESTDIR/data/foo"
+
+    IMPLEMENTS GIVEN an empty backup repository
+    rm -rf "$TESTDIR/repo"
+    mkdir "$TESTDIR/repo"
+
+    IMPLEMENTS WHEN a backup is made
+    backup-program -r "$TESTDIR/repo" "$TESTDIR/data"
+
+    IMPLEMENTS THEN the data can be restored
+    mkdir "$TESTDIR/restored"
+    restore-program -r "$TESTDIR/repo" "$TESTDIR/restored"
+    diff -rq "$TESTDIR/data" "$TESTDIR/restored"
+
+Each "IMPLEMENT GIVEN" (or WHEN, THEN) is followed by a regular
+expression on the same line, and then a shell script that gets executed
+to implement any step that matches the regular expression.  The
+implementation can extract data from the match as well: for example,
+the regular expression might allow a file size to be specified.
+
+The above example is a bit silly, of course: why go to the effort
+to obfuscate the various steps? The answer is that the various
+steps, implemented using IMPLEMENTS, can be combined in many
+ways, to test different aspects of the program being tested.
+
+Moreover, by making the step descriptions be human language
+text, matched by regular expressions, most of the test can
+hopefully be written, and understood, by non-programmers. Someone
+who understands what a program should do, could write tests
+to verify its behaviour. The implementations of the various
+steps need to be implemented by a programmer, but given a
+well-designed set of steps, with enough flexibility in their
+implementation, that quite a good test suite can be written.
+
+Test language specification
+---------------------------
+
+A test document is written in [Markdown][markdown], with block
+quoted code blocks being interpreted specially. Each block
+must follow the syntax defined here.
+
+* Every step in a story is one line, and starts with a keyword.
+* Each implementation (IMPLEMENTS) starts as a new block, and
+  continues until there is a block that starts with another
+  keyword.
+
+The following keywords are defined.
+
+* **STORY** starts a new story. The rest of the line is the name of
+  the story. The name is used for documentation and reporting
+  purposes only and has no semantic meaning. STORY MUST be the
+  first keyword in a story, with the exception of IMPLEMENTS.
+  The set of documents passed in a test run may define any number of
+  stories between them, but there must be at least one or it is a
+  test failure. The IMPLEMENTS sections are shared between the
+  documents and stories.
+
+* **ASSUMING** defines a condition for the story. The rest of the
+  line is "matched text", which gets implemented by an
+  IMPLEMENTS section. If the code executed by the implementation
+  fails, the story is skipped.
+
+* **GIVEN** prepares the world for the test to run. If
+  the implementation fails, the story fails.
+
+* **WHEN** makes the change to the world that is to be tested.
+  If the code fails, the story fails.
+
+* **THEN** verifies that the changes made by the GIVEN steps
+  did the right thing. If the code fails, the story fails.
+
+* **FINALLY** specifies how to clean up after a story. If the
+  code fails, the story fails. All FINALLY blocks get run either when
+  encountered in the story flow, or at the end of the story, regardless
+  of whether the story is failing or not.
+
+* **AND** acts as ASSUMING, GIVEN, WHEN, THEN, or FINALLY: whichever
+  was used last. It must not be used unless the previous step was
+  one of those, or another AND.
+
+* **IMPLEMENTS** is followed by one of ASSUMING, GIVEN, WHEN, or
+  THEN, and a PCRE regular expression, and then shell commands until
+  the end of the block quoted code block. Markdown is unclear whether
+  an empty line (no characters, not even whitespace) between two
+  block quoted code blocks starts a new one or not, so we resolve the
+  ambiguity by specifiying that a code block directly following a code
+  block is a continuation unless it starts with one of the story
+  testing keywords.
+
+  The shell commands get parenthesised parts of the match of the
+  regular expression as environment variables (`$MATCH_1` etc). For
+  example, if the regexp is "a (\d+) byte file", then `$MATCH_1` gets
+  set to the number matched by `\d+`.
+
+  The test runner creates a temporary directory, whose name is
+  given to the shell code in the `TESTDIR` environment variable.
+
+  The shell commands get invoked with `/bin/sh -eu`, and need to
+  be written accordingly. Be careful about commands that return
+  a non-zero exit code. There will be a library of shell functions
+  supplied which allow handling the testing of non-zero exit codes
+  cleanly. In addition functions for handling stdout and stderr will
+  be provided.
+
+  The code block of an IMPLEMENTS block fails if the shell
+  invocation exits with a non-zero exit code. Output to stderr is
+  not an indication of failure. Any output to stdout or stderr may
+  or may not be shown to the user.
+
+Semantics:
+
+* The name of each story (given with STORY) must be unique.
+* All names will be normalised before use (whitespace collapse, leading
+  and trailing whitespace
+* Every ASSUMING, GIVEN, WHEN, THEN, FINALLY must be matched by
+  exactly one IMPLEMENTS. The test runner checks this before running
+  any code.
+* Every IMPLEMENTS may match any number of ASSUMING, GIVEN, WHEN,
+  THEN, or FINALLY. The test runner can warn if an IMPLEMENTS is unused.
+* If ASSUMING fails, that story is skipped, and any FINALLY steps
+  are not run.
+
+See also
+--------
+
+Wikipedia has an article on [Behaviour Driven Development][BDD],
+which can provide background and further explanation to what this
+tools tries to do.
+
+[BDD]: https://en.wikipedia.org/wiki/Behavior-driven_development
+[Markdown]: http://daringfireball.net/projects/markdown/
+
+TODO
+----
+
+* Add DEFINING, PRODUCING, if they turn out to be useful.
+* Need something like ASSUMING, except fail the story if the
+  pre-condition is not true. Useful for testing that you can ssh
+  to localhost when flinging, for example.
+  **DJAS**: We think this might be 'REQUIRING' and it still does
+  not run the FINALLY group.
+* Consider the ordering some more.  IMPLEMENTS can come anywhere
+  but otherwise stories are defined as:
+    * STORY
+    * 0+: ASSUMING
+    * 1+:
+        * 1+: GIVEN
+        * 1+:
+            * 1+: WHEN
+            * 1+: THEN
+        * 0+: FINALLY
diff --git a/setup.py b/setup.py
index 108841a..2822b84 100644
--- a/setup.py
+++ b/setup.py
@@ -32,7 +32,7 @@ class GenerateManpage(build):
     def run(self):
         build.run(self)
         print 'building manpages'
-        for x in ['cmdtest']:
+        for x in ['cmdtest', 'yarn']:
             with open('%s.1' % x, 'w') as f:
                 subprocess.check_call(['python', x,
                                        '--generate-manpage=%s.1.in' % x,
@@ -51,17 +51,19 @@ class CleanMore(clean):
 class Check(Command):
 
     user_options = []
-    
+
     def initialize_options(self):
         pass
-        
+
     def finalize_options(self):
         pass
 
     def run(self):
-        subprocess.check_call(['python', '-m', 'CoverageTestRunner'])
+        subprocess.check_call(
+            ['python', '-m', 'CoverageTestRunner',
+             '--ignore-missing-from', 'without-tests'])
         os.remove('.coverage')
-        
+
         subprocess.check_call(['./cmdtest', 'echo-tests'])
         subprocess.check_call(['./cmdtest', 'sort-tests'])
 
diff --git a/simple.story b/simple.story
new file mode 100644
index 0000000..995318c
--- /dev/null
+++ b/simple.story
@@ -0,0 +1,32 @@
+A simple story test
+===================
+
+This is a very simple story test, which exists only to test
+the story test runner itself.
+
+    STORY a simple story
+
+The following is the actual test in this story:
+
+    GIVEN a clean slate
+    WHEN nothing happens
+    THEN everything is OK
+    AND not all is well
+    FINALLY cleanup
+
+And the implementations follow.
+
+    IMPLEMENTS GIVEN a clean slate
+    echo a clean slate!
+
+    IMPLEMENTS WHEN nothing happens
+    true
+
+    IMPLEMENTS THEN everything is OK
+    echo OK!
+
+    IMPLEMENTS THEN not all is well
+    false
+
+    IMPLEMENTS FINALLY cleanup
+    echo cleaning up
diff --git a/without-tests b/without-tests
new file mode 100644
index 0000000..6aa40f1
--- /dev/null
+++ b/without-tests
@@ -0,0 +1,3 @@
+yarnlib/__init__.py
+setup.py
+yarnlib/elements.py
diff --git a/yarn b/yarn
new file mode 100755
index 0000000..4c2f533
--- /dev/null
+++ b/yarn
@@ -0,0 +1,173 @@
+#!/usr/bin/python
+# Copyright 2013  Lars Wirzenius
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# =*= License: GPL-3+ =*=
+
+
+import cliapp
+import logging
+import os
+import re
+import shutil
+import tempfile
+import time
+import ttystatus
+
+import cmdtestlib
+import yarnlib
+
+
+class YarnRunner(cliapp.Application):
+
+    def setup(self):
+        self.ts = ttystatus.TerminalStatus(period=0.001)
+        self.ts.format(
+            '%ElapsedTime() %Index(story,stories): %String(story_name): '
+            'step %Index(step,steps): %String(step_name)')
+
+    def process_args(self, args):
+        stories, implementations = self.parse_stories(args)
+        self.connect_implementations(stories, implementations)
+
+        self.ts['stories'] = stories
+        self.ts['num_stories'] = len(stories)
+        logging.info('Found %d stories' % len(stories))
+
+        start_time = time.time()
+        failed_stories = []
+        for story in stories:
+            if not self.run_story(story):
+                failed_stories.append(story)
+        duration = time.time() - start_time
+
+        self.ts.clear()
+        self.ts.finish()
+
+        if failed_stories:
+            raise cliapp.AppException(
+                'Test suite FAILED in %s stories' % len(failed_stories))
+
+        print ('Story test suite PASS, with %d stories, in %.1f seconds' %
+            (len(stories), duration))
+
+    def parse_stories(self, filenames):
+        mdparser = yarnlib.MarkdownParser()
+        for filename in filenames:
+            mdparser.parse_file(filename)
+
+        block_parser = yarnlib.BlockParser()
+        block_parser.parse_blocks(mdparser.blocks)
+
+        return block_parser.stories, block_parser.implementations
+
+    def connect_implementations(self, stories, implementations):
+        for story in stories:
+            for step in story.steps:
+                self.connect_implementation(story, step, implementations)
+
+    def connect_implementation(self, story, step, implementations):
+        matching = [i for i in implementations
+                    if step.what == i.what and
+                       re.match('(%s)$' % i.regexp, step.text, re.I)]
+
+        if len(matching) == 0:
+            raise cliapp.AppException(
+                'Story %s, step "%s %s" has no matching '
+                'implementation' %
+                (story.name, step.what, step.text))
+        if len(matching) > 1:
+            s = '\n'.join(
+                'IMPLEMENTS %s %s' % (i.what, i.regexp)
+                for i in matching)
+            raise cliapp.AppException(
+                'Story "%s", step "%s %s" has more than one '
+                'matching implementations:\n%s' %
+                (story.name, step.what, step.text, s))
+
+        assert step.implementation is None
+        step.implementation = matching[0]
+
+    def run_story(self, story):
+        logging.info('Running story %s' % story.name)
+        self.ts['story'] = story
+        self.ts['story_name'] = story.name
+        self.ts['steps'] = story.steps
+
+        datadir = tempfile.mkdtemp()
+
+        cleanup = [s for s in story.steps if s.what == 'FINALLY']
+        normal = [s for s in story.steps if s not in cleanup]
+
+        ok = True
+
+        for step in normal:
+            exit = self.run_step(datadir, story, step)
+            if exit != 0:
+                ok = False
+                break
+
+        for step in cleanup:
+            exit = self.run_step(datadir, story, step)
+            if exit != 0:
+                ok = False
+                break
+
+        shutil.rmtree(datadir)
+
+        return ok
+
+    def run_step(self, datadir, story, step):
+        logging.info('Running step "%s %s"' % (step.what, step.text))
+        logging.info('DATADIR is %s' % datadir)
+        self.ts['step'] = step
+        self.ts['step_name'] = '%s %s' % (step.what, step.text)
+
+        m = re.match(step.implementation.regexp, step.text)
+        assert m is not None
+        env = os.environ.copy()
+        env['DATADIR'] = datadir
+        for i, match in enumerate(m.groups('')):
+            env['MATCH_%d' % (i+1)] = match
+
+        exit, stdout, stderr = cliapp.runcmd_unchecked(
+            ['sh', '-euc', step.implementation.shell], env=env)
+
+        logging.debug('Exit code: %d' % exit)
+        if stdout:
+            logging.debug('Standard output:\n%s' % self.indent(stdout))
+        else:
+            logging.debug('Standard output: empty')
+        if stderr:
+            logging.debug('Standard error:\n%s' % self.indent(stderr))
+        else:
+            logging.debug('Standard error: empty')
+
+        if exit != 0:
+            self.ts.error(
+                'ERROR: In story "%s"\nstep "%s %s" failed,\n'
+                'with exit code %d:\n'
+                'Standard output from shell command:\n%s'
+                'Standard error from shell command:\n%s' %
+                (story.name, step.what, step.text, exit, self.indent(stdout),
+                 self.indent(stderr)))
+
+        return exit
+
+    def indent(self, s):
+        return ''.join('    %s\n' % line for line in s.splitlines())
+
+
+YarnRunner(version=cmdtestlib.__version__).run()
diff --git a/yarn.1.in b/yarn.1.in
new file mode 100644
index 0000000..ead182d
--- /dev/null
+++ b/yarn.1.in
@@ -0,0 +1,115 @@
+.\" Copyright 2013 Lars Wirzenius <liw@liw.fi>
+.\"
+.\" This program is free software: you can redistribute it and/or modify
+.\" it under the terms of the GNU General Public License as published by
+.\" the Free Software Foundation, either version 3 of the License, or
+.\" (at your option) any later version.
+.\"
+.\" This program is distributed in the hope that it will be useful,
+.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
+.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+.\" GNU General Public License for more details.
+.\"
+.\" You should have received a copy of the GNU General Public License
+.\" along with this program.  If not, see <http://www.gnu.org/licenses/>.
+.\"
+.TH YARN 1
+.SH NAME
+yarn \- story testing of Unix command line tools
+.SH SYNOPSIS
+.SH DESCRIPTION
+.B yarn
+is a story testing tool:
+you write a story describing how a user uses your software
+and what should happen,
+and express,
+using very lightweight syntax,
+the story in such a way that it can be tested automatically.
+The story has a simple, but strict structure:
+.IP
+.nf
+GIVEN some setup for the test
+WHEN thing that is to be tested happens
+THEN the post-conditions must be true
+.fi
+.PP
+As an example, consider a very short test story for verifying that
+a backup program works, at least for one simple case.
+.IP
+.nf
+GIVEN some live data in a directory
+AND an empty backup repository
+WHEN a backup is made
+THEN the data case be restored
+.nf
+.PP
+(Note the addition of AND: you can have multiple GIVEN, WHEN, and
+THEN statements. The AND keyword makes the text be more readable.)
+.PP
+Stories are meant to be written in somewhat human readable language.
+However, they are not free form text.
+In addition to the GIVEN/WHEN/THEN structure,
+the text for each of the steps needs a computer-executable implementation.
+This is done by using IMPLEMENTS.
+The backup story from above might be implemented as follows:
+.IP
+.nf
+IMPLEMENTS GIVEN some live data in a directory
+rm -rf "$TESTDIR/data"
+mkdir "$TESTDIR/data"
+echo foo > "$TESTDIR/data/foo"
+.IP
+IMPLEMENTS GIVEN an empty backup repository
+rm -rf "$TESTDIR/repo"
+mkdir "$TESTDIR/repo"
+.IP
+IMPLEMENTS WHEN a backup is made
+backup-program -r "$TESTDIR/repo" "$TESTDIR/data"
+.IP
+IMPLEMENTS THEN the data can be restored
+mkdir "$TESTDIR/restored"
+restore-program -r "$TESTDIR/repo" "$TESTDIR/restored"
+diff -rq "$TESTDIR/data" "$TESTDIR/restored"
+.fi
+.PP
+Each "IMPLEMENT GIVEN" (or WHEN, THEN) is followed by a regular
+expression on the same line,
+and then a shell script that gets executed to implement any step
+that matches the regular expression.
+The implementation can extract data from the match as well:
+for example, the regular expression might allow a file size to be specified.
+.PP
+The above example is a bit silly, of course:
+why go to the effort to obfuscate the various steps?
+The answer is that the various steps,
+implemented using IMPLEMENTS,
+can be combined in many ways,
+to test different aspects of the program being tested.
+.PP
+Moreover,
+by making the step descriptions be human language text,
+matched by regular expressions,
+most of the test can hopefully be written,
+and understood,
+by non-programmers.
+Someone who understands what a program should do,
+could write tests to verify its behaviour.
+The implementations of the various steps need to be implemented
+by a programmer,
+but given a well-designed set of steps,
+with enough flexibility in their implementation,
+that quite a good test suite can be written.
+.SH OPTIONS
+.SH EXAMPLE
+To run
+.B yarn
+on all the stories in your current directory:
+.IP
+.nf
+yarn *.story
+.fi
+.PP
+All the files will be treated together as if they had been one file.
+.SH "SEE ALSO"
+.BR cmdtest (1),
+.BR cliapp (5).
diff --git a/yarn.tests/simple.exit b/yarn.tests/simple.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/yarn.tests/simple.exit
@@ -0,0 +1 @@
+1
diff --git a/yarn.tests/simple.script b/yarn.tests/simple.script
new file mode 100755
index 0000000..83714f9
--- /dev/null
+++ b/yarn.tests/simple.script
@@ -0,0 +1,5 @@
+#!/bin/sh
+
+set -eu
+
+./yarn simple.story
diff --git a/yarn.tests/simple.stderr b/yarn.tests/simple.stderr
new file mode 100644
index 0000000..db4fde4
--- /dev/null
+++ b/yarn.tests/simple.stderr
@@ -0,0 +1,7 @@
+ERROR: In story "a simple story"
+step "THEN not all is well" failed,
+with exit code 1:
+Standard output from shell command:
+Standard error from shell command:
+
+ERROR: Test suite FAILED in 1 stories
diff --git a/yarnlib/__init__.py b/yarnlib/__init__.py
new file mode 100644
index 0000000..07045fc
--- /dev/null
+++ b/yarnlib/__init__.py
@@ -0,0 +1,21 @@
+# Copyright 2013  Lars Wirzenius
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# =*= License: GPL-3+ =*=
+
+
+from mdparser import MarkdownParser
+from elements import Story, StoryStep, Implementation
+from block_parser import BlockParser, BlockError
diff --git a/yarnlib/block_parser.py b/yarnlib/block_parser.py
new file mode 100644
index 0000000..1a600ee
--- /dev/null
+++ b/yarnlib/block_parser.py
@@ -0,0 +1,138 @@
+# Copyright 2013  Lars Wirzenius
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# =*= License: GPL-3+ =*=
+
+
+import cliapp
+
+import yarnlib
+
+
+class BlockError(cliapp.AppException):
+
+    pass
+
+
+# Parse a sequence of textual blocks into Story and Implementation
+# objects, and their constituent objects.
+
+class BlockParser(object):
+
+    def __init__(self):
+        self.stories = []
+        self.implementations = []
+        self.line_parsers = {
+            'STORY': self.parse_story,
+            'GIVEN': self.parse_given,
+            'WHEN': self.parse_when,
+            'THEN': self.parse_then,
+            'FINALLY': self.parse_finally,
+            'AND': self.parse_and,
+            'IMPLEMENTS': self.parse_implementing,
+        }
+
+    def parse_blocks(self, blocks):
+        while blocks:
+            blocks = self.parse_one(blocks)
+
+    def parse_one(self, blocks):
+        assert blocks
+        block = blocks[0]
+        assert block
+        t = block.split('\n', 1)
+        assert len(t) in [1,2]
+        if len(t) == 1:
+            line1 = block
+            block = ''
+        else:
+            line1, block = t
+        if block:
+            blocks[0] = block
+        else:
+            del blocks[0]
+
+        words = line1.split()
+        if not words:
+            return blocks
+        rest = ' '.join(words[1:])
+
+        for keyword in self.line_parsers:
+            if words[0] == keyword:
+                return self.line_parsers[keyword](rest, blocks)
+
+        raise BlockError("Syntax error: unknown step: %s" % line1)
+
+    def parse_story(self, line, blocks):
+        self.stories.append(yarnlib.Story(line))
+        return blocks
+
+    def parse_simple(self, what, line, blocks):
+        if not self.stories:
+            raise BlockError('Syntax errror: %s before STORY' % what)
+        step = yarnlib.StoryStep(what, line)
+        self.stories[-1].steps.append(step)
+        return blocks
+
+    def parse_given(self, line, blocks):
+        return self.parse_simple('GIVEN', line, blocks)
+
+    def parse_when(self, line, blocks):
+        return self.parse_simple('WHEN', line, blocks)
+
+    def parse_then(self, line, blocks):
+        return self.parse_simple('THEN', line, blocks)
+
+    def parse_finally(self, line, blocks):
+        return self.parse_simple('FINALLY', line, blocks)
+
+    def parse_and(self, line, blocks):
+        if not self.stories:
+            raise BlockError('Syntax errror: AND before STORY')
+        story = self.stories[-1]
+        if not story.steps:
+            raise BlockError(
+                'Syntax errror: AND before what it would continue')
+        step = story.steps[-1]
+        assert step.what in self.line_parsers
+        return self.line_parsers[step.what](line, blocks)
+
+    def parse_implementing(self, line, blocks):
+        words = line.split()
+        if len(words) < 2:
+            raise BlockError(
+                'Syntax error: IMPLEMENTS must have what and regexp')
+        what = words[0]
+        regexp = ' '.join(words[1:])
+        if blocks:
+            block = blocks[0]
+            shell = []
+            rest = []
+            for block_line in block.splitlines():
+                if rest or block_line.startswith('IMPLEMENTS'):
+                    rest.append(block_line)
+                else:
+                    shell.append(block_line)
+            shell = '\n'.join(shell)
+            if rest:
+                blocks[0] = '\n'.join(rest)
+            else:
+                del blocks[0]
+        else:
+            shell = ''
+        implementation = yarnlib.Implementation(what, regexp, shell)
+        self.implementations.append(implementation)
+        return blocks
+
diff --git a/yarnlib/block_parser_tests.py b/yarnlib/block_parser_tests.py
new file mode 100644
index 0000000..975444f
--- /dev/null
+++ b/yarnlib/block_parser_tests.py
@@ -0,0 +1,132 @@
+# Copyright 2013  Lars Wirzenius
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# =*= License: GPL-3+ =*=
+
+
+import unittest
+
+import yarnlib
+
+
+class BlockParserTests(unittest.TestCase):
+
+    def setUp(self):
+        self.parser = yarnlib.BlockParser()
+
+    def test_is_initially_empty(self):
+        self.assertEqual(self.parser.stories, [])
+        self.assertEqual(self.parser.implementations, [])
+
+    def test_parses_simple_elements(self):
+        self.parser.parse_blocks(
+            ['STORY foo', 'GIVEN bar',
+             'WHEN foobar\nTHEN yoyo\nFINALLY yay\nAND yeehaa'])
+
+        self.assertEqual(len(self.parser.stories), 1)
+        self.assertEqual(len(self.parser.implementations), 0)
+
+        story = self.parser.stories[0]
+        self.assertEqual(story.name, 'foo')
+        self.assertEqual(len(story.steps), 5)
+        self.assertEqual(story.steps[0].what, 'GIVEN')
+        self.assertEqual(story.steps[0].text, 'bar')
+        self.assertEqual(story.steps[1].what, 'WHEN')
+        self.assertEqual(story.steps[1].text, 'foobar')
+        self.assertEqual(story.steps[2].what, 'THEN')
+        self.assertEqual(story.steps[2].text, 'yoyo')
+        self.assertEqual(story.steps[3].what, 'FINALLY')
+        self.assertEqual(story.steps[3].text, 'yay')
+        self.assertEqual(story.steps[4].what, 'FINALLY')
+        self.assertEqual(story.steps[4].text, 'yeehaa')
+
+    def test_normalises_whitespace(self):
+        self.parser.parse_blocks(['STORY   foo   bar   '])
+        self.assertEqual(self.parser.stories[0].name, 'foo bar')
+
+    def test_handles_empty_line(self):
+        self.parser.parse_blocks(['STORY foo\n\nGIVEN bar\nTHEN foobar'])
+        self.assertEqual(len(self.parser.stories), 1)
+
+    def test_raises_error_for_unknown_step(self):
+        self.assertRaises(
+            yarnlib.BlockError,
+            self.parser.parse_blocks,
+            ['STORY foo\nblah'])
+
+    def test_raises_error_for_step_outside_story(self):
+        self.assertRaises(
+            yarnlib.BlockError,
+            self.parser.parse_blocks,
+            ['GIVEN foo'])
+
+    def test_raises_error_for_AND_before_story(self):
+        self.assertRaises(
+            yarnlib.BlockError,
+            self.parser.parse_blocks,
+            ['AND bar'])
+
+    def test_raises_error_for_AND_before_step(self):
+        self.assertRaises(
+            yarnlib.BlockError,
+            self.parser.parse_blocks,
+            ['STORY foo\nAND bar'])
+
+    def test_parses_implements_in_a_block_by_itself(self):
+        self.parser.parse_blocks(['IMPLEMENTS GIVEN foo\ntrue'])
+        impls = self.parser.implementations
+        self.assertEqual(len(impls), 1)
+        self.assertEqual(impls[0].what, 'GIVEN')
+        self.assertEqual(impls[0].regexp, 'foo')
+        self.assertEqual(impls[0].shell, 'true')
+
+    def test_parses_implements_with_empty_shell_text(self):
+        self.parser.parse_blocks(['IMPLEMENTS GIVEN foo'])
+        impls = self.parser.implementations
+        self.assertEqual(len(impls), 1)
+        self.assertEqual(impls[0].what, 'GIVEN')
+        self.assertEqual(impls[0].regexp, 'foo')
+        self.assertEqual(impls[0].shell, '')
+
+    def test_parses_two_implements_in_a_code_block(self):
+        self.parser.parse_blocks(
+            ['IMPLEMENTS GIVEN foo\ntrue\nIMPLEMENTS WHEN bar\ncat /dev/null'])
+        impls = self.parser.implementations
+        self.assertEqual(len(impls), 2)
+        self.assertEqual(impls[0].what, 'GIVEN')
+        self.assertEqual(impls[0].regexp, 'foo')
+        self.assertEqual(impls[0].shell, 'true')
+        self.assertEqual(impls[1].what, 'WHEN')
+        self.assertEqual(impls[1].regexp, 'bar')
+        self.assertEqual(impls[1].shell, 'cat /dev/null')
+
+    def test_raises_error_for_implements_with_no_args(self):
+        self.assertRaises(
+            yarnlib.BlockError,
+            self.parser.parse_blocks,
+            ['IMPLEMENTS'])
+
+    def test_raises_error_for_implements_with_one_args(self):
+        self.assertRaises(
+            yarnlib.BlockError,
+            self.parser.parse_blocks,
+            ['IMPLEMENTS GIVEN'])
+
+    def test_raises_error_for_implements_with_first_args_not_a_keyword(self):
+        self.assertRaises(
+            yarnlib.BlockError,
+            self.parser.parse_blocks,
+            ['IMPLEMENTS foo'])
+
diff --git a/yarnlib/elements.py b/yarnlib/elements.py
new file mode 100644
index 0000000..a1dea0d
--- /dev/null
+++ b/yarnlib/elements.py
@@ -0,0 +1,51 @@
+# Copyright 2013  Lars Wirzenius
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# =*= License: GPL-3+ =*=
+
+
+# This is a step in a story: GIVEN, WHEN, THEN, etc.
+
+class StoryStep(object):
+
+    def __init__(self, what, text):
+        self.what = what
+        self.text = text
+        self.implementation = None
+
+
+# This is the story itself.
+
+class Story(object):
+
+    def __init__(self, name):
+        self.name = name
+        self.steps = []
+
+
+# This is an IMPLEMENTS chunk.
+
+class Implementation(object):
+
+    def __init__(self, what, regexp, shell):
+        self.what = what
+        self.regexp = regexp
+        self.shell = shell
+
+    def execute(self):
+        exit, out, err = cliapp.runcmd_unchecked(
+            ['sh', '-c', 'set -eu\n' + self.shell])
+        return exit
+
diff --git a/yarnlib/mdparser.py b/yarnlib/mdparser.py
new file mode 100644
index 0000000..6d16772
--- /dev/null
+++ b/yarnlib/mdparser.py
@@ -0,0 +1,75 @@
+# Copyright 2013  Lars Wirzenius
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# =*= License: GPL-3+ =*=
+
+
+import logging
+import markdown
+import StringIO
+from markdown.treeprocessors import Treeprocessor
+
+
+#
+# Classes for Markdown parsing. See python-markdown documentation
+# for details. We want to find all top level code blocks (indented
+# four spaces in the Markdown), which we'll parse for story test
+# stuff later on. We create a Python markdown extension and use
+# "tree processor" to analyse the parsed ElementTree at the right
+# moment for top level <pre> blocks.
+#
+
+# This is a Treeprocessor that iterates over the parsed Markdown,
+# as an ElementTree, and finds all top level code blocks.
+
+class GatherCodeBlocks(Treeprocessor):
+
+    def __init__(self, blocks):
+        self.blocks = blocks
+
+    def run(self, root):
+        for child in root.getchildren():
+            if child.tag == 'pre':
+                code = child.find('code')
+                self.blocks.append(code.text)
+        return root
+
+# This is the Python Markdown extension to call the code block
+# gatherer at the right time. It stores the list of top level
+# code blocks as the blocks attribute.
+
+class ParseStoryTestBlocks(markdown.extensions.Extension):
+
+    def extendMarkdown(self, md, md_globals):
+        self.blocks = []
+        self.gatherer = GatherCodeBlocks(self.blocks)
+        md.treeprocessors.add('gathercode', self.gatherer, '_end')
+
+
+class MarkdownParser(object):
+
+    def __init__(self):
+        self.blocks = []
+
+    def parse_string(self, text):
+        ext = ParseStoryTestBlocks()
+        f = StringIO.StringIO()
+        markdown.markdown(text, output=f, extensions=[ext])
+        self.blocks = ext.blocks
+
+    def parse_file(self, filename): # pragma: no cover
+        with open(filename) as f:
+            self.parse_string(f.read())
+
diff --git a/yarnlib/mdparser_tests.py b/yarnlib/mdparser_tests.py
new file mode 100644
index 0000000..af0c7af
--- /dev/null
+++ b/yarnlib/mdparser_tests.py
@@ -0,0 +1,84 @@
+# Copyright 2013  Lars Wirzenius
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# =*= License: GPL-3+ =*=
+
+
+import unittest
+
+import yarnlib
+
+
+class MarkdownParserTests(unittest.TestCase):
+
+    def setUp(self):
+        self.parser = yarnlib.MarkdownParser()
+
+    def test_finds_code_block(self):
+        self.parser.parse_string('''
+This is blah blah text.
+
+    this is a code block
+
+More text.
+''')
+        self.assertEqual(self.parser.blocks, ['this is a code block\n'])
+
+    def test_finds_consecutive_code_blocks_as_one(self):
+        self.parser.parse_string('''
+This is blah blah text.
+
+    this is a code block
+
+    this is a second code block
+
+More text.
+''')
+        self.assertEqual(
+            self.parser.blocks,
+            ['this is a code block\n\nthis is a second code block\n'])
+
+    def test_finds_code_blocks_with_text_in_between_as_two_blocks(self):
+        self.parser.parse_string('''
+This is blah blah text.
+
+    this is a code block
+
+Blah.
+
+    this is a second code block
+
+More text.
+''')
+        self.assertEqual(
+            self.parser.blocks,
+            ['this is a code block\n', 'this is a second code block\n'])
+
+    def test_only_finds_top_level_code_blocks(self):
+        self.parser.parse_string('''
+This is blah blah text.
+
+    this is a code block
+
+And now a list:
+
+*   list item
+
+        this is a second level code block
+
+More text.
+''')
+        self.assertEqual(self.parser.blocks, ['this is a code block\n'])
+