1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
|
.\" Copyright 2013 Lars Wirzenius <liw@liw.fi>
.\"
.\" This program is free software: you can redistribute it and/or modify
.\" it under the terms of the GNU General Public License as published by
.\" the Free Software Foundation, either version 3 of the License, or
.\" (at your option) any later version.
.\"
.\" This program is distributed in the hope that it will be useful,
.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
.\" GNU General Public License for more details.
.\"
.\" You should have received a copy of the GNU General Public License
.\" along with this program. If not, see <http://www.gnu.org/licenses/>.
.\"
.TH YARN 1
.SH NAME
yarn \- story testing of Unix command line tools
.SH SYNOPSIS
.SH DESCRIPTION
.B yarn
is a story testing tool:
you write a story describing how a user uses your software
and what should happen,
and express,
using very lightweight syntax,
the story in such a way that it can be tested automatically.
The story has a simple, but strict structure:
.IP
.nf
GIVEN some setup for the test
WHEN thing that is to be tested happens
THEN the post-conditions must be true
.fi
.PP
As an example, consider a very short test story for verifying that
a backup program works, at least for one simple case.
.IP
.nf
GIVEN some live data in a directory
AND an empty backup repository
WHEN a backup is made
THEN the data case be restored
.nf
.PP
(Note the addition of AND: you can have multiple GIVEN, WHEN, and
THEN statements. The AND keyword makes the text be more readable.)
.PP
Stories are meant to be written in somewhat human readable language.
However, they are not free form text.
In addition to the GIVEN/WHEN/THEN structure,
the text for each of the steps needs a computer-executable implementation.
This is done by using IMPLEMENTS.
The backup story from above might be implemented as follows:
.IP
.nf
IMPLEMENTS GIVEN some live data in a directory
rm -rf "$TESTDIR/data"
mkdir "$TESTDIR/data"
echo foo > "$TESTDIR/data/foo"
.IP
IMPLEMENTS GIVEN an empty backup repository
rm -rf "$TESTDIR/repo"
mkdir "$TESTDIR/repo"
.IP
IMPLEMENTS WHEN a backup is made
backup-program -r "$TESTDIR/repo" "$TESTDIR/data"
.IP
IMPLEMENTS THEN the data can be restored
mkdir "$TESTDIR/restored"
restore-program -r "$TESTDIR/repo" "$TESTDIR/restored"
diff -rq "$TESTDIR/data" "$TESTDIR/restored"
.fi
.PP
Each "IMPLEMENT GIVEN" (or WHEN, THEN) is followed by a regular
expression on the same line,
and then a shell script that gets executed to implement any step
that matches the regular expression.
The implementation can extract data from the match as well:
for example, the regular expression might allow a file size to be specified.
.PP
The above example is a bit silly, of course:
why go to the effort to obfuscate the various steps?
The answer is that the various steps,
implemented using IMPLEMENTS,
can be combined in many ways,
to test different aspects of the program being tested.
.PP
Moreover,
by making the step descriptions be human language text,
matched by regular expressions,
most of the test can hopefully be written,
and understood,
by non-programmers.
Someone who understands what a program should do,
could write tests to verify its behaviour.
The implementations of the various steps need to be implemented
by a programmer,
but given a well-designed set of steps,
with enough flexibility in their implementation,
that quite a good test suite can be written.
.SH OPTIONS
.SH EXAMPLE
To run
.B yarn
on all the stories in your current directory:
.IP
.nf
yarn *.story
.fi
.PP
All the files will be treated together as if they had been one file.
.SH "SEE ALSO"
.BR cmdtest (1),
.BR cliapp (5).
|
