summaryrefslogtreecommitdiff
path: root/yarns/010-tests.yarn
blob: e2ec72d9f8963b0b491386efca889aa65fe4cdd4 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
% copyright-statement-lint test suite

Introduction
============

`copyright-statement-lint` checks source files for copyright
statements, and checks that their copyright year matches the
latest git commit. As such, this test suite needs to verify several
different cases: no copyright statement, wrong year, multiple
statements, etc. Also various forms of copyright statements.

All of these are fairly simple, however. The test creates a file with
a suitable statement, runs the tool against the file, and checks that
the output of the tool is as expected.

Scenarios
=========

No copyright file
-----------------

    SCENARIO no copyright file
    GIVEN an empty file FOOBAR
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 1
    AND stderr contains "FOOBAR.*no copyright statement"

Simple copyright statement
--------------------------

    SCENARIO simple copyright statement
    GIVEN a file FOOBAR containing "Copyright 2014 Foo Bar"

    GIVEN commit year is 2013
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 1
    AND stderr contains "FOOBAR.*2013 not contained"
    
    GIVEN commit year is 2014
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 0

    GIVEN commit year is 2015
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 1
    AND stderr contains "FOOBAR.*2015 not contained"

Itemised copyright years
------------------------

    SCENARIO itemised copyright years
    GIVEN a file FOOBAR containing "Copyright 1999,2014 Foo Bar"

    GIVEN commit year is 1998
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 1
    AND stderr contains "FOOBAR.*1998 not contained"
    
    GIVEN commit year is 1999
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 0

    GIVEN commit year is 2000
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 1
    AND stderr contains "FOOBAR.*2000 not contained"

    GIVEN commit year is 2013
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 1
    AND stderr contains "FOOBAR.*2013 not contained"
    
    GIVEN commit year is 2014
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 0

    GIVEN commit year is 2015
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 1
    AND stderr contains "FOOBAR.*2015 not contained"

Range of copyright years
------------------------

    SCENARIO range of copyright years
    GIVEN a file FOOBAR containing "Copyright 1999-2014 Foo Bar"

    GIVEN commit year is 1998
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 1
    AND stderr contains "FOOBAR.*1998 not contained"
    
    GIVEN commit year is 1999
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 0

    GIVEN commit year is 2000
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 0

    GIVEN commit year is 2013
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 0
    
    GIVEN commit year is 2014
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 0

    GIVEN commit year is 2015
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 1
    AND stderr contains "FOOBAR.*2015 not contained"

Combined itemised and range of copyright years
----------------------------------------------

    SCENARIO combined itemised and range of copyright years
    GIVEN a file FOOBAR containing "Copyright 1999-2000, 2014 Foo Bar"

    GIVEN commit year is 1998
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 1
    AND stderr contains "FOOBAR.*1998 not contained"
    
    GIVEN commit year is 1999
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 0

    GIVEN commit year is 2000
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 0

    GIVEN commit year is 2001
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 1
    AND stderr contains "FOOBAR.*2001 not contained"
    
    GIVEN commit year is 2013
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 1
    AND stderr contains "FOOBAR.*2013 not contained"
    
    GIVEN commit year is 2014
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 0

    GIVEN commit year is 2015
    WHEN copyright-statement-lint is run against FOOBAR
    THEN exit code is 1
    AND stderr contains "FOOBAR.*2015 not contained"


Implementations
===============

File creation
-------------

These are all pretty straightforward. Any files are created 
in `$DATADIR` using names given by the user.

    IMPLEMENTS GIVEN an empty file (\S+)
    touch "$DATADIR/$MATCH_1"

    IMPLEMENTS GIVEN a file (\S+) containing "(.*)"
    printf '%s\n' "$MATCH_2" > "$DATADIR/$MATCH_1"

Pretending what the commit year is
----------------------------------

    IMPLEMENTS GIVEN commit year is (\d+)
    printf '[config]\ndebug-commit-year = %s\n' "$MATCH_1" \
        > "$DATADIR/test.conf"

Running and checking the results of `copyright-statement-lint`
--------------------------------------------------------------

We only have this one way of running the tool. We capture stdout,
stderr, and the exit code. Note: we do not fail the step even if the
tool fails.

    IMPLEMENTS WHEN copyright-statement-lint is run against (\S+)
    # Given set -e, we need to use an if to capture non-zero exit
    # code. Don't want to turn set -e off, so we capture other errors.
    if "$SRCDIR/copyright-statement-lint" \
        --no-default-config "$DATADIR/$MATCH_1" \
        --config "$DATADIR/test.conf" \
        > "$DATADIR/stdout" \
        2> "$DATADIR/stderr"
    then
        exit=0
    else
        exit=$?
    fi
    echo "$exit" > "$DATADIR/exit"

Exit code checking. Since we don't fail the running of the tool,
every scenario is expected to check the exit code.

    IMPLEMENTS THEN exit code is (\d+)
    cat "$DATADIR/exit"
    grep -Fx "$MATCH_1" "$DATADIR/exit"

Check stdout/stderr.

    IMPLEMENTS THEN (\S+) contains "(.*)"
    cat "$DATADIR/$MATCH_1"
    grep "$MATCH_2" "$DATADIR/$MATCH_1"