1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
|
[[!meta title="Iteration planning notes: January 18"]]
[[!tag meeting]]
[[!meta date="2020-01-18 20:26"]]
What has happened
=============================================================================
* Lars has improved the code generator, and polished the document
generator, and written the first self-tests for Subplot.
* There is now a site for automatically generated example subplots:
<https://examples.subplot.liw.fi/>.
* Overall, Subplot is starting to be ready for some careful real use.
Lars will start using it at work for internal tooling.
Discussion
=============================================================================
* Introducing Subplot to co-workers.
* Lars will write the simplest subplot for an internal tool that
is actually useful, and present Subplot to co-workers, and then
try iterate fast to make it a more impressive use of the tool.
* The internal subplot document can be an example when giving a
tutorial to co-workers.
* Lars will also ask co-workers to read Subplot's own
documentation, and gather feedback from them.
* Approaching other people for feedback on Subplot.
* After Lars is back from his work trip, he and Daniel will
together draft a blog post to introduce Subplot and invite
people to try it for real, with the understanding that Subplot
is at alpha level.
* The goal is to find one or two people who're willing to give
Subplot a real try, and willing to tell us what they think, and
report any problems they have.
* Improving the website.
* <https://subplot.liw.fi/> is plain and boring, but that's OK.
It's structure, what information it presents, in what order, and
in what way, could be improved a lot. We feel it's not something
we're good at.
* In the long term, some user testing would be a good idea, but a
lot of work.
* In the short term, we'll try to attract a contributor or two to
work on the website.
* We'll collect questions people have that aren't answered well on
the site, in lieu of actual user testing.
* Issue tracking, changing the software.
* We'll start advetising the gitlab repository for Subplot
(<https://gitlab.com/larswirzenius/subplot>) as the place to
file issues and merge requests.
* We'll start filing issues there ourselves.
* We'll start changing code using merge requests ourselves.
* For now, self-merging is OK. We'll figure out more formal
processes later, if and when there are more contributors, and we
get more experience with gitlab.
* Handling errors in docgen.
* Currently docgen either fails to produce output (pdflatex not
installed)ΒΈ or embeds error messages in the output document.
* In the medium term it'd be good to introduce the concepts of
warnings and errors. Warnings get embedded in the output
document, errors prevent an output document from happening.
Also, the user should have a way to run docgen in a strict mode,
where all warnings are errors.
* Using `pandoc_ast` for parsing Markdown.
* Both docgen and codegen are built around the Pandoc abstract
syntax tree. Docgen mutates the AST to affect the output
document; codegen examines the AST to extract scenarios and
information from the document.
* This is OK otherwise, but Pandoc doesn't enable very good error
messages: nodes in the AST don't carry any information on where
in the source document they come from. Especially for codegen
this means the end user running the generated test program has
trouble relating a test failure back to the right line in the
source document.
* On the other hand, having docgen and codegen use different
parsers might result in them understanding the same input
differently, and that would be a problem.
* The current situation is good enough for alpha.
* We can improve error messages by giving more structural
locations (headings etc).
* Pandoc may some day give better location information:
<https://github.com/jgm/pandoc/issues/4565>.
* The <https://docs.rs/pulldown-cmark/0.6.1/pulldown_cmark/> crate
for Rust would be good if we want to use another parser.
* Document titles.
* At the moment the document title in the YAML metadata block is
optional. It is treated as the empty string, if not present.
* For sp-filter, which is a Pandoc filter, and heavily used by
Lars for websites via ikiwiki, this is the right thing to do.
ikiwiki has another way to specify a page title, and requiring
YAML metadata blocks is not acceptable.
* For docgen and codegen, however, we could make it a warning or
an error to not have a title, or default the title to the
document filename.
* Adding support for other languages.
* codegen now supports Python. The output program is generated
using a template, so in principle supporting other languages is
straightforward.
* Some languages, including Rust, would need codegen to support
output program consisting of many files.
* Shell would be an obvious next language to support. Adding the
second language would mean adding the infrastructure for codegen
to support more than one language (so the user would need to
specify the language, etc). After that, more languages should
only be a matter of adding a new template.
* Other possible languages: Rust, Go, Haskell, Perl, Ruby, PHP,
Cucumber/Gherkin.
* Drawing the line between functional tests and acceptance tests
* Acceptance tests tend to be functional tests as well, but not
all functional tests are acceptance tests.
* A test is an acceptance test is it verifies acceptance criteria
by stakeholders. Brugally, if the test checks something the
client demands, it's an acceptance test.
* Depending on how a project using Subplot structured and
organized, some or all functional tests may be implemented using
Subplot. Non-acceptance functional tests might be in a second
subplot document, which shares the bindings and step function
implementations of the acceptance test subplot.
Subplot tries to be neutral for process and project
organization.
* Subplot marketing.
* We have a tentative almost-offer of a marketing video for
Subplot. We would like that and give the person making the offer
free hands, but are available for feedback and discussion as
needed.
* Test program output.
* At the moment the generated test program output is unstructured.
This works fine, for now.
* There are many test output formats in use in the industry. One
simple on is TAP, the Test Anything Protocol.
* Structure will be necessary once the test programs start running
scenarios concurrently, but until then, the current approach
will do.
* Community building.
* We recognize that we aren't any good at community building.
* We hope to start attracting real users relatively soon.
* We hope that at least some of the users also become contributors
and we welcome contributions in any form, not just code.
* We have a bi-weekly video chat, and would be happy to open that
to contributors. We regret that we don't have the time and
energy to support users via video chat.
Actions
=============================================================================
* Lars to introduce Subplot at work.
* Lars to change website to point people at gitlab for issues, merge
requests.
* Lars to file issue about docgen, codegen needing to handle missing
titles in a better way.
* Lars to file issue about codegen needing to support at least shell
as another language.
* Lars to file issue that Python template should randomize order in
which scenarios are run.
|
