diff options
Diffstat (limited to 'subplot.md')
| -rw-r--r-- | subplot.md | 1401 |
1 files changed, 939 insertions, 462 deletions
@@ -1,22 +1,3 @@ ---- -title: "Subplot" -author: The Subplot project -bindings: -- subplot.yaml -- lib/runcmd.yaml -- lib/files.yaml -impls: - python: - - subplot.py - - lib/files.py - - lib/runcmd.py - rust: - - subplotlib/subplot-rust-support.rs -classes: -- json -... - - # Introduction Subplot is software to help capture and communicate acceptance @@ -39,20 +20,20 @@ document. We define the various concepts relevant to Subplot as follows: -* **Acceptance criteria**: What the stakeholders require of the system +* **Acceptance criteria:** What the stakeholders require of the system for them to be happy with it and use it. -* **Stakeholder**: Someone with a keen interest in the success of a +* **Stakeholder:** Someone with a keen interest in the success of a system. They might be a paying client, someone who uses the system, or someone involved in developing the system. Depending on the system and project, some stakeholders may have a bigger say than others. -* **Acceptance test**: How stakeholders verify that the system +* **Acceptance test:** How stakeholders verify that the system fulfills the acceptance criteria, in an automated way. Some criteria may not be possible to verify automatically. -* **Scenario**: In Subplot, the acceptance criteria are written as +* **Scenario:** In Subplot, the acceptance criteria are written as freeform prose, with diagrams, etc. The scenarios, which are embedded blocks of Subplot scenario language, capture the mechanisms of verifying that criteria are met - the acceptance tests - showing @@ -82,7 +63,7 @@ technical text that's aimed at all your stakeholders. ## Subplot architecture Subplot reads an input document, in Markdown, and generates a typeset -output document, as PDF or HTML, for all stakeholders to understand. +output document, as HTML, for all stakeholders to understand. Subplot also generates a test program, in Python, that verifies the acceptance criteria are met, for developers and testers and auditors to verify the system under test meets its acceptance criteria. The @@ -104,9 +85,6 @@ impl [shape=box]; subplot [label="Subplot"]; subplot [shape=ellipse]; -pdf [label="foo.pdf \n PDF (generated)"] -pdf [shape=note]; - html [label="foo.html \n HTML (generated)"] html [shape=note]; @@ -119,26 +97,19 @@ report [shape=note]; md -> subplot; bindings -> subplot; impl -> subplot; -subplot -> pdf; subplot -> html; subplot -> testprog; testprog -> report; } ``` -[Pandoc]: https://pandoc.org/ - -Subplot uses the [Pandoc][] software for generating PDF and HTML -output documents. In fact, any output format supported by Pandoc can -be requested by the user. Depending on the output format, Pandoc may -use, for example, LaTeX. Subplot interprets parts of the Markdown -input file itself. +Subplot generated HTML itself. Subplot actually consists mainly of two separate programs: **subplot docgen** for generating output documents, and **subplot codegen** for generating the test program. There are a couple of additional tools (**subplot metadata** for reporting meta data about a Subplot document, and -**subplot-filter** for doing the document generation as a Pandoc filter). +**subplot extract** for extracting embedded files from a subplot document. Thus a more detailed architecture view is shown below. @@ -159,9 +130,6 @@ docgen [shape=ellipse]; codegen [label="subplot codegen"]; codegen [shape=ellipse]; -pdf [label="foo.pdf \n PDF (generated)"] -pdf [shape=note]; - html [label="foo.html \n HTML (generated)"] html [shape=note]; @@ -176,7 +144,6 @@ bindings -> docgen; md -> codegen; bindings -> codegen; impl -> codegen; -docgen -> pdf; docgen -> html; codegen -> testprog; testprog -> report; @@ -334,56 +301,56 @@ tests for Subplot](#acceptance). Each requirement here is given a unique mnemonic id for easier reference in discussions. -**UnderstandableTests** +* **UnderstandableTests** -: Acceptance tests should be possible to express in a way that's - easily understood by all stakeholders, including those who are - not software developers. + Acceptance tests should be possible to express in a way that's + easily understood by all stakeholders, including those who are not + software developers. _Done_ but requires the Subplot document to be written with care. -**EasyToWriteDocs** +* **EasyToWriteDocs** -: The markup language for writing documentation should be easy to + The markup language for writing documentation should be easy to write. _Done_ by using Markdown. -**AidsComprehension** +* **AidsComprehension** -: The formatted human-readable documentation should use good layout + The formatted human-readable documentation should use good layout and typography to enhance comprehension. - _In progress_ — typesetting via Pandoc works, but may need - review and improvement. + _In progress_ — we currently only output HTML, but may add + PDF output back later. -**CodeSeparately** +* **CodeSeparately** -: The code to implement the acceptance criteria should not be + The code to implement the acceptance criteria should not be embedded in the documentation source, but be in separate files. This makes it easier to edit without specialised tooling. _Done_ by keeping scenario step implementations in a separate file. -**AnyProgammingLanguage** +* **AnyProgammingLanguage** -: The developers implementing the acceptance tests should be free to + The developers implementing the acceptance tests should be free to use a language they're familiar and comfortable with. Subplot should not require them to use a specific language. _Not done_ — only Python supported at the moment. -**FastTestExecution** +* **FastTestExecution** -: Executing the acceptance tests should be fast. + Executing the acceptance tests should be fast. _Not done_ — the generated Python test program is simplistic and linear. -**NoDeployment** +* **NoDeployment** -: The acceptance test tooling should assume the system under test is + The acceptance test tooling should assume the system under test is already deployed and available. Deploying is too big of a problem space to bring into the scope of acceptance testing, and there are already good tools for deployment. @@ -391,9 +358,9 @@ reference in discussions. _Done_ by virtue of letting those who implement the scenario steps worry about it. -**MachineParseableResults** +* **MachineParseableResults** -: The tests should produce a machine parseable result that can be + The tests should produce a machine parseable result that can be archived, post-processed, and analyzed in ways that are of interest to the project using Subplot. For example, to see trends in how long tests take, how often tests fail, to find regressions, @@ -406,16 +373,13 @@ reference in discussions. Subplot reads three input files, each in a different format: -* The document file, which uses the Markdown dialects understood by - Pandoc. +* The document file in [GitHub Flavored Markdown](https://github.github.com/gfm/). * The bindings file, in YAML. * The functions file, in Bash or Python. Subplot interprets marked parts of the input document -specially. It does this via the Pandoc abstract syntax tree, rather -than text manipulation, and thus anything that Pandoc understands is -understood by Subplot. We will not specify Pandoc's dialect of -Markdown here, only the parts Subplot pays attention to. +specially. These are fenced code blocks tagged with the `sceanrio`, +`file`, or `example` classes. ## Scenario language @@ -540,14 +504,9 @@ will deal with formatting that nicely for you. ## Document markup -[Pandoc]: https://pandoc.org/ +Subplot parses Markdown input files using GitHub-flavored Markdown. -Subplot uses [Pandoc][], the universal document converter, to parse -the Markdown file, and thus understands the variants of Markdown that -Pandoc supports. This includes traditional Markdown, CommonMark, and -GitHub-flavored Markdown. - -[fenced code blocks]: https://pandoc.org/MANUAL.html#fenced-code-blocks +[fenced code blocks]: https://github.github.com/gfm/#fenced-code-blocks Subplot extends Markdown by treating certain certain tags for [fenced code blocks][] specially. A scenario, for example, would look like @@ -569,6 +528,11 @@ block for the test program. Snippets under the same heading belong together; the next heading of the same or a higher level ends the scenario. +For `scenario` blocks you may not use any attributes. All attributes +are reserved for Subplot. Subplot doesn't define any attributes yet, +but by reserving all of them, it can add them later without it being +a breaking change. + For embedding test data files in the Markdown document, Subplot understands the `file` tag: @@ -580,11 +544,7 @@ This data is accessible to the test program as 'filename'. The `.file` attribute is necessary, as is the identifier, here `#filename`. The generated test program can access the data using the -identifier (without the #). The mechanism used is generic to Pandoc, -and can be used to affect the typesetting by adding more attributes. -For example, Pandoc can typeset the data in the code block using -syntax highlighting, if the language is specified: `.markdown`, -`.yaml`, or `.python`, for example. +identifier (without the #). Subplot also understands the `dot` and `roadmap` tags, and can use the Graphviz dot program, or the [roadmap][] Rust crate, to produce @@ -640,30 +600,23 @@ given file not-numbered-lines.txt ## Document metadata -Pandoc supports, and Subplot makes use of, a [YAML metadata block][] in a -Markdown document. This can and should be used to set the document -title, authors, date (version), and can be used to control some of the -typesetting. Crucially for Subplot, the bindings and functions files -are named in the metadata block, rather than Subplot deriving them -from the input file name. - -[YAML metadata block]: https://pandoc.org/MANUAL.html#extension-yaml_metadata_block - -As an example, the metadata block for the Subplot document might look -as follows. The `---` before and `...` after the block are mandatory: -they are how Pandoc recongizes the block. +Document metadata is read from a YAML file. This can used to set the +document title, authors, date (version), and more. Crucially for +Subplot, the bindings and functions files are named in the metadata +block, rather than Subplot deriving them from the input file name. -~~~{.yaml .numberLines} ---- +~~~{.file .yaml .numberLines} title: "Subplot" -author: The Subplot project +authors: +- The Subplot project date: work in progress +markdowns: +- subplot.md bindings: - subplot.yaml impls: python: - subplot.py -... ~~~ There can be more than one bindings or functions file: use a YAML @@ -734,6 +687,24 @@ implementation functions: * A binding for a "then everything is OK" step, which captures nothing, and calls the `check_everything_is_ok` function. +## Step functions and cleanup + +A step function must be atomic: either it completes successfully, or +it cleans up any changes it made before returning an indication of +failure. + +A cleanup function is only called for successfully executed step +functions. + +For example, consider a step that creates and starts a virtual +machine. The step function creates the VM, then starts it, and if both +actions succeeds, the step succeeds. A cleanup function for that step +will stop and delete the VM. The cleanup is only called if the step +succeeded. If the step function manages to create the VM, but not +start it, it's the step function's responsibility to delete the VM, +before it signals failure. The cleanup function won't be called in +that case. + ### Simple patterns The simple patterns are of the form `{name}` and match a single word @@ -790,7 +761,7 @@ will emit a warning if the file is not found, and subplot codegen will emit an e Bindings can contain an `impl` map which connects the binding with zero or more language templates. If a binding has no `impl` entries then it can still be -used to `docgen` a PDF or HTML document from a subplot document. This permits a +used to `docgen` a HTML document from a subplot document. This permits a workflow where requirements owners / architects design the validations for a project and then engineers implement the step functions to permit the validations to work. @@ -815,24 +786,24 @@ Here is an example of a binding from one of those libraries: ### Embedded file name didn't match ```scenario +given file badfilename.subplot given file badfilename.md and file b.yaml and file f.py and an installed subplot -when I run subplot docgen --merciful badfilename.md -o foo.pdf -then file foo.pdf exists when I try to run subplot codegen --run badfilename.md -o test.py then command fails ``` -~~~{#badfilename.md .file .markdown .numberLines} ---- +~~~{#badfilename.subplot .file .yaml .numberLines} title: Bad filenames in matched steps do not permit codegen +markdowns: [badfilename.md] bindings: [b.yaml] impls: python: [f.py] -... +~~~ +~~~{#badfilename.md .file .markdown .numberLines} # Bad filename ```scenario @@ -841,6 +812,74 @@ given file missing.md ~~~ +### Bindings file strictness - given when then + +The bindings file is semi-strict. For example you must have only one +of `given`, `when`, or `then` in your binding. + + +```scenario +given file badbindingsgwt.subplot +and file badbindingsgwt.md +and file badbindingsgwt.yaml +and an installed subplot +when I try to run subplot docgen --output ignored.html badbindingsgwt.subplot +then command fails +and stderr contains "binding has more than one keyword" +``` + +~~~{#badbindingsgwt.subplot .file .yaml .numberLines} +title: Bad bindings cause everything to fail +markdowns: [badbindingsgwt.md] +bindings: [badbindingsgwt.yaml] +~~~ + +~~~{#badbindingsgwt.md .file .markdown .numberLines} +# Bad bindings +```scenario +given we won't reach here +``` +~~~ + +~~~{#badbindingsgwt.yaml .file .yaml .numberLines} +- given: we won't reach here + then: we won't reach here +~~~ + +### Bindings file strictness - unknown field + +The bindings file is semi-strict. For example, you must not have keys +in the bindings file which are not known to Subplot. + + +```scenario +given file badbindingsuf.subplot +and file badbindingsuf.md +and file badbindingsuf.yaml +and an installed subplot +when I try to run subplot docgen --output ignored.html badbindingsuf.subplot +then command fails +and stderr contains "unknown field `function`" +``` + +~~~{#badbindingsuf.subplot .file .yaml .numberLines} +title: Bad bindings cause everything to fail +markdowns: [badbindingsuf.md] +bindings: [badbindingsuf.yaml] +~~~ + +~~~{#badbindingsuf.md .file .markdown .numberLines} +# Bad bindings +```scenario +given we won't reach here +``` +~~~ + +~~~{#badbindingsuf.yaml .file .yaml .numberLines} +- given: we won't reach here + function: old_school_function +~~~ + ## Functions file Functions implementing steps are supported in Bash and Python. The @@ -1010,14 +1049,16 @@ They're separate from the scenarios so that the scenarios are shorter and clearer, but also so that the input files do not need to be duplicated for each scenario. -~~~~{#simple.md .file .markdown .numberLines} ---- +~~~~{#simple.subplot .file .yaml .numberLines} title: Test scenario +markdowns: +- simple.md bindings: [b.yaml] impls: python: [f.py] -... +~~~~ +~~~~{#simple.md .file .markdown .numberLines} # Simple This is the simplest possible test scenario @@ -1088,21 +1129,20 @@ def foobar_was_done(ctx): ### Smoke test The scenario below uses the input files defined above to run some tests -to verify that Subplot can build a PDF and an HTML document, and +to verify that Subplot can build an HTML document, and execute a simple scenario successfully. The test is based on generating the test program from an input file, running the test program, and examining the output. ~~~scenario +given file simple.subplot given file simple.md and file b.yaml and file f.py and an installed subplot -when I run subplot docgen simple.md -o simple.pdf -then file simple.pdf exists -when I run subplot docgen simple.md -o simple.html +when I run subplot docgen simple.subplot -o simple.html then file simple.html exists -when I run subplot codegen --run simple.md -o test.py +when I run subplot codegen --run simple.subplot -o test.py then scenario "Simple" was run and step "given precondition foo" was run and step "when I do bar" was run @@ -1110,31 +1150,153 @@ and step "then bar was done" was run and command is successful ~~~ +## Indented scenario steps are not allowed + +_Requirement: A scenario step starts at the beginning of the line._ + +Justification: We may want to allow continuing a step to the next +line, but as of June, 2023, we haven't settled on a syntax for this. +However, whatever syntax we do eventually choose, it will be easier +to add that if scenario steps start at the beginning of a line, +without making a breaking change. + +~~~scenario +given file indented-step.subplot +given file indented-step.md +given file b.yaml +given an installed subplot +when I try to run subplot docgen indented-step.subplot -o foo.html +then command fails +and stderr contains "indented" +~~~ + +~~~{#indented-step.subplot .file .yaml .numberLines} +title: Indented scenario step +markdowns: + - indented-step.md +bindings: + - b.yaml +~~~ + +~~~~~~{#indented-step.md .file .markdown .numberLines} +# This is a title + +~~~scenario + given precondition +~~~ +~~~~~~ + +## Named code blocks must have an appropriate class + +_Requirement: Named code blocks must carry an appropriate class such as file or example_ + +Justification: Eventually we may want to add other meanings to named blocks, +currently the identifier cannot be used except to refer to the block as a named file, +but we may want to in the future so this is here to try and prevent any future +incompatibilities. + +~~~scenario +given file named-code-blocks-appropriate.subplot +given file named-code-blocks-appropriate.md +given file b.yaml +given an installed subplot +when I try to run subplot docgen named-code-blocks-appropriate.subplot -o foo.html +then command fails +and stderr contains "#example-1 at named-code-blocks-appropriate.md:7:1" +and stderr doesn't contain "example-2" +and stderr doesn't contain "example-3" +~~~ + +~~~{#named-code-blocks-appropriate.subplot .file .yaml .numberLines} +title: Named code blocks carry appropriate classes step +markdowns: + - named-code-blocks-appropriate.md +bindings: + - b.yaml +~~~ + +~~~~~~{#named-code-blocks-appropriate.md .file .markdown .numberLines} +# This is a title + +~~~scenario +given precondition +~~~ + +~~~{#example-1 .numberLines} +This example is bad +~~~ + +~~~{#example-2 .file .numberLines} +This example is OK because of .file +~~~ + +~~~{#example-3 .example .numberLines} +This example is OK because of .example +~~~ + +~~~~~~ + ## No scenarios means codegen fails If you attempt to `subplot codegen` on a document which contains no scenarios, the tool will fail to execute with a reasonable error message. ~~~scenario +given file noscenarios.subplot given file noscenarios.md and an installed subplot -when I try to run subplot codegen noscenarios.md -o test.py +when I try to run subplot codegen noscenarios.subplot -o test.py then command fails and stderr contains "no scenarios were found" ~~~ -~~~{#noscenarios.md .file .markdown .numberLines} ---- +~~~{#noscenarios.subplot .file .yaml .numberLines} title: No scenarios in here +markdowns: [noscenarios.md] impls: { python: [] } -... +~~~ +~~~{#noscenarios.md .file .markdown .numberLines} # This is a title But there are no scenarios in this file, and thus nothing can be generated in a test suite. ~~~ +## No template means you can docgen but not codegen + +When running `docgen` you do not **need** a template to have been defined in the +subplot input document. If you have template-specific bindings then you **should** +provide one, but if not, then it is unnecessary. This means you can use `docgen` +to build documents before you have any inkling of the implementation language +necessary to validate the scenarios. + +~~~scenario +given file notemplate.subplot +given file notemplate.md +and an installed subplot +when I run subplot docgen notemplate.subplot -o notemplate.html +then file notemplate.html exists +when I try to run subplot codegen notemplate.subplot -o test.py +then command fails +and stderr contains "document has no template" +~~~ + +~~~{#notemplate.subplot .file .yaml .numberLines} +title: No templates in here +markdowns: [notemplate.md] +impls: { } +~~~ + +~~~{#notemplate.md .file .markdown .numberLines} +# This is a title + +```scenario +then failure ensues +``` + +~~~ + ## Keywords Subplot supports the keywords **given**, **when**, and **then**, and @@ -1146,13 +1308,12 @@ combinations. ### All the keywords ~~~scenario +given file allkeywords.subplot given file allkeywords.md and file b.yaml and file f.py and an installed subplot -when I run subplot docgen allkeywords.md -o foo.pdf -then file foo.pdf exists -when I run subplot codegen --run allkeywords.md -o test.py +when I run subplot codegen --run allkeywords.subplot -o test.py then scenario "All keywords" was run and step "given precondition foo" was run and step "when I do bar" was run @@ -1160,14 +1321,16 @@ and step "then bar was done" was run and command is successful ~~~ -~~~{#allkeywords.md .file .markdown .numberLines} ---- +~~~{#allkeywords.subplot .file .yaml .numberLines} title: All the keywords scenario +markdowns: +- allkeywords.md bindings: [b.yaml] impls: python: [f.py] -... +~~~ +~~~{#allkeywords.md .file .markdown .numberLines} # All keywords This uses all the keywords. @@ -1181,6 +1344,7 @@ but foobar was done ``` ~~~ +<!-- disabled until Lars fixes typesetting of scenarios ### Keyword aliases in output We support **and** and **but** in input lines, and we always render @@ -1188,11 +1352,12 @@ scenarios in output so they are used when they are allowed. This scenario verifies that this happens. ~~~scenario +given file aliases.subplot given file aliases.md given file b.yaml given file f.py given an installed subplot -when I run subplot docgen --merciful aliases.md -o aliases.html +when I run subplot docgen --merciful aliases.subplot -o aliases.html then command is successful then file aliases.html matches regex /given<[^>]*> precondition foo/ then file aliases.html matches regex /when<[^>]*> I do bar/ @@ -1201,13 +1366,16 @@ then file aliases.html matches regex /then<[^>]*> bar was done/ then file aliases.html matches regex /and<[^>]*> foobar was done/ ~~~ -~~~{#aliases.md .file .markdown .numberLines} ---- +~~~{#aliases.subplot .file .yaml .numberLines} title: Keyword aliases +markdowns: +- aliases.md bindings: [b.yaml] -functions: [f.py] -... +impls: + python: [f.py] +~~~ +~~~{#aliases.md .file .markdown .numberLines} # Aliases ```scenario @@ -1218,6 +1386,7 @@ then bar was done then foobar was done ``` ~~~ +--> ### Misuse of continuation keywords @@ -1227,24 +1396,25 @@ subplot will be unable to determine what kind of keyword they are meant to be continuing. ~~~scenario +given file continuationmisuse.subplot given file continuationmisuse.md and file b.yaml and file f.py and an installed subplot -when I run subplot docgen continuationmisuse.md -o foo.pdf -then file foo.pdf exists -when I try to run subplot codegen --run continuationmisuse.md -o test.py +when I try to run subplot codegen --run continuationmisuse.subplot -o test.py then command fails ~~~ -~~~{#continuationmisuse.md .file .markdown .numberLines} ---- +~~~{#continuationmisuse.subplot .file .yaml .numberLines} title: Continuation keyword misuse +markdowns: +- continuationmisuse.subplot bindings: [b.yaml] impls: python: [f.py] -... +~~~ +~~~{#continuationmisuse.md .file .markdown .numberLines} # Continuation keyword misuse This scenario should fail to parse because we misuse a @@ -1264,36 +1434,113 @@ It is OK to use markup in document titles, in the YAML metadata section. This scenario verifies that all markup works. ~~~scenario +given file title-markup.subplot given file title-markup.md given an installed subplot -when I run subplot docgen title-markup.md -o foo.pdf -then file foo.pdf exists +when I run subplot docgen title-markup.subplot -o foo.html +then file foo.html exists ~~~ -~~~~{#title-markup.md .file .markdown .numberLines} ---- +~~~~{#title-markup.subplot .file .yaml .numberLines} title: This _uses_ ~~all~~ **most** inline `markup` subtitle: H~2~O is not 2^10^ +markdowns: [title-markup.md] impls: { python: [] } -... +~~~~ +~~~~{#title-markup.md .file .markdown .numberLines} # Introduction ~~~~ +## Scenario titles + +A scenario gets its title from the lowest level of section heading +that applies to it. The heading can use markup. + +~~~scenario +given file scenario-titles.subplot +given file scenario-titles.md +given file b.yaml +given file f.py +given an installed subplot +when I run subplot metadata scenario-titles.subplot +then stdout contains "My fun scenario title" +~~~ + +~~~~{#scenario-titles.subplot .file .yaml .numberLines} +title: Test scenario +markdowns: +- scenario-titles.md +bindings: [b.yaml] +impls: + python: [f.py] +~~~~ + +~~~~{#scenario-titles.md .file .markdown .numberLines} +# My **fun** _scenario_ `title` + +```scenario +given precondition foo +when I do bar +then bar was done +``` +~~~~ + +## Duplicate scenario titles + +_Requirement: Subplot treats it as an error if two scenarios have the +same title._ + +Justification: the title is how a scenario is identified, and the user +needs to be able to do so unambiguously. + +~~~scenario +given file duplicate-scenario-titles.subplot +given file duplicate-scenario-titles.md +given file b.yaml +given file f.py +given an installed subplot +when I try to run subplot metadata duplicate-scenario-titles.subplot +then command fails +then stderr contains "duplicate" +~~~ + +~~~~{#duplicate-scenario-titles.subplot .file .yaml .numberLines} +title: Test scenario +markdowns: +- duplicate-scenario-titles.md +bindings: [b.yaml] +impls: + python: [f.py] +~~~~ + +~~~~{#duplicate-scenario-titles.md .file .markdown .numberLines} +# My sceanrio + +```scenario +when I do bar +``` + +# My sceanrio + +```scenario +when I do bar +``` +~~~~ + ## Empty lines in scenarios -This scenario verifies that empty lines in scenarios are ignored. +This scenario verifies that empty lines in scenarios are OK. ~~~scenario +given file emptylines.subplot given file emptylines.md and file b.yaml and file f.py and an installed subplot -when I run subplot docgen emptylines.md -o emptylines.pdf -then file emptylines.pdf exists -when I run subplot docgen emptylines.md -o emptylines.html +when I run subplot docgen emptylines.subplot -o emptylines.html then file emptylines.html exists -when I run subplot codegen --run emptylines.md -o test.py +when I run subplot codegen --run emptylines.subplot -o test.py then scenario "Simple" was run and step "given precondition foo" was run and step "when I do bar" was run @@ -1301,14 +1548,16 @@ and step "then bar was done" was run and command is successful ~~~ -~~~~{#emptylines.md .file .markdown .numberLines} ---- +~~~~{#emptylines.subplot .file .yaml .numberLines} title: Test scenario +markdowns: +- emptylines.md bindings: [b.yaml] impls: python: [f.py] -... +~~~~ +~~~~{#emptylines.md .file .markdown .numberLines} # Simple This is the simplest possible test scenario @@ -1397,11 +1646,12 @@ failure_cleanup() { ### Cleanup functions gets called on success (Python) ~~~scenario +given file cleanup-success-python.subplot given file cleanup-success-python.md and file cleanup.yaml and file cleanup.py and an installed subplot -when I run subplot codegen --run cleanup-success-python.md -o test.py +when I run subplot codegen --run cleanup-success-python.subplot -o test.py then scenario "Cleanup" was run and step "given foo" was run, and then step "given bar" and cleanup for "given bar" was run, and then for "given foo" @@ -1409,14 +1659,17 @@ and command is successful ~~~ -~~~~~{#cleanup-success-python.md .file .markdown .numberLines} ---- +~~~~~{#cleanup-success-python.subplot .file .yaml .numberLines} title: Cleanup +markdowns: +- cleanup-success-python.md bindings: [cleanup.yaml] impls: python: [cleanup.py] -... +~~~~~ + +~~~~~{#cleanup-success-python.md .file .markdown .numberLines} # Cleanup ~~~scenario @@ -1429,11 +1682,12 @@ given bar ### Cleanup functions get called on failure (Python) ~~~scenario +given file cleanup-fail-python.subplot given file cleanup-fail-python.md and file cleanup.yaml and file cleanup.py and an installed subplot -when I try to run subplot codegen --run cleanup-fail-python.md -o test.py +when I try to run subplot codegen --run cleanup-fail-python.subplot -o test.py then scenario "Cleanup" was run and step "given foo" was run, and then step "given bar" and cleanup for "given bar" was run, and then for "given foo" @@ -1441,14 +1695,16 @@ and cleanup for "given failure" was not run and command fails ~~~ -~~~~~{#cleanup-fail-python.md .file .markdown .numberLines} ---- +~~~~~{#cleanup-fail-python.subplot .file .yaml .numberLines} title: Cleanup +markdowns: +- cleanup-fail-python.md bindings: [cleanup.yaml] impls: python: [cleanup.py] -... +~~~~~ +~~~~~{#cleanup-fail-python.md .file .markdown .numberLines} # Cleanup ~~~scenario @@ -1462,17 +1718,27 @@ given failure ### Cleanup functions gets called on success (Bash) ~~~scenario +given file cleanup-success-bash.subplot given file cleanup-success-bash.md and file cleanup.yaml and file cleanup.sh and an installed subplot -when I run subplot codegen --run cleanup-success-bash.md -o test.sh +when I run subplot codegen --run cleanup-success-bash.subplot -o test.sh then scenario "Cleanup" was run and step "given foo" was run, and then step "given bar" and cleanup for "given bar" was run, and then for "given foo" and command is successful ~~~ +~~~~~{#cleanup-success-bash.subplot .file .yaml .numberLines} +title: Cleanup +markdowns: +- cleanup-success-bash.md +bindings: [cleanup.yaml] +impls: + bash: [cleanup.sh] +~~~~~ + ~~~~~{#cleanup-success-bash.md .file .markdown .numberLines} --- title: Cleanup @@ -1496,11 +1762,12 @@ If a step fails, all the cleanups for the preceding steps are still called, in reverse order. ~~~scenario +given file cleanup-fail-bash.subplot given file cleanup-fail-bash.md and file cleanup.yaml and file cleanup.sh and an installed subplot -when I try to run subplot codegen --run cleanup-fail-bash.md -o test.sh +when I try to run subplot codegen --run cleanup-fail-bash.subplot -o test.sh then scenario "Cleanup" was run and step "given foo" was run, and then step "given bar" and cleanup for "given bar" was run, and then for "given foo" @@ -1508,14 +1775,16 @@ and cleanup for "given failure" was not run and command fails ~~~ -~~~~~{#cleanup-fail-bash.md .file .markdown .numberLines} ---- +~~~~~{#cleanup-fail-bash.subplot .file .yaml .numberLines} title: Cleanup +markdowns: +- cleanup-fail-bash.md bindings: [cleanup.yaml] impls: bash: [cleanup.sh] -... +~~~~~ +~~~~~{#cleanup-fail-bash.md .file .markdown .numberLines} # Cleanup ~~~scenario @@ -1538,24 +1807,26 @@ the `TMPDIR` environment variable to point at the data directory. This scenario verifies that it happens. ~~~scenario +given file tmpdir.subplot given file tmpdir.md and file tmpdir.yaml and file tmpdir.py and an installed subplot -when I run subplot codegen --run tmpdir.md -o test.py +when I run subplot codegen --run tmpdir.subplot -o test.py then command is successful and scenario "TMPDIR" was run and step "then TMPDIR is set" was run ~~~ -~~~~{#tmpdir.md .file .markdown .numberLines} ---- +~~~~{#tmpdir.subplot .file .yaml .numberLines} title: TMPDIR +markdowns: [tmpdir.md] bindings: [tmpdir.yaml] impls: python: [tmpdir.py] -... +~~~~ +~~~~{#tmpdir.md .file .markdown .numberLines} # TMPDIR ~~~scenario @@ -1589,25 +1860,28 @@ can be done using regular expressions or "simple patterns". ### Capture using simple patterns ~~~scenario +given file simplepattern.subplot given file simplepattern.md and file simplepattern.yaml and file capture.py and an installed subplot -when I run subplot codegen --run simplepattern.md -o test.py +when I run subplot codegen --run simplepattern.subplot -o test.py then scenario "Simple pattern" was run and step "given I am Tomjon" was run and stdout contains "function got argument name as Tomjon" and command is successful ~~~ -~~~~{#simplepattern.md .file .markdown .numberLines} ---- +~~~~{#simplepattern.subplot .file .yaml .numberLines} title: Simple pattern capture +markdowns: +- simplepattern.md bindings: [simplepattern.yaml] impls: python: [capture.py] -... +~~~~ +~~~~{#simplepattern.md .file .markdown .numberLines} # Simple pattern ~~~scenario @@ -1635,23 +1909,26 @@ expression meta characters unless the rule is explicitly marked as not being a regular expression pattern. ~~~scenario +given file confusedpattern.subplot given file confusedpattern.md and file confusedpattern.yaml and file capture.py and an installed subplot -when I try to run subplot codegen --run confusedpattern.md -o test.py +when I try to run subplot codegen --run confusedpattern.subplot -o test.py then command fails and stderr contains "simple pattern contains regex" ~~~ -~~~~{#confusedpattern.md .file .markdown .numberLines} ---- +~~~~{#confusedpattern.subplot .file .yaml .numberLines} title: Simple pattern capture +markdowns: +- confusedpattern.md bindings: [confusedpattern.yaml] impls: python: [capture.py] -... +~~~~ +~~~~{#confusedpattern.md .file .markdown .numberLines} # Simple pattern ~~~scenario @@ -1669,22 +1946,25 @@ given I* am Tomjon ### Simple patterns with regex metacharacters: allowed case ~~~scenario +given file confusedbutok.subplot given file confusedbutok.md and file confusedbutok.yaml and file capture.py and an installed subplot -when I run subplot codegen --run confusedbutok.md -o test.py +when I run subplot codegen --run confusedbutok.subplot -o test.py then command is successful ~~~ -~~~~{#confusedbutok.md .file .markdown .numberLines} ---- +~~~~{#confusedbutok.subplot .file .yaml .numberLines} title: Simple pattern capture +markdowns: +- confusedbutok.md bindings: [confusedbutok.yaml] impls: python: [capture.py] -... +~~~~ +~~~~{#confusedbutok.md .file .markdown .numberLines} # Simple pattern ~~~scenario @@ -1703,25 +1983,28 @@ given I* am Tomjon ### Capture using regular expressions ~~~scenario +given file regex.subplot given file regex.md and file regex.yaml and file capture.py and an installed subplot -when I run subplot codegen --run regex.md -o test.py +when I run subplot codegen --run regex.subplot -o test.py then scenario "Regex" was run and step "given I am Tomjon" was run and stdout contains "function got argument name as Tomjon" and command is successful ~~~ -~~~~{#regex.md .file .markdown .numberLines} ---- +~~~~{#regex.subplot .file .yaml .numberLines} title: Regex capture +markdowns: +- regex.md bindings: [regex.yaml] impls: python: [capture.py] -... +~~~~ +~~~~{#regex.md .file .markdown .numberLines} # Regex ~~~scenario @@ -1753,31 +2036,32 @@ expansions don't accidentally refer to values meant for another purpose. ~~~scenario +given file values.subplot given file values.md and file values.yaml and file values.py and an installed subplot -when I run subplot codegen values.md -o test.py +when I run subplot codegen values.subplot -o test.py when I run python3 test.py then command is successful ~~~ -~~~~~~{#values.md .file .markdown .numberLines} ---- +~~~~~~{#values.subplot .file .yaml .numberLines} title: Values +markdowns: +- values.md bindings: [values.yaml] impls: python: [values.py] -... - +~~~~~~ +~~~~~~{#values.md .file .markdown .numberLines} # Values ~~~scenario when I remember foo as bar then expanded "${foo}" is bar ~~~ - ~~~~~~ ~~~{#values.yaml .file .yaml .numberLines} @@ -1823,11 +2107,12 @@ There is currently no equivalent functionality for the generated Bash test program. Patches for that are welcome. ~~~scenario +given file env.subplot given file env.md and file env.yaml and file env.py and an installed subplot -when I run subplot codegen env.md -o test.py +when I run subplot codegen env.subplot -o test.py when I try to run python3 test.py then command fails when I try to run python3 test.py --env FOO=foo @@ -1836,14 +2121,16 @@ when I try to run python3 test.py --env FOO=bar then command is successful ~~~ -~~~~~~{#env.md .file .markdown .numberLines} ---- +~~~~~~{#env.subplot .file .yaml .numberLines} title: Environment variables +markdowns: +- env.md bindings: [env.yaml] impls: python: [env.py] -... +~~~~~~ +~~~~~~{#env.md .file .markdown .numberLines} # Test ~~~scenario then environment variable FOO is set to "bar" @@ -1866,77 +2153,6 @@ def is_set_to(ctx, name=None, value=None): -## Avoid changing typesetting output file needlessly - -### Avoid typesetting if output is newer than source files - -This scenario make sure that if docgen generates the bitwise identical -output to the existing output file, it doesn't actually write it to -the output file, including its timestamp. This avoids triggering -programs that monitor the output file for changes. - -~~~scenario -given file simple.md -and file b.yaml -and file f.py -and an installed subplot -when I run subplot docgen simple.md -o simple.pdf -then file simple.pdf exists -when I remember metadata for file simple.pdf -and I wait until 1 second has passed -and I run subplot docgen simple.md -o simple.pdf -then file simple.pdf has same metadata as before -and only files simple.md, b.yaml, f.py, simple.pdf exist -~~~ - -### Do typeset if output is older than markdown - -~~~scenario -given file simple.md -and file b.yaml -and file f.py -and an installed subplot -when I run subplot docgen simple.md -o simple.pdf -then file simple.pdf exists -when I remember metadata for file simple.pdf -and I wait until 1 second has passed -and I touch file simple.md -and I run subplot docgen simple.md -o simple.pdf -then file simple.pdf has changed from before -~~~ - -### Do typeset if output is older than functions - -~~~scenario -given file simple.md -and file b.yaml -and file f.py -and an installed subplot -when I run subplot docgen simple.md -o simple.pdf -then file simple.pdf exists -when I remember metadata for file simple.pdf -and I wait until 1 second has passed -and I touch file f.py -and I run subplot docgen simple.md -o simple.pdf -then file simple.pdf has changed from before -~~~ - -### Do typeset if output is older than bindings - -~~~scenario -given file simple.md -and file b.yaml -and file f.py -and an installed subplot -when I run subplot docgen simple.md -o simple.pdf -then file simple.pdf exists -when I remember metadata for file simple.pdf -and I wait until 1 second has passed -and I touch file b.yaml -and I run subplot docgen simple.md -o simple.pdf -then file simple.pdf has changed from before -~~~ - ## Document structure Subplot uses chapters and sections to keep together scenario snippets @@ -1949,24 +2165,26 @@ higher level starts a new scenario. ### Lowest level heading is name of scenario ~~~scenario +given file scenarioislowest.subplot given file scenarioislowest.md and file b.yaml and file f.py and an installed subplot -when I run subplot codegen --run scenarioislowest.md -o test.py +when I run subplot codegen --run scenarioislowest.subplot -o test.py then scenario "heading 1.1.1" was run and command is successful ~~~ -~~~~{#scenarioislowest.md .file .markdown .numberLines} - ---- +~~~~{#scenarioislowest.subplot .file .yaml .numberLines} title: Test scenario +markdowns: +- scenarioislowest.md bindings: [b.yaml] impls: python: [f.py] -... +~~~~ +~~~~{#scenarioislowest.md .file .markdown .numberLines} # heading 1 ## heading 1.1 ### heading 1.1.1 @@ -1979,24 +2197,26 @@ given precondition foo ### Subheadings don't start new scenario ~~~scenario +given file subisnotnewscenario.subplot given file subisnotnewscenario.md and file b.yaml and file f.py and an installed subplot -when I run subplot codegen --run subisnotnewscenario.md -o test.py +when I run subplot codegen --run subisnotnewscenario.subplot -o test.py then scenario "heading 1.1a" was run and command is successful ~~~ -~~~~{#subisnotnewscenario.md .file .markdown .numberLines} - ---- +~~~~{#subisnotnewscenario.subplot .file .yaml .numberLines} title: Test scenario +markdowns: +- subisnotnewscenario.md bindings: [b.yaml] impls: python: [f.py] -... +~~~~ +~~~~{#subisnotnewscenario.md .file .markdown .numberLines} # heading 1 ## heading 1.1a @@ -2006,31 +2226,33 @@ given precondition foo ### heading 1.1.1 ### heading 1.1.2 - ~~~~ ### Next heading at same level starts new scenario ~~~scenario +given file samelevelisnewscenario.subplot given file samelevelisnewscenario.md and file b.yaml and file f.py and an installed subplot -when I run subplot codegen --run samelevelisnewscenario.md -o test.py +when I run subplot codegen --run samelevelisnewscenario.subplot -o test.py then scenario "heading 1.1.1" was run and scenario "heading 1.1.2" was run and command is successful ~~~ -~~~~{#samelevelisnewscenario.md .file .markdown .numberLines} - ---- +~~~~{#samelevelisnewscenario.subplot .file .yaml .numberLines} title: Test scenario +markdowns: +- samelevelisnewscenario.md bindings: [b.yaml] impls: python: [f.py] -... +~~~~ + +~~~~{#samelevelisnewscenario.md .file .markdown .numberLines} # heading 1 ## heading 1.1 ### heading 1.1.1 @@ -2049,25 +2271,27 @@ given precondition foo ### Next heading at higher level starts new scenario ~~~scenario +given file higherisnewscenario.subplot given file higherisnewscenario.md and file b.yaml and file f.py and an installed subplot -when I run subplot codegen --run higherisnewscenario.md -o test.py +when I run subplot codegen --run higherisnewscenario.subplot -o test.py then scenario "heading 1.1.1" was run and scenario "heading 1.2" was run and command is successful ~~~ -~~~~{#higherisnewscenario.md .file .markdown .numberLines} - ---- +~~~~{#higherisnewscenario.subplot .file .yaml .numberLines} title: Test scenario +markdowns: +- higherisnewscenario.md bindings: [b.yaml] impls: python: [f.py] -... +~~~~ +~~~~{#higherisnewscenario.md .file .markdown .numberLines} # heading 1 ## heading 1.1 ### heading 1.1.1 @@ -2087,30 +2311,29 @@ given precondition foo The document and code generators require a document title, because it's a common user error to not have one, and Subplot should help make -good documents. The Pandoc filter, however, mustn't require a document -title, because it's used for things like formatting websites using -ikiwiki, and ikiwiki has a different way of specifying page titles. +good documents. #### Document generator gives an error if input document lacks title ~~~scenario +given file notitle.subplot given file notitle.md and an installed subplot -when I try to run subplot docgen notitle.md -o foo.md +when I try to run subplot docgen notitle.subplot -o foo.md then command fails ~~~ -~~~{#notitle.md .file .markdown .numberLines} ---- +~~~{#notitle.subplot .file .yaml .numberLines} +markdowns: +- notitle.md bindings: [b.yaml] functions: [f.py] -... - +~~~ +~~~{#notitle.md .file .markdown .numberLines} # Introduction -This is a very simple Markdown file without a YAML metadata block, -and thus also no document title. +This is a very simple Markdown file without a document title. ```scenario given precondition foo @@ -2121,9 +2344,10 @@ then bar was done #### Code generator gives an error if input document lacks title ~~~scenario +given file notitle.subplot given file notitle.md and an installed subplot -when I try to run subplot codegen --run notitle.md -o test.py +when I try to run subplot codegen --run notitle.subplot -o test.py then command fails ~~~ @@ -2134,25 +2358,27 @@ Markdown allows using any inline markup in document titles and chapter and section headings. Verify that Subplot accepts them. ~~~scenario +given file fancytitle.subplot given file fancytitle.md and file b.yaml and file f.py and an installed subplot -when I try to run subplot docgen fancytitle.md -o foo.md +when I try to run subplot docgen fancytitle.subplot -o foo.md then command is successful -when I try to run subplot codegen fancytitle.md -o foo.md +when I try to run subplot codegen fancytitle.subplot -o foo.md then command is successful ~~~ -~~~~~~{#fancytitle.md .file .markdown .numberLines} ---- +~~~~~~{#fancytitle.subplot .file .yaml .numberLines} title: Plain *emph* **strong** ~~strikeout~~ superscript^10^ subscript~10~ +markdowns: +- fancytitle.md bindings: [b.yaml] impls: python: [f.py] -... - +~~~~~~ +~~~~~~{#fancytitle.md .file .markdown .numberLines} # `code` [smallcaps]{.smallcaps} $$2^10$$ ## "double quoted" @@ -2168,11 +2394,11 @@ This is a very simple Markdown file that uses every kind of inline markup in the title and chapter heading. To satisfy codegen, we *MUST* have a scenario here + ~~~~scenario when I do bar then bar was done ~~~~ - ~~~~~~ @@ -2188,25 +2414,28 @@ This verifies that the generated Python test program can run only chosen scenarios. ~~~scenario +given file twoscenarios-python.subplot given file twoscenarios-python.md and file b.yaml and file f.py and an installed subplot -when I run subplot codegen twoscenarios-python.md -o test.py +when I run subplot codegen twoscenarios-python.subplot -o test.py and I run python3 test.py on then scenario "One" was run and scenario "Two" was not run and command is successful ~~~ -~~~{#twoscenarios-python.md .file .markdown .numberLines} ---- +~~~{#twoscenarios-python.subplot .file .yaml .numberLines} title: Test scenario +markdowns: +- twoscenarios-python.md bindings: [b.yaml] impls: python: [f.py] -... +~~~ +~~~{#twoscenarios-python.md .file .markdown .numberLines} # One ```scenario @@ -2230,11 +2459,12 @@ This verifies that the generated Bash test program can run only chosen scenarios. ~~~scenario +given file twoscenarios-bash.subplot given file twoscenarios-bash.md and file b.yaml and file f.sh and an installed subplot -when I run subplot codegen twoscenarios-bash.md -o test.sh +when I run subplot codegen twoscenarios-bash.subplot -o test.sh and I run bash test.sh on then scenario "One" was run and scenario "Two" was not run @@ -2242,14 +2472,16 @@ and command is successful ~~~ -~~~{#twoscenarios-bash.md .file .markdown .numberLines} ---- +~~~{#twoscenarios-bash.subplot .file .yaml .numberLines} title: Test scenario +markdowns: +- twoscenarios-bash.md bindings: [b.yaml] impls: bash: [f.sh] -... +~~~ +~~~{#twoscenarios-bash.md .file .markdown .numberLines} # One ```scenario @@ -2322,9 +2554,11 @@ This scenario tests that the `date` field in metadata is used if specified. ~~~scenario +given file metadate.subplot given file metadate.md and an installed subplot -when I run subplot docgen metadate.md -o metadate.html +when I run subplot docgen metadate.subplot -o metadate.html +when I run cat metadate.html then file metadate.html exists and file metadate.html contains "<title>The Fabulous Title</title>" and file metadate.html contains "Alfred Pennyworth" @@ -2332,14 +2566,17 @@ and file metadate.html contains "Geoffrey Butler" and file metadate.html contains "WIP" ~~~ -~~~{#metadate.md .file .markdown .numberLines} ---- +~~~{#metadate.subplot .file .yaml .numberLines} title: The Fabulous Title -author: +authors: - Alfred Pennyworth - Geoffrey Butler date: WIP -... +markdowns: +- metadate.md +~~~ + +~~~{#metadate.md .file .markdown .numberLines} # Introduction This is a test document. That's all. ~~~ @@ -2349,9 +2586,10 @@ This is a test document. That's all. This scenario tests that the `--date` command line option is used. ~~~scenario +given file dateless.subplot given file dateless.md and an installed subplot -when I run subplot docgen dateless.md -o dateoption.html --date=FANCYDATE +when I run subplot docgen dateless.subplot -o dateoption.html --date=FANCYDATE then file dateoption.html exists and file dateoption.html contains "<title>The Fabulous Title</title>" and file dateoption.html contains "Alfred Pennyworth" @@ -2359,13 +2597,16 @@ and file dateoption.html contains "Geoffrey Butler" and file dateoption.html contains "FANCYDATE" ~~~ -~~~{#dateless.md .file .markdown .numberLines} ---- +~~~{#dateless.subplot .file .yaml .numberLines} title: The Fabulous Title -author: +authors: - Alfred Pennyworth - Geoffrey Butler -... +markdowns: +- dateless.md +~~~ + +~~~{#dateless.md .file .markdown .numberLines} # Introduction This is a test document. It has no date metadata. ~~~ @@ -2378,10 +2619,11 @@ modification time of the input file, and shall have the date in ISO 8601 format, with time to the minute. ~~~scenario +given file dateless.subplot given file dateless.md and file dateless.md has modification time 2020-02-26 07:53:17 and an installed subplot -when I run subplot docgen dateless.md -o mtime.html +when I run subplot docgen dateless.subplot -o mtime.html then file mtime.html exists and file mtime.html contains "<title>The Fabulous Title</title>" and file mtime.html contains "Alfred Pennyworth" @@ -2395,18 +2637,23 @@ If a bindings file is missing, the error message should name the missing file. ~~~scenario +given file missing-binding.subplot given file missing-binding.md and an installed subplot -when I try to run subplot docgen missing-binding.md -o foo.html +when I try to run subplot docgen missing-binding.subplot -o foo.html then command fails -and stderr contains ": missing-binding.yaml:" +and stderr contains "could not be found" +and stderr contains "missing-binding.yaml" ~~~ -~~~{#missing-binding.md .file .markdown .numberLines} ---- +~~~{#missing-binding.subplot .file .yaml .numberLines} title: Missing binding +markdowns: +- missing-binding.md bindings: [missing-binding.yaml] -... +~~~ + +~~~{#missing-binding.md .file .markdown .numberLines} ~~~ ### Missing functions file @@ -2415,23 +2662,30 @@ If a functions file is missing, the error message should name the missing file. ~~~scenario +given file missing-functions.subplot given file missing-functions.md and file b.yaml and an installed subplot -when I try to run subplot codegen --run missing-functions.md -o foo.py +when I try to run subplot codegen --run missing-functions.subplot -o foo.py then command fails -and stderr contains ": missing-functions.py:" +and stderr contains "could not be found" +and stderr contains "missing-functions.py" ~~~ -~~~{#missing-functions.md .file .markdown .numberLines} +~~~{#missing-functions.subplot .file .yaml .numberLines} --- title: Missing functions +markdowns: +- missing-functions.md bindings: [b.yaml] impls: python: [missing-functions.py] ... ~~~ +~~~{#missing-functions.md .file .markdown .numberLines} +~~~ + ### Extracting metadata from a document The **subplot metadata** program extracts metadata from a document. It is @@ -2452,35 +2706,33 @@ This scenario check subplot metadata works. Note that it requires the bindings or functions files. ~~~scenario +given file images.subplot given file images.md and file b.yaml and file other.yaml and file f.py and file other.py -and file foo.bib -and file bar.bib and file expected.json and an installed subplot -when I run subplot metadata images.md +when I run subplot metadata images.subplot then stdout contains "source: images.md" and stdout contains "source: b.yaml" and stdout contains "source: other.yaml" and stdout contains "source: f.py" and stdout contains "source: other.py" -and stdout contains "source: foo.bib" -and stdout contains "source: bar.bib" and stdout contains "source: image.gif" and stdout contains "bindings: b.yaml" and stdout contains "bindings: other.yaml" and stdout contains "functions[python]: f.py" -when I run subplot metadata images.md -o json +when I run subplot metadata images.subplot -o json then JSON output matches expected.json ~~~ -~~~{#images.md .file .markdown .numberLines} ---- +~~~{#images.subplot .file .yaml .numberLines} title: Document refers to external images +markdowns: +- images.md bindings: - b.yaml - other.yaml @@ -2488,9 +2740,9 @@ impls: python: - f.py - other.py -bibliography: [foo.bib, bar.bib] -... +~~~ +~~~{#images.md .file .markdown .numberLines}  ~~~ @@ -2501,36 +2753,15 @@ bibliography: [foo.bib, bar.bib] ~~~{#other.py .file .python .numberLines} ~~~ -~~~{#foo.bib .file .numberLines} -@book{foo2020, - author = "James Random", - title = "The Foo book", - publisher = "The Internet", - year = 2020, - address = "World Wide Web", -} -~~~ - -~~~{#bar.bib .file .numberLines} -@book{foo2020, - author = "James Random", - title = "The Bar book", - publisher = "The Internet", - year = 2020, - address = "World Wide Web", -} -~~~ - ~~~{#expected.json .file .json} { "title": "Document refers to external images", "sources": [ "b.yaml", - "bar.bib", "f.py", - "foo.bib", "image.gif", "images.md", + "images.subplot", "other.py", "other.yaml" ], @@ -2544,27 +2775,60 @@ bibliography: [foo.bib, bar.bib] "other.py" ] }, - "bibliographies": [ - "bar.bib", - "foo.bib" - ], "files": [], "scenarios": [] } ~~~ +### Multiple markdown files + +This scenario tests that the `markdowns` field in metadata can specify +more than one markdown file. + +~~~scenario +given file multimd.subplot +given file md1.md +given file md2.md +given an installed subplot +when I run subplot docgen multimd.subplot -o multimd.html +when I run cat multimd.html +then file multimd.html exists +and file multimd.html contains "<title>The Fabulous Title</title>" +and file multimd.html contains "First markdown file." +and file multimd.html contains "Second markdown file." +~~~ + +~~~{#multimd.subplot .file .yaml .numberLines} +title: The Fabulous Title +authors: +- Alfred Pennyworth +- Geoffrey Butler +date: WIP +markdowns: +- md1.md +- md2.md +~~~ + +~~~{#md1.md .file .markdown .numberLines} +First markdown file. +~~~ + +~~~{#md2.md .file .markdown .numberLines} +Second markdown file. +~~~ + + ## Embedded files Subplot allows data files to be embedded in the input document. This is handy for small test files and the like. -Handling of a newline character on the last line is tricky. Pandoc -doesn't include a newline on the last line. Sometimes one is -needed—but sometimes it's not wanted. A newline can be added by -having an empty line at the end, but that is subtle and easy to miss. -Subplot helps the situation by allowing a `add-newline=` class to be added -to the code blocks, with one of three allowed cases: +Handling of a newline character on the last line is tricky. The block +ends in a newline on the last line. Sometimes one is needed—but +sometimes it's not wanted. Subplot helps the situation by allowing a +`add-newline=` class to be added to the code blocks, with one of three +allowed cases: * no `add-newline` class—default handling: same as `add-newline=auto` * `add-newline=auto`—add a newline, if one isn't there @@ -2580,18 +2844,21 @@ This scenario checks that an embedded file can be extracted, and used in a subplot. ~~~scenario +given file embedded.subplot given file embedded.md and an installed subplot -when I run subplot docgen --merciful embedded.md -o foo.html +when I run subplot docgen --merciful embedded.subplot -o foo.html then file foo.html exists and file foo.html matches regex /embedded\.txt/ ~~~ -~~~~~~~{#embedded.md .file .markdown .numberLines} ---- +~~~~~~~{#embedded.subplot .file .yaml .numberLines} title: One embedded file -... +markdowns: +- embedded.md +~~~~~~~ +~~~~~~~{#embedded.md .file .markdown .numberLines} ~~~{#embedded.txt .file} This is the embedded file. ~~~ @@ -2761,19 +3028,22 @@ This scenario checks that we get warnings, when using a subplot with embedded files that aren't used. ~~~scenario +given file unusedfile.subplot given file unusedfile.md and an installed subplot -when I try to run subplot docgen --merciful unusedfile.md -o unusedfile.html +when I try to run subplot docgen --merciful unusedfile.subplot -o unusedfile.html then command is successful and file unusedfile.html exists and stderr contains "thisisnotused.txt" ~~~ -~~~~{#unusedfile.md .file .markdown .numberLines} ---- +~~~~{#unusedfile.subplot .file .yaml .numberLines} title: Embedded file is not used by a scenario -... +markdowns: +- unusedfile.md +~~~~ +~~~~{#unusedfile.md .file .markdown .numberLines} ```{#thisisnotused.txt .file} This is the embedded file. ``` @@ -2789,19 +3059,22 @@ subject to the same naming constraints (caseless uniqueness). ### Examples may be unused ~~~scenario +given file unusedexample.subplot given file unusedexample.md and an installed subplot -when I try to run subplot docgen --merciful unusedexample.md -o unusedexample.html +when I try to run subplot docgen --merciful unusedexample.subplot -o unusedexample.html then command is successful and file unusedexample.html exists and stderr doesn't contain "thisisnotused.txt" ~~~ -~~~{#unusedexample.md .file .markdown .numberLines} ---- +~~~{#unusedexample.subplot .file .yaml .numberLines} title: Example is not an embedded file -... +markdowns: +- unusedexample.md +~~~ +~~~{#unusedexample.md .file .markdown .numberLines} ```{#thisisnotused.txt .example} This is the embedded example. ``` @@ -2810,20 +3083,24 @@ This is the embedded example. ### Examples are not files ~~~scenario +given file examplesnotfiles.subplot given file examplesnotfiles.md and an installed subplot -when I try to run subplot codegen examplesnotfiles.md -t python -o examplesnotfiles.html +when I try to run subplot codegen examplesnotfiles.subplot -t python -o examplesnotfiles.html then command fails and file examplesnotfiles.html does not exist and stderr contains "thisisanexample.txt" ~~~ -~~~{#examplesnotfiles.md .file .markdown .numberLines} ---- +~~~{#examplesnotfiles.subplot .file .yaml .numberLines} title: Examples are not files +markdowns: +- examplesnotfiles.md impls: python: [] -... +~~~ + +~~~{#examplesnotfiles.md .file .markdown .numberLines} # Try and use an example as a file @@ -2834,7 +3111,6 @@ given file thisisanexample.txt ```{#thisisanexample.txt .example} This is an embedded example ``` - ~~~ ## Steps must match bindings @@ -2863,12 +3139,15 @@ binding. ### Steps which do not match bindings do not work -~~~~{#nobinding.md .file .markdown} ---- +~~~~{#nobinding.subplot .file .yaml} title: No bindings available +markdowns: +- nobinding.md bindings: - badbindings.yaml -... +~~~~ + +~~~~{#nobinding.md .file .markdown} # Broken scenario because step has no binding ```scenario @@ -2878,22 +3157,26 @@ then nothing works ~~~~ ```scenario +given file nobinding.subplot given file nobinding.md and file badbindings.yaml and an installed subplot -when I try to run subplot codegen --run nobinding.md -o test.py +when I try to run subplot codegen --run nobinding.subplot -o test.py then command fails ``` ### Steps which do not case-sensitively match sensitive bindings do not work -~~~~{#casemismatch.md .file .markdown} ---- +~~~~{#casemismatch.subplot .file .yaml} title: Case sensitivity mismatch +markdowns: +- casemismatch.md impls: { python: [] } bindings: - badbindings.yaml -... +~~~~ + +~~~~{#casemismatch.md .file .markdown} # Broken scenario because step has a case mismatch with sensitive binding ```scenario @@ -2902,23 +3185,27 @@ given a capitalised binding ~~~~ ```scenario +given file casemismatch.subplot given file casemismatch.md and file badbindings.yaml and an installed subplot -when I try to run subplot codegen --run casemismatch.md -o test.py +when I try to run subplot codegen --run casemismatch.subplot -o test.py then command fails ``` ### Steps which match more than one binding do not work -~~~~{#twobindings.md .file .markdown} ---- +~~~~{#twobindings.subplot .file .yaml} title: Two bindings match +markdowns: +- twobindings.md bindings: - twobindings.yaml impls: python: [a_function.py] -... +~~~~ + +~~~~{#twobindings.md .file .markdown} # Broken scenario because step has two possible bindings ```scenario @@ -2943,11 +3230,12 @@ def a_function(ctx): ~~~ ```scenario +given file twobindings.subplot given file twobindings.md and file twobindings.yaml given file a_function.py and an installed subplot -when I try to run subplot codegen --run twobindings.md -o test.py +when I try to run subplot codegen --run twobindings.subplot -o test.py then command fails then stderr contains "xyzzy" then stderr contains "plugh" @@ -2959,18 +3247,21 @@ then stderr contains "plugh" The `subplot metadata` command lists embedded files in its output. ~~~scenario +given file two-embedded.subplot given file two-embedded.md and an installed subplot -when I run subplot metadata --merciful two-embedded.md +when I run subplot metadata --merciful two-embedded.subplot then stdout contains "foo.txt" and stdout contains "bar.yaml" ~~~ -~~~~~~{#two-embedded.md .file .markdown .numberLines} ---- +~~~~~~{#two-embedded.subplot .file .yaml .numberLines} title: Two embedded files -... +markdowns: +- two-embedded.md +~~~~~~ +~~~~~~{#two-embedded.md .file .markdown .numberLines} ~~~{#foo.txt .file} ~~~ @@ -2994,7 +3285,7 @@ into SVGs such as this one. ~~~pikchr arrow right 200% "Markdown" "Source" box rad 10px "Subplot" "Document Generator" "(subplot docgen)" fit -arrow right 200% "HTML+SVG/PDF" "Output" +arrow right 200% "HTML+SVG" "Output" arrow <-> down 70% from last box.s box same "Pikchr" "Formatter" "(docs.rs/pikchr)" fit ~~~ @@ -3003,15 +3294,17 @@ The scenario checks that a diagram is generated and embedded into the HTML outpu and is not referenced as an external image. ~~~scenario +given file pikchr.subplot given file pikchr.md and an installed subplot -when I run pandoc --filter subplot-filter pikchr.md -o pikchr.html -then file pikchr.html matches regex /img src="data:image/svg\+xml;base64,/ +when I run subplot docgen pikchr.subplot -o pikchr.html +then file pikchr.html matches regex /src="data:image/svg\+xml;base64,/ ~~~ -The sample input file **pikchr.md**: +The sample input file **pikchr.md:** ~~~~~~~~{#pikchr.md .file .markdown .numberLines} +--- This is an example markdown file that embeds a simple Pikchr diagram. ~~~pikchr @@ -3024,6 +3317,12 @@ box same "Pikchr" "Formatter" "(docs.rs/pikchr)" fit ~~~~~~~~ +~~~~~~~~{#pikchr.subplot .file .yaml .numberLines} +title: Pikchr test +markdowns: +- pikchr.md +~~~~~~~~ + ### Dot [Graphviz]: http://www.graphviz.org/ @@ -3041,14 +3340,15 @@ The scenario checks that a diagram is generated and embedded into the HTML output, not referenced as an external image. ~~~scenario +given file dot.subplot given file dot.md and file b.yaml and an installed subplot -when I run pandoc --filter subplot-filter dot.md -o dot.html -then file dot.html matches regex /img src="data:image/svg\+xml;base64,/ +when I run subplot docgen dot.subplot -o dot.html +then file dot.html matches regex /src="data:image/svg\+xml;base64,/ ~~~ -The sample input file **dot.md**: +The sample input file **dot.md:** ~~~~~~~~{#dot.md .file .markdown .numberLines} This is an example Markdown file, which embeds a diagram using dot markup. @@ -3060,6 +3360,12 @@ thing -> other ~~~ ~~~~~~~~ +~~~~~~~~{#dot.subplot .file .yaml .numberLines} +title: Dot test +markdowns: +- dot.md +~~~~~~~~ + ### PlantUML @@ -3083,14 +3389,15 @@ The scenario below checks that a diagram is generated and embedded into the HTML output, not referenced as an external image. ~~~scenario +given file plantuml.subplot given file plantuml.md and file b.yaml and an installed subplot -when I run pandoc --filter subplot-filter plantuml.md -o plantuml.html -then file plantuml.html matches regex /img src="data:image/svg\+xml;base64,/ +when I run subplot docgen plantuml.subplot -o plantuml.html +then file plantuml.html matches regex /src="data:image/svg\+xml;base64,/ ~~~ -The sample input file **plantuml.md**: +The sample input file **plantuml.md:** ~~~~~~~~{#plantuml.md .file .markdown .numberLines} This is an example Markdown file, which embeds a diagram using @@ -3107,6 +3414,12 @@ Alice <-- Bob: Another authentication Response ~~~ ~~~~~~~~ +~~~~~~~~{#plantuml.subplot .file .yaml .numberLines} +title: Plantuml test +markdowns: +- plantuml.md +~~~~~~~~ + ### Roadmap @@ -3164,14 +3477,15 @@ This scenario checks that a diagram is generated and embedded into the HTML output, not referenced as an external image. ~~~scenario +given file roadmap.subplot given file roadmap.md and file b.yaml and an installed subplot -when I run pandoc --filter subplot-filter roadmap.md -o roadmap.html -then file roadmap.html matches regex /img src="data:image/svg\+xml;base64,/ +when I run subplot docgen roadmap.subplot -o roadmap.html +then file roadmap.html matches regex /src="data:image/svg\+xml;base64,/ ~~~ -The sample input file **roadmap.md**: +The sample input file **roadmap.md:** ~~~~~~~~{#roadmap.md .file .markdown .numberLines} This is an example Markdown file, which embeds a roadmap. @@ -3220,115 +3534,278 @@ blocked: ~~~ ~~~~~~~~ +~~~~~~~~{#roadmap.subplot .file .yaml .numberLines} +title: Roadmap test +markdowns: +- roadmap.md +~~~~~~~~ + ### Class name validation When Subplot loads a document it will validate that the block classes match a known set. Subplot has a built-in set which it treats as special, -and it knows some pandoc-specific classes and a number of file type classes. +and it knows some custom classes and a number of file type classes. If the author of a document wishes to use additional class names then they can include a `classes` list in the document metadata which subplot will treat as valid. ~~~scenario +given file unknown-class-name.subplot given file unknown-class-name.md +and file known-class-name.subplot and file known-class-name.md and file b.yaml and an installed subplot -when I try to run subplot docgen unknown-class-name.md -o unknown-class-name.html +when I try to run subplot docgen unknown-class-name.subplot -o unknown-class-name.html then command fails and file unknown-class-name.html does not exist and stderr contains "Unknown classes found in the document: foobar" -when I run subplot docgen known-class-name.md -o known-class-name.html +when I run subplot docgen known-class-name.subplot -o known-class-name.html then file known-class-name.html exists ~~~ -~~~~~~~~{#unknown-class-name.md .file .markdown .numberLines} ---- +~~~~~~~~{#unknown-class-name.subplot .file .yaml .numberLines} title: A document with an unknown class name -... +markdowns: +- unknown-class-name.md +~~~~~~~~ +~~~~~~~~{#unknown-class-name.md .file .markdown .numberLines} ```foobar This content is foobarish ``` - ~~~~~~~~ -~~~~~~~~{#known-class-name.md .file .markdown .numberLines} ---- +~~~~~~~~{#known-class-name.subplot .file .yaml .numberLines} title: A document with a previously unknown class name +markdowns: +- known-class-name.md classes: - foobar -... +~~~~~~~~ +~~~~~~~~{#known-class-name.md .file .markdown .numberLines} ```foobar This content is foobarish ``` - ~~~~~~~~ -## Using as a Pandoc filter +## Extract embedded files + +`subplot extract` extracts embedded files from a subplot file. + +~~~scenario +given file embedded-file.subplot +given file embedded-file.md +and file expected.txt +and an installed subplot +when I run subplot extract --merciful embedded-file.subplot foo.txt -d . +then files foo.txt and expected.txt match +~~~ + +~~~~~~{#embedded-file.subplot .file .yaml .numberLines} +title: Embedded file +markdowns: +- embedded-file.md +~~~~~~ + +~~~~~~{#embedded-file.md .file .markdown .numberLines} +~~~{#foo.txt .file} +This is a test file. +~~~ +~~~~~~ -Subplot can be used as a Pandoc _filter_, which means Pandoc can allow -Subplot to modify the document while it is being converted or typeset. -This can useful in a variety of ways, such as when using Pandoc to -improve Markdown processing in the [ikiwiki][] blog engine. +~~~{#expected.txt .file} +This is a test file. +~~~ +## Mistakes in markdown -[ikiwiki]: http://ikiwiki.info/ +When there are mistakes in the markdown input, Subplot should report +the location (filename, line, column) where the mistake is, and what +the mistake is. The scenarios in this section verify that. -The way filters work is that Pandoc parses the input document into an -abstract syntax tree, serializes that into JSON, gives that to the -filter (via the standard input), gets a modified abstract syntax tree -(again as JSON, via the filter's standard output). +### Scenario before the first heading -Subplot supports this via the **subplot-filter** executable. It is built -using the same internal logic as Subplot's docgen. The interface is -merely different to be usable as a Pandoc filter. +_Requirement: A scenario must follow a heading._ -This scenarios verifies that the filter works at all. More -importantly, it does that by feeding the filter a Markdown file that -does not have a YAML metadata block. For the ikiwiki use case, that's -what the input files are like. +Justification: the heading can be used as the title for the scenario. ~~~scenario -given file justdata.md -and an installed subplot -when I run pandoc --filter subplot-filter justdata.md -o justdata.html -then file justdata.html matches regex /does not have a YAML metadata/ +given an installed subplot +given file scenario-before-heading.subplot +given file scenario-before-heading.md +when I try to run subplot docgen scenario-before-heading.subplot -o /dev/null +then command fails +then stderr contains "ERROR: scenario-before-heading.md:1:1: first scenario is before first heading" ~~~ -The input file **justdata.md**: +~~~{#scenario-before-heading.subplot .file .yaml} +title: Foo +markdowns: + - scenario-before-heading.md +~~~ -~~~~~~~~{#justdata.md .file .markdown .numberLines} -This is an example Markdown file. -It does not have a YAML metadata block. -~~~~~~~~ +~~~~~~{#scenario-before-heading.md .file .markdown} +~~~scenario +~~~ +~~~~~~ +### Attempt to use definition list -## Extract embedded files +_Requirement: Attempt to use definition lists is reported._ -`subplot extract` extracts embedded files from a subplot file. +Justification: the markdown parser we use in Subplot doesn't support +them, and it would be unhelpful to not tell the user if they try to +use them. ~~~scenario -given file embedded-file.md -and file expected.txt -and an installed subplot -when I run subplot extract --merciful embedded-file.md foo.txt -d . -then files foo.txt and expected.txt match +given an installed subplot +given file dl.subplot +given file dl.md +when I try to run subplot docgen dl.subplot -o /dev/null +then command fails +then stderr contains "ERROR: dl.md:3:1: attempt to use definition lists in Markdown" ~~~ -~~~~~~{#embedded-file.md .file .markdown .numberLines} ---- -title: Embedded file -... +~~~{#dl.subplot .file .yaml} +title: Foo +markdowns: + - dl.md +~~~ -~~~{#foo.txt .file} -This is a test file. +~~~~~~{#dl.md .file .markdown} +# Foo + +Some term +: Definition of term. +~~~~~~ + +### Bad "add-newline" value + +_Requirement: Only specific values for the "add-newline" attribute are +allowed for an embedded file._ + +~~~scenario +given an installed subplot +given file add-newline.subplot +given file add-newline.md +when I try to run subplot docgen add-newline.subplot -o /dev/null +then command fails +then stderr contains "ERROR: add-newline.md:1:1: value of add-newline attribute is not understood: xyzzy" +~~~ + +~~~{#add-newline.subplot .file .yaml} +title: Foo +markdowns: + - add-newline.md ~~~ +~~~~~~{#add-newline.md .file .markdown} +~~~{#foo.txt .file add-newline=xyzzy} +~~~ ~~~~~~ -~~~{#expected.txt .file} -This is a test file. +## HTML output + +### Embedded CSS + +_Requirement:_ The user can specify CSS files to embed in the HTML +output. + +Justification: We want to allow production of self-standing output +with user-defined styling. + +~~~scenario +given file embedded-css.subplot +given file embedded-css.md +given file embedded-css.css +given file b.yaml +given an installed subplot +when I run subplot docgen embedded-css.subplot -o foo.html +then file foo.html contains "silly: property;" +~~~ + +~~~{#embedded-css.subplot .file .yaml .numberLines} +title: Embedded CSS +markdowns: + - embedded-css.md +bindings: + - b.yaml +css_embed: + - embedded-css.css +~~~ + +~~~~~~{#embedded-css.md .file .markdown .numberLines} +# This is a title + +~~~scenario +given precondition +~~~ +~~~~~~ + +~~~{#embedded-css.css .file .css .numberLines} +html { + silly: property; +} +~~~ + +### CSS URLs + +_Requirement:_ The user can specify CSS URLs to add in the HTML +output. + +Justification: We want to allow users to specify non-embedded CSS. + +~~~scenario +given file css-urls.subplot +given file css-urls.md +given file b.yaml +given an installed subplot +when I run subplot docgen css-urls.subplot -o foo.html +then file foo.html contains "https://example.com/flushing.css" +~~~ + +~~~{#css-urls.subplot .file .yaml .numberLines} +title: Embedded CSS +markdowns: + - css-urls.md +bindings: + - b.yaml +css_urls: + - https://example.com/flushing.css +~~~ + +~~~~~~{#css-urls.md .file .markdown .numberLines} +# This is a title + +~~~scenario +given precondition +~~~ +~~~~~~ + + +## Running Subplot + +The scenarios in this section verify that the Subplot tool can be run +in various specific ways. + +### Files not in current working directory + +_Requirement: Subplot can process a subplot that is not in the current +working directory._ + +~~~scenario +given file x/simple.subplot from simple.subplot +given file x/simple.md from simple.md +given file x/b.yaml from b.yaml +given file x/f.py from f.py +given an installed subplot +when I run subplot metadata x/simple.subplot +then command is successful +when I run subplot codegen x/simple.subplot -o test.py +then file test.py exists +when I run subplot docgen x/simple.subplot -o simple.html +then file simple.html exists ~~~ |