path: root/riki.md

[ikiwiki]: http://ikiwiki.info/
[Subplot]: https://subplot.tech/

# Introduction

`riki` is a small subset of [ikiwiki][] rewritten in Rust, for
speed. This document describes the requirements and acceptance
criteria for the software, and how to verify that riki meets them in
an automated way. This is done using the [Subplot][] software.

# Software architecture

`riki` converts files in a source tree into a files that form a static
website in an output, or target, tree. The files in the source tree
are either "pages" or "blobs". The files in the target tree are HTML
files or blobs.

Source pages contain "wiki text", which
adds on top of Markdown syntax for *wiki links* and *directives*:

* plain wiki link: `[[pagename]]`
  - this corresponds to Markdown: `[pagename](pagename)`
  - or HTML: `<a href="pagename">pagename</a>`
* wiki link with link text: ``[[link text|pagename]]`
  - this corresponds to Markdown: `[link text](pagename)`
  - or HTML: `<a href="pagename">link text</a>`
* directive: `[[!foo arg other="value for other" more="""value for
  more"""]]`
  - directive arguments may contain values
  - values may be single or triple quoted: triple quoted may span
    multiple lines

Directives cause some processing to be done. That processing may take
as its input only values of arguments, or the page, where the
directive is used, or all other files in the site.

Wiki text is converted into plain Markdown by replacing wiki links and
directives with Markdown text, before the whole page is parsed into
HTML, which gets written to the target directory.

Blobs are copied to the target directory as-is, without any
processing.

## Processing pipeline

~~~dot
digraph "processing" {
    source [shape=folder]
    target [shape=folder]
    blob [shape=note]
    wikitext [shape=note]
    html [shape=note]

    source -> blob
    source -> wikitext

    wikitext -> markdown
    wikitext -> wikilink
    wikitext -> directive
    wikilink -> markdown
    directive -> markdown

    markdown -> html
    html -> target

    blob -> target
}
~~~

When the site is processed from source to target, the processing
pipeline is roughly like this:

* read in all files in the source tree
* parse each page of wiki text into snippets
  - a snippet is plain markdown, a wiki link, or a directive
* prepare directives in each page
  - this collects or produces data needed for processing the
    directive, but doesn't produce output
* process directives in each page
  - this produces markdown output
* process wiki links in each page
* combine all processed snippets into one markdown string for each
  page and parse that into HTML
* write HTML files to target tree
* copy all blobs to target tree

Note that when preparing or processing directives, directives on all
pages are prepared first, before any directives are processed. This
allows things like defining shortcuts on any page of the site: the
shortcut definitions are recognized and obeyed during the preparation
stage.

# Verification scenarios

The approach used for verifying acceptance criteria is to run `riki`
against known inputs, and check that the output is as expected.
Specifically this is done by comparing the Pandoc abstract syntax
trees of the input and output. Pandoc is a well-known, well-respected
tool that we rely on as an "oracle".

## Markdown features

### Empty Markdown file

_Requirement: Given an empty input Markdown file, the output must
be an empty HTML file._

~~~scenario
given an installed riki
given file site/empty.mdwn from empty
when I run riki build --plain-body site output
then AST of site/empty.mdwn matches that of output/empty/index.html
~~~


~~~{#empty .file}
~~~

### Plain text

_Requirement: Given a Markdown file with plain text, the output must
be an HTML file with the same text, without extra elements._

~~~scenario
given an installed riki
given file site/page.mdwn from para
when I run riki build --plain-body site output
then AST of site/page.mdwn matches that of output/page/index.html
~~~


~~~{#para .file}
Hello, world.

There are two paragraphs.
~~~


### Quoted block

_Requirement: Given a Markdown file with an quoted block of text, the
output must have a blockquote element.

~~~scenario
given an installed riki
given file site/page.mdwn from blockquote
when I run riki build --plain-body site output
then AST of site/page.mdwn matches that of output/page/index.html
~~~


~~~{#blockquote .file}
> This is a quoted block.
~~~


### Indented code block

_Requirement: Given a Markdown file with an indented code block, the
output must have a pre element.

~~~scenario
given an installed riki
given file site/page.mdwn from indented-code
when I run riki build --plain-body site output
then AST of site/page.mdwn matches that of output/page/index.html
~~~


~~~{#indented-code .file}
    This is indented by four spaces.
~~~


### Fenced code block

_Requirement: Given a Markdown file with a fenced code block, the
output must have a pre element.

~~~scenario
given an installed riki
given file site/page.mdwn from fenced-code
when I run riki build --plain-body site output
then AST of site/page.mdwn matches that of output/page/index.html
~~~


~~~{#fenced-code .file}
```
This is a fenced code block.
```
~~~



### Image link

_Requirement: Given a Markdown file linking to an image, the output
must have an img element.

~~~scenario
given an installed riki
given file site/page.mdwn from image-link
when I run riki build --plain-body site output
then AST of site/page.mdwn matches that of output/page/index.html
~~~


~~~{#image-link .file}
![my kitten](cat.jpg "image-title")
~~~

### Emphasized text

_Requirement: Inline markup for emphasis must result in an em element
in HTML output._

~~~scenario
given an installed riki
given file site/page.mdwn from emph
when I run riki build --plain-body site output
then AST of site/page.mdwn matches that of output/page/index.html
~~~


~~~{#emph .file}
There is *emphasized*, and so is _this_.
~~~

### Strongly emphasised text

_Requirement: Inline markup for strong emphasis must result in a
strong element in HTML output._

~~~scenario
given an installed riki
given file site/page.mdwn from strong
when I run riki build --plain-body site output
then AST of site/page.mdwn matches that of output/page/index.html
~~~


~~~{#strong .file}
There is **emphasized**, and so is __this__.
~~~

### Strike through in text

_Requirement: Inline markup for strike through must result in a del
element in HTML output._

~~~scenario
given an installed riki
given file site/page.mdwn from strike
when I run riki build --plain-body site output
then AST of site/page.mdwn matches that of output/page/index.html
~~~


~~~{#strike .file}
There is ~~struck through~~.
~~~

### Headings

_Requirement: Given a Markdown file with headings of various levels,
the output must be an HTML file with corresponding `h1`, `h2`, etc,
elements, without extra elements. Up to six levels of headings must be
supported._

~~~scenario
given an installed riki
given file site/page.mdwn from headings
when I run riki build --plain-body site output
then AST of site/page.mdwn matches that of output/page/index.html
~~~


~~~{#headings .file}
# Heading one
## Heading two
### Heading three
#### Heading four
##### Heading five
###### Heading six
~~~

### Inline code

_Requirement: Inline code markup with backticks must result in a code
element in HTML output._

~~~scenario
given an installed riki
given file site/page.mdwn from backticks
when I run riki build --plain-body site output
then AST of site/page.mdwn matches that of output/page/index.html
~~~


~~~{#backticks .file}
There is `code` lurking here.
~~~

### Table

_Requirement: Markup of a table result in a table element in HTML
output._

**Note: This is disabled. Pandoc doesn't seem to handle the HTML table OK.***

~~~
given an installed riki
given file site/page.mdwn from table
when I run riki build --plain-body site output
then AST of site/page.mdwn matches that of output/page/index.html
~~~


<!--
~~~{#table .file}
| foo | bar |
| --- | --- |
|   1 |   2 |
~~~
-->

### Horizontal rule

_Requirement: Markup of a horizontal rule must result in hr element in
HTML output._

~~~scenario
given an installed riki
given file site/page.mdwn from rule
when I run riki build --plain-body site output
then AST of site/page.mdwn matches that of output/page/index.html
~~~


~~~{#rule .file}
foo

---------------------------------------------------------------------------------------

bar
~~~

### Unordered list

_Requirement: Markup of an unordered list must result in a ul element
in HTML output._

~~~scenario
given an installed riki
given file site/page.mdwn from ul
when I run riki build --plain-body site output
then AST of site/page.mdwn matches that of output/page/index.html
~~~


~~~{#ul .file}
* first
* second
~~~

### Ordered list

_Requirement: Markup of an ordered list must result in an ol element
in HTML output._

**Note: This is disabled. Pandoc doesn't seem to parse the HTML list
the same as the Markdown.***

~~~
given an installed riki
given file site/page.mdwn from ol
when I run riki build --plain-body site output
then AST of site/page.mdwn matches that of output/page/index.html
~~~

<!--
~~~{#ol .file}
1. first
2. second
~~~
-->

### Task list

_Requirement: Markup of a task list must result in a ul element
in HTML output._

~~~scenario
given an installed riki
given file site/page.mdwn from tasklist
when I run riki build --plain-body site output
then AST of site/page.mdwn matches that of output/page/index.html
~~~


~~~{#tasklist .file}
* [ ] not done
* [x] done
~~~

### Definition list

_Requirement: Markup indicating use of a definition list should be
flagged as an error._

Justification: Neither the CommonMark specification, nor GitHub
Flavored Markdown, supports definition lists, even though some
Markdown variants do. The Markdown parser Riki uses doesn't support
it.

~~~scenario
given an installed riki

given file site/page.mdwn from dl-1
when I try to run riki build --plain-body site output
then command fails
then stderr contains "definition list"

given file site/page.mdwn from dl-2
when I try to run riki build --plain-body site output
then command fails
then stderr contains "definition list"

given file site/page.mdwn from dl-3
when I run riki build --plain-body site output
then file output/page/index.html contains ": bar"
~~~

~~~{#dl-1 .file}
foo
: bar
~~~

~~~{#dl-2 .file}
foo

: bar
~~~

~~~{#dl-3 .file}
foo

<!-- no colon at beginning of line here -->: bar
~~~


### Wiki links to other pages on the site

_Requirement: Pages can link to other pages on the site, the same
way ikiwiki does, including subpages._

~~~scenario
given an installed riki
given file site/dir/foo.mdwn from foo
given file site/absolute.mdwn from empty
given file site/dir/sibling.mdwn from empty
given file site/dir/foo/child.mdwn from empty
given file site/dir/foo/child/grandchild.mdwn from empty
when I run riki build --plain-body site output
then file output/dir/foo/index.html contains "href="../absolute""
then file output/dir/foo/index.html contains "href="sibling""
then file output/dir/foo/index.html contains "href="foo/child""
then file output/dir/foo/index.html contains "href="foo/child/grandchild""
~~~

Note the uppercase link to the `child` page in the test page below.

~~~{#foo .file .markdown}
[[/absolute]]
[[sibling]]
[[child]]
[[child/grandchild]]
[[CHILD]]
~~~

### Wiki links to pages that don't exist

_Requirement: Linking to a page that doesn't exist is an error._

~~~scenario
given an installed riki
given file site/dir/foo.mdwn from badlink
when I try to run riki build --plain-body site output
then command fails
~~~

~~~{#badlink .file .markdown}
[[missing]]
~~~

## Directives

### `img`

The [ikiwiki img directive][] allow including images in the site
source tree.

[ikiwiki img directive](http://ikiwiki.info/ikiwiki/directive/img/)

#### Simple image inclusion

_Requirement: the `img` directive embeds an image in the generated
HTML page._

~~~scenario
given an installed riki
given file site/index.mdwn from img
given file site/img.jpg from jpeg
when I run riki build site output
then file output/index.html contains "<img src="img.jpg""
~~~

~~~{#img .file .markdown}
[[!img img.jpg]]
~~~

~~~{#jpeg .file}
This is a dummy JPEG image.
~~~

#### Image size

_Requirement: the `img` directive can set size._

From the [ikiwiki img directive][] documentation:

> `size`---The size parameter is optional, defaulting to full size.
> You can specify only the width or the height, and the other value
>  will be calculated based on it: "200x", "x200".

~~~scenario
given an installed riki
given file site/index.mdwn from img-size
given file site/a.jpg from jpeg
given file site/b.jpg from jpeg
given file site/c.jpg from jpeg
when I run riki build site output
then file output/index.html contains "<img src="a.jpg" width="100" height="200">"
then file output/index.html contains "<img src="b.jpg" width="100">"
then file output/index.html contains "<img src="c.jpg" height="200">"
~~~

~~~{#img-size .file .markdown}
[[!img a.jpg size="100x200"]]
[[!img b.jpg size="100x"]]
[[!img c.jpg size="x200"]]
~~~

#### Image attributes

_Requirement: the `img` directive allows useful attributes to be set._

The [ikiwiki img
directive](http://ikiwiki.info/ikiwiki/directive/img/) allows
arguments:

> `alt`, `title`, `class`, `align`, `id`, `hspace`, and
> `vspace`---These are passed through unchanged to the html img tag.

~~~scenario
given an installed riki
given file site/index.mdwn from img-attr
given file site/img.jpg from jpeg
when I run riki build site output
when I run cat output/index.html
then file output/index.html contains "<img src="img.jpg""
then file output/index.html contains "alt="halt malt""
then file output/index.html contains "title="tightle""
then file output/index.html contains "class="klass""
then file output/index.html contains "align="malign""
then file output/index.html contains "id="kid""
then file output/index.html contains "hspace="hspc""
then file output/index.html contains "vspace="vspc""
~~~

~~~{#img-attr .file .markdown}
[[!img img.jpg alt="halt malt" title="tightle" class="klass" align="malign"
   id="kid" hspace="hspc" vspace="vspc"]]
~~~

#### Image link generation

_Requirement: the `img` directive can make an image be a link._

The [ikiwiki img
directive](http://ikiwiki.info/ikiwiki/directive/img/) allows
arguments:

> The link parameter is used to control whether the scaled image links
> to the full size version. By default it does; set "link=somepage" to
> link to another page instead, or "link=no" to disable the link, or
> "link=http://url" to link to a given url.

~~~scenario
given an installed riki
given file site/index.mdwn from img-link
given file site/a.jpg from jpeg
given file site/b.jpg from jpeg
when I run riki build site output
when I run cat output/index.html
then file output/index.html contains "<a href="a.jpg"><img src="a.jpg""
then file output/index.html doesn't contain "<a href="b.jpg"><img src="b.jpg""
~~~

~~~{#img-link .file .markdown}
[[!img a.jpg]]
[[!img b.jpg link=no]]
~~~



### meta title

_Requirement: the `meta title` directive sets page title._

~~~scenario
given an installed riki
given file site/index.mdwn from meta
when I run riki build site output
then file output/index.html contains "<TITLE>Yo</TITLE>"
~~~

~~~{#meta .file .markdown}
[[!meta title=Yo]]]
~~~

### shortcut

_Requirement: the `shortcut` directive created a shortcut that looks
like a directive._

~~~scenario
given an installed riki
given file site/a.mdwn from use_shortcut
given file site/b.mdwn from define_shortcut
when I run riki build site output
when I run cat output/a/index.html
then file output/a/index.html contains "<A href="https://example.com/foo/123">foo!123</A>"
~~~

~~~{#use_shortcut .file .markdown}
[[!foo 123]]
~~~

~~~{#define_shortcut .file .markdown}
[[!shortcut name="foo" url="https://example.com/foo/%s" desc="foo!%s"]]
~~~

### table

_Requirement: the `table` directive creates a simple table._

~~~scenario
given an installed riki
given file site/index.mdwn from table
when I run riki build site output
when I run cat output/index.html
then file output/index.html contains "<TABLE>"
then file output/index.html contains "<TH><TD>Greeting</TD>"
then file output/index.html contains "<TD>Greetee</TD>"
then file output/index.html contains "<TR><TD>hello</TD>"
then file output/index.html contains "<TD>world</TD>"
then file output/index.html contains "<TR><TD>goodbye</TD>"
then file output/index.html contains "<TD>cruel world</TD>"
~~~

~~~{#table .file .markdown}
[[!table data="""
Greeting | Greetee
hello | world
goodbye | cruel world
"""]]
~~~

### toc

_Requirement: the `toc` directive creates a table of contents._

~~~scenario
given an installed riki
given file site/index.mdwn from toc
when I run riki build site output
when I run cat output/index.html
then file output/index.html contains "<LI>Introduction</LI>"
then file output/index.html contains "<LI>Acknowledgements</LI>"
~~~

~~~{#toc .file .markdown}
[[!toc]]

# Introduction
## Acknowledgements
~~~

## Source file tree

### Listing source files

_Requirement: source files can be listed._

~~~scenario
given an installed riki
given file site/index.mdwn from empty
given file site/img.jpg from empty
when I run riki list site
then stdout contains "img.jpg"
then stdout contains "index.mdwn"
~~~

### Exclude unusual files

_Requirement: files and directories that aren't meant to be part of
the site content should be excluded._

~~~scenario
given an installed riki
given file site/index.mdwn from empty
given file site/img.jpg from empty
given file site/.git from empty
given file site/index.mdwn~ from empty
given file site/#index.mdwn# from empty
when I run riki list site
then stdout contains "img.jpg"
then stdout contains "index.mdwn"
then stdout doesn't contain ".git"
then stdout doesn't contain "index.mdwn~"
then stdout doesn't contain "#index.mdwn#"
~~~



## Input files other than Markdown

_Requirement: Input files that aren't Markdown files must be copied
into the destination directory as-is._

~~~scenario
given an installed riki
given file site/image.jpg from image
when I run riki build --plain-body site output
then files site/image.jpg and output/image.jpg match
~~~


~~~{#image .file}
# Dummy
Pretend this is an image.
~~~

## Input files in sub-directories

_Requirement: If an source page or file is in a sub-directory, it
should be put in the corresponding sub-directory in the target
directory._

~~~scenario
given an installed riki
given file site/foo/page.mdwn from image
given file site/bar/image.jpg from para
when I run riki build --plain-body site output
then AST of site/foo/page.mdwn matches that of output/foo/page/index.html
then files site/bar/image.jpg and output/bar/image.jpg match
~~~

## Output directory tree

### No markdown files in output tree

_Requirement: Markdown files are not copied to the output tree._

~~~scenario
given an installed riki
given file site/index.mdwn from empty
when I run riki build site output
then file output/index.html exists
then file output/index.mdwn does not exist
~~~

### Output files have source file modification times

_Requirement: Files in the output directory have the same time stamp
as the corresponding files in the source directory._

Note that due to limitations in the Subplot `lib/files` library, our
check for modification times is imprecise.

~~~scenario
given an installed riki
given file site/index.mdwn from empty
given file site/index.mdwn has modification time 1970-01-01 00:00:00
given file site/index.jpg from empty
given file site/index.jpg has modification time 1970-01-01 00:00:00
when I run riki build site output
then file output/index.html has a very old modification time
then file output/index.jpg has a very old modification time
~~~

### Output files have source `meta date` modification times

_Requirement: Files in the output directory have the time stamp
specified in a `meta date` directive._

Note that due to limitations in the Subplot `lib/files` library, our
check for modification times is imprecise.

~~~scenario
given an installed riki
given file site/index.mdwn from dated
given file site/index.mdwn has modification time 2022-02-02 01:02:03
when I run riki build site output
then file output/index.html has a very old modification time
~~~


~~~{#dated .file .markdown}
[[!meta date="1970-01-01 00:00:00"]]

Hello.
~~~