summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorLars Wirzenius <liw@liw.fi>2019-06-09 08:43:19 +0300
committerLars Wirzenius <liw@liw.fi>2019-06-09 08:43:19 +0300
commit87c39a5b41ab5d775177524e88fbcf12a4c452c1 (patch)
treeec475e4862c2549a29746284d4ff90df592ca159
parent7456120b2c7f0572e76fb81901413abc9092a482 (diff)
downloadfable-87c39a5b41ab5d775177524e88fbcf12a4c452c1.tar.gz
Drop: fable arch doc, now that it's in fable-poc.gitHEADmaster
-rw-r--r--Makefile11
-rw-r--r--README3
-rw-r--r--arch.dot29
-rw-r--r--fable.md290
-rw-r--r--fable.yaml0
5 files changed, 2 insertions, 331 deletions
diff --git a/Makefile b/Makefile
deleted file mode 100644
index 1392c21..0000000
--- a/Makefile
+++ /dev/null
@@ -1,11 +0,0 @@
-.SUFFIXES: .md .pdf .dot .svg
-
-all: fable.pdf
-
-fable.pdf: Makefile fable.md arch.svg
-
-.md.pdf:
- pandoc -o $@ $<
-
-.dot.svg:
- dot -Tsvg -o $@ $<
diff --git a/README b/README
index 756b1f9..5ef7ce9 100644
--- a/README
+++ b/README
@@ -1,4 +1,5 @@
fable - automated acceptance testing tool
=============================================================================
-FIXME
+This used to have the arch doc. It is moved, for now, to
+fable-poc.git.
diff --git a/arch.dot b/arch.dot
deleted file mode 100644
index 8c8e780..0000000
--- a/arch.dot
+++ /dev/null
@@ -1,29 +0,0 @@
-digraph arch {
- md [label="document source\n(Markdown)"];
- md [shape=box];
-
- bindings [label="bindings file\n(YAML)"];
- bindings [shape=box];
-
- impl [label="step implementations\n(Python, Rust, ...)"]
- impl [shape=box];
-
- fable [label="Fable"];
- fable [shape=ellipse];
-
- pdf [label="PDF document"]
- pdf [shape=box];
-
- testprog [label="Test program\n(generated)"]
- testprog [shape=box];
-
- report [label="Test report"]
- report [shape=box];
-
- md -> fable;
- bindings -> fable;
- impl -> fable;
- fable -> pdf;
- fable -> testprog;
- testprog -> report;
-}
diff --git a/fable.md b/fable.md
deleted file mode 100644
index e1158fa..0000000
--- a/fable.md
+++ /dev/null
@@ -1,290 +0,0 @@
----
-title: Acceptance testing using Fable
-author:
-- Lars Wirzenius
-- Daniel Silverstone
-date: WORK IN PROGRESS
-keywords:
-- automated acceptance testing
-- scenario testing
-- gherkin
-- cucumber
-abstract: |
- Fable is a tool that supports acceptance testing in two ways: it is a
- way to write and run automated acceptance tests, and also
- presents the acceptance test suite to non-expert readers as a
- human-readable text document.
-
- This document explains Fable, its architecture, and its input
- language.
-...
-
-# Introduction
-
-Fable is a tool for acceptance testing of software. This means it helps
-software developers to make sure their software fulfils the
-acceptance criteria for the software. Such criteria may come from
-users, the developers, their employers, or elsewhere.
-
-Fable is specifically meant for automated acceptance testing. It takes a
-two-pronged approach, where it lets developers write automated
-tests for all the acceptance criteria they have, and runs the tests.
-On the other hand, Fable also produces a PDF file, which documents the
-automated tests for non-technical stakeholders, such as project
-managers and clients.
-
-More concretely, Fable helps developers implement and document their
-automated acceptance tests in a way that, at the same time, helps the
-developers automatically test their software, and write documentation
-for the tests in way that doesn't require programming knowledge to
-understand.
-
-Fable is meant to be a tool for developers, who use it to produce a
-document, which is meant to facilitate communication between various
-shareholders of the software being developed.
-
-Fable's overall working principle is that the tests are implemented and
-documented in a number of source files, which Fable reads to execute
-tests, and to produce a PDF document for non-developer consumption.
-
-# Fable architecture
-
-![Fable architecture](arch.svg)\
-
-Fable's architecture it to read input files, and produce two outputs.
-On the one hand, it outputs a program that executes the tests
-specified in the input files. The person running Fable then runs the
-test program to get a test report, with results of each of the test
-scenarios. On the other hand it outputs a human-readable document (as
-PDF), for communicating what is being tested.
-
-Fable is able to produce the test program in various languages, using
-a templating system to make it simple to add support for more
-languages. Fable comes with support for Python and Rust. It's the
-user's choice which language they're most comfortable with.
-
-Acceptance tests are expressed to Fable in the form of test scenarios,
-in which a sequence of actions are taken, and then the results are
-checked. If the checks fail, the scenario fails.
-
-Fable runs the scenarios concurrently (but see the USING keyword),
-within the constrains of hardware resources. If Fable determines it
-doesn't have all the resources to run all scenarios at once, it will
-run fewer, but randomly chosen scenarios concurrently, to more likely
-to detect unintentional dependencies between scenarios.
-
-# The Fable input language
-
-Fable input consists of three types of files:
-
-* one or more markdown files which document that acceptance tests
-* a binding file (in YAML) which binds scenario steps to their
- implementations
-* scenario step implementations, which are implemented in some
- programming language (e.g., Python or Rust); Fable will combine this
- code with some scaffolding provided by Fable itself
-
-The input files for a simple acceptance test suite for Fable would be
-divided into three files: `foo.md`, `foo.yaml`, and `foo.py` (assuming
-step implementation in Python).
-
-## Markdown files
-
-[fenced code blocks]: https://pandoc.org/MANUAL.html#fenced-code-blocks
-[Gherkin]: https://en.wikipedia.org/wiki/Cucumber_(software)#Gherkin_language
-[Cucumber]: https://en.wikipedia.org/wiki/Cucumber_(software)
-
-The Fable input language is markdown files using [fenced code blocks][]
-with backticks. The code blocks MUST indicate that they contain Fable
-language:
-
- ```fable
- given a service
- and I am Tomjon
- when I access the service
- then it's OK
- ```
-
-Any other code blocks are ignored by Fable and can be used for, say,
-code examples.
-
-FIXME: Later, we may want to add support for embedding
-PlantUML or Graphviz Dot markup, and generate images from that
-automatically. This would look like the example below, and Fable would
-replace the code block with the generated image in the PDF output.
-
- ```dot
- foo -> bar
- ```
-
-Fable understands the full Markdown language, by ignoring everything
-except its own code blocks and headings. It uses [Pandoc][] to produce
-PDF files, and anything that Pandoc supports is OK to use.
-
-[Pandoc]: https://pandoc.org/
-
-Fable treats multiple Markdown files as one, as if they had been
-concatenated with the **cat**(1) utility. Within the logical file,
-normal Markdown and Pandoc markup can be used to structure the
-document in any way that aids human understanding of the acceptance
-test suite, which the caveat that chapter or section headings are used
-by Fable to group code blocks into scenarios.
-
-All code blocks for the same scenario MUST be grouped under a single
-heading. Sub-headings are permitted within a scenario, but the next
-heading at the same or a higher level will end the scenario. This
-allows for scenarios to begin at any level, but not to leak into a
-wider scope within the acceptance document. For example, a scenario
-which starts after a level 2 heading may have subdivisions marked with
-level 3 or below headings, but will end at the next level 2 or level 1
-heading.
-
-Within the Fable code blocks, Fable understands a special language,
-derived from [Gherkin][], as defined by the [Cucumber][] testing tool.
-The language understood by Fable has the following general structure:
-
-* each logical line starts with a keyword at the beginning of the line
-* logical lines may be broken into physical lines, by starting the
- continuation lines with one or more space or TAB characters; the
- physical line break and white space characters are preserved
-* logical lines define steps in a test scenario
-* the meaning and implementation of the steps are defined by other
- Fable input files
-* the keywords are: ASSUMING, USING, GIVEN, WHEN, THEN, AND, with
- meanings defined below; keywords can be written in upper or lower
- case, or mixes, Fable doesn't care
-
-The keywords have the following meanings:
-
-* ASSUMING&mdash;a condition for the scenario; all ASSUMING steps MUST
- be at the beginning of the scenario, and may not be preceded by
- other steps; if the condition fails, the scenario is skipped.
-
- This is used for skipping test scenarios that require specific
- software to be installed in the test environment, or access to
- external services, but which can't be required for all runs of the
- acceptance tests.
-
- All ASSUMING steps MUST come first in the scenario.
-
-* USING&mdash;indicate that the scenario uses a resource such as a
- database, that's constrained and can't be used by all scenarios if
- they run concurrently. When scenarios declare the resource, Fable can
- limit which scenarios run concurrently.
-
- For example, if several scenarios require uncontested use of the
- GPU, of which there is typically only one per machine, they can all
- declare "using the graphical processing unit", and Fable will run
- them one at a time.
-
- (This is an intentionally simplistic way of controlling concurrency.
- The goal is to be simple and correct rather then achievee maximal
- concurrency.)
-
- The actual management of resources belongs to the generated test
- program at runtime, not the Fable compiler.
-
- All USING steps MUST come at the beginning of the scenario, except
- the ASSUMING steps may precede them.
-
-* GIVEN&mdash;set up the test environment for the action (WHEN). This
- might create files, start a background process, or something like
- that. This also sets up the reversal of the setup, so that any
- background processes are stopped automatically after the scenario
- has ended. The setup and cleanup MUST succeed, or the scenario will
- fail.
-
- The cleanups are executed in the reverse order of the GIVENs, and
- they're done always, whether the scenario succeeds or not.
-
-* WHEN&mdash;perform the action that is being tested. This MUST
- succeed. This might, for example, execute a command line program,
- and capture its output and exit code.
-
-* THEN&mdash;test the results of the action. This would examine the
- output and exit code of the program run in a WHEN step, or examine
- current content of the database, or whatever is needed.
-
-* AND&mdash;this keyword exists to make scenarios "read" better in
- English. The keyword indicates that this step should use the same
- keyword as the previous step, whatever that keyword is. For example,
- a step "THEN output is empty" might be followed by "AND the exit
- code is 0" rather than "THEN the exit code is 0".
-
-## Bindings
-
-FIXME: The binding specification needs thought. This is just a sketch.
-
-Binding files match scenario steps to functions that implement them,
-using regular expressions. The bindings may also extract parts of the
-steps, and pass them onto the functions as parameters.
-
-Binding files are YAML files, with lists of bindings, each binding
-being a dict. For example:
-
-```yaml
-- define:
- name: string
- exit_code: int
-
-- pattern: given a service
- function: start_service
- cleanup: stop_service
-
-- pattern: given I am (?P<name\S+)
- function: set_name
- produces: [name]
-
-- pattern: when I access the service
- function: access_service
- requires: [name]
- produces: [exit_code]
-
-- pattern: then it's OK
- function: check_access_was_ok
- requires: [exit_code]
-```
-
-In the example above, the "I am" step extracts the name of the user
-from the step. It's type is declared, and the value is saved for use
-by a later step.
-
-The "I access" step expects the name to have been set by a previous
-step. Fable will check that the name is set, and give an error if it
-isn't, before any scenario runs. If name is set, it is given to the
-function to be called as a function argument.
-
-The "I access" step further sets the variable "exit_code", and the
-"it's OK" step expects it to be set.
-
-## Step implementations
-
-Continuing the example from the previous section, the following Python
-code might implement the functions:
-
-```python
-def start_service():
- ...
-
-def set_name(**matches):
- ...
-
-def access_service(name):
- ...
- return {
- 'exit_code': 0,
- }
-
-def check_access_was_ok(exit_code):
- assert exit_code == 0
-```
-
-With these bindings, Fable produces a Python program, which calls
-these functions in order, and passes values between them via function
-arguments and return values. The generated program will handle running
-scenarios concurrently, and taking care of USING constraints, and
-other resource constraints.
-
-# Acceptance tests for Fable
-
-FIXME: This needs to be written eventually, when Fable is implemented.
diff --git a/fable.yaml b/fable.yaml
deleted file mode 100644
index e69de29..0000000
--- a/fable.yaml
+++ /dev/null