Add docs on the yarn tests and more build tests

4 files changed, 115 insertions, 18 deletions

diff --git a/doc/overview.rst b/doc/overview.rst
index 0ae0e2f..c38dbfd 100644
--- a/doc/overview.rst
+++ b/doc/overview.rst
@@ -173,21 +173,6 @@ Peformance
                        of: none, simple, meliae, or heapy (default: simple)
  --memory-dump-interval=SECONDS
                        make memory profiling dumps at least SECONDS apart
-
-Developing
-**********
-
-.. index:: pre-commit
-
-.. _pre_commit_hook:
-
-Testing vmdebootstrap from git
-==============================
-
-::
-
- ln -s ../../pre-commit.sh .git/hooks/pre-commit
-
 .. index:: networking
 
 .. _networking:
@@ -415,3 +400,89 @@ a specific offset instead of using a normal partition will
 **not** be supportable by vmdebootstrap. Similarly, devices which support
 hypervisor will only be supported using virtual machine images, unless
 the bootloader can be executed from a normal partition.
+
+.. index:: developing
+
+.. _developing:
+
+Developing
+**********
+
+.. index:: pre-commit
+
+.. _pre_commit_hook:
+
+Testing vmdebootstrap from git
+==============================
+
+``vmdebootstrap`` uses ``yarn`` for the test suite, available in the
+`cmdtest <https://tracker.debian.org/pkg/cmdtest>`_ package. YARN
+is a scenario testing tool. Scenarios are written in mostly human
+readable language, however, they are not free form text. For more
+information on YARN see `the homepage <http://liw.fi/cmdtest/README.yarn/>`_::
+
+ $ sudo apt -y install cmdtest
+
+All commits must pass at least the fast tests. All merges into master
+need to pass a full test. All additions of new functionality must add
+fast and build tests - fast tests for any new options and build tests
+which exercise the new functionality. Build tests can add checks for
+particular support on the machine running the test and skip if not
+found or add new environment settings to selectively run some build
+tests instead of all.
+
+If no arguments are given, the full test suite will be executed::
+
+ $ yarns/run-tests
+
+.. warning:: Do not run the full test suite if your connection to a
+   Debian mirror is limited or metered. Each build requires a minimum
+   of 2Gb free space in tmpfs. A full test takes at least 10 minutes.
+
+When limiting the run to specific tests, each ``--env`` option needs
+to be specified separately::
+
+ $ sudo yarns/run-tests --env TESTS=build --env MIRROR=http://mirror/debian
+
+pre-commit
+----------
+
+All vmdebootstrap developers need to run the fast tests as a pre-commit
+hook - any patches which fail this test will be rejected::
+
+ $ ln -s ../../pre-commit.sh .git/hooks/pre-commit
+
+The pre-commit hook just runs the fast tests which do not require
+``sudo``.
+
+Fast tests
+-----------
+
+The fast checks validate the handling of incompatible option arguments::
+
+ $ yarns/run-tests --env TESTS=fast
+
+Fast tests typically take a few seconds to run.
+
+Build tests
+-----------
+
+The slow / build tests build multiple images and use ``sudo`` - a local
+mirror is strongly recommended.
+
+::
+
+ $ sudo yarns/run-tests --env TESTS=build --env MIRROR=http://mirror/debian
+
+If ``MIRROR`` is not specified, a default mirror of ``http://http.debian.net/debian/``
+will be used.
+
+LAVA tests
+----------
+
+There is an example file:`lava-submit.py` script which can be edited
+to automatically submit QEMU tests to a specified LAVA instance. The
+images themselves will use local ``file://`` URLs and therefore the
+``lava-dispatcher`` needs to be installed locally. Configuring LAVA
+for these tests is a separate topic - please ask on the `vmdebootstrap
+mailing list <https://lists.alioth.debian.org/mailman/listinfo/vmdebootstrap-devel>`_.
diff --git a/yarns/300-slow-build-tests.yarn b/yarns/300-slow-build-tests.yarn
index 41d11a3..2ceae72 100644
--- a/yarns/300-slow-build-tests.yarn
+++ b/yarns/300-slow-build-tests.yarn
@@ -14,7 +14,26 @@ These tests are slow, since building images is slow.
     GIVEN user wants to build an image FOO.img that is 2GiB in size
     WHEN the user runs vmdebootstrap --sparse --extlinux
     THEN the image has the correct size
-    AND the image has one partition
+    AND the partition count of the image is 1
     AND partition 1 has the boot flag set
     AND partition 1 has an ext4 filesystem
     AND partition 1 has file /etc/debian_version matching ^8\..*$
+
+    SCENARIO build a Debian 8 image with grub
+    ASSUMING build tests are requested
+    GIVEN user wants to build an image FOO.img that is 2GiB in size
+    WHEN the user runs vmdebootstrap --grub
+    THEN the image has the correct size
+    AND the partition count of the image is 1
+    AND partition 1 has the boot flag set
+    AND partition 1 has an ext4 filesystem
+    AND partition 1 has file /boot/grub/grub.cfg matching ^### BEGIN /etc/grub.d/00_header ###$
+
+    SCENARIO build a Debian 8 image with uefi
+    ASSUMING build tests are requested
+    GIVEN user wants to build an image FOO.img that is 2GiB in size
+    WHEN the user runs vmdebootstrap --grub --use-uefi
+    THEN the image has the correct size
+    AND the partition count of the image is 2
+    AND partition 1 has an vfat filesystem
+    AND partition 2 has file /boot/grub/grub.cfg matching ^### BEGIN /etc/grub.d/00_header ###$
diff --git a/yarns/800-future.yarn b/yarns/800-future.yarn
index 1368b8f..a6ca722 100644
--- a/yarns/800-future.yarn
+++ b/yarns/800-future.yarn
@@ -7,3 +7,5 @@
   More detailed testing of built systems may or may not be
   appropriate: a lot of more intricate testing may be beyond the scope
   of testing of vmdebootstrap.
+* Use the lava-submit.py script to submit a local file to LAVA using
+  the locally installed lava-dispatcher.
diff --git a/yarns/900-implements.yarn b/yarns/900-implements.yarn
index d4c6a7a..9806619 100644
--- a/yarns/900-implements.yarn
+++ b/yarns/900-implements.yarn
@@ -21,6 +21,10 @@ to allow them to run or not to run.
     IMPLEMENTS ASSUMING (fast|build) tests are requested
     test_requested "$MATCH_1"
 
+if `MIRROR` is set, use the default http mirror
+
+    IMPLEMENTS ASSUMING (local|http) tests are requested
+    mirror_requested "$MATCH_1"
 
 ## Building an image, or attempting to
 
@@ -37,6 +41,7 @@ but it's actually restricted to vmdebootstrap in the source tree.
     IMPLEMENTS WHEN the user runs vmdebootstrap (.*)
     PYTHONPATH="$SRCDIR" "$SRCDIR/bin/vmdebootstrap" \
         --image "$IMAGE" \
+        --mirror ${MIRROR:=http://http.debian.net/debian/} \
         --size "$IMAGE_SIZE" \
         $MATCH_1
 
@@ -82,11 +87,11 @@ the image.
 
 Check the partition table on the image.
 
-    IMPLEMENTS THEN the image has one partition
+    IMPLEMENTS THEN the partition count of the image is (\d+)
     parted --script "$IMAGE" print |
         sed '1,/^Number/d' |
         grep -c . |
-        grep -Fx 1
+        grep -Fx $MATCH_1
 
 Check partition boot flag.