From 66614ed9bc09d6ad817df655cd7249c2104b7ac7 Mon Sep 17 00:00:00 2001 From: Neil Williams Date: Sat, 7 Nov 2015 13:27:03 +0000 Subject: Add docs on the yarn tests and more build tests --- doc/overview.rst | 101 ++++++++++++++++++++++++++++++++++------ yarns/300-slow-build-tests.yarn | 21 ++++++++- yarns/800-future.yarn | 2 + yarns/900-implements.yarn | 9 +++- 4 files changed, 115 insertions(+), 18 deletions(-) diff --git a/doc/overview.rst b/doc/overview.rst index 0ae0e2f..c38dbfd 100644 --- a/doc/overview.rst +++ b/doc/overview.rst @@ -173,21 +173,6 @@ Peformance of: none, simple, meliae, or heapy (default: simple) --memory-dump-interval=SECONDS make memory profiling dumps at least SECONDS apart - -Developing -********** - -.. index:: pre-commit - -.. _pre_commit_hook: - -Testing vmdebootstrap from git -============================== - -:: - - ln -s ../../pre-commit.sh .git/hooks/pre-commit - .. index:: networking .. _networking: @@ -415,3 +400,89 @@ a specific offset instead of using a normal partition will **not** be supportable by vmdebootstrap. Similarly, devices which support hypervisor will only be supported using virtual machine images, unless the bootloader can be executed from a normal partition. + +.. index:: developing + +.. _developing: + +Developing +********** + +.. index:: pre-commit + +.. _pre_commit_hook: + +Testing vmdebootstrap from git +============================== + +``vmdebootstrap`` uses ``yarn`` for the test suite, available in the +`cmdtest `_ package. YARN +is a scenario testing tool. Scenarios are written in mostly human +readable language, however, they are not free form text. For more +information on YARN see `the homepage `_:: + + $ sudo apt -y install cmdtest + +All commits must pass at least the fast tests. All merges into master +need to pass a full test. All additions of new functionality must add +fast and build tests - fast tests for any new options and build tests +which exercise the new functionality. Build tests can add checks for +particular support on the machine running the test and skip if not +found or add new environment settings to selectively run some build +tests instead of all. + +If no arguments are given, the full test suite will be executed:: + + $ yarns/run-tests + +.. warning:: Do not run the full test suite if your connection to a + Debian mirror is limited or metered. Each build requires a minimum + of 2Gb free space in tmpfs. A full test takes at least 10 minutes. + +When limiting the run to specific tests, each ``--env`` option needs +to be specified separately:: + + $ sudo yarns/run-tests --env TESTS=build --env MIRROR=http://mirror/debian + +pre-commit +---------- + +All vmdebootstrap developers need to run the fast tests as a pre-commit +hook - any patches which fail this test will be rejected:: + + $ ln -s ../../pre-commit.sh .git/hooks/pre-commit + +The pre-commit hook just runs the fast tests which do not require +``sudo``. + +Fast tests +----------- + +The fast checks validate the handling of incompatible option arguments:: + + $ yarns/run-tests --env TESTS=fast + +Fast tests typically take a few seconds to run. + +Build tests +----------- + +The slow / build tests build multiple images and use ``sudo`` - a local +mirror is strongly recommended. + +:: + + $ sudo yarns/run-tests --env TESTS=build --env MIRROR=http://mirror/debian + +If ``MIRROR`` is not specified, a default mirror of ``http://http.debian.net/debian/`` +will be used. + +LAVA tests +---------- + +There is an example file:`lava-submit.py` script which can be edited +to automatically submit QEMU tests to a specified LAVA instance. The +images themselves will use local ``file://`` URLs and therefore the +``lava-dispatcher`` needs to be installed locally. Configuring LAVA +for these tests is a separate topic - please ask on the `vmdebootstrap +mailing list `_. diff --git a/yarns/300-slow-build-tests.yarn b/yarns/300-slow-build-tests.yarn index 41d11a3..2ceae72 100644 --- a/yarns/300-slow-build-tests.yarn +++ b/yarns/300-slow-build-tests.yarn @@ -14,7 +14,26 @@ These tests are slow, since building images is slow. GIVEN user wants to build an image FOO.img that is 2GiB in size WHEN the user runs vmdebootstrap --sparse --extlinux THEN the image has the correct size - AND the image has one partition + AND the partition count of the image is 1 AND partition 1 has the boot flag set AND partition 1 has an ext4 filesystem AND partition 1 has file /etc/debian_version matching ^8\..*$ + + SCENARIO build a Debian 8 image with grub + ASSUMING build tests are requested + GIVEN user wants to build an image FOO.img that is 2GiB in size + WHEN the user runs vmdebootstrap --grub + THEN the image has the correct size + AND the partition count of the image is 1 + AND partition 1 has the boot flag set + AND partition 1 has an ext4 filesystem + AND partition 1 has file /boot/grub/grub.cfg matching ^### BEGIN /etc/grub.d/00_header ###$ + + SCENARIO build a Debian 8 image with uefi + ASSUMING build tests are requested + GIVEN user wants to build an image FOO.img that is 2GiB in size + WHEN the user runs vmdebootstrap --grub --use-uefi + THEN the image has the correct size + AND the partition count of the image is 2 + AND partition 1 has an vfat filesystem + AND partition 2 has file /boot/grub/grub.cfg matching ^### BEGIN /etc/grub.d/00_header ###$ diff --git a/yarns/800-future.yarn b/yarns/800-future.yarn index 1368b8f..a6ca722 100644 --- a/yarns/800-future.yarn +++ b/yarns/800-future.yarn @@ -7,3 +7,5 @@ More detailed testing of built systems may or may not be appropriate: a lot of more intricate testing may be beyond the scope of testing of vmdebootstrap. +* Use the lava-submit.py script to submit a local file to LAVA using + the locally installed lava-dispatcher. diff --git a/yarns/900-implements.yarn b/yarns/900-implements.yarn index d4c6a7a..9806619 100644 --- a/yarns/900-implements.yarn +++ b/yarns/900-implements.yarn @@ -21,6 +21,10 @@ to allow them to run or not to run. IMPLEMENTS ASSUMING (fast|build) tests are requested test_requested "$MATCH_1" +if `MIRROR` is set, use the default http mirror + + IMPLEMENTS ASSUMING (local|http) tests are requested + mirror_requested "$MATCH_1" ## Building an image, or attempting to @@ -37,6 +41,7 @@ but it's actually restricted to vmdebootstrap in the source tree. IMPLEMENTS WHEN the user runs vmdebootstrap (.*) PYTHONPATH="$SRCDIR" "$SRCDIR/bin/vmdebootstrap" \ --image "$IMAGE" \ + --mirror ${MIRROR:=http://http.debian.net/debian/} \ --size "$IMAGE_SIZE" \ $MATCH_1 @@ -82,11 +87,11 @@ the image. Check the partition table on the image. - IMPLEMENTS THEN the image has one partition + IMPLEMENTS THEN the partition count of the image is (\d+) parted --script "$IMAGE" print | sed '1,/^Number/d' | grep -c . | - grep -Fx 1 + grep -Fx $MATCH_1 Check partition boot flag. -- cgit v1.2.1