Merge branch 'docs' into 'main'

better

See merge request larswirzenius/v-i!7

7 files changed, 722 insertions, 284 deletions

diff --git a/README.md b/README.md
index 5640fa6..0e5ba18 100644
--- a/README.md
+++ b/README.md
@@ -1,94 +1,333 @@
-Install Debian using vmdb2 (v-i) onto my Thinkpad x220 laptop
-=============================================================================
+# v-i---a Debian installer using vmdb2
 
-WARNING!
------------------------------------------------------------------------------
+**WARNING: Running v-i is like waking up after an alien invasion, in a
+post-apocalyptic world, with everything you knew or owned gone
+forever. When you run v-i, it *will* wipe away everything on that
+computer. All volume groups will be deleted, all storage drives
+emptied. Any existing partitions will be lost. Forget any data you
+used to have, and operating systems you used to have installed. If you
+don't know what you're doing, leave.**
 
-**Warning:** this is an operating system installer. It will overwrite your
-hard drive and annihilate anything that used to be there. There are no
-safety rails. You will lose all your data. Make sure your backups
-work. You have been warned.
+**v-i** installs Debian onto a PC. It's entirely non-interactive,
+dangerous, unhelpful, and may only work for the author's PCs. The
+[author][] wrote it so that repeated installations would be less of a
+chore, for them, than using the official Debian installer. (Actually,
+the author thought it'd be a quick, easy hack, and was too stubborn to
+give up, when it turned out to be a bit tricky.)
 
+[**vmdb2**][] is a program to create a disk image virtual machines
+with Debian, by the same author. It "installs Debian" to a file
+representing a hard drive. It's basically [debootstrap][], except the
+target is a disk image instead of a directory. vmdb2 has been quite
+useful for generating virtual machine images. It's also used to create
+[Debian images for Raspberry Pis][]. **v-i** uses **vmdb2** to install
+onto bare metal hardware.
 
-Introduction
------------------------------------------------------------------------------
+To use **v-i** to install Debian on a PC:
 
-This is a hack for my own amusement. It's not meant to be taken
-seriously.
+* Boot the target machine off a live system that has **v-i**
+  installed.
+  - the author uses a USB stick with an image built with the
+    [`build-installer.sh`][] script
+  - the author logs into the installer system via SSH
+* Create a v-i target specification file. See below for an example.
+* Run the command: `v-i --verbose exolobe5.yaml`
+* See `installer.log` for what happened during the installation.
 
-This is a rudimentary installer of Debian based on vmdb2. It has
-nothing to do with debian-installer, the official Debian installer,
-known as d-i. I use d-i, but would like something better:
+Example target specification file (see [exolobe5.yaml](exolobe5.yaml)
+for the version used in production):
 
-* I'd like something I can easily modify. d-i requires building
-  special udeb packages for any software that's to be part of the
-  installer. v-i is happy with normal debs.
+```yaml
+drive: /dev/nvme0n1
+extra_drives:
+  - /dev/nvme1n1
+hostname: exolobe5
+extra_playbooks:
+  - exolobe5.yml
+ansible_vars:
+  user_pub: |
+   ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPQe6lsTapAxiwhhEeE/ixuK+5N8esCsMWoekQqjtxjP liw personal systems
+extra_lvs:
+  - name: vms
+    size: 1T
+    fstype: ext4
+    mounted: /mnt/vms
+```
 
-* d-i uses preseeding for automating an installation. Preseeding means
-  providing answers, in a file, to questions the package may ask
-  during its installation. This is fine, if a little cumbersome, but
-  only helps to answer questions the packages ask. v-i lets you have
-  the full power of Ansible during initial installation.
+Explanation:
 
-On the other hand, d-i is mature software and tested by a very large
-number of people, on a very large number of different hardware. v-i
-can just barely install my own Thinkpad x220 laptop, using UEFI.
+* `hostname`---the hostname of the installed system. This is so that
+  when the installed system boots, and gets a network address using
+  DHCP, it can provide a name. The author's home network setup
+  automatically adds that hostname to the internal DNS. This avoids a
+  manual DNS configuration step, and the author is lazy.
+* `drive`---the main drive to install to. This will have the EFI and
+  `/boot` partitions, and have GRUB installed. The rest of the drive
+  will be a physical volume for LVM2.
+* `extra_drives`---any additional physical volumes for LVM2. These
+  will not be partitioned.
+* `luks`---the password for full disk encryption for LVM2 physical
+   volumes. If not set, LUKS is not used. This is a single, fixed
+   password that is in cleartext. You are expected to change it after
+   the system is installed and boots. If you'd rather use, say, a
+   hardware token's challenge/response feature or TPM for LUKS, that's
+   better done on a running system.
+* `extra_playbooks`---additional Ansible playbooks to use on the
+  installed system. **v-i** comes with a "standard playbook" (in
+  [`std.yml`][]) that it uses unconditionally, to set up a "standard
+  system" that the author likes. You can provide additional playbooks,
+  for additional configuration at installation time.
+* `ansible_vars`---variables to set for Ansible playbooks.
 
-v-i is not currently expected to work with any other hardware. It
-might not even work on your x220.
+With all this configuration in a file, which you can keep in git, you
+can install a base system repeatedly to a specific computer, and do it
+the same way every time.
 
+(Caveat: **v-i** does nothing to configure your BIOS/UEFI. It can't.
+You have to manually configure it the way you want it to be. For
+example, one of the author's machines needs to have its boot order
+adjusted after every operating system installation. It's quite
+tedious.)
 
-Architecture
------------------------------------------------------------------------------
+[**vmdb2**]: https://vmdb2.liw.fi/
+[debootstrap]: https://wiki.debian.org/Debootstrap
+[Debian images for Raspberry Pis]: https://raspi.debian.net/
+[`build-installer.sh`]: build-installer.sh
+[`v-i`]: v-i
+[`std.yml`]: std.yml
+[author]: https://liw.fi/
+[Debian installer]: https://www.debian.org/devel/debian-installer/
+[preseed files]: https://wiki.debian.org/DebianInstaller/Preseed
+[udeb]: https://en.wikipedia.org/wiki/Deb_(file_format)
 
-The vmdb2 installer builds a bootable disk image with Debian. The
-image boots with UEFI, and has vmdb2 installed. You write the image to
-a USB drive, boot off that, and run vmdb2 in the booted system to
-install Debian onto the hard drive.
+## Motivation
 
-Thus:
+The official [Debian installer][] is often referred to as _d-i_. It
+works quite well, for almost any hardware Debian can run on, and
+supports a lot of languages, and if flexible enough to be acceptable
+for nearly every use case. Millions upon millions of people are
+satisfied users of it. It is a great achievement of Debian, and the
+people of the `debian-boot` team.
 
-- build system -> installer image
-- installer image -> USB drive
-- boot from USB drive
-- run vmdb2 to install onto laptop hard drive
-  - the /root/x220.sh script does this, and installs a basic Debian
-    system using my Ansible playbooks
-- boot off the laptop hard drive
+However, the **v-i** author felt it could be improved upon for them:
 
-The image building uses vmdb2 as well. Things get a little recursive.
+* d-i is not entirely easy to understand and modify. It requires
+  building special [udeb][] packages for any software that's to be
+  part of the installer environment, which makes it harder to make
+  changes without collaboration from maintainers of those packages.
+  The architecture of d-i is also a little non-linear. d-i also needs
+  to support a very wide variety of hardware and use cases, which has
+  made it large and complex.
 
+  **v-i** is happy with normal deb packages, and is a thin Python
+  wrapper script around **vmdb2**, making it reasonably easy to
+  understand and change.
 
-Hacking
------------------------------------------------------------------------------
+* d-i is primarily meant to be used interactively, but it does support
+  [preseed files][] for automating an installation. Preseeding means
+  providing answers, in a file, to questions a package being installed
+  may ask during its installation. This is fine, if a little
+  cumbersome, but only helps to answer questions the packages ask when
+  installed.
 
-The main files are:
+  **v-i** lets you have the full power of Ansible during initial
+  installation.
 
-* v-i.vmdb -- the vmdb2 specification file for the installer image
-* v-i.yml -- the Ansible playbook used by v-i.vmdb
-* x220.vmdb -- the vmdb2 specification file to install onto an x220;
-  this is used from the system booted from the image built by v-i.vmdb
-* x220.yml -- the Ansible playbook used by x220.vmdb
+If **v-i** isn't suitable for your uses, that's OK. The author is
+happy with his toy.
 
-To build the installer image, run (the cached tarball will be created
-if it doesn't exist, but you need to choose the location):
 
-~~~sh
-sudo ./v-i.sh /path/to/rootfs/tarball/for/caching/debootstrap.tar.gz
-~~~
+## Architecture
 
-To write the resulting v-i.img to a USB drive:
+**vmdb2** is given a sequence of _steps_ to execute: create this
+partition, make that file system, install those packages, etc.
+**vmdb2** runs the steps against a disk image or physical hard drive,
+with a chroot of the file systems, to do things like installing a
+package in the system being installed.
 
-~~~sh
-sudo dd if=v-i.img of=/dev/path/to/usb/drive/device status=progress oflag=direct
-~~~
+**v-i** defines a fairly minimal _standard install_, whose goal is to
+get the target system into a state where it boots from its own,
+internal storage, and where the rest of the system configuration can
+be finished using your configuration management system of choice.
 
-Plug in the USB drive to the laptop, boot off that. Log into the
-installer as root (no password), and run
+While **vmdb2** can, and does, run Ansible to configure the system
+being installed, in practice some things work better if most
+configuration is done to a running system. The goal of **v-i** is to
+get a system into that state as quickly and easily as possible. For
+example, the Ansible module to set a hostname on a system with systemd
+requires systemd to be running. That's awkward while the system is
+still being installed in a chroot.
 
-~~~sh
-./x220.sh
-~~~
+Thus, **v-i** does the following:
 
-Reboot laptop from its hard drive and you should have a bare bones
-Debian system installed.
+* delete any trace of LVM2 from all drives, wipe all SSDs, and
+  generally reset the system to as close to a blank state as possible
+* create a partition table ("label") on the target drive
+* create EFI and boot partitions, needed to boot with UEFI and LUKS
+* create a physical volume for LVM2, and a logical volume for the root
+  file system
+  - add any additional drives as physical volumes to the volume group
+  - optionally use LUKS for full disk encryption for each physical
+    volume
+* install the Debian base system
+  - run `debootstrap`, install a boot loader, and create fstab and
+    `crypttab` files
+* run the standard Ansible playbook (see [`std.yml`][])
+  - set hostname
+  - set keyboard layout
+  - configure networking (using systemd-networkd)
+  - install an SSH server
+  - add a chosen SSH public key to the root user's authorized keys
+    file
+* run any additional playbooks
+
+**v-i** uses the **vmdb2** caching feature, where the results of
+`debootstrap` and some other steps get stored in a compressed tar
+archive. On subsequent runs, if the cache file exists, it's unpacked,
+instead of running the commands. This speeds things up a bit: running
+**v-i** without the cache file takes the author about 5 minutes; with
+the cache file it takes about 1.5 minutes. This matters if there is a
+need to do many installations.
+
+
+## Hacking
+
+The main files of **v-i** are:
+
+* [`v-i`][]---the actual installer, a Python script
+* [`std.yml`][]---the Ansible playbook to configure a standard install
+
+Also, to build an image to boot off for running the installer:
+
+* `build-installer.sh`---build a disk image where **v-i** can be run
+  - put image on a USB drive, boot off that drive, run installer
+  - note that you can use any live image with **vmdb2** installed; the
+    image built with this is just the easiest for the author
+* `installer.vmdb`---the **vmdb2** specification file for creating the
+  installer image
+* `installer.yml`---the Ansible playbook for creating the installer
+  image
+
+You'll want to build your own installer image, if you want to log into
+it over SSH, so that you can install your own SSH key. If you log in
+via the Linux virtual console, you don't need that: there's no
+password for root (but SSH login for root doesn't work with
+passwords).
+
+You probably mostly only need to modify `v-i` and `std.yml`. The rest
+is to get you and your target machine into a state where you can run
+the installer.
+
+
+
+## FAQ
+
+This section is prescient: the author hasn't been asked any questions
+yet, but expects the following to be asked.
+
+### What version of Debian does v-i install?
+
+**v-i** installs Debian 11 (bullseye).
+
+### What about other releases of Debian?
+
+The Debian 11 (bullseye) release is the earliest release the author
+has gotten to work with **v-i**, and is the only release the author is
+installing on bare metal systems. Later versions of Debian may work,
+we will see.
+
+### Is only UEFI supported?
+
+Yes.
+
+### What about multi-boot?
+
+**v-i** doesn't support installing more than one operating system on
+one computer.
+
+### What about installing something else than Debian?
+
+The author only cares about Debian, but in principle, fairly little of
+**vmdb2** and **v-i** are specific to Debian. It should be possible to
+add support for other operating systems to be installed, at least ones
+based on Linux. If you're interested, you need to change or replace at
+least the following steps in **vmdb2** code, and then change the
+[`v-i`][] script to generate a specification using those steps:
+
+* `debootstrap`---install base operating system into a directory
+  - after this step, all the files in a base installation should be in
+    the specified directory tree, except the boot loader and kernel
+  - could probably be replaced with providing **v-i** with a pre-built
+    cache tar archive
+* `apt`---install packages
+  - whatever package manager the system has probably works
+  - you can probably run the package manager from a chroot step
+* `grub`---install boot loader
+  - this chooses the appropriate Debian package automatically
+  - might possibly be doable as a chroot step
+  - this is likely the trickiest bit: booting is _intricate_
+* `cryptsetup`---format a drive for full disk encryption
+  - this just runs the `cryptsetup` program and tells the fstab step
+    to create a crypttab file
+  - might just work
+* `vgcreate` and `lvcreate`---create LVM2
+  - these just run the relevant LVM2 commands
+  - might just work
+
+### What about other kinds of computers than PCs?
+
+The author only uses 64-bit PC computers (`amd64` arhitecture in
+Debian; also known as x86-64). **v-i** may well work for other
+kinds of computers, as long as they can boot off an installer image
+("live image"), and use GRUB for booting. The author would be
+interested to hear if that is the case.
+
+### Why is the LUKS password in cleartext?
+
+It would be ideal if **v-i** (or **vmdb2**) got the LUKS password for
+full disk encryption in a secure way from a secure source, but that
+turned out to be tricky to do. The author felt it was too tricky to do
+well in the installer environment, while it's pretty easy to do in a
+running system. Thus, the cleartext password _in the installer_ is a
+compromise. You're expected to change the password after the
+installation is done.
+
+It would be possible to ask the person doing the installation to enter
+the password manually, but this would mean the installation would not
+be fully automated. The author didn't want that.
+
+### Do I have to use Ansible?
+
+No. Use whatever you like once you've installed a system with **v-i**
+and booted it. **v-i** itself uses Ansible, because that was easy for
+the author to use.
+
+### I'd like to use v-i, but I need changes
+
+If you can make the changes yourself, go ahead: this is free and open
+source software, have at it. If you don't have the skill or time to
+make changes yourself, you'll need to find someone else to make them.
+This might require paying them.
+
+The author is, unfortunately, probably not willing to spend their free
+time to make changes that don't benefit them directly, for free.
+Sorry. They _are_ willing to review and merge changes that would make
+the software better.
+
+
+# Legalese
+
+Copyright 2018-2022  Lars Wirzenius
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
diff --git a/exolobe5.yaml b/exolobe5.yaml
new file mode 100644
index 0000000..df59ac0
--- /dev/null
+++ b/exolobe5.yaml
@@ -0,0 +1,14 @@
+drive: /dev/nvme0n1
+extra_drives:
+  - /dev/nvme1n1
+hostname: exolobe5
+extra_playbooks:
+  - exolobe5.yml
+ansible_vars:
+  user_pub: |
+   ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPQe6lsTapAxiwhhEeE/ixuK+5N8esCsMWoekQqjtxjP liw personal systems
+extra_lvs:
+  - name: vms
+    size: 1T
+    fstype: ext4
+    mounted: /mnt/vms
diff --git a/exolobe5.yml b/exolobe5.yml
new file mode 100644
index 0000000..fad2643
--- /dev/null
+++ b/exolobe5.yml
@@ -0,0 +1,46 @@
+- hosts: image
+  tasks:
+    - name: "delete external.network"
+      file:
+        path: /etc/systemd/network/external.network
+        state: absent
+    - name: "add br0.netdev"
+      copy:
+        content: |
+          [NetDev]
+          Name=br0
+          Kind=bridge
+        dest: /etc/systemd/network/br0.netdev
+    - name: "bind eth0 to bridge br0"
+      copy:
+        content: |
+          [Match]
+          Name=eth0
+
+          [Network]
+          Bridge=br0
+        dest: /etc/systemd/network/bind.network
+    - name: "have br0 get address with DHCP"
+      copy:
+        content: |
+          [Match]
+          Name=br0
+
+          [Network]
+          DHCP=yes
+        dest: /etc/systemd/network/br0.network
+    - name: "format LV for VM images"
+      filesystem:
+        dev: /dev/vg0/vms
+        fstype: ext4
+    - name: "create mount point for VM images"
+      file:
+        state: directory
+        path: /mnt/vms
+    - name: "mount LV for VM images"
+      mount:
+        src: /dev/vg0/vms
+        path: /mnt/vms
+        fstype: ext4
+        opts: defaults
+        state: mounted
diff --git a/installer.vmdb b/installer.vmdb
index 218a809..9379f1b 100644
--- a/installer.vmdb
+++ b/installer.vmdb
@@ -15,7 +15,7 @@ steps:
     device: "{{ output }}"
     start: 1G
     end: 100%
-    tag: /
+    tag: root
 
   - kpartx: "{{ output }}"
 
@@ -23,24 +23,24 @@ steps:
     partition: efi
 
   - mkfs: ext4
-    partition: /
+    partition: root
 
-  - mount: /
+  - mount: root
 
-  - unpack-rootfs: /
+  - unpack-rootfs: root
 
   - debootstrap: bullseye
     mirror: http://deb.debian.org/debian
-    target: /
+    target: root
     unless: rootfs_unpacked
 
   - apt: install
     packages:
       - linux-image-amd64
-    fs-tag: /
+    fs-tag: root
     unless: rootfs_unpacked
 
-  - cache-rootfs: /
+  - cache-rootfs: root
     unless: rootfs_unpacked
 
   - apt: install
@@ -55,18 +55,23 @@ steps:
       - lvm2
       - cryptsetup
       - cryptsetup-initramfs
-#      - pass
       - dosfstools
-#      - emacs
-#      - gpg
-#      - scdaemon
-    tag: /
+    tag: root
 
-  - ansible: /
-    playbook: v-i.yml
+  - ansible: root
+    playbook: installer.yml
 
-  - fstab: /
+  - fstab: root
+
+  - copy-file: /root/vi
+    src: v-i
+    perm: 0755
+
+  - copy-file: /root/std.yml
+    src: std.yml
+
+  - zerofree: root
 
   - grub: uefi
-    tag: /
+    tag: root
     efi: efi
diff --git a/std.yml b/std.yml
index f2911ad..cb62c82 100644
--- a/std.yml
+++ b/std.yml
@@ -1,4 +1,7 @@
 # Ansible playbook to install stuff for a standard install with v-i.
+# You should inspect the user_* variables at the end, and override
+# them with "ansible_vars" in the system spec file. v-i sets the
+# hostname variable automatically.
 
 - hosts: image
   tasks:
@@ -8,10 +11,6 @@
           {{ hostname }}
         dest: /etc/hostname
 
-    - name: "disable root password"
-      shell: |
-        passwd -l root
-
     - name: "create ~root/.ssh"
       file:
         state: directory
@@ -56,58 +55,32 @@
           {{ user_locale }}
         dest: /etc/profile.d/finnish.sh
 
-    - name: "configure Ethernet networking"
+    - name: "remove ifupdown"
+      apt:
+        name: ifupdown
+        state: absent
+
+    - name: "configure networkd"
       copy:
         content: |
-          auto eth0
-          iface eth0 inet dhcp
-          iface eth0 inet6 auto
-        dest: /etc/network/interfaces.d/wired
+          [Match]
+          Name=eth0
 
-    # - name: "restrict root logins over ssh"
-    #   lineinfile:
-    #     path: /etc/ssh/sshd_config
-    #     regex: "#* *PasswordAuthentication"
-    #     line: "PasswordAuthentication no"
+          [Network]
+          DHCP=yes
+        dest: /etc/systemd/network/external.network
+
+    - name: "enable networkd"
+      systemd:
+        name: systemd-networkd
+        enabled: yes
 
   vars:
-    hostname: v-i
-    user_pub: |
-      ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPQe6lsTapAxiwhhEeE/ixuK+5N8esCsMWoekQqjtxjP liw personal systems
+    ansible_python_interpreter: /usr/bin/python3
+
+    # You may want to override these.
     user_locale: |
       LC_CTYPE=fi_FI.UTF8
     user_keyboard_model: pc105
     user_keyboard_layout: fi
     user_console_codeset: Lat15
-
-    ansible_python_interpreter: /usr/bin/python3
-    ci_prod_signing_key: |
-        -----BEGIN PGP PUBLIC KEY BLOCK-----
-
-        mQINBFrLO7kBEADdz6mHstYmKU5Dp6OSjxWtWaqTDOX1sJdmmaIK/9EKVIH0Maxp
-        5kvVO5G6mULLAjv/kLG0MxasHPrq8I2A/y8AqKAGVL8QelwLjQMIFZ30/VbGQPHS
-        +T5TZXEnoQtNce1GUhFwJ38ZyjjwHBFV9tSec7rZ2Q3YeM3nNnGPf6DacXGfEOPO
-        HIN4sXAN2hzNXNjKRzTIvxQseb6nr7afUh/SlZ3yhQOCrIzmYlD7tP9WJe7ofL0p
-        JY4pDQYw8rT6nC2BE/ioemh84kERCT1vCe+OVFlSRuMlqfEv+ZpKQ+itOmPDQ/lM
-        jpUm1K2hrW/lWpxT/ZxHKo/w1K36J5WshgMZxfUu5BMCL9LMqMcrXNhNjDMfxDMM
-        3yBPOvQ4ls6fecOZ/bsFo1p8VzMk/w/eG8vPs5yuNa5XxN95yFMXoOHGb5Xbu8D4
-        6yiW+Af70LbiSNpGdmNdneiGB2fY38NxBukPw5u3S5qG8HedSmMr1RvSr5kHoAAe
-        UbOY+BYaaKsTAT7+1skUW1o3FJSqoRKCHAzTsMWC6zzhR8hRn7jVrrguH1hGbqq5
-        TZSCFQZExuTJ7uXrTLG0WoBXIjB5wWNcSeXn8myUWYB51nJNF4tJBouZOz9JwWGl
-        kiAQkrHnBttLQWdW9FyjbIoTZMtpvVx+m6ObGTGdGL1cNlLAvWprMXGc+QARAQAB
-        tDJJY2sgQVBUIHJlcG9zaXRvcnkgc2lnbmluZyBrZXkgKDIwMTgpIDxsaXdAbGl3
-        LmZpPokCTgQTAQgAOBYhBKL1uyDoXyxUH3O717Wr+TZVS6PGBQJayzu5AhsDBQsJ
-        CAcCBhUICQoLAgQWAgMBAh4BAheAAAoJELWr+TZVS6PGB5QQANTcikhRUHwt9N4h
-        dGc/Hp6CbqdshMoWlwpFskttoVDxQG5OAobuZl5XyzGcmja1lT85RGkZFfbca0IZ
-        LnXOLLSAu51QBkXNaj4OhjK/0uQ+ITrvL6RQSXNgHiUTR/W2XD1GIUq6nBqe2GSN
-        31S1baYKKVj5QIMsi7Dq8ls3BBXuPCE+xTSaNmGWjes2t9pPidcRvxsksCLY1qgw
-        P1GFXBeMkBQ29kBP87SUL15SIk7OiQLlEURCy5iRls5rt/YEsdEpRWIb0Tm5Nrjv
-        2M3VM+iBhfNXTwj0rJ34mlycF1qQmA7YcTEobT7z587GPY0VWzBpQUnEQj7rQWPM
-        cDYY0b+I6kQ8VKOaL4wVAtE98d7HzFIrIrwhTKufnrWrVDPYsmLZ+LPC1jiF7JBD
-        SR6Vftb+SdDR9xoE1yRuXbC6IfoW+5/qQNrdQ2mm9BFw5jOonBqchs18HTTf3441
-        6SWwP9fY3Vi+IZphPPi0Gf85oMStgnv/Wnw6LacEL32ek39Desero/D8iGLZernK
-        Q2mC9mua5A/bYGVhsNWyURNFkKdbFa+/wW3NfdKYyZnsSfo+jJ2luNewrhAY7Kod
-        GWXTer9RxzTGA3EXFGvNr+BBOOxSj0SfWTl0Olo7J5dnxof+jLAUS1VHpceHGHps
-        GSJSdir7NkZidgwoCPA7BTqsb5LN
-        =dXB0
-        -----END PGP PUBLIC KEY BLOCK-----
diff --git a/tox.ini b/tox.ini
new file mode 100644
index 0000000..5cf16d3
--- /dev/null
+++ b/tox.ini
@@ -0,0 +1,3 @@
+[flake8]
+max-line-length = 999
+ignore = E203, E302, W503
\ No newline at end of file
diff --git a/v-i b/v-i
index ce26933..506a807 100755
--- a/v-i
+++ b/v-i
@@ -2,12 +2,13 @@
 
 import argparse
 import glob
+import logging
 import os
 import shutil
+import subprocess
 import sys
 import tempfile
 import yaml
-from subprocess import run
 
 
 verbose = False
@@ -16,6 +17,12 @@ verbose = False
 def log(msg):
     if verbose:
         print("INSTALLER:", msg)
+    logging.info(msg)
+
+
+def run(argv, **kwargs):
+    log(f"RUN: {argv} {kwargs}")
+    return subprocess.run(argv, **kwargs)
 
 
 def physical_volumes():
@@ -82,15 +89,20 @@ def is_luks(path):
     return p.returncode == 0
 
 
-def clean_up_disks():
-    log("clean up disks from old installs")
+def clean_up_disks(drives):
+    log(f"clean system LVM2 and drives from everything: {drives}")
 
     mounts = mount_points()
     pvs = physical_volumes()
     lvs = logical_volumes()
     vgs = volume_groups()
 
+    log(f"PVs: {pvs}")
+    log(f"VGs: {vgs}")
+    log(f"LVs: {lvs}")
+
     for lv in lvs:
+        log(f"LV: {lv}")
         for m in find_mount_points(mounts, lv["path"]):
             log(f"unmount {m['mount']}")
             run(["umount", m["mount"]], capture_output=True)
@@ -110,115 +122,224 @@ def clean_up_disks():
             log(f"open LUKS volume {mapping} (just in case it is one)")
             run(["cryptsetup", "close", mapping], check=False, capture_output=True)
 
+    for drive in drives:
+        log(f"blkdiscard {drive}")
+        run(
+            ["blkdiscard", "--force", drive],
+            check=True,
+            capture_output=True,
+        )
 
-def vmdb_spec(cryptsetup_password, playbook, extra_vars):
-    device = "{{ image }}"
-    spec = {
-        "steps": [
-            {
-                "mklabel": "gpt",
-                "device": device,
-            },
-            {
-                "mkpart": "primary",
-                "device": device,
-                "start": "0%",
-                "end": "500M",
-                "tag": "efi",
-            },
-            {
-                "mkpart": "primary",
-                "device": device,
-                "start": "500M",
-                "end": "1G",
-                "tag": "boot",
-            },
-        ]
+
+def mklabel(device):
+    return {
+        # Create a GPT partition table. It works well for all modern
+        # systems.
+        "mklabel": "gpt",
+        "device": device,
+    }
+
+
+def mkpart(device, tag, start, end):
+    return {
+        "mkpart": "primary",
+        "device": device,
+        "start": start,
+        "end": end,
+        "tag": tag,
+    }
+
+
+def mkfs(tag, fstype):
+    return {
+        "mkfs": fstype,
+        "partition": tag,
+    }
+
+
+def cryptsetup(tag, password, name):
+    return {
+        "cryptsetup": "cryptsetup0",
+        "password": password,
+        "name": name,
+    }
+
+
+def vgcreate(tag, drives):
+    return {
+        "vgcreate": tag,
+        "physical": drives,
+    }
+
+
+def lvcreate(vg, name, size):
+    return {
+        "lvcreate": vg,
+        "name": name,
+        "size": size,
     }
 
+
+def mount(tag, dirname=None, mount_on=None):
+    d = {"mount": tag}
+    if dirname is not None:
+        d["dirname"] = dirname
+    if mount_on is not None:
+        d["mount-on"] = mount_on
+    return d
+
+
+def unpack_rootfs(tag):
+    return {
+        "unpack-rootfs": tag,
+    }
+
+
+def cache_rootfs(tag):
+    return {
+        "cache-rootfs": tag,
+        "unless": "rootfs_unpacked",
+    }
+
+
+def debootstrap(tag):
+    return {
+        "debootstrap": "bullseye",
+        "mirror": "http://deb.debian.org/debian",
+        "target": tag,
+        "unless": "rootfs_unpacked",
+    }
+
+
+def apt(tag, packages, unless=True):
+    d = {
+        "apt": "install",
+        "packages": [
+            "console-setup",
+            "dosfstools",
+            "linux-image-amd64",
+            "locales-all",
+            "lvm2",
+            "psmisc",
+            "python3",
+            "ssh",
+            "strace",
+        ],
+        "tag": tag,
+    }
+    if unless:
+        d["unless"] = "rootfs_unpacked"
+    return d
+
+
+def virtual_filesystems(tag):
+    return {
+        "virtual-filesystems": "root",
+    }
+
+
+def fstab(tag):
+    return {
+        "fstab": tag,
+    }
+
+
+def grub(device, root, efi):
+    return {
+        "grub": "uefi",
+        "tag": root,
+        "efi": efi,
+        "quiet": True,
+        "image-dev": device,
+    }
+
+
+def vmdb_spec(system, ansible_vars):
+    device = "{{ image }}"
+    steps = [
+        mklabel(device),
+        # Create a partition for UEFI to boot from.
+        mkpart(device, "efi", "0%", "500M"),
+        # Create a separate /boot partition, in case we want to have LUKS.
+        mkpart(device, "boot", "500M", "1G"),
+        # Format the EFI partition. This MUST be vfat.
+        mkfs("efi", "vfat"),
+        # Format /boot. This is conventionally ext2. This file system
+        # gets very little I/O, but it MUST be supported by GRUB, so
+        # ext2 seems like a nice, safe choice.
+        mkfs("boot", "ext2"),
+    ]
+
     # Set up pv0 for lvm2, either encrypted or cleartext.
-    if cryptsetup_password:
-        spec["steps"].extend(
+    if system.luks:
+        steps.extend(
             [
-                {
-                    "mkpart": "primary",
-                    "device": device,
-                    "start": "1G",
-                    "end": "100%",
-                    "tag": "cryptsetup0",
-                },
-                {
-                    "cryptsetup": "cryptsetup0",
-                    "password": cryptsetup_password,
-                    "name": "pv0",
-                },
+                mkpart(device, "cryptsetup0", "1G", "100%"),
+                cryptsetup("cryptsetup0", system.luks, "pv0"),
             ]
         )
+        for (i, drive) in enumerate(system.extra_drives):
+            steps.append(cryptsetup(f"cryptsetuo{i+1}", system.luks, f"pv{i+1}"))
     else:
-        spec["steps"].extend(
+        steps.append(mkpart(device, "pv0", "1G", "100%"))
+        for (i, drive) in enumerate(system.extra_drives):
+            steps.extend(
+                [
+                    mklabel(drive),
+                    mkpart(drive, f"pv{i+1}", "0%", "100%"),
+                ]
+            )
+
+    # Create file systems and install Debian.
+    steps.extend(
+        [
+            # Create an LVM2 volume group using pv0, which we create
+            # earlier. At this point, if LUKS is used, pv0 is the unlocked,
+            # open, cleartext block device.
+            vgcreate(
+                "vg0",
+                [f"pv{i}" for i in range(len([system.drive] + system.extra_drives))],
+            ),
+            # Create a 20 gigabyte LV for the root file system. That's big
+            # enough for a desktop system.
+            lvcreate("vg0", "root", "20G"),
+            # format the root file system. This gets a fair bit of use, so
+            # ext4 seems like a safe choice. If you wanted another file
+            # system, sorry.
+            mkfs("root", "ext4"),
+            # Mount the root file system.
+            mount("root"),
+            # Mount /boot on top of the root file system.
+            mount("boot", dirname="/boot", mount_on="root"),
+            # Mount /boot/efi.
+            mount("efi", dirname="/boot/efi", mount_on="boot"),
+        ]
+    )
+
+    # Add any additional LVs.
+    for lv in system.extra_lvs:
+        steps.extend(
             [
-                {
-                    "mkpart": "primary",
-                    "device": device,
-                    "start": "1G",
-                    "end": "100%",
-                    "tag": "pv0",
-                },
+                lvcreate("vg0", lv["name"], lv["size"]),
+                mkfs(lv["name"], "ext4"),
+                mount(lv["name"], dirname=lv["mounted"], mount_on="root"),
             ]
         )
 
-    # Create file systems and install Debian.
-    spec["steps"].extend(
+    steps.extend(
         [
-            {
-                "mkfs": "vfat",
-                "partition": "efi",
-            },
-            {
-                "mkfs": "ext2",
-                "partition": "boot",
-            },
-            {
-                "vgcreate": "vg0",
-                "physical": ["pv0"],
-            },
-            {
-                "lvcreate": "vg0",
-                "name": "root",
-                "size": "10G",
-            },
-            {
-                "mkfs": "ext4",
-                "partition": "root",
-            },
-            {
-                "mount": "root",
-            },
-            {
-                "mount": "boot",
-                "dirname": "/boot",
-                "mount-on": "root",
-            },
-            {
-                "mount": "efi",
-                "dirname": "/boot/efi",
-                "mount-on": "boot",
-            },
-            {
-                "unpack-rootfs": "root",
-            },
-            {
-                "debootstrap": "bullseye",
-                "mirror": "http://deb.debian.org/debian",
-                "target": "root",
-                "unless": "rootfs_unpacked",
-            },
-            {
-                "apt": "install",
-                "packages": [
+            # If we have a cached version of the installed system, unpack
+            # it now. Otherwise do nothing. Note that if you make any
+            # changes to the steps marked "unless: rootfs_unpacked", you
+            # have to remember to manually remove the cache file. v-i or
+            # vmdb2 won't do that automatically for you.
+            unpack_rootfs("root"),
+            debootstrap("root"),
+            apt(
+                "root",
+                [
                     "console-setup",
                     "dosfstools",
-                    "ifupdown",
                     "linux-image-amd64",
                     "locales-all",
                     "lvm2",
@@ -227,82 +348,120 @@ def vmdb_spec(cryptsetup_password, playbook, extra_vars):
                     "ssh",
                     "strace",
                 ],
-                "tag": "root",
-                "unless": "rootfs_unpacked",
-            },
-            {
-                "cache-rootfs": "root",
-                "unless": "rootfs_unpacked",
-            },
-            {
-                # This MUST be after the debootstrap step.
-                "virtual-filesystems": "root",
-            },
-            {
-                "fstab": "root",
-            },
-            {
-                # These MUST come after the fstab step so that they add the
-                # crypttab in the initramfs.
-                "apt": "install",
-                "packages": [
+            ),
+            # If we didn't unpack an existing cache archive, make one now.
+            # Otherwise, skip this step.
+            cache_rootfs("root"),
+            # This MUST be after the debootstrap step.
+            virtual_filesystems("root"),
+            # Create /etc/fstab (and, if LUKS is used, /etc/crypttab).
+            fstab("root"),
+            # These MUST come after the fstab step so that they add the
+            # crypttab in the initramfs. We install them regardless of
+            # whether LUKS is used: they're harmless if LUKS isn't used.
+            apt(
+                "root",
+                [
                     "cryptsetup",
                     "cryptsetup-initramfs",
                 ],
-                "tag": "root",
-            },
-            {
-                # This also MUST come outside the rootfs caching, as it install
-                # things outside the file systems.
-                "grub": "uefi",
-                "tag": "root",
-                "efi": "efi",
-                "quiet": True,
-                "image-dev": device,
-            },
+                unless=False,
+            ),
+            # This also MUST come outside the rootfs caching, as it install
+            # things outside the file systems, and those won't be in the
+            # cache.
+            grub(device, "root", "efi"),
         ]
     )
 
-    # If a playbook has been specified, add an ansible step.
-    if playbook:
-        spec["steps"].append(
-            {"ansible": "root", "playbook": playbook, "extra_vars": extra_vars}
-        )
+    # If playbooks have been specified, add ansible steps.
+    for p in ["std.yml"] + system.extra_playbooks:
+        if p:
+            steps.append({"ansible": "root", "playbook": p, "extra_vars": ansible_vars})
+
+    return {"steps": steps}
+
+
+class SystemSpec:
+    def __init__(self, filename):
+        REQUIRED = "required"
+        self._obj = {
+            "hostname": REQUIRED,
+            "drive": REQUIRED,
+            "extra_drives": [],
+            "extra_lvs": [],
+            "extra_playbooks": [],
+            "ansible_vars": {},
+            "luks": None,
+        }
+        with open(filename) as f:
+            obj = yaml.safe_load(f)
+
+        # Check for unknown keys.
+        for key in obj:
+            if key not in self._obj:
+                sys.exit(f"spec has unknown key: {key}")
 
-    return spec
+        # Check for missing required keys.
+        for key in self._obj:
+            if self._obj[key] == REQUIRED and key not in obj:
+                sys.exit(f"spec lacks required key {key}")
+
+        # Check for types of values.
+        for key in self._obj:
+            if key in obj:
+                e = type(self._obj[key])
+                a = type(obj[key])
+                if a != e:
+                    sys.exit(f"spec key {key} has unexpected type {a}, wanted {e}")
+
+        self._obj.update(obj)
+
+        for key in self._obj:
+            setattr(self, key, self._obj[key])
+        del self._obj
+
+    def __repr__(self):
+        r = {key: getattr(self, key) for key in dir(self) if not key.startswith("_")}
+        return repr(r)
 
 
 def main():
     p = argparse.ArgumentParser()
     p.add_argument("--verbose", action="store_true")
+    p.add_argument("--very-verbose", action="store_true")
     p.add_argument("--log", default="install.log")
     p.add_argument("--cache", default="cache.tar.gz")
-    p.add_argument("--playbook")
-    p.add_argument("--vars")
-    p.add_argument("--luks")
-    p.add_argument("device")
+    p.add_argument("spec")
     args = p.parse_args()
 
-    if args.verbose:
-        global verbose
-        verbose = args.verbose
+    logging.basicConfig(
+        filename=args.log,
+        level=logging.DEBUG,
+        format="%(asctime)s %(levelname)s %(message)s",
+    )
+
+    global verbose
+    verbose = args.verbose
 
-    extra_vars = {}
-    if args.vars:
-        with open(args.vars) as f:
-            extra_vars = yaml.safe_load(f)
+    log("v-i starts")
 
-    clean_up_disks()
+    system = SystemSpec(args.spec)
+    log(f"spec: {system!r}")
 
-    spec = vmdb_spec(args.luks, args.playbook, extra_vars)
+    clean_up_disks([system.drive] + system.extra_drives)
+
+    ansible_vars = dict(system.ansible_vars)
+    ansible_vars["hostname"] = system.hostname
+    vmdb = vmdb_spec(system, ansible_vars)
     tmp = tempfile.mkdtemp()
     specfile = os.path.join(tmp, "spec.yaml")
-    if args.verbose:
-        yaml.dump(spec, stream=sys.stdout, indent=4)
+    if args.very_verbose:
+        yaml.dump(vmdb, stream=sys.stdout, indent=4)
     with open(specfile, "w") as f:
-        yaml.dump(spec, stream=f, indent=4)
+        yaml.dump(vmdb, stream=f, indent=4)
 
-    log(f"run vmdb2 to install on {args.device}")
+    log(f"run vmdb2 to install on {system.drive}")
     env = dict(os.environ)
     env["ANSIBLE_STDOUT_CALLBACK"] = "yaml"
     env["ANSIBLE_NOCOWS"] = "1"
@@ -312,12 +471,11 @@ def main():
         "vmdb2",
         f"--rootfs-tarball={args.cache}",
         f"--log={args.log}",
-        f"--image={args.device}",
+        f"--image={system.drive}",
         specfile,
     ]
     if verbose:
         argv.append("--verbose")
-
     run(argv, check=True, capture_output=True)
 
     log("cleanup")