docs: rewrite the README

Sponsored-by: author

1 files changed, 306 insertions, 67 deletions

diff --git a/README.md b/README.md
index 5640fa6..0e5ba18 100644
--- a/README.md
+++ b/README.md
@@ -1,94 +1,333 @@
-Install Debian using vmdb2 (v-i) onto my Thinkpad x220 laptop
-=============================================================================
+# v-i---a Debian installer using vmdb2
 
-WARNING!
------------------------------------------------------------------------------
+**WARNING: Running v-i is like waking up after an alien invasion, in a
+post-apocalyptic world, with everything you knew or owned gone
+forever. When you run v-i, it *will* wipe away everything on that
+computer. All volume groups will be deleted, all storage drives
+emptied. Any existing partitions will be lost. Forget any data you
+used to have, and operating systems you used to have installed. If you
+don't know what you're doing, leave.**
 
-**Warning:** this is an operating system installer. It will overwrite your
-hard drive and annihilate anything that used to be there. There are no
-safety rails. You will lose all your data. Make sure your backups
-work. You have been warned.
+**v-i** installs Debian onto a PC. It's entirely non-interactive,
+dangerous, unhelpful, and may only work for the author's PCs. The
+[author][] wrote it so that repeated installations would be less of a
+chore, for them, than using the official Debian installer. (Actually,
+the author thought it'd be a quick, easy hack, and was too stubborn to
+give up, when it turned out to be a bit tricky.)
 
+[**vmdb2**][] is a program to create a disk image virtual machines
+with Debian, by the same author. It "installs Debian" to a file
+representing a hard drive. It's basically [debootstrap][], except the
+target is a disk image instead of a directory. vmdb2 has been quite
+useful for generating virtual machine images. It's also used to create
+[Debian images for Raspberry Pis][]. **v-i** uses **vmdb2** to install
+onto bare metal hardware.
 
-Introduction
------------------------------------------------------------------------------
+To use **v-i** to install Debian on a PC:
 
-This is a hack for my own amusement. It's not meant to be taken
-seriously.
+* Boot the target machine off a live system that has **v-i**
+  installed.
+  - the author uses a USB stick with an image built with the
+    [`build-installer.sh`][] script
+  - the author logs into the installer system via SSH
+* Create a v-i target specification file. See below for an example.
+* Run the command: `v-i --verbose exolobe5.yaml`
+* See `installer.log` for what happened during the installation.
 
-This is a rudimentary installer of Debian based on vmdb2. It has
-nothing to do with debian-installer, the official Debian installer,
-known as d-i. I use d-i, but would like something better:
+Example target specification file (see [exolobe5.yaml](exolobe5.yaml)
+for the version used in production):
 
-* I'd like something I can easily modify. d-i requires building
-  special udeb packages for any software that's to be part of the
-  installer. v-i is happy with normal debs.
+```yaml
+drive: /dev/nvme0n1
+extra_drives:
+  - /dev/nvme1n1
+hostname: exolobe5
+extra_playbooks:
+  - exolobe5.yml
+ansible_vars:
+  user_pub: |
+   ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPQe6lsTapAxiwhhEeE/ixuK+5N8esCsMWoekQqjtxjP liw personal systems
+extra_lvs:
+  - name: vms
+    size: 1T
+    fstype: ext4
+    mounted: /mnt/vms
+```
 
-* d-i uses preseeding for automating an installation. Preseeding means
-  providing answers, in a file, to questions the package may ask
-  during its installation. This is fine, if a little cumbersome, but
-  only helps to answer questions the packages ask. v-i lets you have
-  the full power of Ansible during initial installation.
+Explanation:
 
-On the other hand, d-i is mature software and tested by a very large
-number of people, on a very large number of different hardware. v-i
-can just barely install my own Thinkpad x220 laptop, using UEFI.
+* `hostname`---the hostname of the installed system. This is so that
+  when the installed system boots, and gets a network address using
+  DHCP, it can provide a name. The author's home network setup
+  automatically adds that hostname to the internal DNS. This avoids a
+  manual DNS configuration step, and the author is lazy.
+* `drive`---the main drive to install to. This will have the EFI and
+  `/boot` partitions, and have GRUB installed. The rest of the drive
+  will be a physical volume for LVM2.
+* `extra_drives`---any additional physical volumes for LVM2. These
+  will not be partitioned.
+* `luks`---the password for full disk encryption for LVM2 physical
+   volumes. If not set, LUKS is not used. This is a single, fixed
+   password that is in cleartext. You are expected to change it after
+   the system is installed and boots. If you'd rather use, say, a
+   hardware token's challenge/response feature or TPM for LUKS, that's
+   better done on a running system.
+* `extra_playbooks`---additional Ansible playbooks to use on the
+  installed system. **v-i** comes with a "standard playbook" (in
+  [`std.yml`][]) that it uses unconditionally, to set up a "standard
+  system" that the author likes. You can provide additional playbooks,
+  for additional configuration at installation time.
+* `ansible_vars`---variables to set for Ansible playbooks.
 
-v-i is not currently expected to work with any other hardware. It
-might not even work on your x220.
+With all this configuration in a file, which you can keep in git, you
+can install a base system repeatedly to a specific computer, and do it
+the same way every time.
 
+(Caveat: **v-i** does nothing to configure your BIOS/UEFI. It can't.
+You have to manually configure it the way you want it to be. For
+example, one of the author's machines needs to have its boot order
+adjusted after every operating system installation. It's quite
+tedious.)
 
-Architecture
------------------------------------------------------------------------------
+[**vmdb2**]: https://vmdb2.liw.fi/
+[debootstrap]: https://wiki.debian.org/Debootstrap
+[Debian images for Raspberry Pis]: https://raspi.debian.net/
+[`build-installer.sh`]: build-installer.sh
+[`v-i`]: v-i
+[`std.yml`]: std.yml
+[author]: https://liw.fi/
+[Debian installer]: https://www.debian.org/devel/debian-installer/
+[preseed files]: https://wiki.debian.org/DebianInstaller/Preseed
+[udeb]: https://en.wikipedia.org/wiki/Deb_(file_format)
 
-The vmdb2 installer builds a bootable disk image with Debian. The
-image boots with UEFI, and has vmdb2 installed. You write the image to
-a USB drive, boot off that, and run vmdb2 in the booted system to
-install Debian onto the hard drive.
+## Motivation
 
-Thus:
+The official [Debian installer][] is often referred to as _d-i_. It
+works quite well, for almost any hardware Debian can run on, and
+supports a lot of languages, and if flexible enough to be acceptable
+for nearly every use case. Millions upon millions of people are
+satisfied users of it. It is a great achievement of Debian, and the
+people of the `debian-boot` team.
 
-- build system -> installer image
-- installer image -> USB drive
-- boot from USB drive
-- run vmdb2 to install onto laptop hard drive
-  - the /root/x220.sh script does this, and installs a basic Debian
-    system using my Ansible playbooks
-- boot off the laptop hard drive
+However, the **v-i** author felt it could be improved upon for them:
 
-The image building uses vmdb2 as well. Things get a little recursive.
+* d-i is not entirely easy to understand and modify. It requires
+  building special [udeb][] packages for any software that's to be
+  part of the installer environment, which makes it harder to make
+  changes without collaboration from maintainers of those packages.
+  The architecture of d-i is also a little non-linear. d-i also needs
+  to support a very wide variety of hardware and use cases, which has
+  made it large and complex.
 
+  **v-i** is happy with normal deb packages, and is a thin Python
+  wrapper script around **vmdb2**, making it reasonably easy to
+  understand and change.
 
-Hacking
------------------------------------------------------------------------------
+* d-i is primarily meant to be used interactively, but it does support
+  [preseed files][] for automating an installation. Preseeding means
+  providing answers, in a file, to questions a package being installed
+  may ask during its installation. This is fine, if a little
+  cumbersome, but only helps to answer questions the packages ask when
+  installed.
 
-The main files are:
+  **v-i** lets you have the full power of Ansible during initial
+  installation.
 
-* v-i.vmdb -- the vmdb2 specification file for the installer image
-* v-i.yml -- the Ansible playbook used by v-i.vmdb
-* x220.vmdb -- the vmdb2 specification file to install onto an x220;
-  this is used from the system booted from the image built by v-i.vmdb
-* x220.yml -- the Ansible playbook used by x220.vmdb
+If **v-i** isn't suitable for your uses, that's OK. The author is
+happy with his toy.
 
-To build the installer image, run (the cached tarball will be created
-if it doesn't exist, but you need to choose the location):
 
-~~~sh
-sudo ./v-i.sh /path/to/rootfs/tarball/for/caching/debootstrap.tar.gz
-~~~
+## Architecture
 
-To write the resulting v-i.img to a USB drive:
+**vmdb2** is given a sequence of _steps_ to execute: create this
+partition, make that file system, install those packages, etc.
+**vmdb2** runs the steps against a disk image or physical hard drive,
+with a chroot of the file systems, to do things like installing a
+package in the system being installed.
 
-~~~sh
-sudo dd if=v-i.img of=/dev/path/to/usb/drive/device status=progress oflag=direct
-~~~
+**v-i** defines a fairly minimal _standard install_, whose goal is to
+get the target system into a state where it boots from its own,
+internal storage, and where the rest of the system configuration can
+be finished using your configuration management system of choice.
 
-Plug in the USB drive to the laptop, boot off that. Log into the
-installer as root (no password), and run
+While **vmdb2** can, and does, run Ansible to configure the system
+being installed, in practice some things work better if most
+configuration is done to a running system. The goal of **v-i** is to
+get a system into that state as quickly and easily as possible. For
+example, the Ansible module to set a hostname on a system with systemd
+requires systemd to be running. That's awkward while the system is
+still being installed in a chroot.
 
-~~~sh
-./x220.sh
-~~~
+Thus, **v-i** does the following:
 
-Reboot laptop from its hard drive and you should have a bare bones
-Debian system installed.
+* delete any trace of LVM2 from all drives, wipe all SSDs, and
+  generally reset the system to as close to a blank state as possible
+* create a partition table ("label") on the target drive
+* create EFI and boot partitions, needed to boot with UEFI and LUKS
+* create a physical volume for LVM2, and a logical volume for the root
+  file system
+  - add any additional drives as physical volumes to the volume group
+  - optionally use LUKS for full disk encryption for each physical
+    volume
+* install the Debian base system
+  - run `debootstrap`, install a boot loader, and create fstab and
+    `crypttab` files
+* run the standard Ansible playbook (see [`std.yml`][])
+  - set hostname
+  - set keyboard layout
+  - configure networking (using systemd-networkd)
+  - install an SSH server
+  - add a chosen SSH public key to the root user's authorized keys
+    file
+* run any additional playbooks
+
+**v-i** uses the **vmdb2** caching feature, where the results of
+`debootstrap` and some other steps get stored in a compressed tar
+archive. On subsequent runs, if the cache file exists, it's unpacked,
+instead of running the commands. This speeds things up a bit: running
+**v-i** without the cache file takes the author about 5 minutes; with
+the cache file it takes about 1.5 minutes. This matters if there is a
+need to do many installations.
+
+
+## Hacking
+
+The main files of **v-i** are:
+
+* [`v-i`][]---the actual installer, a Python script
+* [`std.yml`][]---the Ansible playbook to configure a standard install
+
+Also, to build an image to boot off for running the installer:
+
+* `build-installer.sh`---build a disk image where **v-i** can be run
+  - put image on a USB drive, boot off that drive, run installer
+  - note that you can use any live image with **vmdb2** installed; the
+    image built with this is just the easiest for the author
+* `installer.vmdb`---the **vmdb2** specification file for creating the
+  installer image
+* `installer.yml`---the Ansible playbook for creating the installer
+  image
+
+You'll want to build your own installer image, if you want to log into
+it over SSH, so that you can install your own SSH key. If you log in
+via the Linux virtual console, you don't need that: there's no
+password for root (but SSH login for root doesn't work with
+passwords).
+
+You probably mostly only need to modify `v-i` and `std.yml`. The rest
+is to get you and your target machine into a state where you can run
+the installer.
+
+
+
+## FAQ
+
+This section is prescient: the author hasn't been asked any questions
+yet, but expects the following to be asked.
+
+### What version of Debian does v-i install?
+
+**v-i** installs Debian 11 (bullseye).
+
+### What about other releases of Debian?
+
+The Debian 11 (bullseye) release is the earliest release the author
+has gotten to work with **v-i**, and is the only release the author is
+installing on bare metal systems. Later versions of Debian may work,
+we will see.
+
+### Is only UEFI supported?
+
+Yes.
+
+### What about multi-boot?
+
+**v-i** doesn't support installing more than one operating system on
+one computer.
+
+### What about installing something else than Debian?
+
+The author only cares about Debian, but in principle, fairly little of
+**vmdb2** and **v-i** are specific to Debian. It should be possible to
+add support for other operating systems to be installed, at least ones
+based on Linux. If you're interested, you need to change or replace at
+least the following steps in **vmdb2** code, and then change the
+[`v-i`][] script to generate a specification using those steps:
+
+* `debootstrap`---install base operating system into a directory
+  - after this step, all the files in a base installation should be in
+    the specified directory tree, except the boot loader and kernel
+  - could probably be replaced with providing **v-i** with a pre-built
+    cache tar archive
+* `apt`---install packages
+  - whatever package manager the system has probably works
+  - you can probably run the package manager from a chroot step
+* `grub`---install boot loader
+  - this chooses the appropriate Debian package automatically
+  - might possibly be doable as a chroot step
+  - this is likely the trickiest bit: booting is _intricate_
+* `cryptsetup`---format a drive for full disk encryption
+  - this just runs the `cryptsetup` program and tells the fstab step
+    to create a crypttab file
+  - might just work
+* `vgcreate` and `lvcreate`---create LVM2
+  - these just run the relevant LVM2 commands
+  - might just work
+
+### What about other kinds of computers than PCs?
+
+The author only uses 64-bit PC computers (`amd64` arhitecture in
+Debian; also known as x86-64). **v-i** may well work for other
+kinds of computers, as long as they can boot off an installer image
+("live image"), and use GRUB for booting. The author would be
+interested to hear if that is the case.
+
+### Why is the LUKS password in cleartext?
+
+It would be ideal if **v-i** (or **vmdb2**) got the LUKS password for
+full disk encryption in a secure way from a secure source, but that
+turned out to be tricky to do. The author felt it was too tricky to do
+well in the installer environment, while it's pretty easy to do in a
+running system. Thus, the cleartext password _in the installer_ is a
+compromise. You're expected to change the password after the
+installation is done.
+
+It would be possible to ask the person doing the installation to enter
+the password manually, but this would mean the installation would not
+be fully automated. The author didn't want that.
+
+### Do I have to use Ansible?
+
+No. Use whatever you like once you've installed a system with **v-i**
+and booted it. **v-i** itself uses Ansible, because that was easy for
+the author to use.
+
+### I'd like to use v-i, but I need changes
+
+If you can make the changes yourself, go ahead: this is free and open
+source software, have at it. If you don't have the skill or time to
+make changes yourself, you'll need to find someone else to make them.
+This might require paying them.
+
+The author is, unfortunately, probably not willing to spend their free
+time to make changes that don't benefit them directly, for free.
+Sorry. They _are_ willing to review and merge changes that would make
+the software better.
+
+
+# Legalese
+
+Copyright 2018-2022  Lars Wirzenius
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.