path: root/README.md

# v-i---a Debian installer using vmdb2

**WARNING: Running v-i is like waking up after an alien invasion, in a
post-apocalyptic world, with everything you knew or owned gone
forever. When you run v-i, it *will* wipe away everything on that
computer. All volume groups will be deleted, all storage drives
emptied. Any existing partitions will be lost. Forget any data you
used to have, and operating systems you used to have installed. If you
don't know what you're doing, leave.**

**v-i** installs Debian onto a PC. It's entirely non-interactive,
dangerous, unhelpful, and may only work for the author's PCs. The
[author][] wrote it so that repeated installations would be less of a
chore, for them, than using the official Debian installer. (Actually,
the author thought it'd be a quick, easy hack, and was too stubborn to
give up, when it turned out to be a bit tricky.)

[**vmdb2**][] is a program to create a disk image virtual machines
with Debian, by the same author. It "installs Debian" to a file
representing a hard drive. It's basically [debootstrap][], except the
target is a disk image instead of a directory. vmdb2 has been quite
useful for generating virtual machine images. It's also used to create
[Debian images for Raspberry Pis][]. **v-i** uses **vmdb2** to install
onto bare metal hardware.

To use **v-i** to install Debian on a PC:

* Boot the target machine off a live system that has **v-i**
  installed.
  - the author uses a USB stick with an image built with the
    [`build-installer.sh`][] script
  - the author logs into the installer system via SSH
* Create a v-i target specification file. See below for an example.
* Run the command: `v-i --verbose exolobe5.yaml`
* See `installer.log` for what happened during the installation.

Example target specification file:

```yaml
drive: /dev/nvme0n1
extra_drives:
  - /dev/nvme1n1
hostname: exolobe5
extra_playbooks:
  - exolobe5-ansible.yml
ansible_vars:
  user_pub: |
   ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPQe6lsTapAxiwhhEeE/ixuK+5N8esCsMWoekQqjtxjP liw personal systems
extra_lvs:
  - name: vms
    size: 1T
    fstype: ext4
    mounted: /mnt/vms
```

Explanation:

* `hostname`---the hostname of the installed system. This is so that
  when the installed system boots, and gets a network address using
  DHCP, it can provide a name. The author's home network setup
  automatically adds that hostname to the internal DNS. This avoids a
  manual DNS configuration step, and the author is lazy.
  ([dnsmasq](https://dnsmasq.org/) is lovely.)
* `drive`---the main drive to install to. This is where the EFI and
  `/boot` partitions are created, and where GRUB gets installed. The
  rest of the drive will be a physical volume for LVM2.
* `extra_drives`---any additional physical volumes for LVM2. These
  will not be partitioned, and will be used entirely as physical
  volumes.
* `luks`---the password for full disk encryption for all LVM2 physical
   volumes. If not set, LUKS is not used. This is a single, fixed
   password that is in cleartext. You are expected to change it after
   the system is installed and boots. If you'd rather use, say, a
   hardware token's challenge/response feature or TPM for LUKS, that's
   better done on a running system.
* `extra_playbooks`---additional Ansible playbooks to use on the
  installed system. **v-i** comes with a "standard playbook" (in
  [`std.yml`][]) that it uses unconditionally, to set up a "standard
  system" that the author likes. You can provide additional playbooks,
  for additional configuration at installation time.
* `ansible_vars`---variables to set for Ansible playbooks.
  - the `user_pub` variable contains an SSH public key that gets
    installed into the `root` user `authorized_keys` file on the
    installed system by the standard playbook
  - the `user_ca_pubkey` variable contains public key for an SSH CA
    whose user certificates are to be trusted

With all this configuration in a file, which you can keep in git, you
can install a base system repeatedly to a specific computer, and do it
the same way every time. If that's not something you do, then you may
want to use the official Debian installer instead.

(Caveat: **v-i** does nothing to configure your BIOS/UEFI. It can't.
You have to manually configure it the way you want it to be. For
example, one of the author's machines needs to have its boot order
adjusted after every operating system installation. It's quite
tedious.)

[**vmdb2**]: https://vmdb2.liw.fi/
[debootstrap]: https://wiki.debian.org/Debootstrap
[Debian images for Raspberry Pis]: https://raspi.debian.net/
[`build-installer.sh`]: build-installer.sh
[`v-i`]: v-i
[`std.yml`]: std.yml
[author]: https://liw.fi/
[Debian installer]: https://www.debian.org/devel/debian-installer/
[preseed files]: https://wiki.debian.org/DebianInstaller/Preseed
[udeb]: https://en.wikipedia.org/wiki/Deb_(file_format)

## Motivation

The official [Debian installer][] is often referred to as _d-i_. It
works quite well, for almost any hardware Debian can run on, and
supports a lot of languages, and it's flexible enough to be acceptable
for nearly every use case. Millions upon millions of people are
satisfied users of it. It is a great achievement of Debian, and the
people of the `debian-boot` team.

However, the **v-i** author felt it could be improved upon slightly
for their use case:

* d-i is not entirely easy to understand and modify. It requires
  building special [udeb][] packages for any software that's to be
  part of the installer environment, which makes it harder to make
  changes without co-operation from maintainers of those packages. The
  architecture of d-i is also a little non-linear. d-i also needs to
  support a very wide variety of hardware and use cases, which has
  made it large and complex.

  **v-i** is happy with normal deb packages, and is a thin Python
  wrapper script around **vmdb2**, making it reasonably easy to
  understand and change. Well, easy for its author.

* d-i is primarily meant to be used interactively, but it does support
  [preseed files][] for automating an installation. Preseeding means
  providing answers, in a file, to questions a package being installed
  may ask during its installation. This is fine, if a little
  cumbersome, but only helps to answer questions the packages ask when
  installed.

  **v-i** lets you have the full power of Ansible during initial
  installation.

If **v-i** isn't suitable for your uses, that's OK. The author is
happy with his toy.


## Architecture

**vmdb2** is given a sequence of _steps_ to execute: create this
partition, make that file system, install those packages, etc.
**vmdb2** runs the steps against a disk image or physical hard drive,
with a chroot of the file systems, to do things like installing a
package in the system being installed.

**v-i** defines a fairly minimal _standard install_, whose goal is to
get the target system into a state where it boots from its own,
internal storage, and where the rest of the system configuration can
be finished using your configuration management system of choice.

While **vmdb2** can, and does, run Ansible to configure the system
being installed, in practice some things work better if most
configuration is done to a running system; if nothing else, some
Ansible modules don't work well in a chroot. The goal of **v-i** is to
get a system into that state as quickly and easily as possible. For
example, the Ansible module to set a hostname on a system with systemd
requires systemd to be running. That's awkward while the system is
still being installed in a chroot.

Thus, **v-i** does the following:

* delete any trace of LVM2 from all drives, wipe all SSDs, and
  generally reset the system to as close to a blank state as possible
  - there is no question "are you sure?" to give the user a chance to
    repent: as soon as you run **v-i**, you've lost all your data
* create a partition table ("label") on the target drive
* create EFI and boot partitions, needed to boot with UEFI and LUKS
* create a physical volume for LVM2, and a logical volume for the root
  file system
  - add any additional drives as physical volumes to the volume group
  - optionally use LUKS for full disk encryption for each physical
    volume
* install the Debian base system
  - run `debootstrap`, install a boot loader, and create fstab and
    `crypttab` files
* run the standard Ansible playbook (see [`std.yml`][])
  - set hostname
  - set keyboard layout
  - configure networking (using systemd-networkd)
  - install an SSH server
  - add a chosen SSH public key to the root user's authorized keys
    file
* run any additional playbooks

**v-i** uses the **vmdb2** caching feature, where the results of
`debootstrap` and some other steps get stored in a compressed tar
archive. On subsequent runs, if the cache file exists, it's unpacked,
instead of running the commands. This speeds things up a bit: running
**v-i** without the cache file takes the author about 5 minutes; with
the cache file it takes about 1.5 minutes. This matters if there is a
need to do many installations. It also matters if you're developing
an installer and need to run it tens of times a day.


## Hacking

The main files of **v-i** are:

* [`v-i`][]---the actual installer, a Python script
* [`std.yml`][]---the Ansible playbook to configure a standard install

Also, to build an image to boot off for running the installer:

* `build-installer.sh`---build a disk image where **v-i** can be run
  - put image on a USB drive, boot off that drive, run installer
  - note that you can use any live image with **vmdb2** installed; the
    image built with this is just the easiest for the author
* `installer.vmdb`---the **vmdb2** specification file for creating the
  installer image
* `installer.yml`---the Ansible playbook for creating the installer
  image

See the tutorial about ways to add your SSH public key to the image so
that you can log into the installer via SSH.

You probably mostly only need to modify `v-i` and `std.yml`. The rest
is to get you and your target machine into a state where you can run
the installer.



## FAQ

This section is prescient: the author hasn't been asked any questions
yet, but expects the following to be asked.

### What version of Debian does v-i install?

**v-i** installs Debian 11 (bullseye).

### What about other releases of Debian?

The Debian 11 (bullseye) release is the earliest release the author
has gotten to work with **v-i**, and is the only release the author is
installing on bare metal systems. Later versions of Debian may work,
we will see.

### Is only UEFI supported?

Yes.

### Why is BIOS (without UEFI) not supported?

All of the author's PCs have UEFI, and the author doesn't care to do
the work to add support for BIOS.

### What about multi-boot?

**v-i** doesn't support installing more than one operating system on
one computer. The author has no need for this.

### What about installing something else than Debian?

The author only cares about Debian, but in principle, fairly little of
**vmdb2** and **v-i** are specific to Debian. It should be possible to
add support for other operating systems to be installed, at least ones
based on Linux. If you're interested, you need to change or replace at
least the following steps in **vmdb2** code, and then change the
[`v-i`][] script to generate a specification using those steps:

* `debootstrap`---install base operating system into a directory
  - after this step, all the files in a base installation should be in
    the specified directory tree, except the boot loader and kernel
  - could probably be replaced with providing **v-i** with a pre-built
    cache tar archive
* `apt`---install packages
  - whatever package manager the system has probably works
  - you can probably run the package manager from a **vmdb2** `chroot` step
* `grub`---install boot loader
  - this chooses the appropriate Debian package automatically
  - might possibly be doable as a `shell` step
  - this is likely the trickiest bit: booting is _intricate_
* `cryptsetup`---format a drive for full disk encryption
  - this just runs the `cryptsetup` program and tells the fstab step
    to create a crypttab file
  - might just work
* `vgcreate` and `lvcreate`---create LVM2
  - these just run the relevant LVM2 commands
  - might just work

### What about other kinds of computers than PCs?

The author only uses 64-bit PC computers (`amd64` arhitecture in
Debian; also known as x86-64). **v-i** may well work for other
kinds of computers, as long as they can boot off an installer image
("live image"), and use GRUB for booting. The author would be
interested to hear if that is the case.

### Why is the LUKS password in cleartext?

It would be ideal if **v-i** (or **vmdb2**) got the LUKS password for
full disk encryption in a secure way from a secure source, but that
turned out to be tricky to do. The author felt it was too tricky to do
well in the installer environment, while it's pretty easy to do in a
running system. Thus, the cleartext password _in the installer_ is a
compromise. You're expected to change the password after the
installation is done.

It would be possible to ask the person doing the installation to enter
the password manually, but this would mean the installation would not
be fully automated. The author didn't want that.

### Do I have to use Ansible?

No. Use whatever you like once you've installed a system with **v-i**
and booted it. **v-i** itself uses Ansible, because that was easy for
the author to use.

### I'd like to use v-i, but I need changes

If you can make the changes yourself, go ahead: this is free and open
source software, have at it. If you don't have the skill or time to
make changes yourself, you'll need to find someone else to make them.
This might require paying them.

The author is, unfortunately, probably not willing to spend their free
time to make changes that don't benefit them directly, for free.
Sorry. They _are_ willing to review and merge changes that would make
the software better.


# Legalese

Copyright 2018-2022  Lars Wirzenius

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.