---
title: Building Debian system images with vmdb2
author: Lars Wirzenius
date: work-in-progress
...

# Introduction

`vmdb2` installes [Debian][] **system images**. The image may be in a
disk image file or a block device, such as a real hard disk, a
partition, a logical volume, or similar.

Other similar software exists, such as `vmdebootstrap`, an earlier
attempt by the same author.

[Debian]: https://www.debian.org

## Why vmdb2 given vmdebootstrap already existed

`vmdebootstrap` was the first attempt by Lars Wirzenius to write a
tool to build system images. It turned out to not be well designed.
Specifically, it was not easily extensible to be as flexible as a tool
of this sort should be.

## Why vmdb2 given other tools already exist

Lars likes to write tools for himself and had some free time. He
sometimes prefers to write his own tools rather than spend time and
energy evaluating existing tools.

Also, he felt ashamed of how messy `vmdebootstrap` turned out to be.

If nobody else likes `vmdb2`, that just means Lars had some fun on his
own.

# Installation

You can install `vmdb2` from a .deb package on [code.liw.fi/debian][],
Lars's personal APT repository. You can also get the source code with
git from [vmdb2 git][], and run `./vmdb2` from the source tree: the
program doesn't need to be installed anywhere.

[code.liw.fi/debian]: http://code.liw.fi/debian/
[vmdb2 git]: http://git.liw.fi/vmdb2

# Quick start

- sample .vmdb2
- command to run vmdb2 and produce an image
- command to run vm using image

`vmdb2` requires an **image specification file** as input. Here is a
small example:

~~~
steps:
  - mkimg: "{{ output }}"
    size: 4G

  - mklabel: msdos
    device: "{{ output }}"

  - mkpart: primary
    device: "{{ output }}"
    start: 0%
    end: 100%
    part-tag: root-part

  - mkfs: ext4
    partition: root-part

  - mount: root-part
    fs-tag: root-fs

  - unpack-rootfs: root-fs

  - debootstrap: stretch
    mirror: http://http.debian.net/debian
    target: root-fs
    unless: rootfs_unpacked

  - apt: linux-image-amd64
    fs-tag: root-fs
    unless: rootfs_unpacked

  - cache-rootfs: root-fs
    unless: rootfs_unpacked

  - chroot: root-fs
    shell: |
      apt -y install python

  - shell: |
      printf '[pc]\n%s hostname=pc\n' "$ROOT" > pc.inventory
      ansible-playbook -i pc.inventory -c chroot pc.yml
    root-fs: root-fs

  - grub: bios
    root-fs: root-fs
    root-part: root-part
    device: "{{ output }}"
~~~

It's a bit of a mouthful, and needs to be unpacked to be understood,
though every part is really quite simple. First of all, the file is in
[YAML][] format, which is a standard format not specific to `vmdb2` at
all. The top level is the `steps` key, which has as its value a list
of **steps**. The steps instruct `vmdb2` what it should do, and in
which order, to build the image.

[YAML]: https://en.wikipedia.org/wiki/YAML

The previous `vmdebootstrap` program hard coded the sequence of steps
to build an image, and provided command line options to vary the steps
in minor ways. The `vmdb2` approach gives more flexibility with less
complexity, and is more easily testable.

Each step consists of a set of key/value pairs. For example, the first
step creates a disk image file:

      - mkimg: "{{ output }}"
        size: 4G

`mkimg` identifies the step. The value is a [Jinja2][] template. In
this case, the template expands to the value of the `output` variable,
which contains the value of the `--output` command line option,
meaning the name of the output file. Jinja2 is a powerful templating
engine, and `vmdb2` uses it, amongst other reasons, to avoid
implementing a custom language for using variables.

[Jinja2]: http://jinja.pocoo.org/

The `size` key specifies the size of the disk image file that gets
created. `vmdb2` uses the `qemu-img` tool to create the image file and
the size value can be anything that `qemu-img` accepts.

The rest of the steps create a partition table (`mklabel`), a
partition (`mkpart`), and a filesystem (`mkfs`), as well as mount the
filesystem as the root filesystem. Partitions and mount points get
assigned **tags**, which allow later steps to refer to them, without
having to hardcode a directory or device name. Those tend to be a bit
random and vary from run to run, so hardcoding wouldn't work anyway.

The `debootstrap` step runs `debootstrap`, i.e., actually installs
Debian into the image. This can take a while, so `vmdb2` provides a
caching mechanism, to speed up iterations (unlike you, Lars never gets
things right the first one). This caching is done by using the
`--rootfs-tarball` command line option and the `unpack-rootfs` step,
and the `unless: rootfs_unpacked` key/value pair in selected steps.
Caching works like this: the filename given to `--rootfs-tarball` is
the cache filename. The `unpack-rootfs` looks for the cache file; if
it exists, it unpacks it, and sets the `rootfs_unpacked` variable to a
Boolean true value. Any other step can be tagged with `unless`, and if
the named value is true, the step is skipped. Thus, if the cache file
exists, the `debootstrap` step is skipped. Finally, the `cache-rootfs`
step creates a compressed tar archive of the filesystem, and that step
is also skipped if the `unless` tells `vmdb2` to skip it.

At the end, the `grub` step installs the GRUB bootloader. The example
installs a version compatible with traditional PC BIOS systems, which
works fine with virtual machine providers. A variabnt to support UEFI
systems is also available.

To run `vmdb2` with the above example, run a command like this:

    vmdb2 pc.vmdb --output foo.img --rootfs-tarball foo.tar.gz

This tells `vmdb2` to read the image specification file from
`pc.vmdb`, place the output in `foo.img` and use `foo.tar.gz` as the
cache tar archive.


# Configuration

- command line options, list them and give summary of use
- cliapp config files, ini and yaml, and the fact that a long
  option is always an acceptable variable in a config file

# A .vmdb2 image specification file

- yaml, jinja2 syntax
- steps to list what needs to be done to build an image
- setting jinja2 variables from the command line

# List of available steps and their configuration

- list each step
- what is the step for? examples of how it could be used
- what mandatory and optional key/value pairs does it
  understand? examples of their use
- the tag concept

# Examples

- various spec files, explained in detail
- simple image to run under kvm, qemu, or similar, boots with
  normal grub (bios)
- same but with uefi boot
- same but software installed with ansible
- an LVM enabled image with encrypted / but cleartext /boot