1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
|
---
title: Building Debian system images with vmdb2
author: Lars Wirzenius
date: work-in-progress
...
# Introduction
`vmdb2` installes [Debian][] **system images**. The image may be in a
disk image file or a block device, such as a real hard disk, a
partition, a logical volume, or similar.
Other similar software exists, such as `vmdebootstrap`, an earlier
attempt by the same author.
[Debian]: https://www.debian.org
## Why vmdb2 given vmdebootstrap already existed
`vmdebootstrap` was the first attempt by Lars Wirzenius to write a
tool to build system images. It turned out to not be well designed.
Specifically, it was not easily extensible to be as flexible as a tool
of this sort should be.
## Why vmdb2 given other tools already exist
Lars likes to write tools for himself and had some free time. He
sometimes prefers to write his own tools rather than spend time and
energy evaluating existing tools.
Also, he felt ashamed of how messy `vmdebootstrap` turned out to be.
If nobody else likes `vmdb2`, that just means Lars had some fun on his
own.
# Installation
You can install `vmdb2` from a .deb package on [code.liw.fi/debian][],
Lars's personal APT repository. You can also get the source code with
git from [vmdb2 git][], and run `./vmdb2` from the source tree: the
program doesn't need to be installed anywhere.
[code.liw.fi/debian]: http://code.liw.fi/debian/
[vmdb2 git]: http://git.liw.fi/vmdb2
# Quick start
- sample .vmdb2
- command to run vmdb2 and produce an image
- command to run vm using image
`vmdb2` requires an **image specification file** as input. Here is a
small example:
~~~
steps:
- mkimg: "{{ output }}"
size: 4G
- mklabel: msdos
device: "{{ output }}"
- mkpart: primary
device: "{{ output }}"
start: 0%
end: 100%
part-tag: root-part
- mkfs: ext4
partition: root-part
- mount: root-part
fs-tag: root-fs
- unpack-rootfs: root-fs
- debootstrap: stretch
mirror: http://http.debian.net/debian
target: root-fs
unless: rootfs_unpacked
- apt: linux-image-amd64
fs-tag: root-fs
unless: rootfs_unpacked
- cache-rootfs: root-fs
unless: rootfs_unpacked
- chroot: root-fs
shell: |
apt -y install python
- shell: |
printf '[pc]\n%s hostname=pc\n' "$ROOT" > pc.inventory
ansible-playbook -i pc.inventory -c chroot pc.yml
root-fs: root-fs
- grub: bios
root-fs: root-fs
root-part: root-part
device: "{{ output }}"
~~~
It's a bit of a mouthful, and needs to be unpacked to be understood,
though every part is really quite simple. First of all, the file is in
[YAML][] format, which is a standard format not specific to `vmdb2` at
all. The top level is the `steps` key, which has as its value a list
of **steps**. The steps instruct `vmdb2` what it should do, and in
which order, to build the image.
[YAML]: https://en.wikipedia.org/wiki/YAML
The previous `vmdebootstrap` program hard coded the sequence of steps
to build an image, and provided command line options to vary the steps
in minor ways. The `vmdb2` approach gives more flexibility with less
complexity, and is more easily testable.
Each step consists of a set of key/value pairs. For example, the first
step creates a disk image file:
- mkimg: "{{ output }}"
size: 4G
`mkimg` identifies the step. The value is a [Jinja2][] template. In
this case, the template expands to the value of the `output` variable,
which contains the value of the `--output` command line option,
meaning the name of the output file. Jinja2 is a powerful templating
engine, and `vmdb2` uses it, amongst other reasons, to avoid
implementing a custom language for using variables.
[Jinja2]: http://jinja.pocoo.org/
The `size` key specifies the size of the disk image file that gets
created. `vmdb2` uses the `qemu-img` tool to create the image file and
the size value can be anything that `qemu-img` accepts.
The rest of the steps create a partition table (`mklabel`), a
partition (`mkpart`), and a filesystem (`mkfs`), as well as mount the
filesystem as the root filesystem. Partitions and mount points get
assigned **tags**, which allow later steps to refer to them, without
having to hardcode a directory or device name. Those tend to be a bit
random and vary from run to run, so hardcoding wouldn't work anyway.
The `debootstrap` step runs `debootstrap`, i.e., actually installs
Debian into the image. This can take a while, so `vmdb2` provides a
caching mechanism, to speed up iterations (unlike you, Lars never gets
things right the first one). This caching is done by using the
`--rootfs-tarball` command line option and the `unpack-rootfs` step,
and the `unless: rootfs_unpacked` key/value pair in selected steps.
Caching works like this: the filename given to `--rootfs-tarball` is
the cache filename. The `unpack-rootfs` looks for the cache file; if
it exists, it unpacks it, and sets the `rootfs_unpacked` variable to a
Boolean true value. Any other step can be tagged with `unless`, and if
the named value is true, the step is skipped. Thus, if the cache file
exists, the `debootstrap` step is skipped. Finally, the `cache-rootfs`
step creates a compressed tar archive of the filesystem, and that step
is also skipped if the `unless` tells `vmdb2` to skip it.
At the end, the `grub` step installs the GRUB bootloader. The example
installs a version compatible with traditional PC BIOS systems, which
works fine with virtual machine providers. A variabnt to support UEFI
systems is also available.
To run `vmdb2` with the above example, run a command like this:
vmdb2 pc.vmdb --output foo.img --rootfs-tarball foo.tar.gz
This tells `vmdb2` to read the image specification file from
`pc.vmdb`, place the output in `foo.img` and use `foo.tar.gz` as the
cache tar archive.
# Configuration
- command line options, list them and give summary of use
- cliapp config files, ini and yaml, and the fact that a long
option is always an acceptable variable in a config file
`vmdb2` has a number of command line options. Run `vmdb2 --help` to
see them all, or look at the manual page (`man vmdb2`).
Each long option (`--output` for example) can be a **configuration
variable** in a configuration file. The program looks up and reads
several configration files, such as `~/.config/vmdb2/*.conf` and
`*.yaml` in the same directory. `.yaml` files use a YAML syntax,
everything else uses INI file syntax.
Example: Store the following as `~/.config/vmdb2/vmdb2.yaml`:
config:
output: /tmp/vmdb2.img
log: /tmp/vmdb2.log
rootfs-tarball: /tmp/vmdb2.tar.gz
This is equivalent of running `vmdb2--output=/tmp/vmdb2.img
--log=/tmp/vmdb2.log --rootfs-tarball=/tmp/vmdb2.tar.gz`.
See **cliapp**(5) for more detailed information about configuration
files. cliapp is a Python library that `vmdb2` uses for command line
handling, configuration files, and such things.
# A .vmdb2 image specification file
- yaml, jinja2 syntax
- steps to list what needs to be done to build an image
- setting jinja2 variables from the command line
# List of available steps and their configuration
- list each step
- what is the step for? examples of how it could be used
- what mandatory and optional key/value pairs does it
understand? examples of their use
- the tag concept
# Examples
- various spec files, explained in detail
- simple image to run under kvm, qemu, or similar, boots with
normal grub (bios)
- same but with uefi boot
- same but software installed with ansible
- an LVM enabled image with encrypted / but cleartext /boot
|
