1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
|
NEWS for vmdb2, the Debian disk image builder
=============================================================================
Version 0.22+git, not yet released
-----------------------------------------------------------------------------
* Matthias Klein added the optional `options` field to the `mkfs`
step to set aditional options for mkfs.
Version 0.22, released 2021-01-16
-----------------------------------------------------------------------------
* This release has a lot of changes from Ryutaroh Matsumoto and
Christian Kastner to fix and clean up things, and to add support for
building images for I385 (UEFI and BIOS), armhf+UEFI, and ppc64el.
Version 0.21, released 2020-12-18
-----------------------------------------------------------------------------
* The grub plugin is able to install the UEFI version of grub on arm64
targets, patch from David Edmondson.[
* The documentation tile for the `resize_rootfs` step is now fixed.
Version 0.20, released 2020-10-18
-----------------------------------------------------------------------------
* All use of the Python `cliapp` library has been dropped from vmdb2 now.
* New step `resize-rootfs` adds a systemd service to resize the root
file system to fit the actual drive on boot.
Version 0.19, released 2020-09-16
-----------------------------------------------------------------------------
Change in this release are entirely internal code changes that should
not affect users of vmdb2.
* Python Code has been formatted with the Black code formatting tool.
* Changes have been made towards dropping the dependency on the
`cliapp` library: the Python standard library module `subprocess` is
used instead of `cliapp.runcmd`, and the `cliapp` plugin system has
been replaced with a custom one.
Version 0.18, released 2020-08-13
-----------------------------------------------------------------------------
* The way the manual was formatted could sometimes lead to different
plugins' documentation being merged into the same section. This has
now been fixed.
Version 0.17, released 2020-07-13
-----------------------------------------------------------------------------
* The luks plugin now runs the `cryptsetup` utility with the `-q`
(alias for `--batch-mode`) option for batch mode, to prevent the
utility from asking any questions of the user. Fix by Birger
Schacht.
Version 0.16, released 2020-06-06
-----------------------------------------------------------------------------
* Progress output is now flushed after every write. This matters, when
the output goes to a pipe, because then Python would buffer the
output, meaning that progress output gets delayed until the buffer
fills up, and that's useless.
* The documentation is now formatted using Subplot. Previously, the
manual and the acceptance test suite were split into separate, but
confusingly named files (vmdb2.md vs vmdb2.mdwn); now they are
combined. The manual has been cleaned up a bit.
* Add a `quiet` field to the **grub step** to configure the kernel
boot to be quiet or not. Default is now not quiet, which is a change
from before.
* Add a `timeout` field to the **grub step** to configure grub menu
timeout. Default is now zero seconds, which is a change from the
Debian default of five seconds.
* Symbolic links can now be used in the **mkpart step** for device
files, thanks to Tim Small.
Version 0.15, released 2020-05-05
-----------------------------------------------------------------------------
### New and changed features
* Document and rename the `virtual-filesystems` plugin. This allows
installation of Debian packages that require `/proc` to be mounted
to install, such as `ca-certificates-java`.
Undo previous changes to have `/proc` automatically mounted in the
chroot, when running commands in the chroot. That wasn't enough
(e.g., it didn't cover Ansible), and `virtual-filesystems` seems
generally like the more general and better solution.
The `grub` step handles the filesystems it needs itself, but doesn't
mind if they're already mounted.
This change should NOT require changes to existing, working .vmdb
files. If it does, please report a bug.
* The .vmdb file can now be passed in via the standard input, by
specifying `-` as the filename. Patch from voney.
* The manual now documents that un-partitioned images are not
supported, at least not yet.
* The "All went fine" message is now written at the very end of the
vmdb2 run, if at all. Previously all the messages from teardown
parts of steps were printed after it, hiding it effectively.
* The standard output and error output of the commands vmdb2 runs are
no longer logged in real time, only after the command has finished.
This is a side effect of the `cliapp.runcmd` helper function no
longer being used. The Python standard library subprocess module is
used directly instead. This change is a step to drop the dependency
on the cliapp library.
### Bug fixes
* The `grub` step documentation now uses the correct key for
specifying the image device (`image-dev`). Thanks: Sebastian
Bachmann.
* The `mkpart` step now waits for the new device node to be created.
Previously it sometimes took enough time to be created that the next
step failed. This only affects when the output disk is a real block
device, not a disk image file.
### Changes mainly affecting vmdb2 and plugin developers
* The `./check` script only runs the yarn integration testing tool if
it's new enough to have the yarnutils Python library included. Yarn
is being replaced with Subplot.
* Step runners must now use a new way of specifying expected and
allowed keys for steps. The new way specifies types for mandatory
keys, and default values for optional keys. The old
`get_required_keys` method is gone. This change makes mistakes in
step runner code a little less likely, and opens up the possibility
of generating documentation about step keys automatically.
* Failures when un-mounting filesystems are now ignored. Such failures
can happen for various reasons, but by far the most common reason is
because the mount had already been unmounted and it's simpler to
just ignore failures than keep meticulous track of what's been
unmounted already.
Version 0.14.1
-----------------------------------------------------------------------------
This is a bug fix release to fix bugs in the 0.14 release that slipped
through due to insufficient testing.
* The shell plugin source file was misnamed. Fixed now.
* Fixes to problems found while building real images. The new
`check-external` script builds my own images, and Debian's Raspberry
Pi images.
Version 0.14, released 2020-03-21
-----------------------------------------------------------------------------
It has been two years since the previous release. These release notes
are a little sketchy.
* All plugins now provide exactly one type of step. This is a little
simpler than having multiple steps provided by the same plugin.
Simplicity is good.
* There is now some better automated testing.
* vmdb2 now checks at start time that all required keys are set for
all steps and that keys have values of the correct type.
* Source code for vmdb2 is now hosted on gitlab.com and git.liw.fi,
moved there from GitHub.
* Fixed smoke test timeout for starting a VM to 5 min. The previous
timeout (30 s) was short enough that tests often failed because of
it.
* The `debootstrap` action now finishes with `apt-get update`. For
some reason thing had changed so that this became necessary.
* Christian Schlüter added the optional `label` field to the `mkfs`
step to set the filesystem label upon creation.
* Changed the `grub` step to take an optional `image-dev` field to
override the heuristics for finding the device where GRUB is to be
installed. This was necessary to allow vmdb2 to install Debian onto
real hardware.
* The smoke test now tests a UEFI image as well.
* The rootfs unpacking action now copies `/etc/resolv.conf` into the
chroot, overwriting what the tarball has. This allows vmdb2 to work
with a tarball generated in a different network location.
* Add plugins for doing full-disk encryption via cryptsetup, and LVM2.
* There is now some documentation.
* Christian Schlüter added support for f2fs labels in the `mkfs` step.
* Step runners may now implement the `run_even_if_skipped` method to
have code that is run whether the step is skipped or not. The
`debootstrap` step now uses this to run `apt-get update` always.
This allows a rootfs tarball that is old to be used, without the
Packages files being too old to be usable.
* The `cache_rootfs` step now caches all the explicitly mounted
filesystems, not just the root filesystem.
Version 0.13.2, released 2018-05-06
------------------------------------
* New build, after CI screwed up.
Version 0.13.1, released 2018-04-30
----------------------------------
* Fix build-dependency on pandoc.
Version 0.13.1, released 2018-04-30
----------------------------------
Version 0.12, released 2018-02-24
----------------------------------
* Minor Debian packaging fixes.
Version 0.11, released 2018-02-10
----------------------------------
* Build fixes so that .deb doesn't contain Python 2.7 stuff, which
isn't wanted or used.
Version 0.10, released 2018-02-10
---------------------------------
* Build tweaks to make builds work on Debian unstable.
Version 0.9, released 2017-10-11
---------------------------------
* Stuart Prescott added a `variant` parameter to the `vmdebootstrap`
plugin.
* Stuart and Lars documented some of the dependencies in README. If
they're insufficient, report a bug please.
* Stuart added a `qemu-debootstrap` plugin to build images for a
foreign architecture.
* Lars Wirzenius added the function `vmdb.runcm_chroot` for executing
programs inside a chroot. It's a short wrapper around `vmdb.runcmd`,
but is a little easier to get right than invoking **chroot**(8) via
`runcmd`. Michael Stapelberg fixed the function.
* Lars Wirzenius changed it so that `.vmdb` files can have structured
YAML values and their constituent strings are used as Jinja2
templates. Previously vmdb2 would just crash. This opens a
possibility to have things like "list of packages to install" for
the `apt` step as a YAML list, instead of having the step runner
parse a space delimited list.
* Michael Stapelberg added the `fs-type` parameter to the `mkpart`
step so that Raspberry Pi images can be created.
* Lars Wirzenius changed the `apt` step. Previously the `apt` field
values was the name of one package to install. Now the value MUST be
`install` and a separate field `packages` is a YAML list of package
names to install. These will all be installed at once.
* Michael Stapelberg and Lars Wirzenius added support for a
`components` list to the `qemu-debootstrap` step, to allow giving a
list of components to the debootstrap run by the step.
Version 0.8, released 2017-07-29
---------------------------------
* The `mount` plugin can now specify a mount point on an already
mounted filesystem, such as /boot inside the root filesystem already
mounted. This allows constructing a system with a separate /boot
partition. Thanks to Stuart Prescott for reporting the lack of this
feature.
Version 0.7, released 2017-06-18
---------------------------------
* New plugin `virtuals` provides step `mount-virtual-filesystems` for
mounting virtual filesystems such as `/proc` and `/dev` as well.
* The `apt` step installs `eatmydata` and runs `apt` under it, to
speed up package installs.
Version 0.6, released 2017-06-11
---------------------------------
* setup.py now installs the plugins, making the .deb package actually
be usable.
Version 0.5, released 2017-06-04
---------------------------------
* The "unless:" part of vmdb spec files now actually works. Previously
it was entrely unimplmented (there was code to implement the check,
but it was never actually called). The apt and debootstrap plugins
have been fixed to not do "has rootfs tarball been unpacked" checks
themselves.
Version 0.4, released 2017-06-03
---------------------------------
* Add a rudimenteary smoke test yarn for vmdb2-built images.
* Add a plugin to run Ansible to configure an image at build time.
Doing this via Ansible saves me from having to write equivalent
functionality directly into vmdb2, which would be quite a lot of
work. (If someone wants puppet, a plugin for that is probably easily
doable, assuming it can run against a chroot. I don't now, I've
never used puppet.)
Version 0.3, released 2017-05-21
---------------------------------
* Simplify progress reporting to go to stdout, plus stop logging
progress reports ar as errors.
* Add a BIOS flavor for installing GRUB.
Version 0.2, released 2017-05-14
---------------------------------
* Add plugin to provide steps to cache the rootfs, and to unpack the
rootfs from the cache, instead of running debootstrap or installing
packages. This speeds up iteration time from about 9 minutes to 40
seconds on my laptop. See small.vmdb for an example.
* Add a generic "unless this condition is true" functionality to
steps. If a step has a field "unless: foo" it is skipped the
variable foo exists and is true. The variables are set by steps, and
currently only the rootfs unpacking step sets a variable (the
`rootfs_unpacked` variable). This allows debootstrap to be skipped
if the rootfs has already been created by unpacking a cached
tarball.
Version 0.1, released 2017-05-13
-----------------------------------------------------------------------------
This is the first release. It can build a UEFI image for the amd64
architecture. It's not meant to really be useful for other people.
|