hetznertool - create VMs in the Hetzner Cloud for QvarnLabs Ab ============================================================================= `hetznertool` is a little utility to make it easier to manage VMs in the Hetzner Cloud. It's inspired by the OpenStack Heat tooling we've used before at QvarnLabs. * The user is expected to create Hetzner Cloud "projects" manually via the web interface: create project, add ssh keys, create an access token. * Everything else can be done with hetznertool, which wraps around the Hetzner hcloud tool. * User uses hetznertool to create a "context", giving it a name, and the access token from the web UI. hetznertool configures hcloud to use this context. * A new project and context is expected for each group of servers, similar to a "stack" in OpenStack. Hetznertool creates all servers in a group in the same context, and deletes all servers in a context. * Hetznertool reads server descriptitions from a "foo.hz" file, see below. This is needed for creation also. * Hetznertool updates DNS via the QvarnLabs DNS server, or via the dns-api.com service. * Hetznertool creates Ansible inventory files ("hosts files"). Requirements ============================================================================= * Python 3 * `hcloud` from * Download, unpack tarball from above URL. * Move the unpacked `bin/hcloud` into a location on the PATH. * `pass` for password storage. * `expect` for creating contexts (can be done manually with hcloud, without expect) * Get `hetznertool` with git clone from `ssh://git@git.qvarnlabs.net/hetznertool` * Maybe copy it somewhere on the PATH. Preparation: configure hetznertool ----------------------------------------------------------------------------- Create `~/.config/hetznertool/hetznertool.yaml` as a dict, where each dict entry is a "profile". You can have one profile for work and another for persoanl use. For each profile, define the full set of configutation variable. liw: ssh-key: liw-openpgp pass-dir: /home/liw/.password-store pass-subdir: hetzner ns1: '' domain: vm.liw.fi dnszone-dir: /home/liw/data/dns-api.com/zones dnszone-file: vm.liw.fi ansible-inventory-dir: /home/liw/code/ick/ick2-ansible style: dns-api ql: ssh-key: liw-openpgp pass-dir: /home/liw/qvarnlabs/code/qvarnlabs-secrets pass-subdir: infra/hetzner ns1: root@ns1.qvarnlabs.net domain: h.qvarnlabs.eu dnszone-dir: /home/liw/qvarnlabs/code/dnszone dnszone-file: db.h.qvarnlabs.eu ansible-inventory-dir: /home/liw/qvarnlabs/code/qvarn-prov/ansible style: qvarnlabs Variables: * `ssh-key`: name of ssh key to install for root on servers; this needs to be the name defined in the Hetzner Cloud web UI for the key * `pass-dir`: where the `pass` directory is * `pass-subdfir`: sub-directory where `pass` stores context tokens * `ns1`: ssh target (user@addr) where the primary name server runs, when using a self-hosted name server (e.g., QvarnLabs ns1.qvarnlabs.net); can be the empty string if `style` is not `qvarnlabs` * `style`: which style of DNS updates to do, must be one of `dns-api` or `qvarnlabs` * `domain`: domain under which DNS entries are created * `dnszone-dir`: directory where DNS zone files are created; it is assumed to be a git checkout * `dnszone-file`: file for the DNS zone file * `ansible-inventory-dir`: directory where inventory files are to be created Preparation: Hetzner project creation ----------------------------------------------------------------------------- Log into https://console.hetzner.cloud/projects Create projects as needed, add ssh keys to them, and create access tokens for each. Use the access tokens to create "contexts" by running `hetznercloud create-context FOO TOKEN`. Hetznertool puts the token in a password store (using `pass`), so it is shared among users of the same organisation. Use the access token at once when creating it: it cannot be recovered later (but a new one can be created at will). Not that it's not necessary to store the access token elsewhere, but do guard the `.config/hcloud/cli.toml` file, which is where `hcloud` stores the access tokens. `hcloud` does not seem to have a way to store that in `pass` or similar. Avoid sharing `cli.toml` with anyone. This only needs to be done once per project. Use ============================================================================= All operations MUST give the `-p profile` option, where the name of the profile is defined in the config file (see above). List ----------------------------------------------------------------------------- List all servers, with DNS names and IPv4 addresses: hetznercloud -p ql list Create ----------------------------------------------------------------------------- Create a server specification file (`qvarn.hz`): defaults: type: cx11 image: debian-9 hosts: - name: haproxy - name: qvarn1 - name: qvarn2 - name: qvarn3 - name: qvarn4 - name: qvisvqe - name: postgres type: cx51 Values from `defaults` are used if a dict in the list in `hosts` doesn't specify a value. Create servers (may take a minute per server): hetznertool -p ql create mydev qvarn.hz (where `mydev` is the name of an existing context). This will write inventory files and update DNS, creating names like `mydev-haproxy.h.qvarnlabs.eu`. Delete ----------------------------------------------------------------------------- Delete sll servers in a context: hetznertool -p ql delete mydev This will also rewrite zone and inventory files to update them to any remaining servers. To run Ansible ----------------------------------------------------------------------------- ansible-playbook -i hosts.mydev playbook.yml Problems? ============================================================================= Ask in the company Matrix room.