Add: README

1 files changed, 184 insertions, 0 deletions

diff --git a/README b/README
new file mode 100644
index 0000000..4ecd086
--- /dev/null
+++ b/README
@@ -0,0 +1,184 @@
+hetznertool - create VMs in the Hetzner Cloud for QvarnLabs Ab
+=============================================================================
+
+For Hetzner we want something to allow us to easy create VMs, run
+Ansible against them, and have them have DNS names. Here's a first
+stab at what it might look like. First some general observations:
+
+* Hetzner does not separate between projects and stacks. There is a
+  project abstraction, but no stack abstraction. However, unlike with
+  OpenStack, we can add projects easily ourselves. Hetzner calls
+  projects "contexts", in some places.
+
+  Projects need to be created via the web interface ("cloud console").
+  Also via the web interface, ssh keys are uploaded (separately for
+  each project), and access tokens generated (also for each project).
+  Each user should have their unique access token.
+
+  After a project is created, the `hcloud` software is used to create
+  a "context", creation of which requires the access token.
+
+  This part is all manual, at least for now, but it's a one-time
+  operation.
+
+* Hetzner also doesn't seem to care about more than VMs. Each VM gets
+  a public IP, and there's no firewall etc rules to handle.
+
+* Hetzner has an API, but also has a command line tool (`hcloud`),
+  which seems like it's going to be nicer to use, at least initially.
+  (We had small issues with the OpenStack API changing from underneath
+  us.)
+
+The workflow with Hetzner could look like this:
+
+* We create a "project" in the Hetzner cloud. We create per-project
+  access tokens, and store those using `pass` (in a personal instance,
+  not shared).
+
+* We write a tool that reads a description of the set of VMs to be
+  created for a project. The descrioption might look like this:
+
+        defaults:
+            image: debian9
+            type: cx11
+        hosts:
+          - name: haproxy
+            type: cx51
+          - name: qvarn1
+          - name: qvarn2
+
+* VM creation tool would be invoked like this:
+
+        hetznertool create liwproject foo.yaml --ssh-key liw-openpgp
+
+  Here `liwproject` is the (short) name for the project (context) in
+  which to create the VMs, `foo.yaml` is the description file, and
+  `liw-openpgp` is the ssh key name to install in the new VM by
+  default.
+
+* The tool will run `hcloud` with the appropriate parameters, read
+  from the description file and command line options, and also create
+  corresponding DNS entries: `liwproject-haproxy.h.qvarnlabs.eu`.
+  Additionally it will write an Ansible inventory file that can be
+  used with `ansible-playbook -i` (one inventory file per Hetzner
+  context, listing VMs in the context in a way that our playbooks can
+  refer to them with generic names).
+
+* The tool will wait until the VMs are created.
+
+* Later on, we can add support for deleting all the VMs in context:
+
+        hetznertool delete liwproject
+
+In more detail
+=============================================================================
+
+Preparation: Hetzner project creation
+-----------------------------------------------------------------------------
+
+Log into
+
+    https://console.hetzner.cloud/projects
+
+Create projects as needed, add ssh keys to them, and create access
+tokens for each. Use the access tokens to create "contexts" for
+`hcloud` (run `hcloud context create FOO`, it will ask for the access
+token).
+
+Use the ssh token at once when creating it: it cannot be recovered
+later (but a new one can be created at will). Not that it's not
+necessary to store the access token elsewhere, but do guard the
+`.config/hcloud/cli.toml` file, which is where `hcloud` stores the
+access tokens. `hcloud` does not seem to have a way to store that in
+`pass` or similar. Avoid sharing `cli.toml` with anyone.
+
+This only needs to be done once per project.
+
+Preparation: Configure `hetznertool`
+-----------------------------------------------------------------------------
+
+Create `~/.config/heznertool/hetznertool.yaml` with the following
+content (except adjust paths for your files):
+
+    dnszone-dir: ~/qvarnlabs/code/dnszone
+    dnszone-file: db.hetzner
+    ansible-inventory-dir: .
+
+The tool will write zone file (`db.hetzner`) in the zone directory
+(`~/qvarnlabs/code/dnszone`), git comit that, and do a git push.
+
+The tool will write the Ansible inventory files to the directory
+specified with `ansible-inventory-dir` (defaulting to the current
+working directory).
+
+Input file: VM descriptions
+-----------------------------------------------------------------------------
+
+The `default` item has defaults, which will be used of a host doesn't
+override them.
+
+* `image` specifies the VM image to use.
+* `type` is the type of VM to create (CPUs, RAM, disk)
+
+Under `hosts`, a list of dicts, each dict describing one VM:
+
+* `name` is the name of the VM
+* `image`, `type` are optional: if present, they override the value
+  from `defaults`, and if missing, the value from `defaults` is used
+
+There may later be more parameters, if they're needed.
+
+Action: create VMs
+-----------------------------------------------------------------------------
+
+For each VM in the description file, compute the full description
+(fill in missing values from `defaults`), and then run:
+
+    hcloud context use CONTEXT
+    hcloud server create --image IMAGE --name NAME --type TYPE
+
+The `CONTEXT` comes from the `hetznertool` command line.
+
+
+Action: update DNS
+-----------------------------------------------------------------------------
+
+Get list of servers by running `hcloud server list`, which gives
+output like this:
+
+    ID       NAME         STATUS    IPV4              IPV6
+    642236   controller   running   94.130.180.87     2a01:4f8:1c0c:4054::/64
+    642237   qvisqve      running   195.201.99.89     2a01:4f8:1c0c:75f4::/64
+    642238   apt          running   195.201.42.198    2a01:4f8:1c0c:7e66::/64
+    642239   artifacts    running   94.130.184.219    2a01:4f8:1c0c:69ba::/64
+    642240   worker1      running   195.201.112.70    2a01:4f8:1c1c:1871::/64
+    642241   worker2      running   195.201.144.105   2a01:4f8:1c1c:1872::/64
+
+Parse this to extract the name and IPv4 addresses. Do this for every
+context (`hcloud context list`).
+
+Update `dnszones.git`, file `db.hetzner` to add a line for each
+VM in each context:
+
+    {CONTEXT}-{NAME} IN A {IPV4}
+
+(It is assumed that the `qvarnlabs.eu` DNS servers are configured to
+honor the `db.hetzner` file.)
+
+Commit the new file to git, and push to the server.
+
+Copy the new file to `ns1.qvarnlabs.net` and `ns2.qvarnlabs.net` and
+restart the Bind server on `ns1`.
+
+
+Action: write inventory files
+-----------------------------------------------------------------------------
+
+For each context in the Hetzner cloud, write an Ansible inventory
+file `hosts.{CONTEXT}`:
+
+    {SERVER} ansible_ssh_host={IPV4}
+
+Write these files to the current working directory. The
+`ansible_ssh_host` is there so Ansible can be run at once and does not
+need to wait for DNS to be updated.