summaryrefslogtreecommitdiff
path: root/README
blob: d48f2bc963a8aa7ec766f380ee9040978e4e17e1 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
hetznertool - create VMs in the Hetzner Cloud for QvarnLabs Ab
=============================================================================

`hetznertool` is a little utility to make it easier to manage VMs in
the Hetzner Cloud. It's inspired by the OpenStack Heat tooling we've
used before at QvarnLabs.

* The user is expected to create Hetzner Cloud "projects" manually via
  the web interface: create project, add ssh keys, create an access
  token.

* Everything else can be done with hetznertool, which wraps around the
  Hetzner hcloud tool.

* User uses hetznertool to create a "context", giving it a name, and
  the access token from the web UI. hetznertool configures hcloud to
  use this context.

* A new project and context is expected for each group of servers,
  similar to a "stack" in OpenStack. Hetznertool creates all servers
  in a group in the same context, and deletes all servers in a
  context.

* Hetznertool reads server descriptitions from a "foo.hz" file, see
  below. This is needed for creation also.

* Hetznertool updates DNS via the QvarnLabs DNS server, or via the
  dns-api.com service.

* Hetznertool creates Ansible inventory files ("hosts files").


Requirements
=============================================================================

* Python 3
* `hcloud` from <https://github.com/hetznercloud/cli/releases>
  * Download, unpack tarball from above URL.
  * Move the unpacked `bin/hcloud` into a location on the PATH.
* `pass` for password storage.
* Get `hetznertool` with git clone from
  `ssh://git@git.qvarnlabs.net/hetznertool`
  * Maybe copy it somewhere on the PATH.


Preparation: configure hetznertool
-----------------------------------------------------------------------------

Create `~/.config/hetznertool/hetznertool.yaml` as a dict, where each
dict entry is a "profile". You can have one profile for work and
another for persoanl use. For each profile, define the full set of
configutation variable.

    liw:
      ssh-key: liw-openpgp
      pass-dir: /home/liw/.password-store
      pass-subdir: hetzner
      ns1: ''
      domain: vm.liw.fi
      dnszone-dir: /home/liw/data/dns-api.com/zones
      dnszone-file: vm.liw.fi
      ansible-inventory-dir: /home/liw/code/ick/ick2-ansible
      style: dns-api
    ql:
      ssh-key: liw-openpgp
      pass-dir: /home/liw/qvarnlabs/code/qvarnlabs-secrets
      pass-subdir: infra/hetzner
      ns1: root@ns1.qvarnlabs.net
      domain: h.qvarnlabs.eu
      dnszone-dir: /home/liw/qvarnlabs/code/dnszone
      dnszone-file: db.h.qvarnlabs.eu
      ansible-inventory-dir: /home/liw/qvarnlabs/code/qvarn-prov/ansible
      style: qvarnlabs

Variables:

* `ssh-key`: name of ssh key to install for root on servers; this needs
  to be the name defined in the Hetzner Cloud web UI for the key

* `pass-dir`: where the `pass` directory is

* `pass-subdfir`: sub-directory where `pass` stores context tokens

* `ns1`: ssh target (user@addr) where the primary name server runs,
  when using a self-hosted name server (e.g., QvarnLabs
  ns1.qvarnlabs.net); can be the empty string if `style` is not `qvarnlabs`

* `style`: which style of DNS updates to do, must be one of `dns-api`
  or `qvarnlabs`

* `domain`: domain under which DNS entries are created

* `dnszone-dir`: directory where DNS zone files are created; it is
  assumed to be a git checkout

* `dnszone-file`: file for the DNS zone file

* `ansible-inventory-dir`: directory where inventory files are to be
  created


Preparation: Hetzner project creation
-----------------------------------------------------------------------------

Log into

    https://console.hetzner.cloud/projects

Create projects as needed, add ssh keys to them, and create access
tokens for each. Use the access tokens to create "contexts" by running
`hetznercloud create-context FOO TOKEN`. Hetznertool puts the token in
a password store (using `pass`), so it is shared among users of the
same organisation.

Use the access token at once when creating it: it cannot be recovered
later (but a new one can be created at will). Not that it's not
necessary to store the access token elsewhere, but do guard the
`.config/hcloud/cli.toml` file, which is where `hcloud` stores the
access tokens. `hcloud` does not seem to have a way to store that in
`pass` or similar. Avoid sharing `cli.toml` with anyone.

This only needs to be done once per project.


Use
=============================================================================

All operations MUST give the `-p profile` option, where the name of
the profile is defined in the config file (see above).


List
-----------------------------------------------------------------------------

List all servers, with DNS names and IPv4 addresses:

    hetznercloud -p ql list

Create
-----------------------------------------------------------------------------

Create a server specification file (`qvarn.hz`):

    defaults:
      type: cx11
      image: debian-9
    hosts:
      - name: haproxy
      - name: qvarn1
      - name: qvarn2
      - name: qvarn3
      - name: qvarn4
      - name: qvisvqe
      - name: postgres
        type: cx51

Values from `defaults` are used if a dict in the list in `hosts`
doesn't specify a value.

Create servers (may take a minute per server):

    hetznertool -p ql create mydev qvarn.hz

(where `mydev` is the name of an existing context). This will write
inventory files and update DNS, creating names like
`mydev-haproxy.h.qvarnlabs.eu`.

Delete
-----------------------------------------------------------------------------

Delete sll servers in a context:

    hetznertool -p ql delete mydev

This will also rewrite zone and inventory files to update them to any
remaining servers.

To run Ansible
-----------------------------------------------------------------------------

    ansible-playbook -i hosts.mydev playbook.yml


Problems?
=============================================================================

Ask in the company Matrix room.