1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
|
Continuous fun with Ick
-----------------------------------------------------------------------------
Lars Wirzenius
---
Demo
-----------------------------------------------------------------------------
Ask me for access to demo instance
Note:
* apologize for the 90s aesthetic
* show web interface
* projects, a project
* builds
* build log
---
Ick?
-----------------------------------------------------------------------------
* Continuous integration, delivery, and deployment
* See "**Continuous Delivery**", by Humble, Farley
* Aims to be nice to install, run, admin, use
* Aims to be powerful but simple
* Not there yet
Note:
* you push changes to git server
* this triggers ick
* ick gets code from git server, builds, tests, deploys
vvv
WHY?
-----------------------------------------------------------------------------
* Needed something for myself
* Fed up with Jenkins
* implementation irritants
* architecture annoyances.
* Wrote a simplistic replacement in two weeks
* command line tool, not service
* fragile, but worked well enough for me
vvv
* Decided to make a **good** CI engine and service
* this will be the best software I ever write
* "NIH is strong with this one."
* did not like anything I looked at, but not an extensive survey
* language (don't like Java, Go)
* architecture (don't like Docker)
* My current hobby project. One of many.
* a couple of contributors, aiming to grow
* my new main project, after retiring Obnam
Note:
* I like to write my own crappy code, not configure other people's
silly stuff.
* Turns out there's many options now, and I can afford to pay for
cloudy stuff. Might not start Ick now.
---
Not ready for you
-----------------------------------------------------------------------------
* Seriously not ready
* Unless you like fixing things
* ALPHA quality software
* slow, lacks features, buggy, ...
---
Highlights
-----------------------------------------------------------------------------
* Builds run natively on the host, or in a systemd-nspawn container
* In my own active use
* builds, tests all my software
* builds, publishes .deb packages of my software
* builds my websites from source in git
* A demo instance with restricted access
Note:
* still feeling my way forward to what a good solution should be like
* on purpose not using Docker: it's too complicated
---
Tech stack
-----------------------------------------------------------------------------
* Python 3, gunicorn, bottle.py, python-requests
* haproxy for TLS
* data storage
* files on disk, for now
* switching to Muck, a custom key-to-JSON store
* Looking at switching to Rust instead of Python
---
The Ick Project
-----------------------------------------------------------------------------
* source: [git.liw.fi](http://git.liw.fi/ick2/)
* homepage: [ick.liw.fi](https://ick.liw.fi/)
* [bugs](https://ick.liw.fi/issues/),
[contact](https://ick.liw.fi/contact/),
[roadmap](https://ick.liw.fi/roadmap/)
* license: Affero GPL v3 or later
* governance based on lazy consensus, voting
* Contributor Covenant Code of Conduct
* explicit development iterations
* weird bug tracker (email+git)
---
Some day in the future
-----------------------------------------------------------------------------
* **Hostable**: secure, safe, reliable
* **Hosted**: open registration, free for free stuff
* **Distributed, federated**: a healthy ecosystem of providers of
components
* **Scalable**: large projects (Debian), many projects ("the world's CI")
* **Fast**: nearly no overhead over local build+test
* **Fun**: to use, not just to develop
vvv
<img src="roadmap.png" />
---
Architecture
-----------------------------------------------------------------------------
<img src="arch.png" />
vvv
* multiple components, potentially running on different hosts,
communicating over HTTPS using RESTful APIs and JSON, authenticating
via OAuth2, OpenID Connect
* **controller** - what to build, what's building now
* worker and **worker-manager** - actually build
* **qvisqve** - authenticate users, API clients (OAuth2, OIDC)
* **artifact store** - stores workspace, any build artifacts
* **apt** - .deb package repository
* **notification service** - tell people builds have finished
---
Data model
-----------------------------------------------------------------------------
* User defines **projects**
* A project invokes **pipelines** and defines **parameters**
* A pipeline consists of a sequence of **actions**
* Each action is executed by the worker-manger
* Action are affected by parameters
* Which git repo to clone, where to publish artifacts, ...
vvv
```yaml
project: hello
parameter:
whom: world
pipelines:
- hello
pipeline: hello
actions:
- where: host
python: |
whom = params['whom']
print('hello', whom)
```
---
Near future plans
-----------------------------------------------------------------------------
* good web UI - react; work is starting
* trigger service - tells controller when to start builds
* gets change events from git server, etc
* only for change to master?
* every night?
* when dependencies build successfully?
* yuck - replace qvisqve for authentication
* rock - new artifact store
---
Help?
-----------------------------------------------------------------------------
* use, give feedback
* improve documentation
* fix bugs
* adopt a component
* make new components
* security reviews
* ...
---
Thank you
-----------------------------------------------------------------------------
|
