1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
|
Return-Path: <ick-discuss-bounces@ick.liw.fi>
X-Original-To: distix@pieni.net
Delivered-To: distix@pieni.net
Received: from yaffle.pepperfish.net (yaffle.pepperfish.net [88.99.213.221])
by pieni.net (Postfix) with ESMTPS id D902C42F0D
for <distix@pieni.net>; Tue, 26 Jun 2018 16:10:48 +0000 (UTC)
Received: from platypus.pepperfish.net (unknown [10.112.101.20])
by yaffle.pepperfish.net (Postfix) with ESMTP id 9F812417C0
for <distix@pieni.net>; Tue, 26 Jun 2018 17:10:48 +0100 (BST)
Received: from ip6-localhost.nat ([::1] helo=platypus.pepperfish.net)
by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
id 1fXqYK-0003jg-Is; Tue, 26 Jun 2018 17:10:48 +0100
Received: from koom.pieni.net ([88.99.190.206] helo=pieni.net)
by platypus.pepperfish.net with esmtpsa (Exim 4.80 #2 (Debian))
id 1fXqYI-0003jJ-SN
for <ick-discuss@ick.liw.fi>; Tue, 26 Jun 2018 17:10:46 +0100
Received: from exolobe3 (62-78-212-250.bb.dnainternet.fi [62.78.212.250])
by pieni.net (Postfix) with ESMTPSA id 4BB2842C80
for <ick-discuss@ick.liw.fi>; Tue, 26 Jun 2018 16:10:46 +0000 (UTC)
Message-ID: <ef63cf721d3bc2accd26c8d86ee322dcbde246aa.camel@liw.fi>
From: Lars Wirzenius <liw@liw.fi>
To: ick discussions <ick-discuss@ick.liw.fi>
Date: Tue, 26 Jun 2018 19:10:41 +0300
X-Mailer: Evolution 3.28.2-1
Mime-Version: 1.0
X-Pepperfish-Transaction: 4507-3886-d519-a9cc
X-Pepperfish-Transaction-By: platypus
Subject: Distributed CI
X-BeenThere: ick-discuss@ick.liw.fi
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: discussions about the ick CI system <ick-discuss-ick.liw.fi>
List-Unsubscribe: <https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi>,
<mailto:ick-discuss-request@ick.liw.fi?subject=unsubscribe>
List-Archive: <http://listmaster.pepperfish.net/pipermail/ick-discuss-ick.liw.fi>
List-Post: <mailto:ick-discuss@ick.liw.fi>
List-Help: <mailto:ick-discuss-request@ick.liw.fi?subject=help>
List-Subscribe: <https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi>,
<mailto:ick-discuss-request@ick.liw.fi?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============3970042335603488410=="
Mime-version: 1.0
Sender: ick-discuss-bounces@ick.liw.fi
Errors-To: ick-discuss-bounces@ick.liw.fi
--===============3970042335603488410==
Content-Type: multipart/signed; micalg="pgp-sha512";
protocol="application/pgp-signature"; boundary="=-/eOJnGcNVZ8bINwC8SEr"
--=-/eOJnGcNVZ8bINwC8SEr
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
We've had a short disucssion on IRC about maybe adding support to Gitano (a
git server) and ick for ActivityPub instead of having explicit triggers
configured in Gitano to trigger builds in ick. I thought I'd expand on that
a little bit.
The thing I'm grasping at is that currently, each git server instance needs
to notify each CI instance separetely, creating a fairly tight coupling.
Alternatively, each CI instance needs to poll each git server, which is
unworkable at scale.
So currently, if there are three users, one using github.com, one using
gitlab.com, and one using the Debian gitlab instance (salsa), and each user
is developing their own software, then each user needs to configure their
own git server to notify their own CI server. So far, so good.
However, if each of the three also wants to build and deploy each other's
software, so that if user A makes a change, it gets put through CI by A's,
B', and C's CI, and then deployed to A's, B's, and C's production server,=
=20
and likewise for B's and C's software, then each of the three needs to add
a trigger hook on each of th three git repositories. That's three
repositories, three CI servers, and a total of nine trigger hooks.
That's still manageable, but there are problems. Not all git servers allow
random strangers to add trigger hooks on each repository. Also, three
repositories and three CI servers is small potatoes. Imagine having fifty
thousand repositories! Debian has about that many binary packages.
Also imagine that those fifty thousand repositories are used by a million
users. Not all of them by all users, but many repositories by each of the
million. Having millions of hooks does not scale. Having that much polling
also doesn't scale.
So here's the solution I'm thinkin of: instead of having the git server
notify each CI server, have the git server send a message to an ActivityPub
server, and have each CI server listen on such messages on their own
ActivityPub server. The AP servers federate (send messages to each other),
so each git server and each CI server only needs to post to and listen on
one AP server.
With this design, many-to-lots communication becomes feasible.
A makes a change, and pushes to their git server. The git server posts a
message to its own AP server, saying "this branch on this repo on this git
server has changed, new commit it CAFEBEEF." The AP server knows who's
"following" A's repository, and so the message flows out to the AP server
network to every AP server where a CI server is interested in that
repository. Each of those CI servers can then trigger a build and deploy of
A's software.
I don't actually know much about ActivityPub yet. It's the protocol behind
the Mastodon system (https://joinmastodon.org/), and it works fine for a
Twitter clone. I don't know yet if it would work for what I'm talking about
above, but I think so.
Of course, doing distributed CI at this scale might not be something that
anyone actually cares about. We'll see. But even at a small scale, I'd like
to break the necessity of the git server having to know about each CI
server that is interested in each of its repositories.
Also, this needs thinking for non-public repositories.
--=-/eOJnGcNVZ8bINwC8SEr
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEETNTnrewG6wEE1EJ3bC+mFux6IDEFAlsyZYEACgkQbC+mFux6
IDHDoxAAk/e441kEhfF9vQl3BCMddPUvAoiPGeLLY1GSw+LU5BcBb/SVTwNlQ7uC
bVut6uMDlIFrs4YKfqfWRBOLGz0ymtlaql8lDAb/wg8yfFehz1As073lZHbccmoI
Ld8R+Cq9gg7hMt0lOMWhjgvGPy4eCcpy8cP7yaonp3Jb1eXd3NbqUMWo04YKQClH
4ZyKW7S0M3yXgC+yubZQMDwouzLD1elKduZnFw3v1sftaEobBmZ0t7bWlJL8/hD/
40INEpTecAvfhIIDyqPIqljDANZpMJlHxQZLSZjwEAxCd7IILTvHIxojz97Vri7j
KE7t86FkDngVP3cyDit4wscPsLcdRfmJXgXrbxht4fWlLkRjB5ARy4uTUYpwp71i
VwiSbNPFD5G0klKpxGNI8fVIh/kV+fYdJ1c1OQA1cSvIZ6TevN5AXcd4yxchUum5
XsLfWHdp3S5Q1ulTYKBYjCZ05f9toydXmKXtRBRlAvBluqIV75NGCGkjn44xm+P9
6yfOp4u3S4Qspw5oCqd/CaUokK84301Ed9r5gZpeDp8mSR+nxGGt/6thOfEDghyM
aSGIHF99VtAPZPYTb5zSFpT6jbckU6dDVmBkFIQHvNDFJZFGk74G+yDPTmGv55bs
uCGj3KtGfAqVPLLAQyrovUl12XPUkeWKo5FSIBqfCbBf1N5ppeI=
=obrr
-----END PGP SIGNATURE-----
--=-/eOJnGcNVZ8bINwC8SEr--
--===============3970042335603488410==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
ick-discuss mailing list
ick-discuss@ick.liw.fi
https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi
--===============3970042335603488410==--
|
