1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
|
Return-Path: <ick-discuss-bounces@ick.liw.fi>
X-Original-To: distix@pieni.net
Delivered-To: distix@pieni.net
Received: from yaffle.pepperfish.net (yaffle.pepperfish.net [88.99.213.221])
by pieni.net (Postfix) with ESMTPS id EB9594366F
for <distix@pieni.net>; Thu, 5 Apr 2018 12:32:59 +0000 (UTC)
Received: from platypus.pepperfish.net (unknown [10.112.101.20])
by yaffle.pepperfish.net (Postfix) with ESMTP id 8E754417BF
for <distix@pieni.net>; Thu, 5 Apr 2018 13:32:59 +0100 (BST)
Received: from ip6-localhost.nat ([::1] helo=platypus.pepperfish.net)
by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
id 1f444Z-0006Ip-HA; Thu, 05 Apr 2018 13:32:59 +0100
Received: from koom.pieni.net ([88.99.190.206] helo=pieni.net)
by platypus.pepperfish.net with esmtpsa (Exim 4.80 #2 (Debian))
id 1f444Y-0006IZ-8F
for <ick-discuss@ick.liw.fi>; Thu, 05 Apr 2018 13:32:58 +0100
Received: from exolobe3 (unknown [194.111.46.68])
by pieni.net (Postfix) with ESMTPSA id D2AFA406CF
for <ick-discuss@ick.liw.fi>; Thu, 5 Apr 2018 12:32:57 +0000 (UTC)
Message-ID: <1522931576.3943.6.camel@liw.fi>
From: Lars Wirzenius <liw@liw.fi>
To: ick-discuss@ick.liw.fi
Date: Thu, 05 Apr 2018 15:32:56 +0300
In-Reply-To: <1522918875.22924.0@ssh.steve.org.uk>
References: <1522917706.3679.0.camel@liw.fi>
<1522918875.22924.0@ssh.steve.org.uk>
X-Mailer: Evolution 3.22.6-1+deb9u1
Mime-Version: 1.0
X-Pepperfish-Transaction: 90b6-2916-27a0-7aff
X-Pepperfish-Transaction-By: platypus
Subject: Re: Architecture doc updated - feedback welcom
X-BeenThere: ick-discuss@ick.liw.fi
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: discussions about the ick CI system <ick-discuss-ick.liw.fi>
List-Unsubscribe: <https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi>,
<mailto:ick-discuss-request@ick.liw.fi?subject=unsubscribe>
List-Archive: <http://listmaster.pepperfish.net/pipermail/ick-discuss-ick.liw.fi>
List-Post: <mailto:ick-discuss@ick.liw.fi>
List-Help: <mailto:ick-discuss-request@ick.liw.fi?subject=help>
List-Subscribe: <https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi>,
<mailto:ick-discuss-request@ick.liw.fi?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============4343454811967208175=="
Mime-version: 1.0
Sender: ick-discuss-bounces@ick.liw.fi
Errors-To: ick-discuss-bounces@ick.liw.fi
--===============4343454811967208175==
Content-Type: multipart/signed; micalg="pgp-sha512";
protocol="application/pgp-signature"; boundary="=-5WtIjmDdBLMEPqvNwMkn"
--=-5WtIjmDdBLMEPqvNwMkn
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
On Thu, 2018-04-05 at 09:01 +0000, Steve Kemp wrote:
> > Any review and feedback would be welcome. Especially about things that
> > are unclear or missing.
>=20
> In your example project you use inline python to execute the
> three jobs "get_source", "build_ikiwiki_site" & "publish_html",
> but I see that the action of the last task is defined as:
>=20
> actions:
> - shell: |
>=20
> Which looks like a mistake, as it doesn't match the previous
> entries.
Correct, it should've been python. I've changed it to use a shell
snippet instead.
> In the section "Getting an access token" you have a broken/bogus
> markdown link "[Qvisqve][]".
Fixed. Thanks.
> The first choice, to let all things work on the same worker, is
> obviously simplest. It means that the existing workspace is present
> and you don't need to use your artifact storage-system to upload
> any in-progress work, and then refetch it. But I suspect that=20
> in complex pipelines running jobs in parallel would be nice.
Aye, ick will need to learn how to do builds concurrently on more than
one worker at a time. http://ick.liw.fi/blog/2018/03/04/thoughts_on_co
ncurrency_in_ick/ is my latest write-up on that. It's not going to
happen for ALPHA-6, however.
> With regard to credentials I wonder how you'd clone the source
> of a private repository, via git. I could imagine a (horrid)
> hack using:
>=20
> echo "ssh ..." > .ssh/id_rsa
> ssh-keyscan git.example.com > .ssh/known_hosts
> git clone ..
Currently, what I'm doing, is to have an ssh key on each build host,
copy that into the container, where a build is actually run, and then
use that key to access the git server, or the web server (most of my
ick jobs are about rendering static sites with ikiwiki and rsyncing
over ssh to publish the result).
This isn't very good design and will need to be fixed. After ALPHA-6,
though.
I'm currently thinking about a setup with "trusted" workers, which can
be trusted with credential for external systems such as git and web
servers. Or PGP keys. Normal building would happen on normal workers,
which only ever have credentials for getting an access token from the
IDP. They would upload build artifacts (say, the ikiwiki-generated
HTML files) to the artifact store, and the trusted worker would get
them from there and push them out using its ssh key.
Likewise, the trusted worker would be doing the git clone or git pull,
if it needs an ssh key.
--=-5WtIjmDdBLMEPqvNwMkn
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEETNTnrewG6wEE1EJ3bC+mFux6IDEFAlrGF3gACgkQbC+mFux6
IDHawxAAoZ5psftV7kk59Mi1FwJf8Vfv6ezM8w+hsC8NCsABxx18Z8SCRarQry12
uWtLHLQivMmhVKVzwhDl32g3NJRtg8s+o9InuLbjDg0kE0iq5qB1bgDjl8FBwNHZ
6lqIAWHThx5sNvTPg4oHELRgnM/JsVFI/q1WNY1rxpLTMXRXf9LpChv37QtSwg6Y
kCnymlopsYAxWG+5bwWhmDY/8+4Z5y6kk9htfzhsVNitDVHgHetSN3PaaWe94NjQ
QqCcS19PzHWI1+rJy8tw/TcIBNm8Lq+83ojmnKcvC/6CwbMilpIzU1B8a980etCj
S6MJCO86Td6UdmFUFB3xG+u8ZVeH4P6GQv7tBbpUQnrXloIaXM8zs0c+v2q9iCKd
EKZ97p4xQzBd9tYhbPCOcHmHoFL1Ccmp9GTQnP4namm4KE1P4UOoUXm2p/XFIYMi
g7IsKwfeuBFUJmolIS1rJ2yzEYkX1EQdU+pxZwXXa1TbTPAUj93TwopbzfEJ2df6
U6OZnviM5iie1W77GfoDvHFVDK8lwShudYmDNEZWa6l3x20MAQVo16WQ3l6ReZUw
E8E+gKrPOutoD9QUTxVJYCiDRgAkqBPyCPYiQdOE4NHSXXAD2RYTza6SZwWx2uf4
vIS1diwxYrXIl4crQbgspPxuzZXekivaXXRQn2vOMunwxUpoKQk=
=XlXF
-----END PGP SIGNATURE-----
--=-5WtIjmDdBLMEPqvNwMkn--
--===============4343454811967208175==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
ick-discuss mailing list
ick-discuss@ick.liw.fi
https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi
--===============4343454811967208175==--
|
