1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
|
Return-Path: <ick-discuss-bounces@ick.liw.fi>
X-Original-To: distix@pieni.net
Delivered-To: distix@pieni.net
Received: from yaffle.pepperfish.net (yaffle.pepperfish.net [88.99.213.221])
by pieni.net (Postfix) with ESMTPS id BE78243816
for <distix@pieni.net>; Thu, 26 Jul 2018 14:13:38 +0000 (UTC)
Received: from platypus.pepperfish.net (unknown [10.112.101.20])
by yaffle.pepperfish.net (Postfix) with ESMTP id 7F21F41468
for <distix@pieni.net>; Thu, 26 Jul 2018 15:13:38 +0100 (BST)
Received: from ip6-localhost.nat ([::1] helo=platypus.pepperfish.net)
by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
id 1fih1O-0003jX-ED; Thu, 26 Jul 2018 15:13:38 +0100
Received: from koom.pieni.net ([88.99.190.206] helo=pieni.net)
by platypus.pepperfish.net with esmtpsa (Exim 4.80 #2 (Debian))
id 1fih1M-0003io-MB
for <ick-discuss@ick.liw.fi>; Thu, 26 Jul 2018 15:13:36 +0100
Received: from exolobe3.liw.fi (mobile-access-5d6a6b-181.dhcp.inet.fi
[93.106.107.181]) by pieni.net (Postfix) with ESMTPSA id E321040A82
for <ick-discuss@ick.liw.fi>; Thu, 26 Jul 2018 14:13:35 +0000 (UTC)
Received: from exolobe3.liw.fi (localhost [127.0.0.1])
by exolobe3.liw.fi (Postfix) with ESMTPS id 0634488041E
for <ick-discuss@ick.liw.fi>; Thu, 26 Jul 2018 17:13:34 +0300 (EEST)
Date: Thu, 26 Jul 2018 17:13:32 +0300
From: Lars Wirzenius <liw@liw.fi>
To: ick-discuss@ick.liw.fi
Message-ID: <20180726141332.GA32297@exolobe3.liw.fi>
References: <20180726130528.xgtftruk6srzgi4x@somnambulist.local>
MIME-Version: 1.0
In-Reply-To: <20180726130528.xgtftruk6srzgi4x@somnambulist.local>
User-Agent: Mutt/1.10.1 (2018-07-13)
X-Pepperfish-Transaction: 209b-579f-e244-76b2
X-Pepperfish-Transaction-By: platypus
Subject: Re: Proposal: an icktool lint subcommand
X-BeenThere: ick-discuss@ick.liw.fi
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: discussions about the ick CI system <ick-discuss-ick.liw.fi>
List-Unsubscribe: <https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi>,
<mailto:ick-discuss-request@ick.liw.fi?subject=unsubscribe>
List-Archive: <http://listmaster.pepperfish.net/pipermail/ick-discuss-ick.liw.fi>
List-Post: <mailto:ick-discuss@ick.liw.fi>
List-Help: <mailto:ick-discuss-request@ick.liw.fi?subject=help>
List-Subscribe: <https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi>,
<mailto:ick-discuss-request@ick.liw.fi?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============2072883612514617144=="
Mime-version: 1.0
Sender: ick-discuss-bounces@ick.liw.fi
Errors-To: ick-discuss-bounces@ick.liw.fi
--===============2072883612514617144==
Content-Type: multipart/signed; micalg=pgp-sha512;
protocol="application/pgp-signature"; boundary="qMm9M+Fa2AknHoGS"
Content-Disposition: inline
--qMm9M+Fa2AknHoGS
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Thu, Jul 26, 2018 at 02:05:28PM +0100, Daniel Silverstone wrote:
> I have noticed, over time, that it's possible to get into situations
> where the controller is in a bit of an inconsistent or messy state.
>=20
> To that end I'd like to suggest the introduction of a set of lints which
> can be run against the controller to tell you about it (or a subset of it=
).
I am in full support of this idea. I have some suggestions. We should
definitely do some form of this idea, preferably sooner. I too have
spent long evening trying to get all the details right.
> * Projects refer to pipelines which do not exist
> * Projects refer to pipelines but do not have all the requisite parameters
> * Pipelines exist which are not referred to by any project
> * Pipelines exist with no actions
I would add the following check:
* Project defines a parameter that none of its pipelines ask for.
The first implementation thought I have is that we should make it easy
to write checks. If it's easy, we'll have more, and they'll more
likely be correct.
My second implementation thought is it would be nice to be able to
run the checks before putting the projects and pipelines into the
controller. Thus, I'd do it client side. It would be nice if the
client could read the projects and resources from the controller, or
=66rom a local file, though. Local file is more important: if we have
that, we can use "icktool show" to list all projects and pipelines.
My third implementation thought is that I'd like this to not be part
of icktool, which is growing fairly large already. Thus, a new tool,
icklint, would be called for. It can be added to ick2.git, or be kept
in its own repository. Could be written in Python or any other
suitable language, such as Rust. (I'd like to learn Rust more.)
Preferably a language I understand.
Alternatively, we could have a set of checks, and a way to run them
against a set of project and pipeline objects, and then use them both
=66rom icktool, and in the controller, runnig the checks at build
trigger time.
Since I don't yet know enough Rust to write this, I sketch in Python:
assume a main program that reads files with projects and pipelines,
and then runs though a number of checks. Each check is implemented as
an object with the following interface:
class IckLintCheck:
def set_projects(self, projects):
def set_pipelines(self, projects):
def get_projects(self):
def get_pipelines(self):
def find_lint(self):
Here projects and pipelines are dicts indexed with the name of a
project or pipeline, and find_lint yields a sequence of issues. An issue
is a string to be displayed to the user.
Example:
class CheckProjects(IckLintCheck):
def find_lint(self):
checks =3D [self.check_pipeline_exists, self.check_params_given]
for p in self.get_projects():
for name in p['pipelines']:
for check in checks:
for issue in check(p, name):
yield issue
def check_pipeline_exists(self, p, name):
if name not in self.get_pipelines():
yield 'Project {} uses pipeline {}, which does not exist'.f=
ormat(
p['project'], name)
def check_params_given(self, p, name):
pl =3D self.get_pipelines()[name]
for param in pl['parameters']:
if param not in p['parameters']:
yield 'Project {} does not define {}, needed by {}.form=
at('
p['project'], param, name)
It is probably possible to reduct boilerplate, but I hope you get the
idea from the example.
Does that seem reasonable?
--=20
I want to build worthwhile things that might last. --joeyh
--qMm9M+Fa2AknHoGS
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEETNTnrewG6wEE1EJ3bC+mFux6IDEFAltZ1wwACgkQbC+mFux6
IDEPgBAAvf156XGwL1KunkCfrwbgn+hR14XUOIyG21/+jM27vD8verLNafwMp1C7
lp4uVZcXwznUG7tehY6fabrJBtNFQIP07BZB4ZXSVU1rJkbADATqP+oR8kPnSKNF
e4QET2nTdvTapUmvTBe/lLNA1ipE9ugt5wzz61S6WfJoy+F4ikWnt725arg6qRMd
oeestUQB12NDlZlh/XQnpabgwv9AJEjaZTEFiPMDNOM4CI8Kj4JDjHxl2CZlz+u5
xcCY5qQh1Ha374YkbFGL0Ba5uNV2Jo1xP45I6E1G5MPclp8m5CnhSzdEh1cfWZx7
qSWtSzhTWxxEYNvnQlzGwH2fpeXazjJzNEUQSZBYSLO5cHJFr3Q95uvcvX+6rvZN
nOTRIaUA6X062B6VAGy3Qds5Sb6Hb5XUtzfMh+PTy37NxwTkkrvd6H+FY73OYTJ4
FO8dnW04tuQF9QVF4BNoebZ3WR10EiGwBVixLbX7dIIexVKntDHkTiFhQwPBINAd
jOzC9y5eAY65PsYsY4FL/Z/Envsnnr8Shomieyevkpxs7aL+bBBZS2RsVaurqEV/
csoddc6p3cgPNy+XoJAabcky1NI7aExVVGTfYsROI2gj+p3nk42/TKqEim9Xae6m
91dOR2zYpkVINRQOyBQjbCyIS+2u1cy4+0eZkOiIIiuWicEIUUk=
=uvAM
-----END PGP SIGNATURE-----
--qMm9M+Fa2AknHoGS--
--===============2072883612514617144==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
ick-discuss mailing list
ick-discuss@ick.liw.fi
https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi
--===============2072883612514617144==--
|
