Date: Thu, 26 Jul 2018 17:13:32 +0300
From: Lars Wirzenius <liw@liw.fi>
To: ick-discuss@ick.liw.fi
Message-ID: <20180726141332.GA32297@exolobe3.liw.fi>
References: <20180726130528.xgtftruk6srzgi4x@somnambulist.local>
MIME-Version: 1.0
In-Reply-To: <20180726130528.xgtftruk6srzgi4x@somnambulist.local>
User-Agent: Mutt/1.10.1 (2018-07-13)
Subject: Re: Proposal: an icktool lint subcommand
Precedence: list
Content-Type: multipart/mixed; boundary="===============2072883612514617144=="
Mime-version: 1.0
Sender: ick-discuss-bounces@ick.liw.fi
Errors-To: ick-discuss-bounces@ick.liw.fi


--===============2072883612514617144==
Content-Type: multipart/signed; micalg=pgp-sha512;
 protocol="application/pgp-signature"; boundary="qMm9M+Fa2AknHoGS"
Content-Disposition: inline


--qMm9M+Fa2AknHoGS
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Thu, Jul 26, 2018 at 02:05:28PM +0100, Daniel Silverstone wrote:
> I have noticed, over time, that it's possible to get into situations
> where the controller is in a bit of an inconsistent or messy state.
>=20
> To that end I'd like to suggest the introduction of a set of lints which
> can be run against the controller to tell you about it (or a subset of it=
).

I am in full support of this idea. I have some suggestions. We should
definitely do some form of this idea, preferably sooner. I too have
spent long evening trying to get all the details right.

> * Projects refer to pipelines which do not exist
> * Projects refer to pipelines but do not have all the requisite parameters
> * Pipelines exist which are not referred to by any project
> * Pipelines exist with no actions

I would add the following check:

* Project defines a parameter that none of its pipelines ask for.

The first implementation thought I have is that we should make it easy
to write checks. If it's easy, we'll have more, and they'll more
likely be correct.

My second implementation thought is it would be nice to be able to
run the checks before putting the projects and pipelines into the
controller. Thus, I'd do it client side. It would be nice if the
client could read the projects and resources from the controller, or
=66rom a local file, though. Local file is more important: if we have
that, we can use "icktool show" to list all projects and pipelines.

My third implementation thought is that I'd like this to not be part
of icktool, which is growing fairly large already. Thus, a new tool,
icklint, would be called for. It can be added to ick2.git, or be kept
in its own repository. Could be written in Python or any other
suitable language, such as Rust. (I'd like to learn Rust more.)
Preferably a language I understand.

Alternatively, we could have a set of checks, and a way to run them
against a set of project and pipeline objects, and then use them both
=66rom icktool, and in the controller, runnig the checks at build
trigger time.

Since I don't yet know enough Rust to write this, I sketch in Python:
assume a main program that reads files with projects and pipelines,
and then runs though a number of checks. Each check is implemented as
an object with the following interface:

    class IckLintCheck:
        def set_projects(self, projects):
        def set_pipelines(self, projects):
        def get_projects(self):
        def get_pipelines(self):
        def find_lint(self):

Here projects and pipelines are dicts indexed with the name of a
project or pipeline, and find_lint yields a sequence of issues. An issue
is a string to be displayed to the user.

Example:

    class CheckProjects(IckLintCheck):

        def find_lint(self):
            checks =3D [self.check_pipeline_exists, self.check_params_given]
            for p in self.get_projects():
                for name in p['pipelines']:
                    for check in checks:
                        for issue in check(p, name):
                            yield issue

        def check_pipeline_exists(self, p, name):
            if name not in self.get_pipelines():
                yield 'Project {} uses pipeline {}, which does not exist'.f=
ormat(
                    p['project'], name)

        def check_params_given(self, p, name):
            pl =3D self.get_pipelines()[name]
            for param in pl['parameters']:
                if param not in p['parameters']:
                    yield 'Project {} does not define {}, needed by {}.form=
at('
                        p['project'], param, name)

It is probably possible to reduct boilerplate, but I hope you get the
idea from the example.

Does that seem reasonable?

--=20
I want to build worthwhile things that might last. --joeyh

--qMm9M+Fa2AknHoGS
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEETNTnrewG6wEE1EJ3bC+mFux6IDEFAltZ1wwACgkQbC+mFux6
IDEPgBAAvf156XGwL1KunkCfrwbgn+hR14XUOIyG21/+jM27vD8verLNafwMp1C7
lp4uVZcXwznUG7tehY6fabrJBtNFQIP07BZB4ZXSVU1rJkbADATqP+oR8kPnSKNF
e4QET2nTdvTapUmvTBe/lLNA1ipE9ugt5wzz61S6WfJoy+F4ikWnt725arg6qRMd
oeestUQB12NDlZlh/XQnpabgwv9AJEjaZTEFiPMDNOM4CI8Kj4JDjHxl2CZlz+u5
xcCY5qQh1Ha374YkbFGL0Ba5uNV2Jo1xP45I6E1G5MPclp8m5CnhSzdEh1cfWZx7
qSWtSzhTWxxEYNvnQlzGwH2fpeXazjJzNEUQSZBYSLO5cHJFr3Q95uvcvX+6rvZN
nOTRIaUA6X062B6VAGy3Qds5Sb6Hb5XUtzfMh+PTy37NxwTkkrvd6H+FY73OYTJ4
FO8dnW04tuQF9QVF4BNoebZ3WR10EiGwBVixLbX7dIIexVKntDHkTiFhQwPBINAd
jOzC9y5eAY65PsYsY4FL/Z/Envsnnr8Shomieyevkpxs7aL+bBBZS2RsVaurqEV/
csoddc6p3cgPNy+XoJAabcky1NI7aExVVGTfYsROI2gj+p3nk42/TKqEim9Xae6m
91dOR2zYpkVINRQOyBQjbCyIS+2u1cy4+0eZkOiIIiuWicEIUUk=
=uvAM
-----END PGP SIGNATURE-----

--qMm9M+Fa2AknHoGS--


--===============2072883612514617144==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
ick-discuss mailing list
ick-discuss@ick.liw.fi
https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi

--===============2072883612514617144==--