summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--tickets/d20aa3eb7bb042c2ba3912f8086deef7/Maildir/new/1530689407.M245846P12632Q1.koom340
1 files changed, 340 insertions, 0 deletions
diff --git a/tickets/d20aa3eb7bb042c2ba3912f8086deef7/Maildir/new/1530689407.M245846P12632Q1.koom b/tickets/d20aa3eb7bb042c2ba3912f8086deef7/Maildir/new/1530689407.M245846P12632Q1.koom
new file mode 100644
index 0000000..b75c95f
--- /dev/null
+++ b/tickets/d20aa3eb7bb042c2ba3912f8086deef7/Maildir/new/1530689407.M245846P12632Q1.koom
@@ -0,0 +1,340 @@
+Return-Path: <ick-discuss-bounces@ick.liw.fi>
+X-Original-To: distix@pieni.net
+Delivered-To: distix@pieni.net
+Received: from yaffle.pepperfish.net (yaffle.pepperfish.net [88.99.213.221])
+ by pieni.net (Postfix) with ESMTPS id 5C3CC41271
+ for <distix@pieni.net>; Wed, 4 Jul 2018 07:29:17 +0000 (UTC)
+Received: from platypus.pepperfish.net (unknown [10.112.101.20])
+ by yaffle.pepperfish.net (Postfix) with ESMTP id 35BC3415BE
+ for <distix@pieni.net>; Wed, 4 Jul 2018 08:29:17 +0100 (BST)
+Received: from ip6-localhost.nat ([::1] helo=platypus.pepperfish.net)
+ by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+ id 1facE1-0002fK-5n; Wed, 04 Jul 2018 08:29:17 +0100
+Received: from koom.pieni.net ([88.99.190.206] helo=pieni.net)
+ by platypus.pepperfish.net with esmtpsa (Exim 4.80 #2 (Debian))
+ id 1facE0-0002f8-Df
+ for <ick-discuss@ick.liw.fi>; Wed, 04 Jul 2018 08:29:16 +0100
+Received: from exolobe3.liw.fi (62-78-212-250.bb.dnainternet.fi
+ [62.78.212.250]) by pieni.net (Postfix) with ESMTPSA id 0DE4641271
+ for <ick-discuss@ick.liw.fi>; Wed, 4 Jul 2018 07:29:16 +0000 (UTC)
+Received: from exolobe3.liw.fi (localhost [127.0.0.1])
+ by exolobe3.liw.fi (Postfix) with ESMTPS id 238388811CB
+ for <ick-discuss@ick.liw.fi>; Wed, 4 Jul 2018 10:29:15 +0300 (EEST)
+Date: Wed, 4 Jul 2018 10:29:14 +0300
+From: Lars Wirzenius <liw@liw.fi>
+To: ick-discuss@ick.liw.fi
+Message-ID: <20180704072913.GA7501@exolobe3.liw.fi>
+References: <20180702152447.GA2363@exolobe3.liw.fi>
+ <20180703122826.siolafamphg7mpfo@somnambulist.local>
+MIME-Version: 1.0
+In-Reply-To: <20180703122826.siolafamphg7mpfo@somnambulist.local>
+User-Agent: Mutt/1.10.0 (2018-05-17)
+X-Pepperfish-Transaction: 8fe6-8500-7903-5a56
+X-Pepperfish-Transaction-By: platypus
+Subject: Re: Making the "git" action in ick more useful
+X-BeenThere: ick-discuss@ick.liw.fi
+X-Mailman-Version: 2.1.5
+Precedence: list
+List-Id: discussions about the ick CI system <ick-discuss-ick.liw.fi>
+List-Unsubscribe: <https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi>,
+ <mailto:ick-discuss-request@ick.liw.fi?subject=unsubscribe>
+List-Archive: <http://listmaster.pepperfish.net/pipermail/ick-discuss-ick.liw.fi>
+List-Post: <mailto:ick-discuss@ick.liw.fi>
+List-Help: <mailto:ick-discuss-request@ick.liw.fi?subject=help>
+List-Subscribe: <https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi>,
+ <mailto:ick-discuss-request@ick.liw.fi?subject=subscribe>
+Content-Type: multipart/mixed; boundary="===============0125566059845942189=="
+Mime-version: 1.0
+Sender: ick-discuss-bounces@ick.liw.fi
+Errors-To: ick-discuss-bounces@ick.liw.fi
+
+
+--===============0125566059845942189==
+Content-Type: multipart/signed; micalg=pgp-sha512;
+ protocol="application/pgp-signature"; boundary="dDRMvlgZJXvWKvBx"
+Content-Disposition: inline
+
+
+--dDRMvlgZJXvWKvBx
+Content-Type: text/plain; charset=us-ascii
+Content-Disposition: inline
+Content-Transfer-Encoding: quoted-printable
+
+On Tue, Jul 03, 2018 at 01:28:26PM +0100, Daniel Silverstone wrote:
+> On Mon, Jul 02, 2018 at 18:24:48 +0300, Lars Wirzenius wrote:
+> > However, the above design isn't flexible enough for Daniel, who needs
+> > to access multiple git repositories, not just once, and build a tree
+> > of clones. I'll let Daniel tell us what his use-case is like. (I think
+> > I know, but it's better if he tells it himself. I can be a bit of a
+> > broken telephone.)
+>
+> [description of use cases omitted]
+
+> I have many projects, luxio, lua-scrypt, clod, gall, supple, lace, tongue,
+> and gitano, all of which can be built with essentially the same pipeline
+> stages, so long as they are parameterised by the project name. As such,
+> my projects have a reponame parameter. Given that, if we assumed that
+> the git action could be parameterised in the pipelines:
+>=20
+> - action: git
+> repo: "git://git.gitano.org.uk/{{ reponame }}.git"
+> ref: "{{ refname }}"
+> path: "{{ reponame }}"
+> - action: git
+> repo: "git://git.gitano.org.uk/{{ reponame }}/debian.git"
+> ref: "{% if debianrefname %}{{ debianrefname }}{% else %}{{ refname }}{=
+% endif %}"
+> path: "{{ reponame }}/debian"
+> - action: git
+> repo: git://git.gitano.org.uk/gp-packaging-tools.git
+> ref: master
+> path: gp-packaging-tools
+>=20
+> would be approximately what I'd need for my git checkout pipeline.
+>=20
+> If "optional" parameters is not a thing, then the debianrefname parameter=
+ would
+> just have to be provided at all times, defaulting to master as refname wo=
+uld.
+>=20
+> If, on the other hand, the ref was irrelevant because it was expected that
+> there'd be a stage doing `git checkout ${refname}` or similar later, then=
+ I can
+> handle the case of debianrefname being empty/not-provided in shell.
+>=20
+> My "out-there" use-case is what I call, in Gitano, system branches. That=
+ works
+> by having branches of the same name in any/all of the above repos, and th=
+en
+> they're checked out together (along with master for any repo without the
+> branch) and all built together for test purposes. This project would not
+> produce any debs because it purely exists to prove out the test suite on a
+> system other than my development system, and I have tooling to produce the
+> checkouts and build the code, providing that it can run somewhere with ne=
+twork
+> access. We'd need something like:
+>=20
+> - action: git
+> repo: "git://git.gitano.org.uk/gitano.git"
+> ref: "{{ sysbranchref }}"
+> fallback-ref: master
+> path: gitano
+>=20
+> Where fallback-ref is checked out if ref isn't available in the remote.
+>=20
+> I don't know how much of this you, or others, would want in Ick, but ther=
+e you
+> have it.
+>=20
+> For me, the obviously desirable option is (a) that the git action can take
+> repo, ref, and path directly in the pipeline stage, and (b) that we can u=
+se
+> some kind of syntax to interpolate parameters into those strings (using j=
+inja2
+> seems obvious, though requiring that at the ick controller end might be a
+> smidge iffy for maintaining the possibility of replacing the python
+> implementation in the future).
+>=20
+> I look forward to your take on all the above,
+>=20
+> D.
+>=20
+> --=20
+> Daniel Silverstone http://www.digital-scurf.org/
+> PGP mail accepted and encouraged. Key Id: 3CCE BABE 206C 3B69
+>=20
+> _______________________________________________
+> ick-discuss mailing list
+> ick-discuss@ick.liw.fi
+> https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ic=
+k.liw.fi
+>
+
+Thanks, Daniel. That's illuminating and interesting.
+
+I'm going to rephrase what you wrote, to show how I understood it, and
+to suggest something.
+
+First, a comment on adding a templating language on top of YAML. I
+first encountered this with Ansible, and I've since used in my own
+vmdb2 project. It's a powerful concept, but I find it to sometimes be
+problematic: the input files aren't actually YAML anymore, and they
+become harder to understand and harder to debug. Sometimes that's OK,
+the added power is worth it. However, I'd like to avoid that for ick,
+for the time being.
+
+(Also, as you say, using jinja2 for templating might become
+problematic if and when ick gets rewritten in another language.)
+
+Thus, I'll write out your examples without templating. They'll be more
+verbose, and more repetitive, but should be more obvious as to
+meaning.
+
+That also means I'm, at this time, not ready to have action attributes
+that vary from project to project. Perhaps that'd be a useful addition
+later on, but it seems to require some form of (templating) language
+on top of YAML.
+
+Your first use case seems to me to be like this:
+
+* all your projects live on the same git server, next to each other
+* your main repository for a component is foo.git, branch master
+* its Debian packaging is in foo-debian.git, branch master for unstable
+* additionally, you have some common build tooling in
+* gp-packaging-tools.git
+
+I note that this has a fairly tight coupling between the repositories
+for the main code and the Debian packaging. For ick, I'd prefer to
+avoid mandating that: one use case might be that I want to package
+your code and I put the Debian packaging on my server, while your code
+remains on yours.
+
+My suggestion would be to change the git action to look for a project
+parameter "git" (or "sources"?), which would be a *list* of
+url/ref/dir tuples. For you, each project would have a parameter like
+this:
+
+ - project: foo
+ parameters:
+ git:
+ - url: git://git.gitano.org.uk/foo.git
+ ref: master
+ dir: src
+ - url: git://git.gitano.org.uk/foo-debian.git
+ ref: debian/unstable
+ dir: src/debian
+ - url: git://git.gitano.org.uk/gp-packaging-tools.git
+ ref: master
+ dir: gp-packaging-tools
+
+Which would result in a directory tree like this:
+
+ /workspace
+ /workspace/src
+ /workspace/src/.git
+ /workspace/src/debian
+ /workspace/src/debian/.git
+ /workspace/gp-packaging-tools
+ /workspace/gp-packaging-tools/.git
+
+I admit the git parameter gets a bit repetitive. If that's enough of a
+problem, I'd be willing to make the git action understand the optional
+parameter "git_server" (or "git_base_url"):
+
+ - project: foo
+ parameters:
+ git_server: git:///git.gitano.org.uk
+ git:
+ - repo: foo.git
+ ref: master
+ dir: src
+ - repo: foo-debian.git
+ ref: debian/unstable
+ dir: src/debian
+ - repo: gp-packaging-tools.git
+ ref: master
+ dir: gp-packaging-tools
+
+In the tuples in the git parameter, url would still work and would be
+the full URL to the repository. Otherwise, the repo field would be
+appended to the git_server value.
+
+Also, we could make "master" be the default branch. Actually, that
+suggests to me that a mechanism for providing defaults might be
+useful. Maybe something like this:
+
+ - project: foo
+ parameters:
+ git_defaults:
+ server: git:///git.gitano.org.uk
+ ref: master
+ git:
+ - repo: foo.git
+ dir: src
+ - repo: foo-debian.git
+ ref: debian/unstable
+ dir: src/debian
+ - repo: gp-packaging-tools.git
+ dir: gp-packaging-tools
+ - server: git://git.liw.fi
+ repo: liw-stuff.git
+
+In thie example, the url/ref/dir tuple is replaced with a
+server/repo/ref/dir tuple and the full repo URL is built by combining
+server and repo fields. (Though it might be helpful to provide a url
+field that can be used instead of server+repo.) See below for
+overriding values in triggers.
+
+Handling a list of dicts in shell with jq is not necessarily the kind
+of thing I enjoy doing. I'd do it in Python instead, where it's not as
+awkward.
+
+Your system branch use case is interesting. My initial suggestion for
+that would be change ick to allow a trigger call, which starts a new
+build, to override project parameters for a specific build. Currnetly
+it's just a simple GET:
+
+ GET /projects/foo/+trigger
+
+We could add another way to do triggers:
+
+ POST /projects/foo/+trigger
+ Content-Type: application/json
+
+ {
+ "parameters": {
+ "git_defaults": {
+ "ref": "liw/fix"
+ }
+ }
+ }
+
+This would set the "ref" field in the "git_defaults" parameter to
+"liw/fix". The "server" field would not be changed. The
+foo-debian.git still use the debian/unstable branch.
+
+While this would not be as powerful as full jinja2 support, it would,
+I think, be a useful feature, entirely separate from templating.
+
+What I suggest here isn't really what you suggested, but what do you
+think? Would this be workable?
+
+--=20
+I want to build worthwhile things that might last. --joeyh
+
+--dDRMvlgZJXvWKvBx
+Content-Type: application/pgp-signature; name="signature.asc"
+
+-----BEGIN PGP SIGNATURE-----
+
+iQIzBAABCgAdFiEETNTnrewG6wEE1EJ3bC+mFux6IDEFAls8d0kACgkQbC+mFux6
+IDFK2w//fIADlC7DdPcm8qORw0V8Cxr0L3XojFWVxeSdCV2aZdc6SXSraxczDzix
+LP4911zPdi/l6qalFQNlsWBwjWkF3AEvKvwfrc5bMduI351VOm8bs/px9qjPY+6e
+phDkmN7UdkGmvBArlFmDgfi+U3wYI8nhkMCSMh4vz+GfLWkugBmCcyTJzBvWa034
+Q/0xYqewOgSXbrSMV6U7lXlqZDL6im4/Nbz0KpLZI0KmQZFOuL+Is7DQLIJnV3m9
+a/cBGcb0frJQf2kb9da/knSPt3GcZFG3DjNdROsBzJvnQIOQkNIxeF2ituWPCTiv
+mSc4ApudXWVUKVGVrRQA1t/m0rAqItXznewSHp15uBu9PS6HZ1AtDNavbPXuIFC9
+YkPsb9PauTQnsKirjFFOBHSo1AmMK+lDXd7xi5Y0IEU+rxiL63dhpQYRbg9MLx+O
+jaqFPIy/2oay1kcQgtEvAodS38KFft93ayUlqLZfNBAJyM+aMSO3kNvisECyxNj+
+pGcqeE9zOdCcQMb95kgEhbq0RERnQoYp/XNKPwTyhYFVTykXuSqtM7rWSfE+UQmp
+Dqi7kAJOVnvRLTm1FSY6lQ/rIyUny5+IbubLUVmknvJa38Tu+UlBj/Xiw9hbHtHP
+KgoUYcXpZkLdryJ0AcFx8j7Lemh8AP6qsDrYpPfmWuZa5sVIIfA=
+=32jP
+-----END PGP SIGNATURE-----
+
+--dDRMvlgZJXvWKvBx--
+
+
+--===============0125566059845942189==
+Content-Type: text/plain; charset="us-ascii"
+MIME-Version: 1.0
+Content-Transfer-Encoding: 7bit
+Content-Disposition: inline
+
+_______________________________________________
+ick-discuss mailing list
+ick-discuss@ick.liw.fi
+https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi
+
+--===============0125566059845942189==--
+