From fd3532cd3b12ee24fc7e6313257469f8655aed8f Mon Sep 17 00:00:00 2001 From: distix ticketing system Date: Thu, 5 Jul 2018 10:36:06 +0000 Subject: imported mails --- .../Maildir/new/1530786966.M510719P15377Q1.koom | 216 +++++++++++++++++++++ 1 file changed, 216 insertions(+) create mode 100644 tickets/d20aa3eb7bb042c2ba3912f8086deef7/Maildir/new/1530786966.M510719P15377Q1.koom diff --git a/tickets/d20aa3eb7bb042c2ba3912f8086deef7/Maildir/new/1530786966.M510719P15377Q1.koom b/tickets/d20aa3eb7bb042c2ba3912f8086deef7/Maildir/new/1530786966.M510719P15377Q1.koom new file mode 100644 index 0000000..86be98f --- /dev/null +++ b/tickets/d20aa3eb7bb042c2ba3912f8086deef7/Maildir/new/1530786966.M510719P15377Q1.koom @@ -0,0 +1,216 @@ +Return-Path: +X-Original-To: distix@pieni.net +Delivered-To: distix@pieni.net +Received: from yaffle.pepperfish.net (yaffle.pepperfish.net [88.99.213.221]) + by pieni.net (Postfix) with ESMTPS id 78237433A3 + for ; Thu, 5 Jul 2018 10:35:34 +0000 (UTC) +Received: from platypus.pepperfish.net (unknown [10.112.101.20]) + by yaffle.pepperfish.net (Postfix) with ESMTP id 1FA29424AA + for ; Thu, 5 Jul 2018 11:35:34 +0100 (BST) +Received: from ip6-localhost.nat ([::1] helo=platypus.pepperfish.net) + by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian)) + id 1fb1bq-0001mG-0v; Thu, 05 Jul 2018 11:35:34 +0100 +Received: from [148.252.241.226] (helo=somnambulist.local) + by platypus.pepperfish.net with esmtpsa (Exim 4.80 #2 (Debian)) + id 1fb1bo-0001lf-Lg + for ; Thu, 05 Jul 2018 11:35:32 +0100 +Date: Thu, 5 Jul 2018 11:35:31 +0100 +From: Daniel Silverstone +To: ick-discuss@ick.liw.fi +Message-ID: <20180705103531.v2jauqhcgcbhs7oo@somnambulist.local> +References: <20180702152447.GA2363@exolobe3.liw.fi> + <20180703122826.siolafamphg7mpfo@somnambulist.local> + <20180704072913.GA7501@exolobe3.liw.fi> +MIME-Version: 1.0 +Content-Type: text/plain; charset=us-ascii +Content-Disposition: inline +In-Reply-To: <20180704072913.GA7501@exolobe3.liw.fi> +User-Agent: NeoMutt/20170113 (1.7.2) +X-Pepperfish-Transaction: e55a-5c23-b40b-adb7 +X-Pepperfish-Transaction-By: platypus +Subject: Re: Making the "git" action in ick more useful +X-BeenThere: ick-discuss@ick.liw.fi +X-Mailman-Version: 2.1.5 +Precedence: list +List-Id: discussions about the ick CI system +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +Sender: ick-discuss-bounces@ick.liw.fi +Errors-To: ick-discuss-bounces@ick.liw.fi + +On Wed, Jul 04, 2018 at 10:29:14 +0300, Lars Wirzenius wrote: +> Thanks, Daniel. That's illuminating and interesting. + +I'm glad :-) + +> I'm going to rephrase what you wrote, to show how I understood it, and +> to suggest something. + +Fair, I'll respond inline here :-) + +> First, a comment on adding a templating language on top of YAML. I +> first encountered this with Ansible, and I've since used in my own +> vmdb2 project. It's a powerful concept, but I find it to sometimes be +> problematic: the input files aren't actually YAML anymore, and they +> become harder to understand and harder to debug. Sometimes that's OK, +> the added power is worth it. However, I'd like to avoid that for ick, +> for the time being. +> +> (Also, as you say, using jinja2 for templating might become +> problematic if and when ick gets rewritten in another language.) + +Yes, absolutely. At the "worst" we can come up with a convention by which +icktool expands things before it sends them to the controller, but let's avoid +templating languages for now. + +> Your first use case seems to me to be like this: +> +> * all your projects live on the same git server, next to each other +> * your main repository for a component is foo.git, branch master +> * its Debian packaging is in foo-debian.git, branch master for unstable +> * additionally, you have some common build tooling in +> * gp-packaging-tools.git + +Yep (though foo/debian.git) :-) + +> I note that this has a fairly tight coupling between the repositories +> for the main code and the Debian packaging. For ick, I'd prefer to +> avoid mandating that: one use case might be that I want to package +> your code and I put the Debian packaging on my server, while your code +> remains on yours. + +Yes, the coupling is tight for the packages I have. + +> My suggestion would be to change the git action to look for a project +> parameter "git" (or "sources"?), which would be a *list* of +> url/ref/dir tuples. For you, each project would have a parameter like +> this: +> +> - project: foo +> parameters: +> git: +> - url: git://git.gitano.org.uk/foo.git +> ref: master +> dir: src +> - url: git://git.gitano.org.uk/foo-debian.git +> ref: debian/unstable +> dir: src/debian +> - url: git://git.gitano.org.uk/gp-packaging-tools.git +> ref: master +> dir: gp-packaging-tools +> +> Which would result in a directory tree like this: +> +> /workspace +> /workspace/src +> /workspace/src/.git +> /workspace/src/debian +> /workspace/src/debian/.git +> /workspace/gp-packaging-tools +> /workspace/gp-packaging-tools/.git + +That looks acceptable (presumably there'd need to be an action:git pipeline +step somewhere too?) + +> I admit the git parameter gets a bit repetitive. If that's enough of a +> problem, I'd be willing to make the git action understand the optional +> parameter "git_server" (or "git_base_url"): +> +> - project: foo +> parameters: +> git_server: git:///git.gitano.org.uk +> git: +> - repo: foo.git +> ref: master +> dir: src +> - repo: foo-debian.git +> ref: debian/unstable +> dir: src/debian +> - repo: gp-packaging-tools.git +> ref: master +> dir: gp-packaging-tools + +I'd call it repo_base and expect that repos which aren't absolute are urljoin'd +onto it as you suggest. + +> Also, we could make "master" be the default branch. Actually, that +> suggests to me that a mechanism for providing defaults might be +> useful. Maybe something like this: +[snip] + +Sounds good, though again I'd call it default_ref so that it's clear what its +purpose is. + +> In thie example, the url/ref/dir tuple is replaced with a +> server/repo/ref/dir tuple and the full repo URL is built by combining +> server and repo fields. (Though it might be helpful to provide a url +> field that can be used instead of server+repo.) See below for +> overriding values in triggers. + +As I suggested, I'd just go with "if repo is absolute, use it, otherwise +urljoin" as a non-surprising behaviour. + +> Handling a list of dicts in shell with jq is not necessarily the kind +> of thing I enjoy doing. I'd do it in Python instead, where it's not as +> awkward. + +Fair. + +> Your system branch use case is interesting. My initial suggestion for +> that would be change ick to allow a trigger call, which starts a new +> build, to override project parameters for a specific build. Currnetly +> it's just a simple GET: +> +> GET /projects/foo/+trigger +> +> We could add another way to do triggers: +> +> POST /projects/foo/+trigger +> Content-Type: application/json +> +> { +> "parameters": { +> "git_defaults": { +> "ref": "liw/fix" +> } +> } +> } +> +> This would set the "ref" field in the "git_defaults" parameter to +> "liw/fix". The "server" field would not be changed. The +> foo-debian.git still use the debian/unstable branch. + +So long as you have a mechanism for removing values as well, that'd work quite +well for the most part. It still lacks the fallback for repos which lack the +ref, sadly, but perhaps that can't be usefully managed generically? + +> While this would not be as powerful as full jinja2 support, it would, +> I think, be a useful feature, entirely separate from templating. +> +> What I suggest here isn't really what you suggested, but what do you +> think? Would this be workable? + +For my primary use-case this would be perfectly fine. For the system branch +use-case it's still not quite enough to prevent a need for an additional +pipeline step to handle the checkouts. + +I think, for now, we should go with the primary-use case stuff, and worry about +the complex trigger solution in another iteration in the future. Do you want +to combine all of the stuff into a specification and then I'll review that so +we can close this work item off? + +D. + +-- +Daniel Silverstone http://www.digital-scurf.org/ +PGP mail accepted and encouraged. Key Id: 3CCE BABE 206C 3B69 + +_______________________________________________ +ick-discuss mailing list +ick-discuss@ick.liw.fi +https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi -- cgit v1.2.1