Message-ID: <86bf91e609a4e96d53d7eff272be189d589f2ae4.camel@liw.fi>
From: Lars Wirzenius <liw@liw.fi>
To: ick discussions <ick-discuss@ick.liw.fi>
Date: Thu, 30 Aug 2018 18:22:29 +0300
User-Agent: Evolution 3.29.92-1 
Mime-Version: 1.0
Subject: Federated CI
Precedence: list
Content-Type: multipart/mixed; boundary="===============8348942345006944693=="
Mime-version: 1.0
Sender: ick-discuss-bounces@ick.liw.fi
Errors-To: ick-discuss-bounces@ick.liw.fi


--===============8348942345006944693==
Content-Type: multipart/signed; micalg="pgp-sha512";
 protocol="application/pgp-signature"; boundary="=-1eQMbzR54IKbfTBNQXA2"


--=-1eQMbzR54IKbfTBNQXA2
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

(Also posted as https://blog.liw.fi/posts/2018/08/30/federated_ci/ if
that's easier to read.)

In the modern world, a lot of computing happens on other people's
computers. We use a lot of services provided by various parties. This
is a problem for user freedom and software freedom. For example, when
I use Twitter, the software runs on Twitter's servers, and it's
entirely proprietary. Even if it were free software, even if it were
using the Affero GPL license (AGPL), my freedom would be limited by
the fact that I can't change the software running on Twitter's
servers.

If I could, it would be a fairly large security problem. If I could,
then anyone could, and they might not be good people like I am.

If the software were free, instead of proprietary, I could run it on
my own server, or find someone else to run the software for me. This
would make me more free.

That still leaves the data. My calendars would still be on Twitter's
servers: all my tweets, direct messages, the lists of people I follow,
or who follow me. Probably other things as well.

For true freedom in this context, I would need to have a way to
migrate my data from Twitter to another service. For practical
freedom, the migration should not be excessively much work, or be
excessively expensive, not just possible in principle.

For Twitter specifically, there's free-er alternatives, such as
Mastodon.

For ick, my CI / CD engine, here is my current thinking: ick should
not be a centralised service. It should be possible to pick and choose
between instances of its various components: the controller, the
workers, the artifact store, and Qvisqve (authentication server).
Ditto for any additional components in the future.

Since users and the components need to have some trust in each other,
and there may be payment involved, this may need some co-ordination,
and it may not be possible to pick entirely freely. However, as a
thought experiment, let's consider a scenario.

Alice has a bunch of non-mainstream computers she doesn't use herself
much: Arm boards, RISCV boards, PowerPC Macs, Amigas, etc. All in good
working condition. She'd be happy to set them up as build workers, and
let people use them, for a small fee to cover her expenses.

Bettina has a bunch of servers with lots of storage space. She'd be
happy to let people use them as artifact stores, for a fee.

Cecilia has a bunch of really fast x86-64 machines, with lots of RAM
and very fast NVMe disks. She'd also be happy to rent them out as
build workers.

Dinah needs a CI system, but only has one small server, which would
work fine as a controller for her own projects, but is too slow to
comfortably do any actual building.

Eliza also needs a CI system, but wants to keep her projects separate
from Dinah's, so wants to have her own controller. (Eliza and Dinah
can't tolerate each other and do not trust each other.)

Fatima is trusted by everyone, except Eliza, and would be happy to run
a secure server with Qvisqve.

Georgina is like Fatima, except Eliza trusts her, and Dinah doesn't.

The setup would be like this:

* Alice and Cecilia run build workers. The workers trust both Fatima's
  and Georgina's Qvisqves. All of their workers are registered with
  both Qvisqves, and both Dinah's and Eliza's controllers.

* Bettina's artifact store also trusts both Qvisqves.

* Dinah creates an account on Fatima's Qvisqve. Eliza on Georgina's
  Qvisqve. They each get an API token from the respective Qvisqve.

* When Dinah's project builds, her controller uses the API token to
  get an identity token from Fatima's Qvisqve, and gives that to each
  worker used in her builds. The worker checks the ID token, and then
  accepts work from Dinah's controller. The worker reports the time
  used to do the work to its billing system, and Alice or Cecilia uses
  that information to bill Dinah.

* If a build needs to use an artifact store, the ID token is again
  used to bill Dinah.

* For Eliza, the same thing happens, except with another Qvisqve, and
  costs from he builds go to her, not Dinah.

This can be generalised to any number of ick components, which can be
used criss-cross. Each component needs to be configured as to which
Qvisqves it trusts.

I think this would be a nicer thing to have than the centralised
hosted ick I've been thinking about so far. Much more complicated, and
much more work, of course. But interesting.

There are some interesting and difficult questions about security to
solve. I don't want to start thinking about the details yet, I'll play
with the general idea first.

What do you think?

--=-1eQMbzR54IKbfTBNQXA2
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEETNTnrewG6wEE1EJ3bC+mFux6IDEFAluIC7UACgkQbC+mFux6
IDFNyBAAwR7rsZZxKXEHxPJA45KMeirHPcvPQNUxTEjKLWDh4te/vKZTYK/eGUrD
l2WV/C5ZqwTieXCjEM0ke9Mxr5p9WY1VICp6Elq6yPob3zxKhrq4dULH2UknS4m3
aEChP42GPtomfsbOgp9Y1z7hksfpiH8LpyJh6Og73E/5oJQERDKPAX8X62RL+eud
H6qX3VVkFu/dWDEuEdHDY67A40TtvPPe/Kg2dcWn8KVSQLsOsA0DhWqCwpTU5luo
/4H/6O16wNTN2V/rKt/dtXvl/uNrDQX3QPMdI6oyACr59oZ95NohgXMo6rfhr0X3
SLcpyNCScS1BRWu50YuLogJMV66zAtqIjHaawIysynzcAFsi8zDbMDNu3mhJAFpQ
3iXEIsxJlJOnAd9Gcf6MRBE8g3fx38lgJq5SjLJTIXSPn28AM5LUf+nPdTiAxEgw
C0coksHF2Rjjl3BLdTa3dOoLEkUNJKbJNQHJYcJ3PqTA6ZhMgfep7hRWl35I/1h8
lLpiUWANl5ciAZgVGy5mIeGC05HjXCYZM0k3+ahlaciv8rnV0k/JieE8XSpN4rR9
2dXLrf07jUqcZw2VQr2Lz98EwPhG0JIRFMk/UyoBtROuJuYAOUeeqWujTDF6XlsD
l0vdO1nOe+eGrf9BIc/axzmnH/KLfhdYH73Mst/79Iue23z0wqw=
=Mgz3
-----END PGP SIGNATURE-----

--=-1eQMbzR54IKbfTBNQXA2--


--===============8348942345006944693==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
ick-discuss mailing list
ick-discuss@ick.liw.fi
https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi

--===============8348942345006944693==--