Date: Sun, 11 Aug 2019 12:26:31 +0300
From: Lars Wirzenius <liw@liw.fi>
To: Ick discussions <ick-discuss@ick.liw.fi>
Message-ID: <20190811092631.GC4376@exolobe1.liw.fi>
MIME-Version: 1.0
User-Agent: Mutt/1.10.1 (2018-07-13)
Subject: Distributed CI and threat modelling
Precedence: list
Content-Type: multipart/mixed; boundary="===============2386904406100824455=="
Mime-version: 1.0
Sender: ick-discuss-bounces@ick.liw.fi
Errors-To: ick-discuss-bounces@ick.liw.fi


--===============2386904406100824455==
Content-Type: multipart/signed; micalg=pgp-sha512;
 protocol="application/pgp-signature"; boundary="hYooF8G/hrfVAmum"
Content-Disposition: inline


--hYooF8G/hrfVAmum
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Also avilable at:
https://ick.liw.fi/blog/2019/08/11/distributed_ci_and_threat_modelling/

---

Some thoughts about making a distributed CI system secure.

* outline of system
    * version control system hold source code
    * IDP authenticates and authorizes users, system components
    * controller co-ordinates builds, collects build logs
    * artifact store holds build artifacts
    * workers (many) do the actual building, are told by controller
      what to do, fetch source from version control system, upload
      artifacts to artifact store

* entitites in the system that need to be protected:
    * the person using CI
    * the person running the IDP, controller, and artifact store (for
      simplicity, assume they're all run by the same person, although
      they could each be run by separate people)
    * the people running runners

* threats to person using CI
    * malicious workers, which embeds unwanted code in build artifact
        * mitigation: use reproducible builds and build on at least
          two workers to detect unwanted changes in artifacts; this
          would work OK, if there are relatively few malicious workers
    * many malicious workers, or workers that become malicious after a
      long period of working fine
        * mitigation: have at least one trusted worker, which might be
          slow, but whose output is required for a build to be trusted
            * artifacts from maybe-trusted workers can't be used for
              deployment, but could be used with sufficient isolation
              to speed things up, e.g., to do heavy testing: if the
              trusted worker later confirms the binaries are
              trustworthy (bitwise identical), then the test results
              can be trusted, too
        * variant of mitigation: require at least N maybe-trusted
          workers to produce bitwise identical build artifacts, where
          N is set by the person running the CI or whose project is
          being built
        * rejected: a karma or reputation system based on past
          behaviour: this makes long-lived workers valuable targets,
          and years of good behaviour won't protect if the worker gets
          hijacked

* threats to person running IDP, controller, artifact store
    * there are no new threats to these that come due to the
      distributed nature of CI
    * all the usual threats apply, of course

* threats to those running workers
    * build uses too much CPU or RAM
        * mitigation: enable person running worker to set limits and
          priorities so that the build doesn't use resources needed by
          other things
    * build attacks remote hosts (e.g., DDoS)
        * mitigation: prevent build from accessing any network hosts,
          except version control server, controller, artifact store
    * build attacks host where worker runs
        * mitigation: run build in a VM, using the best avilable
          isolation techniques, such as carefully configured qemu/KVM
          to implement the VM, and keeping all related software up to
          date


--=20
I want to build worthwhile things that might last. --joeyh

--hYooF8G/hrfVAmum
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEETNTnrewG6wEE1EJ3bC+mFux6IDEFAl1P30YACgkQbC+mFux6
IDEDDRAAt3jVvvk/p0ikerkUzglq380nnds075YwQnCNl1HsWdnER11jc5OKrVFZ
R3TxD0mQpccN67CmYFzNKXIHRM7csYSL3e8p2dZ1VqrHkjY7GzzqpoiGjpCZ7VQ8
Eo8MWbgDTT3erdTmDl4HuVK4LpySXQDLa2mM9Y19Dw0mxDnnNVlaykhkTAU/oWnh
0vKgBpTiRZlY9HVl1DqEIG1UjqY6dRXkAamp3d7qJvZVPsnNyv9jiRxOThjCxf+B
pq4pRFUFawOr5fm4h15SRbYR/Q+kT3Ow08Imj19+xALA3KWQkZqROQwSqly92Xh6
EzdFlM6mNLTc4QPe0zKbAiY6mvCSSNt18ojBdCYSJqOrZLEsooetDdDQ7Qs3O2du
NikrGbwoael06pPqJgRO9dAq6oxyHCGPmlAczrxkPzzPWIEniSfiB2cdeZGRFuIE
rx35YPshfxHK4Ju+/fmXg/dUToUWYBmSxyzO9RNLi11yIPHWhI1T88sjROgT0kaE
59HU4JUmTTbhy28aMR8e+2KIYAQX2ghE4uyxEUKlxM1er95Afj2x4pT2HEy7hxSQ
bUkxyjqI/HSXK9n8fSi30RLzjF+Dx55OFRmH7Fy8q61bt6pb76TQI9whg8WzvND8
9+tfalrPs86d2X1xHb75TlIV4LlfoULBzYj1gBn7Nvh+4nsLyCk=
=wuWp
-----END PGP SIGNATURE-----

--hYooF8G/hrfVAmum--


--===============2386904406100824455==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
ick-discuss mailing list
ick-discuss@ick.liw.fi
https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi

--===============2386904406100824455==--