summaryrefslogtreecommitdiff
path: root/blog/2019/08/11/distributed_ci_and_threat_modelling.mdwn
blob: 4d5b52119a13c778de61f2aa5d36d749fe68c05c (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
[[!meta title="Distributed CI and threat modelling"]]
[[!tag distributed reproducible-build]]
[[!meta date="2019-08-11 12:07"]]

Some thoughts about making a distributed CI system secure.

* outline of system
    * version control system hold source code
    * IDP authenticates and authorizes users, system components
    * controller co-ordinates builds, collects build logs
    * artifact store holds build artifacts
    * workers (many) do the actual building, are told by controller
      what to do, fetch source from version control system, upload
      artifacts to artifact store

* entitites in the system that need to be protected:
    * the person using CI
    * the person running the IDP, controller, and artifact store (for
      simplicity, assume they're all run by the same person, although
      they could each be run by separate people)
    * the people running runners

* threats to person using CI
    * malicious workers, which embeds unwanted code in build artifact
        * mitigation: use reproducible builds and build on at least
          two workers to detect unwanted changes in artifacts; this
          would work OK, if there are relatively few malicious workers
    * many malicious workers, or workers that become malicious after a
      long period of working fine
        * mitigation: have at least one trusted worker, which might be
          slow, but whose output is required for a build to be trusted
            * artifacts from maybe-trusted workers can't be used for
              deployment, but could be used with sufficient isolation
              to speed things up, e.g., to do heavy testing: if the
              trusted worker later confirms the binaries are
              trustworthy (bitwise identical), then the test results
              can be trusted, too
        * variant of mitigation: require at least N maybe-trusted
          workers to produce bitwise identical build artifacts, where
          N is set by the person running the CI or whose project is
          being built
        * rejected: a karma or reputation system based on past
          behaviour: this makes long-lived workers valuable targets,
          and years of good behaviour won't protect if the worker gets
          hijacked

* threats to person running IDP, controller, artifact store
    * there are no new threats to these that come due to the
      distributed nature of CI
    * all the usual threats apply, of course

* threats to those running workers
    * build uses too much CPU or RAM
        * mitigation: enable person running worker to set limits and
          priorities so that the build doesn't use resources needed by
          other things
    * build attacks remote hosts (e.g., DDoS)
        * mitigation: prevent build from accessing any network hosts,
          except version control server, controller, artifact store
    * build attacks host where worker runs
        * mitigation: run build in a VM, using the best avilable
          isolation techniques, such as carefully configured qemu/KVM
          to implement the VM, and keeping all related software up to
          date