Change: stuff

2 files changed, 89 insertions, 68 deletions

diff --git a/Makefile b/Makefile
index 3084eaf..d851fec 100644
--- a/Makefile
+++ b/Makefile
@@ -3,4 +3,4 @@
 all: ci-arch.html
 
 .mdwn.html:
-	pandoc -o $@ $< 
+	pandoc --standalone --toc -o $@ $< 
diff --git a/ci-arch.mdwn b/ci-arch.mdwn
index cfc7259..43ff6e8 100644
--- a/ci-arch.mdwn
+++ b/ci-arch.mdwn
@@ -3,7 +3,7 @@ TITLE: Future WMF CI architecture
 
 # Introduction
 
-* CI WG proposes replacement of current WMF CI system with one of
+* CI WG plans replacement of its current WMF CI system with one of
   Argo, GitLab CI, Zuul v3.
 
 * This document goes into more detail of how the new CI system should
@@ -63,6 +63,9 @@ TITLE: Future WMF CI architecture
 
 ## Hard requirements
 
+* These are not absolute requirements, and can be negotiated, but only
+  to a minor degree.
+
 * (FAST) Must be fast enough that it isn't perceived as a bottleneck
   by developers.
 
@@ -75,6 +78,8 @@ TITLE: Future WMF CI architecture
   stages ("can get source from git", "can build", "can run unit
   tests", etc.).
 
+* (FEEDBACK2) Must support reporting to Gerrit, IRC, and Phabricator.
+
 * (SECURE) Must be secure enough that we can open it to community
   developers to use without too much supervision.
 
@@ -141,88 +146,96 @@ TITLE: Future WMF CI architecture
 
 * (TESTVC) Must support storing tests in version control.
 
-* () Must support reporting to Gerrit, IRC, and Phabricator.
+* (BUILDDEPS) Must have some way to declare dependent repositories /
+  software needed for testing.
 
-* () Must have some way to declare dependent repositories / software
-  needed for testing.
+* (TESTSERVICES) Must support services for tests — i.e., some PHPUnit
+  tests require MySQL.
 
-* () Must support services for tests — i.e., some PHPUnit tests
-  require MySQL.
+* (OTHERGITORTICKETING) Must allow changing git repository, code
+  review, and ticketing systems from Gerrit and Phabricator.
 
-* () Must allow changing git repository, code review, and ticketing
-  systems from Gerrit and Phabricator.
+* (PROTECTPRODUCTION) Must protect production by detecting problems
+  before they're deployed, and must in general support a sensible
+  CI/CD pipeline.
 
-* () Must protect production by detecting problems before they're
-  deployed, and must in general support a sensible CI/CD pipeline.
+* (ENFORCETESTS) Must allow Release Engineering team to enforce tests
+  on top of what a self-serving developer specifies, to allow us to
+  set minimal technical standards.
 
-* () Must allow Release Engineering team to enforce tests on top of
-  what a self-serving developer specifies, to allow us to set minimal
-  technical standards.
-
-* () Must support dependency caching – we have castor, maybe we could
-  do better? Maybe some CI systems have this figured out?
+* (CACHEDEPS) Must support dependency caching – we have castor, maybe
+  we could do better? Maybe some CI systems have this figured out?
 
 ## Softer requirements
 
-* () Should have a hosted instance we can play with for evaluation to
-  avoid having to install everything from scratch.
+* These requirements are even more easily negotiated.
+
+* (HOSTED) Should have a hosted instance we can play with for
+  evaluation to avoid having to install everything from scratch.
 
-* () Should not require software development from the Foundation.
+* (UPSTREAM) Should not require software development from the
+  Foundation.
 
-* () Should allow builds to happen in K8s containers, but probably
-  should also support running jobs on bare metal or VMs. For example,
-  building Docker containers can't happen inside Docker containers.
+* (BUILDINKUBERNETES) Should allow builds to happen in K8s containers,
+  but probably should also support running jobs on bare metal or VMs.
+  For example, building Docker containers can't happen inside Docker
+  containers.
 
-* () Should allow the developers to define or declare at least parts
-  of the pipeline jobs in the repository: what commands to run for
-  building, testing, etc.
+* (SELFSERVE2) Should allow the developers to define or declare at
+  least parts of the pipeline jobs in the repository: what commands to
+  run for building, testing, etc.
 
-* () Should be highly available - can restart any component without
+* (HA) Should be highly available - can restart any component without
   disrupting service.
 
-* () Should be fast at checking out code and running tests to give
-  quick feedback to developers.
+* (FAST2) Should be fast at checking out code and running tests to
+  give quick feedback to developers.
 
-* () Should have live console output of build.
+* (LIVELOG) Should have live console output of build.
 
-* () Should have build timeouts.
+* (MAXBUILDTIME) Should have build timeouts.
 
-* () Should support secure storage of credentials / secrets.
+* (SECRETS) Should support secure storage of credentials / secrets.
 
-* () Should provide a clean workspace for each test run - either a
-  clean VM or container.
+* (CLEANWORKSPACE) Should provide a clean workspace for each test
+  run - either a clean VM or container.
 
-* () Should allow archiving build logs and possible artifacts for a
-  long period, to allow extracting metrics from a long time period.
+* (ARTIFACTARCHIVE) Should allow archiving build logs and possible
+  artifacts for a long period, to allow extracting metrics from a long
+  time period.
 
-* () Should have rate limiting - one user/project can not take over
-  most/all resources.
+* (RATELIMIT) Should have rate limiting - one user/project can not
+  take over most/all resources.
 
-* () Should support validation and creation of GPG/PGP-signed git
-  commits
+* (CHECKSIG) Should support validation and creation of GPG/PGP-signed
+  git commits
 
 ## Would be nice
 
-* () Would be nice for Release Engineering team to not be a
-  bottleneck: we should not be required to approve a change to how a
-  job runs for a repo.
+* These are so soft they aren't even requirements, and more wish list
+  items.
 
-* () Would be nice for tests not to have to be written in shell
-  script.
+* (RELENGNOTBOTTLENECK) Would be nice for Release Engineering team to
+  not be a bottleneck: we should not be required to approve a change
+  to how a job runs for a repo.
 
-* () Would be nice for tests not to have to be written in Groovy
+* (NOTINSH) Would be nice for tests not to have to be written in shell
   script.
 
-* () Would be nice for tests not (likely) be written in a "language"
-  but an abstraction.
+* (NOTINGROOVY) Would be nice for tests not to have to be written in
+  Groovy script.
+
+* (TESTABSTRACTION) Would be nice for tests not (likely) be written in
+  a "language" but an abstraction.
 
      * The abstraction should be easily extended in a "real" language;
        more than shell, ideally.
 
-* () Would be nice for test abstractions to limit boiler-plate, i.e.,
-  all of our services are tested roughly the same way.
+* (LIMITBOILERPLATE) Would be nice for test abstractions to limit
+  boiler-plate, i.e., all of our services are tested roughly the same
+  way.
 
-* () Would be nice to prioritize jobs.
+* (PRIORITIZEJOBS) Would be nice to prioritize jobs.
 
      * Use case: if there is a queue of jobs, there should be some
        mechanism of jumping that queue for jobs that have a higher
@@ -231,35 +244,37 @@ TITLE: Future WMF CI architecture
      * We currently have a Gating queue that is a higher priority than
        periodic jobs that calculate Code Coverage.
 
-* () Would be nice to support isolation / sandboxing.
+* (ISOLATION) Would be nice to support isolation / sandboxing.
 
      * Jobs should be isolated from one another.
 
      * Jobs should be able to install apt-packages without affecting
        dependencies of other jobs.
 
-* () Would be nice to have configurable job requirements/affinity.
+* (CONTROLAFFINITY) Would be nice to have configurable job
+  requirements/affinity.
 
      * Be able to schedule a job only on nodes that have at least X
        available disk space/ram/cpu/whatever OR try to schedule on
        nodes where a current build of this job isn't already running.
 
-* () Would be nice to build artifacts suitable for production.
+* (BUILDFORPRODUCTION) Would be nice to build artifacts suitable for
+  production.
 
      * Currently we only do container images in a limited way – nice
        to haves: deb packages, java jars, go binaries, packagist
        downloads.
 
-* () Would be nice to make it easy for developers to recreate failures
-  locally.
+* (REPRODUCELOCALLY) Would be nice to make it easy for developers to
+  recreate failures locally.
 
-* () Would be nice to post-merge git-bisect to find patch that caused
-  a particular problem with a Selenium test.
+* (POSTMERGEBISECT) Would be nice to post-merge git-bisect to find
+  patch that caused a particular problem with a Selenium test.
 
-* () Would be nice to have a mechanism for deployment to staging,
-  production, pypi, packagist, toollabs.
+* (DEPLOYWHEREVER) Would be nice to have a mechanism for deployment to
+  staging, production, pypi, packagist, toollabs.
 
-* () Would be nice to have efficient matrix builds.
+* (MATRIXBUILDS) Would be nice to have efficient matrix builds.
 
      * E.g., we currently run phpunit tests and browser tests for the
        Cartesian product of [PHP7 PHP7.1 PHP7.2 HHVM][MySQL, SQLite,
@@ -267,16 +282,20 @@ TITLE: Future WMF CI architecture
        setup/git clone for all of those tests. Doing that in a space
        and time efficient way would be good.
 
-* () Developers should have an option to ssh to VM/container that CI
-  used to run the tests for debugging.
+* (DEVSSH) Developers should have an option to ssh to VM/container
+  that CI used to run the tests for debugging.
 
-* () Would be nice to support building and testing mobile applications
-  (at minimum for iOS and Android).
+* (MOBILE) Would be nice to support building and testing mobile
+  applications (at minimum for iOS and Android).
 
-* () Would be nice to be able to run for secret/security patches.
+* (EMBARGO) Would be nice to be able to run for secret/security
+  patches.
 
 # Important use cases
 
+These are some of the important use cases for the CI system, and how
+we plan CI to implement them.
+
 ## Normal change to an individual component
 
 * a developer pushes a change to one program that runs in production
@@ -290,8 +309,10 @@ TITLE: Future WMF CI architecture
   review, is enough to deploy this to production
 
 * developer pushes change, this trigger commit and acceptance stages,
-  which pass, which trigger code review, and reviewers vote +2, which
-  triggers a deployment to production
+  which pass, which triggers code review requests to be sent to
+  reviewers
+
+* reviewers vote +2, which triggers a deployment to production
 
 * this is the simplest possible use case for CI