Change: formatting, bullet points to paragraphs, etc

1 files changed, 134 insertions, 105 deletions

diff --git a/ci-arch.mdwn b/ci-arch.mdwn
index c89fd96..07d4694 100644
--- a/ci-arch.mdwn
+++ b/ci-arch.mdwn
@@ -501,114 +501,143 @@ would prefer to avoid having such changes.
 
 # Architecture: The WMF development ecosystem
 
-![The WMF development ecosystem, roughly](ecosystem.svg)
-
-* the above figure is simplistic, but gives the general idea
-    * developer pushes a change to Gerrit
-    * CI builds and tests change (commit stage)
-    * CI deploys to a test environment, runs tests against that
-      (acceptance test stage)
-    * CI can deploy to an environment dedicated for manual testing
-    * after successful code review, CI merges changes to the master
-      branch, run all automated tests again, and deploys to the
-      production environment
-
-* the commit and acceptance stages are triggered as soon as developer
-  pushes changes to be reviewed; human reviews won't be requested
-  until the two stages pass, as there's no point in spending human
-  attention on things that are not going to be candidates for
-  deployment to production; they may be re-run after code review
-  accepts the changes, to make sure nothing unforeseen has changed
-  while review took place
-
-* other stages may run in parallel with code review, but if they fail
-  they may nullify candidacy? for example, stages for manual and
-  capacity testing, and security test/review; depending on the change
-  and the component in question, some or all of these may be necessary
+![The WMF development ecosystem, roughly](ecosystem.svg){ height=25% }
+
+The figure above is simplistic, but gives the general idea of what
+happens when a developer is finished with a change:
+
+1. developer pushes a change to Gerrit, which trigger CI
+1. CI builds and tests change (commit stage)
+1. CI deploys to a test environment, runs tests against that
+   (acceptance test stage); if everythins is OK, Gerrit is notified
+   and requests code reviews from relevant parties
+1. testers can request CI it deploy the change to an environment
+   dedicated for manual testing
+1. after a successful code review, CI merges changes to the master
+   branch, runs all automated tests again, and deploys to the
+   production environment
+
+The commit and acceptance stages are triggered as soon as developer
+pushes changes to be reviewed. Human reviews won't be requested until
+the two stages pass, as there's no point in spending human attention
+on things that are not going to be candidates for deployment to
+production. The two stages may be re-run after code review, to make
+sure nothing unforeseen has changed while the review took place.
+
+Other stages may run in parallel with code review, and if they fail
+they may nullify the release candidacy of the change. For example,
+stages for manual and capacity testing, and security test/review;
+depending on the change and the component in question, some or all of
+these may be necessary.
 
 # The (default?) pipeline
 
-![The default pipeline](pipeline.svg)
-
-* CI will provide a default pipeline for all projects
-
-  * divided into several stages
-
-  * mandatory stages: commit, acceptance; other stages may be added to
-    other projects as needed
-
-  * the goal is that if commit + acceptance stages pass, the project
-    has a candidate that can be deployed to production, unless the
-    project is such that it needs (say) manual testing or other human
-    decision for the production deployment decision
-
-  * if commit or acceptance stage fails, there is not production
-    candidate
-
-* commit stage
-
-  * builds all the artifacts that will be used by later stages
-
-  * runs unit tests
-
-  * other tests, possibly integration tests
-
-  * code health checks
-
-  * the commit stage is expected to be fast, aiming at less than five
-    minutes, so that we can expect developers to wait for it to pass
-    successfully
-
-  * the commands to build (compile) or run automated tests are stored
-    in the repository, either explicity, or by indicating the type of
-    build needed; for example, the repository may specify "make" as
-    the command to run, or it may specify that it's a Go project, and
-    CI would know how to build a Go project; in the latter case we can
-    change the commands to build a Go project by changing CI only,
-    without having to change each git repository with a Go program
-
-  * only the declarative style is possible for building Docker images,
-    as we want control over how that is done
-
-  * CI may enforce specific additional commands to run, to build or
-    test further things; this can be used by RelEng to enforce
-    specific code health checks, for example, or to enable (or
-    disable) debug symbols in all builds
-
-  * all tests run in an isolated build tree, and may not use anything
-    outside the tree, including databases or other backing services
-
-  * any build dependencies must be specified explicitly; for example,
-    which version of Go should be installed in the build environment,
-    or if a project build-depends on another project, which artifacts
-    it needs installed from the other project; explicit is more work,
-    but results in fewer problems due to broken heuristics
-
-* acceptance tests
-
-  * deploys artifacts from commit stage to containers in special test
-    environments, runs tests against deployed artifacts
-
-  * possibly run slow tests from the build tree as well, if they don't
-    fit into the commit stage's time budget
-
-  * this stage can be slower than the commit stage, but should still
-    pass in, say, an hour, instead of taking days
-
-* capacity tests
-
-  * these are tests that benchmark the system as deployed into an
-    environment that's sufficiently production-like also as far as
-    hardware resources are concerned
-
-* manual (exploratory) tests
-
-  * testers will have dedicated environments to which they can trigger
-    deployment of specific builds, and in which they can, for example,
-    test that specific bugs are fixed
-
-  * this can also be used to demonstrate upcoming features that are
-    not yet enabled in production
+![The default pipeline](pipeline.svg){ height=25% }
+
+CI will provide a default pipeline for all projects. Projects may use
+that or specify another one.
+
+The pipeline will be divided into several stages. Mandatory stages for
+all changes and all projects are commit and acceptance stage. Other
+stages may be added to specific changes projects as needed.
+
+The goal is that if the commit and acceptance stages pass, the change
+is a candidate that can be deployed to production, unless the
+project is such that it needs (say) manual testing or other human
+decision for the production deployment decision. Likewise, if the
+component or the change is particularly security or performance
+sensitive, stages that check those aspects may be required. CI will
+have ways of indicating the required changes per component, and also
+per change. (It is unclear how this will be managed.)
+
+If the commit or acceptance stage fails, there is not production
+candidate. The pipeline as a whole fails. Any artifacts built by the
+pipeline will not be deployable to production, but they may be
+deployable to test environments, or downloaded by developers for
+inspection.
+
+## The commit stage
+
+The commit stage builds any deployable artifacts, such as executable
+binaries, minimized Javascript, translation files, or Docker images.
+It is important that artifacts don't get rebuilt by later stages,
+because rebuilding does not always result in bitwise identical output.
+Instead the goal is to build once, test the artifacts, and deploy the
+tested artifacts, instead of rebuilding and maybe deploying something
+different than what was tested.
+
+The commit stage also runs unit tests, and any other tests that can be
+run in isolation from other parts of the system, and that also are
+quick. The commit stage does not have access to backing services, such
+as databases or other components of the overall system. For example,
+when the pipeline processes a change to a MediaWiki extenasion, the
+commit stage doesn't have access to MediaWiki core or the MariaDB
+MediaWiki uses. Integration or system tests should be done in the
+acceptance test stage.
+
+The commit stage also runs code health checks.
+
+The commit stage is expected to be fast, aiming at less than five
+minutes, so that we can expect developers to wait for it to pass
+successfully. This will be a new requirement on our developers.
+
+The commands to build (compile) or run automated tests are stored in
+the repository, either explicity, or by indicating the type of build
+needed. There might be a `.pipeline/config.yaml` file in the
+repository, which specifies that `make` is the command that builds the
+artifacts. Otherwise, the file may specify that it's a Go project, and
+CI would know how to build a Go project. In this case we can change
+the commands to build a Go project by changing CI only, without having
+to change each git repository with a Go program.
+
+Only the declarative style will be possible for building Docker images,
+as we want control over how that is done (**SECURE** requirement).
+
+CI may enforce specific additional commands to run, to build or test
+further things; this can be used by RelEng to enforce certain things.
+For example, we may enforce code health checks, or to enable (or
+disable) debug symbols in all builds. Such enforcement will be done in
+collaboration with our developers.
+
+Any build dependencies needed during the commit stage must be
+specified explicitly. For example, the minimum required version of Go
+that should be installed in the build environment would be a build
+dependency. If a project build-depends on another project, it needs to
+specify which project, and which artifacts it needs installed from the
+other project. Explicit build-dependencies is more work, but results
+in fewer problems due to broken heuristics.
+
+## The acceptance stage
+
+During the acceptance stage CI deploys artifacts built in the commit
+stage to a production-like system that has the same versions of all
+sofware as production, except for the changes being processed by the
+pipeline. CI will then run automated acceptance tests, and other
+integration and system tests, against the deployed software. The test
+environment is clean and empty, and well-known, unless and until the
+test suite inserts data or makes changes.
+
+The acceptance stage can take time. Developers are not expected to
+wait until it is finished before they move on to working on something
+else.
+
+## Manual tests
+
+Testers may instruct CI to deploy any recent built set of artifacts to
+a dedicated test environment, and can use the software in that
+environment where it is isolated from others, and won't suddenly
+change underneath them. The details of how this will be implemented
+are to be determined later.
+
+This feature of the CI can also be used to demonstrate upcoming
+features that are not yet ready to be deployed to or enabled in
+production.
+
+## Capacity tests, non-functional requirements
+
+Capacity tests, and other tests for non-functional requirements, will
+also be done in dedicated, isolated production-like environments.
+RelEng will work with the performance team to sort out the details.
 
 # Architecture: internals