Change: rambling

1 files changed, 43 insertions, 11 deletions

diff --git a/ci-arch.mdwn b/ci-arch.mdwn
index 98ece80..2d13816 100644
--- a/ci-arch.mdwn
+++ b/ci-arch.mdwn
@@ -481,34 +481,66 @@ FIXME
 
   * builds all the artifacts that will be used by later stages
 
-  * all tests run in an isolated build tree, and may not use anything
-    outside the tree, including databases or other backing services
-
   * runs unit tests
 
   * other tests, possibly integration tests
 
   * code health checks
 
-  * is fast (aim at less than five minutes)
+  * the commit stage is expected to be fast, aiming at less than five
+    minutes, so that we can expect developers to wait for it to pass
+    successfully
 
-* acceptance tests
+  * the commands to build (compile) or run automated tests are stored
+    in the repository, either explicity, or by indicating the type of
+    build needed; for example, the repository may specify "make" as
+    the command to run, or it may specify that it's a Go project, and
+    CI would know how to build a Go project; in the latter case we can
+    change the commands to build a Go project by changing CI only,
+    without having to change each git repository with a Go program
+
+  * only the declarative style is possible for building Docker images,
+    as we want control over how that is done
+
+  * CI may enforce specific additional commands to run, to build or
+    test further things; this can be used by RelEng to enforce
+    specific code health checks, for example, or to enable (or
+    disable) debug symbols in all builds
+
+  * all tests run in an isolated build tree, and may not use anything
+    outside the tree, including databases or other backing services
 
-  * automated acceptance tests
+  * any build dependencies must be specified explicitly; for example,
+    which version of Go should be installed in the build environment,
+    or if a project build-depends on another project, which artifacts
+    it needs installed from the other project; explicit is more work,
+    but results in fewer problems due to broken heuristics
+
+* acceptance tests
 
-  * deploy artifacts from commit stage, run tests to deployed
-    artifacts
+  * deploys artifacts from commit stage to containers in special test
+    environments, runs tests against deployed artifacts
 
   * possibly run slow tests from the build tree as well, if they don't
     fit into the commit stage's time budget
 
+  * this stage can be slower than the commit stage, but should still
+    pass in, say, an hour, instead of taking days
+
 * capacity tests
 
+  * these are tests that benchmark the system as deployed into an
+    environment that's sufficiently production-like also as far as
+    hardware resources are concerned
+
 * manual (exploratory) tests
 
-  * testers will be able to deploy any candidate a test environment,
-    with the push of a button; the test env is very like production,
-    except for capacity and possibly data
+  * testers will have dedicated environments to which they can trigger
+    deployment of specific builds, and in which they can, for example,
+    test that specific bugs are fixed
+
+  * this can also be used to demonstrate upcoming features that are
+    not yet enabled in production
 
 # Architecture: CI in an ecosystem