Change: move pipeline to before architecture

1 files changed, 117 insertions, 117 deletions

diff --git a/ci-arch.mdwn b/ci-arch.mdwn
index 56d7501..d3ea8d7 100644
--- a/ci-arch.mdwn
+++ b/ci-arch.mdwn
@@ -451,7 +451,123 @@ items.
   patches. This means CI should be able to build and deploy changes
   that can't be made public yet, for security embargo reasons.
 
-# Architecture
+# The CI pipeline
+
+![The default pipeline](pipeline.svg){ height=50% }
+
+CI will provide a default pipeline for all projects. Projects may use
+that or specify another one.
+
+The pipeline will be divided into several stages. Mandatory stages for
+all changes and all projects are commit, acceptance stage, and
+deployment to production. Other stages may be added to specific
+changes or projects as needed.
+
+The goal is that if the commit and acceptance stages pass, the change
+is a candidate that can be deployed to production, unless the project
+is such that it needs (say) manual testing or other human decision for
+the production deployment decision. Likewise, if the component or the
+change is particularly security or performance sensitive, stages that
+check those aspects may be required. CI will have ways of indicating
+the required changes per component, and also per change. (It is
+unclear how this will be managed.)
+
+If the commit or acceptance stage fails, there is not a production
+candidate. The pipeline as a whole fails. Any artifacts built by the
+pipeline will not be deployable to production, but they may be
+deployable to test environments, or downloaded by developers for
+inspection.
+
+## The commit stage
+
+The commit stage builds any deployable artifacts, such as executable
+binaries, minimized JavaScript, translation files, or Docker images.
+It is important that artifacts don't get rebuilt by later stages,
+because rebuilding does not always result in bitwise identical output.
+Instead the goal is to build once, test the artifacts, and deploy the
+tested artifacts, instead of rebuilding and maybe deploying something
+different than what was tested.
+
+The commit stage also runs unit tests, and any other tests that can be
+run in isolation from other parts of the system, and that are also
+quick. The commit stage does not have access to backing services, such
+as databases or other components of the overall system. For example,
+when the pipeline processes a change to a MediaWiki extension, the
+commit stage doesn't have access to MediaWiki core or the MariaDB
+MediaWiki uses. Integration or system tests should be done in the
+acceptance test stage.
+
+The commit stage also runs code health checks.
+
+The commit stage is expected to be fast, aiming at less than five
+minutes, so that we can expect developers to wait for it to pass
+successfully. This will be a new requirement on our developers.
+
+The commands to build (compile) or run automated tests are stored in
+the repository, either explicitly, or by indicating the type of build
+needed. There might be a `.pipeline/config.yaml` file in the
+repository, which specifies that `make` is the command that builds the
+artifacts. Otherwise, the file may specify that it's a Go project, and
+CI would know how to build a Go project. In this case we can change
+the commands to build a Go project by changing CI only, without having
+to change each git repository with a Go program.
+
+Only the declarative style will be possible for building Docker images,
+as we want control over how that is done (**SECURE** requirement).
+
+CI may enforce specific additional commands to run, to build or test
+further things; this can be used by RelEng to enforce certain things.
+For example, we may enforce code health checks, or enable (or
+disable) debug symbols in all builds. Such enforcement will be done in
+collaboration with our developers.
+
+Any build dependencies needed during the commit stage must be
+specified explicitly. For example, the minimum required version of Go
+that should be installed in the build environment would be a build
+dependency. If a project build-depends on another project, it needs to
+specify which project, and which artifacts it needs installed from the
+other project. Explicit build-dependencies are more work, but result
+in fewer problems due to broken heuristics.
+
+## The acceptance stage
+
+During the acceptance stage CI deploys artifacts built in the commit
+stage to a production-like system that has the same versions of all
+software as production, except for the changes being processed by the
+pipeline. CI will then run automated acceptance tests, and other
+integration and system tests, against the deployed software. The test
+environment is clean and empty, and well-known, unless and until the
+test suite inserts data or makes changes.
+
+The acceptance stage can take time. Developers are not expected to
+wait until it is finished before they move on to working on something
+else.
+
+## Deployment to production
+
+If prior stages have passed successfully, and manual code review
+("Gerrit CR:+2 vote") has approved the change, this stage deploys the
+change to production.
+
+## Manual tests
+
+Testers may instruct CI to deploy any recently built set of artifacts
+to a dedicated test environment, and can use the software in that
+environment where it is isolated from others, and won't suddenly
+change underneath them. The details of how this will be implemented
+are to be determined later.
+
+This feature of the CI can also be used to demonstrate upcoming
+features that are not yet ready to be deployed to or enabled in
+production.
+
+## Capacity tests, non-functional requirements
+
+Capacity tests, and other tests for non-functional requirements, will
+also be done in dedicated, isolated production-like environments.
+RelEng will work with the performance team to sort out the details.
+
+# CI Architecture
 
 ## The WMF development ecosystem
 
@@ -705,122 +821,6 @@ credentials it needs for accessing the artifacts and doing the
 deployment. There may be several deployment services, for deploying to
 environments with different security needs.
 
-# The CI pipeline
-
-![The default pipeline](pipeline.svg){ height=50% }
-
-CI will provide a default pipeline for all projects. Projects may use
-that or specify another one.
-
-The pipeline will be divided into several stages. Mandatory stages for
-all changes and all projects are commit, acceptance stage, and
-deployment to production. Other stages may be added to specific
-changes or projects as needed.
-
-The goal is that if the commit and acceptance stages pass, the change
-is a candidate that can be deployed to production, unless the project
-is such that it needs (say) manual testing or other human decision for
-the production deployment decision. Likewise, if the component or the
-change is particularly security or performance sensitive, stages that
-check those aspects may be required. CI will have ways of indicating
-the required changes per component, and also per change. (It is
-unclear how this will be managed.)
-
-If the commit or acceptance stage fails, there is not a production
-candidate. The pipeline as a whole fails. Any artifacts built by the
-pipeline will not be deployable to production, but they may be
-deployable to test environments, or downloaded by developers for
-inspection.
-
-## The commit stage
-
-The commit stage builds any deployable artifacts, such as executable
-binaries, minimized JavaScript, translation files, or Docker images.
-It is important that artifacts don't get rebuilt by later stages,
-because rebuilding does not always result in bitwise identical output.
-Instead the goal is to build once, test the artifacts, and deploy the
-tested artifacts, instead of rebuilding and maybe deploying something
-different than what was tested.
-
-The commit stage also runs unit tests, and any other tests that can be
-run in isolation from other parts of the system, and that are also
-quick. The commit stage does not have access to backing services, such
-as databases or other components of the overall system. For example,
-when the pipeline processes a change to a MediaWiki extension, the
-commit stage doesn't have access to MediaWiki core or the MariaDB
-MediaWiki uses. Integration or system tests should be done in the
-acceptance test stage.
-
-The commit stage also runs code health checks.
-
-The commit stage is expected to be fast, aiming at less than five
-minutes, so that we can expect developers to wait for it to pass
-successfully. This will be a new requirement on our developers.
-
-The commands to build (compile) or run automated tests are stored in
-the repository, either explicitly, or by indicating the type of build
-needed. There might be a `.pipeline/config.yaml` file in the
-repository, which specifies that `make` is the command that builds the
-artifacts. Otherwise, the file may specify that it's a Go project, and
-CI would know how to build a Go project. In this case we can change
-the commands to build a Go project by changing CI only, without having
-to change each git repository with a Go program.
-
-Only the declarative style will be possible for building Docker images,
-as we want control over how that is done (**SECURE** requirement).
-
-CI may enforce specific additional commands to run, to build or test
-further things; this can be used by RelEng to enforce certain things.
-For example, we may enforce code health checks, or enable (or
-disable) debug symbols in all builds. Such enforcement will be done in
-collaboration with our developers.
-
-Any build dependencies needed during the commit stage must be
-specified explicitly. For example, the minimum required version of Go
-that should be installed in the build environment would be a build
-dependency. If a project build-depends on another project, it needs to
-specify which project, and which artifacts it needs installed from the
-other project. Explicit build-dependencies are more work, but result
-in fewer problems due to broken heuristics.
-
-## The acceptance stage
-
-During the acceptance stage CI deploys artifacts built in the commit
-stage to a production-like system that has the same versions of all
-software as production, except for the changes being processed by the
-pipeline. CI will then run automated acceptance tests, and other
-integration and system tests, against the deployed software. The test
-environment is clean and empty, and well-known, unless and until the
-test suite inserts data or makes changes.
-
-The acceptance stage can take time. Developers are not expected to
-wait until it is finished before they move on to working on something
-else.
-
-## Deployment to production
-
-If prior stages have passed successfully, and manual code review
-("Gerrit CR:+2 vote") has approved the change, this stage deploys the
-change to production.
-
-## Manual tests
-
-Testers may instruct CI to deploy any recently built set of artifacts
-to a dedicated test environment, and can use the software in that
-environment where it is isolated from others, and won't suddenly
-change underneath them. The details of how this will be implemented
-are to be determined later.
-
-This feature of the CI can also be used to demonstrate upcoming
-features that are not yet ready to be deployed to or enabled in
-production.
-
-## Capacity tests, non-functional requirements
-
-Capacity tests, and other tests for non-functional requirements, will
-also be done in dedicated, isolated production-like environments.
-RelEng will work with the performance team to sort out the details.
-
 # CI implementation
 
 FIXME This needs to be written, but it needs a lot of thinking first