summaryrefslogtreecommitdiff
path: root/ci-arch.mdwn
diff options
context:
space:
mode:
authorLars Wirzenius <lwirzenius@wikimedia.org>2019-05-06 18:03:45 +0300
committerLars Wirzenius <lwirzenius@wikimedia.org>2019-05-06 18:03:45 +0300
commit5b023bd11d751779006f073f45bf8d862f7ac1a6 (patch)
tree0ecf075b74571d8463e596496bd6049bdbe2b8b3 /ci-arch.mdwn
parentf4863818374398e9c5b2a89f687fd6aedba49e33 (diff)
downloadwmf-ci-arch-5b023bd11d751779006f073f45bf8d862f7ac1a6.tar.gz
Change: format requirements formatting
Diffstat (limited to 'ci-arch.mdwn')
-rw-r--r--ci-arch.mdwn116
1 files changed, 58 insertions, 58 deletions
diff --git a/ci-arch.mdwn b/ci-arch.mdwn
index 5c9e6f4..df00710 100644
--- a/ci-arch.mdwn
+++ b/ci-arch.mdwn
@@ -76,55 +76,55 @@ and have been changed a little compared to that (as of the 21 March
## Very hard requirements
-* These are non-negotiable requirement that must all be fulfilled by a
- our future CI system.
+These are non-negotiable requirement that must all be fulfilled by our
+future CI system.
-* **(SELFHOSTABLE)** Must be hostable by the Foundation. It's not
+* **SELFHOSTABLE** Must be hostable by the Foundation. It's not
acceptable for WMF to rely on an outside service for this.
- * **(FREESOFTWARE)** Must be free software / open source. "Open
+ * **FREESOFTWARE** Must be free software / open source. "Open
core" like GitLab is be good enough, as long as we only need the
parts that provide software freedom.
This is partly due to the **SELFHOSTABLE** requirement, but also
because a WMF value is to prefer open source.
-* **(GITSUPPORT)** Must support git. We're not switching version control
+* **GITSUPPORT** Must support git. We're not switching version control
systems for CI.
-* **(UNDERSTANDABLE)** Must be understandable without too much effort to
+* **UNDERSTANDABLE** Must be understandable without too much effort to
our developers so that they can use CI/CD productively.
-* **(SELFSERVE)** Must support self-serve CI, meaning we don't block
+* **SELFSERVE** Must support self-serve CI, meaning we don't block
people if they want CI for a new repo. Due to **PROTECTPRODUCTION**,
there will probably to be some human approval requirement for new
projects, but as much as possible, people should be allowed to do
their work without having to ask permission.
- * **(SELFSERVE2)** Should allow the developers to define or
+ * **SELFSERVE2** Should allow the developers to define or
declare at least parts of the pipeline jobs in the repository:
what commands to run for building, testing, etc.
## Hard requirements
-* These are not absolute requirements, and can be negotiated, but only
- to a minor degree.
+These are not absolute requirements, and can be negotiated, but only
+to a minor degree.
-* **(FAST)** Must be fast enough that it isn't perceived as a bottleneck
+* **FAST** Must be fast enough that it isn't perceived as a bottleneck
by developers. We will need a metric for this.
- * **(SHORTCYCLETIME)** Must enable us to have a short cycle time
+ * **SHORTCYCLETIME** Must enable us to have a short cycle time
(from idea to running in production). CI is not the only thing
that affects this, but it is an important factor. We probably
need a metric for this.
-* **(TRANSPARENT)** Must make its status and what-is-going-on visible so
+* **TRANSPARENT** Must make its status and what-is-going-on visible so
that its operation can be monitored and so that our developers can
check the status of their builds themselves. Also the overall status
of CI, for example, so they can see if their build is blocked by
waiting on others.
-* **(FEEDBACK)** Must provide feedback to the developers as early as
+* **FEEDBACK** Must provide feedback to the developers as early as
possible for the various stages of a build, especially the early
stages ("can get source from git", "can build", "can run unit
tests", etc.).
@@ -132,98 +132,98 @@ and have been changed a little compared to that (as of the 21 March
The goal is to give feedback as soon as possible, especially in the
case of the build failing.
- * **(FEEDBACK2)** Must support providing feedback via Gerrit, IRC, and
+ * **FEEDBACK2** Must support providing feedback via Gerrit, IRC, and
Phabricator, at the very least. These are our current main feedback
channels.
-* **(SECURE)** Must be secure enough that we can open it to community
+* **SECURE** Must be secure enough that we can open it to community
developers to use without too much supervision.
-* **(MAINTAINED)** Must be maintained and supported upstream. The CI
+* **MAINTAINED** Must be maintained and supported upstream. The CI
system should not require substantial development from the
Foundation. Some customization is expected to be necessary.
-* **(MANYREPOS)** Must be able to handle the number of repositories,
+* **MANYREPOS** Must be able to handle the number of repositories,
projects, builds, and deployments that we have, and will have in the
foreseeable future.
-* **(METRICS)** Must enable us to instrument it to get metrics for CI use
+* **METRICS** Must enable us to instrument it to get metrics for CI use
and effectiveness as we need. Things like cycle times, build times,
build failures, etc.
-* **(GERRIT)** Must work with Gerrit as well as other self-hostable
+* **GERRIT** Must work with Gerrit as well as other self-hostable
code-review systems (e.g., GitLab), if we decide to move to that
later. This means, code review happens on Gerrit, after building and
automated tests pass, and positive code review triggers deployment
to production.
-* **(NOREBUILDING)** Must promote (copy) Docker images and other build
+* **NOREBUILDING** Must promote (copy) Docker images and other build
artifacts from "testing" to "staging" to "production", rather than
rebuilding them, since rebuilding takes time and can fail. Once a
binary, Docker image, or other build artifact has been built,
exactly that artifact should be tested, and eventually deployed to
production.
-* **(LOCALTESTS)** Must allow developer to replicate locally the tests
+* **LOCALTESTS** Must allow developer to replicate locally the tests
that CI runs. This is necessary to allow lower friction in
development, as well as to aid debugging. For example, if CI builds
and tests using Docker container, a developer should be able to
download the same image and run the tests locally.
-* **(AUTOMATEDEPLOYMENT)** Must allow deployment to be fully automated.
+* **AUTOMATEDEPLOYMENT** Must allow deployment to be fully automated.
- * **(AUTOMATEDSELFDEPLOYMENT)** Must be automatically deployable
+ * **AUTOMATEDSELFDEPLOYMENT** Must be automatically deployable
by us or SRE, onto a fresh server.
-* **(HSCALABLE)** Must be horizontally scalable: we need to be able to add
+* **HSCALABLE** Must be horizontally scalable: we need to be able to add
more hardware easily to get more capacity. This is particularly
important for build workers, which are the mostly likely bottleneck.
Also, probably environments used for testing.
-* **(PROGLANGS)** Must be able to support all programming languages we
+* **PROGLANGS** Must be able to support all programming languages we
currently support or are likely to support in the future. These
include, at least, shell, Python, Ruby, Java, PHP, and Go. Some
languages may be needed in several versions.
-* **(OUTPUTLINKS)** Must support HTTP linking to build results for
+* **OUTPUTLINKS** Must support HTTP linking to build results for
easier reference and discussion. This way a build log, or a build
artifact, can be reference using a simple HTTP (or HTTPS) link.
-* **(ARTIFACTARCHIVE)** Should allow archiving build logs,
+* **ARTIFACTARCHIVE** Should allow archiving build logs,
executables, Docker images, and other build artifacts for a long
period.
- * **(RETENTION)** The retention period should be configurable
+ * **RETENTION** The retention period should be configurable
based on artifact type, and whether the build ended up being
deployed to production.
-* **(CONFIGVC)** Must keep configuration in version control. This is
+* **CONFIGVC** Must keep configuration in version control. This is
needed so that we can track changes over time.
-* **(GATING)** Must support gating / pre-merge testing. FIXME: This
+* **GATING** Must support gating / pre-merge testing. FIXME: This
needs to be explained.
-* **(PERIODICBUILDS)** Must support periodic / scheduled testing. This
+* **PERIODICBUILDS** Must support periodic / scheduled testing. This
is needed so that we can test that changes to the environment
haven't broken anything. An example would be changes to Debian, upon
which we base our container images.
-* **(POSTMERGETESTS)** Must support post-merge testing. FIXME: This
+* **POSTMERGETESTS** Must support post-merge testing. FIXME: This
needs to be explained.
-* **(CIMERGES)** Must support tooling to do the merging, instead of
+* **CIMERGES** Must support tooling to do the merging, instead of
developers. We don't want developer merging by hand and pushing the
merges. CI should test changes and merge only if tests pass, so that
the branches for main lines of development are always releaseble.
-* **(TESTVC)** Must support storing tests in version control. This is
+* **TESTVC** Must support storing tests in version control. This is
probably best achieved by having tests be stored in the same git
repository where the code is.
-* **(BUILDDEPS)** Must have some way to declare dependent repositories /
+* **BUILDDEPS** Must have some way to declare dependent repositories /
software needed for testing. FIXME: This needs to be explained.
-* **(TESTSERVICES)** Must support services for tests — i.e., some
+* **TESTSERVICES** Must support services for tests — i.e., some
PHPUnit tests require MySQL. These are most important for
integration tests. Proper unit tests do not depend on any external
stuff. However, integration tests may well need MediaWiki, some
@@ -231,24 +231,24 @@ and have been changed a little compared to that (as of the 21 March
services, and possibly more. CI needs to be able to provide such
environments for testing.
-* **(OTHERGITORTICKETING)** Must allow changing git repository, code
+* **OTHERGITORTICKETING** Must allow changing git repository, code
review, and ticketing systems from Gerrit and Phabricator. We are
not currently looking at switching away from Gerrit and Phabricator,
but the future CI solution should not lock us into specific code
review or ticketing solutions.
-* **(PROTECTPRODUCTION)** Must protect production by detecting problems
+* **PROTECTPRODUCTION** Must protect production by detecting problems
before they're deployed, and must in general support a sensible
CI/CD pipeline. This is necessary both for the safety and security
of our production systems, a higher speed of development, and higher
productivity. The protection brings developer confidence, which
tends to bring speed and productivity.
- * **(ENFORCETESTS)** Must allow Release Engineering team to
+ * **ENFORCETESTS** Must allow Release Engineering team to
enforce tests on top of what a self-serving developer specifies,
to allow us to set minimal technical standards.
-* **(CACHEDEPS)** Must support dependency caching – we have castor, maybe
+* **CACHEDEPS** Must support dependency caching – we have castor, maybe
we could do better? Maybe some CI systems have this figured out?
This means, for example, caching npm and PyPI packages so that every
build doesn't need to download them directly from the centralised
@@ -256,38 +256,38 @@ and have been changed a little compared to that (as of the 21 March
## Softer requirements
-* These requirements are even more easily negotiated.
+These requirements are even more easily negotiated.
-* **(HA)** Should be highly available - can restart any component without
+* **HA** Should be highly available - can restart any component without
disrupting service.
-* **(LIVELOG)** Should have live console output of build.
+* **LIVELOG** Should have live console output of build.
-* **(MAXBUILDTIME)** Should have build timeouts so that a build may
+* **MAXBUILDTIME** Should have build timeouts so that a build may
fail if it takes too long. Among other reasons, this is useful to
automatically work around builds that get "stuck" indefinitely.
-* **(CLEANWORKSPACE)** Should provide a clean workspace for each test
+* **CLEANWORKSPACE** Should provide a clean workspace for each test
run - either a clean VM or container.
-* **(RATELIMIT)** Should have rate limiting - one user/project can not
+* **RATELIMIT** Should have rate limiting - one user/project can not
take over most/all resources.
-* **(CHECKSIG)** Should support validation and creation of GPG/PGP-signed
+* **CHECKSIG** Should support validation and creation of GPG/PGP-signed
git commits
-* **(SECRETS)** Should support secure storage of credentials / secrets.
+* **SECRETS** Should support secure storage of credentials / secrets.
## Would be nice
* These are so soft they aren't even requirements, and more wish list
items.
-* **(LIMITBOILERPLATE)** Would be nice for test abstractions to limit
+* **LIMITBOILERPLATE** Would be nice for test abstractions to limit
boiler-plate, i.e., all of our services are tested roughly the same
way without having to copy instructions to every repository.
-* **(PRIORITIZEJOBS)** Would be nice to prioritize jobs.
+* **PRIORITIZEJOBS** Would be nice to prioritize jobs.
* Use case: if there is a queue of jobs, there should be some
mechanism of jumping that queue for jobs that have a higher
@@ -296,31 +296,31 @@ and have been changed a little compared to that (as of the 21 March
* We currently have a Gating queue that is a higher priority than
periodic jobs that calculate Code Coverage.
-* **(ISOLATION)** Would be nice to support isolation / sandboxing.
+* **ISOLATION** Would be nice to support isolation / sandboxing.
* Jobs should be isolated from one another.
* Jobs should be able to install apt-packages without affecting
dependencies of other jobs.
-* **(CONTROLAFFINITY)** Would be nice to have configurable job
+* **CONTROLAFFINITY** Would be nice to have configurable job
requirements/affinity.
* Be able to schedule a job only on nodes that have at least X
available disk space/ram/cpu/whatever OR try to schedule on
nodes where a current build of this job isn't already running.
-* **(POSTMERGEBISECT)** Would be nice to post-merge git-bisect to find
+* **POSTMERGEBISECT** Would be nice to post-merge git-bisect to find
patch that caused a particular problem with a Selenium test.
-* **(DEPLOYWHEREVER)** Would be nice to have a mechanism for
+* **DEPLOYWHEREVER** Would be nice to have a mechanism for
deployment to staging, production, pypi, packagist, toollabs. We
could do with a way to deploy to any of several possible
environments, for various use cases, such as bug repoduction, manual
exploratory testing, capacity testing, and production. FIXME: what
do pypi and packagist do in the list?
-* **(MATRIXBUILDS)** Would be nice to have efficient matrix builds.
+* **MATRIXBUILDS** Would be nice to have efficient matrix builds.
* E.g., we currently run phpunit tests and browser tests for the
Cartesian product of {PHP7 PHP7.1 PHP7.2 HHVM} x {MySQL,
@@ -328,10 +328,10 @@ and have been changed a little compared to that (as of the 21 March
perform setup/git clone for all of those tests. Doing that in a
space and time efficient way would be good.
-* **(MOBILE)** Would be nice to support building and testing mobile
+* **MOBILE** Would be nice to support building and testing mobile
applications (at minimum for iOS and Android).
-* **(EMBARGO)** Would be nice to be able to run for secret/security
+* **EMBARGO** Would be nice to be able to run for secret/security
patches. This means CI should be able to build and deploy changes
that can't be made public yet, for security embargo reasons.