From 76825e87ef8a6866b2f47636a31e13a082d232b2 Mon Sep 17 00:00:00 2001 From: Lars Wirzenius Date: Sun, 1 Apr 2018 19:50:20 +0300 Subject: Update: roadmap for making ick usable by others --- roadmap.mdwn | 189 +++++++++++++++++++++++++++++++++++------------------------ 1 file changed, 114 insertions(+), 75 deletions(-) (limited to 'roadmap.mdwn') diff --git a/roadmap.mdwn b/roadmap.mdwn index ba8d18b..9870f15 100644 --- a/roadmap.mdwn +++ b/roadmap.mdwn @@ -5,90 +5,129 @@ because predicting the future is difficult. This road map also doesn't give any dates, this is a hobby project. It's also open for discussion. -ALPHA-3 +ALPHA-4 ============================================================================= -Fix blob service to work reliably across networks. Fix other bugs. -Good candidates for bugs to fix: +Use [Qvisqve][] as the IDP. Use [Let's Encrypt][] for TLS +certificates. Update ick [[architecture]] document so it describes a +system that others than Lars can be expected to use. -* [No way to force a running job to stop](http://ick-support.liw.fi/47096bf68fef414296097806d2a647f3.html) -* [Ick component startup code is messy](http://ick-support.liw.fi/cd4979712d0544c285cf025a40a0a203.html) -* [blob service fails to transfer large blobs completely, except over localhost](http://ick-support.liw.fi/67234c63a5e5452e8729f9aa4731fb39.html) +[Qvisqve]: http://www.qvarn.org/qvisqve/ +[Let's Encrypt]: https://letsencrypt.org/ -ALPHA-4 +ALPHA-5 ============================================================================= -Add [Qvisqve][] as an IDP. Drop all client-side token generation. Have -workers be identified by the JWT token created by the IDP, instead of -trusting the workers identify themsselves correctly. +Add simple notifications of finished builds via emails. The SMTP +server and other sender information will be configured via a +configuration file. The recipients will be configured in projects in +the controller. The emails will say which project build finished, +whether the build succeded or failed, and have the build log as an +attachment. The notification will be triggered automatically by the +controller. + +The controller will construct a complete build graph when a project +build is triggered. The build graph will be a list of of all the +actions in all the pipelines of the project (later this will be a +directed acyclic graph to allow for more concurrency). For the build +to succeed, all actions must be executed successfully. Pipelines will +not be visible in the graph anymore. + +Projects may reuse the workspace contents from an earlier build, by +populating the workspace using a blob containing an archive of the +workspace, uploaded at the end of a previous build. Pipelines need to +be written in such a way that if the blob doesn't exist, they'll do a +fresh build from scratch. + + +ALPHA-6 +============================================================================= -[Qvisqve]: http://qvarn.org/qvisqve/ +This version will be usable by others than Lars. It will fix any bugs +and packaging problems and add or update installation and other +documentation to allow that to happen. +This will require help from others, volunteers willing to try +installing ick, and report any issues or questions they have. -ALPHA-5 + +Roadmap ============================================================================= -Add the following changes: - -> Ick builds and publishes its own .deb packages. It can build both CI -> builds, and release builds. It maintains its own APT repository with -> both CI and release builds. Release builds are built from a PGP -> signed git tag of a specific form. CI builds are built from the tip -> of the master branch, and the upstream and Debian version numbers -> are changed to include the build number so that different CI builds -> can be told apart. - -> The builds happen in containers. The container contains the output -> for `debootstrap` with the `build-essential` package installed. Any -> other build dependencies are installed automatically into the -> container, based on information in the `debian/control` file. - -> A new component is added to do trusted operatations as part of the -> build: ssh access for fetching from git, ssh access for package -> uploads, and PGP signing of package uploads. A project can specify -> which APT repository the .deb uploads should go to. - -The high-level architecture will look like this: - - @startuml - component Git - component APT - package "Trusted ick" { - component Controller - component Trusted - component Blobs - } - package "Semi-trusted ick" { - component Worker - } - - Git -> Controller : 1. trigger build - Controller -down-> Trusted : 2. give work - Git -> Trusted : 3. git clone - Trusted -> Blobs : 4. upload workspace - Controller -> Worker : 5. give work - Blobs -> Worker : 6. get workspace - Worker --> Blobs : 7. upload .deb - Blobs --> Trusted : 8. download .deb - Trusted -> APT : 9. upload signed .deb - @enduml - -Major changes: - -* A **trusted worker** component is added. It never runs untrusted - code, which means code that gets downloaded from the git server. - This means it can hold and use secrets such as SSH and PGP keys. - -* The controller will direct the trusted worker to download source - code from the git server, and upload it to the blob service. - Further, the controller will direct the normal worker to download - the source code blob from the blob service, and to upload the build - artifacts to the blob service. Finally, the controller will direct - the trusted worker to download artifacts from the blob service, PGP - sign the .deb files, and upload them to an APT repository. - -* The **APT repository server** component is added. It will only - accept uploads from the trusted worker. It will have both CI and - release builds, but keeps them separate so that users may choose - which to install. + @startroadmap + alpha6: + label: ALPHA-6 + depends: + - alpha5 + - docs + + alpha5: + label: ALPHA-5 + depends: + - alpha4 + - notifications + - build_graph + - incremental + + alpha4: + label: ALPHA-4 + depends: + - qvisqve + - letsencrypt + - self_hosting + - archdocs + + self_hosting: + label: | + ick builds, + publishes + its own .debs + + notifications: + label: | + ick has a + rudimentary + notification + system + + qvisqve: + label: | + ick uses Qvisqve + as an IDP + + build_graph: + label: | + ick constructs a + build graph from + all pipelines at + trigger time + + incremental: + label: | + ick can do + incremental + builds (reuse + workspaces + across builds) + + letsencrypt: + label: | + ick deploys + Let's Encrypt + TLS certificates + + docs: + label: | + there's sufficient + docs for others to + install ick for + themselves + + archdocs: + label: | + the arch doc + describes ick + that others can + use + @endroadmap -- cgit v1.2.1