NEWS for ick2, a CI server ============================================================================= Copyright 2017-2018 Lars Wirzenius This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. You should have received a copy of the GNU Affero General Public License along with this program. If not, see . Version 0.46+git, not yet released ---------------------------------- * Worker manager now sets `LC_ALL` and `DEBIAN_FRONTEND` variables inside the container. * Each action in a pipeline MUST now have a `where` field. Previously you could leave it out and it would act as if `where: host` was specified. * All commands run by worker manager now get their stdin redirected from /dev/null. Version 0.46, released 2018-04-22 ---------------------------------- * The `populate_systree` action now takes the name from the `systree_name` parameter, if no `systree_name` field is given, or its value is `auto`. * New command: `icktool show-latest-log PROJECT`. * `icktool make-it-so` now accepts filename from which to read project and pipeline descriptions. * `icktool --no-verify-tls` now actually works. Version 0.45, released 2018-04-21 ---------------------------------- * Force new release, since the previous one failed to build right. Again. This is why I am writing CI software. Version 0.44, released 2018-04-21 ---------------------------------- * Force new release, since the previous one failed to build right. Version 0.43, released 2018-04-21 ---------------------------------- * `icktool status` is now faster. Previously it was linear to the number of projects, now it is as fast for one as for a hundred. Also, the list of projects is now sorted by project name. * `icktool show-log` has been added. * `icktool show` can show any JSON resource (not logs) and needs to be given the URL path component. * `icktool make-it-so` now updates existing projects and pipelines. * `icktool trigger` now takes any number of projects to trigger on the command line. Version 0.42, released 2018-04-20 ---------------------------------- * `icktool status`, `icktool trigger` commands added. * `icktool build-graph PROJECT [BUILD-ID]` command added. It writes out a .dot file for showing the build graph for a build. Feed that to **dot**(1) for processing. Works for a running build as well as older builds. Version 0.41, released 2018-04-20 ---------------------------------- * `icktool` is now the rewritten version. * The controller now allows triggering an entire project, instead of a pipeline in a project. Additionally, when the project build starts, all actions from all pipelines are queued for the build. Version 0.40, released 2018-04-16 ------------------------------------ * This is the ALPHA-4 version. Version 0.39.2, released 2018-04-16 ------------------------------------ * Fix build problem. Version 0.39.1, released 2018-04-16 ---------------------------------- * Fix build problem. Version 0.39, released 2018-04-16 ---------------------------------- * Fix bug in worker-manager in how `BUILD_NUMBER` is set in the environment when running command in container. Version 0.38, released 2018-04-15 ---------------------------------- * Fix bug in worker-manager to set `BUILD_NUMBER` in environment when running command in container. Version 0.37, released 2018-04-15 ---------------------------------- * `icktool trigger` now works. There was a bug in how the controller response was printed out. * Worker-manager now gives all the child processes it runs the build number as the `BUILD_NUMBER` environment variable. This lets pipeline actions do things like embed it in Debian package versions for CI builds. Version 0.36, released 2018-04-08 ---------------------------------- * `icktool` is again mostly functional. Deleting resources (projects, pipelines, etc) hasn't been implemented yet. Neither has handling blobs. These will be added later, when some one needs them. Version 0.35, released 2018-04-07 ---------------------------------- * Controller now knows the URL to the identity provider's authentication endpoint. This is reported via `/version`, which now doesn't require authentication. * Worker-manager and `icktool` now fetch an access token from the IDP, instead of generating token themselves. * `icktool` has been substantcially rewritten, with much loss of functionality. Lost functionality will be added back as the loss is noticed by users. Version 0.34, released 2018-04-05 ---------------------------------- * `icktool --verify-tls` now works as intended. `worker-manager` now has a `--verify-tls` option. For both programs, the default is "verify". Use `--no-verify-tls` to turn it off. Version 0.33, released 2018-04-01 ---------------------------------- * This is the ALPHA-3 release of ick. The major change from ALPHA-2 is that ick can now actually control multiple workers. Also, the blob service has been renamed to artifact store; the new name is supposed to be clearer to the uninitiated. Version 0.32, released 2018-03-30 ---------------------------------- * Fix: empty workspace when build starts. Again. Version 0.31, released 2018-03-30 ---------------------------------- * Fix: workspace is now emptied when build starts. Version 0.30, released 2018-03-30 ---------------------------------- * Worker manager no longer mounts proc and sys into the chroot. Version 0.29, released 2018-03-30 ---------------------------------- * "Blob service" has been renamed to "artifact store". * Artifact store can handle large artifacts now. Tested with 100 gigabytes. * The worker manager has been largely rewritten, with (unit) tests added. The new code is more maintainable and more easily extensible. Version 0.28, released 2018-02-24 ---------------------------------- * This is the ALPHA-2 release of ick. The major change from ALPHA-1 is that ick can now more easily be deployed so that it has more than one worker (and has also been tested that way). See the ick2-ansible.git repository for how. Version 0.27, released 2018-02-24 ------------------------------------ * `check` now allows only the yarn scenarios to be run. This speeds up the edit-test iteration cycle when changing yarns. * `icktool` now gets the blob service URL from the controller, since it can now be different from the controller URL. Version 0.24.*, multiple intermin internal releases --------------------------------------------------- * All systemd .service files now fail to start if the corresponding configuration file is missing. This allows ick2 .deb to be installed without any services starting, and using configuration management to configure and enable different services on different hosts. * Fix: when worker manager fails to register itself, it terminates. Systemd will restart it after a short while, and then maybe it works, e.g., if the controller is ready then. Can't use systemd unit dependencies, when the units run on different hosts. * Controller now reports URL to blob service. Worker manager gets it from there. This allows controller and blob service to be on different hosts. * Fix worker manager service unit to refer to the correct configuration file. Also, don't install the configuration file. Version 0.24, released 2018-02-07 ---------------------------------- * The workspace is now mounted as `/workspace` in a container. If `where` is `host` or `chroot`, it's still at `/var/lib/ick/workspace`. * The `debootstrap` action now defaults to using the `deb.debian.org` mirror, if the action doesn't specify one. * The `debootstrap` action now accepts a suite value of `auto`, which makes it get the suite name from the `debian_codename` pipeline parameter. The old behaviour of having the suite name as an argument to the `debootstrap` field in the action still works. * When an action is run in a chroot, the `/proc` and `/sys` filesystems get bind mounted temporarily into the chroot. * Shell and python actions (in chroot, container, or host) now get the environment variable `LC_ALL` set to `C`, and `DEBIAN_FRONTEND` set to `noninteractive`. * The systemd unit for the worker manager now restarts the worker manager automatically if it crashes, after 10 seconds. * The service files are now called ick-controller, ick-worker, and ick-blob-service. They don't start unless the corresponding configuration file in /etc exist. This makes the ick Debian package safe to install, if one only wants icktool. Version 0.23, released 2018-01-22 ---------------------------------- * This is the ALPHA-1 version of ick. It is the first publicly advertised version. * The `name` field in pipeline resources is now called `pipeline`. * `icktool status` output now has a new column to indicate latest build status. * Builds are now named `foo/123` (build 123 of project foo). Logs similarly. * New command: `icktool show-latest-log foo` shows the latest log for the `foo` project. * Worker manager does systree blob uploads and downloads via curl, so it doesn't need to keep a large blob in memory. Version 0.22, released 2018-01-15 ---------------------------------- * The worker manager can now run things in a chroot, using the workspace as the root directory. Add a `where: chroot` field to the `shell` or `python` pipeline actions. * The worker manager can now archive the contents of the workspace and store it in the blob service. It can also retrieve a blob an unpack it into `/var/lib/ick/systree`, and run builds using that directory as the root of a container. * The worker manager is now significantly more efficient when reporting build output to the controller. Previously, this would be done every time the stdout or stderr of the build command produced any output. In practice this meant an HTTP request every few bytes of output. Output is now buffered (up to a kibibyte per output stream), reducing the overhead rather a lot. * `icktool` has a new subcommand, `make-it-so`, which reads a YAML file which lists all projects and pipelines, and creates or updates them via the controller API. Version 0.21, released 2017-12-27 ---------------------------------- * Project may now specify parameters, and pipelines in them may use the parameter values, both in shell and Python actions. Version 0.20, released 2017-12-16 ---------------------------------- * A blob service has been added. It is so simple you will weep. * `icktool` has two new subcommands `put-blob` and `get-blob`. * The worker manager now automatically registers itself to the controller. It is no longer necessary to do that manually. Version 0.19, released 2017-11-29 ---------------------------------- * Project may now specify a JSON object as project parameters. All work steps that the worker-manager gets, include the parameters. Worker-manager does not do anything with them yet. * Pipelines are now shared between projects, and projects refer to them by name. Pipeline status is per project, however, so if two project refer to the same pipeline, they can trigger it separately. * Pipeline steps may now be implemented in Python 3. * Pipelines are now run in `/var/lib/ick/workspace`, and it gets emptied when a pipeline starts to build. * Pipelines may now use a `debootstrap` step to have worker-manager create a system tree with `debootsrap`. This requires the `_ickwm` user (which worker-manager runs as) to have password-less sudo access. The packaging does not arrange that yet. Version 0.18, released 2017-11-25 ---------------------------------- * Add subommand `icktool status` which shows current status of every pipeline in every project. * `POST /projects` (or `icktool create-project`) with a project description that names an already existing project fails. Likewise, `PUT /projrect/foo` (or `icktool update-project`) when no project foo exists is now an error. Version 0.17, released 2017-11-19 ---------------------------------- * Added temporarily an endpoint to trigger a pipeline without authorization. Version 0.16, released 2017-11-19 ---------------------------------- * icktool now generates the token itself, instead of running the `create-token` script. * icktool now has a very rudimentary manual page. It's not really of much use, but it's there and it's easier to add things to a dummy file than to start a new file. * icktool now reads YAML, instead of JSON. However, as YAML is a superset of JSON, old inputs will still wors. * Some authorization fixes for the controller. Version 0.15, released 2017-11-13 ---------------------------------- * If the controller is told a build step has failed (ended with an integer, non-zero exit code), it terminates the build there. * Build ids are now incremented. Version 0.14, released 2017-11-12 ---------------------------------- * The contoller can now schedule work for a worker, to run a build. * `icktool` can now show and set the status of a pipeline, show builds, and show logs. * `worker-manager` works with the new version of the controller. The .deb package installs a worker-manager with the controller. Version 0.13, released 2017-10-16 ---------------------------------- * Added the `/workers` endpoint, for managing information about workers. * The `icktool` utility can now list, create, show, update, and delete projects and workers. Version 0.12, released 2017-10-15 ---------------------------------- * Some code refactoring, for clarity, plus some clarifications to yarn test suite added. No functional changes. Version 0.11, released 2017-10-15 ---------------------------------- * Add an `icktool` script. Currently it can generate a token or ask the controller for the controller's version. Version 0.10, released 2017-10-14 --------------------------------- * The yarn test suite can be run so it starts the ick controller locally, or against a pre-existing one that runs remotely. Version 0.9, released 2017-10-14 --------------------------------- * Debian package creates /etc/ick. Version 0.8, released 2017-10-14 --------------------------------- * Debian binary package has its dependencies specified. * The Debian binary package creates `/var/lib/ick/state`. Version 0.7, released 2017-10-14 --------------------------------- * `ick2version` has been removed, the canonical location for the version is not `ick2.__version__`. * Debian package now creates an `_ick` user and group and runs service under that account. Previously it was using the `www-data` account, which can be used by other services as well. This way provides more isolation. * Logs are now in `/var/log/ick`, not `/tmp`. Version 0.6, released 2017-10-13 --------------------------------- * Don't start gunicorn3 in daemon mod. Version 0.5, released 2017-10-13 --------------------------------- * More bug fixes for deploment: Add `start_ick` and make `ick2.service` use it. Version 0.4, released 2017-10-13 --------------------------------- * Bug fixes to allow Ansible to deploy ick2 on a host as a service: * The `ick2version` Python package gets installed. * Temporarily put logs in /tmp. I don't know why logging to syslog fails. * The `create-token` script is a little easier to use now. * The controller logs all exceptions to the `logging` module, instead of `slog`, so they're logged **somewhere** when the program is starting. Version 0.3, released 2017-10-13 ----------------------------------- * Add rudimentary controller and worker manager. The controller supports multiple concurrent builds by different workers. * Add Python3, and gunicorn support, including in the .deb packaging. Once the .deb is installed, the ick2 controller can be started manully (doesn't get enabled or started automatically, nor is there a default project list). * License for all code is now AGPL3+. * All code should now work in Python 3. * /version added to API. Version 0.2.1, released 2017-04-01 --------------------------------- Version 0.2, released 2017-04-01 --------------------------------- Version 0.1, released 2017-04-01 ----------------------------------------------------------------------------- This is the first version.