diff options
author | Lars Wirzenius <liw@liw.fi> | 2018-07-29 13:47:54 +0300 |
---|---|---|
committer | Lars Wirzenius <liw@liw.fi> | 2018-07-29 13:47:54 +0300 |
commit | a1fa5f264234b3c6cb816fccf1d1b5066b1d85ad (patch) | |
tree | 7406c811833a106797a5fa69238a2194260c7dab /icktool.mdwn | |
parent | bdb116582b5cb1e83dbb8cb5948e1b5a96cd9ac3 (diff) | |
download | ick.liw.fi-a1fa5f264234b3c6cb816fccf1d1b5066b1d85ad.tar.gz |
Change: split icktool page into smaller pages
Diffstat (limited to 'icktool.mdwn')
-rw-r--r-- | icktool.mdwn | 244 |
1 files changed, 0 insertions, 244 deletions
diff --git a/icktool.mdwn b/icktool.mdwn index 5c7d178..5d8d2d3 100644 --- a/icktool.mdwn +++ b/icktool.mdwn @@ -105,253 +105,9 @@ Get help on command line use: ./icktool help make-it-so -Ick files ------------------------------------------------------------------------------ - -An `.ick` file is the input to the `icktool make-it-so` command. It -uses YAML syntax, and has two top-level entries: `projects` and -`pipelines`. Each of those is a list, one of projecs to build, and one -of pipelines. - -A project has a name, optionally defines some parameters, and lists -one or more pipelines. A pipeline is a re-useable sequence of build -actions to achieve a goal, such a checking out code from version -control, or building binaries from source code. - -You can roughly think of a pipeline as a subroutine that the project -can call. Each pipeline gets all the project parameters, and can do -things based on the parameter values. - -In the example above, there is one project, called `say_hello`, which -defines one parameters, `target`, with the value `world`. It uses one -pipeline, `greet`. - -There is one pipeline, `greet`, which accepts the paremeter `target`, -and consists of one action, which consists of running a snippet of -shell code on the build host. The snippet extracts the value of the -parameter. The snippet uses a pre-defined shell function `params`, -which outputs the values of all parameters, and extracts the value of -the `target` parameter with the `jq` tool. - -Actions are implemented and executed by the worker manager, which is -an agent running on each build host. The worker manager queries the -controller for an action to execute, executes it, and reports any -output and exit code back to the controller. The controller keeps -track of all projects, pipelines, builds, build output, and build -results. - -Note that `icktool make-it-so` does not touch in any way project or -pipeline resources not mentioned in the .ick file. If the controller -knows of a project foo, but it is not in the .ick file, the foo -project is not modified, or deleted. It remains exactly as it was. -Also, projects may reference any pipelines the controller knows about, -even if not mentioned in the .ick file. - - -Pipelines: "where" ------------------------------------------------------------------------------ - -Each action in a pipeline MUST specify where it is run. There are -three possibilities: - -* `host` – directly on the host -* `chroot` – in a chroot of the workspace -* `container` – in a container using a defined system tree - (systree) and workspace bind-mounted at /workspace - -You should strive to run all actions in conatiners, since that makes -it hardest to mess up the worker host. - -Actions are run as root in the container, and in a chroot. On the -host, they can use `sudo` to gain root access. - - -Pipelines: actions ------------------------------------------------------------------------------ - -The worker manager defines (at least) the following actions: - -* `shell: snippet` - - Run `snippet` using the shell. The cwd is the workspace. The shell - function `params` outputs all parameters as JSON. The `jq` tool - can be used to extract the value of a specific parameter. - -* `python: snippet` - - Run `snippet` using Python 3. The cwd is the workspace. The global - variable `params` is a dict with all parameters. - -* `debootstrap: auto` - - Run the debootstrap command in the workspace. If the value given - to the `debootstrap` key is is `auto`, the Debian dist installed - into the workspace is taken from the parameter `debian_codename`. - The dist can also be named explicitly as the value. - - The mirror defaults to `http://deb.debian.org/debian`, but can be - given explicitly, if needed. - -* `archive: workspace` - - Take the current content of the workspace, put it in a compressed - tar archive, and uplad it to the artifact store. Get the name of - the artifact from the parameter named in the `name_from` field, if - given, and the`artifact_name` parameter if not. - - Optionally archive only parts of the workspace. User can give a - list of globs in the `globs` field, and anything that matches any - of the shell filename patterns will be included. Default is to - archive everything. - - Example: - - - archive: workspace - name_from: debian_packages - globs: - - "*_*" - - This would archive everything at the root of the workspace with an - underscore in the name. The artifact will be named using the value - of the `debian_packages` project parameter. - -* `archive: systree` - - This is identical to `archive: workspace`, except it archives the - system tree instead of the workspace. - -* `action: populate_systree` - - Get an artifact, from the artifact store, and unpack it as the - systree. The artifact is assumed to be a compressed tar archive. - The name of the artifact comes from the `systree_name` parameter, - or the parameter named in the `name_from` field. - - If the artifact does not exist, end the build with failure. - -* `action: populate_workspace` - - Identical to `action: populate_systree`, except it unpacks to the - workspace. The default parameter name is `workspace_name`. If the - artifact does not exist, do nothing. - -* `action: git` - - Clone the git repository given in `git_url` into the directory - named in `git_dir`, checking out the branch/tag/commit named in - `git_ref`. If the directory already exists, do a `git remote - update` inside it instead. - - This is intended to do all the networking parts of getting source - code from a git server. It should be run with `where: host`, so it - has network access and can use the worker's ssh key (for - non-public repositories). - - Further actions, inside a container, can do other operations. - -* `action: git_mirror` - - This will be replacing the `git` action. It has not yet been - implemented. You can use the `git` action as before, for now. - Eventually the `git` action will be removed, after a suitable - transition period. - - Mirror one or more git repositories specified in the `sources` - project variable. The parameter is expected to have as its value a - list of dicts, each dict containing the following fields: - - * `name` — name of the repository - * `repo` — URL of the repository - * `location` — ignored by `git_mirror`, but can be used to - specify where the clone'd repository is to be checked out - * `ref` — ignored by `git_mirror`, but can be used to - specify which ref (branch or tag or commit) should be checked out - - Additionally, `git_mirror` will use the `git_repo_base` project - parameter: if the `repo` field is a relative URL, it will be - joined with the `git_repo_base` value to form the full URL. If - `repo` is a full URL, it is used as is. - - Note that `git_mirror` does NOT do the checking out, only the - initial mirroring (as if by `git clone --mirror $repo - .mirror/$name`) or updating of the mirror (as if by `cd - .mirror/$name && git remote update --prune`). - - Ick provides a pipeline that uses the `git_mirror` action and - addtional `shell` or `python` actions to do the checkout. - -* `action: rsync` - - Copy the content of a directory in the workspace (named in - `rsync_src`) to a remote server (named in `rsync_target`. This - should be run on the host, to have network access and use of the - worker's ssh key. - -* `action: dput` - - Upload all Debian .changes files to the APT repository that is - part of an ick cluster. No parameters needed. - - This should be run on the host, to have network access and use of - the worker's ssh key. - -* `action: notify` - - Use the notification service to send notifications of the current - state of the build. You usually don't need to use these actions - manually, the controller adds a notification action automatically - when a build ends. - - -Notifications ------------------------------------------------------------------------------ - -Ick can send an email when a build ends. This is done automatically. -Set the project parameter `notify` to a list of email addresses that -should be notified. - - Triggering builds from version control servers, etc ----------------------------------------------------------------------------- To trigger a build of project foo, access the `https://controller/projects/foo/+trigger` (HTTP GET method, no authentication needed). - - -Standard pipelines ------------------------------------------------------------------------------ - -Ick comes with a set of "standard" pipelines, named `ick/get_sources` -and the like. They are stored and documented in the ick source code: -<http://git.liw.fi/ick2/tree/pipelines> - -The standard pipelines are not installed automatically. The ick -instance admin needs to install and update them manually. - -Example .ick files ------------------------------------------------------------------------------ - -The <http://git.liw.fi/liw-ci/tree/> repository has some .ick files, -used by ick's author for his personal instances. - - -General advice ------------------------------------------------------------------------------ - -On each ick instance, set up one or more projects that build systrees. -Build one for each target Debian release: `jessie`, `stretch`, -`unstable`, etc. Build each of these projects to populate the artifact -store with systree artifacts. Each actual project should use one of -the artifacts to do the build in a container. - -When building in a container, you can install more things inside the -container. If you build repeatedly, it may be worth having more than -just the minimal container. Instead have an artifact with all the -build dependencies installed, so each build does not have to start -with installing them, saving time. - -Ick can build several things at once. As many things as there are -workers. However, each build (currently) runs on one worker. - -There's currently no way to stop a triggered build. |