Add: some explanation of .ick files and actions

1 files changed, 134 insertions, 3 deletions

diff --git a/icktool.mdwn b/icktool.mdwn
index 5efa4a9..45128b4 100644
--- a/icktool.mdwn
+++ b/icktool.mdwn
@@ -15,6 +15,10 @@ code repository:
 
 [ick2.git]: http://git.liw.fi/ick2
 
+Alternatively, you can install the Debian package from
+<http://code.liw.fi/debian/>.
+
+
 Configuration
 -----------------------------------------------------------------------------
 
@@ -35,7 +39,8 @@ content:
 Change the `controller` line and the first line of `credentials.conf`
 to be the URL to your controller.
 
-Change `verify-tls` to `yes` if you have a CA-signed TLS certificate.
+Change `verify-tls` to `yes` if you have a CA-signed TLS certificate,
+such as by Let's Encrypt.
 
 
 Using
@@ -49,7 +54,7 @@ To see all projects and pipelines use the `status` command:
     $ 
 
 There should be no projects initially. To create some, create a file
-`ick.yaml` with the following content:
+`hello.ick` with the following content:
 
     projects:
 
@@ -78,7 +83,7 @@ a shell function provided by ick that outputs all parameters as JSON.
 
 Tell the controller about the new project:
 
-    ./icktool make-it-so ick.yaml
+    ./icktool make-it-so hello.ick
 
 Trigger a build:
 
@@ -98,3 +103,129 @@ Get help on command line use:
     ./icktool --help
     ./icktool help
     ./icktool help make-it-so
+
+
+Ick files
+-----------------------------------------------------------------------------
+
+An `.ick` file is the input to the `icktool make-it-so` command. It
+uses YAML syntax, and has two top-level entries: `projects` and
+`pipelines`. Each of those is a list, one of projecs to build, and one
+of pipelines.
+
+A project has a name, optionally defines some parameters, and lists
+one or more pipelines. A pipeline is a re-useable sequence of build
+actions to achieve a goal, such a checking out code from version
+control, or building binaries from source code.
+
+You can roughly think of a pipeline as a subroutine that the project
+can call. Each pipeline gets all the project parameters, and can do
+things based on the parameter values.
+
+In the example above, there is one project, called `say_hello`, which
+defines one parameters, `target`, with the value `world`. It uses one
+pipeline, `greet`.
+
+There is one pipeline, `greet`, which accepts the paremeter `target`,
+and consists of one action, which consists of running a snippet of
+shell code on the build host. The snippet extracts the value of the
+parameter. The snippet uses a pre-defined shell function `params`,
+which outputs the values of all parameters, and extracts the value of
+the `target` parameter with the `jq` tool.
+
+Actions are implemented and executed by the worker manager, which is
+an agent running on each build host. The worker manager queries the
+controller for an action to execute, executes it, and reports any
+output and exit code back to the controller. The controller keeps
+track of all projects, pipelines, builds, build output, and build
+results.
+
+
+Pipelines: "where"
+-----------------------------------------------------------------------------
+
+Each action in a pipeline MUST specify where it is run. There are
+three possibilities:
+
+* `host` &ndash; directly on the host
+* `chroot` &ndash; in a chroot of the workspace
+* `container` &ndash; in a container using a defined system tree
+  (systree) and workspace bind-mounted at /workspace
+
+You should strive to run all actions in conatiners, since that makes
+it hardest to mess up the worker host.
+
+Actions are run as root in the container.
+
+
+Pipelines: actions
+-----------------------------------------------------------------------------
+
+The worker manager defines (at least) the following actions:
+
+* `shell: snippet`
+
+    Run `snippet` using the shell. The cwd is the workspace. The shell
+    function `params` outputs all parameters.
+
+* `python: snippet`
+
+    Run `snippet` using Python 3. The cwd is the workspace. The global
+    variable `params` is a dict with all parameters.
+
+* `debootstrap: auto`
+
+    Run the debootstrap command in the workspace. If the value given
+    to the `debootstrap` key is is `auto`, the Debian dist installed
+    into the workspace is taken from the parameter `debian_codename`.
+    The dist can also be named explicitly as the value.
+
+    The mirror defaults to `http://deb.debian.org/debian`, but can be
+    given explicitly, if needed.
+
+* `archive: workspace`
+
+    Take the current content of the workspace, put it in a compressed
+    tar archive, and uplad it to the artifact store. Get the name of
+    the artifact from the `artifact_name` parameter.
+
+* `action: populate_systree`
+
+    Get the artifact named by `systree_name` and unpack it as the
+    systree. If the artifact is a valid operating system tree, later
+    actions can be run in containers.
+
+* `action: populate_workspace`
+
+    Get the artifact named by `workspace_name` and unpack it into the
+    workspace. If no such artifact exists, do nothing.
+
+* `action: git`
+
+    Clone the git repository given in `git_url` into the directory
+    named in `git_dir`, checking out the branch/tag/commit named in
+    `git_ref`. If the directory already exists, do a `git remote
+    update` inside it instead.
+
+    This is intended to do all the networking parts of getting source
+    code from a git server. It should be run with `where: host`, so it
+    has network access and can use the worker's ssh key (for
+    non-public repositories).
+
+    Further actions, inside a container, can do other operations.
+
+* `action: rsync`
+
+    Copy the content of a directory in the workspace (named in
+    `rsync_src`) to a remote server (named in `rsync_target`. This
+    should be run on the host, to have network access and use of the
+    worker's ssh key.
+
+* `action: dput`
+
+    Upload all Debian .changes files to the APT repository that is
+    part of an ick cluster. No parameters needed.
+
+    This should be run on the host, to have network access and use of
+    the worker's ssh key.
+