path: root/blog/2021/10/23/meeting.mdwn

[[!meta title="Iteration planning: October 23rd"]]
[[!tag meeting]]
[[!meta date="Sat, 23 Oct 2021 13:30:00 +0100"]]

[[!toc levels=2]]

# Review of actions from last meeting

Actions from last time, and their status are:

- Lars to create a label for blocking-high-profile-projects (name to
  be decided by Lars when he does it) and to label at least the
  Sequoia-PGP blockers if not also anything else he thinks may be
  blocking projects such as Rustup. **Half done, carrying**

- Lars to record decision about MSRV in `DECISIONS.md` in the
  `subplot` repository. **DONE**

- Lars and/or Daniel to add a Docker image for CI with Rust 1.48.0
  installed, or add it to the main image. **DONE**

- Lars and/or Daniel to change `.gitlab-ci.yml` to build and test
  using the 1.48.0 toolchain. **DONE**

- Lars and Daniel to discuss whether to change `./check` to take an
  option to specify which toolchain to use or to keep that outside of
  the script. **DONE** (We selected external control)

- Lars to refactor the changes for `build.rs` support, as discussed,
  and push an MR for it. **DONE**

- Lars should file an issue about the `get_basedir_from` changes, if
  the problem can be reproduced with the original code.
  **No reproduction, no issue filed**

- Lars to document decision that it's the user's responsibility to set
  up the environment in which tests are run. **DONE**

# Review of the iteration that has ended

[[!milestone 37]] had the following issued closed:

- [[!issue 219]] -- _Needs a release after 0.3.0 to test release automation_

Some progress was made on:

- [[!issue 198]] -- _Unify language handling for multi-impl-lang test suites_

The following issues remain open:

- [[!issue 198]] -- _Unify language handling for multi-impl-lang test suites_
- [[!issue 207]] -- _Release process is too manual, CI build of release .deb changes files_
- [[!issue 218]] -- _Automatable parts of release process are manual_
- [[!issue 238]] -- _Doesn't support `abstract` in document metadata_
- [[!issue 239]] -- _In metadata, `author` is a single string_

We moved the non-release-process remaining issues to the next milestone ([[!milestone 38]]) and then
closed off this iteration.

We removed [[!issue 207]] and [[!issue 218]] from the future iterations, and removed
[[!issue 207]] from the current goal.

# Review of the repositories

There are no MRs currently open in `subplot`, `subplot-web`, or `subplot-container-images`.

The `subplot-container-images` and `subplot-web` repositories do not
extra branches. The `subplot` repository has extra
branches:

- `subplot-rust` -- Daniel is keeping this around until he has
  completed the work on [[!issue 198]].
- `polyglot-docs` -- Daniel is working on [[!issue 198]] here.
- `docgen-cmark` -- Lars is working on making docgen use cmark
  rather than pandoc for the initial parse and processing.

We triggered the pipeline to build the container image for Subplot.
All `subplot` pipelines are clean.

# Current goal (goal 2; not changed for this iteration)

Subplot provides a set of libraries with identical capabilities in
each of the supported languages. Python remains a supported language.
Rust is promoted to supported-language status. Subplot will be tested
with all supported languages. In addition, any quality of life
improvements which can be done shall be done. This goal will be
considered complete when a release of Subplot has been made with the
unified language handling support complete.

# Issue review

We did a full review of all non-`someday-maybe` issues.

Changes we made include:

- [[!issue 126]] -- _Create a CI cronjob which attempts to `cargo update` everything_
  We opted to close this issue as we think the current release process
  is sufficient for now.
- [[!issue 133]] -- _No feature table comparing scenario runners from different codegen templates_
  We opted to mark this issue as blocked by [[!issue 198]] (multi-impl-lang docs)
- [[!issue 141]] -- _subplotlib: Needs better test suite_
  We opted to mark this as blocked by [[!issue 198]] and labelled it as `tracking`
  since we'll close it when the same subplot documents are testing runcmd, files,
  etc. for all language templates.
- [[!issue 180]] -- _Doesn't warn about unused embedded files_
  We agreed we should now be able to lint this as part of `codegen` so tentatively
  assigned it to Lars for this iteration.
- [[!issue 188]] -- _Old Subplot libraries have bindings with regex that could now be simple patterns_
  We tentatively assigned this to Lars for this iteration.
- [[!issue 213]] -- \_"Failure: document has no title" when there's duplicate keys"
  We are going to defer this to the new goal-3 as per below discussion notes
- [[!issue 214]] -- _Lacks examples and docs for using Rust step functions_
  We marked this as blocked by [[!issue 198]]
- [[!issue 220]] -- _Generated Rust test programs are hard to debug_
  We marked this as blocked by [[!issue 198]]
- [[!issue 221]] -- _Command line tool as insufficient logging_
  We made a note on the issue to the effect of the next steps.
  We added this to the current goal, but not to the iteration.
- [[!issue 222]] -- _Typeset output could indicate snippets are a continuation of the scenario and not a new scenario_
  We added the `typesetting` label to this and also `someday-maybe`.
- [[!issue 223]] -- _lib/runcmd lacks a generic "given an installed {name}" step_
  We added a blocked-by on [[!issue 198]]. We still need further discussion on this.
- [[!issue 224]] -- _Logging feature in the `subplot` binary needs tests_
  We updated the title to better indicate that this is about testing the behaviour of
  the `SUBPLOT_LOG*` environment variables.
  We also labelled it for goal 3.
- [[!issue 229]] -- _Licensing of Subplot libraries and subplotlib needs to be clarified_
  We agreed on MIT, Daniel removed the `actively-discussed` label. Daniel will take
  on the task of reaching out (and subsequently changing the licence terms) and so we
  assigned it to Daniel for this iteration.
- [[!issue 238]] -- _Doesn't support `abstract` in document metadata_
  Unassigned from Lars for now.
- [[!issue 239]] -- _In metadata, `author` is a single string_
  Unassigned from Lars for now.
- [[!issue 240]] -- _Could we mangle generated scenario function names in the Rust template?_
  We decided that the suggestions made on-issue were sufficient and closed the issue
- [[!issue 241]] and [[!issue 242]] -- Related to checking subplot, we made the
  two issues be linked. We assigned 242 to Lars for this iteration.

We left the docker-for-subplot issue alone, and subplot-web had no issues.

# Other discussion

## Reworking for goal 3

After significant discussion we agreed that it would be good to replace
the current approach of metadata-in-document with a separated metadata
YAML file. Essentially that file would be the input to Subplot, and it
would list, among other things, the set of markdown files which comprise
the scenario input.

Doing this would allow us to have a strict parsing of Subplot specific
metadata, but a more liberal parsing of optional Pandoc-specific metadata.
The `docgen` process would then include conversion and aggregation of this
metadata into the AST passed to Pandoc.

To this end, we decided that we should defer things currently labelled
as goal 3 until a putative goal 4, and we should define goal 3 as the
above work. We recorded some actions (see below) to this effect. We hope
that this will be the last major breaking change, but we cannot make that
guarantee.

One potentially beneficial side-effect of this work is that you will be
able to be confident of the action of running Pandoc against any individual
markdown file; though if you opted to include Pandoc specific metadata
in one of them, the result after `docgen` would be quite confusing because
we intend to use the `pulldown_cmark` crate for all Markdown parsing and it
has no provision for YAML metadata blocks. Part of this goal would be to
simplify our metadata handling so we would not want to try and support this
in-place metadata at all.

# Plan for next iteration

We opened [[!milestone 38]] to cover this iteration, with the
following issues:

- [[!issue 198]] -- _Unify language handling for multi-impl-lang test suites_
- [[!issue 229]] -- _Licensing of Subplot libraries and subplotlib needs to be clarified_
- [[!issue 242]] -- _`subplotlib` doesn't use `build.rs` script_
- [[!issue 188]] -- _Old Subplot libraries have bindings with regex that could now be simple patterns_
- [[!issue 180]] -- _Doesn't warn about unused embedded files_

# Other business

- We only just made a release, so no release planned for this iteration.

- We are not yet ready to file an RFP bug to get Subplot packaged for
  Debian. It will happen after we think we won't be making breaking
  changes anymore.

- We are not yet ready to make a whole code base review of Subplot.
  However we feel at least part of the new goal 3 will be reviewing a portion of the code.

- We will switch an issue based agenda when other people join the
  meeting.

- We relabelled our "reachout set for goal 2" to be for our new goal 3
  and we agreed to not reach out after this goal because our new goal 3
  is such a large break in API.

- Regarding a talk at FOSDEM, we have agreed that we want to give a
  talk, though we're not certain if it'll be at FOSDEM and we would
  want it to be suitable to give at other places as well. Given that
  we intend to set up a shared space to brainstorm the audience,
  and the intended knowledge that audience will gain from the talk.

- We agreed to continue with merge commits, and noted `git log --graph`
  as helpful, along with `git log --merges`. Once either someone
  queries this decision, or we have more project policy to document,
  we will write this down either in `CONTRIBUTING.md` or `DECISIONS.md`.

# AOB

- Regarding the `media` repository question - we believe this is
  premature at this point - once we are ready to write the talk
  mentioned above, it will be time for the repository.

- We are not going to do a release in this iteration anyway, so
  Lars suggests that whoever does the next release plays with
  `cargo release` since he has checked and its README suggests that
  it should support workspaces.

- We continue to believe that in part, helping high profile projects
  such as Sequoia-PGP or Rustup remains a big part of helping to
  convince people to use Subplot. Since we are about to make some
  more large breaking changes we do not want to convince more people
  into this camp yet. Writing the talk will be another way to
  increase visiblility of the project. More "real" examples may
  also be a good way to show the capabilities of Subplot.
  All of these things will be a lot of work, so we should be careful
  to not burn ourselves out on this.

# Actions

- Lars (and Daniel) to label at least the
  Sequoia-PGP blockers if not also anything else he thinks may be
  blocking projects such as Rustup.

- Daniel to shuffle goal3 to goal4 and create goal3 label for the
  purpose of tracking the work we discussed around redoing metadata
  as a separate file.

- Daniel to file a tracking issue for the metadata work, and assign
  it to Lars. Lars to then break down the work at some future point.

- Lars to set up a space for brainstorming the talk we want to give.

- Lars to file an issue with details of `cargo audit` and `cargo deny`
  and the intention to have a CI pipeline step which runs them.
  This may include filing an issue on the docker container to add the
  tooling.