[[!meta title="Iteration planning: October 23rd"]] [[!tag meeting]] [[!meta date="Sat, 23 Oct 2021 13:30:00 +0100"]] [[!toc levels=2]] # Review of actions from last meeting Actions from last time, and their status are: - Lars to create a label for blocking-high-profile-projects (name to be decided by Lars when he does it) and to label at least the Sequoia-PGP blockers if not also anything else he thinks may be blocking projects such as Rustup. **Half done, carrying** - Lars to record decision about MSRV in `DECISIONS.md` in the `subplot` repository. **DONE** - Lars and/or Daniel to add a Docker image for CI with Rust 1.48.0 installed, or add it to the main image. **DONE** - Lars and/or Daniel to change `.gitlab-ci.yml` to build and test using the 1.48.0 toolchain. **DONE** - Lars and Daniel to discuss whether to change `./check` to take an option to specify which toolchain to use or to keep that outside of the script. **DONE** (We selected external control) - Lars to refactor the changes for `build.rs` support, as discussed, and push an MR for it. **DONE** - Lars should file an issue about the `get_basedir_from` changes, if the problem can be reproduced with the original code. **No reproduction, no issue filed** - Lars to document decision that it's the user's responsibility to set up the environment in which tests are run. **DONE** # Review of the iteration that has ended [[!milestone 37]] had the following issued closed: - [[!issue 219]] -- _Needs a release after 0.3.0 to test release automation_ Some progress was made on: - [[!issue 198]] -- _Unify language handling for multi-impl-lang test suites_ The following issues remain open: - [[!issue 198]] -- _Unify language handling for multi-impl-lang test suites_ - [[!issue 207]] -- _Release process is too manual, CI build of release .deb changes files_ - [[!issue 218]] -- _Automatable parts of release process are manual_ - [[!issue 238]] -- _Doesn't support `abstract` in document metadata_ - [[!issue 239]] -- _In metadata, `author` is a single string_ We moved the non-release-process remaining issues to the next milestone ([[!milestone 38]]) and then closed off this iteration. We removed [[!issue 207]] and [[!issue 218]] from the future iterations, and removed [[!issue 207]] from the current goal. # Review of the repositories There are no MRs currently open in `subplot`, `subplot-web`, or `subplot-container-images`. The `subplot-container-images` and `subplot-web` repositories do not extra branches. The `subplot` repository has extra branches: - `subplot-rust` -- Daniel is keeping this around until he has completed the work on [[!issue 198]]. - `polyglot-docs` -- Daniel is working on [[!issue 198]] here. - `docgen-cmark` -- Lars is working on making docgen use cmark rather than pandoc for the initial parse and processing. We triggered the pipeline to build the container image for Subplot. All `subplot` pipelines are clean. # Current goal (goal 2; not changed for this iteration) Subplot provides a set of libraries with identical capabilities in each of the supported languages. Python remains a supported language. Rust is promoted to supported-language status. Subplot will be tested with all supported languages. In addition, any quality of life improvements which can be done shall be done. This goal will be considered complete when a release of Subplot has been made with the unified language handling support complete. # Issue review We did a full review of all non-`someday-maybe` issues. Changes we made include: - [[!issue 126]] -- _Create a CI cronjob which attempts to `cargo update` everything_ We opted to close this issue as we think the current release process is sufficient for now. - [[!issue 133]] -- _No feature table comparing scenario runners from different codegen templates_ We opted to mark this issue as blocked by [[!issue 198]] (multi-impl-lang docs) - [[!issue 141]] -- _subplotlib: Needs better test suite_ We opted to mark this as blocked by [[!issue 198]] and labelled it as `tracking` since we'll close it when the same subplot documents are testing runcmd, files, etc. for all language templates. - [[!issue 180]] -- _Doesn't warn about unused embedded files_ We agreed we should now be able to lint this as part of `codegen` so tentatively assigned it to Lars for this iteration. - [[!issue 188]] -- _Old Subplot libraries have bindings with regex that could now be simple patterns_ We tentatively assigned this to Lars for this iteration. - [[!issue 213]] -- \_"Failure: document has no title" when there's duplicate keys" We are going to defer this to the new goal-3 as per below discussion notes - [[!issue 214]] -- _Lacks examples and docs for using Rust step functions_ We marked this as blocked by [[!issue 198]] - [[!issue 220]] -- _Generated Rust test programs are hard to debug_ We marked this as blocked by [[!issue 198]] - [[!issue 221]] -- _Command line tool as insufficient logging_ We made a note on the issue to the effect of the next steps. We added this to the current goal, but not to the iteration. - [[!issue 222]] -- _Typeset output could indicate snippets are a continuation of the scenario and not a new scenario_ We added the `typesetting` label to this and also `someday-maybe`. - [[!issue 223]] -- _lib/runcmd lacks a generic "given an installed {name}" step_ We added a blocked-by on [[!issue 198]]. We still need further discussion on this. - [[!issue 224]] -- _Logging feature in the `subplot` binary needs tests_ We updated the title to better indicate that this is about testing the behaviour of the `SUBPLOT_LOG*` environment variables. We also labelled it for goal 3. - [[!issue 229]] -- _Licensing of Subplot libraries and subplotlib needs to be clarified_ We agreed on MIT, Daniel removed the `actively-discussed` label. Daniel will take on the task of reaching out (and subsequently changing the licence terms) and so we assigned it to Daniel for this iteration. - [[!issue 238]] -- _Doesn't support `abstract` in document metadata_ Unassigned from Lars for now. - [[!issue 239]] -- _In metadata, `author` is a single string_ Unassigned from Lars for now. - [[!issue 240]] -- _Could we mangle generated scenario function names in the Rust template?_ We decided that the suggestions made on-issue were sufficient and closed the issue - [[!issue 241]] and [[!issue 242]] -- Related to checking subplot, we made the two issues be linked. We assigned 242 to Lars for this iteration. We left the docker-for-subplot issue alone, and subplot-web had no issues. # Other discussion ## Reworking for goal 3 After significant discussion we agreed that it would be good to replace the current approach of metadata-in-document with a separated metadata YAML file. Essentially that file would be the input to Subplot, and it would list, among other things, the set of markdown files which comprise the scenario input. Doing this would allow us to have a strict parsing of Subplot specific metadata, but a more liberal parsing of optional Pandoc-specific metadata. The `docgen` process would then include conversion and aggregation of this metadata into the AST passed to Pandoc. To this end, we decided that we should defer things currently labelled as goal 3 until a putative goal 4, and we should define goal 3 as the above work. We recorded some actions (see below) to this effect. We hope that this will be the last major breaking change, but we cannot make that guarantee. One potentially beneficial side-effect of this work is that you will be able to be confident of the action of running Pandoc against any individual markdown file; though if you opted to include Pandoc specific metadata in one of them, the result after `docgen` would be quite confusing because we intend to use the `pulldown_cmark` crate for all Markdown parsing and it has no provision for YAML metadata blocks. Part of this goal would be to simplify our metadata handling so we would not want to try and support this in-place metadata at all. # Plan for next iteration We opened [[!milestone 38]] to cover this iteration, with the following issues: - [[!issue 198]] -- _Unify language handling for multi-impl-lang test suites_ - [[!issue 229]] -- _Licensing of Subplot libraries and subplotlib needs to be clarified_ - [[!issue 242]] -- _`subplotlib` doesn't use `build.rs` script_ - [[!issue 188]] -- _Old Subplot libraries have bindings with regex that could now be simple patterns_ - [[!issue 180]] -- _Doesn't warn about unused embedded files_ # Other business - We only just made a release, so no release planned for this iteration. - We are not yet ready to file an RFP bug to get Subplot packaged for Debian. It will happen after we think we won't be making breaking changes anymore. - We are not yet ready to make a whole code base review of Subplot. However we feel at least part of the new goal 3 will be reviewing a portion of the code. - We will switch an issue based agenda when other people join the meeting. - We relabelled our "reachout set for goal 2" to be for our new goal 3 and we agreed to not reach out after this goal because our new goal 3 is such a large break in API. - Regarding a talk at FOSDEM, we have agreed that we want to give a talk, though we're not certain if it'll be at FOSDEM and we would want it to be suitable to give at other places as well. Given that we intend to set up a shared space to brainstorm the audience, and the intended knowledge that audience will gain from the talk. - We agreed to continue with merge commits, and noted `git log --graph` as helpful, along with `git log --merges`. Once either someone queries this decision, or we have more project policy to document, we will write this down either in `CONTRIBUTING.md` or `DECISIONS.md`. # AOB - Regarding the `media` repository question - we believe this is premature at this point - once we are ready to write the talk mentioned above, it will be time for the repository. - We are not going to do a release in this iteration anyway, so Lars suggests that whoever does the next release plays with `cargo release` since he has checked and its README suggests that it should support workspaces. - We continue to believe that in part, helping high profile projects such as Sequoia-PGP or Rustup remains a big part of helping to convince people to use Subplot. Since we are about to make some more large breaking changes we do not want to convince more people into this camp yet. Writing the talk will be another way to increase visiblility of the project. More "real" examples may also be a good way to show the capabilities of Subplot. All of these things will be a lot of work, so we should be careful to not burn ourselves out on this. # Actions - Lars (and Daniel) to label at least the Sequoia-PGP blockers if not also anything else he thinks may be blocking projects such as Rustup. - Daniel to shuffle goal3 to goal4 and create goal3 label for the purpose of tracking the work we discussed around redoing metadata as a separate file. - Daniel to file a tracking issue for the metadata work, and assign it to Lars. Lars to then break down the work at some future point. - Lars to set up a space for brainstorming the talk we want to give. - Lars to file an issue with details of `cargo audit` and `cargo deny` and the intention to have a CI pipeline step which runs them. This may include filing an issue on the docker container to add the tooling.