summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorLars Wirzenius <liw@liw.fi>2021-05-10 09:02:44 +0300
committerLars Wirzenius <liw@liw.fi>2021-05-10 09:02:44 +0300
commitd5ae9be30877427c5de55fc609af5b76d3a286b4 (patch)
treed886954062c9135e72dc195b5493106eb01d2b83
parent5486f3441ed45a86350c1b83b368170f53a76fc2 (diff)
downloadobnam2-d5ae9be30877427c5de55fc609af5b76d3a286b4.tar.gz
chore: use Subplot's built-in copies of its libraries
Drop the copies in subplot/vendor. Subplot will now use its built-in copies by default.
-rw-r--r--obnam.md10
-rw-r--r--subplot/vendored/daemon.md116
-rw-r--r--subplot/vendored/daemon.py262
-rw-r--r--subplot/vendored/daemon.yaml37
-rw-r--r--subplot/vendored/files.md105
-rw-r--r--subplot/vendored/files.py194
-rw-r--r--subplot/vendored/files.yaml83
-rw-r--r--subplot/vendored/runcmd.md194
-rw-r--r--subplot/vendored/runcmd.py252
-rw-r--r--subplot/vendored/runcmd.yaml83
10 files changed, 5 insertions, 1331 deletions
diff --git a/obnam.md b/obnam.md
index 36a46bf..b9ae57a 100644
--- a/obnam.md
+++ b/obnam.md
@@ -1704,16 +1704,16 @@ bindings:
- subplot/server.yaml
- subplot/client.yaml
- subplot/data.yaml
- - subplot/vendored/files.yaml
- - subplot/vendored/runcmd.yaml
+ - lib/files.yaml
+ - lib/runcmd.yaml
template: python
functions:
- subplot/server.py
- subplot/client.py
- subplot/data.py
- - subplot/vendored/daemon.py
- - subplot/vendored/files.py
- - subplot/vendored/runcmd.py
+ - lib/daemon.py
+ - lib/files.py
+ - lib/runcmd.py
classes:
- json
abstract: |
diff --git a/subplot/vendored/daemon.md b/subplot/vendored/daemon.md
deleted file mode 100644
index 9484926..0000000
--- a/subplot/vendored/daemon.md
+++ /dev/null
@@ -1,116 +0,0 @@
-# Introduction
-
-The [Subplot][] library `daemon` for Python provides scenario steps
-and their implementations for running a background process and
-terminating at the end of the scenario.
-
-[Subplot]: https://subplot.liw.fi/
-
-This document explains the acceptance criteria for the library and how
-they're verified. It uses the steps and functions from the
-`lib/daemon` library. The scenarios all have the same structure: run a
-command, then examine the exit code, verify the process is running.
-
-# Daemon is started and terminated
-
-This scenario starts a background process, verifies it's started, and
-verifies it's terminated after the scenario ends.
-
-~~~scenario
-given there is no "/bin/sleep 12765" process
-when I start "/bin/sleep 12765" as a background process as sleepyhead
-then a process "/bin/sleep 12765" is running
-when I stop background process sleepyhead
-then there is no "/bin/sleep 12765" process
-~~~
-
-
-# Daemon takes a while to open its port
-
-[netcat]: https://en.wikipedia.org/wiki/Netcat
-
-This scenario verifies that if the background process never starts
-listening on its port, the daemon library handles that correctly. We
-do this by using [netcat][] to start a dummy daemon, after a short
-delay. The lib/daemon code will wait for netcat to open its port, by
-connecting to the port. It then closes the port, which causes netcat
-to terminate.
-
-~~~scenario
-given a daemon helper shell script slow-start-daemon.sh
-given there is no "slow-start-daemon.sh" process
-when I try to start "./slow-start-daemon.sh" as slow-daemon, on port 8888
-when I stop background process slow-daemon
-then there is no "slow-start-daemon.sh" process
-~~~
-
-~~~{#slow-start-daemon.sh .file .sh .numberLines}
-#!/bin/bash
-
-set -euo pipefail
-
-sleep 2
-netcat -l 8888 > /dev/null
-echo OK
-~~~
-
-# Daemon never opens the intended port
-
-This scenario verifies that if the background process never starts
-listening on its port, the daemon library handles that correctly.
-
-~~~scenario
-given there is no "/bin/sleep 12765" process
-when I try to start "/bin/sleep 12765" as sleepyhead, on port 8888
-then starting daemon fails with "ConnectionRefusedError"
-then a process "/bin/sleep 12765" is running
-when I stop background process sleepyhead
-then there is no "/bin/sleep 12765" process
-~~~
-
-
-# Daemon stdout and stderr are retrievable
-
-Sometimes it's useful for the step functions to be able to retrieve
-the stdout or stderr of of the daemon, after it's started, or even
-after it's terminated. This scenario verifies that `lib/daemon` can do
-that.
-
-~~~scenario
-given a daemon helper shell script chatty-daemon.sh
-given there is no "chatty-daemon" process
-when I start "./chatty-daemon.sh" as a background process as chatty-daemon
-when daemon chatty-daemon has produced output
-when I stop background process chatty-daemon
-then there is no "chatty-daemon" process
-then daemon chatty-daemon stdout is "hi there\n"
-then daemon chatty-daemon stderr is "hola\n"
-~~~
-
-We make for the daemon to exit, to work around a race condition: if
-the test program retrieves the daemon's output too fast, it may not
-have had time to produce it yet.
-
-
-~~~{#chatty-daemon.sh .file .sh .numberLines}
-#!/bin/bash
-
-set -euo pipefail
-
-trap 'exit 0' TERM
-
-echo hola 1>&2
-echo hi there
-~~~
-
-
----
-title: Acceptance criteria for the lib/daemon Subplot library
-author: The Subplot project
-bindings:
-- daemon.yaml
-template: python
-functions:
-- daemon.py
-- runcmd.py
-...
diff --git a/subplot/vendored/daemon.py b/subplot/vendored/daemon.py
deleted file mode 100644
index 11f65bf..0000000
--- a/subplot/vendored/daemon.py
+++ /dev/null
@@ -1,262 +0,0 @@
-import logging
-import os
-import signal
-import socket
-import subprocess
-import time
-
-
-# A helper function for testing lib/daemon itself.
-def _daemon_shell_script(ctx, filename=None):
- get_file = globals()["get_file"]
- data = get_file(filename)
- with open(filename, "wb") as f:
- f.write(data)
- os.chmod(filename, 0o755)
-
-
-# Start a daemon that will open a port on localhost.
-def daemon_start_on_port(ctx, path=None, args=None, name=None, port=None):
- _daemon_start(ctx, path=path, args=args, name=name)
- daemon_wait_for_port("localhost", port)
-
-
-# Start a daemon after a little wait. This is used only for testing the
-# port-waiting code.
-def _daemon_start_soonish(ctx, path=None, args=None, name=None, port=None):
- _daemon_start(ctx, path=os.path.abspath(path), args=args, name=name)
- daemon = ctx.declare("_daemon")
-
- # Store the PID of the process we just started so that _daemon_stop_soonish
- # can kill it during the cleanup phase. This works around the Subplot
- # Python template not giving the step captures to cleanup functions. Note
- # that this code assume at most one _soonish function is called.
- daemon["_soonish"] = daemon[name]["pid"]
-
- try:
- daemon_wait_for_port("localhost", port)
- except Exception as e:
- daemon["_start_error"] = repr(e)
-
- logging.info("pgrep: %r", _daemon_pgrep(path))
-
-
-def _daemon_stop_soonish(ctx, path=None, args=None, name=None, port=None):
- ns = ctx.declare("_daemon")
- pid = ns["_soonish"]
- logging.debug(f"Stopping soonishly-started daemon, {pid}")
- signo = signal.SIGKILL
- try:
- os.kill(pid, signo)
- except ProcessLookupError:
- logging.warning("Process did not actually exist (anymore?)")
-
-
-# Start a daeamon, get its PID. Don't wait for a port or anything. This is
-# meant for background processes that don't have port. Useful for testing the
-# lib/daemon library of Subplot, but not much else.
-def _daemon_start(ctx, path=None, args=None, name=None):
- runcmd_run = globals()["runcmd_run"]
- runcmd_exit_code_is = globals()["runcmd_exit_code_is"]
- runcmd_get_exit_code = globals()["runcmd_get_exit_code"]
- runcmd_get_stderr = globals()["runcmd_get_stderr"]
- runcmd_prepend_to_path = globals()["runcmd_prepend_to_path"]
-
- path = os.path.abspath(path)
- argv = [path] + args.split()
-
- logging.debug(f"Starting daemon {name}")
- logging.debug(f" ctx={ctx.as_dict()}")
- logging.debug(f" name={name}")
- logging.debug(f" path={path}")
- logging.debug(f" args={args}")
- logging.debug(f" argv={argv}")
-
- ns = ctx.declare("_daemon")
-
- this = ns[name] = {
- "pid-file": f"{name}.pid",
- "stderr": f"{name}.stderr",
- "stdout": f"{name}.stdout",
- }
-
- # Debian installs `daemonize` to /usr/sbin, which isn't part of the minimal
- # environment that Subplot sets up. So we add /usr/sbin to the PATH.
- runcmd_prepend_to_path(ctx, "/usr/sbin")
- runcmd_run(
- ctx,
- [
- "daemonize",
- "-c",
- os.getcwd(),
- "-p",
- this["pid-file"],
- "-e",
- this["stderr"],
- "-o",
- this["stdout"],
- ]
- + argv,
- )
-
- # Check that daemonize has exited OK. If it hasn't, it didn't start the
- # background process at all. If so, log the stderr in case there was
- # something useful there for debugging.
- exit = runcmd_get_exit_code(ctx)
- if exit != 0:
- stderr = runcmd_get_stderr(ctx)
- logging.error(f"daemon {name} stderr: {stderr}")
- runcmd_exit_code_is(ctx, 0)
-
- # Get the pid of the background process, from the pid file created by
- # daemonize. We don't need to wait for it, since we know daemonize already
- # exited. If it isn't there now, it's won't appear later.
- if not os.path.exists(this["pid-file"]):
- raise Exception("daemonize didn't create a PID file")
-
- this["pid"] = _daemon_wait_for_pid(this["pid-file"], 10.0)
-
- logging.debug(f"Started daemon {name}")
- logging.debug(f" pid={this['pid']}")
- logging.debug(f" ctx={ctx.as_dict()}")
-
-
-def _daemon_wait_for_pid(filename, timeout):
- start = time.time()
- while time.time() < start + timeout:
- with open(filename) as f:
- data = f.read().strip()
- if data:
- return int(data)
- raise Exception("daemonize created a PID file without a PID")
-
-
-def daemon_wait_for_port(host, port, timeout=5.0):
- addr = (host, port)
- until = time.time() + timeout
- while True:
- try:
- s = socket.create_connection(addr, timeout=timeout)
- s.close()
- return
- except socket.timeout:
- logging.error(
- f"daemon did not respond at port {port} within {timeout} seconds"
- )
- raise
- except socket.error as e:
- logging.info(f"could not connect to daemon at {port}: {e}")
- pass
- if time.time() >= until:
- logging.error(
- f"could not connect to daemon at {port} within {timeout} seconds"
- )
- raise ConnectionRefusedError()
- # Sleep a bit to avoid consuming too much CPU while busy-waiting.
- time.sleep(0.1)
-
-
-# Stop a daemon.
-def daemon_stop(ctx, path=None, args=None, name=None):
- logging.debug(f"Stopping daemon {name}")
-
- ns = ctx.declare("_daemon")
- logging.debug(f" ns={ns}")
- pid = ns[name]["pid"]
- signo = signal.SIGTERM
-
- this = ns[name]
- data = open(this["stdout"]).read()
- logging.debug(f"{name} stdout, before: {data!r}")
- data = open(this["stderr"]).read()
- logging.debug(f"{name} stderr, before: {data!r}")
-
- logging.debug(f"Terminating process {pid} with signal {signo}")
- try:
- os.kill(pid, signo)
- except ProcessLookupError:
- logging.warning("Process did not actually exist (anymore?)")
-
- while True:
- try:
- os.kill(pid, 0)
- logging.debug(f"Daemon {name}, pid {pid} still exists")
- time.sleep(1)
- except ProcessLookupError:
- break
- logging.debug(f"Daemon {name} is gone")
-
- data = open(this["stdout"]).read()
- logging.debug(f"{name} stdout, after: {data!r}")
- data = open(this["stderr"]).read()
- logging.debug(f"{name} stderr, after: {data!r}")
-
-
-def daemon_no_such_process(ctx, args=None):
- assert not _daemon_pgrep(args)
-
-
-def daemon_process_exists(ctx, args=None):
- assert _daemon_pgrep(args)
-
-
-def _daemon_pgrep(pattern):
- logging.info(f"checking if process exists: pattern={pattern}")
- exit = subprocess.call(
- ["pgrep", "-laf", pattern], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL
- )
- logging.info(f"exit code: {exit}")
- return exit == 0
-
-
-def daemon_start_fails_with(ctx, message=None):
- daemon = ctx.declare("_daemon")
- error = daemon["_start_error"]
- logging.debug(f"daemon_start_fails_with: error={error!r}")
- logging.debug(f"daemon_start_fails_with: message={message!r}")
- assert message.lower() in error.lower()
-
-
-def daemon_get_stdout(ctx, name):
- return _daemon_get_output(ctx, name, "stdout")
-
-
-def daemon_get_stderr(ctx, name):
- return _daemon_get_output(ctx, name, "stderr")
-
-
-def _daemon_get_output(ctx, name, which):
- ns = ctx.declare("_daemon")
- this = ns[name]
- filename = this[which]
- data = open(filename).read()
- logging.debug(f"Read {which} of daemon {name} from {filename}: {data!r}")
- return data
-
-
-def daemon_has_produced_output(ctx, name=None):
- started = time.time()
- timeout = 5.0
- while time.time() < started + timeout:
- stdout = daemon_get_stdout(ctx, name)
- stderr = daemon_get_stderr(ctx, name)
- if stdout and stderr:
- break
- time.sleep(0.1)
-
-
-def daemon_stdout_is(ctx, name=None, text=None):
- daemon_get_stdout = globals()["daemon_get_stdout"]
- _daemon_output_is(ctx, name, text, daemon_get_stdout)
-
-
-def daemon_stderr_is(ctx, name=None, text=None):
- daemon_get_stderr = globals()["daemon_get_stderr"]
- _daemon_output_is(ctx, name, text, daemon_get_stderr)
-
-
-def _daemon_output_is(ctx, name, text, getter):
- assert_eq = globals()["assert_eq"]
- text = bytes(text, "UTF-8").decode("unicode_escape")
- output = getter(ctx, name)
- assert_eq(text, output)
diff --git a/subplot/vendored/daemon.yaml b/subplot/vendored/daemon.yaml
deleted file mode 100644
index 4fab1f6..0000000
--- a/subplot/vendored/daemon.yaml
+++ /dev/null
@@ -1,37 +0,0 @@
-- given: there is no "{args:text}" process
- function: daemon_no_such_process
-
-- given: a daemon helper shell script {filename}
- function: _daemon_shell_script
-
-- when: I start "{path}{args:text}" as a background process as {name}, on port {port}
- function: daemon_start_on_port
-
-- when: I try to start "{path}{args:text}" as {name}, on port {port}
- function: _daemon_start_soonish
- cleanup: _daemon_stop_soonish
-
-- when: I start "{path}{args:text}" as a background process as {name}
- function: _daemon_start
-
-- when: I stop background process {name}
- function: daemon_stop
-
-- when: daemon {name} has produced output
- function: daemon_has_produced_output
-
-- then: a process "{args:text}" is running
- function: daemon_process_exists
-
-- then: there is no "{args:text}" process
- function: daemon_no_such_process
-
-- then: starting daemon fails with "{message:text}"
- function: daemon_start_fails_with
-
-- then: daemon {name} stdout is "{text:text}"
- function: daemon_stdout_is
-
-- then: daemon {name} stderr is "{text:text}"
- function: daemon_stderr_is
-
diff --git a/subplot/vendored/files.md b/subplot/vendored/files.md
deleted file mode 100644
index 823c760..0000000
--- a/subplot/vendored/files.md
+++ /dev/null
@@ -1,105 +0,0 @@
-# Introduction
-
-The [Subplot][] library `files` provides scenario steps and their
-implementations for managing files on the file system during tests.
-The library consists of a bindings file `lib/files.yaml` and
-implementations in Python in `lib/files.py`.
-
-[Subplot]: https://subplot.liw.fi/
-
-This document explains the acceptance criteria for the library and how
-they're verified. It uses the steps and functions from the `files`
-library.
-
-# Create on-disk files from embedded files
-
-Subplot allows the source document to embed test files, and the
-`files` library provides steps to create real, on-disk files from
-the embedded files.
-
-~~~scenario
-given file hello.txt
-then file hello.txt exists
-and file hello.txt contains "hello, world"
-and file other.txt does not exist
-given file other.txt from hello.txt
-then file other.txt exists
-and files hello.txt and other.txt match
-and only files hello.txt, other.txt exist
-~~~
-
-~~~{#hello.txt .file .numberLines}
-hello, world
-~~~
-
-
-# File metadata
-
-These steps create files and manage their metadata.
-
-~~~scenario
-given file hello.txt
-when I remember metadata for file hello.txt
-then file hello.txt has same metadata as before
-
-when I write "yo" to file hello.txt
-then file hello.txt has different metadata from before
-~~~
-
-# File modification time
-
-These steps manipulate and test file modification times.
-
-~~~scenario
-given file foo.dat has modification time 1970-01-02 03:04:05
-then file foo.dat has a very old modification time
-
-when I touch file foo.dat
-then file foo.dat has a very recent modification time
-~~~
-
-
-# File contents
-
-These steps verify contents of files.
-
-~~~scenario
-given file hello.txt
-then file hello.txt contains "hello, world"
-and file hello.txt matches regex "hello, .*"
-and file hello.txt matches regex /hello, .*/
-~~~
-
-# Directories
-
-There are also a large number of directory based steps and some directory
-based behaviour available in creating files which are available in the files
-library.
-
-```scenario
-given a directory first
-then directory first exists
-and directory first is empty
-and directory second does not exist
-when I remove directory first
-then directory first does not exist
-when I create directory second
-then directory second exists
-and directory second is empty
-given file second/third/hello.txt from hello.txt
-then directory second is not empty
-and directory second/third exists
-and directory second/third is not empty
-when I remove directory second
-then directory second does not exist
-```
-
----
-title: Acceptance criteria for the files Subplot library
-author: The Subplot project
-template: python
-bindings:
-- files.yaml
-functions:
-- files.py
-...
diff --git a/subplot/vendored/files.py b/subplot/vendored/files.py
deleted file mode 100644
index dd5b9f8..0000000
--- a/subplot/vendored/files.py
+++ /dev/null
@@ -1,194 +0,0 @@
-import logging
-import os
-import re
-import shutil
-import time
-
-
-def files_create_from_embedded(ctx, filename=None):
- files_make_directory(ctx, path=os.path.dirname(filename) or ".")
- files_create_from_embedded_with_other_name(
- ctx, filename_on_disk=filename, embedded_filename=filename
- )
-
-
-def files_create_from_embedded_with_other_name(
- ctx, filename_on_disk=None, embedded_filename=None
-):
- get_file = globals()["get_file"]
-
- files_make_directory(ctx, path=os.path.dirname(filename_on_disk) or ".")
- with open(filename_on_disk, "wb") as f:
- f.write(get_file(embedded_filename))
-
-
-def files_create_from_text(ctx, filename=None, text=None):
- files_make_directory(ctx, path=os.path.dirname(filename) or ".")
- with open(filename, "w") as f:
- f.write(text)
-
-
-def files_make_directory(ctx, path=None):
- path = "./" + path
- if not os.path.exists(path):
- os.makedirs(path)
-
-
-def files_remove_directory(ctx, path=None):
- path = "./" + path
- shutil.rmtree(path)
-
-
-def files_file_exists(ctx, filename=None):
- assert_eq = globals()["assert_eq"]
- assert_eq(os.path.exists(filename), True)
-
-
-def files_file_does_not_exist(ctx, filename=None):
- assert_eq = globals()["assert_eq"]
- assert_eq(os.path.exists(filename), False)
-
-
-def files_directory_exists(ctx, path=None):
- assert_eq = globals()["assert_eq"]
- assert_eq(os.path.isdir(path), True)
-
-
-def files_directory_does_not_exist(ctx, path=None):
- assert_eq = globals()["assert_eq"]
- assert_eq(os.path.isdir(path), False)
-
-
-def files_directory_is_empty(ctx, path=None):
- assert_eq = globals()["assert_eq"]
- assert_eq(os.listdir(path), [])
-
-
-def files_directory_is_not_empty(ctx, path=None):
- assert_ne = globals()["assert_ne"]
- assert_ne(os.listdir(path), False)
-
-
-def files_only_these_exist(ctx, filenames=None):
- assert_eq = globals()["assert_eq"]
- filenames = filenames.replace(",", "").split()
- assert_eq(set(os.listdir(".")), set(filenames))
-
-
-def files_file_contains(ctx, filename=None, data=None):
- assert_eq = globals()["assert_eq"]
- with open(filename, "rb") as f:
- actual = f.read()
- actual = actual.decode("UTF-8")
- assert_eq(data in actual, True)
-
-
-def files_file_matches_regex(ctx, filename=None, regex=None):
- assert_eq = globals()["assert_eq"]
- with open(filename) as f:
- content = f.read()
- m = re.search(regex, content)
- if m is None:
- logging.debug(f"files_file_matches_regex: no match")
- logging.debug(f" filenamed: {filename}")
- logging.debug(f" regex: {regex}")
- logging.debug(f" content: {regex}")
- logging.debug(f" match: {m}")
- assert_eq(bool(m), True)
-
-
-def files_match(ctx, filename1=None, filename2=None):
- assert_eq = globals()["assert_eq"]
- with open(filename1, "rb") as f:
- data1 = f.read()
- with open(filename2, "rb") as f:
- data2 = f.read()
- assert_eq(data1, data2)
-
-
-def files_touch_with_timestamp(
- ctx,
- filename=None,
- year=None,
- month=None,
- day=None,
- hour=None,
- minute=None,
- second=None,
-):
- t = (
- int(year),
- int(month),
- int(day),
- int(hour),
- int(minute),
- int(second),
- -1,
- -1,
- -1,
- )
- ts = time.mktime(t)
- _files_touch(filename, ts)
-
-
-def files_touch(ctx, filename=None):
- _files_touch(filename, None)
-
-
-def _files_touch(filename, ts):
- if not os.path.exists(filename):
- open(filename, "w").close()
- times = None
- if ts is not None:
- times = (ts, ts)
- os.utime(filename, times=times)
-
-
-def files_mtime_is_recent(ctx, filename=None):
- st = os.stat(filename)
- age = abs(st.st_mtime - time.time())
- assert age < 1.0
-
-
-def files_mtime_is_ancient(ctx, filename=None):
- st = os.stat(filename)
- age = abs(st.st_mtime - time.time())
- year = 365 * 24 * 60 * 60
- required = 39 * year
- logging.debug(f"ancient? mtime={st.st_mtime} age={age} required={required}")
- assert age > required
-
-
-def files_remember_metadata(ctx, filename=None):
- meta = _files_remembered(ctx)
- meta[filename] = _files_get_metadata(filename)
- logging.debug("files_remember_metadata:")
- logging.debug(f" meta: {meta}")
- logging.debug(f" ctx: {ctx}")
-
-
-# Check that current metadata of a file is as stored in the context.
-def files_has_remembered_metadata(ctx, filename=None):
- assert_eq = globals()["assert_eq"]
- meta = _files_remembered(ctx)
- logging.debug("files_has_remembered_metadata:")
- logging.debug(f" meta: {meta}")
- logging.debug(f" ctx: {ctx}")
- assert_eq(meta[filename], _files_get_metadata(filename))
-
-
-def files_has_different_metadata(ctx, filename=None):
- assert_ne = globals()["assert_ne"]
- meta = _files_remembered(ctx)
- assert_ne(meta[filename], _files_get_metadata(filename))
-
-
-def _files_remembered(ctx):
- ns = ctx.declare("_files")
- return ns.get("remembered-metadata", {})
-
-
-def _files_get_metadata(filename):
- st = os.lstat(filename)
- keys = ["st_dev", "st_gid", "st_ino", "st_mode", "st_mtime", "st_size", "st_uid"]
- return {key: getattr(st, key) for key in keys}
diff --git a/subplot/vendored/files.yaml b/subplot/vendored/files.yaml
deleted file mode 100644
index f18b8cd..0000000
--- a/subplot/vendored/files.yaml
+++ /dev/null
@@ -1,83 +0,0 @@
-- given: file {filename}
- function: files_create_from_embedded
- types:
- filename: file
-
-- given: file {filename_on_disk} from {embedded_filename}
- function: files_create_from_embedded_with_other_name
- types:
- embedded_filename: file
-
-- given: file {filename} has modification time {year}-{month}-{day} {hour}:{minute}:{second}
- function: files_touch_with_timestamp
-
-- when: I write "(?P<text>.*)" to file (?P<filename>\S+)
- regex: true
- function: files_create_from_text
-
-- when: I remember metadata for file {filename}
- function: files_remember_metadata
-
-- when: I touch file {filename}
- function: files_touch
-
-- then: file {filename} exists
- function: files_file_exists
-
-- then: file {filename} does not exist
- function: files_file_does_not_exist
-
-- then: only files (?P<filenames>.+) exist
- function: files_only_these_exist
- regex: true
-
-- then: file (?P<filename>\S+) contains "(?P<data>.*)"
- regex: true
- function: files_file_contains
-
-- then: file (?P<filename>\S+) matches regex /(?P<regex>.*)/
- regex: true
- function: files_file_matches_regex
-
-- then: file (?P<filename>\S+) matches regex "(?P<regex>.*)"
- regex: true
- function: files_file_matches_regex
-
-- then: files {filename1} and {filename2} match
- function: files_match
-
-- then: file {filename} has same metadata as before
- function: files_has_remembered_metadata
-
-- then: file {filename} has different metadata from before
- function: files_has_different_metadata
-
-- then: file {filename} has changed from before
- function: files_has_different_metadata
-
-- then: file {filename} has a very recent modification time
- function: files_mtime_is_recent
-
-- then: file {filename} has a very old modification time
- function: files_mtime_is_ancient
-
-- given: a directory {path}
- function: files_make_directory
-
-- when: I create directory {path}
- function: files_make_directory
-
-- when: I remove directory {path}
- function: files_remove_directory
-
-- then: directory {path} exists
- function: files_directory_exists
-
-- then: directory {path} does not exist
- function: files_directory_does_not_exist
-
-- then: directory {path} is empty
- function: files_directory_is_empty
-
-- then: directory {path} is not empty
- function: files_directory_is_not_empty
diff --git a/subplot/vendored/runcmd.md b/subplot/vendored/runcmd.md
deleted file mode 100644
index 4615f69..0000000
--- a/subplot/vendored/runcmd.md
+++ /dev/null
@@ -1,194 +0,0 @@
-# Introduction
-
-The [Subplot][] library `runcmd` for Python provides scenario steps
-and their implementations for running Unix commands and examining the
-results. The library consists of a bindings file `lib/runcmd.yaml` and
-implementations in Python in `lib/runcmd.py`. There is no Bash
-version.
-
-[Subplot]: https://subplot.liw.fi/
-
-This document explains the acceptance criteria for the library and how
-they're verified. It uses the steps and functions from the
-`lib/runcmd` library. The scenarios all have the same structure: run a
-command, then examine the exit code, standard output (stdout for
-short), or standard error output (stderr) of the command.
-
-The scenarios use the Unix commands `/bin/true` and `/bin/false` to
-generate exit codes, and `/bin/echo` to produce stdout. To generate
-stderr, they use the little helper script below.
-
-~~~{#err.sh .file .sh .numberLines}
-#!/bin/sh
-echo "$@" 1>&2
-~~~
-
-# Check exit code
-
-These scenarios verify the exit code. To make it easier to write
-scenarios in language that flows more naturally, there are a couple of
-variations.
-
-## Successful execution
-
-~~~scenario
-when I run /bin/true
-then exit code is 0
-and command is successful
-~~~
-
-## Failed execution
-
-~~~scenario
-when I try to run /bin/false
-then exit code is not 0
-and command fails
-~~~
-
-# Check we can prepend to $PATH
-
-This scenario verifies that we can add a directory to the beginning of
-the PATH environment variable, so that we can have `runcmd` invoke a
-binary from our build tree rather than from system directories. This
-is especially useful for testing new versions of software that's
-already installed on the system.
-
-~~~scenario
-given executable script ls from ls.sh
-when I prepend . to PATH
-when I run ls
-then command is successful
-then stdout contains "custom ls, not system ls"
-~~~
-
-~~~{#ls.sh .file .sh .numberLines}
-#!/bin/sh
-echo "custom ls, not system ls"
-~~~
-
-# Check output has what we want
-
-These scenarios verify that stdout or stderr do have something we want
-to have.
-
-## Check stdout is exactly as wanted
-
-Note that the string is surrounded by double quotes to make it clear
-to the reader what's inside. Also, C-style string escapes are
-understood.
-
-~~~scenario
-when I run /bin/echo hello, world
-then stdout is exactly "hello, world\n"
-~~~
-
-## Check stderr is exactly as wanted
-
-~~~scenario
-given helper script err.sh for runcmd
-when I run sh err.sh hello, world
-then stderr is exactly "hello, world\n"
-~~~
-
-## Check stdout using sub-string search
-
-Exact string comparisons are not always enough, so we can verify a
-sub-string is in output.
-
-~~~scenario
-when I run /bin/echo hello, world
-then stdout contains "world\n"
-and exit code is 0
-~~~
-
-## Check stderr using sub-string search
-
-~~~scenario
-given helper script err.sh for runcmd
-when I run sh err.sh hello, world
-then stderr contains "world\n"
-~~~
-
-## Check stdout using regular expressions
-
-Fixed strings are not always enough, so we can verify output matches a
-regular expression. Note that the regular expression is not delimited
-and does not get any C-style string escaped decoded.
-
-~~~scenario
-when I run /bin/echo hello, world
-then stdout matches regex world$
-~~~
-
-## Check stderr using regular expressions
-
-~~~scenario
-given helper script err.sh for runcmd
-when I run sh err.sh hello, world
-then stderr matches regex world$
-~~~
-
-# Check output doesn't have what we want to avoid
-
-These scenarios verify that the stdout or stderr do not
-have something we want to avoid.
-
-## Check stdout is not exactly something
-
-~~~scenario
-when I run /bin/echo hi
-then stdout isn't exactly "hello, world\n"
-~~~
-
-## Check stderr is not exactly something
-
-~~~scenario
-given helper script err.sh for runcmd
-when I run sh err.sh hi
-then stderr isn't exactly "hello, world\n"
-~~~
-
-## Check stdout doesn't contain sub-string
-
-~~~scenario
-when I run /bin/echo hi
-then stdout doesn't contain "world"
-~~~
-
-## Check stderr doesn't contain sub-string
-
-~~~scenario
-given helper script err.sh for runcmd
-when I run sh err.sh hi
-then stderr doesn't contain "world"
-~~~
-
-## Check stdout doesn't match regular expression
-
-~~~scenario
-when I run /bin/echo hi
-then stdout doesn't match regex world$
-
-~~~
-
-## Check stderr doesn't match regular expressions
-
-~~~scenario
-given helper script err.sh for runcmd
-when I run sh err.sh hi
-then stderr doesn't match regex world$
-~~~
-
-
----
-title: Acceptance criteria for the lib/runcmd Subplot library
-author: The Subplot project
-template: python
-bindings:
-- runcmd.yaml
-- runcmd_test.yaml
-functions:
-- runcmd.py
-- runcmd_test.py
-- files.py
-...
diff --git a/subplot/vendored/runcmd.py b/subplot/vendored/runcmd.py
deleted file mode 100644
index a2564c6..0000000
--- a/subplot/vendored/runcmd.py
+++ /dev/null
@@ -1,252 +0,0 @@
-import logging
-import os
-import re
-import shlex
-import subprocess
-
-
-#
-# Helper functions.
-#
-
-# Get exit code or other stored data about the latest command run by
-# runcmd_run.
-
-
-def _runcmd_get(ctx, name):
- ns = ctx.declare("_runcmd")
- return ns[name]
-
-
-def runcmd_get_exit_code(ctx):
- return _runcmd_get(ctx, "exit")
-
-
-def runcmd_get_stdout(ctx):
- return _runcmd_get(ctx, "stdout")
-
-
-def runcmd_get_stdout_raw(ctx):
- return _runcmd_get(ctx, "stdout.raw")
-
-
-def runcmd_get_stderr(ctx):
- return _runcmd_get(ctx, "stderr")
-
-
-def runcmd_get_stderr_raw(ctx):
- return _runcmd_get(ctx, "stderr.raw")
-
-
-def runcmd_get_argv(ctx):
- return _runcmd_get(ctx, "argv")
-
-
-# Run a command, given an argv and other arguments for subprocess.Popen.
-#
-# This is meant to be a helper function, not bound directly to a step. The
-# stdout, stderr, and exit code are stored in the "_runcmd" namespace in the
-# ctx context.
-def runcmd_run(ctx, argv, **kwargs):
- ns = ctx.declare("_runcmd")
-
- # The Subplot Python template empties os.environ at startup, modulo a small
- # number of variables with carefully chosen values. Here, we don't need to
- # care about what those variables are, but we do need to not overwrite
- # them, so we just add anything in the env keyword argument, if any, to
- # os.environ.
- env = dict(os.environ)
- for key, arg in kwargs.pop("env", {}).items():
- env[key] = arg
-
- pp = ns.get("path-prefix")
- if pp:
- env["PATH"] = pp + ":" + env["PATH"]
-
- logging.debug(f"runcmd_run")
- logging.debug(f" argv: {argv}")
- logging.debug(f" env: {env}")
- p = subprocess.Popen(
- argv, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env, **kwargs
- )
- stdout, stderr = p.communicate("")
- ns["argv"] = argv
- ns["stdout.raw"] = stdout
- ns["stderr.raw"] = stderr
- ns["stdout"] = stdout.decode("utf-8")
- ns["stderr"] = stderr.decode("utf-8")
- ns["exit"] = p.returncode
- logging.debug(f" ctx: {ctx}")
- logging.debug(f" ns: {ns}")
-
-
-# Step: prepend srcdir to PATH whenever runcmd runs a command.
-def runcmd_helper_srcdir_path(ctx):
- srcdir = globals()["srcdir"]
- runcmd_prepend_to_path(ctx, srcdir)
-
-
-# Step: This creates a helper script.
-def runcmd_helper_script(ctx, filename=None):
- get_file = globals()["get_file"]
- with open(filename, "wb") as f:
- f.write(get_file(filename))
-
-
-#
-# Step functions for running commands.
-#
-
-
-def runcmd_prepend_to_path(ctx, dirname=None):
- ns = ctx.declare("_runcmd")
- pp = ns.get("path-prefix", "")
- if pp:
- pp = f"{pp}:{dirname}"
- else:
- pp = dirname
- ns["path-prefix"] = pp
-
-
-def runcmd_step(ctx, argv0=None, args=None):
- runcmd_try_to_run(ctx, argv0=argv0, args=args)
- runcmd_exit_code_is_zero(ctx)
-
-
-def runcmd_try_to_run(ctx, argv0=None, args=None):
- argv = [shlex.quote(argv0)] + shlex.split(args)
- runcmd_run(ctx, argv)
-
-
-#
-# Step functions for examining exit codes.
-#
-
-
-def runcmd_exit_code_is_zero(ctx):
- runcmd_exit_code_is(ctx, exit=0)
-
-
-def runcmd_exit_code_is(ctx, exit=None):
- assert_eq = globals()["assert_eq"]
- assert_eq(runcmd_get_exit_code(ctx), int(exit))
-
-
-def runcmd_exit_code_is_nonzero(ctx):
- runcmd_exit_code_is_not(ctx, exit=0)
-
-
-def runcmd_exit_code_is_not(ctx, exit=None):
- assert_ne = globals()["assert_ne"]
- assert_ne(runcmd_get_exit_code(ctx), int(exit))
-
-
-#
-# Step functions and helpers for examining output in various ways.
-#
-
-
-def runcmd_stdout_is(ctx, text=None):
- _runcmd_output_is(runcmd_get_stdout(ctx), text)
-
-
-def runcmd_stdout_isnt(ctx, text=None):
- _runcmd_output_isnt(runcmd_get_stdout(ctx), text)
-
-
-def runcmd_stderr_is(ctx, text=None):
- _runcmd_output_is(runcmd_get_stderr(ctx), text)
-
-
-def runcmd_stderr_isnt(ctx, text=None):
- _runcmd_output_isnt(runcmd_get_stderr(ctx), text)
-
-
-def _runcmd_output_is(actual, wanted):
- assert_eq = globals()["assert_eq"]
- wanted = bytes(wanted, "utf8").decode("unicode_escape")
- logging.debug("_runcmd_output_is:")
- logging.debug(f" actual: {actual!r}")
- logging.debug(f" wanted: {wanted!r}")
- assert_eq(actual, wanted)
-
-
-def _runcmd_output_isnt(actual, wanted):
- assert_ne = globals()["assert_ne"]
- wanted = bytes(wanted, "utf8").decode("unicode_escape")
- logging.debug("_runcmd_output_isnt:")
- logging.debug(f" actual: {actual!r}")
- logging.debug(f" wanted: {wanted!r}")
- assert_ne(actual, wanted)
-
-
-def runcmd_stdout_contains(ctx, text=None):
- _runcmd_output_contains(runcmd_get_stdout(ctx), text)
-
-
-def runcmd_stdout_doesnt_contain(ctx, text=None):
- _runcmd_output_doesnt_contain(runcmd_get_stdout(ctx), text)
-
-
-def runcmd_stderr_contains(ctx, text=None):
- _runcmd_output_contains(runcmd_get_stderr(ctx), text)
-
-
-def runcmd_stderr_doesnt_contain(ctx, text=None):
- _runcmd_output_doesnt_contain(runcmd_get_stderr(ctx), text)
-
-
-def _runcmd_output_contains(actual, wanted):
- assert_eq = globals()["assert_eq"]
- wanted = bytes(wanted, "utf8").decode("unicode_escape")
- logging.debug("_runcmd_output_contains:")
- logging.debug(f" actual: {actual!r}")
- logging.debug(f" wanted: {wanted!r}")
- assert_eq(wanted in actual, True)
-
-
-def _runcmd_output_doesnt_contain(actual, wanted):
- assert_ne = globals()["assert_ne"]
- wanted = bytes(wanted, "utf8").decode("unicode_escape")
- logging.debug("_runcmd_output_doesnt_contain:")
- logging.debug(f" actual: {actual!r}")
- logging.debug(f" wanted: {wanted!r}")
- assert_ne(wanted in actual, True)
-
-
-def runcmd_stdout_matches_regex(ctx, regex=None):
- _runcmd_output_matches_regex(runcmd_get_stdout(ctx), regex)
-
-
-def runcmd_stdout_doesnt_match_regex(ctx, regex=None):
- _runcmd_output_doesnt_match_regex(runcmd_get_stdout(ctx), regex)
-
-
-def runcmd_stderr_matches_regex(ctx, regex=None):
- _runcmd_output_matches_regex(runcmd_get_stderr(ctx), regex)
-
-
-def runcmd_stderr_doesnt_match_regex(ctx, regex=None):
- _runcmd_output_doesnt_match_regex(runcmd_get_stderr(ctx), regex)
-
-
-def _runcmd_output_matches_regex(actual, regex):
- assert_ne = globals()["assert_ne"]
- r = re.compile(regex)
- m = r.search(actual)
- logging.debug("_runcmd_output_matches_regex:")
- logging.debug(f" actual: {actual!r}")
- logging.debug(f" regex: {regex!r}")
- logging.debug(f" match: {m}")
- assert_ne(m, None)
-
-
-def _runcmd_output_doesnt_match_regex(actual, regex):
- assert_eq = globals()["assert_eq"]
- r = re.compile(regex)
- m = r.search(actual)
- logging.debug("_runcmd_output_doesnt_match_regex:")
- logging.debug(f" actual: {actual!r}")
- logging.debug(f" regex: {regex!r}")
- logging.debug(f" match: {m}")
- assert_eq(m, None)
diff --git a/subplot/vendored/runcmd.yaml b/subplot/vendored/runcmd.yaml
deleted file mode 100644
index 48dde90..0000000
--- a/subplot/vendored/runcmd.yaml
+++ /dev/null
@@ -1,83 +0,0 @@
-# Steps to run commands.
-
-- given: helper script {filename} for runcmd
- function: runcmd_helper_script
-
-- given: srcdir is in the PATH
- function: runcmd_helper_srcdir_path
-
-- when: I run (?P<argv0>\S+)(?P<args>.*)
- regex: true
- function: runcmd_step
-
-- when: I try to run (?P<argv0>\S+)(?P<args>.*)
- regex: true
- function: runcmd_try_to_run
-
-# Steps to examine exit code of latest command.
-
-- then: exit code is {exit}
- function: runcmd_exit_code_is
-
-- then: exit code is not {exit}
- function: runcmd_exit_code_is_not
-
-- then: command is successful
- function: runcmd_exit_code_is_zero
-
-- then: command fails
- function: runcmd_exit_code_is_nonzero
-
-# Steps to examine stdout/stderr for exact content.
-
-- then: stdout is exactly "(?P<text>.*)"
- regex: true
- function: runcmd_stdout_is
-
-- then: "stdout isn't exactly \"(?P<text>.*)\""
- regex: true
- function: runcmd_stdout_isnt
-
-- then: stderr is exactly "(?P<text>.*)"
- regex: true
- function: runcmd_stderr_is
-
-- then: "stderr isn't exactly \"(?P<text>.*)\""
- regex: true
- function: runcmd_stderr_isnt
-
-# Steps to examine stdout/stderr for sub-strings.
-
-- then: stdout contains "(?P<text>.*)"
- regex: true
- function: runcmd_stdout_contains
-
-- then: "stdout doesn't contain \"(?P<text>.*)\""
- regex: true
- function: runcmd_stdout_doesnt_contain
-
-- then: stderr contains "(?P<text>.*)"
- regex: true
- function: runcmd_stderr_contains
-
-- then: "stderr doesn't contain \"(?P<text>.*)\""
- regex: true
- function: runcmd_stderr_doesnt_contain
-
-# Steps to match stdout/stderr against regular expressions.
-
-- then: stdout matches regex (?P<regex>.*)
- regex: true
- function: runcmd_stdout_matches_regex
-
-- then: stdout doesn't match regex (?P<regex>.*)
- regex: true
- function: runcmd_stdout_doesnt_match_regex
-
-- then: stderr matches regex (?P<regex>.*)
- regex: true
- function: runcmd_stderr_matches_regex
-
-- then: stderr doesn't match regex (?P<regex>.*)
- regex: true
- function: runcmd_stderr_doesnt_match_regex