From b9de4d5e810d07b679ad9ab9e46d58e4c76213cc Mon Sep 17 00:00:00 2001 From: Daniel Silverstone Date: Sat, 1 Jan 2022 18:09:35 +0000 Subject: tests: Make runcmd.md a common test set now Signed-off-by: Daniel Silverstone --- subplotlib/runcmd.md | 187 --------------------------------------------------- 1 file changed, 187 deletions(-) delete mode 100644 subplotlib/runcmd.md (limited to 'subplotlib') diff --git a/subplotlib/runcmd.md b/subplotlib/runcmd.md deleted file mode 100644 index f6b9d85..0000000 --- a/subplotlib/runcmd.md +++ /dev/null @@ -1,187 +0,0 @@ ---- -title: Acceptance criteria for the runcmd step library for subplotlib. -author: The Subplot project -bindings: - - lib/runcmd.yaml - - lib/files.yaml -impls: - rust: [] -... - -# Introduction - -The [Subplot][] step library `runcmd` for Rust provides scenario steps -and their implementations for running Unix commands and examining the -results. The library consists of a bindings file `steplibrary/runcmd.yaml` and -implementations inside the `subplotlib` crate itself. - -[subplot]: https://subplot.liw.fi/ - -This document explains the acceptance criteria for the library and how -they're verified. It uses the steps and functions from the -`steplibrary/runcmd` library. The scenarios all have the same structure: run a -command, then examine the exit code, standard output (stdout for -short), or standard error output (stderr) of the command. - -The scenarios use the Unix commands `true` and `false` to -generate exit codes, and `echo` to produce stdout. To generate -stderr, they use the little helper script below. - -```{#err.sh .file .sh .numberLines} -#!/bin/sh -echo "$@" 1>&2 -``` - -# Check exit code - -These scenarios verify the exit code. To make it easier to write -scenarios in language that flows more naturally, there are a couple of -variations. - -## Successful execution - -```scenario -when I run true -then exit code is 0 -and command is successful -``` - -## Successful execution in a sub-directory - -```scenario -given a directory xyzzy -when I run, in xyzzy, pwd -then exit code is 0 -then command is successful -then stdout contains "/xyzzy" -``` - -## Failed execution - -```scenario -when I try to run false -then exit code is not 0 -and command fails -``` - -## Failed execution in a sub-directory - -```scenario -given a directory xyzzy -when I try to run, in xyzzy, false -then exit code is not 0 -and command fails -``` - -# Check output has what we want - -These scenarios verify that stdout or stderr do have something we want -to have. - -## Check stdout is exactly as wanted - -Note that the string is surrounded by double quotes to make it clear -to the reader what's inside. Also, C-style string escapes are -understood. - -```scenario -when I run echo hello, world -then stdout is exactly "hello, world\n" -``` - -## Check stderr is exactly as wanted - -```scenario -given helper script err.sh for runcmd -when I run sh err.sh hello, world -then stderr is exactly "hello, world\n" -``` - -## Check stdout using sub-string search - -Exact string comparisons are not always enough, so we can verify a -sub-string is in output. - -```scenario -when I run echo hello, world -then stdout contains "world\n" -and exit code is 0 -``` - -## Check stderr using sub-string search - -```scenario -given helper script err.sh for runcmd -when I run sh err.sh hello, world -then stderr contains "world\n" -``` - -## Check stdout using regular expressions - -Fixed strings are not always enough, so we can verify output matches a -regular expression. Note that the regular expression is not delimited -and does not get any C-style string escaped decoded. - -```scenario -when I run echo hello, world -then stdout matches regex world$ -``` - -## Check stderr using regular expressions - -```scenario -given helper script err.sh for runcmd -when I run sh err.sh hello, world -then stderr matches regex world$ -``` - -# Check output doesn't have what we want to avoid - -These scenarios verify that the stdout or stderr do not -have something we want to avoid. - -## Check stdout is not exactly something - -```scenario -when I run echo hi -then stdout isn't exactly "hello, world\n" -``` - -## Check stderr is not exactly something - -```scenario -given helper script err.sh for runcmd -when I run sh err.sh hi -then stderr isn't exactly "hello, world\n" -``` - -## Check stdout doesn't contain sub-string - -```scenario -when I run echo hi -then stdout doesn't contain "world" -``` - -## Check stderr doesn't contain sub-string - -```scenario -given helper script err.sh for runcmd -when I run sh err.sh hi -then stderr doesn't contain "world" -``` - -## Check stdout doesn't match regular expression - -```scenario -when I run echo hi -then stdout doesn't match regex world$ - -``` - -## Check stderr doesn't match regular expressions - -```scenario -given helper script err.sh for runcmd -when I run sh err.sh hi -then stderr doesn't match regex world$ -``` -- cgit v1.2.1