diff options
author | Lars Wirzenius <liw@liw.fi> | 2013-12-04 09:39:49 +0000 |
---|---|---|
committer | Lars Wirzenius <liw@liw.fi> | 2013-12-04 09:39:49 +0000 |
commit | eda192f41a28ec1b28bae9fb1b8de97f93fb48ab (patch) | |
tree | 41d61e38cd2a065e66a19000102a411de08586cc /yarns/0010-introduction.yarn | |
parent | 4482cac447db4f4539d5ef8df806001f1e9d4d56 (diff) | |
download | obnam-eda192f41a28ec1b28bae9fb1b8de97f93fb48ab.tar.gz |
Add leading zero to filename
This allows more chapters to added later.
Diffstat (limited to 'yarns/0010-introduction.yarn')
-rw-r--r-- | yarns/0010-introduction.yarn | 109 |
1 files changed, 109 insertions, 0 deletions
diff --git a/yarns/0010-introduction.yarn b/yarns/0010-introduction.yarn new file mode 100644 index 00000000..0947f1a0 --- /dev/null +++ b/yarns/0010-introduction.yarn @@ -0,0 +1,109 @@ +% Obnam integration test suite +% http://liw.fi/obnam + +Introduction +============ + +This is the Obnam integration test suite. Obnam is a backup program. +The test suite is implemented using yarn, which is a black box testing +tool for Unix programs, inspired by the BDD tools used by the Ruby +community (Cucumber, Gherkin). + +Obnam has extensive unit tests, which ensure individual functions, +classes, and method work in isolation. The goal of the integration +test suite is to make sure all the pieces work together. Thus, a +typical integration test is to run Obnam in a specific kind of way, or +against a specific kind of data, and then verify that the data can be +restored correctly and that the repository is correct. + +With yarn, tests are written up as "scenarios", and each scenario may +consist of several steps. Each scenario, or some group of steps within +a scenario, may tests one aspect of Obnam, or one way to use Obnam, or +one error situation. + +This test suite is meant to be comprehensible to those who would use +Obnam, but aren't programmers, and would not understand the quite +low-level unit tests. Test scenarios written for yarn are written as a +document (this document), and each scenario consists of two parts: the +scenario itself, and the nitty-gritty implementation part. The +scenario is for everyone to understand, while the implementation part +is meant for programmers, and others who understand shell script. The +scenario describes, to any Obnam user, what is being tested, and at a +very high level how, without having to understand the implementation +part. + +For more information: + +* Obnam: <http://liw.fi/obnam/> +* Yarn: <http://liw.fi/cmdtest/> + +FIXME: Outline of test suite +============================ + +This chapter will be removed, later, when all the outlined parts have +been implemented. + +* Test environment and setup + - where is live data? how is live data generated? + - where is repo? + - how is repo accessed? local fs vs sftp + - how is obnam configured for tests? how can tests change the + configuration? + - encryption keys + - verification + - IMPLEMENTS library; scenario specific implments +* Basic operation: backup, restore, verify + - all types, sizes of files with all kinds of metadata + - single generation +* Multiple generations + - multiple generations + - forget, individual generations + - forget, automatically according to --keep + - generations + - genids + - ls + - diff +* Multiple clients +* De-duplication + - single client vs multiple clients + - across generations + - with encryption +* Compression + - generate interesting set of test data, backup once +* Encryption + - add-key + - client-keys + - list-keys + - list-toplevels + - remove-key +* Lock handling + - force-lock +* FUSE + - mount + - unmount +* Repository management + - fsck + - remove client + - list clients +* System administration + - nagios-last-backup-age +* Test implementation + - shell library + - generic implements for all tests + +Open questions: + +* how to test compression and encryption? should I re-run all tests + plain, compressed, encrypted, and compressed+encrypted? also, + network tests? do I need some kind of include mechanism for this? + - I can probably manage just fine with a simpler set of sets for + compression, encryption and networking, no need to go over all + the cases that plain backups go through: if things work for + plain, they probably work for the other cases, if those other + cases work at all + - then add regression tests as need be +* test accessing live data over sftp +* what errors should I test? can I test? +* multiple clients? +* different filesystems? run test suite multiple times, and set TMPDIR + to point at a particular filesystem each time |