Implement support for listing a generation in kdirstat.cache format

On Sat, 2014-12-06 at 19:15 +0200, Lars Wirzenius wrote:
> On Sun, Nov 16, 2014 at 11:24:11AM +0000, Ian Campbell wrote:
> > This implements an ls variant whose output format is compatible with the
> > kdirstat cache format. The best reference I can find for this is from an old fork:
> > https://github.com/thomas-joiner/k4dirstat/blob/master/kdirstat/cache-file-format.txt
> >
> > Recent versions of k4dirstat appear to be able to read the files produced just
> > fine, although this document is not present in their source AFAICT.
> >
> > I find k4dirstat to be an excellent way to visualise what is included in a
> > backup, e.g. for the purposes of excluding file types or marking directories
> > with a CACHEDIR.TAG. My previous solution involved post-processing the output
> > of ls which require a ssh to the machine with the files (for stat() purposes).
> > This approach pulls all of the data out of the obnam repo, so is much faster,
> > less hacky and gets the status of the actual generation and not the current
> > files etc.
>
> Thank you, Ian. I've applied the patch and pushed the changes to
> master on git.liw.fi.

Thanks.

> It would be nice if someone wrote a test case for this (in yarn form),
> so that I'll notice when I break it.

Your wish etc etc... (it even found a bug!)

8<----------------

>From 8f1aecfee8ec34e5d060f81ee47f9fd8aacdc201 Mon Sep 17 00:00:00 2001
From: Ian Campbell <ijc@hellion.org.uk>
Date: Sun, 7 Dec 2014 15:19:57 +0000
Subject: [PATCH] Add a yarn for testing kdirstat cachefile generation.

Rather simplistic, but caught one bug (unwanted newline at start of file) which
is also fixed here.

2 files changed, 47 insertions, 0 deletions

diff --git a/yarns/0300-kdirstat-integration.yarn b/yarns/0300-kdirstat-integration.yarn
new file mode 100644
index 00000000..a2ff8e69
--- /dev/null
+++ b/yarns/0300-kdirstat-integration.yarn
@@ -0,0 +1,25 @@
+kdirstat integration: producing kdirstat cache files
+====================================================
+
+Obnam implements an ls variant whose output format is compatible with
+the kdirstat cache format.
+
+Create a simple cache file
+--------------------------
+
+Here we create a simple backup and dump the output as a kdirstat cache
+file.
+
+    SCENARIO create and do a simple check of a kdirstat cache
+    GIVEN 10kB of new data in directory L
+    WHEN user U backs up directory L to repository R
+    AND user U creates a kdirstat cache file C of repository R
+    THEN first line of C matches [kdirstat 4.0 cache file]
+    AND for each file in directory L a line in the kdircache C matches it
+
+Validating the cache file
+-------------------------
+
+Sadly there seems to be no CLI usable program to read or validate the
+produced cache files. If there were we would do a more thorough test
+of the syntax here.
diff --git a/yarns/9000-implements.yarn b/yarns/9000-implements.yarn
index 1a86fbe7..de26a8b3 100644
--- a/yarns/9000-implements.yarn
+++ b/yarns/9000-implements.yarn
@@ -573,6 +573,11 @@ Does any line match?
     IMPLEMENTS THEN (\S+) matches (.*)$
     grep -E -e "$MATCH_2" -- "$DATADIR/$MATCH_1"
 
+Does first line match?
+
+    IMPLEMENTS THEN first line of (\S+) matches (.*)$
+    head -n1 "$DATADIR/$MATCH_1" | grep -E -e "$MATCH_2" --
+
 Do all lines match?
 
     IMPLEMENTS THEN all lines in (\S+) match (\S+)
@@ -628,3 +633,20 @@ Run the Nagios monitoring subcommand.
     IMPLEMENTS WHEN user (\S+) attempts nagios-last-backup-age against repository (\S+)
     attempt run_obnam "$MATCH_1" nagios-last-backup-age \
         -r "$DATADIR/$MATCH_2"
+
+kdirstat
+--------
+
+Create a kdirstat cache.
+
+    IMPLEMENTS WHEN user (\S+) creates a kdirstat cache file (\S+) of repository (\S+)
+    run_obnam "$MATCH_1" -r "$DATADIR/$MATCH_3" kdirstat > "$DATADIR/$MATCH_2"
+
+Check that the cache mentions each file in the repository. Since grep
+-E cannot specifically match a tab we check only for one character of
+whitespace.
+
+    IMPLEMENTS THEN for each file in directory (\S+) a line in the kdircache (\S+) matches it
+    find "$DATADIR/$MATCH_1" -type f | while read f ; do \
+        grep -E -e "^F[[:space:]]$f" "$DATADIR/$MATCH_2" || exit 1; \
+    done