summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--tickets/2173d4e7d8a4497da071ca9d46be611b/Maildir/new/1499089770.M541712P12423Q1.koom500
1 files changed, 500 insertions, 0 deletions
diff --git a/tickets/2173d4e7d8a4497da071ca9d46be611b/Maildir/new/1499089770.M541712P12423Q1.koom b/tickets/2173d4e7d8a4497da071ca9d46be611b/Maildir/new/1499089770.M541712P12423Q1.koom
new file mode 100644
index 0000000..4b1249e
--- /dev/null
+++ b/tickets/2173d4e7d8a4497da071ca9d46be611b/Maildir/new/1499089770.M541712P12423Q1.koom
@@ -0,0 +1,500 @@
+Return-Path: <obnam-dev-bounces@obnam.org>
+X-Original-To: distix@pieni.net
+Delivered-To: distix@pieni.net
+Received: from yaffle.pepperfish.net (yaffle.pepperfish.net [88.99.213.221])
+ by pieni.net (Postfix) with ESMTPS id 6C2EF44F5F
+ for <distix@pieni.net>; Mon, 3 Jul 2017 13:42:57 +0000 (UTC)
+Received: from platypus.pepperfish.net (unknown [10.112.101.20])
+ by yaffle.pepperfish.net (Postfix) with ESMTP id E859441CD6;
+ Mon, 3 Jul 2017 14:42:56 +0100 (BST)
+Received: from ip6-localhost.nat ([::1] helo=platypus.pepperfish.net)
+ by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+ id 1dS1cu-0007pv-Ta; Mon, 03 Jul 2017 14:42:56 +0100
+Received: from [10.112.101.21] (helo=inmail2.pepperfish.net)
+ by platypus.pepperfish.net with esmtps (Exim 4.80 #2 (Debian))
+ id 1dS1cu-0007pj-Bk
+ for <obnam-dev@obnam.org>; Mon, 03 Jul 2017 14:42:56 +0100
+Received: from mta01.univpm.it ([193.205.128.17])
+ by inmail2.pepperfish.net with esmtps
+ (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89)
+ (envelope-from <m.alessandrini@univpm.it>) id 1dS1co-0007z5-Tz
+ for obnam-dev@obnam.org; Mon, 03 Jul 2017 14:42:56 +0100
+Received: from [193.205.130.201] (thor.dii.univpm.it [193.205.130.201])
+ by mta01.univpm.it (8.14.8/8.14.8) with ESMTP id v63DgfVx002630;
+ Mon, 3 Jul 2017 15:42:41 +0200
+To: Lars Wirzenius <liw@liw.fi>
+References: <0dc53aa0-b510-8607-ba63-f527f49bdf50@univpm.it>
+ <20170703053310.7y5qri5l6mduhuli@liw.fi>
+ <d62610ae-0749-ec34-eb6a-625ad2107e77@univpm.it>
+From: Michele Alessandrini <m.alessandrini@univpm.it>
+Organization: =?UTF-8?Q?DIBET_-_Universit=c3=a0_Politecnica_delle_Marche?=
+Message-ID: <6e754eb6-34a3-a1e6-9c10-dc4b053ee954@univpm.it>
+Date: Mon, 3 Jul 2017 15:42:26 +0200
+User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
+ Thunderbird/45.8.0
+MIME-Version: 1.0
+In-Reply-To: <d62610ae-0749-ec34-eb6a-625ad2107e77@univpm.it>
+Content-Type: multipart/mixed; boundary="------------D34B4AF69ADA0C644386FED6"
+X-Greylist: inspected by milter-greylist-4.5.12 (mta01.univpm.it
+ [193.205.128.17]);
+ Mon, 03 Jul 2017 15:42:41 +0200 (CEST) for IP:'193.205.130.201'
+ DOMAIN:'thor.dii.univpm.it' HELO:'[193.205.130.201]'
+ FROM:'m.alessandrini@univpm.it' RCPT:''
+X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.12
+ (mta01.univpm.it [193.205.128.17]); Mon, 03 Jul 2017 15:42:41 +0200 (CEST)
+X-Virus-Scanned: clamav-milter 0.98.7 at mta01.univpm.it
+X-Virus-Status: Clean
+X-Scanned-By: MIMEDefang 2.78 on 193.205.128.17
+X-Pepperfish-Transaction: ba3b-0bcc-1918-0d90
+X-Spam-Score: -4.9
+X-Spam-Score-int: -48
+X-Spam-Bar: ----
+X-Scanned-By: pepperfish.net, Mon, 03 Jul 2017 14:42:56 +0100
+X-Spam-Report: Content analysis details: (-4.9 points)
+ pts rule name description
+ ---- ---------------------- --------------------------------------------------
+ -0.5 PPF_USER_AGENT User-Agent: exists
+ -1.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
+ -1.5 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/, medium
+ trust [193.205.128.17 listed in list.dnswl.org]
+ -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
+ [score: 0.0000]
+X-ACL-Warn: message may be spam
+X-Scan-Signature: 5bb76d4b0387b43dc4e28efac2549394
+Cc: obnam-dev@obnam.org
+Subject: patch N.2 for "exclude-if-present" option
+X-BeenThere: obnam-dev@obnam.org
+X-Mailman-Version: 2.1.5
+Precedence: list
+List-Id: Obnam development discussions <obnam-dev-obnam.org>
+List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
+List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
+List-Post: <mailto:obnam-dev@obnam.org>
+List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
+List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=subscribe>
+Sender: obnam-dev-bounces@obnam.org
+Errors-To: obnam-dev-bounces@obnam.org
+
+This is a multi-part message in MIME format.
+--------------D34B4AF69ADA0C644386FED6
+Content-Type: text/plain; charset=utf-8; format=flowed
+Content-Transfer-Encoding: 7bit
+
+Hi, I made some changes to the first patch:
+- modified python code as suggested by Lars
+- improved option description string for man page
+- added a test in the yarns
+
+I managed to add a test among the other scenarios by using only the
+existing "IMPLEMENTS" recipe. I'm confident it passes (it was not
+passing in the first tries for syntax mistakes). I confirm that I have
+an error for 2 scenarios only, I report the output here if you are
+interested.
+
+Bye and thanks, let me know!
+Michele
+
+
+----------------------------------------------------------
+
+ERROR: In scenario "use repository format 6"
+step "THEN restored data in R1 matches T/manifest-1" failed,
+with exit code 1:
+Standard output from shell command:
+ ---
+/tmp/tmpWC8Ic9/use_repository_format_6/datadir/original-manifest
+2017-07-03 15:28:21.712932607 +0200
+ +++
+/tmp/tmpWC8Ic9/use_repository_format_6/datadir/restored-manifest
+2017-07-03 15:28:21.708932607 +0200
+ @@ -1,7 +1,7 @@
+ Name: .
+ Mtime: 2013-12-28 21:11:07 +0000
+ Mode: 40775
+ -Nlink: 4
+ +Nlink: 1
+
+ Name: fifo
+ Mtime: 2013-12-28 21:11:07 +0000
+ @@ -111,10 +111,10 @@
+ Name: dir-empty
+ Mtime: 2013-12-28 21:11:07 +0000
+ Mode: 40775
+ -Nlink: 2
+ +Nlink: 1
+
+ Name: symlink-target-dir
+ Mtime: 2013-12-28 21:11:07 +0000
+ Mode: 40775
+ -Nlink: 2
+ +Nlink: 1
+
+
+Standard error from shell command:
+ + GNUPGHOME=/tmp/tmpWC8Ic9/use_repository_format_6/datadir/dot-gnupg
+ + mkdir -p /tmp/tmpWC8Ic9/use_repository_format_6/datadir/dot-gnupg
+ + rsync -a --delete /home/michele/devel/obnam/test-gpghome/.
+/tmp/tmpWC8Ic9/use_repository_format_6/datadir/dot-gnupg/.
+ + cd /tmp/tmpWC8Ic9/use_repository_format_6/datadir/R1
+ + true
+ + ls
+ + wc -l
+ + cd tmp
+ + true
+ + ls
+ + wc -l
+ + cd tmpbr5YS5
+ + true
+ + + wcls
+ -l
+ + cd data
+ + true
+ + ls
+ + wc -l
+ + break
+ + summain -r --exclude=Ino --exclude=Dev --exclude=Uid
+--exclude=Username --exclude=Gid --exclude=Group --checksum=SHA1 .
+ + normalise_manifest_times
+ + sed /^Mtime:/s/\.[0-9]* / /
+ + normalise_manifest_times
+/tmp/tmpWC8Ic9/use_repository_format_6/datadir/T/manifest-1
+ + sed /^Mtime:/s/\.[0-9]* / /
+/tmp/tmpWC8Ic9/use_repository_format_6/datadir/T/manifest-1
+ + diff -u
+/tmp/tmpWC8Ic9/use_repository_format_6/datadir/original-manifest
+/tmp/tmpWC8Ic9/use_repository_format_6/datadir/restored-manifest
+
+ERROR: In scenario "use repository format 6 with in-tree data"
+step "THEN restored data in R1 matches T/manifest-1" failed,
+with exit code 1:
+Standard output from shell command:
+ ---
+/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/original-manifest
+2017-07-03 15:28:23.648932644 +0200
+ +++
+/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/restored-manifest
+2017-07-03 15:28:23.644932644 +0200
+ @@ -1,7 +1,7 @@
+ Name: .
+ Mtime: 2014-01-07 18:36:19 +0000
+ Mode: 40775
+ -Nlink: 4
+ +Nlink: 1
+
+ Name: fifo
+ Mtime: 2014-01-07 18:36:19 +0000
+ @@ -111,10 +111,10 @@
+ Name: dir-empty
+ Mtime: 2014-01-07 18:36:19 +0000
+ Mode: 40775
+ -Nlink: 2
+ +Nlink: 1
+
+ Name: symlink-target-dir
+ Mtime: 2014-01-07 18:36:19 +0000
+ Mode: 40775
+ -Nlink: 2
+ +Nlink: 1
+
+
+Standard error from shell command:
+ +
+GNUPGHOME=/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/dot-gnupg
+ + mkdir -p
+/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/dot-gnupg
+ + rsync -a --delete /home/michele/devel/obnam/test-gpghome/.
+/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/dot-gnupg/.
+ + cd
+/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/R1
+ + true
+ + + wc -l
+ ls
+ + cd tmp
+ + true
+ + + wc -l
+ ls
+ + cd tmpvR_VsS
+ + true
+ + ls+
+ wc -l
+ + cd data
+ + true
+ + + wc -l
+ ls
+ + break
+ + summain -r --exclude=Ino --exclude=Dev+ --exclude=Uid
+--exclude=Username --exclude=Gid --exclude=Group --checksum=SHA1 .
+ normalise_manifest_times
+ + sed /^Mtime:/s/\.[0-9]* / /
+ + normalise_manifest_times
+/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/T/manifest-1
+ + sed /^Mtime:/s/\.[0-9]* / /
+/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/T/manifest-1
+ + diff -u
+/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/original-manifest
+/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/restored-manifest
+
+Failed scenarios:
+ - use repository format 6
+ - use repository format 6 with in-tree data
+ERROR: Test suite FAILED in 2 scenarios
+ERROR: Command '['yarn', '-s', 'yarns/obnam.sh', '--env',
+'REPOSITORY_FORMAT=6', 'yarns/0010-introduction.yarn',
+'yarns/0020-test-environment.yarn', 'yarns/0040-generations.yarn',
+'yarns/0050-multiple-clients.yarn', 'yarns/0060-encryption.yarn',
+'yarns/0070-compression.yarn', 'yarns/0080-verify.yarn',
+'yarns/0090-lock-handling.yarn', 'yarns/0100-fuse.yarn',
+'yarns/0110-administration.yarn', 'yarns/0120-corrupt-repo.yarn',
+'yarns/0200-repo-formats.yarn', 'yarns/0300-kdirstat-integration.yarn',
+'yarns/9000-implements.yarn', 'yarns/0030-basics.yarn']' returned
+non-zero exit status 1
+
+
+
+
+
+
+La seguente informativa e' inserita in automatico dal sistema al fine esclusivo della realizzazione dei fini istituzionali dell'ente.
+
+
+INVESTI NELLA RICERCA
+
+il 5 per mille all'Universita' Politecnica delle Marche e' un investimento per i giovani, per il loro futuro - C.F. 00382520427
+
+http://www.univpm.it/5_per_mille
+
+--------------D34B4AF69ADA0C644386FED6
+Content-Type: text/x-patch;
+ name="0000-cover-letter.patch"
+Content-Transfer-Encoding: 7bit
+Content-Disposition: attachment;
+ filename="0000-cover-letter.patch"
+
+>From 22f9c4450f331d2d057f357cd1e767c50a0ca0ae Mon Sep 17 00:00:00 2001
+From: Michele Alessandrini <michelealessandrini74@gmail.com>
+Date: Mon, 3 Jul 2017 15:34:19 +0200
+Subject: [PATCH 0/2] *** SUBJECT HERE ***
+
+*** BLURB HERE ***
+
+Michele Alessandrini (2):
+ Add "exclude-if-present" option to exclude directories containing give
+ filename(s)
+ improved python and added yarn for exclude-if-present
+
+ manual/en/060-backing-up.mdwn | 4 ++
+ obnamlib/plugins/exclude_if_present_plugin.py | 53 +++++++++++++++++++++++++++
+ without-tests | 1 +
+ yarns/0030-basics.yarn | 26 +++++++++++++
+ 4 files changed, 84 insertions(+)
+ create mode 100644 obnamlib/plugins/exclude_if_present_plugin.py
+
+--
+2.13.2
+
+
+--------------D34B4AF69ADA0C644386FED6
+Content-Type: text/x-patch;
+ name="0001-Add-exclude-if-present-option-to-exclude.patch"
+Content-Transfer-Encoding: 7bit
+Content-Disposition: attachment;
+ filename="0001-Add-exclude-if-present-option-to-exclude.patch"
+
+>From 50fbcd08554c798456d1693d1a409badc12f6c8c Mon Sep 17 00:00:00 2001
+From: Michele Alessandrini <michelealessandrini74@gmail.com>
+Date: Thu, 29 Jun 2017 12:00:09 +0200
+Subject: [PATCH 1/2] Add "exclude-if-present" option to exclude directories
+ containing give filename(s)
+
+---
+ manual/en/060-backing-up.mdwn | 4 +++
+ obnamlib/plugins/exclude_if_present_plugin.py | 52 +++++++++++++++++++++++++++
+ without-tests | 1 +
+ 3 files changed, 57 insertions(+)
+ create mode 100644 obnamlib/plugins/exclude_if_present_plugin.py
+
+diff --git a/manual/en/060-backing-up.mdwn b/manual/en/060-backing-up.mdwn
+index ee28d6e2..f0edb38f 100644
+--- a/manual/en/060-backing-up.mdwn
++++ b/manual/en/060-backing-up.mdwn
+@@ -112,6 +112,10 @@ backups:
+ are usually not important to back up, and tagging the directory
+ can be easier than constructing a regular expression for
+ `--exclude`.
++* The `--exclude-if-present` setting excludes directories that contain a
++ given file name. This option can be used more than once to specify
++ several file names, if one of them is present the directory gets
++ excluded.
+ * The `--one-file-system` setting excludes any mount points and the
+ contents of the mounted filesystem. This is useful for skipping,
+ for example, virtual filesystems such as `/proc`, remote filesystems
+diff --git a/obnamlib/plugins/exclude_if_present_plugin.py b/obnamlib/plugins/exclude_if_present_plugin.py
+new file mode 100644
+index 00000000..3af81711
+--- /dev/null
++++ b/obnamlib/plugins/exclude_if_present_plugin.py
+@@ -0,0 +1,52 @@
++# Copyright (C) 2015 Lars Wirzenius
++# Copyright (C) 2017 Michele Alessandrini
++#
++# This program is free software: you can redistribute it and/or modify
++# it under the terms of the GNU General Public License as published by
++# the Free Software Foundation, either version 3 of the License, or
++# (at your option) any later version.
++#
++# This program is distributed in the hope that it will be useful,
++# but WITHOUT ANY WARRANTY; without even the implied warranty of
++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
++# GNU General Public License for more details.
++#
++# You should have received a copy of the GNU General Public License
++# along with this program. If not, see <http://www.gnu.org/licenses/>.
++
++
++import logging
++import os
++import stat
++
++import obnamlib
++
++
++class ExcludeIfPresentPlugin(obnamlib.ObnamPlugin):
++
++ def enable(self):
++ backup_group = obnamlib.option_group['backup'] = 'Backing up'
++
++ self.app.settings.string_list(
++ ['exclude-if-present'],
++ 'exclude directories (and their subdirs) '
++ 'that contain the given file (can be '
++ 'used multiple times)',
++ group=backup_group)
++
++ self.app.hooks.add_callback('config-loaded', self.config_loaded)
++
++ def config_loaded(self):
++ if len(self.app.settings['exclude-if-present']) > 0:
++ self.app.hooks.add_callback('backup-exclude', self.exclude)
++
++ def exclude(self, fs=None, pathname=None, stat_result=None, exclude=None,
++ **kwargs):
++ if stat.S_ISDIR(stat_result.st_mode):
++ for filename in self.app.settings['exclude-if-present']:
++ filename_path = os.path.join(pathname, filename)
++ if fs.exists(filename_path):
++ logging.debug('Excluding (exclude-if-present): %s',
++ pathname)
++ exclude[0] = True
++ break
+diff --git a/without-tests b/without-tests
+index f5979198..15d447bf 100644
+--- a/without-tests
++++ b/without-tests
+@@ -21,6 +21,7 @@ obnamlib/plugins/dump_repo_plugin.py
+ obnamlib/plugins/encryption_plugin.py
+ obnamlib/plugins/exclude_caches_plugin.py
+ obnamlib/plugins/exclude_pathnames_plugin.py
++obnamlib/plugins/exclude_if_present_plugin.py
+ obnamlib/plugins/force_lock_plugin.py
+ obnamlib/plugins/forget_plugin.py
+ obnamlib/plugins/fsck_plugin.py
+--
+2.13.2
+
+
+--------------D34B4AF69ADA0C644386FED6
+Content-Type: text/x-patch;
+ name="0002-improved-python-and-added-yarn-for-exclude-if-presen.patch"
+Content-Transfer-Encoding: 7bit
+Content-Disposition: attachment;
+ filename*0="0002-improved-python-and-added-yarn-for-exclude-if-presen.pa";
+ filename*1="tch"
+
+>From 22f9c4450f331d2d057f357cd1e767c50a0ca0ae Mon Sep 17 00:00:00 2001
+From: Michele Alessandrini <michelealessandrini74@gmail.com>
+Date: Mon, 3 Jul 2017 15:32:36 +0200
+Subject: [PATCH 2/2] improved python and added yarn for exclude-if-present
+
+---
+ obnamlib/plugins/exclude_if_present_plugin.py | 5 +++--
+ yarns/0030-basics.yarn | 26 ++++++++++++++++++++++++++
+ 2 files changed, 29 insertions(+), 2 deletions(-)
+
+diff --git a/obnamlib/plugins/exclude_if_present_plugin.py b/obnamlib/plugins/exclude_if_present_plugin.py
+index 3af81711..598306d1 100644
+--- a/obnamlib/plugins/exclude_if_present_plugin.py
++++ b/obnamlib/plugins/exclude_if_present_plugin.py
+@@ -30,14 +30,15 @@ class ExcludeIfPresentPlugin(obnamlib.ObnamPlugin):
+ self.app.settings.string_list(
+ ['exclude-if-present'],
+ 'exclude directories (and their subdirs) '
+- 'that contain the given file (can be '
++ 'that contain the given filename (can be '
+ 'used multiple times)',
++ metavar='FILENAME',
+ group=backup_group)
+
+ self.app.hooks.add_callback('config-loaded', self.config_loaded)
+
+ def config_loaded(self):
+- if len(self.app.settings['exclude-if-present']) > 0:
++ if self.app.settings['exclude-if-present']:
+ self.app.hooks.add_callback('backup-exclude', self.exclude)
+
+ def exclude(self, fs=None, pathname=None, stat_result=None, exclude=None,
+diff --git a/yarns/0030-basics.yarn b/yarns/0030-basics.yarn
+index 226ec2a8..7590115f 100644
+--- a/yarns/0030-basics.yarn
++++ b/yarns/0030-basics.yarn
+@@ -273,6 +273,32 @@ Time to backup.
+ AND user U restores their latest generation in repository R into X
+ THEN L, restored to X, matches manifest M
+
++Exclude directories containing given filename(s)
++------------------------------------------------
++
++The `--exclude-if-present` option tells Obnam to exclude any
++directories containing the given filename.
++
++ SCENARIO exclude directories containing given filename
++ GIVEN 1k of new data in directory L/wanted
++ AND 1k of new data in directory L/unwanted
++ AND file L/unwanted/_nobackup_ with permissions 664
++
++We'll now create the manifest, but remove `L/unwanted` (and files in
++`L/unwanted`) so that it matches what we need. We do it this instead of
++creating the manifest before `L/unwanted`, because creating `L/unwanted`
++changes the timestamp of `L`.
++
++ AND a manifest of L in M
++ AND unwanted is removed from manifest M
++
++Time to backup.
++
++ AND user U sets configuration exclude-if-present to _nobackup_
++ WHEN user U backs up directory L to repository R
++ AND user U restores their latest generation in repository R into X
++ THEN L, restored to X, matches manifest M
++
+ Excluded, already backed up files, are not included in next generation
+ ----------------------------------------------------------------------
+
+--
+2.13.2
+
+
+--------------D34B4AF69ADA0C644386FED6
+Content-Type: text/plain; charset="us-ascii"
+MIME-Version: 1.0
+Content-Transfer-Encoding: 7bit
+Content-Disposition: inline
+
+_______________________________________________
+obnam-dev mailing list
+obnam-dev@obnam.org
+http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org
+
+--------------D34B4AF69ADA0C644386FED6--
+