path: root/tickets/2173d4e7d8a4497da071ca9d46be611b/Maildir/new/1498731008.M794712P3216Q1.koom

Return-Path: <obnam-dev-bounces@obnam.org>
X-Original-To: distix@pieni.net
Delivered-To: distix@pieni.net
Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16])
	by pieni.net (Postfix) with ESMTPS id 6DDE04374E
	for <distix@pieni.net>; Thu, 29 Jun 2017 10:08:54 +0000 (UTC)
Received: from platypus.pepperfish.net (unknown [10.112.100.20])
	by bagpuss.pepperfish.net (Postfix) with ESMTP id E45B6BA9;
	Thu, 29 Jun 2017 11:08:53 +0100 (BST)
Received: from ip6-localhost ([::1] helo=platypus.pepperfish.net)
	by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
	id 1dQWNZ-0000im-OT; Thu, 29 Jun 2017 11:08:53 +0100
Received: from yaffle.pepperfish.net ([88.99.213.221]
 helo=inmail2.pepperfish.net)
 by platypus.pepperfish.net with esmtps (Exim 4.80 #2 (Debian))
 id 1dQWNX-0000iO-O5
 for <obnam-dev@obnam.org>; Thu, 29 Jun 2017 11:08:51 +0100
Received: from mta01.univpm.it ([193.205.128.17])
 by inmail2.pepperfish.net with esmtps
 (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89)
 (envelope-from <m.alessandrini@univpm.it>) id 1dQWNV-0007ka-7u
 for obnam-dev@obnam.org; Thu, 29 Jun 2017 11:08:51 +0100
Received: from [193.205.130.201] (thor.dii.univpm.it [193.205.130.201])
 by mta01.univpm.it (8.14.8/8.14.8) with ESMTP id v5TA8egs001910
 for <obnam-dev@obnam.org>; Thu, 29 Jun 2017 12:08:40 +0200
To: obnam-dev@obnam.org
From: Michele Alessandrini <m.alessandrini@univpm.it>
Organization: =?UTF-8?Q?DIBET_-_Universit=c3=a0_Politecnica_delle_Marche?=
Message-ID: <0dc53aa0-b510-8607-ba63-f527f49bdf50@univpm.it>
Date: Thu, 29 Jun 2017 12:08:29 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Thunderbird/45.8.0
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="------------1A8E1BD52A0B7D67E2F905BD"
X-Greylist: inspected by milter-greylist-4.5.12 (mta01.univpm.it
 [193.205.128.17]);
 Thu, 29 Jun 2017 12:08:40 +0200 (CEST) for IP:'193.205.130.201'
 DOMAIN:'thor.dii.univpm.it' HELO:'[193.205.130.201]'
 FROM:'m.alessandrini@univpm.it' RCPT:''
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.12
 (mta01.univpm.it [193.205.128.17]); Thu, 29 Jun 2017 12:08:40 +0200 (CEST)
X-Virus-Scanned: clamav-milter 0.98.7 at mta01.univpm.it
X-Virus-Status: Clean
X-Scanned-By: MIMEDefang 2.78 on 193.205.128.17
X-Pepperfish-Transaction: 6864-af2f-b93d-e758
X-Spam-Score: -4.9
X-Spam-Score-int: -48
X-Spam-Bar: ----
X-Scanned-By: pepperfish.net, Thu, 29 Jun 2017 11:08:51 +0100
X-Spam-Report: Content analysis details: (-4.9 points)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -1.5 RCVD_IN_DNSWL_MED      RBL: Sender listed at http://www.dnswl.org/, medium
 trust [193.205.128.17 listed in list.dnswl.org]
 -0.5 PPF_USER_AGENT         User-Agent: exists
 -1.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
 -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
 [score: 0.0000]
X-ACL-Warn: message may be spam
X-Scan-Signature: 517f0562ca426c231cd3a90ab8fa0eca
Subject: "exclude-if-present" option
X-BeenThere: obnam-dev@obnam.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Obnam development discussions <obnam-dev-obnam.org>
List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
 <mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
List-Post: <mailto:obnam-dev@obnam.org>
List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
 <mailto:obnam-dev-request@obnam.org?subject=subscribe>
Sender: obnam-dev-bounces@obnam.org
Errors-To: obnam-dev-bounces@obnam.org

This is a multi-part message in MIME format.
--------------1A8E1BD52A0B7D67E2F905BD
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi, I just implemented the "exclude-if-present" feature we were 
discussing yesterday. Is the patch format correct?
I'd be happy if you can review it and consider if it's worth. I tested 
it and it seems to work correctly.
Actually it's not been very difficult, because I copied the 
"exclude-caches" plugin (very similar) and adapted for the new case.

The "./check" suite mostly passes (I had to fight a little with pep8), 
but it gives some errors at the end, I suspect they're not related to 
code, I report the output at the end. Maybe I have not all the needed 
programs installed? I followed the README, and by the way I use debian 
testing so I conveniently found all your packages right in aptitude.

About unit tests, I found there is no similar tests for exclusion cases, 
so it would be difficult for me to write one from scratch, and I added 
it to "without-tests" file together with the other exclude plugins. I 
hope this is not a big problem.

Now about the documentation: I modified the english markdown manual, 
obviously I cannot modify german and french. By the way I'm Italian, it 
would be nice if I could contribute a translation, but I'm not sure I'd 
have the needed time, and secondly I think technical things are better 
in original english. But let me know what you think.

A note: the German PDF fails to build on my computer.

A doubt: where is the man page? Because it needs to be updated too, but 
I can't find its sources.

A suggestion: adding "*.pyc" to .gitignore, otherwise the git status is 
a mess with all the compiled python files.

Bye and thanks, hope to hear from you
Michele (male, if you are wondering)

-------------------------------------------------------------------------------------------
Failed scenarios:
     - backup non-basic filesystem objects
     - backup notices when extended attribute value changes
     - Browsing backups with FUSE plugin
     - use repository format 6
     - use repository format 6 with in-tree data
     - create and do a check of a kdirstat cache of interesting objects
ERROR: Test suite FAILED in 6 scenarios
ERROR: Command '['yarn', '-s', 'yarns/obnam.sh', '--env', 
'REPOSITORY_FORMAT=6', '--require-assumptions', 
'yarns/0010-introduction.yarn', 'yarns/0020-test-environment.yarn', 
'yarns/0030-basics.yarn', 'yarns/0040-generations.yarn', 
'yarns/0050-multiple-clients.yarn', 'yarns/0060-encryption.yarn', 
'yarns/0070-compression.yarn', 'yarns/0080-verify.yarn', 
'yarns/0090-lock-handling.yarn', 'yarns/0100-fuse.yarn', 
'yarns/0110-administration.yarn', 'yarns/0120-corrupt-repo.yarn', 
'yarns/0200-repo-formats.yarn', 'yarns/0300-kdirstat-integration.yarn', 
'yarns/9000-implements.yarn']' returned non-zero exit status 1

 
 
 
 
La seguente informativa e' inserita in automatico dal sistema al fine esclusivo della realizzazione dei fini istituzionali dell'ente. 
 
 
INVESTI NELLA RICERCA 
 
il 5 per mille all'Universita' Politecnica delle Marche e' un investimento per i giovani, per il loro futuro - C.F. 00382520427 
 
http://www.univpm.it/5_per_mille 

--------------1A8E1BD52A0B7D67E2F905BD
Content-Type: text/x-patch;
 name="0000-cover-letter.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="0000-cover-letter.patch"

>From 50fbcd08554c798456d1693d1a409badc12f6c8c Mon Sep 17 00:00:00 2001
From: Michele Alessandrini <michelealessandrini74@gmail.com>
Date: Thu, 29 Jun 2017 12:03:29 +0200
Subject: [PATCH 0/1] *** SUBJECT HERE ***

*** BLURB HERE ***

Michele Alessandrini (1):
  Add "exclude-if-present" option to exclude directories containing give
    filename(s)

 manual/en/060-backing-up.mdwn                 |  4 +++
 obnamlib/plugins/exclude_if_present_plugin.py | 52 +++++++++++++++++++++++++++
 without-tests                                 |  1 +
 3 files changed, 57 insertions(+)
 create mode 100644 obnamlib/plugins/exclude_if_present_plugin.py

-- 
2.11.0


--------------1A8E1BD52A0B7D67E2F905BD
Content-Type: text/x-patch;
 name="0001-Add-exclude-if-present-option-to-exclude.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="0001-Add-exclude-if-present-option-to-exclude.patch"

>From 50fbcd08554c798456d1693d1a409badc12f6c8c Mon Sep 17 00:00:00 2001
From: Michele Alessandrini <michelealessandrini74@gmail.com>
Date: Thu, 29 Jun 2017 12:00:09 +0200
Subject: [PATCH 1/1] Add "exclude-if-present" option to exclude directories
 containing give filename(s)

---
 manual/en/060-backing-up.mdwn                 |  4 +++
 obnamlib/plugins/exclude_if_present_plugin.py | 52 +++++++++++++++++++++++++++
 without-tests                                 |  1 +
 3 files changed, 57 insertions(+)
 create mode 100644 obnamlib/plugins/exclude_if_present_plugin.py

diff --git a/manual/en/060-backing-up.mdwn b/manual/en/060-backing-up.mdwn
index ee28d6e2..f0edb38f 100644
--- a/manual/en/060-backing-up.mdwn
+++ b/manual/en/060-backing-up.mdwn
@@ -112,6 +112,10 @@ backups:
   are usually not important to back up, and tagging the directory
   can be easier than constructing a regular expression for
   `--exclude`.
+* The `--exclude-if-present` setting excludes directories that contain a
+  given file name. This option can be used more than once to specify
+  several file names, if one of them is present the directory gets
+  excluded.
 * The `--one-file-system` setting excludes any mount points and the
   contents of the mounted filesystem. This is useful for skipping,
   for example, virtual filesystems such as `/proc`, remote filesystems
diff --git a/obnamlib/plugins/exclude_if_present_plugin.py b/obnamlib/plugins/exclude_if_present_plugin.py
new file mode 100644
index 00000000..3af81711
--- /dev/null
+++ b/obnamlib/plugins/exclude_if_present_plugin.py
@@ -0,0 +1,52 @@
+# Copyright (C) 2015  Lars Wirzenius
+# Copyright (C) 2017  Michele Alessandrini
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+import logging
+import os
+import stat
+
+import obnamlib
+
+
+class ExcludeIfPresentPlugin(obnamlib.ObnamPlugin):
+
+    def enable(self):
+        backup_group = obnamlib.option_group['backup'] = 'Backing up'
+
+        self.app.settings.string_list(
+            ['exclude-if-present'],
+            'exclude directories (and their subdirs) '
+            'that contain the given file (can be '
+            'used multiple times)',
+            group=backup_group)
+
+        self.app.hooks.add_callback('config-loaded', self.config_loaded)
+
+    def config_loaded(self):
+        if len(self.app.settings['exclude-if-present']) > 0:
+            self.app.hooks.add_callback('backup-exclude', self.exclude)
+
+    def exclude(self, fs=None, pathname=None, stat_result=None, exclude=None,
+                **kwargs):
+        if stat.S_ISDIR(stat_result.st_mode):
+            for filename in self.app.settings['exclude-if-present']:
+                filename_path = os.path.join(pathname, filename)
+                if fs.exists(filename_path):
+                    logging.debug('Excluding (exclude-if-present): %s',
+                                  pathname)
+                    exclude[0] = True
+                    break
diff --git a/without-tests b/without-tests
index f5979198..15d447bf 100644
--- a/without-tests
+++ b/without-tests
@@ -21,6 +21,7 @@ obnamlib/plugins/dump_repo_plugin.py
 obnamlib/plugins/encryption_plugin.py
 obnamlib/plugins/exclude_caches_plugin.py
 obnamlib/plugins/exclude_pathnames_plugin.py
+obnamlib/plugins/exclude_if_present_plugin.py
 obnamlib/plugins/force_lock_plugin.py
 obnamlib/plugins/forget_plugin.py
 obnamlib/plugins/fsck_plugin.py
-- 
2.11.0


--------------1A8E1BD52A0B7D67E2F905BD
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
obnam-dev mailing list
obnam-dev@obnam.org
http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org

--------------1A8E1BD52A0B7D67E2F905BD--