To: obnam-dev@obnam.org
From: Michele Alessandrini <m.alessandrini@univpm.it>
Organization: =?UTF-8?Q?DIBET_-_Universit=c3=a0_Politecnica_delle_Marche?=
Message-ID: <0dc53aa0-b510-8607-ba63-f527f49bdf50@univpm.it>
Date: Thu, 29 Jun 2017 12:08:29 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Thunderbird/45.8.0
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="------------1A8E1BD52A0B7D67E2F905BD"
Subject: "exclude-if-present" option
Precedence: list
Sender: obnam-dev-bounces@obnam.org
Errors-To: obnam-dev-bounces@obnam.org

This is a multi-part message in MIME format.
--------------1A8E1BD52A0B7D67E2F905BD
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi, I just implemented the "exclude-if-present" feature we were 
discussing yesterday. Is the patch format correct?
I'd be happy if you can review it and consider if it's worth. I tested 
it and it seems to work correctly.
Actually it's not been very difficult, because I copied the 
"exclude-caches" plugin (very similar) and adapted for the new case.

The "./check" suite mostly passes (I had to fight a little with pep8), 
but it gives some errors at the end, I suspect they're not related to 
code, I report the output at the end. Maybe I have not all the needed 
programs installed? I followed the README, and by the way I use debian 
testing so I conveniently found all your packages right in aptitude.

About unit tests, I found there is no similar tests for exclusion cases, 
so it would be difficult for me to write one from scratch, and I added 
it to "without-tests" file together with the other exclude plugins. I 
hope this is not a big problem.

Now about the documentation: I modified the english markdown manual, 
obviously I cannot modify german and french. By the way I'm Italian, it 
would be nice if I could contribute a translation, but I'm not sure I'd 
have the needed time, and secondly I think technical things are better 
in original english. But let me know what you think.

A note: the German PDF fails to build on my computer.

A doubt: where is the man page? Because it needs to be updated too, but 
I can't find its sources.

A suggestion: adding "*.pyc" to .gitignore, otherwise the git status is 
a mess with all the compiled python files.

Bye and thanks, hope to hear from you
Michele (male, if you are wondering)

-------------------------------------------------------------------------------------------
Failed scenarios:
     - backup non-basic filesystem objects
     - backup notices when extended attribute value changes
     - Browsing backups with FUSE plugin
     - use repository format 6
     - use repository format 6 with in-tree data
     - create and do a check of a kdirstat cache of interesting objects
ERROR: Test suite FAILED in 6 scenarios
ERROR: Command '['yarn', '-s', 'yarns/obnam.sh', '--env', 
'REPOSITORY_FORMAT=6', '--require-assumptions', 
'yarns/0010-introduction.yarn', 'yarns/0020-test-environment.yarn', 
'yarns/0030-basics.yarn', 'yarns/0040-generations.yarn', 
'yarns/0050-multiple-clients.yarn', 'yarns/0060-encryption.yarn', 
'yarns/0070-compression.yarn', 'yarns/0080-verify.yarn', 
'yarns/0090-lock-handling.yarn', 'yarns/0100-fuse.yarn', 
'yarns/0110-administration.yarn', 'yarns/0120-corrupt-repo.yarn', 
'yarns/0200-repo-formats.yarn', 'yarns/0300-kdirstat-integration.yarn', 
'yarns/9000-implements.yarn']' returned non-zero exit status 1

 
La seguente informativa e' inserita in automatico dal sistema al fine esclusivo della realizzazione dei fini istituzionali dell'ente. 
 
 
INVESTI NELLA RICERCA 
 
il 5 per mille all'Universita' Politecnica delle Marche e' un investimento per i giovani, per il loro futuro - C.F. 00382520427 
 
http://www.univpm.it/5_per_mille 

--------------1A8E1BD52A0B7D67E2F905BD
Content-Type: text/x-patch;
 name="0000-cover-letter.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="0000-cover-letter.patch"

>From 50fbcd08554c798456d1693d1a409badc12f6c8c Mon Sep 17 00:00:00 2001
From: Michele Alessandrini <michelealessandrini74@gmail.com>
Date: Thu, 29 Jun 2017 12:03:29 +0200
Subject: [PATCH 0/1] *** SUBJECT HERE ***

*** BLURB HERE ***

Michele Alessandrini (1):
  Add "exclude-if-present" option to exclude directories containing give
    filename(s)

 manual/en/060-backing-up.mdwn                 |  4 +++
 obnamlib/plugins/exclude_if_present_plugin.py | 52 +++++++++++++++++++++++++++
 without-tests                                 |  1 +
 3 files changed, 57 insertions(+)
 create mode 100644 obnamlib/plugins/exclude_if_present_plugin.py

-- 
2.11.0


--------------1A8E1BD52A0B7D67E2F905BD
Content-Type: text/x-patch;
 name="0001-Add-exclude-if-present-option-to-exclude.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="0001-Add-exclude-if-present-option-to-exclude.patch"

>From 50fbcd08554c798456d1693d1a409badc12f6c8c Mon Sep 17 00:00:00 2001
From: Michele Alessandrini <michelealessandrini74@gmail.com>
Date: Thu, 29 Jun 2017 12:00:09 +0200
Subject: [PATCH 1/1] Add "exclude-if-present" option to exclude directories
 containing give filename(s)

---
 manual/en/060-backing-up.mdwn                 |  4 +++
 obnamlib/plugins/exclude_if_present_plugin.py | 52 +++++++++++++++++++++++++++
 without-tests                                 |  1 +
 3 files changed, 57 insertions(+)
 create mode 100644 obnamlib/plugins/exclude_if_present_plugin.py

diff --git a/manual/en/060-backing-up.mdwn b/manual/en/060-backing-up.mdwn
index ee28d6e2..f0edb38f 100644
--- a/manual/en/060-backing-up.mdwn
+++ b/manual/en/060-backing-up.mdwn
@@ -112,6 +112,10 @@ backups:
   are usually not important to back up, and tagging the directory
   can be easier than constructing a regular expression for
   `--exclude`.
+* The `--exclude-if-present` setting excludes directories that contain a
+  given file name. This option can be used more than once to specify
+  several file names, if one of them is present the directory gets
+  excluded.
 * The `--one-file-system` setting excludes any mount points and the
   contents of the mounted filesystem. This is useful for skipping,
   for example, virtual filesystems such as `/proc`, remote filesystems
diff --git a/obnamlib/plugins/exclude_if_present_plugin.py b/obnamlib/plugins/exclude_if_present_plugin.py
new file mode 100644
index 00000000..3af81711
--- /dev/null
+++ b/obnamlib/plugins/exclude_if_present_plugin.py
@@ -0,0 +1,52 @@
+# Copyright (C) 2015  Lars Wirzenius
+# Copyright (C) 2017  Michele Alessandrini
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+import logging
+import os
+import stat
+
+import obnamlib
+
+
+class ExcludeIfPresentPlugin(obnamlib.ObnamPlugin):
+
+    def enable(self):
+        backup_group = obnamlib.option_group['backup'] = 'Backing up'
+
+        self.app.settings.string_list(
+            ['exclude-if-present'],
+            'exclude directories (and their subdirs) '
+            'that contain the given file (can be '
+            'used multiple times)',
+            group=backup_group)
+
+        self.app.hooks.add_callback('config-loaded', self.config_loaded)
+
+    def config_loaded(self):
+        if len(self.app.settings['exclude-if-present']) > 0:
+            self.app.hooks.add_callback('backup-exclude', self.exclude)
+
+    def exclude(self, fs=None, pathname=None, stat_result=None, exclude=None,
+                **kwargs):
+        if stat.S_ISDIR(stat_result.st_mode):
+            for filename in self.app.settings['exclude-if-present']:
+                filename_path = os.path.join(pathname, filename)
+                if fs.exists(filename_path):
+                    logging.debug('Excluding (exclude-if-present): %s',
+                                  pathname)
+                    exclude[0] = True
+                    break
diff --git a/without-tests b/without-tests
index f5979198..15d447bf 100644
--- a/without-tests
+++ b/without-tests
@@ -21,6 +21,7 @@ obnamlib/plugins/dump_repo_plugin.py
 obnamlib/plugins/encryption_plugin.py
 obnamlib/plugins/exclude_caches_plugin.py
 obnamlib/plugins/exclude_pathnames_plugin.py
+obnamlib/plugins/exclude_if_present_plugin.py
 obnamlib/plugins/force_lock_plugin.py
 obnamlib/plugins/forget_plugin.py
 obnamlib/plugins/fsck_plugin.py
-- 
2.11.0


--------------1A8E1BD52A0B7D67E2F905BD
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
obnam-dev mailing list
obnam-dev@obnam.org
http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org

--------------1A8E1BD52A0B7D67E2F905BD--