From d4dbc955429129e1ad6ca751c0006993740f984e Mon Sep 17 00:00:00 2001 From: distix ticketing system Date: Mon, 3 Jul 2017 13:49:30 +0000 Subject: imported mails --- .../Maildir/new/1499089770.M541712P12423Q1.koom | 500 +++++++++++++++++++++ 1 file changed, 500 insertions(+) create mode 100644 tickets/2173d4e7d8a4497da071ca9d46be611b/Maildir/new/1499089770.M541712P12423Q1.koom (limited to 'tickets') diff --git a/tickets/2173d4e7d8a4497da071ca9d46be611b/Maildir/new/1499089770.M541712P12423Q1.koom b/tickets/2173d4e7d8a4497da071ca9d46be611b/Maildir/new/1499089770.M541712P12423Q1.koom new file mode 100644 index 0000000..4b1249e --- /dev/null +++ b/tickets/2173d4e7d8a4497da071ca9d46be611b/Maildir/new/1499089770.M541712P12423Q1.koom @@ -0,0 +1,500 @@ +Return-Path: +X-Original-To: distix@pieni.net +Delivered-To: distix@pieni.net +Received: from yaffle.pepperfish.net (yaffle.pepperfish.net [88.99.213.221]) + by pieni.net (Postfix) with ESMTPS id 6C2EF44F5F + for ; Mon, 3 Jul 2017 13:42:57 +0000 (UTC) +Received: from platypus.pepperfish.net (unknown [10.112.101.20]) + by yaffle.pepperfish.net (Postfix) with ESMTP id E859441CD6; + Mon, 3 Jul 2017 14:42:56 +0100 (BST) +Received: from ip6-localhost.nat ([::1] helo=platypus.pepperfish.net) + by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian)) + id 1dS1cu-0007pv-Ta; Mon, 03 Jul 2017 14:42:56 +0100 +Received: from [10.112.101.21] (helo=inmail2.pepperfish.net) + by platypus.pepperfish.net with esmtps (Exim 4.80 #2 (Debian)) + id 1dS1cu-0007pj-Bk + for ; Mon, 03 Jul 2017 14:42:56 +0100 +Received: from mta01.univpm.it ([193.205.128.17]) + by inmail2.pepperfish.net with esmtps + (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) + (envelope-from ) id 1dS1co-0007z5-Tz + for obnam-dev@obnam.org; Mon, 03 Jul 2017 14:42:56 +0100 +Received: from [193.205.130.201] (thor.dii.univpm.it [193.205.130.201]) + by mta01.univpm.it (8.14.8/8.14.8) with ESMTP id v63DgfVx002630; + Mon, 3 Jul 2017 15:42:41 +0200 +To: Lars Wirzenius +References: <0dc53aa0-b510-8607-ba63-f527f49bdf50@univpm.it> + <20170703053310.7y5qri5l6mduhuli@liw.fi> + +From: Michele Alessandrini +Organization: =?UTF-8?Q?DIBET_-_Universit=c3=a0_Politecnica_delle_Marche?= +Message-ID: <6e754eb6-34a3-a1e6-9c10-dc4b053ee954@univpm.it> +Date: Mon, 3 Jul 2017 15:42:26 +0200 +User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 + Thunderbird/45.8.0 +MIME-Version: 1.0 +In-Reply-To: +Content-Type: multipart/mixed; boundary="------------D34B4AF69ADA0C644386FED6" +X-Greylist: inspected by milter-greylist-4.5.12 (mta01.univpm.it + [193.205.128.17]); + Mon, 03 Jul 2017 15:42:41 +0200 (CEST) for IP:'193.205.130.201' + DOMAIN:'thor.dii.univpm.it' HELO:'[193.205.130.201]' + FROM:'m.alessandrini@univpm.it' RCPT:'' +X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.12 + (mta01.univpm.it [193.205.128.17]); Mon, 03 Jul 2017 15:42:41 +0200 (CEST) +X-Virus-Scanned: clamav-milter 0.98.7 at mta01.univpm.it +X-Virus-Status: Clean +X-Scanned-By: MIMEDefang 2.78 on 193.205.128.17 +X-Pepperfish-Transaction: ba3b-0bcc-1918-0d90 +X-Spam-Score: -4.9 +X-Spam-Score-int: -48 +X-Spam-Bar: ---- +X-Scanned-By: pepperfish.net, Mon, 03 Jul 2017 14:42:56 +0100 +X-Spam-Report: Content analysis details: (-4.9 points) + pts rule name description + ---- ---------------------- -------------------------------------------------- + -0.5 PPF_USER_AGENT User-Agent: exists + -1.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain + -1.5 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/, medium + trust [193.205.128.17 listed in list.dnswl.org] + -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% + [score: 0.0000] +X-ACL-Warn: message may be spam +X-Scan-Signature: 5bb76d4b0387b43dc4e28efac2549394 +Cc: obnam-dev@obnam.org +Subject: patch N.2 for "exclude-if-present" option +X-BeenThere: obnam-dev@obnam.org +X-Mailman-Version: 2.1.5 +Precedence: list +List-Id: Obnam development discussions +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +Sender: obnam-dev-bounces@obnam.org +Errors-To: obnam-dev-bounces@obnam.org + +This is a multi-part message in MIME format. +--------------D34B4AF69ADA0C644386FED6 +Content-Type: text/plain; charset=utf-8; format=flowed +Content-Transfer-Encoding: 7bit + +Hi, I made some changes to the first patch: +- modified python code as suggested by Lars +- improved option description string for man page +- added a test in the yarns + +I managed to add a test among the other scenarios by using only the +existing "IMPLEMENTS" recipe. I'm confident it passes (it was not +passing in the first tries for syntax mistakes). I confirm that I have +an error for 2 scenarios only, I report the output here if you are +interested. + +Bye and thanks, let me know! +Michele + + +---------------------------------------------------------- + +ERROR: In scenario "use repository format 6" +step "THEN restored data in R1 matches T/manifest-1" failed, +with exit code 1: +Standard output from shell command: + --- +/tmp/tmpWC8Ic9/use_repository_format_6/datadir/original-manifest +2017-07-03 15:28:21.712932607 +0200 + +++ +/tmp/tmpWC8Ic9/use_repository_format_6/datadir/restored-manifest +2017-07-03 15:28:21.708932607 +0200 + @@ -1,7 +1,7 @@ + Name: . + Mtime: 2013-12-28 21:11:07 +0000 + Mode: 40775 + -Nlink: 4 + +Nlink: 1 + + Name: fifo + Mtime: 2013-12-28 21:11:07 +0000 + @@ -111,10 +111,10 @@ + Name: dir-empty + Mtime: 2013-12-28 21:11:07 +0000 + Mode: 40775 + -Nlink: 2 + +Nlink: 1 + + Name: symlink-target-dir + Mtime: 2013-12-28 21:11:07 +0000 + Mode: 40775 + -Nlink: 2 + +Nlink: 1 + + +Standard error from shell command: + + GNUPGHOME=/tmp/tmpWC8Ic9/use_repository_format_6/datadir/dot-gnupg + + mkdir -p /tmp/tmpWC8Ic9/use_repository_format_6/datadir/dot-gnupg + + rsync -a --delete /home/michele/devel/obnam/test-gpghome/. +/tmp/tmpWC8Ic9/use_repository_format_6/datadir/dot-gnupg/. + + cd /tmp/tmpWC8Ic9/use_repository_format_6/datadir/R1 + + true + + ls + + wc -l + + cd tmp + + true + + ls + + wc -l + + cd tmpbr5YS5 + + true + + + wcls + -l + + cd data + + true + + ls + + wc -l + + break + + summain -r --exclude=Ino --exclude=Dev --exclude=Uid +--exclude=Username --exclude=Gid --exclude=Group --checksum=SHA1 . + + normalise_manifest_times + + sed /^Mtime:/s/\.[0-9]* / / + + normalise_manifest_times +/tmp/tmpWC8Ic9/use_repository_format_6/datadir/T/manifest-1 + + sed /^Mtime:/s/\.[0-9]* / / +/tmp/tmpWC8Ic9/use_repository_format_6/datadir/T/manifest-1 + + diff -u +/tmp/tmpWC8Ic9/use_repository_format_6/datadir/original-manifest +/tmp/tmpWC8Ic9/use_repository_format_6/datadir/restored-manifest + +ERROR: In scenario "use repository format 6 with in-tree data" +step "THEN restored data in R1 matches T/manifest-1" failed, +with exit code 1: +Standard output from shell command: + --- +/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/original-manifest +2017-07-03 15:28:23.648932644 +0200 + +++ +/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/restored-manifest +2017-07-03 15:28:23.644932644 +0200 + @@ -1,7 +1,7 @@ + Name: . + Mtime: 2014-01-07 18:36:19 +0000 + Mode: 40775 + -Nlink: 4 + +Nlink: 1 + + Name: fifo + Mtime: 2014-01-07 18:36:19 +0000 + @@ -111,10 +111,10 @@ + Name: dir-empty + Mtime: 2014-01-07 18:36:19 +0000 + Mode: 40775 + -Nlink: 2 + +Nlink: 1 + + Name: symlink-target-dir + Mtime: 2014-01-07 18:36:19 +0000 + Mode: 40775 + -Nlink: 2 + +Nlink: 1 + + +Standard error from shell command: + + +GNUPGHOME=/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/dot-gnupg + + mkdir -p +/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/dot-gnupg + + rsync -a --delete /home/michele/devel/obnam/test-gpghome/. +/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/dot-gnupg/. + + cd +/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/R1 + + true + + + wc -l + ls + + cd tmp + + true + + + wc -l + ls + + cd tmpvR_VsS + + true + + ls+ + wc -l + + cd data + + true + + + wc -l + ls + + break + + summain -r --exclude=Ino --exclude=Dev+ --exclude=Uid +--exclude=Username --exclude=Gid --exclude=Group --checksum=SHA1 . + normalise_manifest_times + + sed /^Mtime:/s/\.[0-9]* / / + + normalise_manifest_times +/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/T/manifest-1 + + sed /^Mtime:/s/\.[0-9]* / / +/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/T/manifest-1 + + diff -u +/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/original-manifest +/tmp/tmpYuAW3Y/use_repository_format_6_with_in-tree_data/datadir/restored-manifest + +Failed scenarios: + - use repository format 6 + - use repository format 6 with in-tree data +ERROR: Test suite FAILED in 2 scenarios +ERROR: Command '['yarn', '-s', 'yarns/obnam.sh', '--env', +'REPOSITORY_FORMAT=6', 'yarns/0010-introduction.yarn', +'yarns/0020-test-environment.yarn', 'yarns/0040-generations.yarn', +'yarns/0050-multiple-clients.yarn', 'yarns/0060-encryption.yarn', +'yarns/0070-compression.yarn', 'yarns/0080-verify.yarn', +'yarns/0090-lock-handling.yarn', 'yarns/0100-fuse.yarn', +'yarns/0110-administration.yarn', 'yarns/0120-corrupt-repo.yarn', +'yarns/0200-repo-formats.yarn', 'yarns/0300-kdirstat-integration.yarn', +'yarns/9000-implements.yarn', 'yarns/0030-basics.yarn']' returned +non-zero exit status 1 + + + + + + +La seguente informativa e' inserita in automatico dal sistema al fine esclusivo della realizzazione dei fini istituzionali dell'ente. + + +INVESTI NELLA RICERCA + +il 5 per mille all'Universita' Politecnica delle Marche e' un investimento per i giovani, per il loro futuro - C.F. 00382520427 + +http://www.univpm.it/5_per_mille + +--------------D34B4AF69ADA0C644386FED6 +Content-Type: text/x-patch; + name="0000-cover-letter.patch" +Content-Transfer-Encoding: 7bit +Content-Disposition: attachment; + filename="0000-cover-letter.patch" + +>From 22f9c4450f331d2d057f357cd1e767c50a0ca0ae Mon Sep 17 00:00:00 2001 +From: Michele Alessandrini +Date: Mon, 3 Jul 2017 15:34:19 +0200 +Subject: [PATCH 0/2] *** SUBJECT HERE *** + +*** BLURB HERE *** + +Michele Alessandrini (2): + Add "exclude-if-present" option to exclude directories containing give + filename(s) + improved python and added yarn for exclude-if-present + + manual/en/060-backing-up.mdwn | 4 ++ + obnamlib/plugins/exclude_if_present_plugin.py | 53 +++++++++++++++++++++++++++ + without-tests | 1 + + yarns/0030-basics.yarn | 26 +++++++++++++ + 4 files changed, 84 insertions(+) + create mode 100644 obnamlib/plugins/exclude_if_present_plugin.py + +-- +2.13.2 + + +--------------D34B4AF69ADA0C644386FED6 +Content-Type: text/x-patch; + name="0001-Add-exclude-if-present-option-to-exclude.patch" +Content-Transfer-Encoding: 7bit +Content-Disposition: attachment; + filename="0001-Add-exclude-if-present-option-to-exclude.patch" + +>From 50fbcd08554c798456d1693d1a409badc12f6c8c Mon Sep 17 00:00:00 2001 +From: Michele Alessandrini +Date: Thu, 29 Jun 2017 12:00:09 +0200 +Subject: [PATCH 1/2] Add "exclude-if-present" option to exclude directories + containing give filename(s) + +--- + manual/en/060-backing-up.mdwn | 4 +++ + obnamlib/plugins/exclude_if_present_plugin.py | 52 +++++++++++++++++++++++++++ + without-tests | 1 + + 3 files changed, 57 insertions(+) + create mode 100644 obnamlib/plugins/exclude_if_present_plugin.py + +diff --git a/manual/en/060-backing-up.mdwn b/manual/en/060-backing-up.mdwn +index ee28d6e2..f0edb38f 100644 +--- a/manual/en/060-backing-up.mdwn ++++ b/manual/en/060-backing-up.mdwn +@@ -112,6 +112,10 @@ backups: + are usually not important to back up, and tagging the directory + can be easier than constructing a regular expression for + `--exclude`. ++* The `--exclude-if-present` setting excludes directories that contain a ++ given file name. This option can be used more than once to specify ++ several file names, if one of them is present the directory gets ++ excluded. + * The `--one-file-system` setting excludes any mount points and the + contents of the mounted filesystem. This is useful for skipping, + for example, virtual filesystems such as `/proc`, remote filesystems +diff --git a/obnamlib/plugins/exclude_if_present_plugin.py b/obnamlib/plugins/exclude_if_present_plugin.py +new file mode 100644 +index 00000000..3af81711 +--- /dev/null ++++ b/obnamlib/plugins/exclude_if_present_plugin.py +@@ -0,0 +1,52 @@ ++# Copyright (C) 2015 Lars Wirzenius ++# Copyright (C) 2017 Michele Alessandrini ++# ++# This program is free software: you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation, either version 3 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program. If not, see . ++ ++ ++import logging ++import os ++import stat ++ ++import obnamlib ++ ++ ++class ExcludeIfPresentPlugin(obnamlib.ObnamPlugin): ++ ++ def enable(self): ++ backup_group = obnamlib.option_group['backup'] = 'Backing up' ++ ++ self.app.settings.string_list( ++ ['exclude-if-present'], ++ 'exclude directories (and their subdirs) ' ++ 'that contain the given file (can be ' ++ 'used multiple times)', ++ group=backup_group) ++ ++ self.app.hooks.add_callback('config-loaded', self.config_loaded) ++ ++ def config_loaded(self): ++ if len(self.app.settings['exclude-if-present']) > 0: ++ self.app.hooks.add_callback('backup-exclude', self.exclude) ++ ++ def exclude(self, fs=None, pathname=None, stat_result=None, exclude=None, ++ **kwargs): ++ if stat.S_ISDIR(stat_result.st_mode): ++ for filename in self.app.settings['exclude-if-present']: ++ filename_path = os.path.join(pathname, filename) ++ if fs.exists(filename_path): ++ logging.debug('Excluding (exclude-if-present): %s', ++ pathname) ++ exclude[0] = True ++ break +diff --git a/without-tests b/without-tests +index f5979198..15d447bf 100644 +--- a/without-tests ++++ b/without-tests +@@ -21,6 +21,7 @@ obnamlib/plugins/dump_repo_plugin.py + obnamlib/plugins/encryption_plugin.py + obnamlib/plugins/exclude_caches_plugin.py + obnamlib/plugins/exclude_pathnames_plugin.py ++obnamlib/plugins/exclude_if_present_plugin.py + obnamlib/plugins/force_lock_plugin.py + obnamlib/plugins/forget_plugin.py + obnamlib/plugins/fsck_plugin.py +-- +2.13.2 + + +--------------D34B4AF69ADA0C644386FED6 +Content-Type: text/x-patch; + name="0002-improved-python-and-added-yarn-for-exclude-if-presen.patch" +Content-Transfer-Encoding: 7bit +Content-Disposition: attachment; + filename*0="0002-improved-python-and-added-yarn-for-exclude-if-presen.pa"; + filename*1="tch" + +>From 22f9c4450f331d2d057f357cd1e767c50a0ca0ae Mon Sep 17 00:00:00 2001 +From: Michele Alessandrini +Date: Mon, 3 Jul 2017 15:32:36 +0200 +Subject: [PATCH 2/2] improved python and added yarn for exclude-if-present + +--- + obnamlib/plugins/exclude_if_present_plugin.py | 5 +++-- + yarns/0030-basics.yarn | 26 ++++++++++++++++++++++++++ + 2 files changed, 29 insertions(+), 2 deletions(-) + +diff --git a/obnamlib/plugins/exclude_if_present_plugin.py b/obnamlib/plugins/exclude_if_present_plugin.py +index 3af81711..598306d1 100644 +--- a/obnamlib/plugins/exclude_if_present_plugin.py ++++ b/obnamlib/plugins/exclude_if_present_plugin.py +@@ -30,14 +30,15 @@ class ExcludeIfPresentPlugin(obnamlib.ObnamPlugin): + self.app.settings.string_list( + ['exclude-if-present'], + 'exclude directories (and their subdirs) ' +- 'that contain the given file (can be ' ++ 'that contain the given filename (can be ' + 'used multiple times)', ++ metavar='FILENAME', + group=backup_group) + + self.app.hooks.add_callback('config-loaded', self.config_loaded) + + def config_loaded(self): +- if len(self.app.settings['exclude-if-present']) > 0: ++ if self.app.settings['exclude-if-present']: + self.app.hooks.add_callback('backup-exclude', self.exclude) + + def exclude(self, fs=None, pathname=None, stat_result=None, exclude=None, +diff --git a/yarns/0030-basics.yarn b/yarns/0030-basics.yarn +index 226ec2a8..7590115f 100644 +--- a/yarns/0030-basics.yarn ++++ b/yarns/0030-basics.yarn +@@ -273,6 +273,32 @@ Time to backup. + AND user U restores their latest generation in repository R into X + THEN L, restored to X, matches manifest M + ++Exclude directories containing given filename(s) ++------------------------------------------------ ++ ++The `--exclude-if-present` option tells Obnam to exclude any ++directories containing the given filename. ++ ++ SCENARIO exclude directories containing given filename ++ GIVEN 1k of new data in directory L/wanted ++ AND 1k of new data in directory L/unwanted ++ AND file L/unwanted/_nobackup_ with permissions 664 ++ ++We'll now create the manifest, but remove `L/unwanted` (and files in ++`L/unwanted`) so that it matches what we need. We do it this instead of ++creating the manifest before `L/unwanted`, because creating `L/unwanted` ++changes the timestamp of `L`. ++ ++ AND a manifest of L in M ++ AND unwanted is removed from manifest M ++ ++Time to backup. ++ ++ AND user U sets configuration exclude-if-present to _nobackup_ ++ WHEN user U backs up directory L to repository R ++ AND user U restores their latest generation in repository R into X ++ THEN L, restored to X, matches manifest M ++ + Excluded, already backed up files, are not included in next generation + ---------------------------------------------------------------------- + +-- +2.13.2 + + +--------------D34B4AF69ADA0C644386FED6 +Content-Type: text/plain; charset="us-ascii" +MIME-Version: 1.0 +Content-Transfer-Encoding: 7bit +Content-Disposition: inline + +_______________________________________________ +obnam-dev mailing list +obnam-dev@obnam.org +http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org + +--------------D34B4AF69ADA0C644386FED6-- + -- cgit v1.2.1