imported mails

14 files changed, 1356 insertions, 0 deletions

diff --git a/tickets/858f91e5841e407699ff9abd95089c64/Maildir/cur/.this-dir-not-empty/.empty/empty-file b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/cur/.this-dir-not-empty/.empty/empty-file
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/cur/.this-dir-not-empty/.empty/empty-file
diff --git a/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/.this-dir-not-empty/.empty/empty-file b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/.this-dir-not-empty/.empty/empty-file
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/.this-dir-not-empty/.empty/empty-file
diff --git a/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998994.M508337P17339Q7.exolobe1 b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998994.M508337P17339Q7.exolobe1
new file mode 100644
index 0000000..79556cd
--- /dev/null
+++ b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998994.M508337P17339Q7.exolobe1
@@ -0,0 +1,100 @@
+Return-Path: <obnam-dev-bounces@obnam.org>
+X-Original-To: distix@pieni.net
+Delivered-To: distix@pieni.net
+Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16])
+	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+	(No client certificate requested)
+	by pieni.net (Postfix) with ESMTPS id F1BD22C3E6
+	for <distix@pieni.net>; Sat, 29 Aug 2015 13:21:44 +0200 (CEST)
+Received: from platypus.pepperfish.net (unknown [10.112.100.20])
+	by bagpuss.pepperfish.net (Postfix) with ESMTP id 81EE0EC0;
+	Sat, 29 Aug 2015 12:21:44 +0100 (BST)
+Received: from ip6-localhost ([::1] helo=platypus.pepperfish.net)
+	by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+	id 1ZVeCe-0000fB-Cs; Sat, 29 Aug 2015 12:21:44 +0100
+Received: from inmail0 ([10.112.100.10] helo=mx0.pepperfish.net)
+ by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+ id 1ZVeCc-0000f5-LU
+ for <obnam-dev@obnam.org>; Sat, 29 Aug 2015 12:21:42 +0100
+Received: from pieni.net ([95.142.166.37] ident=postfix)
+ by mx0.pepperfish.net with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256)
+ (Exim 4.80) (envelope-from <liw@liw.fi>) id 1ZVeCZ-0004vh-SE
+ for obnam-dev@obnam.org; Sat, 29 Aug 2015 12:21:42 +0100
+Received: from exolobe1.liw.fi (82-181-8-107.bb.dnainternet.fi [82.181.8.107])
+ (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+ (No client certificate requested)
+ by pieni.net (Postfix) with ESMTPSA id BADCF2C3E6;
+ Sat, 29 Aug 2015 13:21:33 +0200 (CEST)
+Received: from exolobe1.liw.fi (localhost [127.0.0.1])
+ by exolobe1.liw.fi (Postfix) with ESMTPS id 1633A4033C;
+ Sat, 29 Aug 2015 14:21:33 +0300 (EEST)
+Date: Sat, 29 Aug 2015 14:21:32 +0300
+From: Lars Wirzenius <liw@liw.fi>
+To: Antoine Brenner <brenner+obnam@gymglish.com>
+Message-ID: <20150829112131.GE2246@exolobe1.liw.fi>
+References: <20150404201239.GL6211@exolobe1.liw.fi>
+ <CAExbbMyWptQkOUpyO7TDD=_hgfdi5RjGSPn0HDVFA+jkW+YCsg@mail.gmail.com>
+ <20150405160221.GM6211@exolobe1.liw.fi>
+ <20150405193823.Horde.WbSqs4Wyz21q7mOCcp522w1@szepe.net>
+ <CAPzomaPO78aQTjyUxzzUMjYui51=WbxOZJxq1KDgTDp=ccr+fw@mail.gmail.com>
+MIME-Version: 1.0
+Content-Type: text/plain; charset=us-ascii
+Content-Disposition: inline
+In-Reply-To: <CAPzomaPO78aQTjyUxzzUMjYui51=WbxOZJxq1KDgTDp=ccr+fw@mail.gmail.com>
+User-Agent: Mutt/1.5.23 (2014-03-12)
+X-Spam-Score: -3.4
+X-Spam-Score-int: -33
+X-Spam-Bar: ---
+X-Scanned-By: pepperfish.net, Sat, 29 Aug 2015 12:21:42 +0100
+X-Spam-Report: Content analysis details: (-3.4 points)
+ pts rule name              description
+ ---- ---------------------- --------------------------------------------------
+ -1.0 PPF_USER_AGENT_MUTT    User-Agent: contains Mutt (Mutt isn't a spam
+ tool) -0.5 PPF_USER_AGENT         User-Agent: exists
+ -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
+ [score: 0.0000]
+X-ACL-Warn: message may be spam
+X-Scan-Signature: e606bea56426b7460dd1a96e279d41e5
+Cc: =?iso-8859-1?Q?Sz=E9pe?= Viktor <viktor@szepe.net>, obnam-dev@obnam.org
+Subject: Re: Fundamental problems in Obnam and what should be done about them
+X-BeenThere: obnam-dev@obnam.org
+X-Mailman-Version: 2.1.5
+Precedence: list
+List-Id: Obnam development discussions <obnam-dev-obnam.org>
+List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
+List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
+List-Post: <mailto:obnam-dev@obnam.org>
+List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
+List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=subscribe>
+Sender: obnam-dev-bounces@obnam.org
+Errors-To: obnam-dev-bounces@obnam.org
+
+On Wed, Apr 08, 2015 at 01:04:07PM +0200, Antoine Brenner wrote:
+> As mentioned by Itamar Turner-Trauring on this very mailing-list in
+> Dec. 2013, better deduplication than eg rsync rolling checksum
+> algorithm can most probably be achieved by supporting variable length
+> chunks. (cf bottom left paragraph of page 4 of
+> http://www.news.cs.nyu.edu/~jinyang/sp07/papers/lbfs.pdf . What they
+> do is called Rabin fingerprinting [1])
+
+This is something I want to do, but it requires making various Obnam
+data structures much faster than they are now. This is because the
+number of file data chunks will grow a lot, and without better data
+structures Obnam will struggle even more than it currently does.
+
+> So, at this design stage, I think that making sure that all chunks can
+> have a variable length (even eg within one big file) would be a good
+> idea.
+
+This has, in fact, always been the case. It's not done at the moment,
+but that's purely because of the performance aspect.
+
+-- 
+sic transit disci mundi, ergo obnam
+
+_______________________________________________
+obnam-dev mailing list
+obnam-dev@obnam.org
+http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org
diff --git a/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998995.M106667P17339Q24.exolobe1 b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998995.M106667P17339Q24.exolobe1
new file mode 100644
index 0000000..873175e
--- /dev/null
+++ b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998995.M106667P17339Q24.exolobe1
@@ -0,0 +1,114 @@
+Return-Path: <obnam-dev-bounces@obnam.org>
+X-Original-To: distix@pieni.net
+Delivered-To: distix@pieni.net
+Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16])
+	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+	(No client certificate requested)
+	by pieni.net (Postfix) with ESMTPS id 2B8882C39F
+	for <distix@pieni.net>; Sat, 29 Aug 2015 13:16:07 +0200 (CEST)
+Received: from platypus.pepperfish.net (unknown [10.112.100.20])
+	by bagpuss.pepperfish.net (Postfix) with ESMTP id 96A71EC0;
+	Sat, 29 Aug 2015 12:16:06 +0100 (BST)
+Received: from ip6-localhost ([::1] helo=platypus.pepperfish.net)
+	by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+	id 1ZVe7C-0000Ir-Ek; Sat, 29 Aug 2015 12:16:06 +0100
+Received: from inmail0 ([10.112.100.10] helo=mx0.pepperfish.net)
+ by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+ id 1ZVe7A-0000Ik-BQ
+ for <obnam-dev@obnam.org>; Sat, 29 Aug 2015 12:16:04 +0100
+Received: from pieni.net ([95.142.166.37] ident=postfix)
+ by mx0.pepperfish.net with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256)
+ (Exim 4.80) (envelope-from <liw@liw.fi>) id 1ZVe77-0004mq-Oy
+ for obnam-dev@obnam.org; Sat, 29 Aug 2015 12:16:04 +0100
+Received: from exolobe1.liw.fi (82-181-8-107.bb.dnainternet.fi [82.181.8.107])
+ (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+ (No client certificate requested)
+ by pieni.net (Postfix) with ESMTPSA id A5CBA2C39F;
+ Sat, 29 Aug 2015 13:15:54 +0200 (CEST)
+Received: from exolobe1.liw.fi (localhost [127.0.0.1])
+ by exolobe1.liw.fi (Postfix) with ESMTPS id E54454033C;
+ Sat, 29 Aug 2015 14:15:53 +0300 (EEST)
+Date: Sat, 29 Aug 2015 14:15:52 +0300
+From: Lars Wirzenius <liw@liw.fi>
+To: =?iso-8859-1?Q?Sz=E9pe?= Viktor <viktor@szepe.net>
+Message-ID: <20150829111552.GD2246@exolobe1.liw.fi>
+References: <20150404201239.GL6211@exolobe1.liw.fi>
+ <CAExbbMyWptQkOUpyO7TDD=_hgfdi5RjGSPn0HDVFA+jkW+YCsg@mail.gmail.com>
+ <20150405160221.GM6211@exolobe1.liw.fi>
+ <20150405193823.Horde.WbSqs4Wyz21q7mOCcp522w1@szepe.net>
+MIME-Version: 1.0
+Content-Type: text/plain; charset=iso-8859-1
+Content-Disposition: inline
+Content-Transfer-Encoding: 8bit
+In-Reply-To: <20150405193823.Horde.WbSqs4Wyz21q7mOCcp522w1@szepe.net>
+User-Agent: Mutt/1.5.23 (2014-03-12)
+X-Spam-Score: -2.4
+X-Spam-Score-int: -23
+X-Spam-Bar: --
+X-Scanned-By: pepperfish.net, Sat, 29 Aug 2015 12:16:04 +0100
+X-Spam-Report: Content analysis details: (-2.4 points)
+ pts rule name              description
+ ---- ---------------------- --------------------------------------------------
+ 1.0 PPF_RU_URL             BODY: Contains a URL with a .ru domain
+ -1.0 PPF_USER_AGENT_MUTT    User-Agent: contains Mutt (Mutt isn't a spam
+ tool) -0.5 PPF_USER_AGENT         User-Agent: exists
+ -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
+ [score: 0.0000]
+X-ACL-Warn: message may be spam
+X-Scan-Signature: 8887c701ef4198506b295ef73f0e38b1
+Cc: obnam-dev@obnam.org
+Subject: Re: Fundamental problems in Obnam and what should be done about them
+X-BeenThere: obnam-dev@obnam.org
+X-Mailman-Version: 2.1.5
+Precedence: list
+List-Id: Obnam development discussions <obnam-dev-obnam.org>
+List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
+List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
+List-Post: <mailto:obnam-dev@obnam.org>
+List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
+List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=subscribe>
+Sender: obnam-dev-bounces@obnam.org
+Errors-To: obnam-dev-bounces@obnam.org
+
+Hi, I'm returning to this old thread after months. Sorry about the
+late delays, but that's the reality of a hobby project that one does
+in one's free time.
+
+On Sun, Apr 05, 2015 at 07:38:23PM +0200, Szépe Viktor wrote:
+> Please consider Content-Aware compression and deduplication, thus a per file
+> decision, like in pcompress
+> http://moinakg.github.io/pcompress/
+> JPG and MP3 compression: http://packjpg.encode.ru/?page_id=17
+
+Sure, that kind of thing should be farily simple. It's also not
+related to the repository format, and can be handled at a higher
+layer. In other words, it's possible to do this completely as a plugin
+that just transforms data at backup and restore times. I think.
+
+I think something like this might be added:
+
+    whole-file-compresssion = flac:\.wav$, xz:\.log$, gzip:\.img$
+
+This would compress WAV files with flac, log files with xz, and disk
+images with gzip.
+
+However, I'm not going to be working on this, and will hope that
+someone else writes a clean patch (with tests) to do this.
+
+> The second thing is a weird question.
+> Do you think it is possible to implement duplicity's backends in obnam?
+> duplicity supports numerous network storage types.
+
+I doubt that will be doable in a way that satisfies me. I'd rather
+implement support for those backends in Obnam, in a way that includes
+proper testing so that the backends are supportable.
+
+-- 
+sic transit disci mundi, ergo obnam
+
+_______________________________________________
+obnam-dev mailing list
+obnam-dev@obnam.org
+http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org
diff --git a/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998995.M204968P17339Q29.exolobe1 b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998995.M204968P17339Q29.exolobe1
new file mode 100644
index 0000000..32f090d
--- /dev/null
+++ b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998995.M204968P17339Q29.exolobe1
@@ -0,0 +1,187 @@
+Return-Path: <obnam-dev-bounces@obnam.org>
+X-Original-To: distix@pieni.net
+Delivered-To: distix@pieni.net
+Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16])
+	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+	(No client certificate requested)
+	by pieni.net (Postfix) with ESMTPS id 11ABA2DFF6
+	for <distix@pieni.net>; Sat,  4 Apr 2015 22:15:01 +0200 (CEST)
+Received: from platypus.pepperfish.net (unknown [10.112.100.20])
+	by bagpuss.pepperfish.net (Postfix) with ESMTP id 9FEA0BAE;
+	Sat,  4 Apr 2015 21:15:00 +0100 (BST)
+Received: from localhost ([::1] helo=platypus.pepperfish.net)
+	by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+	id 1YeUT6-00017d-Hr; Sat, 04 Apr 2015 21:15:00 +0100
+Received: from inmail ([10.112.100.10] helo=mx0.pepperfish.net)
+ by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+ id 1YeUT5-00017P-Ix
+ for <obnam-dev@obnam.org>; Sat, 04 Apr 2015 21:14:59 +0100
+Received: from xvm-166-37.ghst.net
+ ([95.142.166.37] helo=pieni.net ident=postfix)
+ by mx0.pepperfish.net with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256)
+ (Exim 4.80) (envelope-from <liw@liw.fi>) id 1YeUT3-0000bv-GV
+ for obnam-dev@obnam.org; Sat, 04 Apr 2015 21:14:59 +0100
+Received: from exolobe1.liw.fi (82-181-8-107.bb.dnainternet.fi [82.181.8.107])
+ (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+ (No client certificate requested)
+ by pieni.net (Postfix) with ESMTPSA id 2BAA42E009
+ for <obnam-dev@obnam.org>; Sat,  4 Apr 2015 22:14:54 +0200 (CEST)
+Received: from exolobe1.liw.fi (localhost [127.0.0.1])
+ by exolobe1.liw.fi (Postfix) with ESMTPS id 9AC4A5522C
+ for <obnam-dev@obnam.org>; Sat,  4 Apr 2015 23:12:40 +0300 (EEST)
+Date: Sat, 4 Apr 2015 23:12:39 +0300
+From: Lars Wirzenius <liw@liw.fi>
+To: Obnam development <obnam-dev@obnam.org>
+Message-ID: <20150404201239.GL6211@exolobe1.liw.fi>
+MIME-Version: 1.0
+Content-Type: text/plain; charset=us-ascii
+Content-Disposition: inline
+User-Agent: Mutt/1.5.23 (2014-03-12)
+Subject: Fundamental problems in Obnam and what should be done about them
+X-BeenThere: obnam-dev@obnam.org
+X-Mailman-Version: 2.1.5
+Precedence: list
+List-Id: Obnam development discussions <obnam-dev-obnam.org>
+List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
+List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
+List-Post: <mailto:obnam-dev@obnam.org>
+List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
+List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=subscribe>
+Sender: obnam-dev-bounces@obnam.org
+Errors-To: obnam-dev-bounces@obnam.org
+
+I want to work on some fundamental design problems in Obnam. While
+Obnam works well enough for a whole bunch of people, it's also clearly
+not good enough for a lot more people. I want to address these issues
+honestly, not hiding any problems.
+
+Here's my list of big picture problems:
+
+* Performance is inadequate. I've always favoured correctness over
+  performance, when developing Obnam, and I have no regrets about
+  that. However, it's also true that a correct solution isn't good
+  enough, if it's so slow it can't be used. The performance goal needs
+  to be quantified, and below is my first cut at that.
+
+    - Backing up the first generation should be take at most 200% of
+      the time it takes to rsync the data, when Obnam and rsync are
+      writing to the same repository via the same communications
+      channel. If Obnam accesses a remote server over sftp, rsync
+      should do the same thing.
+
+    - Backing up a second generation, without changes, should take at
+      most 200% of rsyncing the data. Again, this should happen over
+      the same communications channel.
+
+    - As a start for benchmarks, I'm thinking there should be two: one
+      with just a single, large file of 100 gigabytes (little or no
+      duplication in the data), the other should be 100 thousand empty
+      files. Possibly add a couple of zeroes to those numbers, but
+      that will exceed the capabilities of the hardware I have for
+      benchmarking.
+
+    - Performance is about time, memory usage, CPU usage, repository
+      size, and bandwidth use. All of these should be measured, but
+      time is the most crucial one, I think, which is why I frame the
+      goal in terms of it.
+
+* Reliability isn't good enough. When I started using the
+  copy-on-write B-trees that became known as Larch, they were a good
+  idea. However, in order to get performance out Larch, its
+  implementation became quite intricate. Intricate is a swearword. I
+  don't like intricate. I would like to bring simplicity and do away
+  with Larch.
+
+    - The current repository format ("6") relies on Larch. Larch will
+      stay around until Obnam can drop that format. However, any new
+      repository formats shouldn't use Larch.
+
+    - Larch is not the only reliability concern, but it is perhaps the
+      biggest single component causing me to worry. Another one is the
+      lack of a good fsck that can not just find, but also fix
+      problems. However, this ties into Larch, since fixing a
+      repository that has two abstraction levels (Larch B-trees and
+      then the structure on top of them that Obnam adds) makes things
+      more complicated. Performance affects this as well: fsck is so
+      slow, few people even let it finish, and so fsck is currently
+      basically pointless.
+
+    - As an extra aspect, it's possible that fsck is a thing someone
+      else should write, since they are not so blind to flaws in the
+      structures that Obnam uses, and the code that implements them.
+
+* Code isn't simple enough. I've been working on Obnam for a long time
+  now, and I've not always kept the code clean and simple. It shows,
+  and a very real consequence is that some changes are much harder to
+  do than they should be.
+
+    - Luckily, the test suite is fairly adequate, making it possible 
+      to refactor with confidence.
+
+* Inadequate extensibility in the repository format. The current
+  format is not designed to be extensible. In fact, any changes to it
+  will cause older versions of Obnam to either not work, or to mess up
+  the repository data structures.
+
+    - As a concrete example, I'd like to add support for the chattr
+      attributes used in the ext2, ext3, and ext4 filesystems.
+      However, doing that would require adding a new piece of
+      metadata, and that would break older versions of Obnam, and
+      that's not acceptable. However, a format designed for
+      extensiblity would make this a non-problem.
+
+Here's my current thinking on how to address these problems:
+
+* Set up automated benchmarking, following the master branch. Do
+  measurements to see where bottlenecks are. This is a required
+  pre-condition to doing performance work.
+
+    - There is some start of this in the source tree already.
+
+    - I would like to be able to run the benchmarks on a private
+      machine, and have the results be a set of static files that can
+      be rsync'd to a public web server for everyone to see.
+
+* Design a new repository format, for both extensiblity and speed. The
+  new format should keep down the number of files in the repository,
+  and allow cacheability by avoiding overwriting of the same files.
+  The new format should allow new backups to be streamed to the
+  repository, without a lot of back-and-forth communication. Data
+  should be stored in something akin to JSON or YAML, so new fields
+  can be added easily.
+
+* Think very hard about how the new format can be made resilient to
+  software bugs and accidental corruption in the repository, and how
+  problems can be made fixable by fsck.
+
+* Refactor existing code to be simpler, cleaner, more easily
+  understood. Possibly add some architecture and implementation
+  documentation to make it easier to understand the internals of
+  Obnam.
+
+Those are quite big goals, requiring quite a lot of work, though they
+fit into a few lines of text. I'd very much like help.
+
+As a first step, what do others on obnam-dev think about the above? Do
+you agree with the problems I've raised? Have I missed some important
+problems? What could you to help fix the problems?
+
+Would you like, say, help improve the Obnam benchmark suite (see
+obnam-benchmark and obnam-benchmark-summary in the source tree), to be
+more useful and to present the results in a better form? I am not good
+with gathering measurements or presenting them, so help with this
+would be very useful.
+
+I will, hopefully soon, finish up an email outlining my more detailed
+thoughts on a new repository format, and will be sending it to
+obnam-dev, but let the discussion begin already.
+
+-- 
+Happiness is a tested backup.
+
+_______________________________________________
+obnam-dev mailing list
+obnam-dev@obnam.org
+http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org
diff --git a/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998995.M95218P17339Q23.exolobe1 b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998995.M95218P17339Q23.exolobe1
new file mode 100644
index 0000000..79556cd
--- /dev/null
+++ b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998995.M95218P17339Q23.exolobe1
@@ -0,0 +1,100 @@
+Return-Path: <obnam-dev-bounces@obnam.org>
+X-Original-To: distix@pieni.net
+Delivered-To: distix@pieni.net
+Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16])
+	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+	(No client certificate requested)
+	by pieni.net (Postfix) with ESMTPS id F1BD22C3E6
+	for <distix@pieni.net>; Sat, 29 Aug 2015 13:21:44 +0200 (CEST)
+Received: from platypus.pepperfish.net (unknown [10.112.100.20])
+	by bagpuss.pepperfish.net (Postfix) with ESMTP id 81EE0EC0;
+	Sat, 29 Aug 2015 12:21:44 +0100 (BST)
+Received: from ip6-localhost ([::1] helo=platypus.pepperfish.net)
+	by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+	id 1ZVeCe-0000fB-Cs; Sat, 29 Aug 2015 12:21:44 +0100
+Received: from inmail0 ([10.112.100.10] helo=mx0.pepperfish.net)
+ by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+ id 1ZVeCc-0000f5-LU
+ for <obnam-dev@obnam.org>; Sat, 29 Aug 2015 12:21:42 +0100
+Received: from pieni.net ([95.142.166.37] ident=postfix)
+ by mx0.pepperfish.net with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256)
+ (Exim 4.80) (envelope-from <liw@liw.fi>) id 1ZVeCZ-0004vh-SE
+ for obnam-dev@obnam.org; Sat, 29 Aug 2015 12:21:42 +0100
+Received: from exolobe1.liw.fi (82-181-8-107.bb.dnainternet.fi [82.181.8.107])
+ (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+ (No client certificate requested)
+ by pieni.net (Postfix) with ESMTPSA id BADCF2C3E6;
+ Sat, 29 Aug 2015 13:21:33 +0200 (CEST)
+Received: from exolobe1.liw.fi (localhost [127.0.0.1])
+ by exolobe1.liw.fi (Postfix) with ESMTPS id 1633A4033C;
+ Sat, 29 Aug 2015 14:21:33 +0300 (EEST)
+Date: Sat, 29 Aug 2015 14:21:32 +0300
+From: Lars Wirzenius <liw@liw.fi>
+To: Antoine Brenner <brenner+obnam@gymglish.com>
+Message-ID: <20150829112131.GE2246@exolobe1.liw.fi>
+References: <20150404201239.GL6211@exolobe1.liw.fi>
+ <CAExbbMyWptQkOUpyO7TDD=_hgfdi5RjGSPn0HDVFA+jkW+YCsg@mail.gmail.com>
+ <20150405160221.GM6211@exolobe1.liw.fi>
+ <20150405193823.Horde.WbSqs4Wyz21q7mOCcp522w1@szepe.net>
+ <CAPzomaPO78aQTjyUxzzUMjYui51=WbxOZJxq1KDgTDp=ccr+fw@mail.gmail.com>
+MIME-Version: 1.0
+Content-Type: text/plain; charset=us-ascii
+Content-Disposition: inline
+In-Reply-To: <CAPzomaPO78aQTjyUxzzUMjYui51=WbxOZJxq1KDgTDp=ccr+fw@mail.gmail.com>
+User-Agent: Mutt/1.5.23 (2014-03-12)
+X-Spam-Score: -3.4
+X-Spam-Score-int: -33
+X-Spam-Bar: ---
+X-Scanned-By: pepperfish.net, Sat, 29 Aug 2015 12:21:42 +0100
+X-Spam-Report: Content analysis details: (-3.4 points)
+ pts rule name              description
+ ---- ---------------------- --------------------------------------------------
+ -1.0 PPF_USER_AGENT_MUTT    User-Agent: contains Mutt (Mutt isn't a spam
+ tool) -0.5 PPF_USER_AGENT         User-Agent: exists
+ -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
+ [score: 0.0000]
+X-ACL-Warn: message may be spam
+X-Scan-Signature: e606bea56426b7460dd1a96e279d41e5
+Cc: =?iso-8859-1?Q?Sz=E9pe?= Viktor <viktor@szepe.net>, obnam-dev@obnam.org
+Subject: Re: Fundamental problems in Obnam and what should be done about them
+X-BeenThere: obnam-dev@obnam.org
+X-Mailman-Version: 2.1.5
+Precedence: list
+List-Id: Obnam development discussions <obnam-dev-obnam.org>
+List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
+List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
+List-Post: <mailto:obnam-dev@obnam.org>
+List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
+List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=subscribe>
+Sender: obnam-dev-bounces@obnam.org
+Errors-To: obnam-dev-bounces@obnam.org
+
+On Wed, Apr 08, 2015 at 01:04:07PM +0200, Antoine Brenner wrote:
+> As mentioned by Itamar Turner-Trauring on this very mailing-list in
+> Dec. 2013, better deduplication than eg rsync rolling checksum
+> algorithm can most probably be achieved by supporting variable length
+> chunks. (cf bottom left paragraph of page 4 of
+> http://www.news.cs.nyu.edu/~jinyang/sp07/papers/lbfs.pdf . What they
+> do is called Rabin fingerprinting [1])
+
+This is something I want to do, but it requires making various Obnam
+data structures much faster than they are now. This is because the
+number of file data chunks will grow a lot, and without better data
+structures Obnam will struggle even more than it currently does.
+
+> So, at this design stage, I think that making sure that all chunks can
+> have a variable length (even eg within one big file) would be a good
+> idea.
+
+This has, in fact, always been the case. It's not done at the moment,
+but that's purely because of the performance aspect.
+
+-- 
+sic transit disci mundi, ergo obnam
+
+_______________________________________________
+obnam-dev mailing list
+obnam-dev@obnam.org
+http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org
diff --git a/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998996.M41193P17339Q43.exolobe1 b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998996.M41193P17339Q43.exolobe1
new file mode 100644
index 0000000..80d103a
--- /dev/null
+++ b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998996.M41193P17339Q43.exolobe1
@@ -0,0 +1,110 @@
+Return-Path: <obnam-dev-bounces@obnam.org>
+X-Original-To: distix@pieni.net
+Delivered-To: distix@pieni.net
+Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16])
+	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+	(No client certificate requested)
+	by pieni.net (Postfix) with ESMTPS id 92CA620BB8
+	for <distix@pieni.net>; Sun,  5 Apr 2015 18:04:47 +0200 (CEST)
+Received: from platypus.pepperfish.net (unknown [10.112.100.20])
+	by bagpuss.pepperfish.net (Postfix) with ESMTP id 223EF16E5;
+	Sun,  5 Apr 2015 17:04:47 +0100 (BST)
+Received: from ip6-localhost ([::1] helo=platypus.pepperfish.net)
+	by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+	id 1Yen2V-0008JP-1A; Sun, 05 Apr 2015 17:04:47 +0100
+Received: from inmail ([10.112.100.10] helo=mx0.pepperfish.net)
+ by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+ id 1Yen2S-0008JA-Hw
+ for <obnam-dev@obnam.org>; Sun, 05 Apr 2015 17:04:44 +0100
+Received: from xvm-166-37.ghst.net
+ ([95.142.166.37] helo=pieni.net ident=postfix)
+ by mx0.pepperfish.net with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256)
+ (Exim 4.80) (envelope-from <liw@liw.fi>) id 1Yen2Q-0001Cb-5o
+ for obnam-dev@obnam.org; Sun, 05 Apr 2015 17:04:44 +0100
+Received: from exolobe1.liw.fi (82-181-8-107.bb.dnainternet.fi [82.181.8.107])
+ (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+ (No client certificate requested)
+ by pieni.net (Postfix) with ESMTPSA id 3B5AF20BB8;
+ Sun,  5 Apr 2015 18:04:37 +0200 (CEST)
+Received: from exolobe1.liw.fi (localhost [127.0.0.1])
+ by exolobe1.liw.fi (Postfix) with ESMTPS id 1A44155232;
+ Sun,  5 Apr 2015 19:02:23 +0300 (EEST)
+Date: Sun, 5 Apr 2015 19:02:21 +0300
+From: Lars Wirzenius <liw@liw.fi>
+To: SanskritFritz <sanskritfritz@gmail.com>
+Message-ID: <20150405160221.GM6211@exolobe1.liw.fi>
+References: <20150404201239.GL6211@exolobe1.liw.fi>
+ <CAExbbMyWptQkOUpyO7TDD=_hgfdi5RjGSPn0HDVFA+jkW+YCsg@mail.gmail.com>
+MIME-Version: 1.0
+Content-Type: text/plain; charset=us-ascii
+Content-Disposition: inline
+In-Reply-To: <CAExbbMyWptQkOUpyO7TDD=_hgfdi5RjGSPn0HDVFA+jkW+YCsg@mail.gmail.com>
+User-Agent: Mutt/1.5.23 (2014-03-12)
+X-Spam-Score: -3.4
+X-Spam-Score-int: -33
+X-Spam-Bar: ---
+X-Scanned-By: pepperfish.net, Sun, 05 Apr 2015 17:04:44 +0100
+X-Spam-Report: Content analysis details: (-3.4 points)
+ pts rule name              description
+ ---- ---------------------- --------------------------------------------------
+ -1.0 PPF_USER_AGENT_MUTT    User-Agent: contains Mutt (Mutt isn't a spam
+ tool) -0.5 PPF_USER_AGENT         User-Agent: exists
+ -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
+ [score: 0.0000]
+X-ACL-Warn: message may be spam
+Cc: Obnam development <obnam-dev@obnam.org>
+Subject: Re: Fundamental problems in Obnam and what should be done about them
+X-BeenThere: obnam-dev@obnam.org
+X-Mailman-Version: 2.1.5
+Precedence: list
+List-Id: Obnam development discussions <obnam-dev-obnam.org>
+List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
+List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
+List-Post: <mailto:obnam-dev@obnam.org>
+List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
+List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=subscribe>
+Sender: obnam-dev-bounces@obnam.org
+Errors-To: obnam-dev-bounces@obnam.org
+
+On Sun, Apr 05, 2015 at 05:35:58PM +0200, SanskritFritz wrote:
+> Hi, thanks for those thoughts. I don't know if you have plans to address
+> this problem: http://obnam.org/faq/dedup/
+> Apparently Attic and Zbackup have successfully solved that with a "rolling
+> checksum" algorithm which is beyond my knowledge now. As far as I
+> understand, they use a rolling window on the files instead of fixed chunks,
+> but I can be entirelly wrong.
+
+I think the way Obnam breaks up a file into chunks needs to be
+changed, and the rolling checksum approach of Attic and Zbackup seems
+like a good idea. I've mainly been thinking about things that affect
+the backup repository, so far, and it turns out that the chunking
+algorithm is independent of that.
+
+The way to do this is to change the backup_file_contents method in the
+backup plugin to not hardcode the way chunking happens, but instead
+move that into a hook. Different plugins can then provide different
+algorithms, and one of the plugins gets to enable its algorithm. This
+makes it possible to experiment with different approaches, and moves
+the chunk decision making into its own code module, where it doesn't
+obscure other parts.
+
+Being able to choose the chunking algorithm seems useful to me, since
+different data has different needs. for example, if Obnam were to
+learn how to backup raw disks, it's possible that the only useful
+chunk would be the filesystem block size. Other people might prefer
+the current approach which is very light on CPU usage. The default
+should probably be the rolling checksum approach.
+
+> I can help with testing new solutions.
+
+Excellent.
+
+-- 
+Happiness is a tested backup.
+
+_______________________________________________
+obnam-dev mailing list
+obnam-dev@obnam.org
+http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org
diff --git a/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998997.M912960P17339Q76.exolobe1 b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998997.M912960P17339Q76.exolobe1
new file mode 100644
index 0000000..0de347c
--- /dev/null
+++ b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998997.M912960P17339Q76.exolobe1
@@ -0,0 +1,105 @@
+Return-Path: <obnam-dev-bounces@obnam.org>
+X-Original-To: distix@pieni.net
+Delivered-To: distix@pieni.net
+Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16])
+	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+	(No client certificate requested)
+	by pieni.net (Postfix) with ESMTPS id 9A4282C1EB
+	for <distix@pieni.net>; Sat, 29 Aug 2015 14:04:29 +0200 (CEST)
+Received: from platypus.pepperfish.net (unknown [10.112.100.20])
+	by bagpuss.pepperfish.net (Postfix) with ESMTP id C449FEBE;
+	Sat, 29 Aug 2015 13:04:28 +0100 (BST)
+Received: from ip6-localhost ([::1] helo=platypus.pepperfish.net)
+	by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+	id 1ZVes0-0003mW-MY; Sat, 29 Aug 2015 13:04:28 +0100
+Received: from inmail0 ([10.112.100.10] helo=mx0.pepperfish.net)
+ by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+ id 1ZVerz-0003mG-ET
+ for <obnam-dev@obnam.org>; Sat, 29 Aug 2015 13:04:27 +0100
+Received: from mail-io0-f174.google.com ([209.85.223.174])
+ by mx0.pepperfish.net with esmtps (TLS1.2:RSA_ARCFOUR_SHA1:128)
+ (Exim 4.80) (envelope-from <jean.jordaan@gmail.com>)
+ id 1ZVerx-00064R-GT
+ for obnam-dev@obnam.org; Sat, 29 Aug 2015 13:04:27 +0100
+Received: by ioej130 with SMTP id j130so35688247ioe.3
+ for <obnam-dev@obnam.org>; Sat, 29 Aug 2015 05:04:13 -0700 (PDT)
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
+ h=mime-version:in-reply-to:references:from:date:message-id:subject:to
+ :content-type; bh=pIdhvnV1vHi2R3crxi4MS5kuR8GLu+6lDselk7NM2EA=;
+ b=hvsVPBJkWX0xtEpppmT+WEhCyOsrGS69s/ZENJYpHFcraxn/Z4M52RgRgT0M90rYKp
+ VQrqGyZUpDAEPmoi/Gr68F8E1smK57SZIP3IhWM0f1ZsKmFp/m1ywJ05hdw+0b6XuquV
+ ZsfceQTCvFQ3M3xG3ab/cWTKJ9EVug474HTR0veOJK/AqzFK0Mrjvki5Hu5WpbF3ZjHQ
+ sfnNvHdmOx4j7HZSLsipTysVP2q1zoK4so7nYrLDTwyx9/GxMKzY6Q6B+aH342tmM/rw
+ ppBl4gLsdmfqadFXvt6GnzT+3pvqap/dbV2ZI5q69V7EorUIlHWGKKOCXgYx+Zm5fvRT
+ B4RQ==
+X-Received: by 10.107.41.76 with SMTP id p73mr16829359iop.58.1440849853100;
+ Sat, 29 Aug 2015 05:04:13 -0700 (PDT)
+MIME-Version: 1.0
+Received: by 10.79.37.15 with HTTP; Sat, 29 Aug 2015 05:03:43 -0700 (PDT)
+In-Reply-To: <20150829111552.GD2246@exolobe1.liw.fi>
+References: <20150404201239.GL6211@exolobe1.liw.fi>
+ <CAExbbMyWptQkOUpyO7TDD=_hgfdi5RjGSPn0HDVFA+jkW+YCsg@mail.gmail.com>
+ <20150405160221.GM6211@exolobe1.liw.fi>
+ <20150405193823.Horde.WbSqs4Wyz21q7mOCcp522w1@szepe.net>
+ <20150829111552.GD2246@exolobe1.liw.fi>
+From: Jean Jordaan <jean.jordaan@gmail.com>
+Date: Sat, 29 Aug 2015 15:03:43 +0300
+Message-ID: <CAJqnznA_ZGznKuBjuuUK-P5Ccow3iB14iLnsQX74VVUmi+jDgQ@mail.gmail.com>
+To: obnam-dev@obnam.org
+Content-Type: text/plain; charset=UTF-8
+X-Spam-Score: -0.0
+X-Spam-Score-int: 0
+X-Spam-Bar: /
+X-Scanned-By: pepperfish.net, Sat, 29 Aug 2015 13:04:27 +0100
+X-Spam-Report: Content analysis details: (-0.0 points)
+ pts rule name              description
+ ---- ---------------------- --------------------------------------------------
+ -0.7 RCVD_IN_DNSWL_LOW      RBL: Sender listed at http://www.dnswl.org/, low
+ trust [209.85.223.174 listed in list.dnswl.org]
+ 0.5 PPF_RECEIVED_HTTP      Received header mentions http
+ 1.0 PPF_FROM_CONTAINS_MAIL The From header contains 'mail'
+ 1.2 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
+ (jean.jordaan[at]gmail.com)
+ -0.0 SPF_PASS               SPF: sender matches SPF record
+ -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
+ [score: 0.0000]
+ -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's
+ domain
+ 0.1 DKIM_SIGNED            Message has a DKIM or DK signature,
+ not necessarily valid
+ -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
+X-ACL-Warn: message may be spam
+X-Scan-Signature: da03fc744e23195c883997a4bc59532e
+Subject: Re: Fundamental problems in Obnam and what should be done about them
+X-BeenThere: obnam-dev@obnam.org
+X-Mailman-Version: 2.1.5
+Precedence: list
+List-Id: Obnam development discussions <obnam-dev-obnam.org>
+List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
+List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
+List-Post: <mailto:obnam-dev@obnam.org>
+List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
+List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=subscribe>
+Sender: obnam-dev-bounces@obnam.org
+Errors-To: obnam-dev-bounces@obnam.org
+
+On Sat, Aug 29, 2015 at 2:15 PM, Lars Wirzenius <liw@liw.fi> wrote:
+[...]
+>
+> I think something like this might be added:
+>
+>     whole-file-compresssion = flac:\.wav$, xz:\.log$, gzip:\.img$
+
+It would be good if the specification is based on sniffing the
+filetype (libmagic), rather than hoping that the extension reflects
+the filetype.
+
+-- 
+jean                                              . .. .... //\\\oo///\\
+
+_______________________________________________
+obnam-dev mailing list
+obnam-dev@obnam.org
+http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org
diff --git a/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998999.M149148P17339Q91.exolobe1 b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998999.M149148P17339Q91.exolobe1
new file mode 100644
index 0000000..1dc15df
--- /dev/null
+++ b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455998999.M149148P17339Q91.exolobe1
@@ -0,0 +1,159 @@
+Return-Path: <obnam-dev-bounces@obnam.org>
+X-Original-To: distix@pieni.net
+Delivered-To: distix@pieni.net
+Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16])
+	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+	(No client certificate requested)
+	by pieni.net (Postfix) with ESMTPS id A2E212DE0F
+	for <distix@pieni.net>; Sun,  5 Apr 2015 19:38:29 +0200 (CEST)
+Received: from platypus.pepperfish.net (unknown [10.112.100.20])
+	by bagpuss.pepperfish.net (Postfix) with ESMTP id 213D41AD4;
+	Sun,  5 Apr 2015 18:38:29 +0100 (BST)
+Received: from ip6-localhost ([::1] helo=platypus.pepperfish.net)
+	by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+	id 1YeoVB-00061C-0w; Sun, 05 Apr 2015 18:38:29 +0100
+Received: from inmail ([10.112.100.10] helo=mx0.pepperfish.net)
+ by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+ id 1YeoV9-00060x-Fv
+ for <obnam-dev@obnam.org>; Sun, 05 Apr 2015 18:38:27 +0100
+Received: from szepe.net ([95.140.33.67])
+ by mx0.pepperfish.net with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256)
+ (Exim 4.80) (envelope-from <viktor@szepe.net>) id 1YeoV7-0003vT-AS
+ for obnam-dev@obnam.org; Sun, 05 Apr 2015 18:38:27 +0100
+Received: from localhost (localhost [127.0.0.1]) (uid 33)
+ by szepe.net with local; Sun, 05 Apr 2015 19:38:23 +0200
+ id 002407B2.5521730F.00006DAE
+Received: from 94-21-114-237.pool.digikabel.hu
+ (94-21-114-237.pool.digikabel.hu [94.21.114.237]) by szepe.net (Horde
+ Framework) with HTTP; Sun, 05 Apr 2015 19:38:23 +0200
+Date: Sun, 05 Apr 2015 19:38:23 +0200
+Message-ID: <20150405193823.Horde.WbSqs4Wyz21q7mOCcp522w1@szepe.net>
+From: =?utf-8?b?U3rDqXBl?= Viktor <viktor@szepe.net>
+To: obnam-dev@obnam.org
+References: <20150404201239.GL6211@exolobe1.liw.fi>
+ <CAExbbMyWptQkOUpyO7TDD=_hgfdi5RjGSPn0HDVFA+jkW+YCsg@mail.gmail.com>
+ <20150405160221.GM6211@exolobe1.liw.fi>
+In-Reply-To: <20150405160221.GM6211@exolobe1.liw.fi>
+User-Agent: Internet Messaging Program (IMP) H5 (6.1.7)
+Accept-Language: en,hu
+Content-Type: text/plain; charset=UTF-8; format=flowed; DelSp=Yes
+MIME-Version: 1.0
+Content-Disposition: inline
+Content-Transfer-Encoding: quoted-printable
+X-Spam-Score: -0.9
+X-Spam-Score-int: -8
+X-Spam-Bar: /
+X-Scanned-By: pepperfish.net, Sun, 05 Apr 2015 18:38:27 +0100
+X-Spam-Report: Content analysis details: (-0.9 points)
+ pts rule name              description
+ ---- ---------------------- --------------------------------------------------
+ 1.0 PPF_RU_URL             BODY: Contains a URL with a .ru domain
+ -0.5 PPF_USER_AGENT         User-Agent: exists
+ 0.5 PPF_RECEIVED_HTTP      Received header mentions http
+ -0.0 SPF_HELO_PASS          SPF: HELO matches SPF record
+ -0.0 T_RP_MATCHES_RCVD      Envelope sender domain matches handover relay
+ domain
+ -0.0 SPF_PASS               SPF: sender matches SPF record
+ -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
+ [score: 0.0000]
+X-ACL-Warn: message may be spam
+Subject: Re: Fundamental problems in Obnam and what should be done about them
+X-BeenThere: obnam-dev@obnam.org
+X-Mailman-Version: 2.1.5
+Precedence: list
+List-Id: Obnam development discussions <obnam-dev-obnam.org>
+List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
+List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
+List-Post: <mailto:obnam-dev@obnam.org>
+List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
+List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=subscribe>
+Sender: obnam-dev-bounces@obnam.org
+Errors-To: obnam-dev-bounces@obnam.org
+
+Good evening Lars and everyone who reads this!
+
+It is very useful that you publicly tell you opinion about your=20=20
+project.=20Thank you!
+
+Please consider Content-Aware compression and deduplication, thus a=20=20
+per=20file decision, like in pcompress
+http://moinakg.github.io/pcompress/
+JPG and MP3 compression: http://packjpg.encode.ru/?page_id=3D17
+
+
+The second thing is a weird question.
+Do you think it is possible to implement duplicity's backends in obnam?
+duplicity supports numerous network storage types.
+
+I am waiting for obnam2 because I am stuck in the decision of backup soluti=
+on:
+https://github.com/szepeviktor/debian-server-tools/tree/master/backup#backu=
+p-criteria-list
+
+All the best wishes to you!
+
+
+
+
+Id=C3=A9zem/Quoting Lars Wirzenius <liw@liw.fi>:
+
+> On Sun, Apr 05, 2015 at 05:35:58PM +0200, SanskritFritz wrote:
+>> Hi, thanks for those thoughts. I don't know if you have plans to address
+>> this problem: http://obnam.org/faq/dedup/
+>> Apparently Attic and Zbackup have successfully solved that with a "rolli=
+ng
+>> checksum" algorithm which is beyond my knowledge now. As far as I
+>> understand, they use a rolling window on the files instead of fixed chun=
+ks,
+>> but I can be entirelly wrong.
+>
+> I think the way Obnam breaks up a file into chunks needs to be
+> changed, and the rolling checksum approach of Attic and Zbackup seems
+> like a good idea. I've mainly been thinking about things that affect
+> the backup repository, so far, and it turns out that the chunking
+> algorithm is independent of that.
+>
+> The way to do this is to change the backup_file_contents method in the
+> backup plugin to not hardcode the way chunking happens, but instead
+> move that into a hook. Different plugins can then provide different
+> algorithms, and one of the plugins gets to enable its algorithm. This
+> makes it possible to experiment with different approaches, and moves
+> the chunk decision making into its own code module, where it doesn't
+> obscure other parts.
+>
+> Being able to choose the chunking algorithm seems useful to me, since
+> different data has different needs. for example, if Obnam were to
+> learn how to backup raw disks, it's possible that the only useful
+> chunk would be the filesystem block size. Other people might prefer
+> the current approach which is very light on CPU usage. The default
+> should probably be the rolling checksum approach.
+>
+>> I can help with testing new solutions.
+>
+> Excellent.
+>
+> --
+> Happiness is a tested backup.
+>
+> _______________________________________________
+> obnam-dev mailing list
+> obnam-dev@obnam.org
+> http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam=
+.org
+
+
+Sz=C3=A9pe Viktor
+--=20
++36-20-4242498=20 sms@szepe.net  skype: szepe.viktor
+Budapest, XX. ker=C3=BClet
+
+
+
+
+
+_______________________________________________
+obnam-dev mailing list
+obnam-dev@obnam.org
+http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org
diff --git a/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455999000.M396907P17339Q107.exolobe1 b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455999000.M396907P17339Q107.exolobe1
new file mode 100644
index 0000000..824b1d4
--- /dev/null
+++ b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455999000.M396907P17339Q107.exolobe1
@@ -0,0 +1,231 @@
+Return-Path: <obnam-dev-bounces@obnam.org>
+X-Original-To: distix@pieni.net
+Delivered-To: distix@pieni.net
+Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16])
+	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+	(No client certificate requested)
+	by pieni.net (Postfix) with ESMTPS id C00092E242
+	for <distix@pieni.net>; Wed,  8 Apr 2015 13:04:43 +0200 (CEST)
+Received: from platypus.pepperfish.net (unknown [10.112.100.20])
+	by bagpuss.pepperfish.net (Postfix) with ESMTP id 0D9A8EE3;
+	Wed,  8 Apr 2015 12:04:43 +0100 (BST)
+Received: from ip6-localhost ([::1] helo=platypus.pepperfish.net)
+	by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+	id 1Yfnmk-0005rz-Ue; Wed, 08 Apr 2015 12:04:42 +0100
+Received: from inmail ([10.112.100.10] helo=mx0.pepperfish.net)
+ by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+ id 1Yfnmj-0005rt-OB
+ for <obnam-dev@obnam.org>; Wed, 08 Apr 2015 12:04:41 +0100
+Received: from mail-ig0-f173.google.com ([209.85.213.173])
+ by mx0.pepperfish.net with esmtps (TLS1.2:RSA_ARCFOUR_SHA1:128)
+ (Exim 4.80) (envelope-from <brenner@a9group.com>) id 1Yfnmh-0001IA-3h
+ for obnam-dev@obnam.org; Wed, 08 Apr 2015 12:04:41 +0100
+Received: by iget9 with SMTP id t9so27009137ige.1
+ for <obnam-dev@obnam.org>; Wed, 08 Apr 2015 04:04:29 -0700 (PDT)
+X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
+ d=1e100.net; s=20130820;
+ h=x-gm-message-state:mime-version:in-reply-to:references:from:date
+ :message-id:subject:to:cc:content-type:content-transfer-encoding;
+ bh=73l0yqMklfSExUMEEIPyli6sQXPmGp7Li7AJuNTFD0s=;
+ b=ecLqbZ74cQ4IYuhnBEX0hGmb7cF0JCavookcmtg0owryiSUS1HwPQMBytfAH9N8pH5
+ y4//A5N1gLMtbEskczYPoXH8oTtDazmwh2sPGnTMoYw46irYWVs8booovwgcnVuCLXnD
+ X/dh4dIwAK8pHNUAYdEy+bTfSdUbfJshVclwlrYw68Jl7lBli4rPLJ+5P1Z3fn05dTj1
+ lNgVSZCFb7rJdGUjQZj7m+IGqKhULn3PSiNa9EO84xEcRVhU74ZMoa5I1Wr6YNbuaYQu
+ Zt8ggvSDuAx86YtJ9Dy5GSp84PYY8VO0H++Pc3nt/O75Pbe5LQ8GhzfKh3zsHz/0keWB
+ /CaQ==
+X-Gm-Message-State: ALoCoQmwVUId/oV21QipmOYYiXooytebPqudfqjGtqMoHY1LxKb7Z0LLjtSiGFvZtcg1UBlis2cT
+X-Received: by 10.107.39.72 with SMTP id n69mr37011948ion.8.1428491068223;
+ Wed, 08 Apr 2015 04:04:28 -0700 (PDT)
+MIME-Version: 1.0
+Received: by 10.107.137.224 with HTTP; Wed, 8 Apr 2015 04:04:07 -0700 (PDT)
+In-Reply-To: <20150405193823.Horde.WbSqs4Wyz21q7mOCcp522w1@szepe.net>
+References: <20150404201239.GL6211@exolobe1.liw.fi>
+ <CAExbbMyWptQkOUpyO7TDD=_hgfdi5RjGSPn0HDVFA+jkW+YCsg@mail.gmail.com>
+ <20150405160221.GM6211@exolobe1.liw.fi>
+ <20150405193823.Horde.WbSqs4Wyz21q7mOCcp522w1@szepe.net>
+From: Antoine Brenner <brenner+obnam@gymglish.com>
+Date: Wed, 8 Apr 2015 13:04:07 +0200
+Message-ID: <CAPzomaPO78aQTjyUxzzUMjYui51=WbxOZJxq1KDgTDp=ccr+fw@mail.gmail.com>
+To: =?UTF-8?Q?Sz=C3=A9pe_Viktor?= <viktor@szepe.net>
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: quoted-printable
+X-Spam-Score: -1.1
+X-Spam-Score-int: -10
+X-Spam-Bar: -
+X-Scanned-By: pepperfish.net, Wed, 08 Apr 2015 12:04:41 +0100
+X-Spam-Report: Content analysis details: (-1.1 points)
+ pts rule name              description
+ ---- ---------------------- --------------------------------------------------
+ 1.0 PPF_RU_URL             BODY: Contains a URL with a .ru domain
+ 0.5 PPF_RECEIVED_HTTP      Received header mentions http
+ -0.7 RCVD_IN_DNSWL_LOW      RBL: Sender listed at http://www.dnswl.org/, low
+ trust [209.85.213.173 listed in list.dnswl.org]
+ -0.0 SPF_PASS               SPF: sender matches SPF record
+ -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
+ [score: 0.0000]
+X-ACL-Warn: message may be spam
+Cc: obnam-dev@obnam.org
+Subject: Re: Fundamental problems in Obnam and what should be done about them
+X-BeenThere: obnam-dev@obnam.org
+X-Mailman-Version: 2.1.5
+Precedence: list
+List-Id: Obnam development discussions <obnam-dev-obnam.org>
+List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
+List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
+List-Post: <mailto:obnam-dev@obnam.org>
+List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
+List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=subscribe>
+Sender: obnam-dev-bounces@obnam.org
+Errors-To: obnam-dev-bounces@obnam.org
+
+Hello everyone,
+
+It is very nice to discuss this on this list indeed; thanks.
+
+
+As mentioned by Itamar Turner-Trauring on this very mailing-list in
+Dec. 2013, better deduplication than eg rsync rolling checksum
+algorithm can most probably be achieved by supporting variable length
+chunks. (cf bottom left paragraph of page 4 of
+http://www.news.cs.nyu.edu/~jinyang/sp07/papers/lbfs.pdf . What they
+do is called Rabin fingerprinting [1])
+
+This approach also seems a perfect match for copy-on-write B-tree
+content storage addressed by chunk checksums (assuming that checksum
+collisions do not happen however, like the git people do).
+
+So, at this design stage, I think that making sure that all chunks can
+have a variable length (even eg within one big file) would be a good
+idea.
+
+Using Rabin fingerprinting at the core of obnam might also be worth conside=
+ring.
+
+Best regards,
+Antoine
+
+
+Note1;
+pcompress [2] (looks great, thanks for pointing it out Viktor)
+supports deduplication based on Rabin fingerprinting too, so maybe
+just using it to find and compress the chunks could help.
+
+
+Note2:
+In the meantime, gzip such as shipped with the rsyncable patch [3] in
+Fedora allows what is called rsyncable-gzip output [4] where slightly
+different big files once compressed still have most of their (fixed
+size) chunks shared. This helps a lot when both compression
+and deduplication are wished.
+
+[1] http://en.wikipedia.org/wiki/Rabin_fingerprint
+[2] http://moinakg.github.io/pcompress/
+[3] http://pkgs.fedoraproject.org/cgit/gzip.git/tree/gzip-1.3.13-rsync.patc=
+h
+[4] https://beeznest.wordpress.com/2005/02/03/rsyncable-gzip/
+
+
+
+On Sun, Apr 5, 2015 at 7:38 PM, Sz=C3=A9pe Viktor <viktor@szepe.net> wrote:
+> Good evening Lars and everyone who reads this!
+>
+> It is very useful that you publicly tell you opinion about your project.
+> Thank you!
+>
+> Please consider Content-Aware compression and deduplication, thus a per f=
+ile
+> decision, like in pcompress
+> http://moinakg.github.io/pcompress/
+> JPG and MP3 compression: http://packjpg.encode.ru/?page_id=3D17
+>
+>
+> The second thing is a weird question.
+> Do you think it is possible to implement duplicity's backends in obnam?
+> duplicity supports numerous network storage types.
+>
+> I am waiting for obnam2 because I am stuck in the decision of backup
+> solution:
+> https://github.com/szepeviktor/debian-server-tools/tree/master/backup#bac=
+kup-criteria-list
+>
+> All the best wishes to you!
+>
+>
+>
+>
+> Id=C3=A9zem/Quoting Lars Wirzenius <liw@liw.fi>:
+>
+>
+>> On Sun, Apr 05, 2015 at 05:35:58PM +0200, SanskritFritz wrote:
+>>>
+>>> Hi, thanks for those thoughts. I don't know if you have plans to addres=
+s
+>>> this problem: http://obnam.org/faq/dedup/
+>>> Apparently Attic and Zbackup have successfully solved that with a
+>>> "rolling
+>>> checksum" algorithm which is beyond my knowledge now. As far as I
+>>> understand, they use a rolling window on the files instead of fixed
+>>> chunks,
+>>> but I can be entirelly wrong.
+>>
+>>
+>> I think the way Obnam breaks up a file into chunks needs to be
+>> changed, and the rolling checksum approach of Attic and Zbackup seems
+>> like a good idea. I've mainly been thinking about things that affect
+>> the backup repository, so far, and it turns out that the chunking
+>> algorithm is independent of that.
+>>
+>> The way to do this is to change the backup_file_contents method in the
+>> backup plugin to not hardcode the way chunking happens, but instead
+>> move that into a hook. Different plugins can then provide different
+>> algorithms, and one of the plugins gets to enable its algorithm. This
+>> makes it possible to experiment with different approaches, and moves
+>> the chunk decision making into its own code module, where it doesn't
+>> obscure other parts.
+>>
+>> Being able to choose the chunking algorithm seems useful to me, since
+>> different data has different needs. for example, if Obnam were to
+>> learn how to backup raw disks, it's possible that the only useful
+>> chunk would be the filesystem block size. Other people might prefer
+>> the current approach which is very light on CPU usage. The default
+>> should probably be the rolling checksum approach.
+>>
+>>> I can help with testing new solutions.
+>>
+>>
+>> Excellent.
+>>
+>> --
+>> Happiness is a tested backup.
+>>
+>> _______________________________________________
+>> obnam-dev mailing list
+>> obnam-dev@obnam.org
+>>
+>> http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obna=
+m.org
+>
+>
+>
+> Sz=C3=A9pe Viktor
+> --
+> +36-20-4242498  sms@szepe.net  skype: szepe.viktor
+> Budapest, XX. ker=C3=BClet
+>
+>
+>
+>
+>
+>
+> _______________________________________________
+> obnam-dev mailing list
+> obnam-dev@obnam.org
+> http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam=
+.org
+
+_______________________________________________
+obnam-dev mailing list
+obnam-dev@obnam.org
+http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org
diff --git a/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455999003.M632745P17339Q137.exolobe1 b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455999003.M632745P17339Q137.exolobe1
new file mode 100644
index 0000000..d4bc1fc
--- /dev/null
+++ b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455999003.M632745P17339Q137.exolobe1
@@ -0,0 +1,96 @@
+Return-Path: <obnam-dev-bounces@obnam.org>
+X-Original-To: distix@pieni.net
+Delivered-To: distix@pieni.net
+Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16])
+	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+	(No client certificate requested)
+	by pieni.net (Postfix) with ESMTPS id 3B3322BEC5
+	for <distix@pieni.net>; Sat, 29 Aug 2015 15:07:09 +0200 (CEST)
+Received: from platypus.pepperfish.net (unknown [10.112.100.20])
+	by bagpuss.pepperfish.net (Postfix) with ESMTP id 863C7EC0;
+	Sat, 29 Aug 2015 14:07:08 +0100 (BST)
+Received: from ip6-localhost ([::1] helo=platypus.pepperfish.net)
+	by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+	id 1ZVfqe-0000NA-EQ; Sat, 29 Aug 2015 14:07:08 +0100
+Received: from inmail0 ([10.112.100.10] helo=mx0.pepperfish.net)
+ by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+ id 1ZVfqc-0000Mx-4k
+ for <obnam-dev@obnam.org>; Sat, 29 Aug 2015 14:07:06 +0100
+Received: from pieni.net ([95.142.166.37] ident=postfix)
+ by mx0.pepperfish.net with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256)
+ (Exim 4.80) (envelope-from <liw@liw.fi>) id 1ZVfqZ-0007vr-VM
+ for obnam-dev@obnam.org; Sat, 29 Aug 2015 14:07:06 +0100
+Received: from exolobe1.liw.fi (82-181-8-107.bb.dnainternet.fi [82.181.8.107])
+ (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+ (No client certificate requested)
+ by pieni.net (Postfix) with ESMTPSA id 75E3D2BEC5
+ for <obnam-dev@obnam.org>; Sat, 29 Aug 2015 15:06:57 +0200 (CEST)
+Received: from exolobe1.liw.fi (localhost [127.0.0.1])
+ by exolobe1.liw.fi (Postfix) with ESMTPS id D20C640161
+ for <obnam-dev@obnam.org>; Sat, 29 Aug 2015 16:06:56 +0300 (EEST)
+Date: Sat, 29 Aug 2015 16:06:55 +0300
+From: Lars Wirzenius <liw@liw.fi>
+To: obnam-dev@obnam.org
+Message-ID: <20150829130655.GH2246@exolobe1.liw.fi>
+References: <20150404201239.GL6211@exolobe1.liw.fi>
+ <CAExbbMyWptQkOUpyO7TDD=_hgfdi5RjGSPn0HDVFA+jkW+YCsg@mail.gmail.com>
+ <20150405160221.GM6211@exolobe1.liw.fi>
+ <20150405193823.Horde.WbSqs4Wyz21q7mOCcp522w1@szepe.net>
+ <20150829111552.GD2246@exolobe1.liw.fi>
+ <CAJqnznA_ZGznKuBjuuUK-P5Ccow3iB14iLnsQX74VVUmi+jDgQ@mail.gmail.com>
+MIME-Version: 1.0
+Content-Type: text/plain; charset=us-ascii
+Content-Disposition: inline
+In-Reply-To: <CAJqnznA_ZGznKuBjuuUK-P5Ccow3iB14iLnsQX74VVUmi+jDgQ@mail.gmail.com>
+User-Agent: Mutt/1.5.23 (2014-03-12)
+X-Spam-Score: -3.4
+X-Spam-Score-int: -33
+X-Spam-Bar: ---
+X-Scanned-By: pepperfish.net, Sat, 29 Aug 2015 14:07:06 +0100
+X-Spam-Report: Content analysis details: (-3.4 points)
+ pts rule name              description
+ ---- ---------------------- --------------------------------------------------
+ -1.0 PPF_USER_AGENT_MUTT    User-Agent: contains Mutt (Mutt isn't a spam
+ tool) -0.5 PPF_USER_AGENT         User-Agent: exists
+ -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
+ [score: 0.0000]
+X-ACL-Warn: message may be spam
+X-Scan-Signature: 2e65f5179848bd99ed9586f9b1c7ce7e
+Subject: Re: Fundamental problems in Obnam and what should be done about them
+X-BeenThere: obnam-dev@obnam.org
+X-Mailman-Version: 2.1.5
+Precedence: list
+List-Id: Obnam development discussions <obnam-dev-obnam.org>
+List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
+List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
+List-Post: <mailto:obnam-dev@obnam.org>
+List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
+List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=subscribe>
+Sender: obnam-dev-bounces@obnam.org
+Errors-To: obnam-dev-bounces@obnam.org
+
+On Sat, Aug 29, 2015 at 03:03:43PM +0300, Jean Jordaan wrote:
+> On Sat, Aug 29, 2015 at 2:15 PM, Lars Wirzenius <liw@liw.fi> wrote:
+> [...]
+> >
+> > I think something like this might be added:
+> >
+> >     whole-file-compresssion = flac:\.wav$, xz:\.log$, gzip:\.img$
+> 
+> It would be good if the specification is based on sniffing the
+> filetype (libmagic), rather than hoping that the extension reflects
+> the filetype.
+
+Sure, that can be done too. It's going to make things slower, of
+course, since it requires opening every file and reading parts of
+them. However, we'll leave that for whoever implements this.
+
+-- 
+sic transit disci mundi, ergo obnam
+
+_______________________________________________
+obnam-dev mailing list
+obnam-dev@obnam.org
+http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org
diff --git a/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455999006.M830313P17339Q172.exolobe1 b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455999006.M830313P17339Q172.exolobe1
new file mode 100644
index 0000000..721c199
--- /dev/null
+++ b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/new/1455999006.M830313P17339Q172.exolobe1
@@ -0,0 +1,150 @@
+Return-Path: <obnam-dev-bounces@obnam.org>
+X-Original-To: distix@pieni.net
+Delivered-To: distix@pieni.net
+Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16])
+	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+	(No client certificate requested)
+	by pieni.net (Postfix) with ESMTPS id 6E8CF2E027
+	for <distix@pieni.net>; Sun,  5 Apr 2015 17:36:10 +0200 (CEST)
+Received: from platypus.pepperfish.net (unknown [10.112.100.20])
+	by bagpuss.pepperfish.net (Postfix) with ESMTP id AA36A128B;
+	Sun,  5 Apr 2015 16:36:09 +0100 (BST)
+Received: from ip6-localhost ([::1] helo=platypus.pepperfish.net)
+	by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+	id 1Yeman-0006a3-IS; Sun, 05 Apr 2015 16:36:09 +0100
+Received: from inmail ([10.112.100.10] helo=mx0.pepperfish.net)
+ by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+ id 1Yemam-0006Zm-Bj
+ for <obnam-dev@obnam.org>; Sun, 05 Apr 2015 16:36:08 +0100
+Received: from mail-wi0-f178.google.com ([209.85.212.178])
+ by mx0.pepperfish.net with esmtps (TLS1.2:RSA_ARCFOUR_SHA1:128)
+ (Exim 4.80) (envelope-from <sanskritfritz@gmail.com>)
+ id 1Yemak-0000Sn-7N
+ for obnam-dev@obnam.org; Sun, 05 Apr 2015 16:36:08 +0100
+Received: by wiun10 with SMTP id n10so13613700wiu.1
+ for <obnam-dev@obnam.org>; Sun, 05 Apr 2015 08:36:00 -0700 (PDT)
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
+ h=mime-version:in-reply-to:references:date:message-id:subject:from:to
+ :cc:content-type;
+ bh=ybE61+WSS/6f8VIOCRqZiOr7Le95g+0ptwayeuljRQI=;
+ b=oegm5MEiXCEgacjYwnQZ0ORJPUjN8WWoSUacn2SyIQf1T54hheY2fNdIpkhYbQhhAX
+ eUw9eZDwtbN1aNjZcvHmQnJRFc+lnp+0Aexog7tdBFdZAO6Ufg/YnsJAv8ourAQNXh/5
+ MaT6Sczvkd++IZQcwUHVmikTKorPM/FCTt39CpGQ0b1WzexvIWra5B1rRO5mRioP0W3l
+ pafzpJHtb9n6sxWKA0Gh/gETpJ0qNP+mvl25x0E9QI9SZfbamnvvB1onw+q0+lY1cBDT
+ 89QWepUdElj5Di898z9ztldCZv0OKVGhcbiHbugC7WSEHl7J41IbUSRgaRn78m5F4jnW
+ shgQ==
+MIME-Version: 1.0
+X-Received: by 10.194.104.99 with SMTP id gd3mr22453738wjb.114.1428248158857; 
+ Sun, 05 Apr 2015 08:35:58 -0700 (PDT)
+Received: by 10.194.172.170 with HTTP; Sun, 5 Apr 2015 08:35:58 -0700 (PDT)
+In-Reply-To: <20150404201239.GL6211@exolobe1.liw.fi>
+References: <20150404201239.GL6211@exolobe1.liw.fi>
+Date: Sun, 5 Apr 2015 17:35:58 +0200
+Message-ID: <CAExbbMyWptQkOUpyO7TDD=_hgfdi5RjGSPn0HDVFA+jkW+YCsg@mail.gmail.com>
+From: SanskritFritz <sanskritfritz@gmail.com>
+To: Lars Wirzenius <liw@liw.fi>
+X-Spam-Score: 1.7
+X-Spam-Score-int: 17
+X-Spam-Bar: +
+X-Scanned-By: pepperfish.net, Sun, 05 Apr 2015 16:36:08 +0100
+X-Spam-Report: Content analysis details: (1.7 points)
+ pts rule name              description
+ ---- ---------------------- --------------------------------------------------
+ -0.7 RCVD_IN_DNSWL_LOW      RBL: Sender listed at http://www.dnswl.org/, low
+ trust [209.85.212.178 listed in list.dnswl.org]
+ 0.2 PPF_ALTERNATIVES       Body in multipart/alternative
+ 0.5 PPF_RECEIVED_HTTP      Received header mentions http
+ 1.0 PPF_FROM_CONTAINS_MAIL The From header contains 'mail'
+ 1.2 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
+ (sanskritfritz[at]gmail.com)
+ -0.0 SPF_PASS               SPF: sender matches SPF record
+ -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
+ [score: 0.0000]
+ 0.3 HTML_MESSAGE           BODY: HTML included in message
+ 0.2 PPF_NUMERIC_ENTITY     RAW: Body contains numeric HTML entities
+ 1.0 PPF_BLANK              RAW: Body contains _blank
+ -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's
+ domain
+ 0.1 DKIM_SIGNED            Message has a DKIM or DK signature,
+ not necessarily valid
+ -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
+X-ACL-Warn: message may be spam
+Cc: Obnam development <obnam-dev@obnam.org>
+Subject: Re: Fundamental problems in Obnam and what should be done about them
+X-BeenThere: obnam-dev@obnam.org
+X-Mailman-Version: 2.1.5
+Precedence: list
+List-Id: Obnam development discussions <obnam-dev-obnam.org>
+List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
+List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
+List-Post: <mailto:obnam-dev@obnam.org>
+List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
+List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=subscribe>
+Content-Type: multipart/mixed; boundary="===============2687456440786176051=="
+Mime-version: 1.0
+Sender: obnam-dev-bounces@obnam.org
+Errors-To: obnam-dev-bounces@obnam.org
+
+--===============2687456440786176051==
+Content-Type: multipart/alternative; boundary=089e010d879c28eef60512fbf081
+
+--089e010d879c28eef60512fbf081
+Content-Type: text/plain; charset=UTF-8
+
+On Sat, Apr 4, 2015 at 10:12 PM, Lars Wirzenius <liw@liw.fi> wrote:
+
+>
+> As a first step, what do others on obnam-dev think about the above? Do
+> you agree with the problems I've raised? Have I missed some important
+> problems? What could you to help fix the problems?
+>
+
+Hi, thanks for those thoughts. I don't know if you have plans to address
+this problem: http://obnam.org/faq/dedup/
+Apparently Attic and Zbackup have successfully solved that with a "rolling
+checksum" algorithm which is beyond my knowledge now. As far as I
+understand, they use a rolling window on the files instead of fixed chunks,
+but I can be entirelly wrong.
+I can help with testing new solutions.
+
+--089e010d879c28eef60512fbf081
+Content-Type: text/html; charset=UTF-8
+Content-Transfer-Encoding: quoted-printable
+
+<div dir=3D"ltr"><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">=
+On Sat, Apr 4, 2015 at 10:12 PM, Lars Wirzenius <span dir=3D"ltr">&lt;<a hr=
+ef=3D"mailto:liw@liw.fi" target=3D"_blank">liw@liw.fi</a>&gt;</span> wrote:=
+<br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bor=
+der-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
+As a first step, what do others on obnam-dev think about the above? Do<br>
+you agree with the problems I&#39;ve raised? Have I missed some important<b=
+r>
+problems? What could you to help fix the problems?<br></blockquote></div><b=
+r></div><div class=3D"gmail_extra">Hi, thanks for those thoughts. I don&#39=
+;t know if you have plans to address this problem: <a href=3D"http://obnam.=
+org/faq/dedup/">http://obnam.org/faq/dedup/</a><br></div><div class=3D"gmai=
+l_extra">Apparently Attic and Zbackup have successfully solved that with a =
+&quot;rolling checksum&quot; algorithm which is beyond my knowledge now. As=
+ far as I understand, they use a rolling window on the files instead of fix=
+ed chunks, but I can be entirelly wrong.<br></div><div class=3D"gmail_extra=
+">I can help with testing new solutions.<br></div><div class=3D"gmail_extra=
+"><br></div></div>
+
+--089e010d879c28eef60512fbf081--
+
+
+--===============2687456440786176051==
+Content-Type: text/plain; charset="us-ascii"
+MIME-Version: 1.0
+Content-Transfer-Encoding: 7bit
+Content-Disposition: inline
+
+_______________________________________________
+obnam-dev mailing list
+obnam-dev@obnam.org
+http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org
+
+--===============2687456440786176051==--
+
diff --git a/tickets/858f91e5841e407699ff9abd95089c64/Maildir/tmp/.this-dir-not-empty/.empty/empty-file b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/tmp/.this-dir-not-empty/.empty/empty-file
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/tickets/858f91e5841e407699ff9abd95089c64/Maildir/tmp/.this-dir-not-empty/.empty/empty-file
diff --git a/tickets/858f91e5841e407699ff9abd95089c64/ticket.yaml b/tickets/858f91e5841e407699ff9abd95089c64/ticket.yaml
new file mode 100644
index 0000000..f649034
--- /dev/null
+++ b/tickets/858f91e5841e407699ff9abd95089c64/ticket.yaml
@@ -0,0 +1,4 @@
+ticket-id:
+- 858f91e5841e407699ff9abd95089c64
+title:
+- 'Re: Fundamental problems in Obnam and what should be done about them'