1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
|
Return-Path: <obnam-support-bounces@obnam.org>
X-Original-To: distix@pieni.net
Delivered-To: distix@pieni.net
Received: from yaffle.pepperfish.net (yaffle.pepperfish.net [88.99.213.221])
by pieni.net (Postfix) with ESMTPS id 5EBA84020D
for <distix@pieni.net>; Mon, 24 Jul 2017 17:21:03 +0000 (UTC)
Received: from platypus.pepperfish.net (unknown [10.112.101.20])
by yaffle.pepperfish.net (Postfix) with ESMTP id 4677B4189F;
Mon, 24 Jul 2017 18:21:03 +0100 (BST)
Received: from ip6-localhost.nat ([::1] helo=platypus.pepperfish.net)
by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
id 1dZh2V-0007OV-8e; Mon, 24 Jul 2017 18:21:03 +0100
Received: from [10.112.101.21] (helo=mx3.pepperfish.net)
by platypus.pepperfish.net with esmtps (Exim 4.80 #2 (Debian))
id 1dZh2U-0007O4-ID
for <obnam-support@obnam.org>; Mon, 24 Jul 2017 18:21:02 +0100
Received: from koom.pieni.net ([88.99.190.206] helo=pieni.net)
by mx3.pepperfish.net with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.89) (envelope-from <liw@liw.fi>) id 1dZh2S-0004PI-U8
for obnam-support@obnam.org; Mon, 24 Jul 2017 18:21:02 +0100
Received: from exolobe3.liw.fi (82-181-57-241.bb.dnainternet.fi
[82.181.57.241]) by pieni.net (Postfix) with ESMTPSA id A40C5405A8;
Mon, 24 Jul 2017 17:20:50 +0000 (UTC)
Received: from liw.fi (localhost [127.0.0.1])
by exolobe3.liw.fi (Postfix) with ESMTPS id CA4A61200EA;
Mon, 24 Jul 2017 20:20:44 +0300 (EEST)
Date: Mon, 24 Jul 2017 20:20:43 +0300
From: Lars Wirzenius <liw@liw.fi>
To: "Laurence Perkins (OE)" <lperkins@openeye.net>
Message-ID: <20170724172043.s2ykrfwcusyzdcgd@liw.fi>
References: <1500484994.13826.5.camel@openeye.net>
<20170719181232.sdqihqdqldsgzmtd@liw.fi>
<1500571405.13826.8.camel@openeye.net>
<20170722141756.yzxatuvogrdsh4jv@liw.fi>
<1500916329.13826.13.camel@openeye.net>
MIME-Version: 1.0
In-Reply-To: <1500916329.13826.13.camel@openeye.net>
User-Agent: NeoMutt/20170113 (1.7.2)
X-Pepperfish-Transaction: d932-fcaa-e364-43eb
X-Spam-Score: -3.4
X-Spam-Score-int: -33
X-Spam-Bar: ---
X-Scanned-By: pepperfish.net, Mon, 24 Jul 2017 18:21:02 +0100
X-Spam-Report: Content analysis details: (-3.4 points)
pts rule name description
---- ---------------------- --------------------------------------------------
-0.5 PPF_USER_AGENT User-Agent: exists
-1.0 PPF_USER_AGENT_MUTT User-Agent: contains Mutt (Mutt isn't a spam
tool)
-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
[score: 0.0000]
X-ACL-Warn: message may be spam
X-Scan-Signature: 8e53bf13cb5fa4952a081d65efed4e37
Cc: "obnam-support@obnam.org" <obnam-support@obnam.org>
Subject: Re: Variable Chunksize
X-BeenThere: obnam-support@obnam.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Obnam backup software discussion <obnam-support-obnam.org>
List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-support-obnam.org>,
<mailto:obnam-support-request@obnam.org?subject=unsubscribe>
List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-support-obnam.org>
List-Post: <mailto:obnam-support@obnam.org>
List-Help: <mailto:obnam-support-request@obnam.org?subject=help>
List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-support-obnam.org>,
<mailto:obnam-support-request@obnam.org?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============8879255905254956802=="
Mime-version: 1.0
Sender: obnam-support-bounces@obnam.org
Errors-To: obnam-support-bounces@obnam.org
--===============8879255905254956802==
Content-Type: multipart/signed; micalg=pgp-sha512;
protocol="application/pgp-signature"; boundary="jfkiq2sxlnslxmsw"
Content-Disposition: inline
--jfkiq2sxlnslxmsw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Mon, Jul 24, 2017 at 05:12:15PM +0000, Laurence Perkins (OE) wrote:
> Smaller chunk size makes deduplication more precise regardless of the
> type of splitting, but it should generate some pretty big savings on
> similar data without reducing the chunk size because it will be better
> at finding identical chunks of data since it's not relying on them
> being at fixed offsets.
If you have actual measurements of this, please report them. Some
years ago when this idea first came up in the Obnam context, using
the proposed type of chunking without reducing chunk size
significatnly didn't much help in de-duplication. Only when the
average chunk size became much smaller, did de-deuplication get a lot
better, but then the number of chunks became a problem.
Guessing isn't helpful here. Even if it were, now is not a good time
for me to spend any time on this, and I don't want to even consider a
patch for this until green albatross is in shape.
--=20
I want to build worthwhile things that might last. --joeyh
--jfkiq2sxlnslxmsw
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEETNTnrewG6wEE1EJ3bC+mFux6IDEFAll2LGoACgkQbC+mFux6
IDFfGxAAidaVMXQm4i+pWmu/07be3VuWFJmFgS8Yvp8uXvVd835agOQqypsk775l
jS0KnrXazzVv6lg0PjXOsLQONYRJ9IQQ6wwEBieWc2Q22cSJFyH+VMibIPb3+Tnj
Gu6OA161CCAK0qpnaq6fo7fJsf9JYWcEMYWUhM9u43YVuULVlYZf3OFlXueJTt0y
yaXQfHZYYLoJUz84nIhj4bxBCyL9GjMYfY9FowyDb24pz9CkGtcOubAiskt4hs0f
tL2n05hfnN1fUfv7ukXC1vdbPlGGCLzZmfvBg4grqP8crAqW3P5fYMhci5rQsllh
6e03f+Mwh/w+oFGsff86mwpBCgCpFhU6a4RmREPB4n2A1apmtlhXTXwAioryZIg2
gKlMWXe3OcZAlrZyKrM2lZK56B9M96G2JKIOY2bytquTWDzaEdxTxOHZgnPVVLQx
fPNZvl8tBvFp7XYNtgI0hiyZn/wD5mHR1eC3ji7KiEuKnq3rG9QK9JgI3OfJ95eG
abJdZ/YqtebSZkJytbSKV18nTFffvazf3k+sDpkWFHXAC7YgaNM8OR7gb4HXiNqY
V0iNDLV88XMNX3lNv3DJEks+5ejAPo932kbHrYQFiofTlMnRWRx9wVAPcGcD+dv+
TlrnyhLrZg3uJxxGfSg2mQ7Xq07+FO2ELqlheOSu0tnuq2GNEW4=
=2hi6
-----END PGP SIGNATURE-----
--jfkiq2sxlnslxmsw--
--===============8879255905254956802==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
obnam-support mailing list
obnam-support@obnam.org
http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-support-obnam.org
--===============8879255905254956802==--
|
