1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
|
Return-Path: <obnam-dev-bounces@obnam.org>
X-Original-To: distix@pieni.net
Delivered-To: distix@pieni.net
Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by pieni.net (Postfix) with ESMTPS id 483C52E2E1
for <distix@pieni.net>; Sat, 17 Oct 2015 19:55:06 +0200 (CEST)
Received: from platypus.pepperfish.net (unknown [10.112.100.20])
by bagpuss.pepperfish.net (Postfix) with ESMTP id AE4D2BBD;
Sat, 17 Oct 2015 18:55:05 +0100 (BST)
Received: from ip6-localhost ([::1] helo=platypus.pepperfish.net)
by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
id 1ZnVhB-0004Vo-JJ; Sat, 17 Oct 2015 18:55:05 +0100
Received: from inmail0 ([10.112.100.10] helo=mx0.pepperfish.net)
by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
id 1ZnVhA-0004VM-7u
for <obnam-dev@obnam.org>; Sat, 17 Oct 2015 18:55:04 +0100
Received: from mailout.easymail.ca ([64.68.201.169])
by mx0.pepperfish.net with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256)
(Exim 4.80) (envelope-from <hsivonen@hsivonen.fi>)
id 1ZnVh7-0006Ku-1D
for obnam-dev@obnam.org; Sat, 17 Oct 2015 18:55:04 +0100
Received: from localhost (localhost [127.0.0.1])
by mailout.easymail.ca (Postfix) with ESMTP id AD899EAF8
for <obnam-dev@obnam.org>; Sat, 17 Oct 2015 13:54:50 -0400 (EDT)
X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca
X-Spam-Flag: NO
X-Spam-Score: -3.681
X-Spam-Level:
X-Spam-Status: No, score=-3.681 required=5 tests=[ALL_TRUSTED=-1.8, AWL=0.026,
BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692]
Received: from mailout.easymail.ca ([127.0.0.1])
by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new,
port 10024) with ESMTP id q5i63X3--m9U for <obnam-dev@obnam.org>;
Sat, 17 Oct 2015 13:54:50 -0400 (EDT)
Received: from mail-io0-f174.google.com (mail-io0-f174.google.com
[209.85.223.174]) (using TLSv1 with cipher RC4-SHA (128/128 bits))
(No client certificate requested)
by mailout.easymail.ca (Postfix) with ESMTPSA id 5223BEA8D
for <obnam-dev@obnam.org>; Sat, 17 Oct 2015 13:54:50 -0400 (EDT)
Received: by iodv82 with SMTP id v82so153037402iod.0
for <obnam-dev@obnam.org>; Sat, 17 Oct 2015 10:54:50 -0700 (PDT)
MIME-Version: 1.0
X-Received: by 10.107.18.91 with SMTP id a88mr3405229ioj.91.1445104490055;
Sat, 17 Oct 2015 10:54:50 -0700 (PDT)
Received: by 10.107.142.141 with HTTP; Sat, 17 Oct 2015 10:54:50 -0700 (PDT)
Date: Sat, 17 Oct 2015 20:54:50 +0300
Message-ID: <CAJQvAufu-qnC=nrR87XN3fkHu6K7p3-Hu5A71GL+uXZ8EKCaUw@mail.gmail.com>
From: Henri Sivonen <hsivonen@hsivonen.fi>
To: obnam-dev@obnam.org
Content-Type: text/plain; charset=UTF-8
X-Spam-Score: -2.1
X-Spam-Score-int: -20
X-Spam-Bar: --
X-Scanned-By: pepperfish.net, Sat, 17 Oct 2015 18:55:04 +0100
X-Spam-Report: Content analysis details: (-2.1 points)
pts rule name description
---- ---------------------- --------------------------------------------------
0.5 PPF_RECEIVED_HTTP Received header mentions http
-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low
trust [64.68.201.169 listed in list.dnswl.org]
-0.0 SPF_PASS SPF: sender matches SPF record
-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
[score: 0.0000]
X-ACL-Warn: message may be spam
X-Scan-Signature: fc828f4ee22ea391207ca0493f0eb918
Subject: Enhancement: Avoid storing compressed data when compression would
expand data
X-BeenThere: obnam-dev@obnam.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Obnam development discussions <obnam-dev-obnam.org>
List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
<mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
List-Post: <mailto:obnam-dev@obnam.org>
List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
<mailto:obnam-dev-request@obnam.org?subject=subscribe>
Sender: obnam-dev-bounces@obnam.org
Errors-To: obnam-dev-bounces@obnam.org
I have made a minor improvement to avoid storing compressed data even
when compression has been enabled in cases where compression (plus
tagging) would end up expanding the data. This avoids space waste upon
transfer and storage and compute waste upon reading.
The change is available for pulling from the "no-expansion" branch of
the repo https://github.com/hsivonen/obnam.git (the branch contains
nothing else). Web view of the changeset:
https://github.com/hsivonen/obnam/commit/5002965ad28c89a08c370efa880a060cf7835dbd
Patch format:
https://github.com/hsivonen/obnam/commit/5002965ad28c89a08c370efa880a060cf7835dbd.patch
I have tested the patch manually with a directory containing two files
generated with
$ head -c 1024 /dev/urandom > rand
$ head -c 1024 /dev/zero > zero
(rand doesn't compress and zero compresses very well.)
However, I'm currently unfamiliar with the workings of yarn, so I
haven't added automated tests. If the patch isn't acceptable without
tests, I'd appreciate advice on how to do the kind of testing that
looks inside the chunk data stored in the repo.
--
Henri Sivonen
hsivonen@hsivonen.fi
https://hsivonen.fi/
_______________________________________________
obnam-dev mailing list
obnam-dev@obnam.org
http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org
|
