1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
|
Return-Path: <obnam-dev-bounces@obnam.org>
X-Original-To: distix@pieni.net
Delivered-To: distix@pieni.net
Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by pieni.net (Postfix) with ESMTPS id B3F4A20F8A
for <distix@pieni.net>; Sun, 19 Jun 2016 19:27:43 +0200 (CEST)
Received: from platypus.pepperfish.net (unknown [10.112.100.20])
by bagpuss.pepperfish.net (Postfix) with ESMTP id 3A7775BF;
Sun, 19 Jun 2016 18:27:43 +0100 (BST)
Received: from ip6-localhost ([::1] helo=platypus.pepperfish.net)
by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
id 1bEgVb-00074z-0g; Sun, 19 Jun 2016 18:27:43 +0100
Received: from inmail0 ([10.112.100.10] helo=mx0.pepperfish.net)
by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
id 1bEgVY-00074i-EY
for <obnam-dev@obnam.org>; Sun, 19 Jun 2016 18:27:40 +0100
Received: from smtp.gentoo.org ([140.211.166.183])
by mx0.pepperfish.net with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256)
(Exim 4.80) (envelope-from <robbat2@gentoo.org>) id 1bEgVW-0004Tf-0E
for obnam-dev@obnam.org; Sun, 19 Jun 2016 18:27:40 +0100
Received: from grubbs.orbis-terrarum.net (localhost [127.0.0.1])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA (256/256 bits))
(No client certificate requested)
by smtp.gentoo.org (Postfix) with ESMTPS id 0755D340831
for <obnam-dev@obnam.org>; Sun, 19 Jun 2016 17:27:25 +0000 (UTC)
Received: (qmail 5243 invoked by uid 129); 19 Jun 2016 17:27:25 -0000
X-HELO: bohr-int.orbis-terrarum.net
Authentication-Results: orbis-terrarum.net; auth=pass (plain)
smtp.auth=robbat2-bohr@orbis-terrarum.net
Received: from Unknown (HELO bohr-int.orbis-terrarum.net) (2001:470:e889:1::8)
by orbis-terrarum.net (qpsmtpd/0.95) with ESMTPSA
(ECDHE-RSA-AES256-GCM-SHA384 encrypted); Sun, 19 Jun 2016 17:27:25 +0000
Received: (nullmailer pid 9310 invoked by uid 10000);
Sun, 19 Jun 2016 17:27:22 -0000
From: "Robin H. Johnson" <robbat2@gentoo.org>
To: obnam-dev@obnam.org
Date: Sun, 19 Jun 2016 10:27:17 -0700
Message-Id: <20160619172717.18445-2-robbat2@gentoo.org>
X-Mailer: git-send-email 2.9.0
In-Reply-To: <20160619172717.18445-1-robbat2@gentoo.org>
References: <robbat2-20160619T083928-020175588Z@orbis-terrarum.net>
<20160619172717.18445-1-robbat2@gentoo.org>
X-Virus-Checked: Checked by ClamAV on orbis-terrarum.net
X-Spam-Score: -8.2
X-Spam-Score-int: -81
X-Spam-Bar: --------
X-Scanned-By: pepperfish.net, Sun, 19 Jun 2016 18:27:40 +0100
X-Spam-Report: Content analysis details: (-8.2 points)
pts rule name description
---- ---------------------- --------------------------------------------------
-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high
trust [140.211.166.183 listed in list.dnswl.org]
-0.0 SPF_PASS SPF: sender matches SPF record
-1.4 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
[score: 0.0000]
0.1 PPF_SPLIT_TAG RAW: Body contains a split HTML tag
X-ACL-Warn: message may be spam
X-Scan-Signature: ebfe5e7f0df0b84657034def484e72a4
Cc: "Robin H. Johnson" <robbat2@gentoo.org>
Subject: [PATCH] encryption: boost GPG performance.
X-BeenThere: obnam-dev@obnam.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Obnam development discussions <obnam-dev-obnam.org>
List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
<mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
List-Post: <mailto:obnam-dev@obnam.org>
List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
<mailto:obnam-dev-request@obnam.org?subject=subscribe>
Sender: obnam-dev-bounces@obnam.org
Errors-To: obnam-dev-bounces@obnam.org
Boost GPG performance:
- disabling compression during symmetric encryption.
- tuning symmetric key handling.
Also adds configuration options symmetric-cipher and symmetric-digest
for tuning GPG behavior.
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
---
obnamlib/encryption.py | 38 +++++++++++++++++++++++++----
obnamlib/plugins/encryption_plugin.py | 44 ++++++++++++++++++++++++++++++----
test-gpghome/random_seed | Bin 600 -> 600 bytes
3 files changed, 72 insertions(+), 10 deletions(-)
diff --git a/obnamlib/encryption.py b/obnamlib/encryption.py
index d2c78d0..d79f21f 100644
--- a/obnamlib/encryption.py
+++ b/obnamlib/encryption.py
@@ -102,13 +102,41 @@ def _gpg_pipe(args, data, passphrase, gpghome=None):
return out
-def encrypt_symmetric(cleartext, key, gpghome=None):
+def encrypt_symmetric(
+ cleartext, key,
+ gpghome=None,
+ cipher=None, # pylint: disable=unused-argument
+ digest=None): # pylint: disable=unused-argument
'''Encrypt data with symmetric encryption.'''
- return _gpg_pipe(['-c'], cleartext, key, gpghome=gpghome)
-
-
-def decrypt_symmetric(encrypted, key, gpghome=None):
+ opts = [
+ # Perform symmetric encryption
+ '-c',
+ # Disable compression as our data will be pre-compressed.
+ '--compress-algo', 'none',
+ # Key stretching is generally required as keys are raw random data.
+ # But, in case other parts of obnam reuse this key, salting it provides
+ # more variation than iteration, as on a single system the iterative
+ # count will generally remain consistent, as well as being being
+ # faster.
+ '--s2k-mode', '1',
+ ]
+ if cipher:
+ opts += ['--s2k-cipher-algo', cipher]
+ if digest:
+ opts += ['--s2k-digest-algo', digest]
+
+ return _gpg_pipe(opts, cleartext, key, gpghome=gpghome)
+
+
+def decrypt_symmetric(
+ encrypted, key,
+ gpghome=None,
+ cipher=None, # pylint: disable=unused-argument
+ digest=None): # pylint: disable=unused-argument
'''Decrypt encrypted data with symmetric encryption.'''
+ # cipher and digest are unused with GPG, as the values used to encrypt the
+ # data are stored in the S2K packet data.
+ # the parameters are here for future interface symmetry.
return _gpg_pipe(['-d'], encrypted, key, gpghome=gpghome)
diff --git a/obnamlib/plugins/encryption_plugin.py b/obnamlib/plugins/encryption_plugin.py
index 8c8eecf..a48c329 100644
--- a/obnamlib/plugins/encryption_plugin.py
+++ b/obnamlib/plugins/encryption_plugin.py
@@ -55,6 +55,18 @@ class EncryptionPlugin(obnamlib.ObnamPlugin):
metavar='HOMEDIR',
group=encryption_group,
default=None)
+ self.app.settings.string(
+ ['symmetric-cipher'],
+ 'GPG symmetric encryption cipher',
+ metavar='CIPHER',
+ group=encryption_group,
+ default=None)
+ self.app.settings.string(
+ ['symmetric-digest'],
+ 'GPG symmetric encryption passphrase digest',
+ metavar='DIGEST',
+ group=encryption_group,
+ default=None)
self.tag = "encrypt1"
@@ -112,6 +124,18 @@ class EncryptionPlugin(obnamlib.ObnamPlugin):
def symmetric_key_bits(self):
return int(self.app.settings['symmetric-key-bits'] or '256')
+ @property
+ def symmetric_cipher(self):
+ '''Get the symmetric cipher from config.
+ Return None for GPG default.'''
+ return self.app.settings['symmetric-cipher']
+
+ @property
+ def symmetric_digest(self):
+ '''Get the symmetric digest from config.
+ Return None for GPG default.'''
+ return self.app.settings['symmetric-digest']
+
def _write_file(self, repo, pathname, contents):
repo.get_fs().write_file(pathname, contents)
@@ -133,20 +157,30 @@ class EncryptionPlugin(obnamlib.ObnamPlugin):
self._write_file(repo, os.path.join(toplevel, 'key'), encrypted)
encoded = str(pubkeys)
- encrypted = obnamlib.encrypt_symmetric(encoded, symmetric_key)
+ encrypted = obnamlib.encrypt_symmetric(
+ encoded, symmetric_key,
+ gpghome=self.gnupghome,
+ cipher=self.symmetric_cipher,
+ digest=self.symmetric_digest)
self._write_file(repo, os.path.join(toplevel, 'userkeys'), encrypted)
def filter_read(self, encrypted, repo, toplevel):
symmetric_key = self.get_symmetric_key(repo, toplevel)
- return obnamlib.decrypt_symmetric(encrypted, symmetric_key,
- gpghome=self.gnupghome)
+ return obnamlib.decrypt_symmetric(
+ encrypted, symmetric_key,
+ gpghome=self.gnupghome,
+ cipher=self.symmetric_cipher,
+ digest=self.symmetric_digest)
def filter_write(self, cleartext, repo, toplevel):
if not self.keyid:
return cleartext
symmetric_key = self.get_symmetric_key(repo, toplevel)
- return obnamlib.encrypt_symmetric(cleartext, symmetric_key,
- gpghome=self.gnupghome)
+ return obnamlib.encrypt_symmetric(
+ cleartext, symmetric_key,
+ gpghome=self.gnupghome,
+ cipher=self.symmetric_cipher,
+ digest=self.symmetric_digest)
def get_symmetric_key(self, repo, toplevel):
key = self._symkeys.get(repo, toplevel)
diff --git a/test-gpghome/random_seed b/test-gpghome/random_seed
index f4ad4794cccb730cf965ead333d93493c4a721e4..421d386505c9f9dc662682137c0f4e3e4a31e917 100644
GIT binary patch
literal 600
zcmV-e0;l~%M+tI7i&1?H@__g{3A^S4QdWHxSy4^tdPW5AO6XxaV}F4g3=K|#f~8E(
zG<NQkhr2b`o5Ex|+6dFrV*VCX!6FgdP6NntyBbFzr03{>$dGz<^_n0T)IFj%Lq!}y
z6@=($YE82yjJ%b)1-Qd6ce1h@Mjf|NSShPUxU02=gR8VXYyB`CaX|<lfW=4X!l5cP
zqOE!2?*8ylTBcg~3iNx_tAmTv@j6(p{mnA1^g-6ObQ^?Az(GYxyxFM*$i)~RZhQBt
z$*&%$F^g~#{js4R4=8!TiiKemGHUaeHstb~8#V=!#IuNIXzu#&ev^tB@blYBOCF@<
zlvrln5H3}O_8@Cd%0&XBvOpVfBN=i(#KRHcm;DQtWra*ay@(-vypI~Nep3nvAJu+r
zPci|<3A=~}&Q^~`yxtN5veUfsUGitie`Nhah*oElplzCxi#OTngzkk`TK2uF5#eU+
zFotoVAbCU||9g2?^(OX$r+I<gqmJ-MUiNYv;-WZ~wEiPZm+8)`O%|{@!VnTpQk(nq
z=Thba*YJjhEJdiaEvBh=vw#;UhQ6Ivq+<sfhS4Hxg9{KN^k7%UxW&X!Fl}5FQe!_q
z$^^aFwEs*VELfZ@<cKYdd}=rke!}VKc9Gk6q9PzmQO3$PQsk&AF)}r!OADF4E`d@D
zTqW%AC*oqTjup^cN2xLilFnjMkJ0&4+GV9zCy-W#Hu_O~&gYNH;r_3&yne3Mms-XQ
mscB$LN?EyuGrJ~4{^Iw6-PpQ<0N_YCa$W;xu8J=XN<bjVPbI+s
literal 600
zcmV-e0;l~}85vVB9Ex=H1apCxKGXX|SEYg=Lp8dOc04?w%qBu*jTn)qxqrlORdsTD
ziH}6XMN%^O6xLUKCb8%Bc$h->lyA_*ers$|M_RqipCd)DSy^Cr=oF-0MAse>Y1px!
z$ZwMKyUpk!S3!PB?ou%H6iJe^N@3ygk#2{e9efyQ>Ef0Zrg=In)ja37N0V@Keh%^W
zs^I9qS4k(G>Fa|qOM@gT^N6CEh)-qBPB?Pm*yzNZ<SI`fB?T@&YzmfO1~N@X9g;g8
zNUX>6YK@lIPh08yoOX5F5%=5bv8#fOmlgEJ+^%wmmhYhoi;^F4L63KdG{q5WpG<?U
zGubx>>crD6fspt_xxX1I@HBZ9Iq;F0S=}&qh?2s-NmrRbx6!;Spp0TP0U5u7;F1y9
zZ8SrIXtL1NB|#FUU7d(!*7g#eTR4s#>frt|DJ-Kk)&{KMrTQRJ+g+Wu?kHaN06~i}
zz`2kqB*v`rrR_kO7EC#IdpZ=AlEo;O$V#4S%*cf%?J4m4M_v@0prK3=t2hs#*p*R4
zoQ}lQ1-nLHG_Xw7+zu36jNGWS_Gx3u6D~x&2SY)QtvHR<LhW*Y+4Rph%+wYDbIsv}
z?ld{7A7r?xPY+K&be>Lj$JnJ|0OksX<A8o<E1z-NgYxR#7X!oHxET>J=WLo5oMh9{
z1UyfWe!aA9?cWcTnSwH_^2rHP6?mvJk8S8KQt1}b$_cQ&_#C{To(bos5=|xFwMqG?
mEtumkn|KB_R5k5WtSjRBQOZQzHC@2Ny+3eufrduCa3Xsieky4I
--
2.9.0
_______________________________________________
obnam-dev mailing list
obnam-dev@obnam.org
http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org
|
