1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
|
Return-Path: <obnam-dev-bounces@obnam.org>
X-Original-To: distix@pieni.net
Delivered-To: distix@pieni.net
Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by pieni.net (Postfix) with ESMTPS id CCB862268D
for <distix@pieni.net>; Sun, 19 Jun 2016 10:39:27 +0200 (CEST)
Received: from platypus.pepperfish.net (unknown [10.112.100.20])
by bagpuss.pepperfish.net (Postfix) with ESMTP id 3AED25BC;
Sun, 19 Jun 2016 09:39:27 +0100 (BST)
Received: from ip6-localhost ([::1] helo=platypus.pepperfish.net)
by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
id 1bEYGN-0002Y5-2C; Sun, 19 Jun 2016 09:39:27 +0100
Received: from inmail0 ([10.112.100.10] helo=mx0.pepperfish.net)
by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
id 1bEYGL-0002Xn-Q5
for <obnam-dev@obnam.org>; Sun, 19 Jun 2016 09:39:25 +0100
Received: from smtp.gentoo.org ([140.211.166.183])
by mx0.pepperfish.net with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256)
(Exim 4.80) (envelope-from <robbat2@gentoo.org>) id 1bEYGH-00068d-FU
for obnam-dev@obnam.org; Sun, 19 Jun 2016 09:39:25 +0100
Received: from grubbs.orbis-terrarum.net (localhost [127.0.0.1])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA (256/256 bits))
(No client certificate requested)
by smtp.gentoo.org (Postfix) with ESMTPS id C008834067D
for <obnam-dev@obnam.org>; Sun, 19 Jun 2016 08:39:04 +0000 (UTC)
Received: (qmail 8958 invoked by uid 10000); 19 Jun 2016 08:39:04 -0000
Date: Sun, 19 Jun 2016 08:39:04 +0000
From: "Robin H. Johnson" <robbat2@gentoo.org>
To: obnam-support@obnam.org, obnam-dev@obnam.org
Message-ID: <20160619083904.GA18768@orbis-terrarum.net>
MIME-Version: 1.0
User-Agent: Mutt/1.5.24 (2015-08-30)
X-Spam-Score: -9.8
X-Spam-Score-int: -97
X-Spam-Bar: ---------
X-Scanned-By: pepperfish.net, Sun, 19 Jun 2016 09:39:25 +0100
X-Spam-Report: Content analysis details: (-9.8 points)
pts rule name description
---- ---------------------- --------------------------------------------------
-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high
trust [140.211.166.183 listed in list.dnswl.org]
-1.0 PPF_USER_AGENT_MUTT User-Agent: contains Mutt (Mutt isn't a spam
tool) -0.5 PPF_USER_AGENT User-Agent: exists
-0.0 SPF_PASS SPF: sender matches SPF record
-1.4 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
[score: 0.0000]
X-ACL-Warn: message may be spam
X-Scan-Signature: 9b2f0c8d06ad4d4ac6b47be2df4622cd
Subject: [1/2] GPG & performance: a deep-dive
X-BeenThere: obnam-dev@obnam.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Obnam development discussions <obnam-dev-obnam.org>
List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
<mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
List-Post: <mailto:obnam-dev@obnam.org>
List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
<mailto:obnam-dev-request@obnam.org?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============8343931285586773396=="
Mime-version: 1.0
Sender: obnam-dev-bounces@obnam.org
Errors-To: obnam-dev-bounces@obnam.org
--===============8343931285586773396==
Content-Type: multipart/signed; micalg=pgp-sha1;
protocol="application/pgp-signature"; boundary="APlYHCtpeOhspHkB"
Content-Disposition: inline
--APlYHCtpeOhspHkB
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Hi all,
I've been looking at backup options for a deployment, and in considering
obnam, I like it's general speed, but found that it dropped unacceptably
when encryption was enabled.
TL;DR: suggestions
- Right now: set '-z 0' in obnam symmetric crypto call, immediate 10%
performance boost.
- Plan for moving to PyCrypto or other for symmetric crypto
A first pass examination pointed strongly to obnam's of GPG symmetric
encryption.=20
I improved the obnam-benchmark tool to help take these measurements
below, the changes are on GitHub [1]; but first let's look at how GPG
does symmetric encryption.
GPG symmetric encryption (S2K) does the following:
- takes a passphrase & data input,
- optionally transforms the passphrase.
(see s2k-digest-algo, s2k-mode, s2k-count)
- optionally compresses the input
(compress-level)
- enciphers the output
(see s2k-cipher-algo)
- emit output in the S2K structure
This records all of the above s2k-* parameters, as well.
Stock obnam simply calls 'gpg -c' for symmetric encryption.
In the absence of any other configuration, this generally has the
following defaults:
- s2k-digest-algo=3DSHA1
- s2k-mode=3D3 (key stretching by repeated hashing)
- s2k-count=3Dvaries, my systems are 25M..65M
- compress-level=3D6 (ZLIB level 6)
- s2k-cipher-algo=3DAES128
It uses the cipher in a modified CFB mode [RFC4880, sec 5.7, ... "Tag 9"]
Naively, you might think that GPG is fast enough. Sure, take a 1GB
incompressible input, as a single file.
0.6s | cat in > out
0.8s | gpg --store -z 0
22.3s | gpg --store -z 6
5.6s | gpg --symmetric -z 0
27.4s | gpg --symmetric -z 6 # Default settings!
S2K packet encoding: 33% slower
Compression, used by default: 5-28x performance hit
Symmetric enciphering: ~7x performance hit
Overall: 45x slower
The catch is that obnam calls gpg many many times, with much smaller
inputs, so we have to pay the startup costs many times over.
I set out to measure the cost breakdown of using gpg:
- exec overhead
- S2K packet overhead
- symmetric encryption
- S2K compression
With the stock codebase, the gpg encryption plugin has this approx
performance effect for me:
- many_files benchmark, it's only a 20% hit, but there are only 256
unique values.=20
- On the big_file benchmark, it's ~45x slower (than cat)
First, the reference/stock runs:
A: rsync -a live backup && rsync -a backup restore
B: stock obnam, run with obnam-benchmark, production.yaml, no encryption
C: stock as B, with gpg encryption (compressed symmetric encryption)
Now the modified code variants:
W: gpg, symmetric encryption, uncompressed
X: gpg, s/--symmetric/--store/, compressed
Y: gpg, s/--symmetric/--store/, uncompressed
Z: HACK gpg-symmetric to just return the raw block
PYC: quick hack for PyCrypto AES-256-CTR
B->Z: obnam's overhead in asymmetric encryption only.
Z->Y: this is the overhead added by obnam using GPG symmetric encryption
(on top of the asymmetric management).
Y->X,=20
C->W: this is the overhead added by S2K compression on stored &
enciphering.
Y->W: this is the overhead added by enciphering, with no compression
Timing data:
------------
in seconds, average of 3 runs.
B1 =3D many_files
B2 =3D one_big_file
Benchmark
Test| B1 | B2
----+------+-------
A | 15.0| 5.1
B | 225.4| 10.2
C | 284.8| 272.5
----+------+------
W | 288.3| 246.4
X | 266.1| 57.7
Y | 266.4| 33.9
Z | 249.9| 14.4
----+------+------
PYC | 236.4| 33.6
----+------+------
[1] https://github.com/robbat2/obnam-benchmarks/tree/robbat2/flexibility
--=20
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Trustee & Treasurer
E-Mail : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136
--APlYHCtpeOhspHkB
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.1
Comment: Robbat2 @ Orbis-Terrarum Networks - The text below is a digital signature. If it doesn't make any sense to you, ignore it.
iKYEARECAGYFAldmWihfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl
bnBncC5maWZ0aGhvcnNlbWFuLm5ldDc1OTQwNEJFQkQ0MUY3MTIzODIzODZFRjNF
OTIyQzIyMzIzM0MyMkMACgkQPpIsIjIzwixPYACgzqgcY2uxMJwMmjBYG/HPXYAu
jo4AoNhPOQjVu1CF7eE8RPG2cndRd2Fk
=ukL+
-----END PGP SIGNATURE-----
--APlYHCtpeOhspHkB--
--===============8343931285586773396==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
obnam-dev mailing list
obnam-dev@obnam.org
http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org
--===============8343931285586773396==--
|
