1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
|
Return-Path: <obnam-dev-bounces@obnam.org>
X-Original-To: distix@pieni.net
Delivered-To: distix@pieni.net
Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by pieni.net (Postfix) with ESMTPS id C410822F92
for <distix@pieni.net>; Sun, 19 Jun 2016 11:10:09 +0200 (CEST)
Received: from platypus.pepperfish.net (unknown [10.112.100.20])
by bagpuss.pepperfish.net (Postfix) with ESMTP id 36CF95B8;
Sun, 19 Jun 2016 10:10:09 +0100 (BST)
Received: from ip6-localhost ([::1] helo=platypus.pepperfish.net)
by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
id 1bEYk5-0005m3-0E; Sun, 19 Jun 2016 10:10:09 +0100
Received: from inmail0 ([10.112.100.10] helo=mx0.pepperfish.net)
by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
id 1bEYk2-0005lq-VO
for <obnam-dev@obnam.org>; Sun, 19 Jun 2016 10:10:07 +0100
Received: from smtp.gentoo.org ([140.211.166.183])
by mx0.pepperfish.net with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256)
(Exim 4.80) (envelope-from <robbat2@orbis-terrarum.net>)
id 1bEYk0-00071R-Id
for obnam-dev@obnam.org; Sun, 19 Jun 2016 10:10:06 +0100
Received: from grubbs.orbis-terrarum.net (localhost [127.0.0.1])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA (256/256 bits))
(No client certificate requested)
by smtp.gentoo.org (Postfix) with ESMTPS id 789EB3406DD
for <obnam-dev@obnam.org>; Sun, 19 Jun 2016 09:09:47 +0000 (UTC)
Received: (qmail 4870 invoked by uid 10000); 19 Jun 2016 09:09:47 -0000
Date: Sun, 19 Jun 2016 09:09:47 +0000
From: "Robin H. Johnson" <robbat2@orbis-terrarum.net>
To: "Robin H. Johnson" <robbat2@gentoo.org>
Message-ID: <robbat2-20160619T083928-020175588Z@orbis-terrarum.net>
MIME-Version: 1.0
User-Agent: Mutt/1.5.24 (2015-08-30)
X-Spam-Score: -8.4
X-Spam-Score-int: -83
X-Spam-Bar: --------
X-Scanned-By: pepperfish.net, Sun, 19 Jun 2016 10:10:06 +0100
X-Spam-Report: Content analysis details: (-8.4 points)
pts rule name description
---- ---------------------- --------------------------------------------------
-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high
trust [140.211.166.183 listed in list.dnswl.org]
-1.0 PPF_USER_AGENT_MUTT User-Agent: contains Mutt (Mutt isn't a spam
tool) -0.5 PPF_USER_AGENT User-Agent: exists
-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
[score: 0.0000]
X-ACL-Warn: message may be spam
X-Scan-Signature: a02e829f6933b0f49c4716235214d89f
Cc: obnam-support@obnam.org, obnam-dev@obnam.org
Subject: [2/2] GPG & performance: future options
X-BeenThere: obnam-dev@obnam.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Obnam development discussions <obnam-dev-obnam.org>
List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
<mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
List-Post: <mailto:obnam-dev@obnam.org>
List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
<mailto:obnam-dev-request@obnam.org?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============7343743822476944253=="
Mime-version: 1.0
Sender: obnam-dev-bounces@obnam.org
Errors-To: obnam-dev-bounces@obnam.org
--===============7343743822476944253==
Content-Type: multipart/signed; micalg=pgp-sha1;
protocol="application/pgp-signature"; boundary="it/zdz3K1bH9Y8/E"
Content-Disposition: inline
--it/zdz3K1bH9Y8/E
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
In part #1, we looked at GPG performance, and some of what could be done
to speed it up right away.
The root problem here is that GPG is expensive:
- execve() has a cost [2]
- all of the initialization has a cost [2]
- the S2K encoding has a cost.
I did a very quick hack to use PyCrypto's AES-256-CTR [3] for the symmetric
layer, and it showed tremendous promise (one_big_file: 10s baseline, 33s
pycrypto, 250-270s gpg)
What this DOES require, is that we need one of the following:
A) enciphered chunks must be self-describing (eg GPG S2K packet format
that bundles the parameters)
B) chunk encipherment parameters must be stored in a manner they can be
associated per-block.
Example parameters to store:
- Method (GPG, PyCrypto, Keyczar, NaCl...)
- cipher & blocksize
- key stretching if any [1]
- IV/counter
- checksum/HMAC of enciphered data block [without header] for
validation.
- compression layer under the encipherment
It's theoretically safe to have all of these parameters public, but I
don't see a significant loss if they were wrapped with the master key
[1] S2K does key-stretching by default, but since our symmetric keys are
actually purely random, we could consider turning it off and using pure
binary keys.
[2] I did look at ways of getting around the GPG startup overhead, and
the Assuan protocol is _very_ promising [in fact it'll help the
asymmetric operations a lot]. However it doesn't support symmetric modes
at all, and can't do parallizable-AES like CTR & GCM.
[3] I used what was available off-the-shelf in PyCrypto and known to be
fast. PyCrypto doesn't include any of the newer Authenticated modes like
GCM/EAX/CCM in a stable release yet, it got stuck on 2.7alpha1 :-(.
--=20
Robin Hugh Johnson
E-Mail : robbat2@orbis-terrarum.net
Home Page : http://www.orbis-terrarum.net/?l=3Dpeople.robbat2
ICQ# : 30269588 or 41961639
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
--it/zdz3K1bH9Y8/E
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.1
Comment: Robbat2 @ Orbis-Terrarum Networks - The text below is a digital signature. If it doesn't make any sense to you, ignore it.
iKYEARECAGYFAldmYVpfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl
bnBncC5maWZ0aGhvcnNlbWFuLm5ldDc1OTQwNEJFQkQ0MUY3MTIzODIzODZFRjNF
OTIyQzIyMzIzM0MyMkMACgkQPpIsIjIzwixbRgCgx0zn2ZHtlZ+lkAuK1+nfziEB
ZgYAoPKOYPQ8ocaTZciWy76XzwifUHI+
=pQMQ
-----END PGP SIGNATURE-----
--it/zdz3K1bH9Y8/E--
--===============7343743822476944253==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
obnam-dev mailing list
obnam-dev@obnam.org
http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org
--===============7343743822476944253==--
|
