summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authordistix ticketing system <distix@pieni.net>2017-07-19 18:21:15 +0000
committerdistix ticketing system <distix@pieni.net>2017-07-19 18:21:15 +0000
commitd724406b066d1b790ac584d9c8bb3a3d13aaf7a6 (patch)
treee3b7842d598681346ec2265a434a0f369f22cac3
parent4cc2974d9ebbf9cdf7a23858361c2cbb973f61ac (diff)
downloadobnam-dev-distix-d724406b066d1b790ac584d9c8bb3a3d13aaf7a6.tar.gz
imported mails
-rw-r--r--tickets/6e2a44fcd630472b82cf898f26a7f936/Maildir/new/1500488475.M165390P28121Q1.koom240
-rw-r--r--tickets/6e2a44fcd630472b82cf898f26a7f936/ticket.yaml2
2 files changed, 242 insertions, 0 deletions
diff --git a/tickets/6e2a44fcd630472b82cf898f26a7f936/Maildir/new/1500488475.M165390P28121Q1.koom b/tickets/6e2a44fcd630472b82cf898f26a7f936/Maildir/new/1500488475.M165390P28121Q1.koom
new file mode 100644
index 0000000..7a38577
--- /dev/null
+++ b/tickets/6e2a44fcd630472b82cf898f26a7f936/Maildir/new/1500488475.M165390P28121Q1.koom
@@ -0,0 +1,240 @@
+Return-Path: <obnam-dev-bounces@obnam.org>
+X-Original-To: distix@pieni.net
+Delivered-To: distix@pieni.net
+Received: from yaffle.pepperfish.net (yaffle.pepperfish.net [88.99.213.221])
+ by pieni.net (Postfix) with ESMTPS id 39B3F415D4
+ for <distix@pieni.net>; Wed, 19 Jul 2017 18:18:54 +0000 (UTC)
+Received: from platypus.pepperfish.net (unknown [10.112.101.20])
+ by yaffle.pepperfish.net (Postfix) with ESMTP id 6B84741879;
+ Wed, 19 Jul 2017 19:18:46 +0100 (BST)
+Received: from ip6-localhost.nat ([::1] helo=platypus.pepperfish.net)
+ by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
+ id 1dXtYc-0005NF-DX; Wed, 19 Jul 2017 19:18:46 +0100
+Received: from [10.112.101.21] (helo=mx3.pepperfish.net)
+ by platypus.pepperfish.net with esmtps (Exim 4.80 #2 (Debian))
+ id 1dXtYa-0005Mm-HX
+ for <obnam-dev@obnam.org>; Wed, 19 Jul 2017 19:18:44 +0100
+Received: from koom.pieni.net ([88.99.190.206] helo=pieni.net)
+ by mx3.pepperfish.net with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
+ (Exim 4.89) (envelope-from <liw@liw.fi>) id 1dXtYY-000300-Es
+ for obnam-dev@obnam.org; Wed, 19 Jul 2017 19:18:44 +0100
+Received: from exolobe3.liw.fi (82-181-57-241.bb.dnainternet.fi
+ [82.181.57.241]) by pieni.net (Postfix) with ESMTPSA id B2D52415D4;
+ Wed, 19 Jul 2017 18:18:36 +0000 (UTC)
+Received: from liw.fi (localhost [127.0.0.1])
+ by exolobe3.liw.fi (Postfix) with ESMTPS id 5A589121DE2;
+ Wed, 19 Jul 2017 21:18:35 +0300 (EEST)
+Date: Wed, 19 Jul 2017 21:18:34 +0300
+From: Lars Wirzenius <liw@liw.fi>
+To: Dionisus Torimens <DJtm@gmx.net>
+Message-ID: <20170719181834.pboi4gfnb3vshi3b@liw.fi>
+References: <trinity-fc08f006-f2ea-48b7-bbe8-5be4e764c20a-1492598661322@3capp-gmx-bs37>
+MIME-Version: 1.0
+In-Reply-To: <trinity-fc08f006-f2ea-48b7-bbe8-5be4e764c20a-1492598661322@3capp-gmx-bs37>
+User-Agent: NeoMutt/20170113 (1.7.2)
+X-Pepperfish-Transaction: cc71-fbaa-98a7-c677
+X-Spam-Score: -2.8
+X-Spam-Score-int: -27
+X-Spam-Bar: --
+X-Scanned-By: pepperfish.net, Wed, 19 Jul 2017 19:18:44 +0100
+X-Spam-Report: Content analysis details: (-2.8 points)
+ pts rule name description
+ ---- ---------------------- --------------------------------------------------
+ 0.2 PPF_REFERENCES_NODOTS References contains no dots after the @
+ 0.2 PPF_INREPLYTO_NODOTS In-Reply-To contains no dots after the @
+ -0.5 PPF_USER_AGENT User-Agent: exists
+ -1.0 PPF_USER_AGENT_MUTT User-Agent: contains Mutt (Mutt isn't a spam
+ tool)
+ -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
+ [score: 0.0000]
+ 0.2 PPF_NUMERIC_ENTITY RAW: Body contains numeric HTML entities
+X-ACL-Warn: message may be spam
+X-Scan-Signature: e056c7610997a266061b7a558bad18c3
+Cc: obnam-dev@obnam.org
+Subject: Re: [rfc] [PATCH] parallelize listdir
+X-BeenThere: obnam-dev@obnam.org
+X-Mailman-Version: 2.1.5
+Precedence: list
+List-Id: Obnam development discussions <obnam-dev-obnam.org>
+List-Unsubscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=unsubscribe>
+List-Archive: <http://listmaster.pepperfish.net/pipermail/obnam-dev-obnam.org>
+List-Post: <mailto:obnam-dev@obnam.org>
+List-Help: <mailto:obnam-dev-request@obnam.org?subject=help>
+List-Subscribe: <http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org>,
+ <mailto:obnam-dev-request@obnam.org?subject=subscribe>
+Content-Type: multipart/mixed; boundary="===============9175575107155843625=="
+Mime-version: 1.0
+Sender: obnam-dev-bounces@obnam.org
+Errors-To: obnam-dev-bounces@obnam.org
+
+
+--===============9175575107155843625==
+Content-Type: multipart/signed; micalg=pgp-sha512;
+ protocol="application/pgp-signature"; boundary="zkikvhrkqmxob2zj"
+Content-Disposition: inline
+
+
+--zkikvhrkqmxob2zj
+Content-Type: text/plain; charset=us-ascii
+Content-Disposition: inline
+Content-Transfer-Encoding: quoted-printable
+
+Thanks for the suggestion.
+
+After FORMAT GREEN ALBATROSS is ready for production, I'm happy to
+start thining about other changes, particularly architectural ones. I
+would like, for example, to allow the user to tell Obnam that a
+particular file may have changed and have Obnam back up that file
+only, and assume no other changes to the file system. This could then
+be run by a program that hooks into the kernel to notice changes to
+the filesystem. You change a file, Obnam backs it up about instantly.
+
+But not now, the wide-winged bird is mocking me too much, I'm afraid.
+
+On Wed, Apr 19, 2017 at 12:44:21PM +0200, Dionisus Torimens wrote:
+> <html><head></head><body><div style=3D"font-family: Verdana;font-size: 12=
+=2E0px;"><div>Hi,</div>
+>=20
+> <div>&nbsp;</div>
+>=20
+> <div>an initial try to use a bit of&nbsp;multiprocessing. Makes obnam a b=
+it faster in scannig. No change in the algorithms. I&#39;d love to see some=
+ benchmarking, it feels a bit snappier. The pathos library is very helpful,=
+ but not in most distribution packages afaik.</div>
+>=20
+> <div>&nbsp;</div>
+>=20
+> <div>I&#39;ve been thinking and I guess a good solution in the longer ter=
+m would be to have</div>
+>=20
+> <div>- a scanning process&nbsp;to create a list with changes,</div>
+>=20
+> <div>- another to prepare the uploads and</div>
+>=20
+> <div>- another to actually upload files to the repository.</div>
+>=20
+> <div>&nbsp;</div>
+>=20
+> <div>Cheers</div></div></body></html>
+> >From fe5333545cda1ac13fb10ae7b93c5f022f518f05 Mon Sep 17 00:00:00 2001
+> From: djtm <djtm@users.noreply.github.com>
+> Date: Wed, 19 Apr 2017 12:28:23 +0200
+> Subject: [PATCH] *** SUBJECT HERE ***
+>=20
+> *** BLURB HERE ***
+>=20
+> djtm (1):
+> parallelize listdir
+>=20
+> obnamlib/vfs_local.py | 25 +++++++++++++++----------
+> 1 file changed, 15 insertions(+), 10 deletions(-)
+>=20
+> --=20
+> 2.7.4
+>=20
+> >From fe5333545cda1ac13fb10ae7b93c5f022f518f05 Mon Sep 17 00:00:00 2001
+> From: djtm <djtm@users.noreply.github.com>
+> Date: Tue, 18 Apr 2017 02:54:18 +0200
+> Subject: [PATCH] parallelize listdir
+>=20
+> ---
+> obnamlib/vfs_local.py | 25 +++++++++++++++----------
+> 1 file changed, 15 insertions(+), 10 deletions(-)
+>=20
+> diff --git a/obnamlib/vfs_local.py b/obnamlib/vfs_local.py
+> index e01617e..697ef51 100644
+> --- a/obnamlib/vfs_local.py
+> +++ b/obnamlib/vfs_local.py
+> @@ -24,6 +24,9 @@ import tempfile
+> import time
+> import tracing
+> =20
+> +from pathos.multiprocessing import ProcessingPool
+> +from functools import partial
+> +
+> import obnamlib
+> =20
+> =20
+> @@ -411,17 +414,19 @@ class LocalFS(obnamlib.VirtualFileSystem):
+> def listdir(self, dirname):
+> return os.listdir(self.join(dirname))
+> =20
+> + def listdir2helper(self, dirname, name):
+> + try:
+> + st =3D self.lstat(os.path.join(dirname, name))
+> + except OSError, e: # pragma: no cover
+> + st =3D e
+> + ino =3D -1
+> + else:
+> + ino =3D st.st_ino
+> + return (ino, name, st)
+> +
+> def listdir2(self, dirname):
+> - result =3D []
+> - for name in self.listdir(dirname):
+> - try:
+> - st =3D self.lstat(os.path.join(dirname, name))
+> - except OSError, e: # pragma: no cover
+> - st =3D e
+> - ino =3D -1
+> - else:
+> - ino =3D st.st_ino
+> - result.append((ino, name, st))
+> + pool =3D ProcessingPool()
+> + result =3D pool.map(partial(self.listdir2helper, dirname), self.=
+listdir(dirname))
+> =20
+> # We sort things in inode order, for speed when doing name looku=
+ps
+> # when backing up.
+> --=20
+> 2.7.4
+>=20
+
+> _______________________________________________
+> obnam-dev mailing list
+> obnam-dev@obnam.org
+> http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam=
+=2Eorg
+
+
+--=20
+I want to build worthwhile things that might last. --joeyh
+
+--zkikvhrkqmxob2zj
+Content-Type: application/pgp-signature; name="signature.asc"
+
+-----BEGIN PGP SIGNATURE-----
+
+iQIzBAABCgAdFiEETNTnrewG6wEE1EJ3bC+mFux6IDEFAllvonkACgkQbC+mFux6
+IDEqPQ/+N7ZCdOviAvqpYGR3sMGHSkd5ZO22sE4ysCOTFtO6+pDg2K1y29u8lMVQ
+KOK9cNxzhglMMKKGcegq/u4DZaW2w0PG+tiZ+9xD60+PcshfsQxdKRKZVzc5Zgw/
+SpIpho7kRHJmXNvUpR417BdXhb5BZv24HEezhXA2WVtKxOl4R1U5TrqhucVXKgVi
+sE5QrPL9l4Oj20cGHIaxorEPWlx/RDqLFzsnty7v6NjXg7FzR/M2RQHwJiawgeMW
+J6c3e92V4qvqUy4EGt5ekpzs/ianx0dQrjim0+lcXCtL5PuyfNXDNT8wzqpfAA0r
+x33E72DyGg8Jpa34AEJ0M/H9tDkhgHIv4/pENF0uukba06RdEUN8u7dARJNiTqlp
+T+zIIJOkal/fTMkMbQLy0Btq3gpfnnV576JLF6/9A6L5qxM/eKa+XmEBEtmxquJB
+CchKdLs50ZCIue+K/4b+CmRqrpiz4s9465hqXGtjKklSdzhgVdNy7wMYFvv2N1Tg
+Kow/W8Y1y/Bh+DBGnsEbPDXh6eeHI3h962eFJsAPxCfj/Yo3JMZ01Dgl4nr7/NLZ
+nYCkERmskDo9DpF9OCZgBqd1y3KMibM5kCK6j3pVKnzH5MZ/O68H09f+l1HARM03
+9cUgU2vWE1a/cBFdLaYtUXrdYF4Q+Ph/WhSVSOFEZs9vs7FaSW4=
+=GZud
+-----END PGP SIGNATURE-----
+
+--zkikvhrkqmxob2zj--
+
+
+--===============9175575107155843625==
+Content-Type: text/plain; charset="us-ascii"
+MIME-Version: 1.0
+Content-Transfer-Encoding: 7bit
+Content-Disposition: inline
+
+_______________________________________________
+obnam-dev mailing list
+obnam-dev@obnam.org
+http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org
+
+--===============9175575107155843625==--
+
diff --git a/tickets/6e2a44fcd630472b82cf898f26a7f936/ticket.yaml b/tickets/6e2a44fcd630472b82cf898f26a7f936/ticket.yaml
index 29c4424..9268707 100644
--- a/tickets/6e2a44fcd630472b82cf898f26a7f936/ticket.yaml
+++ b/tickets/6e2a44fcd630472b82cf898f26a7f936/ticket.yaml
@@ -1,3 +1,5 @@
+status:
+- ''
ticket-id:
- 6e2a44fcd630472b82cf898f26a7f936
title: