From 8c087e41fdda5d9508b9c95ea57fbd079142633d Mon Sep 17 00:00:00 2001 From: Lars Wirzenius Date: Sat, 20 Feb 2016 22:10:13 +0200 Subject: imported mails --- .../cur/.this-dir-not-empty/.empty/empty-file | 0 .../new/.this-dir-not-empty/.empty/empty-file | 0 .../new/1455998999.M19944P17339Q86.exolobe1 | 111 +++++++++++++++++++++ .../new/1455998999.M64424P17339Q88.exolobe1 | 111 +++++++++++++++++++++ .../tmp/.this-dir-not-empty/.empty/empty-file | 0 .../d8f289b47b7a4b6893c47db4d7eba444/ticket.yaml | 4 + 6 files changed, 226 insertions(+) create mode 100644 tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/cur/.this-dir-not-empty/.empty/empty-file create mode 100644 tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/new/.this-dir-not-empty/.empty/empty-file create mode 100644 tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/new/1455998999.M19944P17339Q86.exolobe1 create mode 100644 tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/new/1455998999.M64424P17339Q88.exolobe1 create mode 100644 tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/tmp/.this-dir-not-empty/.empty/empty-file create mode 100644 tickets/d8f289b47b7a4b6893c47db4d7eba444/ticket.yaml (limited to 'tickets/d8f289b47b7a4b6893c47db4d7eba444') diff --git a/tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/cur/.this-dir-not-empty/.empty/empty-file b/tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/cur/.this-dir-not-empty/.empty/empty-file new file mode 100644 index 0000000..e69de29 diff --git a/tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/new/.this-dir-not-empty/.empty/empty-file b/tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/new/.this-dir-not-empty/.empty/empty-file new file mode 100644 index 0000000..e69de29 diff --git a/tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/new/1455998999.M19944P17339Q86.exolobe1 b/tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/new/1455998999.M19944P17339Q86.exolobe1 new file mode 100644 index 0000000..520173a --- /dev/null +++ b/tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/new/1455998999.M19944P17339Q86.exolobe1 @@ -0,0 +1,111 @@ +Return-Path: +X-Original-To: distix@pieni.net +Delivered-To: distix@pieni.net +Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16]) + (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) + (No client certificate requested) + by pieni.net (Postfix) with ESMTPS id AC4DE2062E + for ; Sat, 1 Nov 2014 22:51:22 +0100 (CET) +Received: from platypus.pepperfish.net (unknown [10.112.100.20]) + by bagpuss.pepperfish.net (Postfix) with ESMTP id 44281345B; + Sat, 1 Nov 2014 21:51:22 +0000 (GMT) +Received: from localhost ([::1] helo=platypus.pepperfish.net) + by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian)) + id 1XkgZu-0005gG-55; Sat, 01 Nov 2014 21:51:22 +0000 +Received: from inmail ([10.112.100.10] helo=mx0.pepperfish.net) + by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian)) + id 1XkgZs-0005g9-Pm + for ; Sat, 01 Nov 2014 21:51:20 +0000 +Received: from smtp5-g21.free.fr ([212.27.42.5]) + by mx0.pepperfish.net with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256) + (Exim 4.80) (envelope-from ) + id 1XkgZo-0003PE-Cr + for obnam-dev@obnam.org; Sat, 01 Nov 2014 21:51:20 +0000 +Received: from [10.0.0.5] (unknown [62.147.215.125]) + by smtp5-g21.free.fr (Postfix) with ESMTP id AB21ED48044 + for ; Sat, 1 Nov 2014 22:49:07 +0100 (CET) +Message-ID: <545555CF.4020405@pcedev.com> +Date: Sat, 01 Nov 2014 22:51:11 +0100 +From: Olivier Jolly +User-Agent: Mozilla/5.0 (X11; Linux x86_64; + rv:24.0) Gecko/20100101 Thunderbird/24.6.0 +MIME-Version: 1.0 +To: obnam-dev@obnam.org +Content-Type: text/plain; charset=ISO-8859-1; format=flowed +Content-Transfer-Encoding: 7bit +X-Spam-Score: -3.1 +X-Spam-Score-int: -30 +X-Spam-Bar: --- +X-Scanned-By: pepperfish.net, Sat, 01 Nov 2014 21:51:19 +0000 +X-Spam-Report: Content analysis details: (-3.1 points) + pts rule name description + ---- ---------------------- -------------------------------------------------- + -0.5 PPF_USER_AGENT User-Agent: exists + -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low + trust [212.27.42.5 listed in list.dnswl.org] + -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% + [score: 0.0000] +Subject: R43272X "fix" proposal +X-BeenThere: obnam-dev@obnam.org +X-Mailman-Version: 2.1.5 +Precedence: list +List-Id: Obnam development discussions +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +Sender: obnam-dev-bounces@obnam.org +Errors-To: obnam-dev-bounces@obnam.org + +hi, + regarding the R43272X error about missing chunk which is being +discussed recently on the support mailing list, I've also been impacted +by it and spent a bit of time investigating it but would prefer to have +some feedback to know whether I should keep going in this direction or not. + + My understanding is that this error indicate a missing file which +holds a chunk of data for one or more files (or part of one or more files). + Basically, it's an unrecoverable error. + + However, as it is coded today, when this condition is met, this error +leads to a stop of the current sub command (like forget, probably +restore and mount). + + In order to recover from it as good as possible, I wondered whether +the fsck sub command may dereference the file pointing to the missing +chunk. To do this, I looked at the fsck code and the chunk checking work +item can be given the file+genid related to the checked chunk to have +knowledge of the file referencing the missing data. + At this point, as far as I understood the repo interface doesn't have +a working method to remove a file from a past generation (there are +several comment about repo alteration methods only working on the last +generation). + + Reading the Btree/larch documentation and code, I think that the +removal of the key for a file in a given generation (ie the given +generation Btree) should be working. So I tried to perform the +generation btree loading, perform the removal on the tree (pasting most +of the existing file 'remove' method) and then commiting the journal via +the commit method on the client metadata (where I'm unsure it would +propagate to the btree I've altered). + + At this point, it didn't change anything and before diving into the +code again, I wanted to have some feedback regarding the overall +approach of dealing with R43272X (because it could be handled by making +this error not fatal and implying soft "empty/missing file" behaviour +hence leading to a fresh backup on the next backup attempt to mitigate +the data loss). And if the reference removal is agreed on, I wanted to +know if I was right in thinking that reference removal in the given +generation would be a potential fix (potentially with a file salvage +option to recover the valid chunks for the corrupted file). + + Thanks in advance, +Olivier Jolly + +_______________________________________________ +obnam-dev mailing list +obnam-dev@obnam.org +http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org diff --git a/tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/new/1455998999.M64424P17339Q88.exolobe1 b/tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/new/1455998999.M64424P17339Q88.exolobe1 new file mode 100644 index 0000000..520173a --- /dev/null +++ b/tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/new/1455998999.M64424P17339Q88.exolobe1 @@ -0,0 +1,111 @@ +Return-Path: +X-Original-To: distix@pieni.net +Delivered-To: distix@pieni.net +Received: from bagpuss.pepperfish.net (bagpuss.pepperfish.net [148.251.8.16]) + (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) + (No client certificate requested) + by pieni.net (Postfix) with ESMTPS id AC4DE2062E + for ; Sat, 1 Nov 2014 22:51:22 +0100 (CET) +Received: from platypus.pepperfish.net (unknown [10.112.100.20]) + by bagpuss.pepperfish.net (Postfix) with ESMTP id 44281345B; + Sat, 1 Nov 2014 21:51:22 +0000 (GMT) +Received: from localhost ([::1] helo=platypus.pepperfish.net) + by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian)) + id 1XkgZu-0005gG-55; Sat, 01 Nov 2014 21:51:22 +0000 +Received: from inmail ([10.112.100.10] helo=mx0.pepperfish.net) + by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian)) + id 1XkgZs-0005g9-Pm + for ; Sat, 01 Nov 2014 21:51:20 +0000 +Received: from smtp5-g21.free.fr ([212.27.42.5]) + by mx0.pepperfish.net with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256) + (Exim 4.80) (envelope-from ) + id 1XkgZo-0003PE-Cr + for obnam-dev@obnam.org; Sat, 01 Nov 2014 21:51:20 +0000 +Received: from [10.0.0.5] (unknown [62.147.215.125]) + by smtp5-g21.free.fr (Postfix) with ESMTP id AB21ED48044 + for ; Sat, 1 Nov 2014 22:49:07 +0100 (CET) +Message-ID: <545555CF.4020405@pcedev.com> +Date: Sat, 01 Nov 2014 22:51:11 +0100 +From: Olivier Jolly +User-Agent: Mozilla/5.0 (X11; Linux x86_64; + rv:24.0) Gecko/20100101 Thunderbird/24.6.0 +MIME-Version: 1.0 +To: obnam-dev@obnam.org +Content-Type: text/plain; charset=ISO-8859-1; format=flowed +Content-Transfer-Encoding: 7bit +X-Spam-Score: -3.1 +X-Spam-Score-int: -30 +X-Spam-Bar: --- +X-Scanned-By: pepperfish.net, Sat, 01 Nov 2014 21:51:19 +0000 +X-Spam-Report: Content analysis details: (-3.1 points) + pts rule name description + ---- ---------------------- -------------------------------------------------- + -0.5 PPF_USER_AGENT User-Agent: exists + -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low + trust [212.27.42.5 listed in list.dnswl.org] + -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% + [score: 0.0000] +Subject: R43272X "fix" proposal +X-BeenThere: obnam-dev@obnam.org +X-Mailman-Version: 2.1.5 +Precedence: list +List-Id: Obnam development discussions +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +Sender: obnam-dev-bounces@obnam.org +Errors-To: obnam-dev-bounces@obnam.org + +hi, + regarding the R43272X error about missing chunk which is being +discussed recently on the support mailing list, I've also been impacted +by it and spent a bit of time investigating it but would prefer to have +some feedback to know whether I should keep going in this direction or not. + + My understanding is that this error indicate a missing file which +holds a chunk of data for one or more files (or part of one or more files). + Basically, it's an unrecoverable error. + + However, as it is coded today, when this condition is met, this error +leads to a stop of the current sub command (like forget, probably +restore and mount). + + In order to recover from it as good as possible, I wondered whether +the fsck sub command may dereference the file pointing to the missing +chunk. To do this, I looked at the fsck code and the chunk checking work +item can be given the file+genid related to the checked chunk to have +knowledge of the file referencing the missing data. + At this point, as far as I understood the repo interface doesn't have +a working method to remove a file from a past generation (there are +several comment about repo alteration methods only working on the last +generation). + + Reading the Btree/larch documentation and code, I think that the +removal of the key for a file in a given generation (ie the given +generation Btree) should be working. So I tried to perform the +generation btree loading, perform the removal on the tree (pasting most +of the existing file 'remove' method) and then commiting the journal via +the commit method on the client metadata (where I'm unsure it would +propagate to the btree I've altered). + + At this point, it didn't change anything and before diving into the +code again, I wanted to have some feedback regarding the overall +approach of dealing with R43272X (because it could be handled by making +this error not fatal and implying soft "empty/missing file" behaviour +hence leading to a fresh backup on the next backup attempt to mitigate +the data loss). And if the reference removal is agreed on, I wanted to +know if I was right in thinking that reference removal in the given +generation would be a potential fix (potentially with a file salvage +option to recover the valid chunks for the corrupted file). + + Thanks in advance, +Olivier Jolly + +_______________________________________________ +obnam-dev mailing list +obnam-dev@obnam.org +http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/obnam-dev-obnam.org diff --git a/tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/tmp/.this-dir-not-empty/.empty/empty-file b/tickets/d8f289b47b7a4b6893c47db4d7eba444/Maildir/tmp/.this-dir-not-empty/.empty/empty-file new file mode 100644 index 0000000..e69de29 diff --git a/tickets/d8f289b47b7a4b6893c47db4d7eba444/ticket.yaml b/tickets/d8f289b47b7a4b6893c47db4d7eba444/ticket.yaml new file mode 100644 index 0000000..db90eb6 --- /dev/null +++ b/tickets/d8f289b47b7a4b6893c47db4d7eba444/ticket.yaml @@ -0,0 +1,4 @@ +ticket-id: +- d8f289b47b7a4b6893c47db4d7eba444 +title: +- R43272X "fix" proposal -- cgit v1.2.1