From 1de57d9df3992af95e5eacd8d0f149dd94c8875c Mon Sep 17 00:00:00 2001 From: Lars Wirzenius Date: Tue, 20 May 2014 15:00:48 +0100 Subject: Add README --- README | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 README diff --git a/README b/README new file mode 100644 index 0000000..44ea171 --- /dev/null +++ b/README @@ -0,0 +1,19 @@ +A proof of concept for transferring sparse files +================================================ + +In Baserock, we often need to transfer sparse files across the +network. At the moment we do that by ignoring that they're sparse: +just read the holes as long sequences of zero bytes. That works, but +is inefficient. + +This directory contains a couple of scripts: xfer-hole.py and +recv-hole that transfer holes more efficiently. They use the +`SEEK_DATA` and `SEEK_HOLE` options to the `lseek` system call. +xfer-hole.py produces an encoded form of the input file, where holes +are encoded in an efficient way. recv-hole decodes that and reproduces +the original file. The nifty trick is that recv-hole is a pure shell +script, and so will hopefully work on most target systems, instead of +requiring a custom program there. + +This is a proof of concept. It is not ready for production use. +I wrote this to prove it's possible. -- cgit v1.2.1