summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorLars Wirzenius <lars.wirzenius@codethink.co.uk>2014-05-20 15:00:48 +0100
committerLars Wirzenius <lars.wirzenius@codethink.co.uk>2014-05-20 15:00:48 +0100
commit1de57d9df3992af95e5eacd8d0f149dd94c8875c (patch)
treed2c8c7c0df4dbd95d3110d1ef10e9669820cb90f
parent2704e51ad487b240ef6da939d385ae411059e503 (diff)
downloadxfer-hole-master.tar.gz
Add READMEHEADmaster
-rw-r--r--README19
1 files changed, 19 insertions, 0 deletions
diff --git a/README b/README
new file mode 100644
index 0000000..44ea171
--- /dev/null
+++ b/README
@@ -0,0 +1,19 @@
+A proof of concept for transferring sparse files
+================================================
+
+In Baserock, we often need to transfer sparse files across the
+network. At the moment we do that by ignoring that they're sparse:
+just read the holes as long sequences of zero bytes. That works, but
+is inefficient.
+
+This directory contains a couple of scripts: xfer-hole.py and
+recv-hole that transfer holes more efficiently. They use the
+`SEEK_DATA` and `SEEK_HOLE` options to the `lseek` system call.
+xfer-hole.py produces an encoded form of the input file, where holes
+are encoded in an efficient way. recv-hole decodes that and reproduces
+the original file. The nifty trick is that recv-hole is a pure shell
+script, and so will hopefully work on most target systems, instead of
+requiring a custom program there.
+
+This is a proof of concept. It is not ready for production use.
+I wrote this to prove it's possible.