diff options
-rw-r--r-- | README | 19 |
1 files changed, 19 insertions, 0 deletions
@@ -0,0 +1,19 @@ +A proof of concept for transferring sparse files +================================================ + +In Baserock, we often need to transfer sparse files across the +network. At the moment we do that by ignoring that they're sparse: +just read the holes as long sequences of zero bytes. That works, but +is inefficient. + +This directory contains a couple of scripts: xfer-hole.py and +recv-hole that transfer holes more efficiently. They use the +`SEEK_DATA` and `SEEK_HOLE` options to the `lseek` system call. +xfer-hole.py produces an encoded form of the input file, where holes +are encoded in an efficient way. recv-hole decodes that and reproduces +the original file. The nifty trick is that recv-hole is a pure shell +script, and so will hopefully work on most target systems, instead of +requiring a custom program there. + +This is a proof of concept. It is not ready for production use. +I wrote this to prove it's possible. |