Fix Unicode/plain string confusion from sftp listdir.

This manifested itself when using root on sftp, and the
live data containing filenames such as Mäkelä. sftp listdir
would return a unicode string, and later on, the sftp plugin
would feed that to hashlib.md5, which can only deal with
plain strings.

We fix the problem by forcing listdir to always return 
plain strings, by converting any unicode ones to plain ones
in utf-8.

Thanks to Tapani Tarvainen for reporting the problem and
finding an easily reproducible test case.

3 files changed, 13 insertions, 1 deletions

diff --git a/NEWS b/NEWS
index dff9576f..972bc61b 100644
--- a/NEWS
+++ b/NEWS
@@ -8,6 +8,11 @@ This file summarizes changes between releases of Obnam.
 Version 0.20, released UNRELEASED; a BETA release
 -------------------------------------------------
 
+BUG FIXES:
+
+* Non-ASCII filenames over SFTP root now work. (Thanks, Tapani Tarvainen,
+  for the reproducible bug report.)
+
 USER VISIBLE CHANGES:
 
 * The output of `obnam ls` now formats the columns a little prettier,
diff --git a/obnamlib/plugins/sftp_plugin.py b/obnamlib/plugins/sftp_plugin.py
index fc41db58..d7242cdc 100644
--- a/obnamlib/plugins/sftp_plugin.py
+++ b/obnamlib/plugins/sftp_plugin.py
@@ -220,7 +220,8 @@ class SftpFS(obnamlib.VirtualFileSystem):
 
     @ioerror_to_oserror
     def listdir(self, pathname):
-        return self.sftp.listdir(pathname)
+        return [x if type(x) == str else x.encode('utf-8')
+                 for x in self.sftp.listdir(pathname)]
 
     def lock(self, lockname):
         try:
diff --git a/obnamlib/vfs.py b/obnamlib/vfs.py
index 25c443c2..48ed46a5 100644
--- a/obnamlib/vfs.py
+++ b/obnamlib/vfs.py
@@ -373,6 +373,12 @@ class VfsTests(object): # pragma: no cover
         self.fs.mkdir('foo')
         self.assert_(self.fs.isdir('foo'))
 
+    def test_listdir_returns_plain_strings_only(self):
+        self.fs.write_file(u'M\u00E4kel\u00E4', 'data')
+        names = self.fs.listdir('.')
+        types = [type(x) for x in names]
+        self.assertEqual(types, [str])
+
     def test_listdir_raises_oserror_if_directory_does_not_exist(self):
         self.assertRaises(OSError, self.fs.listdir, 'foo')