diff options
Diffstat (limited to 'bugs/Performance_impact_of_backup_generations__63__.mdwn')
| -rw-r--r-- | bugs/Performance_impact_of_backup_generations__63__.mdwn | 55 |
1 files changed, 0 insertions, 55 deletions
diff --git a/bugs/Performance_impact_of_backup_generations__63__.mdwn b/bugs/Performance_impact_of_backup_generations__63__.mdwn deleted file mode 100644 index dda828c..0000000 --- a/bugs/Performance_impact_of_backup_generations__63__.mdwn +++ /dev/null @@ -1,55 +0,0 @@ -[[!tag obnam-1.0-blocker]] - -I'm not sure what to do with this or if it even qualifies as a bug but here we go... - -I back up my notebook using obnam for a while now so I have about 10 clean generations, the repo is 118G big. One of the files in the backup is a 32G VM file that changes slightly between backups, but not during backups. Until a few days ago this worked really well and fast. - -Then I tried to do a partial backup via a very slow line, with snapshots every 5MB. This was interrupted (ctrl+c) after a few hours. - -The next day I continued the backup using the local network. I forgot to change the snapshot interval, so it was still 5MB. I expected the backup to finish within a few minutes as it did in the past, but after crawling through the first couple of GB of the VM file at 50MB/s the speed went down to below 1MB/s, doing a huge amount of IO on the local machine. I haven't manged to complete a backup since. - -Nearly everything I try to do with the repo now takes a lot of time. Listing the huge amount of generations takes over an hour (it lists at a rate of about 1 generation every 10 seconds or so). Listing only the genids is still very quick. - -I'm not sure what goes wrong here. Is the massive amount of generations killing the performance? - --- weinzwang - ---- - -The number of generations shouldn't impact the speed of backups. It does have an impact on -the speed of removing generations, so if you backup stalls at the end when it removes checkpoints, -then you can avoid that with the `--leave-checkpoints` option. - -Could you check logs to see where time is spent? Or even run with profiling enabled when you -list generations? `OBNAM_PROFILE=obnam.prof obnam generations` should do that. Then use -a Python script like the following to get it in cleartext: - - import pstats - import sys - - if len(sys.argv) not in [2, 3]: - sys.stderr.write('Usage: viewprof foo.prof [sort-order]\n') - sys.exit(1) - - if len(sys.argv) == 3: - order = sys.argv[2] - else: - order = 'cumulative' - - p = pstats.Stats(sys.argv[1]) - p.strip_dirs() - p.sort_stats(order) - p.print_stats() - p.print_callees() - -Then mail me the output, and I'll have a look. Thanks. --liw - ---- - -Since this problematic repository doesn't work with recent versions of obnam due to -on-disk format changes, I can't reproduce this anymore. I haven't encountered this effect -with any other repositories I have, so I think the bug can be closed. --weinzwang - --- - -Ack, closing bug. [[done]] --liw |