rsync Problem “Solved”

Having followed this guide to mounting a account in linux, I tried rsyncing a bunch of photos to my account. It kept bombing out partway through, and subsequent attempts to resume the rsync would re-transfer the same files over and over again.

rsync: rename "/mnt/box/bwca/08/09/.IMG_3084_CR2_embedded.jpg.AigcyF" -> "08/09/IMG_3084_CR2_embedded.jpg": Input/output error (5)
rsync: rename "/mnt/box/bwca/08/09/.IMG_3085.CR2.SEk0pG" -> "08/09/IMG_3085.CR2": Input/output error (5)
rsync: mkstemp "/mnt/box/bwca/08/09/.IMG_3085_CR2_embedded.jpg.injOBG" failed: Permission denied (13)
rsync: mkstemp "/mnt/box/bwca/08/09/.IMG_3086_CR2.jpg.OzVzik" failed: Permission denied (13)

I haven’t solved the issue with rsync dying before the end of the transfer (it’s probably related to this syslog message: mount.davfs: open files exceed max cache size by 25 MiBytes), but I have at least solved the issue with re-transferring files over and over again with each subsequent attempt. The two answers were pretty straightforward, but I figured I’d post them anyway, for posterity’s sake.

It appears that rsync’s attempt to apply/maintain the modification time to the davfs2 mount doesn’t work.

david@sydney:~/Pictures$ ls -l /mnt/box/bwca/08/09/IMG_3084.CR2
-rw-r--r-- 1 david david 12564061 Aug 14 16:08 /mnt/box/bwca/08/09/IMG_3084.CR2
david@sydney:~/Pictures$ ls -l 2012/08/09/IMG_3084.CR2
-rw-rw-r-- 1 david david 12564061 Aug  9 13:00 2012/08/09/IMG_3084.CR2

Using the -c flag on rsync will skip re-transferring files based on their checksum, which is the right way to solve the problem, I think, but comes at the expense of greatly reduced speed with each transfer, since now rsync needs to download a copy of every file from in your rsync destination in order to verify its checksum. It’ll still likely be considerably faster than re-uploading everything, as most people have faster download speeds than upload speeds, but if you’re reasonably satisfied with the chances that your data integrity is fine, you can take a shortcut by using –size-only. My full rsync command is:

rsync -avhP --size-only --bwlimit=64 2012/08 /mnt/box/bwca/

2 Responses to “ rsync Problem “Solved””

  1. Tim Says:

    thanks for this. i was going back and reading up on rsync because it had been a while when i came across your post. i appreciate you doing a bit of the “leg work” for me.

  2. David Ramirez Says:

    Addressing the part of mount.davfs: open files exceed max cache size by XXX MiBytes – check the cache under /var/cache/davfs2 (all unsent files hanging there) – then /etc/davfs2/davfs2.conf for the cache size (default: 50 MB).

Leave a Reply