Box.com rsync Problem “Solved”
Having followed this guide to mounting a box.com account in linux, I tried rsyncing a bunch of photos to my account. It kept bombing out partway through, and subsequent attempts to resume the rsync would re-transfer the same files over and over again.
rsync: rename "/mnt/box/bwca/08/09/.IMG_3084_CR2_embedded.jpg.AigcyF" -> "08/09/IMG_3084_CR2_embedded.jpg": Input/output error (5) rsync: rename "/mnt/box/bwca/08/09/.IMG_3085.CR2.SEk0pG" -> "08/09/IMG_3085.CR2": Input/output error (5) rsync: mkstemp "/mnt/box/bwca/08/09/.IMG_3085_CR2_embedded.jpg.injOBG" failed: Permission denied (13) rsync: mkstemp "/mnt/box/bwca/08/09/.IMG_3086_CR2.jpg.OzVzik" failed: Permission denied (13)
I haven’t solved the issue with rsync dying before the end of the transfer (it’s probably related to this syslog message: mount.davfs: open files exceed max cache size by 25 MiBytes), but I have at least solved
the issue with re-transferring files over and over again with each subsequent attempt. The two answers were pretty straightforward, but I figured I’d post them anyway, for posterity’s sake.
It appears that rsync’s attempt to apply/maintain the modification time to the box.com davfs2 mount doesn’t work.
david@sydney:~/Pictures$ ls -l /mnt/box/bwca/08/09/IMG_3084.CR2 -rw-r--r-- 1 david david 12564061 Aug 14 16:08 /mnt/box/bwca/08/09/IMG_3084.CR2 david@sydney:~/Pictures$ ls -l 2012/08/09/IMG_3084.CR2 -rw-rw-r-- 1 david david 12564061 Aug 9 13:00 2012/08/09/IMG_3084.CR2
Using the -c flag on rsync will skip re-transferring files based on their checksum, which is the right
way to solve the problem, I think, but comes at the expense of greatly reduced speed with each transfer, since now rsync needs to download a copy of every file from box.com in your rsync destination in order to verify its checksum. It’ll still likely be considerably faster than re-uploading everything, as most people have faster download speeds than upload speeds, but if you’re reasonably satisfied with the chances that your data integrity is fine, you can take a shortcut by using –size-only. My full rsync command is:
rsync -avhP --size-only --bwlimit=64 2012/08 /mnt/box/bwca/
thanks for this. i was going back and reading up on rsync because it had been a while when i came across your post. i appreciate you doing a bit of the “leg work” for me.
Addressing the part of mount.davfs: open files exceed max cache size by XXX MiBytes – check the cache under /var/cache/davfs2 (all unsent files hanging there) – then /etc/davfs2/davfs2.conf for the cache size (default: 50 MB).