{"id":225,"date":"2012-08-15T12:04:33","date_gmt":"2012-08-15T17:04:33","guid":{"rendered":"http:\/\/www.p14nd4.com\/blog\/?p=225"},"modified":"2012-08-15T13:26:48","modified_gmt":"2012-08-15T18:26:48","slug":"box-com-rsync-problem-solved","status":"publish","type":"post","link":"http:\/\/www.p14nd4.com\/blog\/2012\/08\/15\/box-com-rsync-problem-solved\/","title":{"rendered":"Box.com rsync Problem “Solved”"},"content":{"rendered":"

Having followed this guide to mounting a box.com account in linux<\/a>, I tried rsyncing a bunch of photos to my account. It kept bombing out partway through, and subsequent attempts to resume the rsync would re-transfer the same files over and over again.<\/p>\n

\n
rsync: rename "\/mnt\/box\/bwca\/08\/09\/.IMG_3084_CR2_embedded.jpg.AigcyF" -> "08\/09\/IMG_3084_CR2_embedded.jpg": Input\/output error (5)\r\nrsync: rename "\/mnt\/box\/bwca\/08\/09\/.IMG_3085.CR2.SEk0pG" -> "08\/09\/IMG_3085.CR2": Input\/output error (5)\r\nrsync: mkstemp "\/mnt\/box\/bwca\/08\/09\/.IMG_3085_CR2_embedded.jpg.injOBG" failed: Permission denied (13)\r\nrsync: mkstemp "\/mnt\/box\/bwca\/08\/09\/.IMG_3086_CR2.jpg.OzVzik" failed: Permission denied (13)<\/pre>\n<\/blockquote>\n
I haven’t solved the issue with rsync dying before the end of the transfer (it’s probably related to this syslog message: mount.davfs: open files exceed max cache size by 25 MiBytes<\/span>), but I have at least solved<\/q> the issue with re-transferring files over and over again with each subsequent attempt. The two answers were pretty straightforward, but I figured I’d post them anyway, for posterity’s sake.<\/p>\n
<\/p>\n
It appears that rsync’s attempt to apply\/maintain the modification time to the box.com davfs2 mount doesn’t work.<\/p>\n
\n
david@sydney:~\/Pictures$ ls -l \/mnt\/box\/bwca\/08\/09\/IMG_3084.CR2\r\n-rw-r--r-- 1 david david 12564061 Aug 14 16:08 \/mnt\/box\/bwca\/08\/09\/IMG_3084.CR2\r\ndavid@sydney:~\/Pictures$ ls -l 2012\/08\/09\/IMG_3084.CR2\r\n-rw-rw-r-- 1 david david 12564061 Aug  9 13:00 2012\/08\/09\/IMG_3084.CR2<\/pre>\n<\/blockquote>\n
Using the -c<\/span> flag on rsync will skip re-transferring files based on their checksum, which is the right<\/q> way to solve the problem, I think, but comes at the expense of greatly reduced speed with each transfer, since now rsync needs to download a copy of every file from box.com in your rsync destination in order to verify its checksum. It’ll still likely be considerably faster than re-uploading everything, as most people have faster download speeds than upload speeds, but if you’re reasonably satisfied with the chances that your data integrity is fine, you can take a shortcut by using –size-only<\/span>. My full rsync command is:<\/p>\n
\n
rsync -avhP --size-only --bwlimit=64 2012\/08 \/mnt\/box\/bwca\/<\/pre>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"
Having followed this guide to mounting a box.com account in linux, I tried rsyncing a bunch of photos to my account. It kept bombing out partway through, and subsequent attempts to resume the rsync would re-transfer the same files over and over again. rsync: rename "\/mnt\/box\/bwca\/08\/09\/.IMG_3084_CR2_embedded.jpg.AigcyF" -> "08\/09\/IMG_3084_CR2_embedded.jpg": Input\/output error (5) rsync: rename "\/mnt\/box\/bwca\/08\/09\/.IMG_3085.CR2.SEk0pG" -> […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-225","post","type-post","status-publish","format-standard","hentry","category-general"],"_links":{"self":[{"href":"http:\/\/www.p14nd4.com\/blog\/wp-json\/wp\/v2\/posts\/225","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.p14nd4.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.p14nd4.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.p14nd4.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.p14nd4.com\/blog\/wp-json\/wp\/v2\/comments?post=225"}],"version-history":[{"count":6,"href":"http:\/\/www.p14nd4.com\/blog\/wp-json\/wp\/v2\/posts\/225\/revisions"}],"predecessor-version":[{"id":229,"href":"http:\/\/www.p14nd4.com\/blog\/wp-json\/wp\/v2\/posts\/225\/revisions\/229"}],"wp:attachment":[{"href":"http:\/\/www.p14nd4.com\/blog\/wp-json\/wp\/v2\/media?parent=225"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.p14nd4.com\/blog\/wp-json\/wp\/v2\/categories?post=225"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.p14nd4.com\/blog\/wp-json\/wp\/v2\/tags?post=225"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}