p14nd4's blog is the best blog ever.

Hahaha. So I’ll try to keep this brief, but I just wanted to post the problem I recently found with VMware Server (1.0.4, 1.0.5) on linux (perhaps others), and what I eventually found to be the cause and workaround.

Problem

Running VMware server on linux, I edit virtual machine settings, open the CD-ROM device, select Use ISO image, and Browse. I navigate to my smbmount (actually a cifs mount line in my /etc/fstab), and VMware hangs for about three or four minutes before displaying the contents of the directory. The following lines are written to /var/log/syslog:

Mar 24 16:54:30 nine kernel:  CIFS VFS: server not responding
Mar 24 16:54:30 nine kernel:  CIFS VFS: No response to cmd 47 mid 16
Mar 24 16:54:30 nine kernel:  CIFS VFS: Write2 ret -11, written = 0
Mar 24 16:55:30 nine kernel:  CIFS VFS: server not responding
Mar 24 16:55:30 nine kernel:  CIFS VFS: No response to cmd 47 mid 28
Mar 24 16:55:30 nine kernel:  CIFS VFS: Write2 ret -11, written = 0

Tracking down the problem

I started by just googling these errors for a while, with no particular luck. The closest solution I found was someone getting errors like this, and eventually resolving them with a new NIC and/or switch … some network problem. I was hopeful this might be my problem, since I was connecting at 100/Half through an old hub (not switch) on all machines exhibiting the problem (though, they did have various types of NICs, so I didn’t think it was a driver bug). Unfortunately, even after I was able to get a switch instead of a hub, the problem persisted. The #samba IRC channel was no help, and I wasn’t ready to bother the developers.

I put the problem aside for a few days, and gave it another shot on a clean state of mind. This time, I used lsof to figure out which pid had an open handle on my smbmount. I then attached strace to the process and tried to open a directory, and was greeted with this little gem (comments added by me):

/* create .vmBigFileTest0 for LARGEFILE access */
open("/blah/blah/.vmBigFileTest0", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600) = 36
 
/* override the regular signal handler for 'exceeds file size' with an
 * 'ignore signal' handler
 */
rt_sigaction(SIGXFSZ, {SIG_IGN}, {0x80c9f00, ~[KILL STOP RTMIN RT_1], SA_RESTART}, 8) = 0
 
/* set the read/write offset to 2 GiB */
_llseek(36, 2147483648, [2147483648], SEEK_SET) = 0
 
/* write a byte */
write(36, "\\245", 1)                    = 1
 
/* close ... which also means flush buffers to 'disk.'
 * this call hangs for a long time
 */
close(36 <unfinished ...></unfinished>
 

Download this code: vmware-strace.c

To summarize: VMware apparently thinks it’s necessary to check if the underlying filesystem of its ISO mount location has a 2 GiB (or smaller, I suppose) filesize limitation. While this is a very legitimate concern when browsing for the path to create virtual disk files, it strikes me as pretty irrelevant for mounting an ISO, which should be a read-only operation. To make matters worse, it doesn’t do this by checking the filesystem type in /proc/mounts or something quick and easy like that … it does this by effectively writing 2 GiB of data. (In fairness, their way is more reliable and future-proof, but come on…)

Solution

Once I saw what was going on, the solution to this problem was pretty trivial. I remounted my smbmount to be read-only, rather than read-write, which prevents VMware from getting the chance to try to write 2 GiB of data. In case you really don’t want to do any more research yourself, this means either passing the -o ro option to the mount command, or adding ro to the comma-separated options list of the fstab entry (right after the fs type entry … i.e. cifs).

This entry was posted on Monday, March 24th, 2008 at 18:29 and is filed under Technical. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.