VMware 4.5

Question

Hello fellow archers.  I'm having a difficult time getting vmware running my system.  I've tried the steps in the wiki, as well as the steps in:
http://bbs.archlinux.org/viewtopic.php? … ght=vmware
I've also tried the wiki steps with VMware 5.  Here is my problem:
The installation goes great until I run vmware-config.pl.  When the script comes to the part where it compiles the modules it hangs up on vmnet with the following error:
Building the vmnet module.
Building for VMware Workstation 4.5.2 or VMware GSX Server 3.1.0.
Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-config1/vmnet-only'
make -C /lib/modules/2.6.13-rc1-mm1/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. modules
make[1]: Entering directory `/usr/src/linux-2.6.13-rc1-mm1'
CC [M] /tmp/vmware-config1/vmnet-only/driver.o
/tmp/vmware-config1/vmnet-only/driver.c: In function 'VNetProcessOwnsPort':
/tmp/vmware-config1/vmnet-only/driver.c:1816: error: 'struct files_struct' has no member named 'max_fds'
make[2]: *** [/tmp/vmware-config1/vmnet-only/driver.o] Error 1
make[1]: *** [_module_/tmp/vmware-config1/vmnet-only] Error 2
make[1]: Leaving directory `/usr/src/linux-2.6.13-rc1-mm1'
make: *** [vmnet.ko] Error 2
make: Leaving directory `/tmp/vmware-config1/vmnet-only'
Unable to build the vmnet module.
For more information on how to troubleshoot module-related problems, please
visit our Web site at "http://www.vmware.com/download/modules/modules.html" and
"http://www.vmware.com/support/reference/linux/prebuilt_modules_linux.html".
Execution aborted.
I originally saw this problem with kernel 2.6.12-rc5-mm2.  Thinking it was simply a change in the kernel code I didn't think much of it until now.  I'm running 2.6.13-rc1-mm1.  Since I don't read of anyone else having this problem, I wonder if maybe it has to do with my kernel config.  Any help would be much appreciated.

Answer 1

I kind of forgot about this problem until it recently became necessary for me to get vmware working for work purposes.  These instructions appear to only be necessary if you are running the mm series kernel or the files-break-up-files-struct.patch patch.  The problem is that the mm series kernel no longer has a variable max_fds in files_struct.  The variable now lives in fdtable.  Here is how I got everything working:
Obtain vmware-any-any-update93.tar.gz from http://ftp.cvut.cz/vmware/
Bust that open and locate vmnet.tar.  Untar vmnet.tar and locate driver.c and apply the following patch:
--- driver.c.orig 2005-06-18 21:00:03.000000000 -0400
+++ driver.c 2005-09-09 12:02:36.000000000 -0400
@@ -1803,6 +1803,7 @@
VNetPort *port) // IN: port
struct file *file;
+ struct fdtable *fdt;
int fd;
int maj;
@@ -1813,7 +1814,8 @@
fileTraversalLock(&p->files->file_lock);
- for (fd = 0; fd < p->files->max_fds; fd++) {
+ fdt = files_fdtable(p->files);
+ for (fd = 0; fd < fdt->max_fds; fd++) {
#ifdef KERNEL_2_3_99
file = fcheck_files(p->files, fd);
#else
Retar vmnet.tar, and tar/gzip vmware-any-any-update93.tar.gz.
Now, follow the instructions at:
http://bbs.archlinux.org/viewtopic.php? … ght=vmware
but instead of using vmware-any-any-update89.tar.gz use vmware-any-any-update93.tar.gz
I haven't even looked at vmware 5 yet.  A solution might be just as simple.  Hope this helps someone.