VMware causes kernel oops [SOLVED]

Message

thedopefishlives · Post by **thedopefishlives** » Tue Apr 18, 2006 1:41 pm

Hey all. I am trying to get VMware running on my AMD64 box. It emerges just fine, it builds the modules just fine, but it refuses to run; it just hangs, and the following error appears in my system logs:

Code: Select all

Unable to handle kernel NULL pointer dereference at 0000000000000090 RIP: 
<ffffffff881a483f>{:snd_pcm_oss:snd_pcm_oss_ioctl+1039}
PGD 22755067 PUD 21eb1067 PMD 0 
Oops: 0000 [3] 
last sysfs file: /devices/system/cpu/cpu0/cpufreq/scaling_governor
CPU 0 
Modules linked in: vmnet vmmon snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss fuse ipv6 ac thermal processor battery bridge llc snd_atiixp snd_atiixp_modem snd_ac97_codec snd_ac97_bus snd_pcm snd_timer snd soundcore snd_page_alloc ehci_hcd bcm43xx firmware_class ieee80211softmac ieee80211 ieee80211_crypt sdhci mmc_core yenta_socket rsrc_nonstatic pcmcia_core ohci_hcd usbcore 8139too
Pid: 23814, comm: vmplayer Tainted: P      2.6.17-rc1-mm2 #10
RIP: 0010:[<ffffffff881a483f>] <ffffffff881a483f>{:snd_pcm_oss:snd_pcm_oss_ioctl+1039}
RSP: 0018:ffff81002c9ebeb8  EFLAGS: 00010246
RAX: 0000000000000000 RBX: 00000000fffffff7 RCX: ffffffff881a4430
RDX: ffffffffffffffea RSI: ffff81002c9ebed0 RDI: 0000000000000000
RBP: 00000000ffffb0f4 R08: 00000000ffffb0f4 R09: 00000000ffffb0c8
R10: ffff81002c9ea000 R11: 0000000000000246 R12: ffff81003de7bf40
R13: 0000000000000017 R14: 00000000ffffb0f4 R15: 0000000000000000
FS:  00002b0b46b1d150(0000) GS:ffffffff804fc000(0063) knlGS:00000000f705a6b0
CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
CR2: 0000000000000090 CR3: 0000000022759000 CR4: 00000000000006e0
Process vmplayer (pid: 23814, threadinfo ffff81002c9ea000, task ffff81003fdb8840)
Stack: 0000000000008800 ffff8100218640c0 00000000fffffff7 0000000000000000 
       0000000000000017 00000000ffffb050 00000000fffffff7 00000000c0045004 
       ffff8100218640c0 0000000000000017 
Call Trace: <ffffffff80291a74>{compat_sys_ioctl+196}
       <ffffffff8021bec4>{cstar_do_call+27}

Code: 48 8b 80 90 00 00 00 8b 80 08 03 00 00 85 c0 48 63 d0 0f 88 
RIP <ffffffff881a483f>{:snd_pcm_oss:snd_pcm_oss_ioctl+1039} RSP <ffff81002c9ebeb8>
CR2: 0000000000000090
 <1>Unable to handle kernel NULL pointer dereference at 0000000000000000 RIP: 
<ffffffff881a4390>{:snd_pcm_oss:snd_pcm_oss_release+48}
PGD 3f738067 PUD 3eaff067 PMD 0 
Oops: 0000 [4] 
last sysfs file: /devices/system/cpu/cpu0/cpufreq/scaling_governor
CPU 0 
Modules linked in: vmnet vmmon snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss fuse ipv6 ac thermal processor battery bridge llc snd_atiixp snd_atiixp_modem snd_ac97_codec snd_ac97_bus snd_pcm snd_timer snd soundcore snd_page_alloc ehci_hcd bcm43xx firmware_class ieee80211softmac ieee80211 ieee80211_crypt sdhci mmc_core yenta_socket rsrc_nonstatic pcmcia_core ohci_hcd usbcore 8139too
Pid: 23814, comm: vmplayer Tainted: P      2.6.17-rc1-mm2 #10
RIP: 0010:[<ffffffff881a4390>] <ffffffff881a4390>{:snd_pcm_oss:snd_pcm_oss_release+48}
RSP: 0018:ffff81002c9ebbf8  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff8100218640c0 RCX: ffff8100218640c0
RDX: 0000000000000000 RSI: ffff8100218640c0 RDI: ffff81003e7f0fc8
RBP: ffff81003de7bf40 R08: 0000000000000000 R09: 00000000ffffffff
R10: 000000000000000a R11: ffffffff80282cb0 R12: ffff81003fede1c0
R13: ffff8100218640c0 R14: 0000000000000001 R15: ffff81002c9ebe08
FS:  00002b0b46b1d150(0000) GS:ffffffff804fc000(0000) knlGS:00000000f705a6b0
CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
CR2: 0000000000000000 CR3: 000000003e59b000 CR4: 00000000000006e0
Process vmplayer (pid: 23814, threadinfo ffff81002c9ea000, task ffff81003fdb8840)
Stack: 0000000000000292 ffff8100218640c0 ffff81003e7f0fc8 ffff81003fede1c0 
       ffff81003ddca618 ffffffff8026b1d6 0000000000000296 ffff8100218640c0 
       ffff81003fc45340 0000000000000000 
Call Trace: <ffffffff8026b1d6>{__fput+102} <ffffffff80269bde>{filp_close+110}
       <ffffffff8022a76b>{put_files_struct+107} <ffffffff8022b161>{do_exit+577}
       <ffffffff80366557>{do_unblank_screen+135} <ffffffff8021ac22>{do_page_fault+1938}
       <ffffffff802252c0>{default_wake_function+0} <ffffffff80269966>{get_unused_fd+102}
       <ffffffff8020a2a1>{error_exit+0} <ffffffff881a4430>{:snd_pcm_oss:snd_pcm_oss_ioctl+0}
       <ffffffff881a483f>{:snd_pcm_oss:snd_pcm_oss_ioctl+1039}
       <ffffffff881a4836>{:snd_pcm_oss:snd_pcm_oss_ioctl+1030}
       <ffffffff80291a74>{compat_sys_ioctl+196} <ffffffff8021bec4>{cstar_do_call+27}

Code: 4c 8b 20 48 89 ef e8 55 f3 ff ff 49 8d 9c 24 78 01 00 00 48 
RIP <ffffffff881a4390>{:snd_pcm_oss:snd_pcm_oss_release+48} RSP <ffff81002c9ebbf8>
CR2: 0000000000000000
 <1>Fixing recursive fault but reboot is needed!

Heh heh heh... Any ideas, folks?

Emerge --info:

Gentoo Base System version 1.12.0_pre17
Portage 2.1_pre7-r5 (default-linux/amd64/2006.0, gcc-3.4.6, glibc-2.4-r1, 2.6.17-rc1-mm2 x86_64)
=================================================================
System uname: 2.6.17-rc1-mm2 x86_64 AMD Athlon(tm) 64 Processor 3800+
dev-lang/python: 2.3.4-r1, 2.4.2-r1
sys-apps/sandbox: 1.2.17
sys-devel/autoconf: 2.13, 2.59-r7
sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils: 2.16.1-r2
sys-devel/libtool: 1.5.22
virtual/os-headers: 2.6.11-r3
ACCEPT_KEYWORDS="amd64 ~amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=athlon64 -mtune=athlon64 -O3 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/X11/xkb /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/eselect/compiler /etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/env.d"
CXXFLAGS="-march=athlon64 -mtune=athlon64 -O3 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks metadata-transfer sfperms strict"
GENTOO_MIRRORS="http://gentoo.osuosl.org/ ftp://distro.ibiblio.org/pub/linux/dist ... ns/gentoo/"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage /usr/local/portage-xgl"
SYNC="rsync://rsync.us.gentoo.org/gentoo-portage"
USE="amd64 X acpi alsa apache2 avi berkdb bitmap-fonts cairo cdr cli crypt ctype cups dba dvd dvdr dvdread eds emboss encode esd expat fastbuild firefox foomaticdb force-cgi-redirect fortran ftp gd gif glitz gnome gtk gtk2 imlib ipv6 isdnlog jpeg lzw lzw-tiff memlimit mozilla mp3 mpeg ncurses nptl nptlonly opengl pam pcre pdflib perl png posix pppd python qt readline sdl session simplexml soap sockets spell spl ssl tcpd threads tiff tokenizer truetype truetype-fonts type1-fonts usb xml xpm xsl xv zlib elibc_glibc input_devices_keyboard input_devices_mouse input_devices_synaptics kernel_linux userland_GNU video_cards_ati"
Unset: ASFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS

eldad · Post by **eldad** » Tue Apr 18, 2006 2:25 pm

Hmm, skimming over this I see that the NULL pointer ref was to OSS... perhaps OSS is causing you some trouble.
try running vmware without sound or compile a kernel with ALSA alone...

too bad your vmplayer taints your kernel, so lkml won't help you here.

thedopefishlives · Post by **thedopefishlives** » Tue Apr 18, 2006 10:25 pm

I saw that, I tried that, and it did work. I don't like it, but I didn't need sound for what I needed to do, so it was fine. Thanks.

eldad · Post by **eldad** » Wed Apr 19, 2006 1:50 am

Did try you compiling your kernel with ALSA and enabling sound for vmware?
my guess is that since OSS is deprecated, it doesn't fare well with amd64 arch...

hanj · Post by **hanj** » Wed Apr 26, 2006 3:19 pm

Hello

I just ran into a similar problem. I'm not sure where the reference is pointing to.. but it's looking like lsmod??.. any help is greatly appreciated..

Code: Select all

vmmon: no version magic, tainting kernel.
vmmon: module license 'unspecified' taints kernel.
/dev/vmmon[17076]: Module vmmon: registered with major=10 minor=165
/dev/vmmon[17076]: Module vmmon: initialized
vmnet: no version magic, tainting kernel.
Unable to handle kernel NULL pointer dereference at virtual address 00000000
 printing eip:
c0157d9e
*pgd =    0
*pmd =    0
Oops: 0000 [#1]
Modules linked in: vmnet vmmon
CPU:    0
EIP:    0060:[<c0157d9e>]    Tainted: PF     VLI
EFLAGS: 00010246   (2.6.14-hardened-r7) 
eax: 00000000   ebx: 00000000   ecx: c039fc47   edx: 00001000
esi: f8c232e0   edi: f7492340   ebp: f8c23160   esp: f708fee8
ds: 007b   es: 007b   ss: 0068
Process lsmod (pid: 22852, threadinfo=f708e000 task=f7de7570)
Stack: f7492340 c039fc44 0000000c 00000001 f8c23164 f8c23160 f7492340 00000000 
       c015999e f7492340 f8c23160 f8c2316c 00006772 f8c23164 f7492340 f8c23164 
       c019a5b9 f7492340 f8c23164 00000003 00000000 00000000 00000000 00000000 
Call Trace:
 [<c015999e>]
 [<c019a5b9>]
 [<c017ab05>]
 [<c017ae31>]
 [<c012daf9>]
 [<c012007b>]
Code: 01 00 00 39 de 74 2e c7 44 24 0c 01 00 00 00 8b 43 08 89 3c 24 c7 44 24 04 44 fc 39 c0 83 c0 0c 89 44 24 08 e8 ef 2d 04 00 8b 1b <8b> 03 0f 18 00 90 39 de 75 da 8b 85 d0 00 00 00 85 c0 75 3c 8b

vmware cannot start, and it will lock the terminal.

Here is my emerge info...
app-emulation/vmware-server-1.0.0.22874

Code: Select all

Portage 2.0.54 (default-linux/x86/2005.1, gcc-3.4.5, glibc-2.3.6-r3, 2.6.14-hardened-r7 i686)
=================================================================
System uname: 2.6.14-hardened-r7 i686 Intel(R) Pentium(R) 4 CPU 2.80GHz
Gentoo Base System version 1.6.14
dev-lang/python:     2.3.5-r2, 2.4.2
sys-apps/sandbox:    1.2.12
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1
sys-devel/binutils:  2.16.1
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r2
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=pentium4 -O2 -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/eselect/compiler /etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O2 -mcpu=i686 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks sandbox sfperms strict"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://172.16.0.2/gentoo-portage"
USE="x86 apm arts avi berkdb bitmap-fonts bzip2 cli crypt cups dri eds emboss encode expat foomaticdb fortran gdbm gif gpm gstreamer imlib ipv6 isdnlog jpeg libg++ libwww mad mikmod motif mp3 mpeg ncurses nls ogg oggvorbis opengl oss pam pcre pdflib perl png pppd python qt quicktime readline reflection sdl session spell spl ssl tcpd truetype truetype-fonts type1-fonts udev vorbis xml2 xmms xorg xv zlib userland_GNU kernel_linux elibc_glibc"
Unset:  ASFLAGS, CTARGET, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS

thanks
hanji

hanj · Post by **hanj** » Wed Apr 26, 2006 3:39 pm

Hello

lsmod only has problems after vmware attempts to start. If I remove vmware from run level and reboot.. lsmod works properly. I wonder if it's related to the 'taint' messages...

Code: Select all

vmmon: no version magic, tainting kernel.
vmmon: module license 'unspecified' taints kernel.
vmnet: no version magic, tainting kernel.

Initially, this didn't hose things, until I built in module unload support into the kernel. I'm removing that again, and rebuilding the kernel for testing. Without module unload vmware was having problem restarting (or stopping) since it couldn't remove vmmon and vmnet.

Thanks!
hanji

hanj · Post by **hanj** » Wed Apr 26, 2006 3:49 pm

Removing 'module unload support' from the kernel.. I can 'start' vmware.. sorta.. now I get a panic on vmnet

Code: Select all

vmmon: no version magic, tainting kernel.
vmmon: module license 'unspecified' taints kernel.
/dev/vmmon[6009]: Module vmmon: registered with major=10 minor=165
/dev/vmmon[6009]: Module vmmon: initialized
vmnet: no version magic, tainting kernel.
/dev/vmnet: open called by PID 6047 (vmnet-bridge)
Unable to handle kernel NULL pointer dereference at virtual address 000000b4
 printing eip:
f8c47fd6
*pgd =    0
*pmd =    0
Oops: 0000 [#1]
Modules linked in: vmnet vmmon
CPU:    0
EIP:    0060:[<f8c47fd6>]    Tainted: PF     VLI
EFLAGS: 00010206   (2.6.14-hardened-r7) 
EIP is at VNetSetMACUnique+0x37/0xfffdb061 [vmnet]
eax: 00000006   ebx: f7163e84   ecx: 00000006   edx: 0015eb98
esi: 000000b4   edi: f7163e84   ebp: f700fd80   esp: f7163e64
ds: 007b   es: 007b   ss: 0068
Process vmnet-bridge (pid: 6047, threadinfo=f7162000 task=f710c5a0)
Stack: 00000074 f7163e84 f700fd80 000003e8 f7fa96fc f8c47d52 f700fd80 f7163e84 
       eb565000 f700ac98 f700fd80 f8c4af1e 00000000 f700fd80 f700fdce f8c499d7 
       f700fd80 f700fd88 00008000 f700fda0 f7e5d740 00000000 f8c47163 f7fa96fc 
Call Trace:
 [<f8c47d52>] VNet_MakeMACAddress+0x5e/0xfffdb30c [vmnet]
 [<f8c4af1e>] VNetProc_MakeEntry+0x29/0xfffd810b [vmnet]
 [<f8c499d7>] VNetUserIf_Create+0x143/0xfffd976c [vmnet]
 [<f8c47163>] VNetFileOpOpen+0x0/0xfffdbe9d [vmnet]
 [<f8c47163>] VNetFileOpOpen+0x0/0xfffdbe9d [vmnet]
 [<f8c471c1>] VNetFileOpOpen+0x5e/0xfffdbe9d [vmnet]
 [<f8c47163>] VNetFileOpOpen+0x0/0xfffdbe9d [vmnet]
 [<c018222b>] chrdev_open+0x97/0x13d
 [<c01789f5>] __dentry_open+0x179/0x1b3
 [<c0178ac3>] filp_open+0x94/0xa4
 [<c0178b38>] get_unused_fd+0x2c/0xe0
 [<c0178c9b>] do_sys_open+0x5b/0xff
 [<c012da79>] syscall_call+0x7/0xb
 [<c012007b>] isapnp_proc_init+0x10/0x53
Code: 5c 24 1c ff 0d 08 30 c2 f8 0f 88 4a 03 00 00 a1 44 32 c2 f8 eb 1f 39 2c 24 74 14 8b 34 24 b8 06 00 00 00 fc 83 c6 40 89 df 89 c1 <f3> a6 74 30 8b 04 24 8b 40 50 85 c0 89 04 24 75 da 8b 03 89 45 
 <7>/dev/vmnet: open called by PID 6211 (vmnet-bridge)

I wonder if I should change my kernel. Any suggestions for working kernels?? Will gentoo-sources work?

Thanks!
hanji

hanj · Post by **hanj** » Wed Apr 26, 2006 4:40 pm

Switching kernels to gentoo-sources-2.6.16-r3.. I get the following on vmware-server start...

Code: Select all

 * Re-caching dependency info (mtimes differ)...
 * Re-caching dependency info (mtimes differ)...
 * Starting xinetd ...                                                                                                                                   [ ok ]
 * Re-caching dependency info (mtimes differ)...
 * Re-caching dependency info (mtimes differ)...
 * Starting VMware services:                                                                                                                             [ ok ]
 *   Virtual machine monitor                                                                                                                             [ ok ]
 *   Virtual ethernet                                                                                                                                    [ ok ]
Apr 26 10:34:50 compname Oops: 0000 [#1]
Apr 26 10:34:50 compname CPU:    0
Apr 26 10:34:50 compname EIP is at print_unload_info+0x39/0xef
Apr 26 10:34:50 compname eax: 00000000   ebx: 00000000   ecx: c0361c86   edx: 00001000
Apr 26 10:34:50 compname esi: f8c35f20   edi: f756cd00   ebp: f8c35f20   esp: f7105ef8
Apr 26 10:34:50 compname ds: 007b   es: 007b   ss: 0068
Apr 26 10:34:50 compname Process lsmod (pid: 6621, threadinfo=f7104000 task=f70ef5b0)
Apr 26 10:34:50 compname Stack: <0>f756cd00 c0361c82 00000000 00000000 f8c35f24 f8c35f20 f756cd00 00000400 
Apr 26 10:34:50 compname c012e734 f756cd00 f8c35f20 f8c35f2c 000080c4 f8c35f24 f756cd00 f8c35f24 
Apr 26 10:34:50 compname c016d11d f756cd00 f8c35f24 00000001 00000000 00000000 00000000 00000000 
Apr 26 10:34:50 compname Call Trace:
Apr 26 10:34:50 compname [<c012e734>] m_show+0x4d/0xba
Apr 26 10:34:50 compname [<c016d11d>] seq_read+0xe6/0x2e0
Apr 26 10:34:50 compname [<c014e7e2>] vfs_read+0xaa/0x1a2
Apr 26 10:34:50 medusa [<c014ebff>] sys_read+0x51/0x80
Apr 26 10:34:50 medusa [<c01029c9>] syscall_call+0x7/0xb
Apr 26 10:34:50 compname Code: 24 c7 44 24 0c 00 00 00 00 89 2c 24 e8 2f fe ff ff 89 3c 24 89 44 24 08 c7 44 24 04 82 1c 36 c0 e8 fb 16 04 00 8b 9d 80 01 00 00 <8b> 03 0f 18 00 90 8d b5 80 01 00 00 39 de 74 2e c7 44 24 0c 01 
 *   Module vmnet is not loaded. Please verify that it is loaded before                                                                                  [ !! ]
 *   running this script.

So it's still crashing on the lsmod. Again, if I remove module unload from the kernel, vmware can be started.. but not restarted.

Thanks
hanji

Xithix · Post by **Xithix** » Fri May 26, 2006 12:43 pm

If you change the kernel, you may have to rebuild the modules. Re-emerge vmware-modules. Edit: When I changed kernels, running the config script wasn't enough; it didn't actually rebuild the modules, it only tweaked the init script.

dweigert · Post by **dweigert** » Sat May 27, 2006 5:48 pm

A better otion might be to :

Code: Select all

cd /opt/vmware/workstaton/bin
./vmware-config.pl

Saves from having to re-emerge the whole thing.