vmware-server fails to run with hardened-sources-2.6.24

[nlsa8z6zoz7lyih3ap]

Vmware-server virtual machines fail to run with hardened-sources-2.6.24
on the host,
EVEN WHEN GRSECURITY AND PAX are disabled before compiling the kernel!!!!!
Dmesg gives the following when I try to power on a virtual machine:

(This strikes me as strange as I am getting a pax error message even when pax is disabled in the kernel.)
(With hardened-sources-2.6.25, my computer appears to crash shortly after I try to start a virtual machine.)
(vmware machines work just fine with a gentoo-sources-2.6.24 kernel on the host.)
Any ideas as to what is happening. Could this be a hardened-sources bug?


PAX: vmware-vmx:6825, uid/euid: 1000/1000, invalid execution attempt at ffffc20002367010 RIP:
[<ffffc20002367010>]
PGD 8000000000685063 PUD 21fc46067 PMD 21b93f063 PTE c09063
Oops: 0011 [1] SMP
CPU 1
Modules linked in: vmnet(P) vmmon(P) iptable_raw iptable_mangle ipt_REJECT iptable_filter ip_tables x_tables vboxdrv pwc nvidia(P) i2c_core ohci1394 ieee1394 loop
Pid: 6825, comm: vmware-vmx Tainted: P 2.6.24-hardened-r3-nogrsecpax-a #3
RIP: 0010:[<ffffc20002367010>] [<ffffc20002367010>]
RSP: 0000:ffff81020c679c50 EFLAGS: 00013046
RAX: ffffc20002367010 RBX: ffffc20002367000 RCX: ffffc20002367000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 00002b3679f195b0 R08: 0000000000000001 R09: ffff8102101779e8
R10: ffff81020c678000 R11: ffffffff803cc8a0 R12: ffff81021fc68dc0
R13: 00000000f2dd8b90 R14: 0000000000000000 R15: 0000000000000063
FS: 00002b3679f195b0(0000) GS:ffff81021fc68dc0(0063) knlGS:00000000f2dd8b90
CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b
CR2: ffffc20002367010 CR3: 0000000210165000 CR4: 0000000000000660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process vmware-vmx (pid: 6825, threadinfo ffff81020c678000, task ffff81021d5be040)
Stack: ffffffff88c25f47 0000000200000000 0000000000000000 ffff81021016c000
0000000000003286 000000008005003b 00000000f2dd8de0 00000000000006e0
0000000000000000 ffff8068e000007f 000000000000ffff 0000000000000000
Call Trace:
[<ffffffff88c25f47>] :vmmon:Task_Switch_S1B1+0x2f7/0xab0
[<ffffffff88c33296>] :vmmon:Vmx86_RunVM_S1B1+0xb6/0x210
[<ffffffff88c1c63a>] :vmmon:init_module+0x11aa/0x4210
[<ffffffff88c1d5ee>] :vmmon:init_module+0x215e/0x4210
[<ffffffff88c1f2f0>] :vmmon:init_module+0x3e60/0x4210
[<ffffffff802b7c23>] compat_sys_ioctl+0x1a3/0x3f0
[<ffffffff802567da>] compat_sys_setitimer+0x14a/0x160
[<ffffffff802193d2>] ia32_sysret+0x0/0xa


Code: 48 8b 91 54 07 00 00 ff 34 24 48 89 54 24 08 48 8d 91 2c 07
RIP [<ffffc20002367010>]
RSP <ffff81020c679c50>
CR2: ffffc20002367010
---[ end trace 8f76695c37811e6f ]---

[nlsa8z6zoz7lyih3ap]

The PaX Team has explained to me that PAX_CONFIG=N does not turn a PaX patched kernel (or a Grescurity one)
into a vanilla kernel. Moreover they have given some interesting and compelling reasons as to why this is so.

This topic is covered in the grsecuity forums at http://forums.grsecurity.net/viewtopic.php?f=3&t=2001&start=0&st=0&sk=t&sd=a



WARNING: In my experience running a vmware-sever virtual machine with the 2.6.25 series PaX patched kernels
damages the virtual machine so that it can no be powered on later, even with an unpatched kernel. This just show the need of making
backups before experimenting with new software versions.

The PaX team will look into modifying PaX to enable vmware-server to run, particularly the vmmon module.
Until (and if) they succeed, persons wanting to run vmaware-server with a hardened kernel will have to stick with the 2.6.23 series or lower.

I suspect (but have not confirmed) that the same situation pertains to all other vmware virtual products.

[wyv3rn]

The issue you are running into is AMD64 specific. vmware-server works fine on x86 (2.6.24/2.6.25 too), even with most of the PaX/grsecurity options enabled. PaX Team is looking into solving the issue on AMD64. I won't speak for them, but judging by how well they support their product historically, you should have a fix soon.

[fixinko]

Have you tried to build your kernel with

[nlsa8z6zoz7lyih3ap]

Have you tried to build your kernel with
Quote:

CONFIG_PAX_KERNEXEC=n

?


Yes. there is a fuller discussion in the grsecurity forms. (See link in earlier posting.)

[dtjohnst]

Any chance this is working properly now with 2.6.27-r7?

[nlsa8z6zoz7lyih3ap]

Last that I heard (October 13, 2008) would indicate "no."
See http://forums.grsecurity.net/viewtopic.php?f=3&t=2001&start=0&st=0&sk=t&sd=a
If you choose to test this be sure to do so with a copy of your virtual machine, as the test may damage it.
Please post your test results, if you do test it.

I think that it is a shame that the grsecurity kernel>=2.6.24 kills vmware virtual machines
as it means that people who use them can not even use pax.
The PaX team is really helpful and hardworking. Perhaps if a large number of people
posted on the grsecurity forums that they cared about this issue, it would encourage
the PaX team to work on it.