X fails to start, black screens, 100% CPU, nvidia-drivers

[aktivb]

Asus Geforce GTX650Ti
Works fine with GTX275

nvidia-drivers 313.30
xorg-server 1.13.4
VIDEO_CARDS="nvidia"

Xorg.log:
[ 434.370]
X.Org X Server 1.13.4
Release Date: 2013-04-17
[ 434.370] X Protocol Version 11, Revision 0
[ 434.370] Build Operating System: Linux 3.8.10-gentoo-yang-xen-r10 x86_64 Gentoo
[ 434.370] Current Operating System: Linux yang 3.8.10-gentoo-yang-xen-r10 #3 SMP Thu May 2 16:48:08 CEST 2013 x86_64
[ 434.371] Kernel command line: root=/dev/md124 rootfstype=ext4 softlevel=xen xen-pciback.permissive xen-pciback.hide=(00:1a.0)(05:00.0)(05:00.1)
[ 434.371] Build Date: 29 April 2013 09:15:14PM
[ 434.371]
[ 434.371] Current version of pixman: 0.28.0
[ 434.371] Before reporting problems, check http://wiki.x.org
to make sure that you have the latest version.
[ 434.371] Markers: (--) probed, (**) from config file, (==) default setting,
(++) from command line, (!!) notice, (II) informational,
(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
[ 434.372] (==) Log file: "/var/log/Xorg.0.log", Time: Fri May 3 17:48:42 2013
[ 434.372] (==) Using config file: "/etc/X11/xorg.conf"
[ 434.372] (==) Using system config directory "/usr/share/X11/xorg.conf.d"
[ 434.372] (==) No Layout section. Using the first Screen section.
[ 434.372] (==) No screen section available. Using defaults.
[ 434.372] (**) |-->Screen "Default Screen Section" (0)
[ 434.372] (**) | |-->Monitor "<default monitor>"
[ 434.373] (==) No device specified for screen "Default Screen Section".
Using the first device section listed.
[ 434.373] (**) | |-->Device "Device0"
[ 434.373] (==) No monitor specified for screen "Default Screen Section".
Using a default monitor configuration.
[ 434.373] (==) Automatically adding devices
[ 434.373] (==) Automatically enabling devices
[ 434.373] (==) Automatically adding GPU devices
[ 434.373] (WW) The directory "/usr/share/fonts/TTF/" does not exist.
[ 434.373] Entry deleted from font path.
[ 434.373] (WW) The directory "/usr/share/fonts/OTF/" does not exist.
[ 434.373] Entry deleted from font path.
[ 434.373] (WW) The directory "/usr/share/fonts/Type1/" does not exist.
[ 434.373] Entry deleted from font path.
[ 434.373] (WW) `fonts.dir' not found (or not valid) in "/usr/share/fonts/100dpi/".
[ 434.373] Entry deleted from font path.
[ 434.373] (Run 'mkfontdir' on "/usr/share/fonts/100dpi/").
[ 434.373] (WW) `fonts.dir' not found (or not valid) in "/usr/share/fonts/75dpi/".
[ 434.373] Entry deleted from font path.
[ 434.373] (Run 'mkfontdir' on "/usr/share/fonts/75dpi/").
[ 434.373] (==) FontPath set to:
/usr/share/fonts/misc/
[ 434.373] (==) ModulePath set to "/usr/lib64/xorg/modules"
[ 434.373] (II) The server relies on udev to provide the list of input devices.
If no devices become available, reconfigure udev or disable AutoAddDevices.
[ 434.373] (II) Loader magic: 0x806be0
[ 434.373] (II) Module ABI versions:
[ 434.373] X.Org ANSI C Emulation: 0.4
[ 434.373] X.Org Video Driver: 13.1
[ 434.374] X.Org XInput driver : 18.0
[ 434.374] X.Org Server Extension : 7.0
[ 434.382] (--) PCI: (0:5:0:0) 1002:6798:1043:0448 rev 0, Mem @ 0xc0000000/268435456, 0xfb700000/262144, I/O @ 0x0000d000/256, BIOS @ 0x????????/131072
[ 434.382] (--) PCI:*(0:6:0:0) 10de:11c6:1043:8446 rev 161, Mem @ 0xfa000000/16777216, 0xd0000000/134217728, 0xd8000000/33554432, I/O @ 0x0000c000/128, BIOS @ 0x????????/524288
[ 434.383] (II) Open ACPI successful (/var/run/acpid.socket)
[ 434.383] Initializing built-in extension Generic Event Extension
[ 434.383] Initializing built-in extension SHAPE
[ 434.383] Initializing built-in extension MIT-SHM
[ 434.383] Initializing built-in extension XInputExtension
[ 434.383] Initializing built-in extension XTEST
[ 434.383] Initializing built-in extension BIG-REQUESTS
[ 434.383] Initializing built-in extension SYNC
[ 434.383] Initializing built-in extension XKEYBOARD
[ 434.383] Initializing built-in extension XC-MISC
[ 434.383] Initializing built-in extension XINERAMA
[ 434.383] Initializing built-in extension XFIXES
[ 434.383] Initializing built-in extension RENDER
[ 434.383] Initializing built-in extension RANDR
[ 434.383] Initializing built-in extension COMPOSITE
[ 434.383] Initializing built-in extension DAMAGE
[ 434.383] Initializing built-in extension MIT-SCREEN-SAVER
[ 434.383] Initializing built-in extension DOUBLE-BUFFER
[ 434.383] Initializing built-in extension RECORD
[ 434.383] Initializing built-in extension DPMS
[ 434.383] Initializing built-in extension X-Resource
[ 434.383] Initializing built-in extension XVideo
[ 434.383] Initializing built-in extension XVideo-MotionCompensation
[ 434.383] Initializing built-in extension XFree86-VidModeExtension
[ 434.383] Initializing built-in extension XFree86-DGA
[ 434.383] Initializing built-in extension XFree86-DRI
[ 434.383] Initializing built-in extension DRI2
[ 434.383] (II) "glx" will be loaded by default.
[ 434.383] (II) LoadModule: "dri2"
[ 434.383] (II) Module "dri2" already built-in
[ 434.383] (II) LoadModule: "glamoregl"
[ 434.383] (II) Loading /usr/lib64/xorg/modules/libglamoregl.so
[ 434.400] (EE) Failed to load /usr/lib64/xorg/modules/libglamoregl.so: /usr/lib64/xorg/modules/libglamoregl.so: undefined symbol: _glapi_tls_Context
[ 434.400] (II) UnloadModule: "glamoregl"
[ 434.400] (II) Unloading glamoregl
[ 434.400] (EE) Failed to load module "glamoregl" (loader failed, 7)
[ 434.400] (II) LoadModule: "glx"
[ 434.400] (II) Loading /usr/lib64/xorg/modules/extensions/libglx.so
[ 434.416] (II) Module glx: vendor="NVIDIA Corporation"
[ 434.416] compiled for 4.0.2, module version = 1.0.0
[ 434.416] Module class: X.Org Server Extension
[ 434.416] (II) NVIDIA GLX Module 313.30 Wed Mar 27 15:51:21 PDT 2013
[ 434.416] Loading extension GLX
[ 434.416] (II) LoadModule: "nvidia"
[ 434.416] (II) Loading /usr/lib64/xorg/modules/drivers/nvidia_drv.so
[ 434.427] (II) Module nvidia: vendor="NVIDIA Corporation"
[ 434.427] compiled for 4.0.2, module version = 1.0.0
[ 434.427] Module class: X.Org Video Driver
[ 434.428] (II) NVIDIA dlloader X Driver 313.30 Wed Mar 27 15:33:21 PDT 2013
[ 434.428] (II) NVIDIA Unified Driver for all Supported NVIDIA GPUs
[ 434.429] (--) using VT number 7

[ 434.432] (II) Loading sub module "wfb"
[ 434.432] (II) LoadModule: "wfb"
[ 434.432] (II) Loading /usr/lib64/xorg/modules/libwfb.so
[ 434.433] (II) Module wfb: vendor="X.Org Foundation"
[ 434.433] compiled for 1.13.4, module version = 1.0.0
[ 434.433] ABI class: X.Org ANSI C Emulation, version 0.4
[ 434.433] (II) Loading sub module "ramdac"
[ 434.433] (II) LoadModule: "ramdac"
[ 434.433] (II) Module "ramdac" already built-in
[ 434.434] (II) NVIDIA(0): Creating default Display subsection in Screen section
"Default Screen Section" for depth/fbbpp 24/32
[ 434.434] (==) NVIDIA(0): Depth 24, (==) framebuffer bpp 32
[ 434.434] (==) NVIDIA(0): RGB weight 888
[ 434.434] (==) NVIDIA(0): Default visual is TrueColor
[ 434.434] (==) NVIDIA(0): Using gamma correction (1.0, 1.0, 1.0)
[ 434.434] (**) NVIDIA(0): Enabling 2D acceleration
[ 434.763] (II) NVIDIA(GPU-0): Display (Samsung SMB2440L (DFP-0)) does not support NVIDIA 3D
[ 434.763] (II) NVIDIA(GPU-0): Vision stereo.
[ 434.768] (II) NVIDIA(GPU-0): Display (Samsung SyncMaster (DFP-2)) does not support NVIDIA
[ 434.768] (II) NVIDIA(GPU-0): 3D Vision stereo.
[ 434.770] (II) NVIDIA(0): NVIDIA GPU GeForce GTX 650 Ti (GK106) at PCI:6:0:0 (GPU-0)
[ 434.770] (--) NVIDIA(0): Memory: 2097152 kBytes
[ 434.770] (--) NVIDIA(0): VideoBIOS: 80.06.21.00.b7
[ 434.770] (II) NVIDIA(0): Detected PCI Express Link width: 16X
[ 434.771] (--) NVIDIA(0): Valid display device(s) on GeForce GTX 650 Ti at PCI:6:0:0
[ 434.771] (--) NVIDIA(0): CRT-0
[ 434.771] (--) NVIDIA(0): Samsung SMB2440L (DFP-0) (connected)
[ 434.771] (--) NVIDIA(0): DFP-1
[ 434.771] (--) NVIDIA(0): Samsung SyncMaster (DFP-2) (connected)
[ 434.771] (--) NVIDIA(0): CRT-0: 400.0 MHz maximum pixel clock
[ 434.771] (--) NVIDIA(0): Samsung SMB2440L (DFP-0): 330.0 MHz maximum pixel clock
[ 434.771] (--) NVIDIA(0): Samsung SMB2440L (DFP-0): Internal Dual Link TMDS
[ 434.771] (--) NVIDIA(0): DFP-1: 165.0 MHz maximum pixel clock
[ 434.771] (--) NVIDIA(0): DFP-1: Internal Single Link TMDS
[ 434.771] (--) NVIDIA(0): Samsung SyncMaster (DFP-2): 330.0 MHz maximum pixel clock
[ 434.771] (--) NVIDIA(0): Samsung SyncMaster (DFP-2): Internal Dual Link TMDS
[ 434.771] (**) NVIDIA(0): Using HorizSync/VertRefresh ranges from the EDID for display
[ 434.771] (**) NVIDIA(0): device Samsung SMB2440L (DFP-0) (Using EDID frequencies
[ 434.771] (**) NVIDIA(0): has been enabled on all display devices.)
[ 434.772] (WW) NVIDIA(GPU-0): The EDID for Samsung SMB2440L (DFP-0) contradicts itself: mode
[ 434.772] (WW) NVIDIA(GPU-0): "1920x1080" is specified in the EDID; however, the EDID's
[ 434.772] (WW) NVIDIA(GPU-0): valid VertRefresh range (56.000-75.000 Hz) would exclude
[ 434.772] (WW) NVIDIA(GPU-0): this mode's VertRefresh (50.0 Hz); ignoring VertRefresh
[ 434.772] (WW) NVIDIA(GPU-0): check for mode "1920x1080".
[ 434.772] (WW) NVIDIA(GPU-0): The EDID for Samsung SMB2440L (DFP-0) contradicts itself: mode
[ 434.772] (WW) NVIDIA(GPU-0): "1280x720" is specified in the EDID; however, the EDID's
[ 434.772] (WW) NVIDIA(GPU-0): valid VertRefresh range (56.000-75.000 Hz) would exclude
[ 434.772] (WW) NVIDIA(GPU-0): this mode's VertRefresh (50.0 Hz); ignoring VertRefresh
[ 434.772] (WW) NVIDIA(GPU-0): check for mode "1280x720".
[ 434.772] (WW) NVIDIA(GPU-0): The EDID for Samsung SMB2440L (DFP-0) contradicts itself: mode
[ 434.772] (WW) NVIDIA(GPU-0): "720x576" is specified in the EDID; however, the EDID's
[ 434.772] (WW) NVIDIA(GPU-0): valid VertRefresh range (56.000-75.000 Hz) would exclude
[ 434.772] (WW) NVIDIA(GPU-0): this mode's VertRefresh (50.0 Hz); ignoring VertRefresh
[ 434.772] (WW) NVIDIA(GPU-0): check for mode "720x576".
[ 434.773] (**) NVIDIA(0): Using HorizSync/VertRefresh ranges from the EDID for display
[ 434.773] (**) NVIDIA(0): device Samsung SyncMaster (DFP-2) (Using EDID frequencies
[ 434.773] (**) NVIDIA(0): has been enabled on all display devices.)
[ 434.774] (==) NVIDIA(0):
[ 434.774] (==) NVIDIA(0): No modes were requested; the default mode "nvidia-auto-select"
[ 434.774] (==) NVIDIA(0): will be used as the requested mode.
[ 434.774] (==) NVIDIA(0):
[ 434.774] (II) NVIDIA(0): Validated MetaModes:
[ 434.774] (II) NVIDIA(0): "DFP-0:nvidia-auto-select,DFP-2:nvidia-auto-select"
[ 434.774] (II) NVIDIA(0): Virtual screen size determined to be 3600 x 1080
[ 434.808] (--) NVIDIA(0): DPI set to (93, 94); computed from "UseEdidDpi" X config
[ 434.808] (--) NVIDIA(0): option
[ 434.808] (--) Depth 24 pixmap format is 32 bpp
[ 434.808] (II) NVIDIA: Using 3072.00 MB of virtual memory for indirect memory
[ 434.808] (II) NVIDIA: access.
[ 434.812] (II) NVIDIA(0): Setting mode "DFP-0:nvidia-auto-select,DFP-2:nvidia-auto-select"

[Maleita]

is a new instalation ?

[chithanh]

[aktivb]

Sorry about the late follow-up, but exams are coming up and this takes a long time to try to work out.

It is not a new installation.

I have not tested with a non-xen kernel, but the kernel and driver play nice and everything works if I boot 'bare-metal', without the xen mini kernel.
Except that does not get me what I want, because I want/need xen.

A little more background:
I have a Radeon HD7970/SI Tahiti XT2 I pass through to a Win7 HVM guest. That works nicely.
However, I have a hard time getting another card to run X in the dom0. I have 3 monitors, 2 rotated, and I have a few candidates for dom0 cards:
A Radeon HD7770/SI Cape Verde XT
A Geforce GTX650Ti
A Geforce GTX275 + Geforce GT610

I initially tried the HD7770 with the binary driver, gave me issues similar to those described here (black screens, 100% cpu)
Tried the open-source radeon driver, which for acceleration (needed for monitor rotation) required glamor + mesa 9.1 and a patched llvm. Gave me 5 FPS in glxgears which was unworkable.
Tried the GTX275+GT610. Worked with the binary driver, so tried the GTX650, resulted in what I initially described. I now get the same issue with the GTX275+GT610 combo.

I currently have the HD7770 + GT610 in my box, so I can try to get one driver to work without having to switch cards.
Both binary drivers (nvidia and fglrx) seem to be having trouble with the xen mini kernel, and the radeon driver's performance was abysmal, so I'd say there's trouble there too.
There is some documentation/posts on the nvidia binary driver not playing nice with xen, but most of it is old (2.6~ kernels, 190~ driver)

I have not tried primary passthrough, leaving just the 7970. That is an option, I don't really _need_ X running while I'm in windows, but I would prefer two cards and it's nicer not having to restart everything in X if I need to pop into windows.

I would like some guidance on what areas to experiment and try next. To my limited knowledge, I'm not seeing any real errors in the X.log, and black screens and freeze-ups is not giving me much to work with.

[chithanh]

Support for the radeon 7770/7970 in mesa-9.1 is still very preliminary. You may achieve better results with live ebuilds of mesa and llvm.

Another thing you can try is a different virtualization method instead of Xen.

[aktivb]

I just installed the fglrx driver and booted the kernel bare-metal, and tried X. Same issue.

dmesg:
[ 41.953998] fglrx_pci 0000:06:00.0: irq 61 for MSI/MSI-X
[ 41.954785] <6>[fglrx] Firegl kernel thread PID: 3161
[ 41.954915] <6>[fglrx] Firegl kernel thread PID: 3162
[ 41.955071] <6>[fglrx] Firegl kernel thread PID: 3163
[ 41.955211] <6>[fglrx] IRQ 61 Enabled
[ 41.968845] <6>[fglrx] Gart USWC size:1280 M.
[ 41.968850] <6>[fglrx] Gart cacheable size:508 M.
[ 41.968856] <6>[fglrx] Reserved FB block: Shared offset:0, size:1000000
[ 41.968859] <6>[fglrx] Reserved FB block: Unshared offset:f8fc000, size:404000
[ 41.968862] <6>[fglrx] Reserved FB block: Unshared offset:3ffee000, size:12000
[ 41.997538] dmar: DRHD: handling fault status reg 2
[ 41.997549] dmar: DMAR:[DMA Read] Request device [06:00.0] fault addr 83755d000
DMAR:[fault reason 02] Present bit in context entry is clear
[ 221.998008] <6>[fglrx] ASIC hang happened
[ 221.998017] Pid: 3157, comm: X Tainted: P O 3.8.10-gentoo-yang-xen-r10 #4
[ 221.998019] Call Trace:
[ 221.998076] [<ffffffffa00076a9>] KCL_DEBUG_OsDump+0x9/0x10 [fglrx]
[ 221.998122] [<ffffffffa0015dfc>] firegl_hardwareHangRecovery+0x1c/0x60 [fglrx]
[ 221.998206] [<ffffffffa00a2d19>] ? _ZN4Asic9WaitUntil15ResetASICIfHungEv+0x9/0x10 [fglrx]
[ 221.998288] [<ffffffffa00a2ca3>] ? _ZN4Asic9WaitUntil15WaitForCompleteEv+0x93/0x100 [fglrx]
[ 221.998378] [<ffffffffa00b32d7>] ? _ZN15ExecutableUnits10CPRingIdleE15idle_WaitMethod12_QS_CP_RING_+0x127/0x1f0 [fglrx]
[ 221.998467] [<ffffffffa00b0001>] ? _ZN3CMM12readRegistryEbb+0x841/0xb00 [fglrx]
[ 221.998555] [<ffffffffa00b3c9a>] ? _ZN21ExecutableUnitsCayman14AllCPRingsIdleE15idle_WaitMethod+0x1a/0x90 [fglrx]
[ 221.998642] [<ffffffffa00b315b>] ? _ZN15ExecutableUnits7PM4idleE15idle_WaitMethod+0x4b/0x90 [fglrx]
[ 221.998728] [<ffffffffa00b41fd>] ? _ZN17ExecutableUnitsSI9assertPM4Eb+0x6d/0x80 [fglrx]
[ 221.998827] [<ffffffffa00d1269>] ? _ZN8AsicR6009assertPM4Eb+0x39/0x80 [fglrx]
[ 221.998899] [<ffffffffa00843ce>] ? CMMQS_Initialize_WA+0x20e/0x240 [fglrx]
[ 221.998952] [<ffffffffa003c8c2>] ? firegl_cmmqs_init+0x682/0xc30 [fglrx]
[ 221.998998] [<ffffffffa0018c94>] ? firegl_init_iommu+0x94/0x170 [fglrx]
[ 221.999064] [<ffffffffa0034545>] ? hal_init_gpu+0x215/0x4b0 [fglrx]
[ 221.999108] [<ffffffffa0010827>] ? firegl_open+0x2e7/0x320 [fglrx]
[ 221.999153] [<ffffffffa0001a42>] ? ip_firegl_open+0x12/0x10e0 [fglrx]
[ 221.999198] [<ffffffffa0000f19>] ? ip_firegl_lseek+0xf19/0x15f0 [fglrx]
[ 221.999210] [<ffffffff8115c9d8>] ? chrdev_open+0x98/0x180
[ 221.999215] [<ffffffff8115c940>] ? cdev_put+0x30/0x30
[ 221.999221] [<ffffffff8115680e>] ? do_dentry_open+0x20e/0x290
[ 221.999226] [<ffffffff811569f0>] ? finish_open+0x30/0x40
[ 221.999232] [<ffffffff81165f2e>] ? do_last.isra.52+0x26e/0xcc0
[ 221.999237] [<ffffffff81163163>] ? inode_permission+0x13/0x50
[ 221.999242] [<ffffffff8116320a>] ? link_path_walk+0x6a/0x8e0
[ 221.999247] [<ffffffff81166a2b>] ? path_openat.isra.53+0xab/0x490
[ 221.999253] [<ffffffff811671bd>] ? do_filp_open+0x3d/0xa0
[ 221.999259] [<ffffffff811737a5>] ? __alloc_fd+0x45/0x120
[ 221.999265] [<ffffffff81157cc9>] ? do_sys_open+0xf9/0x1e0
[ 221.999271] [<ffffffff81014926>] ? do_notify_resume+0x66/0x70
[ 221.999276] [<ffffffff81157dcc>] ? sys_open+0x1c/0x20
[ 221.999283] [<ffffffff818080e9>] ? system_call_fastpath+0x16/0x1b
[ 221.999289] pubdev:0xffffffffa0450460, num of device:2 , name:fglrx, major 9, minor 1.
[ 221.999292] device 0 : 0xffff880835f00000 .
[ 221.999296] Asic ID:0x683d, revision:0x29, MMIOReg:0xffffc90014e00000.
[ 221.999299] FB phys addr: 0xb0000000, MC :0xf400000000, Total FB size :0x40000000.
[ 221.999303] gart table MC:0xf40f8fc000, Physical:0xbf8fc000, size:0x403000.
[ 221.999306] mc_node :FB, total 1 zones
[ 221.999309] MC start:0xf400000000, Physical:0xb0000000, size:0xfd00000.
[ 221.999313] Mapped heap -- Offset:0x0, size:0xf8fc000, reference count:1, mapping count:0,
[ 221.999316] Mapped heap -- Offset:0x0, size:0x1000000, reference count:1, mapping count:0,
[ 221.999319] Mapped heap -- Offset:0xf8fc000, size:0x404000, reference count:1, mapping count:0,
[ 221.999322] mc_node :INV_FB, total 1 zones
[ 221.999325] MC start:0xf40fd00000, Physical:0xbfd00000, size:0x30300000.
[ 221.999329] Mapped heap -- Offset:0x302ee000, size:0x12000, reference count:1, mapping count:0,
[ 221.999332] mc_node :GART_USWC, total 3 zones
[ 221.999334] MC start:0xffa0100000, Physical:0x0, size:0x50000000.
[ 221.999337] Mapped heap -- Offset:0x0, size:0x2000000, reference count:1, mapping count:0,
[ 221.999340] mc_node :GART_CACHEABLE, total 3 zones
[ 221.999343] MC start:0xff70400000, Physical:0x0, size:0x2fd00000.
[ 221.999346] Mapped heap -- Offset:0x0, size:0x200000, reference count:1, mapping count:0,
[ 221.999350] Mapped heap -- Offset:0xef000, size:0x11000, reference count:1, mapping count:0,
[ 221.999353] mc_node :PEER_FB_GART, total 1 zones
[ 221.999356] MC start:0xfff0100000, Physical:0x0, size:0x1000.
[ 221.999360] GRBM : 0xa0003028, SRBM : 0x200000c0 .
[ 221.999365] CP_RB_BASE : 0xffa01000, CP_RB_RPTR : 0x40 , CP_RB_WPTR :0x40.
[ 221.999369] CP_IB1_BUFSZ:0x0, CP_IB1_BASE_HI:0x0, CP_IB1_BASE_LO:0x0.
[ 221.999372] last submit IB buffer -- MC :0x0. Can't found mapped physical page for this MC .
[ 221.999376] device 1 : not been initialized ,continue.
[ 221.999378] Dump the trace queue.
[ 221.999380] End of dump