View previous topic :: View next topic |
Author |
Message |
dmpogo Advocate

Joined: 02 Sep 2004 Posts: 3511 Location: Canada
|
Posted: Sun Nov 19, 2006 9:33 pm Post subject: [WORKAROUND] modular AGP/ disabled IOMMU in 2.6.18 on amd64 |
|
|
I don't find a way to disable IOMMU in the new 2.6.18-r2 kernel on amd64 platform.
As I see, the old CONFIG_GART_IOMMU option is split into CONFIG_IOMMU and CONFIG_CALGARY_IOMMU.
menuconfig gives a handle for CONFIG_CALGARY_IOMMU but not for CONFIG_IOMMU which is always set.
Is this by design or a bug ?
Since CONFIG_IOMMU is set, kernel based AGP is forced, and, say, nvidia AGP can't be used.
My problem is however different. On my quad-opteron headless server I found (over a year ago) that without IOMMU=off and AGP=off
(there is no AGP bus at all on it) I expereienced some instabilities, so I'm a bit scared to have it enabled.
Last edited by dmpogo on Tue Nov 21, 2006 2:04 am; edited 2 times in total |
|
Back to top |
|
 |
IanPo n00b

Joined: 30 May 2006 Posts: 64 Location: Moscow, Russia
|
Posted: Mon Nov 20, 2006 8:45 am Post subject: |
|
|
my kernel is 2.6.18-r2, too.
I tried to set the option mentioned off manually in .config as well (was fighting with ati-drivers; according to the guide CONFIG_GART_IOMMU was to be set to "n").
Having "make" started i read something about "writing in .config" in console output.
I stopped compilation and peeped into .config - the option CONFIG_IOMMU was "y" again.
So i could've set option CALGARY_IOMMU but CONFIG_IOMMU.
My opinion: it's not a bug, it's a new feature.
In older kernels it was different: http://gentoo-wiki.com/HOWTO_ATI_on_amd64
Here's some explanations about IOMMU: http://www.linuxhq.com/kernel/v2.6/18-rc2/arch/x86_64/Kconfig
Last edited by IanPo on Mon Nov 20, 2006 8:56 pm; edited 1 time in total |
|
Back to top |
|
 |
nationdemon n00b


Joined: 15 Nov 2006 Posts: 30 Location: Dresden
|
Posted: Mon Nov 20, 2006 11:11 am Post subject: |
|
|
Got the same prob on my girls amd64 the only one with such a god damned ati... sorry but these cards are annoying. It took me 3 days to get it running with glx and than after a month without probs the new kernel hits ground. I've found the header files of both recommended modules (agpgart and amd64-agp)... If I step back to 2.6.17-r8 Iam able to start X with real good results on glxgears and fgl_glxgears (and ati instead of mesa in glxinfo) after reinstalling ati-drivers. But with 2.6.18-r2 nothing works
I've tried to edit the .config setting the options to m and funny it is still m but neither agpgart nor amd64-agp had been installed.
ps.: plz excuse my worse english |
|
Back to top |
|
 |
dmpogo Advocate

Joined: 02 Sep 2004 Posts: 3511 Location: Canada
|
Posted: Mon Nov 20, 2006 5:55 pm Post subject: |
|
|
IanPo wrote: | my kernel is 2.6.18-r2, too.
I tried to set the option mentioned off manually in .config as well (was fighting with ati-drivers; according to the guide CONFIG_GART_IOMMU was to be set to "n").
Having "make" started i read something about "writing in .config" in console output.
I stopped compilation and peeped into .config - the option CONFIG_GART_IOMMU was "y" again.
So i could've set option CALGARY IOMMU but CONFIG_GART_IOMMU.
My opinion: it's not a bug, it's a new feature.
In older kernels it was different: http://gentoo-wiki.com/HOWTO_ATI_on_amd64
Here's some explanations about IOMMU: http://www.linuxhq.com/kernel/v2.6/18-rc2/arch/x86_64/Kconfig |
Bloody hell Let see what can be done with EMBEDDED flag
I guess if you replace in /usr/src/linux-2.6.18-gentoo-r2/arch/x86_64/Kconfig the line
Code: |
bool "IOMMU support" if EMBEDDED
|
by
Code: |
bool "IOMMU support"
|
you will be able to disable IOMMU
Also, it seems that if you choose EMBEDDED flag, you will get an option to disable IOMMU (but what other side effects of that ?)
Last edited by dmpogo on Mon Nov 20, 2006 9:09 pm; edited 4 times in total |
|
Back to top |
|
 |
dmpogo Advocate

Joined: 02 Sep 2004 Posts: 3511 Location: Canada
|
Posted: Mon Nov 20, 2006 6:00 pm Post subject: |
|
|
IanPo wrote: | my kernel is 2.6.18-r2, too.
I tried to set the option mentioned off manually in .config as well (was fighting with ati-drivers; according to the guide CONFIG_GART_IOMMU was to be set to "n").
Having "make" started i read something about "writing in .config" in console output.
I stopped compilation and peeped into .config - the option CONFIG_GART_IOMMU was "y" again.
So i could've set option CALGARY IOMMU but CONFIG_GART_IOMMU.
My opinion: it's not a bug, it's a new feature.
In older kernels it was different: http://gentoo-wiki.com/HOWTO_ATI_on_amd64
Here's some explanations about IOMMU: http://www.linuxhq.com/kernel/v2.6/18-rc2/arch/x86_64/Kconfig |
Did you mean CONFIG_IOMMU, right ? old CONFIG_GART_IOMMU is now CONFIG_IOMMU and CONFIG_CALGARY_IOMMU |
|
Back to top |
|
 |
IanPo n00b

Joined: 30 May 2006 Posts: 64 Location: Moscow, Russia
|
Posted: Mon Nov 20, 2006 8:55 pm Post subject: |
|
|
Quote: | Did you mean CONFIG_IOMMU, right ? |
Yes. Little mistake, sorry.
I'm gonna correct my previous posts.
This is how my Kernel Configuration looks:
Code: | [ ] Support for hot-pluggable CPUs (EXPERIMENTAL)
[*] Provide RTC interrupt
[ ] IBM Calgary IOMMU support
--- Machine check support
[ ] Intel MCE features
[*] AMD MCE features |
And now a piece of my .config:
Code: | CONFIG_HPET_EMULATE_RTC=y
CONFIG_IOMMU=y <--------------------------------------- yeah, i meant this
# CONFIG_CALGARY_IOMMU is not set
CONFIG_SWIOTLB=y |
Thanks for the hint about EMBEDDED flag. I'll try it. |
|
Back to top |
|
 |
nationdemon n00b


Joined: 15 Nov 2006 Posts: 30 Location: Dresden
|
Posted: Mon Nov 20, 2006 9:08 pm Post subject: |
|
|
@ dmpogo THX, it works again... Such an easy solution  |
|
Back to top |
|
 |
dmpogo Advocate

Joined: 02 Sep 2004 Posts: 3511 Location: Canada
|
Posted: Mon Nov 20, 2006 11:55 pm Post subject: |
|
|
To summarize the workaround for those who wants/needs the ability for in-kernel AGP to be disabled or compiled as a module and/or IOMMU disabled on AMD64 under 2.6.18 kernels
In /usr/src/linux-2.6.18-gentoo-r2/arch/x86_64/Kconfig replace the line 406
Code: |
bool "IOMMU support" if EMBEDDED
|
by
Code: |
bool "IOMMU support"
|
The IOMMU option will appear in kernel configuration and if disabled, AGP will not be forced.
It is up to you to decide whether you should disable IOMMU on your system.
I have filed a bug https://bugs.gentoo.org/show_bug.cgi?id=155805 but was sent upstream.
Last edited by dmpogo on Tue Nov 21, 2006 4:07 pm; edited 1 time in total |
|
Back to top |
|
 |
IanPo n00b

Joined: 30 May 2006 Posts: 64 Location: Moscow, Russia
|
Posted: Tue Nov 21, 2006 7:37 am Post subject: |
|
|
Blimey, i was doing it wrong! Value "n" is illegal in .config, should've used "is not set" instead. Those "y"s and "m"s led me the wrong way.
No matter if you deleting "if EMBEDDED" as written above, just for the record. |
|
Back to top |
|
 |
IanPo n00b

Joined: 30 May 2006 Posts: 64 Location: Moscow, Russia
|
Posted: Tue Nov 21, 2006 6:31 pm Post subject: |
|
|
Now testing.
Before:
.config
Code: |
CONFIG_HPET_EMULATE_RTC=y
CONFIG_IOMMU is not set <----------------- set it manually in editor
# CONFIG_CALGARY_IOMMU is not set |
console:
Code: |
sniffer linux # make
HOSTCC scripts/basic/fixdep
HOSTCC scripts/basic/docproc
HOSTCC scripts/kconfig/conf.o
HOSTCC scripts/kconfig/kxgettext.o
HOSTCC scripts/kconfig/mconf.o
HOSTCC scripts/kconfig/zconf.tab.o
HOSTLD scripts/kconfig/conf
scripts/kconfig/conf -s arch/x86_64/Kconfig
#
# configuration written to .config
#
CHK include/linux/version.h
CHK include/linux/utsrelease.h
CC arch/x86_64/kernel/asm-offsets.s
make[1]: *** õÄÁÌÑÅÔÓÑ ÆÁÊÌ `arch/x86_64/kernel/asm-offsets.s' <-------------------------------- interrupting make
make[1]: *** [arch/x86_64/kernel/asm-offsets.s] ðÒÅÒÙ×ÁÎÉÅ
make: *** [prepare0] ðÒÅÒÙ×ÁÎÉÅ
|
.config after make:
Code: |
CONFIG_HPET_EMULATE_RTC=y
CONFIG_IOMMU=y <---------------------- is set to default by script as it's defined in Kconfig
# CONFIG_CALGARY_IOMMU is not set
|
Deleting "if EMBEDDED":
Code: |
# Mark as embedded because too many people got it wrong.
# The code disables itself when not needed.
config IOMMU
bool "IOMMU support"
#if EMBEDDED
default y
select SWIOTLB
select AGP
depends on PCI
help
Support for full DMA access of devices with 32bit memory access only
on systems with more than 3GB. This is usually needed for USB,
sound, many IDE/SATA chipsets and some other devices.
Provides a driver for the AMD Athlon64/Opteron/Turion/Sempron GART
based IOMMU and a software bounce buffer based IOMMU used on Intel
systems and as fallback.
The code is only active when needed (enough memory and limited
device) unless CONFIG_IOMMU_DEBUG or iommu=force is specified
too.
|
Well, there's only 2 Gb of memory installed on my PC, so i don't need it anyway.
Now we do:
make clean
make menuconfig
and see what's happened to Kernel Configuration:
Code: |
(2) Maximum number of CPUs (2-256)
[ ] Support for hot-pluggable CPUs (EXPERIMENTAL)
[*] Provide RTC interrupt
[ ] IOMMU support
[ ] IBM Calgary IOMMU support
|
Now we can disable it
Code: |
< > Siemens R3964 line discipline
< > Applicom intelligent fieldbus card support
Ftape, the floppy tape device driver --->
<*> /dev/agpgart (AGP Support)
<M> AMD Opteron/Athlon64 on-CPU GART support
< > Intel 440LX/BX/GX, I8xx and E7x05 chipset support
< > SiS chipset support
< > VIA chipset support
|
AGPGART as well (the option AMD Opteron/Athlon64 on-CPU GART support was hidden before and /dev/agpgart (AGP Support) one was always enabled) |
|
Back to top |
|
 |
dmpogo Advocate

Joined: 02 Sep 2004 Posts: 3511 Location: Canada
|
Posted: Tue Nov 21, 2006 9:07 pm Post subject: |
|
|
IanPo,
There is one fine point. I don't know if it makes sense (or even can cause problems) to have CALGARY_IOMMU enabled while IOMMU disabled.
Kconfig does not condition CONFIG_CALGARY_IOMMU to be dependent on CONFIG_IOMMU, but this may be because they thought that IOMMU is forced anyway !
So I would be disabling CALGARY every time I disable IOMMU just in case. As you did, of course |
|
Back to top |
|
 |
IanPo n00b

Joined: 30 May 2006 Posts: 64 Location: Moscow, Russia
|
Posted: Wed Nov 22, 2006 12:40 pm Post subject: |
|
|
I agree. For we're tweaking the option manually we also have to set all the dependent ones in accordance. |
|
Back to top |
|
 |
dmpogo Advocate

Joined: 02 Sep 2004 Posts: 3511 Location: Canada
|
Posted: Sun Nov 26, 2006 3:27 am Post subject: |
|
|
IanPo wrote: | I agree. For we're tweaking the option manually we also have to set all the dependent ones in accordance. |
I was told on Linux kernel that CONFIG_CALGARY_IOMMU is completely independent from CONFIG_IOMMU,
despite that the name makes us believe it is a subset |
|
Back to top |
|
 |
Aitikin Apprentice

Joined: 14 May 2005 Posts: 227
|
Posted: Thu Nov 30, 2006 5:25 pm Post subject: |
|
|
I've tried this fix myself and I can't get it to work. My make menuconfig still does not have the IOMMU option available or the agpgart option available. I wish I knew what was goingg wrong. I'm going to try putting the default to N and see what that does. _________________ "it's like sex, except over ssh" ~Valkura
"I should go to bitch, I get up at 6:20" ~Valkura |
|
Back to top |
|
 |
dmpogo Advocate

Joined: 02 Sep 2004 Posts: 3511 Location: Canada
|
Posted: Thu Nov 30, 2006 9:17 pm Post subject: |
|
|
Aitikin wrote: | I've tried this fix myself and I can't get it to work. My make menuconfig still does not have the IOMMU option available or the agpgart option available. I wish I knew what was goingg wrong. I'm going to try putting the default to N and see what that does. |
You are doing something wrong, perhaps you need to run something like make mrproper before you see the changes |
|
Back to top |
|
 |
IanPo n00b

Joined: 30 May 2006 Posts: 64 Location: Moscow, Russia
|
Posted: Sun Dec 03, 2006 7:59 pm Post subject: |
|
|
dmpogo wrote: | IanPo wrote: | I agree. For we're tweaking the option manually we also have to set all the dependent ones in accordance. |
I was told on Linux kernel that CONFIG_CALGARY_IOMMU is completely independent from CONFIG_IOMMU,
despite that the name makes us believe it is a subset |
my bad.
Support for hardware IOMMUs in IBM's xSeries x366 and x460 systems - IOMMU Calgary. Dunno what those systems are. At least, my PC's neither of them
2 Aitikin
If you're still interested in solution, could you post what you did step-by-step here (like i did)? |
|
Back to top |
|
 |
eNTi Veteran


Joined: 20 Oct 2002 Posts: 1011 Location: Salzburg, Austria
|
Posted: Thu Dec 07, 2006 8:19 am Post subject: |
|
|
sorry to ask a little offtopic here, but isn't there a reason, why it's forced into the kernel? can one really use NvAGP on amd64? if yes, that would solve my suspend2 issues, i think. _________________ If you fall off a cliff, you might as well try to fly. After all, you got nothing to lose.
-- John Sheridan - Babylon 5, Season 4 |
|
Back to top |
|
 |
dmpogo Advocate

Joined: 02 Sep 2004 Posts: 3511 Location: Canada
|
Posted: Thu Dec 07, 2006 7:14 pm Post subject: |
|
|
eNTi wrote: | sorry to ask a little offtopic here, but isn't there a reason, why it's forced into the kernel? can one really use NvAGP on amd64? if yes, that would solve my suspend2 issues, i think. |
The reason as stated in Kconfig is "many people get it wrong", so that is a precaution against misconfiguration. I don't know whether it is warranted or not. This is regarding IOMMU, not AGP, AGP is just a byproduct of forcing IOMMU (GART IOMMU to be exact which is just called IOMMU now, CALGARY stuff is completely independent).
What I do know
1) IOMMU is probably safe to disable if you have < 4Gb RAM. It has some uses even for small RAM, but not critical. With > 4Gb some PCI devices may not work properly without IOMMU.
2) I have used my nvidia with both NvAGP=1 and NvAGP=2 (and, surely 3) so I can live with kernel-based agpgart. However, I never push my card to compare performances. But it can be used.
3) Several people with ATI cards seems remarked that some ATI configurations require you to disable kernel-based agpgart driver and that they don't have an option to choose as nvidia people are.
4) In my case my interest to the problem came from other direction. I have a practically headless server with some small video but no AGP bridge at all. Two years ago I experienced some memory related instabilities and I found a stable configuration with AGP off. It is most probably red herring, but I don't want to play these games again with a production server so I have standard config with IOMMU=off and AGP=off. So upgrading to new kernel I was surprised that I don't have this option anymore.
Last edited by dmpogo on Tue Dec 19, 2006 11:22 pm; edited 3 times in total |
|
Back to top |
|
 |
Stolz Moderator


Joined: 19 Oct 2003 Posts: 3028 Location: Hong Kong
|
Posted: Tue Dec 19, 2006 4:44 pm Post subject: |
|
|
Thanks for the workaround, dmpogo.
I have an AMD64 with nVidia proprietary drivers. Since 2.6.18, as soon as I start X, the system starts randomly stalling/freezing, the hard disk light stays lit, and X performance becomes crappy. Disabling IOMMU with the workaround enables again Nvidia's AGP and solves the problem  |
|
Back to top |
|
 |
eNTi Veteran


Joined: 20 Oct 2002 Posts: 1011 Location: Salzburg, Austria
|
Posted: Wed Dec 20, 2006 11:50 am Post subject: |
|
|
yeah it's good, to know, that there is something you can do, to use nvagp after all, but i wouldn't recommend it to any gamers out there. it's very slow on my machine. _________________ If you fall off a cliff, you might as well try to fly. After all, you got nothing to lose.
-- John Sheridan - Babylon 5, Season 4 |
|
Back to top |
|
 |
jonnevers Veteran


Joined: 02 Jan 2003 Posts: 1594 Location: Gentoo64 land
|
Posted: Sat Dec 23, 2006 4:21 pm Post subject: |
|
|
thanks!
I tried everything to get my system to use a kernel > 2.6.17. removing 'if embedded' and then disabling IOMMU and the kernel's AGPART in 'make menuconfig' worked like a charm.
AMD64 xorg-x11 7.1, nvidia-drivers 9742, and beyond-sources 2.6.19-beyond2, fully working! |
|
Back to top |
|
 |
Decibels Veteran


Joined: 16 Aug 2002 Posts: 1630 Location: U.S.A.
|
Posted: Sat Dec 30, 2006 5:25 pm Post subject: |
|
|
Yes, thanks. I have always had to disable IOMMU on my amd64 to get video to work with nvidia driver. I really dislike the fact they made it embedded and not allow us the option to disable it.
I had some trouble with grub and splashutils on 2.6.18-gentoo-r2 kernel so couldn't boot. Ends up that was grub, didn't like some command always had in there for others, but just get black screen when kernel loading. So thought that had to do with the IOMMU thing. Tried the 'nv' driver to get past it, but since figured out it was grub haven't retried with IOMMU enabled and 'nv' driver since removed the embedded.
Once I got past the grub problem, would boot up fine until starting X. Then nothing. That is when removed the embedded on the IOMMU and just incase agpgart was screwed up made it modular so could use the nvagp if needed to try. BUT NO, once disabled IOMMU everything worked fine. Left agpgart as a module and didn't have to try the nvagp. IOMMU is what screwed up the whole process and it appears the kernel maintainers are not going to change it because people were choosing Y when meant N,... _________________ Support bacteria – they’re the only culture some people have.”
– Steven Wright
Last edited by Decibels on Wed Jan 03, 2007 12:14 am; edited 1 time in total |
|
Back to top |
|
 |
Decibels Veteran


Joined: 16 Aug 2002 Posts: 1630 Location: U.S.A.
|
Posted: Sun Dec 31, 2006 6:35 pm Post subject: |
|
|
I went back and left IOMMU enabled (embedded) and tried variouse iommu kernel options. It still appears that disabling by changing Kconfig is the best option.
passing:
iommu=off doesn't work. Still no X with nvidia binary.
Also tried size, allowed, noforce, noagp, iommu=memaper=1 & 2 couldn't get X with nvidia binary.
Only one that works is iommu=noaperture.
Then checked dmesg to see and still get aperture:
Code: | agpgart: Detected AGP bridge 0
agpgart: Setting up Nforce3 AGP.
agpgart: AGP aperture is 32M @ 0xf0000000
PCI-DMA: Disabling IOMMU
Linux agpgart interface v0.101 (c) Dave Jones
[agpgart: Found an AGP 3.0 compliant device at 0000:00:00.0.
agpgart: Putting AGP V3 device at 0000:00:00.0 into 8x mode
agpgart: Putting AGP V3 device at 0000:01:00.0 into 8x mode |
*Later: Upon further checking figure out another option. So here are the options as I see it.
1) You can pass the iommu=noaperture and it works. Saw on nvnews forums that iommu=off may not work and noaperture does in disabling iommu. That appears to be the case with mine also. It must let the bios choose or something. But as you can see I do get aperture of 32Mb.
2) But I don't want to pass iommu=noaperture to kernel. Found out that if I change the bios agp aperture to 64Mb
it will work without having to pass the kernel iommu param. Don't know why yet, but by changing the aperture to
64Mb in bios and leaving Kconfig alone (ie iommu enabled and agpgart compiled in) works. Leaving bios aperture at 32Mb doesn't (have to pass iommu=noaperture).
3) Or each time you recompile new kernel you will have to edit Kconfig to remove the embedded on iommu.
4) Use the 'nv' open-source driver. Most stuff seems to work you will have to resize some of your fonts and opengl stuff probably won't work. But otherwise seems ok. _________________ Support bacteria – they’re the only culture some people have.”
– Steven Wright |
|
Back to top |
|
 |
Decibels Veteran


Joined: 16 Aug 2002 Posts: 1630 Location: U.S.A.
|
Posted: Wed Jan 03, 2007 12:24 am Post subject: |
|
|
Found out that if you have bios agp aperture set to 32Mb that the kernel will:
Code: | If the system's default IOMMU is smaller than 64 MB, the Linux kernel automatically replaces it with a 64 MB IOMMU. |
http://us.download.nvidia.com/XFree86/Linux-x86_64/1.0-9746/README/appendix-aa.html
One way I found to get around it was to change the bios agp aperture to 64Mb. With that, the binary Nvidia driver will work and don't get black screen X lockup. Also if disabled agp in xorg.conf Nvidia driver will work, but that isn't what I want.
I tried to get help with this on nvnews, but can't get any nvidia-bug-reports from the system. Tried while logged on via ssh, open another console, nothing, ssh locks up and can't get to the other console after startx. The lockup is hard and have to hit the reset button.
So with IOMMU embedded, if you don't want to have to edit Kconfig for each new kernel now. You might want to see if increasing your bios agp aperture will help. Works on mine. The kernel is going to take 64Mb anyway for aperture. _________________ Support bacteria – they’re the only culture some people have.”
– Steven Wright |
|
Back to top |
|
 |
Shiznitz Tux's lil' helper

Joined: 04 Jun 2004 Posts: 106 Location: Northern California
|
Posted: Sat Jan 06, 2007 4:55 pm Post subject: |
|
|
Would this be the only workaround you guys can find? It seems as if my Motherboard's BIOS does not let me edit the AGP aperture.
Shiznitz |
|
Back to top |
|
 |
|