Well having had a 64bit machine, every since the socket 754 processors became available, I must admit the 64bit installation of gentoo provides very little hassle, is very stable, this machine has been running for days, and apart from a reboot a few days ago to replace the kernel, it has been running for a couple of weeks prior with no problems.
Sure, a year ago there were problems, and I guess there are still one or two now, but no real show stoppers, apart from one that I have found (see the thread on ghostscript in the amd64 forum).
The only real problems with speed on the amd64 in 64bit was the lack of mmx/3dnow/sse/sse2 optimised software when compiling/running software in 64bit mode, which resulted in everything being run from standard code output from gcc. Recently optimised software, such as mplayer/mythtv has started to appear, which results in a lot lower cpu usage, as you'd expect in the presently dominant 32bit environment.
As more software is optimised, and as the gcc compiler get changed to output better (see the thread on gcc4.1 somewhere in the forum), ie, optimised/vectorised code, then I seriously think the only advantage that the 32bit environment presently happens will be finally eroded and become irrelevent.
Rob.
Article: 64-bit performance in Gentoo Linux
Message
I notice all the benchmarks compare AMD chips running in two different modes, using GCC. I wonder what the results would be if they bench-marked 64bit and 32bit Intel chips using ICC/IFC? Properly optimised ICC binaries can be shown to comfortably outperform GCC binaries on 32bit Intel chips, try THIS article for a demonstration. (It also gets interesting results on AMD chips.)