Possible to replace gcc with AMD's x86 Open64?

Message

phucnguyen · Post by **phucnguyen** » Tue Oct 12, 2010 12:29 pm

Hi,

I used to be a Gentoo user for a while, then switched to Ubuntu. Now I'm considering going back to Gentoo.

I just came across this compiler called Open64 I'd never heard of, and got curious. Looks like AMD has one fork called "x86 Open64" here http://developer.amd.com/cpu/open64/Pages/default.aspx

From this page, it looks like the compiler is supposed to have a certain level of compatibility with GCC. I wonder how compatible it is. My question is, have any of you had experience with it?

How does compiled code's performance compare to GCC's, especially CPU intensive apps on AMD processors, and memory consumption in general?
Is it possible to replace GCC entirely with this compiler? If not, is there a list of applications that can be compiled with it, like the list for ICC?

Many thanks

dol-sen · Post by **dol-sen** » Tue Oct 12, 2010 2:08 pm

Interesting. I have yet to find a license listed anywhere, but I did find a page with limitations.

http://developer.amd.com/cpu/open64/ass ... eNotes.txt

xaviermiller · Post by **xaviermiller** » Tue Oct 12, 2010 2:34 pm

It looks like a patched GCC-4.2

Why didn't do AMD a contribution to GCC so that it is centralized into the main development branch ?

dol-sen · Post by **dol-sen** » Wed Oct 13, 2010 12:16 am

ah, found it. Open64 is the main site.

about-open64.html wrote:Formerly known as Pro64, Open64 was initially created by SGI and licensed under the GNU Public License (GPL v2). It was derived from SGI's MIPSPro compiler.

and HP is working to

replace Open64's front end (derived from GCC 2.95's frontend) with the current GCC front end.

|Quantum| · Post by **|Quantum|** » Sun Feb 20, 2011 11:01 am

I wrote a script based on the HOWTO_ICC_and_Portage howto.
Atm I only adapted the script which uses GCC by default (only selected packages will be built by Open64).

Follow the ICC guide but replace the bashrc script by the following code:

[ -r ${ROOT}/etc/portage/package.open64 ] || return 0
while read -a target; do
if [ "${target}" = "${CATEGORY}/${PN}" ]; then
export OCC="opencc"
export OCXX="openCC"
export CFLAGS=${OPEN64CFLAGS}
export CXXFLAGS=${OPEN64CXXFLAGS}
if [ -r ${ROOT}/etc/portage/package.open64-cflags ]; then
while read target flags; do
if [ "${target}" = "${CATEGORY}/${PN}" ]; then
export CFLAGS="$flags"
export CXXFLAGS="$CFLAGS"
break
fi
done < ${ROOT}/etc/portage/package.open64-cflags
fi

break
fi
done < ${ROOT}/etc/portage/package.open64

if [ -r ${ROOT}/etc/portage/package.gcc-cflags ]; then
while read target flags; do
if [ "${target}" = "${CATEGORY}/${PN}" ]; then
export CFLAGS="$flags"
export CXXFLAGS="$CFLAGS"
break
fi
done < ${ROOT}/etc/portage/package.gcc-cflags
fi

if [ "${OCC}" != "" ]; then
export CC_FOR_BUILD="${OCC}" #workaround gcc detection function in toolchain-funcs.eclass
fi

Furthermore, replace package.icc by package.open64 and package.icc-cflags by package.open64-cflags.
In /etc/make.conf, define OPEN64CFLAGS and OPEN64CXXFLAGS.
Be careful though, some CFLAGS do not follow the GCC rules (for example, I had to change -march=athlon-64 to -march=athlon64fx).

Good luck!

|Quantum| · Post by **|Quantum|** » Sun Feb 20, 2011 11:02 am

Oh yes, I managed to compile xpdf already. I'm gonna test several more packages right now.

|Quantum| · Post by **|Quantum|** » Sun Feb 20, 2011 12:02 pm

At the moment, most builds fail with:

undefined reference to `__builtin_object_size'

This seems to be a known problem, but portage is a little behind on the version (4.2.1 as for now).

Also I found this:

o Open64 does not support the following GCC extensions:
- complex integer data types
- decimal floating point types
- GCC vector types containing a single element
- GCC vector types whose base type is char and whose overall size is less
than 8 bytes
- the __builtin_object_size function
- the __builtin_apply function
- vector types other than MMX (8-byte) and SSE (16-bit)

The __builtin_return_address function is supported only with no
optimization.

(from: http://developer.amd.com/cpu/open64/ass ... enotes.txt).

|Quantum| · Post by **|Quantum|** » Tue Feb 22, 2011 12:17 am

I noticed that most of the bugs come from glibc. Perhaps this could be resolved by recompiling glibc with -fno-builtin, if it weren't for the fact that portage strips most of the flags...

|Quantum| · Post by **|Quantum|** » Tue Feb 22, 2011 1:13 pm

Opened Bug 356063.

Arnaudv6 · Post by **Arnaudv6** » Tue Feb 22, 2011 5:58 pm

Thread date set it just in advance compared to this article:
A Linux Compiler Deathmatch: GCC, LLVM, DragonEgg, Open64, Etc...
Which might answer part of your performance questions.

codestr0m · Post by **codestr0m** » Tue Feb 22, 2011 11:44 pm

Arnaudv6 wrote:Thread date set it just in advance compared to this article:
A Linux Compiler Deathmatch: GCC, LLVM, DragonEgg, Open64, Etc...
Which might answer part of your performance questions.

/* disclaimer - I work for PathScale who is the main contributor for Path64 */

May I interject a little here..

#1 Path64 is a full replacement to Open64
a. Has a real open source community (#pathscale, mailing lists, open development, we actually fix bugs etc)
b. Doesn't have the braindead SPEC hacks from AMD
c. Doesn't have the regressions caused by poor merging of PathScale code
d. Portable across Solaris/*BSD/Linux.. etc.. (soon OSX/Win) So may be additionally interesting to the prefix team
e. Very clean build system. (Open64 is 1G source to download and build.. tons of cruft.. Path64 is around 100MB or less)
f. With 1 patch and some super glue we build the linux kernel /* still needs a lot of work though */
g. glibc build is almost done

#2 Path64 has had an ebuild for the longest time.. ping that lazy bonsaikitten who has it stashed away in his overlay
#3 Our performance tuning is balanced between Intel/AMD

Lets put Open64 to rest and please help us build a community and interest around Path64..
https://github.com/path64/compiler

Thanks

./C

/*
Anyone who is doing open source development for gentoo feel free to ping me and I can likely set you up with a free copy for the EKOPath compiler.
*/

Arnaudv6 · Post by **Arnaudv6** » Wed Feb 23, 2011 8:34 am

From the COPYING file,

All files in this source package marked as (C) PathScale, (C) Cray or (C) STMicroelectronics
are licensed to you under the terms of version 3 of the GPL,
regardless of any contrary statement in any license header.

Sounds great indeed.

username234 · Post by **username234** » Wed Feb 23, 2011 11:11 am

|Quantum| wrote:I noticed that most of the bugs come from glibc. Perhaps this could be resolved by recompiling glibc with -fno-builtin, if it weren't for the fact that portage strips most of the flags...

setting -fno-builtin won't help you with respect to __builtin_object_size(). You could try -U_FORTIFY_SOURCE* as that should disable all calls to that function in glibc.

for the record, -fno-builtin disables all automatic conversions from libc functions to their gcc builtin variants. Manual conversions can still be performed by prefixing the desired function with __builtin_ . Note, however, that only some functions have __builtin_ variants. GNU keeps a list somewhere...

*This implies that -D_FORTIFY_SOURCE is being set. GCC does this automatically in most Linux environments, so I'm a bit curious what is setting it for your compiler (assuming it is set).

|Quantum| · Post by **|Quantum|** » Wed Feb 23, 2011 2:34 pm

username234 wrote: for the record, -fno-builtin disables all automatic conversions from libc functions to their gcc builtin variants. Manual conversions can still be performed by prefixing the desired function with __builtin_ . Note, however, that only some functions have __builtin_ variants. GNU keeps a list somewhere...

You mean, set _FORTIFY_SOURCE manually in the Open64 command line for all builds?

username234 · Post by **username234** » Wed Feb 23, 2011 10:01 pm

|Quantum| wrote:You mean, set _FORTIFY_SOURCE manually in the Open64 command line for all builds?

No, I actually mean to make sure that it is not set by passing in -U_FORTIFY_SOURCE (or the compiler's equivalent).

Put simply, every time __builtin_object_size() is seen in the glibc source it should be wrapped with #if _FORTIFY_SOURCE ... #endif

Passing -in -U_FORTIFY_SOURCE to gcc ensures that the _FORTIFY_SOURCE macro is not set. I am assuming your compiler uses the same option (-U) to unset macros.

|Quantum| · Post by **|Quantum|** » Wed Feb 23, 2011 10:08 pm

username234 wrote:
|Quantum| wrote:I noticed that most of the bugs come from glibc. Perhaps this could be resolved by recompiling glibc with -fno-builtin, if it weren't for the fact that portage strips most of the flags...
setting -fno-builtin won't help you with respect to __builtin_object_size(). You could try -U_FORTIFY_SOURCE* as that should disable all calls to that function in glibc.

for the record, -fno-builtin disables all automatic conversions from libc functions to their gcc builtin variants. Manual conversions can still be performed by prefixing the desired function with __builtin_ . Note, however, that only some functions have __builtin_ variants. GNU keeps a list somewhere...

*This implies that -D_FORTIFY_SOURCE is being set. GCC does this automatically in most Linux environments, so I'm a bit curious what is setting it for your compiler (assuming it is set).

IT WORKS!!!

flybyray · Post by **flybyray** » Sun Mar 06, 2011 11:24 pm

|Quantum| wrote: IT WORKS!!!

What works? Compiling glibc with open64?
How does that work? How did you manage the cpuid.h problem

Code: Select all

checking for cpuid.h... no
configure: error: gcc must provide the <cpuid.h> header

what cpuid.h did you provide or what are your configure flags?

perhaps you did that workaround: http://bugs.gentoo.org/show_bug.cgi?id=292174#c8

|Quantum| · Post by **|Quantum|** » Fri Mar 18, 2011 11:40 am

flybyray wrote:
|Quantum| wrote: IT WORKS!!!
What works? Compiling glibc with open64?
How does that work? How did you manage the cpuid.h problem
Code: Select all
checking for cpuid.h... no
configure: error: gcc must provide the <cpuid.h> header
what cpuid.h did you provide or what are your configure flags?

perhaps you did that workaround: http://bugs.gentoo.org/show_bug.cgi?id=292174#c8

Nono, it actually worked only partially to compile some sources depending on glibc.
The true problem was that I had to switch the gcc-profile back to 4.2.4.
Now I just installed the opencc 4.2.3 binary with gcc-profile 4.4.5 which seems to work for about 60% of the sources.

After some testing, I got numerous other errors from packages depending on specific GCC compiler flags (especially -f flags). Ergo, the GCC compatibility is still not perfect. This might be circumvented with a wrapper script, though.
I'm sorry for the scarce information here, I'll post more details later but right now I'm in the middle of something.

xming · Post by **xming** » Tue Jun 14, 2011 11:46 am

EKOPath going open source.

E001754 · Post by **E001754** » Tue Jun 14, 2011 10:29 pm

xming wrote:EKOPath going open source.

Apart from Phoronix spotlight, do you guys really think the EKOPath that is talked about on being much faster than gcc could a viable solution for a full distro compile ?
I mean, I guess that certainly there would be some (many) compilation breakage with EKOPath that don't appear with gcc, but is that compiler really as faster as it's described in phoronix ?
For day-to-day programs, is it making sense even thinking about it ?

Sorry for the question, but I don't know a shit in programming, except for VBA Excel/Word, which is far from enough to understand things in C/C++.

I guess reality is far from being as simple as Phoronix tends to describe it. Some gains, but I guess also some losses with EKOPath. And in performances too.

Am I right?

Lori · Post by **Lori** » Thu Jun 16, 2011 9:54 am

It would be interesting to test building some performance critical packages with EKOPath and benchmark them. I for one will try if I get the time to build MPlayer, which is already superoptimized (with -O3, regardless of your make.conf, and a lot of assembly), and if I see gains there, I will be impressed.

So more independent benchmarks are required to draw any conclusions.

Desti² · Post by **Desti²** » Sat Aug 13, 2011 2:22 am

Is there a quick solution to switch portage between GCC and ekopath compiler, or a how -to to use ekopath within portage at all?

Etal · Post by **Etal** » Sat Aug 13, 2011 3:52 am

Desti² wrote:Is there a quick solution to switch portage between GCC and ekopath compiler, or a how -to to use ekopath within portage at all?

You can set your CC and CXX variables to pathcc and pathCC

# CC=pathcc CXX=pathCC emerge -av package

DaggyStyle · Post by **DaggyStyle** » Sat Aug 13, 2011 9:34 am

Desti² wrote:Is there a quick solution to switch portage between GCC and ekopath compiler, or a how -to to use ekopath within portage at all?

I would not recommend it as ekopath doesn't compiles the kernel successfully yet.

Desti² · Post by **Desti²** » Sun Aug 14, 2011 3:10 am

Etal wrote:
Desti² wrote:Is there a quick solution to switch portage between GCC and ekopath compiler, or a how -to to use ekopath within portage at all?
You can set your CC and CXX variables to pathcc and pathCC

# CC=pathcc CXX=pathCC emerge -av package

Ah, thanks, I tried it, but only got an error

checking for x86_64-pc-linux-gnu-gcc... /opt/ekopath/bin/pathcc
checking whether the C compiler works... no
configure: error: in `/var/tmp/portage/media-libs/libtheora-1.1.1/work/libtheora-1.1.1':
configure: error: C compiler cannot create executables

Have I missed a step?