View previous topic :: View next topic |
Author |
Message |
Desperadoss n00b
Joined: 19 Jun 2008 Posts: 7
|
Posted: Thu Jun 19, 2008 7:19 am Post subject: Re: [HOWTO] Intel C++ Compiler and per-package CFLAGS |
|
|
|mattst88| wrote: | I've rewritten Gentoo-Wiki's HOWTO on using the Intel C++ Compiler.
http://gentoo-wiki.com/HOWTO_Intel_C_Compiler
The old guide was terribly written and horribly messy.
It also made Portage compile with icc unless you explicitly told it to use gcc. This is bad. Far too many packages don't work with icc to use this behavior. My guide makes Portage compile only specified packages with icc and all others with gcc which will save a few headaches in the long run. It also will give your system the ability to use per-package CFLAGS for gcc and icc!
Need feedback on the guide. Suggestions are welcome. |
Really Thanx ! |
|
Back to top |
|
|
mattst88 Developer
Joined: 28 Oct 2004 Posts: 422
|
Posted: Sun Jun 29, 2008 4:27 am Post subject: |
|
|
To fix the limits.h bug, update to icc-10.1.017. I've already filed a version bump request in bugzilla. |
|
Back to top |
|
|
drakhain n00b
Joined: 29 Jun 2008 Posts: 4
|
Posted: Sun Jun 29, 2008 11:46 pm Post subject: |
|
|
|mattst88| wrote: | To fix the limits.h bug, update to icc-10.1.017. I've already filed a version bump request in bugzilla. |
Hello,
I've installed Gentoo 64 on my XPS M1710 with icc 10.1.17 and i'm still having problem with that limits.h. Since i'm not able to compile blender from the portage tree ( the method explained with bashrc and package.icc/gcc didn't worked for me, meaning that whatever is in those files it is being compiled with icc all the time) i've downloaded the source from blender.org.
Can someone help me compiling blender 2.46 with Icc please? |
|
Back to top |
|
|
mattst88 Developer
Joined: 28 Oct 2004 Posts: 422
|
Posted: Mon Jun 30, 2008 12:41 am Post subject: |
|
|
drakhain wrote: | |mattst88| wrote: | To fix the limits.h bug, update to icc-10.1.017. I've already filed a version bump request in bugzilla. |
Hello,
I've installed Gentoo 64 on my XPS M1710 with icc 10.1.17 and i'm still having problem with that limits.h. Since i'm not able to compile blender from the portage tree ( the method explained with bashrc and package.icc/gcc didn't worked for me, meaning that whatever is in those files it is being compiled with icc all the time) i've downloaded the source from blender.org.
Can someone help me compiling blender 2.46 with Icc please? |
It looks to me like Blender's build system is hardcoded to use gcc.
It cannot be compiled by icc using this guide alone. _________________ My Wiki page |
|
Back to top |
|
|
mattst88 Developer
Joined: 28 Oct 2004 Posts: 422
|
Posted: Mon Jun 30, 2008 1:32 am Post subject: |
|
|
Well, crap. By updating to 10.1.017, it fixed the php build on my AMD64 system, but not on my Thinkpad Core2 Duo.. _________________ My Wiki page |
|
Back to top |
|
|
drakhain n00b
Joined: 29 Jun 2008 Posts: 4
|
Posted: Mon Jun 30, 2008 5:45 am Post subject: |
|
|
Yes by default blender is build with gcc. to enable the built to be done with icc i just edited the nan_complile file under the blender2.blabla/source directory. After that i compile it with the make command. I tried to use scons after editing it's files but i had to hardcode icc in it too (even if it has some compiler detection in Sconstruct i wasn't able to use it by doing things like: OCC=icc). |
|
Back to top |
|
|
drakhain n00b
Joined: 29 Jun 2008 Posts: 4
|
Posted: Tue Jul 01, 2008 3:27 pm Post subject: |
|
|
So?, there is no workaround for using icc without being bored by a "can't open limits.h" or "limits.h trying to call himself"? |
|
Back to top |
|
|
mattst88 Developer
Joined: 28 Oct 2004 Posts: 422
|
Posted: Tue Jul 01, 2008 3:35 pm Post subject: |
|
|
I haven't found it yet if there is an existing fix.
The icc-10.1.017 readme says php now compiles on AMD64.
icc-10.1.015 will not compile php on either my desktop or my laptop due to the limits.h error.
icc-10.1.017 will compile php on my desktop, but still fails on my laptop due to the limits.h error.
Bizarre. _________________ My Wiki page |
|
Back to top |
|
|
drakhain n00b
Joined: 29 Jun 2008 Posts: 4
|
Posted: Tue Jul 01, 2008 3:49 pm Post subject: |
|
|
Ok :s, but is it possible that includes from gcc and include from icc are interfering when icc is parsing the sources? |
|
Back to top |
|
|
mattst88 Developer
Joined: 28 Oct 2004 Posts: 422
|
Posted: Tue Jul 01, 2008 4:17 pm Post subject: |
|
|
drakhain wrote: | Ok :s, but is it possible that includes from gcc and include from icc are interfering when icc is parsing the sources? |
It is possible, but I think it is some kind of icc bug. _________________ My Wiki page |
|
Back to top |
|
|
brian33x51 Tux's lil' helper
Joined: 16 Jun 2002 Posts: 118
|
Posted: Thu Jul 10, 2008 4:49 pm Post subject: |
|
|
Yup I'll verify this is still a problem (10.0.017 x86_64). I can't compile since one of the headers in one of the libraries I depend on includes this file. |
|
Back to top |
|
|
DDMask n00b
Joined: 10 Jun 2008 Posts: 4
|
Posted: Sun Jul 13, 2008 9:30 pm Post subject: |
|
|
So I did my own dumb benchmark thing, and bzipping a 2GB file on a Pentium-M (Celeron) went from 21 mins to 16.5 minutes with icc, so I'm feeling it's worthwhile for me to recompile a few more items in order to get this clunky laptop running faster. However my Core2Duo running at 2.6Ghz showed a lesser improvement by going from a bzip2 compression time of 30s on a different file to 26s.
Would any of you smarty-pants be willing to throw together a script which could modify /etc/portage/package.gcc based on failures during emerge? Maybe in the meantime a more comprehensive blacklist could be created on the wiki with not just programs that could bork a system, but also packages that will fail with icc 100% of the time.
The 2% or so gained in speed by using Gentoo is nice I guess, but a gain of >10% is substantial, even if it only applies to some programs. |
|
Back to top |
|
|
mattst88 Developer
Joined: 28 Oct 2004 Posts: 422
|
Posted: Mon Jul 14, 2008 3:15 am Post subject: |
|
|
DDMask wrote: | So I did my own dumb benchmark thing, and bzipping a 2GB file on a Pentium-M (Celeron) went from 21 mins to 16.5 minutes with icc, so I'm feeling it's worthwhile for me to recompile a few more items in order to get this clunky laptop running faster. However my Core2Duo running at 2.6Ghz showed a lesser improvement by going from a bzip2 compression time of 30s on a different file to 26s. |
21 minutes to 16.5 is a speed improvement of 21%. 30 seconds to 26 seconds is still a speed improvement of 13%. Not bad at all. Did you compile bzip2 with -xT -ip -O3 or similar on the Core 2 Duo? _________________ My Wiki page |
|
Back to top |
|
|
DDMask n00b
Joined: 10 Jun 2008 Posts: 4
|
Posted: Mon Jul 14, 2008 11:38 am Post subject: |
|
|
|mattst88| wrote: | Did you compile bzip2 with -xT -ip -O3 or similar on the Core 2 Duo? |
I just used ICCCFLAGS="-O2 -xT -ip -gcc" maybe -O3 would be a better choice.
EDIT: I'm bored so here's my C2D with bzip2 compiled with -O2 -xT -ip -gcc:
real 3m11.639s
user 2m56.118s
sys 0m3.346s
with -O3 -xT -ipo -gcc:
real 3m0.241s
user 2m55.174s
sys 0m2.933s
with gcc, -march=nocona -02 -pipe
real 9m20.463s
user 9m17.525s
sys 0m2.969s
Ummm... that ain't right. 3 tries with gcc and they were all the same. I guess I screwed something up somewhere. Oh well, I'll just go back to an icc build. |
|
Back to top |
|
|
Gregoire Apprentice
Joined: 15 Apr 2006 Posts: 292
|
Posted: Tue Aug 26, 2008 1:23 pm Post subject: |
|
|
Is someone using icc and bashrcng ? |
|
Back to top |
|
|
jdoe Tux's lil' helper
Joined: 02 Jun 2003 Posts: 122 Location: Italy - Florence - Empoli
|
Posted: Wed Aug 27, 2008 8:22 am Post subject: |
|
|
is there a way to use icc on a per package basis with paludis? _________________ me |
|
Back to top |
|
|
a.b. Apprentice
Joined: 23 Mar 2008 Posts: 218 Location: Anus Mundi, Germany
|
Posted: Thu Aug 28, 2008 10:42 pm Post subject: |
|
|
jdoe wrote: | is there a way to use icc on a per package basis with paludis? |
I's amazing. I just have finished a bashrc for paludis and now I see your post
I've uploaded it together with some example config files here: http://ihatenicks.bplaced.net/paludis-bashrc/ Read the config files and the comments in the script for instructions.
I haven't done much testing yet, feedback is welcome. |
|
Back to top |
|
|
Gregoire Apprentice
Joined: 15 Apr 2006 Posts: 292
|
Posted: Tue Sep 02, 2008 7:46 am Post subject: |
|
|
Great, I'll emerge paludis this afternoon and I will try this new bashrc
In the meantime, I am for a few days under x11-base/xorg-server-1.4.2 compiled without debug (which fails with gcc) working great compiled with ICCCFLAGS="-O2 -xT -gcc -mtune=core2".
Also media-video/ffmpeg-20089999 from berkano overlay compil and run very well with -O3 -xT -gcc -mtune=core2.
Sofar I can't compil mplayer because of my system having freetype-1 and freetype-2 : https://forums.gentoo.org/viewtopic-t-508174-postdays-0-postorder-asc-start-475.html
Great !!!
Thank for all work that have been done |
|
Back to top |
|
|
steveb Advocate
Joined: 18 Sep 2002 Posts: 4564
|
Posted: Thu Sep 04, 2008 5:38 pm Post subject: |
|
|
I don't know when I installed ICC and used the unofficial Gentoo wiki to copy the script from there and use it. Anyway... after some time I started to modify the script from there. One bit there, another bit there. This is what I use now: Code: | ##
## Set which compiler to favorize
##
primary_compiler="gcc"
secondary_compiler="icc"
###########################################################
##
## DO NOT CONFIGURE ANYTHING BELOW THIS LINE
##
###########################################################
##
## Try to exit as fast as needed
##
[ -z "${CATEGORY}" -o -z "${PN}" ] && return 0
[ "${primary_compiler}" == "gcc" -a -z "${secondary_compiler}" ] && secondary_compiler="icc"
[ "${primary_compiler}" == "icc" -a -z "${secondary_compiler}" ] && secondary_compiler="gcc"
[ "${secondary_compiler}" != "" -a ! -r "${ROOT}"/etc/portage/package.${secondary_compiler} ] && secondary_compiler=""
[ "${secondary_compiler}" == "" -a ! -r "${ROOT}"/etc/portage/package.${primary_compiler} ] && return 0
handled_by_primary=0
handled_by_secondary=0
##
## Function for exporting CFLAGS
##
export_CFLAGS() {
local compiler="gcc"
[ "${@}" == "icc" ] && compiler="icc"
if [ -r "${ROOT}"/etc/portage/package.${compiler}-cflags ]
then
while read target cflags
do
if [ "${target}" == "${CATEGORY}/${PN}" ]
then
export CFLAGS="${cflags}"
break
fi
done < "${ROOT}"/etc/portage/package.${compiler}-cflags
fi
}
##
## Function for exporting CXXFLAGS
##
export_CXXFLAGS() {
local compiler="gcc"
[ "${@}" == "icc" ] && compiler="icc"
if [ -r "${ROOT}"/etc/portage/package.${compiler}-cxxflags ]
then
while read target cxxflags
do
if [ "${target}" == "${CATEGORY}/${PN}" ]
then
export CXXFLAGS="${cxxflags}"
break
fi
done < "${ROOT}"/etc/portage/package.${compiler}-cxxflags
fi
}
##
## Function for exporting LDFLAGS
##
export_LDFLAGS() {
local compiler="gcc"
[ "${@}" == "icc" ] && compiler="icc"
if [ -r "${ROOT}"/etc/portage/package.${compiler}-ldflags ]
then
while read target ldflags
do
if [ "${target}" == "${CATEGORY}/${PN}" ]
then
export LDFLAGS="${ldflags}"
break
fi
done < "${ROOT}"/etc/portage/package.${compiler}-ldflags
fi
}
##
## Check if primary compiler will handle the package
##
if [ -r "${ROOT}"/etc/portage/package.${primary_compiler} ]
then
while read -a target
do
if [ "${target}" == "${CATEGORY}/${PN}" ]
then
handled_by_primary=1
if [ "${primary_compiler}" == "icc" ]
then
export OCC="icc"
export OCXX="icpc"
#export F77="ifort"
#export FC="ifort"
export CFLAGS=${ICCCFLAGS}
export CXXFLAGS=${ICCCXXFLAGS}
export LDFLAGS=${ICCLDFLAGS}
fi
export_CFLAGS ${primary_compiler}
export CXXFLAGS="${CFLAGS}"
export_CXXFLAGS ${primary_compiler}
export_LDFLAGS ${primary_compiler}
break
fi
done < "${ROOT}"/etc/portage/package.${primary_compiler}
fi
##
## Check if secondary compiler should handle the package
##
if [ "${handled_by_primary}" == "0" -a "${secondary_compiler}" != "" -a -r "${ROOT}"/etc/portage/package.${secondary_compiler} ]
then
while read -a target
do
if [ "${target}" == "${CATEGORY}/${PN}" ]
then
handled_by_secondary=1
if [ "${secondary_compiler}" == "icc" ]
then
export OCC="icc"
export OCXX="icpc"
#export F77="ifort"
#export FC="ifort"
export CFLAGS=${ICCCFLAGS}
export CXXFLAGS=${ICCCXXFLAGS}
export LDFLAGS=${ICCLDFLAGS}
fi
export_CFLAGS ${secondary_compiler}
export CXXFLAGS="${CFLAGS}"
export_CXXFLAGS ${secondary_compiler}
export_LDFLAGS ${secondary_compiler}
break
fi
done < "${ROOT}"/etc/portage/package.${secondary_compiler}
fi
##
## System compiler will handle the package
##
[ "${handled_by_primary}" == "0" -a "${handled_by_secondary}" == "0" ] && return 0
##
## workaround gcc detection function in toolchain-funcs.eclass
##
if [ "${OCC}" != "" ]
then
export CC_FOR_BUILD="${OCC}"
fi |
I have another one but that one is not 100% finished and working the way I would like it to work. Anyway... maybe some one will find this one more flexible then the one in the unofficial Gentoo wiki? It works +/- the same way as the one in the unofficial Gentoo wiki except that it allows you as well to use different flags for the C++ compiler (/etc/portage/package.{gcc,icc}-cxxflags) and if needed LDFLAGS (/etc/portage/package.{gcc,icc}-ldflags). The CFLAGS can be specified in /etc/portage/package.{gcc,icc}-cflags. If no C++ flags are specified, then the script will use the one specified for the C compiler. Some stuff looks freaky but I tried to avoid running into problems by using external programs like sed, awk, grep, etc...
// SteveB |
|
Back to top |
|
|
tranquilcool Veteran
Joined: 25 Mar 2005 Posts: 1179
|
Posted: Thu Sep 04, 2008 11:43 pm Post subject: |
|
|
steveb wrote: | I don't know when I installed ICC and used the unofficial Gentoo wiki to copy the script from there and use it. Anyway... after some time I started to modify the script from there. One bit there, another bit there. This is what I use now: Code: | ##
## Set which compiler to favorize
##
primary_compiler="gcc"
secondary_compiler="icc"
###########################################################
##
## DO NOT CONFIGURE ANYTHING BELOW THIS LINE
##
###########################################################
##
## Try to exit as fast as needed
##
[ -z "${CATEGORY}" -o -z "${PN}" ] && return 0
[ "${primary_compiler}" == "gcc" -a -z "${secondary_compiler}" ] && secondary_compiler="icc"
[ "${primary_compiler}" == "icc" -a -z "${secondary_compiler}" ] && secondary_compiler="gcc"
[ "${secondary_compiler}" != "" -a ! -r "${ROOT}"/etc/portage/package.${secondary_compiler} ] && secondary_compiler=""
[ "${secondary_compiler}" == "" -a ! -r "${ROOT}"/etc/portage/package.${primary_compiler} ] && return 0
handled_by_primary=0
handled_by_secondary=0
##
## Function for exporting CFLAGS
##
export_CFLAGS() {
local compiler="gcc"
[ "${@}" == "icc" ] && compiler="icc"
if [ -r "${ROOT}"/etc/portage/package.${compiler}-cflags ]
then
while read target cflags
do
if [ "${target}" == "${CATEGORY}/${PN}" ]
then
export CFLAGS="${cflags}"
break
fi
done < "${ROOT}"/etc/portage/package.${compiler}-cflags
fi
}
##
## Function for exporting CXXFLAGS
##
export_CXXFLAGS() {
local compiler="gcc"
[ "${@}" == "icc" ] && compiler="icc"
if [ -r "${ROOT}"/etc/portage/package.${compiler}-cxxflags ]
then
while read target cxxflags
do
if [ "${target}" == "${CATEGORY}/${PN}" ]
then
export CXXFLAGS="${cxxflags}"
break
fi
done < "${ROOT}"/etc/portage/package.${compiler}-cxxflags
fi
}
##
## Function for exporting LDFLAGS
##
export_LDFLAGS() {
local compiler="gcc"
[ "${@}" == "icc" ] && compiler="icc"
if [ -r "${ROOT}"/etc/portage/package.${compiler}-ldflags ]
then
while read target ldflags
do
if [ "${target}" == "${CATEGORY}/${PN}" ]
then
export LDFLAGS="${ldflags}"
break
fi
done < "${ROOT}"/etc/portage/package.${compiler}-ldflags
fi
}
##
## Check if primary compiler will handle the package
##
if [ -r "${ROOT}"/etc/portage/package.${primary_compiler} ]
then
while read -a target
do
if [ "${target}" == "${CATEGORY}/${PN}" ]
then
handled_by_primary=1
if [ "${primary_compiler}" == "icc" ]
then
export OCC="icc"
export OCXX="icpc"
#export F77="ifort"
#export FC="ifort"
export CFLAGS=${ICCCFLAGS}
export CXXFLAGS=${ICCCXXFLAGS}
export LDFLAGS=${ICCLDFLAGS}
fi
export_CFLAGS ${primary_compiler}
export CXXFLAGS="${CFLAGS}"
export_CXXFLAGS ${primary_compiler}
export_LDFLAGS ${primary_compiler}
break
fi
done < "${ROOT}"/etc/portage/package.${primary_compiler}
fi
##
## Check if secondary compiler should handle the package
##
if [ "${handled_by_primary}" == "0" -a "${secondary_compiler}" != "" -a -r "${ROOT}"/etc/portage/package.${secondary_compiler} ]
then
while read -a target
do
if [ "${target}" == "${CATEGORY}/${PN}" ]
then
handled_by_secondary=1
if [ "${secondary_compiler}" == "icc" ]
then
export OCC="icc"
export OCXX="icpc"
#export F77="ifort"
#export FC="ifort"
export CFLAGS=${ICCCFLAGS}
export CXXFLAGS=${ICCCXXFLAGS}
export LDFLAGS=${ICCLDFLAGS}
fi
export_CFLAGS ${secondary_compiler}
export CXXFLAGS="${CFLAGS}"
export_CXXFLAGS ${secondary_compiler}
export_LDFLAGS ${secondary_compiler}
break
fi
done < "${ROOT}"/etc/portage/package.${secondary_compiler}
fi
##
## System compiler will handle the package
##
[ "${handled_by_primary}" == "0" -a "${handled_by_secondary}" == "0" ] && return 0
##
## workaround gcc detection function in toolchain-funcs.eclass
##
if [ "${OCC}" != "" ]
then
export CC_FOR_BUILD="${OCC}"
fi |
I have another one but that one is not 100% finished and working the way I would like it to work. Anyway... maybe some one will find this one more flexible then the one in the unofficial Gentoo wiki? It works +/- the same way as the one in the unofficial Gentoo wiki except that it allows you as well to use different flags for the C++ compiler (/etc/portage/package.{gcc,icc}-cxxflags) and if needed LDFLAGS (/etc/portage/package.{gcc,icc}-ldflags). The CFLAGS can be specified in /etc/portage/package.{gcc,icc}-cflags. If no C++ flags are specified, then the script will use the one specified for the C compiler. Some stuff looks freaky but I tried to avoid running into problems by using external programs like sed, awk, grep, etc...
// SteveB |
what is this script? is it bashrc to put in /etc/portage?
i don't understand where you put or how you use this script. _________________ this is a strange strange world. |
|
Back to top |
|
|
steveb Advocate
Joined: 18 Sep 2002 Posts: 4564
|
Posted: Thu Sep 04, 2008 11:52 pm Post subject: |
|
|
tranquilcool wrote: | what is this script? is it bashrc to put in /etc/portage? | Yes. It is a bashrc for Portage.
tranquilcool wrote: | i don't understand where you put or how you use this script. | Read the unofficial Gentoo wiki article Intel C++ Compiler how to use it.
// SteveB |
|
Back to top |
|
|
a.b. Apprentice
Joined: 23 Mar 2008 Posts: 218 Location: Anus Mundi, Germany
|
Posted: Thu Sep 04, 2008 11:53 pm Post subject: |
|
|
Gregoire wrote: | Great, I'll emerge paludis this afternoon and I will try this new bashrc |
It should actually work with portage as well, though it's kinda untested and portage complains about the fact I've used sed.
Quote: | Thank for all work that have been done |
My pleasure.
Last edited by a.b. on Thu Sep 04, 2008 11:57 pm; edited 2 times in total |
|
Back to top |
|
|
steveb Advocate
Joined: 18 Sep 2002 Posts: 4564
|
Posted: Thu Sep 04, 2008 11:56 pm Post subject: |
|
|
a.b. wrote: | ... and portage complains about the fact I've used sed. | That's the reason I tried to avoid external programs in my bashrc above.
// SteveB |
|
Back to top |
|
|
a.b. Apprentice
Joined: 23 Mar 2008 Posts: 218 Location: Anus Mundi, Germany
|
Posted: Fri Sep 05, 2008 12:00 am Post subject: |
|
|
The fact portage complains about the usage of external programs in the bashrc seems like a bug to me. Ebuilds aren't supposed to use anything other than bash code but I see no actual reason to restrict that in a configuration file. |
|
Back to top |
|
|
steveb Advocate
Joined: 18 Sep 2002 Posts: 4564
|
Posted: Fri Sep 05, 2008 12:23 am Post subject: |
|
|
a.b. wrote: | The fact portage complains about the usage of external programs in the bashrc seems like a bug to me. Ebuilds aren't supposed to use anything other than bash code but I see no actual reason to restrict that in a configuration file. | Wrong. If you ever have build Gentoo for an embedded system, then you will see that bash is not the number one choice for an shell. So stuff like: Code: | function debug(){
if [[ -n "$debug" && "$debug" == "1" ]]; then
echo "$(date +%T) $1" >> /tmp/pbshrc-test
fi
} | Will not work. What is the reason you use "[[" ... "]]"? Would it not be better and more portable to use (untested but I think it should work): Code: | function debug(){
if [ -n "${debug}" -a "${debug}" == "1" ]; then
echo "$(date +%T) ${1}" >> /tmp/pbshrc-test
fi
} |
And stuff like this: Code: | while read atom data; do
data="$(echo $data | sed -e 's/#.*//')" #Strip comments
if matchatom "$atom"; then
if [ $append -eq 0 ]; then
ret="$data"
else
ret="$ret $data"
fi
fi
done < $1 |
Could be replaced by this (untested but I think it should work. It should be even faster since calling the replace (in your code it is a call to GNU sed) is only done when needed): Code: | while read atom data; do
if matchatom "${atom}"; then
if [ ${append} -eq 0 ]; then
ret="${data%%#*}"
else
ret="${ret} ${data%%#*}"
fi
fi
done < ${1} |
// SteveB |
|
Back to top |
|
|
|