Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
gcc-4.8.0-rc1 lto experience
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3  
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
costel78
Guru
Guru


Joined: 20 Apr 2007
Posts: 402

PostPosted: Sat Jun 15, 2013 10:30 am    Post subject: Reply with quote

Hmmm.... I don't pretend I know for sure the cause but your list of failed packages is big.
Are you using --hash-style=gnu in your ldflags ? because if you compile those packages without lto at all and -Wl,--hash-style=gnu in ldflags, many of them are not reporting as not respecting ldflags, so... it's a little bit strange.
I presume that all packages which fail with -lto -fno-fat-lto-objects have a problem to respect ldflags and portage should report this at the end of emerge.

Later edit:
Code:
gentoo ~ # emerge --info app-arch/bzip2
Portage 2.2.0_alpha179 (default/linux/amd64/13.0/desktop/gnome, gcc-4.8.1, glibc-2.17, 3.9.6-gentoo-costel x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-3.9.6-gentoo-costel-x86_64-Intel-R-_Core-TM-_i7_CPU_860_@_2.80GHz-with-gentoo-2.2
KiB Mem:    16429576 total,   9622356 free
KiB Swap:          0 total,         0 free
Timestamp of tree: Mon, 10 Jun 2013 00:45:01 +0000
ld GNU ld (GNU Binutils) 2.23.1
app-shells/bash:          4.2_p45
dev-lang/python:          2.7.5, 3.3.2
dev-util/cmake:           2.8.11.1
dev-util/pkgconfig:       0.28
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.11.8
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.13.2
sys-devel/binutils:       2.23.1
sys-devel/gcc:            4.8.1
sys-devel/gcc-config:     1.8
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.9 (virtual/os-headers)
sys-libs/glibc:           2.17
Repositories: gentoo added
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=native -mtune=native -Wno-error -w -flto=8 -fno-fat-lto-objects -fuse-linker-plugin -ftracer -floop-interchange -floop-strip-mine -floop-block"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -march=native -mtune=native -Wno-error -w -flto=8 -fno-fat-lto-objects -fuse-linker-plugin -ftracer -floop-interchange -floop-strip-mine -floop-block"
DISTDIR="/mnt/linux/distfiles"
EMERGE_DEFAULT_OPTS="--jobs=4 --load-average=10 --keep-going --with-bdeps=y --complete-graph --quiet-build=n --autounmask-write"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs candy collision-protect config-protect-if-modified distlocks fail-clean fixlafiles merge-sync news nodoc noinfo parallel-fetch parallel-install preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://ftp.romnet.org/gentoo/ http://mirrors.xservers.ro/gentoo/ ftp://ftp.romnet.org/gentoo/ http://ftp.roedu.net/pub/mirrors/gentoo.org/ http://distfiles.gentoo.org/"
LANG="ro_RO.utf8"
LC_ALL="ro_RO.UTF-8"
LDFLAGS="-Wl,-O1,--sort-common,--hash-style=gnu,--warn-once,--as-needed,-z,now -O2 -pipe -march=native -mtune=native -Wno-error -w -flto=8 -fno-fat-lto-objects -fuse-linker-plugin -ftracer -floop-interchange -floop-strip-mine -floop-block"
MAKEOPTS="-j8 --load-average=10"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_EXTRA_OPTS="--human-readable --delete-before --progress"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/portage/local/added"
SYNC="rsync://127.0.0.1/gentoo-portage"
USE="X a52 aac acl acpi alsa amd64 berkdb branding bzip2 cairo cdda cdr cli colord consolekit cracklib crypt cxx dbus development dri dts dvd dvdr eds emboss encode evo exif fam firefox flac gdbm gif gmp gnome gnome-keyring gnome-online-accounts gpm gstreamer gtk gtk3 iconv ipv6 ithreads jpeg lcms libnotify libsecret logrotate mad mmx mng modules mp3 mp4 mpeg mudflap multilib mysql nautilus ncurses networkmanager nls nptl ogg opengl openmp pam pango pch pcre pdf png policykit postgres ppds pulseaudio qt3support readline sdl session socialweb spell sse sse2 ssl ssse3 startup-notification svg systemd tcpd theora threads tiff truetype udev udisks unicode upower usb vdpau vhosts vorbis wxwidgets x264 xcb xml xv xvid xvmc zlib" ABI_X86="64" ALSA_CARDS="virtuoso hda-intel" APACHE2_MODULES="actions alias auth_basic auth_digest authn_alias authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers ident imagemap include info log_config log_forensic logio mime mime_magic negotiation reqtimeout rewrite setenvif speling status substitute unique_id userdir usertrack version vhost_alias access_compat authn_core authz_core proxy proxy_http socache_shmcb unixd" APACHE2_MPMS="worker" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog apache bind conntrack cpu disk dns email hddtemp iptables mysql network nginxping postgresql processes sensors uptime users" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="ro en" NGINX_MODULES_HTTP="access auth_basic autoindex browser charset empty_gif fastcgi geo gunzip gzip limit_conn limit_req map memcached naxsi proxy referer rewrite scgi split_clients ssi upstream_ip_hash userid uwsgi addition cache_purge dav fancyindex flv geoip gzip_static headers_more image_filter mp4 perl push random_index realip secure_link spdy stub_status sub upload_progress xslt" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7 python 3_3" PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby19" USERLAND="GNU" VIDEO_CARDS="nv nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, USE_PYTHON

=================================================================
                        Package Settings
=================================================================

app-arch/bzip2-1.0.6-r3 was built with the following:
USE="(multilib) -static -static-libs" ABI_X86="64"


For example, app-arch/bzip2 does not fail in my case, and there are more packages in this situation. O2 vs O3, maybe ?
_________________
Sorry for my English. I'm still learning this language.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Sat Jun 15, 2013 1:45 pm    Post subject: Reply with quote

drhirsch wrote:
Actually I would consider "-flto" without "-fno-fat-lto-objects" even more dangerous.

No, it should be completely safe (up to unknown compiler bugs which of course might always occur).
Quote:
If you only use "-flto" you will never know what you actually get (although it seems to compile fine, you may get runtime errors or completely unoptimized code or other soft errors).

AFAIK, this is false. It is clear what you actually get: Object files compiled with the compiler options which you specified plus additional LTO information, i.e. the only difference with -flto is that LTO information is added. The only disadvantage which you have when you specify -flto at compile time but not at link time (compared to never specifying -flto) is a longer compile time and longer .o and .a files (only the matter actually makes a difference for the installed package).
costel78 wrote:
Yes, packages whose does not respect ldflags and compiled with -flto does not get any optimization at all.

AFAIK, this is false. They get exactly those optimizations which you specified in CFLAGS (but not those which you specified in LDFLAGS). So if you want to be sure about optimization just specify it in both variables.
Back to top
View user's profile Send private message
drhirsch
n00b
n00b


Joined: 08 May 2004
Posts: 66
Location: Germany

PostPosted: Sat Jun 15, 2013 4:58 pm    Post subject: Reply with quote

Quote:
Quote:
If you only use "-flto" you will never know what you actually get (although it seems to compile fine, you may get runtime errors or completely unoptimized code or other soft errors).

AFAIK, this is false. It is clear what you actually get: Object files compiled with the compiler options which you specified plus additional LTO information, i.e. the only difference with -flto is that LTO information is added.


This should be the case, unfortunately it doesn't seem to work reliably yet. I did notice it first time when I compiled x264 with -flto: It was so slow it was unusable for me (I use it for live video streaming). Inspecting the binaries with objdump, I saw that they were completely unoptmizied (as with -O0). In that case the culprit was the build system, silently stripping the optimization flags from the linker command, but leaving -flto alone.

I see it the other way around: By using "-flto" only in conjunction with "-fno-fat-lto-objects" I lose nothing: The packages which now fail to build wouldn't have been LTOed anyway, so they should be not different to packages built with "-fno-lto", the remainder gets compiled somewhat faster, and it avoids possible problems with packages which seem to compile fine, but show runtime failures later.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Sat Jun 15, 2013 5:44 pm    Post subject: Reply with quote

drhirsch wrote:
This should be the case, unfortunately it doesn't seem to work reliably yet.

What you describe does not contradict what I had written:
Quote:
In that case the culprit was the build system, silently stripping the optimization flags from the linker command, but leaving -flto alone.

So the .o files were correctly optimized, but linking with bad options led to bad results. -fno-fat-lto-objects would not have saved you from that, either. (It would only have saved you if the build system would also have stripped -flto from the LDFLAGS, but in this case omitting -fno-fail-lto-objects would just have produced the same (optimized) code as if you would not have used -flto at all).
Quote:
By using "-flto" only in conjunction with "-fno-fat-lto-objects" I lose nothing

You just gave a counterexample where you get a worse result with -flto despite using -fno-fat-lto-objects (if it is really the case that the built system kept -flto but removed other LDFLAGS). So -fno-fat-lto-objects saves you from nothing:
Quote:
the remainder gets compiled somewhat faster

This is really the only advantage.
Quote:
The packages which now fail to build wouldn't have been LTOed anyway

Not necessarily: Some part of the packages (typically the library parts) would not have used LTO; the rest might have used it.
Quote:
and it avoids possible problems with packages which seem to compile fine, but show runtime failures later.

There are no such "runtime failures later" - either LTO is applied during linking or not; neither produces code with runtime failures, especially if LTO is not used during linking.

To make it clear once more: If -flto with -fnot-fat-lto-objects gives you a compilation failure then -flto without -fno-fat-lto-objects either gives you also a compilation failure or does not produce worse/broken code. (Well - of course, there can be always cases where -flto can produce worse/broken code due to compiler bugs, but if this happens for a particular package this is just an accident and is not logically related with omitting -fnot-fat-lto-objects).
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Sat Jun 15, 2013 5:53 pm    Post subject: Reply with quote

For packages which install .a files, using -fno-fat-objects can be even harmfull: Although the package installs properly, it can be that packages using the installed .a files cannot be compiled at all...

Of course, to get the list which you have posted, using -fno-fat-objects is convenient: In this way, one can see which packages have a problematic build system, and it is probably better in this list to include a false positive. In contrast, to actually use the system, I would not recommend to use -fno-fat-objects, currently. In the future, when most packages have been properly fixed, omitting -fno-fat-objects is just a waste of disk space for .a files; but I guess when this happens in some years, this option will be default in gcc anyway...
Back to top
View user's profile Send private message
costel78
Guru
Guru


Joined: 20 Apr 2007
Posts: 402

PostPosted: Sun Jun 16, 2013 5:33 am    Post subject: Reply with quote

mv wrote:
costel78 wrote:
Yes, packages whose does not respect ldflags and compiled with -flto does not get any optimization at all.

AFAIK, this is false. They get exactly those optimizations which you specified in CFLAGS (but not those which you specified in LDFLAGS). So if you want to be sure about optimization just specify it in both variables.

You are correct. I think I should say if for some reason CGLAGS are stripped and LDFLAGS are not respected package could not get any optimization at all, but this does not depend by lto, it depends by build system.
But, at the end of the day I have two practical reasons to use both -flto and -fno-fat-lto-objects now:
1. I have a dual boot system on a 80GB SSD - so space it's a premium. Last time I checked lto save me ~600MB per total. And this came with a bonus of a minor speed improvement.
2. -flto alone it's time consuming. -fno-fat-lto-objects almost every time greatly reduce time required for package to compile at a similar level as without -flto.

For now -fno-fat-lto it's a headache:
Code:
gentoo portage # grep no-lto.conf package.env | wc -l
27
gentoo portage # grep no-fatlto.conf package.env | wc -l
45

but I think it worth the trouble.
_________________
Sorry for my English. I'm still learning this language.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Sun Jun 16, 2013 4:34 pm    Post subject: Reply with quote

costel78 wrote:
-flto alone it's time consuming. -fno-fat-lto-objects almost every time greatly reduce time required for package to compile at a similar level as without -flto

Currently, ccache saves sometimes a lot of time for generating .o files, but ccache cannot help (currently) for the linking phase
This is why the time in the compile phase does not appear as severe to me as the time needed in the link phase (at least for packages which I reemerge several times).
Back to top
View user's profile Send private message
PaulBredbury
Watchman
Watchman


Joined: 14 Jul 2005
Posts: 7310

PostPosted: Sun Aug 04, 2013 3:05 pm    Post subject: Reply with quote

Surprisingly, the -O2 needs to be in LDFLAGS also. As an example:

CFLAGS="-O2 -march=native -fomit-frame-pointer -pipe -flto"
CXXFLAGS=$CFLAGS
LDFLAGS="-O2 -Wl,-O1 -s -flto=2"

Filesize of /usr/bin/less, after compiling less version 460:
Code:
With -O2 in LDFLAGS:  135184
Without:              147596


evolution-data-server 3.6.4 cannot be compiled with LTO, otherwise evolution shows the dates of new emails simply as: ?
Back to top
View user's profile Send private message
_______0
Guru
Guru


Joined: 15 Oct 2012
Posts: 521

PostPosted: Tue Nov 05, 2013 11:12 pm    Post subject: Reply with quote

let me get this straight.

So far now I've installed the entire X stack with graphite flags under 4.7 and have no probs. And with any other packages so far, that I've been able to notice. Perhaps the only failing is blender with horrible runtime side-effects.

Does the state of graphite change with gcc 4.8?

According to some posters on this thread mention that graphite on 4.8 is useless.

What should I do?

thanks

UPDATE:

Interesting working result so far:

gcc 4.7
Code:
ls -s /usr/bin/mplayer
3040 /usr/bin/mplayer


gcc 4.8
Code:
ls -s /usr/bin/mplayer
3068 /usr/bin/mplayer


damn!! 4.8 builds mplayer 28 kb fatter!!
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Wed Nov 06, 2013 7:54 am    Post subject: Reply with quote

_______0 wrote:
Does the state of graphite change with gcc 4.8?

I had so many runtime issues with graphite in gcc-4.8 and gcc-4.8.1 that I decided to disable it. With gcc-4.7* I had much less problems with graphite.
Back to top
View user's profile Send private message
PaulBredbury
Watchman
Watchman


Joined: 14 Jul 2005
Posts: 7310

PostPosted: Wed Nov 06, 2013 1:16 pm    Post subject: Reply with quote

FWIW, I gave up on compiling with LTO - it seemed to mis-compile too often :(
Back to top
View user's profile Send private message
trubicoid
Tux's lil' helper
Tux's lil' helper


Joined: 04 Aug 2009
Posts: 79

PostPosted: Wed Apr 01, 2015 1:04 pm    Post subject: Reply with quote

PaulBredbury wrote:
Surprisingly, the -O2 needs to be in LDFLAGS also. As an example:

CFLAGS="-O2 -march=native -fomit-frame-pointer -pipe -flto"
CXXFLAGS=$CFLAGS
LDFLAGS="-O2 -Wl,-O1 -s -flto=2"

Filesize of /usr/bin/less, after compiling less version 460:
Code:
With -O2 in LDFLAGS:  135184
Without:              147596




actually, the -O2 has no effect in LDFLAGS, maybe you have prelink active?

with or without -O2 in LDFLAGS I get exactly 149664 (on K10, less 458)

without flto both in CFLAGS and LDFLAGS it is 157800
Back to top
View user's profile Send private message
haarp
Guru
Guru


Joined: 31 Oct 2007
Posts: 535

PostPosted: Fri Apr 17, 2015 5:11 pm    Post subject: Reply with quote

trubicoid wrote:
PaulBredbury wrote:
Surprisingly, the -O2 needs to be in LDFLAGS also. As an example:

CFLAGS="-O2 -march=native -fomit-frame-pointer -pipe -flto"
CXXFLAGS=$CFLAGS
LDFLAGS="-O2 -Wl,-O1 -s -flto=2"

Filesize of /usr/bin/less, after compiling less version 460:
Code:
With -O2 in LDFLAGS:  135184
Without:              147596




actually, the -O2 has no effect in LDFLAGS, maybe you have prelink active?

with or without -O2 in LDFLAGS I get exactly 149664 (on K10, less 458)

without flto both in CFLAGS and LDFLAGS it is 157800

This is interesting (and confusing). From the gcc manpage:
Quote:
Additionally, the optimization flags used to compile individual files are not necessarily
related to those used at link time. For instance,

gcc -c -O0 -flto foo.c
gcc -c -O0 -flto bar.c
gcc -o myprog -flto -O3 foo.o bar.o

This produces individual object files with unoptimized assembler code, but the resulting binary
myprog is optimized at -O3. If, instead, the final binary is generated without -flto, then
myprog is not optimized.

If I'm reading this correctly, there actually is a point in having optimization in LDFLAGS. But what happens if it's not specified? Does it assume the same level that was used in CFLAGS? Does it not optimize at all?


edit: Seems to depend on the build script. All scripts I've observed so far tend to add CFLAGS even during the final link, so whatever O-level is used there is also used during the link, unless overridden by another O-flag in LDFLAGS. I'm guessing that PaulBredbury was testing on some application that did NOT add CFLAGS during linktime.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Goto page Previous  1, 2, 3
Page 3 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum