Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
LTO: We are almost there
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6  
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
IvanMajhen
Guru
Guru


Joined: 10 Jun 2006
Posts: 392
Location: Croatia

PostPosted: Sat May 27, 2017 9:57 am    Post subject: Reply with quote

I am getting this for libreoffice:
Code:
/[LNK] Library/libscnlo.so
/var/tmp/portage-ondisk/portage/app-office/libreoffice-5.3.3.2/temp/ccKfBKCF.ltrans20.ltrans.o: In function `MultiLineEdit::~MultiLineEdit()':
<artificial>:(.text+0x213): undefined reference to `vtable for MultiLineEdit'
<artificial>:(.text+0x239): undefined reference to `VTT for MultiLineEdit'
/var/tmp/portage-ondisk/portage/app-office/libreoffice-5.3.3.2/temp/ccKfBKCF.ltrans20.ltrans.o: In function `dp_gui::LicenseView::~LicenseView()':
<artificial>:(.text+0x2fb): undefined reference to `vtable for MultiLineEdit'
<artificial>:(.text+0x332): undefined reference to `vtable for MultiLineEdit'
/var/tmp/portage-ondisk/portage/app-office/libreoffice-5.3.3.2/temp/ccKfBKCF.ltrans20.ltrans.o: In function `MultiLineEdit::~MultiLineEdit()':
<artificial>:(.text+0x3b3): undefined reference to `vtable for MultiLineEdit'
<artificial>:(.text+0x3e0): undefined reference to `VTT for MultiLineEdit'
/var/tmp/portage-ondisk/portage/app-office/libreoffice-5.3.3.2/temp/ccKfBKCF.ltrans20.ltrans.o: In function `makeLicenseView':
<artificial>:(.text+0x7be): undefined reference to `vtable for MultiLineEdit'
collect2: error: ld returned 1 exit status

It worked before with same compiler (gcc-6.3)
Back to top
View user's profile Send private message
costel78
Guru
Guru


Joined: 20 Apr 2007
Posts: 305

PostPosted: Thu Jun 01, 2017 2:58 pm    Post subject: Reply with quote

Yes, it is the same error I got on libreoffice here, but is gone with gcc-7.1.0 and libreoffice-5.3.3.2
It was a gcc error since the error persist with libreoffice-5.3.3.2 and gcc-6.3.1
_________________
Sorry for my English. I'm still learning this language.
Back to top
View user's profile Send private message
CaptainBlood
l33t
l33t


Joined: 24 Jan 2010
Posts: 821

PostPosted: Wed Aug 16, 2017 1:44 pm    Post subject: Reply with quote

My no-lto:
Code:
dev-lang/ruby:2.1 no-lto.conf ## LDFLAGS check error
dev-lang/ruby:2.2 no-lto.conf ## LDFLAGS check error
dev-cpp/gtkmm:2.4 no-lto.conf ## ardour! k3d?
dev-lang/php:7.0 no-lto.conf no-tmpfs-4-small-mem.conf
dev-lang/php:7.1 no-lto.conf no-tmpfs-4-small-mem.conf
dev-lang/python:2.7 no-openmp.conf no-lto.conf ## 6.4
dev-lang/python:3.4 no-openmp.conf no-lto.conf ## 6.4
dev-lang/python:3.5 no-openmp.conf no-lto.conf ## 6.4
dev-lang/python:3.6 no-openmp.conf no-lto.conf ## 6.4
dev-lang/rust no-tmpfs.conf no-graphite.conf no-lto.conf no-gcc-lto-level-1.conf ld-bfd.conf no-ld-default.conf
dev-lang/spidermonkey no-lto.conf no-tmpfs-4-small-mem.conf
dev-libs/libaio no-lto.conf ## xen, qemu
dev-libs/libgcrypt no-lto.conf
dev-libs/log4cxx no-lto.conf
dev-libs/wayland no-lto.conf
dev-qt/qtscript:4 no-lto.conf ## kdelibs # as of 2017/08/14
dev-qt/qtscript:5 no-lto.conf
dev-qt/qtwebkit:4 no-lto.conf no-openmp.conf #no-openmp seems required, seems patchable for lto, hardly
dev-qt/qtwebkit:5 no-lto.conf no-openmp.conf ## unoptimized
dev-util/cargo no-lto.conf ld-gold-time.conf ## time to check
media-gfx/kpovmodeler gcc-no-graphite-identity.conf no-lto.conf no-openmp.conf
media-gfx/splashutils no-lto.conf no-ld-bfd-fix.conf ## bfd must be manually set as default
media-libs/alsa-lib no-lto.conf ## well know reluctant
media-libs/flac no-lto.conf
media-libs/libomxil-bellagio no-lto.conf ## as of 2017/10/05
media-libs/mesa gcc-no-graphite-identity.conf no-lto.conf no-openmp.conf
media-sound/pulseaudio gcc-no-graphite-identity.conf no-lto.conf
net-fs/samba gcc-no-graphite-identity.conf no-lto.conf
sci-electronics/gazebo no-lto.conf no-tmpfs.conf ## 6.4.0
sci-libs/pcl no-lto.conf no-tmpfs.conf
sys-libs/ncurses no-lto.conf ## as of 2017/05/11
x11-drivers/xf86-video-intel no-lto.conf
x11-libs/cairo no-lto.conf ## 6.4.0
My fat-lto:
Code:
dev-lang/tcl fat-lto.conf ## lto bugs dev-lang/tk
dev-libs/beignet fat-lto.conf
dev-qt/designer:4 fat-lto.conf ## builds full lto, required by kdelibs
dev-qt/designer:5 fat-lto.conf ## builds full lto, required by kde-frameworks/kross
dev-qt/qtcore:4 fat-lto.conf ## full lto fails! # kdelibs # !hydrogen
dev-qt/qtgui:4 fat-lto.conf ## essai kdelibs # hydrogen # could try to full lto + no-as-needed.conf
dev-qt/qtgui:5 fat-lto.conf ## qtwayland as of 2017/07/20
gnome-base/orbit fat-lto.conf ## libbonobo!
media-libs/x264 fat-lto.conf ## full lto => config error
sci-libs/mpir fat-lto.conf
x11-libs/wxGTK:2.8 fat-lto.conf no-tmpfs.conf no-tmpfs-4-small-mem.conf ## codeblocks
x11-libs/wxGTK:3.0 fat-lto.conf no-tmpfs.conf no-tmpfs-4-small-mem.conf ## check!

with gcc 6.4.0.
Any ideas to fix some of them?
Thks 4 ur attention.
Back to top
View user's profile Send private message
costel78
Guru
Guru


Joined: 20 Apr 2007
Posts: 305

PostPosted: Wed Aug 16, 2017 6:47 pm    Post subject: Reply with quote

Sadly, it's quite a long list.
I am currently on gcc-7.2.0 but, maybe some fixes applies anyway.

php build with:
CFLAGS="${CFLAGS} -flto-partition=max"
CXXFLAGS="${CXXFLAGS} -flto-partition=max"
LDFLAGS="${LDFLAGS} -flto-partition=max"

ncurses, I presume, it's failing during configure and it needs:
LDFLAGS="${LDFLAGS} -ldl"

I have no problems with ruby:2.4 ( I am not using other slot), libaio, flac, mesa, samba, gtkmm and cairo.
spidermonkey:24 patch:
Code:
--- ./js/src/builtin/Intl.cpp.orig      2017-06-25 11:04:52.150266554 +0200
+++ ./js/src/builtin/Intl.cpp   2017-06-25 11:36:01.103553866 +0200
@@ -998,7 +998,7 @@
     if (!chars2)
         return false;

-    UCollationResult uresult = ucol_strcoll(coll, chars1, length1, chars2, length2);
+    UCollationResult uresult = ucol_strcoll(coll, (const UChar*)chars1, length1, (const UChar*)chars2, length2);

     int32_t res;
     switch (uresult) {
@@ -1333,7 +1333,7 @@
         currency = value.toString();
         MOZ_ASSERT(currency->length() == 3, "IsWellFormedCurrencyCode permits only length-3 strings");
         // uCurrency remains owned by currency.
-        uCurrency = JS_GetStringCharsZ(cx, currency);
+        uCurrency = (const UChar*)JS_GetStringCharsZ(cx, currency);
         if (!uCurrency)
             return NULL;

@@ -1440,12 +1440,12 @@
     if (!chars.resize(INITIAL_STRING_BUFFER_SIZE))
         return false;
     UErrorCode status = U_ZERO_ERROR;
-    int size = unum_formatDouble(nf, x, chars.begin(), INITIAL_STRING_BUFFER_SIZE, NULL, &status);
+    int size = unum_formatDouble(nf, x, (UChar*)chars.begin(), INITIAL_STRING_BUFFER_SIZE, NULL, &status);
     if (status == U_BUFFER_OVERFLOW_ERROR) {
         if (!chars.resize(size))
             return false;
         status = U_ZERO_ERROR;
-        unum_formatDouble(nf, x, chars.begin(), size, NULL, &status);
+        unum_formatDouble(nf, x, (UChar*)chars.begin(), size, NULL, &status);
     }
     if (U_FAILURE(status)) {
         JS_ReportErrorNumber(cx, js_GetErrorMessage, NULL, JSMSG_INTERNAL_INTL_ERROR);
@@ -1800,7 +1800,7 @@
     if (!skeleton)
         return false;
     SkipRoot skip(cx, &skeleton);
-    uint32_t skeletonLen = u_strlen(skeleton);
+    uint32_t skeletonLen = u_strlen((const UChar*)skeleton);

     UErrorCode status = U_ZERO_ERROR;
     UDateTimePatternGenerator *gen = udatpg_open(icuLocale(locale.ptr()), &status);
@@ -1810,7 +1810,7 @@
     }
     ScopedICUObject<UDateTimePatternGenerator> toClose(gen, udatpg_close);

-    int32_t size = udatpg_getBestPattern(gen, skeleton, skeletonLen, NULL, 0, &status);
+    int32_t size = udatpg_getBestPattern(gen, (const UChar*)skeleton, skeletonLen, NULL, 0, &status);
     if (U_FAILURE(status) && status != U_BUFFER_OVERFLOW_ERROR) {
         JS_ReportErrorNumber(cx, js_GetErrorMessage, NULL, JSMSG_INTERNAL_INTL_ERROR);
         return false;
@@ -1820,13 +1820,13 @@
         return false;
     pattern[size] = '\0';
     status = U_ZERO_ERROR;
-    udatpg_getBestPattern(gen, skeleton, skeletonLen, pattern, size, &status);
+    udatpg_getBestPattern(gen, (const UChar*)skeleton, skeletonLen, pattern, size, &status);
     if (U_FAILURE(status)) {
         JS_ReportErrorNumber(cx, js_GetErrorMessage, NULL, JSMSG_INTERNAL_INTL_ERROR);
         return false;
     }

-    RootedString str(cx, JS_NewUCStringCopyZ(cx, pattern));
+    RootedString str(cx, JS_NewUCStringCopyZ(cx, (const jschar *)((const UChar *)pattern)));
     if (!str)
         return false;
     args.rval().setString(str);
@@ -1874,7 +1874,7 @@
         if (!JSObject::getProperty(cx, internals, internals, cx->names().timeZone, &value))
             return NULL;
         if (!value.isUndefined()) {
-            uTimeZone = JS_GetStringCharsZ(cx, value.toString());
+            uTimeZone = (const UChar *)JS_GetStringCharsZ(cx, value.toString());
             if (!uTimeZone)
                 return NULL;
             uTimeZoneLength = u_strlen(uTimeZone);
@@ -1882,7 +1882,7 @@
     }
     if (!JSObject::getProperty(cx, internals, internals, cx->names().pattern, &value))
         return NULL;
-    uPattern = JS_GetStringCharsZ(cx, value.toString());
+    uPattern = (const UChar *)JS_GetStringCharsZ(cx, value.toString());
     if (!uPattern)
         return NULL;
     uPatternLength = u_strlen(uPattern);
@@ -1921,12 +1921,12 @@
     if (!chars.resize(INITIAL_STRING_BUFFER_SIZE))
         return false;
     UErrorCode status = U_ZERO_ERROR;
-    int size = udat_format(df, x, chars.begin(), INITIAL_STRING_BUFFER_SIZE, NULL, &status);
+    int size = udat_format(df, x, (UChar*)chars.begin(), INITIAL_STRING_BUFFER_SIZE, NULL, &status);
     if (status == U_BUFFER_OVERFLOW_ERROR) {
         if (!chars.resize(size))
             return false;
         status = U_ZERO_ERROR;
-        udat_format(df, x, chars.begin(), size, NULL, &status);
+        udat_format(df, x, (UChar*)chars.begin(), size, NULL, &status);
     }
     if (U_FAILURE(status)) {
         JS_ReportErrorNumber(cx, js_GetErrorMessage, NULL, JSMSG_INTERNAL_INTL_ERROR);


wayland patch:
Code:
--- org/tests/test-runner.h     2015-07-06 22:38:51.000000000 +0300
+++ new/tests/test-runner.h     2017-05-09 13:57:50.935848215 +0300
@@ -41,7 +41,7 @@
        static void name(void);                                 \
                                                                \
        const struct test test##name                            \
-                __attribute__ ((section ("test_section"))) = { \
+                __attribute__ ((used, section ("test_section"))) = {   \
                #name, name, 0                                  \
        };                                                      \
                                                                \
@@ -51,7 +51,7 @@
        static void name(void);                                 \
                                                                \
        const struct test test##name                            \
-                __attribute__ ((section ("test_section"))) = { \
+                __attribute__ ((used, section ("test_section"))) = {   \
                #name, name, 1                                  \
        };                                                      \
                                                                \


I am not using anymore x11-drivers/xf86-video-intel, yes I am on intel GPU and qt

For python I am using a modified ebuild to enable lto
emake profile-opt for python-2.7.13 and emake profile-opt CPPFLAGS= CFLAGS= LDFLAGS= for python 3.6.2

If it helps, package.env:
Code:
dev-lang/python no-sandbox
dev-libs/libgcrypt no-lto
dev-python/notify-python no-lto
dev-util/cmocka no-lto
media-libs/alsa-lib no-lto
media-sound/pulseaudio no-lto
gnome-base/orbit no-lto
gnome-base/libbonobo no-lto
dev-libs/elfutils no-lto
net-libs/nodejs no-lto
dev-util/cargo no-lto
media-libs/adplug O2
media-libs/libvorbis O2
media-video/ffmpeg O2


and emerge --info:
Code:
Portage 2.3.7 (python 3.6.2-final-0, hardened/linux/amd64/no-multilib, gcc-7.2.0, glibc-2.26, 4.12.7-gentoo-costel x86_64)
=================================================================
System uname: Linux-4.12.7-gentoo-costel-x86_64-Intel-R-_Core-TM-_i7-6700K_CPU_@_4.00GHz-with-gentoo-2.4.1
KiB Mem:    31844056 total,  14384064 free
KiB Swap:          0 total,         0 free
Timestamp of repository gentoo: Wed, 16 Aug 2017 17:30:01 +0000
Head commit of repository gentoo: edad608296c71126f8617012a70a481c659f9461
sh bash 4.4_p12
ld GNU gold (Gentoo 2.29 p1.0 2.29) 1.14
app-shells/bash:          4.4_p12::gentoo
dev-java/java-config:     2.2.0-r3::gentoo
dev-lang/perl:            5.26.0::gentoo
dev-lang/python:          2.7.13::newer, 3.6.2::newer
dev-util/cmake:           3.9.1::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.4.1-r1::gentoo
sys-apps/sandbox:         2.10-r4::gentoo
sys-devel/autoconf:       2.13::gentoo, 2.69-r4::gentoo
sys-devel/automake:       1.13.4-r1::gentoo, 1.15.1::gentoo
sys-devel/binutils:       2.29::gentoo
sys-devel/gcc:            7.2.0::newer
sys-devel/gcc-config:     1.8-r1::gentoo
sys-devel/libtool:        2.4.6-r4::gentoo
sys-devel/make:           4.2.1-r1::gentoo
sys-kernel/linux-headers: 4.11::gentoo (virtual/os-headers)
sys-libs/glibc:           2.26::newer
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.ro.gentoo.org/gentoo-portage/
    priority: -1000
    sync-rsync-extra-opts: --human-readable --delete-before --progress
    sync-rsync-vcs-ignore: true

added
    location: /usr/portage/local/added
    masters: gentoo
    priority: 30

newer
    location: /usr/portage/local/newer
    masters: gentoo
    priority: 100

python
    location: /usr/portage/local/python
    masters: gentoo
    priority: 100

Installed sets: @desktop, @dev, @minimal
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O3 -pipe -march=native -mtune=skylake -fomit-frame-pointer -w -flto=8 -fuse-linker-plugin -floop-nest-optimize"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php7.1/ext-active/ /etc/php/cgi-php7.1/ext-active/ /etc/php/cli-php7.1/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O3 -pipe -march=native -mtune=skylake -fomit-frame-pointer -w -flto=8 -fuse-linker-plugin -floop-nest-optimize"
DISTDIR="/mnt/linux/distfiles"
EMERGE_DEFAULT_OPTS="--jobs=4 --load-average=11 --keep-going --with-bdeps=y --complete-graph --quiet-build=n --autounmask-write"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs candy cgroup collision-protect config-protect-if-modified distlocks fail-clean fixlafiles ipc-sandbox merge-sync multilib-strict news nodoc noinfo parallel-fetch parallel-install preserve-libs protect-owned sandbox sfperms strict unknown-features-filter unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://mirrors.evowise.com/gentoo/ http://ftp.romnet.org/gentoo/ ftp://ftp.romnet.org/gentoo/ http://distfiles.gentoo.org/"
LANG="ro_RO.UTF-8"
LC_ALL="ro_RO.UTF-8"
LDFLAGS="-Wl,-O1,--sort-common,--hash-style=gnu,--as-needed,-z,relro,-z,now -O3 -pipe -march=native -mtune=skylake -fomit-frame-pointer -w -flto=8 -fuse-linker-plugin -floop-nest-optimize"
MAKEOPTS="-j8 --load-average=10"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_EXTRA_OPTS="--human-readable --delete-before --progress"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="X a52 aac acl acpi aes alsa amd64 avx avx2 branding bzip2 cairo caps cli colord cracklib crypt cups custom-cflags custom-optimization cxx dbus dri egl encode exif fam flac fma3 gif glamor gles gpm gstreamer gtk gtk3 hardened iconv icu idn ipv6 jemalloc jpeg justify lcms libnotify libressl lz4 mad mmx mmxext modules mp3 mp4 mpeg ncurses nls nptl ogg opengl openmp opus pam pango pax_kernel pcre pdf pie png policykit popcnt pulseaudio readline sdl seccomp server session sse sse2 sse3 sse4_1 sse4_2 ssl ssp ssse3 svg systemd tcpd threads tiff truetype udev udisks unicode urandom usb vaapi vdpau vhosts vorbis wayland x264 xattr xcb xtpax xvid zlib" ABI_X86="64" ALSA_CARDS="hda-intel usb-audio" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 fma3 mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" CURL_SSL="libressl" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev keyboard mouse" KERNEL="linux" L10N="ro en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="ro en" NETBEANS_MODULES="apisupport extide php webcommon" NGINX_MODULES_HTTP="access auth_basic autoindex browser charset fastcgi gzip limit_conn limit_req memcached referer rewrite split_clients userid gunzip naxsi realip" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-1" POSTGRES_TARGETS="postgres9_5" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby24" USERLAND="GNU" VIDEO_CARDS="intel i965" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS


Best of luck!
_________________
Sorry for my English. I'm still learning this language.
Back to top
View user's profile Send private message
hceline
n00b
n00b


Joined: 30 Aug 2015
Posts: 26

PostPosted: Tue Sep 05, 2017 7:24 pm    Post subject: Reply with quote

After 4 days of research and recompiling I must thank you all for the info, and add what I learned:

Some of the remaining lto errors comes for this issue: https://bugs.gentoo.org/show_bug.cgi?id=603594

TL;DR version is you can replace nolto with FEATURE=nostirp (or even better add "*.a" to $STRIP_MASK) on some packages, which include (but not limited to) the following:

dev-qt/qtgui
dev-qt/designer
dev-libs/crossguid
dev-lang/tcl
dev-lang/tk
dev-qt/qtcore
gnome-base/orbit
Back to top
View user's profile Send private message
Tom_
Guru
Guru


Joined: 20 May 2004
Posts: 376
Location: France

PostPosted: Sat Sep 30, 2017 10:14 am    Post subject: Reply with quote

Are you all using ~amd64?
Do you think that there is a chance of enabling lto on a stable gentoo system (with an up-to-date toolchain - gcc 7.2 for example) ?
Back to top
View user's profile Send private message
hceline
n00b
n00b


Joined: 30 Aug 2015
Posts: 26

PostPosted: Mon Oct 02, 2017 11:28 am    Post subject: Reply with quote

Tom_ wrote:
Are you all using ~amd64?
Do you think that there is a chance of enabling lto on a stable gentoo system (with an up-to-date toolchain - gcc 7.2 for example) ?


I am on a mix of amd64 and ~amd64, pure amd64 would not work with 7.2 toolchain as some packages would fail to compile(as of about 3 weeks ago at-least).
Back to top
View user's profile Send private message
CaptainBlood
l33t
l33t


Joined: 24 Jan 2010
Posts: 821

PostPosted: Mon Oct 02, 2017 5:12 pm    Post subject: Reply with quote

hceline wrote:
Some of the remaining lto errors comes for this issue: https://bugs.gentoo.org/show_bug.cgi?id=603594

TL;DR version is you can replace nolto with FEATURE=nostirp (or even better add "*.a" to $STRIP_MASK) on some packages
Here strip is achieved as
Code:
LDFLAGS+="-Wl,-s".

no strip is achieved by removal of that string from LDFLAGS

Are the use of $STRIP_MASK or $FEATURE identical in effect?

Thks 4 ur attention, interest & support.
Back to top
View user's profile Send private message
hceline
n00b
n00b


Joined: 30 Aug 2015
Posts: 26

PostPosted: Mon Oct 02, 2017 6:17 pm    Post subject: Reply with quote

CaptainBlood wrote:

Are the use of $STRIP_MASK or $FEATURE identical in effect?




Yes, as far as I can tell they are.
Edit: Sorry, just woke up, the above is not correct.

No;
The $FEATURE disable strip for whole system or all executables single ebuild, depending if set in make.conf or package.env.
With $STRIP_MASK one can set nostrip for all .a files system-wide or pr. ebuild with package.env.

I believe the latter to be better as it only affects .a files, which are the ones that makes issues if stripped.
Back to top
View user's profile Send private message
costel78
Guru
Guru


Joined: 20 Apr 2007
Posts: 305

PostPosted: Sun Nov 05, 2017 5:50 pm    Post subject: Reply with quote

Some more optimizations.
python:
Code:
--- /usr/portage/dev-lang/python/python-3.6.3.ebuild    2017-10-31 19:35:06.000000000 +0200
+++ /usr/portage/local/newer/dev-lang/python/python-3.6.3.ebuild        2017-10-12 19:12:38.000000000 +0300
@@ -109,12 +109,13 @@
        fi

        filter-flags -malign-double
+       append-cflags "-I/usr/include/tirpc"

        # https://bugs.gentoo.org/show_bug.cgi?id=50309
-       if is-flagq -O3; then
-               is-flagq -fstack-protector-all && replace-flags -O3 -O2
-               use hardened && replace-flags -O3 -O2
-       fi
+       # if is-flagq -O3; then
+       #       is-flagq -fstack-protector-all && replace-flags -O3 -O2
+       #       use hardened && replace-flags -O3 -O2
+       # fi

        # Export CXX so it ends up in /usr/lib/python3.X/config/Makefile.
        tc-export CXX
@@ -143,6 +144,7 @@
                --without-ensurepip
                --with-system-expat
                --with-system-ffi
+               --enable-optimizations
        )

        OPT="" econf "${myeconfargs[@]}"
@@ -270,7 +272,7 @@
        # if not using a cross-compiler, use the fresh binary
        if ! tc-is-cross-compiler; then
                local -x PYTHON=./python
-               local -x LD_LIBRARY_PATH=${LD_LIBRARY_PATH+${LD_LIBRARY_PATH}:}${PWD}
+               local -x LD_LIBRARY_PATH=${LD_LIBRARY_PATH+${LD_LIBRARY_PATH}:}.
        else
                vars=( PYTHON "${vars[@]}" )
        fi


Code:
--- /usr/portage/dev-lang/python/python-2.7.14.ebuild   2017-11-02 23:12:54.000000000 +0200
+++ /usr/portage/local/newer/dev-lang/python/python-2.7.14.ebuild       2017-10-12 19:12:18.000000000 +0300
@@ -16,7 +16,7 @@

 LICENSE="PSF-2"
 SLOT="2.7"
-KEYWORDS="~alpha amd64 ~arm ~arm64 ~hppa ia64 ~m68k ~mips ppc ppc64 ~s390 ~sh ~sparc x86 ~amd64-fbsd ~sparc-fbsd ~x86-fbsd"
+KEYWORDS="~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64 ~s390 ~sh ~sparc ~x86 ~amd64-fbsd ~sparc-fbsd ~x86-fbsd"
 IUSE="-berkdb build doc elibc_uclibc examples gdbm hardened ipv6 libressl +ncurses +readline sqlite +ssl +threads tk +wide-unicode wininst +xml"

 # Do not add a dependency on dev-lang/python to this ebuild.
@@ -148,12 +148,13 @@
        fi

        filter-flags -malign-double
+       append-cflags "-I/usr/include/tirpc"

        # https://bugs.gentoo.org/show_bug.cgi?id=50309
-       if is-flagq -O3; then
-               is-flagq -fstack-protector-all && replace-flags -O3 -O2
-               use hardened && replace-flags -O3 -O2
-       fi
+       #if is-flagq -O3; then
+       #       is-flagq -fstack-protector-all && replace-flags -O3 -O2
+       #       use hardened && replace-flags -O3 -O2
+       #fi

        if tc-is-cross-compiler; then
                # Force some tests that try to poke fs paths.
@@ -213,7 +214,7 @@
        touch Include/graminit.h Python/graminit.c

        cd "${BUILD_DIR}" || die
-       emake
+       emake profile-opt

        # Work around bug 329499. See also bug 413751 and 457194.
        if has_version dev-libs/libffi[pax_kernel]; then
@@ -312,7 +313,7 @@
        # if not using a cross-compiler, use the fresh binary
        if ! tc-is-cross-compiler; then
                local -x PYTHON=./python
-               local -x LD_LIBRARY_PATH=${LD_LIBRARY_PATH+${LD_LIBRARY_PATH}:}${PWD}
+               local -x LD_LIBRARY_PATH=${LD_LIBRARY_PATH+${LD_LIBRARY_PATH}:}.
        else
                vars=( PYTHON "${vars[@]}" )
        fi


If you are not on hardened profile O3 is safe. Otherwise not. Be warned!
Some downsides: FEATURES="-ipc-sandbox -sandbox -usersandbox" is required for tests to be able to perform. PGO Python use make test phase to collect infos at runtime.

And now some really experimental stuff. Surprisingly, no errors so far. I am using it since a month. Use at your own risk.
Code:
--- /usr/portage/eclass/toolchain.eclass        2017-10-24 10:33:45.000000000 +0300
+++ /usr/portage/local/newer/eclass/toolchain.eclass    2017-10-04 13:15:50.000000000 +0300
@@ -239,7 +239,7 @@
 gentoo_urls() {
        local devspace="HTTP~vapier/dist/URI HTTP~rhill/dist/URI
        HTTP~zorry/patches/gcc/URI HTTP~blueness/dist/URI
-       HTTP~tamiko/distfiles/URI HTTP~slyfox/distfiles/URI"
+       HTTP~tamiko/distfiles/URI"
        devspace=${devspace//HTTP/https:\/\/dev.gentoo.org\/}
        echo mirror://gentoo/$1 ${devspace//URI/$1}
 }
@@ -907,7 +907,7 @@
        if tc_version_is_at_least 3.4 ; then
                # The "release" keyword is new to 4.0. #551636
                local off=$(tc_version_is_at_least 4.0 && echo release || echo no)
-               confgcc+=( --enable-checking="${GCC_CHECKS_LIST:-$(usex debug yes ${off})}" )
+               confgcc+=( --enable-checking="${GCC_CHECKS_LIST:-$(usex debug no ${off})}" )
        fi

        # Branding
@@ -1606,11 +1606,15 @@

        pushd "${WORKDIR}"/build >/dev/null

+       einfo "Costel here..."
        emake \
-               LDFLAGS="${LDFLAGS}" \
+               CFLAGS="${CFLAGS}" \
+               LIBCFLAGS="${CFLAGS}" \
+               LIBCXXFLAGS="${CFLAGS}" \
+               LDFLAGS="${LDFLAGS} ${BOOT_CFLAGS} -flto=4 -fuse-linker-plugin -floop-nest-optimize" \
                STAGE1_CFLAGS="${STAGE1_CFLAGS}" \
                LIBPATH="${LIBPATH}" \
-               BOOT_CFLAGS="${BOOT_CFLAGS}" \
+               BOOT_CFLAGS="${BOOT_CFLAGS} -flto=4 -fuse-linker-plugin -floop-nest-optimize" \
                ${GCC_MAKE_TARGET} \
                || die "emake failed with ${GCC_MAKE_TARGET}"



To be clear:
  • --enable-optimizations and emake profile-opt are perfectly safe for python, tested and maintained by upstream.
  • O3 in python is safe only for no-hardened profiles.
  • gcc with lto it is crazy! Period. But there can not be progress without experiments.

_________________
Sorry for my English. I'm still learning this language.
Back to top
View user's profile Send private message
incripshin
n00b
n00b


Joined: 07 Oct 2005
Posts: 47
Location: Seattle, WA, US

PostPosted: Sun Nov 12, 2017 8:46 am    Post subject: Reply with quote

I was ready to post a bug about this but then realized it's probably hopeless. Thankfully this seems like the place.

It looks like LTO breaks AC_CHECK_FUNC(S) in autoconf, causing it to always return success. The only fix is to disable LTO during AC_CHECK_FUNC, which means (1) a patch to autoconf and (2) every single package using autoconf needs to use eautoreconf when LTO is enabled. Or maybe gentoo could use a wrapper around the compiler during the configure phase to filter out any LTO-related flags (which may have repurcussions).

Anyway, my story. Ncurses failed to build because it tries to use getttynam, which doesn't exist in Musl. The configure script is using the macro AC_CHECK_FUNCS(getttynam ...). The config.log shows (I'm transcribing from another computer):

Code:
checking for getttynam
clang -o conftest -march=native -O2 -pipe -flto -Qunused-arguments -DXTSTRINGDEFINES -DNDEBUG -fuse-ld=lld -Wl,-O2 -Wl,--as-needed conftest.c >&5
$? = 0
test -s conftest
$? = 0
result: yes


Basically this is what conftest.c looks like:

Code:
char getttynam ();
char (*f) ();
int main() {
  f = getttynam;
  return 0;
}


The only thing it is checking for is: at link-time, is there a symbol getttynam? The problem is that there is no reason for an optimizing compiler to care about whether or not it exists. This seems a lot like undefined behavior (though I can't find anything that says whether or not it is).

So I crafted this loop to figure out what leads to this situation (I have two available compilers, three linkers, lto enabled/disabled, and four optimization levels):

Code:
for cc in clang gcc; do
  for ld in bfd gold lld; do
    for lto in lto no-lto; do
      for opt in 0 1 2 3; do
        $cc conftest.c -O$opt -f$lto -fuse-ld=$ld -o conftest-$cc-$lto-$opt-$ld
      done
    done
  done
done


It looks like every possible configuration that uses LTO links cleanly; the rest do not. If I further break up the compilation & linking into two commands (intermediate object file), then clang doesn't work with gold/bfd (bfd = original GNU ld) (gcc never worked with lld since it only allows GNU linkers).

It must be that the linker is only capable of knowing that the symbol isn't needed when LTO is enabled. This seems to be confirmed when I make f a static variable, it always links (so -fno-lto exhibits the same 'broken' behavior).
Back to top
View user's profile Send private message
costel78
Guru
Guru


Joined: 20 Apr 2007
Posts: 305

PostPosted: Sun Nov 12, 2017 11:27 am    Post subject: Reply with quote

There are two things to be taken into account: Link Time optimization it is considered experimental, even after all these years. And the true is that IT IS still a work in progress; the second aspect is that lto is know to expose errors in code. Even the thread title contain the word "almost". :)

Now, I haven't tried musl or uclib and lld was not ready in my case, at least.
I may be wrong, but, more or less, the same undefined behavior it is present with glibc and gcc on ncurses, gimp, gettext and efl.
As a solution to your problem, I recall that x265 or x264 needed to disable lto for a part of configure phase. You may look there to see how they solved or circumvent the issue.
_________________
Sorry for my English. I'm still learning this language.
Back to top
View user's profile Send private message
Naib
Watchman
Watchman


Joined: 21 May 2004
Posts: 5078
Location: Removed by Neddy

PostPosted: Sun Nov 12, 2017 11:44 am    Post subject: Reply with quote

Is there a way to list all packages that were built with LTO?

I started messing with it with 7.2 once I had my ryzen system nice and stable. @system was rebuilt a number of times as I weeded out the troublesome packages & then I deal with @world via attrition.

It maybe worth rebuilding what I have manually to ensure complete coverage

so far these are what I have had to disable lto on

Code:

media-libs/alsa-lib no-lto
media-libs/x264 no-lto
media-libs/mesa no-lto
x11-libs/gtk no-lto
app-text/mupdf no-lto
media-sound/pulseaudio no-lto
dev-lang/spidermonkey no-lto
dev-lang/tk no-lto
dev-qt/qtscript no-lto
dev-qt/qtcore no-lto
dev-qt/qtwebkit no-lto
app-office/libreoffice no-lto

net-libs/webkit-gtk no-lto
dev-lang/php no-lto


_________________
The best argument against democracy is a five-minute conversation with the average voter
Great Britain is a republic, with a hereditary president, while the United States is a monarchy with an elective king
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 5750

PostPosted: Sun Nov 12, 2017 5:25 pm    Post subject: Reply with quote

incripshin wrote:
It looks like LTO breaks AC_CHECK_FUNC(S) in autoconf, causing it to always return success.

This is not the case here (autoconf-2.69-r4). Maybe you are using an earlier autoconf version?
Quote:
Basically this is what conftest.c looks like

The test script generated by autoconf-2.69-r4 basically looks like this:
Code:
char foobar ();
int main ()
{
return foobar ();
  ;
  return 0;
}

Quote:
The problem is that there is no reason for an optimizing compiler to care about whether or not it exists

Indeed, an optimizing compiler will optimize out the variable f and thus also the assignment. In my opinion this is a clear bug of autoconf, relying on certain non-optimizations to happen.
Back to top
View user's profile Send private message
incripshin
n00b
n00b


Joined: 07 Oct 2005
Posts: 47
Location: Seattle, WA, US

PostPosted: Mon Nov 13, 2017 1:01 am    Post subject: Reply with quote

mv wrote:
This is not the case here (autoconf-2.69-r4). Maybe you are using an earlier autoconf version?


You're right. AC_CHECK_FUNC has only worked properly since Autoconf 2.60. But first, I think I got the conftest.c wrong before. It actually was (from configure):

Code:
char getttynam ();
char (*f) ();
int main() {
  f = getttynam; /* workaround for ICC 12.0.3 */ if (f == 0) return 1;
  return 0;
}


It doesn't seem to change anything from before. I don't think that was a standard build of autoconf-2.52, though. I do see that string show up in other packages, so I suspect a popular distribution patched autoconf.

I read through a bunch of autoconf source. It's quite depressing looking at all those macros. I don't understand it entirely, but anyway. Autoconf 2.57 does something slightly different from predecessors, but it's basically the version just above. It looks something like:

Code:
char getttynam ();
char (*f) () = getttynam;
int main() {
  return f != getttynam;
}


Still broken with LTO. The next major change came in Autoconf 2.60. This is the version of conftest.c you posted. It always fails to link! Finally.

I can see only a couple of different policies Portage could adopt:

  • Run eautoreconf if the configure was generated with an old autoconf (less than some stable version; there is a comment Generated by Autoconf 2.52.20141204 in Ncurses' configure)
  • Run eautoreconf always


It definitely needs to do one of these. I'd be happier with always regenerating configure, since it's practically a binary blob. On the other hand, it could be that a configure cannot be regenerated due to changes in autoconf behavior. It's the right thing to do, but it could be a very painful road to go down.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 5750

PostPosted: Mon Nov 13, 2017 9:10 am    Post subject: Reply with quote

incripshin wrote:
I can see only a couple of different policies Portage could adopt

Neither is correct. The correct solution is to ask upstream to fix it:
If upstream is distributing a broken ./configure (caused by using a buggy autoconf), it should get informed so that this can be fixed.
It was always gentoo policy to send patches/fixes upstream, and there is no reason why the bug of using a broken autoconf version should be handled differently.

For packages with dead or uncooperative upstream, a different solution must be used (presumably by calling eautoreconf in the ebuild), but this is always the case in such situations.

Doing routinely patching is not only a waste of resources and calling for other troubles but also hides an upstream problem tacitly, which is not good.
Back to top
View user's profile Send private message
Naib
Watchman
Watchman


Joined: 21 May 2004
Posts: 5078
Location: Removed by Neddy

PostPosted: Fri Dec 08, 2017 9:37 pm    Post subject: Reply with quote

new profile (ie -pie) seems to have broken alot
_________________
The best argument against democracy is a five-minute conversation with the average voter
Great Britain is a republic, with a hereditary president, while the United States is a monarchy with an elective king
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 5750

PostPosted: Sat Dec 09, 2017 6:32 am    Post subject: Reply with quote

Naib wrote:
new profile (ie -pie) seems to have broken alot

I doubt that pie and lto cause problems for each other. Quite the opposite, mixing e.g. pie code and no-pie static libs might work with lto.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6
Page 6 of 6

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum