Firefox and Thunderbird on arm64

Message

Post by **NeddySeagoon** » Mon May 22, 2017 8:47 am

Team,

Both Firefox and Thunderbird on a Raspberry Pi 3 in 64 bit mode appear to stall at

        --optimizejars \
         \
         \
        package-manifest ../../dist ../../dist/thunderbird \
        --non-resource defaults/messenger/mailViews.dat
 * Unable to trace static ELF: /sbin/ldconfig: /sbin/ldconfig -p
 * Unable to trace static ELF: /sbin/ldconfig: /sbin/ldconfig -p
Warning: /var/tmp/portage/mail-client/thunderbird-52.1.0/work/thunderbird-52.1.0/tbird/mail/installer/package-manifest:241: Missing file(s): bin/components/mozldap.xpt
Warning: /var/tmp/portage/mail-client/thunderbird-52.1.0/work/thunderbird-52.1.0/tbird/mail/installer/package-manifest:243: Missing file(s): bin/components/nsAbLDAPAutoCompleteSearch.js
Warning: /var/tmp/portage/mail-client/thunderbird-52.1.0/work/thunderbird-52.1.0/tbird/mail/installer/package-manifest:244: Missing file(s): bin/components/nsLDAPProtocolHandler.js
Warning: /var/tmp/portage/mail-client/thunderbird-52.1.0/work/thunderbird-52.1.0/tbird/mail/installer/package-manifest:245: Missing file(s): bin/components/ldapComponents.manifest
Warning: /var/tmp/portage/mail-client/thunderbird-52.1.0/work/thunderbird-52.1.0/tbird/mail/installer/package-manifest:246: Missing file(s): bin/libldap60.so
Warning: /var/tmp/portage/mail-client/thunderbird-52.1.0/work/thunderbird-52.1.0/tbird/mail/installer/package-manifest:247: Missing file(s): bin/libldif60.so
Warning: /var/tmp/portage/mail-client/thunderbird-52.1.0/work/thunderbird-52.1.0/tbird/mail/installer/package-manifest:248: Missing file(s): bin/libprldap60.so
Executing /var/tmp/portage/mail-client/thunderbird-52.1.0/work/thunderbird-52.1.0/tbird/dist/bin/xpcshell -g /var/tmp/portage/mail-client/thunderbird-52.1.0/work/thunderbird-52.1.0/tbird/dist/bin/ -a /var/tmp/portage/mail-client/thunderbird-52.1.0/work/thunderbird-52.1.0/tbird/dist/bin/ -f /var/tmp/portage/mail-client/thunderbird-52.1.0/work/thunderbird-52.1.0/mozilla/toolkit/mozapps/installer/precompile_cache.js -e precompile_startupcache("resource://gre/");

I left firefox for over 48h.
There is no CPU activity, 148M in swap.

It appears to want to rum some java and there is a selection of java-vm installed.

Code: Select all

Pi3 64bit ~ # eselect java-vm list
Available Java Virtual Machines:
  [1]   icedtea-7  system-vm
  [2]   icedtea-8 
  [3]   oracle-jdk-bin-1.8

Any hints as to how to investigate this?

chithanh · Post by **chithanh** » Tue May 23, 2017 7:15 am

I think this problem is not 64 bit specific. A user reported this last year on arm, but did not follow up on it:
https://archives.gentoo.org/gentoo-user ... c6032c81bf

Also at least one other user has independently rediscovered this issue, but nobody bothered to report a bug apparently: My first idea would be to check if a manual build works, and if it does disable Gentoo specific stuff like sandbox.

Post by **NeddySeagoon** » Tue May 23, 2017 8:35 am

chithanh,

Thanks, I'll try that when libreoffice finishes building. That probably won't be today.

wrc1944 · Post by **wrc1944** » Mon Jun 05, 2017 9:11 pm

Neddy,
I just had the same thing happen with thunderbird on my Ryzen ~amd64 systems. There's a post on bugzilla that said downgrading sandbox to 2.10-r1 worked for some, so I tried it but failed with exactly the same error.

https://bugs.gentoo.org/show_bug.cgi?id=580726 I am running gcc-7.1.0-r1, so there's that....

Post by **NeddySeagoon** » Mon Jun 05, 2017 9:21 pm

wrc1944,

There is also https://bugs.gentoo.org/show_bug.cgi?id=617882.

As I posted on the bug, there is no logic behind the workaround there.

thumper · Post by **thumper** » Mon Jun 05, 2017 11:07 pm

Just to chime in...

I'm also seeing this on Ryzen with gcc-7.1.0-r1, have not had time to look onto it much, and did not record the error message in dmesg, but easy to reproduce, oom killer kicks it out which was a surprise since I have 32G of ram.

George

thumper · Post by **thumper** » Tue Jun 06, 2017 12:47 am

well FWIW setting USE="custom-cflags custom-optimization" for firefox and emerging with:

Code: Select all

CXXFLAGS="${CXXFLAGS} -mno-avx" emerge -1v firefox

Worked fine here.

George

wrc1944 · Post by **wrc1944** » Tue Jun 06, 2017 8:40 pm

thumper,
On thunderbird-52.1.1

Code: Select all

CXXFLAGS="${CXXFLAGS} -mno-avx" emerge thunderbird

worked perfectly for me on ~amd64 Ryzen system, gcc-7.1.0-r1.

Thanks much to you and Neddy for the tips. Much appreciated.

thumper · Post by **thumper** » Fri Jun 16, 2017 1:00 am

FWIW.

For firefox-54.0 and thunderbird-52.2.0 I had to change it to the following to get them installed.

Code: Select all

CXXFLAGS="${CXXFLAGS} -Os -mno-avx" USE="custom-cflags custom-optimization" emerge -1v firefox
CXXFLAGS="${CXXFLAGS} -Os -mno-avx" USE="custom-cflags custom-optimization" emerge -1v thunderbird

YMMV.

George

mDup · Post by **mDup** » Mon Jun 19, 2017 9:24 pm

thumper wrote:FWIW.

For firefox-54.0 and thunderbird-52.2.0 I had to change it to the following to get them installed.
Code: Select all
CXXFLAGS="${CXXFLAGS} -Os -mno-avx" USE="custom-cflags custom-optimization" emerge -1v firefox
CXXFLAGS="${CXXFLAGS} -Os -mno-avx" USE="custom-cflags custom-optimization" emerge -1v thunderbird
YMMV.

George

Did your emerge for firefox-54.0 also require rust and cargo?
Emering rust itself requires rustc so I first built it from git src and installed in /usr/local.
But now (unlike build from git src) emerge for rust fails with errors like:

Code: Select all

# grep error.*: build.log | sort | uniq -c
     74 error[E0433]: failed to resolve. Could not find `Try` in `ops`
      1 error: requires `freeze` lang_item
      5 error: unused macro definition

here is the emerge -pv of firefox:

Code: Select all

# emerge -pv firefox

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N     ] dev-lang/rust-1.16.0:stable/1.16::gentoo  USE="-clang -debug (-doc) -libcxx" 27,804 KiB
[ebuild  N     ] dev-util/cargo-0.17.0::gentoo  USE="-debug (-doc) (-libressl)" 14,253 KiB
[ebuild     U  ] www-client/firefox-54.0::gentoo [50.1.0-r1::rpi3] USE="dbus gmp-autoupdate hwaccel jemalloc pulseaudio system-harfbuzz system-icu system-jpeg system-libevent system-libvpx system-sqlite -bindist -custom-cflags -custom-optimization -debug (-hardened) -jack -neon -nsplugin% (-pgo) (-selinux) -startup-notification (-system-cairo) {-test} -wifi (-gtk2%) (-jit%*) (-skia%)" L10N="-ach -af -an -ar -as -ast -az -bg -bn-BD -bn-IN -br -bs -ca -cak -cs -cy -da -de -dsb -el -en-GB -en-ZA -eo -es-AR -es-CL -es-ES -es-MX -et -eu -fa -ff -fi -fr -fy -ga -gd -gl -gn -gu -he -hi -hr -hsb -hu -hy -id -is -it -ja -ka% -kab% -kk -km -kn -ko -lij -lt -lv -mai -mk -ml -mr -ms -nb -nl -nn -or -pa -pl -pt-BR -pt-PT -rm -ro -ru -si -sk -sl -son -sq -sr -sv -ta -te -th -tr -uk -uz -vi -xh -zh-CN -zh-TW (-be%)" 223,816 KiB

Total: 3 packages (1 upgrade, 2 new), Size of downloads: 265,872 KiB

Post by **NeddySeagoon** » Mon Jun 19, 2017 9:35 pm

mDup,

The rust and cargo ebuilds fetch the birnaries that they need to build with.
If this is arm64 they both need patches but they are keyworded for ~arm64 and ~x86.

The arm64 patches and instructions are on the Gentoo Wiki

thumper · Post by **thumper** » Mon Jun 19, 2017 9:50 pm

My emerge log for that day shows:

Code: Select all

     Thu Jun 15 17:38:11 2017 >>> dev-lang/rust-1.16.0
     Thu Jun 15 17:41:35 2017 >>> dev-util/cargo-0.17.0
     Thu Jun 15 20:14:42 2017 >>> www-client/firefox-54.0
     Thu Jun 15 20:49:22 2017 >>> mail-client/thunderbird-52.2.0

I'm on x86_64 (Ryzen) and the build for firefox and thunderbird failed with the same error as mentioned in this post.
I would guess that ARM has the avx processor flag as well that seems to be at the root of the problem.

According to this rustc is part of rust:

Code: Select all

#  equery belongs /usr/bin/rustc
 * Searching for /usr/bin/rustc ... 
dev-lang/rust-1.16.0 (/usr/bin/rustc-1.16.0)

George

Post by **NeddySeagoon** » Mon Jun 19, 2017 9:52 pm

thumper,

Nope. Setting -fno-avx on arm64 upsets gcc. Its not a legal setting.

mDup · Post by **mDup** » Mon Jun 19, 2017 9:54 pm

NeddySeagoon wrote:mDup,

The rust and cargo ebuilds fetch the birnaries that they need to build with.
If this is arm64 they both need patches but they are keyworded for ~arm64 and ~x86.

The arm64 patches and instructions are on the Gentoo Wiki

I will give that a try.
Keep you informed.

thumper · Post by **thumper** » Mon Jun 19, 2017 10:04 pm

NeddySeagoon wrote:thumper,

Nope. Setting -fno-avx on arm64 upsets gcc. Its not a legal setting.

Interesting, same failure, different cause, well, not being a user of ARM I will step out.

Cheers;

George

wrc1944 · Post by **wrc1944** » Tue Jun 20, 2017 8:07 pm

Hmmmm.. FWIW, one of my ~amd64 Ryzen systems also pulled in rust (didn't notice cargo though). Guess it was firefox that needed it, but I do also have Thunderbird on one of my Ryzen systems. IIRC rust took quite a while to compile, even on a pretty powerful system.

Post by **NeddySeagoon** » Tue Jun 20, 2017 8:32 pm

wrc1944,

That's the bundled LLVM.

Code: Select all

aarch64 NeddySeagoon_Static / # genlop -t rust cargo firefox
 * dev-lang/rust

     Sun Jun 18 04:30:42 2017 >>> dev-lang/rust-1.16.0-r100
       merge time: 7 hours, 8 minutes and 6 seconds.

 * dev-util/cargo

     Sun Jun 18 10:07:42 2017 >>> dev-util/cargo-0.17.0
       merge time: 36 minutes and 7 seconds.

 * www-client/firefox

     Tue Jun 20 21:14:21 2017 >>> www-client/firefox-54.0
       merge time: 2 hours, 51 minutes and 8 seconds.

--- edit ---

Posting this update from firefox-54.0 built with rust and cargo in an aarch64 chroot then installed on a 64 bit Raspberry Pi 3.

Code: Select all

FEATURES="-sandbox -usersandbox" MAKEOPTS="-j8" emerge firefox

is ugly though.

mDup · Post by **mDup** » Tue Jun 20, 2017 10:52 pm

mDup wrote:
NeddySeagoon wrote:mDup,

The rust and cargo ebuilds fetch the birnaries that they need to build with.
If this is arm64 they both need patches but they are keyworded for ~arm64 and ~x86.

The arm64 patches and instructions are on the Gentoo Wiki
I will give that a try.
Keep you informed.

rust and cargo emerge fine natively

native firefox 54.0 emerge has same problem as reported by others (and myself) elsewhere

Code: Select all

Executing /var/tmp/portage/www-client/firefox-54.0/work/firefox-54.0/ff/dist/bin/xpcshell -g /var/tmp/portage/www-client/firefox-54.0/work/firefox-54.0/ff/dist/bin/ -a /var/tmp/portage/www-client/firefox-54.0/work/firefox-54.0/ff/dist/bin/ -f /var/tmp/portage/www-client/firefox-54.0/work/firefox-54.0/toolkit/mozapps/installer/precompile_cache.js -e precompile_startupcache("resource://gre/");

takes forever

Post by **NeddySeagoon** » Tue Jun 20, 2017 11:14 pm

mDup,

Turn off the sandbox.

Code: Select all

FEATURES="-sandbox -usersandbox" emerge firefox -av

worked for me.
Firefox-54.0 runs on a Pi3 too. I have not tested audio yet.

I also have

Code: Select all

--- ./browser/config/mozconfigs/linux32/common-opt.orig 2017-03-27 16:47:57.039428674 -0700
+++ ./browser/config/mozconfigs/linux32/common-opt      2017-03-27 16:43:26.004591584 -0700
@@ -3,6 +3,7 @@
 ac_add_options --enable-update-channel=${MOZ_UPDATE_CHANNEL}
 ac_add_options --with-google-api-keyfile=/builds/gapi.data
 ac_add_options --with-mozilla-api-keyfile=/builds/mozilla-desktop-geoloc-api.key
+ac_add_options --disable-startupcache
 
 . $topsrcdir/build/unix/mozconfig.linux32
 
--- ./browser/config/mozconfigs/linux64/common-opt.orig 2017-03-27 16:46:54.794982020 -0700
+++ ./browser/config/mozconfigs/linux64/common-opt      2017-03-27 16:47:05.345170624 -0700
@@ -3,6 +3,7 @@
 ac_add_options --enable-update-channel=${MOZ_UPDATE_CHANNEL}
 ac_add_options --with-google-api-keyfile=/builds/gapi.data
 ac_add_options --with-mozilla-api-keyfile=/builds/mozilla-desktop-geoloc-api.key
+ac_add_options --disable-startupcache
 
 . $topsrcdir/build/unix/mozconfig.linux

as that was supposed to fix the stall on its own.

mDup · Post by **mDup** » Wed Jun 21, 2017 1:05 pm

NeddySeagoon wrote:mDup,
Turn off the sandbox.
Code: Select all
FEATURES="-sandbox -usersandbox" emerge firefox -av
worked for me.
Firefox-54.0 runs on a Pi3 too. I have not tested audio yet.

@NeddySeagoon
Thanks for expert advice. firefox-54.0 builds now (no patch needed).
Unfortunately I regularly get 'Segmentation fault's. Even with entire mozilla files cleaned.
But that may be my specific problem.
update
firefox-54.0 runs fine under 4.12.x (of course: no audio as it is WIP on amlogic mainline)