| View previous topic :: View next topic |
| Author |
Message |
Sakaki
Guru
Joined: 21 May 2014
Posts: 409
|
 Posted: Tue Oct 02, 2018 12:00 pm Post subject: Emerging firefox-62.0.2 on arm64 / clang (patch) |
 |
|
Hello,
I've been trying to build www-client/firefox-62.0.2 on arm64 over the last few days (for an RPi3, active USE flags: bindist clang dbus eme-free hwaccel lto screenshot startup-notification), and have kept running into problems with webrtc. Following upstream, I've been building with sys-devel/clang (7.0.0, lto) rather than gcc - not sure if this is contributing to the problem or completely orthogonal to it.
Anyway, there appear to be at least three issues:- Firstly, in some moz.builds, the same .c files are being pulled in twice - once due to the OS_TARGET being Linux, and the second time, due to the CPU_ARCH being aarch64. Since the included files are not protected by the usual #include guard, this results in redefined symbol errors (e.g., "WebRtcSpl_SqrtFloor"). For example, /media/webrtc/trunk/webrtc/common_audio/signal_processing/complex_bit_reverse.c is pulled in twice by media/webrtc/trunk/webrtc/common_audio/common_audio_c_gn/moz.build.
- Secondly, in other moz builds a default 'C' implementation for some functions is first pulled in due to the OS_TARGET being Linux, and then a (conflicting prototype) optimized version is also included, due to the CPU_ARCH being aarch64. For example, in media/webrtc/trunk/webrtc/modules/audio_processing/audio_processing_c_gn/moz.build, both media/webrtc/trunk/webrtc/modules/audio_processing/ns/ns_core.c and media/webrtc/trunk/webrtc/modules/audio_processing/ns/nsx_core.c are pulled in, but they have conflicting prototypes for the UpdateNoiseEstimate(...) function.
- Thirdly, although in various moz.build files, CPU_ARCH being aarch64 defines WEBRTC_HAS_NEON, some common files required for NEON are not included by media/webrtc/trunk/moz.build, causing the final link to fail (due to undefined symbols, such as e.g. "WebRtcSpl_MaxAbsValueW16Neon" etc).
So, I have put together a short patch to address these issues. It's a bit of a hack tbh, as presumably the GN config files (from which these moz.build files are automatically created) are where the real prolbem lies, but I don't have enough knowledge of the firefox build system to fix things at that level. Here's the patch:
| Code: | diff -uNr a/media/webrtc/trunk/moz.build b/media/webrtc/trunk/moz.build
--- a/media/webrtc/trunk/moz.build 2018-10-01 13:05:43.719382546 +0100
+++ b/media/webrtc/trunk/moz.build 2018-10-01 17:31:42.168723566 +0100
@@ -252,6 +252,18 @@
"/media/webrtc/trunk/webrtc/modules/video_processing/video_processing_neon_gn"
]
+# ensure neon components present for aarch64 Linux
+if CONFIG["CPU_ARCH"] == "aarch64" and CONFIG["OS_TARGET"] == "Linux":
+
+ DIRS += [
+ "/media/webrtc/trunk/webrtc/common_audio/common_audio_neon_c_gn",
+ "/media/webrtc/trunk/webrtc/common_audio/common_audio_neon_gn",
+ "/media/webrtc/trunk/webrtc/modules/audio_coding/isac_neon_gn",
+ "/media/webrtc/trunk/webrtc/modules/audio_processing/audio_processing_neon_c_gn",
+ "/media/webrtc/trunk/webrtc/modules/audio_processing/audio_processing_neon_gn",
+ "/media/webrtc/trunk/webrtc/modules/video_processing/video_processing_neon_gn"
+ ]
+
if CONFIG["CPU_ARCH"] == "arm" and CONFIG["OS_TARGET"] == "NetBSD":
DIRS += [
diff -uNr a/media/webrtc/trunk/webrtc/common_audio/common_audio_c_gn/moz.build b/media/webrtc/trunk/webrtc/common_audio/common_audio_c_gn/moz.build
--- a/media/webrtc/trunk/webrtc/common_audio/common_audio_c_gn/moz.build 2018-10-01 13:05:43.789381037 +0100
+++ b/media/webrtc/trunk/webrtc/common_audio/common_audio_c_gn/moz.build 2018-10-01 13:30:31.227200556 +0100
@@ -196,11 +196,13 @@
DEFINES["WEBRTC_ARCH_ARM64"] = True
DEFINES["WEBRTC_HAS_NEON"] = True
- UNIFIED_SOURCES += [
- "/media/webrtc/trunk/webrtc/common_audio/signal_processing/complex_bit_reverse.c",
- "/media/webrtc/trunk/webrtc/common_audio/signal_processing/filter_ar_fast_q12.c",
- "/media/webrtc/trunk/webrtc/common_audio/signal_processing/spl_sqrt_floor.c"
- ]
+ # Linux already pulls the following sources in unconditionally
+ if CONFIG["OS_TARGET"] != "Linux":
+ UNIFIED_SOURCES += [
+ "/media/webrtc/trunk/webrtc/common_audio/signal_processing/complex_bit_reverse.c",
+ "/media/webrtc/trunk/webrtc/common_audio/signal_processing/filter_ar_fast_q12.c",
+ "/media/webrtc/trunk/webrtc/common_audio/signal_processing/spl_sqrt_floor.c"
+ ]
if CONFIG["CPU_ARCH"] == "arm":
diff -uNr a/media/webrtc/trunk/webrtc/modules/audio_coding/isac_fix_c_gn/moz.build b/media/webrtc/trunk/webrtc/modules/audio_coding/isac_fix_c_gn/moz.build
--- a/media/webrtc/trunk/webrtc/modules/audio_coding/isac_fix_c_gn/moz.build 2018-10-01 13:05:43.779381251 +0100
+++ b/media/webrtc/trunk/webrtc/modules/audio_coding/isac_fix_c_gn/moz.build 2018-10-01 13:30:44.116918435 +0100
@@ -206,13 +206,15 @@
DEFINES["WEBRTC_ARCH_ARM64"] = True
DEFINES["WEBRTC_HAS_NEON"] = True
- SOURCES += [
- "/media/webrtc/trunk/webrtc/modules/audio_coding/codecs/isac/fix/source/pitch_filter_c.c"
- ]
+ # Linux already pulls the following sources in unconditionally
+ if CONFIG["OS_TARGET"] != "Linux":
+ SOURCES += [
+ "/media/webrtc/trunk/webrtc/modules/audio_coding/codecs/isac/fix/source/pitch_filter_c.c"
+ ]
- UNIFIED_SOURCES += [
- "/media/webrtc/trunk/webrtc/modules/audio_coding/codecs/isac/fix/source/lattice_c.c"
- ]
+ UNIFIED_SOURCES += [
+ "/media/webrtc/trunk/webrtc/modules/audio_coding/codecs/isac/fix/source/lattice_c.c"
+ ]
if CONFIG["CPU_ARCH"] == "arm":
diff -uNr a/media/webrtc/trunk/webrtc/modules/audio_processing/audio_processing_c_gn/moz.build b/media/webrtc/trunk/webrtc/modules/audio_processing/audio_processing_c_gn/moz.build
--- a/media/webrtc/trunk/webrtc/modules/audio_processing/audio_processing_c_gn/moz.build 2018-10-01 13:05:43.759381684 +0100
+++ b/media/webrtc/trunk/webrtc/modules/audio_processing/audio_processing_c_gn/moz.build 2018-10-01 13:31:07.996395775 +0100
@@ -168,11 +168,13 @@
DEFINES["WEBRTC_ARCH_ARM64"] = True
DEFINES["WEBRTC_HAS_NEON"] = True
- UNIFIED_SOURCES += [
- "/media/webrtc/trunk/webrtc/modules/audio_processing/ns/noise_suppression_x.c",
- "/media/webrtc/trunk/webrtc/modules/audio_processing/ns/nsx_core.c",
- "/media/webrtc/trunk/webrtc/modules/audio_processing/ns/nsx_core_c.c"
- ]
+ # Linux already pulls in a generic set, with differing prototypes
+ if CONFIG["OS_TARGET"] != "Linux":
+ UNIFIED_SOURCES += [
+ "/media/webrtc/trunk/webrtc/modules/audio_processing/ns/noise_suppression_x.c",
+ "/media/webrtc/trunk/webrtc/modules/audio_processing/ns/nsx_core.c",
+ "/media/webrtc/trunk/webrtc/modules/audio_processing/ns/nsx_core_c.c"
+ ]
if CONFIG["CPU_ARCH"] == "arm":
|
(Note that this does use the generic, rather than optimized, noise suppression code, but I couldn't get the latter to link correctly.)
To apply the patch, create the directory /etc/portage/patches/www-client/firefox-62.0.2, and then save the text above as (e.g.) firefox-62.0.2-fix_webrtc_sourcelists_on_arm64_linux.patch within that directory (for more details on using /etc/portage/patches, please see these notes). (You can also download the patch from here.)
Then emerge firefox-62.0.2 again, it should build cleanly this time (note that you need the gold USE flag on sys-devel/llvm, if you use clang). I don't know if the patch works - or is even needed - under gcc (the build cycles for this package being so long, I don't have time to test that right now ><)
If you'd prefer to use a binary package, I have one available for firefox-62.0.2 on arm64 here. I have also pushed a patched www-client/firefox-62.0.2 ebuild to the rpi3-overlay used by the gentoo-on-rpi3-64bit image, so if you are using this image, on your next weekly genup run firefox should upgrade automatically to 62.0.2 (using the binhost binary package).
If anyone out there has this working with gcc (or has a nicer patch!) please chime in!
PS building with lto active takes a long time, and a lot of memory... I had to resort to a user mode binfmt_misc qemu chroot to avoid stalling on swap.
_________________
Regards,
sakaki |
|
| Back to top |
|
 |
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54216
Location: 56N 3W
|
| Posted: Sat Oct 20, 2018 7:32 pm Post subject: |
|
|
Sakaki,
Your patch works for firefox-62.0.3 too. Thank you.
| Code: | | [ebuild R *] www-client/firefox-62.0.3::gentoo USE="bindist dbus hwaccel screenshot startup-notification system-harfbuzz system-icu system-jpeg system-libevent system-libvpx -clang -custom-cflags -custom-optimization -debug -eme-free -geckodriver -gmp-autoupdate (-hardened) -jack -lto -neon -pulseaudio (-selinux) -system-sqlite -test -wifi" |
That build is -clang -lto, which is the default/linux/arm64/17.0/desktop default just now.
I've just started | Code: | | USE=lto emerge www-client/firefox |
-- edit --
That built but its not run time tested yet. My Pi3 is in a mess again.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
Sakaki
Guru
Joined: 21 May 2014
Posts: 409
|
| Posted: Sun Oct 21, 2018 12:26 pm Post subject: |
|
|
Diffing firefox-62.0.{2,3}.ebuild shows, interestingly, a move to using thin LTO, which should save significantly on build resources; also that --disable-webrtc is now used for arm (though not arm64) due to bug 667642.
_________________
Regards,
sakaki |
|
| Back to top |
|
|
Sakaki
Guru
Joined: 21 May 2014
Posts: 409
|
| Posted: Sun Oct 21, 2018 6:33 pm Post subject: |
|
|
Just built 62.0.3 under gcc 8.2.0. Seems to run OK (very limited testing though) on an RPi3; pushed to binhost, and patched ebuild to overlay.
Couldn't get this version to build under clang, strangely - got a error during the config step. May be something up with my user mode qemu chroot though so I'll check that before reporting anything.
_________________
Regards,
sakaki |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Gentoo on ARM
