Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Emerging firefox-62.0.2 on arm64 / clang (patch)
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Gentoo on ARM
View previous topic :: View next topic  
Author Message
Sakaki
Apprentice
Apprentice


Joined: 21 May 2014
Posts: 231

PostPosted: Tue Oct 02, 2018 12:00 pm    Post subject: Emerging firefox-62.0.2 on arm64 / clang (patch) Reply with quote

Hello,

I've been trying to build www-client/firefox-62.0.2 on arm64 over the last few days (for an RPi3, active USE flags: bindist clang dbus eme-free hwaccel lto screenshot startup-notification), and have kept running into problems with webrtc. Following upstream, I've been building with sys-devel/clang (7.0.0, lto) rather than gcc - not sure if this is contributing to the problem or completely orthogonal to it.

Anyway, there appear to be at least three issues:
  • Firstly, in some moz.builds, the same .c files are being pulled in twice - once due to the OS_TARGET being Linux, and the second time, due to the CPU_ARCH being aarch64. Since the included files are not protected by the usual #include guard, this results in redefined symbol errors (e.g., "WebRtcSpl_SqrtFloor"). For example, /media/webrtc/trunk/webrtc/common_audio/signal_processing/complex_bit_reverse.c is pulled in twice by media/webrtc/trunk/webrtc/common_audio/common_audio_c_gn/moz.build.
  • Secondly, in other moz builds a default 'C' implementation for some functions is first pulled in due to the OS_TARGET being Linux, and then a (conflicting prototype) optimized version is also included, due to the CPU_ARCH being aarch64. For example, in media/webrtc/trunk/webrtc/modules/audio_processing/audio_processing_c_gn/moz.build, both media/webrtc/trunk/webrtc/modules/audio_processing/ns/ns_core.c and media/webrtc/trunk/webrtc/modules/audio_processing/ns/nsx_core.c are pulled in, but they have conflicting prototypes for the UpdateNoiseEstimate(...) function.
  • Thirdly, although in various moz.build files, CPU_ARCH being aarch64 defines WEBRTC_HAS_NEON, some common files required for NEON are not included by media/webrtc/trunk/moz.build, causing the final link to fail (due to undefined symbols, such as e.g. "WebRtcSpl_MaxAbsValueW16Neon" etc).

So, I have put together a short patch to address these issues. It's a bit of a hack tbh, as presumably the GN config files (from which these moz.build files are automatically created) are where the real prolbem lies, but I don't have enough knowledge of the firefox build system to fix things at that level. Here's the patch:

Code:
diff -uNr a/media/webrtc/trunk/moz.build b/media/webrtc/trunk/moz.build
--- a/media/webrtc/trunk/moz.build   2018-10-01 13:05:43.719382546 +0100
+++ b/media/webrtc/trunk/moz.build   2018-10-01 17:31:42.168723566 +0100
@@ -252,6 +252,18 @@
         "/media/webrtc/trunk/webrtc/modules/video_processing/video_processing_neon_gn"
     ]
 
+# ensure neon components present for aarch64 Linux
+if CONFIG["CPU_ARCH"] == "aarch64" and CONFIG["OS_TARGET"] == "Linux":
+
+    DIRS += [
+        "/media/webrtc/trunk/webrtc/common_audio/common_audio_neon_c_gn",
+        "/media/webrtc/trunk/webrtc/common_audio/common_audio_neon_gn",
+        "/media/webrtc/trunk/webrtc/modules/audio_coding/isac_neon_gn",
+        "/media/webrtc/trunk/webrtc/modules/audio_processing/audio_processing_neon_c_gn",
+        "/media/webrtc/trunk/webrtc/modules/audio_processing/audio_processing_neon_gn",
+        "/media/webrtc/trunk/webrtc/modules/video_processing/video_processing_neon_gn"
+    ]
+
 if CONFIG["CPU_ARCH"] == "arm" and CONFIG["OS_TARGET"] == "NetBSD":
 
     DIRS += [
diff -uNr a/media/webrtc/trunk/webrtc/common_audio/common_audio_c_gn/moz.build b/media/webrtc/trunk/webrtc/common_audio/common_audio_c_gn/moz.build
--- a/media/webrtc/trunk/webrtc/common_audio/common_audio_c_gn/moz.build   2018-10-01 13:05:43.789381037 +0100
+++ b/media/webrtc/trunk/webrtc/common_audio/common_audio_c_gn/moz.build   2018-10-01 13:30:31.227200556 +0100
@@ -196,11 +196,13 @@
     DEFINES["WEBRTC_ARCH_ARM64"] = True
     DEFINES["WEBRTC_HAS_NEON"] = True
 
-    UNIFIED_SOURCES += [
-        "/media/webrtc/trunk/webrtc/common_audio/signal_processing/complex_bit_reverse.c",
-        "/media/webrtc/trunk/webrtc/common_audio/signal_processing/filter_ar_fast_q12.c",
-        "/media/webrtc/trunk/webrtc/common_audio/signal_processing/spl_sqrt_floor.c"
-    ]
+    # Linux already pulls the following sources in unconditionally
+    if CONFIG["OS_TARGET"] != "Linux":
+        UNIFIED_SOURCES += [
+            "/media/webrtc/trunk/webrtc/common_audio/signal_processing/complex_bit_reverse.c",
+            "/media/webrtc/trunk/webrtc/common_audio/signal_processing/filter_ar_fast_q12.c",
+            "/media/webrtc/trunk/webrtc/common_audio/signal_processing/spl_sqrt_floor.c"
+        ]
 
 if CONFIG["CPU_ARCH"] == "arm":
 
diff -uNr a/media/webrtc/trunk/webrtc/modules/audio_coding/isac_fix_c_gn/moz.build b/media/webrtc/trunk/webrtc/modules/audio_coding/isac_fix_c_gn/moz.build
--- a/media/webrtc/trunk/webrtc/modules/audio_coding/isac_fix_c_gn/moz.build   2018-10-01 13:05:43.779381251 +0100
+++ b/media/webrtc/trunk/webrtc/modules/audio_coding/isac_fix_c_gn/moz.build   2018-10-01 13:30:44.116918435 +0100
@@ -206,13 +206,15 @@
     DEFINES["WEBRTC_ARCH_ARM64"] = True
     DEFINES["WEBRTC_HAS_NEON"] = True
 
-    SOURCES += [
-        "/media/webrtc/trunk/webrtc/modules/audio_coding/codecs/isac/fix/source/pitch_filter_c.c"
-    ]
+    # Linux already pulls the following sources in unconditionally
+    if CONFIG["OS_TARGET"] != "Linux":
+        SOURCES += [
+            "/media/webrtc/trunk/webrtc/modules/audio_coding/codecs/isac/fix/source/pitch_filter_c.c"
+        ]
 
-    UNIFIED_SOURCES += [
-        "/media/webrtc/trunk/webrtc/modules/audio_coding/codecs/isac/fix/source/lattice_c.c"
-    ]
+        UNIFIED_SOURCES += [
+            "/media/webrtc/trunk/webrtc/modules/audio_coding/codecs/isac/fix/source/lattice_c.c"
+        ]
 
 if CONFIG["CPU_ARCH"] == "arm":
 
diff -uNr a/media/webrtc/trunk/webrtc/modules/audio_processing/audio_processing_c_gn/moz.build b/media/webrtc/trunk/webrtc/modules/audio_processing/audio_processing_c_gn/moz.build
--- a/media/webrtc/trunk/webrtc/modules/audio_processing/audio_processing_c_gn/moz.build   2018-10-01 13:05:43.759381684 +0100
+++ b/media/webrtc/trunk/webrtc/modules/audio_processing/audio_processing_c_gn/moz.build   2018-10-01 13:31:07.996395775 +0100
@@ -168,11 +168,13 @@
     DEFINES["WEBRTC_ARCH_ARM64"] = True
     DEFINES["WEBRTC_HAS_NEON"] = True
 
-    UNIFIED_SOURCES += [
-        "/media/webrtc/trunk/webrtc/modules/audio_processing/ns/noise_suppression_x.c",
-        "/media/webrtc/trunk/webrtc/modules/audio_processing/ns/nsx_core.c",
-        "/media/webrtc/trunk/webrtc/modules/audio_processing/ns/nsx_core_c.c"
-    ]
+    # Linux already pulls in a generic set, with differing prototypes
+    if CONFIG["OS_TARGET"] != "Linux":
+        UNIFIED_SOURCES += [
+            "/media/webrtc/trunk/webrtc/modules/audio_processing/ns/noise_suppression_x.c",
+            "/media/webrtc/trunk/webrtc/modules/audio_processing/ns/nsx_core.c",
+            "/media/webrtc/trunk/webrtc/modules/audio_processing/ns/nsx_core_c.c"
+        ]
 
 if CONFIG["CPU_ARCH"] == "arm":
 


(Note that this does use the generic, rather than optimized, noise suppression code, but I couldn't get the latter to link correctly.)

To apply the patch, create the directory /etc/portage/patches/www-client/firefox-62.0.2, and then save the text above as (e.g.) firefox-62.0.2-fix_webrtc_sourcelists_on_arm64_linux.patch within that directory (for more details on using /etc/portage/patches, please see these notes). (You can also download the patch from here.)

Then emerge firefox-62.0.2 again, it should build cleanly this time (note that you need the gold USE flag on sys-devel/llvm, if you use clang). I don't know if the patch works - or is even needed - under gcc (the build cycles for this package being so long, I don't have time to test that right now ><)

If you'd prefer to use a binary package, I have one available for firefox-62.0.2 on arm64 here. I have also pushed a patched www-client/firefox-62.0.2 ebuild to the rpi3-overlay used by the gentoo-on-rpi3-64bit image, so if you are using this image, on your next weekly genup run firefox should upgrade automatically to 62.0.2 (using the binhost binary package).

If anyone out there has this working with gcc (or has a nicer patch!) please chime in!

PS building with lto active takes a long time, and a lot of memory... I had to resort to a user mode binfmt_misc qemu chroot to avoid stalling on swap.
_________________
Regards,

sakaki
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 41562
Location: 56N 3W

PostPosted: Sat Oct 20, 2018 7:32 pm    Post subject: Reply with quote

Sakaki,

Your patch works for firefox-62.0.3 too. Thank you.
Code:
[ebuild   R   *] www-client/firefox-62.0.3::gentoo  USE="bindist dbus hwaccel screenshot startup-notification system-harfbuzz system-icu system-jpeg system-libevent system-libvpx -clang -custom-cflags -custom-optimization -debug -eme-free -geckodriver -gmp-autoupdate (-hardened) -jack -lto -neon -pulseaudio (-selinux) -system-sqlite -test -wifi"


That build is -clang -lto, which is the default/linux/arm64/17.0/desktop default just now.

I've just started
Code:
USE=lto emerge www-client/firefox


-- edit --
That built but its not run time tested yet. My Pi3 is in a mess again.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Sakaki
Apprentice
Apprentice


Joined: 21 May 2014
Posts: 231

PostPosted: Sun Oct 21, 2018 12:26 pm    Post subject: Reply with quote

Diffing firefox-62.0.{2,3}.ebuild shows, interestingly, a move to using thin LTO, which should save significantly on build resources; also that --disable-webrtc is now used for arm (though not arm64) due to bug 667642.
_________________
Regards,

sakaki
Back to top
View user's profile Send private message
Sakaki
Apprentice
Apprentice


Joined: 21 May 2014
Posts: 231

PostPosted: Sun Oct 21, 2018 6:33 pm    Post subject: Reply with quote

Just built 62.0.3 under gcc 8.2.0. Seems to run OK (very limited testing though) on an RPi3; pushed to binhost, and patched ebuild to overlay.

Couldn't get this version to build under clang, strangely - got a error during the config step. May be something up with my user mode qemu chroot though so I'll check that before reporting anything.
_________________
Regards,

sakaki
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo on ARM All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum