View previous topic :: View next topic |
Author |
Message |
bpoint Tux's lil' helper
Joined: 07 Oct 2008 Posts: 84 Location: Japan
|
Posted: Sun Oct 06, 2019 8:41 am Post subject: Gentoo with clang/lld/libc++/compiler-rt/libunwind |
|
|
There doesn't seem to be much recent info/details available on switching over Gentoo to use clang instead of gcc, so I decided to try a completely fresh, minimal install of Gentoo (with current clang-8.0.1) to see how well it worked out. (I'm a big fan of clang.)
For the most part, it works quite well and only a small handful of packages required using gcc as a fallback. Below are my findings.
1. USE Flag Setup
First, enable the following USE flags to allow clang to build with LLVM that depends on it's own components (using LLVM-provided libc++ instead of stdlibc++):
Code: | USE="clang compiler-rt default-compiler-rt default-libcxx libcxx libcxxabi libunwind" |
2. Bootstrap clang
Initially using gcc/g++, build clang with all the required dependencies:
Code: | # emerge -vt clang llvm libcxx libcxxabi compiler-rt libunwind lld |
3. Switch to Using clang/lld/libc++ as the Global Compiler/Linker/C++ Library
In /etc/portage/make.conf:
Code: | CC="clang"
CXX="clang++"
CXXFLAGS="-stdlib=libc++ ${CFLAGS}"
LDFLAGS="-fuse-ld=lld -rtlib=compiler-rt -unwindlib=libunwind" |
4. Setup gcc Fallbacks and Other Necessary Package Tweaks
/etc/portage/env/compiler-gcc.conf:
Code: | CC="gcc"
CXX="g++"
CXXFLAGS="${CFLAGS}"
LDFLAGS="" |
/etc/portage/env/with-lgcc_s.conf:
Code: | LDFLAGS="${LDFLAGS} -lgcc_s" |
/etc/portage/env/with-lm.conf:
Code: | LDFLAGS="${LDFLAGS} -lm" |
/etc/portage/package.env: (notes included, see below)
Code: | # build llvm with -lgcc_s included to resolve __register_frame / __deregister_frame undefined symbols
sys-devel/llvm with-lgcc_s.conf
# build sysvinit with gcc to workaround "cannot specify -o when generating multiple output files"
sys-apps/sysvinit compiler-gcc.conf
# use gcc with builds which ignore/filter out use of lld linker (which causes "not an object or archive" when linking with GNU ld)
dev-libs/popt compiler-gcc.conf
sys-libs/binutils-libs compiler-gcc.conf
# build sandbox with gcc as it fails during configure with "configure: error: Unable to determine LIBC PATH ( )"
sys-apps/sandbox compiler-gcc.conf
# use gcc for packages that have inline assembler that doesn't work with clang
dev-libs/libgcrypt compiler-gcc.conf
# use gcc for efibootmgr since the compile process fails with clang
sys-boot/efibootmgr compiler-gcc.conf
# build gettext with -lm included to resolve "undefined symbol: sqrt" and "undefined symbol: log10" link errors
sys-devel/gettext with-lm.conf
# build busybox with gcc since it has some odd math-related link errors (even with -lm)
sys-apps/busybox compiler-gcc.conf
# build elfutils with gcc since configure says "gcc with GNU99 support required"
dev-libs/elfutils compiler-gcc.conf
# build gcc with gcc since building with clang spawns a massive amount of warnings related to rtl.h (maybe not strictly required, but does drastically slow down the compile process)
sys-devel/gcc compiler-gcc.conf
# build glibc with gcc since configure fails with "These critical programs are missing or too old: compiler"
sys-libs/glibc compiler-gcc.conf
|
5. Rebuild clang
Now that clang/llvm/etc have successfully built, rebuild clang with itself:
Code: | # emerge -vt clang llvm libcxx libcxxabi compiler-rt libunwind lld |
6. Rebuild World
At this point, clang has successfully built all of the dependencies it depends on and has rebuilt itself using those dependencies. From here you can rebuild world with:
Code: | # emerge -vt -e @world |
Notes / Issues
For some reason, sys-devel/llvm still has a dependency on libgcc_s.so even though it's being built to use compiler-rt (which should remove the gcc dependency). During building, there are two linker errors with unresolved symbols for __register_frame and __deregister_frame, so the fix I went with was just to pass -lgcc_s to the linker, but only when building llvm.
sys-devel/gettext throws math-related linker errors (undefined symbol: sqrt and log10) when building with clang, which can be fixed by passing -lm to the linker. I found a related bug report/issue here.
So the current list of packages which do NOT build with clang and require gcc (either fail during configure, or fail to build at all) are:
Code: | dev-libs/elfutils
dev-libs/libgcrypt
dev-libs/popt
sys-apps/busybox
sys-apps/sandbox
sys-apps/sysvinit
sys-boot/efibootmgr
sys-libs/binutils-libs
sys-libs/glibc |
Hopes this info comes in useful for anyone else out there who also wants to switch over to clang. One day it would be nice to be able to unmerge gcc
References
1. https://wiki.gentoo.org/wiki/Clang
2. https://forums.gentoo.org/viewtopic-t-1084292-start-0.html |
|
Back to top |
|
|
elsandosgrande Tux's lil' helper
Joined: 18 May 2019 Posts: 144 Location: Sarajevo 71000, Bosnia and Herzegovina
|
Posted: Sun Oct 06, 2019 9:56 pm Post subject: |
|
|
I had this idea pop into my head a month or two ago, but I still fail to find definitive proof that Clang is overall better than GCC. So, what did you gain by doing this, other than killing some time and getting some Gentoo/Clang practice in?
I know of one other guy that switched his system over to Clang, but he is basically running a FreeBSD and Gentoo hybrid, with as many GNU components replaced by non-GNU equivalents. For some reason, he says that "GNU is not usable". |
|
Back to top |
|
|
bpoint Tux's lil' helper
Joined: 07 Oct 2008 Posts: 84 Location: Japan
|
Posted: Thu Oct 10, 2019 5:22 am Post subject: |
|
|
I'm by no means anti-GNU or anything, but clang tends to stick to the standards more than gcc does (with regards to extensions), which makes for more compatible code across both platforms and architectures. This can help eliminate bugs and is generally good for everyone.
If you're looking for a specific reason to use clang because it's "better" than gcc, I doubt you'll find one. At best, you might be able to have slightly smaller executables with clang's optimizer (especially when using -Oz), but that certainly doesn't apply for every package. |
|
Back to top |
|
|
Juippisi Developer
Joined: 30 Sep 2005 Posts: 722 Location: /home
|
Posted: Thu Oct 10, 2019 5:32 pm Post subject: |
|
|
Thanks, very informative! Also by the look of that list, state of clang is looking better than it did last time I tried this (2014 or so).
I was curious to switch my system compiler into https://developer.amd.com/amd-aocc/ aocc with my new Ryzen, but its always a major version behind so its gains are already implemented in newest clang (there's no performance difference according to Phoronix tests).
Honestly, your package.env looks so clean might try. Any idea what are the clang alternatives to LTO/graphite? -flto=thin? |
|
Back to top |
|
|
elsandosgrande Tux's lil' helper
Joined: 18 May 2019 Posts: 144 Location: Sarajevo 71000, Bosnia and Herzegovina
|
Posted: Thu Oct 10, 2019 9:04 pm Post subject: |
|
|
Since that's the case, I guess that I'll be sticking to GCC for the foreseeable future. Thanks for the info bpoint! |
|
Back to top |
|
|
bpoint Tux's lil' helper
Joined: 07 Oct 2008 Posts: 84 Location: Japan
|
Posted: Sat Oct 12, 2019 8:00 am Post subject: |
|
|
Juippisi wrote: | Honestly, your package.env looks so clean might try. Any idea what are the clang alternatives to LTO/graphite? -flto=thin? |
I've expanded on testing out LTO and am actually using -flto=thin for clang, and -flto for gcc (with gold linker) with this configuration now:
Code: | CC="clang"
CXX="clang++"
CFLAGS="-march=native -O2 -pipe -fomit-frame-pointer -ffunction-sections -fdata-sections -flto=thin"
CXXFLAGS="-stdlib=libc++ ${CFLAGS}"
LDFLAGS="-flto=thin -fuse-ld=lld -rtlib=compiler-rt -unwindlib=libunwind -Wl,--gc-sections -Wl,--as-needed -Wl,--strip-all" |
So far so good here, so I'd definitely say to give it a try! |
|
Back to top |
|
|
12101111 n00b
Joined: 01 Nov 2019 Posts: 2
|
Posted: Fri Nov 01, 2019 2:53 pm Post subject: |
|
|
I'm using clang 9 as system compiler (and musl as libc).
I found that when llvm is linked with libgcc, almost all C++ and Rust packages in the system are linked with libgcc.
Https://bugs.llvm.org/show_bug.cgi?id=43411 describes how to build llvm without relying on libgcc, but I can't reproduce it (don't known how to modify clang/llvm's ebuild) |
|
Back to top |
|
|
Perfect Gentleman Veteran
Joined: 18 May 2014 Posts: 1245
|
Posted: Fri Nov 01, 2019 3:33 pm Post subject: |
|
|
@bpoint, have you built Chromium with these flags ? |
|
Back to top |
|
|
Ginta Apprentice
Joined: 17 Feb 2006 Posts: 152 Location: Spain
|
Posted: Mon Dec 09, 2019 3:56 pm Post subject: Re: Gentoo with clang/lld/libc++/compiler-rt/libunwind |
|
|
bpoint wrote: |
Notes / Issues
For some reason, sys-devel/llvm still has a dependency on libgcc_s.so even though it's being built to use compiler-rt (which should remove the gcc dependency). During building, there are two linker errors with unresolved symbols for __register_frame and __deregister_frame, so the fix I went with was just to pass -lgcc_s to the linker, but only when building llvm.
sys-devel/gettext throws math-related linker errors (undefined symbol: sqrt and log10) when building with clang, which can be fixed by passing -lm to the linker. I found a related bug report/issue here.
|
I've been testing the same than you on parallel, then I found this thread, and managed to build LLVM without these undefined references to libgcc_s
* In order not to have to pass -lgcc_s to LLVM, you have to use llvm-libunwind instead of "normal" libunwind, that contains the missing symbols.
* Your issue with gettext is caused because somehow lto missdrops dependencies on -lm, just disable LTO to compile it
Furthermore, the flag unwindlib is mostly filtered out in many ebuilds, a fix in https://bugs.gentoo.org/702344 (or at least the idea to fix it)
For example, one of the ebuilds filtering that flag is binutils, which with my ebuild it can be now compiled with clang.
I hope this helps. _________________ Some people should use emerge brain on theirselves... |
|
Back to top |
|
|
Perfect Gentleman Veteran
Joined: 18 May 2014 Posts: 1245
|
Posted: Tue Dec 10, 2019 4:07 am Post subject: |
|
|
Code: | sys-devel/llvm-9.0.1_rc2::pg_overlay was built with the following:
USE="gold libffi ncurses -debug -doc -exegesis -libedit -test -xar -xml -z3" LLVM_TARGETS="(X86) -AArch64 -AMDGPU -ARM -BPF -Hexagon -Lanai -MSP430 -Mips -NVPTX -PowerPC -RISCV -Sparc -SystemZ -WebAssembly -XCore"
CFLAGS="-march=native -mtune=native -O2 -pipe -fomit-frame-pointer -fno-plt -fno-stack-protector -flto=thin"
CXXFLAGS="-march=native -mtune=native -O2 -pipe -fomit-frame-pointer -fno-plt -fno-stack-protector -flto=thin -stdlib=libc++"
FEATURES="ipc-sandbox parallel-fetch assume-digests unmerge-logs binpkg-logs userfetch usersandbox userpriv news config-protect-if-modified binpkg-dostrip distlocks merge-sync unmerge-orphans sfperms preserve-libs unknown-features-warn network-sandbox multilib-strict xattr sandbox fixlafiles strict binpkg-docompress usersync protect-owned ebuild-locks parallel-install pid-sandbox"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--strip-debug -flto=thin -Wl,--thinlto-jobs=7 -fuse-ld=lld -rtlib=compiler-rt -unwindlib=libunwind"
sys-devel/clang-9.0.1_rc2::pg_overlay was built with the following:
USE="default-compiler-rt default-libcxx -debug -doc -static-analyzer -test -xml" LLVM_TARGETS="(X86) -AArch64 -AMDGPU -ARM -BPF -Hexagon -Lanai -MSP430 -Mips -NVPTX -PowerPC -RISCV -Sparc -SystemZ -WebAssembly -XCore" PYTHON_SINGLE_TARGET="python3_8 -python2_7 -python3_6 -python3_7" PYTHON_TARGETS="python3_8 -python2_7 -python3_6 -python3_7"
CFLAGS="-march=native -mtune=native -O2 -pipe -fomit-frame-pointer -fno-plt -fno-stack-protector -flto=thin"
CXXFLAGS="-march=native -mtune=native -O2 -pipe -fomit-frame-pointer -fno-plt -fno-stack-protector -flto=thin -stdlib=libc++"
FEATURES="xattr unknown-features-warn news binpkg-docompress userpriv ebuild-locks parallel-install distlocks ipc-sandbox sandbox multilib-strict fixlafiles sfperms strict pid-sandbox protect-owned unmerge-logs parallel-fetch merge-sync userfetch usersync preserve-libs network-sandbox assume-digests binpkg-dostrip unmerge-orphans usersandbox config-protect-if-modified binpkg-logs"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--strip-debug -flto=thin -Wl,--thinlto-jobs=7 -fuse-ld=lld -rtlib=compiler-rt -unwindlib=libunwind |
|
|
Back to top |
|
|
Ginta Apprentice
Joined: 17 Feb 2006 Posts: 152 Location: Spain
|
Posted: Tue Dec 10, 2019 10:30 pm Post subject: |
|
|
Perfect Gentleman wrote: | Code: | sys-devel/llvm-9.0.1_rc2::pg_overlay was built with the following:
USE="gold libffi ncurses -debug -doc -exegesis -libedit -test -xar -xml -z3" LLVM_TARGETS="(X86) -AArch64 -AMDGPU -ARM -BPF -Hexagon -Lanai -MSP430 -Mips -NVPTX -PowerPC -RISCV -Sparc -SystemZ -WebAssembly -XCore"
CFLAGS="-march=native -mtune=native -O2 -pipe -fomit-frame-pointer -fno-plt -fno-stack-protector -flto=thin"
CXXFLAGS="-march=native -mtune=native -O2 -pipe -fomit-frame-pointer -fno-plt -fno-stack-protector -flto=thin -stdlib=libc++"
FEATURES="ipc-sandbox parallel-fetch assume-digests unmerge-logs binpkg-logs userfetch usersandbox userpriv news config-protect-if-modified binpkg-dostrip distlocks merge-sync unmerge-orphans sfperms preserve-libs unknown-features-warn network-sandbox multilib-strict xattr sandbox fixlafiles strict binpkg-docompress usersync protect-owned ebuild-locks parallel-install pid-sandbox"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--strip-debug -flto=thin -Wl,--thinlto-jobs=7 -fuse-ld=lld -rtlib=compiler-rt -unwindlib=libunwind"
sys-devel/clang-9.0.1_rc2::pg_overlay was built with the following:
USE="default-compiler-rt default-libcxx -debug -doc -static-analyzer -test -xml" LLVM_TARGETS="(X86) -AArch64 -AMDGPU -ARM -BPF -Hexagon -Lanai -MSP430 -Mips -NVPTX -PowerPC -RISCV -Sparc -SystemZ -WebAssembly -XCore" PYTHON_SINGLE_TARGET="python3_8 -python2_7 -python3_6 -python3_7" PYTHON_TARGETS="python3_8 -python2_7 -python3_6 -python3_7"
CFLAGS="-march=native -mtune=native -O2 -pipe -fomit-frame-pointer -fno-plt -fno-stack-protector -flto=thin"
CXXFLAGS="-march=native -mtune=native -O2 -pipe -fomit-frame-pointer -fno-plt -fno-stack-protector -flto=thin -stdlib=libc++"
FEATURES="xattr unknown-features-warn news binpkg-docompress userpriv ebuild-locks parallel-install distlocks ipc-sandbox sandbox multilib-strict fixlafiles sfperms strict pid-sandbox protect-owned unmerge-logs parallel-fetch merge-sync userfetch usersync preserve-libs network-sandbox assume-digests binpkg-dostrip unmerge-orphans usersandbox config-protect-if-modified binpkg-logs"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--strip-debug -flto=thin -Wl,--thinlto-jobs=7 -fuse-ld=lld -rtlib=compiler-rt -unwindlib=libunwind |
|
As said in previous post, default-compiler-rt still misses some defaults, like --unwindlib option being filtered out in many ebuilds, like binutils for example.
Furthermore, it needs further testing, because when tested the modification, I got lots of SIGSEGV on basic programs (tested on a chrooted stage3) that made system unusable due to this compiler-rt usage.
If someone could clarify/help the issue, it would be good to have it added to the previously mentioned bugreport. _________________ Some people should use emerge brain on theirselves... |
|
Back to top |
|
|
Perfect Gentleman Veteran
Joined: 18 May 2014 Posts: 1245
|
Posted: Wed Dec 11, 2019 3:28 am Post subject: |
|
|
add libunwind in env
Code: | ~ $ cat /etc/portage/env/llvm
CMAKE_EXTRA_CACHE_FILE="/etc/portage/env/llvm_cmake"
~ $ cat /etc/portage/env/llvm_cmake
SET (BUILD_SHARED_LIBS ON CACHE BOOL "Build all libraries as shared libraries instead of static" FORCE)
SET (LLVM_ENABLE_LIBCXX ON CACHE BOOL "Use libc++ if available." FORCE)
SET (LLVM_ENABLE_LLD ON CACHE BOOL "Use lld as C and C++ linker." FORCE)
SET (LLVM_ENABLE_LTO THIN CACHE STRING "Build LLVM with LTO. May be specified as Thin or Full to use a particular kind of LTO" FORCE)
SET (LLVM_ENABLE_UNWIND_TABLES OFF CACHE BOOL "Emit unwind tables for the libraries" FORCE)
SET (LLVM_INCLUDE_BENCHMARKS OFF CACHE BOOL "Generate benchmark targets. If OFF, benchmarks can't be built." FORCE)
SET (LLVM_INCLUDE_DOCS OFF CACHE BOOL "Generate build targets for llvm documentation." FORCE)
SET (LLVM_INCLUDE_EXAMPLES OFF CACHE BOOL "Generate build targets for the LLVM examples" FORCE)
SET (LLVM_INCLUDE_TESTS OFF CACHE BOOL "Generate build targets for the LLVM unit tests." FORCE)
SET (LLVM_INCLUDE_GO_TESTS OFF CACHE BOOL "Include the Go bindings tests in test build targets." FORCE)
SET (LLVM_OPTIMIZED_TABLEGEN ON CACHE BOOL "Force TableGen to be built with optimization" FORCE)
SET (CLANG_DEFAULT_UNWINDLIB "libunwind" CACHE STRING "Default unwind library to use (\"none\" \"libgcc\" or \"libunwind\", empty to match runtime library.)" FORCE)
SET (CMAKE_BUILD_TYPE RELEASE CACHE STRING "" FORCE)
SET (CLANG_DEFAULT_LINKER lld CACHE STRING "" FORCE)
~ $ cat /etc/portage/env/clang_compiler
CC="clang"
CXX="clang++"
COMMON_FLAGS="-march=native -mtune=native -O2 -pipe -flto=thin -fno-plt -fno-stack-protector -fno-unwind-tables -fomit-frame-pointer"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS} -stdlib=libc++"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--strip-debug -flto=thin -Wl,--thinlto-jobs=7 -fuse-ld=lld -rtlib=compiler-rt -unwindlib=libunwind"
AR="llvm-ar"
NM="llvm-nm"
RANLIB="llvm-ranlib"
~ $ cat /etc/portage/package.env/llvm
sys-devel/clang clang_compiler llvm
sys-devel/clang-common clang_compiler llvm
sys-devel/clang-runtime clang_compiler llvm
sys-devel/lld clang_compiler llvm
sys-devel/llvm clang_compiler llvm_2
sys-devel/llvm-common clang_compiler llvm
sys-devel/llvmgold clang_compiler llvm
sys-libs/compiler-rt clang_compiler llvm
sys-libs/compiler-rt-sanitizers clang_compiler llvm
sys-libs/libcxx clang_compiler llvm
sys-libs/libcxxabi clang_compiler llvm
sys-libs/libomp clang_compiler llvm
sys-libs/llvm-libunwind clang_compiler llvm |
|
|
Back to top |
|
|
NTU Apprentice
Joined: 17 Jul 2015 Posts: 187
|
Posted: Fri Feb 21, 2020 9:21 am Post subject: |
|
|
12101111 wrote: | I'm using clang 9 as system compiler (and musl as libc).
I found that when llvm is linked with libgcc, almost all C++ and Rust packages in the system are linked with libgcc.
Https://bugs.llvm.org/show_bug.cgi?id=43411 describes how to build llvm without relying on libgcc, but I can't reproduce it (don't known how to modify clang/llvm's ebuild) |
Try adding this line:
Code: | -DLIBCXXABI_USE_LLVM_UNWINDER=YES | To mycmakeargs, like so: Code: | local mycmakeargs=(
-DLIBCXXABI_USE_LLVM_UNWINDER=YES
# disable appending VCS revision to the version to improve
# direct cache hit ratio |
To this file:
https://gitweb.gentoo.org/repo/gentoo.git/tree/sys-devel/llvm/llvm-9.0.1.ebuild
Or whatever version of LLVM you're working on. Look for "multilib_src_configure" in the LLVM ebuild and drop it in there. Let me know how it goes!
Try doing the same thing for Clang, drop the -DLIBCXXABI_USE_LLVM_UNWINDER=YES line into "mycmakeargs" in the "multilib_src_configure" secton like the other ones into the ebuild.
Make sure you do this in a local overlay, like so:
https://wiki.gentoo.org/wiki/Custom_repository#Creating_a_local_repository
Best of luck!
edit: I am going to be hosting an overlay with these changes to LLVM's build thrown in:
-DLIBCXXABI_USE_COMPILER_RT=ON
-DLIBUNWIND_USE_COMPILER_RT=ON
-DLIBCXXABI_USE_LLVM_UNWINDER=ON
This should take care of all that. Will post links when finished |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|