Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
LLVM targets
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
axl
Veteran
Veteran


Joined: 11 Oct 2002
Posts: 1075
Location: Romania

PostPosted: Wed Feb 06, 2019 5:41 pm    Post subject: LLVM targets Reply with quote

hi folks. I was trying to compile some cross platform code using llvm and clang.

I'm starting slow. with a helloworld.c program.

the sys-devel/llvm package has a LLVM_TARGETS flag which mentions ARM and aarch64 and powepc. clang has one.

however compiler-rt doesn't. As a result it only builds libclang_rt.builtins-i386.a and libclang_rt.builtins-x86_64.a.

as a result when you try:

clang -target aarch64-unknown-linux-gnu -arch aarch64-unknown-linux-gnu test.c -o test -I/usr/aarch64-unknown-linux-gnu/usr/ -L/usr/aarch64-unknown-linux-gnu/usr/lib64 -fuse-ld=lld -v

you get something like:
Code:
clang version 7.0.1 (tags/RELEASE_701/final)
Target: aarch64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm/7/bin
Selected GCC installation: /usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0
Candidate multilib: .;@m64
Selected multilib: .;@m64
 "/usr/lib64/llvm/7/bin/clang-7" -cc1 -triple aarch64-unknown-linux-gnu -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name test.c -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -fuse-init-array -target-cpu cyclone -target-feature +fp-armv8 -target-feature +neon -target-feature +crypto -target-feature +zcm -target-feature +zcz -target-abi aapcs -fallow-half-arguments-and-returns -dwarf-column-info -debugger-tuning=gdb -v -resource-dir /usr/lib64/llvm/7/bin/../../../../lib/clang/7.0.1 -I /usr/aarch64-unknown-linux-gnu/usr/ -internal-isystem /usr/local/include -internal-isystem /usr/lib64/llvm/7/bin/../../../../lib/clang/7.0.1/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdebug-compilation-dir /root -ferror-limit 19 -fmessage-length 238 -fno-signed-char -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/test-198016.o -x c test.c -faddrsig
clang -cc1 version 7.0.1 based upon LLVM 7.0.1 default target x86_64-pc-linux-gnu
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/aarch64-unknown-linux-gnu/usr
 /usr/lib64/llvm/7/bin/../../../../lib/clang/7.0.1/include
 /usr/include
End of search list.
 "/usr/bin/ld.lld" --eh-frame-hdr -m aarch64linux -dynamic-linker /lib/ld-linux-aarch64.so.1 -o test /usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0/../../../../lib64/crt1.o /usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0/../../../../lib64/crti.o /usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0/crtbegin.o -L/usr/aarch64-unknown-linux-gnu/usr/lib64 -L/usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0 -L/usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0/../../../../aarch64-unknown-linux-gnu/lib/../lib64 -L/usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0/../../../../lib64 -L/usr/lib64/llvm/7/bin/../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0/../../../../aarch64-unknown-linux-gnu/lib -L/usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0/../../.. -L/lib -L/usr/lib /tmp/test-198016.o /usr/lib64/llvm/7/bin/../../../../lib/clang/7.0.1/lib/linux/libclang_rt.builtins-aarch64.a -lc /usr/lib64/llvm/7/bin/../../../../lib/clang/7.0.1/lib/linux/libclang_rt.builtins-aarch64.a /usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0/crtend.o /usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0/../../../../lib64/crtn.o
ld.lld: error: cannot open /usr/lib64/llvm/7/bin/../../../../lib/clang/7.0.1/lib/linux/libclang_rt.builtins-aarch64.a: No such file or directory
ld.lld: error: cannot open /usr/lib64/llvm/7/bin/../../../../lib/clang/7.0.1/lib/linux/libclang_rt.builtins-aarch64.a: No such file or directory
clang-7: error: linker command failed with exit code 1 (use -v to see invocation)


I'm tired, and I feel like I've been running in circles for a few hours. If llvm and clang have a LLVM_TARGET which allows cross compile, shouldn't rt-compiler have one too? what am I missing?
Back to top
View user's profile Send private message
axl
Veteran
Veteran


Joined: 11 Oct 2002
Posts: 1075
Location: Romania

PostPosted: Fri Feb 08, 2019 12:41 am    Post subject: Reply with quote

I suck at writing. I started looking into llvm and clang for all the wrong reasons. I was trying to compile some stuff for my jailbroken ipad. Once you look into this, you find out that currently it is easier to compile stuff for apple using clang and llvm than gcc. Another bad reason was that I was curious if clang can compile kernels. Cross-compile code. Compile code through distcc. And last but not least, if I could speed up compiling at least some stuff, using a GPU instead of a cpu. These were my targets going into this thing.

I already had a toe in. I have multiple crossdev toolchains installed on my machine. the arm64, armv6, armv7, powerpc, powerpc64. once you install with crossdev the initial packages, you can at least easily cross compile kernels. easily. even using genkernel. as long as you set everything right, you can do it. and even with distcc on top. which is absolutely great. I think, if you look at what crossdev does... it installs a binutils aimed at manipulating arm binaries, but it runs on x86, just as gcc (runs on x86 - but makes stuff for arm) and finally glibc. that is installed in /usr/$CHOST/lib and is arm code.

At this point, all binutils (the original binutils for the host system, the cross-arm-binutils, the cross-armv7a, the cross-aarch64 and so on) were compiled with multitarget useflag, and llvm / clang with gold, and also compiled lld. on the host. if you look at what I posted yesterday, at this point, if you try to use clang to build code for another platform, it will complain of 2 things. 1 is that it can't find libclang_rt.builtins-aarch64.a. or armhf.a. You can just ignore that. In fact this is one of the issues I have to solve tomorrow. Can't figure out what is pulling that and how to disable it. Because it will work without it.

The second thing it will complain is the fact that which ever files I tried to pull for crtbegin.o crtend.o (I think this is how they are called, they are gcc files for sure and are usually in /usr/$CHOST/lib/gcc/$VER/) and I tried them all... clang would complain that it can't mix arm code with x86 code and those files are not good. I was pointing out to clang using -L paramenter. and he didn't like any of them, because all of them even those compiled by crossdev are just x86 files. so I would need their version in arm. so after cross-fix-root and adding distcc to features in make.conf CHOST=$CHOST cross-emerge gcc. now we have the arm versions or the gcc crt files, right there in where clangs expects them to be to link properly. So now when I run:
Code:
clang --target=aarch64-unknown-linux-gnu --sysroot=/usr/aarch64-unknown-linux-gnu/ -march=armv8 -O2 -pipe  -fuse-ld=lld test.c -o test -v
I get

Code:
clang version 7.0.1 (tags/RELEASE_701/final)
Target: aarch64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm/7/bin
Selected GCC installation: /usr/aarch64-unknown-linux-gnu//usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0
Candidate multilib: .;@m64
Selected multilib: .;@m64
 "/usr/lib64/llvm/7/bin/clang-7" -cc1 -triple aarch64-unknown-linux-gnu -emit-obj -disable-free -disable-llvm-verifier -discard-value-names -main-file-name test.c -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -fuse-init-array -target-cpu generic -target-feature +neon -target-abi aapcs -fallow-half-arguments-and-returns -dwarf-column-info -debugger-tuning=gdb -v -resource-dir /usr/lib64/llvm/7/bin/../../../../lib/clang/7.0.1 -isysroot /usr/aarch64-unknown-linux-gnu/ -internal-isystem /usr/aarch64-unknown-linux-gnu//usr/local/include -internal-isystem /usr/lib64/llvm/7/bin/../../../../lib/clang/7.0.1/include -internal-externc-isystem /usr/aarch64-unknown-linux-gnu//include -internal-externc-isystem /usr/aarch64-unknown-linux-gnu//usr/include -O2 -fdebug-compilation-dir /root -ferror-limit 19 -fmessage-length 238 -fno-signed-char -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -o /tmp/test-20f104.o -x c test.c -faddrsig
clang -cc1 version 7.0.1 based upon LLVM 7.0.1 default target x86_64-pc-linux-gnu
ignoring nonexistent directory "/usr/aarch64-unknown-linux-gnu//usr/local/include"
ignoring nonexistent directory "/usr/aarch64-unknown-linux-gnu//include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib64/llvm/7/bin/../../../../lib/clang/7.0.1/include
 /usr/aarch64-unknown-linux-gnu//usr/include
End of search list.
 "/usr/bin/ld.lld" --sysroot=/usr/aarch64-unknown-linux-gnu/ --eh-frame-hdr -m aarch64linux -dynamic-linker /lib/ld-linux-aarch64.so.1 -o test /usr/aarch64-unknown-linux-gnu//usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0/../../../../lib64/crt1.o /usr/aarch64-unknown-linux-gnu//usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0/../../../../lib64/crti.o /usr/aarch64-unknown-linux-gnu//usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0/crtbegin.o -L/usr/aarch64-unknown-linux-gnu//usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0 -L/usr/aarch64-unknown-linux-gnu//usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0/../../../../lib64 -L/usr/aarch64-unknown-linux-gnu//lib/../lib64 -L/usr/aarch64-unknown-linux-gnu//usr/lib/../lib64 -L/usr/aarch64-unknown-linux-gnu//usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0/../../.. -L/usr/aarch64-unknown-linux-gnu//lib -L/usr/aarch64-unknown-linux-gnu//usr/lib /tmp/test-20f104.o /usr/lib64/llvm/7/bin/../../../../lib/clang/7.0.1/lib/linux/libclang_rt.builtins-aarch64.a -lc /usr/lib64/llvm/7/bin/../../../../lib/clang/7.0.1/lib/linux/libclang_rt.builtins-aarch64.a /usr/aarch64-unknown-linux-gnu//usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0/crtend.o /usr/aarch64-unknown-linux-gnu//usr/lib/gcc/aarch64-unknown-linux-gnu/8.2.0/../../../../lib64/crtn.o
Fri Feb 08 - 02:23:38
[root@magdalina:~]# file test
test: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, not stripped


the file will run fine on rpi3b+. So getting back to where I was yesterday. Didn't use the sysroot parameter, and didn't have gcc compiled in arm mode with cross-emerge installed in /usr/$CHOST. That was one problem.

The second problem is the files I was trying to build in the first place. Not sure why they are needed, how they are pulled into commands. Let's start yesterday when I didn't have the file. I'ma just copy paste from the console.

Code:
[root@magdalina:~]# mv /usr/lib64/clang/7.0.1/lib/linux/libclang_rt.builtins-aarch64.a /
Fri Feb 08 - 02:32:37
[root@magdalina:~]# /usr/lib/llvm/7/bin/clang --target=aarch64-unknown-linux-gnu --sysroot=/usr/aarch64-unknown-linux-gnu/ -march=armv8 -O2 -pipe  -fuse-ld=lld test.c -o test
ld.lld: error: cannot open /usr/lib64/llvm/7/bin/../../../../lib/clang/7.0.1/lib/linux/libclang_rt.builtins-aarch64.a: No such file or directory
ld.lld: error: cannot open /usr/lib64/llvm/7/bin/../../../../lib/clang/7.0.1/lib/linux/libclang_rt.builtins-aarch64.a: No such file or directory
clang-7: error: linker command failed with exit code 1 (use -v to see invocation)
Fri Feb 08 - 02:32:40
[root@magdalina:~]# touch /usr/lib64/clang/7.0.1/lib/linux/libclang_rt.builtins-aarch64.a
Fri Feb 08 - 02:32:47
[root@magdalina:~]# /usr/lib/llvm/7/bin/clang --target=aarch64-unknown-linux-gnu --sysroot=/usr/aarch64-unknown-linux-gnu/ -march=armv8 -O2 -pipe  -fuse-ld=lld test.c -o test
Fri Feb 08 - 02:32:50
[root@magdalina:~]# /usr/lib/llvm/7/bin/clang --target=aarch64-unknown-linux-gnu --sysroot=/usr/aarch64-unknown-linux-gnu/ -march=armv8 -O2 -pipe  -fuse-ld=gold test.c -o test
/usr/bin/aarch64-unknown-linux-gnu-ld.gold: error: /usr/lib64/llvm/7/bin/../../../../lib/clang/7.0.1/lib/linux/libclang_rt.builtins-aarch64.a: file is empty
/usr/bin/aarch64-unknown-linux-gnu-ld.gold: error: /usr/lib64/llvm/7/bin/../../../../lib/clang/7.0.1/lib/linux/libclang_rt.builtins-aarch64.a: file is empty
clang-7: error: linker command failed with exit code 1 (use -v to see invocation)
Fri Feb 08 - 02:32:56
[root@magdalina:~]# mv /libclang_rt.builtins-aarch64.a /usr/lib64/clang/7.0.1/lib/linux/
Fri Feb 08 - 02:33:19
[root@magdalina:~]# /usr/lib/llvm/7/bin/clang --target=aarch64-unknown-linux-gnu --sysroot=/usr/aarch64-unknown-linux-gnu/ -march=armv8 -O2 -pipe  -fuse-ld=gold test.c -o test


Again, I don't exactly understand why those files are needed. I suspect it has a lot to do with the other parameter I played around with and that is gcc-toolchain. Those missing builtin files I got to build in 2 ways. 3 if you include touch. First way was by compiling it on the pi itself. And it works just as expected. The other way was to compile by hand compiler-rt. first ebuild compiler-rt prepare, then
Code:
cd /var/tmp/portage/sys-libs/compiler-rt-7.0.1/work/compiler-rt-7.0.1.src/
mkdir build
cd build
cmake -G "Unix Makefiles" \
   -DCOMPILER_RT_BUILD_BUILTINS=ON \
   -DCOMPILER_RT_BUILD_SANITIZERS=OFF \
   -DCOMPILER_RT_BUILD_XRAY=OFF \
   -DCOMPILER_RT_BUILD_LIBFUZZER=OFF \
   -DCOMPILER_RT_BUILD_PROFILE=OFF \
   -DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON \
   -DLLVM_CONFIG_PATH=`which llvm-config` \
   -DCMAKE_C_COMPILER=`which clang` \
   -DCMAKE_AR=`which llvm-ar` \
   -DCMAKE_NM=`which llvm-nm` \
   -DCMAKE_RANLIB=`which llvm-ranlib` \
   -DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=gold" \
   -DCMAKE_C_COMPILER_TARGET="armv7a-linux-gnueabihf" \
   -DCMAKE_ASM_COMPILER_TARGET="armv7a-linux-gnueabihf" \
   -DCMAKE_C_FLAGS="--target=armv7a-unknown-linux-gnueabihf --gcc-toolchain=/usr/armv7a-unknown-linux-gnueabihf/usr --sysroot=/usr/armv7a-unknown-linux-gnueabihf -march=armv7a -O2 -pipe" \
   -DCMAKE_CXX_FLAGS="--target=armv7a-unknown-linux-gnueabihf --gcc-toolchain=/usr/armv7a-unknown-linux-gnueabihf/usr --sysroot=/usr/armv7a-unknown-linux-gnueabihf -march=armv7a -O2 -pipe" \
   -DCMAKE_ASM_FLAGS="--target=armv7a-unknown-linux-gnueabihf --gcc-toolchain=/usr/armv7a-unknown-linux-gnueabihf/usr --sysroot=/usr/armv7a-unknown-linux-gnueabihf -march=armv7a -O2 -pipe" \
   -DCMAKE_C_COMPILER_EXTERNAL_TOOLCHAIN=/usr/armv7a-unknown-linux-gnueabihf/usr \
   -DCMAKE_SYSROOT=/usr/armv7a-unknown-linux-gnueabihf \
   ../
make -j4
make install
mv /usr/local/lib/linux/libclang_rt.builtins-armhf.a /usr/lib/clang/7.0.1/lib/linux/libclang_rt.builtins-armhf.a


Again, 2 questions at this point.

1. can clang compile arm code on x86 without all that crossdev stuff. just by itself. I'm pretty sure I'm doing something wrong.
2. and I suspect it has something to do with question 1, why is that builtins file being pulled in if it's not actually necessary?

PS. I'm sorry. I know I suck at writing. I did my best.
Back to top
View user's profile Send private message
nikolis
Apprentice
Apprentice


Joined: 21 Aug 2003
Posts: 153
Location: Athens

PostPosted: Wed Nov 18, 2020 12:32 pm    Post subject: Reply with quote

Question with the same title

Code:
~ $ equery u llvm
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for sys-devel/llvm-10.0.1:
 U I
 - - abi_x86_32               : 32-bit (x86) libraries
 - - debug                    : Enable extra debug codepaths, like asserts and extra output. If you want
                                to get meaningful backtraces see
                                https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces
 - - doc                      : Build and install the HTML documentation and regenerate the man pages
 - - exegesis                 : Enable performance counter support for llvm-exegesis tool that can be
                                used to measure host machine instruction characteristics
 - - gold                     : Build the gold linker plugin
 - - libedit                  : Use the libedit library (replacement for readline)
 + + libffi                   : Enable support for Foreign Function Interface library
 - - llvm_targets_AArch64     : AArch64 CPU target (arm64 in Gentoo)
 + + llvm_targets_AMDGPU      : AMDGPU target (supports R600 and GCN GPUs)
 - - llvm_targets_ARC         : ARC (Argonaut RISC Core) embedded CPU target [EXPERIMENTAL]
 - - llvm_targets_ARM         : ARM CPU target
 - - llvm_targets_AVR         : 8-bit Atmel AVR microcontroller target [EXPERIMENTAL]
 + + llvm_targets_BPF         : Berkeley Packet Filter target
 - - llvm_targets_Hexagon     : Qualcomm Hexagon DSP target
 - - llvm_targets_Lanai       : Lanai CPU target
 - - llvm_targets_MSP430      : MSP430 CPU target (experimental)
 - - llvm_targets_Mips        : MIPS CPU target (includes MIPS64)
 + + llvm_targets_NVPTX       : NVIDIA PTX (GPU) target (32-bit and 64-bit)
 - - llvm_targets_PowerPC     : PowerPC CPU target (PPC32 and PPC64)
 - - llvm_targets_RISCV       : RISC-V CPU target
 - - llvm_targets_Sparc       : Sparc CPU target
 - - llvm_targets_SystemZ     : SystemZ (s390x) CPU target
 - - llvm_targets_WebAssembly : WebAssembly backend
 - - llvm_targets_XCore       : XCore CPU target
 + + ncurses                  : Support querying terminal properties using ncurses' terminfo
 - - test                     : Enable dependencies and/or preparations necessary to run tests (usually
                                controlled by FEATURES=test but can be toggled independently)
 - - xar                      : Support dumping LLVM bitcode sections in Mach-O files (uses
                                app-arch/xar)
 + + xml                      : Add support for XML files
 - - z3                       : Enable support for sci-mathematics/z3 constraint solver
~ $

How do i turn it off ?
Code:
llvm_targets_NVPTX,
llvm_targets_AMDGPU
Back to top
View user's profile Send private message
The Main Man
Veteran
Veteran


Joined: 27 Nov 2014
Posts: 1061
Location: /run/user/1000

PostPosted: Wed Nov 18, 2020 12:37 pm    Post subject: Reply with quote

In package.use
Code:
sys-devel/llvm -llvm_targets_NVPTX
sys-devel/clang -llvm_targets_NVPTX


I'm not sure if clang one is needed but why not
Do the same with other targets you want to add or remove
Back to top
View user's profile Send private message
Ionen
Developer
Developer


Joined: 06 Dec 2018
Posts: 2073

PostPosted: Wed Nov 18, 2020 12:42 pm    Post subject: Reply with quote

It can be used by rust as well, I feel it makes more sense to just set it globally, like:
Code:
*/* LLVM_TARGETS: -NVPTX -AMDGPU
With that syntax it's even possible to have "LLVM_TARGETS: -*" if need be and then enable only what you need (on a amd64 profile X86 is forced so it stays either way).

Note that these gpu targets can be needed by the likes of mesa or ffmpeg depending on USE, but portage will complain if that's a problem, not that either can be relevant if have neither a nvidia nor amd gpu.


Last edited by Ionen on Sun Nov 29, 2020 10:51 am; edited 1 time in total
Back to top
View user's profile Send private message
nikolis
Apprentice
Apprentice


Joined: 21 Aug 2003
Posts: 153
Location: Athens

PostPosted: Wed Nov 18, 2020 1:36 pm    Post subject: Reply with quote

I turned off the targets, temporarily do not need them because i run gentoo in virtualbox.
Thank you.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum