Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Playing with LLVM, gold linker and CLANG
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
Ginta
Tux's lil' helper
Tux's lil' helper


Joined: 17 Feb 2006
Posts: 138
Location: Spain

PostPosted: Mon Jul 23, 2018 3:01 am    Post subject: Playing with LLVM, gold linker and CLANG Reply with quote

I installed a fresh stage3 on a chroot to play and test with LLVM, gold linker and CLANG, with the goal of testing to compile as much as possible with clang instead of gcc.

Once installed llvm and clang, I set the default compiler as clang in /etc/portage/make.conf as follows:

Code:

CC="clang"
CXX="clang++"
AR="llvm-ar"
NM="llvm-nm"
RANLIB="llvm-ranlib"


Of course, I also enabled gold linker by setting the following to binutils' environment:
Code:
EXTRA_ECONF="--enable-gold=default"


Then I thought, like gcc that compiles with itself... may it be possible to compile LLVM and CLANG with CLANG? Then I ran a
Code:
emerge -1 llvm
using
Code:
clang[default-compiler-rt default-libcxx static-analyzer xml]
and found some things:

a) With sys-libs/libcxxabi[libunwind] + sys-libs/libcxx[libunwind] combination makes LLVM fail compilation with undefined references.
b) With sys-libs/libcxxabi[-libunwind] + sys-libs/libcxx[-libunwind] combination LLVM compiles fine, but then clang is unusable because of:
Code:
clang: symbol lookup error: clang: undefined symbol: _ZN4llvm3sys2fs17getMainExecutableB5cxx11EPKcPv


I played with disabled as-needed, with using bfd linker instead of gold linker to link itself, and the result is the same, LLVM compiled ok, its executables seem to run properly, but clang is broken.

So I thought, ok, maybe I can rebuild clang with gcc and then rebuild with itself as a second stage? Then I ran

Code:

CC="gcc" CXX="g++" AR="ar" NM="nm" RANLIB="ranlib" LDFLAGS="${LDFLAGS} -fuse-ld=bfd" emerge -1 clang


And then it failed to compile with:
Code:

[28/1289] : && /usr/bin/g++  -march=native -O2 -pipe -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -Wdelete-non-virtual-dtor -Wno-comment -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -fno-strict-aliasing -pedantic -Wno-long-long  -fuse-ld=bfd     -Wl,-O3 -Wl,--gc-sections utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangASTNodesEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangAttrEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangCommentCommandInfoEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangCommentHTMLNamedCharacterReferenceEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangCommentHTMLTagsEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangDataCollectorsEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangDiagnosticsEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangOptionDocEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangSACheckersEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/NeonEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/TableGen.cpp.o  -o bin/clang-tblgen -L/usr/lib64/llvm/6/lib64 -Wl,-rpath,"\$ORIGIN/../lib64:/usr/lib64/llvm/6/lib64" /usr/lib/llvm/6/lib64/libLLVMSupport.so.6.0.1 /usr/lib/llvm/6/lib64/libLLVMTableGen.so.6.0.1 -lpthread && :
FAILED: bin/clang-tblgen
: && /usr/bin/g++  -march=native -O2 -pipe -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -Wdelete-non-virtual-dtor -Wno-comment -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -fno-strict-aliasing -pedantic -Wno-long-long  -fuse-ld=bfd     -Wl,-O3 -Wl,--gc-sections utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangASTNodesEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangAttrEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangCommentCommandInfoEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangCommentHTMLNamedCharacterReferenceEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangCommentHTMLTagsEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangDataCollectorsEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangDiagnosticsEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangOptionDocEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangSACheckersEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/NeonEmitter.cpp.o utils/TableGen/CMakeFiles/clang-tblgen.dir/TableGen.cpp.o  -o bin/clang-tblgen -L/usr/lib64/llvm/6/lib64 -Wl,-rpath,"\$ORIGIN/../lib64:/usr/lib64/llvm/6/lib64" /usr/lib/llvm/6/lib64/libLLVMSupport.so.6.0.1 /usr/lib/llvm/6/lib64/libLLVMTableGen.so.6.0.1 -lpthread && :
utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangAttrEmitter.cpp.o: In function `(anonymous namespace)::PragmaClangAttributeSupport::generateParsingHelpers(llvm::raw_ostream&)':
ClangAttrEmitter.cpp:(.text._ZN12_GLOBAL__N_127PragmaClangAttributeSupport22generateParsingHelpersERN4llvm11raw_ostreamE+0x997): undefined reference to `llvm::Twine::str[abi:cxx11]() const'
utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangAttrEmitter.cpp.o: In function `clang::CalculateDiagnostic(llvm::Record const&)':
ClangAttrEmitter.cpp:(.text._ZN5clangL19CalculateDiagnosticERKN4llvm6RecordE+0x7b8): undefined reference to `llvm::Twine::str[abi:cxx11]() const'
utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangDiagnosticsEmitter.cpp.o: In function `groupDiagnostics(std::vector<llvm::Record*, std::allocator<llvm::Record*> > const&, std::vector<llvm::Record*, std::allocator<llvm::Record*> > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, (anonymous namespace)::GroupInfo, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, (anonymous namespace)::GroupInfo> > >&) [clone .constprop.521]':
ClangDiagnosticsEmitter.cpp:(.text._ZL16groupDiagnosticsRKSt6vectorIPN4llvm6RecordESaIS2_EES6_RSt3mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN12_GLOBAL__N_19GroupInfoESt4lessISD_ESaISt4pairIKSD_SF_EEE.constprop.521+0xf19): undefined reference to `llvm::Twine::str[abi:cxx11]() const'
utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangDiagnosticsEmitter.cpp.o: In function `clang::EmitClangDiagsDefs(llvm::RecordKeeper&, llvm::raw_ostream&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
ClangDiagnosticsEmitter.cpp:(.text._ZN5clang18EmitClangDiagsDefsERN4llvm12RecordKeeperERNS0_11raw_ostreamERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x61): undefined reference to `llvm::StringRef::upper[abi:cxx11]() const'
utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangDiagnosticsEmitter.cpp.o: In function `clang::EmitClangDiagDocs(llvm::RecordKeeper&, llvm::raw_ostream&)':
ClangDiagnosticsEmitter.cpp:(.text._ZN5clang17EmitClangDiagDocsERN4llvm12RecordKeeperERNS0_11raw_ostreamE+0xcd6): undefined reference to `llvm::Twine::str[abi:cxx11]() const'
utils/TableGen/CMakeFiles/clang-tblgen.dir/NeonEmitter.cpp.o: In function `(anonymous namespace)::Intrinsic::DagEmitter::emitDag(llvm::DagInit*)':
NeonEmitter.cpp:(.text._ZN12_GLOBAL__N_19Intrinsic10DagEmitter7emitDagEPN4llvm7DagInitE+0xfda): undefined reference to `llvm::SetTheory::addOperator(llvm::StringRef, std::unique_ptr<llvm::SetTheory::Operator, std::default_delete<llvm::SetTheory::Operator> >)'
NeonEmitter.cpp:(.text._ZN12_GLOBAL__N_19Intrinsic10DagEmitter7emitDagEPN4llvm7DagInitE+0x101c): undefined reference to `llvm::SetTheory::addOperator(llvm::StringRef, std::unique_ptr<llvm::SetTheory::Operator, std::default_delete<llvm::SetTheory::Operator> >)'
NeonEmitter.cpp:(.text._ZN12_GLOBAL__N_19Intrinsic10DagEmitter7emitDagEPN4llvm7DagInitE+0x106a): undefined reference to `llvm::SetTheory::addOperator(llvm::StringRef, std::unique_ptr<llvm::SetTheory::Operator, std::default_delete<llvm::SetTheory::Operator> >)'
NeonEmitter.cpp:(.text._ZN12_GLOBAL__N_19Intrinsic10DagEmitter7emitDagEPN4llvm7DagInitE+0x10c3): undefined reference to `llvm::SetTheory::addExpander(llvm::StringRef, std::unique_ptr<llvm::SetTheory::Expander, std::default_delete<llvm::SetTheory::Expander> >)'
utils/TableGen/CMakeFiles/clang-tblgen.dir/NeonEmitter.cpp.o: In function `(anonymous namespace)::Intrinsic::DagEmitter::emitDagCall(llvm::DagInit*)':
NeonEmitter.cpp:(.text._ZN12_GLOBAL__N_19Intrinsic10DagEmitter11emitDagCallEPN4llvm7DagInitE+0x3a1): undefined reference to `llvm::Twine::str[abi:cxx11]() const'
utils/TableGen/CMakeFiles/clang-tblgen.dir/TableGen.cpp.o: In function `llvm::cl::opt<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, false, llvm::cl::parser<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::~opt()':
TableGen.cpp:(.text._ZN4llvm2cl3optINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEELb0ENS0_6parserIS7_EEED2Ev[_ZN4llvm2cl3optINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEELb0ENS0_6parserIS7_EEED5Ev]+0x3): undefined reference to `vtable for llvm::cl::opt<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, false, llvm::cl::parser<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >'
utils/TableGen/CMakeFiles/clang-tblgen.dir/TableGen.cpp.o: In function `__static_initialization_and_destruction_0(int, int) [clone .constprop.52]':
TableGen.cpp:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.52+0xc4e): undefined reference to `vtable for llvm::cl::OptionValue<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >'
TableGen.cpp:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.52+0xc7d): undefined reference to `vtable for llvm::cl::opt<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, false, llvm::cl::parser<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >'
TableGen.cpp:(.text.startup._Z41__static_initialization_and_destruction_0ii.constprop.52+0xca5): undefined reference to `vtable for llvm::cl::parser<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >'
utils/TableGen/CMakeFiles/clang-tblgen.dir/TableGen.cpp.o: In function `llvm::cl::opt<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, false, llvm::cl::parser<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::~opt()':
TableGen.cpp:(.text._ZN4llvm2cl3optINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEELb0ENS0_6parserIS7_EEED0Ev[_ZN4llvm2cl3optINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEELb0ENS0_6parserIS7_EEED5Ev]+0x3): undefined reference to `vtable for llvm::cl::opt<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, false, llvm::cl::parser<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >'
utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangOptionDocEmitter.cpp.o: In function `clang::docs::(anonymous namespace)::emitOptionName(llvm::StringRef, llvm::Record const*, llvm::raw_ostream&)':
ClangOptionDocEmitter.cpp:(.text._ZN5clang4docs12_GLOBAL__N_114emitOptionNameEN4llvm9StringRefEPKNS2_6RecordERNS2_11raw_ostreamE+0x266): undefined reference to `llvm::Twine::str[abi:cxx11]() const'
utils/TableGen/CMakeFiles/clang-tblgen.dir/ClangOptionDocEmitter.cpp.o: In function `clang::docs::(anonymous namespace)::emitDocumentation(int, clang::docs::(anonymous namespace)::Documentation const&, llvm::Record const*, llvm::raw_ostream&)':
ClangOptionDocEmitter.cpp:(.text._ZN5clang4docs12_GLOBAL__N_117emitDocumentationEiRKNS1_13DocumentationEPKN4llvm6RecordERNS5_11raw_ostreamE+0x1f9): undefined reference to `llvm::Twine::str[abi:cxx11]() const'
ClangOptionDocEmitter.cpp:(.text._ZN5clang4docs12_GLOBAL__N_117emitDocumentationEiRKNS1_13DocumentationEPKN4llvm6RecordERNS5_11raw_ostreamE+0x40e): undefined reference to `llvm::Twine::str[abi:cxx11]() const'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.


So yes, this is unsupported probably, and it is just a test playground for learning, but, since it is already out of my knowledge, any hint for successfully building clang and llvm with clang and its libc++, or is it simply not prepared for doing so (a thing I can't believe)?

Many thanks for the hints!
_________________
Somebody should use emerge brain on theirselves...
Back to top
View user's profile Send private message
virtguru
Tux's lil' helper
Tux's lil' helper


Joined: 14 Aug 2010
Posts: 146
Location: Eire

PostPosted: Mon Jul 23, 2018 1:09 pm    Post subject: Reply with quote

Code:
llvm::Twine::str[abi:cxx11]() const


maybe an ABI issue , so you may have to build llvm with same compiler ?
Back to top
View user's profile Send private message
Ginta
Tux's lil' helper
Tux's lil' helper


Joined: 17 Feb 2006
Posts: 138
Location: Spain

PostPosted: Mon Jul 23, 2018 4:01 pm    Post subject: Reply with quote

I followed up with the tests, indeed, it seems a problem with libcxx:

I recompiled clang with -default-libcxx and -default-compiler-rt so it uses gcc's C++ libraries, and then I could build both, LLVM and clang with clang.

So now I am curious, because I have a kind of academic question:

If clang provides a libc++ replacement for libstdc++ why it can't be used to bootstrap itself?
_________________
Somebody should use emerge brain on theirselves...
Back to top
View user's profile Send private message
virtguru
Tux's lil' helper
Tux's lil' helper


Joined: 14 Aug 2010
Posts: 146
Location: Eire

PostPosted: Tue Jul 24, 2018 10:06 pm    Post subject: Reply with quote

I'd be interested to know the answer to that as well I don't know clang / llvm that well.

Cheers
Back to top
View user's profile Send private message
Ginta
Tux's lil' helper
Tux's lil' helper


Joined: 17 Feb 2006
Posts: 138
Location: Spain

PostPosted: Wed Jul 25, 2018 9:28 pm    Post subject: Reply with quote

I've been digging, reading and thinking.

Of course it makes sense that compiling LLVM with libc++ instead of libstdc++ will cause clang not to be buildable with gcc and libstdc++, and I am suspecting this is the cause.

I made up to this thinking, because when recompiling LLVM, then clang got unusable and I had to recompile with gcc (even I was using libstdc++), so I just thought that some other way to bootstrap it would be neccesary.

And then I found up this: https://libcxx.llvm.org/docs/UsingLibcxx.html#using-libc-with-gcc

So what I am going to try next (when I have time) is to rebuild clang with default libc++ and bootstrap it with:

Code:

CFLAGS="${CFLAGS}  -nostdinc++ -I<libcxx-install-prefix>/include/c++/v1"
CXXFLAGS="${CFLAGS} -nostdinc++ -I<libcxx-install-prefix>/include/c++/v1"
LDFLAGS="${LDFLAGS}  -nodefaultlibs -lc++ -lc++abi -lm -lc -lgcc_s -lgcc"


And then having a clang bootstraped with gcc, hopefully I could rebuild clang with itself..

Will tell the results of the tests asap I can do them :)
_________________
Somebody should use emerge brain on theirselves...
Back to top
View user's profile Send private message
Dr.Willy
Guru
Guru


Joined: 15 Jul 2007
Posts: 504
Location: NRW, Germany

PostPosted: Thu Jul 26, 2018 8:46 am    Post subject: Reply with quote

I remember there being a topic about LLVM/Clang here on the forums, where people ran into similar issues.
IIRC the solution was to build llvm:5 and clang:5 using gcc and then use llvm:5 and clang:5 to build llvm:6 and clang:6 or something along those lines.
Back to top
View user's profile Send private message
Ginta
Tux's lil' helper
Tux's lil' helper


Joined: 17 Feb 2006
Posts: 138
Location: Spain

PostPosted: Thu Jul 26, 2018 10:47 am    Post subject: Reply with quote

Dr.Willy wrote:
I remember there being a topic about LLVM/Clang here on the forums, where people ran into similar issues.
IIRC the solution was to build llvm:5 and clang:5 using gcc and then use llvm:5 and clang:5 to build llvm:6 and clang:6 or something along those lines.


Thanks for the hint, I am currently trying what I said in my previous post, since this is about learning more than other thing, will keep you all updated, and if fails, will try what you say :)
_________________
Somebody should use emerge brain on theirselves...
Back to top
View user's profile Send private message
Perfect Gentleman
l33t
l33t


Joined: 18 May 2014
Posts: 908

PostPosted: Thu Jul 26, 2018 1:23 pm    Post subject: Reply with quote

i've to build llvm/clang with clang when i'd build llvm/clang statically. but the downfall is no possibility to build mesa with use=llvm
Back to top
View user's profile Send private message
Ginta
Tux's lil' helper
Tux's lil' helper


Joined: 17 Feb 2006
Posts: 138
Location: Spain

PostPosted: Mon Aug 06, 2018 1:38 pm    Post subject: Reply with quote

Perfect Gentleman wrote:
i've to build llvm/clang with clang when i'd build llvm/clang statically. but the downfall is no possibility to build mesa with use=llvm


There is no need to build them statically, but they need to be correctly bootstrapped.

I finally made them work and compile clang with clang :)

Following these steps:


  • Compile LLVM and CLANG with gcc like normal, paying attention to enable default-compiler-rt default-libcxx USE flags
  • Backup packages in case of fire
    Code:

    quickpkg llvm clang sys-libs/libomp sys-libs/libunwind sys-libs/libcxxabi sys-libs/netbsd-csu sys-libs/libcxx sys-libs/compiler-rt sys-devel/clang-common sys-libs/compiler-rt-sanitizers sys-devel/clang-runtime # in case we need it urgently

  • Prepare make.conf to make use of CLANG as system's compiler
    Code:

    CC="clang"
    CXX="clang++"
    AR="llvm-ar"
    NM="llvm-nm"
    RANLIB="llvm-ranlib"
    FCFLAGS="-march=native -O2 -pipe"
    CFLAGS_BASE="-march=native -O2 -pipe"
    CFLAGS_LTO="-flto=thin"
    CFLAGS="${CFLAGS_BASE} ${CFLAGS_LTO}"
    CXXFLAGS="${CFLAGS}"
    CHOST="x86_64-pc-linux-gnu"
    LDFLAGS_BASE="${LDFLAGS} -march=native -Wl,--sort-common -Wl,-O3"
    LDFLAGS_STRIP="-Wl,-S"
    LDFLAGS_BFD="-fuse-ld=bfd"
    LDFLAGS_GOLD="-fuse-ld=gold"
    LDFLAGS="${LDFLAGS_BASE} ${LDFLAGS_GOLD} ${LDFLAGS_STRIP}"

  • Compile LLVM with CLANG
    Code:
    emerge -1 llvm clang

    And ignore the error, because once LLVM is installed being compiled with CLang, then Clang will be broken, we'll fix this in the next step.
    If LLVM fails, recompile clang's libraries without libunwind support (it will use gcc_s then, but it causes problems with some packages) and retry LLVM compilation
  • Recompile Clang again with GCC, but using libc++ instead of libstdc++ from glibc:
    Code:
    CC="gcc" CXX="g++" CFLAGS="-march=native -O2 -pipe -nostdinc++ -I/usr/include/c++/v1" CXXFLAGS="-march=native -O2 -pipe -nostdinc++ -I/usr/include/c++/v1" LDFLAGS="-lc++ -lc++abi -lm -lc" emerge -1 clang

    This will compile a working version of clang using the newly compiled llvm with libc++ support.
    If it succedes, we can now compile clang with itself as normal:
    Code:
    emerge -1 clang



Be aware that there are some packages failing to compile with clang (I made some patches/reports about some of them), I am still trying to recompile the whole world with clang

I will post an entry in my blog soon with a better redaction, but I hope this can help somebody while I do so.

Quote:
Please notice that this bootstrap process is needed whenever a new major version of LLVM and Clang is released

_________________
Somebody should use emerge brain on theirselves...
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum