View previous topic :: View next topic |
Author |
Message |
smartding Tux's lil' helper
Joined: 22 Jan 2021 Posts: 129
|
Posted: Mon Mar 07, 2022 3:01 pm Post subject: [solved] LLVM_TARGETS not respected? |
|
|
I have LLVM_TARGETS="X86 BPF" in my /etc/portage/make.conf
Today, after upgrading to llvm 13.0.1, running llc --version shows:
Code: |
llc --version
LLVM (http://llvm.org/):
LLVM version 13.0.1
Optimized build.
Default target: x86_64-pc-linux-gnu
Host CPU: tigerlake
Registered Targets:
aarch64 - AArch64 (little endian)
aarch64_32 - AArch64 (little endian ILP32)
aarch64_be - AArch64 (big endian)
amdgcn - AMD GCN GPUs
arm - ARM
arm64 - ARM64 (little endian)
arm64_32 - ARM64 (little endian ILP32)
armeb - ARM (big endian)
avr - Atmel AVR Microcontroller
bpf - BPF (host endian)
bpfeb - BPF (big endian)
bpfel - BPF (little endian)
hexagon - Hexagon
lanai - Lanai
mips - MIPS (32-bit big endian)
mips64 - MIPS (64-bit big endian)
mips64el - MIPS (64-bit little endian)
mipsel - MIPS (32-bit little endian)
msp430 - MSP430 [experimental]
nvptx - NVIDIA PTX 32-bit
nvptx64 - NVIDIA PTX 64-bit
ppc32 - PowerPC 32
ppc32le - PowerPC 32 LE
ppc64 - PowerPC 64
ppc64le - PowerPC 64 LE
r600 - AMD GPUs HD2XXX-HD6XXX
riscv32 - 32-bit RISC-V
riscv64 - 64-bit RISC-V
sparc - Sparc
sparcel - Sparc LE
sparcv9 - Sparc V9
systemz - SystemZ
thumb - Thumb
thumbeb - Thumb (big endian)
wasm32 - WebAssembly 32-bit
wasm64 - WebAssembly 64-bit
x86 - 32-bit X86: Pentium-Pro and above
x86-64 - 64-bit X86: EM64T and AMD64
xcore - XCore
|
Ilvm now supports all those targets and my global LLVM_TARGETS setting is not respected.
Also, running equery u llvm now shows:
Code: |
equery u llvm
[ Legend : U - final flag setting for installation]
[ : I - package is installed with flag ]
[ Colors : set, unset ]
* Found these USE flags for sys-devel/llvm-13.0.1:
U I
- - abi_x86_32 : 32-bit (x86) libraries
+ + binutils-plugin : Build the binutils plugin
- - debug : Enable extra debug codepaths, like asserts and extra output. If you want to get meaningful backtraces see https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces
- - doc : Build and install the HTML documentation and regenerate the man pages
- - exegesis : Enable performance counter support for llvm-exegesis tool that can be used to measure host machine instruction characteristics
- - libedit : Use the libedit library (replacement for readline)
+ + libffi : Enable support for Foreign Function Interface library
+ + ncurses : Support querying terminal properties using ncurses' terminfo
- - test : Enable dependencies and/or preparations necessary to run tests (usually controlled by FEATURES=test but can be toggled independently)
- - xar : Support dumping LLVM bitcode sections in Mach-O files (uses app-arch/xar)
- - xml : Add support for XML files
- - z3 : Enable support for sci-mathematics/z3 constraint solver
|
I think I used to have flags such as llvm_targets_AArch64, llvm_targets_ARM, they are no longer there.
[Moderator edit: changed [quote] tags to [code] tags to preserve output layout. -Hu]
Last edited by smartding on Tue Mar 08, 2022 2:50 am; edited 1 time in total |
|
Back to top |
|
|
Ionen Developer
Joined: 06 Dec 2018 Posts: 2874
|
Posted: Mon Mar 07, 2022 3:59 pm Post subject: |
|
|
Given disabling targets breaks ABI, they're now forced by default as a precaution (i.e. could break clang/rust/lld and -U/-N to force rebuild on LLVM_TARGETS changes doesn't always work out so well). Packages like zig also need all targets enabled and don't support these being arbitrarily disabled.
If you /really/ want to disable them, you'll need to unforce it through /etc/portage/profile/package.use.force by adding e.g. */* -llvm_targets_AArch64 and so on at your own risks (will at least need to ensure clang was rebuilt after llvm -- if before it'll be broken, and manually rebuild lld -- if using system-llvm on rust it'll likely be broken too if rebuilt before, any other package using llvm libraries may also have problems even if they don't use LLVM_TARGETS at all). |
|
Back to top |
|
|
Hu Administrator
Joined: 06 Mar 2007 Posts: 22778
|
Posted: Mon Mar 07, 2022 4:13 pm Post subject: |
|
|
Ionen addressed the immediate problem. As a general tip, when reporting that flags are not what you want, you should show the Portage output confirming that. In this case, emerge --verbose --info sys-devel/llvm would have been nice to have. |
|
Back to top |
|
|
sdauth l33t
Joined: 19 Sep 2018 Posts: 659 Location: Ásgarðr
|
Posted: Mon Mar 07, 2022 4:28 pm Post subject: |
|
|
Thanks Ionen for the explanation. I was having the same issue as OP. It would be nice to add a note to eselect news to warn users about it. |
|
Back to top |
|
|
figueroa Advocate
Joined: 14 Aug 2005 Posts: 3007 Location: Edge of marsh USA
|
Posted: Mon Mar 07, 2022 6:09 pm Post subject: |
|
|
+1 -- I came here this morning specifically looking for this thread. _________________ Andy Figueroa
hp pavilion hpe h8-1260t/2AB5; spinning rust x3
i7-2600 @ 3.40GHz; 16 gb; Radeon HD 7570
amd64/23.0/split-usr/desktop (stable), OpenRC, -systemd -pulseaudio -uefi |
|
Back to top |
|
|
dmpogo Advocate
Joined: 02 Sep 2004 Posts: 3446 Location: Canada
|
Posted: Mon Mar 07, 2022 6:57 pm Post subject: |
|
|
Same as other people, a bit froze seeing all the targets in the today's update |
|
Back to top |
|
|
Ionen Developer
Joined: 06 Dec 2018 Posts: 2874
|
Posted: Mon Mar 07, 2022 7:15 pm Post subject: |
|
|
dmpogo wrote: | Same as other people, a bit froze seeing all the targets in the today's update |
The forcing technically been done early last November, but was set to be only active starting >=llvm-13.0.1_rc, formerly unkeyworded, then non-rc got keyworded, and now llvm-13.0.1 was stabilized today so it's what you're newly seeing. |
|
Back to top |
|
|
dmpogo Advocate
Joined: 02 Sep 2004 Posts: 3446 Location: Canada
|
Posted: Mon Mar 07, 2022 7:30 pm Post subject: |
|
|
Ionen wrote: | dmpogo wrote: | Same as other people, a bit froze seeing all the targets in the today's update |
The forcing technically been done early last November, but was set to be only active starting >=llvm-13.0.1_rc, formerly unkeyworded, then non-rc got keyworded, and now llvm-13.0.1 was stabilized today so it's what you're newly seeing. |
I understand, now that prompts me once again to check whether I can get rid of llvm on my laptop. Only mesa needs it, but I am with intel driver, and though use gallium, do not use opencl. So should check if I can disable llvm, and what will be the outcome. |
|
Back to top |
|
|
smartding Tux's lil' helper
Joined: 22 Jan 2021 Posts: 129
|
Posted: Tue Mar 08, 2022 2:49 am Post subject: |
|
|
Mystery solved.
It would be better if there's some info about this change at the end of emerge. |
|
Back to top |
|
|
Hu Administrator
Joined: 06 Mar 2007 Posts: 22778
|
Posted: Tue Mar 08, 2022 3:59 pm Post subject: |
|
|
I would think that most people who notice this at all would notice it before building LLVM, so a notice logged at the end of the llvm emerge would be too late. As I hinted above, the output of emerge --pretend --verbose should show the flags as forced. |
|
Back to top |
|
|
nagora n00b
Joined: 08 Jul 2014 Posts: 11
|
Posted: Wed Mar 09, 2022 9:27 pm Post subject: |
|
|
Yeah, mystery solved but what a terrible solution! And a huge waste of electricity and processing time compiling targets that I will never use or need.
So, does LLVM_TARGETS just do nothing now? |
|
Back to top |
|
|
Hu Administrator
Joined: 06 Mar 2007 Posts: 22778
|
Posted: Wed Mar 09, 2022 10:15 pm Post subject: |
|
|
As Ionen described, this was an unfortunate necessity due to other packages breaking in response to custom LLVM_TARGETS values. If those breaks were fixed, the flag could probably be made tunable again. |
|
Back to top |
|
|
sam_ Developer
Joined: 14 Aug 2020 Posts: 2008
|
Posted: Thu Mar 10, 2022 3:04 am Post subject: |
|
|
nagora wrote: | Yeah, mystery solved but what a terrible solution! And a huge waste of electricity and processing time compiling targets that I will never use or need.
So, does LLVM_TARGETS just do nothing now? |
They also don't do nothing. You can unforce the targets if you wish in /etc/portage. See bug 767700.
As for news, maybe, although there's a tendency to prefer them for actionable things. In most cases here, no action is actually suggested. But I think it probably would've been worth it on balance because of confusion if nothing else. We still could but someone would need to write it up. |
|
Back to top |
|
|
figueroa Advocate
Joined: 14 Aug 2005 Posts: 3007 Location: Edge of marsh USA
|
Posted: Thu Mar 10, 2022 4:15 am Post subject: |
|
|
Sam, how about: "To prevent breakage of other core packages, a change has been made to now stable sys-devel/llvm- 13.0.1 to force ON all LLVM_TARGETS. Users' custom LLVM_TARGETS are ignored. No action is required on the part of the user. For further information, users may refer to the discussion in the following forum thread:
https://forums.gentoo.org/viewtopic.php?p=8694331 _________________ Andy Figueroa
hp pavilion hpe h8-1260t/2AB5; spinning rust x3
i7-2600 @ 3.40GHz; 16 gb; Radeon HD 7570
amd64/23.0/split-usr/desktop (stable), OpenRC, -systemd -pulseaudio -uefi
Last edited by figueroa on Wed Jul 27, 2022 3:52 pm; edited 1 time in total |
|
Back to top |
|
|
xavier10 Guru
Joined: 19 Jan 2004 Posts: 485 Location: Paris, France
|
Posted: Wed Jul 27, 2022 9:57 am Post subject: |
|
|
Useful discussion, I was looking just for this information.
I was looking for ideas to cut down llvm/clang compilation time and initially thought reducing the list of targets might be a good idea, but I see it is not; is there any other way to reduce the emerge time that I could pursue ? |
|
Back to top |
|
|
rogge Tux's lil' helper
Joined: 13 Oct 2006 Posts: 135 Location: Erfurt
|
Posted: Sat Jul 30, 2022 2:09 pm Post subject: |
|
|
Thanks a lot for explanation! |
|
Back to top |
|
|
njsg Tux's lil' helper
Joined: 17 Dec 2005 Posts: 88
|
Posted: Mon Jul 24, 2023 1:28 pm Post subject: |
|
|
Is rust-bin also affected by this ABI incompatibility? |
|
Back to top |
|
|
RIA77 Guru
Joined: 24 Feb 2016 Posts: 391
|
Posted: Tue Aug 15, 2023 10:17 am Post subject: |
|
|
Why there is no LLM targets respected ? I have still the same problem. |
|
Back to top |
|
|
Hu Administrator
Joined: 06 Mar 2007 Posts: 22778
|
Posted: Tue Aug 15, 2023 12:25 pm Post subject: |
|
|
RIA77 wrote: | Why there is no LLM targets respected ? I have still the same problem. | Your question is answered by multiple people just a few posts up. In short, because respecting LLVM_TARGETS caused other packages to exhibit bad failure modes, so the targets are now forced enabled to avoid that. As I speculated last year when this was asked, that force might be removed if the downstream packages were changed to better handle changes in LLVM_TARGETS. |
|
Back to top |
|
|
Ionen Developer
Joined: 06 Dec 2018 Posts: 2874
|
Posted: Tue Aug 15, 2023 1:51 pm Post subject: |
|
|
Can note that the ABI issues are still valid today and people that un-force LLVM_TARGETS sometime still end up with a broken llvm toolchain (incl. rust if USE=system-llvm) depending on ordering, not that this is a problem if you can fallback to gcc+binutils and rebuild the broken bits (doing this on a llvm profile without gcc could lead to a broken system that will need binpkgs to recover).
If toolchain is broken, it will manifest itself as undefined symbols in clang or lld themselves, and things like firefox[clang] will fail to build. You just need to rebuild lld and clang in that situation, and maybe rust.
But after that's sorted, it should never come back as an issue and you can safely enjoy quicker llvm builds (fine to have less targets on rust too, or USE=system-llvm is fine as well after it is rebuilt -- albeit may as well use rust-bin if goal is to shorten build time)
If understand the risks and in case process is still not clear -- essentially need to revert what /var/db/repos/gentoo/profiles/base/package.use.force does for llvm targets which can be done using /etc/portage/profile/package.use.force (note that this is under the profile/ directory, not top level), aka with: Code: | */* LLVM_TARGETS: -AArch64 -AMDGPU -ARM -AVR -BPF -Hexagon -Lanai -LoongArch -MSP430 -Mips -NVPTX -PowerPC -RISCV -Sparc -SystemZ -VE -WebAssembly -XCore | (this may need updates as new targets are added, X86 is left forced as it'd be a bad idea for most)
After that you can set what you need using /etc/portage/package.use normally, and note you can use -* there (unlike in package.use.force) e.g.: Code: | */* LLVM_TARGETS: -* AMDGPU NVPTX | If you use NVIDIA, may want to enable NVPTX given can be used for some cuda/ffmpeg stuff, and AMDGPU is currently hard required by mesa's ebuild with USE=llvm even if you don't use an amd gpu (note USE=-llvm typically means gimped mesa, albeit with nvidia-drivers it'd at worst just means losing llvmpipe software rendering, unsure for consequences with Intel gpus or Nouveau but I don't think it should be disabled given they can use gallium and llvm is used with it).
If you ever find yourself needing dev-lang/zig which need most llvm targets to be enabled, you can always fallback to dev-lang/zig-bin that comes with its own llvm. Some packages need BPF to be enabled too, and there's chances there's missing dependencies going around given not many test with targets disabled anymore.
Last edited by Ionen on Tue Aug 15, 2023 10:49 pm; edited 2 times in total |
|
Back to top |
|
|
RIA77 Guru
Joined: 24 Feb 2016 Posts: 391
|
Posted: Tue Aug 15, 2023 2:29 pm Post subject: |
|
|
Quote: | Mystery solved.
It would be better if there's some info about this change at the end of emerge. |
Оbviously it's not mystery solved. That's reason why I asked.
And also I didn't know if it's safe to force disabling targets or not. Thank you Lonen for clarifying. |
|
Back to top |
|
|
|