Newbie question about gcc CFLAGS

[Cuong Nguyen]

Have read all related to the question, I am still unsure about how to use march in CFLAGS, to get the most of my cpu

Should I use

Code: Select all

-march=<my-cpu> -mtune=generic

or

Code: Select all

-march=<my-cpu> -mtune=<my-cpu>

The difference between

Code: Select all

-march=<my-cpu>

with/without trailing -m flags as suggested with

Code: Select all

gcc -### -c -march=native /usr/include/stdlib.h

and

Code: Select all

-march=native

Thank you

[Rob Paxon]

Use "march=native" without "mtune" for normal use (if you're compiling software for use on the machine you're compiling it on)

[NeddySeagoon]

Cuong Nguyen,

Code: Select all

-march=native

is as good as it gets on Intel/AMD CPUs and as long as you do not use a distributed build system.
If you don't know what a distributed build system is, you are not using one.

If you have gcc-7.3.0 you might want at add

Code: Select all

-mindirect-branch=thunk

to your CFLAGS too.
It helps defend against Spectre v2

[Cuong Nguyen]

Thanks, I am not building a distro, just trying to put gentoo on all new and old hardware I have. Putting march=native without mtune is enough for portability, as my pcs can build entire systems (emerge -e) with over 250 pks overnight.

Thank you for anti-spectre flags on gcc-7.3.0. I will try it, I am on 8.0.0_pre9999 right now.

[mv]

Code: Select all

-mindirect-branch=thunk

This has no effect without -O2. Moreover, as mentioned in another thread, one probably should also add

Code: Select all

-fno-plt -mfunction-return=thunk

for spectre. But IMHO spectre is overestimated. More valuable are other protection measurements like

Code: Select all

-fstack-protector-strong -pie -fPIE -fstack-check=specific -Wl,-z,now -Wl,-z,relro

in CFLAGS, CXXFLAGS; LDFLAGS. Fortunately, the former 2(or also the 3rd?) are meanwhile default with gcc[ssp pie].

[NeddySeagoon]

mv,

What the reasoning behind

Code: Select all

-fno-plt -mfunction-return=thunk

?

[NeddySeagoon]

Cuong Nguyen,

That sounds like a build farm you have there, so maybe you do want to use distributed compiling.

[mv]

mv,

What the reasoning behind

Code: Select all

-fno-plt -mfunction-return=thunk

?

They eliminate some further prediction not covered by -mindirect-branch=thunk: In binutils, -Wl,-z,retpolineplt was not implemented, because they said that -fno-plt should be used instead.
And concerning -mfunction-return, see e.g. here

[NeddySeagoon]

mv,

Thank you.

[Cuong Nguyen]

Cuong Nguyen,

That sounds like a build farm you have there, so maybe you do want to use distributed compiling.

NeddySeagoon

I understand the concept of distributed compiling, never tried it though. IMHO, it worth to try with 20++ of narrow range or identical CPUs. I have only about half dozen machines 10-5 years old, from Nehalem, Westmere to IvyBridge, Haswell. Yet I have to make a choice, should I build a neutral my-distro to run on all the machines or per-cpu customized.

[Cuong Nguyen]

for spectre. But IMHO spectre is overestimated. More valuable are other protection measurements like

Code: Select all

-fstack-protector-strong -pie -fPIE -fstack-check=specific -Wl,-z,now -Wl,-z,relro

in CFLAGS, CXXFLAGS; LDFLAGS. Fortunately, the former 2(or also the 3rd?) are meanwhile default with gcc[ssp pie].

Is that default for hardened profile? As advised by Gentoo Hardened FAQs: "let the profile do its jobs" https://wiki.gentoo.org/wiki/Hardened/F ... _CFLAGS.3F

AFAIK Arch Linux set fstack-protector-strong -fno-plt in it's CFLAGS and CXXFLAGS as default makepkg.conf

[mv]

for spectre. But IMHO spectre is overestimated. More valuable are other protection measurements like

Code: Select all

-fstack-protector-strong -pie -fPIE -fstack-check=specific -Wl,-z,now -Wl,-z,relro

in CFLAGS, CXXFLAGS; LDFLAGS. Fortunately, the former 2(or also the 3rd?) are meanwhile default with gcc[ssp pie].

Is that default for hardened profile? As advised by Gentoo Hardened FAQs: "let the profile do its jobs"

The hardened gcc uses them implicitly by default (i.e. they do not have to be specified). But hardened uses even -fstack-protector-all which considerably slows down for practically no security gain. Moreover, I had other issues with hardened, and it is not so simple to switch gcc profile package-dependent automatically as it is to filter flags in /etc/portage/bashrc (and when you switch gcc profile you would have to add the non-problematic flags anyway).

[NeddySeagoon]

Cuong Nguyen,

There is a half way house too. You build a core set of packages that will suit all the systems.
This core set is build once, install everywhere.

Where performance matters (if it does), you build the package locally, so it makes best use of the hardware.

[Cuong Nguyen]

Moreover, I had other issues with hardened, and it is not so simple to switch gcc profile package-dependent automatically as it is to filter flags in /etc/portage/bashrc (and when you switch gcc profile you would have to add the non-problematic flags anyway).

I don't like hardened profile, too, as it renders other problems when switching profiles. Now I copy settings from hardened profile (i.e. "hardened" use-flag) to my local /etc/portage/profile or starting with hardened profile, re-emerge toolchain consists of gcc, binutils, glibc, virtual/libc, libtool, protect it from re-emerging by package.provided file.

The otherway is instead of switching profiles I create combined profiles on local and link it with hardened profile.

Now I put fstack flags explicitly into C, CXX and LDFLAGS

Code: Select all

CFLAGS="${CFLAGS} -fstack-protector-strong -pie -fPIE -fstack-check=specific"
CFLAGS="${CFLAGS} -fno-plt -mfunction-return=thunk"

CXXFLAGS="${CFLAGS}"

LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--sort-common -Wl,--hash-style=both -Wl,-z,relro -Wl,-znow"
LDFLAGS="${LDFLAGS} -fstack-protector-strong -pie -fPIE -fstack-check=specific"
LDFLAGS="${LDFLAGS} -fno-plt -mfunction-return=thunk"

I am just wondering why I have to put the same flags into C CXX and LDFLAGS?

Best regards

[Cuong Nguyen]

Code: Select all

-mindirect-branch=thunk

This has no effect without -O2. Moreover, as mentioned in another thread, one probably should also add

Code: Select all

-fno-plt -mfunction-return=thunk

for spectre. But IMHO spectre is overestimated. More valuable are other protection measurements like

Code: Select all

-fstack-protector-strong -pie -fPIE -fstack-check=specific -Wl,-z,now -Wl,-z,relro

in CFLAGS, CXXFLAGS; LDFLAGS. Fortunately, the former 2(or also the 3rd?) are meanwhile default with gcc[ssp pie].

mv,
-pie -fPIE flags break several packages, one of them is coreutils with error:

Code: Select all

../lib64/Scrt1.o:function _start: error: undefined reference to 'main'

Best regards,

[mv]

-pie -fPIE flags break several packages

Yes, for several packages they need to be filtered.
However, you always catch this during compile time.
To be honest, I do not understand why the filtering is necessary while actually sys-devel/gcc[pie] claims that pie should be added automatically, anyway: It is not the case that all the mentioned packages do add -no-pie internally.

[Cuong Nguyen]

-pie -fPIE flags break several packages

Yes, for several packages they need to be filtered.
However, you always catch this during compile time.
To be honest, I do not understand why the filtering is necessary while actually sys-devel/gcc[pie] claims that pie should be added automatically, anyway: It is not the case that all the mentioned packages do add -no-pie internally.

Thank you mv,

As stated the Gentoo Hardened FAQs, gcc (pie) is enabled and it should be passed to packages created with the pie flags as default. Anyway I removed -pie -fPIE flags, keeping stack and anti spectre vuln. as you've suggested. All the packages compiled with no problems.

I am not sure how those hardened improvements will degrade my system yet.

Best regards,

[Cuong Nguyen]

-pie -fPIE flags break several packages

Yes, for several packages they need to be filtered.
However, you always catch this during compile time.
To be honest, I do not understand why the filtering is necessary while actually sys-devel/gcc[pie] claims that pie should be added automatically, anyway: It is not the case that all the mentioned packages do add -no-pie internally.

mv,

Thanks for your great package portage-bashrc-mv, it help me to tidy up all per package config file I used before. I used to put gcc.O2{3}.{,no-}graphite.{,no-}lto.conf in /etc/portage/env and referred by files in/etc/portage/package.env.

One more question, should I stop using graphite flags? As you said in /etc/portage/package.cflags/graphite file?

Best regards,

[Cuong Nguyen]

Cuong Nguyen,

There is a half way house too. You build a core set of packages that will suit all the systems.
This core set is build once, install everywhere.

Where performance matters (if it does), you build the package locally, so it makes best use of the hardware.

How to define a core set to start with? Should I build a neutral, cpu independent toolchain in binary and use it to compile all other packages? like march=x86-64 mtune=generic.

Now I use the following steps.
from stage3
emerge -1 gcc with all optimizations, hardened flags e.g hardened, ssp, pie, graphite
re-emerge binutils, libc, glibc, libtool
re-emerge gcc once more
re-emerge all python ruby perl
emerge all needed packages.

Thank you
Best regards,

[mv]

One more question, should I stop using graphite flags?

That's what you have to decide. I had a lot of instabilities, but that was quite a while ago (perhaps gcc-4). The situation might have been improved, meanwhile.
However, the problem is that these issues usually come up only at runtime, and I have decided that a theoretical slight speed improvement is not worth random runtime segfaults or strange behavior: When it occurs, one has forgotten that the cflags might be the reason. I was hunting such bugs several times and when it turned out once too often that graphite was the reason, I had decided to stop it and evade further trouble.

[Cuong Nguyen]

One more question, should I stop using graphite flags?

However, the problem is that these issues usually come up only at runtime, and I have decided that a theoretical slight speed improvement is not worth random segfaults or strange behavior: When it occurs, one has forgotten that the cflags might be the reason. I was hunting such bugs several times and when it turned out once too often that graphite was the reason, I had decided to stop it and evade further trouble.

Yes I had cannot boot couples of time when I used too aggressive graphite flags, removing some flags helped. I did not notice any performance improvement though, maybe because I mostly tested gentoo on VMs. I dont know any distro utilize graphite, although some user packages on Arch Linux allow to recompile with graphite and lto flags, but Arch Port system not as good as in Gentoo.

Best regards,

[mv]

I did not notice any performance improvement though,

That's also a reason why I think it isn't worth the trouble. For -flto there is quite an impressive reduction in disk space, sometimes (e.g. for eix it was(is?) regularly the case that the binary size reduced to 1/3, probably in combination with -fmerge-all-constants or something similar), but for graphite the award is minimal. Perhaps it makes sense to enable graphite selectively for special packages, e.g. gcc, clang, llvm, ffmpeg and some similar multimedia packages which use the processor heavily. So far, I never played with that.

[Cuong Nguyen]

Dear mv,

Please help me with following questions regarding your portage-bashrc-mv

1. if I use --param in CFLAGS

Code: Select all

--param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=12288

i've seen your portage bashrc will remove duplicate --param, leaving followed by params

Code: Select all

--param l1-cache-size=32 l1-cache-line-size=64 l2-cache-size=12288

that causes error during compilng

2. How to use patch files in /etc/portage/env/patches, I could not find any reference to those patches. How can I create my own patches for being emerged packages? As per gentoo wiki suggests, I create patches under /etc/portage/patches/%{CATEGORY}/%{PF} folders for every package needs patches.

3. How can I monitor build.log for certain messages like lto flags caused plugins required... warnings so I can stop emerging and adjust flags? I use emerge --keep-going --quiet-build=y so all messages are in background.

Thank you,

[mv]

1. if I use --param in CFLAGS

AFAIK every option in gcc can be specified in 1 argument:

Code: Select all

--param=l1-cache-size=32

(There are also other options like -Wl which are handled correctly by the script only if only 1 argument is passed to the linker. This could be polished more, but I think it is better to avoid built-in “knowledge” about the options if possible.)

2. How to use patch files in /etc/portage/env/patches

I suppose that you have found this directory in portage-env-mv. This directory is accessed by the function mv_epatch from env/scripts/mv_patch. The latter file in turn is sourced by e.g. env/%{CATEGORY}/%{PF}. The advantage is that this works reliably also for packages whose EAPI is too old (so that they do not support /etc/portage/patches) and which do not inherit the eutils (or the recent epatch) eclass so that the epatch and epatch_user commands are not available (there do exist some horrible hacks on some gentoo wiki how to source the latter eclasses manually, but it is much cleaner to provide a local patch function instead.)
For a public repository like portage-env-mv, this has the additional advantage that only /etc/portage/env needs to be contained in the repository so that you can checkout the repository directly into your /etc/portage without collisions with your local /etc/portage/patches.

3. How can I monitor build.log for certain messages like lto flags caused plugins required... warnings so I can stop emerging and adjust flags? I use emerge --keep-going --quiet-build=y so all messages are in background

Code: Select all

watch grep 'plugin.*required'  /path/to/build.log

However, there are quite some packages which claim that the plugin is required but which run without problems with flto, because the static libs requiring the plugins are built "in vain" (i.e. neither used nor instaled).

[Cuong Nguyen]

For a public repository like portage-env-mv, this has the additional advantage that only /etc/portage/env needs to be contained in the repository so that you can checkout the repository directly into your /etc/portage without collisions with your local /etc/portage/patches.

mv, thank you very much for your detailed explanation. I was too lazy to learn how to script ebuild hooks, your portage-bashrc-mv saves me tons of time to do per-package building customization. Now I put all my mods in /etc/portage/package.cflags/local.

Best regards,