Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Newbie question about gcc CFLAGS
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
Cuong Nguyen
Tux's lil' helper
Tux's lil' helper


Joined: 18 Jan 2018
Posts: 148

PostPosted: Thu Feb 01, 2018 2:20 am    Post subject: Newbie question about gcc CFLAGS Reply with quote

Have read all related to the question, I am still unsure about how to use march in CFLAGS, to get the most of my cpu

Should I use
Code:
-march=<my-cpu> -mtune=generic
or
Code:
-march=<my-cpu> -mtune=<my-cpu>


The difference between
Code:
-march=<my-cpu>
with/without trailing -m flags as suggested with
Code:
gcc -### -c -march=native /usr/include/stdlib.h

and
Code:
-march=native


Thank you
Back to top
View user's profile Send private message
Rob Paxon
n00b
n00b


Joined: 27 Mar 2006
Posts: 26

PostPosted: Thu Feb 01, 2018 4:51 pm    Post subject: Reply with quote

Use "march=native" without "mtune" for normal use (if you're compiling software for use on the machine you're compiling it on)
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 46365
Location: 56N 3W

PostPosted: Thu Feb 01, 2018 8:18 pm    Post subject: Reply with quote

Cuong Nguyen,

Code:
-march=native
is as good as it gets on Intel/AMD CPUs and as long as you do not use a distributed build system.
If you don't know what a distributed build system is, you are not using one.

If you have gcc-7.3.0 you might want at add
Code:
-mindirect-branch=thunk
to your CFLAGS too.
It helps defend against Spectre v2
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Cuong Nguyen
Tux's lil' helper
Tux's lil' helper


Joined: 18 Jan 2018
Posts: 148

PostPosted: Sat Feb 03, 2018 7:39 am    Post subject: Reply with quote

Thanks, I am not building a distro, just trying to put gentoo on all new and old hardware I have. Putting march=native without mtune is enough for portability, as my pcs can build entire systems (emerge -e) with over 250 pks overnight.

Thank you for anti-spectre flags on gcc-7.3.0. I will try it, I am on 8.0.0_pre9999 right now.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6381

PostPosted: Sat Feb 03, 2018 9:39 am    Post subject: Reply with quote

NeddySeagoon wrote:
Code:
-mindirect-branch=thunk

This has no effect without -O2. Moreover, as mentioned in another thread, one probably should also add
Code:
-fno-plt -mfunction-return=thunk

for spectre. But IMHO spectre is overestimated. More valuable are other protection measurements like
Code:
-fstack-protector-strong -pie -fPIE -fstack-check=specific -Wl,-z,now -Wl,-z,relro
in CFLAGS, CXXFLAGS; LDFLAGS. Fortunately, the former 2(or also the 3rd?) are meanwhile default with gcc[ssp pie].
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 46365
Location: 56N 3W

PostPosted: Sat Feb 03, 2018 5:13 pm    Post subject: Reply with quote

mv,

What the reasoning behind
Code:
-fno-plt -mfunction-return=thunk
?
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 46365
Location: 56N 3W

PostPosted: Sat Feb 03, 2018 5:15 pm    Post subject: Reply with quote

Cuong Nguyen,

That sounds like a build farm you have there, so maybe you do want to use distributed compiling.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6381

PostPosted: Sat Feb 03, 2018 5:42 pm    Post subject: Reply with quote

NeddySeagoon wrote:
mv,

What the reasoning behind
Code:
-fno-plt -mfunction-return=thunk
?

They eliminate some further prediction not covered by -mindirect-branch=thunk: In binutils, -Wl,-z,retpolineplt was not implemented, because they said that -fno-plt should be used instead.
And concerning -mfunction-return, see e.g. here
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 46365
Location: 56N 3W

PostPosted: Sat Feb 03, 2018 5:51 pm    Post subject: Reply with quote

mv,

Thank you.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Cuong Nguyen
Tux's lil' helper
Tux's lil' helper


Joined: 18 Jan 2018
Posts: 148

PostPosted: Mon Feb 05, 2018 7:04 am    Post subject: Reply with quote

NeddySeagoon wrote:
Cuong Nguyen,

That sounds like a build farm you have there, so maybe you do want to use distributed compiling.


NeddySeagoon

I understand the concept of distributed compiling, never tried it though. IMHO, it worth to try with 20++ of narrow range or identical CPUs. I have only about half dozen machines 10-5 years old, from Nehalem, Westmere to IvyBridge, Haswell. Yet I have to make a choice, should I build a neutral my-distro to run on all the machines or per-cpu customized.
Back to top
View user's profile Send private message
Cuong Nguyen
Tux's lil' helper
Tux's lil' helper


Joined: 18 Jan 2018
Posts: 148

PostPosted: Mon Feb 05, 2018 7:23 am    Post subject: Reply with quote

mv wrote:

for spectre. But IMHO spectre is overestimated. More valuable are other protection measurements like
Code:
-fstack-protector-strong -pie -fPIE -fstack-check=specific -Wl,-z,now -Wl,-z,relro
in CFLAGS, CXXFLAGS; LDFLAGS. Fortunately, the former 2(or also the 3rd?) are meanwhile default with gcc[ssp pie].


Is that default for hardened profile? As advised by Gentoo Hardened FAQs: "let the profile do its jobs" https://wiki.gentoo.org/wiki/Hardened/FAQ#Can_I_add_-fstack-protector-all_or_-fstack-protector_in_the_make.conf_CFLAGS.3F

AFAIK Arch Linux set fstack-protector-strong -fno-plt in it's CFLAGS and CXXFLAGS as default makepkg.conf
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6381

PostPosted: Mon Feb 05, 2018 7:29 am    Post subject: Reply with quote

Cuong Nguyen wrote:
mv wrote:

for spectre. But IMHO spectre is overestimated. More valuable are other protection measurements like
Code:
-fstack-protector-strong -pie -fPIE -fstack-check=specific -Wl,-z,now -Wl,-z,relro
in CFLAGS, CXXFLAGS; LDFLAGS. Fortunately, the former 2(or also the 3rd?) are meanwhile default with gcc[ssp pie].


Is that default for hardened profile? As advised by Gentoo Hardened FAQs: "let the profile do its jobs"

The hardened gcc uses them implicitly by default (i.e. they do not have to be specified). But hardened uses even -fstack-protector-all which considerably slows down for practically no security gain. Moreover, I had other issues with hardened, and it is not so simple to switch gcc profile package-dependent automatically as it is to filter flags in /etc/portage/bashrc (and when you switch gcc profile you would have to add the non-problematic flags anyway).
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 46365
Location: 56N 3W

PostPosted: Mon Feb 05, 2018 10:23 am    Post subject: Reply with quote

Cuong Nguyen,

There is a half way house too. You build a core set of packages that will suit all the systems.
This core set is build once, install everywhere.

Where performance matters (if it does), you build the package locally, so it makes best use of the hardware.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Cuong Nguyen
Tux's lil' helper
Tux's lil' helper


Joined: 18 Jan 2018
Posts: 148

PostPosted: Mon Feb 05, 2018 10:28 am    Post subject: Reply with quote

[quote="mv"][quote="Cuong Nguyen"]
mv wrote:

Moreover, I had other issues with hardened, and it is not so simple to switch gcc profile package-dependent automatically as it is to filter flags in /etc/portage/bashrc (and when you switch gcc profile you would have to add the non-problematic flags anyway).


I don't like hardened profile, too, as it renders other problems when switching profiles. Now I copy settings from hardened profile (i.e. "hardened" use-flag) to my local /etc/portage/profile or starting with hardened profile, re-emerge toolchain consists of gcc, binutils, glibc, virtual/libc, libtool, protect it from re-emerging by package.provided file.

The otherway is instead of switching profiles I create combined profiles on local and link it with hardened profile.

Now I put fstack flags explicitly into C, CXX and LDFLAGS
Code:

CFLAGS="${CFLAGS} -fstack-protector-strong -pie -fPIE -fstack-check=specific"
CFLAGS="${CFLAGS} -fno-plt -mfunction-return=thunk"

CXXFLAGS="${CFLAGS}"

LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--sort-common -Wl,--hash-style=both -Wl,-z,relro -Wl,-znow"
LDFLAGS="${LDFLAGS} -fstack-protector-strong -pie -fPIE -fstack-check=specific"
LDFLAGS="${LDFLAGS} -fno-plt -mfunction-return=thunk"

I am just wondering why I have to put the same flags into C CXX and LDFLAGS?

Best regards
Back to top
View user's profile Send private message
Cuong Nguyen
Tux's lil' helper
Tux's lil' helper


Joined: 18 Jan 2018
Posts: 148

PostPosted: Tue Feb 06, 2018 1:08 am    Post subject: Reply with quote

mv wrote:
NeddySeagoon wrote:
Code:
-mindirect-branch=thunk

This has no effect without -O2. Moreover, as mentioned in another thread, one probably should also add
Code:
-fno-plt -mfunction-return=thunk

for spectre. But IMHO spectre is overestimated. More valuable are other protection measurements like
Code:
-fstack-protector-strong -pie -fPIE -fstack-check=specific -Wl,-z,now -Wl,-z,relro
in CFLAGS, CXXFLAGS; LDFLAGS. Fortunately, the former 2(or also the 3rd?) are meanwhile default with gcc[ssp pie].


mv,
-pie -fPIE flags break several packages, one of them is coreutils with error:
Code:
../lib64/Scrt1.o:function _start: error: undefined reference to 'main'


Best regards,
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6381

PostPosted: Tue Feb 06, 2018 7:00 am    Post subject: Reply with quote

Cuong Nguyen wrote:
-pie -fPIE flags break several packages

Yes, for several packages they need to be filtered.
However, you always catch this during compile time.
To be honest, I do not understand why the filtering is necessary while actually sys-devel/gcc[pie] claims that pie should be added automatically, anyway: It is not the case that all the mentioned packages do add -no-pie internally.
Back to top
View user's profile Send private message
Cuong Nguyen
Tux's lil' helper
Tux's lil' helper


Joined: 18 Jan 2018
Posts: 148

PostPosted: Tue Feb 06, 2018 6:39 pm    Post subject: Reply with quote

mv wrote:
Cuong Nguyen wrote:
-pie -fPIE flags break several packages

Yes, for several packages they need to be filtered.
However, you always catch this during compile time.
To be honest, I do not understand why the filtering is necessary while actually sys-devel/gcc[pie] claims that pie should be added automatically, anyway: It is not the case that all the mentioned packages do add -no-pie internally.


Thank you mv,

As stated the Gentoo Hardened FAQs, gcc (pie) is enabled and it should be passed to packages created with the pie flags as default. Anyway I removed -pie -fPIE flags, keeping stack and anti spectre vuln. as you've suggested. All the packages compiled with no problems.

I am not sure how those hardened improvements will degrade my system yet.

Best regards,
Back to top
View user's profile Send private message
Cuong Nguyen
Tux's lil' helper
Tux's lil' helper


Joined: 18 Jan 2018
Posts: 148

PostPosted: Sat Feb 10, 2018 9:24 am    Post subject: Reply with quote

mv wrote:
Cuong Nguyen wrote:
-pie -fPIE flags break several packages

Yes, for several packages they need to be filtered.
However, you always catch this during compile time.
To be honest, I do not understand why the filtering is necessary while actually sys-devel/gcc[pie] claims that pie should be added automatically, anyway: It is not the case that all the mentioned packages do add -no-pie internally.


mv,

Thanks for your great package portage-bashrc-mv, it help me to tidy up all per package config file I used before. I used to put gcc.O2{3}.{,no-}graphite.{,no-}lto.conf in /etc/portage/env and referred by files in/etc/portage/package.env.

One more question, should I stop using graphite flags? As you said in /etc/portage/package.cflags/graphite file?

Best regards,
Back to top
View user's profile Send private message
Cuong Nguyen
Tux's lil' helper
Tux's lil' helper


Joined: 18 Jan 2018
Posts: 148

PostPosted: Sat Feb 10, 2018 9:32 am    Post subject: Reply with quote

NeddySeagoon wrote:
Cuong Nguyen,

There is a half way house too. You build a core set of packages that will suit all the systems.
This core set is build once, install everywhere.

Where performance matters (if it does), you build the package locally, so it makes best use of the hardware.


How to define a core set to start with? Should I build a neutral, cpu independent toolchain in binary and use it to compile all other packages? like march=x86-64 mtune=generic.

Now I use the following steps.
from stage3
emerge -1 gcc with all optimizations, hardened flags e.g hardened, ssp, pie, graphite
re-emerge binutils, libc, glibc, libtool
re-emerge gcc once more
re-emerge all python ruby perl
emerge all needed packages.

Thank you
Best regards,
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6381

PostPosted: Sat Feb 10, 2018 10:14 am    Post subject: Reply with quote

Cuong Nguyen wrote:
One more question, should I stop using graphite flags?

That's what you have to decide. I had a lot of instabilities, but that was quite a while ago (perhaps gcc-4). The situation might have been improved, meanwhile.
However, the problem is that these issues usually come up only at runtime, and I have decided that a theoretical slight speed improvement is not worth random runtime segfaults or strange behavior: When it occurs, one has forgotten that the cflags might be the reason. I was hunting such bugs several times and when it turned out once too often that graphite was the reason, I had decided to stop it and evade further trouble.
Back to top
View user's profile Send private message
Cuong Nguyen
Tux's lil' helper
Tux's lil' helper


Joined: 18 Jan 2018
Posts: 148

PostPosted: Sat Feb 10, 2018 11:06 am    Post subject: Reply with quote

mv wrote:
Cuong Nguyen wrote:
One more question, should I stop using graphite flags?

However, the problem is that these issues usually come up only at runtime, and I have decided that a theoretical slight speed improvement is not worth random segfaults or strange behavior: When it occurs, one has forgotten that the cflags might be the reason. I was hunting such bugs several times and when it turned out once too often that graphite was the reason, I had decided to stop it and evade further trouble.


Yes I had cannot boot couples of time when I used too aggressive graphite flags, removing some flags helped. I did not notice any performance improvement though, maybe because I mostly tested gentoo on VMs. I dont know any distro utilize graphite, although some user packages on Arch Linux allow to recompile with graphite and lto flags, but Arch Port system not as good as in Gentoo.

Best regards,
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6381

PostPosted: Sat Feb 10, 2018 11:46 am    Post subject: Reply with quote

Cuong Nguyen wrote:
I did not notice any performance improvement though,

That's also a reason why I think it isn't worth the trouble. For -flto there is quite an impressive reduction in disk space, sometimes (e.g. for eix it was(is?) regularly the case that the binary size reduced to 1/3, probably in combination with -fmerge-all-constants or something similar), but for graphite the award is minimal. Perhaps it makes sense to enable graphite selectively for special packages, e.g. gcc, clang, llvm, ffmpeg and some similar multimedia packages which use the processor heavily. So far, I never played with that.
Back to top
View user's profile Send private message
Cuong Nguyen
Tux's lil' helper
Tux's lil' helper


Joined: 18 Jan 2018
Posts: 148

PostPosted: Thu Feb 15, 2018 1:24 am    Post subject: Reply with quote

Dear mv,

Please help me with following questions regarding your portage-bashrc-mv

1. if I use --param in CFLAGS
Code:
--param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=12288

i've seen your portage bashrc will remove duplicate --param, leaving followed by params
Code:
--param l1-cache-size=32 l1-cache-line-size=64 l2-cache-size=12288

that causes error during compilng

2. How to use patch files in /etc/portage/env/patches, I could not find any reference to those patches. How can I create my own patches for being emerged packages? As per gentoo wiki suggests, I create patches under /etc/portage/patches/%{CATEGORY}/%{PF} folders for every package needs patches.

3. How can I monitor build.log for certain messages like lto flags caused plugins required... warnings so I can stop emerging and adjust flags? I use emerge --keep-going --quiet-build=y so all messages are in background.

Thank you,
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6381

PostPosted: Thu Feb 15, 2018 8:06 am    Post subject: Reply with quote

Cuong Nguyen wrote:
1. if I use --param in CFLAGS

AFAIK every option in gcc can be specified in 1 argument:
Code:
--param=l1-cache-size=32

(There are also other options like -Wl which are handled correctly by the script only if only 1 argument is passed to the linker. This could be polished more, but I think it is better to avoid built-in “knowledge” about the options if possible.)
Quote:
2. How to use patch files in /etc/portage/env/patches

I suppose that you have found this directory in portage-env-mv. This directory is accessed by the function mv_epatch from env/scripts/mv_patch. The latter file in turn is sourced by e.g. env/%{CATEGORY}/%{PF}. The advantage is that this works reliably also for packages whose EAPI is too old (so that they do not support /etc/portage/patches) and which do not inherit the eutils (or the recent epatch) eclass so that the epatch and epatch_user commands are not available (there do exist some horrible hacks on some gentoo wiki how to source the latter eclasses manually, but it is much cleaner to provide a local patch function instead.)
For a public repository like portage-env-mv, this has the additional advantage that only /etc/portage/env needs to be contained in the repository so that you can checkout the repository directly into your /etc/portage without collisions with your local /etc/portage/patches.
Quote:
3. How can I monitor build.log for certain messages like lto flags caused plugins required... warnings so I can stop emerging and adjust flags? I use emerge --keep-going --quiet-build=y so all messages are in background

Code:
watch grep 'plugin.*required'  /path/to/build.log

However, there are quite some packages which claim that the plugin is required but which run without problems with flto, because the static libs requiring the plugins are built "in vain" (i.e. neither used nor instaled).
Back to top
View user's profile Send private message
Cuong Nguyen
Tux's lil' helper
Tux's lil' helper


Joined: 18 Jan 2018
Posts: 148

PostPosted: Fri Feb 16, 2018 6:57 am    Post subject: Reply with quote

mv wrote:

For a public repository like portage-env-mv, this has the additional advantage that only /etc/portage/env needs to be contained in the repository so that you can checkout the repository directly into your /etc/portage without collisions with your local /etc/portage/patches.


mv, thank you very much for your detailed explanation. I was too lazy to learn how to script ebuild hooks, your portage-bashrc-mv saves me tons of time to do per-package building customization. Now I put all my mods in /etc/portage/package.cflags/local.

Best regards,
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum