Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
CFLAGS, -march, and distcc
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
metafarion
n00b
n00b


Joined: 15 Mar 2012
Posts: 6
Location: Madison, WI

PostPosted: Thu Mar 15, 2012 4:21 am    Post subject: CFLAGS, -march, and distcc Reply with quote

Ok, so for awhile now I've had to choose between using -march=native in my make.conf or having additional compiling power with distcc (really helps on the netbook). My understanding is that you can't use march=native in a distcc environment because gcc will pull in different "native" CFLAGS from all the different distcc nodes depending on their individual architectures. Fair enough.

However, it's also been demonstrated (see here and other pages like it) that -march=native enables additional CFLAGS which -march=atom (for my netbook for example) does not, and therefore you may end up with a more optimized build using march=native.

Ignore for the moment, the argument of whether or not this is true and that this is a dumb Gentoo ricer question, would it not be possible to get the effect of -march=native simply by doing a gcc test with -march=native set, recording the CFLAGS and setting them all explicitly in make.conf? Then you would be free to enable distcc without worrying about CFLAGS from other cpu types creeping in maybe? Am I missing any crucial details in this plan? Does -march affect more than just CFLAGS? Other things that can't be explicitly set in make.conf?


Last edited by metafarion on Thu Mar 15, 2012 4:39 am; edited 1 time in total
Back to top
View user's profile Send private message
cach0rr0
Moderator
Moderator


Joined: 13 Nov 2008
Posts: 4123
Location: Houston, Republic of Texas

PostPosted: Thu Mar 15, 2012 4:36 am    Post subject: Reply with quote

i dont know this is a correct or even consensus way, but what ive done is keep two sets of CFLAGS handy in make.conf for when im doing something that may benefit from distcc:

Code:

CFLAGS="-march=native -O2 -pipe"
#CFLAGS="-march=core2 -m64 -m80387 -m96bit-long-double -malign-stringops -mcx16 -mfancy-math-387 -mfp-ret-in-387 -mfused-madd -mglibc -mhard-float -mieee-fp -mpush-args -mred-zone -msahf -msse -msse2 -msse3 -msse4.1 -mssse3 -mstackrealign -mtls-direct-seg-refs -O2 -pipe"


obtained via looking at output of:

Code:

gcc -Q -march=native --help=target
gcc -### -march=native -E /usr/include/stdlib.h 2>&1 | grep "/usr/libexec/gcc/.*cc1"


hope that helps (?)
im sure others will comment
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash
Back to top
View user's profile Send private message
metafarion
n00b
n00b


Joined: 15 Mar 2012
Posts: 6
Location: Madison, WI

PostPosted: Mon Mar 19, 2012 5:12 am    Post subject: Reply with quote

That sounds very reasonable. Then is it your understanding as well that -march= basically invokes a preset of CFLAGS and that's it? Does it do anything else?
Back to top
View user's profile Send private message
cach0rr0
Moderator
Moderator


Joined: 13 Nov 2008
Posts: 4123
Location: Houston, Republic of Texas

PostPosted: Mon Mar 19, 2012 7:01 am    Post subject: Reply with quote

metafarion wrote:
That sounds very reasonable. Then is it your understanding as well that -march= basically invokes a preset of CFLAGS and that's it? Does it do anything else?


my understanding is it'll attempt to auto-determine the best march for your CPU, and tack on any additional flags your CPU might support.
I mention those separately because "march" itself will expand to other things. So for example having -march=amdfam10 might enable a chunk of things for amdfam10, and then on top of that by virtue of having used native, other things are added as well

caveat being that may be a *wrong* understanding, but at least it'd be a wrong understanding ive derived from attempts to research it rather than guesswork!
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 10051

PostPosted: Mon Mar 19, 2012 10:32 pm    Post subject: Reply with quote

Also, using -march=native will prevent distcc from sending the work to remote machines. It has no way to guarantee that the remote machine will produce the same expanded value as the local machine, so the only safe way for distcc to handle it is to build everything locally. If you intend to use distcc, look up what -march=native does for you and use those options explicitly.
Back to top
View user's profile Send private message
metafarion
n00b
n00b


Joined: 15 Mar 2012
Posts: 6
Location: Madison, WI

PostPosted: Fri Mar 23, 2012 12:41 am    Post subject: Reply with quote

Alright, so based on everything I've heard so far and everything I've read about gcc, I think I have a method for emerging with distcc while still gaining the benefits of -march=native. I'm going to run through this with an Athlon64 X2 3800+ system for criticism and speculation.

The first order of business is to determine what flags -march=native adds to -march=k8-sse3 (the suggested arch for my CPU)

Create dummy files for gcc to process with each -march and send the verbose output to respective .s files:

$ touch native.cc
$ touch k8-sse3.cc
$ gcc -fverbose-asm -march=native native.cc -S
$ gcc -fverbose-asm -march=k8-sse3 k8-sse3.cc -S

Compare the contents of the two output files, specifically the 'options passed' section:

$ diff -u k8-sse3.s native.s

-# options passed: -D_GNU_SOURCE k8-sse3.cc -D_FORTIFY_SOURCE=2
-# -march=k8-sse3 -fverbose-asm
+# options passed: -D_GNU_SOURCE native.cc -D_FORTIFY_SOURCE=2
+# -march=k8-sse3 -msahf --param l1-cache-size=64 --param
+# l1-cache-line-size=64 --param l2-cache-size=512 -mtune=k8 -fverbose-asm

We can easily see that in addition to the flags enabled by -march=k8-sse3, -march=native enables the following: -msahf --param l1-cache-size=64 --param l1-cache-line-size=64 --param l2-cache-size=512 -mtune=k8

These flags can be added to make.conf along with -march=k8-sse3.

Bam. -march=native without -march=native.


Now this seems like a pretty tidy little solution to me, but my suspicion is this: If it's really that simple, then why doesn't distcc do it automatically? What would stop the devs from instructing distcc to gather the target flags from -march=native and pass them to all the compile nodes. Why is it just accepted that march=native is just incompatible with distcc, end-of-story?

Or is there a huge gaping flaw in my analysis here?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 10051

PostPosted: Fri Mar 23, 2012 2:35 am    Post subject: Reply with quote

You ran several processes and performed non-trivial text parsing on the output to derive that list of flags. It is much easier to document that users should avoid -march=native than to implement the corresponding functionality in code. Also, distcc is mostly stateless. It would add significant overhead to have it compute that expression every time.
Back to top
View user's profile Send private message
piedar
Tux's lil' helper
Tux's lil' helper


Joined: 09 Aug 2010
Posts: 80

PostPosted: Sun May 06, 2012 5:24 pm    Post subject: Reply with quote

I have some related distcc questions and rather than start another thread, I'll ask them here. I've roughly followed the distcc docs, but they're outdated and more than a little confusing.

Do the CFLAGS on machines running distccd affect compilation on machines requesting jobs? I have two machines, let's refer to them by their processors: core2 and atom. I want the core2 to compile for the atom, but not the other way around. So in my make.conf on atom, I used metafarion's method to get CFLAGS equivalent to -march=native. But can I leave -march=native in core2's CFLAGS or does that affect the builds for atom? Does core2 need FEATURES=distcc and/or distcc added to its PATH? Does atom need to have the distccd service running (seems to work without it)?

Right now, I have set core2's CFLAGS == atom's CFLAGS just in case.


Bonus: Is there a way to do --load-average per machine?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 10051

PostPosted: Sun May 06, 2012 6:04 pm    Post subject: Reply with quote

piedar wrote:
Do the CFLAGS on machines running distccd affect compilation on machines requesting jobs?
No. The volunteers receive a command line and preprocessed source text from the master, run the compiler specified by the master on the received text, and return the result. They do not inspect local Portage configuration.
piedar wrote:
But can I leave -march=native in core2's CFLAGS or does that affect the builds for atom? Does core2 need FEATURES=distcc and/or distcc added to its PATH? Does atom need to have the distccd service running (seems to work without it)?
As I stated earlier in the thread, if you use -march=native, that will automatically prevent the compilation from being distributed. The master must specify FEATURES=distcc. The volunteers must run distccd, or sshd if you use ssh transport for distcc.
piedar wrote:
Bonus: Is there a way to do --load-average per machine?
I am not aware of any tools which collect and use the load information to achieve this. It would be nice, but it requires integrating the distribution phase, the host list, and the build tool (Make, SCons, waf, etc.) such that the build tool knows when it can create a new job, and where that job can run without exceeding any per-host limits.
Back to top
View user's profile Send private message
metafarion
n00b
n00b


Joined: 15 Mar 2012
Posts: 6
Location: Madison, WI

PostPosted: Fri May 18, 2012 9:27 pm    Post subject: Reply with quote

Hu wrote:
piedar wrote:
But can I leave -march=native in core2's CFLAGS or does that affect the builds for atom? Does core2 need FEATURES=distcc and/or distcc added to its PATH? Does atom need to have the distccd service running (seems to work without it)?
As I stated earlier in the thread, if you use -march=native, that will automatically prevent the compilation from being distributed. The master must specify FEATURES=distcc. The volunteers must run distccd, or sshd if you use ssh transport for distcc.


If I'm understanding Piedar correctly, he's asking if core2 needs to have march=native removed from its make.conf in order to be a viable distcc node for atom and I believe the answer is no. Everything you set in a given host's make.conf affects only its own emerge process and not that of machines connecting to it via distcc.
Back to top
View user's profile Send private message
dacid
n00b
n00b


Joined: 29 May 2004
Posts: 49
Location: Charlotte, NC, USA

PostPosted: Wed Oct 10, 2012 5:57 pm    Post subject: Reply with quote

metafarion wrote:
Alright, so based on everything I've heard so far and everything I've read about gcc, I think I have a method for emerging with distcc while still gaining the benefits of -march=native. I'm going to run through this with an Athlon64 X2 3800+ system for criticism and speculation.

The first order of business is to determine what flags -march=native adds to -march=k8-sse3 (the suggested arch for my CPU)

Create dummy files for gcc to process with each -march and send the verbose output to respective .s files:

$ touch native.cc
$ touch k8-sse3.cc
$ gcc -fverbose-asm -march=native native.cc -S
$ gcc -fverbose-asm -march=k8-sse3 k8-sse3.cc -S

Compare the contents of the two output files, specifically the 'options passed' section:

$ diff -u k8-sse3.s native.s

-# options passed: -D_GNU_SOURCE k8-sse3.cc -D_FORTIFY_SOURCE=2
-# -march=k8-sse3 -fverbose-asm
+# options passed: -D_GNU_SOURCE native.cc -D_FORTIFY_SOURCE=2
+# -march=k8-sse3 -msahf --param l1-cache-size=64 --param
+# l1-cache-line-size=64 --param l2-cache-size=512 -mtune=k8 -fverbose-asm

We can easily see that in addition to the flags enabled by -march=k8-sse3, -march=native enables the following: -msahf --param l1-cache-size=64 --param l1-cache-line-size=64 --param l2-cache-size=512 -mtune=k8

These flags can be added to make.conf along with -march=k8-sse3.

Bam. -march=native without -march=native.


Now this seems like a pretty tidy little solution to me, but my suspicion is this: If it's really that simple, then why doesn't distcc do it automatically? What would stop the devs from instructing distcc to gather the target flags from -march=native and pass them to all the compile nodes. Why is it just accepted that march=native is just incompatible with distcc, end-of-story?

Or is there a huge gaping flaw in my analysis here?


Well, this all made sense to me, but in practice it didn't work (for me at least)
I did as explained above, but when I set the flags from the diff, emerge fails with an error (see bug 437818 for the details). I tried it a couple of times, just can't get it to work.

Dave
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 36544
Location: 56N 3W

PostPosted: Wed Oct 10, 2012 6:13 pm    Post subject: Reply with quote

dacid,

Post your make.conf from the netbook.
Post your distcc setup file drom the netbook.

Confirm you have *identical* versions of gcc on the netbooks and the helper(s).

You may use pump mode too. pump emerge ... so helpers do preprocessing and not just compiling.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 5432

PostPosted: Wed Oct 10, 2012 7:15 pm    Post subject: Reply with quote

dacid wrote:

I did as explained above, but when I set the flags from the diff, emerge fails with an error (see bug 437818 for the details). I tried it a couple of times, just can't get it to work.
Dave


From what i see on your bug report, the bug shouldn't have been mark duplicate but invalid.
You are getting c compiler cannot create executable error because of cflags/cxxflags errors.
Quote:
CFLAGS="march=core2 -O2 -mtune=generic -pipe"
CXXFLAGS="march=core2 -O2 -mtune=generic -pipe"


It's CFLAGS="-march..." notice -march= not march=

What you get is that gcc on any invalid flags (except -no"family" type) throw an error
And all configure try to see if gcc produce a file and watch if the file exist then. If not, C compiler cannot produce executable.
For your case, gcc refuse because march= is an invalid flags.
Back to top
View user's profile Send private message
Slated
n00b
n00b


Joined: 16 Oct 2010
Posts: 12

PostPosted: Sat Dec 03, 2016 9:41 pm    Post subject: Reply with quote

Apologies for resurrecting an old thread, but it's a popular search result, so needs clarification for anyone who might stumble upon it.

The only bit of any actual relevance here is "options enabled", not "options passed", and AFAICT all of the arch specific "options enabled" are determined entirely by "-march=<arch>", so verbosely regurgitating the full arch options list is completely redundant, and might actually cause problems.

The only important step you need to do, to ensure compatible and fully arch-optimised cross-compilation, is to ensure that each machine is using exactly the same version of GCC compiled with the same options, and pass exactly the same (and correct) "-march" to each of them. That is all.

To verify this, simply check the "options enabled" section of the assembler code file in the above examples, prepared on different machines with different architectures, but with the same "-march" value and exactly the same version and spec of GCC, and you will see that this section is identical in each case.

That is the only bit that is actually used for compilation, and the only bit that actually affects the end result. Everything else is completely redundant. To make this even clearer, if it isn't in "options enabled" then, surprisingly, it isn't used. Period. This should be glaringly obvious, but apparently it isn't to everyone, as this and many other threads across the internet demonstrate.

In case you have multiple machines with many different versions of GCC, and you don't want to (or can't) keep them synchronised, then you should use crossdev to maintain a consistent toolchain across the network. This will enable you to cross-compile consistently on the one hand, without affecting the respective native compilers and systems on the other.
_________________
Homer
http://slated.org
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 5432

PostPosted: Sun Dec 04, 2016 2:28 am    Post subject: Reply with quote

Well, you are the one that actually need clarifications.

Quote:
The only bit of any actual relevance here is "options enabled", not "options passed", and AFAICT all of the arch specific "options enabled" are determined entirely by "-march=<arch>", so verbosely regurgitating the full arch options list is completely redundant, and might actually cause problems.

No, the options enable are determined by gcc devs for each arch, but also by options passed.
Why? because for whatever reason you can use an -arch=this that have mmx enable per default for that arch, and the option pass -mno-mmx will disable mmx usage for this arch, even if per default mmx was enable.
But it's not only to disable option, but actually can enable some more, see sse2 below, making your "relevance of options enabled over options passed" wrong.
I also fail to see any danger of redundant passing options: if -msse2 is enable per default for your arch, giving gcc -msse2 will not cause any trouble. The only danger in it is typing error, but copy/paste is your friend.

Quote:

The only important step you need to do, to ensure compatible and fully arch-optimised cross-compilation, is to ensure that each machine is using exactly the same version of GCC compiled with the same options, and pass exactly the same (and correct) "-march" to each of them. That is all

Sorry that is wrong again, you will get fully compatible arch, but not fully arch-optimised result.
Arch is not a cpu type, but a cpu family, many cpu in the same family have different options, and it could be really different, it is possible to have two cpu from the same family but one handling sse2 while the other not.
Because of this, if gcc devs see any cpu of that family doesn't have sse2, the default (because safety) is to disable sse2 for the whole family ; and checking option enable, you will see -mno-sse2 set.
And while building -march=tada on the two cpu result in both results as valid for that family, one cpu that is sse2 capable will endup with code with no sse2 in it because the family is know to have cpu in it that lack sse2. So yes, in both case you endup with code optimize for "tada" family, but no, in no way you endup with code optimize fully for "tada" family. In fact, you endup with the minimal optimized family code (total invert of your "fully arch-optimized" result, as it is "minimal default arch-optimized").

Quote:
To verify this, simply check the "options enabled" section of the assembler code file in the above examples, prepared on different machines with different architectures, but with the same "-march" value and exactly the same version and spec of GCC, and you will see that this section is identical in each case.

You are not asking gcc to show the options set optimize for your arch or cpu, but the options default set for the family. No matter the arch or cpu type you have, the arch family default options are set by gcc devs.

Quote:
That is the only bit that is actually used for compilation, and the only bit that actually affects the end result. Everything else is completely redundant. To make this even clearer, if it isn't in "options enabled" then, surprisingly, it isn't used. Period. This should be glaringly obvious, but apparently it isn't to everyone, as this and many other threads across the internet demonstrate.

Yes if it is not set it will not be use, but that's the whole point of giving gcc hint about what to enable when it is not enable as default for a family.

You misunderstood that -march=native is detecting the family options and (because it also imply -mtune) the cpu options.
Again while -march=tada will not enable sse2 per default, the output of -march=native in this case should output the family options + family optimized options (so, in one cpu the output will remain the same, but for the other cpu that handle sse2, gcc will enable -msse2) ; and also the cpu optimized options, the easiest ones to catch are cache size optimization, but nothing is limiting gcc devs about the trick for your cpu, let's pickup a "simple" example, say you have a core2 class cpu with a flaw coprocessor, it's then doable that for that cpu that -mtune=native will swap -mfpmath=387 for -mfpmath=sse to limit the flaw in it and optimize the code better for that processor.

As you see, -march=core2 is not giving the user the same as -march=native even -march=native will tell user to use -march=core2 because he have a core2 class cpu, gcc will detect the features of the cpu and family and enable them, while -march=core2 will only enable the least common denominators of all core2 class cpu, making a valid code for all core2 cpu, but the least optimize for your cpu.

If we follow your weird logic, then you should just tell anyone to not care about their arch type and just goes with : is your cpu running in 32 or 64, for the first use -march=i686 and the second case, just use -march=x86-64.
It will produce even less optimized code, but code even more compatible with many cpus.

This is also why distcc disallow -march=native, distcc don't know the gcc you use and pass the cflags you set to it.
But distcc devs aren't dumb, and know what -march=native will do, it will detect the cpu and family host, and enable optimization for it, making a big big surprise to a user, because once your pentium cpu is asking your corei7 to build code using -march=native, the result would be code optimize for that corei7, something your pentium will really not like.

So, sorry Slated, it is important for a user to pass the right options for its cpu and family to gcc, he can get them easy by running -march=native test and use the output to build it's cflag for distcc. If he only grab the -march= result from it, he will only produce the least optimized code for his cpu.
This is not really ricing, but just use what you have fully. Else let's all use -march=i686/x86-64 and generic!
Back to top
View user's profile Send private message
Slated
n00b
n00b


Joined: 16 Oct 2010
Posts: 12

PostPosted: Mon Dec 05, 2016 12:18 am    Post subject: Reply with quote

That isn't the use case I'm referring to. I'm responding to the specific case where taking the "options passed" from a "gcc -march=native" test is presumed to somehow affect a different result than the actual "options enabled" from that same test.

Naturally you can add other options, which are either not arch-specific or are for architectures that are not currently defined, but that has nothing to do with the results of the test, since you need to know more than the test results will tell you in order to make that determination.

As an aside, I'm also challenging the assertion that "-march=native" returns a profile that is not defined as an arch that can be simply replicated with "-march=<arch>". If that were so, then "-march=native" simply wouldn't work, since it would have no arch definitions from which to produce its profile.

But again, if you know something about your processor that the gcc developers don't, then you can manually add that option, assuming it's actually supported by gcc ("options enabled" will tell you).

To make this even clearer, "-march=native" will not somehow magically produce a unique profile that does not already exist as a predefined profile which can simply be called with "-march=<arch>" ("Using -mtune=native produces code optimized for the local machine under the constraints of the selected instruction set").

So really, the only purpose of altering the results of "-march=native" is to disable features, since you cannot somehow force your CPU to execute instructions that it has no support for, and again why would you want to do that, unless you have discovered an undocumented bug in that instruction set, or you have a CPU that is not defined by gcc profiles (in which case "-march=native" will not help you)?

Gcc profiles are more fine-grained than you're suggesting. They encompass far more than just top level hierarchies. Nowhere did I suggest that people should be using "i686" as a profile.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum