Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
CFLAGS, -march, and distcc
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
metafarion
n00b
n00b


Joined: 15 Mar 2012
Posts: 6
Location: Madison, WI

PostPosted: Thu Mar 15, 2012 4:21 am    Post subject: CFLAGS, -march, and distcc Reply with quote

Ok, so for awhile now I've had to choose between using -march=native in my make.conf or having additional compiling power with distcc (really helps on the netbook). My understanding is that you can't use march=native in a distcc environment because gcc will pull in different "native" CFLAGS from all the different distcc nodes depending on their individual architectures. Fair enough.

However, it's also been demonstrated (see here and other pages like it) that -march=native enables additional CFLAGS which -march=atom (for my netbook for example) does not, and therefore you may end up with a more optimized build using march=native.

Ignore for the moment, the argument of whether or not this is true and that this is a dumb Gentoo ricer question, would it not be possible to get the effect of -march=native simply by doing a gcc test with -march=native set, recording the CFLAGS and setting them all explicitly in make.conf? Then you would be free to enable distcc without worrying about CFLAGS from other cpu types creeping in maybe? Am I missing any crucial details in this plan? Does -march affect more than just CFLAGS? Other things that can't be explicitly set in make.conf?


Last edited by metafarion on Thu Mar 15, 2012 4:39 am; edited 1 time in total
Back to top
View user's profile Send private message
cach0rr0
Moderator
Moderator


Joined: 13 Nov 2008
Posts: 4122
Location: Houston, Republic of Texas

PostPosted: Thu Mar 15, 2012 4:36 am    Post subject: Reply with quote

i dont know this is a correct or even consensus way, but what ive done is keep two sets of CFLAGS handy in make.conf for when im doing something that may benefit from distcc:

Code:

CFLAGS="-march=native -O2 -pipe"
#CFLAGS="-march=core2 -m64 -m80387 -m96bit-long-double -malign-stringops -mcx16 -mfancy-math-387 -mfp-ret-in-387 -mfused-madd -mglibc -mhard-float -mieee-fp -mpush-args -mred-zone -msahf -msse -msse2 -msse3 -msse4.1 -mssse3 -mstackrealign -mtls-direct-seg-refs -O2 -pipe"


obtained via looking at output of:

Code:

gcc -Q -march=native --help=target
gcc -### -march=native -E /usr/include/stdlib.h 2>&1 | grep "/usr/libexec/gcc/.*cc1"


hope that helps (?)
im sure others will comment
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash
Back to top
View user's profile Send private message
metafarion
n00b
n00b


Joined: 15 Mar 2012
Posts: 6
Location: Madison, WI

PostPosted: Mon Mar 19, 2012 5:12 am    Post subject: Reply with quote

That sounds very reasonable. Then is it your understanding as well that -march= basically invokes a preset of CFLAGS and that's it? Does it do anything else?
Back to top
View user's profile Send private message
cach0rr0
Moderator
Moderator


Joined: 13 Nov 2008
Posts: 4122
Location: Houston, Republic of Texas

PostPosted: Mon Mar 19, 2012 7:01 am    Post subject: Reply with quote

metafarion wrote:
That sounds very reasonable. Then is it your understanding as well that -march= basically invokes a preset of CFLAGS and that's it? Does it do anything else?


my understanding is it'll attempt to auto-determine the best march for your CPU, and tack on any additional flags your CPU might support.
I mention those separately because "march" itself will expand to other things. So for example having -march=amdfam10 might enable a chunk of things for amdfam10, and then on top of that by virtue of having used native, other things are added as well

caveat being that may be a *wrong* understanding, but at least it'd be a wrong understanding ive derived from attempts to research it rather than guesswork!
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash
Back to top
View user's profile Send private message
Hu
Watchman
Watchman


Joined: 06 Mar 2007
Posts: 9057

PostPosted: Mon Mar 19, 2012 10:32 pm    Post subject: Reply with quote

Also, using -march=native will prevent distcc from sending the work to remote machines. It has no way to guarantee that the remote machine will produce the same expanded value as the local machine, so the only safe way for distcc to handle it is to build everything locally. If you intend to use distcc, look up what -march=native does for you and use those options explicitly.
Back to top
View user's profile Send private message
metafarion
n00b
n00b


Joined: 15 Mar 2012
Posts: 6
Location: Madison, WI

PostPosted: Fri Mar 23, 2012 12:41 am    Post subject: Reply with quote

Alright, so based on everything I've heard so far and everything I've read about gcc, I think I have a method for emerging with distcc while still gaining the benefits of -march=native. I'm going to run through this with an Athlon64 X2 3800+ system for criticism and speculation.

The first order of business is to determine what flags -march=native adds to -march=k8-sse3 (the suggested arch for my CPU)

Create dummy files for gcc to process with each -march and send the verbose output to respective .s files:

$ touch native.cc
$ touch k8-sse3.cc
$ gcc -fverbose-asm -march=native native.cc -S
$ gcc -fverbose-asm -march=k8-sse3 k8-sse3.cc -S

Compare the contents of the two output files, specifically the 'options passed' section:

$ diff -u k8-sse3.s native.s

-# options passed: -D_GNU_SOURCE k8-sse3.cc -D_FORTIFY_SOURCE=2
-# -march=k8-sse3 -fverbose-asm
+# options passed: -D_GNU_SOURCE native.cc -D_FORTIFY_SOURCE=2
+# -march=k8-sse3 -msahf --param l1-cache-size=64 --param
+# l1-cache-line-size=64 --param l2-cache-size=512 -mtune=k8 -fverbose-asm

We can easily see that in addition to the flags enabled by -march=k8-sse3, -march=native enables the following: -msahf --param l1-cache-size=64 --param l1-cache-line-size=64 --param l2-cache-size=512 -mtune=k8

These flags can be added to make.conf along with -march=k8-sse3.

Bam. -march=native without -march=native.


Now this seems like a pretty tidy little solution to me, but my suspicion is this: If it's really that simple, then why doesn't distcc do it automatically? What would stop the devs from instructing distcc to gather the target flags from -march=native and pass them to all the compile nodes. Why is it just accepted that march=native is just incompatible with distcc, end-of-story?

Or is there a huge gaping flaw in my analysis here?
Back to top
View user's profile Send private message
Hu
Watchman
Watchman


Joined: 06 Mar 2007
Posts: 9057

PostPosted: Fri Mar 23, 2012 2:35 am    Post subject: Reply with quote

You ran several processes and performed non-trivial text parsing on the output to derive that list of flags. It is much easier to document that users should avoid -march=native than to implement the corresponding functionality in code. Also, distcc is mostly stateless. It would add significant overhead to have it compute that expression every time.
Back to top
View user's profile Send private message
piedar
Tux's lil' helper
Tux's lil' helper


Joined: 09 Aug 2010
Posts: 80

PostPosted: Sun May 06, 2012 5:24 pm    Post subject: Reply with quote

I have some related distcc questions and rather than start another thread, I'll ask them here. I've roughly followed the distcc docs, but they're outdated and more than a little confusing.

Do the CFLAGS on machines running distccd affect compilation on machines requesting jobs? I have two machines, let's refer to them by their processors: core2 and atom. I want the core2 to compile for the atom, but not the other way around. So in my make.conf on atom, I used metafarion's method to get CFLAGS equivalent to -march=native. But can I leave -march=native in core2's CFLAGS or does that affect the builds for atom? Does core2 need FEATURES=distcc and/or distcc added to its PATH? Does atom need to have the distccd service running (seems to work without it)?

Right now, I have set core2's CFLAGS == atom's CFLAGS just in case.


Bonus: Is there a way to do --load-average per machine?
Back to top
View user's profile Send private message
Hu
Watchman
Watchman


Joined: 06 Mar 2007
Posts: 9057

PostPosted: Sun May 06, 2012 6:04 pm    Post subject: Reply with quote

piedar wrote:
Do the CFLAGS on machines running distccd affect compilation on machines requesting jobs?
No. The volunteers receive a command line and preprocessed source text from the master, run the compiler specified by the master on the received text, and return the result. They do not inspect local Portage configuration.
piedar wrote:
But can I leave -march=native in core2's CFLAGS or does that affect the builds for atom? Does core2 need FEATURES=distcc and/or distcc added to its PATH? Does atom need to have the distccd service running (seems to work without it)?
As I stated earlier in the thread, if you use -march=native, that will automatically prevent the compilation from being distributed. The master must specify FEATURES=distcc. The volunteers must run distccd, or sshd if you use ssh transport for distcc.
piedar wrote:
Bonus: Is there a way to do --load-average per machine?
I am not aware of any tools which collect and use the load information to achieve this. It would be nice, but it requires integrating the distribution phase, the host list, and the build tool (Make, SCons, waf, etc.) such that the build tool knows when it can create a new job, and where that job can run without exceeding any per-host limits.
Back to top
View user's profile Send private message
metafarion
n00b
n00b


Joined: 15 Mar 2012
Posts: 6
Location: Madison, WI

PostPosted: Fri May 18, 2012 9:27 pm    Post subject: Reply with quote

Hu wrote:
piedar wrote:
But can I leave -march=native in core2's CFLAGS or does that affect the builds for atom? Does core2 need FEATURES=distcc and/or distcc added to its PATH? Does atom need to have the distccd service running (seems to work without it)?
As I stated earlier in the thread, if you use -march=native, that will automatically prevent the compilation from being distributed. The master must specify FEATURES=distcc. The volunteers must run distccd, or sshd if you use ssh transport for distcc.


If I'm understanding Piedar correctly, he's asking if core2 needs to have march=native removed from its make.conf in order to be a viable distcc node for atom and I believe the answer is no. Everything you set in a given host's make.conf affects only its own emerge process and not that of machines connecting to it via distcc.
Back to top
View user's profile Send private message
dacid
n00b
n00b


Joined: 29 May 2004
Posts: 49
Location: Charlotte, NC, USA

PostPosted: Wed Oct 10, 2012 5:57 pm    Post subject: Reply with quote

metafarion wrote:
Alright, so based on everything I've heard so far and everything I've read about gcc, I think I have a method for emerging with distcc while still gaining the benefits of -march=native. I'm going to run through this with an Athlon64 X2 3800+ system for criticism and speculation.

The first order of business is to determine what flags -march=native adds to -march=k8-sse3 (the suggested arch for my CPU)

Create dummy files for gcc to process with each -march and send the verbose output to respective .s files:

$ touch native.cc
$ touch k8-sse3.cc
$ gcc -fverbose-asm -march=native native.cc -S
$ gcc -fverbose-asm -march=k8-sse3 k8-sse3.cc -S

Compare the contents of the two output files, specifically the 'options passed' section:

$ diff -u k8-sse3.s native.s

-# options passed: -D_GNU_SOURCE k8-sse3.cc -D_FORTIFY_SOURCE=2
-# -march=k8-sse3 -fverbose-asm
+# options passed: -D_GNU_SOURCE native.cc -D_FORTIFY_SOURCE=2
+# -march=k8-sse3 -msahf --param l1-cache-size=64 --param
+# l1-cache-line-size=64 --param l2-cache-size=512 -mtune=k8 -fverbose-asm

We can easily see that in addition to the flags enabled by -march=k8-sse3, -march=native enables the following: -msahf --param l1-cache-size=64 --param l1-cache-line-size=64 --param l2-cache-size=512 -mtune=k8

These flags can be added to make.conf along with -march=k8-sse3.

Bam. -march=native without -march=native.


Now this seems like a pretty tidy little solution to me, but my suspicion is this: If it's really that simple, then why doesn't distcc do it automatically? What would stop the devs from instructing distcc to gather the target flags from -march=native and pass them to all the compile nodes. Why is it just accepted that march=native is just incompatible with distcc, end-of-story?

Or is there a huge gaping flaw in my analysis here?


Well, this all made sense to me, but in practice it didn't work (for me at least)
I did as explained above, but when I set the flags from the diff, emerge fails with an error (see bug 437818 for the details). I tried it a couple of times, just can't get it to work.

Dave
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 32104
Location: 56N 3W

PostPosted: Wed Oct 10, 2012 6:13 pm    Post subject: Reply with quote

dacid,

Post your make.conf from the netbook.
Post your distcc setup file drom the netbook.

Confirm you have *identical* versions of gcc on the netbooks and the helper(s).

You may use pump mode too. pump emerge ... so helpers do preprocessing and not just compiling.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
krinn
Advocate
Advocate


Joined: 02 May 2003
Posts: 4296

PostPosted: Wed Oct 10, 2012 7:15 pm    Post subject: Reply with quote

dacid wrote:

I did as explained above, but when I set the flags from the diff, emerge fails with an error (see bug 437818 for the details). I tried it a couple of times, just can't get it to work.
Dave


From what i see on your bug report, the bug shouldn't have been mark duplicate but invalid.
You are getting c compiler cannot create executable error because of cflags/cxxflags errors.
Quote:
CFLAGS="march=core2 -O2 -mtune=generic -pipe"
CXXFLAGS="march=core2 -O2 -mtune=generic -pipe"


It's CFLAGS="-march..." notice -march= not march=

What you get is that gcc on any invalid flags (except -no"family" type) throw an error
And all configure try to see if gcc produce a file and watch if the file exist then. If not, C compiler cannot produce executable.
For your case, gcc refuse because march= is an invalid flags.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum