Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Where to put a HowTo?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
Goverp
Guru
Guru


Joined: 07 Mar 2007
Posts: 589

PostPosted: Fri Aug 17, 2018 7:07 pm    Post subject: Where to put a HowTo? Reply with quote

I was thinking of writing up the configuration to let my laptop be a "slave" user of my desktop's "master" Gentoo setup, despite having dissimilar chips. (It involves nfs, rsync and binary packages.)

I assumed the appropriate place to put it would be the Gentoo wiki, but I can see neither an appropriate slot in the article hierarchy, nor a "How to" template - everything is fitted into hardware or software. Should I look elsewhere, am I missing something, or would it make sense to start a How to section (and create a template)?

Suggestions please!
_________________
Greybeard
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 41298
Location: 56N 3W

PostPosted: Fri Aug 17, 2018 8:17 pm    Post subject: Reply with quote

Goverp,

For the Wiki, join #gentoo-wiki on freenode and ask there.

It could be a post in Documentation, Tips and Tricks but that's harder to search.

I would create the page in your userspace on the Wiki. Only you can edit it there. Others can use the talk page.
When its well advanced, ask about where it fits.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5122
Location: The Peanut Gallery

PostPosted: Fri Aug 24, 2018 3:34 pm    Post subject: Reply with quote

What Neddy said, though personally I'd start with a post in Docs, Tips and Tricks, since forum bbcode is so much easier to work with, and you can get feedback from other users on the overall content. (And it then remains a good base doc.)
Back to top
View user's profile Send private message
Goverp
Guru
Guru


Joined: 07 Mar 2007
Posts: 589

PostPosted: Sat Aug 25, 2018 9:35 am    Post subject: Reply with quote

I'll (eventually) go with Neddy's solution - I know about as much BB code as wiki. As for feedback, I'll stick my finders in my ears and say "I can't hear you" :-)
_________________
Greybeard
Back to top
View user's profile Send private message
josephg
l33t
l33t


Joined: 10 Jan 2016
Posts: 688

PostPosted: Sat Aug 25, 2018 12:06 pm    Post subject: Re: Where to put a HowTo? Reply with quote

Goverp wrote:
I was thinking of writing up the configuration to let my laptop be a "slave" user of my desktop's "master" Gentoo setup, despite having dissimilar chips. (It involves nfs, rsync and binary packages.)

Please do! I, for one, would be very interested, as I am now at that stage where I need to have a proper strategy for expanding my gentoo across with minimum cruft.
_________________
"Growth for the sake of growth is the ideology of the cancer cell." Edward Abbey
Back to top
View user's profile Send private message
Goverp
Guru
Guru


Joined: 07 Mar 2007
Posts: 589

PostPosted: Tue Aug 28, 2018 6:39 pm    Post subject: Reply with quote

Try this.
_________________
Greybeard
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5122
Location: The Peanut Gallery

PostPosted: Fri Aug 31, 2018 2:41 pm    Post subject: Reply with quote

Goverp wrote:
I'll (eventually) go with Neddy's solution - I know about as much BB code as wiki.
Whatever works for you.
I'd just point out that you've been using bbcode for over a decade already ;-) (It's the forum tags in phpBB.)
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 41298
Location: 56N 3W

PostPosted: Fri Aug 31, 2018 7:39 pm    Post subject: Reply with quote

Goverp,

The configurations above use --march=native on both master and slave. This works for me. Gcc is at liberty to generate code exploiting hardware features on the master that might not exist on the slave. As it works for me, it appears that it only generates feature-sensitive code if the appropriate value is present in CPU_FLAGS_X86, and portage detects packages sensitive to these flags and compiles separately for the slave. Truly magic. I expected this to end in tears.

As --march=native works for you, you are lucky.
The correct approach is to run gcc -### -E - -march=native 2>&1 | sed -r '/cc1/!d;s/(")|(^.* - )|( -mno-[^\ ]+)//g' on all participating machines then set CFLAGS to the lowest common denominator.

CFLAGS gives gcc permission to use the defined instructions, it may choose not to.

CPU_FLAGS_X86, on the other hand, is a USE expand that enables hand optimised code using the listed instruction sets, if a package provides some.
Using incompatible CPU_FLAGS_X86 will break affected code.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Goverp
Guru
Guru


Joined: 07 Mar 2007
Posts: 589

PostPosted: Sat Sep 01, 2018 7:18 am    Post subject: Reply with quote

Neddy,

Thanks for that. One day I'll learn sed and understand what you wrote :-)

As I wrote, I expected it to all end in tears, but it doesn't. Probably there are few cases where the 3DNow! codes are useful; perhaps even gcc never bothered to code for them.

I'll update the item to include your code and an appropriate discussion.

As to CPU_FLAGS_X86, they can differ between the master and slave machines (assuming they're appropriate in each case). Portage spots the resulting differing USE flags, and compiles different versions of packages sensitive to said flags on the slave. Hence the occasional line of green amongst the purple in emerge output (though you can't see it, as I haven't posted any yet. (I need to run emerge update again, and scrape the output, and work out how to colour it in the wiki page, assuming that's not automatic.)

Thanks again.
_________________
Greybeard
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 41298
Location: 56N 3W

PostPosted: Sat Sep 01, 2018 10:30 am    Post subject: Reply with quote

Goverp,

Yep, by default, portage will not merge a binary that was built with different use flags to those requested, so you are protected against USE flag differences. That includes CPU_FLAGS_X86 differences as CPU_FLAGS_X86 is a USE expand.

In practice, very few packages offer explicitly enabled hand optimised code segments. Multimedia and crypto packages are typical consumers.
I didn't write that one liner, I found it on stack exchange, a long time ago.

For completeness, many newer AMD CPUs that don't advertise 3DNow! and 3DNowext can execute the instructions anyway.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 12707

PostPosted: Sat Sep 01, 2018 4:29 pm    Post subject: Reply with quote

Goverp wrote:
Thanks for that. One day I'll learn sed and understand what you wrote :-)
How about today? Let's dissect it:
NeddySeagoon wrote:
gcc -### -E - -march=native 2>&1 | sed -r '/cc1/!d;s/(")|(^.* - )|( -mno-[^\ ]+)//g'
  • -r: per info sed, this is a short name for --regexp-extended. It changes how sed interprets the expression you give it.
  • '/cc1/!d;s/(")|(^.* - )|( -mno-[^\ ]+)//g' - this is a complicated expression, but we can break it down further.
    • '/cc1/!d' - if the line matches cc1, this instruction applies. When it matches, nothing happens. When it fails to match (!), command d applies: the line is deleted.
    • ; - separates multiple sed commands in a single argument.
    • s/(")|(^.* - )|( -mno-[^\ ]+)//g - this is a multi-way (|) match with substitution; what it matches gets replaced by the empty string (since there is no text between the second and third slashes); it may match multiple times in a line, and each time will substitute (g).
      • (") - Match a double quote
      • (^.* - ) - Match zero-or-more anything at the start of the line, then a literal space, dash, space.
      • ( -mno-[^\ ]+) - Match literal space, -mno-, then one-or-more (+) anything except (^) backslash or space.
This is a reconstructed explanation from reading through the expression. I believe it to be right, but I am not the original author of the expression. It makes a bit more sense if you run the gcc part alone without filtering it through sed, so that you can see what is fed into sed. If you want more detail, ask.
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5122
Location: The Peanut Gallery

PostPosted: Sat Sep 01, 2018 11:43 pm    Post subject: Reply with quote

Hu wrote:
Let's dissect it:
NeddySeagoon wrote:
gcc -### -E - -march=native 2>&1 | sed -r '/cc1/!d;s/(")|(^.* - )|( -mno-[^\ ]+)//g'
[list][*]-r: per info sed, this is a short name for --regexp-extended. It changes how sed interprets the expression you give it.
One should use the -E flag instead, since that is specified by POSIX, so it works everywhere. GNU has always supported it, following BSD, but never documented it.
(It is consistent with grep -E.)
Hu wrote:
If you want more detail, ask.
Yeah, ask #sed ;) on IRC: chat.freenode.org or .net
Good explanation, still. Note that sed -n with a q on the matching line after substs and p, is more efficient generally.

Just to summarise (for new users): it's stripping gcc "-no-foo" options, from the line used to drive the base C compiler (which is derived from configured defn files. Zorry is good on those.) The rest are permitted on the machine in question (and must still be matched with other boxes.)
You usually don't want any cache-line settings, for example (unless you are running the exact same CPU on all machines.) cache line-sizes don't tend to match, even where the machines are both eg: core2 or w/e.
You get rid of the -no-foo ones because there are so many of them, essentially forming a report of what the compiler knows about the native build-machine.
NeddySeagoon wrote:
CFLAGS gives gcc permission to use the defined instructions, it may choose not to.
Yes, but equally it presumes you know what you're doing; so if you tell it to use say SSE4.1, it has no way of checking whether the runtime CPU supports that. (Unless it's built for a totally different platform.)
You covered this with "lowest common denominator", ofc: I just think it bears repeating that the flags are the admin's responsibility.
By definition it is not the compiler's job to know where the binary will eventually be run; and illegal insns at runtime are a distinct possibility, if incorrect flags are given.
NeddySeagoon wrote:
Yep, by default, portage will not merge a binary that was built with different use flags to those requested, so you are protected against USE flag differences. That includes CPU_FLAGS_X86 differences as CPU_FLAGS_X86 is a USE expand.
Yeah --binpkg-respect-use is set automatically now. (man emerge)
Don't forget FEATURES="buildpkg binpkg-multi-instance binpkg-logs" on the main CBUILD machine. cf: man make.conf
NeddySeagoon wrote:
For completeness, many newer AMD CPUs that don't advertise 3DNow! and 3DNowext can execute the instructions anyway.
Yes, though amd64/em64t requires SSE, so you have 16 128-bit xmm0..15 floating-point registers (which can used as vectors) on any x86_64 machine (without getting into optional AVX ymm, and AVX2 zmm 512-bit.)
Useful for -m32, though not so much on x32 vs x86.
--
"SSE" = Streaming SIMD Extensions. SIMD has been around since at least the 1970s, if not before (eg: Z80's LDIR CPIR, and DR forms) and means "Single-Insn, Multiple Data".
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum