make.conf to support two differnet cpus

[DaggyStyle]

Greetings,

I'm trying to setup a gentoo chroot that will generate bins that will wor on both ryzen gen 2 and a kaby lake based Pentium setups.
after reviewing both cpus, I've came to a conclsuing I need to define this:

Code: Select all

COMMON_FLAGS="-O2 -pipe -march=x86-64"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"
CPU_FLAGS_X86="aes mmx popcnt rdrand sse sse2 sse4_1 sse4_2 ssse3"

is there anything wrong/missing?

thanks.

[mike155]

It will probably work. But if you care about performance, "x86-64" might be too conservative. "x86-64" won't generate code for SSE and AVX, for example.

See:

Code: Select all

LANG=C gcc -Q -O2 --help=target -march=x86-64

[Anon-E-moose]

Ryzen flags will work for kaby lake cpus, if you remove the sse4a flag (ryzen only)

Ryzen CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3"
KBlake CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3"

as far as a common arch, compatible with both, not sure what I'd choose.

Personally, I do a regular compile for the desktop (ryzen) with a chroot where I create packages for the laptop, so many things get a second compile.
Some things are only on one machine, some on the other, ie. the hardware differences, so I found it hard to create both from a common point cleanly.
I only do the compile for the laptop weekly, as it's a secondary machine and thus not used all the time.

[netfab]

Here is what I do since years to build packages that runs on 3 different CPU :
- Core i5-2500K
- Core i3-3220
- Xeon E5603

Run this on each system, and save the output :

Code: Select all

$ gcc -### -march=native -E /usr/include/stdlib.h 2>&1 | grep "/usr/libexec/gcc/.*cc1"

Once you have each output, you must determine which flags to enable (I wrote a bash script for this).
Here is a part of my make.conf used on each system, created with gcc 9.3.0 outputs :

Code: Select all

# gcc 9.3.0
I_SETS_ON="-mmmx -msse -msse2 -msse3 -mssse3 -mcx16 -msahf -mpclmul -mpopcnt"
I_SETS_ON="${I_SETS_ON} -msse4.2 -msse4.1 -mfxsr"

I_SETS_OFF="-mno-3dnow -mno-sse4a -mno-movbe -mno-aes -mno-sha -mno-abm -mno-lwp"
I_SETS_OFF="${I_SETS_OFF} -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-sgx -mno-bmi2"
I_SETS_OFF="${I_SETS_OFF} -mno-pconfig -mno-wbnoinvd -mno-tbm -mno-avx -mno-avx2"
I_SETS_OFF="${I_SETS_OFF} -mno-lzcnt -mno-rtm -mno-hle -mno-rdrnd -mno-f16c"
I_SETS_OFF="${I_SETS_OFF} -mno-fsgsbase -mno-rdseed -mno-prfchw -mno-adx -mno-xsave"
I_SETS_OFF="${I_SETS_OFF} -mno-xsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd"
I_SETS_OFF="${I_SETS_OFF} -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec"
I_SETS_OFF="${I_SETS_OFF} -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl"
I_SETS_OFF="${I_SETS_OFF} -mno-avx512ifma -mno-avx512vbmi -mno-avx5124fmaps"
I_SETS_OFF="${I_SETS_OFF} -mno-avx5124vnniw -mno-clwb -mno-mwaitx -mno-clzero -mno-pku"
I_SETS_OFF="${I_SETS_OFF} -mno-rdpid -mno-gfni -mno-shstk -mno-avx512vbmi2 -mno-avx512vnni"
I_SETS_OFF="${I_SETS_OFF} -mno-vaes -mno-vpclmulqdq -mno-avx512bitalg -mno-movdiri -mno-movdir64b"
I_SETS_OFF="${I_SETS_OFF} -mno-waitpkg -mno-cldemote -mno-ptwrite"

COMMON_FLAGS="-march=x86-64 -mtune=generic -O2 -pipe"
COMMON_FLAGS="${COMMON_FLAGS} ${I_SETS_ON} -fstack-protector-strong ${I_SETS_OFF}"

CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"

Then you can compile on one system, and deploy packages on the others using -k/-K emerge options. You can also use distcc.

[DaggyStyle]

It will probably work. But if you care about performance, "x86-64" might be too conservative. "x86-64" won't generate code for SSE and AVX, for example.

See:

Code: Select all

LANG=C gcc -Q -O2 --help=target -march=x86-64

here is the output of cpuinfo:

Code: Select all

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 158
model name      : Intel(R) Pentium(R) CPU G4560 @ 3.50GHz
stepping        : 9
microcode       : 0x84
cpu MHz         : 800.492
cache size      : 3072 KB
physical id     : 0
siblings        : 4
core id         : 1
cpu cores       : 2
apicid          : 3
initial apicid  : 3
fpu             : yes
fpu_exception   : yes
cpuid level     : 22
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave rdrand lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust smep erms invpcid mpx rdseed smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit srbds
bogomips        : 7002.48
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:

as you can see, there is not avx support so I don't see any reason to add it.

[DaggyStyle]

Ryzen flags will work for kaby lake cpus, if you remove the sse4a flag (ryzen only)

Ryzen CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3"
KBlake CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3"

as far as a common arch, compatible with both, not sure what I'd choose.

Personally, I do a regular compile for the desktop (ryzen) with a chroot where I create packages for the laptop, so many things get a second compile.
Some things are only on one machine, some on the other, ie. the hardware differences, so I found it hard to create both from a common point cleanly.
I only do the compile for the laptop weekly, as it's a secondary machine and thus not used all the time.

which cpu is the kblake?

[DaggyStyle]

Here is what I do since years to build packages that runs on 3 different CPU :
- Core i5-2500K
- Core i3-3220
- Xeon E5603

Run this on each system, and save the output :

Code: Select all

$ gcc -### -march=native -E /usr/include/stdlib.h 2>&1 | grep "/usr/libexec/gcc/.*cc1"

Once you have each output, you must determine which flags to enable (I wrote a bash script for this).
Here is a part of my make.conf used on each system, created with gcc 9.3.0 outputs :

Code: Select all

# gcc 9.3.0
I_SETS_ON="-mmmx -msse -msse2 -msse3 -mssse3 -mcx16 -msahf -mpclmul -mpopcnt"
I_SETS_ON="${I_SETS_ON} -msse4.2 -msse4.1 -mfxsr"

I_SETS_OFF="-mno-3dnow -mno-sse4a -mno-movbe -mno-aes -mno-sha -mno-abm -mno-lwp"
I_SETS_OFF="${I_SETS_OFF} -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-sgx -mno-bmi2"
I_SETS_OFF="${I_SETS_OFF} -mno-pconfig -mno-wbnoinvd -mno-tbm -mno-avx -mno-avx2"
I_SETS_OFF="${I_SETS_OFF} -mno-lzcnt -mno-rtm -mno-hle -mno-rdrnd -mno-f16c"
I_SETS_OFF="${I_SETS_OFF} -mno-fsgsbase -mno-rdseed -mno-prfchw -mno-adx -mno-xsave"
I_SETS_OFF="${I_SETS_OFF} -mno-xsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd"
I_SETS_OFF="${I_SETS_OFF} -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec"
I_SETS_OFF="${I_SETS_OFF} -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl"
I_SETS_OFF="${I_SETS_OFF} -mno-avx512ifma -mno-avx512vbmi -mno-avx5124fmaps"
I_SETS_OFF="${I_SETS_OFF} -mno-avx5124vnniw -mno-clwb -mno-mwaitx -mno-clzero -mno-pku"
I_SETS_OFF="${I_SETS_OFF} -mno-rdpid -mno-gfni -mno-shstk -mno-avx512vbmi2 -mno-avx512vnni"
I_SETS_OFF="${I_SETS_OFF} -mno-vaes -mno-vpclmulqdq -mno-avx512bitalg -mno-movdiri -mno-movdir64b"
I_SETS_OFF="${I_SETS_OFF} -mno-waitpkg -mno-cldemote -mno-ptwrite"

COMMON_FLAGS="-march=x86-64 -mtune=generic -O2 -pipe"
COMMON_FLAGS="${COMMON_FLAGS} ${I_SETS_ON} -fstack-protector-strong ${I_SETS_OFF}"

CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"

Then you can compile on one system, and deploy packages on the others using -k/-K emerge options. You can also use distcc.

my case is to cpus for different companies, the above applies.

[mike155]

Hm. You wrote:

generate bins that will wor on both ryzen gen 2 and a kaby lake based Pentium setups

Your 'Intel Pentium CPU G4560' doesn't look like a Ryzen or a Kaby Lake CPU.

But you're right: if you have a 'Pentium G4560' CPU, don't enable any of the newer extensions like AVX.

[DaggyStyle]

Hm. You wrote:

generate bins that will wor on both ryzen gen 2 and a kaby lake based Pentium setups

Your 'Intel Pentium CPU G4560' doesn't look like a Ryzen or a Kaby Lake CPU.

But you're right: if you have a 'Pentium G4560' CPU, don't enable any of the newer extensions like AVX.

Wikipedia is not a trustworthy information source, see https://ark.intel.com/content/www/us/en ... 0-ghz.html

[mike155]

Now I understand. Wikipedia writes

In Q1 2017 Intel released the Kaby Lake-based Pentium G4560

But it surely is not a Kaby Lake processor. Because Kaby Lake processors support AVX and AVX2 (see https://en.wikipedia.org/wiki/Kaby_Lake)

I really doubt that programs that were compiled for Sky Lake or Kaby Lake processors will run on the G4560 - because '-march=skylake' will generate AVX and AVX2 instructions.

[Anon-E-moose]

Ryzen flags will work for kaby lake cpus, if you remove the sse4a flag (ryzen only)

Ryzen CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3"
KBlake CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3"

as far as a common arch, compatible with both, not sure what I'd choose.

Personally, I do a regular compile for the desktop (ryzen) with a chroot where I create packages for the laptop, so many things get a second compile.
Some things are only on one machine, some on the other, ie. the hardware differences, so I found it hard to create both from a common point cleanly.
I only do the compile for the laptop weekly, as it's a secondary machine and thus not used all the time.

which cpu is the kblake?

i3-7100u (acer laptop)

Edit to add: So Daggy, given the g4560 is not 100% kabylake, your cut down cpu instruction set is probably what you'll use.

[Anon-E-moose]

Now I understand. Wikipedia writes

In Q1 2017 Intel released the Kaby Lake-based Pentium G4560

But it surely is not a Kaby Lake processor. Because Kaby Lake processors support AVX and AVX2 (see https://en.wikipedia.org/wiki/Kaby_Lake)

I really doubt that programs that were compiled for Sky Lake or Kaby Lake processors will run on the G4560 - because '-march=skylake' will generate AVX and AVX2 instructions.

I agree with that.

Seems the G4560 is based on kaby lake, but they evidently removed some of the instruction set, to get better speed in some areas (typical intel)

[mike155]

Back to the question.

I would run the following command on the Ryzen machine:

Code: Select all

LANG=C gcc -Q -O2 --help=target -march=native >ryzen.txt

and the command below on the Pentium machine

Code: Select all

LANG=C gcc -Q -O2 --help=target -march=native >pentium.txt

After that, I would run

Code: Select all

LANG=C gcc -Q -O2 --help=target -march=x86-64 >x86-64.txt

and compare 'x86-64.txt' with the other two files:

Code: Select all

diff x86-64.txt ryzen.txt   | egrep "^>"
diff x86-64.txt pentium.txt | egrep "^>"

If everything you see is 'enabled', you will know that '-march=x86-64' will work for both machines. Something is wrong if you see 'disabled'.

After that, I would repeat the last step with 'core2', 'sandybridge', 'ivybridge', 'haswell', 'skylake'.

Code: Select all

LANG=C gcc -Q -O2 --help=target -march=core2 >core2.txt
diff core2.txt ryzen.txt   | egrep "^>"
diff core2.txt pentium.txt | egrep "^>"
...

Choose the last / most advanced architecture where you don't see any 'disabled'.

[Anon-E-moose]

The avx instruction was introduced (intel) in the sandybridge line, so you'll have to choose a cpu prior, core2, nehalem, etc.

You'll likely be crippling the ryzen (a lot) by using such a cut down cpu set.

[DaggyStyle]

Ryzen flags will work for kaby lake cpus, if you remove the sse4a flag (ryzen only)

Ryzen CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3"
KBlake CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3"

as far as a common arch, compatible with both, not sure what I'd choose.

Personally, I do a regular compile for the desktop (ryzen) with a chroot where I create packages for the laptop, so many things get a second compile.
Some things are only on one machine, some on the other, ie. the hardware differences, so I found it hard to create both from a common point cleanly.
I only do the compile for the laptop weekly, as it's a secondary machine and thus not used all the time.

which cpu is the kblake?

i3-7100u (acer laptop)

Edit to add: So Daggy, given the g4560 is not 100% kabylake, your cut down cpu instruction set is probably what you'll use.

thanks for the confirmation

[DaggyStyle]

The avx instruction was introduced (intel) in the sandybridge line, so you'll have to choose a cpu prior, core2, nehalem, etc.

You'll likely be crippling the ryzen (a lot) by using such a cut down cpu set.

rhe reason behind this is to create a binpkg host for the g4560 as it boots from a usb thumb drive