| View previous topic :: View next topic |
| Author |
Message |
Elleni
Veteran
Joined: 23 May 2006
Posts: 1270
|
 Posted: Mon Dec 18, 2017 12:55 pm Post subject: [Solved] Compiling binaries for core2 on an ryzen 1600 box |
 |
|
I have a friend of mine, who is a pure user, with no knowledge of Linux whatsoever. But as we both had same hardware, I had installed and am still maintaining and supporting his Gentoo installation. All works quite well, as I do the building on my box, and build binaries to install them on the second machine.
Now my motherboard is starting to make problems (in particular the IME chip is dying). So I decided to buy new hardware. (AMD Ryzen 1600 CPU, RX580 Radeon GPU).
As the old box is an Intel Core2 Duo with an intel onboard gpu I am wondering, how I will still be able to provide the binaries in order to not compile the packages on the old intel box, and thus accelerating upgrades.
Question is: How would I do this easily. Should I create a VM with the environment of the old box, to do the building of the binaries there, or is there a better approach? I used march=native until know, so I guess, I will have to specify march and corresponding useflags.
According to wiki march should be:
| Code: | | CFLAGS="-march=core2 -O2 -pipe" |
And according to cpuinfo2cpuflags-x86 useflags should include:
| Code: | | CPU_FLAGS_X86="mmx mmxext sse sse2 sse3 sse4_1 ssse3" |
And VIDEO_CARDS="intel i965" and ALSA_CARDS="had-intel"
Are those correct options?
Will my brand new AMD box be able to compile binaries for these settings ?
cat /proc/cpuinfo
| Code: | processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Core(TM)2 Duo CPU E8500 @ 3.16GHz
stepping : 10
microcode : 0xa0e
cpu MHz : 3166.000
cache size : 6144 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl cpuid aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm tpr_shadow vnmi flexpriority dtherm
bugs :
bogomips : 6317.45
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual |
Edit: On Core2 machine:
gcc -march=native -E -v - </dev/null 2>&1 | sed -n 's/.* -v - //p'
| Code: | | -march=core2 -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mno-movbe -mno-aes -mno-sha -mno-pclmul -mno-popcnt -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-bmi2 -mno-tbm -mno-avx -mno-avx2 -mno-sse4.2 -msse4.1 -mno-lzcnt -mno-rtm -mno-hle -mno-rdrnd -mno-f16c -mno-fsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mno-xsave -mno-xsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-mwaitx -mno-clzero -mno-pku --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=6144 -mtune=core2 |
Last edited by Elleni on Sun Apr 08, 2018 4:25 pm; edited 4 times in total |
|
| Back to top |
|
 |
DawgG
l33t
Joined: 17 Sep 2003
Posts: 866
|
| Posted: Mon Dec 18, 2017 3:30 pm Post subject: |
|
|
i've done stuff like that a couple of times and it usually worked well.
easiest is probably to set up a chroot on your "compilehost" and just copy the target-system there (minus all the private stuff, /home etc.) adjust some settings inside the chroot's make.conf; eg the number of makes you run (-j17 might be nice on a recent ryzen), various portdir and tmpdir-settings, but don't get too fancy with the cflags.
then compile all the packages and kernel inside the chroot, transfer to the target system and emerge -K there. i ususally compile the kernel inside the chroot (make all) and rsync the whole dir to the target, there i just enter the dir and "make modules_install."
GOOD LUCK!
_________________
DUMM KLICKT GUT. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54237
Location: 56N 3W
|
| Posted: Mon Dec 18, 2017 3:59 pm Post subject: |
|
|
Elleni,
A chroot can work well. There is one drawback. The code you build there has to run on the build host as well as the target.
You need to find CFLAGS= and CPU_FLAGS_X86 settings the will run on both systems.
That's easier that it looks.
Run app-portage/cpuid2cpuflags on both systems. Only set the common ones in the CPU_FLAGS_X86 sutting in the chroot.
Determine the common CFLAGS using | Code: | | gcc -### -E - -march=native 2>&1 | sed -r '/cc1/!d;s/(")|(^.* - )|( -mno-[^\ ]+)//g' |
If you get different -march settings, I suspect you will, you need to dig into what they set, and only choose the common things.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
DawgG
l33t
Joined: 17 Sep 2003
Posts: 866
|
| Posted: Tue Dec 19, 2017 11:45 am Post subject: |
|
|
on the system(s) i use this mainly the processors are very different (intel-core5 vs amd-b350) and it works very well. my cflags are not too fancy, though. i can check the other flags mentioned it you are interested.
GOOD LUCK!
_________________
DUMM KLICKT GUT. |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Tue Dec 19, 2017 1:54 pm Post subject: |
|
|
I use a script that do that myself, please keep in mind both hosts must use the same profile, you better also use same toolchain. If your hosts are somehow quiet similar, you'll be ok.
| Code: | #!/bin/sh
mkdir -p /usr/share/core2build/etc/portage
# sync current value use by the core2 host
scp -r yourotherhost:/etc/portage/* /usr/share/core2build/etc/portage
# remove the profile because it's a relative symlink
rm -rf /usr/share/core2build/etc/portage/make.profile
# recreate the profile using the same profile as the build host
ln -s $(readlink -f /etc/portage/make.profile) /usr/share/core2build/etc/portage/make.profile
# using cores from the build host as it have more than original host
opts=$(cat /proc/cpuinfo | grep "model name" | wc -l)
# disabling PORTAGE_DEFAULT_OPTS to get ride of what the host use, enabling usepkg to faster the process, buildpkgonly to not emerge any package in real, set PORTAGE_TMP to not pollute your build host /var/tmp/portage, set PORTAGE_CONFIG_ROOT to use the make.conf get from the host, and set PKGDIR to not pollute build host binpkg dir.
# note also we will use USE flags as set by the core2 host, which might be quiet different from your build host, making a simple package emerge building in real a lot of packages as your build host didn't use the same USE flag as the core2, hence the use of --usepkg to speed up things
PORTAGE_CONFIGROOT="/usr/share/core2build" PORTAGE_TMP="/tmp/core2build" PKGDIR="/usr/share/core2build/packages" MAKEOPTS="-j${opts}" emerge --ignore-default-opts --buildpkgonly --usepkg "$@"
# we sent back to host the result binaries built
scp -r /usr/share/core2build/packages/* yourotherhost:/usr/portage/packages |
If you don't use ssh remove the two scp lines, and copy your core2 /etc/portage/* files manually to /usr/share/core2build/etc/portage
I don't use CFLAGS="-march=native" on my core2, but if your host have this set, either change its make.conf CFLAGS to match what it use, or change the line to override -march=native, like that
| Code: | | PORTAGE_CONFIGROOT="/usr/share/core2build" PORTAGE_TMP="/tmp/core2build" PKGDIR="/usr/share/core2build/packages" MAKEOPTS="-j${opts}" CFLAGS="-march=core2 -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mno-movbe -mno-aes -mno-sha -mno-pclmul -mno-popcnt -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-bmi2 -mno-tbm -mno-avx -mno-avx2 -mno-sse4.2 -msse4.1 -mno-lzcnt -mno-rtm -mno-hle -mno-rdrnd -mno-f16c -mno-fsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mno-xsave -mno-xsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-mwaitx -mno-clzero -mno-pku --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=6144 -mtune=core2" CXXFLAGS="${CFLAGS}" emerge --ignore-default-opts --buildpkgonly --usepkg "$@" |
|
|
| Back to top |
|
|
Elleni
Veteran
Joined: 23 May 2006
Posts: 1270
|
| Posted: Tue Dec 26, 2017 12:31 am Post subject: |
|
|
Great guys! Thanks for your informations.  |
|
| Back to top |
|
|
Elleni
Veteran
Joined: 23 May 2006
Posts: 1270
|
| Posted: Sat Mar 10, 2018 7:31 pm Post subject: |
|
|
Hey guys,
I finally could tar the core2 target system, and copied it to my new ryzen box and chrooted in. I changed some settings as needed (added buildpkg to features to create binaries, augmented makeopts to -j13 and changed march from native to core2)
cpu_flags_x86 should not be a problem, as all of them are also available on the ryzen box.
As for cflags this is what I get on ryzen:
gcc -### -E - -march=native 2>&1 | sed -r '/cc1/!d;s/(")|(^.* - )|( -mno-[^\ ]+)//g'
| Code: | | -march=znver1 -mmmx -msse -msse2 -msse3 -mssse3 -msse4a -mcx16 -msahf -mmovbe -maes -msha -mpclmul -mpopcnt -mabm -mfma -mbmi -mbmi2 -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mrdrnd -mf16c -mfsgsbase -mrdseed -mprfchw -madx -mfxsr -mxsave -mxsaveopt -mclflushopt -mxsavec -mxsaves -mmwaitx -mclzero --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=512 -mtune=znver1 |
But I forgot to do the same on the core2 box.
Can I assume, that this will not be a problem ?
Earlier in the thread I had done on the core2 box:
gcc -march=native -E -v - </dev/null 2>&1 | sed -n 's/.* -v - //p'
| Code: | -march=core2 -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mno-movbe -mno-aes -mno-sha -mno-pclmul -mno-popcnt -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-bmi2 -mno-tbm -mno-avx -mno-avx2 -mno-sse4.2 -msse4.1 -mno-lzcnt -mno-rtm -mno-hle -mno-rdrnd -mno-f16c -mno-fsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mno-xsave -mno-xsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-mwaitx -mno-clzero -mno-pku --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=6144 -mtune=core2
|
Or should I try to get the result of the following also on the core2 box for comparison and to be sure binaries created on my ryzen chroot work well on the core2 box?
| Code: | | gcc -### -E - -march=native 2>&1 | sed -r '/cc1/!d;s/(")|(^.* - )|( -mno-[^\ ]+)//g' |
Another thing. The core2 box is still on gcc 5.4 and profile 13.0
On the Ryzen box I updated to profile 17.0 and gcc-7.3.0 because of meltdown and spectre stuff. Is this still necessary to mitigate those, or would stable gcc 6.4 be good enough? |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54237
Location: 56N 3W
|
| Posted: Sat Mar 10, 2018 8:10 pm Post subject: |
|
|
Elleni,
The /13.0/ profiles will be going away. You may as well migrate. As for Spectre and Meltdown, you ned to evaluate you risks and act accordingly.
You know the Ryzen will give you | Code: | | -march=znver1 -mmmx -msse -msse2 -msse3 -mssse3 -msse4a -mcx16 -msahf -mmovbe -maes -msha -mpclmul -mpopcnt -mabm -mfma -mbmi -mbmi2 -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mrdrnd -mf16c -mfsgsbase -mrdseed -mprfchw -madx -mfxsr -mxsave -mxsaveopt -mclflushopt -mxsavec -mxsaves -mmwaitx -mclzero --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=512 -mtune=znver1 |
and the core2 wants | Code: | | -march=core2 -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mno-movbe -mno-aes -mno-sha -mno-pclmul -mno-popcnt -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-bmi2 -mno-tbm -mno-avx -mno-avx2 -mno-sse4.2 -msse4.1 -mno-lzcnt -mno-rtm -mno-hle -mno-rdrnd -mno-f16c -mno-fsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mno-xsave -mno-xsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-mwaitx -mno-clzero -mno-pku --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=6144 -mtune=core2 |
There are some mismatches. -march=znver1 and -march=core2.
The core2 will not run code made with -march=znver1
The Ryzen has -msse4a and the core2 wants -mno-sse4a
There are other mismatches too.
I don't know the underlying differences between -march=znver1 and -march=core2. You need to look at that as the core2 code built on the Ryzen has to run on the Ryzen too.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
Elleni
Veteran
Joined: 23 May 2006
Posts: 1270
|
| Posted: Sat Mar 10, 2018 8:21 pm Post subject: |
|
|
I decided to update to 7.3 compiler and to 17.0 profile too on the chrooted environement.
I am not sure, if I understand it correctly Neddy. But I thought, changing CFLAGS to core2 instead of native in make.conf of chrooted environement would be enough to create binaries that would work on the core2 box?
As I dont know how I could check if those would run on ryzen box, maybe I will just follow the try and error approach. For me, the code created in the chroot does not have to run on ryzen box. It would be enough if it runs well on the core2 box of my friends box. I do updates on my ryzen box separately with its own march=native in its respective make.conf |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54237
Location: 56N 3W
|
| Posted: Sat Mar 10, 2018 8:58 pm Post subject: |
|
|
Elleni,
| Elleni wrote: | | I am not sure, if I understand it correctly Neddy. But I thought, changing CFLAGS to core2 instead of native in make.conf of chrooted environement would be enough to create binaries that would work on the core2 box? |
That's correct as far as it goes but its not sufficient.
The binaries, or at least, the toolchain, built for core2 also have to run on the Ryzen inside the chroot, white other packages are being built.
What happens outside the chroot, when you update the Ryzen normally, does not affect the content of the chroot.
Go with the trial and error approach. You will learn a lot.
If gcc gets rebuilt for core2 then won't run in the chroot, you will soon find out. :)
Should there be a problem, there is a compromise, the toolchain in the chroot can be built to run anywhere, then added to package.mask, so its never updated.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
Elleni
Veteran
Joined: 23 May 2006
Posts: 1270
|
| Posted: Sat Mar 10, 2018 9:07 pm Post subject: |
|
|
Ok 
I rebuilt gcc-7.3 in chroot switched to it and rebuilt libtool for core2 and right now I am doing the profile update to 17.0. So I re-compile gcc, binutils glibc and after that I will issue emerge -e @world as suggested in the news comment for the profile update.
As soon as this is all done, I will mark this thread solved. Again Neddy - you are worth gold thanks for your fast and competent help
And also from me congrats on the shortly passed 40'000 posts |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54237
Location: 56N 3W
|
| Posted: Sat Mar 10, 2018 9:26 pm Post subject: |
|
|
Elleni,
Good luck and thank you.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
Elleni
Veteran
Joined: 23 May 2006
Posts: 1270
|
| Posted: Sun Mar 11, 2018 8:02 pm Post subject: |
|
|
Successfully compiled ~1450 packages for core2 box 
For the transfer of the binaries, I use the following command. Can this be optimized, or does it look good ?
| Code: | | rsync -hrtuvzhP --delete /path/to/chroot/usr/portage/packages/ -e 'ssh -p 1234' user@remote.host:/path/to/packages/ |
On the target machine, I then mount packages folder as follows:
| Code: | | sshfs -C -o allow_other -p1234 user@remote.host:/path/to/packages /usr/portage/packages/ |
And upgrade it as follows:
| Code: | | emerge --sync && emerge world -uDNavk --with-bdeps=y && emerge --depclean && revdep-rebuild -i && etc-update |
Plus I compile kernel locally on target box. I think this should work well.
Last edited by Elleni on Thu Mar 22, 2018 9:23 pm; edited 1 time in total |
|
| Back to top |
|
|
Elleni
Veteran
Joined: 23 May 2006
Posts: 1270
|
| Posted: Thu Mar 22, 2018 9:20 pm Post subject: |
|
|
All good. Target machine is updated profile migrated to 17.0 and box is running well. Just a minor issue:
After world update, revdep-rebuild, emerge --depclean and emerge @preserved-rebuild, I still get this.
| Code: |
!!! existing preserved libs:
>>> package: app-emulation/wine-2.3
* - /usr/lib32/libwine.so.1
* - /usr/lib32/libwine.so.1.0
* used by /usr/lib32/wine-any-2.21/wine/acledit.dll.so (app-emulation/wine-any-2.21)
* used by /usr/lib32/wine-any-2.21/wine/aclui.dll.so (app-emulation/wine-any-2.21)
* used by /usr/lib32/wine-any-2.21/wine/activeds.dll.so (app-emulation/wine-any-2.21)
* used by 876 other files
Use emerge @preserved-rebuild to rebuild packages using these libraries |
Would this be solved by emerge -C wine-any and re-emerge it afterwards? Or do I have to manually delete mentioned dll.so files?
On the target machine
cat /sys/devices/system/cpu/vulnerabilities/* shows:
| Code: | Mitigation: PTI
Mitigation: __user pointer sanitization
Mitigation: Full generic retpoline |
I did not do an intel microcode update. Does this mean that there is no need for microcode update or would you recommend to do it anyway? Its an old core2 duo box.
cat /proc/cpuinfo
| Code: | processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz
stepping : 10
microcode : 0xa07
cpu MHz : 2992.601
cache size : 6144 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl cpuid aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm pti tpr_shadow vnmi flexpriority dtherm
bugs : cpu_meltdown spectre_v1 spectre_v2
bogomips : 5985.24
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz
stepping : 10
microcode : 0xa07
cpu MHz : 2992.625
cache size : 6144 KB
physical id : 0
siblings : 2
core id : 1
cpu cores : 2
apicid : 1
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl cpuid aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm pti tpr_shadow vnmi flexpriority dtherm
bugs : cpu_meltdown spectre_v1 spectre_v2
bogomips : 5985.24
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
|
[Moderator edit: changed [quote] tags to [code] tags to preserve output layout. -Hu] |
|
| Back to top |
|
|
Elleni
Veteran
Joined: 23 May 2006
Posts: 1270
|
| Posted: Thu Apr 05, 2018 8:45 pm Post subject: |
|
|
| Anyone an idea why the preserved-rebuild message persists? How could I solve this minor issue? |
|
| Back to top |
|
|
DONAHUE
Watchman
Joined: 09 Dec 2006
Posts: 7651
Location: Goose Creek SC
|
|
| Back to top |
|
|
Elleni
Veteran
Joined: 23 May 2006
Posts: 1270
|
| Posted: Sun Apr 08, 2018 4:25 pm Post subject: |
|
|
| Thank you, that did the trick. |
|
| Back to top |
|
|
|
Installing Gentoo
