Segfaults during compilation on AMD Ryzen.

Kresp · Tux's lil' helper Joined: 17 Oct 2016 Posts: 77

I often encounter segfaults during emerge builds of heavy packages like curl, hgc, llvm:

Logicien · Posted: Sun Apr 02, 2017 7:36 am Post subject:

How do you optimise Gcc compile time in /etc/portage/make.conf?
_________________
Paul

Kresp · Tux's lil' helper Joined: 17 Oct 2016 Posts: 77

Logicien · Posted: Sun Apr 02, 2017 11:09 am Post subject:

NeddySeagoon · Posted: Sun Apr 02, 2017 11:31 am Post subject:

Kresp,

Segfaults during build normally indicate a hardware issue.

However, with such an old gcc on such new hardware, I would not rule out other things.
gcc-4.9.4 does not understand -march=native for Ryzen. That needs gcc-6.3 which is still hard masked in Gentoo.
I'm not suggesting that you upgrade to that. The update is not trivial and gcc-6.3 does not work for everything yet.

That the problem is intermittent points to hardware.

Is your BIOS the latest available version?
There have been a rash of BIOS updates since Rzyen was released.
Update your BIOS as a first step, if there in a newer one for your motherboard.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

Kresp · Tux's lil' helper Joined: 17 Oct 2016 Posts: 77

NeddySeagoon · Posted: Sun Apr 02, 2017 2:20 pm Post subject:

Kresp,

The rebuilds between gcc-4.x and gcc-5.x are due to the C++ ABI change.
Its not required every gcc major version update.
gcc-5.x to gcc-6.x is painless except for packages that have problems with gcc-6.

You are quite right to take things slowly.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

limn · l33t Joined: 13 May 2005 Posts: 997

Those are not compiler faults, at least not directly. It's your shell the kernel is complaining about.

c1pherx · n00b Joined: 02 Apr 2017 Posts: 7

I'm running a Ryzen 1700X with the ASRock Taichi and 32GB of RAM. I've tried GCC 4.9.4, 5.4.0, and 6.3.0 and I've seen some sporadic segmentation faults on all three. Also an up-to-date BIOS, Memtest returns no issues, and stress doesn't seem to cause issues. So far I haven't been able to isolate exactly what is causing it, but I don't think it's hardware. Some things I do know:

gcc-5.4.0 with -march=native is -march=bdver4.
gcc-6.3.0 with -march=native is -march=znver1

I know there are some significant differences between Bulldozer and Zen, but 6.3.0 has also produces the occasional segfault despite the newer -march. Currently re-emerging all of world with 6.3.0 to see if that helps matters.

toralf · Posted: Sun Apr 02, 2017 4:53 pm Post subject: Re: Segfaults during compilation.

Jaglover · Posted: Sun Apr 02, 2017 5:16 pm Post subject:

Memtest results are not conclusive, it can run for days without error and the memory can still be faulty. Only if it tells you the RAM is bad you can believe it.
_________________
My Gentoo installation notes.
Please learn how to denote units correctly!

NeddySeagoon · Posted: Sun Apr 02, 2017 5:35 pm Post subject:

Jaglover,

Its worse than that.

Only if memtest tells you the RAM is bad at the same address on several cycles, is there a good chance its the RAM.
It can also be the memory controller (in the processor on Ryzen) or the local voltage regulator (on the motherboard) for the RAM.

Errors at random addresses are probably not RAM.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

limn · l33t Joined: 13 May 2005 Posts: 997

FMA3 instruction problem?
It is described as intermittent, but also as not affecting Linux.

Kresp · Tux's lil' helper Joined: 17 Oct 2016 Posts: 77

Well, gcc5 was not a panacea - segfaults still happen.

I removed fma3 flag to check if this is related to that CPU bug, but sudo emerge --ask --newuse --update @world did not rebuild anything.

Will now try disabling SMT in UEFI and changing MAKEOPTS to 8. SMT/HT is marketing gimmick anyway.

limn · l33t Joined: 13 May 2005 Posts: 997

emerge --newuse --update does not consider changes to CPU_FLAGS_X86.

Recompile a package that failed an arbitrary number of times with the cpu flag until you have at least one failure. Then remove flag and compile it at least as many times.

Even then you may not know until you apply the FMA3 fix.

NeddySeagoon · Posted: Mon Apr 03, 2017 2:23 pm Post subject:

Kresp, limn,

CPU_FLAGS_X86 applies only to hand optimised code segments where a package advertises that such speedups are available for user selection.
They do not affect the code emitted by gcc.

To stop gcc using FMA3, you need to find the name of the option add add it to cflags.

limn · l33t Joined: 13 May 2005 Posts: 997

Neddy,

Are you saying that CPU_FLAGS_X86 activate optimizations in the binary at run time?

NeddySeagoon · Posted: Mon Apr 03, 2017 5:53 pm Post subject:

limn,

No. In the source at build time. They are exactly like USE flags, which is what they were at one time.

CPU_FLAGS_X86="mmx" will include sections of optional code in the source that have been hand optimised to make use of the mmx instruction set.
CFLAGS="-mmmx" (is that right?) allows gcc to emit mmx instructions in the course of any build.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

liewyec · n00b Joined: 03 Apr 2017 Posts: 9

i got the same problem. I have ryzen 1800x, asus prime x370 pro. I have only two sticks of ram, memtest also no errors.

I can sometimes compile entire chromium with no errors and sometimes it crashes multiple times in a row compiling just a few packages and it is not even running 16 threads, It is really strange. I tried disable optimization but it didn't help.

In the end i wrote a script that will restart emerge few times if it fails.

c1pherx · n00b Joined: 02 Apr 2017 Posts: 7

Just a quick update.

I switched to GCC-6.3.0 and the segfaults continued.

Then I switched from 4x8GB of CMK16GX4M2B3000C15 to 4x8GB of CMK16GX4M2B3200C16. The Segfaults became slightly less frequent (but this may just have been luck). Then I switched down to 2x8GB of CMK16GX4M2B3200C16 and OC'd it to 3200MHz 16-15-15-15-36 @ 1.35V and I haven't seen a Segfault since. I've compiled GHC, Chromium, and Firefox multiple times each.

I have some Ripjaws V arriving tomorrow. Going to see if I can get 32GB stable.

Keepco · n00b Joined: 02 Apr 2017 Posts: 5

Seems like I'm plagued by this as well. My Specs:

Ryzen 7 1700
MSI X370 XPower Titanium
16GB of Corsair Dominator RAM (CMD16GX4M2B3000C15), currently running @ 2133MHz

Just a few "big" packages fail for me, namely:

chromium webkit-gtk electron libreoffice

Is there any other way to get rid of this other than trying different RAM sticks?

c1pherx · n00b Joined: 02 Apr 2017 Posts: 7

Yea. I spoke too soon. I've reduced the frequency of it happening, but it is still happening. On to the next ideas.

One pattern I'm noticing is that now it seems to be happening with builds that use libtool. This may just be a correlation, but my most recent failures were gnutls (first time that's happened) and libseccomp (first time here too). Both use Libtool.

Keepco · n00b Joined: 02 Apr 2017 Posts: 5

What irritates me is that my chromium builds always segfaults with at least the last 2 lines being exactly the same (can't recall the other ones, should've written those down somewhere..)[/code]

c1pherx · n00b Joined: 02 Apr 2017 Posts: 7

Keepco · n00b Joined: 02 Apr 2017 Posts: 5