World update stuck as python3.4 can't compile itself

V10lator · Apprentice Joined: 11 Jul 2004 Posts: 207

The problem is that python just takes forever (I was waiting for 50+ hours) to compile one file:

phajdan.jr · Posted: Sun Feb 05, 2017 10:59 am Post subject:

Try to see with strace/gdb what that process is trying to do.
_________________
http://phajdan-jr.blogspot.com/

ross_anderson · n00b Joined: 13 Feb 2017 Posts: 3

I can confirm this on numerous machines I have similar issues as reported. I tried to attach to the python call once is became stuck and strace provided zero info. Since it's not a compile error I'd welcome some guidance on how to capture the appropriate debug information.

DeIM · Guru Joined: 11 Apr 2006 Posts: 429

Have the same problem on ARM. Both 3.4.5 and ~3.4.6 the same problem.

ross_anderson · n00b Joined: 13 Feb 2017 Posts: 3

I continue to struggle with this on multiple systems. The python build just hangs indefinitely with 1 core locked 100%. I'd appreciate any guidance in getting this problem to a state of either resolution or more information to allow more knowledgeable members to assist.

Irre · Guru Joined: 09 Nov 2013 Posts: 434 Location: Stockholm

Try to kill only the looping process, not emerge!

DeIM · Guru Joined: 11 Apr 2006 Posts: 429

When I kill only the hanging process, it restarts the part of compilation, where is the test_buffer.py

I tried patching the source to use installed python instead of the new compiled for compiling test_buffer.py and it passed and installed. So I tried to recompile package again but it hanged on the same file.

Irre · Guru Joined: 09 Nov 2013 Posts: 434 Location: Stockholm

Irre · Guru Joined: 09 Nov 2013 Posts: 434 Location: Stockholm

dev-lang/python-3.6.0 emerged successfully with no hanging process

Edit:
Did not build on Cubieboard 20 (ARMv7)

ross_anderson · n00b Joined: 13 Feb 2017 Posts: 3

On a hunch I made a custom ENV flag for the build.

Make file has

CFLAGS="-march=native -Ofast -pipe -fno-tree-loop-distribute-patterns"

For python I set

CFLAGS="-O2 -pipe -march=native"

And the build succeeded. Could other users comment on thier build flag?

Syl20 · l33t Joined: 04 Aug 2005 Posts: 619 Location: France

https://wiki.gentoo.org/wiki/Safe_CFLAGS
https://wiki.gentoo.org/wiki/GCC_optimization

If you don't use the recommended safe flags, you will have problems.

Leikor · Tux's lil' helper Joined: 04 Sep 2003 Posts: 123

Removing the "-Ofast" option fix this issue for me.

abusse · n00b Joined: 27 Oct 2017 Posts: 8

I have the same issue with the following cflags:

krinn · Watchman Joined: 02 May 2003 Posts: 7470

You should ask yourself: if -O3 is making everything faster and it is safe to do, why -O3 exist at all, gcc should have nothing and always use -O3 and do the best job.
Because some optimizations are dangerous, they are options, and the proper handling of something like that is that instead of disable -O3 in a package, you should do the contrary, which mean -O2 everywhere and -O3 on per package set ; packages you know are good with -O3 (not only good because they aren't buggy with it, but also good because -O3 do make them faster, which optimizations set by -O3 "should do", but "may not").

There's many reasons why optimizations may break code.
As a "raw" example: in computers shifting bit is something natural, and shifting a bit left is * 2, and shifting right is / 2.
So computer is able to calc : 100 SHR 1 = 50 faster than 100 / 2 = 50. Each time you see : X / 2, you should optimize that to do X SHR 1 and get a faster result.
But it's not always safe:
- Y = X / 2
- Z = 100 / Y
If X=1, with X / 2 you get Y=0.5, and with X SHR 1 you get Y=0, which mean Z = division per 0
I'm not saying gcc do that stupidity, i'm showing you a case where optimization lead to a failure in a program.
You shouldn't enable on all packages an optimization that is only present in -O3 set, gcc devs have put it in -O3 because they knows that optimization is tricky to apply and may lead to trouble.

NeddySeagoon · Posted: Fri Jan 26, 2018 2:33 pm Post subject:

abusse,

Look into

NeddySeagoon · Posted: Fri Jan 26, 2018 2:36 pm Post subject:

krinn,

If X=1, with X / 2 you get Y=0.5 is true only if Y is some sort of floating paint number.
If its an integer, Y=0 is correct, however its calculated.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

krinn · Watchman Joined: 02 May 2003 Posts: 7470

s_j_newbury · n00b Joined: 05 Apr 2007 Posts: 56 Location: UK

Optimisations *almost* always do things that are permitted by the language standards, but aren't *always* expected. Sometimes code relies on undefined behaviour which has produced the right result in the past, until the compiler does something different, but not wrong. The bug is in the code not the compiler.

Occasionally, optimisations do the *wrong* thing, which can break valid code, this is a bug in the compiler unless it is clearly indicated that it is the expected behaviour. GCC has a number of flags which are known to do the *wrong* thing on occasion and are named accordingly.

-ffast-math doesn't do the wrong thing conceptually, it's permitted by the C standard, but it does violate IEEE 754 which developers sometimes depend upon for predictable behaviour.