Profile 17.0

Message

Spargeltarzan · Post by **Spargeltarzan** » Fri Dec 01, 2017 4:51 pm

Hi,

When I read the news about profile 17.0 it is said that gcc 6 is a necessity, but I am unsure if emerge -e is necessary when I now only change to the profile to 17.0 and did already an emerge -e for gcc 6 on profile 13.0.

Post by **NeddySeagoon** » Fri Dec 01, 2017 5:15 pm

Spargeltarzan,

the /17.0/ profiles force the Position Independent Executable use flag from (-pie) to (pie).
This affects gcc and the way it builds everything. pie is now the default everywhere.

Mixing code built with -pie and pie is a very bad thing. It certainly breaks all of your static libraries.
Maybe more, so rebuilding everything is safe.

However, if you used a hardened profile before the change to /17.0/ then pie was already on (due to hardened) and no rebuilds for pie should be required.

The /17.0/ profiles also do other things but thats the major change.
gcc-5 is masked
There is a change to icu. Versions that don't build with gcc-6 are masked.

That's the things I'm aware of.

Spargeltarzan · Post by **Spargeltarzan** » Fri Dec 01, 2017 5:25 pm

Thank you for the detailed answer, I started now my emerge -e. And actually sorry for the question, when reading the news item again it answered it actually in the last paragraph

Spargeltarzan · Post by **Spargeltarzan** » Fri Dec 01, 2017 5:33 pm

Shouldn't I see the PIE flag on every package now? I saw it only in the gcc package

Post by **NeddySeagoon** » Fri Dec 01, 2017 6:14 pm

Spargeltarzan,

Once you compile gcc with USE=pie, it will apply it everywhere else by default.

pie is actually a CFLAG. As a USE flag on gcc, it builds gcc to apply -fpie (it might be -fPIE) everywhere.
This saves you adding it to CFLAGS. It becomes the default behaviour for gcc.

pablo_supertux · Post by **pablo_supertux** » Fri Dec 01, 2017 6:54 pm

I tried reading some articles about PIE (never heard of it before) and I didn't understand a thing

what exactly are these PIE binaries und why are they the new default? As far as I can tell from the answers in this posts, once I've changed my profile to 17.0, I don't have to change my USE flags, right?

Too bad that I made the jump from gcc4 to gcc6 last week, I already had to recompile almost everything, now I have to do that again? That's bad timing for me

McQ · Post by **McQ** » Fri Dec 01, 2017 8:50 pm

pablo_supertux wrote:I tried reading some articles about PIE (never heard of it before) and I didn't understand a thing

what exactly are these PIE binaries und why are they the new default? As far as I can tell from the answers in this posts, once I've changed my profile to 17.0, I don't have to change my USE flags, right?

Too bad that I made the jump from gcc4 to gcc6 last week, I already had to recompile almost everything, now I have to do that again? That's bad timing for me

From a resent Gentoo Commits message

Code: Select all

We have just added (for all arches except arm and mips, these follow
later) a new set of profiles with release version 17.0 to the Gentoo 
repository. These bring three changes:
1) The default C language version for applications is now C14.
   This change is mostly relevant to Gentoo developers. It also
   means, however, that compilers earlier than GCC 6 are masked 
   and not supported for use as a system compiler anymore. Feel 
   free to unmask them if you need them for specific applications.
2) Where supported, GCC will now build position-independent
   executables (PIE) by default. This improves the overall
   security fingerprint. The switch from non-PIE to PIE binaries,
   however, requires some steps by users, as detailed below.
3) Up to now, hardened profiles were separate from the default
   profile tree. Now they are moving into the 17.0 profile
   as a feature there, similar to "no-multilib" and "systemd".

Please migrate away from the 13.0 profiles within the six weeks after
GCC 6.4.0 has been stabilized on your architecture. The 13.0 profiles
will be deprecated then and removed in half a year.

If you are not already running a hardened setup with PIE enabled, then
switching the profile involves the following steps: 
If not already done,
* Use gcc-config to select gcc-6.4.0 or later as system compiler
* Re-source /etc/profile:
    . /etc/profile
* Re-emerge libtool
    emerge -1 sys-devel/libtool
Then, 
* Select the new profile with eselect
* Re-emerge, in this sequence, gcc, binutils, and glibc
    emerge -1 sys-devel/gcc:6.4.0
    emerge -1 sys-devel/binutils
    emerge -1 sys-libs/glibc
* Rebuild your entire system
    emerge -e @world

Switching the profile from 13.0 to 17.0 modifies the settings of 
GCC 6 to generate PIE executables by default; thus, you need to do 
the rebuilds even if you have already used GCC 6 beforehand.
If you do not follow these steps you may get spurious build
failures when the linker tries unsuccessfully to combine non-PIE
and PIE code.

Tony0945 · Post by **Tony0945** » Fri Dec 01, 2017 8:59 pm

Hmmm! Having a little trouble building with pie:

Code: Select all

 # USE="pie" emerge -a1v gcc::gentoo

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] sys-devel/gcc-6.4.0:6.4.0::gentoo [6.4.0:6.4.0::local] USE="cxx fortran (multilib) nptl openmp pch sanitize ssp vtv (-altivec) (-awt) -cilk -debug -doc (-fixed-point) (-gcj) -go -graphite (-hardened) (-jit) (-libssp) -mpx -nls -objc -objc++ -objc-gc -pgo (-pie) -regression-test -vanilla" 13 KiB

Total: 1 package (1 reinstall), Size of downloads: 13 KiB

Would you like to merge these packages? [Yes/No] n

I also have pie enabled in make.conf and package.use, so why does portage want to build with -pie? I'm still on 13.0/desktop.

BTW, I first ran "quickpkg gcc" just in case.

pablo_supertux · Post by **pablo_supertux** » Fri Dec 01, 2017 9:16 pm

McQ wrote:
pablo_supertux wrote:I tried reading some articles about PIE (never heard of it before) and I didn't understand a thing

what exactly are these PIE binaries und why are they the new default? As far as I can tell from the answers in this posts, once I've changed my profile to 17.0, I don't have to change my USE flags, right?

Too bad that I made the jump from gcc4 to gcc6 last week, I already had to recompile almost everything, now I have to do that again? That's bad timing for me
From a resent Gentoo Commits message

Yeah, I read that and because of it I went looking for an explanation of what PIE does, but the articles I've found were too technical and I didn't unterstand everything, that's why I asked.

Tony0945 · Post by **Tony0945** » Fri Dec 01, 2017 9:43 pm

pablo_supertux wrote: Yeah, I read that and because of it I went looking for an explanation of what PIE does, but the articles I've found were too technical and I didn't unterstand everything, that's why I asked.

What bothers me is that it seems to be a RedHat idea. I wonder if I can keep the 13.0 profile as a custom profile. Every RedHat idea for at least the last five years has been a bummer.

Gentlenoob · Post by **Gentlenoob** » Fri Dec 01, 2017 9:44 pm

Dear Gentooers,

I've some fairly old boxes (Atom / Core2 from ~6/10 years ago), so this update, if done as suggested, will take some days. Is there some way to avoid this, e.g. staying at '-pie' and still going to the 17.0 profile?

The news article talks about spurious build failures. Neddys advice sounds much more scary. What happens if for some reason 'emerge -e' fails in the midst and I'm left with said mix of pie and -pie?

Do the failures mentioned in the news affect only building or also runtime? Do they happen at random as 'spurious' may imply? Is important stuff affected or just some fringe packages?

Of course I'll thoroughly backup beforehand anyway.

Thanks a lot,
Ralph

Tony0945 · Post by **Tony0945** » Fri Dec 01, 2017 9:45 pm

google "gcc pie good and bad" and see some scary things.

The Doctor · Post by **The Doctor** » Fri Dec 01, 2017 9:58 pm

@Tony0945
Rest assured that PIE has been around for quite some time. This isn't the good idea fairy visiting Pottering again. As with all security you will take a small performance hit but I doubt you will notice, especially on a modern setup.

Gentlenoob wrote: What happens if for some reason 'emerge -e' fails in the midst and I'm left with said mix of pie and -pie?

Bad things if you don't fix it. You should take advantage of the --resume option to continue the build process.

They failures will be both run and build time. They would affect major packages as well as minor ones. However, there should be no reason why you cannot complete an emerge -e world so this should not affect you.

Gentlenoob wrote: Of course I'll thoroughly backup beforehand anyway.

Always wise.

Soul Reaver · Post by **Soul Reaver** » Fri Dec 01, 2017 11:01 pm

So how does this relate to clang? I use clang as my main compiler, falling back to gcc for some incompatible packages. Is it sufficient to recompile those packages built with gcc? Are clang built binaries even compatible with gcc (+pie) built binaries?

Tony0945 · Post by **Tony0945** » Sat Dec 02, 2017 12:59 am

The Doctor wrote:@Tony0945
Rest assured that PIE has been around for quite some time. This isn't the good idea fairy visiting Pottering again. As with all security you will take a small performance hit but I doubt you will notice, especially on a modern setup.

Thank you. I will accept YOUR judgement.

pablo_supertux · Post by **pablo_supertux** » Sat Dec 02, 2017 1:20 am

Gentlenoob wrote:
I've some fairly old boxes (Atom / Core2 from ~6/10 years ago), so this update, if done as suggested, will take some days. Is there some way to avoid this, e.g. staying at '-pie' and still going to the 17.0 profile?

Unless you create your own profile base on 17.0 and write your own patches, I'm afraid this is not going to end well. The news text says

Please migrate away from the 13.0 profiles within the six weeks after
GCC 6.4.0 has been stabilized on your architecture. The 13.0 profiles
will be deprecated then and removed in half a year.

and that means that the profile 13.0 will be gone in a couple of months.

Do the failures mentioned in the news affect only building or also runtime? Do they happen at random as 'spurious' may imply? Is important stuff affected or just some fringe packages?

I'll first do a backup of the current running system and I'm going to wait a couple of weeks before doing that. I'm afraid that if I do it right now, I'm going to have some issues here and there and is going to be a pain to fix them. I think I'll wait 2 or 3 weeks and then do the update. Most of the initial issues might be resolved by then.

I still wish to know what's exactly PIE binaries do. And which other distributions are doing doing it.

The Doctor · Post by **The Doctor** » Sat Dec 02, 2017 1:44 am

pablo_supertux wrote:I still wish to know what's exactly PIE binaries do. And which other distributions are doing doing it.

Basically the programs can be relocated in memory. This makes certain types of attacks much harder to do. It has been a feature of hardened Gentoo for quite a while.

Aside from Linux Mac OS X and iOS use it. OpenBSD has been using it since 2013 by default. Fedora 23 made it a default option and Ubuntu 17 did as well. Finding a complete list of Linux distros is rather more difficult, but the lit certainly includes Alpine Linux.

Post by Hu » Sat Dec 02, 2017 1:51 am

PIE = Position Independent Executable. As The Doctor says, this makes them trivial to relocate at load time. Malicious code commonly benefits from reusing fragments of the exploited application. By relocating them to an address that an attacker cannot easily predict, the kernel can make it more difficult for malicious code to find those fragments, thus making complex exploits more difficult to write and less reliable to run. (This does nothing to complicate exploits that aim simply to crash the victim program.) This relocation technique is commonly called Address Space Layout Randomization (ASLR). It can be done with non-PIE programs, but it's less efficient.

Starting in Windows Vista, Microsoft implemented a very limited form of ASLR. They may have improved it subsequently, but when last I checked, it had some notable weaknesses relative to the Linux implementation (which Brad Spengler will happily tell you is itself not as good as the implementation he/PaX team use in the now-unavailable grsecurity project).

Tony0945 · Post by **Tony0945** » Sat Dec 02, 2017 2:21 am

OK, I see that pie goes with the profile and you have to change profile before you build with pie.
On my downstairs machine I did the following:

1. quickpkg gcc libtool binutils glibc
2. eselect profile 17 (on my machine selects profile 17.0/desktop
3. bash updateprofile

updateprofilescript is:

Code: Select all

 emerge -1v sys-devel/gcc:6.4.0 && emerge -1v sys-devel/binutils && emerge -1v sys-libs/glibc && emerge -e @world && echo "It worked!"

Running step 3 now. I first took the precaution of shutting down xdm. I'll report sometime tomorrow how it went.
I usually emerge -e system then -e world, but I'm following orders. Sorry I didn't run "time bash updateprofile" but it may be finished when I wake up tomorrow. Maybe not. Usually the last step alone takes about 11 hours.

EDIT: Fourteen hours later and the screen is emerging 328 of 526. Originally there were 1052, so why now 528? I think emerge@preserved-rebuild is running. Maybe it will be done tonight. Maybe. I should have done benchmarks before and after to see what the performance hit is.

Another thought. Perhaps I should rebuild the kernel also. emerge -e will just refetch the source not build the kernel.

Fitzcarraldo · Post by **Fitzcarraldo** » Sat Dec 02, 2017 6:25 am

If anyone arrives in this thread when searching for solutions to the problems listed below that may occur while carrying out the procedure specified in News item "2017-11-30-new-17-profiles"...

In my case the following three packages failed to re-merge during "emerge -e @world" (I used "emerge --resume --skip-first" to continue each time):

a) media-libs/libbluray-1.0.1 "configure: error: could not find jni.h"

See https://bugs.gentoo.org/621992

b) dev-libs/klibc-2.0.4-r2 "kernel/bounds.c:1:0: error: code model kernel does not support PIC mode"

See https://bugs.gentoo.org/621848

I applied the fix in Comment 4 of the bug report to an ebuild for klibc-2.0.4-r2 in my local overlay.

c) app-emulation/virtualbox-modules-5.1.26 "cc1: error: incompatible gcc/plugin versions"

I had to rebuild the kernel (4.12.12-gentoo) in order to be able to re-merge virtualbox-modules successfully.
And to rebuild the kernel I had to use the command "make clean" before "make && make modules_install"
in order to prevent the "make" command failing with the following error message referring to the old version
of gcc instead of 6.4.0:

No rule to make target '/usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0/include/stddef.h',
needed by '/usr/src/linux-4.12.12-gentoo/tools/objtool/fixdep.o'. Stop.

After rebuilding the kernel, I used the command "emerge @module-rebuild" to re-merge
virtualbox-modules-5.1.26 successfully.

EDIT: Corrected '@system' to '@world'.

wenguiGwok · Post by **wenguiGwok** » Sat Dec 02, 2017 6:29 am

i did as instructed after receiving the 17.0 upgrade notif, namely, change the profile, rebuild gcc and toolchain, then emerge -e @wolrd.

when i checked the USE for gcc itself, here is :

Code: Select all

⚡ equery u gcc
...
 * Found these USE flags for sys-devel/gcc-6.4.0:
 U I
...
 - + pie              : Build programs as Position Independent Executables (a security hardening technique)

seems like pie is not set. nor is pie listed in USE shown by `emerge --info`.

however, emerge gives:

Code: Select all

⚡ emerge -qpv gcc         
[ebuild   R   ] sys-devel/gcc-6.4.0  USE="cxx fortran (multilib) nls nptl openmp pch (pie) sanitize ssp vtv (-altivec) (-awt) -cilk -debug -doc (-fixed-point) (-gcj) -go -graphite (-hardened) (-jit) (-libssp) -mpx -objc -objc++ -objc-gc -pgo -regression-test -vanilla"

which indicates pie is set on.

some guys on IRC kindly told me they are having the same results. so just be easy with that and believe emerge not equery this time.
actually the 4th comment in this thread has made it clear.

NeddySeagoon wrote:Spargeltarzan,

Once you compile gcc with USE=pie, it will apply it everywhere else by default.

pie is actually a CFLAG. As a USE flag on gcc, it builds gcc to apply -fpie (it might be -fPIE) everywhere.
This saves you adding it to CFLAGS. It becomes the default behaviour for gcc.

just a memo for those who might have been confused as i was.

teefax · Post by **teefax** » Sat Dec 02, 2017 7:33 am

Is there any way to avoid the @world rebuild?

Post by **fedeliallalinea** » Sat Dec 02, 2017 7:53 am

teefax wrote:Is there any way to avoid the @world rebuild?

If you want switch to 17 no

NeddySeagoon wrote:Mixing code built with -pie and pie is a very bad thing. It certainly breaks all of your static libraries.
Maybe more, so rebuilding everything is safe.

However, if you used a hardened profile before the change to /17.0/ then pie was already on (due to hardened) and no rebuilds for pie should be required.

teefax · Post by **teefax** » Sat Dec 02, 2017 8:03 am

Yes, I've read that. I meant, is it possible to disable PIE?

It seems the PIE use flag is forced for the 17.0 profile. Is it sufficient to add -fno-pie to my CFLAGS in make.conf ?

Myu · Post by **Myu** » Sat Dec 02, 2017 8:49 am

I did the step mentioned in the news item and moved to 17.0/desktop and after 977 packages rebuild overnight, I got my system back, 0 build failures !

Thank you to the Gentoo team/community as a whole ! I'm really impressed and pleased, as I understand such an event (rebuilding world) is not that common (I use Gentoo since 2014)

Profile 17.0

Profile 17.0

pie not shown as set by equery