| View previous topic :: View next topic |
| Author |
Message |
Spargeltarzan
Guru
Joined: 23 Jul 2017
Posts: 317
|
 Posted: Fri Dec 01, 2017 4:51 pm Post subject: Profile 17.0 |
 |
|
Hi,
When I read the news about profile 17.0 it is said that gcc 6 is a necessity, but I am unsure if emerge -e is necessary when I now only change to the profile to 17.0 and did already an emerge -e for gcc 6 on profile 13.0. |
|
| Back to top |
|
 |
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54232
Location: 56N 3W
|
| Posted: Fri Dec 01, 2017 5:15 pm Post subject: |
|
|
Spargeltarzan,
the /17.0/ profiles force the Position Independent Executable use flag from (-pie) to (pie).
This affects gcc and the way it builds everything. pie is now the default everywhere.
Mixing code built with -pie and pie is a very bad thing. It certainly breaks all of your static libraries.
Maybe more, so rebuilding everything is safe.
However, if you used a hardened profile before the change to /17.0/ then pie was already on (due to hardened) and no rebuilds for pie should be required.
The /17.0/ profiles also do other things but thats the major change.
gcc-5 is masked
There is a change to icu. Versions that don't build with gcc-6 are masked.
That's the things I'm aware of.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
Spargeltarzan
Guru
Joined: 23 Jul 2017
Posts: 317
|
| Posted: Fri Dec 01, 2017 5:25 pm Post subject: |
|
|
Thank you for the detailed answer, I started now my emerge -e. And actually sorry for the question, when reading the news item again it answered it actually in the last paragraph  |
|
| Back to top |
|
|
Spargeltarzan
Guru
Joined: 23 Jul 2017
Posts: 317
|
| Posted: Fri Dec 01, 2017 5:33 pm Post subject: |
|
|
| Shouldn't I see the PIE flag on every package now? I saw it only in the gcc package |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54232
Location: 56N 3W
|
| Posted: Fri Dec 01, 2017 6:14 pm Post subject: |
|
|
Spargeltarzan,
Once you compile gcc with USE=pie, it will apply it everywhere else by default.
pie is actually a CFLAG. As a USE flag on gcc, it builds gcc to apply -fpie (it might be -fPIE) everywhere.
This saves you adding it to CFLAGS. It becomes the default behaviour for gcc.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
pablo_supertux
Advocate
Joined: 25 Jan 2004
Posts: 2931
Location: Somewhere between reality and Middle-Earth and in Freiburg (Germany)
|
| Posted: Fri Dec 01, 2017 6:54 pm Post subject: |
|
|
I tried reading some articles about PIE (never heard of it before) and I didn't understand a thing 
what exactly are these PIE binaries und why are they the new default? As far as I can tell from the answers in this posts, once I've changed my profile to 17.0, I don't have to change my USE flags, right?
Too bad that I made the jump from gcc4 to gcc6 last week, I already had to recompile almost everything, now I have to do that again? That's bad timing for me 
_________________
A! Elbereth Gilthoniel!
silivren penna míriel
o menel aglar elenath,
Gilthoniel, A! Elbereth! |
|
| Back to top |
|
|
McQ
n00b
Joined: 27 Jul 2017
Posts: 5
|
| Posted: Fri Dec 01, 2017 8:50 pm Post subject: |
|
|
| pablo_supertux wrote: | I tried reading some articles about PIE (never heard of it before) and I didn't understand a thing
what exactly are these PIE binaries und why are they the new default? As far as I can tell from the answers in this posts, once I've changed my profile to 17.0, I don't have to change my USE flags, right?
Too bad that I made the jump from gcc4 to gcc6 last week, I already had to recompile almost everything, now I have to do that again? That's bad timing for me |
From a resent Gentoo Commits message
| Code: | We have just added (for all arches except arm and mips, these follow
later) a new set of profiles with release version 17.0 to the Gentoo
repository. These bring three changes:
1) The default C language version for applications is now C14.
This change is mostly relevant to Gentoo developers. It also
means, however, that compilers earlier than GCC 6 are masked
and not supported for use as a system compiler anymore. Feel
free to unmask them if you need them for specific applications.
2) Where supported, GCC will now build position-independent
executables (PIE) by default. This improves the overall
security fingerprint. The switch from non-PIE to PIE binaries,
however, requires some steps by users, as detailed below.
3) Up to now, hardened profiles were separate from the default
profile tree. Now they are moving into the 17.0 profile
as a feature there, similar to "no-multilib" and "systemd".
Please migrate away from the 13.0 profiles within the six weeks after
GCC 6.4.0 has been stabilized on your architecture. The 13.0 profiles
will be deprecated then and removed in half a year.
If you are not already running a hardened setup with PIE enabled, then
switching the profile involves the following steps:
If not already done,
* Use gcc-config to select gcc-6.4.0 or later as system compiler
* Re-source /etc/profile:
. /etc/profile
* Re-emerge libtool
emerge -1 sys-devel/libtool
Then,
* Select the new profile with eselect
* Re-emerge, in this sequence, gcc, binutils, and glibc
emerge -1 sys-devel/gcc:6.4.0
emerge -1 sys-devel/binutils
emerge -1 sys-libs/glibc
* Rebuild your entire system
emerge -e @world
Switching the profile from 13.0 to 17.0 modifies the settings of
GCC 6 to generate PIE executables by default; thus, you need to do
the rebuilds even if you have already used GCC 6 beforehand.
If you do not follow these steps you may get spurious build
failures when the linker tries unsuccessfully to combine non-PIE
and PIE code.
|
|
|
| Back to top |
|
|
Tony0945
Watchman
Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA
|
| Posted: Fri Dec 01, 2017 8:59 pm Post subject: |
|
|
Hmmm! Having a little trouble building with pie: | Code: | # USE="pie" emerge -a1v gcc::gentoo
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild R ] sys-devel/gcc-6.4.0:6.4.0::gentoo [6.4.0:6.4.0::local] USE="cxx fortran (multilib) nptl openmp pch sanitize ssp vtv (-altivec) (-awt) -cilk -debug -doc (-fixed-point) (-gcj) -go -graphite (-hardened) (-jit) (-libssp) -mpx -nls -objc -objc++ -objc-gc -pgo (-pie) -regression-test -vanilla" 13 KiB
Total: 1 package (1 reinstall), Size of downloads: 13 KiB
Would you like to merge these packages? [Yes/No] n
|
I also have pie enabled in make.conf and package.use, so why does portage want to build with -pie? I'm still on 13.0/desktop.
BTW, I first ran "quickpkg gcc" just in case. |
|
| Back to top |
|
|
pablo_supertux
Advocate
Joined: 25 Jan 2004
Posts: 2931
Location: Somewhere between reality and Middle-Earth and in Freiburg (Germany)
|
| Posted: Fri Dec 01, 2017 9:16 pm Post subject: |
|
|
| McQ wrote: | | pablo_supertux wrote: | I tried reading some articles about PIE (never heard of it before) and I didn't understand a thing
what exactly are these PIE binaries und why are they the new default? As far as I can tell from the answers in this posts, once I've changed my profile to 17.0, I don't have to change my USE flags, right?
Too bad that I made the jump from gcc4 to gcc6 last week, I already had to recompile almost everything, now I have to do that again? That's bad timing for me |
From a resent Gentoo Commits message
|
Yeah, I read that and because of it I went looking for an explanation of what PIE does, but the articles I've found were too technical and I didn't unterstand everything, that's why I asked.
_________________
A! Elbereth Gilthoniel!
silivren penna míriel
o menel aglar elenath,
Gilthoniel, A! Elbereth! |
|
| Back to top |
|
|
Tony0945
Watchman
Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA
|
| Posted: Fri Dec 01, 2017 9:43 pm Post subject: |
|
|
| pablo_supertux wrote: |
Yeah, I read that and because of it I went looking for an explanation of what PIE does, but the articles I've found were too technical and I didn't unterstand everything, that's why I asked. |
What bothers me is that it seems to be a RedHat idea. I wonder if I can keep the 13.0 profile as a custom profile. Every RedHat idea for at least the last five years has been a bummer. |
|
| Back to top |
|
|
Gentlenoob
n00b
Joined: 10 Apr 2008
Posts: 66
|
| Posted: Fri Dec 01, 2017 9:44 pm Post subject: |
|
|
Dear Gentooers,
I've some fairly old boxes (Atom / Core2 from ~6/10 years ago), so this update, if done as suggested, will take some days. Is there some way to avoid this, e.g. staying at '-pie' and still going to the 17.0 profile?
The news article talks about spurious build failures. Neddys advice sounds much more scary. What happens if for some reason 'emerge -e' fails in the midst and I'm left with said mix of pie and -pie?
Do the failures mentioned in the news affect only building or also runtime? Do they happen at random as 'spurious' may imply? Is important stuff affected or just some fringe packages?
Of course I'll thoroughly backup beforehand anyway.
Thanks a lot,
Ralph |
|
| Back to top |
|
|
Tony0945
Watchman
Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA
|
| Posted: Fri Dec 01, 2017 9:45 pm Post subject: |
|
|
| google "gcc pie good and bad" and see some scary things. |
|
| Back to top |
|
|
The Doctor
Moderator
Joined: 27 Jul 2010
Posts: 2678
|
| Posted: Fri Dec 01, 2017 9:58 pm Post subject: |
|
|
@Tony0945
Rest assured that PIE has been around for quite some time. This isn't the good idea fairy visiting Pottering again. As with all security you will take a small performance hit but I doubt you will notice, especially on a modern setup.
| Gentlenoob wrote: | | What happens if for some reason 'emerge -e' fails in the midst and I'm left with said mix of pie and -pie? |
Bad things if you don't fix it. You should take advantage of the --resume option to continue the build process.
They failures will be both run and build time. They would affect major packages as well as minor ones. However, there should be no reason why you cannot complete an emerge -e world so this should not affect you.
| Gentlenoob wrote: | | Of course I'll thoroughly backup beforehand anyway. |
Always wise.
_________________
First things first, but not necessarily in that order.
Apologies if I take a while to respond. I'm currently working on the dematerialization circuit for my blue box. |
|
| Back to top |
|
|
Soul Reaver
n00b
Joined: 09 Feb 2014
Posts: 10
|
| Posted: Fri Dec 01, 2017 11:01 pm Post subject: |
|
|
| So how does this relate to clang? I use clang as my main compiler, falling back to gcc for some incompatible packages. Is it sufficient to recompile those packages built with gcc? Are clang built binaries even compatible with gcc (+pie) built binaries? |
|
| Back to top |
|
|
Tony0945
Watchman
Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA
|
| Posted: Sat Dec 02, 2017 12:59 am Post subject: |
|
|
| The Doctor wrote: | @Tony0945
Rest assured that PIE has been around for quite some time. This isn't the good idea fairy visiting Pottering again. As with all security you will take a small performance hit but I doubt you will notice, especially on a modern setup. |
Thank you. I will accept YOUR judgement. |
|
| Back to top |
|
|
pablo_supertux
Advocate
Joined: 25 Jan 2004
Posts: 2931
Location: Somewhere between reality and Middle-Earth and in Freiburg (Germany)
|
| Posted: Sat Dec 02, 2017 1:20 am Post subject: |
|
|
| Gentlenoob wrote: |
I've some fairly old boxes (Atom / Core2 from ~6/10 years ago), so this update, if done as suggested, will take some days. Is there some way to avoid this, e.g. staying at '-pie' and still going to the 17.0 profile?
|
Unless you create your own profile base on 17.0 and write your own patches, I'm afraid this is not going to end well. The news text says
| Quote: |
Please migrate away from the 13.0 profiles within the six weeks after
GCC 6.4.0 has been stabilized on your architecture. The 13.0 profiles
will be deprecated then and removed in half a year.
|
and that means that the profile 13.0 will be gone in a couple of months.
| Quote: |
Do the failures mentioned in the news affect only building or also runtime? Do they happen at random as 'spurious' may imply? Is important stuff affected or just some fringe packages?
|
I'll first do a backup of the current running system and I'm going to wait a couple of weeks before doing that. I'm afraid that if I do it right now, I'm going to have some issues here and there and is going to be a pain to fix them. I think I'll wait 2 or 3 weeks and then do the update. Most of the initial issues might be resolved by then.
I still wish to know what's exactly PIE binaries do. And which other distributions are doing doing it.
_________________
A! Elbereth Gilthoniel!
silivren penna míriel
o menel aglar elenath,
Gilthoniel, A! Elbereth! |
|
| Back to top |
|
|
The Doctor
Moderator
Joined: 27 Jul 2010
Posts: 2678
|
| Posted: Sat Dec 02, 2017 1:44 am Post subject: |
|
|
| pablo_supertux wrote: | | I still wish to know what's exactly PIE binaries do. And which other distributions are doing doing it. |
Basically the programs can be relocated in memory. This makes certain types of attacks much harder to do. It has been a feature of hardened Gentoo for quite a while.
Aside from Linux Mac OS X and iOS use it. OpenBSD has been using it since 2013 by default. Fedora 23 made it a default option and Ubuntu 17 did as well. Finding a complete list of Linux distros is rather more difficult, but the lit certainly includes Alpine Linux.
_________________
First things first, but not necessarily in that order.
Apologies if I take a while to respond. I'm currently working on the dematerialization circuit for my blue box. |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21619
|
| Posted: Sat Dec 02, 2017 1:51 am Post subject: |
|
|
PIE = Position Independent Executable. As The Doctor says, this makes them trivial to relocate at load time. Malicious code commonly benefits from reusing fragments of the exploited application. By relocating them to an address that an attacker cannot easily predict, the kernel can make it more difficult for malicious code to find those fragments, thus making complex exploits more difficult to write and less reliable to run. (This does nothing to complicate exploits that aim simply to crash the victim program.) This relocation technique is commonly called Address Space Layout Randomization (ASLR). It can be done with non-PIE programs, but it's less efficient.
Starting in Windows Vista, Microsoft implemented a very limited form of ASLR. They may have improved it subsequently, but when last I checked, it had some notable weaknesses relative to the Linux implementation (which Brad Spengler will happily tell you is itself not as good as the implementation he/PaX team use in the now-unavailable grsecurity project). |
|
| Back to top |
|
|
Tony0945
Watchman
Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA
|
| Posted: Sat Dec 02, 2017 2:21 am Post subject: |
|
|
OK, I see that pie goes with the profile and you have to change profile before you build with pie.
On my downstairs machine I did the following:
1. quickpkg gcc libtool binutils glibc
2. eselect profile 17 (on my machine selects profile 17.0/desktop
3. bash updateprofile
updateprofilescript is: | Code: | emerge -1v sys-devel/gcc:6.4.0 && emerge -1v sys-devel/binutils && emerge -1v sys-libs/glibc && emerge -e @world && echo "It worked!"
|
Running step 3 now. I first took the precaution of shutting down xdm. I'll report sometime tomorrow how it went.
I usually emerge -e system then -e world, but I'm following orders. Sorry I didn't run "time bash updateprofile" but it may be finished when I wake up tomorrow. Maybe not. Usually the last step alone takes about 11 hours.
EDIT: Fourteen hours later and the screen is emerging 328 of 526. Originally there were 1052, so why now 528? I think emerge@preserved-rebuild is running. Maybe it will be done tonight. Maybe. I should have done benchmarks before and after to see what the performance hit is.
Another thought. Perhaps I should rebuild the kernel also. emerge -e will just refetch the source not build the kernel.
Last edited by Tony0945 on Sat Dec 02, 2017 5:09 pm; edited 1 time in total |
|
| Back to top |
|
|
Fitzcarraldo
Advocate
Joined: 30 Aug 2008
Posts: 2034
Location: United Kingdom
|
| Posted: Sat Dec 02, 2017 6:25 am Post subject: |
|
|
If anyone arrives in this thread when searching for solutions to the problems listed below that may occur while carrying out the procedure specified in News item "2017-11-30-new-17-profiles"...
In my case the following three packages failed to re-merge during "emerge -e @world" (I used "emerge --resume --skip-first" to continue each time):
a) media-libs/libbluray-1.0.1 "configure: error: could not find jni.h"
See https://bugs.gentoo.org/621992
b) dev-libs/klibc-2.0.4-r2 "kernel/bounds.c:1:0: error: code model kernel does not support PIC mode"
See https://bugs.gentoo.org/621848
I applied the fix in Comment 4 of the bug report to an ebuild for klibc-2.0.4-r2 in my local overlay.
c) app-emulation/virtualbox-modules-5.1.26 "cc1: error: incompatible gcc/plugin versions"
I had to rebuild the kernel (4.12.12-gentoo) in order to be able to re-merge virtualbox-modules successfully.
And to rebuild the kernel I had to use the command "make clean" before "make && make modules_install"
in order to prevent the "make" command failing with the following error message referring to the old version
of gcc instead of 6.4.0:
No rule to make target '/usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0/include/stddef.h',
needed by '/usr/src/linux-4.12.12-gentoo/tools/objtool/fixdep.o'. Stop.
After rebuilding the kernel, I used the command "emerge @module-rebuild" to re-merge
virtualbox-modules-5.1.26 successfully.
EDIT: Corrected '@system' to '@world'.
_________________
Clevo W230SS: amd64, VIDEO_CARDS="intel modesetting nvidia".
Compal NBLB2: ~amd64, xf86-video-ati. Dual boot Win 7 Pro 64-bit.
OpenRC udev elogind & KDE on both.
Fitzcarraldo's blog
Last edited by Fitzcarraldo on Sat Dec 02, 2017 4:05 pm; edited 1 time in total |
|
| Back to top |
|
|
wenguiGwok
n00b
Joined: 02 Dec 2017
Posts: 4
|
| Posted: Sat Dec 02, 2017 6:29 am Post subject: pie not shown as set by equery |
|
|
i did as instructed after receiving the 17.0 upgrade notif, namely, change the profile, rebuild gcc and toolchain, then emerge -e @wolrd.
when i checked the USE for gcc itself, here is :
| Code: |
⚡ equery u gcc
...
* Found these USE flags for sys-devel/gcc-6.4.0:
U I
...
- + pie : Build programs as Position Independent Executables (a security hardening technique)
|
seems like pie is not set. nor is pie listed in USE shown by `emerge --info`.
however, emerge gives:
| Code: |
⚡ emerge -qpv gcc
[ebuild R ] sys-devel/gcc-6.4.0 USE="cxx fortran (multilib) nls nptl openmp pch (pie) sanitize ssp vtv (-altivec) (-awt) -cilk -debug -doc (-fixed-point) (-gcj) -go -graphite (-hardened) (-jit) (-libssp) -mpx -objc -objc++ -objc-gc -pgo -regression-test -vanilla"
|
which indicates pie is set on.
some guys on IRC kindly told me they are having the same results. so just be easy with that and believe emerge not equery this time.
actually the 4th comment in this thread has made it clear.
| NeddySeagoon wrote: | Spargeltarzan,
Once you compile gcc with USE=pie, it will apply it everywhere else by default.
pie is actually a CFLAG. As a USE flag on gcc, it builds gcc to apply -fpie (it might be -fPIE) everywhere.
This saves you adding it to CFLAGS. It becomes the default behaviour for gcc. |
just a memo for those who might have been confused as i was.
Last edited by wenguiGwok on Sat Dec 02, 2017 11:21 am; edited 1 time in total |
|
| Back to top |
|
|
teefax
n00b
Joined: 14 Jan 2015
Posts: 11
Location: Germany
|
| Posted: Sat Dec 02, 2017 7:33 am Post subject: |
|
|
| Is there any way to avoid the @world rebuild? |
|
| Back to top |
|
|
fedeliallalinea
Administrator
Joined: 08 Mar 2003
Posts: 30905
Location: here
|
| Posted: Sat Dec 02, 2017 7:53 am Post subject: |
|
|
| teefax wrote: | | Is there any way to avoid the @world rebuild? |
If you want switch to 17 no
| NeddySeagoon wrote: | Mixing code built with -pie and pie is a very bad thing. It certainly breaks all of your static libraries.
Maybe more, so rebuilding everything is safe.
However, if you used a hardened profile before the change to /17.0/ then pie was already on (due to hardened) and no rebuilds for pie should be required. |
_________________
Questions are guaranteed in life; Answers aren't. |
|
| Back to top |
|
|
teefax
n00b
Joined: 14 Jan 2015
Posts: 11
Location: Germany
|
| Posted: Sat Dec 02, 2017 8:03 am Post subject: |
|
|
Yes, I've read that. I meant, is it possible to disable PIE?
It seems the PIE use flag is forced for the 17.0 profile. Is it sufficient to add -fno-pie to my CFLAGS in make.conf ? |
|
| Back to top |
|
|
Myu
Apprentice
Joined: 22 Oct 2014
Posts: 164
Location: Belgium
|
| Posted: Sat Dec 02, 2017 8:49 am Post subject: |
|
|
I did the step mentioned in the news item and moved to 17.0/desktop and after 977 packages rebuild overnight, I got my system back, 0 build failures !
Thank you to the Gentoo team/community as a whole ! I'm really impressed and pleased, as I understand such an event (rebuilding world) is not that common (I use Gentoo since 2014)
_________________
Gentoo stable with bits of ~amd64 // Xfce 4.13 + Compiz Reloaded. |
|
| Back to top |
|
|
|
Portage & Programming
