Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Profile 17.0
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next  
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 5754

PostPosted: Tue Dec 05, 2017 4:03 am    Post subject: Reply with quote

eccerr0r wrote:
From what I hear, PIE on Linux is only good for security (and libraries that can be used in multiple programs).

As already mentioned (by me and in more detail by Hu), shared libraries use PIC anyway. PIE only means to do the same also for the main program (up to a minor technical difference).
Quote:
This is once again something done that sacrifices speed to solve a problem that should have been solved by writing the software correctly (i.e., securely).

Like the whole unix permission system. Yes, indeed, if every microsecond counts for your system and if you have a completely isolated system with error-free software, it just burns unnecessary CPU cycles to check for various privileges in the kernel, and using PIE becomes a bad idea.

On all other systems, in particular those which ever connect to other machines (left alone untrusted machines like on the internet), it is a bad advice to avoid privilege separation or PIE even if the former (and in exceptional cases on x86 also the latter) has a small performance impact.

Edit: Fix typo


Last edited by mv on Tue Dec 05, 2017 4:33 am; edited 1 time in total
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 5754

PostPosted: Tue Dec 05, 2017 4:26 am    Post subject: Reply with quote

eccerr0r wrote:
I'm worried how much longer Firefox will take to parse a webpage [...]

What you are doing here is classical spreading of FUD. I will no further comment on these non-examples.
Situations where pie vs. no-pie might be relevant is e.g. if you use an x86 gentoo machine for mining (I will not discuss how clever this is) and had already done everything else to optimize every single CPU cycle for this single process. Of course, then it is not forbidden to pass CFLAGS=-nopie when emerging your particular mining package if you want to rely that this package has no bugs. ;)
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 6361
Location: almost Mile High in the USA

PostPosted: Tue Dec 05, 2017 7:17 am    Post subject: Reply with quote

That's the thing, without testing, we don't know for sure, plus I've been noticing constant slowdown on x86 on things that supposedly hasn't changed much. My specific concern is that programs like Firefox have already been growing so slow that it's already painful. Continually adding mere 1% performance hits here and there for whatever reason, it adds up.

I would imagine that the checks for the unix file permission system is much less than 1% as it only need to be checked when the resource is being used. On the other end of the spectrum, it's also not nearly as bad as if the kernel/cpu needs to trap and emulate frequently used instructions. To implement position independence on x86 it's somewhere in between, but it depends on the actual program to know where. It's several instructions to do the calculation - even if it averages one every 100 instructions, that gets that roughly 1% penalty. But it's good if gcc will try to avoid doing this if it doesn't need to.

If all that's needed is to simply add CFLAGS=-nopie to revert back to old behavior, that's fine. Or if this turns out to be a penalty of 0.1% or less without intervention, but there was a reason why this wasn't done by default a long time ago.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
tholin
Tux's lil' helper
Tux's lil' helper


Joined: 04 Oct 2008
Posts: 97

PostPosted: Tue Dec 05, 2017 9:51 am    Post subject: Reply with quote

eccerr0r wrote:
I'm worried how much longer Firefox will take to parse a webpage, which will be affected by this change. Perhaps something that uses webkit-gtk won't change much, probably already compiled with PIC.

I run a stable system and haven't changed profile yet. I run gcc-5.4.0, firefox-52.4.0 and default/linux/amd64/13.0/desktop.

Code:
# hardening-check /usr/bin/firefox
/usr/bin/firefox:
 Position Independent Executable: yes
 Stack protected: yes
 Fortify Source functions: yes
 Read-only relocations: yes
 Immediate binding: no, not found!


/proc/`pidof firefox`/maps confirms that the executable is mapped to a random location so even without the new profiles firefox is PIE.

Lets say for the sake of argument that firefox was built without PIE, how much speedup would that give you? Firefox is one of those programs that build almost all logic into shared libraries and the main executable is only a small shim for those libraries.

/usr/lib64/firefox/firefox 283K
/usr/lib64/firefox/libxul.so 74M

On a running firefox that shim only makes up 0.18% of the code in the process address space so even if you could make that code execute 1% faster it wouldn't make any difference anyway.
Back to top
View user's profile Send private message
tld
Veteran
Veteran


Joined: 09 Dec 2003
Posts: 1222

PostPosted: Tue Dec 05, 2017 10:29 am    Post subject: Reply with quote

With my x86 systems (especially my MythTV systems) I'm actually more concerned about the need to recompile everything than I am about the potential for a performance hit...though that's a bit of a concern as well.

Recompiling everything is a pretty huge deal on these systems. Even the recompile required for gcc 4 to 5 for the C++11 ABI was significant. The good news there was supposed to be that wasn't needed when going from gcc 5 to 6. Now suddently I find out I need to recompile everything...something I haven't had to do since going from gcc 3 to 4. MythTV unfortunately requires dev-qt/qtwebkit:5. That one package alone takes 7 hours to compile. Horrible. Especially if gcc 6 is even a little slower than previous versions that could take days on those systems (and maybe longer on this one as I have a lot more installed).

I suppose I could compile things niced and continue using the systems and hope for the best but this is pretty awful. I've yet to read anything that's convinced me it's really necessary.

EDIT: Just to clarify: I understand that a mix of pie and -pie is an issue...I just haven't seen anything that convinces me that using -pie globally would be.

Tom
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 6361
Location: almost Mile High in the USA

PostPosted: Tue Dec 05, 2017 6:04 pm    Post subject: Reply with quote

It very well be that we've already experiencing the performance loss by position independence, then. I have never used webkit based browsers on x86-32 but the firefox slowness has been around for a while among other programs simple programs that used to perform acceptably but no longer, even if it's due to code bloat instead of PI, but ultimately PI can only make things worse.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 2395
Location: Canada

PostPosted: Tue Dec 05, 2017 8:15 pm    Post subject: gcc 6 and gcc 5 coexistence under 17.0 profile Reply with quote

So gcc-5 is masked under 17.0 profile. Supposedly, to use it for some programs that need it (say it is the last version to support gcj), I need to umask it. However will it be able to generate working code, given that all the libraries are compiled with gcc-6 and +pie option ?
Back to top
View user's profile Send private message
The Doctor
Moderator
Moderator


Joined: 27 Jul 2010
Posts: 2340

PostPosted: Tue Dec 05, 2017 8:33 pm    Post subject: Re: gcc 6 and gcc 5 coexistence under 17.0 profile Reply with quote

dmpogo wrote:
So gcc-5 is masked under 17.0 profile. Supposedly, to use it for some programs that need it (say it is the last version to support gcj), I need to umask it. However will it be able to generate working code, given that all the libraries are compiled with gcc-6 and +pie option ?
It should. The problem is trying to link a static library into a PIE executable. Linking PIE libraries is business as usual even with gcc-5.
_________________
First things first, but not necessarily in that order.
Back to top
View user's profile Send private message
tholin
Tux's lil' helper
Tux's lil' helper


Joined: 04 Oct 2008
Posts: 97

PostPosted: Wed Dec 06, 2017 9:53 am    Post subject: Re: gcc 6 and gcc 5 coexistence under 17.0 profile Reply with quote

The Doctor wrote:
dmpogo wrote:
So gcc-5 is masked under 17.0 profile. Supposedly, to use it for some programs that need it (say it is the last version to support gcj), I need to umask it. However will it be able to generate working code, given that all the libraries are compiled with gcc-6 and +pie option ?
It should. The problem is trying to link a static library into a PIE executable. Linking PIE libraries is business as usual even with gcc-5.

Makes you wonder what's the point of running emerge -e? On my system 31% of all installed packages are not compiled with gcc at all. They are fonts, icons, time zone data, pure python, perl or java programs. Of the remaining 69% there are a lot of libraries that only install shared .so files and they are PIC already. I would guess only 50% of all packages benefit from being rebuilt with emerge -e, the rest just use up time and SSD write cycles.

There has to be a better way™
Back to top
View user's profile Send private message
The Doctor
Moderator
Moderator


Joined: 27 Jul 2010
Posts: 2340

PostPosted: Wed Dec 06, 2017 4:33 pm    Post subject: Reply with quote

Quote:
Makes you wonder what's the point of running emerge -e? On my system 31% of all installed packages are not compiled with gcc at all. They are fonts, icons, time zone data, pure python, perl or java programs. Of the remaining 69% there are a lot of libraries that only install shared .so files and they are PIC already. I would guess only 50% of all packages benefit from being rebuilt with emerge -e, the rest just use up time and SSD write cycles.
The problem is identifying those packages.
_________________
First things first, but not necessarily in that order.
Back to top
View user's profile Send private message
MrCat
n00b
n00b


Joined: 06 Dec 2017
Posts: 1

PostPosted: Wed Dec 06, 2017 5:22 pm    Post subject: Reply with quote

It all went smooth for me, I even forgot about this pie stuff until today. In case anyone is a dumb as I'm, don't forget to @module-rebuild if you rebuild your kernel using the new profiles... you can imagine what happens if you don't.
Back to top
View user's profile Send private message
wjb
Guru
Guru


Joined: 10 Jul 2005
Posts: 386
Location: Fife, Scotland

PostPosted: Wed Dec 06, 2017 10:05 pm    Post subject: Reply with quote

@The Doctor
If its just needed to rebuild packages that install .a files, I think I have some python that will more or less do that. Take about an hour to bash into shape.

Its taken a day and a half to update the i7, and not looking forward to a full rebuild on the much slower laptop.

EDIT: https://github.com/wwjjbb/rebuild-static-libs


Last edited by wjb on Thu Dec 07, 2017 12:11 am; edited 1 time in total
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 39300
Location: 56N 3W

PostPosted: Wed Dec 06, 2017 10:55 pm    Post subject: Reply with quote

wjb,

I think it wan mv who stated the same thing and gave a one liner.
I'll link to the post if I can find it.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Sembiance
n00b
n00b


Joined: 03 Oct 2003
Posts: 14
Location: Asheville, NC, USA

PostPosted: Thu Dec 07, 2017 2:14 pm    Post subject: Reply with quote

wjb wrote:
@The Doctor
If its just needed to rebuild packages that install .a files, I think I have some python that will more or less do that. Take about an hour to bash into shape.

Its taken a day and a half to update the i7, and not looking forward to a full rebuild on the much slower laptop.

EDIT: https://github.com/wwjjbb/rebuild-static-libs


Or just disable PIE.
Code:
eselect profile set <choose profile 17>
emerge --sync
mkdir -p "/etc/portage/profile/"
echo "sys-devel/gcc -pie" >> /etc/portage/profile/package.use.force

# Add "-pie" to your USE= flag in /etc/portage/make.conf (or /etc/make.conf, whichever exists)

Now just emerge update as normal, like any other sync/update operation.

No need to recompile world, deal with static lib issues, PIE incompatible packages, incompatibility with ThreadSanitizer, and documented PIE performance issues.

This is why Gentoo is great, we have options! :)
Back to top
View user's profile Send private message
tld
Veteran
Veteran


Joined: 09 Dec 2003
Posts: 1222

PostPosted: Thu Dec 07, 2017 3:37 pm    Post subject: Reply with quote

Sembiance wrote:
Or just disable PIE.
I'm most likely taking this option on my x86 machines. as mentioned above I really don't want to recompile those unless there's just no way to avoid it.

Thanks to @klynastor for this original suggestion! I have one question though. I'm having a really hard time finding anything that clarifies the /etc/portage/profile directory. Am I to understand that any files in there are simply added to (possibly overriding) your selected profile? For example, where I could select the x86/17.0/desktop profile, and then just add that file?

Thanks!
Tom
Back to top
View user's profile Send private message
Sembiance
n00b
n00b


Joined: 03 Oct 2003
Posts: 14
Location: Asheville, NC, USA

PostPosted: Thu Dec 07, 2017 3:47 pm    Post subject: Reply with quote

tld wrote:
Sembiance wrote:
Or just disable PIE.
I'm most likely taking this option on my x86 machines. as mentioned above I really don't want to recompile those unless there's just no way to avoid it.

Thanks to @klynastor for this original suggestion! I have one question though. I'm having a really hard time finding anything that clarifies the /etc/portage/profile directory. Am I to understand that any files in there are simply added to (possibly overriding) your selected profile? For example, where I could select the x86/17.0/desktop profile, and then just add that file?

Thanks!
Tom


According to that paper I linked, on x86 you take a 26% performance hit from PIE. Which is, significant.
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 2395
Location: Canada

PostPosted: Thu Dec 07, 2017 3:55 pm    Post subject: Reply with quote

The Doctor wrote:
Quote:
Makes you wonder what's the point of running emerge -e? On my system 31% of all installed packages are not compiled with gcc at all. They are fonts, icons, time zone data, pure python, perl or java programs. Of the remaining 69% there are a lot of libraries that only install shared .so files and they are PIC already. I would guess only 50% of all packages benefit from being rebuilt with emerge -e, the rest just use up time and SSD write cycles.
The problem is identifying those packages.


On page two user mv gave that command

Code:

emerge -1 /lib*/*.a /usr/lib*/*.a


does it miss something, besides kernel and third party kernel modules ?
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 2395
Location: Canada

PostPosted: Thu Dec 07, 2017 4:04 pm    Post subject: Reply with quote

Sembiance wrote:
tld wrote:
Sembiance wrote:
Or just disable PIE.
I'm most likely taking this option on my x86 machines. as mentioned above I really don't want to recompile those unless there's just no way to avoid it.

Thanks to @klynastor for this original suggestion! I have one question though. I'm having a really hard time finding anything that clarifies the /etc/portage/profile directory. Am I to understand that any files in there are simply added to (possibly overriding) your selected profile? For example, where I could select the x86/17.0/desktop profile, and then just add that file?

Thanks!
Tom


According to that paper I linked, on x86 you take a 26% performance hit from PIE. Which is, significant.


Well, on some benchmarks, but average of 10\% is also nontrivial. Interesting from the same paper there is a remark (not as deeply investigated) for 3.64% average
penalty of x64 systems as well.
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 5975

PostPosted: Thu Dec 07, 2017 5:03 pm    Post subject: Reply with quote

The real problem with PIE is that it doesn't protect from malware presence, it only try to make their life harder.
So where PIE would be a saver, well, it would be a real saver when someone is trying to find an hole in your system, which imply someone has access to that system.

And that is what is trouble me then: why PIE on everyone when PIE has specific usage.

When PIE would be of help out of malware: only when user have a weak user password allowing someone to use that account to execute program using that account searching a breach in memory.
This is really not typical use case of a "desktop" computer.

Of course user could have only 1 account, but weak password with it, that might be typical desktop case too, but really, do anyone think if someone has such computer configure to allow access to that user account with weak password, any attacker would had trouble because of PIE? Such bad setup might have way easier "legit" things already setup (lol at weak user password with ssh access and sudo without password because user is in wheel group : free root for everyone!)

So for me, PIE would be of help if you have many accounts and one might be weak and you fail at detecting the weak account, that's something seen on big server computer (because the number of users may bypass your vigilance) ; not really the typical desktop computer anyone use.

So it raise the question: if PIE have a speed impact (and it does, no need to quantify it, we all know it does), and PIE is gold with multi-users, why PIE on everyone per default?
Do you really need PIE on your NAS? on your personal fileserver? on your NFS server? on your desktop? on your DLNA server? ...
If all those case are only aim at YOUR own usage, you don't have internet ssh to them, because you typically don't need any, and they are all normally outside DMZ because they never provide anything to anyone on internet.
Why would my DLNA/NFS/sound streaming whatever personal computer for my own use have PIE enable then?

If i have a computer that do provide internet service (even as little as an ssh access), i might consider making it all PIE enable (well, you better use strong password, ssh-key only login, and a good firewall then PIE in real), but it still might make sense for me. But it's not really typical use case of a computer.
The idea is: if i have a door, even locked, then there is a door and maybe someone might open it. Once he is in, PIE will make his life harder, it's shitty he is in, but at least, i might see his presence while he battle against my programs that are all PIE).
I would also PIE a laptop computer if i move with it, connecting to insecure (anyway random) networks...

But profile 17.0 doesn't make any distinction, everyone must use PIE because, all programs must be secure ; for what? Prevent malware from using your computer as bot? (PIE doesn't prevent malware, it might make malware's life harder, it doesn't prevent you from having one), prevent someone using your computer when you have no internet entry? Oh, to prevent someone that have access to your keyboard to find a way to get root?

So PIE is a good feature in hardened profiles, because you should know if you have need for hardening your computer or not.
But it should remain like it was, a default option that make sense in hardened profiles, a stupid option to make default in non hardened profiles.
What's the next move: make mandatory kernel with selinux?

To cite someone:
http://www.zdnet.com/article/linux-creator-linus-torvalds-this-is-what-drives-me-nuts-about-it-security/ wrote:
"Because in the end, those users really do matter. Without those users, your system may be 'secure', but all your security work was still just masturbation. You didn't do anything useful at all in the end."

Read what he said, because i think he is just pointing what gentoo devs have done with 17.0 profile! Putting on a not useful option on everyone for the sake of security, when this option doesn't add any benefits to most of user of that 17.0 profile!
Worst as it was made available to everyone with a news, and you can see every users just switching to it without thinking about it ; a news told them how, so they do has told without waiting! But the tree is clearly not ready for PIE on per default, and it's a shitty move by our devs, because you can see how many open threads are there with user unable to not fall into some problem.

17.0 profile should default c++14 and make gcc 6.4 mandatory if they wish, but adding PIE as mandatory is really, really, a bad decision, made only to yes: "masturbate with security"
Back to top
View user's profile Send private message
proteusx
Apprentice
Apprentice


Joined: 21 Jan 2008
Posts: 150

PostPosted: Thu Dec 07, 2017 5:06 pm    Post subject: Profile 17.0 without PIE and and 'emerge -e world' Reply with quote

Here is what I did to migrate to profile 17.0 without PIE and without recompiling @world.


  • I do not need PIE
    Code:
    echo "sys-devle/gcc pie" >> /etc/portage/profile/package.use.mask/profile-17.0.mask
    echo "*/* -pie" >> /etc/portage/package.use/global.use


  • Unmask my current compiler sys-devl/gcc:5.4.0 (used by my cross compilers too).
    Code:
    echo "sys-devel/gcc:5.4.0" >> /etc/portage/package.unmask/profile-17.0.unmask


  • I need pdftk which is masked in profile 17.0
    Code:
    echo "app-text/pdftk:0" >> /etc/portage/package.unmask/profile-17.0.unmask


  • I need qtcore:4[icu] because I still use kde4.
    Code:
    echo "dev-qt/qtcore:4 -icu" >> /etc/portage/profile/package.use.mask/profile-17.0.mask


  • Select the new profile and emerge world.
    Code:
    # eselect default/linux/amd64/17.0/desktop
    # emerge -DuNa world  --with-bdeps=y

    Nothing to merge; quitting.


Thus I can continue with the new profile as if nothing happened.
I post this in case it is useful to someone else.
Back to top
View user's profile Send private message
wjb
Guru
Guru


Joined: 10 Jul 2005
Posts: 386
Location: Fife, Scotland

PostPosted: Thu Dec 07, 2017 5:39 pm    Post subject: Reply with quote

dmpogo wrote:

...

Code:

emerge -1 /lib*/*.a /usr/lib*/*.a


does it miss something, besides kernel and third party kernel modules ?


It misses a lot - the libraries in /usr/lib* can be nested quite deep, e.g. dev-python/numpy needs something like /usr/lib*/*/*/*/*/*/*.a.

On my system it finds about a third of the packages with static libs.

EDIT: ok, ignoring all the haskell packages I seem to have, its closer to 70%.
Back to top
View user's profile Send private message
allistarM
Tux's lil' helper
Tux's lil' helper


Joined: 22 Jul 2004
Posts: 76

PostPosted: Thu Dec 07, 2017 8:03 pm    Post subject: Reply with quote

A rebuild of everything will take some time. Am I safe to use distcc for this when one of the build hosts is still on the 13.0 profile using gcc 5? If I switch to gcc 6.4 on all build hosts will that be sufficient or do I need to do this rebuild entirely on a single server? Is the PIE thing a compile time change or a prt of the linking process? If the latter then I expect distcc will be fine.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 5754

PostPosted: Thu Dec 07, 2017 8:17 pm    Post subject: Reply with quote

krinn wrote:
When PIE would be of help out of malware: only when user have a weak user password

It helps in much more situations than you suggest.
A typical situation is when you browse the web and, say (for the sake of example) your jpeg library has a certain security bug like e.g. a buffer overflow. Without pie, it might perhaps be relatively simple to craft a special "picture" to execute arbitrary code (with the permissions of the user who views the picture; but do not forget that there might even be further vulnerabilities which allow to raise privileges).
In contrast, with pie, it is practically^1 excluded that a buffer overflow might be exploited this way.

^1 it depends of course on the type of security bug whether pie actually helps, but for a relatively large class of common bugs it really prevents reliably arbitrary code execution by an attacker.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 11434

PostPosted: Fri Dec 08, 2017 1:51 am    Post subject: Reply with quote

allistarM wrote:
Am I safe to use distcc for this when one of the build hosts is still on the 13.0 profile using gcc 5?
Yes, if you meet certain elaborate conditions that you are unlikely to satisfy. ;) To be safe, you must:
  • Use only volunteers which have installed a compatible gcc (in this case, gcc 6.4 with PIE force-enabled).
  • Prevent distribution of any jobs which would run a gcc other than gcc-6.4-pie. Some build systems run gcc as a bare name, which, lacking corrective wrappers, would run gcc on the volunteer, again as a bare name. If that doesn't map to gcc-6.4-pie, then your volunteer returns code that isn't forced-PIE.

allistarM wrote:
If the latter then I expect distcc will be fine.
PIE is a compile-time property. It affects what code is used to implement the compiled functions.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 39300
Location: 56N 3W

PostPosted: Fri Dec 08, 2017 9:54 am    Post subject: Reply with quote

Hu,

What about adding -fpie to CFLAGS on the client?
That would pass -fpie no the helpers and pie code would arrive back.

Is that the right flag though, I think there is also a -fPIE?

I can see problems with those packages that set their own CFLAGS. The -fpie might get dropped.
Where its an internal gcc setting, that's not possible.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next
Page 5 of 8

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum