Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Ninja vs. PIE
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
NTU
Apprentice
Apprentice


Joined: 17 Jul 2015
Posts: 164

PostPosted: Fri Mar 22, 2019 2:39 am    Post subject: Ninja vs. PIE Reply with quote

Hi,

Projects have been adding ninja support as an alternative to GNU make. The problem is that I'm trying to build a secure system that uses position independent executables, but if you use -fPIE in your CFLAGS and -pie in your LDFLAGS, you get "undefined reference to main" errors if you use ninja instead of GNU make. gdk-pixbuf and pixman are the most recent additions it seems, so I added those to package.env that use nopie.conf instead of my typical flags. This list will continue to grow and eventually, you won't be able to use -fPIE -pie anymore because everyone will be using ninja at this rate.

Either fix ninja to work with position independent executables or give me an option to use make instead. Without copying and pasting hundreds of lines of stuff in my heavily customized /etc/portage dir, in short:

make.conf:
Code:
CFLAGS="-O2 -fPIC -fPIE -fstack-protector-strong -fstack-clash-protection"
CXXFLAGS="${CFLAGS}"
LDFLAGS="${LDFLAGS} ${CFLAGS} -pie -Wl,-z,now -Wl,-z,relro"

env/nopie.conf:
Code:
CFLAGS="-O2 -fPIC -fstack-protector-strong -fstack-clash-protection"
CXXFLAGS="${CFLAGS}"
LDFLAGS="${LDFLAGS} ${CFLAGS} -Wl,-z,now -Wl,-z,relro"

package.env
Code:
some-package/that-uses-ninja-or-does-not-work-with-fpie-pie nopie.conf


I never used Ninja nor really read into it, but I just did now, and I'm not sure if this is actually a Meson issue or a Ninja problem. Meson is the alternative build system to GNU autotools, Ninja uses the Meson build system instead of the GNU build system. I can't see how Meson is the problem here judging from this example:

https://cgit.freedesktop.org/mesa/drm/tree/meson.build

So I'm only assuming that this PIE problem comes from within Ninja and not directly related to Meson.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14159

PostPosted: Fri Mar 22, 2019 4:31 am    Post subject: Reply with quote

Hardened systems normally build everything as position independent. Some of them have installed projects that are ninja-based, so there cannot be a hard incompatibility. Could you provide a minimal project which fails for you?
Back to top
View user's profile Send private message
NTU
Apprentice
Apprentice


Joined: 17 Jul 2015
Posts: 164

PostPosted: Fri Mar 22, 2019 5:47 am    Post subject: Reply with quote

Hu wrote:
Hardened systems normally build everything as position independent. Some of them have installed projects that are ninja-based, so there cannot be a hard incompatibility. Could you provide a minimal project which fails for you?

Libdrm should be a small enough example. Please note that there is a difference between PIC and PIE. PIC always works except in some rare instances like non-PIC friendly ffmpeg or mesa code. PIE and Ninja are mutually exclusive so far in my experience. I rather not have to make a hello world project on Github that uses Meson w/ Ninja to confirm but this is probably the best way to REALLY find out.

I don't use the hardened profile nor the hardened USE flag because the whole concept is broken IMO. A hardened profile should use -fstack-protector-all and force PIC+PIE (according to this page: https://wiki.gentoo.org/wiki/Hardened/Gentoo_Hardened_and_Stack_Clash#Effect_of_Gentoo_Hardened_protections ) but when you use hardening-check, it's actually not. I found that adding -fPIC -fPIE and -fstack-protector-all or -fstack-protector-strong by hand to make.conf does a better job at hardening than the profile defaults.

Packages that are not PIE-friendly are added to package.env with my nopie.conf file amended to them. Since Gentoo has been adopting Meson (due to upstream maintainers adding Meson support in addition to GNU autotools) this file has been growing, and more and more packages are in need of nopie.conf in order to successfully compile.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6293

PostPosted: Fri Mar 22, 2019 6:21 am    Post subject: Reply with quote

I think you are confusing some things. First of all, ninja is not a build system but only the lowest level. It only does what the actual build system tells it to. The actual build system is probably either cmake or meson these days.
Second, you will indeed have a hard time compiling a library like the mentioned pixman or libdrm: As you observed, they have to be compiled with pic, and any attempt to compile them with PIE will fail. This has nothing to do with the build system. The situation is a bit more complicated for gdk-pixbuf, because this project contains libraries and executables, but there do not exist separate C*FLAGS for libraries and executables. If these packages worked previously wth -fPIE, then only because they manually filtered -fPIE in their build system (either globally or at least for the executables). That's why the hardened toolchain is preferable since it does these things without having to patch the build system for every single package. BTW, have you selected the correct gcc profile after compiling with hardened?
Back to top
View user's profile Send private message
NTU
Apprentice
Apprentice


Joined: 17 Jul 2015
Posts: 164

PostPosted: Fri Mar 22, 2019 9:10 am    Post subject: Reply with quote

mv wrote:
I think you are confusing some things. First of all, ninja is not a build system but only the lowest level. It only does what the actual build system tells it to. The actual build system is probably either cmake or meson these days.
Second, you will indeed have a hard time compiling a library like the mentioned pixman or libdrm: As you observed, they have to be compiled with pic, and any attempt to compile them with PIE will fail.

This is false, if you compile libdrm using GNU make and PIE, it works just fine, as have other projects. I know this because up until Gentoo switched from emake to meson_* it worked just fine. pixman-0.36.0 should still work with -fPIE -pie, but I know for a fact that 0.38.0 doesn't. You can test this yourself, and the flags were not filtered by anything.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6293

PostPosted: Fri Mar 22, 2019 9:30 am    Post subject: Reply with quote

NTU wrote:
This is false, if you compile libdrm using GNU make and PIE, it works just fine, as have other projects.

Maybe gcc ignores -fPIE if flags for pic are selected (which it should be for a library).
And this might also be the reason why it does not work for you with meson: Maybe with meson by default your explicitly specified cflags occur after the added pic flags and with autotools they occur before (or vice versa).
In any case, if -fPIE should really become active when compiling a library, the produced library is broken.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14159

PostPosted: Sat Mar 23, 2019 12:19 am    Post subject: Reply with quote

NTU wrote:
Hu wrote:
Could you provide a minimal project which fails for you?
Libdrm should be a small enough example.
No.
Code:
$ du -hs libdrm-2.4.97/
5.8M    libdrm-2.4.97/
$ find libdrm-2.4.97/ -type f | wc
    365     365   13310
Back to top
View user's profile Send private message
NTU
Apprentice
Apprentice


Joined: 17 Jul 2015
Posts: 164

PostPosted: Sat Mar 23, 2019 12:24 am    Post subject: Reply with quote

mv wrote:
In any case, if -fPIE should really become active when compiling a library, the produced library is broken.

Broken how? I thought I offended you like a year ago, you said you weren't going to help me anymore. Anyway, I appreciate it!
Hu wrote:
NTU wrote:
Hu wrote:
Could you provide a minimal project which fails for you?
Libdrm should be a small enough example.
No.
Code:
$ du -hs libdrm-2.4.97/
5.8M    libdrm-2.4.97/
$ find libdrm-2.4.97/ -type f | wc
    365     365   13310

Small is relative so what do you expect..
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14159

PostPosted: Sat Mar 23, 2019 4:28 am    Post subject: Reply with quote

Since this is reported as a problem with position independent executables, I'd expect a ninja-based "Hello, world" should provoke it.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6293

PostPosted: Sat Mar 23, 2019 5:46 am    Post subject: Reply with quote

NTU wrote:
Broken how?

Some assembler experts may fill in details, but the concepts of PIE and pic are the same. There are essentially only 2 differences.
  1. The way how the position-independence is achieved. AFAIK, for PIC libraries a certain register is used which contains the "base" address, and everything is calculated relative to that. For PIE some details differ, depending on the architecture (e.g. perhaps this is a different register on x86_64 architecture - just a guess for simpler explanation). Now if your code expects a library to be PIC-compliant and thus for calling a function of that library uploadsl the "base" address to the corresponding register, but the library is actually PIE and expects it in a different register, the library can do anything unpredictable but most likely will segfault immediately.
  2. The startup-code must be different,
You are happy to apparenly see the effect of 2 during linking; otherwise you would have just broken your system.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14159

PostPosted: Sat Mar 23, 2019 3:52 pm    Post subject: Reply with quote

On x86_64, instruction-pointer-relative addressing is standard for globals, so PIC is natural. On x86_32, a separate register (commonly ebx) is set aside for this purpose. To avoid the problems mv describes, generated code defaults to assuming that PIC is not set up properly, so every function that needs it will initialize it. (There are ways around this, mostly involving when the compiler can prove that the function is only called from contexts where PIC was already initialized correctly and can be safely reused.)

I suspect the reported problem is caused by confusing the compiler into thinking that every target is an executable (not a shared library), so it requires a main in each such target.
Back to top
View user's profile Send private message
NTU
Apprentice
Apprentice


Joined: 17 Jul 2015
Posts: 164

PostPosted: Sun Mar 24, 2019 7:21 pm    Post subject: Reply with quote

I found two smaller examples when compared to libdrm; dev-libs/pugixml and dev-libs/tinyxml. -fPIC -fPIE -pie work fine. These projects don't use ninja though, but it does address the theory that PIE breaks libs.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum