Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
How to convince portage to build a static binary
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
ewildgoose
n00b
n00b


Joined: 02 Mar 2003
Posts: 74

PostPosted: Wed Apr 07, 2021 12:14 pm    Post subject: How to convince portage to build a static binary Reply with quote

Hi, I'm using gentoo as a meta-distribution for an embedded project and I want to build a few tools for use in a custom initramfs (we have to do a bunch of scripts to make sure the system is sane, register with some services during initial manufacturing boot, etc)

So I'm really struggling to figure out how to persuade portage to build me some static binaries... My solution at present is to fork the ebuilds and make a -static version, but it's a fair bit of extra maintenance. Any suggestions?

So lets imagine that I wanted a static "jq" binary. What I tried so far was stuff like:

Code:

LDFLAGS=-static USE=static-libs ebuild ./repos/gentoo/app-misc/jq/jq-1.6-r3.ebuild compile


However, that leaves me with a binary that relies on my libc (musl). Yes it's only 600KB, but for various reasons I need to be frugal, so I want a static compile, run through LTO.

I then tried variations of things like
Code:
  LDFLAGS="-all-static"

However, this breaks the configure stage... ("compiler cannot generate executables")


For this particular package I can fork the ebuild, add
Code:
src_compile() {
   emake LDFLAGS="$LDFLAGS -all-static"
}


However, I feel like there must be a simpler way to change the flags only for the compile phase? Or perhaps I'm attacking this completely the wrong way, is there a different route to go down to build a static binary?

Thanks for ideas
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 16875

PostPosted: Wed Apr 07, 2021 4:29 pm    Post subject: Reply with quote

How many tools are you redoing this way? When you sum them up, will the static linking, and resulting license-compliance requirements, be worth it? I wonder if you would be better off just shipping the dynamic libraries in the initramfs. With dynamic linking, you will pay for one copy of the full dynamic library. With static linking, you will pay for some part of the supporting library in every tool you ship. If you have a tiny number of tools, static linking may work out better. With many, it may not.

Assuming you have checked that and are still determined to go static, you could use a bashrc hook to rewrite LDFLAGS only during the compile phase. You could also consider adding USE=static to the ebuild, guarding the LDFLAGS change accordingly, and then contributing that back to the main tree, so that long term, you do not need to maintain a forked ebuild. You should also check whether the tool's configure script has an option, often named --enable-static, to tell it to build statically. If it does, then upstream is already maintaining the support and you only need to trigger that, which you could do with EXTRA_ECONF (or, again, a forked ebuild that you later contribute back).
Back to top
View user's profile Send private message
ewildgoose
n00b
n00b


Joined: 02 Mar 2003
Posts: 74

PostPosted: Wed Apr 07, 2021 8:45 pm    Post subject: Reply with quote

Thanks for the thoughts. I think I have a couple so far, not many, but each needed quite a bit of work. I think I have busybox, curl, one tool from util-linux (not the whole shibang), just the fsck.ext2 util from e2fs-progs and jq. Several of these want to pull in 10s of MBs of shared libs, so saving is quite high so far.

There are some reasons why a full multi 10s MBs of operating system won't work here. This use case isn't openwrt tight where the whole lot including the final operating system needs to fit in 16MB or 32MB, but we do need to keep an eye on sizes. I'm soft trying to keep under about 60MB total. initramfs/kernel included within that goal. However, initramfs is just dead code really, I need it for a few seconds before we start the full OS. I use a bunch of tricks like reaching into a chrooted version of the full OS to get certain tools, but some use cases are rescue and manufacturing, so can't always do that

I think there is nothing better than bashrc (which I'm already using to hook a few other things). However, I'm reluctant to use it widely as it's difficult to maintain and document. Bit opaque in the whole build process...
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 16875

PostPosted: Wed Apr 07, 2021 10:32 pm    Post subject: Reply with quote

Yes, bashrc may not scale well. That is why I suggested you should modify, and contribute back, the relevant ebuilds. You may also find it useful to submit patches to the upstream projects where necessary, to add/fix a configure-time option to produce static binaries.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7953
Location: almost Mile High in the USA

PostPosted: Thu Apr 08, 2021 2:27 am    Post subject: Reply with quote

Devil's advocate, static libraries can be smaller if they don't include functions like tan() exp() but retain div()...

Ugh.

I can't believe lvm requires libm. Lazy programmers. Grr.
_________________
Intel Core i7 2700K@ 3.6GHz/RadeonHD 5770/8GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 16875

PostPosted: Thu Apr 08, 2021 4:36 pm    Post subject: Reply with quote

Right, linking in only the required functionality can be a savings. My concern above is that since statically linked functionality is by definition duplicated in every executable, then with enough executables using a given feature, that functionality can be duplicated enough times that the required size for all the duplicates exceeds the size required to store a single copy of the shared library, even with all the unnecessary functionality that the shared library would bring along. For maximum savings, OP should use a shared library, and custom build that shared library to have only the features needed by the executables bundled into the initramfs. This is more work than any of the other choices, and more fragile.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum