Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Packaging: What to do if build dir's pathname is too long
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
zwol
n00b
n00b


Joined: 24 May 2024
Posts: 4

PostPosted: Fri May 24, 2024 5:44 pm    Post subject: Packaging: What to do if build dir's pathname is too long Reply with quote

I'm trying to make a local package of the Guix overlay package manager and I've run into a weird problem. This software likes to use very long pathnames for things; in conjunction with the path to the build directory it can hit various upper limits on the total lengths of pathnames. For example, the Linux kernel has a hard upper limit of 256 bytes on the length of a #! line. Guix's test suite therefore refuses to run if the path to the build directory is too long, and it turns out that "/srv/pb/portage/sys-apps/guix-1.4.0/work/guix-1.4.0" is too long for it:

Code:
# ebuild ./guix-1.4.0.ebuild clean test
...
>>> Configuring source in /srv/pb/portage/sys-apps/guix-1.4.0/work/guix-1.4.0 ...
 * econf: updating guix-1.4.0/build-aux/config.sub with /usr/share/gnuconfig/config.sub
 * econf: updating guix-1.4.0/build-aux/config.guess with /usr/share/gnuconfig/config.guess
./configure --prefix=/usr [...]
...
checking for unit test root directory... /srv/pb/portage/sys-apps/guix-1.4.0/work/guix-1.4.0/test-tmp
checking the length of the socket file name used in tests... 92
checking the length of a typical hash bang line... 73
checking the length of a hash bang line used in tests... 129
configure: WARNING: test directory '/srv/pb/portage/sys-apps/guix-1.4.0/work/guix-1.4.0/test-tmp' may lead to overly long hash-bang lines
...
make  check-TESTS check-local
make[3]: Entering directory '/srv/pb/portage/sys-apps/guix-1.4.0/work/guix-1.4.0'

Cannot run tests because file name limits would be exceeded.
Look for 'length' in the 'config.log' file for details.

make[3]: *** [Makefile:6810: check-local] Error 1
make[3]: *** Waiting for unfinished jobs....


The relevant bit of config.log just repeats what was printed during the configure phase.

Notice that this is after I already moved the Portage build root to something with a shorter name than the default. In order to make this work, I think I need to make the "sys-apps/guix-1.4.0/work/guix-1.4.0/test-tmp" piece be shorter. For example, is it possible to get rid of the second "guix-1.4.0" in there? Or the "sys-apps"? The devmanual doesn't discuss the organization of the build root much at all. Any suggestions would be appreciated.

The ebuild I have right now is appended, but it doesn't do anything special, and some of the dev-scheme/* packages it refers to are also local to my installation (guile-avahi and guile-semver). Also you need Guile 3.0 and a patched guile-lib. This yak is so tall.

Code:
# Copyright 2024 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2

EAPI=8

DESCRIPTION="Declarative overlay package manager"
HOMEPAGE="https://guix.gnu.org/"
SRC_URI="https://ftpmirror.gnu.org/gnu/${PN}/${P}.tar.gz"

LICENSE="GPL-3+"
SLOT="0"
KEYWORDS="~amd64"

PATCHES=(
        "${FILESDIR}/guile-launcher-avoid-deprecated-apis.diff"
)

DEPEND="
        app-arch/bzip2
        dev-db/sqlite
        dev-libs/libgcrypt
        dev-scheme/guile
        dev-scheme/guile-avahi
        dev-scheme/guile-gcrypt
        dev-scheme/guile-git
        dev-scheme/guile-gnutls
        dev-scheme/guile-json
        dev-scheme/guile-lib
        dev-scheme/guile-lzlib
        dev-scheme/guile-semver
        dev-scheme/guile-sqlite3
        dev-scheme/guile-ssh
        dev-scheme/guile-zlib
        dev-scheme/guile-zstd
"
RDEPEND="${DEPEND}"
BDEPEND=""
Back to top
View user's profile Send private message
Hu
Administrator
Administrator


Joined: 06 Mar 2007
Posts: 21918

PostPosted: Fri May 24, 2024 6:52 pm    Post subject: Reply with quote

Welcome to the forums.

I suppose it would be unacceptable to ask upstream to stop relying on #! lines and invoke their programs as programs (with the script passed as an argument), which would completely avoid the problem.

Taking your path as an example:
  • /srv/pb: $PORTAGE_TMPDIR. As you found, you can make this a bit better, but there will be a hard minimum of / even if you set PORTAGE_TMPDIR=/ (which may have other interesting and undesirable results). You could recover a bit of space here, but not much.
  • portage/: this comes from the package manager. A hack to the source code could likely change it, but I don't know how many places you might need to change to keep all the computed paths consistent.
  • sys-apps/guix-1.4.0/: $P: renaming the ebuild could help here, but that would be very ugly.
  • work/: this comes from the package manager. As with portage/ above, you could change it, but you might need to change multiple places.
  • guix-1.4.0: I think this is because the archive has that as its first level directory. You could package the archive with a different name, or move this up a level after unpacking.
Back to top
View user's profile Send private message
logrusx
Veteran
Veteran


Joined: 22 Feb 2018
Posts: 1751

PostPosted: Fri May 24, 2024 7:53 pm    Post subject: Reply with quote

zwol wrote:

Code:


Cannot run tests because file name limits would be exceeded.
Look for 'length' in the 'config.log' file for details.





Aaand? What does config.log say?

Best Regards,
Georgi
Back to top
View user's profile Send private message
zwol
n00b
n00b


Joined: 24 May 2024
Posts: 4

PostPosted: Tue May 28, 2024 1:18 pm    Post subject: Reply with quote

logrusx wrote:
zwol wrote:
Code:
Cannot run tests because file name limits would be exceeded.
Look for 'length' in the 'config.log' file for details.


Aaand? What does config.log say?


zwol wrote:
The relevant bit of config.log just repeats what was printed during the configure phase.


i.e.

zwol wrote:
Code:
checking for unit test root directory... /srv/pb/portage/sys-apps/guix-1.4.0/work/guix-1.4.0/test-tmp
checking the length of the socket file name used in tests... 92
checking the length of a typical hash bang line... 73
checking the length of a hash bang line used in tests... 129
configure: WARNING: test directory '/srv/pb/portage/sys-apps/guix-1.4.0/work/guix-1.4.0/test-tmp' may lead to overly long hash-bang lines
Back to top
View user's profile Send private message
zwol
n00b
n00b


Joined: 24 May 2024
Posts: 4

PostPosted: Tue May 28, 2024 1:30 pm    Post subject: Reply with quote

Hu wrote:
I suppose it would be unacceptable to ask upstream to stop relying on #! lines and invoke their programs as programs (with the script passed as an argument), which would completely avoid the problem.


They want to test the same code paths that would be used by the software when it's installed, and use of #! is necessary in the installed software (can't make everyone who wants to run installed scripts type the name of the interpreter) so yeah, that's not happening.

Quote:
$PORTAGE_TMPDIR. As you found, you can make this a bit better, but there will be a hard minimum of / even if you set PORTAGE_TMPDIR=/ (which may have other interesting and undesirable results).


Right, and if I want this package to ever be usable by anyone but myself, it needs to build with at least the default value for $PORTAGE_TMPDIR, which is, um, /var/tmp ... I hadn't realized I only managed to shave one character off relative to the default :oops:

For the same reason, I am reluctant to mess with anything that comes from the guts of the package manager.

Quote:
guix-1.4.0: I think this is because the archive has that as its first level directory. You could package the archive with a different name, or move this up a level after unpacking.


This has some promise, I'll look into that. Relatedly, is there a standard procedure for when you want to do a "split tree" build of an autotools-based package? That is, invoking the 'configure' script with the current working directory outside of the source tree, which causes all the build products to get written to a brand new directory tree starting from the cwd. I could use this to move the build tree up a bit without needing to mess with where the source tree lands.
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9559
Location: beyond the rim

PostPosted: Wed May 29, 2024 8:27 am    Post subject: Reply with quote

If the issue is just the length check in the test routine I'd probably just go remove that via patch. It seems to be wrong anyway, the installed package should not contain any paths involving PORTAGE_TMPDIR.
Back to top
View user's profile Send private message
zwol
n00b
n00b


Joined: 24 May 2024
Posts: 4

PostPosted: Wed May 29, 2024 1:42 pm    Post subject: Reply with quote

Genone wrote:
If the issue is just the length check in the test routine I'd probably just go remove that via patch. It seems to be wrong anyway, the installed package should not contain any paths involving PORTAGE_TMPDIR.


I expect the test suite to throw many failures due to the overlength paths. This package is hairy enough that I really want to make sure it passes its test suite.
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9559
Location: beyond the rim

PostPosted: Wed May 29, 2024 2:45 pm    Post subject: Reply with quote

Ah ok, so that length check is a test for the test ;)

Well, one thing you could try then is to redefine $S (the build directory which defaults to "$WORKDIR/$P") in the ebuild. Don't know if it is possible to relocate that outside of $WORKDIR though. $WORKDIR is readonly, so not much chance there without mangling the ebuild name itself. Also sandbox might complain if you try to write outside $WORKDIR.
But should allow to chop off a few chars of the final guix-1.4.0 part at least.

/srv/pb/portage/sys-apps/guix-1.4.0/work/guix-1.4.0 is just 52 chars, so the majority of the problem is the package itself anyway.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum