my own binhost

[darkseer]

I have some machines that are quite slow and one or two fast ones. In order to make updates faster for the very slow machines I've created a binhost with a chroot environment for each of the slower machines. At 3 am or so a job kicks off and does an emerge -vuND --buildpkg world in the chrooted environments and creates all the necessary binary packages for update. I'm running into an issue with use flags. The profiles, package.use, and package.keywords file are the same in the chrooted environment and the target environment.

If I do a emerge -vp ghostscript-gpl on the target machine I get:

[ebuild R ] app-text/ghostscript-gpl-8.71-r6 USE="X cairo cups gtk jpeg2k -bindist -djvu" LINGUAS="-ja -ko -zh_CN -zh_TW"

If I do emerge -vp ghostscript-gpl on the binhost in the chrooted environment I get:

[ebuild R ] app-text/ghostscript-gpl-8.71-r6 USE="X cairo cups gtk jpeg2k -bindist -djvu" LINGUAS="-ja -ko -zh_CN -zh_TW"

This is expected and good so far. But when I do the following on the target emerge -pv --getbinpkg --usepkgonly ghostscript-gpl I get:

[binary R ] app-text/ghostscript-gpl-8.71-r6 USE="X cups gtk jpeg2k -bindist -cairo* -djvu" LINGUAS="-ja -ko -zh_CN -zh_TW"


Why am I loosing the cairo flag when I try to get the binary package from my binhost?

-
Darkseer

[Hu]

Presumably, the binary was built with USE=-cairo, though the output you have provided so far seems to show that it should be built with USE=cairo. What is the output if you run emerge -pvk =app-text/ghostscript-gpl-8.71-r6 ; emerge --info both on the target machine and in the chroot?

[darkseer]

target:
These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild R ] app-text/ghostscript-gpl-8.71-r6 USE="X cairo cups gtk jpeg2k -bindist -djvu" LINGUAS="-ja -ko -zh_CN -zh_TW" 15,343 kB

Total: 1 package (1 reinstall), Size of downloads: 15,343 kB

* IMPORTANT: 1 news items need reading for repository 'gentoo'.
* Use eselect news to read news items.

Portage 2.1.9.25 (default/linux/amd64/10.0/desktop/kde, gcc-4.4.5, glibc-2.11.3-r0, 2.6.36-gentoo-r5 x86_64)
=================================================================
System uname: Linux-2.6.36-gentoo-r5-x86_64-Intel-R-_Core-TM-2_Duo_CPU_P9600_@_2.66GHz-with-gentoo-1.12.14
Timestamp of tree: Mon, 07 Mar 2011 19:00:01 +0000
app-shells/bash: 4.1_p9
dev-java/java-config: 2.1.11-r3
dev-lang/python: 2.6.6-r2, 3.1.3-r1
dev-util/cmake: 2.8.1-r2
sys-apps/baselayout: 1.12.14-r1
sys-apps/sandbox: 2.4
sys-devel/autoconf: 2.13, 2.65-r1
sys-devel/automake: 1.9.6-r2, 1.10.3, 1.11.1
sys-devel/binutils: 2.20.1-r1
sys-devel/gcc: 4.4.5
sys-devel/gcc-config: 1.4.1
sys-devel/libtool: 2.2.10
sys-devel/make: 3.81-r2
virtual/os-headers: 2.6.36.1 (sys-kernel/linux-headers)
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=nocona"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/openvpn/easy-rsa /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests binpkg-logs distlocks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j3"
PKGDIR="/home/portage/distfiles"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/home/portage/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage/layman/desktop-effects /usr/local/portage/layman/kde /usr/local/portage/layman/multimedia /usr/local/portage/layman/enlightenment /usr/local/portage/layman/mozilla"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X a52 aac aalib acl acpi akonadi alsa amd64 apm archive audiofile augeas bash-completion berkdb bluetooth branding bzip2 cairo cdaudio cdda cdr chipcard cli connection-sharing consolekit cracklib crypt css cups cxx daap dbus dhclient dhcpcd dri dts dv dvd dvdr eap-sim emacs embedded emboss emovix encode escreen exif extensions extrafilters extras faac faad fam fame fat fbsplash ffmpeg ffmpeg-mt firefox fits flac fontconfig fortran ftp fts3 fuse gadu gdal gdbm gdu geos gif gimp git glitz gml gnome gnome-keyring gnutls google-gadgets gpm gps graphviz gstreamer gtk hal hbci hdf5 iconv id3tag imagemagick imlib ipod ipv6 isc java java6 jingle jpeg jpeg2k kde kerberos lame lastfm latex lcms ldap libnotify lm_sensors lzo mad madwifi meanwhile mikmod mjpeg mmx mmxext mng modules motif mp3 mp3tunes mp4 mpeg msn mtp mudflap multilib multimedia musepack musicbrainz mysql ncurses netcdf network nls nptl nptlonly nsplugin nss ntfs nuv nvidia offensive ofx ogdi ogg okular old-daemons openexr opengl openmp oscar otr pam pango pcre pdf perl phonon plasma pmu png policykit ppds pppd python qimageblitz qq qscintilla qt3 qt3support qt4 quicktime quotes qwt rar readline resolvconf rrdtool rss rtc ruby samba script scripts sdk sdl semantic-desktop sensord session skype smp sms sndfile sound spell sql sqlite sse sse2 ssl ssse3 startup-notification static-libs subversion svg sysfs syslog tcl tcpd templates tftp theora threads tiff truetype udev unicode usb v4l v4l2 vboxwebsrv vcd vdpau vim-syntax vim-with-x vnc vorbis webkit webpresence wmf wps x264 xcb xcomposite xml xorg xrandr xulrunner xv xvid xvmc yahoo yv12 zeroconf zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev mouse keyboard synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" SANE_BACKENDS="hp5590" USERLAND="GNU" VIDEO_CARDS="nvidia intel" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

binhost:

emerge -pvk =app-text/ghostscript-gpl-8.71-r6 ; emerge --info

These are the packages that would be merged, in order:

Calculating dependencies... done!
[binary R ] app-text/ghostscript-gpl-8.71-r6 USE="X cups gtk jpeg2k -bindist -cairo* -djvu" LINGUAS="-ja -ko -zh_CN -zh_TW"

Total: 1 package (1 reinstall, 1 binary), Size of downloads: 0 kB

* IMPORTANT: 4 news items need reading for repository 'gentoo'.
* Use eselect news to read news items.

Portage 2.1.9.25 (default/linux/amd64/10.0/desktop/kde, gcc-4.4.5, glibc-2.11.3-r0, 2.6.29-gentoo-r6 x86_64)
=================================================================
System uname: Linux-2.6.29-gentoo-r6-x86_64-Intel-R-_Core-TM-2_CPU_6700_@_2.66GHz-with-gentoo-1.12.14
Timestamp of tree: Wed, 09 Mar 2011 08:00:01 +0000
app-shells/bash: 4.1_p9
dev-java/java-config: 2.1.11-r3
dev-lang/python: 2.6.6-r2, 3.1.3-r1
dev-util/cmake: 2.8.1-r2
sys-apps/baselayout: 1.12.14-r1
sys-apps/sandbox: 2.4
sys-devel/autoconf: 2.13, 2.65-r1
sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.1
sys-devel/binutils: 2.20.1-r1
sys-devel/gcc: 4.4.5
sys-devel/gcc-config: 1.4.1
sys-devel/libtool: 2.2.10
sys-devel/make: 3.81-r2
virtual/os-headers: 2.6.36.1 (sys-kernel/linux-headers)
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=nocona"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests binpkg-logs distlocks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j3"
PKGDIR="/home/portage/distfiles"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X a52 aac aalib acl acpi akonadi alsa amd64 apm archive audiofile augeas bash-completion berkdb bluetooth branding bzip2 cairo cdaudio cdda cdr chipcard cli connection-sharing consolekit cracklib crypt css cups cxx daap dbus dhclient dhcpcd dri dts dv dvd dvdr eap-sim emacs embedded emboss emovix encode escreen exif extensions extrafilters extras faac faad fam fame fat fbsplash ffmpeg ffmpeg-mt firefox fits flac fontconfig fortran ftp fts3 fuse gadu gdal gdbm gdu geos gif gimp git glitz gml gnome gnome-keyring gnutls google-gadgets gpm gps graphviz gstreamer gtk hal hbci hdf5 iconv id3tag imagemagick imlib ipod ipv6 isc java java6 jingle jpeg jpeg2k kde kerberos lame lastfm latex lcms ldap libnotify lm_sensors lzo mad madwifi meanwhile mikmod mjpeg mmx mmxext mng modules motif mp3 mp3tunes mp4 mpeg msn mtp mudflap multilib multimedia musepack musicbrainz mysql ncurses netcdf network nls nptl nptlonly nsplugin nss ntfs nuv nvidia offensive ofx ogdi ogg okular old-daemons openexr opengl openmp oscar otr pam pango pcre pdf perl phonon plasma pmu png policykit ppds pppd python qimageblitz qq qscintilla qt3 qt3support qt4 quicktime quotes qwt rar readline resolvconf rrdtool rss rtc ruby samba script scripts sdk sdl semantic-desktop sensord session skype smp sms sndfile sound spell sql sqlite sse sse2 ssl ssse3 startup-notification static-libs subversion svg sysfs syslog tcl tcpd templates tftp theora threads tiff truetype udev unicode usb v4l v4l2 vboxwebsrv vcd vdpau vim-syntax vim-with-x vnc vorbis webkit webpresence wmf wps x264 xcb xcomposite xml xorg xrandr xulrunner xv xvid xvmc yahoo yv12 zeroconf zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev mouse keyboard synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" SANE_BACKENDS="hp5590" USERLAND="GNU" VIDEO_CARDS="nvidia intel" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY

[darkseer]

I can see on the bin host with emerge -pvk =app-text/ghostscript-gpl-8.71-r6 there is no cairo listed, but if I look in the Packages file in the package directory I see the following:

BUILD_TIME: 1299272934
CPV: app-text/ghostscript-gpl-8.71-r6
DEFINED_PHASES: compile configure install prepare setup
DEPEND: app-text/libpaper media-libs/fontconfig virtual/jpeg >=media-libs/libpng-1.2.42 >=media-libs/tiff-3.9.2 >=sys-libs/zlib-1.2.3 >=net-print/cups-1.3.8 >=x11-libs/gtk+-2.0 media-libs/jasper x11-libs/libXt x11-libs/libXext app-arch/xz-utils dev-util/pkgconfig =sys-devel/automake-1.11* >=sys-devel/autoconf-2.61 >=sys-devel/libtool-2.2.6b
DESC: Ghostscript is an interpreter for the PostScript language and for PDF
EAPI: 3
IUSE: bindist cairo cups djvu gtk jpeg2k X linguas_ja linguas_ko linguas_zh_CN linguas_zh_TW
KEYWORDS: alpha amd64 arm hppa ia64 ~mips ppc ppc64 s390 sh sparc x86 ~sparc-fbsd ~x86-fbsd
LICENSE: GPL-3 CPL-1.0
MD5: ce8be00cf8e30b0f49cc8b3d20059937
MTIME: 1299272949
RDEPEND: app-text/libpaper media-libs/fontconfig virtual/jpeg >=media-libs/libpng-1.2.42 >=media-libs/tiff-3.9.2 >=sys-libs/zlib-1.2.3 >=net-print/cups-1.3.8 >=x11-libs/gtk+-2.0 media-libs/jasper x11-libs/libXt x11-libs/libXext >=media-fonts/urw-fonts-2.4.9
SHA1: e228343263b903ad38d17ab5d2077c98d26d5f38
SIZE: 10360962
USE: X cups gtk jpeg2k
REPO: gentoo


So I'm confused.. It looks like we have cairo according to the manifest, emerge isn't seeing it... :/

[dol-sen]

I believe it is because on the binhost it is not installed, only complied there. For them to be truly synced you need to install the packages on the binhost as well. Then the everything will be in sync.
_________________
Brian
Porthole, the Portage GUI frontend irc@freenode: #gentoo-guis, #porthole, Blog
layman, gentoolkit, CoreBuilder, esearch...

[dol-sen]

an alternative is to use distcc and set it up so that it uses you buildhost for the compiling, and just assemble and merge on the target.
_________________
Brian
Porthole, the Portage GUI frontend irc@freenode: #gentoo-guis, #porthole, Blog
layman, gentoolkit, CoreBuilder, esearch...

[dol-sen]

hmmm, thinking out loud here. What if you mount your targets /var/db/pkgs in your binhost chroot so that it sees the packages that are installed. It might work except for fresh deps in the current compile run.
_________________
Brian
Porthole, the Portage GUI frontend irc@freenode: #gentoo-guis, #porthole, Blog
layman, gentoolkit, CoreBuilder, esearch...

[darkseer]

On the binhost:
emerge -av cairo

* IMPORTANT: 4 news items need reading for repository 'gentoo'.
* Use eselect news to read news items.
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild R ] x11-libs/cairo-1.10.2-r1 USE="X opengl qt4 static-libs svg xcb (-aqua) -debug -directfb -doc (-drm) (-gallium) (-openvg)" 0 kB

The binhost is a complete install. If I were to replace system disk on the target system with the chrooted environment the only difference would be the conf files. I'm going to try copying the tbz2 file over and see what happens... hmmm.


-darkseer

[Hu]

As I read the output, the build chroot has most recently compiled the prebuilt package with USE=-cairo, so that is what the target will get if you install the existing tbz2. However, the output from your first post indicates that were you to emerge --buildpkgonly app-text/ghostscript-gpl now, it would build a package with USE=cairo. Additionally, the currently installed version of app-text/ghostscript-gpl in the build chroot was built with USE=cairo. Try explicitly building a package of it in the chroot.

[darkseer]

ok, there is obviously something different between my chroot and my target environment, and I am having difficulty finding it. Hu, if I run emerge -av --buildpkg ghostscript-gpl in the chroot, its telling me that it is compiling cairo in, but when I try to install the package it produces it tells me cairo is not installed. I'm trying to look at the binary and libs directly to see if cairo was compiled in in the chroot version of ghostscript or not. If it is the Packages file is off some how off, if it is not, emerge is lying to me when it builds..... I'm sure I have some flag, profile, or variable set that is causing this, I just cant find it.

-Darkseer

[darkseer]

OK, if I do an emerge -v --buildpkg ghostscript-gpl on the binhost the resulting .tbz2 file does not have cairo compiled in. If I do a emerge -v ghostscript-gpl; quickpkg ghostscript-gpl on the binhost the resulting package file does have cairo built in..... So if I just emerge the package cairo is built in. If I add the build package option cairo is not built in even though the emerge command says its going to build cairo in. What am I missing?

-Darkseer

[HeXiLeD]

When using binhost is it possible to have portage to use binhost amd having it taking precedence over compiling from source?
In other words; lets see i have a irssi on my binhost and when executing emerge irssi portage will look first in the binhost and if it does not find it there; it will then fetch from source to compile.
I have a few nebooks and this would make life easier in terms of compilation and installation since they all have the same flags.
Whichever netbook compiles whatever package; the others can then fetch from there.

Another question is if it is possible to automatically build packages form de whole system instead of have to execute quickpkg <name> individually.
I tried quickpkg * inside /usr/bin but i am not sure how valid is. (is it ? ) can it be done everywhere there is an executable ?
_________________
443640, Questioning, Unsolved, Configs, BinHost

[few]

[HeXiLeD]