Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[SOLVED] Using 'quickpkg' to make all installed binaries
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
dufeu
l33t
l33t


Joined: 30 Aug 2002
Posts: 757
Location: US-FL-EST

PostPosted: Sat May 11, 2013 11:35 am    Post subject: [SOLVED] Using 'quickpkg' to make all installed binaries Reply with quote

My binary packages server gaves it's last gasp a few months ago after demonstrating the concussive force consequences of gravitic attraction when two objects meet. i.e. It got knocked of it's small table {about 3 ft high} and met the floor .. violently .. while my cat and one of the dogs were chasing each other.

Salvage turned out to be more of a project than expected and while I still have the hard drive, I don't have a system to install it into at the moment.

I figured 'no problem', I'll just make one of my binary package consumers become the new binary package server.

Without thinking about it first, I did a normal/usual type world update on the new host {over 800 packages} without updating /etc/portage/make.conf to have emerge automatically generate binary packages. While I have used 'quickpkg' in the past for a package here and there, I've never used it for all installed packages. After reading the relevant man page, I realize some of my implicit assumptions were wrong.

For one issue, only the options listed in 'quickpkg' are available. While the syntax/definition of package atom are the same as emerge, none of emerge's options are available. {Which actually makes a lot of sense.} However, I was assuming I could use something like:
Code:
# quickpkg --include-unmodified-config y --deep --with-bdeps=y @world

Which if it worked, would do what I want {I think}. However, '--deep' and '--with-bdeps=y' are not quickpkg options. DOH!

In other words, if I run quickpkg for @world, how do I get the required dependencies which aren't in my world list?

Also, how can I restrict quickpkg to only those installed packages which don't yet have binary packages in $PKGDIR {in my case: /usr/local/portage/packages}?

I was thinking of attempting to put together a 'bash' script modelled on something like Sven Vermeulen's Gentoo protip: using buildpkgonly.
Code:
#!/bin/sh

LIST=$(mktemp);

emerge -p --complete-graph=y --with-bdeps=y --color=n --columns --quiet=y @world | awk '{print $2}' > ${LIST};

for PACKAGE in $(cat ${LIST});
do
  printf "Building binary package for ${PACKAGE}... "
  quickpkg --include-unmodified-config=y ${PACKAGE};
  if [[ $? -eq 0 ]];
  then
    echo "ok";
  else
    echo "failed";
  fi
done
But I only know enough 'bash' to be stupidly dangerous. While I can cut and paste Verneleulen's script, I'm not sure how to check for an already existing package.

Some advice and quidance would be appreciated!

BTW - FWIW - Out of curiosity, I did try a simple:
Code:
quickpkg --include-unmodified-config y @world

The results were:
Code:
 * Building package for media-libs/libcanberra-0.30-r1 ...                                                                                          [ ok ]
 * Building package for virtual/glu-9.0 ...                                                                                                         [ ok ]
 * Building package for gnome-base/gconf-2.32.4-r1 ...                                                                                              [ ok ]
 * Building package for app-office/dia-0.97.2-r1 ...                                                                                                [ ok ]
 * Building package for app-editors/ee-1.5.2 ...                                                                                                    [ ok ]
 * Building package for media-video/dvbsnoop-1.4.50 ...                                                                                             [ ok ]
 * Building package for x11-wm/windowmaker-0.95.4-r1 ...                                                                                            [ ok ]
 * Building package for x11-proto/dri2proto-2.8 ...                                                                                                 [ ok ]
 * Building package for media-fonts/font-adobe-utopia-75dpi-1.0.4 ...                                                                               [ ok ]
 * Building package for x11-libs/libSM-1.2.1-r1 ...                                                                                                 [ ok ]
 * Building package for dev-perl/DateManip-6.380.0 ...                                                                                              [ ok ]
 * Building package for media-libs/libsamplerate-0.1.8 ...                                                                                          [ ok ]
 * Building package for media-libs/gst-plugins-base-0.10.36 ...                                                                                     [ ok ]
 * Building package for media-libs/gst-plugins-base-1.0.6 ...                                                                                       [ ok ]
 * Building package for net-mail/mailbase-1.1 ...                                                                                                   [ ok ]
 * Building package for app-crypt/pkcrack-1.2.2-r1 ...                                                                                              [ ok ]
 * Building package for virtual/package-manager-0 ...                                                                                               [ ok ]
 * Building package for perl-core/File-Spec-3.400.0 ...                                                                                             [ ok ]
 * Building package for media-fonts/tengwar-fonts-1.9d ...                                                                                          [ ok ]
 * Building package for xfce-base/xfce4-session-4.10.1 ...                                                                                          [ ok ]
 * Building package for dev-qt/qtgui-4.8.4-r1 ...                                                                                                   [ ok ]
 * Building package for app-arch/unrar-5.0.3 ...                                                                                                    [ ok ]
 * Building package for media-fonts/vdrsymbols-ttf-20100612 ...                                                                                     [ ok ]
 * Building package for x11-misc/driconf-0.9.1-r1 ...                                                                                               [ ok ]
 * Building package for net-libs/gnutls-3.1.10 ...                                                                                                  [ ok ]
 * Building package for dev-util/gtk-doc-am-1.18 ...                                                                                                [ ok ]
 * Building package for app-admin/syslog-ng-3.4.1 ...                                                                                               [ ok ]
 * Building package for x11-libs/libXt-1.1.3-r1 ...                                                                                                 [ ok ]
 * Building package for app-arch/xarchiver-0.5.2-r3 ...                                                                                             [ ok ]
 * Building package for app-misc/tmux-1.8 ...                                                                                                       [ ok ]
 * Building package for x11-misc/xdg-utils-1.1.0_rc1_p20120916 ...                                                                                  [ ok ]
 * Building package for kde-misc/krusader-2.4.0_beta3 ...                                                                                           [ ok ]
 * Building package for dev-python/setuptools-0.6.33 ...                                                                                            [ ok ]
 * Building package for x11-libs/libXi-1.7.1 ...                                                                                                    [ ok ]
 * Building package for media-libs/libpng-1.2.50 ...                                                                                                [ ok ]
 * Building package for media-libs/libpng-1.5.15-r15 ...                                                                                            [ ok ]
 * Building package for media-libs/libpng-1.6.2 ...                                                                                                 [ ok ]
 * Building package for kde-base/kwordquiz-4.10.3 ...                                                                                               [ ok ]
 * Building package for dev-libs/nss-3.14.3 ...                                                                                                     [ ok ]
 * Building package for lxde-base/lxterminal-0.1.11 ...                                                                                             [ ok ]
 * Building package for dev-libs/libIDL-0.8.14 ...                                                                                                  [ ok ]
 * Building package for app-editors/focuswriter-1.4.2 ...                                                                                           [ ok ]
 * dev-db/libdbi-drivers-0.8.3-r2: package was emerged with USE=-bindist!
 * dev-db/libdbi-drivers-0.8.3-r2: it might not be legal to redistribute this.
 * Building package for dev-db/libdbi-drivers-0.8.3-r2 ...                                                                                          [ ok ]
 * Building package for media-fonts/liberation-fonts-2.00.0-r1 ...                                                                                  [ ok ]
 * Building package for media-fonts/sgi-fonts-1.0 ...                                                                                               [ ok ]
 * Building package for kde-base/kdepim-meta-4.10.3 ...                                                                                             [ ok ]
 * Building package for kde-base/kdeadmin-meta-4.10.3 ...                                                                                           [ ok ]
 * Building package for dev-python/pygtksourceview-2.10.1-r1 ...Killed

At the moment, I have no idea where to look to find more information why the process died. Some advice in this area would be appreciated too.
_________________
People whom think M$ is mediocre, don't know the half of it.


Last edited by dufeu on Sun May 12, 2013 5:19 am; edited 1 time in total
Back to top
View user's profile Send private message
krinn
Advocate
Advocate


Joined: 02 May 2003
Posts: 4648

PostPosted: Sat May 11, 2013 1:41 pm    Post subject: Reply with quote

Code:
qlist -I | xargs -I {} echo quickpkg quickpkg_options {}


Something like that ?
Back to top
View user's profile Send private message
Hu
Watchman
Watchman


Joined: 06 Mar 2007
Posts: 9309

PostPosted: Sat May 11, 2013 5:53 pm    Post subject: Reply with quote

You can use eix option --binary to list only packages which have a tbz2. It should be usable in a negative sense, but that did not work for me. As an untested workaround:
Code:
#!/bin/sh

eix -I --only-names > installed-package.txt
eix -I --only-names --binary > tbz2-package.txt
comm -23 installed-package.txt tbz2-package.txt > needed-package.txt
xargs -a needed-package.txt quickpkg --include-config=y

I picked --include-config rather than --include-unmodified-config because it is more complete. However, beware that distributing a tbz2 built using --include-config exposes your local modifications to the recipient. For files which contain passwords, this would give the recipient those passwords.
Back to top
View user's profile Send private message
dufeu
l33t
l33t


Joined: 30 Aug 2002
Posts: 757
Location: US-FL-EST

PostPosted: Sun May 12, 2013 3:13 am    Post subject: Reply with quote

krinn wrote:
Code:
qlist -I | xargs -I {} echo quickpkg quickpkg_options {}


Something like that ?

Unfortunately no. 'quickpkg' only accepts the installed full name with versioning or the @list sets such as @world or @system. As I read the qlist manpage, 'qlist -I' only provides the unversioned package name.

Interesting approach. I had completely forgottent about qlist. Thank you though.
_________________
People whom think M$ is mediocre, don't know the half of it.
Back to top
View user's profile Send private message
dufeu
l33t
l33t


Joined: 30 Aug 2002
Posts: 757
Location: US-FL-EST

PostPosted: Sun May 12, 2013 5:16 am    Post subject: Reply with quote

Hu wrote:
You can use eix option --binary to list only packages which have a tbz2. It should be usable in a negative sense, but that did not work for me. As an untested workaround:
Code:
#!/bin/sh

eix -I --only-names > installed-package.txt
eix -I --only-names --binary > tbz2-package.txt
comm -23 installed-package.txt tbz2-package.txt > needed-package.txt
xargs -a needed-package.txt quickpkg --include-config=y

I picked --include-config rather than --include-unmodified-config because it is more complete. However, beware that distributing a tbz2 built using --include-config exposes your local modifications to the recipient. For files which contain passwords, this would give the recipient those passwords.

After re-reading the 'eix' manpage, the need for including version info and some experimentation, I modified your suggestion to:
Code:
#!/bin/sh

eix '-I*' --format '<installedversions:EQNAMEVERSION>' > installed-package.txt
eix '-I*' --binary --format '<installedversions:EQNAMEVERSION>' > tbz2-package.txt
comm -23 installed-package.txt tbz2-package.txt > needed-package.txt
xargs -a needed-package.txt quickpkg --include-config=y

At this time, the '--binary' option appears to be broken. I did not get expected list of approximately 1200 packages. Instead, I got a list of 15 packages. To add insult to injury, said 15 packages were already replaced and upgraded to later versions. I even ran:
Code:
eix-update
to ensure the eix database was current.

Rather than worry about '--binary' being possibly borked, I simply let all 2143 installed packages (re)generate new binary packages.

Anyway, the concept works well and is pretty much the results I was looking for. Thank you!

I'll probably file a bugzilla report reqarding the '--binary' option after some more testing. I really want to find out where it got it's information from.
_________________
People whom think M$ is mediocre, don't know the half of it.
Back to top
View user's profile Send private message
gentoo-freak
n00b
n00b


Joined: 11 Sep 2009
Posts: 11

PostPosted: Wed Jan 28, 2015 9:19 am    Post subject: Offtopic Reply with quote

just use here the EIX_LIMIT=0 option

Code:
gentoo scripts # EIX_LIMIT=0 eix '-I*' --format '<installedversions:EQNAMEVERSION>' | wc -l
221
Back to top
View user's profile Send private message
mv
Advocate
Advocate


Joined: 20 Apr 2005
Posts: 4602

PostPosted: Wed Jan 28, 2015 9:46 am    Post subject: Re: Offtopic Reply with quote

gentoo-freak wrote:
just use here the EIX_LIMIT=0 option

If the output is not a terminal (as in your example where it is a pipe, or as in the script where it is a file) there is no limitation of the output, hence setting EIX_LIMIT=0 has no effect (and is not necessary).
Quote:
At this time, the '--binary' option appears to be broken

I can only guess what the problem is, because here it works correctly.
However, --binary recognizes only those binary packages which actually appear in the database with the corresponding versions.
For instance, if you have a dev-lang/python-3.2.5-r5.tbz2 but this version was removed from the tree and replaced by dev-lang/python-3.2.5-r6, eix will not "recognize" this *.tbz2 file.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum