Any suggestion to clean the "world" file?

Message

shallpion · Post by **shallpion** » Wed Jan 06, 2010 8:26 am

I carelessly put many packages into world file by emerging without -1 and now this file contains 200 packages around. Any suggestions to clean this file? I learned there are some program(I could not remember their name, sorry) can do this job but they are hard-masked as unstable, so I dare not try them:) Or any guideline to clean this file manually? Thanks.

durian · Post by **durian** » Wed Jan 06, 2010 9:11 am

What's wrong with having them in the world file? That way they get updated.

Otherwise, unmerge them, and "emerge -1" them again :)

-peter

cach0rr0 · Post by **cach0rr0** » Wed Jan 06, 2010 11:06 am

should look at 'regenworld', and the --depclean emerge option

just be careful with depclean, as it isn't perfect. Check out "man emerge" for the full details

Post by **John R. Graham** » Wed Jan 06, 2010 11:12 am

durian wrote:...
Otherwise, unmerge them, and "emerge -1" them again
...

That's way too much work. Just open the world file (/var/lib/portage/world) in your favorite editor and delete the lines corresponding to packages you don't want. It's as simple as that. A subsequent "emerge --depclean" will remove those packages that are not dependencies of packages that are still in the world file and retain all that are.

However, before doing this, please read the warnings about --depclean in the emerge man page.

- John

Nerevar · Post by **Nerevar** » Wed Jan 06, 2010 12:59 pm

Shows packages in world file that have dependencies:

Code: Select all

while read pkg ; do
  DEPS="`equery -q d $pkg`"
  if [ -n "$DEPS" ] ; then
    echo $pkg
  fi
done < /var/lib/portage/world

durian · Post by **durian** » Wed Jan 06, 2010 1:03 pm

john_r_graham wrote:
durian wrote:...
Otherwise, unmerge them, and "emerge -1" them again :)
...
That's way too much work. Just open the world file (/var/lib/portage/world) in your favorite editor and delete the lines corresponding to packages you don't want. It's as simple as that. A subsequent "emerge --depclean" will remove those packages that are not dependencies of packages that are still in the world file and retain all that are.

However, before doing this, please read the warnings about --depclean in the emerge man page.

True, a lot of extra work - but personally I have bad experiences with depclean; it always wants to remove things I want to keep. (I have tried the method you mention, but I didn't like the output of --depclean --pretend.)

-peter

myceliv · Post by **myceliv** » Wed Jan 06, 2010 4:39 pm

Nerevar wrote:Shows packages in world file that have dependencies:

Neat script! (Although it's only a little faster than frackin python-updater, i.e. slower than pouring frozen molasses.)

regenworld actually added some unneeded stuff for me. Guess I'll save that only for emergencies that should never happen.

One substantial benefit to a minimal world is that it lets the package manager resolve dependencies instead of overly specifying things that will likely change over time. That way you're less likely to run into blocks, etc. as packages evolve.

So I vote for the method of

manually edit out obvious mistakes and add useful missing bits like slots for gcc python etc
run emerge -a --depclean, redirecting or processing the output if needed (make sure emerge -uavDN first)
emerge --noreplace anything that shouldn't be removed so its added to world. Then you can run --depclean regularly and get brief, easily scannable output.
from then on use --oneshot when needed to avoid mistakes, peek at the world-with-deps script list every so often to see if some of them aren't really wanted. (May still want to keep some, and overlays seem to need special treatement too.)

Guess I don't see a way to totally automate it still, there's just too much room for individual preferences.
[Edits:] add update warning and s/ghc/gcc/

Mike Hunt · Post by **Mike Hunt** » Wed Jan 06, 2010 4:43 pm

Then with portage-2.2, the sets feature allows even more cleaning of the world file, because many things that would likely remain forever on the system, like app-portage/gentoolkit and so on... can be put into a set instead of the world file.

shallpion · Post by **shallpion** » Wed Jan 06, 2010 6:06 pm

I tried depclean only several times and by now it seems to be working not bad because every time I check very carefully and make sure only the ones I truly don't want are listed.....

shallpion · Post by **shallpion** » Wed Jan 06, 2010 6:09 pm

myceliv wrote:
Nerevar wrote:Shows packages in world file that have dependencies:
Neat script! (Although it's only a little faster than frackin python-updater, i.e. slower than pouring frozen molasses.)

regenworld actually added some unneeded stuff for me. Guess I'll save that only for emergencies that should never happen.

One substantial benefit to a minimal world is that it lets the package manager resolve dependencies instead of overly specifying things that will likely change over time. That way you're less likely to run into blocks, etc. as packages evolve.

So I vote for the method of

manually edit out obvious mistakes and add useful missing bits like slots for gcc python etc

run emerge -a --depclean, redirecting or processing the output if needed (make sure emerge -uavDN first)

emerge --noreplace anything that shouldn't be removed so its added to world. Then you can run --depclean regularly and get brief, easily scannable output.

from then on use --oneshot when needed to avoid mistakes, peek at the world-with-deps script list every so often to see if some of them aren't really wanted. (May still want to keep some, and overlays seem to need special treatement too.)

Guess I don't see a way to totally automate it still, there's just too much room for individual preferences.
[Edits:] add update warning and s/ghc/gcc/

I tried this method before and suddenly realize that I could not do it well manually.

Because there are so many packages that I am not sure if they should be left in world.

ppurka · Post by **ppurka** » Wed Jan 06, 2010 11:40 pm

You know, there is an option to emerge to remove packages from world (without recompilation): --deselect. At least, it is present in portage-2.2. I am pretty sure it can be integrated (an else statement!) with the script provided a few posts earlier so as to remove from the world file the packages which have reverse dependencies.

EDIT:

Code: Select all

~> while read pkg; do [[ -n "$(equery -q depends "$pkg")" ]] && echo emerge --deselect "$pkg"; done < /var/lib/portage/world
emerge --deselect app-arch/rar
emerge --deselect app-cdr/k3b
emerge --deselect app-editors/gvim
emerge --deselect app-editors/vim-core
emerge --deselect app-portage/portage-utils
emerge --deselect app-shells/gentoo-bashcomp
emerge --deselect app-text/iso-codes
emerge --deselect app-text/pdftk
emerge --deselect dev-cpp/eigen
emerge --deselect dev-db/mysql
emerge --deselect dev-libs/apr
emerge --deselect dev-libs/cyrus-sasl
emerge --deselect dev-libs/elfutils
emerge --deselect dev-libs/soprano
emerge --deselect dev-python/setuptools
emerge --deselect dev-tex/xcolor
emerge --deselect dev-texlive/texlive-genericrecommended
emerge --deselect dev-texlive/texlive-latexrecommended

... and still running

shallpion · Post by **shallpion** » Thu Jan 07, 2010 12:30 am

I doubt most of the xxx-libs/xxx package can be removed from world.

Mike Hunt · Post by **Mike Hunt** » Thu Jan 07, 2010 1:25 am

Huh,

Whatever-libs are dependencies which shouldn't be in the world file.
Of course there is an exception - as there always is:

sys-libs/gpm

but that's because it really should be in like sys-apps or app-admin, or something.

Nerevar · Post by **Nerevar** » Thu Jan 07, 2010 3:46 am

Much faster/uglier (feel free to clean it up)...

Code: Select all

xargs -a /var/lib/portage/world equery -N -C depends | while read line ; do
    if [ -n "$line" ] ; then
        PKG="$MATCH"
        MATCH="`echo "$line" | awk '{ if ($2 == "Searching") print $6"/"$4 }'`"
        if [ -z "$MATCH" -a -n "$PKG" ] ; then
            echo "$PKG"
        fi  
    fi  
done

ppurka · Post by **ppurka** » Thu Jan 07, 2010 3:52 am

I had a lot of packages to be removed from world. So, I promptly did

Code: Select all

app-arch/rar app-cdr/k3b app-editors/gvim app-editors/vim-core app-portage/portage-utils app-shells/gentoo-bashcomp app-text/iso-codes app-text/pdftk dev-cpp/eigen dev-db/mysql dev-libs/apr dev-libs/cyrus-sasl dev-libs/elfutils dev-libs/soprano dev-python/setuptools dev-tex/xcolor dev-texlive/texlive-genericrecommended dev-texlive/texlive-latexrecommended dev-util/git dev-util/subversion kde-base/kdemultimedia-kioslaves kde-base/konsole media-sound/phonon media-video/mplayer media-video/nvidia-settings net-misc/ntp net-misc/wicd net-print/hplip rox-base/rox sys-apps/hdparm sys-apps/openrc sys-apps/sdparm sys-devel/binutils-config sys-devel/gcc sys-devel/gcc-config sys-devel/gdb sys-kernel/zen-sources sys-libs/db:4.5 sys-libs/glibc sys-power/acpid sys-process/lsof virtual/perl-libnet www-client/mozilla-firefox x11-apps/mesa-progs x11-apps/xdpyinfo x11-apps/xhost x11-apps/xvinfo x11-apps/xwininfo x11-drivers/xf86-input-synaptics x11-libs/libXxf86dga x11-libs/libvdpau

Then I tried emerge --depclean.

Code: Select all

 app-editors/gvim
 net-print/hplip
 sys-libs/db
 x11-apps/xhost
 net-misc/wicd
 app-arch/rar
 sys-apps/sdparm
 x11-libs/libXxf86dga
 net-misc/ntp
 net-print/foomatic-filters
 net-print/cupsddk
 net-wireless/wpa_supplicant
 net-misc/dhcpcd
 net-wireless/wireless-tools
 dev-python/urwid
 app-vim/ntp-syntax
 dev-libs/libnl
 rox-extra/videothumbnail
 rox-base/mime-editor
 rox-base/thumbs
 rox-base/rox-lib
 rox-base/rox
 rox-base/zeroinstall-injector
 app-shells/gentoo-bashcomp
 app-shells/bash-completion
 sys-apps/miscfiles

Now, I fixed some of them like wicd (solid was emerged with -wicd), but there are a couple of circular dependencies which are not picked up. Examples are rox and mime-editor, cups and hplip, gentoo-bashcomp and bash-completion, etc. It is in these circular dependency cases that the depclean becomes dangerous

Most surprising is gvim:

Code: Select all

~> eqd gvim
 * Searching for gvim ...
app-vim/gentoo-syntax-20090220 (>=app-editors/gvim-7.0)
app-vim/gtk-syntax-20060423 (>=app-editors/gvim-7.0)
app-vim/ntp-syntax-20031001 (>=app-editors/gvim-7.0)
app-vim/pam-syntax-20060424 (>=app-editors/gvim-7.0)
dev-util/cmake-2.6.4-r3 (vim-syntax ? app-editors/gvim)
virtual/editor-0 (app-editors/gvim)
~> eqd editor
 * Searching for editor ...
app-admin/sudo-1.7.2_p1 (virtual/editor)
~> grep sudo /var/lib/portage/world
app-admin/sudo

Mike Hunt · Post by **Mike Hunt** » Thu Jan 07, 2010 4:05 am

Hmmm, that's weird, these (and maybe a couple more) should absolutely not be needed in the world file. If they are then there is a problem someplace else, other than circular dependencies.

rox-extra/videothumbnail
rox-base/mime-editor
rox-base/thumbs
rox-base/rox-lib
rox-base/zeroinstall-injector
x11-libs/libXxf86dga

ppurka · Post by **ppurka** » Thu Jan 07, 2010 4:49 am

Mike Hunt wrote:Hmmm, that's weird, these (and maybe a couple more) should absolutely not be needed in the world file. If they are then there is a problem someplace else, other than circular dependencies.
rox-extra/videothumbnail
rox-base/mime-editor
rox-base/thumbs
rox-base/rox-lib
rox-base/zeroinstall-injector
x11-libs/libXxf86dga

All of the highlighted ones were removed from depclean once I put rox in the world file,- so it was a circular dependency with rox that pulled in all of these in depclean. libXxf86dga is a stray package and I haven't tried unmerging it yet (doing an update world right now).

To fix the original depclean list, I had to add the following packages to world: hplip, xhost, ntp, gentoo-bashcomp.
And I had to modify the following USE flags: +rar in mplayer, +scsi in laptop-mode-tools, +wicd in solid.
Now, what I have in depclean is libXxf86dga and a specific version of db.

Mike Hunt · Post by **Mike Hunt** » Thu Jan 07, 2010 5:46 am

Oh well, strange thing about those circular deps. I thought such things only effected emerge dependency checking installation issues.

At any rate, on my box, x11-libs/libXxf86dga is pulled in by x11-base/xorg-x11 or the dga USE flag in mplayer.

Cheers,

ppurka · Post by **ppurka** » Thu Jan 07, 2010 6:20 am

Mike Hunt wrote:Oh well, strange thing about those circular deps. I thought such things only effected emerge dependency checking installation issues.

Yeah. It is the first time I am seeing it affecting depclean. No wonder blindly running depclean is not advised.

At any rate, on my box, x11-libs/libXxf86dga is pulled in by x11-base/xorg-x11 or the dga USE flag in mplayer.

Cheers,

The former is not emerged and mplayer has -dga. I only wonder why I installed it in the first place.