Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Clean out your world file
View unanswered posts
View posts from last 24 hours

Goto page 1, 2, 3 ... 27, 28, 29  Next  
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
ecatmur
Advocate
Advocate


Joined: 20 Oct 2003
Posts: 3595
Location: Edinburgh

PostPosted: Sat Feb 28, 2004 4:33 am    Post subject: Clean out your world file Reply with quote

Edit: this script also now contains an "emerge depclean" that actually works, an "emerge prune" that actually looks at your dependencies, and a "qpkg -q" that isn't horribly broken! :D

A world file (/var/cache/edb/world) with lots of entries is no good for keeping your system clean and running smooth. Of course you need to go through it now and again to remove apps you don't want, but the job's easier if it contains as few 'extra' packages as possible i.e. packages that are actually deps on other packages.

Download: http://catmur.co.uk/~ed/main/my-bin/dep
Ebuild: http://catmur.co.uk/svn/repos/gentoo/tree/app-portage/udept/
Website: http://catmur.co.uk/gentoo/udept/

Quote:

dep v. 0.3.00 "Level Of Incompetence"
an inappropriate Portage engine!

Usage: dep [OPTION]... [PACKAGE]...
List dependencies and reverse dependencies of packages, and clean the world
file and the set of installed packages.

Task selection:
-h, -?, --help Display this help
-P, --purge Unmerge unneeded SLOTted packages
-d, --depclean Unmerge all unneeded packages
-l, --list-depends (default) List dependencies of PACKAGE
-L, --list-raw-depends Display the dependency strings for PACKAGE
-r, --list-rev-depends List reverse dependencies of PACKAGE
-w, --pruneworld Remove redundant entries from the world file

Options:
-b, --no-build-time Omit dependencies needed solely to build PACKAGE
-o, --original-depends Calculate dependencies as when PACKAGE was merged
-a, --ask (default) For -[Pdw]: ask to perform action
-f, --force For -[Pdw]: perform action without asking
-p, --pretend For -[Pdw]: show what action would be taken

A package is unneeded if there is no installed package that depends on it.
An entry in the world file is redundant if there is another installed package
that depends on it.
When merging the new world file, the merging program used is sdiff(1).
Your world file is located at /var/lib/portage/world.


And here's some sample output:
Code:
# dep -w
!!!REDUNDANT!!! app-admin/gkrellm depended on by:
        x11-plugins/gkrellflynn-0.6
        x11-plugins/gkrellmoon-0.6
        x11-plugins/gkrellmss-2.3
app-admin/syslog-ng
!!!REDUNDANT!!! app-editors/gvim depended on by:
        app-text/vim-latex-1.5_rc1
        app-vim/info-1.7-r1
app-editors/nano
app-emulation/wine
...
x11-themes/mplayer-skins
x11-themes/nautilus-themes
 
145 packages in worldfile: 137 valid, 8 redundant; 70 packages in system.
563 packages installed: 12% in system, 25% in world, 63% deps.


Post back if you like it and if I've missed anything!
_________________
No more cruft
dep: Revdeps that work
Using command-line ACCEPT_KEYWORDS?


Last edited by ecatmur on Mon Mar 20, 2006 12:31 pm; edited 16 times in total
Back to top
View user's profile Send private message
iBormuth
n00b
n00b


Joined: 13 Jan 2004
Posts: 59

PostPosted: Sat Feb 28, 2004 10:21 am    Post subject: Reply with quote

That looks quite good to me so I didn't go through the source yet.

Quote:

although I would *never* do this myself as it's far too unsafe.


What makes you beleve this is that insecure ?

There is just one scenario I could imagine:

- You have package X in world and want it there.
- You install package Y which depends on X
- You run pruneworld which removes X as it is implied by Y
- You run emerge -c -C Y

Now you lost package X.

Should't is be possible to have an additional file (e. g. MyWorld) that holds packages you explicitly want to keep in the system. Your script could make shure to keep ale packages listed in theis file in world.

What do you think ... ?


- Ingo
Back to top
View user's profile Send private message
deepthink
n00b
n00b


Joined: 25 Mar 2003
Posts: 35

PostPosted: Sun Mar 07, 2004 2:34 pm    Post subject: Reply with quote

Great work.

Thanks!
Back to top
View user's profile Send private message
TGL
Bodhisattva
Bodhisattva


Joined: 02 Jun 2002
Posts: 1978
Location: Rennes, France

PostPosted: Sun Mar 07, 2004 6:03 pm    Post subject: Reply with quote

Nice idea and nice script, thanks a lot ecatmur.

iBormuth wrote:
There is just one scenario I could imagine:

- You have package X in world and want it there.
- You install package Y which depends on X
- You run pruneworld which removes X as it is implied by Y
- You run emerge -c -C Y

Now you lost package X.
Imho this scenario is enough to not blindly follow the script recommandations. I've seen many such cases running the script where the Y depending on X was a package I may unmerge someday whereas X was really a must have (for instance, gphoto -> hotplug. I don't need gphoto anymore, and will probably unmerge it, but I really do need hotplug).

Also, the script is not completly safe regarding virtual packages I think. For instance:
Code:
!!!REDUNDANT!!! net-p2p/bittorrent-theshadow depended on by:
        net-p2p/bittorrent-theshadow-5.8.11          (as virtual/bittorrent)
If i remove this one, since nothing depends on the virtual, bittorent will probably get depcleaned. I think virtuals should only count if they are actually in the system class or depended on by another package.

iBormuth wrote:
Should't is be possible to have an additional file (e. g. MyWorld) that holds packages you explicitly want to keep in the system. Your script could make shure to keep ale packages listed in theis file in world.
You only move the issue to another file, but the whole problem remains to maintain such a file. Doing it completly automatically, based on emerge calls, is exactly what world is already. Doing it completly manually is not really realistic: count approx 300 entries for an average desktop system... you will always forget to add some stuff there, etc. Only a mix of automation and manual work seems really practical, and this is exactly the same as using only the world file with an occasional manual cleanup (which becomes easier with this script).


Side note: to keep your world file clean, don't forget the "--oneshot" emerge option. For instance, when you re-emerge a library because it is broken, adding the lib to the world file usually makes no sense. That's exactly what --oneshot is for.
Back to top
View user's profile Send private message
MighMoS
Guru
Guru


Joined: 24 Apr 2003
Posts: 416
Location: @ ~

PostPosted: Sun Mar 07, 2004 6:09 pm    Post subject: Reply with quote

This is a really cool script...Thanks man. I think I'll just use the program as a suggestion guide to things I may wanna take out though.
_________________
jabber: MighMoS@jabber.org

localhost # export HOME=`which heart`
Back to top
View user's profile Send private message
ecatmur
Advocate
Advocate


Joined: 20 Oct 2003
Posts: 3595
Location: Edinburgh

PostPosted: Mon Mar 08, 2004 11:45 am    Post subject: Reply with quote

Bittorrent problem fixed, thanks TGL. (it wasn't removing !pkgspec atoms properly).
Back to top
View user's profile Send private message
dyqik
Tux's lil' helper
Tux's lil' helper


Joined: 08 May 2003
Posts: 120
Location: Oxford, UK

PostPosted: Mon Mar 08, 2004 12:01 pm    Post subject: Reply with quote

I just cut and pasted this into scite and removed the leading spaces on each line. The script crashes with the error "Syntax Error: unexpected end of file" at line 192. Anything else I should do other than cutting and pasting?
Back to top
View user's profile Send private message
iBormuth
n00b
n00b


Joined: 13 Jan 2004
Posts: 59

PostPosted: Mon Mar 08, 2004 12:07 pm    Post subject: Reply with quote

What do you think about that:

Code:

sort /var/cache/edb/world > /tmp/world.sorted ; pruneworld | sort | join /tmp/world.sorted - | less


It gives you the opposite of pruneword. Actually this is the list I would look at and decide wheater I need the packages or not. All the redundant stuff can stay in the world file.

When you run that command every some weeks the system sould stay quite slim.
Back to top
View user's profile Send private message
nickbrown
n00b
n00b


Joined: 07 Apr 2003
Posts: 15
Location: Edinburgh, Scotland.

PostPosted: Fri Mar 12, 2004 12:28 pm    Post subject: Reply with quote

dyqik wrote:
I just cut and pasted this into scite and removed the leading spaces on each line. The script crashes with the error "Syntax Error: unexpected end of file" at line 192. Anything else I should do other than cutting and pasting?


I'm seeing exactly the same problem, although I simply cut and pasted it.
Back to top
View user's profile Send private message
nickbrown
n00b
n00b


Joined: 07 Apr 2003
Posts: 15
Location: Edinburgh, Scotland.

PostPosted: Fri Mar 12, 2004 12:55 pm    Post subject: Reply with quote

nickbrown wrote:
dyqik wrote:
I just cut and pasted this into scite and removed the leading spaces on each line. The script crashes with the error "Syntax Error: unexpected end of file" at line 192. Anything else I should do other than cutting and pasting?


I'm seeing exactly the same problem, although I simply cut and pasted it.


This was solved by removing the trailing space after "END". I had to read the bash manual to find out about here documents.
I'm now seeing this error;
Code:

cat:  : No such file or directory
./pruneworld: line 168: syntax error near unexpected token `|'
./pruneworld: line 168: ` | sort \ '

Any ideas?
Or can you provide a link to the script.
Back to top
View user's profile Send private message
moocha
Watchman
Watchman


Joined: 21 Oct 2003
Posts: 5722

PostPosted: Fri Mar 12, 2004 2:08 pm    Post subject: Reply with quote

Removing all spaces at the end of all lines should do the trick :).
Back to top
View user's profile Send private message
nickbrown
n00b
n00b


Joined: 07 Apr 2003
Posts: 15
Location: Edinburgh, Scotland.

PostPosted: Fri Mar 12, 2004 2:19 pm    Post subject: Reply with quote

moocha wrote:
Removing all spaces at the end of all lines should do the trick :).

I just figured that out. Actually all that is need is remove the spaces from continuation lines. Thanks.
Next up, I think the script has a bug. In it out put I'm seeing
Quote:

!!REDUNDANT!!! net-misc/dhcp depended on by:
SYSTEM PROFILE

dhcp is not in the system profile. I think the scripts grepping of /etc/make.profile/packages must not be rigorous enough. It must be find net-misc/dhcpcd by mistake.
Back to top
View user's profile Send private message
ecatmur
Advocate
Advocate


Joined: 20 Oct 2003
Posts: 3595
Location: Edinburgh

PostPosted: Fri Mar 12, 2004 8:54 pm    Post subject: Reply with quote

Good catch, you're exactly right.

It's fixed in the above (adding \> to the end of the 2 make.profile/packages grep strings)
Back to top
View user's profile Send private message
nx12
Apprentice
Apprentice


Joined: 14 Jan 2004
Posts: 193

PostPosted: Tue Mar 23, 2004 1:54 pm    Post subject: Reply with quote

EDIT:My post is useless now.
_________________
signature sucks


Last edited by nx12 on Thu Jan 13, 2005 1:20 pm; edited 1 time in total
Back to top
View user's profile Send private message
M104
Tux's lil' helper
Tux's lil' helper


Joined: 13 Jan 2003
Posts: 132
Location: Riverside, CA

PostPosted: Thu Mar 25, 2004 7:59 am    Post subject: Reply with quote

My test run resulted in this:
Code:
307 packages in world:  197 valid,  110 redundant;
70 packages in system;
744 packages installed: 9% in system, 41% in world, 64% deps.

I have maintained my world file manually for over a year and I had a hard time believing that 1/3 of my world file entries were redundant. I moved the output to my world file (after a backup, of course) and emerge -pv depclean wanted to remove 39 packages. The script seems to ignore USE flags. Here's an example:
Code:
!!!REDUNDANT!!! app-misc/screen depended on by:
        games-fps/enemy-territory-2.56-r2
        games-fps/quake3-1.32b-r3
        games-fps/quake3-alliance-3.3
        games-fps/quake3-brainworks-0.91
        games-fps/quake3-cpma-1.2
        games-fps/quake3-truecombat-1.1

That's only if the "dedicated" flag is set, which it is not in my case. Here's another example:
Code:
!!!REDUNDANT!!! kde-base/kde depended on by:
        app-crypt/kgpg-1.0.0
        app-editors/kxmleditor-1.0.0
        app-misc/krusader-1.30
        dev-util/kdevelop-3.0.1
        kde-base/kdeadmin-3.2.1
        kde-base/kdeartwork-3.1.5
        kde-base/kdeartwork-3.2.1
        kde-base/kdegraphics-3.2.1
        kde-base/kdemultimedia-3.2.1
        kde-base/kdenetwork-3.1.5
        kde-base/kdenetwork-3.2.1
        kde-base/kdepim-3.2.1
        kde-base/kdeutils-3.2.1

This makes emerge -pv depclean want to remove all of KDE! Has anyone else experienced this?
_________________
"Pulling together is the aim of despotism and tyranny. Free men pull in all kinds of directions."
Terry Pratchett, The Truth
Back to top
View user's profile Send private message
iBormuth
n00b
n00b


Joined: 13 Jan 2004
Posts: 59

PostPosted: Thu Mar 25, 2004 9:19 am    Post subject: Reply with quote

It seems that is a known problem of qpkg -q: https://bugs.gentoo.org/show_bug.cgi?id=27738

I think you would have to put the system use flags into the environment variable USE...
Back to top
View user's profile Send private message
M104
Tux's lil' helper
Tux's lil' helper


Joined: 13 Jan 2003
Posts: 132
Location: Riverside, CA

PostPosted: Thu Mar 25, 2004 9:23 am    Post subject: Reply with quote

iBormuth wrote:
It seems that is a known problem of qpkg -q: https://bugs.gentoo.org/show_bug.cgi?id=27738

I think you would have to put the system use flags into the environment variable USE...

Well that shouldn't be too hard. Thanks for the script ecatmur and keep up the good work. :D
_________________
"Pulling together is the aim of despotism and tyranny. Free men pull in all kinds of directions."
Terry Pratchett, The Truth
Back to top
View user's profile Send private message
Meeuw
Tux's lil' helper
Tux's lil' helper


Joined: 08 Nov 2002
Posts: 77
Location: NL - Oostzaan

PostPosted: Tue May 04, 2004 1:09 pm    Post subject: Reply with quote

this script is great!

after cleaning your world file you can run:
emerge -p depclean

to check for 'lost' packages, you really should unmerge them, because they aren't included in 'world' anymore!
Back to top
View user's profile Send private message
ecatmur
Advocate
Advocate


Joined: 20 Oct 2003
Posts: 3595
Location: Edinburgh

PostPosted: Tue May 04, 2004 4:35 pm    Post subject: Reply with quote

Made some changes, and added quite a bit of code from my own portage-bash library. It should now handle USE flags and the like slightly better - it now uses current USE flag settings rather than those which were used to merge the package.
_________________
No more cruft
dep: Revdeps that work
Using command-line ACCEPT_KEYWORDS?
Back to top
View user's profile Send private message
ecatmur
Advocate
Advocate


Joined: 20 Oct 2003
Posts: 3595
Location: Edinburgh

PostPosted: Tue May 04, 2004 4:50 pm    Post subject: Reply with quote

Now uses environment USE flags as well.

Also extended to make a nifty "qpkg -q -I" replacement - I realised that it actually works unlike qpkg, so now you get:
Code:
$ USE="-esd" ./bin/pruneworld media-sound/esound
preparing USE list.................................................................................................
!!!REDUNDANT!!! media-sound/esound depended on by:
        app-accessibility/gok-0.10.2
        gnome-base/control-center-2.6.1
        gnome-base/gnome-2.6
        gnome-base/gnome-libs-1.4.2
        gnome-base/gnome-session-2.6.1
        gnome-base/libgnome-2.6.0
        gnome-base/libgnomeui-2.6.0
        gnome-base/nautilus-2.6.1
        gnome-extra/gnome-media-2.6.0
        media-plugins/gst-plugins-esd-0.6.4
        media-plugins/gst-plugins-esd-0.8.1
        net-im/gnomemeeting-1.0.2
        x11-plugins/gkrellmss-2.4

_________________
No more cruft
dep: Revdeps that work
Using command-line ACCEPT_KEYWORDS?
Back to top
View user's profile Send private message
Meeuw
Tux's lil' helper
Tux's lil' helper


Joined: 08 Nov 2002
Posts: 77
Location: NL - Oostzaan

PostPosted: Tue May 04, 2004 4:53 pm    Post subject: Reply with quote

Code:
cat: /etc/portage/package.use: No such file or directory
what did i miss? :-/
Back to top
View user's profile Send private message
ecatmur
Advocate
Advocate


Joined: 20 Oct 2003
Posts: 3595
Location: Edinburgh

PostPosted: Tue May 04, 2004 5:29 pm    Post subject: Reply with quote

Oops, that's an optional file. Fixed now, thanks.
_________________
No more cruft
dep: Revdeps that work
Using command-line ACCEPT_KEYWORDS?
Back to top
View user's profile Send private message
ecatmur
Advocate
Advocate


Joined: 20 Oct 2003
Posts: 3595
Location: Edinburgh

PostPosted: Wed May 05, 2004 11:44 pm    Post subject: Reply with quote

Just done some upgrades. It now contains a "qpkg -q" that actually works, and a "emerge depclean" that actually works. All written in the greatest of scripting languages, Bash.
_________________
No more cruft
dep: Revdeps that work
Using command-line ACCEPT_KEYWORDS?
Back to top
View user's profile Send private message
scourage
n00b
n00b


Joined: 15 Aug 2003
Posts: 71
Location: UCLA (upper corner of lower Alabama)

PostPosted: Sun May 09, 2004 10:48 pm    Post subject: Reply with quote

I love your script, but I noticed that it removed a circular dependancy:

!!!REDUNDANT!!! net-im/gaim depended on by:
net-im/gaim-encryption-2.25
!!!REDUNDANT!!! net-im/gaim-encryption depended on by:
net-im/gaim-0.77-r2

and both are removed from the world file. I think that it should leave gaim. It might be a problem with the dependencies of gaim and gaim-encryption, or I borked my world file, but it's something odd I noticed.

Cheers,

Bob
_________________
Halfway down the trail to Hell...
Back to top
View user's profile Send private message
ecatmur
Advocate
Advocate


Joined: 20 Oct 2003
Posts: 3595
Location: Edinburgh

PostPosted: Mon May 10, 2004 12:11 am    Post subject: Reply with quote

Um, circular dependencies are a Bad Thing™. There's really not much I can do about that... unless someone can suggest a good O(n) algorithm.
_________________
No more cruft
dep: Revdeps that work
Using command-line ACCEPT_KEYWORDS?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Goto page 1, 2, 3 ... 27, 28, 29  Next
Page 1 of 29

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum