Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
cleaning the world file (wiki) - check the script
View unanswered posts
View posts from last 24 hours

Goto page 1, 2, 3, 4  Next  
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
Mr. T.
Guru
Guru


Joined: 26 Dec 2016
Posts: 477

PostPosted: Fri Jan 05, 2018 6:46 pm    Post subject: Reply with quote

The script checkworldfile seems weird. Normally, the first test mentions if the package is a dependency for other packages so the second test is useless.


Code:
#!/bin/bash

while read i ; do \
        if [ -n "$(qdepends -Q $i)" ]; then \
                echo '' ; echo 'checking '$i ;
                if [ -n "$(emerge -p --quiet --depclean $i)" ]; then \
                        echo $i' needs to stay in @world'
                else
                        echo $i' can be deselected'
                        echo $i >> /tmp/deselect
                fi
        fi
done < /var/lib/portage/world


Yamakuzure wrote:
There is a very big problem with trying to use qdepends like this: It lists all packages that depend on the package you check, but this includes packages that are *not* in your world file.


The package depends on another package that is a dependency of an installed package (indirect depedency). I agree with khayyam!
Back to top
View user's profile Send private message
Frautoincnam
Apprentice
Apprentice


Joined: 19 May 2017
Posts: 294

PostPosted: Fri Jan 05, 2018 9:17 pm    Post subject: Reply with quote

Code:
checking app-editors/vim
app-editors/vim can be deselected

# emerge --deselect vim
>>> Removing app-editors/vim from "world" favorites file...
# emerge --depclean -a

 * Always study the list of packages to be cleaned for any obvious
 * mistakes. Packages that are part of the world set will always
 * be kept.  They can be manually added to this set with
 * `emerge --noreplace <atom>`.  Packages that are listed in
 * package.provided (see portage(5)) will be removed by
 * depclean, even if they are part of the world set.
 *
 * As a safety measure, depclean will not remove any packages
 * unless *all* required dependencies have been resolved.  As a
 * consequence of this, it often becomes necessary to run
 * `emerge --update --newuse --deep @world` prior to depclean.

Calculating dependencies... done!
>>> Calculating removal order...

>>> These are the packages that would be unmerged:

 dev-python/docutils
    selected: 0.13.1
   protected: none
     omitted: none

 dev-python/pygments
    selected: 2.2.0
   protected: none
     omitted: none

 app-vim/gentoo-syntax
    selected: 20170225
   protected: none
     omitted: none


!!! 'app-editors/vim' (virtual/editor) is part of your system profile.
!!! Unmerging it may be damaging to your system.


 app-editors/vim
    selected: 8.0.1298
   protected: none
     omitted: none

 app-eselect/eselect-vi
    selected: 1.1.9
   protected: none
     omitted: none

 app-editors/vim-core
    selected: 8.0.1298
   protected: none
     omitted: none

 dev-util/ctags
    selected: 20161028
   protected: none
     omitted: none

 app-eselect/eselect-ctags
    selected: 1.18
   protected: none
     omitted: none

All selected packages: =dev-python/docutils-0.13.1 =app-eselect/eselect-ctags-1.18 =dev-util/ctags-20161028 =app-vim/gentoo-syntax-20170225 =app-editors/vim-8.0.1298 =dev-python/pygments-2.2.0 =app-eselect/eselect-vi-1.1.9 =app-editors/vim-core-8.0.1298

>>> 'Selected' packages are slated for removal.
>>> 'Protected' and 'omitted' packages will not be removed.

Would you like to unmerge these packages? [Yes/No] no

Quitting.

Packages installed:   669
Packages in world:    112
Packages in system:   43
Required packages:    661
Number to remove:     8

 * IMPORTANT: 4 news items need reading for repository 'gentoo'.
 * Use eselect news read to view new items.

# emerge --noreplace vim

 * IMPORTANT: 4 news items need reading for repository 'gentoo'.
 * Use eselect news read to view new items.

Calculating dependencies... done!
>>> Recording app-editors/vim in "world" favorites file...
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.
# emerge --depclean -a

 * Always study the list of packages to be cleaned for any obvious
 * mistakes. Packages that are part of the world set will always
 * be kept.  They can be manually added to this set with
 * `emerge --noreplace <atom>`.  Packages that are listed in
 * package.provided (see portage(5)) will be removed by
 * depclean, even if they are part of the world set.
 *
 * As a safety measure, depclean will not remove any packages
 * unless *all* required dependencies have been resolved.  As a
 * consequence of this, it often becomes necessary to run
 * `emerge --update --newuse --deep @world` prior to depclean.

Calculating dependencies... done!
>>> No packages selected for removal by depclean
>>> To see reverse dependencies, use --verbose
Packages installed:   669
Packages in world:    113
Packages in system:   43
Required packages:    669
Number removed:       0
Back to top
View user's profile Send private message
charles17
Advocate
Advocate


Joined: 02 Mar 2008
Posts: 3664

PostPosted: Thu Jan 11, 2018 12:40 pm    Post subject: Reply with quote

helecho,
Please post here your modified complete proposal. So we can cross check.
Back to top
View user's profile Send private message
Mr. T.
Guru
Guru


Joined: 26 Dec 2016
Posts: 477

PostPosted: Thu Jan 11, 2018 12:52 pm    Post subject: cleaning the world file (wiki) - check the script Reply with quote

The script seems odd.

checkworldfile:
#!/bin/bash

while read i ; do \
        if [ -n "$(qdepends -Q $i)" ]; then \
                echo '' ; echo 'checking '$i ;
                if [ -n "$(emerge -p --quiet --depclean $i)" ]; then \
                        echo $i' needs to stay in @world'
                else
                        echo $i' can be deselected'
                        echo $i >> /tmp/deselect
                fi
        fi
done < /var/lib/portage/world


Could someone explain how does it work?
Back to top
View user's profile Send private message
charles17
Advocate
Advocate


Joined: 02 Mar 2008
Posts: 3664

PostPosted: Thu Jan 11, 2018 12:57 pm    Post subject: Re: cleaning the world file (wiki) - check the script Reply with quote

helecho wrote:
Could someone explain how does it work?

Doesn't it work for you? What output is it giving you?
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6227
Location: Room 101

PostPosted: Thu Jan 11, 2018 1:08 pm    Post subject: Re: cleaning the world file (wiki) - check the script Reply with quote

helecho wrote:
Could someone explain how does it work?

helecho ... the 'while' loop reads in /var/lib/portage/world asigning each line the variable 'i', each 'i', or package, is then queried for reverse dependencies, if that test is non-zero, then a '--pretend --depclean' is done, if that returns non-zero nothing is done, if zero then write the $i to /tmp/deselect.

best ... khay
Back to top
View user's profile Send private message
Naib
Watchman
Watchman


Joined: 21 May 2004
Posts: 6051
Location: Removed by Neddy

PostPosted: Thu Jan 11, 2018 1:17 pm    Post subject: Reply with quote

It reads every line of the world file and checks if anything depends on it.
Ideally only user requested packaged reside in the world file
_________________
Quote:
Removed by Chiitoo
Back to top
View user's profile Send private message
Mr. T.
Guru
Guru


Joined: 26 Dec 2016
Posts: 477

PostPosted: Thu Jan 11, 2018 2:00 pm    Post subject: Reply with quote

The script does not work.

Code:
user $ emerge app-arch/upx-ucl
[ebuild  N    ] dev-libs/ucl-1.03-r1  USE="-static-libs"
[ebuild  N    ] app-arch/upx-ucl-3.91  USE="zlib -lzma"
# Currently, there aren't obligatory dependencies in my world file.
user $ ./checkworldfile
# I put the dev-libs/ucl dependency in the world file
user $ emerge --noreplace dev-libs/ucl
# checkworlfile detects the dependency is needed
user $ ./checkworldfile
checking dev-libs/ucl
dev-libs/ucl needs to stay in @world
# The package that pulled the dependency is uninstalled.
user $ emerge --unmerge app-arch/upx-ucl
# The package dev-libs/ucl isn't a dependency but the script does not detect it.
user $ ./checkworldfile


helecho.
Back to top
View user's profile Send private message
Mr. T.
Guru
Guru


Joined: 26 Dec 2016
Posts: 477

PostPosted: Thu Jan 11, 2018 2:11 pm    Post subject: Reply with quote

A package installed by a user may become a dependency after installing another package.

charles17, qdepends -Q only displays build time dependencies. qdepends -aQ displays build, runtime and "post" dependencies.

I think I do not misunderstand the function of the script by reading Naib's post and khayyam's post.
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7470

PostPosted: Thu Jan 11, 2018 3:42 pm    Post subject: Reply with quote

You misunderstood it :)

Code:
if [ -n "$(qdepends -Q $i)" ]; then \

That's the key part you miss, this check if a package in world has a reverse dependency
Why, because you don't need to try guessing what will be result of depclean on a package that nobody use, it's a condition for depclean.

When you remove app-arch/upx-ucl from your system, you end with dev-libs/ucl no more use by anybody, as such the only logic there is: keep it in world or depclean will catch it.

The problem with this script is that to make you happy, it should have an "else" condition to the qdepends check, that just say "keep it".

And the logic is :
-> check if pck-A have a reverse dep
---> no, no more test need, you already know you must keep it in world, else it's a candidate for depclean (and it's there the script miss a "you must keep it" output)
---> yes? ok, pck-B use it, but does pck-B really need it? (which in real is not checking if pck-B need it, but just see if depclean will agree to remove pck-A
----------> yes, it really need it, so depclean will refuse to remove it -> tell user it could remove pck-A from world because pck-B (in real, you don't who protect it, just that it is protect from depclean) protect it
----------> no, pck-B could use pck-C if pck-A is not there -> tell user to keep pck-A in world, else depclean will remove it because pck-B will be happy with pck-C instead. (again in real you have clue who is ok with that, just that depclean is ok to remove pck-A)

And on your test case, when you do --unmerge app-arch/upx-ucl you are making dev-libs/ucl no more use by anyone as it was only that package that use it in your test case.
The qdepends test fail: dev-libs/ucl is not use by anyone, test end here because you already know you must keep it in world, or depclean will catch it.
Back to top
View user's profile Send private message
Mr. T.
Guru
Guru


Joined: 26 Dec 2016
Posts: 477

PostPosted: Thu Jan 11, 2018 4:10 pm    Post subject: Reply with quote

krinn, the goal is rather to remove some dependencies placed in the world file. Thus, potential blockages are reduced.
Back to top
View user's profile Send private message
charles17
Advocate
Advocate


Joined: 02 Mar 2008
Posts: 3664

PostPosted: Thu Jan 11, 2018 4:53 pm    Post subject: Reply with quote

helecho wrote:
krinn, the goal is rather to remove some dependencies placed in the world file. ....

... and to keep packages added intentionally. How else should I do with e.g. media-sound/mpg123?
Code:
$ qdepends -aQ media-sound/mpg123
 * DEPEND
app-eselect/eselect-mpg123-0.1
 * RDEPEND
app-eselect/eselect-mpg123-0.1
 * PDEPEND

So app-eselect/eselect-mpg123 needs it. But --depclean would like to remove it if not registered in @world:
Code:
$ emerge -pvc  media-sound/mpg123

Calculating dependencies... done!
>>> Calculating removal order...

>>> These are the packages that would be unmerged:

 media-sound/mpg123
    selected: 1.25.6
   protected: none
     omitted: none

All selected packages: =media-sound/mpg123-1.25.6

>>> 'Selected' packages are slated for removal.
>>> 'Protected' and 'omitted' packages will not be removed.

Packages installed:   900
Packages in world:    97
Packages in system:   43
Required packages:    899
Number to remove:     1
Back to top
View user's profile Send private message
Mr. T.
Guru
Guru


Joined: 26 Dec 2016
Posts: 477

PostPosted: Thu Jan 11, 2018 5:04 pm    Post subject: Reply with quote

charles17, emerge --depclean will remove it because it is an unneeded "dependency".

app-eselect/eselect-mpg-123 is a dependency of media-sound/mpg123.

Quote:
app-eselect/eselect-mpg-123 Manage /usr/bin/mpg123 symlink

helecho.
Back to top
View user's profile Send private message
Mr. T.
Guru
Guru


Joined: 26 Dec 2016
Posts: 477

PostPosted: Thu Jan 11, 2018 5:23 pm    Post subject: Reply with quote

In fact, qdepends sometimes displays wrong dependencies (e.g. soft blockers, choice between various packages, etc.).

Anyway, these limitations must be circumvented.

If I remember correctly: qdepends -aQ tells that dev-lang/vala is a dependency of dev-libs/gobject-introspection although dev-lang/vala is not installed.

Code:
user $ emerge -pv dev-lang/vala
[N] dev-lang/vala
[N] dev-libs/vala-common
user $ qdepends -aQ dev-lang/vala
dev-libs/gobject-introspection


P.S: I have done a test and I got a lot of false dependencies (more than one hundred). Nonetheless, I made an assumption that was not good to test qdepends.

I believe these limitation may be circumvented.

helecho.


Last edited by Mr. T. on Thu Jan 11, 2018 5:40 pm; edited 1 time in total
Back to top
View user's profile Send private message
Jaglover
Watchman
Watchman


Joined: 29 May 2005
Posts: 8291
Location: Saint Amant, Acadiana

PostPosted: Thu Jan 11, 2018 5:38 pm    Post subject: Reply with quote

I personally do not believe some script can decide for me what I want, thus my method is to look at world file and remove everything I haven't explicitly installed by hand. Over years I've learned to use always emerge -1 when troubleshooting, thanks to this my world files stay clean and do not require maintenance.
_________________
My Gentoo installation notes.
Please learn how to denote units correctly!
Back to top
View user's profile Send private message
Mr. T.
Guru
Guru


Joined: 26 Dec 2016
Posts: 477

PostPosted: Thu Jan 11, 2018 5:42 pm    Post subject: Reply with quote

charles17, you do not want to keep the package (media-sound/mpg123) because it does not depend on another one. In fact, qdepends displays wrong information.

media-sound/mpg123 is a soft blocker and so appears in the RDEPEND variable (look at the app-eselect/eselect-mpg123 ebuild).
Back to top
View user's profile Send private message
charles17
Advocate
Advocate


Joined: 02 Mar 2008
Posts: 3664

PostPosted: Thu Jan 11, 2018 5:51 pm    Post subject: Reply with quote

helecho wrote:
charles17, you do not want to keep the package (media-sound/mpg123) because it does not depend on another one.

I am using it, so I'll keep it.

You now understand the checkworldfile script?
Code:
$ while read i ; do qdepends -Q $i; done < /var/lib/portage/world
app-arch/xz-utils-5.2.3
net-misc/dhcpcd-ui-9999
app-office/lyx-2.2.3-r1
app-office/texmacs-1.99.2-r1
app-eselect/eselect-mpg123-0.1
sys-apps/i2c-tools-3.1.1-r1

And I want to keep all of them!
Back to top
View user's profile Send private message
Mr. T.
Guru
Guru


Joined: 26 Dec 2016
Posts: 477

PostPosted: Thu Jan 11, 2018 5:58 pm    Post subject: Reply with quote

charles17, the script displays information about what package(s) may be "safely" removed.
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7470

PostPosted: Thu Jan 11, 2018 6:00 pm    Post subject: Reply with quote

It's not a false dependency, first because it's a reverse dependency and second because that reverse dep exists, even if its existence is conditional.

pckA need pckB if use="this"
checking -Q pckA you will get
pckB (with may be true or not depending on "this"/"-this")

because even pckA is not install, pckB will be its reverse dependency if you install it.
but this may not be true once it is install, only if user has set USE="this" with pckB.
and the idea is just a quick lookup, if qdepends tells a reverse deps is on pckA, it might not be "real" if the conditional is not met, and depclean test will answer this.
but if qdepends tells "nobody need it", then you take the shortcut with "none use it, i must keep it safe in world", because you are certain depclean will remove it.

I think you've put too much hope on this script: the script have no idea if vlc is a package you have add in world to use vlc ; but it check if vlc is use by someone else, and if it is, it will suggest : "bah, no need for vlc in world, someone protect it from depclean".
I'm taking vlc, because the script suggest i could remove vlc from world, because it have reverse dep to media-plugins/npapi-vlc-2.0.0, when for me, i care about vlc, and don't care about npapi, i use vlc, so i use npapi because i have vlc, but in real, npapi is just there because i could, while i have no real need of that.
but it's true that if i remove vlc from world, it will remains protect from depclean (lol for me, rock solid protect as i don't use depclean that much) ; but this only as long as that npapi is set (something i could removed anytime because i don't care about it).
and if you forget that: you'll end with npapi remove because you don't care about it, and vlc is now candidate for depclean (which is sole protection that it was in world has been removed if you have follow the script advise).
Back to top
View user's profile Send private message
John R. Graham
Administrator
Administrator


Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia

PostPosted: Thu Jan 11, 2018 6:15 pm    Post subject: Reply with quote

helecho wrote:
charles17, the script displays information about what package(s) may be "safely" removed.
Well, not exactly. "Safely deselected" would be more accurate. But it's more nuanced than that (as per krinn).

- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters.
Back to top
View user's profile Send private message
charles17
Advocate
Advocate


Joined: 02 Mar 2008
Posts: 3664

PostPosted: Thu Jan 11, 2018 6:26 pm    Post subject: Reply with quote

John R. Graham wrote:
Well, not exactly. "Safely deselected" would be more accurate. ...

And still wrong. Those are packages I am using. If I deselect them, --depclean would remove them.
Back to top
View user's profile Send private message
Mr. T.
Guru
Guru


Joined: 26 Dec 2016
Posts: 477

PostPosted: Thu Jan 11, 2018 6:26 pm    Post subject: Reply with quote

krinn, the script should display information. qdepends only displays information about installed packages, otherwise, the software would fail.
Back to top
View user's profile Send private message
Mr. T.
Guru
Guru


Joined: 26 Dec 2016
Posts: 477

PostPosted: Thu Jan 11, 2018 6:30 pm    Post subject: Reply with quote

The script does not know if a package is a dependency or if a package is required by a user. Anyway, it displays packages that may be "safely" removed.

Thus, it is easy (using the script) to know if a package can be removed.
Back to top
View user's profile Send private message
charles17
Advocate
Advocate


Joined: 02 Mar 2008
Posts: 3664

PostPosted: Thu Jan 11, 2018 6:36 pm    Post subject: Reply with quote

helecho,

If it makes you happy, go and remove the Keep it clean section from the https://wiki.gentoo.org/wiki/World_set_%28Portage%29 article.
For me it does what I want and I'll keep that script.

ok?
Back to top
View user's profile Send private message
Mr. T.
Guru
Guru


Joined: 26 Dec 2016
Posts: 477

PostPosted: Thu Jan 11, 2018 6:44 pm    Post subject: Reply with quote

Your script does not seem correct.

krinn wrote:
I'm taking vlc, because the script suggest i could remove vlc from world, because it have reverse dep to media-plugins/npapi-vlc-2.0.0


helecho.


Last edited by Mr. T. on Thu Jan 11, 2018 6:44 pm; edited 1 time in total
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Goto page 1, 2, 3, 4  Next
Page 1 of 4

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum