Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Find obsolete USE flags in /etc/portage/package.use [SOLVED]
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
radio_flyer
Guru
Guru


Joined: 04 Nov 2004
Posts: 317
Location: Northern California

PostPosted: Thu Oct 06, 2011 7:22 pm    Post subject: Find obsolete USE flags in /etc/portage/package.use [SOLVED] Reply with quote

I tried searching but came up empty. I'd like to find a way to identify obsolete USE flags in the /etc/portage/package.use file. Here's an example I've been using for test purposes:

Quote:

# emerge -pv noflushd
These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild R ] sys-block/noflushd-2.7.3 0 kB

Total: 1 package (1 reinstall), Size of downloads: 0 kB


In my /etc/portage/package.use:

Quote:

sys-block/noflushd toilet


(Yes, this is just a silly test, but both the package and USE flag exist. The whole package will disappear once I figure out how to flush the obsolete USE flag from package.use...)

As you can see, portage is totally happy to ignore this unneeded use flag:
Quote:

# emerge -uDpv --newuse world

These are the packages that would be merged, in order:

Calculating dependencies... done!

Total: 0 packages, Size of downloads: 0 kB


If I do an eix-update and an eix -Tt, I get:
Quote:

No non-matching entries in /etc/portage/package.keywords.
No non-matching entries in /etc/portage/package.accept_keywords.
No non-matching entries in /etc/portage/package.mask.
No non-matching entries in /etc/portage/package.unmask.
No non-matching or empty entries in /etc/portage/package.use.
No non-matching or empty entries in /etc/portage/package.env.
No non-matching or empty entries in /etc/portage/package.cflags.
The names of all installed packages are in the database.


If I use eix-test-obsolete:
Quote:

No non-matching entries in /etc/portage/package.keywords.
No non-matching entries in /etc/portage/package.accept_keywords.
No non-matching entries in /etc/portage/package.mask.
No non-matching entries in /etc/portage/package.unmask.
No non-matching or empty entries in /etc/portage/package.use.
No non-matching or empty entries in /etc/portage/package.env.
No non-matching or empty entries in /etc/portage/package.cflags.
The names of all installed packages are in the database.

No redundant entries in /etc/portage/package.{,accept_}keywords
No uninstalled entries in /etc/portage/package.{,accept_}keywords
No redundant entries in /etc/portage/package.mask
No uninstalled entries in /etc/portage/package.mask
No redundant entries in /etc/portage/package.unmask
No uninstalled entries in /etc/portage/package.unmask
Skipping check: redundant entries in /etc/portage/package.use
Skipping check: uninstalled entries in /etc/portage/package.use
Skipping check: redundant entries in /etc/portage/package.env
Skipping check: uninstalled entries in /etc/portage/package.env
Skipping check: redundant entries in /etc/portage/package.cflags
Skipping check: uninstalled entries in /etc/portage/package.cflags


I don't understand why eix is saying 'Skipping check', as 'eix --dump-defaults' shows all TEST_* flags set 'TRUE'. I'm not an eix guru though, so perhaps I'm barking up the wrong option tree?

With 'enalyze a use':
Quote:

...
+sysfs .............................. default 1
+t1lib .............................. default 2
+tcl ................................ default 10
+tcpd ............................... default 4
+templates .......................... default 1
+tex4ht ............................. ....... 1
+theora ............................. default 9
+threads ............................ default 35
+threadsafe ......................... ....... 1
+thumbnails ......................... ....... 1
+tiff ............................... default 21
+tk ................................. default 15
+tokenizer .......................... ....... 1
+toolame ............................ ....... 1
+tremor ............................. ....... 1
+truetype ........................... default 22
+twolame ............................ ....... 1
+udev ............................... default 4
...


No mention of 'toilet' at all.

What I'm looking for is something that will report:
Quote:

In package.use: USE flag toilet for package sys-block/noflushd is obsolete


Any ideas?


Last edited by radio_flyer on Fri Oct 07, 2011 4:33 pm; edited 1 time in total
Back to top
View user's profile Send private message
myceliv
Apprentice
Apprentice


Joined: 29 Nov 2007
Posts: 178

PostPosted: Thu Oct 06, 2011 11:34 pm    Post subject: Reply with quote

I believe you could use `enalyze rp use' which would be your minimum set diffed against `enalyze a use' (which analyzes against installed packages.) I just backup package.use and make sure enalyze r result doesn't trigger any rebuilds and move on.
Back to top
View user's profile Send private message
dol-sen
Retired Dev
Retired Dev


Joined: 30 Jun 2002
Posts: 2805
Location: Richmond, BC, Canada

PostPosted: Fri Oct 07, 2011 1:30 am    Post subject: Reply with quote

Yes, "enalyze rebuild use" creates a file/ouptut of the flag settings needed to maintain the current installed state after considering the profile/globals/defaults/make.conf. It totally ignores anything in the current /etc/portage/package.use file(s). One of these days/years I would like to get around to extending the code more to have full validation code able to scan your file(s) and report/update them in place.

What you could do is use that file and diff it with your current one or accumulated one to see what is/might be obsolete.
_________________
Brian
Porthole, the Portage GUI frontend irc@freenode: #gentoo-guis, #porthole, Blog
layman, gentoolkit, CoreBuilder, esearch...
Back to top
View user's profile Send private message
[n00b@localhost]
Apprentice
Apprentice


Joined: 30 Aug 2004
Posts: 266
Location: London, UK

PostPosted: Fri Oct 07, 2011 10:30 am    Post subject: Reply with quote

I use the following script (in ${HOME}/bin/check-use):

Code:
#!/bin/bash

files="/etc/portage/package.use"
if [ -d "${files}" ]
  then
  files="${files}/*"
fi

for file in ${files}
  do
  echo -e "\E[1;33;40m${file}:"
  cat ${file} | while read atom uses
    do
    packages="$(equery -q l ${atom} 2>/dev/null)"
    if [ $? -ne 0 ]
      then
      echo -e "\E[0;31;40mNo package installed matching ${atom}"
    fi
    package_uses=""
    skip=2
    for package in ${packages}
      do
      package_use="$(equery -q u =${package} 2>/dev/null)"
      if [ $? -ne 0 ]
        then
        echo -e "\E[0;31;40mUnable to find use flags for ${package} (the package is not installed or the ebuild for the installed version is missing)"
        if [ ${skip} -ne 0 ]
          then
          skip=1
        fi
      else
        package_uses="${package_uses} $(echo ${package_use} | sed -e's/^-\|+//g')"
        skip=0
      fi
    done
    if [ ${skip} -eq 0 ]
      then
      echo -ne "  \E[0;36;40m$(echo ${atom} | cut -d'/' -f1)/\E[0;32;40m$(echo ${atom} | cut -d'/' -f2)"
      for use in ${uses}
        do
        if [ ! "$(echo ${package_uses} | grep -- $(echo ${use} | sed -e's/^-\|+//g'))" ]
          then
          if [ "$(echo ${use} | cut -c1)" = "-" ]
            then
            echo -ne " \E[1;34;40m${use}"
          else
            echo -ne " \E[1;31;40m${use}"
          fi
        fi
      done
      echo -e "\E[0m"
    fi
  done
done


It lists every installed package in /etc/portage/package.use (or /etc/portage/package.use/*) and outputs any invalid use flags next to each one. It handles multiple packages installed in different slots and packages that have been removed.
Back to top
View user's profile Send private message
radio_flyer
Guru
Guru


Joined: 04 Nov 2004
Posts: 317
Location: Northern California

PostPosted: Fri Oct 07, 2011 4:31 pm    Post subject: Reply with quote

n00b@localhost, excellent script! That's exactly what I was looking for. It found both my test flag and another one I hadn't realized had gone obsolete.

The only problem I encountered running it was that I tend to have blank lines and comment lines documenting why I set the flags the way I do. I easily fixed that by replacing the cat of the package.use file with a simple egrep filtering out comments and blank lines:

Code:

#!/bin/bash

files="/etc/portage/package.use"
if [ -d "${files}" ]
  then
  files="${files}/*"
fi

for file in ${files}
  do
  echo -e "\E[1;33;40m${file}:"
  egrep -v '^[ \t]*#|^$' ${file} | while read atom uses
    do
    packages="$(equery -q l ${atom} 2>/dev/null)"
    if [ $? -ne 0 ]
      then
      echo -e "\E[0;31;40mNo package installed matching ${atom}"
    fi
    package_uses=""
    skip=2
    for package in ${packages}
      do
      package_use="$(equery -q u =${package} 2>/dev/null)"
      if [ $? -ne 0 ]
        then
        echo -e "\E[0;31;40mUnable to find use flags for ${package} (the package is not installed or the ebuild for the installed version is missing)"
        if [ ${skip} -ne 0 ]
          then
          skip=1
        fi
      else
        package_uses="${package_uses} $(echo ${package_use} | sed -e's/^-\|+//g')"
        skip=0
      fi
    done
    if [ ${skip} -eq 0 ]
      then
      echo -ne "  \E[0;36;40m$(echo ${atom} | cut -d'/' -f1)/\E[0;32;40m$(echo ${atom} | cut -d'/' -f2)"
      for use in ${uses}
        do
        if [ ! "$(echo ${package_uses} | grep -- $(echo ${use} | sed -e's/^-\|+//g'))" ]
          then
          if [ "$(echo ${use} | cut -c1)" = "-" ]
            then
            echo -ne " \E[1;34;40m${use}"
          else
            echo -ne " \E[1;31;40m${use}"
          fi
        fi
      done
      echo -e "\E[0m"
    fi
  done
done


I hope that still works for those who have a package.use directory. I don't see why it shouldn't, but I haven't tested it.

Thanks!
Back to top
View user's profile Send private message
Arkhelion
Apprentice
Apprentice


Joined: 07 Sep 2010
Posts: 151
Location: France

PostPosted: Fri Oct 07, 2011 5:03 pm    Post subject: Reply with quote

Other way to do it may be with an /etc/eixrc like this.
Code:
EIX_SYNC_CONF="%{EIX_SYNC_OPTS} *"
CHECK_INSTALLED_OVERLAYS="repository"
NONEXISTENT_IF_OTHER_OVERLAY="true"
REDUNDANT_IF_IN_USE="-some"
REDUNDANT_IF_DOUBLE_USE="some"
REDUNDANT_IF_IN_ENV="-some"
REDUNDANT_IF_DOUBLE_ENV="some"
REDUNDANT_IF_IN_CFLAGS="-some"
REDUNDANT_IF_DOUBLE_CFLAGS="some"


This is mine, the top 3 lines are about checking packages which moved from an overlay, so it's not linked to this issue, but the bottom 6 are supposed to tell eix not to skip the last tests.
_________________
Arkhelion
Back to top
View user's profile Send private message
radio_flyer
Guru
Guru


Joined: 04 Nov 2004
Posts: 317
Location: Northern California

PostPosted: Fri Oct 07, 2011 6:24 pm    Post subject: Reply with quote

Thanks Arkhelion, I was wondering how to enable those skipped checks. Sadly, eix still can't pick up my test entry:

Quote:

sys-block/noflushd toilet


When I run it, I get this:
Quote:

# eix-test-obsolete

No non-matching entries in /etc/portage/package.keywords.
No non-matching entries in /etc/portage/package.accept_keywords.
No non-matching entries in /etc/portage/package.mask.
No non-matching entries in /etc/portage/package.unmask.
No non-matching or empty entries in /etc/portage/package.use.
No non-matching or empty entries in /etc/portage/package.env.
No non-matching or empty entries in /etc/portage/package.cflags.
The names of all installed packages are in the database.

No redundant entries in /etc/portage/package.{,accept_}keywords
No uninstalled entries in /etc/portage/package.{,accept_}keywords
No redundant entries in /etc/portage/package.mask
No uninstalled entries in /etc/portage/package.mask
No redundant entries in /etc/portage/package.unmask
No uninstalled entries in /etc/portage/package.unmask
No redundant entries in /etc/portage/package.use
No uninstalled entries in /etc/portage/package.use
No redundant entries in /etc/portage/package.env
No uninstalled entries in /etc/portage/package.env
No redundant entries in /etc/portage/package.cflags
No uninstalled entries in /etc/portage/package.cflags


As far as I can tell, eix just doesn't seem to pick up the USE flags. When I unmerge noflushd and run eix again, I get:

Quote:

# eix-test-obsolete

No non-matching entries in /etc/portage/package.keywords.
No non-matching entries in /etc/portage/package.accept_keywords.
No non-matching entries in /etc/portage/package.mask.
No non-matching entries in /etc/portage/package.unmask.
No non-matching or empty entries in /etc/portage/package.use.
No non-matching or empty entries in /etc/portage/package.env.
No non-matching or empty entries in /etc/portage/package.cflags.
The names of all installed packages are in the database.

No redundant entries in /etc/portage/package.{,accept_}keywords
No uninstalled entries in /etc/portage/package.{,accept_}keywords
No redundant entries in /etc/portage/package.mask
No uninstalled entries in /etc/portage/package.mask
No redundant entries in /etc/portage/package.unmask
No uninstalled entries in /etc/portage/package.unmask
No redundant entries in /etc/portage/package.use

Not installed but in /etc/portage/package.use:
[N] sys-block/noflushd (2.7.3): A daemon to spin down your disks and force accesses to be cached

No redundant entries in /etc/portage/package.env
No uninstalled entries in /etc/portage/package.env
No redundant entries in /etc/portage/package.cflags
No uninstalled entries in /etc/portage/package.cflags


So eix seems to be able to pick up the package, but not the USE flags. The script, however, very clearly highlights the active USE flags in green and the unused ones in red, and when I run it the 'toilet' use flag clearly stands out.

(And next time I promise to find a better test case, really :roll: )
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Fri Oct 07, 2011 8:32 pm    Post subject: Reply with quote

radio_flyer wrote:
As far as I can tell, eix just doesn't seem to pick up the USE flags

/usr/share/doc/eix-0.23.2/TODO wrote:
Ambitious tasks which will probably not be realized: [...]
- Full support for USE-Flags.
Back to top
View user's profile Send private message
mpagano
Developer
Developer


Joined: 27 Apr 2004
Posts: 197
Location: USA

PostPosted: Thu Dec 15, 2011 8:16 pm    Post subject: Reply with quote

I just committed a new version of portpeek that will clean your package.use file.

This version is 1.9.81

I'm not sure if I have every instance covered, so I have hard masked it for now.

I just committed it minutes ago, so it may take some time to hit the tree.

the options are:

portpeek -s

That will show everything

portpeek -sfq

That will do the fix (f) and ask you first (q)


Code:


15:09:23-root@kiowa:/etc/portage #portpeek -s

package.use:
use flag: systemd is invalid for : sys-fs/udev-171-r4
use flag: java is invalid for : dev-libs/soprano-2.7.0
use flag: gtk is invalid for : app-office/libreoffice-bin-3.4.3.2-r1
use flag: eds is invalid for : app-office/libreoffice-bin-3.4.3.2-r1
use flag: mike is invalid for : sys-apps/pciutils-3.1.7

Package: sys-apps/mike not found. Please check /etc/portage/package.use to validate entry
Done



Last edited by mpagano on Thu Dec 15, 2011 9:04 pm; edited 2 times in total
Back to top
View user's profile Send private message
mpagano
Developer
Developer


Joined: 27 Apr 2004
Posts: 197
Location: USA

PostPosted: Thu Dec 15, 2011 8:22 pm    Post subject: Reply with quote

Its not production ready code yet, but its a first effort.

Make backups first before you change anything.
Back to top
View user's profile Send private message
Rexilion
Veteran
Veteran


Joined: 17 Mar 2009
Posts: 1044

PostPosted: Tue Jun 25, 2013 6:33 pm    Post subject: Reply with quote

Sorry for posting to this thread so late, but I want to say it just helped me a lot...

Just tested portpeek, works like a charm. Only one nitpick: It says pdfimport is not a USE-flag of libreoffice 4.0.9999 while I have 4.0.3.3 installed. All else works.

I checked again with the posted second version of the bash script which I ran after closely inspecting it. All good!
_________________
fs/super.c: "Self-destruct in 5 seconds. Have a nice day...\n"
Back to top
View user's profile Send private message
mwhitlock
Tux's lil' helper
Tux's lil' helper


Joined: 07 Aug 2006
Posts: 87
Location: New Hampshire, USA

PostPosted: Mon Apr 04, 2016 7:04 pm    Post subject: Reply with quote

The Bash script, as modified by radio_flyer, worked on my system, identifying all the obsolete USE flags in /etc/portage/package.use/*, albeit quite slowly due to its calling out to slow Python scripts to do much of its work.

However, I still have no solution for finding redundant USE flags: those that are mentioned in /etc/portage/package.use/* but where such mentions don't alter the effective flag state from that provided by the profile and /etc/portage/make.conf. (I'd also like to see which USE flags mentioned in /etc/portage/make.conf are redundant with respect to the defaults provided by the profile.)
Back to top
View user's profile Send private message
dol-sen
Retired Dev
Retired Dev


Joined: 30 Jun 2002
Posts: 2805
Location: Richmond, BC, Canada

PostPosted: Tue Apr 05, 2016 2:06 am    Post subject: Reply with quote

mwhitlock Both those are capabilities of enalyze.

enalyze analyze -v use

That will list all flags and whether they are default. You will have to manually edit your make.cong USE setting to remove any defaulted duplicates, but that should not be difficult.
You can also use the above report to decide which non-default use flags you want to add to make.conf USE. The regenerate a new package.use (see below)

enalyze rebuild use

will then only add entries to a new package.use file to maintain the installed state after considering everything else.
_________________
Brian
Porthole, the Portage GUI frontend irc@freenode: #gentoo-guis, #porthole, Blog
layman, gentoolkit, CoreBuilder, esearch...
Back to top
View user's profile Send private message
mwhitlock
Tux's lil' helper
Tux's lil' helper


Joined: 07 Aug 2006
Posts: 87
Location: New Hampshire, USA

PostPosted: Tue Apr 05, 2016 7:38 am    Post subject: redundant-use.sh Reply with quote

So, I took the time to write a script for finding redundant USE flags in /etc/portage/make.conf and /etc/portage/package.use/*. I think it's fairly comprehensive, but I make no warranties.

Gist: redundant-use.sh

Features:
  • Considers globally profile-forced USE flags (<profile>/use.force, <profile>/use.stable.force)
  • Considers globally profile-masked USE flags (<profile>/use.mask, <profile>/use.stable.mask)
  • Considers per-package profile-forced USE flags (<profile>/package.use.force, <profile>/package.use.stable.force)
  • Considers per-package profile-masked USE flags (<profile>/package.use.mask, <profile>/package.use.stable.mask)
  • Supports profile overrides (/etc/portage/profile extends /etc/portage/make.profile)
  • Observes ${PORTAGE_CONFIGROOT} environment variable
  • Detects redundant and defunct global USE flags in config files (/etc/portage/make.globals, /etc/portage/make.conf)
  • Detects redundant and defunct per-package USE flags in config files (/etc/portage/package.use/*)
  • Understands version scoping (>=, >, ~, =, <=, <)
  • Understands slot/subslot scoping
  • Understands repository scoping
Sample /etc/portage/make.conf (excerpt):
Code:
USE="apparmor multilib -nls -selinux tcmalloc"

Sample /etc/portage/package.use/demo:
Code:
app-shells/bash:0 -nls readline
media-libs/phonon -zeitgeist
>=media-libs/phonon-4 -zeitgeist
sys-block/noflushd toilet
<sys-block/noflushd-2.7.3 toilet
www-client/chromium tcmalloc

Sample redundant-use.sh output:
Code:
/etc/portage/make.conf:
  redundant USE="multilib" (profile default)
  defunct USE="apparmor" (masked by profile)
  redundant USE="-selinux" (masked by profile)
/etc/portage/package.use/demo:
  [app-shells/bash:0] redundant USE="-nls" (disabled globally)
  [app-shells/bash:0] redundant USE="readline" (profile default)
  [app-shells/bash:0] redundant USE="readline" (forced by profile)
  [media-libs/phonon] redundant USE="-zeitgeist" (masked by profile)
  [>=media-libs/phonon-4] redundant USE="-zeitgeist" (already disabled by this file)
  [<sys-block/noflushd-2.7.3] redundant USE="toilet" (already enabled by this file)
  [www-client/chromium] redundant USE="tcmalloc" (enabled globally)

What it does not do:
  • It does not scan /var/db/pkg or any configured ebuild repositories, so it cannot consider whether configured USE flags actually affect packages. This implies that there is no checking for obsolete/inapplicable USE flags. This also implies that USE flags disabled on a per-package basis in /etc/portage/package.use are not reported as redundant unless the global configuration already disables them, since individual ebuilds may enable specific USE flags by default. For the same reason, USE flags enabled on a per-package basis in /etc/portage/package.use are not reported as redundant unless they are already enabled by the profile or the global configuration.


Last edited by mwhitlock on Tue Apr 05, 2016 6:39 pm; edited 1 time in total
Back to top
View user's profile Send private message
Juippisi
Developer
Developer


Joined: 30 Sep 2005
Posts: 722
Location: /home

PostPosted: Tue Apr 05, 2016 12:40 pm    Post subject: Reply with quote

I just ran redundant-use.sh, and it gave me this:

Code:

# sh redundant-use.sh
/etc/portage/make.conf:
  redundant USE="unicode" (profile default)
  redundant USE="flac" (profile default)
  redundant USE="ogg" (profile default)
  redundant USE="svg" (profile default)
  redundant USE="mp4" (profile default)
  redundant USE="mp3" (profile default)
  redundant USE="X" (profile default)
  redundant USE="dbus" (profile default)
  defunct USE="svga" (masked by profile)
  redundant USE="mad" (profile default)
  redundant USE="udev" (profile default)
  redundant USE="sse" (profile default)
  redundant USE="libnotify" (profile default)
  redundant USE="usb" (profile default)
  redundant USE="mmx" (profile default)
  redundant USE="vorbis" (profile default)
  redundant USE="consolekit" (profile default)
  redundant USE="policykit" (profile default)
  redundant USE="policykit" (forced by profile)
  redundant USE="pdf" (profile default)
  redundant USE="jpeg" (profile default)
  redundant USE="alsa" (profile default)
  redundant USE="x264" (profile default)
  redundant USE="png" (profile default)
/etc/portage/package.use:
  [app-text/ghostscript-gpl] redundant USE="cups" (profile default)
  [dev-libs/quazip] redundant USE="qt5" (profile default)
  [app-office/libreoffice] defunct USE="python_single_target_python3_5" (masked by profile)
  [dev-lang/php] redundant USE="cli" (profile default)
  [gnome-base/gvfs] redundant USE="udev" (enabled globally)
  [gnome-base/gvfs] redundant USE="udev" (profile default)
  [gnome-base/gvfs] redundant USE="udisks" (profile default)
redundant-use.sh: line 253: 10bit: value too great for base (error token is "10bit")


so lots to clean, thanks for the script! Not sure what that 10bit error is though

Code:

# grep -i 10bit /etc/portage/package.use
media-libs/x264 10bit
media-libs/x265 10bit 12bit
Back to top
View user's profile Send private message
charles17
Advocate
Advocate


Joined: 02 Mar 2008
Posts: 3664

PostPosted: Tue Apr 05, 2016 1:52 pm    Post subject: Reply with quote

Try
Code:
qgrep 10bit media-libs/x265
and see it not for all versions. For 1.8-r3 it does not apply.
Back to top
View user's profile Send private message
mwhitlock
Tux's lil' helper
Tux's lil' helper


Joined: 07 Aug 2006
Posts: 87
Location: New Hampshire, USA

PostPosted: Tue Apr 05, 2016 6:15 pm    Post subject: Reply with quote

Juippisi wrote:
Not sure what that 10bit error is though

@Juippisi: Thanks for posting your results. I've revised my gist with a correction that should solve the problem for you.

charles17 wrote:
and see it not for all versions. For 1.8-r3 it does not apply.

@charles17: That doesn't matter to my script, as it doesn't actually examine any ebuilds. You could have totally bogus USE flags in your package.use, and my script would only report them if they're redundant. If you want to find inapplicable USE flags, run the script posted earlier in this thread.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum