Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Reverse dependencies & unneeded packages
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
aborder
n00b
n00b


Joined: 09 Sep 2003
Posts: 13

PostPosted: Sat Sep 13, 2003 8:28 am    Post subject: Reverse dependencies & unneeded packages Reply with quote

If Portage is a great tool to install packages, I think there is still room for improvement with respect to uninstallation.
Consider the following scenario. You want to install a package foo, so you run:
Code:
# ebuild -p foo
[ebuild  N    ] sys-apps/libbar-1.0.0
[ebuild     U ] sys-apps/foo-2.0.0 [1.0.0]

This shows that foo depends on libbar; if you emerge it, you'll end up installing two different packages on your system. Fortunately Portage makes all this automatic.
A week later you realise you don't need foo anymore.
Code:
emerge unmerge foo

and you get rid of it. But wait! What about libbar? The short answer is that it remains on your system even if no other package depends on it. Do this install / uninstall cycle for a few times and the number of "lost" packages becomes annoying.
Right now two utils come to your help. The first is emerge itself, more precisely "emerge depclean". Quoting the man page:
Quote:
depclean
Determines all packages installed on the system that have no
explicit reason for being there. emerge generates a list of
packages which it expects to be installed by checking the system
package list and the world file. It then compares that list to

the list of packages which are actually installed; the differ-
ences are listed as unnecessary packages and are suggested for
unmerging. WARNING: Removing some packages may cause packages
which link to the removed package to stop working and complain
about missing libraries. Re-emerge the complaining package to
fix this issue. Note that changes in USE flags can drastically
affect the output of depclean.

From my point of view this approach has one limit: emerge considers USE flags as they are defined when you run it, not as they where when you compiled each package.
The other useful tool is qpkg. Run:
Code:
qpkg -I -q foo

and you'll get the list of all installed packages depending on foo. qpkg is even more limited than emerge, since it only checks one package at a time and completely ignores USE flags.

Since none of these tools completely satisfied me, I decided to write my own. The result is the the following pkdep Python script. It seems to be more aggressive than emerge, so be careful and think twice before removing anything.
Code:
#!/usr/bin/env python
# Copyright 2003-2004 Anakim Border <aborder@users.sourceforge.net>
# This software is released under the terms of the GPL licence.

# Python 2.3+ required

from sets import Set
import os
import re

# Global vars
pkgdir     = '/var/db/pkg'
pkguse     = 'USE'
pkgprovide = 'PROVIDE'
pkgdepend  = 'RDEPEND'
versionex  = r'([0-9]+)(?:\.([0-9]+)(?:\.([0-9]+))?)?([a-z]?)(?:_(alpha|beta|pre|rc)([0-9]*))?(?:-r([0-9]+))?\*?'

# Functions
def versionMatch(dep, found):
   required = dep.package.version
   op = dep.versionFlags
   flags = [None, 'alpha', 'beta', 'pre', 'rc']
   
   try:
      subverMatch(found.major, op, required.major)
      subverMatch(found.middle, op, required.middle)
      subverMatch(found.minor, op, required.minor)
      subverMatch(found.letter, op, required.letter)
      subverMatch(flags.index(found.flag), op, flags.index(required.flag))
      subverMatch(found.flagnum, op, required.flagnum)
      subverMatch(found.release, op, required.release)
      return True
      
   except VersionMatch:
      return True
   except VersionMismatch:
      return False
   
def subverMatch(x, op, y):
   if x is None and y is None:
      return
   
   if op & PackageDep.VERSION_LT:
      if x is None:
         raise VersionMatch
      elif y is None:
         raise VersionMismatch
      elif x < y:
         raise VersionMatch
   
   elif op & PackageDep.VERSION_GT != 0:
      if x is None:
         raise VersionMismatch
      elif y is None:
         raise VersionMatch
      elif x > y:
         raise VersionMatch
      
   if op & PackageDep.VERSION_EQ != 0:
      if x is None or y is None:
         return
      elif x == y:
         if op & PackageDep.VERSION_NOT != 0:
            raise VersionMismatch
         else:
            return

   raise VersionMismatch
   
class VersionMatch(Exception): pass
class VersionMismatch(Exception): pass

def versionCmp(pkgA, pkgB):
   flags = [None, 'alpha', 'beta', 'pre', 'rc']
   a, b = pkgA.version, pkgB.version
   
   try:
      subverCmp(a.major, b.major)
      subverCmp(a.middle, b.middle)
      subverCmp(a.minor, b.minor)
      subverCmp(a.letter, b.letter)
      subverCmp(flags.index(a.flag), flags.index(b.flag))
      subverCmp(a.flagnum, b.flagnum)
      subverCmp(a.release, b.release)
      return 0
   
   except VersionCompared, e:
      return e.args[0]

def subverCmp(a, b):
   res = cmp(a, b)
   if res != 0:
      raise VersionCompared, res
   
class VersionCompared(Exception): pass

# Package classes
class Package(object):
   def __init__(self, name):
      self.name, self.version = self.parseName(name)

   def parseName(self, name):
      pos = 0
      while True:
         pos = name.find('-', pos+1)
         if pos < 0: return name, None

         if name[pos+1].isdigit():
            return name[:pos], PackageVersion(name[pos+1:])

class PackageVersion(object):
   __slots__ = ('major', 'middle', 'minor', 'letter', 'flag', 'flagnum', 'release')
   parser = re.compile(versionex)

   def __init__(self, version = None):
      for attr in self.__slots__:
         self.__setattr__(attr, None)

      if version:
         try:
            info = self.parser.findall(version)[0]

            self.major = int(info[0])
            if len(info[1]) > 0: self.middle = int(info[1])
            if len(info[2]) > 0: self.minor = int(info[2])
            if len(info[3]) > 0: self.letter = info[3]
            if len(info[4]) > 0: self.flag = info[4]
            if len(info[5]) > 0: self.flagnum = int(info[5])
            if len(info[6]) > 0: self.release = int(info[6])
         except IndexError:
            raise ValueError

   def __repr__(self):
      return '<PackageVersion: %s %s %s %s %s %s %s>' % (self.major, self.middle, self.minor, self.letter, self.flag, self.flagnum, self.release)
      
   def __str__(self):
      res = []
      if self.major is not None:
         res.append(str(self.major))
      if self.middle is not None:
         res.append('.%d' % self.middle)
      if self.minor is not None:
         res.append('.%d' % self.minor)
      if self.letter is not None:
         res.append(self.letter)
      if self.flag is not None:
         res.append('_%s' % self.flag)
      if self.flagnum is not None:
         res.append(str(self.flagnum))
      if self.release is not None:
         res.append('-r%d' % self.release)
      
      return ''.join(res)

class PackageDep(object):
   VERSION_NONE = 0
   VERSION_NOT  = 1
   VERSION_EQ   = 2
   VERSION_LT   = 4
   VERSION_GT   = 8

   def __init__(self):
      self.package = None
      self.versionFlags = self.VERSION_NONE

class PackageDeps(Package):
   def __init__(self, name, basedir):
      super(PackageDeps, self).__init__(name)

      self.useflags = self.parseUseFlags(os.path.join(basedir, pkguse))
      self.aliases = self.parseProvide(os.path.join(basedir, pkgprovide))
      self.deps     = self.parseDeps(os.path.join(basedir, pkgdepend))
      
   def parseUseFlags(self, usefile):
      # ASSUMPTION: USE declaration is on a single row
      useline = ' '.join([ l.strip() for l in file(usefile, 'r') ])

      flags = {}
      for flag in useline.split(' '):
         if flag[0] == '-':
            flags[flag[1:]] = False
         else:
            flags[flag] = True

      return flags

   def parseProvide(self, providefile):
      provides = Set()
      for line in file(providefile, 'r'):
         line = line.strip()
         if len(line) == 0: continue

         p = Package(line)
         provides.add(p.name)
      
      return provides

   def parseDeps(self, filename):
      deps = {}

      for item in self.getDepItem(filename):
         idx = item.find('?')
         if idx >= 0:
            flag = item[:idx]
            if flag[0] == '!':
               flag = flag[1:]
               if self.useflags.has_key(flag) and self.useflags[flag]:
                  continue
            else:
               if not self.useflags.has_key(flag) or not self.useflags[flag]:
                  continue

            item = item[idx+1:].strip()
            if len(item) == 0: continue

         d = PackageDep()
         while item[0] in ('!', '~', '=', '<', '>'):
            if item[0] == '!':
               d.versionFlags |= PackageDep.VERSION_NOT
            if item[0] == '=':
               d.versionFlags |= PackageDep.VERSION_EQ
            elif item[0] == '<':
               d.versionFlags |= PackageDep.VERSION_LT
            elif item[0] == '>':
               d.versionFlags |= PackageDep.VERSION_GT
            elif item[0] == '~':
               pass    # Ignore it

            item = item[1:]

         if d.versionFlags == d.VERSION_NOT :  # '!package' or '!=package'
            continue

         if item[-1] == '*':
            item = item[:-1]
      
         d.package = Package(item)
         deps[d.package.name] = d

      return deps

   def getDepItem(self, filename):
      buf = ''
      for line in file(filename, 'r'):
         buf = ('%s %s' % (buf, line)).lstrip()
         
         while len(buf) > 0:
            idx = self.findWhitespace(buf)
            if idx == -1:
               yield buf
               buf = ''
               break
            
            if buf[idx-1] == '?':
               end = buf.find(')')
               if end == -1: break
               
               dep = buf[:idx]
               start = buf.find('(')
               depbuf = buf[start+1:end].lstrip()
               buf = buf[end+1:].lstrip()
               
               while len(depbuf) > 0:
                  idx = self.findWhitespace(depbuf)
                  if idx == -1:
                     yield '%s %s' % (dep, depbuf)
                     break
                  else:
                     yield '%s %s' % (dep, depbuf[:idx])
                     depbuf = depbuf[idx+1:].lstrip()

            else:
               yield buf[:idx]
               buf = buf[idx+1:].lstrip()

   def findWhitespace(self, data):
      for idx in range(0, len(data)):
         if data[idx] in (' ', '\t', '\n'):
            return idx
      return -1

# Main entry point
if __name__ == '__main__':
   packages = {}
   aliases  = {}
   deps     = []
   cuts = Set()
   
   # Scan pkgdir and build package list
   for category in os.listdir(pkgdir):
      dirname = os.path.join(pkgdir, category)
      if not os.path.isdir(dirname):
         continue

      for pkg in os.listdir(dirname):
         try:
            pd = PackageDeps(os.path.join(category, pkg), os.path.join(dirname, pkg))
   
            packages.setdefault(pd.name, []).append(pd)
            for alias in pd.aliases:
               aliases.setdefault(pd.name, []).append(pd)
            deps.append(pd)
         except IOError:
            pass

   for pd in deps:
      # Scan package dependencies
      for name, dep in pd.deps.iteritems():
         try:
            if packages.has_key(name):
               if dep.package.version != None:
                  matches = [ r for r in packages[name] if versionMatch(dep, r.version) ]
               else:
                  matches = packages[name]
               
               if len(matches) > 0:
                  if len(matches) > 1:
                     # All dependencies on the highest-numbered release
                     matches.sort(versionCmp)
                     cuts.add( (name, matches[-1]) )
                  else:
                     cuts.add( (name, matches[0]) )
                     
            elif aliases.has_key(name):
               for pkg in aliases[name]:
                  if len(packages[pkg]) > 1:
                     # All dependencies on the highest-numbered release
                     packages[pkg].sort(versionCmp)
                     cuts.add( (pkg, packages[pkg][-1]) )
                  else:
                     cuts.add( (pkg, packages[pkg][0]) )
         
         except KeyError:
            pass
            
   for name, release in cuts:
      packages[name].remove(release)
      if len(packages[name]) == 0:
         del packages[name]

   if len(packages) > 0:
      print 'The following packages are not depended upon by anything else:'
      print

      pkgs = packages.keys()
      pkgs.sort()
      for name in pkgs:
         for release in packages[name]:
            print '\t%s-%s' % (name, release.version)

If you give it a try and find a bug, please let me know :)

Edit #1
I probably found the code responsible for the error reported by Ox-. I've changed both versionMatch and PackageDeps.getDepItem functions.

Edit #2
I've changed pkdep to make it work on systems having multiple releases of the same package installed (for example: Berkeley DB). The output now lists both package names and their versions.


Last edited by aborder on Thu Jul 08, 2004 11:08 am; edited 2 times in total
Back to top
View user's profile Send private message
Ox-
Guru
Guru


Joined: 19 Jun 2003
Posts: 305

PostPosted: Sat Sep 13, 2003 10:23 am    Post subject: Reply with quote

Awesome, I was thinking of writing a dependency tracking tool, but don't have any time atm.

I tested your program and it listed 220 of the 630 packages I have installed. A few had been merged from the command line, but most were legitimate dependencies :( I'll see if I can find the problem.
Back to top
View user's profile Send private message
aborder
n00b
n00b


Joined: 09 Sep 2003
Posts: 13

PostPosted: Sat Sep 13, 2003 11:24 am    Post subject: Reply with quote

Ox- wrote:
I tested your program and it listed 220 of the 630 packages I have installed. A few had been merged from the command line, but most were legitimate dependencies :( I'll see if I can find the problem.


Did you try to run "qpkg -I -q package" on some of the packages incorrectly reported as unneeded?
Back to top
View user's profile Send private message
Ox-
Guru
Guru


Joined: 19 Jun 2003
Posts: 305

PostPosted: Sat Sep 13, 2003 12:28 pm    Post subject: Reply with quote

Well I went back and looked closer, and all 220 are in the world file :)

I'm pretty sure I don't have any unneeded packages atm for a couple reasons. One, I have tons of disk space and rarely unmerge stuff, and two because I've been doing some study of the portage dependency system lately by running qpkg -q, etcat depends, emerge --dep-clean, --dups, dep-clean -CRUN, etc.

Just to check, I did pick one from the list, and qpkg -q showed a dependency.

I'm wondering if I might have edited something incorrectly from your script.. because I couldn't just cut+paste into editor window. I had to do Save Page As... and delete the HTML. (For some reason when I cut from mozilla into a console window running emacs it freaks out python-mode).
Back to top
View user's profile Send private message
aborder
n00b
n00b


Joined: 09 Sep 2003
Posts: 13

PostPosted: Sat Sep 13, 2003 2:03 pm    Post subject: Reply with quote

Ox- wrote:
Just to check, I did pick one from the list, and qpkg -q showed a dependency.


"qpkg -q" runs its checks against all packages, no matter if they're installed on not; my script just checks the installed ones.
Another difference is that qpkg completely discards the USE flags. Doing so it assumes that all flags are specified: this greatly increase the number of package interdependencies.

Could you please tell me the name of a package erroneously reported by my script and the ouput of "qpkg -I -q package-name"?
Using that piece of info I can run some more tests...
Back to top
View user's profile Send private message
Ox-
Guru
Guru


Joined: 19 Jun 2003
Posts: 305

PostPosted: Sat Sep 13, 2003 3:19 pm    Post subject: Reply with quote

To be more specific, I did qpkg -I -q...

One of the ones reported by your program was x11-themes/gtk-engines-metal. It is both in the world file and depended on by gtk-themes. I have both slot 1 and 2 installed (of both gtk-engines-... and gtk-themes).

I'm travelling for a few weeks, but I'll help out some more when I return.

Here's some other stats for ya:
Code:
# etcat -v gtk-engines-metal
[ Results for search key : gtk-engines-metal ]
[ Applications found : 1 ]
 
*  x11-themes/gtk-engines-metal :
        [  I] x11-themes/gtk-engines-metal-0.12 (1)
        [   ] x11-themes/gtk-engines-metal-1.9.0 (2)
        [  I] x11-themes/gtk-engines-metal-2.2.0 (2)
 
# etcat -v gtk-themes
[ Results for search key : gtk-themes ]
[ Applications found : 1 ]
 
*  x11-themes/gtk-themes :
        [  I] x11-themes/gtk-themes-1.0 (1)
        [M~ ] x11-themes/gtk-themes-1.0-r1 (1)
        [   ] x11-themes/gtk-themes-2.0 (2)
        [M~ ] x11-themes/gtk-themes-2.0-r1 (2)
        [  I] x11-themes/gtk-themes-2.0-r2 (2)
 
# qpkg -I | wc -l
    630

# emerge -pve world | wc -l
    645

# wc -l /var/cache/edb/world
    531 /var/cache/edb/world

# grep engines-metal /var/cache/edb/world
x11-themes/gtk-engines-metal

# qpkg -I -q x11-themes/gtk-engines-metal
x11-themes/gtk-engines-metal-0.12 *
DEPENDED ON BY:
        x11-themes/gtk-themes-1.0
        x11-themes/gtk-themes-2.0-r2
x11-themes/gtk-engines-metal-2.2.0 *
DEPENDED ON BY:
        x11-themes/gtk-themes-1.0
        x11-themes/gtk-themes-2.0-r2
Back to top
View user's profile Send private message
aborder
n00b
n00b


Joined: 09 Sep 2003
Posts: 13

PostPosted: Sun Sep 14, 2003 8:33 am    Post subject: Reply with quote

I think I've found the cause of the error inside the versionMatch function.
I've edited the original post to fix it.
Back to top
View user's profile Send private message
hensan
l33t
l33t


Joined: 26 Jun 2003
Posts: 868
Location: Sweden

PostPosted: Sun Sep 14, 2003 10:20 am    Post subject: Reply with quote

I like the idea behind this script so I tried it out.

What I noticed was the same thing Ox- mentioned, that the script suggests that I remove half the files in my world file.
Back to top
View user's profile Send private message
aborder
n00b
n00b


Joined: 09 Sep 2003
Posts: 13

PostPosted: Sun Sep 14, 2003 12:11 pm    Post subject: On the world file Reply with quote

hensan wrote:
I like the idea behind this script so I tried it out.

What I noticed was the same thing Ox- mentioned, that the script suggests that I remove half the files in my world file.

This is not by itself a problem; in fact pkdep (my script) does nothing to check the contents of the world file. Since this seems to be an obscure point, let me explain it using an example.
Suppose you want to install the gtk-themes package as reported by Ox-. You run:
Code:
# emerge gtk-themes -p

These are the packages that I would merge, in order:

Calculating dependencies ...done!
[ebuild  N    ] x11-themes/gtk-engines-metal-2.2.0
[ebuild  N    ] x11-themes/gtk-engines-thinice-2.0.2
[ebuild  N    ] x11-themes/gtk-engines-mist-0.8-r2
[ebuild  N    ] x11-themes/gtk-engines-mgicchikn-1.0.0
[ebuild  N    ] x11-themes/gtk-engines-xfce-2.1.1
[ebuild  N    ] x11-themes/gtk-engines-redmond95-2.2.0
[ebuild  N    ] x11-themes/gtk-engines-pixbuf-2.2.0
[ebuild  N    ] x11-themes/gtk-engines-cleanice-1.1.5
[ebuild  N    ] x11-themes/gtk-engines-lighthouseblue-0.4.2
[ebuild  N    ] x11-themes/gtk-engines-flat-2.0
[ebuild  N    ] x11-themes/gtk-themes-2.0-r2

A lot of packages; you may only want to install the metal theme. Running
Code:
# emerge gtk-themes --nodeps -p

These are the packages that I would merge, in order:

Calculating dependencies ...done!
[ebuild  N    ] x11-themes/gtk-themes-2.0-r2

is not the way to go, since it won't install any theme at all.
A solution to the problem is to manually install gtk-engines-metal, then to emerge gtk-themes using the --nodeps switch.
Where is the problem? If you look at the output of the first emerge you'll find something like:
Code:
# emerge gtk-engines-metal
...
>>> x11-themes/gtk-engines-metal-2.2.0 merged.
>>> Recording x11-themes/gtk-engines-metal in "world" favorites file...
...

gtk-engines-metal is recorded in the world file; this means that if you ever unmerge gtk-themes, "emerge depclean" won't give you a notice about the fact that gtk-engines-metal is removable.
Though a little bit extreme, this is the kind of situation that pkdep has been designed to solve and that's why it doesn't check the world file.
Back to top
View user's profile Send private message
xentric
Guru
Guru


Joined: 16 Mar 2003
Posts: 410
Location: Netherlands

PostPosted: Sun Sep 28, 2003 11:27 am    Post subject: Reply with quote

I have a problem running your script:

I just cut&pasted the script above into /usr/local/bin/pkgscan
ran chmod 755 on the file to make it executable
But it produces the following errors:
Code:
root@gentoo xentric $ pkgscan
Traceback (most recent call last):
  File "/usr/local/bin/pkgscan", line 81, in ?
    class PackageVersion:
  File "/usr/local/bin/pkgscan", line 82, in PackageVersion
    parser = re.compile(versionex)
  File "/usr/lib/python2.2/sre.py", line 179, in compile
    return _compile(pattern, flags)
  File "/usr/lib/python2.2/sre.py", line 229, in _compile
    raise error, v # invalid expression
sre_constants.error: bad character range
root@gentoo xentric $
Back to top
View user's profile Send private message
aborder
n00b
n00b


Joined: 09 Sep 2003
Posts: 13

PostPosted: Sun Sep 28, 2003 12:06 pm    Post subject: Reply with quote

xentric wrote:
Code:
root@gentoo xentric $ pkgscan
Traceback (most recent call last):
  File "/usr/local/bin/pkgscan", line 81, in ?
    class PackageVersion:
  File "/usr/local/bin/pkgscan", line 82, in PackageVersion
    parser = re.compile(versionex)
  File "/usr/lib/python2.2/sre.py", line 179, in compile
    return _compile(pattern, flags)
  File "/usr/lib/python2.2/sre.py", line 229, in _compile
    raise error, v # invalid expression
sre_constants.error: bad character range


Looking at the Traceback, it seems that Python is having problems compiling the regular expression defined by the versionex var (you can find it at line 17).
Since versionex definition is quite long, it is possible that your browser (or text editor) applied on it some unwanted word-wrapping. Please try replacing line 17 with the following:
Code:
versionex  = r'([0-9]+)(?:\.([0-9]+)(?:\.([0-9]+))?)?([a-z]?)'
versionex += r'(?:_(alpha|beta|pre|rc)([0-9]*))?(?:-r([0-9]+))?\*?'

After that, head -n20 pkgscan should give you:
Code:
#!/usr/bin/env python
#
# Copyright 2003 Anakim Border <aborder@users.sourceforge.net>
#
# This software is released under the terms of the GPL licence.
#

from __future__ import generators
import os
import re

# Global vars
pkgdir     = '/var/db/pkg'
pkguse     = 'USE'
pkgprovide = 'PROVIDE'
pkgdepend  = 'RDEPEND'
versionex  = r'([0-9]+)(?:\.([0-9]+)(?:\.([0-9]+))?)?([a-z]?)'
versionex += r'(?:_(alpha|beta|pre|rc)([0-9]*))?(?:-r([0-9]+))?\*?'

# Functions

and the script should work.
Back to top
View user's profile Send private message
xentric
Guru
Guru


Joined: 16 Mar 2003
Posts: 410
Location: Netherlands

PostPosted: Sun Sep 28, 2003 10:33 pm    Post subject: Reply with quote

Yay... that worked, (Opera must have borked the script) but the next problem is this:
Code:
root@gentoo root $ pkdep
: No such file or directory
root@gentoo root $

It doesn't do a thing :(
The /var/db/pkg directory is there, what other files/dirs does it look for?
Back to top
View user's profile Send private message
aborder
n00b
n00b


Joined: 09 Sep 2003
Posts: 13

PostPosted: Wed Oct 01, 2003 7:11 am    Post subject: Reply with quote

xentric wrote:
It doesn't do a thing :(
The /var/db/pkg directory is there, what other files/dirs does it look for?

The script recursively walks the /var/db/pkg directory searching for files describing the packages installed on your Gentoo box. The "interesting" files for each package are: USE, PROVIDE and RDEPEND.
The default behaviour of the script is to continue the scan even if it can't find something; if package foo lacks the USE file (this is a common situation, at least on my system), pkgscan will ignore it and continue its work.
Since this strategy doesn't seem to work on your box, I suggest to change pkgscan to make it report errors with greater detail. Around line 245 you'll find the following:
Code:
      for pkg in os.listdir(dirname):
         try:
            pd = PackageDeps(os.path.join(category, pkg), os.path.join(dirname, pkg))

            packages[pd.name] = pd
            for alias in pd.aliases:
               aliases[alias] = pd.name
            deps.append(pd)
         except IOError:
            pass
         except Exception, e:
            print e

Change it to:
Code:
      for pkg in os.listdir(dirname):
         try:
            pd = PackageDeps(os.path.join(category, pkg), os.path.join(dirname, pkg))

            packages[pd.name] = pd
            for alias in pd.aliases:
               aliases[alias] = pd.name
            deps.append(pd)
         except IOError:
            pass
         #except Exception, e:
         #   print e

Then run pkgscan and search the output for a traceback (like the one you reported in your previous message).
If that's not enough, try with:
Code:
      for pkg in os.listdir(dirname):
         pd = PackageDeps(os.path.join(category, pkg), os.path.join(dirname, pkg))

         packages[pd.name] = pd
         for alias in pd.aliases:
            aliases[alias] = pd.name
         deps.append(pd)

but consider this as a last option, since it will probably report too many errors.
Back to top
View user's profile Send private message
xentric
Guru
Guru


Joined: 16 Mar 2003
Posts: 410
Location: Netherlands

PostPosted: Wed Oct 01, 2003 3:12 pm    Post subject: Reply with quote

Quote:

Change it to:
Code:
      for pkg in os.listdir(dirname):
         try:
            pd = PackageDeps(os.path.join(category, pkg), os.path.join(dirname, pkg))

            packages[pd.name] = pd
            for alias in pd.aliases:
               aliases[alias] = pd.name
            deps.append(pd)
         except IOError:
            pass
         #except Exception, e:
         #   print e

Then run pkgscan and search the output for a traceback (like the one you reported in your previous message).

After doing this the script started working, and when I removed the #'s I just put in,
it still worked. Damn... I can't explain why it wouldn't work before but thanks for
your help, it works now. Great script, thanks !
Back to top
View user's profile Send private message
Target
Apprentice
Apprentice


Joined: 25 Apr 2002
Posts: 200

PostPosted: Mon Feb 09, 2004 8:01 pm    Post subject: Reply with quote

Uhh... You should probably give greater emphasis to the "be careful what you remove" disclaimer.

Code:
You don't need the following packages:
...
sys-devel/ld.so
...
Back to top
View user's profile Send private message
aborder
n00b
n00b


Joined: 09 Sep 2003
Posts: 13

PostPosted: Thu Jul 08, 2004 11:21 am    Post subject: Reply with quote

Target wrote:
Code:
You don't need the following packages:
...
sys-devel/ld.so
...

I'm aware of that issue; I'll fix it as soon as I'll introduce system profile support.
Working on it...
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9013
Location: beyond the rim

PostPosted: Thu Jul 08, 2004 4:04 pm    Post subject: Reply with quote

That package isn't even in the tree anymore, wonder how you got it installed in the first place.
Back to top
View user's profile Send private message
aborder
n00b
n00b


Joined: 09 Sep 2003
Posts: 13

PostPosted: Fri Jul 09, 2004 6:45 am    Post subject: Reply with quote

Genone wrote:
That package isn't even in the tree anymore, wonder how you got it installed in the first place.

Code:
# epm -qi ld.so
Name        : ld.so
Version     : 1.9.11
Release     : 2                                   Slot: 0
Install date: Wed Sep 18 21:23:47 2002      Build Host: i686-pc-linux-gnu

As you can see my install dates back to 2002.
If I may suggest a feature to add to Portage, here it is: display a big red label when emerge finds an installed package whose ebuild can't be found neither in /usr/portage nor in PORTDIR_OVERLAY. :twisted:
Back to top
View user's profile Send private message
hepta_sean
Apprentice
Apprentice


Joined: 27 Apr 2004
Posts: 246
Location: Berlin, Germany

PostPosted: Fri Jul 09, 2004 8:29 am    Post subject: Reply with quote

aborder wrote:
If I may suggest a feature to add to Portage, here it is: display a big red label when emerge finds an installed package whose ebuild can't be found neither in /usr/portage nor in PORTDIR_OVERLAY. :twisted:


If you're interested I'm using a script, whose prime objective is to find ebuilds, which changed significantly without a version bump and to just overwrite the ebuild in /var/db/pkg if only the KEYWORDS changed, but as a side effect it also outputs all installed packages, whose ebuild in the tree is gone:
Code:
#!/bin/bash

export PKGDB="/var/db/pkg/"

cd ${PKGDB}
for pkg in `find ./ | grep ebuild$ | xargs -n1 -- dirname | sed 's/^\.\///g'`
do
        CATEGORY=`echo ${pkg} | sed 's/\/.*//g'`
        APP=`echo ${pkg} | sed -e 's/.*\///g' -e 's/-[0-9].*//g'`
        VERSION=`echo ${pkg} | sed "s/${CATEGORY}\/${APP}-//g"`
        EBUILD=$(bunzip2 -c ${PKGDB}/${pkg}/environment.bz2 | sed 's/^EBUILD=\(.*\)/\1/;t;d')
        INSTALLED=${PKGDB}/${pkg}/${APP}-${VERSION}.ebuild

        if [ -f ${EBUILD} ]
        then
                if ! diff -q -b ${EBUILD} ${INSTALLED} >/dev/null
                then
                        if diff -q -b -I ^# -I ^KEYWORDS ${EBUILD} ${INSTALLED} >/dev/null
                        then
                                echo Only insignificant changes for ${pkg}
                        else
                                echo Significant changes - consider to remerge ${pkg}
                        fi
                fi
        else
                echo Ebuild is gone - please repair ${pkg}
        fi
done

The original idea of this script is borrowed from revcheck.sh written by ecatmur.

Also you may find this thread interesting, which treats a tool quite similar to yours, although with a somehow different focus. Perhaps you can borrow some ideas or so.
Back to top
View user's profile Send private message
herlock
Tux's lil' helper
Tux's lil' helper


Joined: 30 Aug 2003
Posts: 141
Location: Bruxelles

PostPosted: Sat Aug 21, 2004 9:54 pm    Post subject: Reply with quote

This look like great.. but I have a question.

You emerged foo and then libbar is emerged to as depency; okay.
Three days after you want to emerge foo2 with libbar as depency but libbar is emerged yet.

One week after you want to unmerged foo but you want keep foo2.

Does the script unmerge libbar anyway ?


PS: I hoop you understand me, because I have a very bad english :/ sorry guy
Back to top
View user's profile Send private message
aborder
n00b
n00b


Joined: 09 Sep 2003
Posts: 13

PostPosted: Sun Aug 22, 2004 11:58 am    Post subject: Reply with quote

herlock wrote:
One week after you want to unmerged foo but you want keep foo2.

Does the script unmerge libbar anyway ?

No, it doesn't.
Each time pkdep runs, it rebuilds the dependency list from scratch. In the case of your example pkdep notices that foo2 depends on libbar, so it considers that library "in use".
Back to top
View user's profile Send private message
eNTi
Veteran
Veteran


Joined: 20 Oct 2002
Posts: 1011
Location: Salzburg, Austria

PostPosted: Sun Aug 22, 2004 6:47 pm    Post subject: Reply with quote

is it possible to get the script via a link? it's kinda inconvenient to copy/paste the whole thing.

i've tried it out and:
Code:

root@eNTi # ./pkdep apache
Traceback (most recent call last):
  File "./pkdep", line 297, in ?
    pd = PackageDeps(os.path.join(category, pkg), os.path.join(dirname, pkg))
  File "./pkdep", line 170, in __init__
    self.useflags = self.parseUseFlags(os.path.join(basedir, pkguse))
  File "./pkdep", line 180, in parseUseFlags
    if flag[0] == '-':
IndexError: string index out of range

_________________
If you fall off a cliff, you might as well try to fly. After all, you got nothing to lose.

-- John Sheridan - Babylon 5, Season 4
Back to top
View user's profile Send private message
duderonomy
Guru
Guru


Joined: 20 Mar 2004
Posts: 340
Location: SF Bay Area

PostPosted: Mon Aug 23, 2004 2:57 am    Post subject: Reply with quote

pkdep is available here:
http://www.mediaweb.com/gentoo/scripts/pkdep/

Cheers,
D
Back to top
View user's profile Send private message
BearingSpacer
n00b
n00b


Joined: 17 Feb 2003
Posts: 19

PostPosted: Tue Aug 24, 2004 6:36 pm    Post subject: Reply with quote

Hi !

no luck, I have the same error, even grabbing the script from the link :(
Code:
Traceback (most recent call last):
  File "./pkdep", line 291, in ?
    pd = PackageDeps(os.path.join(category, pkg), os.path.join(dirname, pkg))
  File "./pkdep", line 172, in __init__
    self.useflags = self.parseUseFlags(os.path.join(basedir, pkguse))
  File "./pkdep", line 182, in parseUseFlags
    if flag[0] == '-':
IndexError: string index out of range


any idea ?
python is still like chinese to me :(
Back to top
View user's profile Send private message
aborder
n00b
n00b


Joined: 09 Sep 2003
Posts: 13

PostPosted: Thu Aug 26, 2004 7:03 am    Post subject: Reply with quote

BearingSpacer wrote:
Code:
IndexError: string index out of range

any idea ?

I've patched pkdep to fix the issue.
You can find the new release (r1.4) at the same address as before: http://www.mediaweb.com/gentoo/scripts/pkdep/
Please let me know if it works.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum