Unmerging and reverse dependencies-- need testers

[neonik]

Uncommenting lines 1971 and 1985 in the latest version would probably help.

Owh... So easy! Ok, next take on that

[spb]

Owh... So easy! Ok, next take on that

I meant help by providing a bit more info. But I've worked out the problem in those two cases (I told you the virtual code might be flaky).

Lines 1916ff:

Code: Select all

					for p in others:
						myprovide = vardbapi.aux_get(cpv, ["PROVIDE"])[0].split()
						if mycp not in provide:
							continue

Change 'provide' in that if to 'myprovide'. Stupid typos. But I'd be interested in that debug info anyway-- I'd like to know why your system is causing it to throw errors on baselayout and coreutils when mine isn't.

I've reuploaded the last (v5) diff with that fix included.

[neonik]

I meant help by providing a bit more info.

I understood that. I was gonna try debugging myself, but no need anymore.

Your patch v5 seems to solve the issue with both packages.


Here the output of v4 with provide instead of myprovide (cut down up to the last reference):

Found revdep sys-apps/portage-2.0.51_pre20 for dev-lang/python-2.3.4
checking sys-apps/file-4.10
Checking atom virtual/libc
Checking atom virtual/python
Found revdep sys-apps/file-4.10 for dev-lang/python-2.3.4
checking sys-kernel/config-kernel-0.3.3
Checking atom virtual/python
Found revdep sys-kernel/config-kernel-0.3.3 for dev-lang/python-2.3.4
checking sys-libs/libcap-1.10-r4
Checking atom >=virtual/python-2.2.1
Traceback (most recent call last):
File "./emerge-rdep", line 2807, in ?
if 1==unmerge(myaction, myfiles):
File "./emerge-rdep", line 2166, in unmerge
myrdepgraph.create(y)
File "./emerge-rdep", line 2030, in create
self.create( mydep, cpv, 1 )
File "./emerge-rdep", line 2030, in create
self.create( mydep, cpv, 1 )
File "./emerge-rdep", line 2029, in create
for mydep in self.direct_revdeps(cpv):
File "./emerge-rdep", line 1987, in direct_revdeps
if self.parse_dep(myatom, cpv, myinstcpvs):
File "./emerge-rdep", line 1919, in parse_dep
if mycp not in provide:
NameError: global name 'provide' is not defined

Changing provide to myprovide solves the issue.


Edit: And trying to unmerge the entire system works now too. Great job, thebell.

[spb]

Your patch v5 seems to solve the issue with both packages.

Good. It was a stupid problem anyway.

Here the output of v4 with provide instead of myprovide (cut down up to the last reference):

<snip>
checking sys-libs/libcap-1.10-r4
Checking atom >=virtual/python-2.2.1
Traceback (most recent call last):
<snip>

I though as much. You've got a package installed that depends on a specific version of a virtual package. I don't, so that branch never got executed on my system.

Edit: And trying to unmerge the entire system works now too. Great job, thebell.

Great! If noone has any more problems with it, I might submit this to bugzilla.

[rcxAsh]

Big Edit
Removed everything I said so that I don't confuse anyone.
lol. I figured out what I did wrong.

Oh. it's a patch for _emerge_.....

Edit 2
I tried it with xorg-x11, but it fails for me.

Code: Select all

lostech bin # emerge -Cp xorg-x11

>>> These are the packages that I would unmerge:

Calculating reverse dependencies \Note: Empty target in string. (Deprecated)
  --> svga? svga? []
Note: Empty target in string. (Deprecated)
  --> X? X? []                                                                 
Note: Empty target in string. (Deprecated)
  --> svga? svga? []
Note: Empty target in string. (Deprecated)
  --> X? X? []
Traceback (most recent call last):
  File "/usr/bin/emerge", line 2815, in ?
    if 1==unmerge(myaction, myfiles):
  File "/usr/bin/emerge", line 2170, in unmerge
    myrdepgraph.create(y)
  File "/usr/bin/emerge", line 2029, in create
    self.create( mydep, cpv, 1 )
  File "/usr/bin/emerge", line 2028, in create
    for mydep in self.direct_revdeps(cpv):
  File "/usr/bin/emerge", line 1986, in direct_revdeps
    if self.parse_dep(myatom, cpv, myinstcpvs):
  File "/usr/bin/emerge", line 1928, in parse_dep
    if not cpv_satisfies_dep(cpv, atom):
  File "/usr/bin/emerge", line 1827, in cpv_satisfies_dep
    return ( len(portage.match_from_list(atom, [cpv])) > 0 )
  File "/usr/lib/portage/pym/portage.py", line 4031, in match_from_list
    cat,pkg = catsplit(mycpv)
ValueError: unpack list of wrong size

I'm using the last emerge-unmerge-rdep-v5.diff that I saw in the thread; for portage 2.0.51-pre20.

Looks really great though! Thanks for the great patch.

Edit 3
I tried it with gtk+ as well, but it failed as well:

Code: Select all

lostech bin # emerge -Cp gtk+

>>> These are the packages that I would unmerge:

Calculating reverse dependencies \Traceback (most recent call last):
  File "/usr/bin/emerge", line 2815, in ?
    if 1==unmerge(myaction, myfiles):
  File "/usr/bin/emerge", line 2170, in unmerge
    myrdepgraph.create(y)
  File "/usr/bin/emerge", line 2028, in create
    for mydep in self.direct_revdeps(cpv):
  File "/usr/bin/emerge", line 1986, in direct_revdeps
    if self.parse_dep(myatom, cpv, myinstcpvs):
  File "/usr/bin/emerge", line 1928, in parse_dep
    if not cpv_satisfies_dep(cpv, atom):
  File "/usr/bin/emerge", line 1827, in cpv_satisfies_dep
    return ( len(portage.match_from_list(atom, [cpv])) > 0 )
  File "/usr/lib/portage/pym/portage.py", line 4031, in match_from_list
    cat,pkg = catsplit(mycpv)
ValueError: unpack list of wrong size

I'm not sure what's wrong with me though, as neonik has already reported that these two packages worked fine.

[neonik]

Please post your emerge info or emerge -v info and try this to see on what package it breaks:

Uncommenting lines 1971 and 1985 in the latest version would probably help.

[rcxAsh]

Alright, thanks for the reply. I uncommented the lines and it seems that it fails on the games-util/qstat package (at least, that's the last thing it prints out).

Output from gtk+:

Code: Select all

lostech root # emerge -Cpv gtk+ 

>>> These are the packages that I would unmerge:

Calculating reverse dependencies \checking app-admin/gkrellm-2.2.2
Checking atom >=sys-apps/sed-4
Checking atom dev-libs/openssl
Checking atom >=x11-libs/gtk+-2.0.5
checking app-cdr/gcombust-0.1.55
Checking atom =x11-libs/gtk+-1.2*
Checking atom sys-devel/gettext
Checking atom app-cdr/cdrtools
checking app-editors/bluefish-0.12
Checking atom >=x11-libs/gtk+-2
checking app-i18n/uim-0.4.1
Checking atom virtual/x11
Checking atom >=x11-libs/gtk+-2
checking app-pda/jpilot-0.99.6-r1
Checking atom >=x11-libs/gtk+-2
checking app-pda/plucker-1.8-r1
Checking atom >=dev-lang/python-1.5.2
Checking atom >=x11-libs/gtk+-2.2
checking app-sci/tilp-6.09
Checking atom dev-libs/libticables
Checking atom dev-libs/libticalcs
Checking atom dev-libs/libtifiles
Checking atom =x11-libs/gtk+-1.2*
checking app-text/gtkspell-2.0.4-r1
Checking atom >=x11-libs/gtk+-2
checking dev-perl/gtk-perl-0.7008-r10
Checking atom media-libs/gdk-pixbuf
Checking atom =x11-libs/gtk+-1.2*
Checking atom dev-perl/XML-Writer
Checking atom dev-perl/XML-Parser
checking dev-python/wxpython-2.4.2.4
Checking atom >=dev-lang/python-2.1
Checking atom >=x11-libs/wxGTK-2.4.2
Checking atom >=x11-libs/gtk+-2.0
checking dev-python/pygtk-2.0.0-r1
Checking atom >=dev-lang/python-2.2
Checking atom >=x11-libs/pango-1
Checking atom >=x11-libs/gtk+-2
checking dev-util/anjuta-1.2.2-r1
Checking atom >=dev-libs/glib-2.0.6
Checking atom >=x11-libs/gtk+-2.0.8
checking games-emulation/epsxe-1.6.0-r3
Checking atom >=dev-libs/glib-1.2
Checking atom =x11-libs/gtk+-1.2*
Checking atom =sys-libs/ncurses-5*
Checking atom =sys-libs/zlib-1*
Checking atom net-misc/wget
Checking atom games-emulation/psemu-peopsspu
checking games-emulation/psemu-peopsspu-1.0.7-r1
Checking atom app-arch/unzip
Checking atom =x11-libs/gtk+-1*
Checking atom sys-devel/automake
checking games-util/xqf-0.9.13
Checking atom >=games-util/qstat-25
Checking atom :
Traceback (most recent call last):
  File "/usr/bin/emerge", line 2815, in ?
    if 1==unmerge(myaction, myfiles):
  File "/usr/bin/emerge", line 2170, in unmerge
    myrdepgraph.create(y)
  File "/usr/bin/emerge", line 2028, in create
    for mydep in self.direct_revdeps(cpv):
  File "/usr/bin/emerge", line 1986, in direct_revdeps
    if self.parse_dep(myatom, cpv, myinstcpvs):
  File "/usr/bin/emerge", line 1928, in parse_dep
    if not cpv_satisfies_dep(cpv, atom):
  File "/usr/bin/emerge", line 1827, in cpv_satisfies_dep
    return ( len(portage.match_from_list(atom, [cpv])) > 0 )
  File "/usr/lib/portage/pym/portage.py", line 4031, in match_from_list
    cat,pkg = catsplit(mycpv)
ValueError: unpack list of wrong size

The output for xorg-x11 is quite long (2937 lines), so instead of making people scroll like crazy, you can see it here: emerge -Cpv xorg-x11

[fctk]

sorry but... i'm a noob and don't know how i can patch portage with the .diff file... i'd really want to test this...
i downloaded the diff and upgraded portage to 51-pre20 and now? thx

[neonik]

So in both cases on >=games-util/qstat-25. Humm... I'll test it.

sorry but... i'm a noob and don't know how i can patch portage with the .diff file... i'd really want to test this...
i downloaded the diff and upgraded portage to 51-pre20 and now? thx

First you have to create a directory where you put the binary executable /usr/bin/emerge:

Code: Select all

$ mkdir ~/portage/

Second step would be changing in the named directory and downlaoding the patch:

Code: Select all

$ cd ~/portage/ && wget http://graphit.sourceforge.net/misc/emerge-unmerge-rdep-v5.diff

After that you copy the file emerge over into this directory:

Code: Select all

$ cp /usr/bin/emerge ./

Now you should patch the file:

Code: Select all

$ patch -b ./emerge -i emerge-unmerge-rdep-v5.diff

That's all. Now you can execute ./emerge or ~/portage/emerge to test it out. You'll eventually have to become root to run emerge.

[neonik]

>=games-util/qstat-25

There's no such package, actually. There's a games-util/qstat-2.6 and games-util/qstat-2.5c in the tree, but nothing like games-util/qstat-25. Trying to emerge it, which should normally give an error message saying you should append a '=', it outputs this error message:

!!! Problem in games-util/qstat-25 dependencies.
!!!
Traceback (most recent call last):
File "/usr/bin/emerge", line 2739, in ?
retval,favorites=mydepgraph.select_files(myfiles)
File "/usr/bin/emerge", line 997, in select_files
print "!!!",e.value,e.__module__
AttributeError: KeyError instance has no attribute 'value'

Which I don't consider normal. I'm no portage developer, but I believe thebell should have a solution.
At the first sight it's a broken dependency, i.e. either a typing mistake or the package was renamed/deleted.

[fctk]

thanks for the help but now i get this:

Code: Select all

bash-2.05b$ patch -p1 -b ./emerge -i emerge-unmerge-rdep-v5.diff 
missing header for unified diff at line 3 of patch
patching file ./emerge

is it ok?

[neonik]

Yeah, sorry, my mistake. -p1 should be there. But it shouldn't matter. Anyway, just to make sure, redo that please without the flag -p1.

For that you can do following:

Code: Select all

$ cd ~/portage/ ; mv ./emerge.orig ./emerge ; patch -b -i emerge-unmerge-rdep-v5.diff

I'll edit the post accordingly.

[rcxAsh]

>=games-util/qstat-25

There's no such package, actually. There's a games-util/qstat-2.6 and games-util/qstat-2.5c in the tree, but nothing like games-util/qstat-25. Trying to emerge it, which should normally give an error message saying you should append a '=', it outputs this error message:

!!! Problem in games-util/qstat-25 dependencies.
!!!
Traceback (most recent call last):
File "/usr/bin/emerge", line 2739, in ?
retval,favorites=mydepgraph.select_files(myfiles)
File "/usr/bin/emerge", line 997, in select_files
print "!!!",e.value,e.__module__
AttributeError: KeyError instance has no attribute 'value'

Which I don't consider normal. I'm no portage developer, but I believe thebell should have a solution.
At the first sight it's a broken dependency, i.e. either a typing mistake or the package was renamed/deleted.

Yah, I noticed that the package didn't exist. I did try copying the 2.5c ebuild naming it 25c (since it seems to be that). However, it still failed.

I just "upgraded" (portage tells me it's downgrading.. because, 25 is greater than 2.6 ) qstat.

Should I need to do anything now? Rebuild some cache or something? Because gtk+ (so I assume xorg-x11 as well) still fails on qstat-25.

What? I thought I got rid of qstat-25c. During the unmerge of the older version I saw this, but don't know if it means anything significant. It just confirms what you said and what I originally suspected:

Code: Select all

!!! aux_get(): ebuild for 'games-util/qstat-25c' does not exist at:
!!!            None

So it didn't exist indeed. But I think that it did unmerge successfully.. I wasn't sure whether qstat-25c was the same as qstat-2.5c. So now I know that it isn't.

Also, even trying to unmerge qstat itself gives me an error breaking on qstat-25. Weird.

BTW, in the future, shouldn't reverse dependancy checking be able to handle a non-existant ebuild without dying? I notice that some ebuilds get renamed from time to time. So, it would be safer if the patched emerge could handle these without choking. (Eg, there was once an ebuild for bittorrent-theshadow, or something similar. But it has since been renamed to bittornado to match the project's new name)

[spb]

Sorry all, but I've been out of the loop for a while, and kinda lost this thread.

rcxAsh: that's an interesting problem. It does seem to choke on some dep atoms that are otherwise accepted, and I can't quite work out why, since it uses Portage's existing atom matching functions. As for dying on non-existant ebuilds, it definitely shouldn't, but I can't see how it would at the moment. The revdep code doesn't even look in the portage tree. I'm on a different computer at the moment, but I'll have a look at that as soon as I get back.

Oh, and for patching it, I usually do

Code: Select all

cp /usr/bin/emerge .
patch emerge <emerge-unmerge-rdep-v<n>.diff

[batzee]

Hmm, the patch (v5) gives a bunch of error messages on my system:

Code: Select all

patch emerge <emerge-unmerge-rdep-v5.diff
patching file emerge
Hunk #1 FAILED at 11.
Hunk #2 succeeded at 31 (offset -4 lines).
Hunk #3 succeeded at 1655 (offset -165 lines).
Hunk #4 succeeded at 1952 (offset -168 lines).
Hunk #5 succeeded at 1984 with fuzz 2 (offset -168 lines).
Hunk #6 FAILED at 1996.
Hunk #7 succeeded at 2019 (offset -175 lines).
2 out of 7 hunks FAILED -- saving rejects to file emerge.rej

This is the emerge.rej:

Code: Select all

% cat emerge.rej
***************
*** 11,17 ****
  from stat import *
  from output import *

- import portage
  import portage_util
  import portage_locks
  import portage_exception
--- 11,17 ----
  from stat import *
  from output import *

+ import portage, portage_dep
  import portage_util
  import portage_locks
  import portage_exception
***************
*** 1772,1778 ****
                                        print yellow("\a!!! This could be damaging to your system.\n")
                                        if "--pretend" not in myopts:
                                                countdown(10,red("Press Ctrl-C to Stop"))
-
                else:
                        #unmerge_action in ["prune", clean"]
                        slotmap={}
--- 1996,2006 ----
                                        print yellow("\a!!! This could be damaging to your system.\n")
                                        if "--pretend" not in myopts:
                                                countdown(10,red("Press Ctrl-C to Stop"))
+                       else:
+                               # Add this package to the revdepgraph.
+                               for y in mymatch:
+                                       myrdepgraph.create(y)
+
                else:
                        #unmerge_action in ["prune", clean"]
                        slotmap={}

BTW, my portage is version 2.0.50-r11

[spb]

The latest patch is against 2.0.51_pre20, so it probably won't apply cleanly to .50. Those two rejects are fairly simple to fix though-- just open up emerge in an editor, remove the lines with a '-' next to them, and add the ones with a '+'.

[Mon]

_pre20 is in Portage now. Gonna try to patch and see how it works.

The idea is great.

I don't see a _pre20 in my tree. 51_r9 is the newest i can and have emerged, ofcourse the patch doesn't apply to this one.
What do i do?

[neonik]

Try to fix the rejects per hand, it shouldn't be hard. If you can't cope that, just give me a note. thebell could, however, have a solution, since it's his work.

[JOS654]

Hi is this project alive yet? i would like to help testing
btw thanks for your work

[spb]

Hi is this project alive yet? i would like to help testing
btw thanks for your work

It worked last time I tried it, but it hasn't been updated to the latest portage versions, so it might not now. I doubt I'll be updating it any further either, since jstubbs is working on a completely new dependency engine for portage-2.1 which will make this unnecessary.

[ecoffey]

Seems like this thread has been quiet for a good long while now.

But if anyone is still reading it, I was curious about when something like this would make into the offical emerge program?

I think its a neat and insanely useful feature, so I would really like to use it full time.

Thanks.

[koniu]

ecoffey, read spb's last post. portage 2.1 seems to be the thing (to come).