| View previous topic :: View next topic |
| Author |
Message |
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 7742
Location: beyond the rim
|
 Posted: Mon Jun 27, 2005 11:23 pm Post subject: two new metadata query tools |
 |
|
Guess it's about time I release my little metadata query tool to the public, I've kept it for myself long enough:
http://dev.gentoo.org/~genone/scripts/metascan
What does it do? It scans the portage tree (or your vardb) for packages/ebuilds matching certain metadata conditions, examples:
| Code: | # - find all packages in the tree supporting the berkdb use flag: metascan IUSE berkdb
# - find all packages in the tree providing an mta: metascan PROVIDE virtual/mta
# - find all packages in the tree providing an mta and not supporting the maildir use flag:
# metascan -a PROVIDE virtual/mta IUSE -maildir
# - find all installed packages containing net-nds/openldap in DEPEND or RDEPEND:
# metascan -i -n DEPEND net-nds/openldap RDEPEND net-nds/openldap
|
Please note that you should be familiar with the basic ebuild syntax to use this tool. If you can't follow the examples above, this tool is not for you.
It's intended as a low-level query tool for maintenance tasks to complement the higher-level functions of equery (in the long run it might also be integrated into equery). I also hope to post the counterpart script auxget (maybe with a different name) shortly (needs some more polishing first).
EDIT Updated to work tihe the new portage-2.1_pre versions (including several refactorings and other enhancements)
Last edited by Genone on Tue Jan 17, 2006 2:30 pm; edited 2 times in total |
|
| Back to top |
|
 |
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 7742
Location: beyond the rim
|
| Posted: Tue Jun 28, 2005 11:45 pm Post subject: |
|
|
Ok, as said I also uploaded auxget now:
http://dev.gentoo.org/~genone/scripts/auxget
It's the complement to metascan as it allows you to read the individual variables for a given package, for example:
| Code: | auxget app-admin/sudo
DEPEND: pam? ( || ( virtual/pam sys-libs/pam ) ) ldap? ( >=net-nds/openldap-2.1.30-r1 ) skey? ( >=app-admin/skey-1.1.5-r1 ) sys-devel/bison virtual/mta !bootstrap? ( sys-devel/patch userland_GNU? ( sys-apps/shadow ) )
RDEPEND: pam? ( || ( virtual/pam sys-libs/pam ) ) ldap? ( >=net-nds/openldap-2.1.30-r1 ) skey? ( >=app-admin/skey-1.1.5-r1 ) sys-devel/bison virtual/mta ldap? ( dev-lang/perl )
SLOT: 0
SRC_URI: ftp://ftp.sudo.ws/pub/sudo/sudo-1.6.8p9.tar.gz
RESTRICT:
HOMEPAGE: http://www.sudo.ws/
LICENSE: Sudo
DESCRIPTION: Allows certain users/groups to run commands as root
KEYWORDS: alpha amd64 hppa ia64 ppc ppc64 sparc x86
INHERITED: multilib pam eutils
IUSE: pam skey offensive ldap
CDEPEND:
PDEPEND:
PROVIDE:
|
Also has a few options to tune the output to your needs. Like metascan this is a low-level tool, so you should know a little bit about portage if you want to use it. |
|
| Back to top |
|
|
TrueDFX
Developer
Joined: 02 Jun 2004
Posts: 1317
|
| Posted: Wed Jun 29, 2005 1:39 am Post subject: |
|
|
Very nice. Two questions though:
I noticed auxget's -r option doesn't respect package.use. Is that intentional? (Also, in combination with -i, -r uses the current USE flags, not those with which the package was installed, but that doesn't matter to me.)
The second question: would it be possible to add a similar option to metascan to search only through the USE-reduced *DEPENDs? |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 7742
Location: beyond the rim
|
| Posted: Wed Jun 29, 2005 5:48 am Post subject: |
|
|
| TrueDFX wrote: | Very nice. Two questions though:
I noticed auxget's -r option doesn't respect package.use. Is that intentional? |
Should work and works for me ...
| Quote: | (Also, in combination with -i, -r uses the current USE flags, not those with which the package was installed, but that doesn't matter to me.)
|
Yeah, support for vardb is limited because the interface is very different.
| Quote: | | The second question: would it be possible to add a similar option to metascan to search only through the USE-reduced *DEPENDs? |
Done (just added that code to auxget a few minutes before posting the link). |
|
| Back to top |
|
|
TrueDFX
Developer
Joined: 02 Jun 2004
Posts: 1317
|
| Posted: Wed Jun 29, 2005 6:54 am Post subject: |
|
|
What am I missing, then? I have nocxx in my USE flags, overridden for gcc with package.use.
| Code: | $ emerge -pv sys-devel/gcc
These are the packages that I would merge, in order:
Calculating dependencies ...done!
[ebuild R ] sys-devel/gcc-4.0.1_pre20050616 (-altivec) -bootstrap -build +fortran -gcj* +gtk -hardened -ip28 (-multilib) -multislot (-n32) (-n64) +nls -nocxx +objc -static 31,018 kB
[ebuild N ] sys-libs/libstdc++-v3-3.3.4 (-multilib) +nls 22,784 kB
Total size of downloads: 53,803 kB
$ python auxget sys-devel/gcc PDEPEND
PDEPEND: sys-devel/gcc-config !nocxx? ( !n32? ( !n64? ( !elibc_uclibc? ( !build? ( sys-libs/libstdc++-v3 ) ) ) ) )
$ python auxget -r sys-devel/gcc PDEPEND
PDEPEND: sys-devel/gcc-config |
And thanks for the metascan improvement, but unfortunately it doesn't work with portage-2.0*: portage_dep is not imported, and if I add that to the import line, I get another error: | Code: | $ python metascan.1 -i -c -p DEPEND dev-lang/python
Reading package list ... done
Scanning packages for ['DEPEND'] ... Traceback (most recent call last):
File "metascan.1", line 194, in ?
pvalues[i] = " ".join(portage.flatten(portage_dep.use_reduce(portage_dep.paren_reduce(pvalues[i]), settings["USE"])))
IndexError: list index out of range |
(Edit: I read the comment in the script, tried adding -r, and success. So I guess importing portage_dep is all. Great job. thanks  ) |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 7742
Location: beyond the rim
|
| Posted: Wed Jun 29, 2005 11:45 am Post subject: |
|
|
| TrueDFX wrote: | | What am I missing, then? I have nocxx in my USE flags, overridden for gcc with package.use. |
Weird. That exact example works here (same gcc version too). |
|
| Back to top |
|
|
TrueDFX
Developer
Joined: 02 Jun 2004
Posts: 1317
|
| Posted: Wed Jun 29, 2005 12:49 pm Post subject: |
|
|
Another portage 2.1* change? I managed to get it working by changing
| Code: | | portage_dep.use_reduce(..., settings["USE"]) |
to
| Code: | | portage_dep.use_reduce(..., " ".split(settings["USE"])) |
So actually, it looks like it had nothing to do with package.use at all, and would probably print wrong results even without package.use. Sorry for the confusion. |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 7742
Location: beyond the rim
|
| Posted: Thu Jun 30, 2005 7:00 am Post subject: |
|
|
| Hmm, might have had a bug there, please try the version I just uploaded (featuring a new --show-cpv option also). |
|
| Back to top |
|
|
TrueDFX
Developer
Joined: 02 Jun 2004
Posts: 1317
|
| Posted: Thu Jun 30, 2005 7:10 am Post subject: |
|
|
| No problem here with your new version. |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 7742
Location: beyond the rim
|
| Posted: Wed Sep 21, 2005 2:35 am Post subject: |
|
|
Added some fixes and enhancements to metascan (most notably a --debug and --regular-expressions switch).
NOTE: the syntax for negated queries has changed as the old syntax didn't allow to query for stuff like KEYWORDS="-amd64".
Old syntax: metascan KEY -value
New syntax: metascan -KEY value |
|
| Back to top |
|
|
dpetka2001
l33t
Joined: 04 Mar 2005
Posts: 795
|
| Posted: Sat Oct 22, 2005 5:24 am Post subject: |
|
|
just downloaded metascan and am not able to use it...it says | Code: | | -bash: /usr/bin/local/metascan: Permission denied |
why is that?? I am logged in as root...and another question...would you suggest metascan to find reverse dependencies for a package compared to unclepine?? thanks... |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 7742
Location: beyond the rim
|
| Posted: Sat Oct 22, 2005 5:39 am Post subject: |
|
|
Already asked in the other thread, but you gave it execute permissions, right?
As for the other question, I don't know unclepine, but metascan does things the right way (as it just queries portage instead of playing around with vdb/porttree directly) |
|
| Back to top |
|
|
dpetka2001
l33t
Joined: 04 Mar 2005
Posts: 795
|
| Posted: Sat Oct 22, 2005 6:01 am Post subject: |
|
|
it was the execute permission that was the problem...fixed it and it works now...so in order to find reverse dependencies for a specific package I have to type | Code: | | metascan -n DEPEND category/application-version |
?? what is it better to use?? the portage tree or the vardb as database?? |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 7742
Location: beyond the rim
|
| Posted: Sat Oct 22, 2005 6:18 am Post subject: |
|
|
Depends: you want the actual reverse deps for a installed package or the potential reverse deps for a non-installed package?
For the first you use vdb, for the second porttree.
Oh, and you don't specify a version when using -n (generally using versions with metascan is tricky). |
|
| Back to top |
|
|
dpetka2001
l33t
Joined: 04 Mar 2005
Posts: 795
|
| Posted: Sat Oct 22, 2005 7:33 am Post subject: |
|
|
so to find the reverse dependencies of an installed package on my system I would use | Code: | | metascan -i -n DEPEND category/package |
right?? is any other Key needed?? what about RDEPEND that you use in your example?? |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 7742
Location: beyond the rim
|
| Posted: Sat Oct 22, 2005 8:42 am Post subject: |
|
|
| dpetka2001 wrote: | so to find the reverse dependencies of an installed package on my system I would use | Code: | | metascan -i -n DEPEND category/package |
right? |
Yep
| Quote: | | is any other Key needed?? what about RDEPEND that you use in your example?? |
Well, there are three different types of dependencies:
- DEPEND: build time dependencies
- RDEPEND: run time dependencies
- PDEPEND: run time dependencies that should be merged after the package depending on them
It depends on the specific situation which key(s) you should use, e.g. for unmerge safety checks you want RDEPEND and PDEPEND but not DEPEND while for test coverage checks you want all of them. |
|
| Back to top |
|
|
dberkholz
Developer
Joined: 18 Mar 2003
Posts: 975
Location: Corvallis, OR, USA
|
| Posted: Mon Dec 19, 2005 10:25 pm Post subject: |
|
|
Downloaded both and neither run, with the portage-2.1:
| Code: | $ metascan --help
Traceback (most recent call last):
File "/home/donnie/dev/public_html/scripts/metascan", line 32, in ?
isvalidatom = portage_dep.isvalidatom
AttributeError: 'module' object has no attribute 'isvalidatom'
|
Ideas? |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 7742
Location: beyond the rim
|
| Posted: Tue Jan 17, 2006 2:32 pm Post subject: |
|
|
| Ups, sorry, forgot to upload the new versions after I put them in subversion. Current versions should work with both 2.0 and new 2.1. |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 7742
Location: beyond the rim
|
| Posted: Mon Feb 13, 2006 12:16 am Post subject: |
|
|
As another addition auxget has now an alternate personality that is triggered when it is invoked as mass_auxget, this allows reading metadata keys for multiple packages at once (instead of calling auxget for each of them), example:
| Code: | $ mass_auxget IUSE LICENSE sys-apps/portage app-portage/gentoolkit app-portage/porthole
sys-apps/portage-2.1_pre4-r1: IUSE="build doc selinux" LICENSE="GPL-2"
app-portage/gentoolkit-0.2.2_pre2: IUSE="" LICENSE="GPL-2"
app-portage/porthole-0.4.1: IUSE="debug" LICENSE="GPL-2" |
When no packages are specified all packages are looked up, same for keys (so without any arguments it will look up all keys for all packages). |
|
| Back to top |
|
|
eyebex
n00b
Joined: 23 Jul 2005
Posts: 63
|
|
| Back to top |
|
|
|
Portage & Programming
