| View previous topic :: View next topic |
| Author |
Message |
vad3r
Guru
Joined: 02 May 2004
Posts: 461
Location: Munich, Germany
|
 Posted: Fri Mar 20, 2009 4:35 pm Post subject: How to get all USE flags: Gentoo management software |
 |
|
Hi all,
we're planning to write a software to manage a huge amount of Gentoo servers and therefore we need to get all available USE flags for each ebuild in a simple way. What's the best approach to get the flags:
- Parse the ebuilds
- Parse emerge/equery output
- Use Python Portage modules
Thanks
vad3R
_________________
-- http://www.linuxaddicted.de/ | Please help to answer open questions
Last edited by vad3r on Mon Mar 23, 2009 9:00 pm; edited 1 time in total |
|
| Back to top |
|
 |
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Fri Mar 20, 2009 5:22 pm Post subject: |
|
|
Another possibility:
| Code: | | PRINT_KEYWORDS=false STYLE_VERSION_LINES=true FORMAT='<availableversionsshort>' eix sys-devel/gcc |
|
|
| Back to top |
|
|
yther
Apprentice
Joined: 25 Oct 2002
Posts: 151
Location: Charlotte, NC (USA)
|
| Posted: Fri Mar 20, 2009 5:52 pm Post subject: |
|
|
Parsing the output of "emerge -pv" might be simplest. The available flags should be easy to parse that way. Example:
| Code: | rassilon@miharu:~$ emerge -pv klipper
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild R ] kde-base/klipper-4.2.1 USE="kdeprefix -debug" 0 kB
Total: 1 package (1 reinstall), Size of downloads: 0 kB |
As they appear within USE="blah bleh" you should be able to grab what you need fairly easily, after stripping the extra lines of output. It could probably be done using awk, but don't ask me how!  |
|
| Back to top |
|
|
cwr
Veteran
Joined: 17 Dec 2005
Posts: 1969
|
| Posted: Sun Mar 22, 2009 11:50 am Post subject: |
|
|
Parse the ebuilds, clean up the data, and stick it into an Sqlite database for
easy access. Not a major task, and saves a lot of time in the long run.
Some ebuilds are pretty weird, so you have to clean the data up a bit before
it can be usefully summarised.
Will |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Sun Mar 22, 2009 1:08 pm Post subject: |
|
|
| cwr wrote: | Parse the ebuilds, clean up the data, and stick it into an Sqlite database for
easy access. Not a major task, and saves a lot of time in the long run. |
"Parsing" the ebuilds to generate metadata is certainly the wrong way; if you do this, you will have to execute them, and this needs a lot of time and has security risks. update-eix has solved this problem by a rich configurable interface for getting the metadata by various means - that's why I suggested the eix solution. Of course, eix only knows the metadta (like IUSE) but has no mechanism to determine which USE-flags are set (i.e. profile-useflags, /etc/portage/package.use, USE environment variable etc. are not considered in this respect). |
|
| Back to top |
|
|
gimpel
Advocate
Joined: 15 Oct 2004
Posts: 2720
Location: Munich, Bavaria
|
| Posted: Sun Mar 22, 2009 1:11 pm Post subject: |
|
|
Use equery.
| Code: | | equery -q -C -N uses <pkgspec>|awk '{print $3}' |
_________________
http://proaudio.tuxfamily.org/wiki - pro-audio software overlay
|
|
| Back to top |
|
|
cwr
Veteran
Joined: 17 Dec 2005
Posts: 1969
|
| Posted: Sun Mar 22, 2009 1:36 pm Post subject: |
|
|
| mv wrote: | | cwr wrote: | Parse the ebuilds, clean up the data, and stick it into an Sqlite database for
easy access. Not a major task, and saves a lot of time in the long run. |
"Parsing" the ebuilds to generate metadata is certainly the wrong way; if you do this, you will have to execute them, and this needs a lot of time and has security risks. update-eix has solved this problem by a rich configurable interface for getting the metadata by various means - that's why I suggested the eix solution. Of course, eix only knows the metadta (like IUSE) but has no mechanism to determine which USE-flags are set (i.e. profile-useflags, /etc/portage/package.use, USE environment variable etc. are not considered in this respect). |
No, there's no reason to execute the ebuilds when parsing them. Why would there be? It's just a question
of reading the strings and deciding if they are of any interest, then carrying out various substitutions. The
initial scan is about as quick as the average emerge, but subsquent lookups in the Sqlite3 database are
extremely fast.
Will |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Sun Mar 22, 2009 2:04 pm Post subject: |
|
|
| cwr wrote: | | No, there's no reason to execute the ebuilds when parsing them. Why would there be? It's just a question of reading the strings |
No, it is not: Especially IUSE strings are often set in eclasses, and, moreover, the shell constructs leading to the final strings are sometimes rather complex. You really have to execute the ebuilds by bash to get a reliable result - an "empric" parsing (i.e. not really evaluating all shell commands by bash) is not enough as you can see if you look at the results of update-eix's cache method "parse" (which bases already on a rather complex parsing algorithm). |
|
| Back to top |
|
|
cwr
Veteran
Joined: 17 Dec 2005
Posts: 1969
|
| Posted: Mon Mar 23, 2009 2:33 pm Post subject: |
|
|
| mv wrote: | | cwr wrote: | | No, there's no reason to execute the ebuilds when parsing them. Why would there be? It's just a question of reading the strings |
No, it is not: Especially IUSE strings are often set in eclasses, and, moreover, the shell constructs leading to the final strings are sometimes rather complex. You really have to execute the ebuilds by bash to get a reliable result - an "empric" parsing (i.e. not really evaluating all shell commands by bash) is not enough as you can see if you look at the results of update-eix's cache method "parse" (which bases already on a rather complex parsing algorithm). |
True, but there are relatively few of these flags; in practice very few dependencies are affected.
Will |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Mon Mar 23, 2009 8:47 pm Post subject: |
|
|
| cwr wrote: | | but there are relatively few of these flags; in practice very few dependencies are affected. |
That's not true. As I suggested, use | Code: | | PORTDIR_CACHE_METHOD=parse OVERLAY_CACHE_METHOD=parse update-eix |
and compare the results. For example, all kde, most qt, most gnome, lots of java, mozilla, appache, texlive packages, and a lot of toolchain will not have all IUSE data, not to speak about missing debug flag in even more pacakges. |
|
| Back to top |
|
|
vad3r
Guru
Joined: 02 May 2004
Posts: 461
Location: Munich, Germany
|
| Posted: Mon Mar 23, 2009 8:57 pm Post subject: |
|
|
A lot of interesting informations so far. Thanks a lot for your suggestions. I'll try eix cause it seems to be reliable, easy and nice to parse
_________________
-- http://www.linuxaddicted.de/ | Please help to answer open questions |
|
| Back to top |
|
|
cwr
Veteran
Joined: 17 Dec 2005
Posts: 1969
|
| Posted: Wed Mar 25, 2009 11:07 am Post subject: |
|
|
| mv wrote: | | cwr wrote: | | but there are relatively few of these flags; in practice very few dependencies are affected. |
That's not true. As I suggested, use | Code: | | PORTDIR_CACHE_METHOD=parse OVERLAY_CACHE_METHOD=parse update-eix |
and compare the results. For example, all kde, most qt, most gnome, lots of java, mozilla, appache, texlive packages, and a lot of toolchain will not have all IUSE data, not to speak about missing debug flag in even more pacakges. |
There are 24813 ebuilds in the 2008.0 portage tree; 146 of these have macros in their IUSE string, 188 macros in total.
(Some of these are IUSE replacement strings). There are 32 different macro strings, including two sets of duplicates
which would appear identical to a shell parser. These are not large percentages.
Will |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Wed Mar 25, 2009 6:01 pm Post subject: |
|
|
| cwr wrote: | | 146 of these have macros in their IUSE string, 188 macros in total. |
I bet that you did not take the IUSE strings set in eclasses into account; moreover, even if there is no $(...) substitution in IUSE, it might still be set conditionally (especially in eclasses). |
|
| Back to top |
|
|
|
Portage & Programming
