Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
emerge -p sanitizing filter
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
ectospasm
l33t
l33t


Joined: 19 Feb 2003
Posts: 711
Location: Mobile, AL, USA

PostPosted: Wed Aug 02, 2006 11:35 am    Post subject: emerge -p sanitizing filter Reply with quote

Many times I want to take the output of emerge -p, and put the list of packages into /etc/portage/package.*. I haven't found an easy way of doing that with emerge itself (-p and -vp give too much info for this purpose), so I wrote a filter in Perl to do it. Basically, at the moment on my system if I issue this command, I get the following output:
Code:
$ emerge -vuDp baselayout

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U ] sys-libs/gpm-1.20.1-r5 [1.20.1-r4] 559 kB
[ebuild     U ] sys-devel/gettext-0.14.5 [0.14.4] USE="nls -doc -emacs -nocxx%" 6,939 kB

Total size of downloads: 7,498 kB


That's great for humans, but that won't work in the /etc/portage/package.* files. Piping the same command through my filter, and it looks much better:
Code:
emerge -vuDp baselayout | scripts/pkg-list.pl
sys-libs/gpm
sys-devel/gettext


You can add keywords to the output, say for adding to package.use or package.keywords, just use the -k or --keyword option, followed by a string (like "~x86" or "gtk -qt"). There's also a -V or --versions option, to display package versions.

The script lives here for the time being: http://blancher.org/trey/scripts/pkg-list.pl

Let me know what you think, if this is overkill, if it's broken, etc.
_________________
Join the adopt an unanswered post initiative today
Join the EFF!
Join the Drug Policy Alliance!
Back to top
View user's profile Send private message
mfyahya
Tux's lil' helper
Tux's lil' helper


Joined: 02 Feb 2006
Posts: 110

PostPosted: Tue Aug 15, 2006 7:16 am    Post subject: Reply with quote

looks cool. How about adding an option to append the version number to packages as well? Useful when one wants to do stuff like >=package-#.#.#
Back to top
View user's profile Send private message
erikm
l33t
l33t


Joined: 08 Feb 2005
Posts: 634

PostPosted: Tue Aug 15, 2006 7:54 am    Post subject: Reply with quote

Hi,

I have no intention to steal the glory of your fine perl work, but I find
Code:
~# emerge -uDNpv world | awk '$1 ~ /^\[ebuild/ {print "<desired prefix here>"gensub("\-[0-9]\.[0-9].*$","",1,$4)"<desired suffix here>" }'
a little niftier.
This also allows for simple addition of pre- and suffixes, as shown.

EDIT: It's even simpler if you wish to keep the version number:
Code:
~# emerge -uDNpv world | awk '$1 ~ /^\[ebuild/ {print "<desired prefix here>"$4"<desired suffix here>" }'
Back to top
View user's profile Send private message
ectospasm
l33t
l33t


Joined: 19 Feb 2003
Posts: 711
Location: Mobile, AL, USA

PostPosted: Tue Aug 15, 2006 10:54 am    Post subject: Reply with quote

mfyahya wrote:
looks cool. How about adding an option to append the version number to packages as well? Useful when one wants to do stuff like >=package-#.#.#


There's already the -V or --versions option, lemme know if it's not working for you. Or do you want a specific version that's not listed in emerge -vp output? I'm not sure that's feasible, since in most cases you won't want the exact same version for all packages in the list (unless you're merging something like KDE or GNOME).

If you can suggest a patch, that'd be great.
_________________
Join the adopt an unanswered post initiative today
Join the EFF!
Join the Drug Policy Alliance!
Back to top
View user's profile Send private message
ectospasm
l33t
l33t


Joined: 19 Feb 2003
Posts: 711
Location: Mobile, AL, USA

PostPosted: Tue Aug 15, 2006 11:00 am    Post subject: Reply with quote

erikm wrote:
Hi,

I have no intention to steal the glory of your fine perl work, but I find <awk pipeline> a little niftier.


It's been my experience that awk is more concise on the command line than Perl, but I've never figured out how to encapsulate awk into a script (of course, that's probably because I learned Perl long before I ever witnessed the power of awk). For something quick and dirty (quicker and dirtier than Perl even), I'd suggest awk. But again, I don't really know awk, so I wouldn't know how to recreate your pipeline from memory.

And I think in Perl now, so it's moot. (-:
_________________
Join the adopt an unanswered post initiative today
Join the EFF!
Join the Drug Policy Alliance!
Back to top
View user's profile Send private message
erikm
l33t
l33t


Joined: 08 Feb 2005
Posts: 634

PostPosted: Tue Aug 15, 2006 11:52 am    Post subject: Reply with quote

ectospasm wrote:
erikm wrote:
Hi,

I have no intention to steal the glory of your fine perl work, but I find <awk pipeline> a little niftier.


It's been my experience that awk is more concise on the command line than Perl, but I've never figured out how to encapsulate awk into a script (of course, that's probably because I learned Perl long before I ever witnessed the power of awk). For something quick and dirty (quicker and dirtier than Perl even), I'd suggest awk. But again, I don't really know awk, so I wouldn't know how to recreate your pipeline from memory.

And I think in Perl now, so it's moot. (-:

A pure awk script:
Code:
#!/bin/awk -f
<awk code>

Encapsulated in a bash script, for instance:
Code:
#!/bin/bash
<bash code>
awk '
<awk code>
'

I don't know perl, but I assume things wouldn't look much different if perl allows for calling external programs...?

EDIT: Oh, and while on the subject, my little pipe is trivial in terms of the power of awk; however, it nicely illustrates the amazing capabilities of regexps... ;)
Back to top
View user's profile Send private message
ectospasm
l33t
l33t


Joined: 19 Feb 2003
Posts: 711
Location: Mobile, AL, USA

PostPosted: Tue Aug 15, 2006 12:12 pm    Post subject: Reply with quote

erikm wrote:
EDIT: Oh, and while on the subject, my little pipe is trivial in terms of the power of awk; however, it nicely illustrates the amazing capabilities of regexps... ;)


regexps kickass. I use them heavily in Perl. I've heard the regexps in Perl are more powerful than in awk, but I have no experience to backup that claim.
_________________
Join the adopt an unanswered post initiative today
Join the EFF!
Join the Drug Policy Alliance!
Back to top
View user's profile Send private message
sageman
Guru
Guru


Joined: 04 May 2005
Posts: 363
Location: New Hampshire

PostPosted: Tue Aug 15, 2006 12:13 pm    Post subject: Reply with quote

I just always used sed. Just do:
Code:

emerge world -up > foo.log
cat foo.log | sed s/^\\[.*\\]\ //g > bar.log

or something like that. Strips out stuff in the [] before the package name. Woo, regex.
_________________
Carlton Stedman
Gentoo Metalheads on Last.fm: http://www.last.fm/group/Gentoo+Metalheads
Back to top
View user's profile Send private message
erikm
l33t
l33t


Joined: 08 Feb 2005
Posts: 634

PostPosted: Tue Aug 15, 2006 1:03 pm    Post subject: Reply with quote

ectospasm wrote:

regexps kickass. I use them heavily in Perl. I've heard the regexps in Perl are more powerful than in awk, but I have no experience to backup that claim.

I always thought they were a standard?
Back to top
View user's profile Send private message
sageman
Guru
Guru


Joined: 04 May 2005
Posts: 363
Location: New Hampshire

PostPosted: Tue Aug 15, 2006 1:13 pm    Post subject: Reply with quote

erikm wrote:
ectospasm wrote:

regexps kickass. I use them heavily in Perl. I've heard the regexps in Perl are more powerful than in awk, but I have no experience to backup that claim.

I always thought they were a standard?


No, there are a number of "standards" for regex. There are regex, and extended regex (which is a POSIX standard), and perl compatible regular expressions (pcre) and (probably) some more. Many GNU apps/tools, however, share some common method for regex, perhaps with a couple of permutations.

Perl does add a bit of functionality to the "typical" regex used in tools like sed and grep (course, grep -E or egrep adds extended regex functionality).

EDIT: Wikipedia explains a few different types of regex: http://en.wikipedia.org/wiki/Regular_expression#Syntax
_________________
Carlton Stedman
Gentoo Metalheads on Last.fm: http://www.last.fm/group/Gentoo+Metalheads
Back to top
View user's profile Send private message
erikm
l33t
l33t


Joined: 08 Feb 2005
Posts: 634

PostPosted: Tue Aug 15, 2006 1:25 pm    Post subject: Reply with quote

Ah, ok.

man awk wrote:
<...>
Regular Expressions
Regular expressions are the extended kind found in egrep.
<...>


Wikipedia wrote:
Perl has a richer and more predictable syntax than even the extended POSIX regexp.

Answers that question, I guess.
Back to top
View user's profile Send private message
sageman
Guru
Guru


Joined: 04 May 2005
Posts: 363
Location: New Hampshire

PostPosted: Tue Aug 15, 2006 3:18 pm    Post subject: Reply with quote

erikm wrote:
Ah, ok.

man awk wrote:
<...>
Regular Expressions
Regular expressions are the extended kind found in egrep.
<...>



Which should be the POSIX standard I beleive. I think all the GNU tools follow this POSIX standard (grep, sed, awk, et cetera).

Quote:

Wikipedia wrote:
Perl has a richer and more predictable syntax than even the extended POSIX regexp.

Answers that question, I guess.


Wikipedia to the rescue :) Seriously, isn't it about time that "wikipedia" got verbed like "google" has? I've been using it ("Just wikipedia that"). Woo for neo-colloquialisms.
_________________
Carlton Stedman
Gentoo Metalheads on Last.fm: http://www.last.fm/group/Gentoo+Metalheads
Back to top
View user's profile Send private message
ectospasm
l33t
l33t


Joined: 19 Feb 2003
Posts: 711
Location: Mobile, AL, USA

PostPosted: Tue Aug 15, 2006 5:35 pm    Post subject: Reply with quote

Quote:
and perl compatible regular expressions (pcre)


I'm very wary of PCRE, mainly because the last time I tried a PCRE in PHP, it was broken for my uses. What worked in Perl didn't work in PHP (unfortunately I can't cite an example now, since I'm no longer with that company and don't have access to my code). And IIRC the regexp I was using wasn't too complicated. Oh, well...
_________________
Join the adopt an unanswered post initiative today
Join the EFF!
Join the Drug Policy Alliance!
Back to top
View user's profile Send private message
LXj
Tux's lil' helper
Tux's lil' helper


Joined: 03 Dec 2005
Posts: 107

PostPosted: Tue Aug 15, 2006 8:32 pm    Post subject: Reply with quote

Hmm I think I posted similar scripts a couple of time ago... Here are the ones I use currently http://skill.ru/artwork/153608.shtml
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum