Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Need some help with sed
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
biggyL
Tux's lil' helper
Tux's lil' helper


Joined: 31 Jan 2005
Posts: 120
Location: Israel

PostPosted: Thu Jun 05, 2008 2:08 pm    Post subject: Need some help with sed Reply with quote

Hello All,

Could you please give me some quick help here?

I've got this file:
Code:

# cat /root/run-to-update.sh
#!/bin/bash
#relevant as for 06/05/08
/usr/bin/glsa-check -f 200805-17
    # emerge --ask --oneshot --verbose ">=dev-lang/perl-5.8.8-r5"
    # emerge --ask --oneshot --verbose ">=sys-devel/libperl-5.8.8-r2"


I need to get from it a list or a file with this content:
dev-lang/perl
sys-devel/libperl

or

perl
libperl

How can I do it with sed or may you have another more efficient tool/suggestion?
Back to top
View user's profile Send private message
cwr
Veteran
Veteran


Joined: 17 Dec 2005
Posts: 1969

PostPosted: Thu Jun 05, 2008 3:08 pm    Post subject: Reply with quote

You could do it all with sed, but I don't have a reference by me to check the exact
pattern match. The quick and dirty way is "grep verbose input.txt >output text",
and then use vim (or sed) to trim the lines in output.txt to the exact data required.

Will
Back to top
View user's profile Send private message
biggyL
Tux's lil' helper
Tux's lil' helper


Joined: 31 Jan 2005
Posts: 120
Location: Israel

PostPosted: Fri Jun 06, 2008 6:01 am    Post subject: Reply with quote

Sorry man I need this code to run inside a script to do it on the fly, so your solution is out of the question :(

Any other idea?
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3256

PostPosted: Fri Jun 06, 2008 6:42 am    Post subject: Reply with quote

Code:
sed -ne '/emerge --ask --oneshot --verbose/s,.*\">=\([^\"]*\)\",\1,p' | xargs qatom
and then interpret the output further :?:
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/
Back to top
View user's profile Send private message
think4urs11
Bodhisattva
Bodhisattva


Joined: 25 Jun 2003
Posts: 6659
Location: above the cloud

PostPosted: Fri Jun 06, 2008 7:48 am    Post subject: Reply with quote

Code:
me@box ~/test $ sed '/=/!d;s/.*=\(.*\)-[0-9]\{1,\}.*$/\1/' /root/run-to-update.sh
dev-lang/perl
sys-devel/libperl

Code:
me@box ~/test $ sed '/=/!d;s/.*=.*\/\(.*\)-[0-9]\{1,\}.*$/\1/'' /root/run-to-update.sh
perl
libperl

_________________
Nothing is secure / Security is always a trade-off with usability / Do not assume anything / Trust no-one, nothing / Paranoia is your friend / Think for yourself
Back to top
View user's profile Send private message
cwr
Veteran
Veteran


Joined: 17 Dec 2005
Posts: 1969

PostPosted: Fri Jun 06, 2008 12:45 pm    Post subject: Reply with quote

OK, try:

sed -e "s/^.*=//" -e "s/.[0-9].*//" infile > outfile

It's often easier, I find, to take two cuts at the more complicated expressions.

Will
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3256

PostPosted: Fri Jun 06, 2008 2:27 pm    Post subject: Reply with quote

Think4UrS11 wrote:
me@box ~/test $ sed '/=/!d;s/.*=\(.*\)-[0-9]\{1,\}.*$/\1/' /root/run-to-update.sh
Could you explain what the bolded portion does? Is this some kind of back reference? I typically don't grep everything before a number since it fails on packages like media-fonts/font-adobe-100dpi where there is a -[0-9] string in the package name itself. But your expression works really nice!

cwr: Your sed expression suffers from exactly the same problem as I have mentioned above. And that is why I recommended qatom which breaks up the package pattern into category, name, version and revision.
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/
Back to top
View user's profile Send private message
think4urs11
Bodhisattva
Bodhisattva


Joined: 25 Jun 2003
Posts: 6659
Location: above the cloud

PostPosted: Fri Jun 06, 2008 4:58 pm    Post subject: Reply with quote

ppurka wrote:
Think4UrS11 wrote:
me@box ~/test $ sed '/=/!d;s/.*=\(.*\)-[0-9]\{1,\}.*$/\1/' /root/run-to-update.sh
Could you explain what the bolded portion does? Is this some kind of back reference?

The general for is x\{m,n\}
means at least m and at most n times the occurance of x - in this case any digit, at least once till the end of the line.
and btw - it works also for media-fonts/font-adobe-100dpi ;)

It works because of the way the versioning is done, like pkg-ver{_suf{#}}{-r#}
see http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=2&chap=1#doc_chap2 for details

Every package has a version number attached and it cuts at the left-most possible postion where nothing else than a version number, revision number and probably some patchlevel (media-video/mplayer-1.0_rc2_p26753-r1) follows.
Version numbers are seperated by '.', revision is always -r.. and patchlevels are '_' so the leftmost position matching is after 100dpi as there is no further '-' right from that followed by a digit.

and yes, it takes some time to get used to this 'search from right to left but think vice versa' (or the other way around *g*) thingie
_________________
Nothing is secure / Security is always a trade-off with usability / Do not assume anything / Trust no-one, nothing / Paranoia is your friend / Think for yourself
Back to top
View user's profile Send private message
cwr
Veteran
Veteran


Joined: 17 Dec 2005
Posts: 1969

PostPosted: Fri Jun 06, 2008 5:59 pm    Post subject: Reply with quote

Yes, I hadn't realised there might be digits in the name. However, full sed parsing
isn't hard if you have the docs by you (I was trying to do it from memory) and so
on the whole I'd use think4urs11's solution. But like most scripts, it needs to be
developed from actual test cases.

Will
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3256

PostPosted: Fri Jun 06, 2008 11:53 pm    Post subject: Reply with quote

Think4UrS11 wrote:
and btw - it works also for media-fonts/font-adobe-100dpi ;)
Thank you for that excellent explanation! And yes, I had checked that media-fonts/font-adobe-100dpi worked fine with your sed expression. Then I realized that the only portion of your sed that I didn't understand was the portion I mentioned in my earlier post (and that was doing most of the magic!).
Because I could not come up with a decent sed expression, I resorted to using qatom in a script I was using. In fact, even qatom does not always give the correct results.

cwr: I think I know only a very small portion of what sed can do... I am still learning when the occasion arises :D
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/
Back to top
View user's profile Send private message
biggyL
Tux's lil' helper
Tux's lil' helper


Joined: 31 Jan 2005
Posts: 120
Location: Israel

PostPosted: Sun Jun 08, 2008 10:41 am    Post subject: Reply with quote

Think4UrS11

Thank you very much!
It worked like a charm :)
Back to top
View user's profile Send private message
truc
Advocate
Advocate


Joined: 25 Jul 2005
Posts: 3199

PostPosted: Sun Jun 08, 2008 8:16 pm    Post subject: Reply with quote

Think4UrS11 wrote:
ppurka wrote:
Think4UrS11 wrote:
me@box ~/test $ sed '/=/!d;s/.*=\(.*\)-[0-9]\{1,\}.*$/\1/' /root/run-to-update.sh
Could you explain what the bolded portion does? Is this some kind of back reference?

The general for is x\{m,n\}


You can also replace \{1,\} with \+
_________________
The End of the Internet!
Back to top
View user's profile Send private message
biggyL
Tux's lil' helper
Tux's lil' helper


Joined: 31 Jan 2005
Posts: 120
Location: Israel

PostPosted: Tue Jun 17, 2008 9:41 am    Post subject: Reply with quote

Sorry to bother you again but it seems that I need additional help :)

running
# glsa-check -p $(glsa-check -t all)
I got this output
Code:

This system is affected by the following GLSAs:
Checking GLSA 200801-09
The following updates will be performed for this GLSA:
     x11-libs/libXfont-1.3.1-r1 (1.3.0)
     x11-base/xorg-server-1.3.0.0-r5 (1.3.0.0-r2)

Checking GLSA 200801-20
The following updates will be performed for this GLSA:
     dev-libs/libxml2-2.6.30-r1 (2.6.28)

Checking GLSA 200803-19
The following updates will be performed for this GLSA:
     www-servers/apache-2.2.8 (2.2.6)

Checking GLSA 200803-24
The following updates will be performed for this GLSA:
     dev-libs/libpcre-7.6-r1 (6.6)

Checking GLSA 200803-30
The following updates will be performed for this GLSA:
     mail-mta/postfix-2.3.8-r1 (2.3.6)
     net-mail/cyrus-imapd-2.3.9-r1 (2.2.13-r1)

Checking GLSA 200711-02
The following updates will be performed for this GLSA:
     net-misc/openssh-4.7_p1-r6 (4.6_p1-r3)

Checking GLSA 200711-07
The following updates will be performed for this GLSA:
     dev-lang/python-2.4.4-r6 (2.4.4-r4)



What I need is some sed expression to extract all package names (net-misc/openssh net-misc/openssh .....) from it
So I could run quickpkg on those packages.

Something like:
quickpkg `sed '/expression/' `glsa-check -p $(glsa-check -t all)``

Any suggestions/help would be welcomed :)
Back to top
View user's profile Send private message
truc
Advocate
Advocate


Joined: 25 Jul 2005
Posts: 3199

PostPosted: Tue Jun 17, 2008 10:18 am    Post subject: Reply with quote

keeping the same idea as above, this might work (just tested with your sample output)

Code:
sed -n '/^[[:space:]]\+/s/[[:space:]]\(.*\)-[0-9]\{1,\}.*$/\1/p'


then something like:
Code:
quickpkg $(glsa-check -p $(glsa-check -t all) | sed -n '/^[[:space:]]\+/s/[[:space:]]\(.*\)-[0-9]\{1,\}.*$/\1/p')

_________________
The End of the Internet!
Back to top
View user's profile Send private message
cwr
Veteran
Veteran


Joined: 17 Dec 2005
Posts: 1969

PostPosted: Tue Jun 17, 2008 10:43 am    Post subject: Reply with quote

Or feed the file through grep to remove the lines you don't want?
(grep -v pattern). It looks as if most of the unwanted lines start
the same way.

Will
Back to top
View user's profile Send private message
truc
Advocate
Advocate


Joined: 25 Jul 2005
Posts: 3199

PostPosted: Tue Jun 17, 2008 11:43 am    Post subject: Reply with quote

cwr wrote:
...


But he also want to remove the version-revision to the package name, and the (old-version-revision) part
_________________
The End of the Internet!
Back to top
View user's profile Send private message
biggyL
Tux's lil' helper
Tux's lil' helper


Joined: 31 Jan 2005
Posts: 120
Location: Israel

PostPosted: Wed Jun 18, 2008 6:44 am    Post subject: Reply with quote

truc wrote:
keeping the same idea as above, this might work (just tested with your sample output)

Code:
sed -n '/^[[:space:]]\+/s/[[:space:]]\(.*\)-[0-9]\{1,\}.*$/\1/p'


then something like:
Code:
quickpkg $(glsa-check -p $(glsa-check -t all) | sed -n '/^[[:space:]]\+/s/[[:space:]]\(.*\)-[0-9]\{1,\}.*$/\1/p')


Thanks truc, thats nice :). But there is this line "This system is affected by the following GLSAs:" that I should rid off. Ah, and also these " " spaces in front of the package names.
What's the needed change in sed exp. to accomplish that, please?

Code:

# glsa-check -p $(glsa-check -t all) | sed -n '/^[[:space:]]\+/s/[[:space:]]\(.*\)-[0-9]\{1,\}.*$/\1/p'
This system is affected by the following GLSAs:
    x11-libs/libXfont
    x11-base/xorg-server
    dev-libs/libxml2
    www-servers/apache
    dev-libs/libpcre
    mail-mta/postfix
    net-mail/cyrus-imapd
    net-misc/openssh
    dev-lang/python
    media-libs/libpng
    app-arch/cpio
    dev-db/mysql
    dev-lang/perl
    dev-libs/libpcre
    sys-devel/libperl
    dev-lang/perl
    app-antivirus/clamav
    dev-libs/openssl
    sys-apps/util-linux
    dev-libs/openssl
    app-arch/bzip2
    net-misc/openssh
    dev-db/mysql
    app-arch/unzip
    media-libs/libpng
    net-misc/rsync
    app-admin/php-toolkit
    sys-apps/portage
    sys-fs/e2fsprogs
    app-admin/syslog-ng
    dev-libs/libxslt
Back to top
View user's profile Send private message
think4urs11
Bodhisattva
Bodhisattva


Joined: 25 Jun 2003
Posts: 6659
Location: above the cloud

PostPosted: Wed Jun 18, 2008 7:23 am    Post subject: Reply with quote

biggyL wrote:
But there is this line "This system is affected by the following GLSAs:" that I should rid off. Ah, and also these " " spaces in front of the package names.
What's the needed change in sed exp. to accomplish that, please?

Without looking for other optimizations
Code:
glsa-check -p $(glsa-check -t all) | sed -n '/^[[:space:]]\+/s/[[:space:]]\(.*\)-[0-9]\{1,\}.*$/\1/;s/^\s*//;1d'

_________________
Nothing is secure / Security is always a trade-off with usability / Do not assume anything / Trust no-one, nothing / Paranoia is your friend / Think for yourself
Back to top
View user's profile Send private message
biggyL
Tux's lil' helper
Tux's lil' helper


Joined: 31 Jan 2005
Posts: 120
Location: Israel

PostPosted: Wed Jun 18, 2008 7:35 am    Post subject: Reply with quote

Thanks Think4UrS11

But I get:
Code:

# glsa-check -p $(glsa-check -t all) | sed -n '/^[[:space:]]\+/s/[[:space:]]\(.*\)-[0-9]\{1,\}.*$/\1/;s/^\s*//;1d'
This system is affected by the following GLSAs:


When I need to get only:
x11-libs/libXfont
x11-base/xorg-server
dev-libs/libxml2
www-servers/apache
Back to top
View user's profile Send private message
truc
Advocate
Advocate


Joined: 25 Jul 2005
Posts: 3199

PostPosted: Wed Jun 18, 2008 8:01 am    Post subject: Reply with quote

keeping the Think4UrS11 idea, this might work
Code:
glsa-check -p $(glsa-check -t all) | sed -n '1b;/^[[:space:]]\+/s/[[:space:]]*\(.*\)-[0-9]\{1,\}.*$/\1/p'

_________________
The End of the Internet!
Back to top
View user's profile Send private message
biggyL
Tux's lil' helper
Tux's lil' helper


Joined: 31 Jan 2005
Posts: 120
Location: Israel

PostPosted: Wed Jun 18, 2008 8:09 am    Post subject: Reply with quote

truc wrote:
keeping the Think4UrS11 idea, this might work
Code:
glsa-check -p $(glsa-check -t all) | sed -n '1b;/^[[:space:]]\+/s/[[:space:]]*\(.*\)-[0-9]\{1,\}.*$/\1/p'


Thank you guys, I almost there:

Code:

# glsa-check -p $(glsa-check -t all) | sed -n '1b;/^[[:space:]]\+/s/[[:space:]]*\(.*\)-[0-9]\{1,\}.*$/\1/p'
This system is affected by the following GLSAs:
x11-libs/libXfont
x11-base/xorg-server
dev-libs/libxml2
www-servers/apache
dev-libs/libpcre
mail-mta/postfix
net-mail/cyrus-imapd
net-misc/openssh
dev-lang/python
media-libs/libpng
app-arch/cpio
dev-db/mysql
dev-lang/perl
dev-libs/libpcre
sys-devel/libperl
dev-lang/perl
app-antivirus/clamav
dev-libs/openssl
sys-apps/util-linux


The only thing I need is to get rid of the first line "This system is affected by the following GLSAs:"
Any suggestions?
Back to top
View user's profile Send private message
truc
Advocate
Advocate


Joined: 25 Jul 2005
Posts: 3199

PostPosted: Wed Jun 18, 2008 8:24 am    Post subject: Reply with quote

wild guess: this line is probably echoed in stderr and not stdout..

try this:
Code:
glsa-check -p $(glsa-check -t all) 2>/dev/null | sed -n '/^[[:space:]]\+/s/[[:space:]]*\(.*\)-[0-9]\{1,\}.*$/\1/p'

_________________
The End of the Internet!
Back to top
View user's profile Send private message
biggyL
Tux's lil' helper
Tux's lil' helper


Joined: 31 Jan 2005
Posts: 120
Location: Israel

PostPosted: Wed Jun 18, 2008 8:51 am    Post subject: Reply with quote

truc wrote:
wild guess: this line is probably echoed in stderr and not stdout..

try this:
Code:
glsa-check -p $(glsa-check -t all) 2>/dev/null | sed -n '/^[[:space:]]\+/s/[[:space:]]*\(.*\)-[0-9]\{1,\}.*$/\1/p'


I got your idea toredirect stderr to /dev/null but this won't help :(

I've also tried this:
Code:

# glsa-check -p $(glsa-check -t all) | sed -n '1b;/^[[:space:]]\+/s/[[:space:]]*\(.*\)-[0-9]\{1,\}.*$/\1/p' | tail --lines=+1
This system is affected by the following GLSAs:
www-servers/apache
net-misc/rsync
media-libs/libpng
x11-libs/libXfont
dev-libs/libxml2
dev-db/mysql
dev-libs/libpcre
net-misc/rsync
app-arch/tar
net-misc/openssh
dev-libs/glib
dev-libs/libpcre
media-libs/libpng
dev-libs/libxslt
media-libs/t1lib
dev-lang/python
app-arch/cpio
app-admin/php-toolkit
app-arch/bzip2
sys-fs/e2fsprogs
sys-devel/libperl
dev-lang/perl
dev-db/mysql
dev-lang/perl
net-misc/openssh
app-admin/syslog-ng
sys-apps/util-linux
web log # glsa-check -p $(glsa-check -t all) | sed -n '1b;/^[[:space:]]\+/s/[[:space:]]*\(.*\)-[0-9]\{1,\}.*$/\1/p' | tail --lines=+2
This system is affected by the following GLSAs:
net-misc/rsync
media-libs/libpng
x11-libs/libXfont
dev-libs/libxml2
dev-db/mysql
dev-libs/libpcre
net-misc/rsync
app-arch/tar
net-misc/openssh
dev-libs/glib
dev-libs/libpcre
media-libs/libpng
dev-libs/libxslt
media-libs/t1lib
dev-lang/python
app-arch/cpio
app-admin/php-toolkit
app-arch/bzip2
sys-fs/e2fsprogs
sys-devel/libperl
dev-lang/perl
dev-db/mysql
dev-lang/perl
net-misc/openssh
app-admin/syslog-ng
sys-apps/util-linux


As you can see, I get the same result, with this freaking "This system is affected by the following GLSAs:" line.
Back to top
View user's profile Send private message
truc
Advocate
Advocate


Joined: 25 Jul 2005
Posts: 3199

PostPosted: Wed Jun 18, 2008 9:09 am    Post subject: Reply with quote

just play with glsa until you find how to hide this line! :p

Code:
glsa-check -p $(glsa-check -t all) 2>/dev/null

or
Code:
glsa-check -p $(glsa-check -t all 2>/dev/null)


something else?
_________________
The End of the Internet!
Back to top
View user's profile Send private message
biggyL
Tux's lil' helper
Tux's lil' helper


Joined: 31 Jan 2005
Posts: 120
Location: Israel

PostPosted: Wed Jun 18, 2008 9:56 am    Post subject: Reply with quote

truc wrote:
just play with glsa until you find how to hide this line! :p

Code:
glsa-check -p $(glsa-check -t all) 2>/dev/null

or
Code:
glsa-check -p $(glsa-check -t all 2>/dev/null)


something else?


Code:

# glsa-check -p $(glsa-check -t all) 2>/dev/null
This system is affected by the following GLSAs:
Checking GLSA 200801-09
The following updates will be performed for this GLSA:
     x11-libs/libXfont-1.3.1-r1 (1.3.0)
     x11-base/xorg-server-1.3.0.0-r5 (1.3.0.0-r2)

Checking GLSA 200801-20
The following updates will be performed for this GLSA:
     dev-libs/libxml2-2.6.30-r1 (2.6.28)

Checking GLSA 200803-19
The following updates will be performed for this GLSA:
     www-servers/apache-2.2.8 (2.2.6)

Checking GLSA 200803-24
The following updates will be performed for this GLSA:
     dev-libs/libpcre-7.6-r1 (6.6)

Checking GLSA 200803-30
The following updates will be performed for this GLSA:
     mail-mta/postfix-2.3.8-r1 (2.3.6)
     net-mail/cyrus-imapd-2.3.9-r1 (2.2.13-r1)


or

Code:

# glsa-check -p $(glsa-check -t all 2>/dev/null)
Checking GLSA 200801-09
The following updates will be performed for this GLSA:
     x11-libs/libXfont-1.3.1-r1 (1.3.0)
     x11-base/xorg-server-1.3.0.0-r5 (1.3.0.0-r2)

Checking GLSA 200801-20
The following updates will be performed for this GLSA:
     dev-libs/libxml2-2.6.30-r1 (2.6.28)

Checking GLSA 200803-19
The following updates will be performed for this GLSA:
     www-servers/apache-2.2.8 (2.2.6)

Checking GLSA 200803-24
The following updates will be performed for this GLSA:
     dev-libs/libpcre-7.6-r1 (6.6)

Checking GLSA 200803-30
The following updates will be performed for this GLSA:
     mail-mta/postfix-2.3.8-r1 (2.3.6)
     net-mail/cyrus-imapd-2.3.9-r1 (2.2.13-r1)
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum