Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
TIP Strip comments when posting config files
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
tomk
Bodhisattva
Bodhisattva


Joined: 23 Sep 2003
Posts: 7221
Location: Sat in front of my computer

PostPosted: Sun Apr 11, 2004 2:22 pm    Post subject: TIP Strip comments when posting config files Reply with quote

I got tired of searching through long config files which have loads of comments looking for some option I'd set. It also annoys me when someone posts their huge config files which are mainly comments, not only does it take up valuable database space it also makes it harder to see what the problem is.

So I wrote a small script called confcat which will just show you the important parts of your config files.

Code:
#!/bin/bash
# Code to cat a config file removing all comments and blank lines.
 
grep -vh '^[[:space:]]*\(#\|$\)' "$@"


Save this to a file called confcat in your $PATH (It's in ~/bin on my box), make it executable:

Code:
chmod 755 /path/to/confcat


Then use it like cat:

Code:
confcat /path/to/configfile


Basically it strips and lines beginning with a hash '#' and any empty lines.

I know it doesn't look much, but I ran a few tests to see how many lines are outputted:

Code:
$ confcat /etc/make.conf | wc -l
      9
$ cat /etc/make.conf | wc -l
    156
$ confcat /etc/X11/XF86Config | wc -l
     82
$ cat /etc/X11/XF86Config | wc -l
    462
$ confcat /etc/mutt/Muttrc | wc -l
     19
$ cat /etc/mutt/Muttrc | wc -l
   3690
$ confcat /usr/src/linux/.config | wc -l
    200
$ cat /usr/src/linux/.config | wc -l
   1119


And an example of it in action:

Code:
$ confcat /etc/make.conf
USE="-* x86 crypt berkdb pam ncurses readline ssl tcltk tcpd X mmx xml2 \
truetype java gtk jpeg tiff png python 3dfx voodoo3 opengl imap apache2 \
postgres sasl"
CHOST="i686-pc-linux-gnu"
CFLAGS="-O3 -march=pentium3 -fprefetch-loop-arrays -funroll-loops -pipe"
CXXFLAGS="${CFLAGS}"
PORTDIR_OVERLAY="/usr/local/portage"
GENTOO_MIRRORS="http://www.mirror.ac.uk/sites/www.ibiblio.org/gentoo/"
SYNC="rsync://rsync.uk.gentoo.org/gentoo-portage"


Hopefully someone will find it useful.

Edit: made it so it includes comments that appear after whitespace at the beginning of the line, lines that only have whitespace and so that it only uses one grep process
_________________
Search | Read | Answer | Report | Strip


Last edited by tomk on Fri Nov 24, 2006 11:59 pm; edited 2 times in total
Back to top
View user's profile Send private message
ikaro
Advocate
Advocate


Joined: 14 Jul 2003
Posts: 2527
Location: Denmark

PostPosted: Sun Apr 11, 2004 2:53 pm    Post subject: Reply with quote

Cool tip.
I had once something similar for vim.
Thanks.
_________________
linux: #232767
Back to top
View user's profile Send private message
Earthwings
Bodhisattva
Bodhisattva


Joined: 14 Apr 2003
Posts: 7753
Location: Germany

PostPosted: Sun Apr 11, 2004 3:27 pm    Post subject: Reply with quote

Nice. Extending it to strip whitespace leaded comments saves me another 70 lines for XF86Config.
Code:

sed -e '/^\(\t\| \)*#.*\|^\(\t\| \)*$/d' -e 's/\(.*\)#.*/\1/' "$@"

This also strips comments at the end of a line, though it looks a little bit ugly :o
Back to top
View user's profile Send private message
tomk
Bodhisattva
Bodhisattva


Joined: 23 Sep 2003
Posts: 7221
Location: Sat in front of my computer

PostPosted: Sun Apr 11, 2004 8:19 pm    Post subject: Reply with quote

Earthwings wrote:
Extending it to strip whitespace leaded comments saves me another 70 lines for XF86Config.


Good idea. I've obviously got some more sed to learn, here's how to do it with grep.

Code:
#!/bin/bash
# Code to cat a config file removing all comments and blank lines.
 
grep -vh '^[[:space:]]*#' "$@" | grep -v '^$'

_________________
Search | Read | Answer | Report | Strip
Back to top
View user's profile Send private message
agmoe
n00b
n00b


Joined: 01 Apr 2003
Posts: 8
Location: Sweden

PostPosted: Mon Apr 12, 2004 1:07 am    Post subject: Reply with quote

Nice, I've been thinking of something like this for a while after going trough some XF86Configs generated with fglrxconfig, containing 95% comments, I've been to lazy to try to do anything myself though :)
_________________
God apa gavs galna anlag svag apa dog
Back to top
View user's profile Send private message
water
Guru
Guru


Joined: 19 Jun 2002
Posts: 387
Location: Zierikzee, The Netherlands

PostPosted: Tue Apr 13, 2004 9:24 am    Post subject: Reply with quote

I've seen something before at this subforum, but it's a good thing.
_________________
Groeten uit Holland
Back to top
View user's profile Send private message
Boohbah
Apprentice
Apprentice


Joined: 17 Oct 2003
Posts: 250
Location: Seattle

PostPosted: Tue Apr 13, 2004 9:37 am    Post subject: Reply with quote

I do the same thing with a perl script i whipped up one night to take the comments out of my kernel .config.
Code:
#!/usr/bin/perl
while(<>) {
        print unless m/^#/ or m/^\s+/;
}

EDIT:
Mine's not as cool as yours though as it only removes comments at the beginning of the line :oops:
_________________
Never try to explain computers to a layman. It's easier to explain sex to a virgin.
-- Robert Heinlein

(Note, however, that virgins tend to know a lot about computers.)
Back to top
View user's profile Send private message
ett_gramse_nap
Apprentice
Apprentice


Joined: 01 Oct 2003
Posts: 252
Location: Göteborg, Sweden

PostPosted: Tue Apr 13, 2004 10:20 am    Post subject: Reply with quote

Thanks! I've thought of this at least a douzen of times but never managed to do something about it...
_________________
Don't bother!
Back to top
View user's profile Send private message
Boohbah
Apprentice
Apprentice


Joined: 17 Oct 2003
Posts: 250
Location: Seattle

PostPosted: Wed Apr 14, 2004 12:15 am    Post subject: Reply with quote

While my script works fine for kernel configs, it doesn't work for X configs because i foolishly remove every line that starts with whitespace :oops: Here is my new and improved version that works by anchoring the whitespace to the whole line:
Code:
#!/usr/bin/perl       
while(<>) { print unless m/^#/ or m/^\s+$/; }

Now i feel better :wink:
_________________
Never try to explain computers to a layman. It's easier to explain sex to a virgin.
-- Robert Heinlein

(Note, however, that virgins tend to know a lot about computers.)
Back to top
View user's profile Send private message
Boohbah
Apprentice
Apprentice


Joined: 17 Oct 2003
Posts: 250
Location: Seattle

PostPosted: Wed Apr 14, 2004 12:20 am    Post subject: Reply with quote

In further testing i realized my script doesn't catch lines beginning with whitespace before a comment, so here is my newer and even more improved version.
Code:
#!/usr/bin/perl
while(<>) { print unless m/^\s*#/ or m/^\s+$/; }

Yes, i am a perl n00b :lol:
_________________
Never try to explain computers to a layman. It's easier to explain sex to a virgin.
-- Robert Heinlein

(Note, however, that virgins tend to know a lot about computers.)
Back to top
View user's profile Send private message
ecatmur
Advocate
Advocate


Joined: 20 Oct 2003
Posts: 3595
Location: Edinburgh

PostPosted: Wed Apr 14, 2004 1:36 am    Post subject: Reply with quote

A shorter sed version:
Code:
#!/bin/sed -f
s/#.*//
/^\s*$/d
or
Code:
sed -e 's/#.*//;/^\s*$/d' "$@"

_________________
No more cruft
dep: Revdeps that work
Using command-line ACCEPT_KEYWORDS?
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9503
Location: beyond the rim

PostPosted: Sun May 02, 2004 1:52 pm    Post subject: Reply with quote

Nice thing, but for make.conf I still recommend `emerge --info`.
Back to top
View user's profile Send private message
Shan
Guru
Guru


Joined: 04 Nov 2003
Posts: 558
Location: /dev/null

PostPosted: Sat Jun 26, 2004 2:27 am    Post subject: Reply with quote

Not meaning to wake up the dead here, but wouldn't it just be easier to put this as an alias in your /etc/profile ? eg
Code:
alias confcat="sed -e 's/#.*//;/^\s*$/d' "$@""

and you can still run it the same way as if you'd put it in a script and chmodded it and all, but it just saves you from having to make another executable file.
_________________
{ NO -U } { STRIP }
{ TINY }
Back to top
View user's profile Send private message
Jazz
Guru
Guru


Joined: 16 Nov 2003
Posts: 543
Location: Melbourne, Australia

PostPosted: Sat Jun 26, 2004 4:34 am    Post subject: Reply with quote

emerge info sux, it also shows things that arent even present in the make.conf, that creates a lot of confusion
_________________
In 2010, M$ Windows will be a quantum processing emulation layer for a 128-bit mod of a 64-bit hack of a 32-bit patch to a 16-bit GUI for an 8-bit operating system written for a 4-bit processor from a 2-bit company that can't stand 1 bit of competition.
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9503
Location: beyond the rim

PostPosted: Sat Jun 26, 2004 2:38 pm    Post subject: Reply with quote

Jazz wrote:
emerge info sux, it also shows things that arent even present in the make.conf, that creates a lot of confusion

Well, we depend on that information for solving bugs (things like kernel version, used profile, gcc/libc versions, ...)
Back to top
View user's profile Send private message
Shan
Guru
Guru


Joined: 04 Nov 2003
Posts: 558
Location: /dev/null

PostPosted: Sun Jun 27, 2004 11:23 pm    Post subject: AR Reply with quote

Jazz wrote:
emerge info sux, it also shows things that arent even present in the make.conf, that creates a lot of confusion


That may be so, but those things that are added are needed for proper debugging / help purposes. For example, it shows your currently running kernel, installed gcc and glibc, as well as automake / autoconf and your baselayout. Tell me how those are "bad" things that are shown?

For instance Not every user "knows" that certain versions of the (now old) KDE3.2 betas had trouble with the ARCH version of autoconf or automake, so when they posted questions, they get a faster answer if they include the output from emerge --info, which shows what version they're using, and thus allows the person replying to simply say "Upgrade auto* to ~ARCH"

And for the record, I answerd my own earlier post asking if it could be used as an alias. It can, and works just the same (atleast with the one ecatmur posted)
_________________
{ NO -U } { STRIP }
{ TINY }
Back to top
View user's profile Send private message
tomk
Bodhisattva
Bodhisattva


Joined: 23 Sep 2003
Posts: 7221
Location: Sat in front of my computer

PostPosted: Mon Jun 28, 2004 11:23 am    Post subject: Reply with quote

Jazz wrote:
emerge info sux, it also shows things that arent even present in the make.conf, that creates a lot of confusion


Most of the things that emerge --info shows which aren't present in make.conf are defaults which are tucked away in other files on your box. emerge --info gives you an easy way to get all this information about your system in one simple command.

Shan wrote:
And for the record, I answerd my own earlier post asking if it could be used as an alias. It can, and works just the same (atleast with the one ecatmur posted)


This works just as well, and the sed version seems to be quicker than the grep version. As I said before I really need to learn more sed.
_________________
Search | Read | Answer | Report | Strip
Back to top
View user's profile Send private message
UberLord
Retired Dev
Retired Dev


Joined: 18 Sep 2003
Posts: 6835
Location: Blighty

PostPosted: Mon Jun 28, 2004 11:49 am    Post subject: Reply with quote

Nice tip :)
_________________
Use dhcpcd for all your automated network configuration needs
Use dhcpcd-ui (GTK+/Qt) as your System Tray Network tool
Back to top
View user's profile Send private message
syn_ack
n00b
n00b


Joined: 26 Jan 2004
Posts: 31

PostPosted: Mon Jul 12, 2004 7:25 am    Post subject: Reply with quote

ecatmur wrote:
A shorter sed version:
Code:
#!/bin/sed -f
s/#.*//
/^\s*$/d
or
Code:
sed -e 's/#.*//;/^\s*$/d' "$@"


I've recently started learning/studying SED and GREP thanks to this thread. I'm unsure if this is correct or not but doesn't the following basically accomplish stripping out "commented lines" as well as "blank ones" in a shorter amount of key strokes? Please let me know what I'm missing here.
Code:


cat .config | sed '/^#/ d' | sed '/^$/ d'

cat .config | sed -e '/^#/ d' -e '/^$/ d'

cat .config | grep  '.' | grep -v '#'

cat .config | grep -v '#' | grep  '[^$]'


And ecatmur, I would just like to say thanks for the awesome work on the "dep script" here. You have seriously motivated me to start learning the power of the BASH.
There have been so many things that I would like to fix or (actually), make better for myself and possibly the Gentoo community. But things seem to all of the sudden start becoming understandable for what ever reason. Your inspirational and make Linux truely that much more fun.

I feel like I'm starting to break some wild horse.. :D Not there yet. But I know if I stick with it and have good teachers like you guys/gals I'll get there.. Trying to stay focused in one Linux area over another seems to be my biggest problem. Kid in the candy store effect.
Back to top
View user's profile Send private message
hardcore
l33t
l33t


Joined: 01 Nov 2003
Posts: 626
Location: MSU, MI

PostPosted: Wed Feb 16, 2005 3:23 am    Post subject: Reply with quote

This is just what i was looking for. Thanks :)
_________________
Nothing can stop me now, cuz I just don't care.
Back to top
View user's profile Send private message
randolph
n00b
n00b


Joined: 16 Mar 2005
Posts: 10
Location: Berlin

PostPosted: Wed Jun 29, 2005 7:20 am    Post subject: Reply with quote

syn_ack wrote:

And ecatmur, I would just like to say thanks for the awesome work on the "dep script" here. You have seriously motivated me to start learning the power of the BASH....
...You're inspirational and make Linux truely much more fun....


I would like to step in line to this. As I am a noob "chmod 755" gave me just the tip I needed. Not knowing how to execute a script. And the rest speeks for it self.

Thank you very much ecatmur
Back to top
View user's profile Send private message
x1jmp
n00b
n00b


Joined: 13 Jun 2005
Posts: 28
Location: March-Hugstetten, Germany

PostPosted: Mon Aug 22, 2005 1:22 pm    Post subject: Reply with quote

Thanks for this tip with sed!

My version is this, which also deletes lines with ";", like the smb.conf
Code:
sed -e 's/[#;].*//;/^\s*$/d'

It's also possible to add exactly this into the .bashrc as an alias.
Back to top
View user's profile Send private message
msalerno
Veteran
Veteran


Joined: 17 Dec 2002
Posts: 1338
Location: Sweating in South Florida

PostPosted: Fri Aug 26, 2005 8:55 pm    Post subject: Reply with quote

A perl update to the one above:

Code:
perl -pne 's/^\s*([#;].*|\s*)\n$//' <filename>
Back to top
View user's profile Send private message
dundas
Guru
Guru


Joined: 16 Dec 2004
Posts: 317
Location: China, Earth

PostPosted: Tue Jan 31, 2006 12:11 pm    Post subject: Reply with quote

cool, will do, thx tomk.
_________________
Appreciate Gentoo: Best Devs, Best Forums. YOU could help too: Help Answer
Back to top
View user's profile Send private message
nanafunk
n00b
n00b


Joined: 29 Jun 2005
Posts: 36

PostPosted: Wed Feb 01, 2006 8:55 am    Post subject: Reply with quote

some of you might find this url useful :)

Useless Use of Cat (cat foo | grep bar). See http://www.ruhr.de/home/smallo/award.html
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum