TIP Strip comments when posting config files

[tomk]

I got tired of searching through long config files which have loads of comments looking for some option I'd set. It also annoys me when someone posts their huge config files which are mainly comments, not only does it take up valuable database space it also makes it harder to see what the problem is.

So I wrote a small script called confcat which will just show you the important parts of your config files.

Code: Select all

#!/bin/bash
# Code to cat a config file removing all comments and blank lines.
 
grep -vh '^[[:space:]]*\(#\|$\)' "$@"

Save this to a file called confcat in your $PATH (It's in ~/bin on my box), make it executable:

Code: Select all

chmod 755 /path/to/confcat

Then use it like cat:

Code: Select all

confcat /path/to/configfile

Basically it strips and lines beginning with a hash '#' and any empty lines.

I know it doesn't look much, but I ran a few tests to see how many lines are outputted:

Code: Select all

$ confcat /etc/make.conf | wc -l
      9
$ cat /etc/make.conf | wc -l
    156
$ confcat /etc/X11/XF86Config | wc -l 
     82
$ cat /etc/X11/XF86Config | wc -l
    462
$ confcat /etc/mutt/Muttrc | wc -l
     19
$ cat /etc/mutt/Muttrc | wc -l
   3690
$ confcat /usr/src/linux/.config | wc -l
    200
$ cat /usr/src/linux/.config | wc -l
   1119

And an example of it in action:

Code: Select all

$ confcat /etc/make.conf
USE="-* x86 crypt berkdb pam ncurses readline ssl tcltk tcpd X mmx xml2 \
truetype java gtk jpeg tiff png python 3dfx voodoo3 opengl imap apache2 \
postgres sasl"
CHOST="i686-pc-linux-gnu"
CFLAGS="-O3 -march=pentium3 -fprefetch-loop-arrays -funroll-loops -pipe"
CXXFLAGS="${CFLAGS}"
PORTDIR_OVERLAY="/usr/local/portage"
GENTOO_MIRRORS="http://www.mirror.ac.uk/sites/www.ibiblio.org/gentoo/"
SYNC="rsync://rsync.uk.gentoo.org/gentoo-portage"

Hopefully someone will find it useful.

Edit: made it so it includes comments that appear after whitespace at the beginning of the line, lines that only have whitespace and so that it only uses one grep process

[ikaro]

Cool tip.
I had once something similar for vim.
Thanks.

[Earthwings]

Nice. Extending it to strip whitespace leaded comments saves me another 70 lines for XF86Config.

Code: Select all

sed -e '/^\(\t\| \)*#.*\|^\(\t\| \)*$/d' -e 's/\(.*\)#.*/\1/' "$@"

This also strips comments at the end of a line, though it looks a little bit ugly

[tomk]

Extending it to strip whitespace leaded comments saves me another 70 lines for XF86Config.

Good idea. I've obviously got some more sed to learn, here's how to do it with grep.

Code: Select all

#!/bin/bash
# Code to cat a config file removing all comments and blank lines.
 
grep -vh '^[[:space:]]*#' "$@" | grep -v '^$' 

[agmoe]

Nice, I've been thinking of something like this for a while after going trough some XF86Configs generated with fglrxconfig, containing 95% comments, I've been to lazy to try to do anything myself though

[water]

I've seen something before at this subforum, but it's a good thing.

[Boohbah]

I do the same thing with a perl script i whipped up one night to take the comments out of my kernel .config.

Code: Select all

#!/usr/bin/perl
while(<>) {
        print unless m/^#/ or m/^\s+/;
}

EDIT:
Mine's not as cool as yours though as it only removes comments at the beginning of the line

[ett_gramse_nap]

Thanks! I've thought of this at least a douzen of times but never managed to do something about it...

[Boohbah]

While my script works fine for kernel configs, it doesn't work for X configs because i foolishly remove every line that starts with whitespace Here is my new and improved version that works by anchoring the whitespace to the whole line:

Code: Select all

#!/usr/bin/perl       
while(<>) { print unless m/^#/ or m/^\s+$/; }

Now i feel better

[Boohbah]

In further testing i realized my script doesn't catch lines beginning with whitespace before a comment, so here is my newer and even more improved version.

Code: Select all

#!/usr/bin/perl
while(<>) { print unless m/^\s*#/ or m/^\s+$/; }

Yes, i am a perl n00b

[ecatmur]

A shorter sed version:

Code: Select all

#!/bin/sed -f
s/#.*//
/^\s*$/d

or

Code: Select all

sed -e 's/#.*//;/^\s*$/d' "$@"

[Genone]

Nice thing, but for make.conf I still recommend `emerge --info`.

[Shan]

Not meaning to wake up the dead here, but wouldn't it just be easier to put this as an alias in your /etc/profile ? eg

Code: Select all

alias confcat="sed -e 's/#.*//;/^\s*$/d' "$@""

and you can still run it the same way as if you'd put it in a script and chmodded it and all, but it just saves you from having to make another executable file.

[Jazz]

emerge info sux, it also shows things that arent even present in the make.conf, that creates a lot of confusion

[Genone]

emerge info sux, it also shows things that arent even present in the make.conf, that creates a lot of confusion

Well, we depend on that information for solving bugs (things like kernel version, used profile, gcc/libc versions, ...)

[Shan]

emerge info sux, it also shows things that arent even present in the make.conf, that creates a lot of confusion

That may be so, but those things that are added are needed for proper debugging / help purposes. For example, it shows your currently running kernel, installed gcc and glibc, as well as automake / autoconf and your baselayout. Tell me how those are "bad" things that are shown?

For instance Not every user "knows" that certain versions of the (now old) KDE3.2 betas had trouble with the ARCH version of autoconf or automake, so when they posted questions, they get a faster answer if they include the output from emerge --info, which shows what version they're using, and thus allows the person replying to simply say "Upgrade auto* to ~ARCH"

And for the record, I answerd my own earlier post asking if it could be used as an alias. It can, and works just the same (atleast with the one ecatmur posted)

[tomk]

emerge info sux, it also shows things that arent even present in the make.conf, that creates a lot of confusion

Most of the things that emerge --info shows which aren't present in make.conf are defaults which are tucked away in other files on your box. emerge --info gives you an easy way to get all this information about your system in one simple command.

And for the record, I answerd my own earlier post asking if it could be used as an alias. It can, and works just the same (atleast with the one ecatmur posted)

This works just as well, and the sed version seems to be quicker than the grep version. As I said before I really need to learn more sed.

[UberLord]

Nice tip

[syn_ack]

A shorter sed version:

Code: Select all

#!/bin/sed -f
s/#.*//
/^\s*$/d

or

Code: Select all

sed -e 's/#.*//;/^\s*$/d' "$@"

I've recently started learning/studying SED and GREP thanks to this thread. I'm unsure if this is correct or not but doesn't the following basically accomplish stripping out "commented lines" as well as "blank ones" in a shorter amount of key strokes? Please let me know what I'm missing here.

Code: Select all

cat .config | sed '/^#/ d' | sed '/^$/ d'

cat .config | sed -e '/^#/ d' -e '/^$/ d'

cat .config | grep  '.' | grep -v '#'

cat .config | grep -v '#' | grep  '[^$]'

And ecatmur, I would just like to say thanks for the awesome work on the "dep script" here. You have seriously motivated me to start learning the power of the BASH.
There have been so many things that I would like to fix or (actually), make better for myself and possibly the Gentoo community. But things seem to all of the sudden start becoming understandable for what ever reason. Your inspirational and make Linux truely that much more fun.

I feel like I'm starting to break some wild horse.. Not there yet. But I know if I stick with it and have good teachers like you guys/gals I'll get there.. Trying to stay focused in one Linux area over another seems to be my biggest problem. Kid in the candy store effect.

[hardcore]

This is just what i was looking for. Thanks

[randolph]

And ecatmur, I would just like to say thanks for the awesome work on the "dep script" here. You have seriously motivated me to start learning the power of the BASH....
...You're inspirational and make Linux truely much more fun....

I would like to step in line to this. As I am a noob "chmod 755" gave me just the tip I needed. Not knowing how to execute a script. And the rest speeks for it self.

Thank you very much ecatmur

[x1jmp]

Thanks for this tip with sed!

My version is this, which also deletes lines with ";", like the smb.conf

Code: Select all

sed -e 's/[#;].*//;/^\s*$/d'

It's also possible to add exactly this into the .bashrc as an alias.

[msalerno]

A perl update to the one above:

Code: Select all

perl -pne 's/^\s*([#;].*|\s*)\n$//' <filename>

[dundas]

cool, will do, thx tomk.

[nanafunk]

some of you might find this url useful

Useless Use of Cat (cat foo | grep bar). See http://www.ruhr.de/home/smallo/award.html