| View previous topic :: View next topic |
| Author |
Message |
as.gentoo
Guru
Joined: 07 Aug 2004
Posts: 319
|
 Posted: Wed Nov 25, 2009 12:03 pm Post subject: [SOLVED] how do I remove color code from log files? |
 |
|
Attached logs - e.g. for bug reports - look like this on my system:
| Code: | >>> [1m[37mcfg-update-1.8.2-r1[0m[0m: Creating checksum index...
>>> Unpacking source...
>>> Unpacking glipper-1.0.tar.bz2 to /var/tmp/portage/x11-misc/glipper-1.0-r2/work
[32;01m*[0m Fixing OMF Makefiles ...
[A[187C [34;01m[ [32;01mok[34;01m ][0m
[32;01m*[0m Running elibtoolize in: glipper-1.0
[32;01m*[0m Applying portage-1.5.10.patch ...
[32;01m*[0m Applying max_cmd_len-1.5.20.patch ...
[32;01m*[0m Applying sed-1.5.6.patch ...
[32;01m*[0m Applying as-needed-1.5.patch ...
[32;01m*[0m Applying glipper-1.0-binary-data.patch ...
[A[187C [34;01m[ [32;01mok[34;01m ][0m
[32;01m*[0m Applying glipper-1.0-transparent.patch ...
[A[187C [34;01m[ [32;01mok[34;01m ][0m
>>> Source unpacked. |
What I need should look like this:
| Code: | >>> cfg-update-1.8.2-r1: Creating checksum index...
>>> Unpacking source...
>>> Unpacking glipper-1.0.tar.bz2 to /var/tmp/portage/x11-misc/glipper-1.0-r2/work
* Fixing OMF Makefiles ...
[ ok ]
* Running elibtoolize in: glipper-1.0
* Applying portage-1.5.10.patch ...
* Applying max_cmd_len-1.5.20.patch ...
* Applying sed-1.5.6.patch ...
* Applying as-needed-1.5.patch ...
* Applying glipper-1.0-binary-data.patch ...
[ ok ]
* Applying glipper-1.0-transparent.patch ...
[ ok ]
>>> Source unpacked. |
I guess that's color and other format signs. cat, more and less know how to interpret that and show colors. When I load the log file in gedit (which understands unicode) it just looks like what I see after uploading that file as plain text attachment to bugs.gentoo.org and watch it in firefox afterwards - just as the first example above!
I think it's possible to disable (showing) colors completely but that's not what I want! It is useful in my xterm! So, is there something like clean -i messy.log -o clean.log?
A few years ago I used a2ps which is/was really powerful regarding text conversion, but I'm not sure if it is capable of doing what I want and I don't want to emerge it just to see that it doesn't do what I want it to. When the text is short I just copy and paste the output of less messy.log to gedit and load the resulting file up, but that can't be the (cumbersome) solution - especially not when the log file contains much more than just a few lines (which is the case, well ... always)! I don't know how to use sed to do that and I've got lots of things to do which have a much higher priority on my to-do list.  There must be an easier way!
I didn't find an answer in http://www.gentoo.org/doc/en/bugzilla-howto.xml neither when I used www.forestle.org or www.znout.org to find the answer in this forum (I did the last one as workaround for the ... um ... first complaint below). 
Please help !
By the way, why doesn't the search work as (IMHO) expected? When I use quotation marks - e.g. typing something like this: "remove color code signs" or "for bug reports" into the search input field the site shows (most probably) every thread which contains all those single words - but not as phrase. Is there an other syntax to define that I'm looking for threads which contain that phrase? I refuse to believe that the user has to check all of the threads headings on perhaps 5 pages to find or not find - a valid result. Well, actually it's even worse: ...has to read every thread in the list because that phrase may be in the content of one of those threads?!
*cough* The search field should be a little wider, but that's not really important.  *cough*
Last edited by as.gentoo on Sat Dec 05, 2009 4:57 am; edited 1 time in total |
|
| Back to top |
|
 |
poly_poly-man
Advocate
Joined: 06 Dec 2006
Posts: 2477
Location: RIT, NY, US
|
| Posted: Wed Nov 25, 2009 12:20 pm Post subject: |
|
|
The search on this site sucks. Use google to search (site:forums.gentoo.org).m
Anyway, you could fix your problem with a clever sed script, but I'm pretty sure I've solved this with some option to cat in the past (cat -n or something). Look at the man pages for both cat and echo.
_________________
iVBORw0KGgoAAAANSUhEUgAAA
avatar: new version of logo - see topic 838248. Potentially still a WiP. |
|
| Back to top |
|
|
toralf
Developer
Joined: 01 Feb 2004
Posts: 3925
Location: Hamburg
|
| Posted: Wed Nov 25, 2009 12:36 pm Post subject: |
|
|
| Think about setting the LESS variable to sth different than the one within /etc/profile.env ? Furthermore hhm, and look for NOCOLOR might help. |
|
| Back to top |
|
|
as.gentoo
Guru
Joined: 07 Aug 2004
Posts: 319
|
| Posted: Wed Nov 25, 2009 2:05 pm Post subject: |
|
|
| poly_poly-man wrote: | | Anyway, you could fix your problem with a clever sed script, but I'm pretty sure I've solved this with some option to cat in the past (cat -n or something). Look at the man pages for both cat and echo. |
I did that for cat yet. The only thing in that direction is cat -v but that makes th output look as it is in firefox/gedit. (-n activates showing line numbers).
man echo wasn't helpfull neither  |
|
| Back to top |
|
|
as.gentoo
Guru
Joined: 07 Aug 2004
Posts: 319
|
| Posted: Wed Nov 25, 2009 2:22 pm Post subject: |
|
|
| toralf wrote: | | Think about setting the LESS variable to sth different than the one within /etc/profile.env ? Furthermore hhm, and look for NOCOLOR might help. |
No, that doesn't help too. |
|
| Back to top |
|
|
as.gentoo
Guru
Joined: 07 Aug 2004
Posts: 319
|
| Posted: Wed Nov 25, 2009 3:01 pm Post subject: |
|
|
This helps a little, but is far from being satisfying...
cat messy.log | col -b > almost_clean.log
the file looks like this now:
| Code: | >>> cfg-update-1.8.2-r1: Creating checksum index...
>>> Unpacking source...
>>> Unpacking glipper-1.0.tar.bz2 to /var/tmp/portage/x11-misc/glipper-1.0-r2/work
32;*Fixing OMF Makefiles ...
A187C 34;[ 32;ok34; ] 32;*Running elibtoolize in: glipper-1.0
32;* Applying portage-1.5.10.patch ...
32;* Applying max_cmd_len-1.5.20.patch ...
32;* Applying sed-1.5.6.patch ...
32;* Applying as-needed-1.5.patch ...
32;*Applying glipper-1.0-binary-data.patch ...
A187C 34;[ 32;ok34; ] 32;*Applying glipper-1.0-transparent.patch ...
A187C 34;[ 32;ok34; ]>>> Source unpacked.
>>> Compiling source in /var/tmp/portage/x11-misc/glipper-1.0-r2/work/glipper-1.0 ...
|
|
|
| Back to top |
|
|
toralf
Developer
Joined: 01 Feb 2004
Posts: 3925
Location: Hamburg
|
| Posted: Wed Nov 25, 2009 3:28 pm Post subject: |
|
|
| Have a look at the command "strings" to reject non-ascii chars. |
|
| Back to top |
|
|
as.gentoo
Guru
Joined: 07 Aug 2004
Posts: 319
|
| Posted: Wed Nov 25, 2009 4:19 pm Post subject: |
|
|
| toralf wrote: | | Have a look at the command "strings" to reject non-ascii chars. |
Well, another incomplete man page... The important thing is only revealed when strings is asked to explain itself (strings -h).
| Code: | strings: supported targets: elf64-x86-64 elf32-i386 a.out-i386-linux efi-app-ia32 efi-app-x86_64 elf64-little elf64-big elf32-little elf32-big srec symbolsrec tekhex binary ihex
Report bugs to <http://www.sourceware.org/bugzilla/> |
Using the strings command with text files outputs nothing...
| Code: | atti@magic ~ $ file x11-misc_glipper-1.0-r2.log
x11-misc_glipper-1.0-r2.log: UTF-8 Unicode English text, with very long lines, with escape sequences
atti@magic ~ $ strings --bytes=3 --all --encoding=l x11-misc_glipper-1.0-r2.log
atti@magic ~ $
|
In contrast to that it works when fed w/ the gimp executable:
| Code: | atti@magic ~ $ strings --bytes=500 --all --encoding=s /usr/bin/gimp
M 509.72445,438.68864 C 501.47706,469.77945 464.95038,491.54566 431.85915,497.74874 C 438.5216,503.01688 442.87782,511.227 442.87782,520.37375 C
...
369.47991,514.47879 C 374.18324,514.47879 378.00043,518.29599 378.00043,522.99931 z
This tool allows intensity levels in the specified drawable to be remapped according to a set of parameters. The low/high input levels specify an initial mapping from the source intensities. The gamma value determines how intensities between the low and high input intensities are interpolated. A gamma value of 1.0 results in a linear interpolation. Higher gamma values result in more high-level intensities. Lower gamma values result in more low-level intensities. The low/high output levels constrain the final intensity mapping--that is, no final intensity will be lower than the low output level and no final intensity will be higher than the high output level. This tool is only valid on RGB color and grayscale images. It will not operate on indexed drawables.
|
Any further suggestions are welcome! In case this thread will not help me with the problem: at least I get to know commands I haven't heard of yet.  |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9709
Location: almost Mile High in the USA
|
| Posted: Fri Dec 04, 2009 9:25 pm Post subject: |
|
|
I use this perl script (filter) to get rid of ANSI escape sequences, IRC bold codes, and ancient man page overstrikes:
| Code: | #!/usr/bin/perl
# StripANSI .. remove ANSI codes from stdin
# Also removes removes backspace bolds and BitchX IRC ^B's
while(<>){s/\e\[[\d;]*.//g;s/(.)\cH\1/$1/g;s/\cB//g;print;}
|
It does _NOT_ handle every formatting issue (try feeding it spinners and animated ANSI, it will do very strange things) but for most simple stuff it works fine.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
as.gentoo
Guru
Joined: 07 Aug 2004
Posts: 319
|
| Posted: Sat Dec 05, 2009 4:56 am Post subject: |
|
|
| Works like a charm! Thank you very much! |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9709
Location: almost Mile High in the USA
|
| Posted: Sat Dec 05, 2009 7:40 am Post subject: |
|
|
I would be slightly concerned that piping emerge output would not automatically disable color code generation, for me, emerge and other portage programs generally can determine piping and redirection, and I don't need my script (as the comments suggest, it's for my IRC logs I want to get rid of ANSI sequences!)... But oh well, as long as it's gone...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
as.gentoo
Guru
Joined: 07 Aug 2004
Posts: 319
|
| Posted: Sat Dec 05, 2009 10:29 am Post subject: |
|
|
I'm not sure what you mean... The following command reproduces color codes as well
atti@magic ~ $ sudo emerge -v kbd | cat -v
These are the packages that would be merged, in order:
Calculating dependencies ^H^H... done!
[ebuild U ] sys-apps/kbd-1.15 [1.13-r1] USE="nls" 0 kB
Total: 1 package (1 upgrade), Size of downloads: 0 kB
>>> Verifying ebuild manifests
>>> Emerging (1 of 1) sys-apps/kbd-1.15
* kbd-1.15.tar.gz RMD160 SHA1 SHA256 size ... [ ok ]
* checking ebuild checksums ... [ ok ]
* checking auxfile checksums ... [ ok ]
* checking miscfile checksums ... [ ok ]
>>> ^[[1m^[[37mcfg-update-1.8.2-r1^[[0m^[[0m: Checksum index is up-to-date ...
>>> Unpacking source...
>>> Unpacking kbd-1.15.tar.gz to /var/tmp/portage/sys-apps/kbd-1.15/work |
|
| Back to top |
|
|
poly_poly-man
Advocate
Joined: 06 Dec 2006
Posts: 2477
Location: RIT, NY, US
|
| Posted: Sat Dec 05, 2009 1:26 pm Post subject: |
|
|
| as.gentoo wrote: | | I'm not sure what you mean... The following command reproduces color codes as well |
he's saying that ideally, it shouldn't, and that should be fixed in portage.
_________________
iVBORw0KGgoAAAANSUhEUgAAA
avatar: new version of logo - see topic 838248. Potentially still a WiP. |
|
| Back to top |
|
|
as.gentoo
Guru
Joined: 07 Aug 2004
Posts: 319
|
| Posted: Sat Dec 05, 2009 9:52 pm Post subject: |
|
|
| poly_poly-man wrote: | | as.gentoo wrote: | | I'm not sure what you mean... The following command reproduces color codes as well |
he's saying that ideally, it shouldn't, and that should be fixed in portage. |
xD ... English is not my native language. I should probably have paid more attention in the lessons.
Now, who's filing the bug report? |
|
| Back to top |
|
|
|
Other Things Gentoo
