Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Non-consistent output from history/grep using utf8-locale?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
freke
Apprentice
Apprentice


Joined: 23 Jan 2003
Posts: 271

PostPosted: Tue Dec 29, 2015 12:02 am    Post subject: Non-consistent output from history/grep using utf8-locale? Reply with quote

I have a weird problem - when locale is set to POSIX this works fine
Code:
history | grep tail


When using an utf8-locale it mostly works but sometimes I get
Code:
ns ~ # history | grep tail
Binary file (standard input) matches
ns ~ # history | grep tail
   55  recal tail
   56  tail -f /var/log/everything/current
   94  recal tail
....

I've never experienced the Binary file (standard input) matches using POSIX-locale!
(recently switched to en_US.utf8 due to equery/python3.4 not behaving using POSIX)
Back to top
View user's profile Send private message
khayyam
Advocate
Advocate


Joined: 07 Jun 2012
Posts: 4496

PostPosted: Tue Dec 29, 2015 12:32 am    Post subject: Re: Non-consistent output from history/grep using utf8-local Reply with quote

freke wrote:
Code:
Binary file (standard input) matches

freke ... which suggests a character sequence in the output of 'history' is causing grep to interpret the output as binary. To avoid this use the '-a,--text' grep option. I suspect the reason is that having set LANG you have LC_COLLATE="en_US.uft8" which is probably best set to C.

/etc/env.d/02locale:
# Configuration file for eselect
# This file has been automatically generated.
LANG="en_US.utf8"
LC_COLLATE="C"

HTH & best ... khay
Back to top
View user's profile Send private message
freke
Apprentice
Apprentice


Joined: 23 Jan 2003
Posts: 271

PostPosted: Tue Dec 29, 2015 8:10 am    Post subject: Re: Non-consistent output from history/grep using utf8-local Reply with quote

khayyam wrote:
freke wrote:
Code:
Binary file (standard input) matches

freke ... which suggests a character sequence in the output of 'history' is causing grep to interpret the output as binary. To avoid this use the '-a,--text' grep option. I suspect the reason is that having set LANG you have LC_COLLATE="en_US.uft8" which is probably best set to C.

/etc/env.d/02locale:
# Configuration file for eselect
# This file has been automatically generated.
LANG="en_US.utf8"
LC_COLLATE="C"

HTH & best ... khay


Thanks - will try when I get back at my machine - what puzzles me is that I can do 'history | grep tail' 5 times in a row - and that 4 times it works but the 5th time it fails.

I might need to reed up on the LC_* stuff :)
Back to top
View user's profile Send private message
khayyam
Advocate
Advocate


Joined: 07 Jun 2012
Posts: 4496

PostPosted: Tue Dec 29, 2015 8:53 am    Post subject: Re: Non-consistent output from history/grep using utf8-local Reply with quote

freke wrote:
[...] - what puzzles me is that I can do 'history | grep tail' 5 times in a row - and that 4 times it works but the 5th time it fails.

freke ... yes, because the content of 'history' is changing, so the output piped to grep is similarly. What specifically grep interprets as binary I can't say, but I'd expect its either some char in .bash_history, or the fact that the env has changed, so how/what grep expects stdin to be doesn't match the locale.

So, I don't think anything specifically puzzling, its simply a matter of there being a change in *how* the data is represented/encoded, and that change is effected by the locale, and the chars generated by that locale. I don't use bash but I'd expect that if I placed unicode data in its history/buffer it wouldn't know what that encoding was if the env changed and was then posix, or iso-8859-1.

best ... khay
Back to top
View user's profile Send private message
freke
Apprentice
Apprentice


Joined: 23 Jan 2003
Posts: 271

PostPosted: Tue Dec 29, 2015 9:25 pm    Post subject: Reply with quote

Thanks - decided to clean out .bash_history now after changing to en_US.utf8 and so far it seems fine and consistent now.

tyvm for the help :)
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum