Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
quietemerge -- provide pretty emerge output
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
JustJoe
Tux's lil' helper
Tux's lil' helper


Joined: 30 Jul 2005
Posts: 80

PostPosted: Tue Apr 06, 2010 9:13 pm    Post subject: Reply with quote

ppurka wrote:
Hi, your proposal seems interesting. However, I don't think I will implement cpu-scaling. People have many different setups. Some people control the scaling using cpufreq, some people use their DE's and so on. It is very easy to go wrong when you mess with others' cpu-scaling.

These type of things are best left to the daemon/DE/manual control as the individual wants.


Fair enough. 8)
I added the performance bit on line 744, just below where mount_tmpfs is being called, and the ondemand bit is at line 901, just before 'exit_prog $exit_status'.
Seems to work fine like that. Any advise on the placement of these commands in quietemerge ?
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3213

PostPosted: Wed Apr 07, 2010 12:06 am    Post subject: Reply with quote

JustJoe wrote:

Fair enough. 8)
I added the performance bit on line 744, just below where mount_tmpfs is being called,
This placement is perfect. If you want to set your cpu frequency, then just before actual emerge makes good sense.
Quote:
and the ondemand bit is at line 901, just before 'exit_prog $exit_status'.
Seems to work fine like that. Any advise on the placement of these commands in quietemerge ?
The ondemand bit is not really well placed. The script can exit even when you press Ctrl+C. The function exit_prog() handles those cases.

So, a correct line in which to place this ondemand bit would be anywhere before unmount_tmpfs in the function exit_prog(). So, for example placing it just before line 412 would be a good option.
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde4 :-/
Back to top
View user's profile Send private message
jw5801
Apprentice
Apprentice


Joined: 12 Jun 2008
Posts: 251
Location: Melbourne, Australia

PostPosted: Wed Apr 07, 2010 12:11 am    Post subject: Reply with quote

ppurka wrote:
JustJoe wrote:

Fair enough. 8)
I added the performance bit on line 744, just below where mount_tmpfs is being called,
This placement is perfect. If you want to set your cpu frequency, then just before actual emerge makes good sense.
Quote:
and the ondemand bit is at line 901, just before 'exit_prog $exit_status'.
Seems to work fine like that. Any advise on the placement of these commands in quietemerge ?
The ondemand bit is not really well placed. The script can exit even when you press Ctrl+C. The function exit_prog() handles those cases.

So, a correct line in which to place this ondemand bit would be anywhere before unmount_tmpfs in the function exit_prog(). So, for example placing it just before line 412 would be a good option.


Hmm... how about a configuration file option to set pre and post merge commands? That could cover this usage scenario as well as a lot of other general situations.
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3213

PostPosted: Wed Apr 07, 2010 12:24 am    Post subject: Reply with quote

jw5801 wrote:
Hmm... how about a configuration file option to set pre and post merge commands? That could cover this usage scenario as well as a lot of other general situations.
That's a possibility. Give me a little time to figure it out.

I think something like this will take care of both scripts and commands:
Code:
# In config file:
precommand="something something"
In my code:
Code:
# If it is executable, then run it. Otherwise it is a command.
[[ -x "$( which "$precommand" )" ]] && "$precommand" || eval "$precommand"

Any comments or suggestions? Any cases where this may fail?
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde4 :-/
Back to top
View user's profile Send private message
JustJoe
Tux's lil' helper
Tux's lil' helper


Joined: 30 Jul 2005
Posts: 80

PostPosted: Wed Apr 07, 2010 9:09 am    Post subject: Reply with quote

ppurka wrote:
The ondemand bit is not really well placed. The script can exit even when you press Ctrl+C. The function exit_prog() handles those cases.

So, a correct line in which to place this ondemand bit would be anywhere before unmount_tmpfs in the function exit_prog(). So, for example placing it just before line 412 would be a good option.

Doh @ self. The obvious exit_prog function. :oops:


ppurka wrote:
jw5801 wrote:
Hmm... how about a configuration file option to set pre and post merge commands? That could cover this usage scenario as well as a lot of other general situations.
That's a possibility. Give me a little time to figure it out.

I think something like this will take care of both scripts and commands:
Code:
# In config file:
precommand="something something"
In my code:
Code:
# If it is executable, then run it. Otherwise it is a command.
[[ -x "$( which "$precommand" )" ]] && "$precommand" || eval "$precommand"

Any comments or suggestions? Any cases where this may fail?

I'd like that. With this there's no need for editing quietemerge every time a new version get released. Good for the lazy. :)
Back to top
View user's profile Send private message
jw5801
Apprentice
Apprentice


Joined: 12 Jun 2008
Posts: 251
Location: Melbourne, Australia

PostPosted: Wed Apr 07, 2010 9:53 am    Post subject: Reply with quote

ppurka wrote:
I think something like this will take care of both scripts and commands:
Code:
# In config file:
precommand="something something"
In my code:
Code:
# If it is executable, then run it. Otherwise it is a command.
[[ -x "$( which "$precommand" )" ]] && "$precommand" || eval "$precommand"

Any comments or suggestions? Any cases where this may fail?


Looks good to me, no obvious points of failure spring to mind.
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3213

PostPosted: Thu Apr 08, 2010 5:08 am    Post subject: Reply with quote

Update: Version 20100407
Changelog:
* Support for pre-commands and post-commands.

To enable this, add the variables PRE_CMD and POST_CMD to the config file in ~/.config. The variables may point to a command or to a script/program.

If you want to migrate your current config to the new one, then it has to be done manually:
Code:
$ mv ~/.config/{quietemerge.config,quietemerge.config.backup}
$ quietemerge test   # This will create the new config file
$ <your favourite diff program> ~/.config/{quietemerge.config,quietemerge.config.backup}

_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde4 :-/
Back to top
View user's profile Send private message
JustJoe
Tux's lil' helper
Tux's lil' helper


Joined: 30 Jul 2005
Posts: 80

PostPosted: Sun Apr 11, 2010 9:13 am    Post subject: Reply with quote

ppurka,

Although the PRE_CMD and POST_CMD variables do work, they do not seem to appear in the config file by default, I had to add them manually.
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3213

PostPosted: Sun Apr 11, 2010 1:49 pm    Post subject: Reply with quote

JustJoe wrote:
ppurka,

Although the PRE_CMD and POST_CMD variables do work, they do not seem to appear in the config file by default, I had to add them manually.
I don't have a mechanism of updating the config file. That's why I wrote that you need to migrate your old config file manually.
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde4 :-/
Back to top
View user's profile Send private message
JustJoe
Tux's lil' helper
Tux's lil' helper


Joined: 30 Jul 2005
Posts: 80

PostPosted: Mon Apr 12, 2010 12:54 pm    Post subject: Reply with quote

ppurka wrote:
JustJoe wrote:
ppurka,

Although the PRE_CMD and POST_CMD variables do work, they do not seem to appear in the config file by default, I had to add them manually.
I don't have a mechanism of updating the config file. That's why I wrote that you need to migrate your old config file manually.

I understand about updating an existing config file, but shouldn't these vars appear in a newly created config file ? (Looks like they don't atm.)
Back to top
View user's profile Send private message
JustJoe
Tux's lil' helper
Tux's lil' helper


Joined: 30 Jul 2005
Posts: 80

PostPosted: Mon Apr 12, 2010 1:07 pm    Post subject: Reply with quote

Mekoryuk wrote:
I'm a little confused as to what constitutes proper behavior for the script. Since day one I've only ever seen "ETA" give an actual value on a handful of packages; most of the time ETA remains blank, and only ever mentions something when the build is almost finished ("any time now"). I've experimented with various values in the config file, but the result remains the same. I suppose it's because that genlop is attempting to calculate time remaining concurrently with the emerge, and that in most cases the emerge will finish long before genlop has a chance to figure it out, but even for large packages where one might expect genlop to figure it out, ETA still remains blank. Is this normal behavior?

I had the same problem. It was only when i ran emerge -p kopete|genlop -pq when i saw a message about genlop not knowing my CPU. It directed me here:
http://gentoo.linuxhowtos.org/compiletimeestimator/
Only a couple of days after i had send the info requested on that page i got a reply that my CPU was added to the database. Now my ETA's are ok (well, at least they're not blank anymore).
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3213

PostPosted: Mon Apr 12, 2010 4:06 pm    Post subject: Reply with quote

JustJoe wrote:
I understand about updating an existing config file, but shouldn't these vars appear in a newly created config file ? (Looks like they don't atm.)
You should follow the instructions I posted here. Don't create a new blank config file!! The script only checks if there is a config file or not. If it is there, even if blank, the script won't touch that file.
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde4 :-/
Back to top
View user's profile Send private message
JustJoe
Tux's lil' helper
Tux's lil' helper


Joined: 30 Jul 2005
Posts: 80

PostPosted: Mon Apr 12, 2010 5:24 pm    Post subject: Reply with quote

ppurka wrote:
JustJoe wrote:
I understand about updating an existing config file, but shouldn't these vars appear in a newly created config file ? (Looks like they don't atm.)
You should follow the instructions I posted here. Don't create a new blank config file!! The script only checks if there is a config file or not. If it is there, even if blank, the script won't touch that file.


Guess we're missing each other here a bit...
When quietemerge is run for the very first time -so no quietemerge.config file exists in ~/.config/ yet-, the default config file that gets created by quietemerge doesn't seem to contain the new PRE_CMD and POST_CMD variables.
Seems logical to me those variables should be in the default quietemerge.config that quietemerge creates on its first run. Looks like they're not:

Code:
joe@homebox ~ $ mv .config/quietemerge.config .config/quietemerge.config.bak
joe@homebox ~ $ quietemerge test
  * /home/joe/.config/quietemerge.config config file not found. Creating ...
                                                                               
  * Open /home/joe/.config/quietemerge.config and modify the settings       
    to your own liking.                                                         
                                                                               
joe@homebox ~ $ grep CMD .config/quietemerge.config                       
joe@homebox ~ $
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3213

PostPosted: Mon Apr 12, 2010 5:34 pm    Post subject: Reply with quote

JustJoe wrote:

Guess we're missing each other here a bit...
When quietemerge is run for the very first time -so no quietemerge.config file exists in ~/.config/ yet-, the default config file that gets created by quietemerge doesn't seem to contain the new PRE_CMD and POST_CMD variables.
Seems logical to me those variables should be in the default quietemerge.config that quietemerge creates on its first run. Looks like they're not:

Code:
joe@homebox ~ $ mv .config/quietemerge.config .config/quietemerge.config.bak
joe@homebox ~ $ quietemerge test
  * /home/joe/.config/quietemerge.config config file not found. Creating ...
                                                                               
  * Open /home/joe/.config/quietemerge.config and modify the settings       
    to your own liking.                                                         
                                                                               
joe@homebox ~ $ grep CMD .config/quietemerge.config                       
joe@homebox ~ $
Well, then you simply have an old quietemerge script in your system! :P Though, I wonder how you are able to run some *_CMD if you have an older script. I just downloaded the script from the google code website and it works fine in mine. Please ensure that you have the latest version:
Code:
~> grep -m1 Version $(which quietemerge)
        #   Version: 20100407                                       #

_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde4 :-/
Back to top
View user's profile Send private message
JustJoe
Tux's lil' helper
Tux's lil' helper


Joined: 30 Jul 2005
Posts: 80

PostPosted: Mon Apr 12, 2010 8:43 pm    Post subject: Reply with quote

Code:
# grep -m1 Version $(which quietemerge)
        #   Version: 20100119                                       #

*crawls back under rock.... :oops:

Guess i totally forgot to actually move the 20100407 version into $PATH after downloading it.
The reason why 20100119 just runs my defined CMD's is not a mystery at all: i had those hard coded in it.

Sorry for the needless fuzz !


edit:

I edited line 447 and 769 from quietemerge to redirect which error msg to /dev/null :
Code:

line 447
[[ "$_POST_CMD" && -x "$( which "$_POST_CMD" 2> /dev/null )" ]] \

line 769
[[ "$_PRE_CMD" && -x "$( which "$_PRE_CMD" 2> /dev/null )" ]] \

This suppresses ugly which output like:
which: no scaling_governor in (./[ -f /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor ] ....<snipped for forum readability>

As far as i know this will not cause any problems with the rest of your code.
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3213

PostPosted: Tue Apr 13, 2010 1:17 am    Post subject: Reply with quote

JustJoe wrote:

I edited line 447 and 769 from quietemerge to redirect which error msg to /dev/null :
Code:

line 447
[[ "$_POST_CMD" && -x "$( which "$_POST_CMD" 2> /dev/null )" ]] \

line 769
[[ "$_PRE_CMD" && -x "$( which "$_PRE_CMD" 2> /dev/null )" ]] \

This suppresses ugly which output like:
which: no scaling_governor in (./[ -f /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor ] ....<snipped for forum readability>

As far as i know this will not cause any problems with the rest of your code.
Yes. I should have done that. Well, the next (non-trivial) revision will include this fix :)
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde4 :-/
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3213

PostPosted: Fri Apr 22, 2011 12:24 pm    Post subject: Version 20110422 Reply with quote

New version! 20110422 :)

Changelog:
  • Use my own genlop function (now dependency on genlop is removed)
    [A bug free genlop functionality will be required for future --jobs support]
  • Redirect error messages to /dev/null (as pointed out by JustJoe)
  • Handle -r resume switch (seems to be a new argument in emerge?)


Google didn't seem to handle my upload of similarly named files correctly this time. I have removed the "quietemerge" file from the download list (and lost all the statistics) :(

New version can be downloaded from here: http://code.google.com/p/quietemerge/downloads/list
Direct file link: http://quietemerge.googlecode.com/files/quietemerge-20110422

Please rename the file to quietemerge after downloading.

EDIT: Added reason for dropping genlop.
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde4 :-/
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3213

PostPosted: Sat Apr 30, 2011 8:56 am    Post subject: Reply with quote

Update: Version 20110430.
Changes:
  • Recognize --color=n in command line and turn off all colors
  • Create config file only if user is root
  • Add option to show per-package ETA. It is config variable ETA_ALL
  • Add rudimentary detection of old config file
  • Bug fix: work around duplicate lines in emerge.log. Notable package is sys-apps/portage.
  • Bug fix: try to fix "any time now" showing up as "any time no"
  • Bug fix: Pass --resume on to emerge command
  • Make sure one tmp file ($tmpT) is deleted just before writing to it.


Direct file link: http://quietemerge.googlecode.com/files/quietemerge-20110430

Please rename the file to quietemerge after downloading.
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde4 :-/
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3213

PostPosted: Wed May 18, 2011 3:08 pm    Post subject: Reply with quote

There is something weird in this screenshot: http://i.imgur.com/pPufG.png
Can anyone spot it? :P :twisted:
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde4 :-/
Back to top
View user's profile Send private message
gmargaro
n00b
n00b


Joined: 12 Feb 2004
Posts: 2

PostPosted: Thu May 19, 2011 3:29 pm    Post subject: Reply with quote

ppurka wrote:
There is something weird in this screenshot: http://i.imgur.com/pPufG.png
Can anyone spot it? :P :twisted:


The --jobs option ??
Back to top
View user's profile Send private message
krinn
Advocate
Advocate


Joined: 02 May 2003
Posts: 4433

PostPosted: Thu May 19, 2011 3:46 pm    Post subject: Reply with quote

yep, your cpu has learn to count total ETA with scotty from the enterprise ship : because only him can says: "it will take 7 days to repair, so it will be ok in 2 days"
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3213

PostPosted: Thu May 19, 2011 3:48 pm    Post subject: Reply with quote

gmargaro wrote:
ppurka wrote:
There is something weird in this screenshot: http://i.imgur.com/pPufG.png
Can anyone spot it? :P :twisted:


The --jobs option ??
It sure is! I am currently planning on running a big uDNv world to test it. I held off updating the few kde apps (okular, digikam, etc) that I have (4.4->4.6) all this while, so that I get an incentive to complete this --jobs support :)
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde4 :-/
Back to top
View user's profile Send private message
gmargaro
n00b
n00b


Joined: 12 Feb 2004
Posts: 2

PostPosted: Thu May 19, 2011 3:51 pm    Post subject: Reply with quote

Great !

I miss the --jobs option since i reinstalled quietemerge.
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3213

PostPosted: Thu May 19, 2011 4:33 pm    Post subject: Reply with quote

krinn wrote:
yep, your cpu has learn to count total ETA with scotty from the enterprise ship : because only him can says: "it will take 7 days to repair, so it will be ok in 2 days"
What does that quote mean? :P
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde4 :-/
Back to top
View user's profile Send private message
krinn
Advocate
Advocate


Joined: 02 May 2003
Posts: 4433

PostPosted: Thu May 19, 2011 5:34 pm    Post subject: Reply with quote

that on left part it shown a per package eta of 12s, 6s, 9s, 26s, 23s, 18s and 11s and a total ETA of 49s
same for right side, same thing is seen. (it's even worst with a funky 15s)
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6  Next
Page 4 of 6

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum