Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
quietemerge -- provide pretty emerge output
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
jw5801
Apprentice
Apprentice


Joined: 12 Jun 2008
Posts: 251
Location: Melbourne, Australia

PostPosted: Tue Nov 03, 2009 12:53 pm    Post subject: Reply with quote

ppurka wrote:
jw5801 wrote:
Ah, cool. No bugs, but a feature request: Would it be possible/simple to print the time taken to emerge each package where you're currently printing 'Done!'?
This is now available in version 20091102 :D


Excellent, thanks very much! I'm getting a syntax error with the new version though, not a very helpful error though...

Code:
jw@Andornor ~/scripts $ bash quietemerge-20091102
quietemerge-20091102: line 673: syntax error: unexpected end of file
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3256

PostPosted: Tue Nov 03, 2009 3:12 pm    Post subject: Reply with quote

jw5801 wrote:

Excellent, thanks very much! I'm getting a syntax error with the new version though, not a very helpful error though...

Code:
jw@Andornor ~/scripts $ bash quietemerge-20091102
quietemerge-20091102: line 673: syntax error: unexpected end of file
I have no idea how that happened. 8O The file quietemerge-20091102 that I uploaded to http://code.google.com/p/quietemerge/downloads/list does not have that error. On the other hand the file quietemerge has that error. Both of them should be the same file.

It is fixed now. It looks like in testing the script or just before uploading I might have pressed a key or something in gvim. That (unintentionally) altered some part of the script. Sorry for this. I hate bugs. :evil:
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/
Back to top
View user's profile Send private message
jprobichaud
Tux's lil' helper
Tux's lil' helper


Joined: 28 Jan 2009
Posts: 81
Location: Montreal, Qc

PostPosted: Wed Nov 04, 2009 10:31 pm    Post subject: Reply with quote

ppurka wrote:
Ha! Bug in genlop!!
Code:
~> genlop -t "=dev-libs/apr-1.3.9" --date 2 minute ago
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1039, <_GEN_0> line 23512.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1040, <_GEN_0> line 23512.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1042, <_GEN_0> line 23512.
 * dev-libs/apr

     Tue Nov  3 00:01:33 2009 >>> dev-libs/apr-1.3.9
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1078, <_GEN_0> line 23512.
       merge time: 5 days, 5 hours, 1 minute and 33 seconds.
~> genlop -t "=dev-libs/apr-1.3.9"                   
 * dev-libs/apr

     Tue Nov  3 00:01:33 2009 >>> dev-libs/apr-1.3.9
       merge time: 1 minute and 31 seconds.


Updated Version 20091102 to work around genlop bug :!:


Which version of genlop do you have? I have 0.30.8-r2 and I can't produce this bug...
Back to top
View user's profile Send private message
jprobichaud
Tux's lil' helper
Tux's lil' helper


Joined: 28 Jan 2009
Posts: 81
Location: Montreal, Qc

PostPosted: Wed Nov 04, 2009 11:07 pm    Post subject: Reply with quote

I spoke too soon... I now have the issue myself, with the latest quietemerge ...

Code:

  * Install  app-emulation/emul-linux-x86-qtlibs-20081109.                                                                                                                              49 seconds.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1075, <_GEN_0> line 103950.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1076, <_GEN_0> line 103950.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1078, <_GEN_0> line 103950.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1114, <_GEN_0> line 103950.
  * Install  net-fs/samba-libs-3.4.3-r1.                                                                                                                6 days, 22 hours, 46 minutes and 3 seconds.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1075, <_GEN_0> line 103959.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1076, <_GEN_0> line 103959.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1078, <_GEN_0> line 103959.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1114, <_GEN_0> line 103959.
  * Install  net-fs/samba-server-3.4.3-r1.                                                                                                             6 days, 22 hours, 57 minutes and 57 seconds.


And obviously, the 6 days isn't real!

I'll try to check genlop to see if I could find the bug
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3256

PostPosted: Wed Nov 04, 2009 11:37 pm    Post subject: Reply with quote

jprobichaud wrote:
I spoke too soon... I now have the issue myself, with the latest quietemerge ...

Code:

  * Install  app-emulation/emul-linux-x86-qtlibs-20081109.                                                                                                                              49 seconds.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1075, <_GEN_0> line 103950.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1076, <_GEN_0> line 103950.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1078, <_GEN_0> line 103950.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1114, <_GEN_0> line 103950.
  * Install  net-fs/samba-libs-3.4.3-r1.                                                                                                                6 days, 22 hours, 46 minutes and 3 seconds.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1075, <_GEN_0> line 103959.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1076, <_GEN_0> line 103959.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1078, <_GEN_0> line 103959.
Use of uninitialized value in subtraction (-) at /usr/bin/genlop line 1114, <_GEN_0> line 103959.
  * Install  net-fs/samba-server-3.4.3-r1.                                                                                                             6 days, 22 hours, 57 minutes and 57 seconds.


And obviously, the 6 days isn't real!

I'll try to check genlop to see if I could find the bug
Hi,
The problem is that genlop gives the correct estimate if I increase the time in
Code:
genlop -t "=cat/pkg-ver"  --date 3 minute ago
This was set to 1 minute in an earlier version of the script, and I noticed that increasing it to 3 minutes is good enough to stop that error from occuring in genlop; except it surely isn't good enough for all cases.

I don't want to increase the time too much since then it would create a problem with my parsing of genlop's output, if you emerged the same package twice in a row within the same time limit. The other option is for me to implement a counter myself. But that is a really clumsy option, especially when genlop already does it.
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/
Back to top
View user's profile Send private message
jprobichaud
Tux's lil' helper
Tux's lil' helper


Joined: 28 Jan 2009
Posts: 81
Location: Montreal, Qc

PostPosted: Thu Nov 05, 2009 1:43 am    Post subject: Reply with quote

ppurka wrote:

I don't want to increase the time too much since then it would create a problem with my parsing of genlop's output, if you emerged the same package twice in a row within the same time limit. The other option is for me to implement a counter myself. But that is a really clumsy option, especially when genlop already does it.


Ah, I understand. What about doing:
Code:

jrobicha@localhost ~ $ genlop -t kde-base/kwrited --date 1 day ago | tail -n 3 | head -n 1
     Wed Nov  4 18:58:13 2009 >>> kde-base/kwrited-4.3.3


Actually, we can even skip the "--date ..." part.
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3256

PostPosted: Thu Nov 05, 2009 1:47 am    Post subject: Reply with quote

jprobichaud wrote:
ppurka wrote:

I don't want to increase the time too much since then it would create a problem with my parsing of genlop's output, if you emerged the same package twice in a row within the same time limit. The other option is for me to implement a counter myself. But that is a really clumsy option, especially when genlop already does it.


Ah, I understand. What about doing:
Code:

jrobicha@localhost ~ $ genlop -t kde-base/kwrited --date 1 day ago | tail -n 3 | head -n 1
     Wed Nov  4 18:58:13 2009 >>> kde-base/kwrited-4.3.3


Actually, we can even skip the "--date ..." part.
Yes, something like that would be an option. Not that I am happy about using a bunch of pipes.
I will update the script with something similar to this. Let's hope genlop doesn't fail me again. :roll:

I don't want to skip the --date part. A simple thing like showing the merged time would be a highly inefficient operation if I don't give the --date part. I expect that genlop will take less time to parse emerge.log if I give it a date argument. If genlop still parses the whole emerge.log, then it is being really inefficient. Some people here have installations which are several years old and hence might have huge emerge.logs.
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/
Back to top
View user's profile Send private message
jw5801
Apprentice
Apprentice


Joined: 12 Jun 2008
Posts: 251
Location: Melbourne, Australia

PostPosted: Thu Nov 05, 2009 2:50 am    Post subject: Reply with quote

ppurka wrote:
jw5801 wrote:

Excellent, thanks very much! I'm getting a syntax error with the new version though, not a very helpful error though...

Code:
jw@Andornor ~/scripts $ bash quietemerge-20091102
quietemerge-20091102: line 673: syntax error: unexpected end of file
I have no idea how that happened. 8O The file quietemerge-20091102 that I uploaded to http://code.google.com/p/quietemerge/downloads/list does not have that error. On the other hand the file quietemerge has that error. Both of them should be the same file.

It is fixed now. It looks like in testing the script or just before uploading I might have pressed a key or something in gvim. That (unintentionally) altered some part of the script. Sorry for this. I hate bugs. :evil:


Yeah, dunno what would make it give that error. Possibly some not properly encoded character somewhere. Anyway, all better now!

Cheers!
jw
Back to top
View user's profile Send private message
jprobichaud
Tux's lil' helper
Tux's lil' helper


Joined: 28 Jan 2009
Posts: 81
Location: Montreal, Qc

PostPosted: Thu Nov 05, 2009 3:07 am    Post subject: Reply with quote

ppurka wrote:


I don't want to skip the --date part. A simple thing like showing the merged time would be a highly inefficient operation if I don't give the --date part. I expect that genlop will take less time to parse emerge.log if I give it a date argument. If genlop still parses the whole emerge.log, then it is being really inefficient. Some people here have installations which are several years old and hence might have huge emerge.logs.


shocking truth: genlop is 'really' inefficient.

I've ran these commands multiple times and I'm giving you the representative numbers:
With the date argument:
Code:

$  time genlop -t kde-base/kwrited --date 1 day ago | tail -n 3 | head -n 1
     Wed Nov  4 18:58:13 2009 >>> kde-base/kwrited-4.3.3

real    0m1.935s
user    0m1.803s
sys     0m0.045s


And without:
Code:

 $ time genlop -t kde-base/kwrited  | tail -n 3 | head -n 1
     Wed Nov  4 18:58:13 2009 >>> kde-base/kwrited-4.3.3

real    0m1.580s
user    0m1.348s
sys     0m0.033s


It's faster because it doesn't have to do date comparison all the time...

I know by experience, genlop is highly inefficient. It's a really good tool that I like alot, but it is not coded in a way that is scalable. I've submitted a patch to improve it but I didn't had an answer yet. It's the 2nd time that I try to contribute to this tool, but to no avail... Maybe I should just 'fork' it.
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3256

PostPosted: Thu Nov 05, 2009 3:49 am    Post subject: Reply with quote

jprobichaud wrote:
shocking truth: genlop is 'really' inefficient.
This is bad. Really bad :(
Quote:
I know by experience, genlop is highly inefficient. It's a really good tool that I like alot, but it is not coded in a way that is scalable. I've submitted a patch to improve it but I didn't had an answer yet. It's the 2nd time that I try to contribute to this tool, but to no avail... Maybe I should just 'fork' it.
Maybe you can contact the author(s) directly by email? And point them to the bug reports perhaps?
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3256

PostPosted: Thu Nov 05, 2009 5:11 am    Post subject: Reply with quote

Update: Version 20091105. Changelog:
* Work around the bug in genlop. Again.
* Make "Enter" select "yes" at the prompt when --ask is used.
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/
Back to top
View user's profile Send private message
jw5801
Apprentice
Apprentice


Joined: 12 Jun 2008
Posts: 251
Location: Melbourne, Australia

PostPosted: Thu Nov 05, 2009 5:22 am    Post subject: Reply with quote

ppurka wrote:
jprobichaud wrote:
shocking truth: genlop is 'really' inefficient.
This is bad. Really bad :(
Quote:
I know by experience, genlop is highly inefficient. It's a really good tool that I like alot, but it is not coded in a way that is scalable. I've submitted a patch to improve it but I didn't had an answer yet. It's the 2nd time that I try to contribute to this tool, but to no avail... Maybe I should just 'fork' it.
Maybe you can contact the author(s) directly by email? And point them to the bug reports perhaps?


Maybe as a rather hacky workaround you could create a second file with just the last however many lines you need to check from /var/log/emerge.log and get genlop to parse that?

Code:
jw@Andornor ~ $ time genlop qt-webkit
 * x11-libs/qt-webkit

     Thu Nov  5 16:10:39 2009 >>> x11-libs/qt-webkit-4.5.3

real    0m2.162s
user    0m0.762s
sys     0m0.038s
jw@Andornor ~ $ tail -n100 /var/log/emerge.log > /tmp/quietemerge-short-emerge.log && time genlop -f /tmp/quietemerge-short-emerge.log qt-webkit
using logfile /tmp/quietemerge-short-emerge.log
 * x11-libs/qt-webkit

     Thu Nov  5 16:10:39 2009 >>> x11-libs/qt-webkit-4.5.3

real    0m0.346s
user    0m0.151s
sys     0m0.025s


Seems to make a lot of difference. I'm currently compiling, so the real time is a bit skewed, but you can see the big drop in user time.
Back to top
View user's profile Send private message
jprobichaud
Tux's lil' helper
Tux's lil' helper


Joined: 28 Jan 2009
Posts: 81
Location: Montreal, Qc

PostPosted: Thu Nov 05, 2009 5:47 am    Post subject: Reply with quote

ppurka wrote:
Maybe you can contact the author(s) directly by email? And point them to the bug reports perhaps?


Every time the bug gets 'touched' they receive an email, maybe there will be some action at some point...

Cheers
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3256

PostPosted: Thu Nov 05, 2009 6:21 am    Post subject: Reply with quote

jw5801 wrote:
Maybe as a rather hacky workaround you could create a second file with just the last however many lines you need to check from /var/log/emerge.log and get genlop to parse that?
Interesting. I will keep this in mind for a later revision.
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3256

PostPosted: Fri Nov 13, 2009 1:17 am    Post subject: Reply with quote

Update to version 20091112. Changelog:
  • Bug fix: ACCESS VIOLATION SUMMARY wasn't being caught. This might still be buggy :-/
  • Bug fix: Some packages would output many lines on the countdown.
  • Bug fix: --resume didn't use to update term title.
  • Show preserved libs found and do conf update even if some package has failed.
  • Show the # of failed, # of skipped and # of packages left to emerge.
  • Thanks to jw5801 and jprobichaud for discussion and ideas regarding usage of genlop. genlop is slightly faster now in showing merge time.
  • Broke off all common portions of the script into their own functions.

The failed packages, etc are shown like this:
Code:
-------------------------------- Starting emerge -------------------------------

  * Install  rox-base/rox-lib-2.0.5-r1.                                  Error!
  * Skipped  rox-base/thumbs-0.1.4 due to --keep-going
  * Install  app-admin/eselect-1.2.3.                                11 seconds.
  * [ ERROR!! ] Emerge: 1 failed, 1 skipped, 0 left
    See /tmp/output.log.
   
  * Preserved libraries found: See /tmp/output.log

_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/
Back to top
View user's profile Send private message
jw5801
Apprentice
Apprentice


Joined: 12 Jun 2008
Posts: 251
Location: Melbourne, Australia

PostPosted: Wed Nov 18, 2009 12:19 am    Post subject: Reply with quote

Hmm... interesting point raised by Patrick Lauer in a blog post.

I've just discovered the EMERGE_DEFAULT_OPTS make.conf variable, which comes in handy (before I was creating an alias to emerge, so that --ask was the default), however I've noticed that your script doesn't handle --ask when it's passed via EMERGE_DEFAULT_OPTS. If you can work out a way to detect --ask being passed in this form, that would be awesome, otherwise it might be best to add "--ignore-default-opts" to the emerge calls in the script.
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3256

PostPosted: Wed Nov 18, 2009 1:28 am    Post subject: Reply with quote

jw5801 wrote:
Hmm... interesting point raised by Patrick Lauer in a blog post.

I've just discovered the EMERGE_DEFAULT_OPTS make.conf variable, which comes in handy (before I was creating an alias to emerge, so that --ask was the default), however I've noticed that your script doesn't handle --ask when it's passed via EMERGE_DEFAULT_OPTS. If you can work out a way to detect --ask being passed in this form, that would be awesome, otherwise it might be best to add "--ignore-default-opts" to the emerge calls in the script.
Does --pretend even work when you have --ask in your EMERGE_DEFAULT_OPTS?

I think I can not even ignore the EMERGE_DEFAULT_OPTS since some people may have options other than --ask (for example --with-bdeps=y) which do not affect this script, and which will actually enhance/modify their emerge behaviour. If I ignore the default opts, then this script will not work as I intended it to,- that is, to be a simple way of showing what emerge is doing without interfering with emerge itself.

EDIT: -p seems to work even with --ask in EMERGE_DEFAULT_OPTS. But, I can see where the problem will be. I will be backgrounding emerge by emerge -q pkg >& log_file and then emerge will wait for an answer. In fact, when I tried it out just now, the emerge process had stopped:
Code:

~>  EMERGE_DEFAULT_OPTS="--ask" emerge -q rox  >& /dev/null &
[1] 10673
~>
[1]  + suspended (tty input)  EMERGE_DEFAULT_OPTS="--ask" emerge -q rox >&/dev/null
So, unless you foreground emerge you will not be able to pass any options to emerge. I think there may be other complications too. Best option would be to launch this script as EMERGE_DEFAULT_OPTS="" quietemerge <other options>. You can create an alias for this.
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/
Back to top
View user's profile Send private message
jw5801
Apprentice
Apprentice


Joined: 12 Jun 2008
Posts: 251
Location: Melbourne, Australia

PostPosted: Wed Nov 18, 2009 1:47 am    Post subject: Reply with quote

ppurka wrote:
jw5801 wrote:
Hmm... interesting point raised by Patrick Lauer in a blog post.

I've just discovered the EMERGE_DEFAULT_OPTS make.conf variable, which comes in handy (before I was creating an alias to emerge, so that --ask was the default), however I've noticed that your script doesn't handle --ask when it's passed via EMERGE_DEFAULT_OPTS. If you can work out a way to detect --ask being passed in this form, that would be awesome, otherwise it might be best to add "--ignore-default-opts" to the emerge calls in the script.

Does --pretend even work when you have --ask in your EMERGE_DEFAULT_OPTS?


Yeah, if you pass both --pretend and --ask to emerge, it ignores --ask.

Quote:
I think I can not even ignore the EMERGE_DEFAULT_OPTS since some people may have options other than --ask (for example --with-bdeps=y) which do not affect this script, and which will actually enhance/modify their emerge behaviour. If I ignore the default opts, then this script will not work as I intended it to,- that is, to be a simple way of showing what emerge is doing without interfering with emerge itself.


Yeah, that's a very good point. I wonder if there's a way to specifically ignore --ask, using sed, or some such? I'll try and hack something together to parse EMERGE_DEFAULT_OPTS and post back.
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3256

PostPosted: Wed Nov 18, 2009 1:50 am    Post subject: Reply with quote

jw5801 wrote:
Yeah, that's a very good point. I wonder if there's a way to specifically ignore --ask, using sed, or some such? I'll try and hack something together to parse EMERGE_DEFAULT_OPTS and post back.
I am sorry, I just tried out some things and edited my earlier post. See the edit for why it is not a good idea to have --ask and background emerge.

BTW, you don't need to hack anything. My script already does all the necessary hacking when it parses the command line arguments. I won't need to rewrite much, but my above comments still hold.
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/
Back to top
View user's profile Send private message
jw5801
Apprentice
Apprentice


Joined: 12 Jun 2008
Posts: 251
Location: Melbourne, Australia

PostPosted: Wed Nov 18, 2009 2:06 am    Post subject: Reply with quote

ppurka wrote:
EDIT: -p seems to work even with --ask in EMERGE_DEFAULT_OPTS. But, I can see where the problem will be. I will be backgrounding emerge by emerge -q pkg >& log_file and then emerge will wait for an answer. In fact, when I tried it out just now, the emerge process had stopped:
Code:

~>  EMERGE_DEFAULT_OPTS="--ask" emerge -q rox  >& /dev/null &
[1] 10673
~>
[1]  + suspended (tty input)  EMERGE_DEFAULT_OPTS="--ask" emerge -q rox >&/dev/null
So, unless you foreground emerge you will not be able to pass any options to emerge. I think there may be other complications too. Best option would be to launch this script as EMERGE_DEFAULT_OPTS="" quietemerge <other options>. You can create an alias for this.


Yeah, that's essentially what I've done.

Quote:
BTW, you don't need to hack anything. My script already does all the necessary hacking when it parses the command line arguments. I won't need to rewrite much, but my above comments still hold.


I was thinking perhaps something to read EMERGE_DEFAULT_OPTS, catch everything that isn't '--ask' and add it to the options passed on the command line, then run emerge with --ignore-default-opts and the options parsed from EMERGE_DEFAULT_OPTS as well as those read from stdin? Obviously if --ask is detected in EMERGE_DEFAULT_OPTS you'd then set e_opt_ask=1.

I guess that's probably similar to what you've done for parsing the command line arguments.
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3256

PostPosted: Wed Nov 18, 2009 4:58 am    Post subject: Reply with quote

jw5801 wrote:
I was thinking perhaps something to read EMERGE_DEFAULT_OPTS, catch everything that isn't '--ask' and add it to the options passed on the command line, then run emerge with --ignore-default-opts and the options parsed from EMERGE_DEFAULT_OPTS as well as those read from stdin? Obviously if --ask is detected in EMERGE_DEFAULT_OPTS you'd then set e_opt_ask=1.

I guess that's probably similar to what you've done for parsing the command line arguments.
Yes. I got another idea with not providing EMERGE_DEFAULT_OPTS in the emerge command line itself. I will try it out and update the script if it works. Otherwise I will use this method that you mention.
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3256

PostPosted: Fri Nov 20, 2009 5:32 am    Post subject: Reply with quote

Update to version 20091120: Changelog:
* Respect --ask in EMERGE_DEFAULT_OPTS

@jw5801 My idea didn't work out :cry: I wanted to do emerge <<<'yes' > log_file &. But it didn't work for some reason. I can't be bothered to figure out why. Your method works just fine.
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/
Back to top
View user's profile Send private message
jw5801
Apprentice
Apprentice


Joined: 12 Jun 2008
Posts: 251
Location: Melbourne, Australia

PostPosted: Fri Nov 20, 2009 5:57 am    Post subject: Reply with quote

ppurka wrote:
Update to version 20091120: Changelog:
* Respect --ask in EMERGE_DEFAULT_OPTS

@jw5801 My idea didn't work out :cry: I wanted to do emerge <<<'yes' > log_file &. But it didn't work for some reason. I can't be bothered to figure out why. Your method works just fine.


Oo, that would be very neat, not sure why it isn't working though... apparently emerge thinks it isn't running in a terminal if you do it that way.
Back to top
View user's profile Send private message
0x4a47
n00b
n00b


Joined: 24 Mar 2004
Posts: 18

PostPosted: Fri Nov 20, 2009 10:39 am    Post subject: Reply with quote

hi,

just found and tried quietemerge (Version: 20091120). i think it doesn't handle fetch restriction packages well:
Code:

# quietemerge -u --deep --newuse world
------------------------------------------------------ Pretended emerge -------------------------------------------------------

These are the packages that would be merged, in order:

Calculating dependencies ... done!
[ebuild  N F  ] dev-java/ibm-jdk-bin-1.4.2.13  USE="X alsa -doc -examples -javacomm (-nsplugin)" 57,710 kB [0]
[ebuild   R   ] mail-client/claws-mail-3.7.3  USE="bogofilter crypt dbus dillo gnome gnutls imap ipv6 ldap nntp session smime spamassassin ssl xface -doc -pda -spell -startup-notification (-kde%*)" 0 kB [0=>1]                                             

Total: 2 packages (1 new, 1 reinstall), Size of downloads: 57,710 kB
Fetch Restriction: 1 package (1 unsatisfied)                       
Portage tree and overlays:                                         
 [0] /usr/portage                                                   
 [1] /usr/local/portage                                             
                                                     Total ETA: 4 minutes.

------------------------------------------------------- Starting emerge -------------------------------------------------------

  * Install  dev-java/ibm-jdk-bin-1.4.2.13.                                                                              Done!


of course nothing has been "done" :) because the package couldn't be downloaded. the other reinstall of "claws-mail" didn't process either of course. if i set "SHOW_MERGE_TIME=1" then it just exits without any information (no "done") of what just happened (because ibm-jdk-bin is new and has never been installed).

the file /tmp/output.log shows the whole message of the ibm-jdk-bin package and where to download it though.
is there a way to make quietemerge more friendly with such packages?


another thing is security related, called "symlink attack" (e.g. see http://www.infosecwriters.com/texts.php?op=display&id=159 ). could you please change the hardcoded TMPFILE=/tmp/output.log in line 314 to TMPFILE=`mktemp`. for users being able to find the tmp file location for debug output you'd have to output the path somewhere while running quietemerge.
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3256

PostPosted: Fri Nov 20, 2009 12:44 pm    Post subject: Reply with quote

0x4a47 wrote:
hi,

just found and tried quietemerge (Version: 20091120). i think it doesn't handle fetch restriction packages well:
Code:

# quietemerge -u --deep --newuse world
------------------------------------------------------ Pretended emerge -------------------------------------------------------

These are the packages that would be merged, in order:

Calculating dependencies ... done!
[ebuild  N F  ] dev-java/ibm-jdk-bin-1.4.2.13  USE="X alsa -doc -examples -javacomm (-nsplugin)" 57,710 kB [0]
[ebuild   R   ] mail-client/claws-mail-3.7.3  USE="bogofilter crypt dbus dillo gnome gnutls imap ipv6 ldap nntp session smime spamassassin ssl xface -doc -pda -spell -startup-notification (-kde%*)" 0 kB [0=>1]                                             

Total: 2 packages (1 new, 1 reinstall), Size of downloads: 57,710 kB
Fetch Restriction: 1 package (1 unsatisfied)                       
Portage tree and overlays:                                         
 [0] /usr/portage                                                   
 [1] /usr/local/portage                                             
                                                     Total ETA: 4 minutes.

------------------------------------------------------- Starting emerge -------------------------------------------------------

  * Install  dev-java/ibm-jdk-bin-1.4.2.13.                                                                              Done!


of course nothing has been "done" :) because the package couldn't be downloaded. the other reinstall of "claws-mail" didn't process either of course. if i set "SHOW_MERGE_TIME=1" then it just exits without any information (no "done") of what just happened (because ibm-jdk-bin is new and has never been installed).

the file /tmp/output.log shows the whole message of the ibm-jdk-bin package and where to download it though.
is there a way to make quietemerge more friendly with such packages?
Very good! I think I don't have any fetch restricted files in my system which is why I never ran into this bug (I recently did a quietemerge -e world). I will update the script soon to catch such events.
Quote:
another thing is security related, called "symlink attack" (e.g. see http://www.infosecwriters.com/texts.php?op=display&id=159 ). could you please change the hardcoded TMPFILE=/tmp/output.log in line 314 to TMPFILE=`mktemp`. for users being able to find the tmp file location for debug output you'd have to output the path somewhere while running quietemerge.
That is something I was unaware of. The TMPFILE I create is created with the id of root and uses default permissions which gives only read attribute to other users. If you have another malicious user with root privileges then it is really unfortunate. I am against changing this to mktemp since it will create a bunch of files (over time) in /tmp which I want to avoid. Secondly there is also the case that the users will have to remember to read my output carefully everytime.

What I will do though is delete the file just before redirecting to it. That should mitigate any security concerns. There is a final security concern and that is what happens when your malicious root user changes the file during compilation. That is some vulnerability which can happen even with mktemp, so I don't think it is something which can be avoided.

I guess I can use mktemp for some other tmpfiles in the script. That I will do.
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6  Next
Page 2 of 6

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum