View previous topic :: View next topic |
Author |
Message |
Gherald2 Guru
Joined: 02 Jul 2003 Posts: 326 Location: Madison, WI USA
|
Posted: Wed Jul 14, 2004 4:03 am Post subject: eprogress: track an ebuild's progress |
|
|
Ever wondered how long an ebuild is going to take? Here's a quick script to determine how far along it is.
Or perhaps you'd like to see a list of all previous failed/interrupted ebuilds? Just run it w/o arguements and you'll get a pretty good idea.
Code: | #!/bin/bash
#eprogress v0.0.1, July 13, 2004
#When given a package name this scipt is will track the progress of an ebuild by counting the number of .c and .cpp files in the portage temp directory and comparing it to the number of .o files that have been compiled so far.
#When run without arguements it's useful for locating previous failed or interrupted ebuilds so they can be attempted again or deleted manually to free space.
cd /var/tmp/portage
printf "%40s%8s%8s%8s%9s" "PACKAGE" "TOTAL" "LEFT" "DONE" "PERCENT"; echo
for i in $1*; do
A=`find $i/ -iname "*.c*" | wc -l`
if test $A -eq 0; then continue; fi
B=`find $i/ -iname "*.o*" | wc -l`
C=$[$A-$B]
D=`expr $B \* 100 / $A`
if test $D -gt 100; then D="??"; fi
printf "%40s%8s%8s%8s%7s" "$i" "$A" "$C" "$B" "$D"; echo
done |
TODO:
-add java and ".C" file support
-add an interactive mode that rescans every n seconds.
-rename variables
Note that someone attempted a full progress bar here but the project seems abandoned
If you know a better way to get this info, let us know! _________________ Unregistered Linux User #17598363
Last edited by Gherald2 on Wed Jul 14, 2004 3:01 pm; edited 1 time in total |
|
Back to top |
|
|
pjp Administrator
Joined: 16 Apr 2002 Posts: 20067
|
Posted: Wed Jul 14, 2004 4:58 am Post subject: |
|
|
I've wanted to see something like this. Just some options to consider:- Listing file sizes of object being compiled. Perhaps a list of current, plus next 3 (or 5, 10, etc.).
- A (1 of 35) type counter.
As for knowing a better way to get the info, perhaps using python and the portage system would be more handy. I've only poked at it a little though, so I can't be specific. _________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
neenee Veteran
Joined: 20 Jul 2003 Posts: 1786
|
Posted: Wed Jul 14, 2004 10:36 am Post subject: |
|
|
here is a thread with two other scripts which show progress.
but i like how the output of your script is more detailed |
|
Back to top |
|
|
nelz n00b
Joined: 24 Apr 2003 Posts: 19 Location: Warrington, UK
|
Posted: Tue Jul 20, 2004 11:04 am Post subject: |
|
|
This script fails if you have PORTAGE_TMPDIR set to anywhere other than /var/tmp.
Replace
Code: | cd /var/tmp/portage |
with
Code: | source /etc/make.conf
cd $PORTAGE_TMPDIR/portage |
[/code] |
|
Back to top |
|
|
Gherald2 Guru
Joined: 02 Jul 2003 Posts: 326 Location: Madison, WI USA
|
Posted: Tue Jul 20, 2004 11:25 am Post subject: |
|
|
Good idea. I'll just leave the post as-is though, unless more people are interested in seeing the script improved.
For such a quick hack, to me it seems usable enough as-is.
But feel free to post any other modifications you think of _________________ Unregistered Linux User #17598363 |
|
Back to top |
|
|
xchris Advocate
Joined: 10 Jul 2003 Posts: 2824
|
Posted: Thu Jul 29, 2004 2:24 pm Post subject: |
|
|
i really like this script!!!
10x _________________ while True:Gentoo() |
|
Back to top |
|
|
dub.wav Tux's lil' helper
Joined: 09 Apr 2003 Posts: 149 Location: Norway
|
Posted: Tue Aug 10, 2004 1:42 pm Post subject: |
|
|
I've written a python version of the bash script above, which does the same thing, except that:
* it adds the following arguments: -i for interactive mode, and -d for number of seconds between updates.
* the TOTAL header comes after DONE.
* it's much faster than the bash script.
example usage: Code: |
eprogress.py gedit
eprogress.py gedit -i -d2 # default delay is 5 seconds
eprogress.py \* # works like the bash script without args. |
Btw, I think that it would be a better idea to use a list of known source extensions/object extensions instead of wildcards.
How many different extensions are there? (For objects I've personally only seen .o, and for source .c, .cpp.)
The script is probably a bit too large to post, but I don't have any webspace to use, so here goes.
Code: | import portage, glob, sys, os, fnmatch, time
from optparse import OptionParser
def err(msg, fatal=True, ret=1):
print >>sys.stderr, "Error:", msg
if fatal:
sys.exit(ret)
if os.getuid() != 0:
err("You have to be root to run this script.")
parser = OptionParser(usage="%prog [options] progname",
version="%prog 0.0.1")
parser.add_option("-i", action="store_true",
dest="interactive", default=False,
help="Interactive mode")
parser.add_option("-d", type="int", dest="delay", default=5,
help="Number of seconds between updates in interactive mode")
options, args = parser.parse_args()
if len(args) != 1:
parser.print_help()
sys.exit(1)
else:
progname = args[0]
for cfg_file in "/etc/make.conf", "/etc/make.globals":
try:
portage_tmpdir = portage.getconfig(cfg_file)["PORTAGE_TMPDIR"]
break
except:
portage_tmpdir = False
if not portage_tmpdir:
err("Could not find PORTAGE_TMPDIR.")
portage_tmpdir += "/portage"
p = os.popen("clear")
clear = p.read()
if p.close() != None:
clear = False
os.chdir(portage_tmpdir)
#
header_format = "%40s%8s%8s%8s%9s"
result_format = "%40s%8s%8s%8s%7s"
# Borrowed from Gherald's bash script.
while True:
dirs = []
dirs_tmp = glob.glob("%s*/work/*" % progname)
for dir in dirs_tmp:
if os.path.isdir(dir):
dirs.append(dir)
if clear:
sys.stdout.write(clear)
for dir in dirs:
package = dir.split('/')[0]
source = []
object = []
print header_format % ("PACKAGE", "DONE", "TOTAL", "LEFT", "PERCENT")
for root, dirs, files in os.walk(dir):
for file in files:
ext = os.path.splitext(file)[1][1:].lower()
if fnmatch.fnmatch(ext, "c*"):
source.append(file)
elif fnmatch.fnmatch(ext, "o"):
object.append(file)
total = len(source)
left = total - len(object)
done = len(object)
try:
percent = done * 100 / total
except:
pass
print result_format % (package, done, total, left, percent)
if not options.interactive:
break
else:
time.sleep(options.delay) |
|
|
Back to top |
|
|
discomfitor l33t
Joined: 21 Feb 2003 Posts: 927 Location: None
|
Posted: Tue Aug 10, 2004 8:35 pm Post subject: |
|
|
Seems like it could use a bit more work; on gnumeric I supposedly had 33 files left to compile and was at 91% as it was unmerging the old one and finishing. I like how it works though...reminds me of this project, but without the cool little bar. Then again, the other one would break a lot of emerges.
Keep up the good ideas! _________________ There is no substitute for experience.
Imperfection indicates a lack of effort. |
|
Back to top |
|
|
Gherald2 Guru
Joined: 02 Jul 2003 Posts: 326 Location: Madison, WI USA
|
Posted: Wed Aug 11, 2004 4:48 am Post subject: |
|
|
Nice going dub.wav, I've been toying with python lately and your version looks like it'll be fun to play around with.
Darckness, these scripts are based on the assumption that the ratio of .o to .c* files will give a crude indication of the progress. There will always be some ebuilds that don't behave this way, but with portage-ng (which will likely have a full-fledged plugin interface) on the roadmap those special cases aren't really worth investigating. And I did mention the project you linked to.. as you said, it breaks a lot of emerges and of course involves editing portage. I wanted something less obtrusive _________________ Unregistered Linux User #17598363 |
|
Back to top |
|
|
discomfitor l33t
Joined: 21 Feb 2003 Posts: 927 Location: None
|
Posted: Wed Aug 11, 2004 6:17 am Post subject: |
|
|
Yeah, my eyes have been failing me in the color spectrum recently, so I missed your reference to the deceased project. It seems that for larger packages it becomes increasingly inaccurate. Mozilla was an entertaining one; it finished at 33%. _________________ There is no substitute for experience.
Imperfection indicates a lack of effort. |
|
Back to top |
|
|
fallow Bodhisattva
Joined: 08 Jan 2004 Posts: 2208 Location: Poland
|
Posted: Mon Sep 06, 2004 7:57 am Post subject: |
|
|
I like the script from Gherald , nice 4 me .
Thx & Greetings _________________ "Time is a companion that goes with us on a journey. It reminds us to cherish each moment, because it will never come again. What we leave behind is not as important as how we have lived" J-L. Picard |
|
Back to top |
|
|
Hackeron Guru
Joined: 01 Nov 2002 Posts: 307
|
Posted: Tue Mar 15, 2005 11:12 am Post subject: |
|
|
dub.wav wrote: | I've written a python version of the bash script above, which does the same thing, except that:
* it adds the following arguments: -i for interactive mode, and -d for number of seconds between updates.
* the TOTAL header comes after DONE.
* it's much faster than the bash script. | What are you smoking? -- 99% of the work is done by os.walk which is about 40x slower than using find. I ran some benchmarks and for a larger build dir like glibc, os.walk takes up to 3 seconds while find takes 0.1 seconds! |
|
Back to top |
|
|
roo_ Tux's lil' helper
Joined: 07 Feb 2004 Posts: 91
|
Posted: Sun Apr 17, 2005 3:18 pm Post subject: |
|
|
Hackeron wrote: | dub.wav wrote: | I've written a python version of the bash script above, which does the same thing, except that:
* it adds the following arguments: -i for interactive mode, and -d for number of seconds between updates.
* the TOTAL header comes after DONE.
* it's much faster than the bash script. | What are you smoking? -- 99% of the work is done by os.walk which is about 40x slower than using find. I ran some benchmarks and for a larger build dir like glibc, os.walk takes up to 3 seconds while find takes 0.1 seconds! |
The bash script is a ton faster. |
|
Back to top |
|
|
radarsat1 n00b
Joined: 13 Jul 2005 Posts: 32
|
Posted: Wed Jul 13, 2005 2:14 pm Post subject: |
|
|
I like this bash script!
I ran it with the following command line:
Code: | watch "sudo ./eprogress \`grep === /var/log/emerge.log | tail -n 1 | cut -f 3 -d / | cut -f 1 -d -\`" |
there's probably a better way to do it. but anyways it lets me watch the progress of the current package being compiled, whatever that happens to be. could probably parse the progress better and pipe it to zenity for a graphical progress bar.
(needs root access unfortunately) |
|
Back to top |
|
|
radarsat1 n00b
Joined: 13 Jul 2005 Posts: 32
|
Posted: Wed Jul 13, 2005 2:32 pm Post subject: |
|
|
hm, actually this one seems to work better:
Code: |
watch "sudo ./eprogress \`grep === /var/log/emerge.log | tail -n 1 | cut -f 3 -d / | cut -f 1 -d :\` 2>&1"
|
|
|
Back to top |
|
|
Earthwings Bodhisattva
Joined: 14 Apr 2003 Posts: 7753 Location: Germany
|
Posted: Wed Jul 13, 2005 2:43 pm Post subject: |
|
|
See also genlop (emerge genlop), especially _________________ KDE |
|
Back to top |
|
|
Cinder6 l33t
Joined: 05 Aug 2004 Posts: 767 Location: California
|
Posted: Thu Jul 14, 2005 3:51 am Post subject: |
|
|
I have always wished that the -q option in emerge would just show a progressbar and write the standard output to a virtual screen, so if the emerge fails, it can still show you the error message. _________________ Knowledge is power.
Power corrupts.
Study hard.
Be evil.
Ugly Overload |
|
Back to top |
|
|
cruise n00b
Joined: 01 Dec 2003 Posts: 48
|
Posted: Tue Sep 06, 2005 10:15 am Post subject: Okay, my tweaked version |
|
|
Code: |
#!/bin/bash
#eprogress v0.0.2, Sep 06, 2005
#When given a package name this scipt is will track the progress of an ebuild by counting the number of .c and .cpp files in the portage temp directory and comparing it to the number of .o files that have been compiled so far.
#When run without arguments it's useful for locating previous failed or interrupted ebuilds so they can be attempted again or deleted manually to free space.
PORTAGE_TMPDIR=/var/tmp
source /etc/make.conf
cd ${PORTAGE_TMPDIR}/portage
printf "%40s%8s%8s%8s%9s" "PACKAGE" "TOTAL" "LEFT" "DONE" "PERCENT"; echo
for i in $1*; do
A=`find $i/ -iname "*.c*" | wc -l`
if test $A -eq 0; then continue; fi
B=`find $i/ -iname "*.o" | egrep '/\.' | wc -l`
C=$[$A-$B]
D=`expr $B \* 100 / $A`
if test $D -gt 100; then D="??"; fi
printf "%40s%8s%8s%8s%7s" "$i" "$A" "$C" "$B" "$D"; echo
done
|
Added the check for a different tmp dir, and included an egrep to ignore anything with a leading '.' The CVS e17 version of edje, for example, seems to have copies of it's .o files in .lib directories, which really screws up the progress counting :P
Again, quick and dirty, but it works on mine system for the moment, so what the heck :D _________________ "quantam sufficit"
[ cruise / casual-tempest.net / transference.org ] |
|
Back to top |
|
|
iplayfast l33t
Joined: 08 Jul 2002 Posts: 642 Location: Cambridge On,CA
|
|
Back to top |
|
|
|