Forums

Skip to content

Advanced search
  • Quick links
    • Unanswered topics
    • Active topics
    • Search
  • FAQ
  • Login
  • Register
  • Board index Assistance Unsupported Software
  • Search

eprogress: track an ebuild's progress

This forum covers all Gentoo-related software not officially supported by Gentoo. Ebuilds/software posted here might harm the health and stability of your system(s), and are not supported by Gentoo developers. Bugs/errors caused by ebuilds from overlays.gentoo.org are covered by this forum, too.
Post Reply
Advanced search
19 posts • Page 1 of 1
Author
Message
Gherald2
Guru
Guru
User avatar
Posts: 326
Joined: Wed Jul 02, 2003 9:09 pm
Location: Madison, WI USA
Contact:
Contact Gherald2
Website

eprogress: track an ebuild's progress

  • Quote

Post by Gherald2 » Wed Jul 14, 2004 4:03 am

Ever wondered how long an ebuild is going to take? Here's a quick script to determine how far along it is.

Or perhaps you'd like to see a list of all previous failed/interrupted ebuilds? Just run it w/o arguements and you'll get a pretty good idea.

Code: Select all

#!/bin/bash
#eprogress v0.0.1, July 13, 2004

#When given a package name this scipt is will track the progress of an ebuild by counting the number of .c and .cpp files in the portage temp directory and comparing it to the number of .o files that have been compiled so far.

#When run without arguements it's useful for locating previous failed or interrupted ebuilds so they can be attempted again or deleted manually to free space.

cd /var/tmp/portage
printf "%40s%8s%8s%8s%9s" "PACKAGE" "TOTAL" "LEFT" "DONE" "PERCENT"; echo 
for i in $1*; do
	A=`find $i/ -iname "*.c*" | wc -l`
	if test $A -eq 0; then continue; fi
	B=`find $i/ -iname "*.o*" | wc -l`
	C=$[$A-$B]
	D=`expr $B \* 100 / $A`
	if test $D -gt 100; then D="??"; fi
	printf "%40s%8s%8s%8s%7s" "$i" "$A" "$C" "$B" "$D"; echo
done
TODO:

-add java and ".C" file support
-add an interactive mode that rescans every n seconds.
-rename variables

Note that someone attempted a full progress bar here but the project seems abandoned :(

If you know a better way to get this info, let us know!
Last edited by Gherald2 on Wed Jul 14, 2004 3:01 pm, edited 1 time in total.
Unregistered Linux User #17598363
Top
pjp
Administrator
Administrator
User avatar
Posts: 20668
Joined: Tue Apr 16, 2002 10:35 pm

  • Quote

Post by pjp » Wed Jul 14, 2004 4:58 am

I've wanted to see something like this. Just some options to consider:
  • Listing file sizes of object being compiled. Perhaps a list of current, plus next 3 (or 5, 10, etc.).
  • A (1 of 35) type counter.
As for knowing a better way to get the info, perhaps using python and the portage system would be more handy. I've only poked at it a little though, so I can't be specific.
Quis separabit? Quo animo?
Top
neenee
Veteran
Veteran
User avatar
Posts: 1786
Joined: Sun Jul 20, 2003 12:15 pm

  • Quote

Post by neenee » Wed Jul 14, 2004 10:36 am

here is a thread with two other scripts which show progress.

but i like how the output of your script is more detailed :wink:
Top
nelz
n00b
n00b
Posts: 19
Joined: Thu Apr 24, 2003 7:30 am
Location: Warrington, UK

  • Quote

Post by nelz » Tue Jul 20, 2004 11:04 am

This script fails if you have PORTAGE_TMPDIR set to anywhere other than /var/tmp.

Replace

Code: Select all

cd /var/tmp/portage
with

Code: Select all

source /etc/make.conf
cd $PORTAGE_TMPDIR/portage
[/code]
Top
Gherald2
Guru
Guru
User avatar
Posts: 326
Joined: Wed Jul 02, 2003 9:09 pm
Location: Madison, WI USA
Contact:
Contact Gherald2
Website

  • Quote

Post by Gherald2 » Tue Jul 20, 2004 11:25 am

Good idea. I'll just leave the post as-is though, unless more people are interested in seeing the script improved.

For such a quick hack, to me it seems usable enough as-is.

But feel free to post any other modifications you think of :)
Unregistered Linux User #17598363
Top
xchris
Advocate
Advocate
User avatar
Posts: 2824
Joined: Thu Jul 10, 2003 10:21 pm

  • Quote

Post by xchris » Thu Jul 29, 2004 2:24 pm

i really like this script!!!
10x
while True:Gentoo()
Top
dub.wav
Tux's lil' helper
Tux's lil' helper
Posts: 149
Joined: Wed Apr 09, 2003 12:55 pm
Location: Norway

  • Quote

Post by dub.wav » Tue Aug 10, 2004 1:42 pm

I've written a python version of the bash script above, which does the same thing, except that:
* it adds the following arguments: -i for interactive mode, and -d for number of seconds between updates.
* the TOTAL header comes after DONE.
* it's much faster than the bash script.
example usage:

Code: Select all

eprogress.py gedit
eprogress.py gedit -i -d2 # default delay is 5 seconds
eprogress.py \* # works like the bash script without args.
Btw, I think that it would be a better idea to use a list of known source extensions/object extensions instead of wildcards.
How many different extensions are there? (For objects I've personally only seen .o, and for source .c, .cpp.)
The script is probably a bit too large to post, but I don't have any webspace to use, so here goes.

Code: Select all

import portage, glob, sys, os, fnmatch, time
from optparse import OptionParser

def err(msg, fatal=True, ret=1):
	print >>sys.stderr, "Error:", msg
	if fatal:
		sys.exit(ret)

if os.getuid() != 0:
	err("You have to be root to run this script.")

parser = OptionParser(usage="%prog [options] progname",
		version="%prog 0.0.1")
parser.add_option("-i", action="store_true", 
		dest="interactive", default=False, 
		help="Interactive mode")
parser.add_option("-d", type="int", dest="delay", default=5,
		help="Number of seconds between updates in interactive mode")

options, args = parser.parse_args()
if len(args) != 1:
	parser.print_help()
	sys.exit(1)
else:
	progname = args[0]

for cfg_file in "/etc/make.conf", "/etc/make.globals":
	try:
		portage_tmpdir = portage.getconfig(cfg_file)["PORTAGE_TMPDIR"]
		break
	except:
		portage_tmpdir = False
		
if not portage_tmpdir:
	err("Could not find PORTAGE_TMPDIR.")

portage_tmpdir += "/portage"

p = os.popen("clear")
clear = p.read()
if p.close() != None:
	clear = False

os.chdir(portage_tmpdir)

#
header_format = "%40s%8s%8s%8s%9s"
result_format = "%40s%8s%8s%8s%7s"
# Borrowed from Gherald's bash script.

while True:
	dirs = []
	dirs_tmp = glob.glob("%s*/work/*" % progname)
	for dir in dirs_tmp:
		if os.path.isdir(dir):
			dirs.append(dir)
	if clear:
		sys.stdout.write(clear)
	for dir in dirs:
		package = dir.split('/')[0]
		source = []
		object = []
		print header_format % ("PACKAGE", "DONE", "TOTAL", "LEFT", "PERCENT")
		for root, dirs, files in os.walk(dir):
			for file in files:
				ext = os.path.splitext(file)[1][1:].lower()
				if fnmatch.fnmatch(ext, "c*"):
					source.append(file)
				elif fnmatch.fnmatch(ext, "o"):
					object.append(file)
						
		total = len(source)
		left = total - len(object)
		done = len(object)
		try:
			percent = done * 100 / total
		except:
			pass
		print result_format % (package, done, total, left, percent)
	if not options.interactive:
		break
	else:
		time.sleep(options.delay)
Top
discomfitor
l33t
l33t
User avatar
Posts: 927
Joined: Fri Feb 21, 2003 11:51 pm
Location: None

  • Quote

Post by discomfitor » Tue Aug 10, 2004 8:35 pm

Seems like it could use a bit more work; on gnumeric I supposedly had 33 files left to compile and was at 91% as it was unmerging the old one and finishing. I like how it works though...reminds me of this project, but without the cool little bar. Then again, the other one would break a lot of emerges.

Keep up the good ideas!
There is no substitute for experience.
Imperfection indicates a lack of effort.
Top
Gherald2
Guru
Guru
User avatar
Posts: 326
Joined: Wed Jul 02, 2003 9:09 pm
Location: Madison, WI USA
Contact:
Contact Gherald2
Website

  • Quote

Post by Gherald2 » Wed Aug 11, 2004 4:48 am

Nice going dub.wav, I've been toying with python lately and your version looks like it'll be fun to play around with.

Darckness, these scripts are based on the assumption that the ratio of .o to .c* files will give a crude indication of the progress. There will always be some ebuilds that don't behave this way, but with portage-ng (which will likely have a full-fledged plugin interface) on the roadmap those special cases aren't really worth investigating. And I did mention the project you linked to.. as you said, it breaks a lot of emerges and of course involves editing portage. I wanted something less obtrusive :)
Unregistered Linux User #17598363
Top
discomfitor
l33t
l33t
User avatar
Posts: 927
Joined: Fri Feb 21, 2003 11:51 pm
Location: None

  • Quote

Post by discomfitor » Wed Aug 11, 2004 6:17 am

Yeah, my eyes have been failing me in the color spectrum recently, so I missed your reference to the deceased project. It seems that for larger packages it becomes increasingly inaccurate. Mozilla was an entertaining one; it finished at 33%.
There is no substitute for experience.
Imperfection indicates a lack of effort.
Top
fallow
Bodhisattva
Bodhisattva
User avatar
Posts: 2208
Joined: Thu Jan 08, 2004 12:53 pm
Location: Poland

  • Quote

Post by fallow » Mon Sep 06, 2004 7:57 am

I like the script from Gherald , nice 4 me .
Thx & Greetings ;)
"Time is a companion that goes with us on a journey. It reminds us to cherish each moment, because it will never come again. What we leave behind is not as important as how we have lived" J-L. Picard ;)
Top
Hackeron
Guru
Guru
Posts: 307
Joined: Fri Nov 01, 2002 2:44 pm

  • Quote

Post by Hackeron » Tue Mar 15, 2005 11:12 am

dub.wav wrote:I've written a python version of the bash script above, which does the same thing, except that:
* it adds the following arguments: -i for interactive mode, and -d for number of seconds between updates.
* the TOTAL header comes after DONE.
* it's much faster than the bash script.
What are you smoking? -- 99% of the work is done by os.walk which is about 40x slower than using find. I ran some benchmarks and for a larger build dir like glibc, os.walk takes up to 3 seconds while find takes 0.1 seconds!
Top
roo_
Tux's lil' helper
Tux's lil' helper
User avatar
Posts: 91
Joined: Sat Feb 07, 2004 7:53 pm

  • Quote

Post by roo_ » Sun Apr 17, 2005 3:18 pm

Hackeron wrote:
dub.wav wrote:I've written a python version of the bash script above, which does the same thing, except that:
* it adds the following arguments: -i for interactive mode, and -d for number of seconds between updates.
* the TOTAL header comes after DONE.
* it's much faster than the bash script.
What are you smoking? -- 99% of the work is done by os.walk which is about 40x slower than using find. I ran some benchmarks and for a larger build dir like glibc, os.walk takes up to 3 seconds while find takes 0.1 seconds!
The bash script is a ton faster.
Top
radarsat1
n00b
n00b
Posts: 32
Joined: Wed Jul 13, 2005 2:11 pm

  • Quote

Post by radarsat1 » Wed Jul 13, 2005 2:14 pm

I like this bash script!
I ran it with the following command line:

Code: Select all

watch "sudo ./eprogress \`grep === /var/log/emerge.log | tail -n 1 | cut -f 3 -d / | cut -f 1 -d -\`"
there's probably a better way to do it. but anyways it lets me watch the progress of the current package being compiled, whatever that happens to be. could probably parse the progress better and pipe it to zenity for a graphical progress bar.
(needs root access unfortunately)
Top
radarsat1
n00b
n00b
Posts: 32
Joined: Wed Jul 13, 2005 2:11 pm

  • Quote

Post by radarsat1 » Wed Jul 13, 2005 2:32 pm

hm, actually this one seems to work better:

Code: Select all

watch "sudo ./eprogress \`grep === /var/log/emerge.log | tail -n 1 | cut -f 3 -d / | cut -f 1 -d :\` 2>&1"
Top
Earthwings
Bodhisattva
Bodhisattva
User avatar
Posts: 7753
Joined: Mon Apr 14, 2003 8:13 pm
Location: Germany

  • Quote

Post by Earthwings » Wed Jul 13, 2005 2:43 pm

See also genlop (emerge genlop), especially

Code: Select all

genlop --current
KDE
Top
Cinder6
l33t
l33t
User avatar
Posts: 767
Joined: Thu Aug 05, 2004 4:33 am
Location: California
Contact:
Contact Cinder6
Website

  • Quote

Post by Cinder6 » Thu Jul 14, 2005 3:51 am

I have always wished that the -q option in emerge would just show a progressbar and write the standard output to a virtual screen, so if the emerge fails, it can still show you the error message.
Knowledge is power.
Power corrupts.
Study hard.
Be evil.

Ugly Overload
Top
cruise
n00b
n00b
Posts: 48
Joined: Mon Dec 01, 2003 5:32 pm
Contact:
Contact cruise
Website

Okay, my tweaked version

  • Quote

Post by cruise » Tue Sep 06, 2005 10:15 am

Code: Select all

#!/bin/bash
#eprogress v0.0.2, Sep 06, 2005

#When given a package name this scipt is will track the progress of an ebuild by counting the number of .c and .cpp files in the portage temp directory and comparing it to the number of .o files that have been compiled so far.

#When run without arguments it's useful for locating previous failed or interrupted ebuilds so they can be attempted again or deleted manually to free space.

PORTAGE_TMPDIR=/var/tmp
source /etc/make.conf
cd ${PORTAGE_TMPDIR}/portage

printf "%40s%8s%8s%8s%9s" "PACKAGE" "TOTAL" "LEFT" "DONE" "PERCENT"; echo
for i in $1*; do
   A=`find $i/ -iname "*.c*" | wc -l`
   if test $A -eq 0; then continue; fi
   B=`find $i/ -iname "*.o" | egrep '/\.' | wc -l`
   C=$[$A-$B]
   D=`expr $B \* 100 / $A`
   if test $D -gt 100; then D="??"; fi
   printf "%40s%8s%8s%8s%7s" "$i" "$A" "$C" "$B" "$D"; echo
done
Added the check for a different tmp dir, and included an egrep to ignore anything with a leading '.' The CVS e17 version of edje, for example, seems to have copies of it's .o files in .lib directories, which really screws up the progress counting :P

Again, quick and dirty, but it works on mine system for the moment, so what the heck :D
"quantam sufficit"
[ cruise / casual-tempest.net / transference.org ]
Top
iplayfast
l33t
l33t
User avatar
Posts: 642
Joined: Mon Jul 08, 2002 5:48 pm
Location: Cambridge On,CA
Contact:
Contact iplayfast
Website

  • Quote

Post by iplayfast » Fri Sep 15, 2006 7:29 pm

Oh man I have to say. I really like what you've done with this. Seems like the original thread still has some activity on it as well. 8O http://forums.gentoo.org/viewtopic-t-13 ... rt-25.html
Top
Post Reply

19 posts • Page 1 of 1

Return to “Unsupported Software”

Jump to
  • Assistance
  • ↳   News & Announcements
  • ↳   Frequently Asked Questions
  • ↳   Installing Gentoo
  • ↳   Multimedia
  • ↳   Desktop Environments
  • ↳   Networking & Security
  • ↳   Kernel & Hardware
  • ↳   Portage & Programming
  • ↳   Gamers & Players
  • ↳   Other Things Gentoo
  • ↳   Unsupported Software
  • Discussion & Documentation
  • ↳   Documentation, Tips & Tricks
  • ↳   Gentoo Chat
  • ↳   Gentoo Forums Feedback
  • ↳   Duplicate Threads
  • International Gentoo Users
  • ↳   中文 (Chinese)
  • ↳   Dutch
  • ↳   Finnish
  • ↳   French
  • ↳   Deutsches Forum (German)
  • ↳   Diskussionsforum
  • ↳   Deutsche Dokumentation
  • ↳   Greek
  • ↳   Forum italiano (Italian)
  • ↳   Forum di discussione italiano
  • ↳   Risorse italiane (documentazione e tools)
  • ↳   Polskie forum (Polish)
  • ↳   Instalacja i sprzęt
  • ↳   Polish OTW
  • ↳   Portuguese
  • ↳   Documentação, Ferramentas e Dicas
  • ↳   Russian
  • ↳   Scandinavian
  • ↳   Spanish
  • ↳   Other Languages
  • Architectures & Platforms
  • ↳   Gentoo on ARM
  • ↳   Gentoo on PPC
  • ↳   Gentoo on Sparc
  • ↳   Gentoo on Alternative Architectures
  • ↳   Gentoo on AMD64
  • ↳   Gentoo for Mac OS X (Portage for Mac OS X)
  • Board index
  • All times are UTC
  • Delete cookies

© 2001–2026 Gentoo Foundation, Inc.

Powered by phpBB® Forum Software © phpBB Limited

Privacy Policy