gest: perl script to estimate emerge times

Message

lanalyst · Post by **lanalyst** » Sat Jul 26, 2003 6:26 pm

I've hacked a perl script to estimate build times based on previous averages taken from emerge.log. To use it, pipe the output of the emerge --pretend command to it and it will tag duration for each package with a total to the output.

Script is here (updated URL 12/05). Requires emerge Time-Duration and chmod +x gest.

Examples:
emerge -up world | gest
emerge -ep | gest
emerge -p xfree | gest

Sample output:
bash-2.05b# emerge -p kdebase | /root/gest

These are the packages that I would merge, in order:

Calculating dependencies ...done!
[ebuild U ] kde-base/arts-1.1.2-r1 [1.1.1] Estimate: 27m12s
[ebuild U ] app-doc/kdelibs-apidocs-3.1.2 [3.1.1a] Estimate: 19m46s
[ebuild U ] kde-base/kdelibs-3.1.2 [3.1.1a] Estimate: 4h11m
[ebuild U ] kde-base/kdebase-3.1.2 [3.1.1a] Estimate: 5h36m

Total estimated update time: 10 hours and 34 minutes

Enjoy

Dave

pYrania · Post by **pYrania** » Sat Jul 26, 2003 7:03 pm

Hm. I don't think my cluster can do it _that_ fast

Code: Select all

$ emerge -p kde | ./gest

These are the packages that I would merge, in order:

Calculating dependencies  ...done!
[ebuild   R  ] kde-base/kde-3.1.2   Estimate: 12s

Total estimated update time: 12 seconds

btw: it is chmod +x to assign execution flag to the file.

lanalyst · Post by **lanalyst** » Sat Jul 26, 2003 7:23 pm

kde-base/kde really isn't much of an ebuild.. it has a few deps:

Code: Select all

% emerge -p  kde-base/kdelibs kde-base/kdebase kde-base/kdeaddons kde-base/kdeadmin kde-base/kdeartwork kde-base/kdeedu kde-base/kdegames kde-base/kdegraphics kde-base/kdemultimedia kde-base/kdenetwork kde-base/kdepim kde-base/kdetoys kde-base/kdeutils | ./gest
 
These are the packages that I would merge, in order:
 
Calculating dependencies  ...done!
[ebuild   R  ] kde-base/kdelibs-3.1.2   Estimate: 1h7m
[ebuild   R  ] kde-base/kdebase-3.1.2   Estimate: 1h49m
[ebuild   R  ] kde-base/kdeaddons-3.1.2   Estimate: 33m22s
[ebuild   R  ] kde-base/kdeadmin-3.1.2   Estimate: 13m32s
[ebuild   R  ] kde-base/kdeartwork-3.1.2   Estimate: 12m44s
[ebuild   R  ] kde-base/kdeedu-3.1.2   Estimate: 33m10s
[ebuild   R  ] kde-base/kdegames-3.1.2   Estimate: 41m32s
[ebuild   R  ] kde-base/kdegraphics-3.1.2   Estimate: 40m20s
[ebuild   R  ] kde-base/kdemultimedia-3.1.2-r1   Estimate: 1h59m
[ebuild   R  ] kde-base/kdenetwork-3.1.2   Estimate: 48m31s
[ebuild   R  ] kde-base/kdepim-3.1.2   Estimate: 1h4m
[ebuild   R  ] kde-base/kdetoys-3.1.2   Estimate: 7m38s
[ebuild   R  ] kde-base/kdeutils-3.1.2   Estimate: 21m34s
 
Total estimated update time: 10 hours and 11 minutes

Thanks for the pointer on the typo..

pYrania · Post by **pYrania** » Sat Jul 26, 2003 8:30 pm

oh right, forgot it's actually just a meta package

Post by **Pythonhead** » Sat Jul 26, 2003 10:06 pm

Very nifty, its working great for me. I added gest to this list of portage utilities that aren't in portage:

http://forums.gentoo.org/viewtopic.php?p=416282#416282

Here's a feature request for people who use ccache.

1) If a package has been emerged more than once, leave out the first compile time in your list of averages because succeeding packages compile in a fraction of the amount of time.

2)If a package has only been emerged once, note that in the output, so they'll be aware the next actual emerge time will be quite lower.

EDIT: Or much easier, just show the number of times its been emerged and ccache users can figure it out themselves.

tomaw · Post by **tomaw** » Sun Jul 27, 2003 8:08 am

Very nice, my suggestion though, somehow not lose the colours of the output - I find them quite useful.

lanalyst · Post by **lanalyst** » Mon Jul 28, 2003 1:55 am

I've posted a new version (at the original link).

I've cleaned the code up a bit and have added a few options (no color.. sorry)...

Usage:
emerge -up world | gest
emerge -pe | gest -V
qpkg -nc -f /sbin/ifconfig | gest
gest -LV
gest packnames.txt

Options:
-l : just a dump sucessful emerges in the emerge.log - date elapsed package
-V : Verbose option (does not apply to -l) lists individual packages
-L : Sorts groups/package with [count] and estimate (average)

<STDIN> : Either output from emerge --pretend or qpkg -nc
**NOTE: a flat list of packages cannot contain version-release numbers

---------------

Output changes: the total number of emerges added before the time estimate

Sample output:

Code: Select all

% gest packnames.txt
app-text/docbook-dsssl-stylesheets [2] Estimate: 34s
app-text/docbook-sgml [1] Estimate: 16s
app-text/docbook-sgml-dtd [10] Estimate: 29s
app-text/docbook-sgml-utils [1] Estimate: 37s
app-text/docbook-xml-dtd [1] Estimate: 20s
app-text/docbook-xml-simple-dtd [1] Estimate: 18s
app-text/docbook-xsl-stylesheets [1] Estimate: 33s
app-text/jadetex [1] Estimate: 26s
app-text/openjade [1] Estimate: 13m27s
app-text/sgml-common [2] Estimate: 32s
app-text/sgmltools-lite [1] Estimate: 35s
dev-libs/libxml [2] Estimate: 5m23s
dev-libs/libxml2 [5] Estimate: 6m24s
app-text/html401 [1] Estimate: 25s
app-text/xhtml1 [1] Estimate: 24s
Total estimated update time: 30 minutes and 44 seconds

Code: Select all

% emerge -up world | gest -V
 
These are the packages that I would merge, in order:
 
Calculating world dependencies  ...done!
[ebuild    U ] sys-devel/distcc-2.8-r2 [2.7.1]  [6] Estimate: 51s
        2003-01-05 12:29      53s    0.14-r1
        2003-01-07 18:57      38s       0.15
        2003-03-15 11:48      50s      1.2.3
        2003-05-17 17:16      50s   2.0.1-r1
        2003-06-14 06:38      52s     2.5-r1
        2003-07-15 20:56     1m7s      2.7.1
 
Total estimated update time: 51 seconds

Code: Select all

% qpkg -nc -f /sbin/ifconfig | gest
sys-apps/net-tools [3] Estimate: 1m7s
        2002-12-13 22:00      49s    1.60-r5
        2003-03-30 14:04    1m11s    1.60-r6
        2003-06-23 23:11    1m22s    1.60-r7
Total estimated update time: 1 minute and 7 seconds

Code: Select all

% gest -l
.........
2003-07-27 02:11    5m48s media-video/ffmpeg-0.4.7_pre20030624
2003-07-27 02:14    2m28s media-libs/fontconfig-2.2.0-r2
2003-07-27 02:36     1m9s media-libs/fontconfig-2.2.0-r2
2003-07-27 05:01     2h8m x11-base/xfree-4.3.0-r2
2003-07-27 11:11    4m38s dev-perl/PDL-2.4.0-r1
2003-07-27 11:14      47s dev-perl/Storable-2.07-r1
2003-07-27 11:14      46s net-libs/libnet-1.0.2a-r2

syscrash · Post by **syscrash** » Tue Aug 19, 2003 9:32 pm

Doesn't seem to be working with k3b for me. It works if I use something like gnome-base/libghttp, but it doesn't give any estimate for kde-libs or anything like that.

insanecanuck · Post by **insanecanuck** » Tue Aug 19, 2003 10:22 pm

Works nice, thanks

lanalyst · Post by **lanalyst** » Wed Aug 20, 2003 6:26 am

syscrash2k wrote:Doesn't seem to be working with k3b for me. It works if I use something like gnome-base/libghttp, but it doesn't give any estimate for kde-libs or anything like that.

The estimates are based on packages already emerged.. that is, packages recorded in /var/log/emerge.log as sucessful...

rizzo · Post by **rizzo** » Wed Aug 20, 2003 6:39 pm

splat does something like this already:

Code: Select all

$ splat -s mozilla-firebird
 = Summary

        Total Builds:  6
        Time Elapsed:  8 hours, 8 minutes, and 11 seconds
        Average Time:  1 hour, 21 minutes, and 21 seconds

        First Build:   net-www/mozilla-firebird-0.6
                       (Tue Jun  3 00:27:04 2003)

        Last Build:    net-www/mozilla-firebird-0.6.1
                       (Wed Jul 30 12:25:43 2003)

gest: perl script to estimate emerge times

gest: perl script to estimate emerge times

gest - update