| View previous topic :: View next topic |
| Author |
Message |
MrBrutico
n00b
Joined: 01 Jan 2015
Posts: 53
|
 Posted: Thu Jul 25, 2019 7:53 pm Post subject: |
 |
|
To my market zero seconds
| Code: | elover # qlist -I | xargs qlop -t | awk '{secs += $2} END { printf("%dh:%dm:%ds\n", secs / 3600, (secs % 3600) / 60, secs % 60); }'
0h:0m:0s
|
|
|
| Back to top |
|
 |
radio_flyer
Guru
Joined: 04 Nov 2004
Posts: 317
Location: Northern California
|
| Posted: Fri Nov 15, 2019 5:39 pm Post subject: |
|
|
For those still playing with this, some notes:
Taking numpy, for example:
| Code: |
$ qlop -t numpy
2012-03-12T17:16:12 >>> dev-python/numpy: 2′56″
2012-06-02T12:34:59 >>> dev-python/numpy: 2′59″
2012-12-07T10:41:48 >>> dev-python/numpy: 3′00″
2013-02-27T08:25:16 >>> dev-python/numpy: 2′17″
2013-05-24T09:00:28 >>> dev-python/numpy: 2′45″
2013-05-24T10:51:23 >>> dev-python/numpy: 4′26″
2013-09-03T13:14:56 >>> dev-python/numpy: 3′29″
2013-09-05T08:35:27 >>> dev-python/numpy: 3′49″
2013-10-31T10:10:45 <<< dev-python/numpy: 4s
2013-10-31T10:38:07 >>> dev-python/numpy: 2′03″
2013-10-31T20:19:37 >>> dev-python/numpy: 2′16″
2013-12-02T00:49:14 >>> dev-python/numpy: 3′49″
2014-02-09T09:50:46 >>> dev-python/numpy: 2′42″
2014-07-06T12:07:37 >>> dev-python/numpy: 1′21″
2014-10-27T14:35:20 >>> dev-python/numpy: 1′25″
2014-10-28T20:37:48 >>> dev-python/numpy: 1′20″
2014-11-23T10:35:55 >>> dev-python/numpy: 1′16″
2014-11-24T09:04:41 >>> dev-python/numpy: 1′28″
2015-03-03T09:50:57 >>> dev-python/numpy: 3′32″
2015-03-31T00:30:17 <<< dev-python/numpy: 4s
2015-03-31T00:35:55 >>> dev-python/numpy: 3′29″
2015-07-26T09:11:41 >>> dev-python/numpy: 2′31″
2015-10-02T03:45:29 >>> dev-python/numpy: 2′46″
2016-01-26T09:59:43 >>> dev-python/numpy: 1′49″
2016-10-28T09:00:12 >>> dev-python/numpy: 1′46″
2016-11-22T09:07:14 >>> dev-python/numpy: 1′54″
2017-04-19T05:32:09 >>> dev-python/numpy: 1′47″
2017-04-20T03:32:36 >>> dev-python/numpy: 1′46″
2017-11-23T04:45:17 >>> dev-python/numpy: 1′46″
2017-11-29T12:14:24 >>> dev-python/numpy: 2′46″
2017-12-02T17:06:06 >>> dev-python/numpy: 2′44″
2018-06-22T11:00:27 >>> dev-python/numpy: 6′36″
2018-06-22T23:23:59 >>> dev-python/numpy: 6′41″
2018-06-23T00:47:55 >>> dev-python/numpy: 2′33″
2018-06-23T10:44:59 >>> dev-python/numpy: 5′03″
2018-07-24T10:36:58 >>> dev-python/numpy: 3′57″
2019-03-23T14:44:31 >>> dev-python/numpy: 4′31″
2019-06-26T10:48:30 >>> dev-python/numpy: 4′12″
2019-07-01T08:47:48 >>> dev-python/numpy: 5′40″
2019-07-01T12:00:21 <<< dev-python/numpy: 5s
2019-07-01T12:14:07 >>> dev-python/numpy: 5′08″
2019-07-01T13:47:51 >>> dev-python/numpy: 5′11″
2019-11-08T13:26:54 >>> dev-python/numpy: 4′23″
2019-11-10T06:39:16 >>> dev-python/numpy: 4′25″
|
Note that the times vary (on the same machine) quite a bit because different amounts of parallelization are needed to get a working build (numpy is very sensitive to this). Also note that some unmerge times are included, because occasionally I had to unmerge numpy to make portage happy, and then merge it back in "carefully". (numpy is also very touchy like this.)
The '-t' option to qlop reports ALL emerge and unmerge times for a package. This is why some folks are getting huge numbers. Also note that the times are listed in field 4, not field 2. On my system, the OP script returns:
| Code: |
$ qlist -I | xargs qlop -t | awk '{secs += $2} END { printf("%dh:%dm:%ds\n", secs / 3600, (secs % 3600) / 60, secs % 60); }'
0h:0m:0s
|
If I change the awk field to 4:
| Code: |
$ qlist -I | xargs qlop -t | awk '{secs += $4} END { printf("%dh:%dm:%ds\n", secs / 3600, (secs % 3600) / 60, secs % 60); }'
211h:59m:59s
|
That explains why some folks are getting no times, while others are getting outrageous numbers. This is with qlop from app-portage/portage-utils-0.80.
With that version, the '-a' option for qlop reports an average:
| Code: |
$ qlop -a numpy
dev-python/numpy: 3′10″ average for 41 merges
dev-python/numpy: 4s average for 3 unmerges
|
Note that the time is now output in field two. There still two entries for emerges and unmerges, but the unmerges tend to be fast and don't effect the overall results too much. The average build time is quicker than my current build times, but not bad as a wag. Numpy is a worst-case package; most package build times don't show this much variation.
Using '-a' instead of '-t' and reverting back to field 2, the script produces:
| Code: |
$ qlist -I | xargs qlop -a | awk '{secs += $2} END { printf("%dh:%dm:%ds\n", secs / 3600, (secs % 3600) / 60, secs % 60); }'
15h:52m:43s
|
This estimate much more closely matches the average world build time that I see. Double that for the time spent resolving various build failures and filing the appropriate bug reports and patches. |
|
| Back to top |
|
|
acidbreez
n00b
Joined: 05 Dec 2006
Posts: 24
|
| Posted: Sat Jul 16, 2022 1:19 am Post subject: |
|
|
I know this is an old post but I figured I would show you guys/gals what I got for numbers too 
| Code: | | $ qlist -I | xargs qlop -t | cut -f4 -d" " | awk '{s+=$1} END {print "secs="s}' > /tmp/btime && eval $(cat /tmp/btime) && printf '%dh:%dm:%ds\n' $(($secs/3600)) $(($secs%3600/60)) $(($secs%60)) && rm /tmp/btime |
43h:26m:45s
| Code: | | $ qlist -I | xargs qlop -a | awk '{secs += $2} END { printf("%dh:%dm:%ds\n", secs / 3600, (secs % 3600) / 60, secs % 60); }' |
7h:1m:40s
Linux Gentoo 5.18.5-gentoo-custom #9 SMP PREEMPT_DYNAMIC Fri Jul 8 10:47:38 MDT 2022 x86_64 AMD Ryzen 9 3900X 12-Core Processor AuthenticAMD GNU/Linux |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Sat Jul 16, 2022 7:50 am Post subject: |
|
|
| acidbreez wrote: | | Code: | | $ qlist -I | xargs qlop -t | cut -f4 -d" " | awk '{s+=$1} END {print "secs="s}' > /tmp/btime && eval $(cat /tmp/btime) && printf '%dh:%dm:%ds\n' $(($secs/3600)) $(($secs%3600/60)) $(($secs%60)) && rm /tmp/btime |
|
I realized that the number shown by this code is much too low (only ~20 hours).
Indeed:
| Code: | $ qlop -t gcc | cut -f4 -d" "
5:21:17
8:14:55
34s
$ qlop -t gcc | cut -f4 -d" "| awk '{s+=$1} END {print "secs="s}
secs=47 |
WTF? Aha:
| Code: | qlop -t gcc | cut -f4 -d" "| awk '{s+=$1;print s}'
5
13
47 |
The largest unit (typically hours or minutes) is just counted as a second and the rest discarded.
A fix is to pass the "-M" option to qlop -t:
| Code: | $ qlist -I | xargs qlop -tM | cut -f4 -d" " | awk '{s+=$1} END {print "secs="s}' > /tmp/btime && eval $(cat /tmp/btime) && printf '%dh:%dm:%ds\n' $(($secs/3600)) $(($secs%3600/60)) $(($secs%60)) && rm /tmp/btime
102h:57m:1s |
This looks more realistic. |
|
| Back to top |
|
|
soundrolf
Tux's lil' helper
Joined: 08 Sep 2016
Posts: 122
Location: Cologne / Germany
|
| Posted: Sat Jul 16, 2022 9:51 am Post subject: |
|
|
This show Compile Time with 'emerge -e @world
| Code: | | qlist -I | xargs qlop -a | awk '{secs += $2} END { printf("%dh:%dm:%ds\n", secs / 1800, (secs % 1800) / 60, secs % 60); }' |
Result: 19h:11m:28s
This show the amount of packages
| Code: | | find /var/db/pkg/ -mindepth 2 -maxdepth 2 -printf "%P\n" | wc -l |
Result: 1631
Regargs
soundrolf
_________________
Regards
soundrolf
Mobo: ASUS PRIME B550M-K BIOS 3302 08/01/2023
CPU: AMD Ryzen 7 5800X 8-Core Processor (16) @ 3800 MHz
GPU: AMD Caicos Radeon HD 6450/7450/8450 / R5 230 OEM 1GB DDR3
RAM: 80GB crucial DDR4 3200 MHz 2x8GB 2x32GB |
|
| Back to top |
|
|
flysideways
Guru
Joined: 29 Jan 2005
Posts: 437
|
| Posted: Sun Oct 23, 2022 5:47 pm Post subject: |
|
|
| Code: | ~ # qlist -I | xargs qlop -t | awk '{secs += $4} END { printf("%dh:%dm:%ds\n", secs / 3600, (secs % 3600) / 60, secs % 60); }'
6h:9m:23s
|
My Gentoo VM running on a usb ssd attached to a new MacBook Pro.
I found this thread while I was looking about for a way to use qlop to predict update times like I used to do with genlop, "emerge -p command | genlop -p".
Any suggestions?
Thanks |
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 3345
Location: Rasi, Finland
|
| Posted: Tue May 09, 2023 8:01 pm Post subject: |
|
|
Slightly out of topic, but | list of packages which consumed the most time to build: | | qlop -mtM | awk '(pkg[$3] < $4) {pkg[$3] = $4} END {for (i in pkg) print i,pkg[i]}' | sort -nk 2 |
Takes the longest build for each package and sorts by time.
You can give -v to qlop to list all the versions of each package.
With | Code: | | qlist -Iv | xargs qlop -mtM |
...one could easily roughly estimate world compile time.
Maybe I should wrap up this in awk or in sh... Oh well...
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 3345
Location: Rasi, Finland
|
| Posted: Tue May 09, 2023 10:28 pm Post subject: |
|
|
| most time consuming builds ever plus total time compiling combined: | | qlop -mtM | sort -nk 4 | awk 'function hms(s) {h=int(s/3600); s=$4%3600; m=int(s/60); s=s%60; return h "h " m "m " s "s"} {total+=$4; print $3,hms($4)} END {print "Total: " hms(total)}' |
And adding qlist -Iv | xargs in front of that "oneliner" will limit to currently installed packages, with the possible caveat of including every build of the same package which version matches the installed one.
Curious ones can add pass -v for qlop to see what versions it really lists.
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
nikolis
Apprentice
Joined: 21 Aug 2003
Posts: 199
Location: Athens
|
| Posted: Wed May 10, 2023 4:26 pm Post subject: |
|
|
| Code: | /Nic # qlist -I | xargs qlop -a | awk '{secs += $2} END { printf("%dh:%dm:%ds\n", secs / 3600, (secs % 3600) / 60, secs % 60); }'
[b]2h:17m:9s[/b]
|
|
|
| Back to top |
|
|
nikolis
Apprentice
Joined: 21 Aug 2003
Posts: 199
Location: Athens
|
| Posted: Wed May 10, 2023 4:29 pm Post subject: |
|
|
| Code: | /Nic # qlist -I | xargs qlop -a | awk '{secs += $2} END { printf("%dh:%dm:%ds\n", secs / 3600, (secs % 3600) / 60, secs % 60); }'
|
2h:17m:9s |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
